Exposing your build system API

Nowadays, there are multiple CI tools: Jenkins/Hudson, Bamboo, TeamCity among others. Some of them are free, some others are licensed and they all offer different sets of features. Any of these tools is a great way for software development organizations to quickly jump on the Continuous Integration/Continuous Delivery train.

This is what the typical first project is organized as far as building and deploying goes:

Screen Shot 2013-02-19 at 11.46.53 AM

You have your project code in source control, being checked out by your CI tool, by a single build plan which executes a set of build scripts that may be either embedded in the build plan, checked into a separate source control repository or saved in some other way.

This works fine until you have to create a second project; then, the chart looks like this:

Screen Shot 2013-02-19 at 11.53.02 AM

Two projects, one CI tool, two build plans which are most likely a copy of each other, and both use the same set of scripts. One important thing to notice here is that the build plans are exactly like each other, but will fall out of sync as soon as the needs for one of them change. Multiple projects lead to build plan duplication and as the number of projects grow, the abstract mapping of projects <-> build plans gets uglier. Sometimes even build scripts get duplicated:

Screen Shot 2013-02-19 at 11.56.44 AM

This is far from ideal.

How can we tackle this with an engineering approach? One thing comes to mind: what if we create a domain-specific language layer on top of our build system? This would expose an API of “verbs” or actions that our build scripts can execute, such as package, build, test, deploy, etc. Then the CI tool would only interact with this API. It would look like this:

Screen Shot 2013-02-19 at 12.10.50 PM

We can even take this idea one step further. What if we package this, call it Build System 1.0 (or whatever quirky name you can think of) and publish it to your artifact repository?. Then we can incorporate it into our build plan such that all it does is “press the buttons” of the actions exposed.

This build system package can then be bootstrapped into the project. The details of doing this will vary depending on what framework is being used (Maven, Ant, Gradle) and in some cases might be tricky, but the concept it’s the same. This is not absolutely necessary and can be skipped if it becomes too complicated to implement.

At this point our map looks similar but with some key differences:

Screen Shot 2013-02-19 at 12.57.31 PM

We still have multiple projects, but the CI tool becomes a mere button-presser or “dumb” broker of the actions that are being invoked. We also still have multiple build plans but they are thinner and we don’t care if they fall out of sync, because they’ll be small enough that the difference can be ignored. And more importantly, the build plans are interacting with our brand new API. Also, since all the logic is encapsulated in one place and the history is tracked by the SCM, we are consolidating the knowledge of our build process.

In this sense, the build system becomes much like a software module:

  • It has an API
  • It’s extensible.
  • It’s releasable.
  • And it can be open sourced (either internally or externally if it’s generic enough).

On top of that, there’s an economic benefit. By reducing your build tool to an on-demand button-presser, there’s no need to spend big bucks on tools that will execute the exact same actions as the free ones.

You’re welcome.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: