Running Jenkins jobs sequentially

I am a huge fan of the Jenkins continuous integration tool, using it not just for continuous integration, but also continuous delivery, server monitoring and performance testing.

Although using Jenkins is normally a breeze, I recently had a need to run several Jenkins jobs sequentially and it proved a little trickier than expected, so I thought worth posting about.

Basically, I wanted to call a database backup job before calling the deploy build job, which obviously shouldn’t be done in parallel. After a bit of research, I found a number of ways to have Jenkins run jobs sequentially.

1) Chain jobs together

Chain Jenkins job together using the “Build other projects” option in the “Post Build Actions” of a job. The downside to this is that it introduces a dependency between jobs. For example, I want to be able to run the database backup job without doing a deployment every time.

2) Parameterized Trigger Plugin

When you have the Parameterized Trigger Plugin installed:

• Create a wrapper job for your sequential jobs
• For each sequential job
• Select Build->Add build step->Trigger/call builds on other projects
• Enter the sequential job name
• Check the ‘Block until the triggered projects finish their builds’ checkbox (this only appears when you have the Parameterized Trigger Plugin installed)

Now when you run the wrapper job, all the triggered jobs will be run in order and sequentially. You of course also have the option of using the parameters functionality of the plugin too.

3) Throttle Concurrent Builds Plugin

The docs for the Throttle Concurrent Builds Plugin seem to be a little lacking, but it works well. With it installed, a “Throttle Concurrent Builds” section shows up in the Jenkins config section (Jenkins -> Manage Jenkins -> Configure System). There, you create a “Multi-Project Throttle Category”
Next, for each job that needs to be run sequentially, you

• Check the ‘Throttle Concurrent Builds’ check box
• Select the ‘Throttle this project as part of one or more categories’ radio button
• Check the checkbox of the “Multi-Project Throttle Category” you created above
• Save

Finally, you create a wrapper job that triggers all your to-be-run-sequentially jobs and when you run it, you should see your jobs running sequentially and in order.
Although a fairly complex option, this approach also allows you to run some jobs sequentially and others in parallel. For example, your wrapper job can run jobs A and B sequentially and jobs C and D sequentially, but trigger A and C in parallel. You can also combine this plugin with the Parameterized Trigger Plugin for maximum flexibility.

4) Join Plugin

Although I haven’t used the Join Plugin myself, it seems worth mentioning. Its docs state that

This plugin allows a job to be run after all the immediate downstream jobs have completed. In this way, the execution can branch out and perform many steps in parallel, and then run a final aggregation step just once after all the parallel work is finished.

 
 
That’s it. Several options for running Jenkins jobs in parallel. Choose the one that best suits your needs…

Tags: agile, ci, continuousintegration, devops, hudson, jenkins