Improving performance of Jenkins builds

For building our software we use the continuous integration build server solution Jenkins CI. In the last week we got problems with the build duration. After analyzing the build progress we found some hints for improving the build performance.

Bottlenecks during compiling

The first hint was the duration time of compiling the source code. Before we added a new HDD, build time over all was 11 minutes. After them build time increased to 15 minutes! Another build job which only compiled an artifact without executing any tests needed 3 minutes more – in comparison with the older configuration.
So we analyzed the complete build progress step by step and found a bottleneck. Our software using grails during build progress. For compiling the source code into java bytecode, jenkins writes into .grails in the home directory. We moved the complete home onto a SSD and got back the lost 3 minutes.

Bottlenecks during testing

Another bottleneck is the working directory in Jenkins called “workspace”. Each job gets its own workspace. During generating the artifacts the complete source code repository will checked out into this workspace. Any test result, compiled class and the artifact itself will be buffered in the workspace directory. After build was successful the artifacts and the test results will be copied to jobs directory. If it is possible to mount the workspace of a build job on a SSD you will get any better performance.

The myth of SSD vs. tmpfs

Maybe in your opinion, it could be much better to mount the workspace and .grails directory and any other highly used directories on a tmpfs. We tried this option and the result is: Yes it will decrease the build time. But not as much as you think. Mounting the workspace and the .grails directory on a tmpfs decreased the build time at maximum 10 seconds in comparison to a SSD. The improvement to mount these directories on a tmpfs would be in reducing the write cycles on the SSD which increases the lifetime of a SSD.

Resume

It is possible to impove the duration time of a Jenkins build. The first approach to improve the duration time of a build is to get out in which directories will jenkins write/delete/copy files. In our example we improved the build time by 33% moving the workspace and all other directories on a SSD.
Another possibility to increase the build time could be a faster CPU. This option would need a build server (maybe a jenkins slave) with a desktop CPU. Because the Intel Xeons are much slower than the most high end desktop CPUs. We didn’t tested this, but one of our developer workstations with an Intel i7 CPU only needs 80% of the time for building the same artifact.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s