Accelerate
Troubleshoot
Observe
We’re excited to announce a big step towards our mission to transform how software is built and shipped. We recognize that not all teams are using Gradle or are in a position to migrate soon. In this blog post, you’ll learn how you can utilize the insights provided by build scans and the acceleration of the build cache for Apache Maven builds.
Maven build scans
Build scans are a record of what happened during a build that you can easily share with coworkers, which often reduces the time spent debugging build problems considerably. In order to generate a build scan for your Maven build, you need to apply the Develocity Maven extension by adding the following configuration block to the .mvn/extensions.xml file in your project.
<extensions>
<extension>
<groupId>com.gradle</groupId>
<artifactId>gradle-enterprise-maven-extension</artifactId>
<version>1.0.6</version>
</extension>
</extensions>We’re going to use the Spring Boot project as an example in this blog post because it has a non-trivial build and a medium size with about 270,000 lines of code. Now that we’re set up, let’s run a build:
$ mvn clean package
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:55 min
[INFO] Finished at: 2019-03-07T13:42:34+01:00
[INFO] ------------------------------------------------------------------------
[INFO] 1154 goals, 1154 executed
Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? (yes/no): yes
[INFO] Gradle Terms of Service accepted.
[INFO]
[INFO] Publishing build scan...
[INFO] https://gradle.com/s/vcsk26inbactc
[INFO]As you can see from the above output, the Develocity Maven extension prints the URL of the build scan at the very end of the build log. In this initial version, each build scan provides you with a detailed timeline of the goals that were executed, performance insights such as cache utilization, an overview of projects and applied plugins as well as the switches that were used in the build and details about the infrastructure that it was executed on. Future versions will bring features currently only available for Gradle builds such as dependency visualization and comparison, the performance dashboard, inputs comparison, and others, to Maven builds.
Maven build cache
Build caching for Gradle has reduced build times for many software teams. For instance, the Gradle team utilizes the build cache to reduce the average build time by more than 80%. Develocity 2019.1 now brings the power of the build cache to Maven builds. As Maven does not have an up-to-date checking feature, the local build cache will serve as a very effective substitute for that. So on average, the impact of the build cache will be even higher for Maven builds. Conceptually, build caching for Maven works the same way as it does for Gradle. Execution of a goal can be avoided by reusing the outputs from a previous identical execution. Whether an execution is identical is determined by computing a strong hash key from the inputs of a goal execution and storing its outputs in the build cache under that key.
While you can submit Maven build scans for free to scans.gradle.com (just like for Gradle), using the build cache for Maven requires a Develocity installation and license. Develocity is a commercial SaaS product that can be hosted on-premises and ships with a build scan server and a multi-node build cache backend.
Thus, in order to use the build cache, you need to specify a Develocity server. In your project directory add the following configuration to the .mvn/gradle-enterprise.xml file:
<gradleEnterprise>
<server>
<url>https://gradle-enterprise.mycompany.com</url>
</server>
</gradleEnterprise>Once you’ve done that, you’re ready to run your first Maven build with build caching enabled.
$ mvn clean package
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17:32 min
[INFO] Finished at: 2019-03-07T14:25:54+01:00
[INFO] ------------------------------------------------------------------------
[INFO] 1154 goals, 1154 executed
[INFO]
[INFO] Publishing build scan...
[INFO] https://gradle.com/s/sgpehtjem32am
[INFO]When a Develocity server is configured, your build scans would normally be uploaded to that server. For the sake of this blog post, we’ve uploaded them to scans.gradle.com so they are easy to access. In the build scan of the above example build, we can see that 1154 goals were executed in 88 projects in 17 minutes and 32 seconds. Since this was our first build using the build cache, all of the goals were executed and none were loaded from cache. When we take a closer look at the “Performance” tab of the build scan, we can see that 231 goal executions were stored in the cache, ready to be re-used by subsequent builds
In its initial version, the Develocity Maven extension supports caching of the compile and testCompile goals of the maven-compiler-plugin, the test goal of the maven-surefire-plugin, and the integration-test goal of the maven-failsafe-plugin. Future versions of the extension will add caching support for more plugins and goals. When we explore the timeline and change the sort order to “Longest”, we can see that test execution using the maven-surefire-plugin was a major contributor to the overall build execution time. Thus, already with the initial set of supported goals, we can expect a considerable speed-up when we execute a second build without making any changes to the project:
$ mvn clean package
...
[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ spring-boot-autoconfigure ---
[INFO] Loaded from the build cache, saving 2m:34s:970ms
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:51 min
[INFO] Finished at: 2019-03-07T14:45:46+01:00
[INFO] ------------------------------------------------------------------------
[INFO] 1154 goals, 924 executed, 230 from cache, saving at least 13m:3s
[INFO]
[INFO] Publishing build scan...
[INFO] https://gradle.com/s/6ndftemyd7aci
[INFO]Instead of almost 18 minutes, the second build completed in 3 minutes and 51 seconds. As we can see in the build scan, 230 goals were loaded from the local build cache. Even though we haven’t changed anything in the project, there was one cache miss. This is why running the build twice without any changes is an insightful first step when introducing the build cache to a new project. Such cache misses are caused by unstable inputs, e.g. timestamps in build artifacts such as manifests of JAR files. The build cache guide provides detailed instructions on how to find such unstable inputs.
Another scenario that shows what you can expect from the build cache is making a small change to a subproject that is changed often. In our case, we’ll simply add a System.out.println("Hello") to the org.springframework.boot.cli.SpringCli class. In the timeline of the resulting build scan, we can see that the compiler:compile goal in the Spring Boot CLI project was executed because its inputs changed. Similarly, the project’s tests were recompiled and executed. However, the outputs of 224 goals did not have to be executed again because their inputs did not change. The overall build time was 4 minutes and 42 seconds.
We evaluated the Maven build cache with a wide range of popular open source projects both small and large. Most projects, including small ones, can expect a significant benefit from the cache. In general, the better your build is modularized the more effective the cache will work for you. For example, if there’s only one module every change requires everything to be recompiled and all tests to be executed again.
Using the remote build cache
So far we’ve only seen the local build cache in action. It allows builds on the same machine to reuse the outputs whenever they execute a goal with the same inputs. This is particularly effective for Maven builds given the lack of incremental building and need to frequently execute clean builds. However, the full benefit of the build cache is realized when also using the remote cache backend that Develocity provides. This remote cache allows you to share cached outputs across your whole team, including local and CI builds.
By default, the Develocity Maven extension queries the remote build cache of your Develocity installation before executing a supported goal. However, since the remote build cache is shared with other developers and CI machines, storing in the remote cache is disabled by default. You can enable it by adding the following configuration to your .mvn/gradle-enterprise.xml:
<gradleEnterprise>
<buildCache>
<remote>
<storeEnabled>true</storeEnabled>
</remote>
</buildCache>
</gradleEnterprise>We recommend enabling storing entries in the remote build cache only on CI servers. This way, all developers can take advantage of goals that have already been executed with the same inputs on the CI server when running builds locally. At the same time, only the controlled environment of a CI server is allowed to created shared build output.
Executing the build of the Spring Boot project on another machine after its outputs have been stored in the remote build cache yields the output below.
$ mvn clean package
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:00 min
[INFO] Finished at: 2019-03-07T15:21:34+01:00
[INFO] ------------------------------------------------------------------------
[INFO] 1154 goals, 924 executed, 230 from cache, saving at least 12m:57s
[INFO]
[INFO] Publishing build scan...
[INFO] https://gradle.com/s/dkvwbtxigilqu
[INFO]As you can see from the resulting build scan, the outputs of 230 goals were loaded from the remote build cache. In addition, they were stored in the local build cache to save the time required for downloading from the remote build cache in future builds on the same machine. In this case, the machine was connected to the build cache backend via the Internet. In a typical setup, where the backend is located in the local intranet, you can expect even less overhead.
Next steps
If you’re interested in learning more about build scans and the build cache for Maven projects, the free Maven build cache deep dive training is a good start. While the user manual documents all available configuration options, the build cache guide provides in-depth explanations on the underlying concepts and a hands-on tutorial that covers all aspects of introducing the build cache to your team or organization. Of course, you can always contact us to learn more about Develocity. If you have feedback or questions regarding free Maven build scans on scans.gradle.com, please visit the forums. We’re looking forward to your feedback!