Accelerate
Troubleshoot
Observe
If some of the most important open source projects in the world are any indication, Develocity is quickly emerging as a staple for build and test data analytics and as a source of acceleration technology for both Gradle and Maven-based builds. Spring, JetBrains (Kotlin), JUnit, and not surprisingly, Gradle Build Tool, all rely on Develocity to improve build and test feedback cycle times and make troubleshooting more efficient with root cause analysis data (Build Scans) and failure analytics. Here’s a brief summary of the role Develocity is playing in each project and some high-level results.
Spring
Spring makes programming Java quicker, easier, and safer for everybody. Spring’s focus on speed, simplicity, and productivity has made it the world’s most popular Java framework. In fact, half of all Java developers use Spring Boot, five times as many as use the next most popular framework (JVM Ecosystem Report, October 2019). Develocity is being used by the Spring Boot, Spring Framework, and Spring Security projects with more teams planning to come online.
According to the Spring Boot team an important driver for Develocity was frustration with the length of the feedback loop when making and testing changes. Further, the time spent waiting for builds to complete was adding to the time taken to fix bugs and implement new features. After migrating from Maven to the Gradle Build Tool and further optimizing the Build Cache effectiveness with Develocity, CI builds now take roughly 20 minutes on average, 3-4 times faster than before. Local builds are taking an average of 2.5 minutes, which is 20-30 times faster than before. More details about the build’s performance are available on Spring’s public instance of Develocity.
Beyond the performance improvements, the team has also started looking at some of the other data available to address flaky tests and other avoidable failures. For example, developers have started to use Develocity Trends Dashboard and Flaky Test Management to identify and help resolve any flaky tests occurring on CI. A next step is to experiment with the new Develocity Test Distribution capability.
JetBrains (Kotlin)
Kotlin is a general-purpose, free, open source, statically typed “pragmatic” programming language initially designed for the Java Virtual Machine (JVM) and Android that combines object-oriented and functional programming features. It is focused on interoperability, safety, clarity, and tooling support. Kotlin is the second most-beloved programming language among developers, according to a recent Stack Overflow survey, behind only Rust, and the most used programming language for building JVM applications after Java.
The JetBrains Kotlin Compiler open source project is built with Develocity about 60,000 times every month. During the build optimization period in early 2020, median build time was reduced from about 10 minutes to about 3.3 minutes. Develocity revealed several significant build cache misses caused by custom tasks that were not cacheable. JetBrains made their custom tasks cacheable and also optimized the configuration phase of the build with the help of Build Scan insights. Currently, on CI, median build time is about 3.75 minutes with average remote build cache savings of 20 minutes per build.
Moving forward the JetBrains team is considering expanding the Develocity footprint beyond the Kotlin Compiler project to other Kotlin projects. You can check out the JetBrains public instance of Develocity here.
JUnit
JUnit is the most widely used testing framework on the JVM. JUnit 5 provides a platform for other testing frameworks to integrate with IDEs and build tools as well as a modern, programmer-friendly API for writing tests. Its extensibility (e.g. to write extensions for third-party frameworks such as Spring) is key to its success.
Given its widespread usage, JUnit needs to be thoroughly tested on all current JDKs and early access versions. Even though most of JUnit’s own tests are quite fast, the CI fanout of building and testing on all these versions takes a toll on overall CI build times. Since JUnit is an open source project without a backing company, it relies on free CI services, currently GitHub Actions, where agents are ephemeral (i.e. no reusable state can be kept between executions). Since the JUnit team enabled the Develocity Remote Build Cache, the average build time decreased by about 50%. In addition, the team enjoys the benefits of the local and remote Build Cache when switching from one feature branch to another which makes for a much-improved developer experience.
Recently, the JUnit team has added the test-retry Gradle plugin to their build. The goal is to stabilize the occasionally flaky tests for timeout assertions and parallel test execution with the help of the Develocity Tests Dashboard.
You can check out the JUnit public instance of Develocity here.
Gradle Build Tool
Gradle has emerged as the build tool of choice for projects within the JVM ecosystem, including Kotlin. It is downloaded on average more than 19 million times per month and has been counted in the Top 20 Most Popular Open Source Projects for IT by TechCrunch.
Gradle Build Tool has a comprehensive automated test suite that covers different operating systems and Java versions. The team runs about 70,000 builds a week.
The build heavily relies on the remote build cache for build acceleration, especially on CI, to avoid recompiling the code for different stages of the CI pipeline and re-running tests that are not relevant to tested changes. The team also relies heavily on Build Scans for finding performance regressions in the build, especially when related to build caching, and for analyzing build failures. The flaky test management capabilities available in Test Failure Analytics are used regularly to identify and prioritize fixing the most flaky tests. The most problematic tests are regularly addressed in a consolidated team effort called “fix it day”.
Most recently, Gradle Build Tool started to use Test Distribution to further reduce build times. For example, the time to run the tests for a key subproject on local builds dropped from 10-20 minutes to 4-6 minutes. Multiple project maintainers called this improvement “life-changing”.
You can check out the Develocity public instance here.