Writing tests is hard. Fixing them is harder. Join the webinar.
Gradle Enterprise 2020.2
Gradle Enterprise 2020.2 introduces a major new feature: Test Distribution. Gradle Enterprise Test Distribution executes your existing test suites faster by distributing across as many machines as you provide.
Background build scan uploading makes capturing build scans for local developer builds seamless as the build now completes before the build scan finishes uploading. This is especially an improvement for developers with poor network connections.
Read on for more details of these and other new features.
Use version 1.5.3 of the Gradle Enterprise Maven extension, and version 3.3.4 of the Gradle Enterprise Gradle plugin for optimal usage of Gradle Enterprise.
Highlights
Test distribution for Gradle builds
Test distribution is a brand new major feature in Gradle Enterprise and is part of the testing add-on package introduced in Gradle Enterprise 2019.5. It allows you to distribute the execution of your build’s tests to as many compute resources, called agents, as you can provide. It is currently only available for Gradle builds, with equivalent Maven functionality to come.
After applying the Gradle Enterprise test distribution plugin to your Gradle Enterprise connected Gradle build, any existing test suite using Gradle’s built-in test task can be configured to be distributed. The plugin ships your tests and the full runtime classpath to any available test distribution agents that are registered with your Gradle Enterprise server to be executed there. Test results including logging are shipped back to the build and are available in build scans and existing reporting such as the results XML file used by many CI servers. If no agents are available, test execution proceeds locally.
The Test Distribution agent is available as a Docker image or standalone fat JAR and is easy to deploy and operate via Kubernetes or any modern compute platform. Registration key secrets – that agents use to connect to the Gradle Enterprise server – can be generated in the Gradle Enterprise admin section. The admin section shows all active jobs, connected agents, and a historical visualization of workload and agent utilization.
See the Gradle Enterprise Test Distribution Gradle Plugin User Manual and Gradle Enterprise Test Distribution Agent User Manual to learn more and get started with Test Distribution.
Improved test result visualization
The test result visualization has been revamped for both Gradle and Maven build scans to make the critical information more accessible and apparent, and to better visualize flaky tests.
The test result page now shows all executions of a test, which streamlines flaky test debugging by putting all information related to the test in one place. For Gradle builds using plugin version 3.3 or later, any logging output for the first successful execution of a test after a failing execution (i.e. a test that succeeded after being retried) is now also displayed, making it easier to identify differences between failed and passed test executions.
Error stacktraces now initially show only the most relevant parts, based on heuristics such as the test class name. The full stacktrace can be viewed by expanding the summarized stacktrace.
The new “Test history” link shows the results of the test over the previous 7 days in the Tests Dashboard, making it easier to understand whether a test’s failure or flakiness is new or pre-existing.
Background build scan publication
Previously, builds waited for build scans to be uploaded before finishing which could cause a delay, particularly for developers with poor network connections. Now, build scans are uploaded in the background after the build has finished which allows IDEs and users to resume control after executing a build much sooner and makes collecting build scans for local development builds seamless. This functionality is available for Gradle and Maven build scans.
Background uploading is now the default behaviour and does not need to be enabled. However, in ephemeral build environments (e.g. CI agents) that terminate as soon as the build finishes it should be disabled to avoid the upload process being terminated early. Please see the corresponding section of the Gradle plugin or Maven extension user manual for more information.
View the dependencies requiring a dependency for Maven builds
The build scan dependency graph visualization allows exploring the dependencies of a module, starting at the top level dependencies and transitively through their dependencies. Now, it is also possible to easily view the dependents of any dependency in the graph via the new “Required by” tab in the detail inspector.
This view shows all of the dependents, which is particularly useful when reasoning about a dependency substitution. When encountering a dependency that has been substituted due to conflict resolution, the “Required by” view can show the dependents of other versions. This is also helpful when viewing dependency graph search results, where the path through the graph to a dependency is less apparent.
Find snapshot dependencies in Maven builds
Dependencies of Maven builds can be filtered by resolution type. Previously, the “dynamic” resolution type filtered to dependencies declared with any kind of non-specific version. This included snapshot dependencies, which have now been broken out into their own type due to their prevalence in Maven builds.
The “dynamic” criteria now includes only range dependencies, and dependencies declared without any specific version number. The previous behaviour can be achieved by including both the “dynamic“ and “snapshot” criteria.
Summarized stacktraces for build failures and Gradle deprecations
As mentioned in “Improved test result visualization“, test failure stacktraces are initially displayed in a summarized form to emphasize the parts of the stacktrace that are most likely to be informative in diagnosing the problem. Similarly, the stracktraces displayed for build failures for both Maven and Gradle are now summarized, along with the stacktraces for Gradle deprecations.
Upgrade notes
Background build scan uploading and CI builds
The new default behavior of the Gradle Enterprise Gradle Plugin and Maven Extension is to upload build scans in a forked background process after the build has completed. This provides a much better experience for developers, but can be problematic in build environments that terminate as soon as the build is finished, such as single-use CI environments, as the upload may be terminated before it completes.
Background upload can be easily disabled, and should be when using environments of this nature. Please see the corresponding section of the Gradle plugin or Maven extension user manual for more information on how to disable background build scan uploading.
Upgrading from versions prior to 2019.1 is no longer supported
As of Gradle Enterprise 2020.2, upgrading directly from versions prior to 2019.1 is no longer supported. If you are using such a version and wish to upgrade, please contact Gradle Enterprise technical support.
Changes
- [FIX] Build deletion is slow when many different tests have been executed
- [FIX] Build scan upload may fail if upload is not initiated within one minute
- [FEATURE] Scan list filtering supports negation in requested tasks/goals field
- [FIX] Tests with & in their name cannot be viewed in build scans
- [FIX] Task inputs comparison fails to render in some cases when file order is different
- [FIX] Refreshing a build scan search with a relative time window may not show new results
- [FIX] Constrained filtering criteria in build scan views are incorrectly prefix matched
- [FIX] Source location for Gradle build failures is not shown when known
- [FIX] Exceptions with many stack frames are not reliably viewable
- [FIX] Console log raw download link is incorrect when filtering a too large console log by task/goal
- [FIX] Dependencies page contains a misleading dash in front of reported resolution times
- [FIX] Support bundles cannot be created for Kubernetes deployments across several nodes
- [FIX] Support bundle generation may fail for Kubernetes deployments due to missing log files
- [FIX] Kubernetes installation scripts are incompatible with OpenShift
- [FIX] Database compatibility check may fail when restoring from a backup
- [FEATURE] Exception stacktrace messages can be linked to
- [FEATURE] Timeline can be filtered by Mojo type in Maven builds
- [FIX] Toggling from relative to fixed build time window in build scan list does not use the same window
- [FIX] Test dashboard displays extremely long test durations incorrectly
- [FIX] Test dashboard shows incorrect build count when a search matches many test classes
- [FIX] Test distribution agent test failures are not visualized in build scans
- [FIX] Task/goal inputs comparison displayed unnecessarily when build tool or plugin versions changed
- [FIX] Linking directly to a dependency in a Gradle build scan may not work reliably
- [FIX] Focusing console log lines is slow for very large console logs
- [FIX] Console log section fails to render when Gradle Enterprise is accessed via plain HTTP
- [FIX] “View task in console log” links are shown for non-task related failures
- [FIX] Detail inspectors are slow to render when sections display a lot of data
- [FIX] Build failures with large exception messages are slow to render
- [FIX] Auto-deletion of build scan performance has regressed since 2020.1
- [FIX] Stopping Gradle Enterprise may cause database index corruption
- [FIX] Corrupt database indexes are not repaired if discovered during daily maintenance
- [FIX] Idle test distribution agents are not always assigned to new build sessions
- [FIX] Build scan search relative time windows do not span entire calendar days
- [FIX] Build scan search tooltips do not explain behavior with regard to and/or
- [FIX] Task inputs comparison does not handle Gradle read-only dependency cache
- [FIX] Test history links do not show history for test across all projects, tasks, and goals
- [FIX] Average transfer speed for build cache and network activity is misleading
- [FIX] Build cache failures do not use consistent stacktrace visualization
- [FIX] Individual frames of stacktraces cannot be focussed
- [FIX] Dependency search results are incorrectly expandable
- [FIX] Console log multi-line focus does not work reliably
- [FIX] Console log renders slowly when there are many lines of logging
- [FIX] Console log for Gradle builds tries to display more lines than browser can render
- [FIX] Results for tests with excessive logging cannot be displayed
- [FIX] Groovy power assert failures in tests are rendered poorly
- [FIX] Kubernetes configuration does not correctly set memory limits for Test distribution component
- [FIX] Test distribution agents cannot connect to server when deployed with Kubernetes
- [FIX] Minimum supported Kubernetes version raised to 1.9
- [FIX] Internal dependencies have reported CVE’s
- [FEATURE] Test distribution for Gradle builds
- [FEATURE] Improved visualization of test results
- [FEATURE] Background upload of build scans when the build has finished
- [FEATURE] Insights on dependencies requiring a given dependency in build scans for Maven
- [FEATURE] Gradle deprecation notices show documentation link when available
- [FIX] Gradle task inputs comparison may fail when comparing lifecycle tasks
- [FIX] Test page may fail when a test suite is executed several times by the same Gradle task
- [FIX] Build scan background callbacks fail when invoked on certain Groovy objects
- [FIX] Build cache events consume too much disk space
- [FIX] Database backup may fail if executed while Gradle Enterprise is stopped