Failure Analytics

Find unreliable builds and tests, which users and environments are affected, identify root cause, and easily collaborate with peers.

Unreliable builds cause downtime, waste compute resources, and are a massive distraction. They also negatively affect the quality of the code that is shipped. Builds become unreliable when problems are too expensive to find, too hard to reproduce for root cause analysis, and when fixes can not be correctly prioritized because their relative impact is unknown. With Develocity, you can leverage analytics to proactively find unreliable builds and tests, learn how many people and environments are affected by the problem, share information about them, and understand the root cause efficiently.

Software testing failures are the most common cause of build failures and many test failures are avoidable. Further, not all avoidable test failures are flaky tests (e.g. unstable environments also cause a large number of test failures).

Build Failure Analytics

Used to avoid local and CI failures. Specifically, it is used to detect, group, and prioritize avoidable build failures for remediation based on impact and determine the root cause of the incident faster. It does this by providing an intuitive dashboard of build and CI failure metrics.

Key Features

• Classify non-verification (a.k.a avoidable) build failures from verification failures (e.g. JUnit, Checkstyle.)
• Fix non-verification failures efficiently using data reported by the Failure Dashboard and associated Build Scan®. This includes failure history across all local and CI builds, common traits among failed runs, differences when compared to successful runs.
• Prioritize remediation for failures based on the frequency of occurrence and impact.
• Search build failures using any part of an error message (as you would on StackOverflow).
• Deliver a non-verification failure history to measure how the negative impact of failures is decreasing or increasing over time.

Test Failure Analytics

Addresses the systemic issues that result when these test failures are left unaddressed and allowed to fester. The result is compounding test suite failures and a lack of developer motivation to address the problems because it's become too difficult to determine where to start. What's needed is an easy way to triage failed tests and separate the healthy tests that detected a real bug in production code, from flaky tests, and completely broken tests that fail frequently and require immediate attention.

Key Features

• Determine priorities for remediation by generating a list of test cases grouped by classes and packages that fail most frequently.
• Get a timeline view of when particular tests started to fail (“before” fix picture) which can be used to understand your starting baseline level of test failure volume and can be helpful in root cause determination.
• Streamline root cause analysis with a listing of build environment metadata for failed test executions.
• View performance characteristics of failed test executions compared to passing ones.
• Verify the establishment of a new reduced test-failure-frequency baseline and monitor on an on-going basis with an extended timeline view (“after” fix picture).
• Analyze test failures by Gradle task or Maven goal with a dashboard of Build & Test Failure Trends for local and CI builds.