Try our free Gradle and Maven training courses on DPE University

Back to Demos

Flaky Test Management

Overview

Quickly detect, prioritize, and fix flaky tests based on frequency, recency, and impact to users and projects, and much more.


A flaky test is a non-deterministic test caused by code that produces both a “passing” and “failing” test result. Develocity detects flakiness for a single build by re-running failed tests and across builds by inspecting test inputs. When they succeed after re-trying, the test is classified as flaky, but it still must be remedied. Flakiness is not necessarily rooted in the test, it can also indicate flaky behavior in your development environment and production code. Without a remedy, you shift the consequences of unreliable code to users and customers and flakiness accumulates, which increases build and test time and wastes valuable compute resources. A less obvious consequence is the degree to which flaky tests poison your culture by reducing confidence in your test suite. This may result in engineers building less often and paying less attention to writing tests.


To address these pains, Develocity provides Flaky Test Management to help you proactively detect flakiness in your application when conducting software testing. This allows you to prioritize which to fix first.

Key Features

  • Detect flaky tests reliably for all Maven, Gradle, and Bazel builds, including local ones by using a retry mechanism and Build Scan®.

  • Prioritize flaky test remediation based on frequency of occurrence, recency, and impact for a custom set of builds.

  • Observe flaky test history to measure how their negative impact is increasing or decreasing over time.

  • Fix flaky tests efficiently using data reported by the Test Failure Dashboard and associated Build Scan. This includes test history across all local and CI builds, common traits among flaky runs, differences when compared to stable runs, and test methods used for specific runs.

Related Feature Info and Customer Stories