Someone in your leadership team asks whether AI is working. The numbers within easy reach are about activity: licences in use, tokens consumed, lines of code suggested, pull requests opened, and how much faster people say they feel. Those counts do not show whether changes reach users sooner and with fewer failures. DORA metrics, from Google Cloud’s software delivery research programme, help answer that question. They describe delivery speed and reliability. You still need to ask whether the product meets users’ needs. Read the measures as a set and keep them away from targets.
One update first. If you learned them as “the four keys”, DORA now names five software delivery performance metrics. The fifth is explained below.
Why the easy numbers mislead
In DORA’s 2025 survey of nearly 5,000 technology professionals, 90% of respondents use AI at work and more than 80% believe it has increased their productivity. That’s a measure of adoption and of how the work feels. It isn’t a measure of delivery.
Activity counts have the same problem. More code, more pull requests and more tokens can all rise while the time it takes to get a change in front of users gets longer, because someone still has to read, test and release everything that’s produced. DORA made the point about token leaderboards in June: “company-wide aggregates are meaningless, and individual scores are harmful”. It still sees some value in a dashboard built and shared responsibly, looked at by team, application or use case rather than ranking people.
This matters more with AI because the pressure runs one way. “AI makes you go faster” quickly turns into an expectation, and activity numbers are the easiest way to look as if you’re meeting it. Delivery measures are much harder to flatter.
The five DORA metrics, in plain English
Picture a bakery that buys a much faster mixer. The mixer’s speed tells you very little about the business. What tells you something is how long an order takes to reach the counter, how many batches go out each day, how quickly a burnt batch is replaced, how many batches come back burnt, and how many unplanned bakes you’re doing to put right yesterday’s problems.
Software delivery works the same way. DORA splits its five metrics into two groups.
Throughput: how many changes move through, and how fast
- Change lead time: how long a change takes to go from committed to version control to deployed in production.
- Deployment frequency: how many deployments you make in a given period, or the time between them.
- Failed deployment recovery time: how long it takes to recover when a deployment fails and needs immediate intervention.
Instability: how well the deployments go
- Change fail rate: the share of deployments that need immediate intervention afterwards, usually a rollback or a hotfix.
- Deployment rework rate: the share of deployments that weren’t planned, but happened because of an incident in production.
If you learned the older set, two things changed. Mean time to recover was renamed and narrowed to failed deployment recovery time in 2023, so it now covers only failures caused by a change. And DORA added a fifth metric, deployment rework rate, in 2024, to measure rework directly instead of inferring it from change fail rate.
What AI does to DORA metrics
DORA’s 2025 report sums up its findings in one line: “AI’s primary role in software development is that of an amplifier.” It magnifies what’s already there, the strengths and the dysfunctions.
In the survey, higher AI adoption went with higher throughput, but also with more instability. In the report’s words, AI adoption “not only fails to fix instability, it is currently associated with increasing instability”. Two cautions: these are associations from a survey, not proof of cause, and the throughput link is small. If your throughput figures rise while your instability figures rise too, check whether failures and rework are offsetting the faster delivery.
One possible explanation is the verification tax, described in DORA’s report on the return from AI-assisted development: the extra time spent reviewing and testing a larger volume of generated code. It doesn’t cost that tax directly. It uses delivery metrics as the stand-in: “the higher the verification tax, the fewer annual software deployments will be possible”, and the longer change lead time gets. So if coding speeds up and lead time doesn’t fall, trace where finished code waits. Review is one place to look. Episode 1 of the video series walks through that queue, and what code review is for when agents write the first draft covers the reviewer’s side.
Plan for the possibility of a dip at the start. The same report forecasts “a temporary productivity dip and period of instability” early in adoption. That is a planning scenario, not a guarantee that every team will dip or recover. Agree when to review the evidence and what would make you continue, change course or stop.
Read them as a set
The value of the five is in the tension between them. Deployment frequency on its own can be pushed up by shipping riskier changes more often. Looking only at change fail rate can encourage teams to avoid releases, leaving users waiting. Together they show whether the whole system is getting better or the problem is just moving somewhere else.
DORA’s data also says speed and stability don’t have to be traded against each other. In the 2025 report, the two best-performing clusters of teams did well on both, and they made up nearly 40% of the sample. The return-on-AI report gives the pairing a job: track change failure rate and rework “to ensure that increased velocity isn’t coming at the cost of stability”.
Use these combinations as prompts for investigation. They show where to ask questions, rather than what AI caused:
- Throughput up, instability flat or down: delivery may be improving. Check whether the work is comparable and users are benefiting before widening an experiment.
- Throughput up, instability up: look at failures and rework. Test smaller changes and stronger checks.
- Coding faster, lead time unchanged: trace where work waits after coding, starting with review, testing and release.
- Everything flat after a few months: check whether AI is aimed at the step holding delivery up. Check the work mix and measurement window before changing direction.
Five rules for using DORA metrics with AI-enabled teams
DORA’s guide lists the common ways these metrics go wrong. Under AI pressure, five matter most.
- Measure one application or service at a time. That’s the level the metrics are designed for. Blending them across a department mixes very different contexts, and comparing a mobile app with a mainframe system can mislead. A blended number can also hide the teams that are struggling.
- Don’t turn them into targets. A goal like “Every application must deploy multiple times per day by year’s end” invites teams to game the number. This is Goodhart’s law: when a measure becomes a target, it stops being a good measure.
- Never use one on its own. Pair a throughput measure with an instability measure every time you report.
- Compare a team with itself over time, not with other teams. Competition turns measures into league tables, and league tables get gamed.
- Start rough. DORA warns against putting measurement ahead of improvement: wiring up every system for precise data may not be worth it at first. It suggests starting with conversations or its free Quick Check. Record that first baseline in a spreadsheet, then improve precision when the questions justify it.
What to do this month
Pick one service per team, and with each team write down rough figures for the five metrics over the last three months. Do it before the next AI rollout, licence review or quarterly report, because once a change lands you can’t go back for the “before”. The return-on-AI report makes the same point: it’s hard to measure AI’s effect without knowing where you started. Use DORA metrics to investigate AI’s impact against that baseline, rather than to claim AI caused every change. Compare similar kinds of work and note changes in demand, staffing and release policy.
Then agree with each team, in a sentence, what they expect AI to change, and look again each month. Two signals deserve the most attention: change lead time that stays flat or grows while coding gets faster, and a rework rate that creeps up.
If you want one lever that tends to move all five, DORA’s guide points to smaller batches: smaller changes are easier to move through delivery and easier to recover from when they fail. Its AI Capabilities Model puts a plain limit on it: a batch of code that needs more than a week to finish is too big, and the aim is work that takes a few hours to two days.
How you’ll know it’s working
Imagine a team that does this for three months (an illustrative example, not a real one). When someone asks whether AI is helping, the team answers with its own numbers rather than impressions: lead time steady or falling, deployments at least as frequent, change fail rate and rework rate flat. And if throughput starts rising alongside instability, the team spots it in the first month rather than after a bad quarter.
For a quick first check on one team, episode 2 of the video series compares a team’s guess about how much faster AI has made them with what the clock says.
For a wider view, the free AI Delivery Health Check asks, among other things, what you can show when someone asks what AI is doing for delivery. What the Health Check covers explains the rest.
Tool note: many delivery platforms can calculate these metrics for you. A spreadsheet is fine to start; pick a tool once you know which questions you want it to answer.
