Hours saved reach the budget only when someone moves them

Opposite findings agree about who gains from assistance

Two of the most cited measurements of AI at work disagree in direction. Brynjolfsson, Li and Raymond studied 5,172 customer-support agents and found issues resolved per hour rose 15 percent on average once the assistant was available. METR randomised 246 real tasks across 16 experienced open-source maintainers working in repositories they knew intimately, and found completion times 19 percent worse with the tools than without.Sources: Quarterly Journal of Economics, Generative AI at Work, 2025, METR, Experienced Developer Productivity Study, 2025

Presented as competing headline percentages the two results are unusable, and averaging them would be worse. Read at the level of who was measured, they converge. The support study found its gains concentrated among the least experienced and lowest-skilled agents, with the most experienced showing small speed gains and small declines in quality. METR measured maintainers who already held years of context in the exact codebase they were asked to change.Sources: Quarterly Journal of Economics, Generative AI at Work, 2025, METR, Experienced Developer Productivity Study, 2025

One moderator transfers between the two studies. Assistance pays most where the worker lacks context the model can supply, and pays least, or costs time, where the worker already holds more context than the model can reconstruct. METR published a second reading of the experiment in February 2026, reporting a smaller and statistically inconclusive slowdown among newly recruited developers. That is the direction the moderator predicts.Sources: METR, Experienced Developer Productivity Study, 2025, METR, Changing our developer productivity experiment design, 2026

For an investment committee the consequence is direct. A percentage lifted from either study describes the workforce that study observed. The equivalent local measurement is eligible volume, the share of it completed through the new route, the effort spent reviewing and correcting output, and elapsed time from request to accepted result, taken over a window long enough to survive one unrepresentative week. Staff estimates cannot stand in for that measurement. METR asked its developers to forecast the effect before the trial and to estimate it afterwards. They predicted a 24 percent speed-up, believed after the fact that they had gained 20 percent, and had in fact lost 19. A programme whose baseline rests on what the team believes the tool saved is reading the instrument that produced that error.Sources: METR, Experienced Developer Productivity Study, 2025

Because the moderator has to be measured locally, the arithmetic that turns a local measurement into money has to be built stage by stage.

Gross hours contract three times before reaching cash

Gross released time is the starting quantity, and three deductions stand between it and a financial consequence. Eligible volume that never reached the new route removes the first slice. Review, exception handling, support and the workarounds people invent for themselves remove the second. Capacity survives both, and capacity becomes money only where a cost leaves the budget or where the freed hours are deliberately pointed at work whose output is already counted. None of the three deductions is exotic. Each is a quantity the workflow can produce once somebody decides that it should.

The exhibit below runs that logic across 100 modelled hours. Its stages matter more than its values. Each stage is a point at which a programme can lose the entire benefit while continuing to report it, and each is answered by a different person holding a different piece of evidence.

Exception effort is the deduction programmes most often omit, because it accrues to people who were never part of the pilot. A checker who reads every output before release, a supervisor who fields the cases the system declines, a colleague who keeps the old spreadsheet alive for the awkward accounts: all three are real hours, all three are caused by the automation, and none of them appears in a before-and-after comparison of task time. Counting that effort requires a decision about scope before the pilot begins, because it stays invisible to any measurement bounded by the automated step.

Redeployment decides whether anything financial happens at all. Released capacity absorbed into general availability leaves no trace in any account, and it is indistinguishable from a benefit that never existed. Capacity pointed at an identified queue, an identified backlog or a piece of demand the business already counts can be observed moving. The ledger structure below shows which reading supports each stage and where the confidence level attaches.

Recording the bridge honestly usually reduces the headline number by a large fraction. A programme reporting a modelled twenty-hour cash equivalent against a hundred theoretical hours is in a defensible position, because it can say where the other eighty went and what would have to change to recover any of them. A committee can interrogate that position. It can ask why adoption stalled short of eligible volume, whether the review step is permanent or an artefact of an immature model, and which queue the redeployed capacity was meant to reach. None of those questions is available against a single multiplied figure.

Following the bridge to its last stage exposes how rarely the final conversion can be evidenced, which turns the method into an argument against the investments it was built to assess.

Requiring a cash trace would decline almost everything

The strongest objection to this method is that almost nothing survives it. DSIT’s UK adoption research found 56 percent of current AI users reporting higher employee productivity while 77 percent reported no change in revenue. Humlum and Vestergaard, using Danish administrative records for occupations with high adoption, found users reporting time savings of about 2.8 percent of work hours and no significant effect on earnings or recorded hours, with confidence intervals ruling out effects above one percent. A nationally representative US survey by Bick, Blandin and Deming put self-reported savings at 5.4 percent of work hours among users. The pattern across all three is the same. Reported hours move and reported money does not.Sources: DSIT AI Adoption Research 2026, Humlum and Vestergaard, Large Language Models, Small Labor Market Effects, NBER, 2025, Bick, Blandin and Deming, The Rapid Adoption of Generative AI, NBER, 2024

MIT’s NANDA review of more than 300 enterprise initiatives arrived at the same place from the opposite direction, reporting that 95 percent of organisations could show no measurable profit-and-loss return from generative AI pilots. Applied as a gate at the point of decision, a cash trace would have stopped the programmes that later mattered alongside the ones that never did, because at that point the two are indistinguishable.Sources: MIT NANDA, The GenAI Divide: State of AI in Business 2025

Benefits that never pass through a salary line can still be the reason to invest. Faster response can change conversion on a pipeline the sales team already quotes. Fewer errors can reduce a remediation cost the business already books. Better-evidenced decisions can lower the probability or the consequence of a control failure. Refusing all of these because they arrive as quality, service or risk would leave a firm holding the most conservative investment case in its market and none of the option value. There is a timing problem underneath this. Capability built on one workflow lowers the cost of the next one, and that transfer is real even though it never appears in the benefits case for the workflow that paid for it.

The concession the method has to make is about pricing, and about how confidence is recorded. A benefit may sit in the ledger with an identified mechanism, an identified manager and a stated result that would disprove it, at low confidence and with no pound attached. The programme in the opening arithmetic reported as cash a quantity it had never observed. That is the defect, and it survives whatever the true benefit turns out to be.

Since a cash test alone would reject benefits the firm has good reason to buy, the ledger has to carry claims it cannot yet price and say plainly that it cannot.

Report confidence alongside every figure in the ledger

Restate the hundred hours as a hypothesis with a date on it. The entry should hold the baseline period and eligible volume, the observed share completed through the new route, the effort spent on review and exceptions, the identified destination of any released capacity, and a confidence level that starts low and moves only as observation replaces assumption. Confidence is the field that does the work, because the next reading exposes any inflation. It also fixes the reporting cadence, since a claim held at low confidence has to state the reading that would raise it and the date by which that reading will exist.

Two disciplines keep the ledger honest at low cost. Measure the workflow in front of you, since the research shows the effect size belongs to the worker and the task and travels badly between them. Then separate the operating change from the financial one at the point of entry, so that an improvement in handling time is never promoted into a saving nobody has found.

Measurement repays its cost when it changes a decision, and the decision it should change most often is how much to claim. A programme that reports a range with its causal chain visible, and marks the stages it has not yet observed, gives an investment committee something it can act on. A precise figure with no route to the accounts gives it something to repeat.