We asked 15 engineering leaders where AI gains disappear. Review, requirements, QA, demand — the bottleneck doesn't vanish, it moves somewhere different.
Earlier this year we ran a field study inside a 250-engineer organisation to find out what AI-assisted development was actually doing to delivery.
The short version: AI produced roughly a 14% gain in implementation time — the stretch from task started to merge request opened. Only about 6% of that reached delivery. The rest was absorbed downstream, mostly in review and rework, which grew enough to eat more than half the gain before anything shipped.
We published the study, and then I did something less rigorous and, it turned out, more informative. I took the finding to fifteen engineering leaders — directors, VPs, heads of delivery, across consultancies, product companies, platform teams — and asked them one question:
Where does it leak for your teams — review, or somewhere else?
I expected most of them to say review. I'd just measured it. I had a chart.
Almost nobody said review.
They didn't disagree with the finding, and several of them independently produced numbers with the same shape. What they disagreed about — without knowing they were disagreeing, since they were all talking to me separately — was where the constraint had landed in their own organisation.
That turned out to be the more useful result.
Review and rework. This is our study's answer and the one I'd assumed was general. Implementation got faster, review got slower, and the second effect was larger than the first. When we asked the 250 developers why they were spending time fixing, checking and redoing AI output, the two dominant reasons were lack of trust in generated code and batch sizes that had grown too large to review properly.
Product definition. An engineering director at a large IT services firm described the opposite problem. His developers' output had roughly quadrupled; effective velocity was up about 30%. And the result was that his backlog was burning down faster than product owners and business analysts could refill it. His attempt to point AI at those roles hadn't been fruitful. His constraint wasn't review — it was the supply of well-formed work.
Demand. An analytics leader in a 6,000-person organisation gave me the answer I hadn't anticipated at all. Delivery got faster, so the organisation asked for more: more documentation, more of everything that scales with output. Productivity rose and net output stayed flat, because the freed capacity was absorbed before it could show up as delivery. Nothing leaked in the pipeline. It leaked on the way in.
Requirements. A director of software development who's been shipping AI-generated code for over six months put the constraint at the very front. Capacity, in his framing, is lost across requirements, standards, architecture, review, QA and deployment — and AI doesn't inherently fix any of them. It can help or exacerbate any of them. He starts at requirements, because if the input to the system is flawed, the output can only be flawed.
Testing. The same IT services director flagged QA as a distinct leak rather than a subset of review: even at 100% API automation, significant manual testing remained, particularly on UI. Generated code arrives faster at a verification step that hasn't automated at the same rate.
Validation and trust. A technical director running agentic end-to-end flows rather than code completion expects the constraint somewhere different again. When an agent produces a whole change rather than a suggestion, the human bottleneck stops being review of code and becomes confidence that the thing is right. Different work, different queue, same underlying shape.
The disagreement is about location. Underneath it there's a striking amount of convergence.
The consultancy director's 4x developer output against 30% effective velocity is our 14-to-6 in different units — a large gain at the point of authorship, a much smaller one at the point of delivery. Another leader pointed me at the broader literature converging on a 5–10% organisation-level boost, which brackets our net figure of about 6% almost exactly.
So the numbers agree. The mechanism agrees too, once you state it generally enough:
AI doesn't remove the constraint. It moves it. Making one step dramatically cheaper doesn't make the system faster — it relocates the bottleneck to whichever step is now the slowest. That step is different in every organisation, because it depends on what was already weakest before AI arrived.
The IT services director is the clearest demonstration, because he's mid-move. His constraint was review. It's now becoming product definition. Nothing about that transition was visible until it was already happening.
This is the part I've changed my mind about.
If you'd asked me a year ago what an engineering organisation should do about AI-driven review load, I'd have given you a confident list. Having had these fifteen conversations, the confident list looks like a category error. An organisation whose leak is in requirements gets nothing from optimising review. An organisation whose leak is in demand absorption can fix review perfectly and watch the org-level number stay exactly where it was.
Everyone in these conversations was running multiple changes in parallel — AI in development, AI-assisted review, AI in QA, automation layers on top of project management, and in several cases the same mindset spreading into marketing, sales and HR. When that much moves at once, the honest question isn't which practice should we adopt. It's which of the things we're already doing is moving delivery, and which is moving numbers that don't matter.
You can't answer that from a blog post, including this one. You can only answer it by being able to see your own flow — which step is slow, in which direction it's moving, and what happened to it after you changed something.
With that caveat firmly attached, four moves came up often enough to be worth naming.
Load standards before coding, not at review. The sharpest challenge I got in any of these conversations was this: if you tell a junior developer to build something and then judge their pull request against standards you never gave them, does that make sense? Team and company conventions need to be present during the work, not applied to it afterwards. Standards checked at the PR are standards introduced too late.
Put a gate before the human. The study organisation kept humans in review deliberately — nobody I spoke to was arguing for removing them — but added an automated self-review pass in front. The effect was that reviewers stopped functioning as the cleanup crew for problems a machine could have caught.
Make small changes a team contract, not a personal habit. One director, looking after a single team, reached this independently and immediately: small items yield more from AI than big batches, so slicing more finely raises the share of the gain that survives. The study organisation arrived at the same conclusion, but it took them 500 developer comments to get there. It was one of only two changes they made.
Keep the human in the loop on purpose. A software architect I spoke to was moving toward AI-assisted review with as many guardrails as possible while remaining explicit that human review isn't going away for a long time. The distinction that matters is between keeping humans in the loop as a decision and keeping them there because nobody got round to changing it.
One friction that nobody had named for me before, and which I now hear everywhere: one model writes the code, another model faults it. Rework generated not by human disagreement but by two systems with different priors, arguing through your pull request.
When I shared the study with one leader, she said what a lot of people say — that the emerging consensus of a 5–10% boost is worthwhile but not revolutionary yet.
I've come to think that framing is the trap, and it's a trap the headline number sets.
The honest headline isn't AI gives you 6%. It's AI gives you 14%, and your delivery pipeline eats more than half of it before anything ships.
Those two statements sound similar and point in opposite directions. The first says wait for better models. The second says the value has already been generated and is being destroyed in transit — in review queues, in rework, in coordination, in whichever step your own organisation happens to be worst at.
That's not a modelling problem. It's an operations problem. Which means, unlike the first framing, it's something you can work on this quarter.
The catch — and it's the entire reason for this piece — is that you have to find out where your value is going, because the fifteen people I asked all had a different answer, and every one of them was right about their own org.
