Key takeaways
- When you automate a step, you also remove whoever was reading it on the way past. That check has no owner, no name and no line in any process document, and it never showed up as a cost because the company was already paying for the work it rode along with.
- Why now: what became automatable is the two ends, reading inputs nobody had tidied up and writing the response. The forecast in the middle was never the hard part. Those two ends were the human step, and the reading rode along with them.
- An afternoon of assembly was never only one job. The assembly was waste. The reading that happened alongside it was not, and only the assembly was ever costed.
- The failure is quiet. Nothing throws an error, because nothing is wrong in the way software understands wrong, and the output that should not have gone looks exactly like the ones that should.
- A system reading its own reporting can tell you it ran. It cannot tell you it was right.
- The reps still review what the system drafts, so a human is in front of the output. But reading a finished draft is not the same act as building one from raw permits: our read is that a draft arriving fluent and usually correct is exactly when people stop reading and start approving.
- So the replacement has to be bought on purpose. Alongside the build cost, ask for a named owner, a separate first-year upkeep figure, and a named person who checks output against something other than the system's own account of itself.
- Make the check specific enough to survive, because a vague one evaporates the same way the old one did. A fixed monthly sample, read against the raw inputs, by someone not measured on throughput, with what was wrong written down.
- The harder case is not the proposal on your desk. It is the system bought two years ago that works well enough that nobody looks, where an unpriced check has had the longest time to be missing. Nobody raises that one, because raising it questions something already counted as a success.
In this article
When a company automates a piece of work, it records what that work produced. It almost never records who was looking at it while they did it.
McKinsey’s “Growth favors the bold: AI as force multiplier”, published on 6 August 2026, describes an industrial distributor whose reps used to find work by hand. The company built a system that reads building permits alongside project activity, its own customer history and its product data, predicts where demand is likely to emerge, and drafts the outreach a rep reviews, edits and sends. McKinsey puts the result at an estimated $300 million to $400 million in incremental gross margin, and both hedges there are McKinsey’s own: they call it an estimate, and they don’t name the client.
What actually changed
Scoring which customers are likely to buy is decades old. Matching permits against a customer list is a database job. If that were all this was, it would be a 2010 project wearing a 2026 label.
What changed, as I read the architecture, sits at the two ends. The system takes public records nobody has tidied up and makes them usable without a person in between, and then it writes the outreach. That’s the AI part, and it isn’t the forecast.
Which raises the obvious question of why a 2010 capability is being credited with $300 to $400 million now. The scoring was always possible. What it never had was a way to be fed without a person in the middle, or acted on without one at the other end. The ends were the constraint, not the middle, and that is why the money shows up in 2026.
The part that was never in the spec
Both of those steps used to be somebody’s afternoon, and an afternoon is never only one job.
While a person was pulling the material together, they were also reading it. Not reviewing it, not signing anything, just seeing it, and noticing the thing that was obviously wrong before it went any further. A permit for a site your own records say was finished last year. A customer three months into a dispute with you. Not subtle, and hard to miss when the material is passing through your hands. Nobody called that a control, and nobody priced it either. That unpriced gap is the one we traced through three consultancy reports that shipped with fabricated citations. It had no owner, no name and no line in a process document, and it cost nothing extra, because the company was already paying for the assembly and the reading came along with it.
Someone will object that the reps still read something, and they’re right. McKinsey says the system was wired into the CRM so reps could review, edit and send the emails, so there’s a human in front of the output.
But reading a finished draft is not the same act as building one from raw permits. My read is that the draft arriving fluent, complete and usually correct is the exact condition under which people stop reading and start approving. What went isn’t the human. It’s the contact with the raw material, and the raw material is where the odd thing used to show up.
Why you won’t notice it’s gone
The failure is quiet. Nothing throws an error, because nothing is wrong in the way software understands wrong. Every input is a real record, and the message that shouldn’t have gone out looks exactly like the ones that should.
If you check at all, you’ll probably check the way the system invites you to, by reading its own reporting. That can tell you it ran. It can’t tell you it was right, and a system grading its own homework is a loop, not a control.
What to ask for before you sign
Most proposals arrive with a build cost attached. Ask for three more things: a named owner, a separate first-year upkeep figure, and a named person who checks the output against something other than the system’s own account of itself.
Push hardest on the last two, because neither tends to be offered. Upkeep goes unmentioned because the sources feeding the thing change format and break quietly, months after anyone is still watching. The check goes unmentioned because at the point of sale the system is being sold to you on removing the very work the check depends on.
Then make the check specific enough to survive, because a vague one evaporates the same way the old one did. Pick a number and hold it. Say twenty real outputs a month, pulled at random and read against the raw inputs they came from, by someone who isn’t measured on the system’s throughput. The number matters less than the fact that it never moves. Write down what was wrong. Put that number somewhere a person has to look at it.
Settle all of that while the vendor still wants the deal. It’s far harder to add once people already trust the tool.
And the harder case isn’t the proposal on your desk. It’s the system you bought two years ago, running well enough that nobody looks at it, which is where an unpriced check has had the longest time to be missing. Nobody will raise that one, because raising it means asking whether something already counted as a success has been quietly wrong for a while. You can find out this month, with the same twenty outputs and the same person.
Whose decision this is
None of this is an IT decision. Who checks the judgment, and what counts as independent, belong to whoever owns the work. Get that backwards and you get a tool bought and called a redesign of the work.
So the question isn’t what your new system produces. It’s who used to read the old version on their way past, and whether anybody thought to tell them they’d stopped.
Questions this article gets
Our people already review the output before it goes. Isn't that the check?
It is if it survives contact with volume. The check that disappears is the one that happened because somebody was already handling the material and noticed something odd. A review step added on top of a system that produces far more than a person used to tends to become approval rather than reading, especially when the output is usually right. Ask what would have to be visibly wrong for someone to stop it, and how often that has actually happened.
How do we make a check independent without hiring someone?
Independent means the checker did not build the thing and is not measured on its throughput. In practice that is usually a sample of real outputs, read by someone from the side of the business that lives with the consequences, on a rhythm that somebody owns by name. It is unglamorous and it is small. The failure mode is not that companies choose a weak check, it is that they never name one.
What usually goes wrong?
The check gets added late, after people already trust the system. At that point it reads as a vote of no confidence in something that appears to be working, and it competes with the efficiency case the tool was bought on. The cheap version then fails quietly: the system reports on its own output, the dashboard looks healthy, and nobody notices it has been surfacing the easy calls.