Two weeks ago, KPMG and researchers at the University of Texas at Austin published a field study of 523 early-career professionals working with AI agents. What makes it useful is the method. They ran the work through the AI first with no human involved and kept that as the baseline. Then they measured people against it. Half beat the machine. A quarter matched it. And 24.1% fell below it.
Why It Matters
That last group is what matters. A quarter handed back work worse than what the AI produced alone, and nothing on their record would have told you. KPMG puts it plainly. Traditional measures of capability did not explain the gap, and those who outperformed the AI looked nearly identical on paper to those who did not. The below-baseline group scored higher on critical thinking, domain knowledge and AI literacy than the group that matched the machine, and level with the group that beat it. What separated them was not what they knew. It was how they directed the machine, and a review of the finished document cannot see that.
The Decision
The question worth asking is what your reviews of junior work measure now that the deliverable is partly the machine’s. A reasonable CEO could call this a coaching problem and leave the process alone. Or decide that no review means much until you know what the work looks like without the person in it.
What To Do This Week
- Pick one recurring junior deliverable. Ask its owner to run it through your AI with no human involved and keep the output. That is your baseline.
- Ask that owner to hold the last three human versions against it and answer one question. What did the person add?
- Ask one manager which of their people direct the AI and which accept its first answer. If nobody knows, start there.
What Not To Do
Do not buy AI literacy training as the answer. AI literacy was one of the three capabilities measured, and it did not predict who beat the machine. Do not read this as a case against junior hiring. Half beat the machine outright, and KPMG says the gap is in how they applied what they knew, which is coachable. And do not keep grading the deliverable alone. A good document no longer proves good judgment.
Signal Boost
KPMG and UT Austin, Shaping Early-Career Success in the Age of AI - the university’s own write-up, free to read. Worth it for what the top group actually did, which was treat the AI as a collaborator needing direction, oversight and judgment.