What Juniors Keep When the AI Is Gone

Look at the work your junior people have handed in since they started using AI. It’s probably cleaner than it was a year ago, better organized, with fewer obvious gaps, and it arrives faster. It’s easy to read that as growth, and in a new field trial, the juniors read it that way themselves.

The trial points to a simple rule. Work done with AI measures the person and the tool together. What the person can do once the tool is closed is a second number, and it doesn’t always move with the first.

To measure that second number, the trial tested the skill AI output demands: looking at a draft a machine wrote and knowing what’s wrong with it. As the trial’s authors put it, “Modern AI systems are easy to operate but hard to evaluate.”

What happened when the AI was taken away

The trial ran for three months at eleven US intellectual property law firms (NBER Working Paper 35720, Autor et al., September 2026). It randomly split 133 practicing patent lawyers into two groups. About two in three got an unreleased AI drafting assistant from Google Labs that could write passages, rewrite selected text and critique a draft, and the rest worked without it. Patent attorneys at an independent firm scored every piece of work without knowing who had used the tool.

With the AI, the work got better, at ten days and again at ninety, and the juniors (under seven years of experience) gained the most. At both check-ins, juniors using AI scored at or above seniors working without it.

At ninety days, everyone also did a task with no AI allowed: take a flawed, AI-written patent draft and mark it up, the kind of review patent lawyers routinely do on their own. The group that had spent three months with the tool did better overall, but the whole of that advantage over the no-AI group came from its seniors. The juniors, a group the authors call relatively small, showed no average gain. Their scores spread out instead, with sharply fewer middling results, more poor ones and more good ones.

The researchers sum it up in one line: “the largest gains from AI … accrued to the lawyers who retained the least.”

What the seniors did differently

To look for a reason, the researchers had an AI model help review how each lawyer marked up that flawed draft, and they report what it found as observations, not measured results. Seniors who’d had the tool tended to restructure the draft to deal with its biggest legal risks, and they explained their edits in writing. Juniors worked through it in order, mostly on style and administrative fixes, and frequently spotted serious problems without fixing them, leaving short notes that read like instructions to an AI assistant that wasn’t there.

The researchers found the same spot-it-but-don’t-fix-it habit among juniors who never had the tool, so it looks like where juniors start. Their reading is that, as a group, three months with AI didn’t move juniors past it. They add a caution of their own: there were relatively few juniors in the study, which makes it hard to compare juniors who had the tool with juniors who didn’t.

Speed tells a similar story. Juniors with the tool worked faster on the drafting tasks, while seniors didn’t get faster with it, and the seniors are the ones who came out better on their own. The paper says this is consistent with the time people put in being part of how skill gets built, and that the trial can’t prove it.

The paper also separates two goals that are easy to confuse: raising measured performance and building expertise. Its tentative reading of this result is that the two goals pull against each other for people with limited experience and work together for people with seasoned judgment.

And, the paper notes, neither the AI-assisted scores nor the lawyers’ own assessment of their work would have forecast what happened. The juniors believed the tool was making them better, and in the work they handed in, it was.

The authors place the finding next to earlier studies in which AI narrowed the gap between newer and more experienced workers, a lift that can make talent signals noisier. This trial goes one step further and measures what’s left of the lift when the tool is closed.

A check you can run, in four steps

Running it takes one senior person per team, one prepared draft a quarter, about an hour of each junior’s time, and the senior’s time to score the checks and talk to anyone they flag.

1. Keep two numbers. Quality with AI and ability without it are separate measures. Anything that looks at work done with the tool, whether it’s a performance review, an output dashboard or the person’s own sense of progress, is reporting the first. Treat an improvement there as a fact about the person and the tool together until the check in step 2 shows the person improving too.

2. Once a quarter, check each junior’s judgment without AI. The trial ran its test without AI once, after about three months, so that’s the only interval it tested. Repeating it every quarter is this article’s choice, and so are the numbers below.

Copy the study’s shape. Have AI draft the kind of work your team really does, a proposal, a contract clause or an analysis, as a practice piece that never goes out. One draft can serve every junior on the team that quarter.

Before the junior sees it, the senior makes sure the draft holds exactly three big problems, meaning problems that would cost the business if it went out as it is, such as a wrong figure, a missing condition or a promise the business can’t keep. If the draft has fewer than three, the senior plants the rest, and if it has more, fixes the extras first. Small issues of style don’t count. To keep one quarter comparable with the next, pick three kinds of problem once and make sure each fresh draft holds one of each. The junior fixes the draft with no AI and writes one line per change saying why. In the trial, lawyers spent about an hour on the review.

The senior marks each big problem as repaired, only flagged or missed, and the score is the number repaired, from 0 to 3. The repaired-or-flagged lens comes from the researchers’ observations, in which juniors often spotted serious problems without fixing them. Name that senior as the owner, because checking AI-written work is a job, and like any verification step nobody has priced, it needs a named person rather than a slot in the process.

3. Read the results person by person, and over time. On average the trial’s juniors did no better than the no-AI group, while their scores spread out, which the authors say is consistent with the tool helping some juniors learn and standing in for learning in others. An average across your team hides exactly that spread. One check can’t sort people on its own either: in the researchers’ observations, spotting problems without fixing them was common among juniors with or without the tool, so a low first score may simply be where someone starts. Read each junior’s scores as a line over time, not against colleagues.

Treat the first check as a starting point that doesn’t count toward any flag. What deserves a closer look is a line that stays at the bottom: no more than one of the three big problems repaired in each of two checks in a row after the first. So the earliest flag comes two quarters after the first check.

A junior who repairs all three gives the check nothing to flag, and a rising score is welcome but doesn’t prove the tool helped, since a quarter’s experience can lift people too. Even a low line proves nothing on its own. The trial can’t say why the tool seemed to help some juniors and not others, so the check only tells the senior where to look. The authors name two things their trial didn’t have and a future one should: a starting test, which would have measured skill directly, and repeated tests without AI over a longer period, which would show whether juniors’ results come back together or keep spreading.

The closer look is short. Two low checks in a row already make one hard draft an unlikely explanation. The senior sits with the junior, goes through the problems they only flagged or missed, and asks how they would fix each one. The conversation shows the senior which kinds of problem to practice first, and step 4 is for that junior. If most juniors on a team stay low together, that’s worth noticing too, since it may say more about how the team works with AI than about any one person.

4. For juniors whose scores stay low, give the saved time back to judgment. The same senior owns this. For those juniors, keep their workload from growing just because AI makes them faster, and put part of the time AI saves them into reviewing and fixing the team’s everyday AI drafts by hand, starting with the kinds of problem they missed or only flagged. This step has the least behind it, so treat it as a cheap hedge rather than a cure.

The support is indirect. The researchers suggest that relying on AI to make the edits may fail to correct juniors’ habit of flagging problems without fixing them. Seniors didn’t get faster with the tool, yet at ninety days those who’d had it did better on the task without AI than seniors who never had it, and in the authors’ summary of earlier studies, setups where people have to interpret, evaluate or verify AI output tend to build skill, while ones that let them accept it uncritically skip the effort learning needs.

The order matters. Keeping two numbers comes first, because the gap may not show up in the measures you already have. The check comes next, because nothing after it can be decided without it. Reading person by person, and over time, comes before any change, because an average and a single reading both hide the spread. Step 4 comes last, and only for juniors whose scores stay low, because it’s the least proven step.

Where this evidence stops

The evidence comes from one trial, in one profession, with one tool. Of the 133 lawyers, 91 finished every task, and the junior result rests on a smaller group still, which the authors flag themselves. The trial’s juniors had less than seven years of experience, which may be a wider band than your company means by junior. Every firm does ongoing patent-drafting work for Google, so they aren’t typical of professional services, and three months is short against the years it takes to build patent expertise. Google built the tool and funded the study, six of the seven authors work there, and the paper is a working paper that hasn’t been peer-reviewed. There was no skills test at the start, so the comparison rests on the random split.

Other research doesn’t all point the same way. The study the authors call closest to theirs found that law students who used AI early did better afterward on tasks without it.

Two things matter most for how you use this. The paper doesn’t know why the tool seemed to help some juniors and not others. And on the one task it tested without AI, after three months, it found no measurable average harm. So the check tells you where to look, and the trial is no case for taking the tool away from your juniors.

The one takeaway the authors call unambiguous is about measurement rather than about lawyers: knowing whether AI builds skill takes a test that separates what the software contributes from what the person knows. A growing body of research measures work done with the tool. The authors call the question of what people learn from it underexplored, and they describe theirs as the first study in law to measure it with practicing lawyers rather than students.