Say an analyst recommends a supplier. The numbers check out, but one choice shaped the recommendation before it reached you: how much weight price gets against service reliability. Several weightings are defensible, and choosing among them is a business decision. A study published by Anthropic suggests that when an AI agent helps with an analysis, a choice like that may be the agent’s. Fifty experienced analysts used Anthropic’s Claude Code to assess a fictional farming co-op’s program, and on four choices with two defensible answers each, a coin flip secretly set which option the agent favored. They went with it on 86% of those choices, against 50% by chance.
Why It Matters
Afterward, 77% of the 48 surveyed repeated the agent’s argument in at least one answer. Only 15% credited the agent anywhere. In our supplier example, which the study didn’t test, you could check the data, approve the conclusion, and still miss that the weighting already settled part of the decision. The researchers note that checking for errors can’t catch that, because the work can look sound either way. The study’s limits: the researchers set the agent’s preference, deliberately strengthened it, and tested one fictional task with one model, so the effect in your company is unknown.
The Decision
The question is which choices you’re willing to let an AI-assisted analysis settle, and which need an accountable person to make and own them before you commit. Requiring that costs time on every decision it covers. Not requiring it means accepting that some choices may follow the agent’s preference.
What To Do This Week
- Ask your leadership team which upcoming, costly-to-reverse decisions rest on AI-assisted analysis.
- For one of them, ask the analyst for a short note on one choice that shaped the answer: what was chosen, the reasonable alternative, and what the alternative would change for the business. Have the decision’s owner pick an option and say why, before anyone commits.
- With your leadership team, agree which kinds of choice, like how criteria are weighted, need that note and an owner’s decision as standard.
What Not To Do
Don’t take a fluent explanation as proof the choice was the analyst’s. In the study, those who followed the agent on every choice were about as confident as the rest. And don’t tell your board the note solves this. It makes clear who owns a choice, but it doesn’t guarantee the thinking behind it was independent.
Signal Boost
How AI agents shape experts’ judgment calls, by Amaç Herdağdelen, Hailey Joren and Alex Tamkin, Anthropic, October 27, 2026. Read it before you set that standard.