← All articles Part of: AI Strategy & ROI

The Answer You Should Have Left Blank

6 min read
A man in a suit, seen from behind, presses a round buzzer on a quiz podium while facing a large bulb-framed screen that reads Artificial Intelligence, with three small cartoon robot heads around the words.

Key takeaways

  • Having an AI answer in view changes whether you say "not yet" at all. In five experiments built on questions the AI almost always got wrong, people with AI advice at hand almost stopped saying "I don't know". Where answers were graded, they got fewer right, and in the study that asked, they also felt far surer.
  • For questions where a wrong answer costs something, write one line before you take in any answer: "I know", "I don't know" or "I'd need X". Let the AI change that line only if it gives you something you can check, and you check it.
  • Make "not yet, missing X, by a date" an accepted answer in your meetings, and write down the answers to decisions that matter with a date to check them. The steps are our design. The one fix the researchers tested, small cash stakes for accuracy, helped only partly.

Someone in the meeting asks a question nobody has the number for. What would it cost, all in, to move the support team to the new vendor?

A few years ago the honest answer was “I don’t know yet, give me until Thursday.” Now there’s a laptop open, and ten seconds later there’s a figure on the screen, stated cleanly, with its reasons attached.

Most of the worry about that moment is whether the figure is right. A new set of experiments points at something that happens a step earlier: whether anyone still says “not yet” at all.

Three researchers in Paris, Rome and Milan ran five experiments with 3,132 people and found that “merely having access to AI nearly eliminated participants’ willingness to suspend judgment” (Marcoccia, Quattrociocchi and Capraro, arXiv, July 2026). People answered more questions and, where it was measured, got fewer of them right and felt much surer of the answers they gave.

“Not yet, I need X” is also an answer, and when a decision hangs on it, it’s often the right one.

Film details the AI got wrong

The design is what makes the result hard to wave away. Leaning on AI advice is usually sensible, because the advice is usually good, so in most studies it’s hard to tell sensible delegation from lost judgment.

The researchers took that ambiguity out. They wrote six questions about small visual details in films, such as the color of the team’s uniform in Bend It Like Beckham, details rare enough online that a language model would likely make something up. The model they used, StepFun’s Step 3.5 Flash, got them wrong “almost without exception.”

Every participant could decline any question. A blank, “I don’t know” or “I’m not sure” counted as holding back, and even a hedged guess like “green?” counted as an answer.

In the first study, people without AI held back on 36% of the questions on average, while people who could ask the AI held back on 6%. A repeat with pre-recorded AI answers found 44% against 3%.

Then the researchers took away the choice of asking. In a later study, the AI’s answer simply appeared next to each question, the way a summary sits at the top of a search page, and without money at stake, holding back fell from 35% to 1%.

In the three studies that graded the answers, people with AI access and no money at stake got 9.2% of the questions right, against 27.5% without it. In the one study that asked about confidence, people with AI and no money at stake rated theirs at 75.9 out of 100 on average, against 29.6 without AI, about two and a half times as high.

Why a wrong answer felt right

The authors’ reading is that a fluent answer in view lowers the bar at which a person decides they know enough to answer. A language model, they note, has to produce something every time, and it doesn’t hold back on its own. Lean on it, and you can take on that habit.

Two details make the result stranger than it first looks. People normally underweight advice: earlier research the authors cite finds people move only about a third of the way toward an advisor’s view. Here, the authors note, people did the opposite.

And advice that clashes with your own hunch would normally make you less sure. Instead, people with the AI were far surer than people without it. Because the AI group answered more questions yet got fewer right, the authors conclude the advice must have turned some answers people knew, or could have reached, into errors.

It’s a different problem from a model that sounds too sure of itself, which we’ve looked at as a boardroom risk of its own. Here the problem sits in the person, in the moment they decide whether they know enough to speak.

The limit of a quiz

The questions were trivia, built so the AI would be wrong. The authors say plainly that it’s “untested whether suspension collapses to the same degree when AI advice is usually correct.”

The participants were people recruited and paid online, US-based in four of the five studies, answering questions about films rather than making decisions with real-world consequences. The paper is a preprint, posted on arXiv with no journal publication listed.

So the study doesn’t show that AI makes you wrong in general. When the AI is usually right, answering more often may be exactly what you want. What it does show is narrower, and for a CEO more useful: an answer in view changes the decision to answer at all, and in the one study that measured confidence, it climbed even when most of the answers were wrong.

What a dime at stake did

The researchers did test one fix. In some conditions people earned $0.10 for each right answer and lost $0.10 for each wrong one, while holding back paid nothing.

It helped, partly. With money at stake, people asked the AI less often (5.27 of the six questions without stakes, 4.53 with them, in one study), and among people with AI, those with money at stake got more answers right.

In that study, holding back among people with AI rose from 2% to 8% of the questions, but that stayed far below the 41% among people with no AI and the same money at stake. People with AI also still got fewer of the questions right, 16% against 27%.

The authors read this as a change in “willingness, not capacity.” People can still call on their own judgment when it matters to them, yet a small cash cost for being wrong brought back only part of it, and bigger or reputational stakes weren’t tested.

Write your line first

The steps below are ours, built on the study’s findings, and the study didn’t test them. The first three are a habit you can start tomorrow, without anyone’s permission. The fourth carries it to your team, which won’t take it up unless “not yet” is safe to say there.

  1. Pick the questions that matter. A wrong answer costs something, you’ll act on it, and you can afford to wait a day: a number going to the board, an answer to a client, a go or no-go on a vendor. Where the AI is usually right and a mistake is cheap, deferring to it is reasonable, so leave those alone.
  2. Write one line before you take in any answer. “I know: ___.” “I don’t know.” Or “I’d need X to know.” Write it the moment the question comes up, before anyone opens a laptop, and if a tool has already put a summary up unasked, write it before you read that. In the study an answer that simply appeared had much the same effect as one people asked for, so the line has to exist before you’ve taken the answer in.
  3. Let the AI change your line only if it gives you something you can check, and you check it. Open the source, trace the figure, redo the calculation. If it can’t be checked, or you haven’t, your line stands, and a “don’t know” or “need X” stays “not yet, missing X, by Thursday.” Feeling surer doesn’t count, because in the study confidence rose sharply while accuracy fell.
  4. Make “not yet” a normal answer in your room. Say it yourself first, accept it from others without a penalty, and write down the answers to decisions that matter with a date to check them. The date gives being wrong a cost you’ll notice, a stake of your own, though whether it works like the dime is untested, and the dime itself helped only partly.

After a week, count the lines that said “don’t know” or “need X” where the answer went out anyway with nothing new checked. That count is your own reading of the effect, in your own work.

But “not yet” has a cost of its own. A leader who never answers stalls the company, so the habit is for questions where waiting a day is affordable. The wider case for using AI less is a separate argument, and this is a check on a handful of questions.

Which answers your room accepts

The method leaves you with two decisions.

The first is which questions in your company used to get “let me check” and now get an AI answer within seconds, where a wrong answer would cost something. That’s your list for step 1.

The second is whether “not yet, missing X” is something people in your meetings can say without looking unprepared. If it isn’t, nobody on your team will say it, however good the habit sounds.

The tools will keep getting better at always having an answer. Keeping “not yet” on the table is the part that stays a leadership decision.

Questions this article gets

Does the study show that AI makes people wrong?

Not in general. The researchers built quiz questions the AI almost always got wrong, so they could separate leaning on the AI from the AI being right. They say it is untested whether people stop saying "I don't know" to the same degree when the AI is usually correct. What the study shows is that having an AI answer available changes the decision to answer at all, and, in the one study that measured it, confidence rose even when most answers were wrong.

What counted as saying "I don't know" in the experiments?

Every participant could decline any of the six questions. A blank, "I don't know", "I'm not sure" or a non-answer counted as holding back. Any real attempt at an answer, even a hedged guess like "green?", counted as answering.

Why write my own line before reading the AI's answer?

Because in the study the effect held even when the AI's answer simply appeared next to the question without being asked for. Once a fluent answer is in view, the authors suggest, the bar for "I know enough" has already moved. That's why the line has to come first. The four steps are our design, built on that finding, and the study didn't test them.

Article Governance 6 min read

Your Next Customer May Arrive as an AI Agent

Amazon blocked Meta's shopping agent, and Meta and Sierra proposed rules for letting agents in. Choose one task a customer's agent may do, and its limits.

Continue reading →

Read the original post on LinkedIn →