A new preprint suggests that access to AI advice makes people far less willing to admit they don't know something — even when the advice on offer is wrong.
Researchers found that participants' willingness to say 'I don't know' collapsed from 44 per cent to just 3 per cent once AI assistance was introduced. Accuracy fell at the same time, while confidence in their own answers rose from roughly 30 out of 100 to 76 out of 100, about two and a half times higher.
The research, led by Valerio Capraro of the University of Milano-Bicocca alongside Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, ran five separate experiments involving 3,132 participants in total.
Four of those experiments were preregistered, meaning the researchers set out their hypotheses and methods before collecting data, and one was a direct replication. Throughout, participants answered difficult questions and were always given the option to decline rather than guess.
Crucially, the researchers deliberately designed the questions so that the AI advice on offer was wrong. That allowed them to examine whether access to AI advice changed people's willingness to suspend judgement, independently of whether the AI itself provided useful information.
To do this, the team used difficult visual trivia questions, including details from films that were largely absent from online text. This made it more likely that AI would produce incorrect answers. Examples included the colour of a football team's kit in Bend It Like Beckham and the vehicle driven by Monica in Like a Cat on a Highway.
The researchers also tested several large language models, including GPT-5.5, Claude 4.6 Sonnet and Gemini 3.5 Flash. While these models performed well on many of the questions, they consistently failed on the hardest question used in the experiment.
The effect held whether participants actively asked for AI advice or simply had it displayed to them without requesting it. The availability of AI advice was enough to shift behaviour. 'People became much worse, the accuracy was only one third, but they were twice as confident,' Capraro said.
Once AI was involved, participants answered more questions overall because fewer opted to decline. But their accuracy fell from 27.5 per cent without AI advice to 9.2 per cent when incorrect AI advice was available. Confidence in those answers, meanwhile, rose sharply, climbing from roughly 30 out of 100 to 76 out of 100.
The researchers also tested whether financial incentives for accuracy would counter the effect. It helped, but only partially. Participants who faced monetary incentives sought and followed AI advice less often, answered more accurately and suspended judgement more frequently than those without such incentives.
Even so, their willingness to suspend judgement remained well below the level recorded when AI advice was unavailable, suggesting that financial incentives did not eliminate the tendency to defer to AI.
AI advice makes people 3× less accurate and 2× more confident at the same time.Our new paper is receiving a lot of attention, perhaps not surprisingly, given how dramatic the results are. If you have not read the article yet, here’s a brief summary:When people receive AI… pic.twitter.com/SnjH4y3vnj
The findings sit within what the researchers describe as 'epistemia', the tendency to accept AI output because of its surface plausibility and linguistic coherence rather than because it has been independently verified.
The researchers argue that AI systems can produce fluent, plausible answers even when they are wrong, potentially encouraging people to treat a coherent response as evidence that they know enough to answer.
Their experiments suggest that access to such advice can lower the threshold at which people decide to suspend judgement, even when the advice itself is unreliable.
