Exclusive: Guardian analysis reveals that dozens of policy submissions from across the political spectrum incorrectly summarise real research and invent or wrongly cite sources
Follow our Australia news live blog for latest updates
Get our breaking news email, free app or daily news podcast
Australia’s government inquiry process is supposed to help parliament make better decisions, hearing from experts and constituents alike. But Guardian Australia can reveal that the system is being flooded with AI-generated material, which is inventing studies and attributing nonexistent research to real academics and authors.
In some cases, committee reports have cited submissions in which the majority of sources appear to be AI-generated “hallucinations”, when large language models (LLMs) invent content that looks real but doesn’t actually exist.
An AI hallucination takes place when an LLM like ChatGPT or Claude outputs text that is plausible but incorrect, or confidently generates information that appears relevant but is actually unrelated.
This happens because these AI systems use complex statistics to figure out the text they generate and they have no innate ability to distinguish between correct and incorrect content. They simply predict the next most likely bit of text based on their training data, with a certain degree of creativity or randomness involved.
Hallucinations can become more common when the AI is asked to generate content on topics that are not well covered in its training data.
These hallucinations are not bugs and although the frequency can be reduced by increasing the amount of training data or adding web-search results and other information, they are an inherent feature of LLMs and can never be fully eliminated.
If the phenomenon continues, there is a significant risk of parliamentarians “making decisions based on evidence that doesn’t exist”, says Christian Downie, a professor in the Australian National University’s school of regulation and global governance.
The situation is made more confusing by AI services including Google search. Previously, looking up an invented reference might produce results that would indicate the reference did not exist. Now Google’s AI summary will sometimes create summaries of a fake reference as though it were genuine.
In some instances, Google’s AI summary and ChatGPT will also cite the inquiry submission with the fake reference as a source, creating an ongoing cycle of misinformation.
One document submitted to an inquiry into family violence and suicide included a hallucinated reference attributed to Divna Haslam, a University of Queensland associate professor and clinical psychologist, and misstated her team’s research findings.
Haslam, who researches child and family adversity and maltreatment, says the reference was “scary” because it looked so real that someone taking a cursory look could be convinced. Google’s AI summary summarised the reference as if it were a real paper.
“It’s very frustrating to … to have invested time, money, effort, expertise in rigorous research,” she says. “Then to see something that’s inaccurate and inappropriately attributed anyway is really concerning.”
