AI is revolutionising information retrieval… but at what price?
Artificial intelligence (AI) search engines promise rapid, synthetic access to information. Gemini, Perplexity, ChatGPT and other generative search tools are positioning themselves as alternatives to traditional search engines. However, a study by Columbia Journalism Review (CJR) revealed a major flaw: these tools struggle to cite their sources correctly. By analysing eight AI engines, the researchers found that they often misattribute news content, sometimes produce non-existent links and rely on syndicated versions of articles, making verification difficult. This study raises a crucial question: how can these tools be integrated into a content strategy without sacrificing the rigour and transparency of the information?
A study on the reliability of quotations from AI engines
CJR’s Tow Center for Digital Journalism evaluated eight AI search engines to examine their ability to correctly cite news sources.
Methodology
- 1,600 requests submitted to these tools
- Analysis of 10 articles from 20 different publishers
- Check that citations are accurate (name of article, publisher and correct URL)
The results are worrying, and highlight major shortcomings in sourcing.
Massive errors and unclear sources
Highly variable accuracy
The study reveals that the majority of AI engines tested produce incorrect answers in more than 60 % of cases. Some tools stand out for their blatant lack of reliability:
- Perplexity: 37 % errors
- Google Gemini: 54 % errors
- Grok 3: 94 % errors
Approximate and sometimes invented quotes
- The engines tested generated incorrect citations and non-existent links.
- Some linked to syndicated versions of articles, making it difficult to identify the original sources.
Overconfidence in the answers
- ChatGPT provided 134 incorrect answers out of 200, but only expressed a doubt in 15 cases.
- None of the chatbots tested refused to respond, even when the information was clearly incorrect or impossible to verify.
Why are these gaps a problem?
A direct impact on the credibility of information
If AI engines cannot guarantee correct citation of sources, their reliability as a research tool is called into question. The average user, reluctant to check every reference, runs the risk of absorbing misattributed or truncated information.
A lack of recognition for content publishers
The media and news publishers invest time and resources in producing reliable information. If AI engines exploit this content without properly crediting it, they deprive these players of legitimate traffic and weaken the information ecosystem.
An increased risk of misinformation
When sources are misattributed or quotes are invented, the risk of disseminating false information increases. These errors, amplified by AI, can undermine understanding of current affairs and fuel mistrust of the media.
AI: a powerful but imperfect tool
This study is a reminder of a fundamental principle: AI, no matter how good it is, is not an option, is no substitute for human interpretation and a well thought-out content strategy, in marketing, for example. So you have to separate the wheat from the chaff.
What AI can do for you
- Rapid analysis and efficient synthesis of information
- The ability to process large volumes of data in a short space of time
- A tool to help you find and structure content
What AI does not replace
- Human expertise in assessing and prioritising sources
- The ability to contextualise and verify information
- A well thought-out editorial strategy, based on a solid structure and co-creation with experts, both inside and outside the company.
Conclusion: AI, a lever to be used with discernment
AI search engines offer interesting prospects for accessing information, but their inability to properly cite sources is a major limitation. Blind use of these tools can undermine the credibility of content and fuel misinformation.
AI should be seen as a assistant, and not as a reliable source in its own right. Rigorous human supervision remains essential to guarantee the quality and accuracy of the content. At Nutrimedia, we are convinced that any content strategy must be structured, sourced and validated by experts.
FAQ — Frequently asked questions
Why do AI search engines cite their sources poorly?
The Columbia Journalism Review (Tow Center for Digital Journalism) study, run across eight engines and 1,600 queries, found that most produce wrong answers in more than 60% of cases: incorrect citations, fabricated links, and redirects to syndicated versions that hide the original source. The chatbots answer even when the information is unverifiable, instead of declining.
Which AI engine is most reliable at citing sources?
In this study, Perplexity performed best with 37% errors, ahead of Google Gemini (54%) and Grok 3 (94%). None reaches a level that removes the need to verify: even the best is wrong more than one time in three.
Can an AI engine be used as a reliable source of information?
No. AI should be treated as a research assistant, not a source. ChatGPT gave 134 incorrect answers out of 200 while expressing doubt in only 15 cases: displayed confidence does not guarantee accuracy. Human oversight remains essential.
How does a brand stay credible in the face of generative AI?
By structuring its content, sourcing every claim and having experts validate it. Solid, traceable and correctly attributed content is more likely to be cited faithfully and holds up better against engines’ attribution errors. It is also the first test when choosing a B2B nutrition agency.
Have you ever noticed misquotes in the answers provided by AI engines? Share your experience in the comments.