Insight 20 October 2026 · updated 14 October 2026

AI hallucinations: 5 tasks to keep in a health claim dossier

AI hallucinations and health claims: invented references, reworded claims, exposed trial data. Five dossier tasks AI must not do alone, and those it speeds up.

Nicolas Rousseau

In brief: In 2023, 47% of the references ChatGPT-3.5 produced for medical texts were fabricated, 46% were real but inaccurate, and only 7% were real and accurate [1]. Another study found 55% fabricated references for GPT-3.5 and 18% for GPT-4 [2]. Models have improved since, but in a health claim dossier, a single fake reference is enough to wreck the credibility of the whole thing. The question is not whether to use AI. It is which tasks to keep away from it.

Both studies looked at 2023 models queried without access to a document base. Today’s tools, connected to PubMed or to your own documents, probably do better, but we found no comparable recent measurement. What has not changed: when a model gets it wrong, it does so with exactly the same confidence. In a blog post, the error costs you a correction. In a dossier submitted to EFSA or sent to an industrial customer, it costs you their trust.

Why a model invents a reference

A language model does not consult a library. It produces the most plausible next words. And a bibliographic reference has a very regular shape: authors, a title, a journal, a year, pages. That makes it easy to imitate. The result looks exactly like a citation and has no guaranteed link to a real paper. This is what is called a hallucination.

The problem does not stop at invented references. In the Scientific Reports study, 43% of GPT-3.5’s real references and 24% of GPT-4’s contained substantive errors [2]. A reference that exists but carries the wrong year, journal or authors will pass a quick check. It will not pass an assessor’s.

Five tasks to keep in human hands

1. Choosing and reading the pivotal studies. AI can suggest a list. It cannot guarantee that each reference exists, that it says what you claim it says, or that it covers your strain, your dose and your population. Every study cited must have been opened and read by someone who signs off on it.

2. Writing the wording of a claim. Regulation (EC) No 1924/2006 tightly governs claims [3], and authorised health claims come with a set wording and conditions of use [4]. A language model rewords by nature. Ask it to “improve” an authorised wording and it will give you an unauthorised claim that reads more smoothly.

3. Extrapolating from one strain to another, or one dose to another. This is the most tempting shortcut and the most dangerous. A model will readily connect similar results. The dossier demands an exact match between the ingredient, the dose and the demonstrated effect.

4. Handling unpublished trial data. Health data from trial participants falls under the special categories of the GDPR [5], and pseudonymised data is still personal data. Pasting it into a consumer assistant means passing it to a third party. Check where the data goes before you open the tool.

5. Taking responsibility. The food business operator must be able to justify its claims [3]. No tool signs on its behalf. If nobody can explain where a sentence comes from, it has no place in the dossier.

What AI does well (and faster than you)

Task	Delegate?	Condition
Formatting the bibliography	Yes	References come from a verified list, not from the model
Consistency checks across documents	Yes	The check flags, a human decides
First screening of a literature search	Yes	Excluded studies remain accessible
Draft translation	Yes	Reviewed by a speaker who knows regulatory vocabulary
Checking structure against the EFSA guidance [6]	Yes	The guidance is given to the tool, not summarised from memory
Choosing pivotal studies	No
Claim wording	No
Raw trial data in a consumer tool	No

The common rule: AI works on materials you supply and have verified. It does not supply the materials.

Checking a reference: four steps

There is nothing sophisticated about verification. It takes discipline, reference by reference.

Open the reference by its identifier (DOI or PubMed ID), never by searching for the title. A plausible title always brings something back.
Compare the sentence in the dossier with the paper: ingredient or strain, dose, population, primary endpoint. All four must match.
Check the status of the publication: corrected, retracted, or followed by a comment that limits its scope.
Record who checked, and when. Without that trace, the check will have to be redone at the next doubt.

It takes time. It is also the one check no tool can certify on your behalf.

The one-page rule, heading by heading

The internal rule does not need to be long. Six headings are enough.

Approved tools, named, with their privacy settings.
Prohibited data: unpublished trial data, personal data, customer documents covered by a confidentiality agreement.
Permitted tasks: those in the table above, and no others without agreement.
Prohibited tasks: choosing studies, claim wordings, extrapolations.
Named review: who signs off each section of the dossier.
Traceability: passages prepared with a tool are marked in the working document.

The framework that is coming anyway

Since 2 August 2026, Article 50 of the EU AI Act [7] has imposed transparency obligations on certain AI-generated texts. There is an exception when the content has undergone human review and a person holds editorial responsibility for it. We covered this in AI Act and health content: the exception that saves your editorial line. For a health claim dossier, that human review is not a formality: it is what makes the work valid.

What this means for your teams

For your regulatory team, the job is to write the rule before everyone invents their own. One page is enough: approved tools, prohibited data, steps that require a named reviewer.

For your marketing team, it draws a clear line. AI speeds up everything to do with form. Substance (the study, the wording, the dose) remains a documented human decision.

Does your team have a written rule today on what can be pasted into an AI assistant?

Sources

FAQ

Do recent models still invent references?

Probably less often than the 2023 models, especially when they are connected to a document base. The risk is not zero, and a model will not tell you on its own that a reference is invented. Every reference must be opened and checked.

Can we use an AI assistant connected to our own documents?

Yes, that is the safest set-up: the assistant answers from your sources and cites the passage it used. Check where the data is hosted and whether it is used to train the model.

Do you have to declare AI use in a dossier submitted to EFSA?

Check the requirements in force at the time of submission. What does not change: every piece of data must be traceable to its source and every statement must be justifiable by the applicant.

Can AI check that a claim matches the authorised wording?

It can compare a text against the exact wording you give it and flag discrepancies. It must not propose a rewording in place of the authorised wording.

On the reliability of AI engines as a source, see when AI blurs the trail. And to make scientific content citable by those engines, our approach to AI visibility (GEO).