Hallucination

It isn't lying, and it isn't malfunctioning

Both of those framings mislead. Lying requires knowing the truth and choosing otherwise. Malfunctioning implies a working state it has fallen away from. Neither applies.

A language model produces text by repeatedly answering one question: given everything so far, what plausibly comes next? That is the whole mechanism. There is no separate stage where a draft gets checked against a store of facts, because there is no store of facts to check it against. What the model has is a vast statistical sense of how language about the world tends to go.

Most of the time, plausible and true coincide. The most plausible completion of "the capital of France is" is "Paris" precisely because that is true and the truth is written down everywhere. The mechanism gets accuracy for free, as a side effect of pursuing plausibility.

But when the truth is obscure, absent, or contested, plausibility keeps working perfectly well without it. Asked for the title of a paper that doesn't exist, the model does exactly what it always does and produces a title that reads like a real one. Nothing has gone wrong internally. The same process that produced "Paris" produced the fake paper. That is the uncomfortable part: fluency and accuracy come out of one process, so you cannot have one without risking the other.

"Confabulation" is the better word

Hallucination is the term that stuck, but it's a poor metaphor: hallucinating means perceiving something that isn't there, and the model isn't perceiving anything. Clinicians have a more precise word for fluently and confidently filling a gap without any awareness that you're filling it, and that word is confabulation. It describes this behaviour far better.

Worth knowing, partly because it's more accurate and partly because it predicts the behaviour better. Confabulation is not random noise. It is coherent, contextually appropriate, and delivered with the same manner as everything else. That's exactly what makes it hard to catch.

Why it sounds so certain

The tone is the trap. A model's confident register is not evidence of internal confidence, because the register is just more generated text. It writes "the study found" the same way whether the study exists or not, since that's how sentences about studies are phrased.

There is genuine uncertainty in the machinery, but by default very little of it surfaces in the prose. Nothing forces a model to write "I'm about 60% sure of this," and quite a lot of its training rewards sounding helpful and fluent. So you get a uniform confident voice across a very wide range of actual reliability, which is the single most dangerous property of these systems for anyone using them for real work.

Where it happens most

This is the practical part, because the risk is not evenly spread. Raise your guard sharply on:

• Specific numbers. Statistics, prices, dates, measurements, scores. A plausible-looking number is trivially easy to generate and almost impossible to spot as fake.
• Citations, URLs and quotes. The worst category by far. These have highly regular formats, so a fabricated one looks structurally perfect. Invented papers with real-sounding authors in real journals are a classic, and dead or wrong URLs are routine.
• Obscure topics. The less written about something is, the thinner the statistical signal, and the more the model is effectively improvising in the right style.
• Anything recent. Past the training cutoff the model has nothing, but it will often answer anyway rather than decline. There's a subtlety here: vendors distinguish a training data cutoff from a reliable knowledge cutoff, the latter being earlier. Events near the boundary are in there thinly, which is arguably worse than absent, because thin coverage produces confident half-knowledge.
• Precise details about your own material. Ask which page a clause appears on and you may get a page number invented to satisfy the question.

The pressure problem

Here is a failure mode most people trip over without noticing. If you push back on a correct answer, a model will frequently abandon it.

Ask "are you sure? I thought it was 2019," and you have not requested a verification. You have supplied a strong hint about what completion you'd find plausible, and plausible-continuation is the only game being played. Agreement is often the most plausible next move. The model will apologise, adopt your date, and sound just as confident as before.

So "are you sure?" is close to useless as a checking tool, and worse than useless if you telegraph the answer you expect. If you must probe, keep it neutral and ask for the basis: what is that based on? A real source can be checked. A fabricated one usually falls apart when you go looking for it.

What actually reduces it

Supply the source material. The single most effective habit, by a distance. A model asked to summarise a document you have pasted in is doing a fundamentally more reliable task than one asked to recall the document from training. Move the work from recall to reading. This is the same insight as → Context windows, and it's why the two pages keep pointing at each other.

Ask for quotes, not summaries, when accuracy is critical. "Quote the sentence that says this" is checkable in seconds. "Summarise what it says about this" is not.

Give it a search tool, or use a product that has one. A model that can look things up and cite what it found is meaningfully more reliable on anything factual or recent. Still verify the citations, because a real search result can be attached to a claim it doesn't support.

Explicitly permit not knowing. Adding "if you're not confident, say so" or "say 'I don't know' rather than guessing" genuinely helps, because it makes admitting ignorance a plausible completion rather than an unhelpful-seeming one.

Ask it to work in the open. Requesting reasoning before the answer, rather than an answer alone, surfaces more of the path and gives you something to inspect.

What doesn't work

• Telling it not to hallucinate. "Do not make anything up" is a reasonable thing to write and does almost nothing, because the model has no reliable internal flag for which of its output is invented. If it could tell, this page wouldn't need to exist.
• Asking "are you sure?" See above. It measures agreeableness, not accuracy.
• Asking it to grade its own confidence. The number you get back is itself generated text, and treating it as a real probability is a category error.
• Assuming a newer or more expensive model has solved it. Better models confabulate less and hide it better, which does not straightforwardly make them safer to trust unverified.

The honest bottom line

This is reducible but not currently eliminable, and any tool promising otherwise is overselling. The workable posture is not distrust, which wastes the tool, but calibrated verification: match your checking to the cost of being wrong.

Brainstorming, drafting, explaining, restructuring something you'll read anyway: use it freely, errors are cheap and you'll catch them. Anything load-bearing where a wrong specific has consequences: verify the specifics against a real source before it leaves your hands. Names, numbers, dates, citations and quotes are the five things to check every time, because they are simultaneously the most likely to be invented and the most damaging when they are.

Sources

The distinction between a training data cutoff and an earlier reliable knowledge cutoff is drawn from Anthropic's models overview, fetched 11 August 2026, which publishes both dates separately for each model. The mechanism described here is uncontroversial and widely documented, but this page currently cites no specific research paper, and should before it loses its provisional status. No hallucination-rate figures are quoted because rates vary enormously by model, task and measurement method, and a single number would be more misleading than no number.

Next → Context windows · Tokens · All → Concepts