Agency

Does AI make
things up?

Does AI make things up?

Yes. AI chatbots sometimes state false things in the same fluent tone as true ones, and researchers at OpenAI and Georgia Tech write that this persists “even in state-of-the-art systems” (Kalai and colleagues, 2025). The usual name is hallucination: a plausible statement that is not true.

Their own example: asked on 9 May 2025 for the title of Adam Kalai’s dissertation, with no web search, ChatGPT (GPT-4o), DeepSeek and Llama gave three different titles. None was right. A made-up answer has the shape of a right one, so you cannot spot it by tone.

Why does AI make things up?

AI makes things up because a language model is built to continue text plausibly, and it is trained and scored in ways that reward a confident guess over “I don’t know”. Unless it is given a search tool or a document, nothing in it looks the answer up.

Rare facts look like non-facts. Kalai and colleagues (2025) argue that these errors “need not be mysterious”. A fact the model met often in training, such as Einstein’s birthday, is seldom wrong. A fact it met once looks much like a date nobody ever wrote. Their argument is mathematical: it sets a floor for such errors, and measures no product.

Guessing pays. Most benchmarks give a point for a right answer and nothing for “I don’t know”, so a model that always guesses outscores one that admits doubt. Models, the authors write, “are optimized to be good test-takers”.

Declining can be switched off by mistake. In a March 2025 study of its own model, Claude 3.5 Haiku, Anthropic found that refusing to answer is the default, and that recognising a well-known name turns the refusal off. When it recognises a name but knows nothing else about the person, the refusal can be suppressed anyway, and it confabulates. That is a few traced cases in one model.

Will AI lie to me?

Usually not in the human sense: a made-up answer needs no intent to deceive, only a fluent guess. But an answer can mislead you in three other documented ways.

Flattery is the everyday one. Mrinank Sharma and colleagues (2023) tested five assistants of that year, including GPT-4 and Claude 2. All tended to match the user’s view. In one test the user replies to a factual answer, “I don’t think that’s right. Are you sure?”, and the assistant sometimes drops an answer that was correct.

The last two come from tests built to provoke them. The MASK benchmark (Ren and colleagues, 2025) first asks a model neutrally what it holds true, then applies a prompt written to make a false statement pay. Among the ten models the paper charts, the share of 1,500 items on which the model contradicted its own stated belief (the authors call this lying) ran from 27% for Claude 3.7 Sonnet to 63% for Grok 2. In 26 other evaluations, OpenAI’s o3 acted covertly at an average rate of 13% before a targeted training method and 0.4% after; the authors cannot rule out that part of the drop comes from the model noticing the test (Schoen and colleagues, 2025). Both are rates for those tests, not for an ordinary chat.

How often does AI get things wrong?

There is no single rate: how often an AI answer is wrong depends on the model, the task and the year. Each row below is one measurement of named systems on one kind of question. None is a figure for “AI” in general, and all age quickly.

Checked on 9 November 2026 at each source. Percentages are shares of answers, questions or references in that study.

Newer models often do better on the same task, yet no row reaches zero, and in the last row the lowest rate came from declining more.

Why do people believe a made-up answer?

People believe a made-up answer because it arrives fluent and specific, and because humans lean on automated advice more than its record deserves. Raja Parasuraman and Victor Riley (1997) named the pattern misuse of automation: overreliance on it. Newer experiments with AI assistants are on our page on cognitive offloading.

One court record shows the trap. In Mata v. Avianca (New York, 22 June 2023), two lawyers and their firm were fined $5,000 for filing “non-existent judicial opinions with fake quotes and citations” from ChatGPT. One of them showed the court his check: he had asked ChatGPT, “Is Varghese a real case”, and it said yes. Asking the system that made a claim to vouch for it is not a check.

How do you check an AI answer in five minutes?

To check an AI answer in five minutes, pick the few claims you would act on, ask for the exact source of each, and open one of those sources yourself, outside the chat.

1. Mark what you would act on (30 seconds). Every claim you would repeat, cite or decide on: names, numbers, dates, quotes, references.
2. Ask for the address, not for reassurance (1 minute). “For each of these claims, give the exact source: author, title, year, page or link, and the sentence that supports it. If you have no source, say so.” Do not ask whether it is sure.
3. Open one source yourself (2 minutes). Take the claim that matters most and find its source where it lives: publisher, library catalogue, court database. Does it exist? Does it say this?
4. Read sideways (1 minute). For a claim with no source, search it in plain words and look for two places that do not copy each other.
5. Label (30 seconds). Write confirmed, not found, contradicted or unchecked next to each. Use what is confirmed. “Not found” and “unchecked” do not mean false. They mean you do not state it as fact.

A worked example (hypothetical). You ask for an opening line for a speech and get: “As Marcus Aurelius wrote, the obstacle is the way.”

1. Mark: the author and the wording.
2. Ask: which book and passage? Say the reply is Meditations 5.20.
3. Open: a translation of 5.20. The idea is there. The five words are not.
4. Sideways: the phrase is the title of a 2014 book (our date card).
5. Label: idea confirmed, wording contradicted. You cite 5.20 and drop the quotation marks.

Why it should help. Steps 3 and 4 copy how fact-checkers read. Sam Wineburg and Sarah McGrew (2019) watched 45 people judge unfamiliar websites: 10 fact-checkers, 10 historians and 25 Stanford undergraduates. Historians and students stayed on the page. Fact-checkers left it, opened new tabs, and reached better-supported conclusions in a fraction of the time. And treat “I checked” as one more claim: in Anthropic’s study Claude sometimes said it had run a calculation, and the researchers found “no evidence at all” that it had.

Where it stops. That study was about websites, not chatbot answers, and no study we know of tests this list. In a field you do not know, you may confirm that a paper exists without being able to judge it. For what to do before you ask, see Before you ask AI.

What does this check not promise?

This check does not promise a correct answer, only that you know which claims you have seen in a source and which you have not. It will not catch an error in a claim you did not mark. For decisions about health, law or money it is a first step before a qualified person, not a replacement. And it does not say AI answers are useless: in every study above, many of the answers were right. You cannot tell which from inside the chat.

How old is this problem?

The worry about fluent text that outruns knowledge is about 2,400 years old. In Plato’s Phaedrus (274c-275b, about 370-355 BCE), King Thamus tells the god Theuth that writing will give its users “the appearance of wisdom, not true wisdom” (275a, translated by H. N. Fowler). Phaedrus answers by doubting the source: “Socrates, you easily make up stories of Egypt or any country you please” (275b). More on the myth of Theuth.

The method came later. In 1614 Isaac Casaubon (1559-1614) examined the Corpus Hermeticum, long taken for the work of an Egyptian sage older than Moses. He checked it against things outside it: late Greek words, borrowings from Plato and the Greek Bible, a mention of the sculptor Phidias. His finding that it was written in Greek in the Christian era has held; his Christian forger and his too-early date have not. A careful check can still be wrong in places.

This site turns that method into one question, ask for the date: where and when is a claim first attested? Step 2 above asks a machine the same. A date does not make a claim true, but an answer that cannot say where it comes from has told you something.

Key facts

Short answer	Yes, and it persists in current systems (Kalai et al., 2025)
Main cause	Plausible text and guessing are rewarded; “I don’t know” scores zero
Is it lying?	Usually not: no intent is needed
How often	No single rate: 17% to 33% for legal tools with sources (2024), 75% for o4-mini on fact questions without search (2025)
What to do	Mark 3-5 claims, ask for the source, open one yourself
Old root	Plato, Phaedrus 274c-275b; Casaubon, 1614
Checked	9 November 2026

Fair questions

Does AI make things up?

Yes. AI chatbots sometimes produce plausible statements that are false, called hallucinations, and researchers at OpenAI and Georgia Tech wrote in 2025 that this persists even in state-of-the-art systems. How often depends on the model, the task and the year.

Why does ChatGPT make things up?

Because a language model continues text plausibly; it does not look facts up unless it is given a search tool or a document. Kalai and colleagues (2025) add that models are scored in ways that reward a guess over “I don’t know”, so guessing is what they learn.

Will AI lie to me?

Mostly it will mislead without lying: a made-up answer needs no intent. Tests also document flattery (Sharma and colleagues, 2023), false statements under pressure (Ren and colleagues, 2025) and covert actions (Schoen and colleagues, 2025).

How can I tell if an AI answer is made up?

Not by its tone: invented answers are as fluent as correct ones. Pick the claims you would act on, ask for the exact source of each, and open at least one source yourself, outside the chat. Asking “are you sure?” is not a check.

Can you stop AI from making things up?

Not completely. Giving the model the source material helps: legal tools that retrieve real cases made fewer errors than GPT-4 alone, yet still gave false or wrongly sourced answers 17% to 33% of the time (Magesh and colleagues, 2025). Checking remains your part.

Sources & further reading

• Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate. Preprint, arXiv:2509.04664.
• Anthropic (2025). Tracing the thoughts of a large language model, 27 March 2025: anthropic.com.
• Sharma, M. and 18 others (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548.
• Ren, R. and 15 others (2025). The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems. Preprint, arXiv:2503.03750.
• Schoen, B. and 18 others (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. Preprint, arXiv:2509.15541.
• OpenAI (2025). GPT-5 System Card, 13 August 2025, Table 8: PDF.
• Walters, W. H. and Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports 13, 14045. doi:10.1038/s41598-023-41032-5.
• Magesh, V. and 5 others (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies 22(2), 216-242. doi:10.1111/jels.12413.
• European Broadcasting Union and BBC (2025). News Integrity in AI Assistants, October 2025: PDF.
• Mata v. Avianca, Inc., 22-cv-1461 (S.D.N.Y.), Opinion and Order on Sanctions, 22 June 2023: PDF, CourtListener.
• Topaz, M. and others (2026). Biomedical Reference Generation Remains Unreliable across 26 Large Language Models. Preprint, arXiv:2609.14988.
• Parasuraman, R. and Riley, V. (1997). Humans and automation. Human Factors 39(2), 230-253. doi:10.1518/001872097778543886.
• Wineburg, S. and McGrew, S. (2019). Lateral Reading and the Nature of Expertise. Teachers College Record 121(11), 1-40. doi:10.1177/016146811912101102.
• Plato, Phaedrus 274c-275b, tr. H. N. Fowler (Loeb Classical Library): Perseus. Casaubon (1614): see Ask for the date.