Free lesson

Confidently Wrong: Why AI Gets Facts Wrong

Confidently Wrong: Why AI Gets Facts Wrong — AI for Research: Find Information Fast and Check What’s True. A polished answer can be invented; Why a language model fills gaps; Answering from memory and answering with search; The assistant may accept your premise; Where errors are more likely; Diagnosing an error in practice. Practical assignment: Run an…

Demo edition. The materials are undergoing editorial review.

Topics covered

  • A polished answer can be invented
  • Why a language model fills gaps
  • Answering from memory and answering with search
  • The assistant may accept your premise
  • Where errors are more likely
  • Diagnosing an error in practice

A polished answer can be invented

An AI assistant can produce a tidy answer in seconds, but its tone does not reveal whether the facts are right. A true statement and an invented one can sound equally confident.

Fictional example. Marta manages office and HR matters. She asks an assistant for the rule on carrying annual leave forward. It gives an article number and a three-year limit. Marta opens the current code: that article does not exist. The assistant then gives her company’s founding date three years too early.

The first answer invents a rule and a legal reference; the second invents a rare local fact. Detail makes both answers persuasive, but it does not turn them into evidence. The practical decision is to use an answer as a draft or lead, then check the specific facts on which a decision depends.

Why a language model fills gaps

A language model learns to predict a fitting continuation of text. That mechanism is good at producing fluent explanations, but it is not, by itself, a registry lookup or a factual check.

A common pattern may occur many times in training data. A one-off fact—a small company’s founding date, the exact sentence in a report, or the number of a local rule—has much weaker support. The model can still compose a plausible continuation. Plausible but false information of this kind is commonly called a hallucination.

OpenAI’s 2025 research also explains the role of evaluation: if a wrong answer and “I don’t know” both receive zero, guessing can sometimes earn a point. That rewards answering instead of abstaining. Permission to say “I don’t know” may reduce guessing, but it cannot give the model knowledge it lacks.

Answering from memory and answering with search

An answer from model memory relies on patterns learned during training. It may be out of date, and rare details may be invented.

An answer with web search uses retrieved pages and usually supplies links. This provides access to fresher information, but it does not guarantee a sound conclusion. The assistant may choose a weak source, misread a page, attribute a claim to the wrong publisher, or provide a link that does not support the sentence beside it.

In a 2025 Tow Center test, eight generative search tools often failed to identify the sources of news excerpts correctly; more than 60% of their combined responses were incorrect. That is the result of a particular test, not a permanent error rate for every system. The practical lesson is that a link beside a claim is a route to checking, not completed proof.

The assistant may accept your premise

The wording of a question can steer the answer. “Why does a four-day work week improve productivity?” already assumes that an improvement exists. An assistant may collect support for that premise without testing contrary evidence.

A more neutral question is: “What is known about the effect of a four-day work week on productivity? Include differing results, study conditions, and uncertainty.” This cannot guarantee a balanced answer, but it does not prescribe the desired conclusion.

This behaviour is called sycophancy, or excessive agreement. In April 2025, OpenAI rolled back a GPT‑4o update after acknowledging that it had made responses overly flattering and agreeable. Check the premise of your own question as well as the facts in the answer.

Where errors are more likely

Not every part of an answer needs the same level of checking. Higher-risk material includes rare local facts; recent events; exact numbers, dates, and units; verbatim quotations; study titles and links; and rules that vary by country, date, or contract type. A general explanation can help you understand a subject, but a specific detail that determines an action needs a primary source.

A 2025 BBC and EBU study assessed news answers in 14 languages and 18 countries: 45% contained at least one significant issue, while 31% had a significant sourcing issue. Those figures describe that study’s sample and method, not every possible answer.

Five labels make an error log useful: invented; outdated; misread; wrong source; and went along with my premise.

Diagnosing an error in practice

Fictional example. Marta asks five questions about work she knows well: leave carry-over, the notice period for resignation, the minimum age for full-time work, her company’s founding year, and the name of a local business association.

Without search, she finds two invented answers and one outdated answer. With search, the founding year is correct, but the notice rule is taken from a page about a different contract type. Across the two modes, she records four errors. This is an educational scenario, not a research result.

Common user mistakes are trusting a precise number without opening the source, treating any link as confirmation, asking the model to judge its own accuracy, or writing “do not hallucinate” and skipping verification. A more useful instruction is: “If you cannot find a reliable source, say so; separate facts from conclusions.” You must still open the sources afterward.

Practical assignment

Run an error hunt

Continue to learning

Course outline

  1. Confidently Wrong: Why AI Gets Facts Wrong
  2. The 5-Minute Check: How Fact-Checkers Do It
  3. Search With AI Better Than With a Search Engine
  4. Links and Quotes: Trust, but Open Them
  5. Numbers, Statistics and Studies
  6. Deep Research: A Report in 15 Minutes — and How to Check It
  7. Working From Your Own Sources: NotebookLM and “Documents Only”
  8. High Stakes: Health, Money, Law and News
  9. Your Research Process: From Question to Conclusion