Hallucinations and How to Ground an LLM
Why LLMs state false things confidently, and the practical techniques that reduce it: grounding, citations, constrained outputs and checks.
Generative AIIntermediate2 min readUpdated 11 Oct 2026
Why it happens
An LLM generates text that is likely given its training and its prompt. It has no built-in check against a source of truth. When it lacks the information, a fluent but wrong answer can still be likely text.
Common triggers:
- questions about facts not in the prompt (recent events, private company data)
- precise details: numbers, dates, names, citations
- prompts that pressure for an answer ("you must answer")
Techniques that help
| Technique | How |
|---|---|
| Grounding (RAG) | Put the relevant source text in the prompt and instruct the model to answer from it |
| Allow "I don't know" | Say explicitly that "not found in the documents" is a valid answer |
| Citations | Ask for the source passage id for each claim, then verify the ids exist |
| Structured output | Constrain answers to a schema or a fixed set of labels |
| Tools for facts | Let the model call a database or calculator instead of recalling numbers |
| Verification step | A second check: does every claim appear in the sources? |
Answer using only the context below. If the answer is not in the context,
reply exactly: "Not found in the provided documents."
Cite the passage id for each statement, like [doc-3].Measure it
Build an evaluation set with questions whose correct answers (and "not in the documents" cases) are known, and track groundedness over time.
Finished this note?