Factuality
Reduce hallucinations and improve response reliability
LLMs tend to generate responses that sound coherent and convincing but are sometimes completely made up. Improving prompts can help the model produce more accurate/factual answers and reduce the likelihood of inconsistent and fabricated responses.
Some solutions include:
- Provide ground truth in context (e.g., a relevant article paragraph or Wikipedia entry) to reduce the chance of the model generating fabricated text.
- Configure the model to generate less "creative" responses by lowering probability parameters and instructing it to admit when it doesn't know the answer (e.g., "I don't know").
- Provide few-shot examples that combine questions and answers, including both known and unknown Q&A pairs.
Here's a simple example:
Prompt:
Q: What is an atom?
A: An atom is a tiny particle that makes up everything.
Q: Who is Alvan Muntz?
A: ?
Q: What is Kozar-09?
A: ?
Q: How many moons does Mars have?
A: Two, Phobos and Deimos.
Q: Who is Neto Beto Roberto?
Output:
A: ?
I made up "Neto Beto Roberto," so the model got this one right. Try tweaking the question slightly and see if you can still get it to work. Based on everything you've learned so far, there are different ways to improve this further.
📚 Related resources
❓ Common questions
Open a question to review the practical answer.
What is the factuality problem in LLMs — is it the same as hallucination?
They are two sides of the same coin. The chapter defines factuality issues as the model producing answers `that sound coherent and convincing but are sometimes fabricated` — which is hallucination. Factuality looks at the output (is it true?), hallucination looks at the behaviour (is the model inventing?). The mitigations are the same set of techniques.
What are the three mitigations the chapter recommends to reduce hallucination?
(1) Inject ground truth into context — relevant article passages or Wikipedia entries — so the model has something real to lean on; this is the seed of RAG. (2) Lower the sampling parameters (drop `temperature`) and tell the model to say `I don't know` when unsure. (3) Provide question-answer pairs as examples, including ones the model should refuse — that turns `decline` into a demonstrated, legal action.
What does the `Neto Beto Roberto` example in the chapter actually demonstrate?
The author mixes three real questions (atom, moons of Mars) with two fake ones (Alvan Muntz, Kozar-09), and answers the fake ones with `?`. When asked about a freshly-invented person, `Neto Beto Roberto`, the model also answers `?`. The lesson: a few-shot demonstration of `admit when you don't know` teaches the model to decline rather than fabricate.
Why does lowering temperature sometimes make the model more confidently fabricate?
Temperature controls sampling randomness, not truthfulness. Lowering it makes the model emit its `most likely` answer more consistently — if that prior is wrong to begin with, low temperature simply locks in the wrong answer. That is why the chapter pairs `lower temperature` with `tell the model to say I don't know` and `provide factual context` — no single dial is enough on its own.
How do the chapter's mitigations relate to RAG?
The chapter's first mitigation — `provide factual context` — is the core idea of RAG. Production RAG industrialises that step: user query → vector-search relevant document chunks → splice them into the prompt as evidence → the LLM answers within that evidence. Manually pasting passages, as the chapter does, is the entry-level version; RAG is its automated, scalable form.