The number-one fear about deploying AI is that it will confidently make things up. You can't eliminate hallucination at the model level — but you can build a system around the model that keeps it grounded and honest. Here's the recipe.
1. Ground it in retrieval
The single biggest lever: don't rely on the model's memory. Use RAG to fetch relevant, authoritative information and have the model answer from that, not from its training. Hallucinations concentrate on rarely-seen facts; retrieval supplies exactly those facts so the model doesn't have to guess. Answer from sources, not from vibes.
A model answering from its memory guesses. A model answering from retrieved documents reads. Design so it reads.
2. Demand citations
Have the model cite which retrieved source each claim comes from. This does two things: it lets users verify, and it disciplines the model — a claim that must be tied to a source is less likely to be invented. If it can't cite, it probably shouldn't assert.
3. Let it say "I don't know"
Explicitly instruct and allow the model to decline when the retrieved context doesn't contain the answer. A chatbot that says "I don't have information on that" is far more trustworthy than one that always produces something. Abstention beats fabrication.
4. Constrain the scope
A focused bot ("answers about our product from our docs") hallucinates far less than an open-ended one, because retrieval covers its whole domain. Narrow scope is a feature, not a limitation.
5. Verify and evaluate
Build an eval set of real questions, including ones the bot shouldn't answer, and measure how often it stays grounded. For high-stakes answers, add a verification step that checks the response against the sources.
The mindset
Treat hallucination as a system-design problem, not a model flaw to prompt away. Ground in retrieval, cite sources, permit "I don't know," scope tightly, and measure relentlessly. Do that, and you get a chatbot people can actually trust — not because the model is perfect, but because the system around it is honest.
