Hallucination

Why It Matters

Hallucination is the Achilles heel of LLMs. Models generate confidently incorrect information (fake statistics, invented citations, fabricated details) with no indication they’re making things up. Users can’t easily distinguish hallucinated content from accurate responses.

This isn’t a bug to be fixed; it’s fundamental to how language models work. They predict probable next tokens based on patterns, not retrieve verified facts. When the model doesn’t “know” something, it doesn’t say “I don’t know” but instead generates plausible-sounding text.

For AI engineers, hallucination mitigation is a core job responsibility. Every production system needs strategies to reduce, detect, and handle hallucinations. Applications in healthcare, legal, and finance have especially low tolerance, as a hallucinated medical recommendation or legal citation can cause real harm.

Implementation Basics

Mitigating hallucination requires multiple approaches:

1. Grounding with RAG The most effective mitigation. Retrieve relevant source documents and include them in the prompt. The model answers based on provided context rather than generating from training data. This dramatically reduces hallucination for domain-specific questions.

2. Prompt Engineering Instruct the model to only answer based on provided context. Use phrases like “If the information isn’t in the documents, say you don’t know.” This doesn’t eliminate hallucination but reduces it.

3. Temperature and Sampling Lower temperature (0.0-0.3) reduces creativity and makes outputs more deterministic. For factual tasks, this reduces hallucination at the cost of less varied responses.

4. Citation Requirements Ask the model to cite specific passages from source documents. This makes hallucinations easier to detect, because if the “citation” doesn’t exist, you know there’s a problem.

5. Verification Loops Use a second LLM call or rule-based system to check facts in the response. Cross-reference claimed statistics, verify URLs exist, check that citations match sources.

6. Confidence Calibration Some approaches use log probabilities or separate verification models to estimate confidence. Low-confidence claims can be flagged for review or filtered out.

No single technique eliminates hallucination entirely. Production systems combine multiple approaches based on the risk tolerance of the application.