Hallucinated Citations
Trusting the model to write its own citations — and shipping fabricated sources to users.
Intent & Description
🎯 Intent
Letting the model format citations inline instead of binding them to actually retrieved documents.
📋 Context
Research, legal, or QA agents are told to “include sources.” The model writes whatever URL or paper title sounds plausible. Nothing verifies it. Users get confident-looking references to documents that don’’t exist — or did exist once, under a different URL.
💡 Solution
Wire citations to retrieved-source IDs from an actual retrieval pipeline. Validate URLs before display. Never trust free-text citation output from the model. See citation-streaming, naive-rag, contextual-retrieval.
Real-world Use Case
- Never use this; cite an example only to label the failure mode.
- Use citation-streaming, naive-rag, or contextual-retrieval to bind citations to retrieved-source IDs.
- Validate URLs and titles against retrieval results before display.
Source
📌 TL;DR
Bind citations to retrieved document IDs from a real retrieval pipeline — never trust the model to write its own links.
Disadvantages
- Trust collapses the first time a user clicks a citation and hits 404
- Legal and regulatory exposure in any domain where sourcing matters