RAFT
Train the model to be robust to irrelevant retrieved documents (distractors) in a domain-specific RAG setting.
Intent & Description
🎯 Intent
Train the model to be robust to irrelevant retrieved documents (distractors) in a domain-specific RAG setting.
📋 Context
A team is using retrieval-augmented generation in a specific domain and has observed that retrieval almost always returns a mix of documents. Some of the retrieved chunks are genuinely relevant to the user’s query; others are topically similar distractors that share keywords or themes but do not actually answer the question. An off-the-shelf retrieval-augmented model attends to all of these chunks and is over-confident on the distractors that look plausible at a glance.
💡 Solution
Construct training examples where some documents are oracle and others are distractors. Train the model to cite oracle documents and ignore distractors. Couples chain-of-thought with citation discipline.
Real-world Use Case
- Domain-specific RAG models drift to topically similar distractors.
- Training data with oracle and distractor documents can be constructed at scale.
- Citation discipline matters and outputs must be traceable to oracle sources.
Source
Advantages
- Robustness to distractor documents in domain RAG.
- Citation discipline improves.
Disadvantages
- Training data effort.
- Domain-specific; transfer between domains is partial.