Fallback Chain
Try a primary handler; on failure or low confidence, fall through to a sequence of fallback handlers.
Intent & Description
🎯 Intent
Try a primary handler; on failure or low confidence, fall through to a sequence of fallback handlers.
📋 Context
An agent in production depends on at least one model or tool that can fail for routine reasons: rate limiting, vendor errors, regional incidents, or outputs the model itself returns with low confidence. End users are sitting on the other end of the call expecting an answer regardless of which upstream had a bad minute. The team has more than one option available — a backup model, a smaller local model, a deterministic rule-based fallback — but those options are not wired in by default.
💡 Solution
Define an ordered chain of handlers. Each handler returns either a confident answer or a failure/low-confidence signal. On failure, the next handler runs. Final fallback is a generic ‘I don’t know’ rather than a wrong answer.
Real-world Use Case
- Single-handler failure would cascade to the user as an outage.
- Multiple handlers exist with meaningful differences in capability or cost.
- Each handler can return a confidence or failure signal that triggers the next.
Source
Advantages
- Graceful degradation under partial failures.
- Each layer can be tuned independently.
Disadvantages
- Cumulative latency on full cascade.
- Hides quality regressions in the primary.