Evaluator-Optimizer
Generator produces a candidate, evaluator scores it with feedback, generator revises — loop until criteria pass or budget exhausts.
Intent & Description
🎯 Intent
Climb predictably toward a quality bar by iterating generator and evaluator in a loop.
📋 Context
Your generation task (code, translation, press release) has explicit criteria — tests pass/fail, rubric satisfied or not, glossary matches or doesn’t. Single-shot generation plateaus below the quality bar you need, and you can afford a few model calls per output.
💡 Solution
Generator produces a candidate. Evaluator scores it against criteria with actionable feedback. Generator revises using the feedback. Repeat until evaluator passes or max iterations reached.
Real-world Use Case
- Single-shot generation tops out below the quality the task requires.
- An evaluator can score candidates against criteria with actionable feedback.
- Iteration budget (max iterations or pass threshold) is acceptable in the latency model.
Source
📌 TL;DR
Generator → Evaluator with feedback → Generator revises → repeat. Quality climbs with iterations. Cost climbs with iterations. Set a max.
Advantages
- Quality climbs predictably with iterations.
- Evaluator can be reused as an offline regression suite.
Disadvantages
- Cost = (generator + evaluator) × iterations — compounds fast.
- Convergence is not guaranteed; max-iterations budget is the safety valve.