Deterministic-LLM Sandwich
Bracket every LLM call with deterministic checks on both sides — pre-check decides if the model should run, post-check validates the output before it lands.
Intent & Description
🎯 Intent
Prevent the model from landing unsafe or malformed output by wrapping it in deterministic validation.
📋 Context
You use an LLM at a point where wrong output causes real damage — a knitting pattern with a bad stitch count, a DB migration that breaks production, an insurance quote missing a required coverage line. Removing the model entirely isn’t an option, but every output is one hallucination away from causing harm.
💡 Solution
Three layers. Pre: deterministic check decides whether the LLM should run at all (e.g., AST parse must succeed). LLM: produces a candidate with a structured-output schema and frozen rubric. Post: deterministic re-validation (parse, type-check, run tests). If post fails, return the original input unchanged.
Real-world Use Case
- LLM output must be checked deterministically before being trusted (AST parse, type-check, test run).
- A pre-check can decide whether the LLM should run at all.
- Returning the original input on post-check failure is acceptable behavior.
Source
📌 TL;DR
Pre-check → LLM → post-check. If post fails, return the original. The model can’t land broken output. The checks, not the model, are where bugs get fixed.
Advantages
- Model cannot land an unsafe artifact — post-check is the hard gate.
- Bug fixes go into the deterministic layer where they’re testable and repeatable.
Disadvantages
- Building the deterministic checks is the bulk of the engineering work.
- Over-strict post-checks reject valid outputs, degrading the model’s utility.