Journaled LLM Call
Record every non-deterministic step (LLM calls, tool results, timestamps) to an append-only journal on first execution — replay crashes by replaying the journal, not re-invoking the model.
Intent & Description
🎯 Intent
Record every non-deterministic step on first execution and replay that recorded value during crash-recovery instead of re-invoking the model.
📋 Context
Durable-execution engines (Temporal, Inngest) recover from crashes by replaying workflow code from a recorded history. If your workflow contains non-deterministic steps — LLM calls, tool results, timestamps, random draws — replaying those steps will produce different values on recovery, causing divergence or duplicate side effects.
💡 Solution
Classify every step as deterministic workflow logic or non-deterministic effect. Run each effect exactly once and append its result to an append-only journal keyed by step position. On crash-recovery, the engine replays workflow code from the start — deterministic logic recomputes freely, but each effect call short-circuits to its journaled output. The model is queried only the first time a step is reached; the recorded response stands in for all subsequent replays. This trades a possibly-stale recorded answer for deterministic, fault-tolerant replay without double-billing.
Real-world Use Case
- The agent runs on a durable-execution engine that recovers by replaying workflow code from a recorded history.
- The workflow contains non-deterministic steps — LLM calls, tool results, timestamps, or random draws.
- A recovered run must follow the same path as the original, and re-invoking the model on recovery is unacceptable on cost or correctness grounds.
Source
📌 TL;DR
Journal every LLM call and replay from the log on crash-recovery — deterministic resume, zero re-billing, and a built-in audit trail as a bonus.
Advantages
- Replay is deterministic — recovered runs follow the identical path the original took.
- Each model call is billed once; recovery reuses the recorded output.
- The journal doubles as an audit trail of every non-deterministic decision the workflow made.
Disadvantages
- Journaled responses can be stale — replay reuses a value the world has since changed.
- Missing one non-deterministic step reintroduces divergence that’s hard to spot and debug.
- The append-only journal grows with every effect and must be stored and garbage-collected.
- Changing workflow code between original run and replay can invalidate journaled step positions.