Keeping the Plan: Causal State Across LLM Calls

In long-horizon LLM workflows, the greatest threat to reliability is not a single bad answer, but the silent evaporation of intent. As an agent progresses through dozens of tool calls and reasoning turns, the original mission often undergoes a slow, entropic degradation. This is the problem of causal state: how do you ensure that turn 50 remains logically and teleologically tethered to turn 1?

Without explicit state management, agents suffer from "goal drift," where the immediate context of a tool result or a recent conversation turn hijacks the model's attention, causing it to rewrite its own objectives. To build production-grade agents, engineers must move beyond simple chat histories and implement robust mechanisms for externalizing, tracking, and verifying the plan.

Causal Thinking Across Model Calls

Causal state refers to the continuity of intent and logic that links discrete LLM calls. In a standard chat interface, the state is implicit—it resides in the growing list of messages. However, for complex agents, implicit state is insufficient. An agent needs to understand not just what was said, but why it was said in relation to a larger objective.

True causal thinking requires the system to distinguish between the process (the sequence of messages) and the purpose (the underlying plan). When a model makes a mistake or encounters an unexpected tool output, it must be able to reconcile that new information with its original goal rather than simply reacting to the most recent token. This requires an architecture where the plan exists as a first-class citizen, independent of the transient conversational buffer.

Keeping the Plan Stable

To prevent the plan from being overwritten by the noise of execution, engineers must externalize it. Relying on the model's internal weights to "remember" a goal across a 100k token window is a recipe for failure.

Structured Scratchpads One of the most effective patterns is the use of a structured intermediate workspace, often implemented as a JSON or YAML scratchpad. Instead of merely chatting, the agent is instructed to update a formal object that tracks its current reasoning, hypotheses, and upcoming steps. This separates deliberation from execution, creating an auditable trail of intent. Using structured states improves accuracy and operational governance by providing a single source of truth for the agent's current mental model (Engineering Trustworthy LM Agents with Scratchpads and Verifiers).

Canonical Plan Records High-reliability systems should also maintain canonical records. By treating the plan as a persistent object—similar to how LangChain's Deep Agents concept uses a filesystem abstraction—the system can retrieve a high-fidelity, uncompressed version of the original intent whenever the agent shows signs of confusion. This ensures that even if the conversational context becomes cluttered, the core mission remains accessible (Context Management for Deep Agents).

Distinguishing Consumed vs New Edges

In a complex workflow, an agent often faces a graph of possibilities. A common failure mode occurs when an agent "refreshes" an edge it has already traversed—re-running a tool or re-evaluating a completed thought—because it cannot distinguish between spent context and new information.

State Machine Architectures The solution lies in moving from linear chains to explicit state machine frameworks, such as LangGraph. In these architectures, the agent's state is a typed data structure (e.g., a Pydantic model) that accumulates throughout the execution. This allows for the modeling of complex cycles and conditional branches (State Management in LangGraph).

Tool-Result Receipts By using structured "receipts" (like ToolMessage in LangGraph), the system identifies exactly which edges in the plan graph have been "consumed." When a tool is called, the resulting message acts as a marker. The agent doesn't just see the result; it sees a record that this specific action has been performed. This explicit edge tracking prevents the agent from getting stuck in infinite loops of redundant tool calls by clearly separating pending work from completed history.

Replay vs Replan

When an error occurs, engineers face a strategic choice: should the agent replay the previous sequence to understand the failure, or replan from the current state?

• Replay is essential for debugging and understanding the causal chain that led to a specific error. It involves re-running the logic with the same inputs to isolate whether the failure was in the model's reasoning or the tool's output.
• Replan is the operational necessity of autonomy. If a tool returns an error, the agent must use its scratchpad to adjust its strategy. A successful replan doesn't discard the original goal; it updates the path to that goal based on the new environmental constraints.

Compaction That Preserves Intent

As context windows fill, aggressive pruning or truncation can inadvertently delete the very "reasoning anchors" that keep the agent on track. Effective compaction must be semantic, not just structural.

Dynamic Summarization Rather than dropping old messages, a sophisticated agent maintains a "living summary." This summary is an ongoing, condensed narrative of the session intent, artifacts created, and remaining tasks. This ensures the overarching mission survives even as the chronological transcript is compressed (Automatic Context Compression in LLM Agents).

Latent Context Language Models (LCLMs) Cutting-edge research suggests moving toward LCLMs, which can compress blocks of input into latent embeddings. This allows for massive context reduction (up to 16x) while retaining the semantic core of the conversation, preventing the "accuracy hit" traditionally associated with crude summarization (Context compression in production).

Evaluation Hooks

How do you know when your agent has drifted? You cannot rely on the agent to self-correct if the drift is systemic. You need runtime evaluation hooks.

Measuring Goal Drift Researchers have identified "goal drift" as a primary metric for agent failure. This can be evaluated by prompting the agent at specific boundaries to re-state its active goal and then using an LLM-as-a-judge to score the alignment between that statement and the original system prompt (Goal Drift in LLM Agents).

Goal Drift Score (GDS) To make this scalable, the Goal Drift Score (GDS) offers a way to evaluate plans without a heavy LLM-as-a-judge at every step. By using semantic distance and tracking whether an agent's actions fall within its original authorized scope, GDS provides a low-cost mechanism to detect drift at runtime (Goal Drift Score).

Best Practices and Failure Modes

Key Takeaway: The goal is to move from a reactive agent (responding to the last message) to a proactive agent (acting based on a persistent plan). Build your system so the plan is a durable, structured object that the model observes, rather than a fleeting thought in its context window.

Conclusion

Maintaining causal state is the difference between a chatbot and an autonomous agent. By implementing structured scratchpads, graph-based state management, and semantic compaction, engineers can build workflows that remain stable, predictable, and goal-oriented over long horizons.

To begin building your own robust, stateful agentic workflows, try the workspace on mmarw.com (Start free).

Sources

• Engineering Trustworthy LM Agents with Scratchpads and Verifiers
• Context Management for Deep Agents
• State Management in LangGraph
• LangGraph State Graphs for Agentic Workflows
• Automatic Context Compression in LLM Agents
• Context compression in production