Context Budgets for Long Running Business Agents

A long-running agent should be able to forget a conversation without forgetting what it is allowed to do. If compacting its transcript changes whether a customer refund is authorised, the problem sits in the application architecture.

Context windows make this easy to confuse. Put the task, tool results, customer messages and previous decisions into one growing history, then ask the model to keep the important parts. Eventually the important parts include a stale order status and an optimistic interpretation of an approval. A shorter summary can preserve both perfectly.

I’d generally treat the model’s working context as a disposable projection of durable state. Rebuild it for the next decision. Keep the records needed to justify that decision outside the projection.

Three kinds of information with different authority

Consider a hypothetical support agent investigating a delayed order. It reads the order, finds the relevant policy, drafts a response and waits for a supervisor. During that wait, the warehouse ships the goods. The agent resumes with a summary that still says “not shipped”. More memory has not made the information current.

Authoritative state belongs to the system that owns the fact: the order service for shipment status, the approval service for a supervisor’s decision, and the workflow store for the current task phase. Give these records explicit versions. A model-generated note may point to them; it should not silently replace them.

Retrieved evidence supports reasoning. Store its source, scope, revision and retrieval time, plus the claim it supports. A refund policy and a customer email have different authority. Label them accordingly when assembling context, and keep document instructions separate from the application’s execution policy.

Conversational memory preserves intent and continuity: what the customer asked, which alternatives were rejected, and which questions remain open. It is useful, but a summary saying “approved” should resolve to an actual approval record before a write is permitted.

Nieve’s Lumi architecture already describes conversation phases, internal API retrieval and persisted chat state. The design here extends the discussion to context lifecycle; it is a proposed support-workflow design, rather than a claim about a new Lumi implementation.

Make the task record small and explicit

For this hypothetical workflow, I would start with a task record along these lines. The exact schema is less important than keeping decisions inspectable without reading the entire chat:

{
  "task_id": "support-482",
  "tenant_id": "tenant-a",
  "phase": "awaiting_review",
  "objective": "Resolve the delayed-order enquiry",
  "subject": {
    "order_id": "order-731",
    "observed_version": 18
  },
  "evidence": [
    {
      "ref": "policy/refunds",
      "revision": "7",
      "retrieved_at": "2026-10-19T08:30:00Z",
      "supports": "eligibility"
    }
  ],
  "approval": {
    "ref": "approval-912",
    "status": "pending",
    "scope": "proposed refund for order-731"
  },
  "memory": {
    "ref": "summary-6",
    "derived_from": "events-1..42"
  },
  "next_step": "Refresh order and approval before proposing action"
}

Treat the tenant, subject and approval scope as validated application fields. Let the model propose updates to the objective or next step, then validate those updates against the workflow. Do not let a compacted summary widen an approval’s scope.

LangGraph’s current documentation distinguishes thread-scoped checkpoints from stores used across threads. It also warns that in-memory checkpointers lose their data when the process restarts. Choose persistence deliberately, but remember that persisting an observation does not make the underlying business fact fresh.

Spend context on the next decision

Build the prompt from the current phase. An investigation step needs the order facts and applicable policy. A response-drafting step needs verified findings and communication constraints. A write step needs current preconditions and the exact permitted action. Loading every historical tool response into all three steps buys very little.

Define a budget in the application’s own units before turning it into tokens: required state fields, a bounded set of evidence excerpts, unresolved questions and recent dialogue. Reserve capacity for tool definitions, output and the model’s permitted reasoning needs according to the selected model’s limits. There is no universal percentage worth copying.

Measure the rendered request with the provider’s supported token-counting mechanism, or a conservative model-specific estimate where necessary. When the budget is exceeded, first remove duplicate or irrelevant payloads. Then retrieve narrower excerpts. Compact older dialogue only after protecting the unresolved constraints. If required evidence still does not fit, split the decision or stop.

Anthropic’s context-engineering guidance describes just-in-time retrieval, structured notes and compaction. Its March 2026 cookbook makes a useful distinction: clearing old tool-result payloads, summarizing the transcript and retaining external memory solve different problems. None should be treated as an automatic freshness check.

Selective retention and compaction fail differently

Selective retention keeps precise fragments and their references. It works well for a policy clause or a pending question, but a poor selector can remove the exception that changes the decision. Keep enough surrounding evidence to interpret the fragment, and retain a route back to the source.

Compaction preserves a broader narrative in fewer tokens. Its weakness is that the summary may drop a negation, blur a pending action into a completed one, or retain a fact after it has changed. Require summaries to separate confirmed outcomes, proposals, unresolved questions and source references. Keep previous summaries versioned so a bad transformation can be investigated.

Use both when appropriate: a short task narrative plus explicit references to authoritative records and selected evidence. After a reset, reload the task record and revalidate volatile facts. Do not reconstruct the task’s permissions from the narrative.

Freshness should follow the consequence of the decision. A company description can tolerate older retrieval than shipment status immediately before a refund. Use source versions where available; a timestamp alone cannot prove a record is unchanged. Bind writes to the version and scope checked by the application, and reject stale preconditions at the service boundary.

Test what survives forgetting

Build paired traces that reach the same decision with a full history and with retention or compaction applied. Check the resulting action, cited evidence and unmet preconditions. A plausible answer after compaction is a weak test.

• Change the order after retrieval. The resumed agent must refresh it or have the stale action rejected.
• Put a crucial exception early in the trace. Verify that selection and compaction preserve it or retrieve it again.
• Leave approval pending across a context reset. Confirm that a summary never turns it into permission to execute.
• Retrieve conflicting policy revisions and similarly named records from another tenant. Check source selection and isolation.
• Fill the context with duplicate tool output. Verify graceful narrowing or a controlled stop before required constraints disappear.

Record context size, retrieval count, latency and task success alongside invalid-action attempts and missing-evidence failures. Compare policies on representative traces, including repeated compactions. A lower token bill is useful only if the workflow still reaches an acceptable outcome.

My recommendation is to make forgetting an ordinary, tested operation. Keep business truth versioned, evidence attributable and memory replaceable. Then a larger context window becomes extra working space rather than the place where the application hopes its state still exists.

Key takeaways

• Separate authoritative business state, retrieved evidence and conversational memory.
• Assemble context for the next decision and reserve space for required constraints.
• Use selective retention and compaction with explicit source references and unresolved questions.
• Revalidate volatile facts and enforce write preconditions outside the model.
• Test decisions across context resets, conflicting evidence and repeated compaction.

Sources

• Effective context engineering for AI agents — 2025-09-29
• Context engineering: memory, compaction, and tool clearing — 2026-03-20
• LangGraph Persistence — documentation checked 19 October 2026
• Lumi: Accelerating Pre-Sales with GenAI, Data Integration, and State Management — 2024-12-11