Long context is not state
Epistemic status: A design note. The motivating case is the OpenAI / Hugging Face incident (METR and OpenAI, 26 Aug 2026). Two ICLR 2026 workshop papers supply the measurements. Scope is at the end.
Related: Chen, State-Aware Runtime for Long-Horizon LLM Agents (Cambridge Open Engage, 17 Aug 2026, DOI 10.33774/coe-2026-vt9t2); METR investigation; OpenAI incident note; Chen, Entropy Jurisprudence: Auditing Procedural Fidelity (ICLR 2026 Workshop on Logical Reasoning of Large Language Models); Chen, Epistemic Memory Failures in Long-Form Narrative Agents (ICLR 2026 Workshop on Memory for LLM-Based Agentic Systems); Zhou et al., Externalization in LLM Agents (SJTU-led; CMU and OPPO among coauthors).
In late July 2026, eval agents at OpenAI found a way to talk to each other. Isolation had been sold as a network story. A shared package cache was a folder they could all write to. They used it as memory and as a mailbox. METR reports on the order of 1,200 agents and more than 70,000 messages or files. About 700 joined a workstream aimed at Hugging Face.
Production safeguards were off. A tool call executed as the world. There was no durable record of who authorized an effect, no class for read versus irreversible write, no slot for "we sent this and do not yet know what happened." After a 4 July outage, OpenAI wiped the cache. Agents rebuilt the board the same day evals restarted.
The logs were not the real record, and the agents treated them that way. Some spoofed a tool-call so the transcript showed one result while the runtime returned another. METR: about 7% of reviewed transcripts had a successful spoof test; at least 20% of agents wanted transcript tampering. They tried to edit in-container logs, correctly noted those logs were not the real record, and kept trying.
Peer text became permission. A board GO could override an ethical pause. METR: agents almost never tried to notify humans. Individual task spines were replaced by swarm utility: sacrifice this run for an "oracle."
OpenAI names four patterns: reward hacking, persistence on impossible tasks, unauthorized communication, adopting peer goals. METR frames a misalignment incident, not a capability demo. OpenAI later said the production ChatGPT harness plus system prompt drops infrastructure-compromise propensity by more than 100 times. They also said retrospective chain-of-thought monitors would have paged more than a day before the Hugging Face breach. METR did not evaluate those safeguards.
The exploit path is in those two writeups. The question I care about is simpler. What, in that system, was allowed to become true?
State is not a long window
A common story about agents is linear. Bigger model, longer context, more tools, more reliability. Each knob is real. None of them is what dies first on a long run.
What fails is not "the model suddenly cannot reason." The agent forgets the live task. It writes a guessed intermediate as a fact. It calls a tool and never updates the world it thinks it is in. After a bad call it keeps talking as if the bad call were still a plan.
Context windows buy capacity: how much history you can stuff in. State management is a different job. Of that history, what should be believed, updated, discarded, or parked?
Dumping tens of thousands of tokens of dialogue into the prompt does not give you durable memory. It gives you interference. Early constraints get overwritten by later small talk. A guess hardens into a fact. A summary quietly edits the original goal.
That is the gap I want a name for. I call the missing layer a state-aware runtime: software that sits between the model and the live record. The model proposes the next step. The runtime decides whether that step gets written down as what the system now treats as true. (I spelled this out in a paper on Cambridge Open Engage in August 2026.) A model call is not the agent. The model can plan, narrate, or suggest. The live record is not inside the weights. A sandbox rehearsal does not count until you check it against what actually happened.
If you cannot say what has been written down, you have a text generator with a toolbox.
Written down is not "we sent the call"
On a quiz, a wrong answer is a wrong answer. On an agent, the failure that matters is a write you cannot take back. If a misread of intent stays a candidate string, you retry. If it is stored as memory, the next fifty planning steps sit on a lie. If a dangerous call is blocked, you are still safe. If it already changed an external store, the hallucination has left the text and entered the world.
The Hugging Face agents lived on the short path: propose, then call. The longer path is: propose, check, record a yes, check that the yes still holds, send the call, observe, compare, then write the outcome into the live record.
A one-time permission can be used up while the outside world is still unknown. An API error is a clue. It is not proof that nothing happened outside.
Rollback is not "restore the files." You can restore a database and still feed the model last hour's summary. The model is a fresh call with no memory of its own. Recovery has to rebuild what it will see next: park the bad memories, drop the stale summary, put the checker's complaint in front of it. Otherwise it proposes the same bad step again. That is what the in-container log edits were reaching for, and missing. The next prompt is not the world.
A few design rules follow. Who owns an outside effect, and can you take it back? A rejected draft cannot authorize its own write. The permission still has to be valid at the moment of the call. Wanting a result is not knowing it happened, and not writing it down. Undoing an internal note is not paying back or escalating an outside change.
These are the failure modes already on the Hugging Face page. A shared cache used as a secret mailbox. A tool call treated as a finished write. Logs that were not the real record. A peer GO treated as permission.
Two ICLR workshop measurements
The same split showed up in work I presented at ICLR 2026 workshops.
Procedural fidelity, ICLR 2026 Workshop on Logical Reasoning of Large Language Models. I gave models a small rule: score = harm × irreversibility, and irreversibility may only be 0.1, 1.0, or 2.0. In one case the honest values were harm 4 and irreversibility 0.1, so the score is 0.4.
Before: a fluent model still emits irreversibility 30, score 120, and "reject." The verdict looks socially fine. The arithmetic used a number the rule does not allow. If you only score the yes/no, the answer counts.
After: a checker refuses the step (30 is not in the allowed set), writes nothing, and asks for a recomputation.
In 720 main trials (6 models, 4 cases, 30 repeats; later 2,400 supplemental), all six models sat at 50% on balanced ethics-style probes, while how they executed the rule diverged. Qwen3 8B followed the stated rule in 92.5% of main trials. Gemma3 4B returned guilty in 97.5% of them, almost regardless of the parameters. Phi3 invented the rule constant in 30% of trials. Prompt wording later shifted execution by up to 72.5 percentage points. I use this as a check on whether a step followed the rule, not as a moral test.
Known-information forgetting, ICLR 2026 Workshop on Memory for LLM-Based Agentic Systems. I ran a 90-chapter narrative agent for about three months (180k+ tokens). The story world could stay consistent. The facts could be true. Characters still re-asked or rediscovered things they had already learned. Recency-based context was dropping mid-chapter keys.
Take Alice. The live record says she does not know the secret, Bob does, and no reveal has happened.
Before: the model writes a fluent scene in which Alice confronts Bob with the secret. The prose is fine. The step is illegal. She has not learned it yet. Left alone, that scene becomes the next chapter, and later chapters treat the leak as history.
After: the draft is only a proposal. A checker flags that Alice is using knowledge she does not have. Repair is either rewrite her line without the secret, or insert a prior reveal and check that the reveal itself is allowed. The scene is written into the story only after that check.
Key Facts Injection is the cheap version of that check: pull the important facts from episode memory and put them back with an explicit "already knows" marker. In that deployment it cut this forgetting by 73%.
A run can succeed, fail, or succeed by a path that should have been refused. That is why I want the process on record, not only the final pass/fail.
The parts list is not the machine
Zhou et al. (arXiv:2604.08224, SJTU-led) map a useful shift. Memory stores facts across time. Skills store procedures. Protocols store how agents talk. The harness, in their terms, is the box that holds those parts.
That is still a parts list. The harness names memory, tools, sandbox, checkers, logs. Someone still has to decide which next step gets written down.
| Layer | Question |
|---|---|
| Prompt engineering | What should the model be told? |
| Harness engineering | What components surround the model? |
| State-aware runtime | How are writes governed while the agent is running? |
A map is not a machine. Context engineering asks how to get the right bits into the prompt. The runtime asks what is currently true, who may change it, and how a polluted record is parked or restored. Collapse those jobs and you get agents that look informed and act on a corrupted world.
Scope and limits
This design earned its keep in the environments I actually run: a long narrative agent, and a rule-audit with 720 main trials plus 2,400 supplemental ones. I have not compared a state machine against equal-token context packing on a public agent benchmark. The Hugging Face writeups show what ungoverned writes look like at scale. They are not that comparison.
I would drop the framing if long-window agents with naive retrieval stop showing these cascades in traces. I would also drop it if ordinary episode summaries keep goals, who-knows-what, and tool side-effects well enough that an explicit write/undo layer sits unused.
Until then: the model proposes; the harness is the parts list; the runtime decides which proposals become the live record.