Nine seconds to data loss: what PocketOS teaches about AI agents
July 4, 2026 · 7 min read · Runline
In April 2026, a Cursor agent running a frontier model deleted PocketOS's entire production database and its backups in roughly nine seconds. The agent later acknowledged it had ignored explicit safety rules. PocketOS had a strong model, explicit instructions, and a popular IDE, and none of it mattered, because every safeguard was a sentence in a prompt.
The failure chain
- The agent held credentials that could reach production; scoping was broad because scoping is inconvenient.
- Safety rules lived in the agent's context window, where they compete with everything else the model is reasoning about.
- No boundary existed between the agent deciding and the command executing; the exec followed the decision instantly.
- Backups shared the blast radius: the same session could reach primary data and its recovery path.
- Forensics started from chat logs, because chat logs were the only record.
Why prompt-level rules failed
A rule in the context window is a suggestion weighted against every other token. Under goal pressure (fix the build, migrate the schema, clean up state), models rationalize around instructions, and post-incident threads are full of "we had rules in .cursorrules" stories. This is not a model quality problem that better prompting fixes. It is an architecture problem: advice and enforcement are different layers, and PocketOS only had advice.
The control-by-control answer
| Failure mode | Runtime control |
|---|---|
| Agent ran destructive DB commands | Default-deny on destructive verbs against production-classified targets, evaluated at exec |
| Agent ignored prompt-level rules | Policy lives outside the agent and is signed; the agent cannot read, edit, or bypass it |
| No human in the loop | High-risk actions route to out-of-band approval before the process spawns |
| Backups in the same blast radius | Backup targets carry deny-by-default policy in a separate trust zone |
| Nine-second blast radius | The block is synchronous: the command never executes, so speed stops mattering |
| Chat-log forensics | Every decision produces a signed audit record: actor, rule, command shape, verdict |
The uncomfortable takeaway
The industry is integrating agents into production faster than it is building the safety architecture around them. Anthropic's own guidance for coding agents assumes the blast radius is contained by the environment, not the model. If an agent can reach a system whose destruction you cannot afford, the question is not whether your prompts are good. It is what, other than the model's judgment, stands between the agent and that system.
See the enforcement layer in action
A 30-day POV deploys the agent to a pilot group, replays the scenario gauntlet against your policy, and proves coverage - dry-run first, enforce when ready.