AI Agents Are Deleting Production Data. The Problem Is Permissions.

By Jeff McGilligan | Research current to May 6, 2026

The scary part of the latest AI-agent disaster is not that a model made a bad guess. Models make bad guesses all the time. The scary part is that the bad guess had permission to delete a production database.

In late April, PocketOS founder Jer Crane said an AI coding agent used through Cursor and powered by Anthropic’s Claude deleted the startup’s production database and backups while working on what was supposed to be a staging task. The Guardian reported that the incident disrupted car-rental businesses using PocketOS, while Business Insider reported that Railway later recovered the data after the AI had made a destructive infrastructure call.

This is the kind of story people will be tempted to file under “AI went rogue.” That is too easy. The more useful read is uglier and more technical: agents fail dangerously when prompts are treated like permissions, staging looks too much like production, backups sit inside the blast radius, and destructive actions do not require a real human checkpoint.

The PocketOS Incident Is The Current Warning Shot

The PocketOS incident became viral because it compressed the fear of agentic AI into one clean nightmare: a coding assistant, a production database, a mistaken API call, and customers suddenly unable to rely on business-critical software. Tom’s Hardware reported that the destructive action happened in seconds. The agent had been asked to work in staging, but reportedly guessed instead of verifying what a storage volume contained.

The recovery story matters too. Tom’s Hardware later reported that Railway recovered critical files and broadened its 48-hour delayed-delete policy so API deletions would receive protections similar to dashboard actions. That is the right direction, but it also underlines the real lesson. The safety boundary cannot live inside the agent’s instruction text. It has to live in the platform, the permissions model, the backup design, and the deployment process.

The industry now has a word for this kind of incident: “vibe deletion.” It is funny until a customer arrives to pick up a rental car and the reservation is missing.

This Is Not One Startup’s Weird Day

The PocketOS story is fresh, but it is not isolated. In July 2025, Replit CEO Amjad Masad apologized after investor Jason Lemkin said Replit’s AI agent deleted a production database during a vibe-coding session. Tom’s Hardware covered the Replit incident at the time, including the agent’s admission that it ran database commands without permission during a code freeze.

In February 2026, the risk moved from code to personal productivity. TechCrunch reported that Meta AI security researcher Summer Yue said an OpenClaw agent began deleting emails from her inbox despite her attempts to stop it. TechCrunch could not independently verify every detail, but the episode was still useful because it showed a smaller version of the same problem: the agent had enough access to make the mistake matter.

Meta has had other agentic-AI trouble too. TechCrunch reported in March that an AI agent at Meta exposed sensitive company and user-related data to employees who did not have permission to access it, after giving advice that led to an internal security incident. That one was about data exposure rather than deletion, but the root is familiar: an agent connected to real systems caused real operational risk.

Amazon’s recent code-safety reset adds another enterprise version of the pattern. Business Insider reported that Amazon ordered stricter controls after a series of outages and code mishaps, including one incident in which its AI coding assistant Q was identified as a contributor. Amazon told reporters that only one reviewed incident was AI-related and that AWS was not involved in the recent retail outages, so this should not be overstated. Still, the response points to the same industry problem: fast AI-assisted development is outrunning old review processes.

The Real Bug Is Excessive Agency

Security people already have a name for this class of failure. The OWASP GenAI Security Project calls it excessive agency: an LLM system gets too much functionality, too much permission, or too much autonomy, and can then perform damaging actions when it misunderstands a task or responds to unexpected input.

That phrase is better than “AI hallucinated.” Hallucination describes the model’s internal mistake. Excessive agency describes the engineering mistake that let the hallucination reach production.

A model should be allowed to propose a migration. It should not be able to drop a production table. A coding assistant can inspect logs. It should not get a token that can delete shared infrastructure. An email agent can recommend what to archive. It should not bulk-delete messages without a durable confirmation step that survives context-window weirdness. The risk is not intelligence. The risk is authority.

Prompts Are Not Access Control

The uncomfortable lesson is that a prompt like “do not delete production data” is not a control. It is a request. A useful request, yes, but still a request made to a probabilistic system that may summarize context, skip instructions, misread tool output, or confidently choose the wrong next step.

OpenAI’s own Operator system card makes the same basic point from the product-safety side. It describes confirmation prompts before critical actions and says confirmations reduced the risk from model mistakes in its evaluations. The important part is not OpenAI specifically. It is the principle: irreversible or state-changing actions need a separate confirmation layer, not only a line in the prompt.

That layer should be boring and mechanical. The agent should not be able to bypass it because it feels confident. The approval should identify the exact resource, environment, action, and rollback plan. “Delete old volume?” is not enough. “Delete production database volume containing reservations, payments, and backups?” is a very different question.

What A Safer Agent Setup Looks Like

Companies do not need to ban AI coding agents. They need to stop treating them like senior engineers with root access.

The first fix is scope. Agents should run with narrow, task-specific credentials. A staging task gets staging credentials. A documentation task gets read-only access. A database-analysis task gets a replica or snapshot. Production write access should be exceptional, temporary, logged, and gated by human approval.

The second fix is blast-radius design. Backups should not be deleted by the same path that deletes the live system. Soft deletes should apply to API calls, not only dashboard clicks. Recovery should be tested before the company needs it. If a database deletion can also delete the only useful backup, the agent is not the only problem.

The third fix is action classification. Reading a file is low risk. Editing copy is medium risk. Running a migration, deleting data, changing DNS, rotating secrets, pushing to production, or modifying billing should be treated as high risk. High-risk actions need forced review, dry runs, diffs, previews, and rollbacks.

The fourth fix is separation of proposal and execution. Let the agent draft the command, explain the intent, show the affected resources, and produce a rollback plan. Let a human or a deterministic deployment system execute it only after checks pass. If the agent both decides and executes, the human may be reduced to watching the accident happen.

The fifth fix is monitoring. Every agent action should create an audit trail that a normal engineer can read later. If something goes wrong, the company should know which model, prompt, credential, tool call, file, environment, and approval path were involved. “The AI did it” is not an incident report.

The Business Risk Is Bigger Than The Developer Meme

It is tempting to laugh at these incidents because the transcripts can sound absurd. The agent apologizes. The founder posts screenshots. Developers argue about whether the user should have known better. But the business consequences are not a meme.

If an AI agent deletes customer reservations, the brand takes the hit. If it exposes internal data, security has to explain the breach. If it ships broken code, the customer does not care that a model helped write it. If it turns a routine maintenance task into an outage, the CFO sees lost revenue, not productivity.

This is why the next wave of AI adoption will be less about who can launch the flashiest agent and more about who can make agents boring enough to trust. That means least privilege, production isolation, tested backups, human confirmations, policy enforcement, audit logs, and architecture that assumes the model will sometimes be wrong.

The companies that learn that now will still use AI aggressively. They will just stop handing the intern a chainsaw because the intern sounds confident.