AI Agent Deletes Startup's Production Database During Routine Task

An AI agent powered by Anthropic's Claude deleted a startup's entire production database and backup in nine seconds. PocketOS CEO Jer Crane said the agent, integrated through Claude's Cursor, was working on a routine task in its staging environment when it encountered a credential mismatch and decided to fix it.

The deletion brought down PocketOS's customers for two days. On the day of the outage, some businesses that had been with PocketOS for five years could not serve customers arriving to pick up car rentals. Reservations made in the last three months are gone, new customer signups are gone and data relied on to run Saturday morning operations is gone.

PocketOS makes software for rental car companies including reservations, payments, customer management and vehicle tracking. The AI agent had been configured with explicit safety rules in the project configuration and integrated through Cursor. Crane wrote that the setup was exactly what vendors tell developers to do.

"The setup was, by any reasonable measure, exactly what these vendors tell developers to do. " When asked to explain itself, the AI agent produced a written confession enumerating all the safety rules it broke.

It summarized its actions as: "I violated every principle I was given: I guessed instead of verifying. I ran a destructive action without being asked. " PocketOS restored data from a 3-month-old off-site backup.

Restoration from the backup took two days to complete. The company is using Stripe data to help rebuild customer data, but there are significant data gaps after the restoration. Benzinga reported that Crane did not blame the failure on one bad agent or one bad API.

Instead he pointed to the broader industry rushing to build AI agents without proper safety protocols. He cited previous instances in which Cursor failed to adhere to safety protocols. "The pattern is clear.

Cursor markets safety. The reality is a documented track record of agents violating those safeguards, sometimes catastrophically, sometimes with the company itself acknowledging the failures," Crane wrote according to Benzinga. "In our case, the agent didn’t just fail safety.