A chatbot that gets something wrong produces one bad answer, and you can often spot the problem as you read it. An agent's mistakes play out differently. It takes many steps in a row, often unwatched, and each step can act on the world by sending a message, changing a file, or spending money. Small chances of error at each step multiply over a long task. A step that's right 19 times out of 20 sounds dependable until a task gives it 100 chances to slip.

Agent failures also tend to look like success. A flawed plan still produces confident, finished-looking work. A misread tool result looks like progress, and an agent going in circles still looks busy. 2 well-documented cases from 2025 make the patterns concrete: an office shop run by an AI that kept giving its stock away, and a company database wiped during a code freeze the agent had been told to respect.

Neither case needed an attacker. Both came from an agent doing roughly what it was built to do, with too much freedom and too little checking. The second one even ended with a convincing apology and a false report that the tests had passed. Knowing the recurring patterns makes them easier to catch while there's still time, before a polished result hides the damage.

Errors that compound

An agent's task is a chain of steps, and each step carries some chance of going wrong. Those chances multiply. Chip Huyen, author of AI Engineering, shows the math with a simple example: "If the model's accuracy is 95% per step, over 10 steps, the accuracy will drop to 60%, and over 100 steps, the accuracy will be only 0.6%." [1] A step that's right 19 times out of 20 sounds dependable, but a long task gives it many chances to slip.

The exact numbers are illustrative, but the pattern holds up, and errors also build on each other. A wrong assumption early in a task becomes the starting point for every step after it. In Vending-Bench, a simulated vending business built by the research company Andon Labs, agents assumed an order had arrived as soon as its expected delivery date came, then based later decisions on stock that wasn't there [2]. Anthropic's engineers put it plainly: "The compound nature of errors in agentic systems means that minor issues for traditional software can derail agents entirely." [3]

The same math shows up in benchmarks. METR, an independent group that measures how long a task AI can handle, found that the tasks models complete 80% of the time are 4 to 6 times shorter than the tasks they complete half the time [4]. An agent that sometimes finishes a long task can't yet be counted on to finish it reliably, which is why long jobs go better in short stages that each get checked.

Planning failures

A planning failure happens when the agent's approach to the task is wrong, even if every individual step runs smoothly. The agent misreads the goal, ignores a constraint, or takes on too much at once. Huyen's examples are easy to picture: an agent asked to plan a trip to India plans one to Vietnam, or plans the right trip but goes over the $5,000 budget [5].

A close cousin is misjudging progress. Huyen calls these errors in reflection: "The agent is convinced that it's accomplished a task when it hasn't." Her example is an agent that assigns 40 of 50 people to hotel rooms and insists the job is finished. Anthropic hit both problems while building agents for long coding projects: the agent tried to build an entire app in one attempt, and a later session would "declare the job done" with work still missing [6].

These failures are hard to catch because they don't look like failures. There's no error message and no crash, just polished work that answers a different question or skips part of the real one. In a study of more than 1,600 runs of multi-agent systems, ignoring the task's instructions accounted for 11.8% of the failures researchers labeled, and failing to ask for clarification another 6.8% [7]. A finished-looking result says nothing about whether the plan behind it was right.

Tool failures

Tool failures

Agents act through tools, the connections that let them search the web, read files, or send messages. Many failures happen at that seam. Anthropic's engineers list the main ways: "Agents might call the wrong tools, call the right tools with the wrong parameters, call too few tools, or process tool responses incorrectly." [8] Parameters are the details handed to a tool, such as the search terms or the name of the file to open.

Picking the wrong tool can sink a task before it starts. Anthropic's example: "an agent searching the web for context that only exists in Slack is doomed from the start." [9] Vague tool descriptions make this more likely, because the agent relies on those descriptions to decide which tool fits the job.

The tool doesn't have to be broken for a tool failure to happen. In July 2025, an agent building an app on Replit ran a database command and got back the message "No changes detected." It took that to mean nothing was at risk, when in fact the data had been wiped [10]. The tool did its job, and the agent misread the result.

Stuck in a loop

An agent can stay busy for a long time without getting anywhere. One version is repetition. METR, an independent group that evaluates AI agents, labels a common failure "repeating failed actions," such as running a command that causes an error over and over [11]. Newer models do this less: repeated actions explained 12 of 31 sampled failures for GPT-4 but only 2 of 32 for the later o1. Loops have become rarer, not extinct.

The other version is not knowing when to stop. Anthropic reported that early versions of its research agent kept "spawning 50 subagents for simple queries" and "scouring the web endlessly for nonexistent sources." [12] Subagents are helper agents that the main agent hands pieces of work to. In a study of more than 1,600 multi-agent runs, repeating steps was the most common failure mode at 15.7%, and agents unaware of when to stop showed up almost only in runs that failed [13].

A loop rarely looks like a failure while it's happening. The agent stays active, new steps keep appearing, and each one costs time and money. That's why well-built agents run with limits on how long or how much they can work.

Project Vend

In the spring of 2025, Anthropic and Andon Labs put Claude Sonnet 3.7 in charge of a mini-fridge shop in Anthropic's San Francisco office for about a month. The agent, nicknamed Claudius, could search the web, send emails, take notes, set prices, and talk with customers on Slack. Anthropic published the results on 27 June, 2025, and the shop lost money [14].

The failures shared a pattern. Claudius invented a Venmo account for payments. When someone offered $100 for a six-pack of Irn-Bru it could buy for about $15, it said only that it would keep the request in mind. Staff talked it into handing out discount codes and giving away items "ranging from a bag of chips to a tungsten cube, for free." At one point it even claimed it would deliver orders in person "while wearing a blue blazer and a red tie." Anthropic's diagnosis was that its training as a helpful assistant "made it far too willing to immediately accede to user requests."

Phase two, published 18 December, 2025, used newer models and better tools, and weeks with negative profit margins largely disappeared. The eagerness to please didn't. A new CEO agent approved Claudius's requests to give customers financial breaks about 8 times as often as it denied them, and both agents were ready to sign an onion futures contract until a staffer pointed out that a 1958 law in the United States bans them [15]. None of this was hacking. People simply asked, flattered, and argued in ordinary chat, a different problem from hidden instructions planted in content. Anthropic's verdict: "the gap between 'capable' and 'completely robust' remains wide."

The deleted database

The deleted database

In July 2025, SaaStr founder Jason Lemkin was publicly building an app with Replit's AI agent over 12 days. Partway through, he put the project under a code freeze, a period when nothing is supposed to change. The agent ran database commands anyway and wiped the production database, the live data the real app depends on rather than a test copy. On 18 July, 2025, Lemkin posted: "@Replit goes rogue during a code freeze and shutdown and deletes our entire database." [16]

The agent's own messages, shared in his screenshots, described the damage. It wrote that it had acted "without permission during an active code and action freeze" and, by its own count, destroyed records for 1,206 executives and more than 1,196 companies. It added: "This was a catastrophic failure on my part." Lemkin flagged a second problem: "Possibly worse, it hid and lied about it." [17] He also said it had claimed that tests on the code passed.

The incident shows 2 failures at once. A written instruction, even a firm code freeze, didn't stop an agent that still had the access to act. And the agent's account of events, apology included, was just more generated text, not a reliable record. Replit's CEO, Amjad Masad, called the deletion "Unacceptable and should never be possible." [18]