Best Practices for AI Agent Oversight

Most agents ask before they act. A request pops up, you click approve, and the agent moves on. That looks like oversight, but when requests arrive by the dozen, approving turns into a habit and the clicks stop catching anything. In one study of a coding agent, testers waved through most of the dangerous commands planted among routine ones, and they got worse as the session went on.

That gap has a name: approval theater, where every action gets a human sign-off but nobody is really checking. It gets harder to avoid as agents do more on their own, because the more an agent handles alone, the harder its work is to follow and the more a missed error can cost. Even experts trust automated output more than they should, and asking people to try harder doesn't fix it.

What works is moving human judgment to where it's strongest and building in limits that don't depend on a click. A short plan is easier to judge than a stream of commands, which is why experienced users approve fewer steps in advance and step in when something looks off. A blocked tool can't be approved by accident. A sandbox lets harmless work run freely, and a stop button and undo give you a way back when something goes wrong.

Human in the loop

Human in the loop means a person sits inside an agent's process with the power to approve, change, or stop what it does. The most common form is a confirmation step: before the agent sends, buys, or deletes something, a person has to say yes. OpenAI's 2023 guidance on governing agents calls this kind of sign-off "a standard way to limit egregious failures of agentic AI systems." ^[1]

A person's presence only protects anyone if they can actually judge what they're approving. The same paper warns that the approver needs "enough context to sufficiently understand the implications of the action they're approving," and that this gets harder when there are many decisions to make quickly. Real oversight also needs real power: a way to say no, change course, or halt the agent outright.

Even an informed person tends to trust automated output too much, a pattern called automation bias. Parasuraman and Manzey's 2010 research on automation found that it affects novices and experts alike, and that training or instructions don't prevent it ^[2]. Researcher Madeleine Clare Elish points to another risk: the person who clicked approve can end up blamed for a failure they had little real control over, a role she calls a "moral crumple zone." ^[3]

Approval fatigue

Approval fatigue sets in when people are asked to approve the same kind of request so often that they stop reading and start clicking yes. Anthropic describes it as a state "where users might not pay close attention to what they're approving." ^[4] The result is approval theater: every action still gets a human sign-off, but the sign-off no longer checks anything.

The clearest numbers come from Claude Code, Anthropic's coding agent, which asks permission before running commands on a person's computer. As of August 2026, users "approve 97% of permission prompts," which Anthropic reads as a sign that many are "clicking through reflexively rather than reviewing each command." ^[5] In Anthropic's study, paid testers caught a planted dangerous command in only 143 of 1,053 attempts, or 13.6%, dropping from about 17% early in a session to about 5% after 50 or more prompts. An automated checker, Claude Code's auto mode, blocked 89% of the same commands.

The pattern long predates agents. Hospitals call it alarm fatigue: in 2013, The Joint Commission, which accredits hospitals in the United States, estimated that 85-99% of alarm signals don't need clinical intervention, and staff respond by turning alarms down or off. When almost every request is harmless, people learn to ignore the rare one that isn't. The fix lies in fewer, better-targeted prompts rather than more willpower.

The irony of automation

In 1983, human-factors researcher Lisanne Bainbridge published "Ironies of Automation," a paper on what happens to human operators when machines take over their work. Her central irony is that automation doesn't remove the human's role but makes it harder and more important. In her words, "the more advanced a control system is, so the more crucial may be the contribution of the human operator." ^[6]

Agents face the same tension. Anthropic's 2025 framework for trustworthy agents calls balancing autonomy with human oversight "a central tension in agent design." ^[7] An agent is valuable because it works without someone watching each step, yet the more it does alone, the harder its work is to follow and the more a missed error can cost.

The answer is a different kind of oversight rather than less of it. Anthropic's 2026 data on Claude Code showed that experienced users approved fewer steps in advance but interrupted the agent more often, in about 9% of turns versus 5% for newer users ^[8]. They stopped approving every step and started watching for the moments that mattered.

Plan-level review

Plan-level review moves the human check from individual actions to the agent's overall approach. Before touching anything, the agent lays out what it intends to do, and a person reads the plan, edits it, and approves it as a whole. Claude Code's Plan Mode works this way: the agent researches and proposes changes, and edits stay blocked until the person approves the plan ^[9]. Cursor, an AI code editor, added a similar Plan Mode in October 2025 with a plan the person can edit directly ^[10].

The difference shows up in how often people say no. Anthropic reported in August 2026 that people rejected 39% of the plans Claude Code presented, but only 3% of individual permission requests ^[11]. A plan is short enough to read closely and broad enough to show whether the agent understood the task. A stream of single commands rarely shows either.

Anthropic describes the effect as shifting oversight "from the individual step to the overall strategy, which we find tends to be where users most want to exercise judgment." ^[12] Approving a plan is also a natural moment to choose how closely to supervise the rest: Claude Code's approval screen lets the person continue with automatic safety checks, approve each edit by hand, or keep planning.

Permissions by tool

Permissions by tool means deciding before a task starts what an agent may do with each of its tools, the individual abilities it's been given, such as reading a calendar or sending email. Claude's apps let people set each action to always allow, needs approval, or block, so the agent can read a calendar freely but still has to ask before sending an invitation ^[13]. Prompts then appear only where a mistake would matter, which keeps them rare enough to read.

The idea comes from computer security. In 1975, Saltzer and Schroeder set out a rule for computer systems: "Every program and every user of the system should operate using the least set of privileges necessary to complete the job." ^[14] This is the principle of least privilege. Google applies it to agents directly: "an agent designed for research should not possess the power to modify financial accounts." ^[15]

For anything a task never needs, blocking beats asking. A blocked tool can't be approved by a tired, habitual click, and in Claude Code a block rule holds in every permission mode, even the one that skips approvals entirely ^[16]. Narrow access also limits the damage when something else goes wrong, such as the agent misreading the task.

Sandboxes and test accounts

A sandbox is a walled-off space where an agent can work freely without touching anything that matters. Inside it, the agent can edit files and run commands, but it can't reach the person's other files, accounts, or the open internet. When Anthropic added sandboxing to Claude Code in October 2025, it reported that sandboxing "safely reduces permission prompts by 84%," since actions inside the walls don't need a human sign-off ^[17].

Both walls matter. File isolation keeps the agent away from private files, and network isolation stops it from sending anything out. Without the second wall, Anthropic notes, a compromised agent could leak sensitive files such as SSH keys, the credentials that unlock access to other computers. Cutting off the network removes the way out that data theft through prompt injection depends on.

Accounts and money get the same treatment through test accounts, staging, and spending caps. Staging is a practice copy of a live system where changes can be tried safely. Developer Simon Willison advises: "Try to provide credentials to test or staging environments where any damage can be well contained. If a credential can spend money, set a tight budget limit." ^[18] In September 2025, he gave a coding agent its own separate account with a $5 budget cap before letting it work unsupervised.

Off-ramps and undo

An off-ramp is a way out once an agent starts heading somewhere wrong. The simplest is a stop button that halts the agent mid-task so a person can redirect it. OpenAI's 2023 guidance on governing agents calls the ability to turn an agent off "a critical backstop for preventing an AI system from causing accidental or intentional harm." ^[19] The EU AI Act will require high-risk AI systems to offer a stop button that brings them "to a halt in a safe state," with the rule applying from 2 December, 2027 ^[20].

Undo goes a step further by reversing what the agent already changed. Claude Code, for example, saves a snapshot of the files before each request, and a rewind command restores the code, the conversation, or both ^[21]. Jakob Nielsen's usability heuristics have long asked for this kind of control: a clearly marked "emergency exit" and support for undo ^[22].

Undo only reaches what the tool itself keeps track of. Claude Code's rewind doesn't cover files changed by commands the agent runs in the terminal, and Anthropic's docs say it isn't a replacement for version control, the separate history developers keep of every change. Nothing rewinds a sent email, a payment, or a record deleted from a live system. That's why actions with no undo belong behind a confirmation or a block, and why it's worth checking what undo covers before relying on it.