How to Spot a Real AI Agent

Test any product marketed as an AI agent against the definition the field settled on

min

exercises

Upd. on

Start my lesson

The word "agent" now shows up on everything from chatbots to simple automations, and the label alone tells you very little about what a product actually does. For years, 2 people could talk about agents and mean entirely different things, and plenty of marketing has happily filled that gap.

By 2025, practitioners had converged on a short, practical definition: an agent runs tools in a loop to achieve a goal. Each part does real work. Tools let the model act instead of only writing text. The loop feeds every result back so the model can decide its next move. The goal tells the loop when to stop. Take away any one of them and you're looking at something else, often a fixed workflow where people chose every step in advance. A product that only writes text back, or runs the same steps every time, fails that test however capable it sounds.

That difference matters before you hand over a real task. Workflows are cheaper and more predictable when the steps never change, while agents earn their extra cost on messy, open-ended problems. Selling one as the other even has a name: agentwashing. Judging a product by what it does after its first reply, not by what its product page claims, is the most reliable way to know which one you're getting.

Tools in a loop

The most widely used definition of an AI agent fits in one sentence. Simon Willison, a software developer who writes extensively about AI, put it this way on 18 September, 2025: "An LLM agent runs tools in a loop to achieve a goal." ^[1] An LLM, or large language model, is the kind of AI model behind chatbots. Tools are the actions it can take, such as searching the web or editing a file. The loop sends each tool's result back to the model so it can work out what to do next, and the goal is what the whole loop works toward.

Reaching that sentence took a while. In October 2024, Willison asked his followers to define "agent" and collected 211 answers, which an AI model then sorted into 13 distinct groups ^[2]. Anthropic's December 2024 guide described agents as "typically just LLMs using tools based on environmental feedback in a loop." ^[3] By May 2025, Anthropic's Hannah Moran was putting it more simply, as reported by Willison: "Agents are models using tools in a loop." ^[4]

Each part of the definition rules something out. Without tools, the model can only write text. Without the loop, it acts once and never sees whether the action worked. Without a goal, the loop has no reason to end. The definition also leaves out a popular idea on purpose: an AI that replaces a human employee, which Willison calls his "least favorite definition." ^[5]

Observe, decide, act, check

Inside the loop, an agent repeats the same 4 moves until the work is done. Anthropic describes the cycle for its own agents as gathering context, taking action, verifying the work, and repeating ^[6].

Each pass through the loop has 4 steps:

Observe: take in where things stand, such as the task, earlier results, or the text of a page
Decide: reason about what those observations mean and pick the next action
Act: use a tool, for example running a search or filling in a form field
Check: look at what the action actually produced before starting the next pass

The decide step is what separates an agent from a simple automatic device like a thermostat, which reacts to readings but never reasons about them. A 2022 research paper called ReAct, short for reasoning and acting, showed that language models perform better when they alternate between thinking through a problem and taking an action, with each observation feeding the next round of reasoning ^[7].

The check step keeps the loop grounded. Anthropic notes that agents need to "gain 'ground truth' from the environment at each step," such as the results of a tool or a piece of code that ran ^[8]. In plain terms, the agent compares its progress against what really happened, like an error message or a page that didn't load, rather than what it expected. That's what lets it notice a mistake and change course on the next pass.

The goal ends the loop

The last part of the definition, "to achieve a goal," is what keeps an agent's loop from running forever. Willison added it to signal that agent loops have a stopping condition ^[9]. The agent keeps observing, deciding, acting, and checking until the result matches the goal, then stops and reports back. OpenAI's guide to building agents says the same thing in different words: an agent "recognizes when a workflow is complete." ^[10]

Willison deliberately left out who sets the goal. He considered adding "set by a user" but dropped it, because some agents now receive their goals from other agents, as when a main agent hands a smaller job to a helper ^[11]. The goal is the planned exit, but builders often add a backstop too, such as a maximum number of steps, to keep control if the goal is never reached ^[12].

Workflows vs. agents

Many AI products that look like agents are actually workflows. In a workflow, people decide the steps in advance and the AI model fills in each one. In an agent, the model decides the next step itself. Anthropic draws the line this way: "Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." ^[13]

Anthropic's own examples of workflows are familiar ones: writing marketing copy and then translating it into another language, or sorting customer service messages into general questions, refund requests, and technical support so each one gets handled the right way ^[14]. Both use AI at every step, but the path never changes. OpenAI makes the same cut from the other side: apps that use a model without letting it steer the work, such as simple chatbots or tools that respond once per request, aren't agents ^[15].

Neither design is better across the board. Anthropic notes that "workflows offer predictability and consistency for well-defined tasks," ^[16] and it groups both under the umbrella term "agentic systems." The real question is which one fits the task at hand.

When a loop is worth it

An agent isn't always the right choice. Every pass through the loop takes time and computing, so agents cost more than a single AI reply. In Anthropic's data, agents typically use about 4 times as many tokens as a chat ^[17]. Tokens are the small chunks of text an AI model reads and writes, and more of them means more cost and waiting. Anthropic recommends "finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all." ^[18]

Agents earn that cost on open-ended problems where, in Anthropic's words, "it's difficult or impossible to predict the required number of steps." ^[19] Hugging Face, an AI platform, gives a simple example. A surf trip company can handle requests like checking trip details with a fixed workflow. But a single message about a forgotten passport, a possible 2-day delay, and cancellation insurance breaks every predefined path, and that's where an agent helps ^[20].

Good agent tasks also have a result that can be checked. Willison suggests looking for problems with clear success criteria, where finding a good solution takes some trial and error ^[21]. When the steps are the same every time, a fixed workflow is cheaper, faster, and more reliable.

Agentwashing

Agentwashing is marketing a product as an AI agent when it doesn't actually run tools in a loop toward a goal. Analysts and journalists began calling it that in mid-2025. Gartner, a research and advisory firm, described it as rebranding existing products, such as AI assistants, chatbots, and robotic process automation (software that repeats scripted clicks and keystrokes), without real agent capabilities. Gartner estimates that only about 130 of the thousands of companies selling agentic AI are the real thing ^[22].

Writing in MIT Technology Review in July 2025, Yoav Shoham, a Stanford professor emeritus and cofounder of the AI company AI21 Labs, said the label was being applied to everything from simple scripts to sophisticated AI workflows, and warned that this kind of agentwashing doesn't just confuse customers but "invites disappointment." ^[23] The problem is the claim, not the design: a workflow sold honestly as a workflow isn't agentwashing.

No regulator has used the term so far, but agent-style claims have started showing up in AI-washing cases, where companies overstate what their AI can do. In 2025, the Federal Trade Commission, the American consumer protection agency, sued Air AI. Its complaint cited marketing claiming the product could act on its own across more than 5,000 applications and do the entire job of a full-time worker ^[24]. The case settled in March 2026 ^[25].

The "does it loop?" test

A product page can't show what software actually does, but a few questions can. The published definitions of an agent combine into a quick test. It's a synthesis drawn from several sources, not a published framework from any one of them.

The 4 questions are:

Does it take actions through tools, or only write text?
Does it choose its own next step, or follow a fixed script?
Does it check the result of each step and adjust?
Does it work toward a goal and recognize when it's done?

Each question traces to a source. Tools and goals come from Willison's definition ^[26], and choosing the next step comes from Anthropic's line between workflows and agents ^[27]. Checking results matches OpenAI's description of an agent that "can proactively correct its actions if needed." ^[28] The AI Agent Index, a public catalog of deployed agents hosted at MIT, uses similar criteria, such as reaching a goal without being told exactly how, to decide which systems to include ^[29].

The test works because it judges a product by what it does after its first output, not by its label. A product that only writes text back fails the first question, however capable it sounds. One that runs the same steps every time fails the second. Words on a product page carry little weight on their own, and Willison warns about "excitable use of the word 'autonomous'" in non-technical settings ^[30].