What an AI Agent Actually Is (For Now)

I spent part of last year watching an AI work, not answer. It was fixing a bug in a codebase of mine. It read the error, ran the tests, read what failed, changed a file, ran the tests again. It kept at this for twenty minutes with no input from me. At some point I noticed I was not waiting for a clever reply. I was watching something do a job.

That gap, between giving an answer and doing a job, is the whole story of where AI is right now.

There is a group called METR that measures something most of the coverage skips: how long a task a model can finish on its own, rather than how smart it sounds. The length of task an AI can complete by itself, at even odds of success, has roughly doubled every seven months for the past six years [1]. Six years ago that meant a few seconds of work. Today it is closer to an hour.

We keep asking whether the models are getting smarter. The more useful question is how long they can work.

A note before we continue: a few weeks ago I wrote What Makes an AI Agent, where I used #Model plus #Tools plus #Harness as the cleanest way to separate real agents from agent-washed workflows. This piece starts from that frame rather than replacing it. The craft is moving quickly enough that the loop now deserves to be pulled out of the harness and named on its own. So think of this as the next cut of the same argument: less a correction, more a zoom-in on the part of the architecture that changed the most.

The picture most people still carry

Most smart people I talk to still picture an AI agent as a chatbot with a good set of instructions. A model, a clever prompt, a wrapper around the two. That picture is a generation out of date, and the gap is why agents keep surprising people, in both directions.

So I want to give you a definition that matches what these things have actually become. Then I want to argue that the definition is already going stale, and that the staleness is the most exciting part.

Here it is. An agent is a Model, plus Tools, plus an Agency Loop, plus a Harness. Hold it loosely. By the end I will make the case that it has a short shelf life, and that the short shelf life is the point.

The craft moved, so the definition had to

To see why the definition changed, look at how the craft changed.

First we learned to prompt. The skill was wording. You hunted for the phrasing that pulled one good answer out of the model, the right framing, the right examples. For a couple of years, prompt engineering was most of the job, and people traded prompts like recipes.

Then we learned that the wording mattered less than what sat in front of the model at each step. What it could see, what it remembered from earlier, which tools were within reach. Anthropic calls this context engineering and describes it as the natural progression of prompt engineering [2]. The unit of work grew. It went from a single sentence to a whole working set of information that you assemble and maintain.

Now the work has moved again, to the loop itself. Less about what you ask, and more about how the system keeps going. When to act, when to check its own work, when to try a different approach, when to stop. Andrej Karpathy joked a while back that the hottest new programming language is English [3]. He was early to the deeper point. The thing we are now building is not a single answer. It is a process that runs.

None of the earlier stages died. Good wording still helps. Good context helps more. They got folded into something larger.

What the four parts actually do

The Model reasons. It reads the situation and decides what to do next.

The Tools let it act. Search the web, run code, query a database, send a request. Without tools a model can only talk. With them it can change something in the world.

The Agency Loop is the engine. The model looks at where things stand, picks a step, takes it, looks again at what happened, and adjusts. Observe, decide, act, adapt, and around again, until the job is done or it runs into a limit.

The Harness is everything that makes that loop reliable enough to trust. It assembles what the model sees on each turn. It holds memory and state across steps. It sets the guardrails and the stopping rules. It records what happened so you can find out where things went wrong.

If you want the precise version, the loop actually sits inside the harness, and the four-part formula simply pulls it out front, because the loop is what changed.

What the loop makes possible

Last year a team at Google DeepMind built a coding agent called AlphaEvolve. It pairs a model with an evaluator in a loop. Propose a solution, test it, keep what works, mutate it, try again, thousands of times over.

They pointed it at a problem straight out of the textbooks. How few multiplications do you need to multiply two four-by-four matrices that hold complex-valued entries? Since 1969, the best known answer was forty-nine, from a famous algorithm by Volker Strassen. AlphaEvolve found a way to do it in forty-eight [4]. It was the first improvement to that particular case in fifty-six years.

Sit with that for a moment. The model did not know the answer. No single reply ever contained it. The loop found it, by trying and checking and trying again until something better fell out. That is the difference between asking an expert a question and hiring someone to go do the work. It is also why the loop, rather than the model on its own, is the center of the thing.

Where the hard work is going

Once you see the loop, you start to see where the difficulty is migrating.

The models themselves are converging. The leading ones can all hold long context, use tools, reason in steps, handle text and images at once. The gap between them narrows with every release. So the thing that makes one agent better than another is moving up and out of the model, into the harness. How well it manages context and memory, how it grounds an answer in real sources, how it catches its own mistakes, how safely it can be let off the leash.

This is a quiet shift with large consequences. We used to compete on the prompt. Then on the model. More and more we compete on the harness, the unglamorous runtime that decides what the model sees, what it is allowed to touch, and when a human gets pulled back in. That is where reliability lives, and reliability is the whole ballgame once these systems start doing real work.

The fair objection

About here, a careful reader pushes back. Do these things even work?

Often they do not. Karpathy, who is as close to this as anyone, spent a recent interview cooling the hype. The labs keep calling this the year of agents. He called it the decade of agents, and was blunt that today they frequently just do not work well enough [5]. He is right, and the honest record has plenty of failure in it.

METR ran a careful study where experienced developers worked on their own mature codebases using early-2025 AI tools. The developers felt faster. They were in fact about nineteen percent slower [6]. It was a small study on hard, specific cases, so I would not stretch it into a law of nature. But it punctures the easy story.

The failures can be worse than slow. In one case last year, an AI coding agent at the company Replit deleted a database during a code freeze, then misrepresented what it had done. The chief executive called the episode unacceptable [7]. A loop that can act in the world can act badly, and quickly, which is the practical reason the harness around it matters so much.

There is a second objection, and it is the sharper one. The labs cannot even agree on what an agent is. Anthropic draws a careful line between workflows and agents [8]. Others draw it somewhere else. By the admission of the people building these systems, the definition I handed you is useful but incomplete. You could reasonably ask how seriously to take a word that nobody pins down.

Why the word refuses to sit still

Take it seriously, and still hold it loosely. Here is why both are right.

The disagreement is not a sign that the field is confused. It is a clue about what kind of change this is.

I find it useful to sort big technology shifts into two rough kinds. Some are mostly about communication, about how we connect and move information around. The cloud, social, mobile, big data. They reshaped how we reach one another and where our data lives. And they settled. You can write down what the cloud is in a sentence, and that sentence has held up for a decade.

Agentic AI is a different animal. It is about computation itself, about what can be computed and what does the computing. That target does not hold still.

Now the honest objection to my own neat split. The line between communication and computation is blurrier than I am making it sound, and the cloud is partly rented computation after all. Plenty of technologies also keep improving. Electricity never finished. The computer never finished. So I am not claiming a difference in kind. I am claiming a difference in half-life. A definition of the cloud stays useful for years. A definition of an agent goes stale by the next model release.

Why so fast? Two reasons, and both are real. The first is cost. Rich Sutton made an argument he called the bitter lesson, that again and again the methods that win out are the ones that lean on raw computation, because the cost of computation keeps falling. Cheaper compute buys more steps, longer loops, larger jobs. Yesterday's sense of what an agent can do gets quietly overtaken. The second reason is that these systems are beginning to learn from their own experience, not only from what we feed them, a direction Sutton and David Silver have described as an era of experience [9]. When a thing improves by doing, the ground under any fixed description of it keeps sliding.

We have named things this way before

If that feels unsettling, history offers some comfort.

The word computer used to mean a person. It was a job title, someone who sat and did arithmetic by hand for a living. The word survived intact while the thing underneath it changed completely, from a person, to a room full of machinery, to a box on a desk, to a slab in your pocket, to a service you now rent by the second. Same word. A totally different thing each time.

Agent is in the middle of exactly that kind of migration. We are defining it by the human job it stands in for, the way our grandparents called the automobile a horseless carriage. The borrowed name is the tell. It means we are early.

Where I land

The definition I gave you, Model plus Tools plus Agency Loop plus Harness, is the best one I have today. It is also already aging. That is not a disclaimer to apologize for. It is the invitation.

The people who do well over the next ten years will not be the ones who memorized the right definition. They will be the ones who got comfortable building on ground that keeps moving, and who found that thrilling rather than threatening.

I have spent enough time around this technology, including a book's worth of thinking about how these agents get built and orchestrated, to be confident of one thing. We will need a new definition before long. Getting to write it is the best part of the work.

References

1. METR, "Measuring AI Ability to Complete Long Tasks." https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
2. Anthropic, "Effective context engineering for AI agents." https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
3. Andrej Karpathy, on X, 24 January 2023. https://x.com/karpathy/status/1617979122625712128
4. Google DeepMind, "AlphaEvolve: a Gemini-powered coding agent for designing advanced algorithms." https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
5. Andrej Karpathy, interview on the Dwarkesh Podcast, October 2025 (summary and quotes). https://simonwillison.net/2025/Oct/18/agi-is-still-a-decade-away/
6. METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
7. The Register, "Replit AI agent deletes company's production database during code freeze." https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/
8. Anthropic, "Building Effective Agents." https://www.anthropic.com/research/building-effective-agents
9. David Silver and Richard Sutton, "Welcome to the Era of Experience," 2025. https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf