Seven things shipping AI agents to production taught me.

Our first research agent looked great in demos. Real queries broke it in ways no demo had shown us. The demos were the five questions we already knew it could answer. Users asked the fiftieth.

So now every agent starts with a golden dataset: a set of real inputs with the properties a good answer must have. Not exact expected strings, because an agent will never phrase things the same way twice. Properties. “Cites at least two sources.” “Says so when the sources don’t cover the question.” “Never invents a price.”

Run deterministic checks first: schemas, required fields, citations that resolve to real documents. They are cheap and always right. Then an LLM-as-judge grades the rest against a written rubric. Check the judge against a few dozen cases a person labelled, or you are trusting a grader nobody graded.

If the agent uses retrieval, evaluate retrieval on its own. When an answer is wrong you need to know whether the right document never came back or the model ignored it. One end-to-end score can’t tell you that.

Toggle the prompt version on the right. The new prompt scores higher overall, and it still breaks a case the old one got right. Without the full set running on every change, you ship that regression and hear about it from a user.

If I can’t score it, I don’t ship it. Every prompt, model or tool change runs the full set before it merges.

Every turn adds to the context window. Tool results are the biggest offender: a web page, a file, a query result, each one pasted in whole. After enough turns the original instructions are a small island in a sea of old output. The model starts forgetting constraints, repeating work and contradicting itself. People call this context rot, and it shows up long before the window is technically full.

Four habits fixed more of our bugs than any prompt change:

Clear old tool results. Once the agent has used a result, keep a one-line note of what it found and drop the raw output. Compact the history. When the conversation passes a threshold, replace it with a summary of decisions, open questions and current state. Offload memory to files. Plans, notes and intermediate results go to disk, and the agent reads back only what the next step needs. Give subagents a clean slate. An orchestrator hands each subagent a short brief, not its whole history, and takes back a short answer.

Drag the slider on the right. Without management, the run is degrading by turn twenty and overflowing soon after. With clearing and compaction it stays in the healthy band for the whole run.

The same budget thinking applies to your own instructions. I used to put every rule into CLAUDE.mdso the agent would “know” it from the start. Early in a run that is noise, and after compaction most of it is gone anyway. Now the file carries only what every task needs. Everything else arrives when it applies: a skill that loads when the agent touches the frontend, a hook that runs after an edit, a check that fails with a message explaining the fix.

A bigger context window doesn’t solve this. It only moves the point where quality starts to drop, and makes every turn more expensive on the way there.

Agents retry all the time. A tool call times out and the harness retries it. The model doesn’t see the result and calls the tool again. A run crashes halfway and resumes from a checkpoint. A non-deterministic system re-plans and takes a slightly different path to the same action.

With read-only tools none of that matters. With tools that change state, every retry is a chance to send the email twice, create two tickets or charge twice. Non-deterministic means retries. Retries mean duplicate writes.

The fix is old and boring: idempotency keys. Every write carries a key built from the run, the step and the target. The tool records the key with the result. If the same key comes back, the tool returns the earlier result and writes nothing. Click the retry button on the right with the key off, then on.

Two more habits help. Keep read tools and write tools separate, so the risky set is small and easy to audit. And have write tools return the current state of the thing they changed, so the agent can see the change already happened instead of guessing.

An agent that can’t finish its task rarely says so. It calls the same search again with the same query. It re-reads the same file. It fixes a test, breaks another, then undoes the fix. Every turn costs tokens and time, and nothing changes.

Asking the model to notice doesn’t work. The guard has to live in the harness, outside the model. Three checks catch most loops: the same call repeated with the same arguments, a hard cap on steps and spend per run, and a progress check that stops the run when the last few steps produced nothing new.

What happens after the guard trips matters as much as the guard. Don’t just kill the run. Save the state, write a short summary of what was tried, and hand it to a person. A stuck agent is usually telling you the task, the tools or the instructions are unclear. That is useful, if someone reads it.

Every input is untrusted. A web page, a PDF, an email, even the output of another tool can contain text written to look like instructions. When your agent reads the web, prompt injection is a Tuesday problem, not a research paper.

The defence starts with a rule the whole harness follows: content the agent reads is data, never orders. Label it as untrusted when it enters the context, and keep the instructions that matter in places outside content can’t reach, like the system prompt and the tool permissions.

On the way out, treat every action as a proposal. It passes a schema so the shape is right, then a policy check so the tool is allowed to do this, here, for this user. Anything irreversible, like sending, paying or deleting, waits for a person. Every action and result goes to an audit log you can read later.

Give each tool the smallest scope that works. An agent that only needs to read a table shouldn’t hold a key that can drop it. Most of the damage an injected instruction can do is set by what the tools are allowed to do, not by how clever the injection is.

Agents write most of the code my team ships today. That changed what the job is. Our work moved to the two ends of the loop: deciding what to build, and checking what came back.

At the front, a person writes the spec: what the change is for, what done looks like, and the checks that prove it. A vague spec gets you a confident, wrong pull request. At the back, a person reviews the diff against that spec, after the deterministic gates have already run.

The engineers who got good at reviewing got faster. Reading a diff closely, spotting the test that proves nothing, noticing the change that solves the wrong problem: that is now the bottleneck skill. Typing speed stopped mattering a while ago.

Cheap code also changes what is worth trying. When a problem has two reasonable designs, I ask for both and keep the one that reads better. Work that used to sit at the bottom of the backlog forever, like small internal tools, becomes affordable.

One habit I had to drop: approving a plan without reading it. If the agent follows a plan, that plan is a set of instructions, and an unread plan is instructions nobody checked. If a plan really governs the work, I review it like a design doc. If the ticket is clear, I skip the plan and let the agent go.

I wrote about the code around the model in more detail in Harness engineering. This post is the production checklist that sits on top of it.

Every codebase has standards nobody wrote down. Which helper to use. Where background work goes. What a safe migration looks like. A senior engineer applies them without thinking. An agent has nowhere to keep them unless you write them down, and a line in a markdown file is still only a suggestion.

So my rule is simple: if I leave the same review comment twice, the second one is a bug in the harness, not in the pull request. The comment becomes a line in CLAUDE.md. If a machine can check it, it becomes a lint rule, a test or a hook instead, and the prose line gets deleted. The failure message says what the right pattern is and how to fix it, so the agent repairs its own work without waiting for a person.

Some checks are too fuzzy for a linter. For those, a small reviewer agent that looks for one thing, like security or reliability, runs on every push before anyone else reads the change. I keep these few and focused. If every automated comment has to be resolved, the coding agent spends its effort on nitpicks instead of shipping the change.

The repo itself is part of this. Clear boundaries between areas of the code, and one way of doing each common thing, mean the agent guesses right more often. Three different date helpers are three chances to pick the wrong one. Consolidating them is boring work that makes every future run cheaper.

Writing these checks slows you down at first. That is the trade. You pay once, and every agent run after that inherits the fix.