Managing AI Agents in Production: The First 90 Days

Managing AI agents in production comes down to three systems: performance metrics with an explicit error budget, human-in-the-loop escalation paths that route edge cases to a named person, and a review cadence with written kill criteria. Put those in place during the first 90 days and the agent becomes a manageable team member. Skip them and you'll learn about failures the worst way possible — from a customer.

Most of the content out there stops at deployment. This is the playbook for what happens after, based on what we see running agent systems for clients in support, operations, and sales workflows.

The mindset, in one paragraph

We made the full argument in an earlier piece, Write the Job Description Before You Deploy the AI Agent: an agent isn't software you install, it's a hire on probation — it needs a defined scope, a manager who reviews its work, escalation rules for situations above its pay grade, and a 90-day check-in where you expand its role, narrow it, or let it go. This article assumes you've bought that premise and picks up where it leaves off: after go-live. Nearly every failure mode we've been called in to fix traces back to a missing piece of that structure, not to the model itself — so here's how to build the structure, month by month.

Days 1–30: instrument before you trust

The first month is about visibility, not optimization. You cannot manage what you can't see, and most teams go live seeing almost nothing.

The five metrics that matter

Track these from day one, per task type, not as one blended average:

Task completion rate — the share of tasks the agent finishes end-to-end without a human stepping in. Blended averages hide problems: to pick illustrative numbers, an agent can run at 92% on password resets and 40% on billing disputes and still show a respectable overall figure. The split matters more than the headline.
Escalation rate — how often it hands off to a human. High isn't automatically bad early on. A rate near zero in week one is actually a red flag: it usually means the agent is bluffing through cases it should be handing off.
Error rate against an error budget — more on this below.
Cost per completed task — model calls plus tool calls plus the human time spent on escalations and corrections. Teams routinely count only the API bill and miss that review labor is often a large share of the real cost.
Latency to resolution — an agent that's accurate but takes four hours on a task a human does in ten minutes isn't saving anything.

Set an error budget, not a perfection target

Borrow the concept from site reliability engineering. Decide, in writing, what failure rate is acceptable for each task category — say, 2% on order-status lookups, 0% on anything touching refunds over a threshold you choose. When the agent is inside budget, leave it alone and resist the urge to tinker. When it burns through the budget, that triggers a defined response: tighten its scope, add a review step, or pull the task category entirely.

Without a budget, every error becomes a debate. With one, the response is automatic and nobody has to relitigate it at 11pm.

Build the escalation path before you need it

Human-in-the-loop escalation is the highest-leverage part of the whole oversight setup, and it's the piece most teams design last. Design it first.

Three triggers that should always route to a human

Low confidence. When the agent can't ground its answer in your data or its retrieval comes back thin, it hands off instead of improvising.
Category rules. Some things escalate regardless of confidence: legal threats, refund requests above a set amount, anything involving a contract, a visibly angry customer. Write the list down. Ours for a typical support deployment runs 8–12 items.
Human request. If a customer asks for a person, they get a person — first time, no deflection loop. Nothing torches trust faster than an agent that argues.

An escalation needs a catcher

An escalation path that ends in a shared inbox nobody owns is not a path; it's a queue where problems age. Assign a named owner per shift, define a response-time target, and — this is the step everyone skips — log why each escalation happened. That log becomes your improvement backlog. In month two, it's the most valuable dataset you have.

Days 31–60: settle into a review cadence

By the second month you have baseline data. Now you manage the agent the way you'd manage the person: on a rhythm.

Daily (5 minutes): glance at completion rate, escalation rate, and error-budget burn. You're looking for spikes, not trends.
Weekly (30–60 minutes): pull a random sample of 20–30 transcripts and actually read them. Not just the failures — random. Metrics tell you that something drifted; transcripts tell you why. This is where you catch the agent being technically correct but tone-deaf, or confidently wrong in ways that never trip an error flag.
Monthly (half a day): review the escalation log, decide what to feed back into prompts, tools, or training data, and formally revisit scope. Expand what's earning trust; restrict what isn't.

The weekly transcript reading is the discipline teams drop first, and it's the one that matters most. Dashboards measure what you predicted could go wrong. Transcripts show you what you didn't predict.

Days 61–90: write your kill criteria

By day 90 you should have a documented answer to a question most founders never ask: what would make us pull this agent? There are three levels of response, and knowing when to retire an agent rather than keep fixing it is what separates disciplined operators from teams running on hope.

Rework when the failures are patterned

If errors cluster around specific topics, phrasings, or data gaps, that's fixable. Update the prompts, add examples, improve retrieval, extend the tools. Rework is the right call when the agent's scope is sound but its execution inside that scope is weak.

Restrict when the failures are categorical

If the agent handles most of its scope well but reliably fumbles one slice — say, anything multi-account, or anything requiring judgment about exceptions — don't keep tuning endlessly. Carve that slice out and route it to humans. A narrower agent that's trustworthy beats a broader one you have to double-check. Scope reduction is a management decision, not a defeat.

Retire when the economics or the risk stop working

Pull the agent when any of these hold for a sustained period: cost per completed task (including human review time) exceeds what a person costs; the error budget is breached for multiple consecutive review cycles despite rework; or a single failure mode carries damage — regulatory, financial, reputational — that no completion rate offsets. Retirement doesn't have to be permanent. Some of our clients pull an agent back to a supervised draft-only mode for a quarter and relaunch it later with better guardrails.

The mistakes we keep seeing

Set-and-forget. The agent works in week one, attention moves elsewhere, and drift goes unnoticed until a customer complains. Agents degrade quietly as your product, policies, and data change around them.
One blended metric. A healthy overall completion rate hiding one catastrophic category.
No named owner. "The team" monitors the agent, which means nobody does. Every production agent needs one accountable human, the same as every direct report needs a manager.
Treating escalations as failures. Early on, escalations are the system working. Punishing the metric teaches your team to suppress handoffs, which converts visible escalations into invisible errors.

Where to start this week

If you already have an agent live, don't rebuild anything yet. Pull 25 random transcripts from the last two weeks and read them — that alone will tell you whether your real problem is metrics, escalation, or scope. If you're still designing your first deployment, build the escalation path and error budget into the spec before go-live; retrofitting oversight is always more expensive than including it. And if you want an experienced set of eyes on either situation, that's the kind of technical direction we give founders every week at Startupp — bring the transcripts, we'll help you read them.