Observability and Cost Control for Agents

Your agent worked in the demo. In production it does something weird on one request in fifty, and your logs show one line: “agent finished, 47 seconds, no error.” That tells you nothing. You cannot see what the model decided, which tools it called, or why the bill for that one request was ten dollars.

Agents need different observability than a normal service. A single request turns into a chain of model calls and tool calls, and the interesting failures live in the middle. This article covers what to capture, how to track cost per run, how to stop runaway loops, and how to evaluate whole runs.

Why agents are hard to observe

A normal HTTP endpoint has one request and one response. You log the input, the output, the latency, done. That breaks the moment you add the agent loop.

One user request becomes many model calls and many tool calls. The user asks “reconcile last month’s invoices,” and under the hood the agent calls the model, runs a database query, calls the model again, hits a payment API, calls it a third time, and answers. If any step goes wrong, a single input/output log cannot show you where.

So the unit you care about is not the request. It is the run: the whole loop from goal to final answer, including every step in between. Trace the run, not the request.

flowchart LR
    REQ([User request]) --> RUN
    subgraph RUN ["One run, one trace"]
        direction LR
        S1[Step 1<br/>model + tool] --> S2[Step 2<br/>model + tool] --> S3[Step 3<br/>model]
    end
    RUN --> ANS([Final answer])

This is the same tracing idea from production AI observability, pushed one level deeper. There, a trace wrapped a single LLM call. Here, a trace wraps a whole loop of them.

What to capture

Structure every run as one trace with an ordered list of steps. For each step, record at least:

The step number and the run ID. So you can order steps and group them back into one run.
The model’s decision. Which tool it chose and the arguments it generated, or that it returned a final answer.
The tool call and its result. The name, the arguments, the raw result, and whether the tool succeeded or threw.
Token usage. Input and output tokens for that model call. Multiply by price to get cost.
Latency. How long the model call and the tool call each took.

The result field matters more than it looks. When an agent goes off the rails, it is usually because a tool returned something the model misread: an empty list, an error string it treated as data, a truncated response. If you log only the model’s decisions and not the tool outputs it saw, you cannot reconstruct why it decided what it did.

Why capture the raw tool result and not just a summary?

The short answer: the model acts on the raw result, so you have to see the raw result to debug it.

A tool that returns {"status": "error", "rows": []} looks fine if you log “query ran, 0 rows.” But the model might read that error and confidently report “there are no invoices,” which is wrong. The bug is in how the model read one specific payload. If your log summarized the payload away, the run looks reasonable and you never find the cause. Store the raw result, truncate very large ones with a note, keep the shape intact.

Cost: an agent loops, so it costs more

A single LLM call has a predictable cost: input tokens plus output tokens, times the price. An agent breaks that intuition, because it calls the model once per loop step, and each call resends the growing conversation.

That second part is the trap. On step five, the model does not just see the latest tool result. It sees the goal, all four previous tool calls, and all four previous results, replayed as input tokens. The input grows every step. A ten-step agent can cost far more than ten single calls, because the later steps carry the weight of everything before them.

So the number you track is cost per run, not cost per call. Sum the token cost across every step and attribute it to the one request that started the run. A run that touches an expensive model ten times with a fat context is where surprise bills come from.

The per-token mechanics and model price differences are covered in cost optimization; the agent-specific point is that the loop multiplies whatever a single call costs.

Why does input cost grow faster than I expect?

Because each loop step resends the whole conversation so far.

The model is stateless. To let it “remember” step three at step four, your code sends the entire message history again on every call. Step one sends a little; step ten sends the goal plus nine rounds of tool calls and results. If a task takes fifteen steps, the early context has been paid for fifteen times. This is why trimming old, irrelevant tool results matters for cost, not just for staying under the context window.

Catching runaway loops

The worst cost failure is the loop that never ends. The model calls tools in circles, each step adding to the context, until you or your bill notices. This is the same runaway loop from the agent’s stop conditions, seen from the cost side.

Use two hard limits, and stop the run when it hits either:

A step limit. Cap the loop iterations. If the agent has not finished in, say, 20 steps, stop and return a partial result. This bounds the worst case.
A budget cap. Track cumulative cost inside the run and stop when it crosses a threshold. This catches both the run that is cheap per step but too long, and the run where a few steps are unexpectedly huge.

The budget cap is the one people skip, and the one that saves you when a single run’s context balloons. Check it inside the loop, before the next model call, not after the run finishes.

flowchart TB
    STEP[Start loop step] --> CHECK{Over step limit<br/>or budget cap?}
    CHECK -->|yes| STOP([Stop, return partial])
    CHECK -->|no| CALL[Model call + tool]
    CALL --> LOG[Log step,<br/>add to cost]
    LOG --> DONE{Final answer?}
    DONE -->|no| STEP
    DONE -->|yes| END([Return answer])

Treat both limits as safety rails you expect to trip occasionally, not rare edge cases. An agent that hits the step limit is telling you something: the task was too hard, a tool is failing silently, or the model is stuck. That signal is only useful if you logged the run.

Evaluating runs, not single outputs

Standard LLM evals score one output against one input: given this prompt, was the answer good? Agents need more, because a run can reach a correct answer through a terrible path, or fail despite every step looking fine on its own.

Evaluate the run on three axes:

Outcome. Did the agent achieve the goal? Closest to a normal eval.
Path. How did it get there? A 3-step run and a 25-step run that reach the same answer are not equally good. The long one costs more and is more likely to break.
Cost and steps. Track average and worst-case cost and step count per task type. A regression here is a real problem even when outcomes stay correct, because it hits your bill and latency.

The practical version: keep a set of representative tasks, run the agent over them, record outcome, step count, and cost for each. When you change a prompt, a tool description, or the model, re-run the set and compare. A change that improves answers but doubles the average steps is a trade you want to make on purpose, not find in the invoice.

A step log in code

Here is the shape of logging one loop step with tokens and cost. Drop it inside your agent loop, right after each model call.

def log_step(run_id, step, response, tool_name, tool_result):
    usage = response.usage
    cost = (usage.input_tokens * PRICE_IN
            + usage.output_tokens * PRICE_OUT)
    logger.info("agent_step", extra={
        "run_id": run_id,
        "step": step,
        "tool": tool_name,
        "tool_ok": tool_result.ok,
        "input_tokens": usage.input_tokens,
        "output_tokens": usage.output_tokens,
        "cost_usd": round(cost, 6),
    })
    return cost  # caller adds this to the run's running total

The caller keeps a running total and checks it against the budget cap before the next step. Copy PRICE_IN and PRICE_OUT from your provider’s current rates rather than guessing; prices change. With this in place, one query by run_id gives you the whole run: every decision, every tool result, and where the money went.

Common beginner mistakes

Logging the request, not the run. A single input/output line hides the entire loop where failures live. Trace every step.
Dropping tool results from logs. The model acts on tool output, so you cannot debug decisions without seeing what it saw.
Tracking cost per call. The loop resends growing context, so per-call numbers understate the real per-run cost.
No budget cap. A step limit alone does not catch a short run with a huge context. Cap cost too.
Evaluating only the final answer. A correct answer reached in 25 expensive steps is still a regression worth catching.
Checking limits after the loop. Check step count and budget before each model call, or the runaway has already run away.

Questions you will face in production

“How much detail should I log per step? Won’t full traces cost a fortune to store?” Log the full structure of every step, including raw tool results, but truncate very large payloads with a note about what was cut. Storage is cheap next to a debugging session with no data. Sample down only if volume is genuinely a problem, and never sample away failed runs; those are the ones you most need.

“What is a sane starting step limit and budget cap?” Start conservative and loosen with data. A step limit around 15 to 20 and a per-run budget a few times your expected cost will catch true runaways without killing legitimate long tasks. If the step limit fires constantly, the task or a tool is the problem, not the limit.

“How do I know if a change made the agent worse?” Keep a fixed set of tasks, re-run the agent over them after each change, recording outcome, steps, and cost, and compare against the previous run. That turns “it feels slower” into a number you can act on.

What to remember

The unit of observability for an agent is the run, not the request; trace the whole loop
Capture per step: the model’s decision, the tool call and its raw result, tokens, and latency
Cost per run, not per call: the loop resends growing context, so later steps are the expensive ones
Use two hard limits, a step limit and a budget cap, and check them inside the loop before each call
Evaluate runs on outcome, path, and cost, not just the final answer
A tripped limit is a signal about the task or a tool, not just a safety net

What to study next

You can now see inside a run and bound what it costs. The next step up in complexity is coordinating several agents at once, where tracing and cost tracking get harder because runs nest inside runs. Read multi-agent systems for how that changes the picture, and keep cost optimization handy for the per-token mechanics that drive every number here.