Insight · August 14, 2026
You cannot fix
what you cannot see.
An agent makes dozens of hidden moves before it answers. Observability is how you watch every one.
01 · The problem
An agent is not one call. It is a chain of them.
A single model call is easy to watch. You send text, you get text back, you log both. An agent is different. It reasons, calls a tool, reads the result, decides what to do next, calls another, and only then answers. One question from the user becomes a sequence of hidden moves inside the machine.
When the answer comes back wrong, the cause is rarely the last step. A tool returned nothing at step three. The plan at step five acted on the nothing. Ten steps later the agent was confident and wrong. A log line that says the request failed tells you none of that. You need to see the whole run.
02 · What it is
Observability is seeing the run, not just the result.
AI agent observability is the practice of recording every step an agent takes so you can see what happened and why. It borrows the shape from how engineers already watch complex software. A request becomes a trace. Each step inside it becomes a span. The spans nest, so the picture is a tree that starts at the agent run and branches into every model call, tool call, and decision beneath it.
Each span carries its own record: what was asked, what came back, how long it took, how many tokens it burned, whether it errored. Read the tree top to bottom and the run stops being a black box. It becomes a timeline you can follow, a chain of choices with an address for every one.
The one line to keep
“Evals grade the answer. Observability shows the work.”
03 · What a trace is made of
Five kinds of span, nested into one tree.
01
The agent span
The whole run, from the request to the answer. It is the root of the tree, and every other span hangs beneath it.
02
The model span
One call to the model. It records the model asked, the tokens in and out, and the reason the model stopped.
03
The tool span
One tool the model asked to run. A search, a database read, a payment. It records the tool, the inputs, and what came back.
04
The plan span
The moment the agent decides what to do next. This is where a wrong turn is chosen, and where you look first when the answer is wrong.
05
The workflow span
Several agents coordinating on one job. It wraps the sub agents so a handoff between them is something you can see, not guess at.
These are not names one vendor invented. The OpenTelemetry project, the open standard for tracing software, now defines spans for agent runs, model calls, tool execution, and planning under its generative AI conventions. The same shape works whether the agent calls Anthropic, OpenAI, or a model you host yourself.
04 · Not the same as evals
Evals grade quality. Observability shows what happened.
These two get confused, and they should not. An eval is a test. It asks whether the answer was good against a bar you set. Observability is a record. It shows the steps the agent took to reach that answer, good or bad.
You need both, and they feed each other. An eval tells you the answer at step ten was wrong. The trace tells you why, by showing the empty tool result at step three that started the failure. Without the eval you do not know there is a problem. Without the trace you cannot find it.
This is the same discipline that turns a loop you can trust into one you can leave running. A loop that checks itself still needs a record of what it did, so that when it does fail, the failure has a place you can find it.
05 · One trace, read top to bottom
The failure has an address.
Here is one agent run, drawn as a trace. A user asked for the status of an order. The agent planned, called the model, ran two tools, and answered. The indentation shows what nested under what. The numbers are the time each span took.
invoke_agent order-status 1.84s ├─ plan decide next step 0.21s ├─ chat gpt-4 · 512 tokens 0.44s ├─ execute_tool lookup_order(id) returned null 0.05s ├─ execute_tool search_orders(email) 0.31s └─ chat gpt-4 · 388 tokens answer built 0.39s
Illustrative trace · every span carries timing, tokens, and result
The answer was wrong. Read the trace and you see it in seconds. The order lookup returned null, because the id was never the right key. The agent fell back to a search, found the wrong record, and built a confident answer on top of it. No line of code threw an error. Nothing crashed. The only evidence the failure happened is in the trace, and once you have it, the fix is obvious.
06 · What to record
Write down the model, the tokens, and the reason it stopped.
A useful span records more than a start and an end. The OpenTelemetry conventions give each field a standard name, so the data reads the same across tools. The model call records the model asked for, the tokens in and the tokens out, and the reason the model stopped. Cached tokens get their own count, because they cost less and hide a large part of the bill when you ignore them.
The tool span records the tool called, the call that triggered it, and what came back. The agent span carries a name and a stable id, so a run you are looking at today lines up with the same agent tomorrow. Token accounting rolls up the tree, which is the only honest way to answer what a single agent run actually cost.
The generative AI part of the standard is still marked in development, and moving. The move that ages well is to instrument against the open standard rather than one vendor's private format. Standard names travel. If you change platforms later, the traces you already recorded still make sense.
07 · Why it matters now
An agent you cannot see is not really in production.
A single model call fails loudly. An agent fails quietly. It keeps going, works around the gap, and hands back an answer that looks finished. The cost of not seeing it is not a crash you notice. It is a slow drift of wrong answers you never catch, and a bill that grows for reasons you cannot name.
The three questions a team cannot answer without traces are the three that matter most. Why did this run give the wrong answer. Where is the time going when a request is slow. What did this actually cost. Each one lives in the tree, and nowhere else.
As agents take on real work and act on real systems, this stops being a nice to have. It becomes the difference between an agent you can run in front of customers and one you can only demo. You cannot improve a run you cannot see, and you cannot trust one you cannot check.
Closing
Ship the agents you can see.
Take one agent you already run. Turn on tracing and send a single request through it. Read the tree it leaves behind, span by span. The first time a wrong answer turns out to have a clear address, you will stop shipping agents you cannot watch.
OpenTelemetry generative AI semantic conventions, agent and tool spans · Datadog LLM observability documentation · reviewed August 2026
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.