Reading an execution trace
An execution trace is the per-turn record of what your agent actually did to produce each reply - every tool it called, every variable it wrote, every routing decision, every knowledge lookup, every guardrail check, in the order they happened. The chat bubble shows you what the bot said; the trace shows you why it said it.
This is the surface you open when a reply is wrong and you need to find out where it went wrong: did the right tool fire? With the right arguments? Did it write the variable the next turn depends on? Did routing hand off to the agent you expected? The trace answers those questions deterministically - it's a literal log of what ran, not a guess.
Note: The trace is structural and deterministic - a tool either was or wasn't called, a variable either was or wasn't written. That's different from the evaluator scores, which are fuzzy LLM judgements about response quality. Both appear in the Trust Centre, but they answer different questions: the trace tells you what happened; the evaluators tell you how good it was.
Where to open a trace
You can read the same kind of trace in three places, depending on what you're doing:
| Where | How you open it | Use it when |
|---|---|---|
| Playground | Click View trace under any bot message in the Playground. | You're iterating live and want to inspect the turn you just saw. |
| Test Case → Conversation tab | The trace is paired inline with each agent message on the Test Case detail panel. | You're reviewing a saved case and want to see the baseline behaviour that was captured. |
| Reports run | Open a simulation row in an Individual Report; each run carries the trace it produced. | You're investigating why a specific run passed or failed. |
It's the same trace shape in all three - once you can read one, you can read all of them.
What a trace shows
For each turn, the trace lists the events that ran, in order. Each event is one thing the agent did:
| Event | What it tells you | Example question it answers |
|---|---|---|
| Tool call | Which tool ran and the exact arguments it was given. | "Did the agent call getOrderStatus, and did it pass the order ID the user typed - or did it hallucinate one?" |
| Tool result | Whether the call succeeded or errored, how long it took, and the value that came back (or the error code). | "Did the order-lookup API time out, or did it return an empty result the agent then misread?" |
| Memory update | Which variables were written or changed this turn - dotted paths like order_id or customer.email - with their before / after values. | "Did the agent actually save customer.email so the next agent can use it, or did that write never happen?" |
| Agent handoff / transition | Which agent the conversation moved from and to, and the reason for the move. | "Why did this route to the Billing agent instead of staying with Support?" |
| Knowledge-base retrieval | The query that was issued, and which documents / chunks came back, each with its match score. | "Did the agent retrieve the refund-policy doc - and did it score high enough to be trusted, or did it grab something off-topic?" |
| Guardrail check | Whether a safety guardrail allowed, blocked, redacted, or flagged the turn. | "Was this reply blocked by a guardrail, or did the agent genuinely decide not to answer?" |
Read the events top to bottom and you can reconstruct the agent's reasoning for that single reply.
Per-turn summary
Each turn also carries a small summary strip alongside its events:
- Latency - how long the turn took, in milliseconds.
- Tokens - tokens in and tokens out for the turn.
- LLM calls / tool calls - how many model calls and how many tool calls the turn made.
- Status - whether the turn completed cleanly, errored, or was cut short.
This is where you catch a turn that's correct but slow - one that quietly made three LLM calls and two tool calls when it should have made one of each.
Conversation summary
When the run ends, a conversation summary rolls the per-turn numbers up: total latency, total tokens, total LLM and tool calls across all turns, and the final status of the whole conversation. Use the per-turn summary to find which turn was expensive; use the conversation summary to judge whether the run as a whole was healthy.
How events anchor to turns
Every event belongs to a specific turn - the reply it helped produce. In the Playground and in the Test Case Conversation tab, that anchoring is visual: the trace for a turn sits directly under (or beside) the agent message it produced, so a multi-turn conversation reads as message → its trace → next message → its trace. This is also why turn-anchored assertions (e.g. "getOrderStatus was called on turn 2") can point at a single reply - they check the events recorded against that turn's slice of the trace.
Redaction and truncation
Two things in the trace are deliberately not shown verbatim, so don't be surprised when you see them:
- Secret-looking values are redacted. Anything that looks like a token, password, or API key shows as
<redacted>rather than its real value. The event still tells you the variable was written or the argument was passed - you just don't see the secret itself. - Very large values are truncated. A tool that returns a huge payload (a long document, a big JSON blob) is clipped so the trace stays readable. You'll see enough to know what came back, not the entire thing byte-for-byte.
If a tool result looks empty or wrong, check whether it was truncated before you assume the tool failed. A clipped large value is not the same as a missing one.
The trace vs the evaluator score
When a Reports run scores a turn, two independent things are happening:
- Assertions check the trace - deterministically. "Was this tool called? Was that variable written? Did routing pick this agent?" These are pass/fail and never disagree with themselves: the event either is in the trace or it isn't.
- Evaluators score the response - fuzzily. An LLM judges qualities like accuracy, helpfulness, or tone and returns a number against a threshold.
A turn can fail an assertion while passing every evaluator (the reply read fine but the agent skipped the tool it was supposed to call), or pass every assertion while failing an evaluator (the right tool fired but the wording was off-policy). When you're debugging, look at the trace first to settle the deterministic question, then read the evaluator scores for the judgement call.
Best practices
- Open the trace before you theorise. The events answer most "why did it do that?" questions outright - wrong tool, missing memory write, an off-topic KB chunk that scored too high.
- Read top to bottom, then check the summary. The event order is the reasoning; the per-turn summary catches the correct-but-slow or correct-but-expensive turns the bubbles hide.
- Separate the two failure modes. A failing assertion is a structural bug in the trace; a failing evaluator is a quality judgement on the words. Fix them differently.
- Don't trust an "empty" value at a glance. Redaction and truncation are intentional - confirm a value is genuinely missing before you chase it as a bug.
Continue to: Debug a failing test case - turn what you read in the trace into a fix.