Reports
A report is what you read after a run: which tests passed and failed, how each one scored, and what the agent actually did. It's where you find out why something broke, not just that it did.
Run history
Open Run history from the Testing Lab to see every run. Above the list are the Pass rate, the number of Failed runs and the Avg duration. Each run shows:
| Column | What it tells you |
|---|---|
| Run | Which run this was. |
| Environment | Whether it ran against Sandbox or Staging. |
| Triggered by / Triggered at | Who started it, and when. |
| Duration | How long it took. |
| Pass | How many of its tests passed and failed. |
Search for a run, filter the list, or group it by day or by dataset. To see every run of one test, open the test and click View all under Recent runs. See Test Case.
Click a run to open its report.
Reading a report
At the top you'll see when the run started, how long it took and who started it, then how many tests it had, how many passed and failed, and the pass rate.
- Retry failed runs the failed tests again as a new run. When nothing failed, the same button reads Re-run.
- Export report downloads the results as a CSV file.
Below is one row per test, with the agent it targets and how long it took. Search the list, or filter it by type, result or agent. Expand a row to see, in this order:
| Section | What it tells you |
|---|---|
| Assertions | The test's checks: whether the right thing happened. See Checks and scores. |
| Evaluators | Each evaluator's score against its threshold, and whether it passed. See Evaluators & Rules. |
| Agent rules | For a test that targets an agent: which of the agent's own rules it followed or broke, with the line that proves it. See Agent rules. |
| Trace events | What the agent did on each turn: tools it called and what they returned, values it saved, hand-offs between agents, knowledge searches and guardrail checks. See Reading an execution trace. |
| AI Summary | Why the reply scored as it did, with Strengths and Areas for improvement. |
| Expected Behaviors | Each behaviour the test expects, whether it held, and what was Observed. |
Click Analyse conversation to put the test - its scenario and expected outcome - side by side with the conversation the run produced. Go to conversation logs opens that conversation in Conversations.
A report keeps the scores and thresholds the run was judged with, so it reads the same even if you change your evaluators later.
Checks and scores
Expanding a test shows two kinds of result, and they answer different questions:
| Checks (under Assertions) | Evaluator scores | |
|---|---|---|
| Question | Did the right thing happen? | Was the reply any good? |
| How it's decided | Either it happened in the run or it didn't | An AI judge reads the conversation and scores it |
| Examples | The refund lookup was called; the order ID was saved | Accuracy 88, Empathy 95 |
A test can score well on every evaluator and still fail a check: the agent wrote a fluent reply but never actually looked anything up. That's exactly the kind of quiet failure checks are there to catch, which is why they're shown above the scores.
Each check shows what it checks - Tool called, Memory set or Max turns - its priority, and the turn it applies to. When it fails, it says what was expected and what was Observed instead. Failures come first, and a failed Informational check is shown in amber rather than red, because it doesn't fail the test.
| Priority | When it fails |
|---|---|
| Critical | The test fails, and the evaluators don't score it |
| Normal | The test fails; the evaluators still score it, for context |
| Informational | It's reported, but the test doesn't fail |
Why a failed test can have no scores
When a critical check fails, the test has already failed, so it isn't scored. You'll see Evaluator scores were skipped in place of the scores. That's expected: look at the check that failed instead.
If you see Evaluator scores are unavailable, the run finished without scoring the test. Run the test again to get them.
Older test cases saved without checks don't show an Assertions section.
When a test errors
A Systematic error means the test couldn't run to the end - for example, because a system the agent calls was down. The report shows the error in place of the results. It isn't a verdict on the agent: fix the cause, or use a mock for that system, and run the test again.
Tips
- Start from the report when you're chasing a regression. It shows exactly what happened in one run - the quickest way to find what flipped from pass to fail.
- Read the checks before the scores. A failed check tells you what went wrong; a low score only tells you the reply wasn't good enough.
- Re-run one test, not the whole set. To check a single fix, open the test in the Testing Lab and click Run test.
- Read some passes too. A pass means the test was satisfied, not that the agent did everything right. See Read a run.
Next: Reading an execution trace.