Testing Lab
The Testing Lab is where your test suite lives. You create tests here, group them into datasets, run them, and come back to see how each one has done over time. Open it from AI Trust Center → Testing lab. New to testing? Get started explains what to test before you create anything.
What you can do here
| To | Do this |
|---|---|
| Find a test | Type in Search test cases, or add a filter: Status, Type, Agent, Dataset or Category. |
| Create tests | Click New test case. See Create test cases. |
| Run tests | Select the tests and click Run. See Run tests. |
| Look back at past runs | Click Run history, then open a run to read its report. |
| Stand in for a system the agent calls | Click Mocks to save responses for your APIs and flows, which a run can use instead of the real thing. |
Read the list
Each test shows its name, type, agent, how many messages it has, its pass rate over recent runs, the datasets it's in, and the result of its latest run:
| Status | What it means | What to do |
|---|---|---|
| Passing | The latest run passed. | Nothing - unless you changed the agent since. Then run it again. |
| Failing | The latest run failed. | Open the run to see why, and decide whether the agent or the test is wrong. See Read a run. |
| Systematic error | The run couldn't finish - for example, a system the agent calls was down. | It isn't a verdict on the agent. Mock that system and run again. |
| Never run | There's no result yet. | Run it. Tests drafted by the Nexus AI stay here until their first run. |
Click a test to see it in detail on the right. At the top are its latest result and pass rate, with Run test, Edit and Delete. The Overview tab shows its expected outcome (you can edit it in place), its datasets, its agent and category, and its recent runs. The Conversation tab shows the conversation and, once the test has a baseline, what the agent did on each turn. To change its checks, use Edit. See Test Case for more.
Act on several tests at once
Tick the tests you want, or use the box at the top of the list to select every loaded test - and then every test that matches your filters. A bar appears with:
- Run - run the selected tests.
- Add to dataset - put them in a dataset.
- Categorize - give them a category, so you can filter by it later.
- Delete - remove them for good.
Datasets
A dataset is a named group of tests you run together. One test can be in several datasets, so you don't have to copy a test to use it in two places.
A layout that works for most bots:
- One dataset per agent, so you can run everything an agent should do after you change it.
- One for routing, on a bot with several agents.
- A short set of your most important tests, to run after every change.
Every test you create in the Testing Lab goes into a dataset - you pick one, or create one, as you go. To change a test's datasets later, open it and use Manage, or select several tests and use Add to dataset. To find tests that aren't in any dataset, filter by Dataset → Unlinked.
To run a dataset, filter by it, select all the tests, and click Run.
What a test holds
| Part | What it's for |
|---|---|
| Name | What the test protects. Name it after the rule it checks - refuses-unoffered-time, not test 1. |
| Conversation | How the customer talks to the agent - one turn or many. |
| Expected outcome | In plain words, what must happen and what must not. Each run is judged against it. |
| Assertions | Optional pass/fail checks on what the agent did: a tool it called, a value it saved, how many turns it took. See Choose the checks to keep. |
| User profile | Optional. The customer the test runs as - for example, a signed-in Gold member. |
| Agent | The agent the test is for. It lets you filter by agent, and it's what lets Nexus preview a Scenario test before saving it. |
| Datasets | The groups it belongs to. |
| Baseline | The first run, kept as a reference for what the agent did. See Baseline capture. |
Keep the suite healthy
- Start with 10 to 30 tests. Cover the agent's main job, each of its rules both ways, each tool it calls, and two or three attempts to push it off course. What to test has the full checklist.
- Turn real failures into tests. A conversation that went wrong is a test waiting to happen. Save it as a test once you've fixed the agent, and it stays fixed.
- Fix inconsistent tests first. A test that fails for no good reason on some runs hides the failures that matter. Run it several times in one run (Iterations) to see whether it flips, then fix it or take it out of the dataset before you rely on the dataset.
- Tighten your checks over time. Start with the expected outcome in plain words. Once that holds, add checks for what you can see the agent do: that it called a tool at the right turn, saved a value, or finished within a set number of turns.
- Clear out tests that check nothing. Duplicates, tests for agents you've switched off, and tests no answer could fail make the suite slower without making it safer.
Next: Create test cases.