Testing Lab
The Testing Lab is where you build the regression suite for your Nexus bot. Open it from AI Trust Centre → Testing Lab.
The empty state says "No test cases yet - Create your first test case to start evaluating your AI agent." From the toolbar:
- Search test cases - full-text search across saved cases.
- Run history - every dataset run with pass / fail counts; click into a run for per-case results.
- New test case - opens the create drawer.
Once you've added cases, the list shows each one with its type icon, recent pass-rate, and dataset chips, and a per-case detail panel on the right.
In this section
- Create test cases - the two ways to author a case (Import content, Scenario), the Trial-run preview, and what gets captured: assertions, baseline, and run status.
- Run tests - run a dataset with personas and iterations, and manage tests without leaving the Conversation Builder.
How a test case is structured
Each saved case carries:
| Field | Notes |
|---|---|
| Name | Human-readable. Use the question the case answers: "billing question routes correctly", "fallback fires on gibberish". Bad: "test 1". |
| User inputs | The messages a real user would send. One or many turns. |
| Initial state | Optional. Memory / user-profile values the test needs (e.g. customer_id, account_tier = "gold"). |
| Expected outcome | What the case is asserting. Plain English plus the assertions you pick (see Assertion picker). |
| Source reference | Which agent / dataset / surface this case belongs to. Used by filters and deep-links. |
| Baseline trace | Captured automatically on creation (for cases created via Trial run) - see Baseline capture. |
| Run status | ready / stale - see Run status semantics. |
Cases live inside datasets. A dataset is a named bundle you run together (e.g. "Pre-release smoke suite", "Adversarial inputs"). One case can belong to multiple datasets.
Best practices
- Aim for 10-30 cases first time. Cover golden paths, one case per Routing Logic rule, one case per tool, and 2-3 adversarial prompts.
- Promote real failures into cases. A bug you found in the Playground is a test case waiting to happen. Use Import content → Chat Transcript to bring in the conversation that exhibited it (the Trial run preview captures its baseline), then assert the fix.
- Don't run noisy datasets nightly. If a single flaky case is generating a pile of low-value failures every run, fix the case before scheduling. (Scheduled runs are a planned follow-up.)
- Tighten assertions over time. Start with "right agent fired"; once that's stable, add "response contained the tracking number"; once that's stable, add latency / cost assertions.
💡 Try the Nexus AI: "Convert this Playground conversation I just had into a Testing Lab Scenario test case."
💡 Try the Nexus AI: "Which tests in my regression suite are most likely to be redundant or flaky? Suggest pruning."
Read next: Evaluators & Rules - tune what counts as a pass on every run.