Testing in Practice: Stack, Prompts and Use Cases
Reproducible test suites that measure LLM output quality across model, prompt and code changes. Evals are the unit tests of AI systems. This guide pulls together everything on Onexial tagged testing — 5 connected nodes across definitions, workflows, tool stacks, comparisons, prompts and applied use cases — and orders it the way you would actually learn it: vocabulary first, then process, then tooling, then execution. Every item below links to a full node with its own examples and connections, so you can go as deep as you need without losing the map.
Core concepts behind Testing
Before wiring anything together, the vocabulary has to be precise. These 2 definitions cover the terms that show up in almost every Testing discussion — each one links to a full entry with an example and its own connections inside the graph.
AI Evals
Reproducible test suites that measure LLM output quality across model, prompt and code changes.
AI Testing
AI testing covers two things: using AI to generate and maintain tests, and testing AI systems whose output is non-deterministic.
Workflows: how Testing runs end to end
Concepts only matter once they become a repeatable process. Below are 1 documented workflows that apply Testing to a concrete problem, with the steps, the tools involved and the variations worth testing.
The Testing tool stack
A stack is a set of tools chosen for one job, not a list of favourites. These 1 stacks show which combinations hold up in production for Testing, and what each layer is actually responsible for.
Prompts you can reuse
Prompts are reusable components. Each of these 1 prompts is written to be dropped into a Testing workflow with minimal editing, including the context it expects and an example output.
Frequently asked questions
- What is AI Evals?
- Evals are the unit tests of AI systems. You define a labeled dataset and scoring functions (exact match, rubric grading, LLM-as-judge) and run them on every change — so you catch regressions when swapping models, tweaking prompts or upgrading a tool.
- What is an example of AI Evals?
- Before promoting a new system prompt, a team runs 200 saved customer questions through both the old and new prompt and compares helpfulness, accuracy and refusal rates side by side.
- Why does AI Evals matter for AI and automation?
- Reproducible test suites that measure LLM output quality across model, prompt and code changes. It connects to the workflows, prompts and tool stacks linked on this page, so you can move from definition to execution without leaving Onexial.
- What is AI Testing?
- Testing with AI means generating unit, integration and edge-case tests from code and specs — cheap coverage, but it must be reviewed or it just encodes current behaviour as correct. Testing AI means grading probabilistic output: assertion-based checks for structure, rubric or model-graded checks for quality, and regression suites of real failures. Snapshot tests break immediately on model output; behavioural assertions survive.
- What is an example of AI Testing?
- An extraction feature is tested with 40 stored documents asserting schema validity and field-level accuracy above 95%, not exact string equality.
- Why does AI Testing matter for AI and automation?
- AI testing covers two things: using AI to generate and maintain tests, and testing AI systems whose output is non-deterministic. It connects to the workflows, prompts and tool stacks linked on this page, so you can move from definition to execution without leaving Onexial.
- What does the Build a Test Suite for a Non-Deterministic AI Feature workflow do?
- Test structure deterministically, quality with rubrics, and regressions with stored real failures.
- What problem does Build a Test Suite for a Non-Deterministic AI Feature solve?
- Traditional assertions fail on model output that is correct but worded differently.