RUBIKSCORPAI systems
How to evaluate AI: a practical test plan before release
AI evaluation is a repeatable comparison between what a system does and what a defined task requires. It needs representative cases, an explicit scoring method and a decision about which failures are acceptable.
Define success before choosing a metric
“Good answers” is too vague to evaluate. For a policy assistant, success might mean selecting the right policy revision, answering only what it supports and escalating when information is missing. A stock lookup has a different target: the correct item, account and current quantity.
Write down the intended user, input, permitted data and allowed actions. Decide which results are drafts and which represent completed operations. This gives reviewers a common object to score.
- Task: what should the system help someone do?
- Baseline: what happens today without the proposed change?
- Evidence: what makes an answer or action correct?
- Failure: what outcome would be costly or unacceptable?
- Owner: who can approve the test criteria and release decision?
Build a test set that represents the work
Include routine cases, ambiguous inputs, missing information and known failures. Add variations that matter to deployment, such as language, document format, customer permissions or factory lighting. Do not let easy cases conceal the situations that cause real harm.
Keep a held-out test set apart from examples used to adjust prompts or train a model. A test repeatedly used to guide changes becomes less independent. For production data, choose a split that reflects time, customers or equipment rather than mechanically splitting near-duplicates.
There is no universal number of examples that proves readiness. Begin with coverage of the important task types, report the number tested and widen the set when failures reveal gaps.
A reusable rubric for a document assistant
This rubric is an original example for planning a test, not an industry certification or a complete security assessment. Adapt the pass conditions with the people responsible for the actual process.
| Criterion | Pass condition | Example failure |
|---|---|---|
| Source selection | Uses the permitted, applicable revision | Quotes an obsolete policy |
| Support | Important claims follow from the source | Invents an exception |
| Access | Uses only records permitted for the user | Exposes another account’s details |
| Uncertainty | Escalates when evidence is absent | Presents a guess as confirmed |
| Action state | Reports what actually completed | Says an email was sent after a failed call |
Combine automated checks and informed review
Deterministic checks are useful for exact fields, required structure and known permissions. Human review is useful when correctness depends on context or interpretation. A model-based judge can assist at scale, but its judgments also need validation against the agreed rubric.
Keep examples where reviewers disagree. Clarify the rubric rather than averaging away a disputed definition. Repeat some cases when generation varies and record the number of runs.
Report both totals and slices. A strong overall score can conceal failure on one language, one document family or one equipment condition. Small samples should remain visibly small rather than be presented as precise general guarantees.
Make a release decision someone can defend
Agree release gates before comparing candidates. For example, an access violation may block release even if writing quality improves. A latency target should name the workload and measurement method. These are process-specific choices, not universal thresholds.
Keep the test cases, outputs, scores and system configuration together. A model, prompt, tool or source change should be traceable. Run the relevant regression checks again when that configuration changes.
After release, sample real failures through an approved process, add representative cases and revisit the evaluation. Testing supports a bounded decision at a point in time; it cannot establish that every future output will be correct.
Common questions
Is one overall score enough?
No. Inspect critical failures, task categories and operating conditions separately.
Can another AI grade all the answers?
It can assist, but its scoring needs validation against the task rubric and informed review.
Is an evaluation the same as a security audit?
No. A general quality evaluation does not replace a separately scoped security assessment.
Sources and notes
- Google Cloud — Generative AI evaluation overview
Evaluation datasets, rubrics and metric types.
- Google — Dividing datasets
Separating training, validation and testing.
- NIST — AI Risk Management Framework
Voluntary guidance for identifying, measuring and managing AI risks.
- NIST AI 600-1 — Generative AI Profile
Risks and management considerations for generative AI systems.
Read next
Have a specific question?
Tell us the task, the information available and what a useful result would look like.