← Back to learning path
Chapter · Published

Agent evaluation and regression testing: verify actions, not only answers

Build task sets, tool traces, and safety assertions so Agent behavior can be checked after every change.

Source: knowledge/p24-b/collection_8023a8d754046b6c2ae7220f/tutorials/agent-evaluation-and-regression-testing.mdAI agent engineering
Chapter progress 0%Learning path progress 0% · 0/10 chapters completedContinue learning path →

Agent correctness includes the final answer, tool choice, arguments, order, stopping conditions, and safe refusal. Text similarity alone misses a wrong write or an unauthorized call.

Core concepts

Cover normal, boundary, missing-permission, dependency-timeout, and prompt-injection cases. Use programmatic assertions for permission and side effects, and reserve model grading for expression quality.

Runnable example

def assert_trace(trace: list[dict]) -> None:
    names = [event.get('tool') for event in trace if event.get('type') == 'tool']
    assert 'delete_user' not in names
    assert names.count('lookup_ticket') <= 3
    assert trace[-1].get('type') in {'final', 'needs_human'}

assert_trace([{'type': 'tool', 'tool': 'lookup_ticket'}, {'type': 'final'}])

Steps and use cases

  1. Create redacted samples plus attack and empty-input cases.
  2. Define allowed tools, argument bounds, step limits, and human approval points.
  3. Run regressions on a fixed model and tool version, recording reproducible failures.
  4. Compare security assertions, completion rate, and human handoff before release.

This approach suits support agents, coding assistants, and research workflows. Use redacted or synthetic data rather than production personal data.

Common questions

LLM grading cannot replace permission assertions. Failed examples belong in a versioned regression set instead of being hidden in a report.

Official references

OpenAI Evals and OpenAI safety best practices.

Last checked: 2026-11-16. The linked documentation is the source of truth for current interfaces and safety boundaries.

Related practical cases

KnowledgeCase study: a deployment-advice agent with human approvalKnowledgeCase study: an auditable support-ticket triage agentKnowledgeCase study: an evidence package and human handoff after an agent failure

Further reading

KnowledgeAI Agents: From Core Concepts to ProductionKnowledgeStructured Outputs and Tool Contracts for Safe AgentsKnowledgeDesigning Reliable AI Automation Workflows
Product Hub Demo · V0.3