Agent correctness includes the final answer, tool choice, arguments, order, stopping conditions, and safe refusal. Text similarity alone misses a wrong write or an unauthorized call.
Core concepts
Cover normal, boundary, missing-permission, dependency-timeout, and prompt-injection cases. Use programmatic assertions for permission and side effects, and reserve model grading for expression quality.
Runnable example
def assert_trace(trace: list[dict]) -> None:
names = [event.get('tool') for event in trace if event.get('type') == 'tool']
assert 'delete_user' not in names
assert names.count('lookup_ticket') <= 3
assert trace[-1].get('type') in {'final', 'needs_human'}
assert_trace([{'type': 'tool', 'tool': 'lookup_ticket'}, {'type': 'final'}])
Steps and use cases
- Create redacted samples plus attack and empty-input cases.
- Define allowed tools, argument bounds, step limits, and human approval points.
- Run regressions on a fixed model and tool version, recording reproducible failures.
- Compare security assertions, completion rate, and human handoff before release.
This approach suits support agents, coding assistants, and research workflows. Use redacted or synthetic data rather than production personal data.
Common questions
LLM grading cannot replace permission assertions. Failed examples belong in a versioned regression set instead of being hidden in a report.
Official references
OpenAI Evals and OpenAI safety best practices.
Last checked: 2026-11-16. The linked documentation is the source of truth for current interfaces and safety boundaries.
