A production Agent release includes the model identifier, prompt version, tool schema, evaluation result, budget configuration, and rollback pointer. During a canary, limit the percentage and risk of traffic and watch refusals, handoffs, timeouts, and side effects.
Core concepts
Treat prompts, tools, permissions, knowledge sources, and model versions as one rollback unit. Keep task state and event records so a rollback does not delete work.
Runnable example
from dataclasses import dataclass
@dataclass(frozen=True)
class AgentRelease:
version: str
model: str
prompt_version: str
tool_schema_version: str
def can_promote(checks: dict[str, bool]) -> bool:
return all(checks.get(name, False) for name in {'eval', 'security', 'rollback', 'observability'})
Release steps
- Freeze the release manifest and run the fixed regression set.
- Start with internal or low-risk traffic and require approval for writes.
- Pause promotion when security, timeout, handoff, or side-effect thresholds fail.
- Keep the previous compatible release and database strategy available for rollback.
This pattern suits support, operations, and internal automation. Payment, deletion, and privilege changes should require a human by default.
Official references
OpenAI production best practices and OpenAI safety best practices.
Last checked: 2026-11-16. Canary percentages and thresholds are risk decisions, not universal defaults.
