← Back to learning path
Chapter · Published

Agent production release: canary, rollback, and human handoff

Create release checks, a canary plan, rollback signals, and a human handoff path before an Agent reaches production.

Source: knowledge/p24-b/collection_8023a8d754046b6c2ae7220f/tutorials/agent-production-release-and-human-handoff.mdAI agent engineering
Chapter progress 0%Learning path progress 0% · 0/10 chapters completedContinue learning path →

A production Agent release includes the model identifier, prompt version, tool schema, evaluation result, budget configuration, and rollback pointer. During a canary, limit the percentage and risk of traffic and watch refusals, handoffs, timeouts, and side effects.

Core concepts

Treat prompts, tools, permissions, knowledge sources, and model versions as one rollback unit. Keep task state and event records so a rollback does not delete work.

Runnable example

from dataclasses import dataclass

@dataclass(frozen=True)
class AgentRelease:
    version: str
    model: str
    prompt_version: str
    tool_schema_version: str

def can_promote(checks: dict[str, bool]) -> bool:
    return all(checks.get(name, False) for name in {'eval', 'security', 'rollback', 'observability'})

Release steps

  1. Freeze the release manifest and run the fixed regression set.
  2. Start with internal or low-risk traffic and require approval for writes.
  3. Pause promotion when security, timeout, handoff, or side-effect thresholds fail.
  4. Keep the previous compatible release and database strategy available for rollback.

This pattern suits support, operations, and internal automation. Payment, deletion, and privilege changes should require a human by default.

Official references

OpenAI production best practices and OpenAI safety best practices.

Last checked: 2026-11-16. Canary percentages and thresholds are risk decisions, not universal defaults.

Related practical cases

KnowledgeCase study: an auditable support-ticket triage agentKnowledgeCase study: a deployment-advice agent with human approvalKnowledgeCase study: an evidence package and human handoff after an agent failure

Further reading

KnowledgeAI Agents: From Core Concepts to ProductionKnowledgeDesigning Reliable AI Automation WorkflowsKnowledgeStructured Outputs and Tool Contracts for Safe Agents
Product Hub Demo · V0.3