← Back to learning path
Chapter · Published

Agent observability and cost controls

Use traces, budgets, and redacted logs to find slow requests, loops, and unexpected model cost.

Source: knowledge/p24-b/collection_8023a8d754046b6c2ae7220f/tutorials/agent-observability-and-cost-controls.mdAI agent engineering
Chapter progress 0%Learning path progress 0% · 0/10 chapters completedContinue learning path →

One request can include several model and tool calls. Keep a run_id, span_id, duration, result class, estimated tokens, and remaining budget. Logs should store a summary or hash instead of raw customer content.

Core concepts

Check the budget before a call and record actual usage after it. Apply limits for steps, tokens, time, and cost per tenant or task type. Alert on timeouts, exhausted budgets, and repeated tool calls.

Runnable example

from time import monotonic

def budgeted_call(call, budget: int):
    started = monotonic()
    result = call()
    elapsed_ms = round((monotonic() - started) * 1000)
    usage = int(result.get('usage', {}).get('total_tokens', 0))
    if usage > budget:
        raise RuntimeError('agent budget exceeded')
    return {'elapsed_ms': elapsed_ms, 'tokens': usage, 'result': result}

Steps and use cases

  1. Define run, model, and tool event fields plus a sampling policy.
  2. Set separate step, token, time, and cost budgets.
  3. Check remaining budget before invocation and record result categories afterward.
  4. Review slow traces and loops without retaining unnecessary personal content.

This is useful for multi-tenant agents, batch jobs, and auditable internal automation. Tokens are usage units, not a universal price; keep provider pricing in configuration.

Official references

OpenAI production best practices and OpenTelemetry.

Last checked: 2026-11-16. Do not treat one local run as a general performance or cost claim.

Related practical cases

KnowledgeCase study: an auditable support-ticket triage agentKnowledgeCase study: an evidence package and human handoff after an agent failureKnowledgeCase study: a deployment-advice agent with human approval

Further reading

KnowledgeDesigning Reliable AI Automation WorkflowsKnowledgeAI Agents: From Core Concepts to ProductionKnowledgeProduction Prompt Engineering: From Instructions to Measurable Contracts
Product Hub Demo · V0.3