One request can include several model and tool calls. Keep a run_id, span_id, duration, result class, estimated tokens, and remaining budget. Logs should store a summary or hash instead of raw customer content.
Core concepts
Check the budget before a call and record actual usage after it. Apply limits for steps, tokens, time, and cost per tenant or task type. Alert on timeouts, exhausted budgets, and repeated tool calls.
Runnable example
from time import monotonic
def budgeted_call(call, budget: int):
started = monotonic()
result = call()
elapsed_ms = round((monotonic() - started) * 1000)
usage = int(result.get('usage', {}).get('total_tokens', 0))
if usage > budget:
raise RuntimeError('agent budget exceeded')
return {'elapsed_ms': elapsed_ms, 'tokens': usage, 'result': result}
Steps and use cases
- Define run, model, and tool event fields plus a sampling policy.
- Set separate step, token, time, and cost budgets.
- Check remaining budget before invocation and record result categories afterward.
- Review slow traces and loops without retaining unnecessary personal content.
This is useful for multi-tenant agents, batch jobs, and auditable internal automation. Tokens are usage units, not a universal price; keep provider pricing in configuration.
Official references
OpenAI production best practices and OpenTelemetry.
Last checked: 2026-11-16. Do not treat one local run as a general performance or cost claim.
