← Back to learning path
Case · Published

Case study: rehearsing an AI code-review gate in an isolated repository

Use a fixed patch, test results, and an approval marker to rehearse a replayable AI-programming delivery gate.

Source: knowledge/p27/ai-programming-engineering/cases/ai-code-review-gate-rehearsal.mdAI programming engineering
Chapter progress 0%Learning path progress 0% · 0/2 chapters completedContinue learning path →

Problem and boundaries

The team wants AI to help fix small defects, but a generated patch must not enter the main branch directly. This case validates file scope, test gates, and human approval in a temporary repository without touching a real repository.

Design

Store the task input, baseline commit, patch hash, and test result as one release-candidate record. The gate reads those results; it does not ask the model to interpret whether tests passed.

Minimal implementation

from dataclasses import dataclass

@dataclass(frozen=True)
class Candidate:
    patch_hash: str
    files: tuple[str, ...]
    tests_passed: bool
    approved: bool

def gate(candidate: Candidate) -> bool:
    return candidate.files and all(path.startswith('src/') for path in candidate.files) and candidate.tests_passed and candidate.approved

assert gate(Candidate('sha256:demo', ('src/app.py',), True, True))

Reproduction steps

  1. Copy an isolated repository and pin its commit; keep real secrets out of the directory.
  2. Generate or write a patch that changes only src/, calculate its hash, and save it.
  3. Run formatting checks and tests. Deliberately add one failing result and confirm that the gate rejects it.
  4. Add the approval marker and run again, confirming that the same patch is replayable without repeating side effects.

Verification checklist

Verify allowed directories, failed tests, missing approval, changed patch hashes, and duplicate replay.

Common questions

Does this prove that the model is always correct? No. It verifies delivery boundaries and replayability only.

Can this be copied directly into production CI? Start with read-only checks, then expand according to repository permissions and audit requirements.

Further reading

Official references

OpenAI Codex documentation; GitHub Actions security hardening.

Last checked: 2026-11-17. Follow the linked sources for current interfaces, model behavior, and pricing; this case does not present one local run as a general performance result.

Further reading

KnowledgeDesigning Reliable AI Automation WorkflowsKnowledgeProduction Prompt Engineering: From Instructions to Measurable Contracts
Product Hub Demo · V0.3