Problem and boundaries
The team wants AI to help fix small defects, but a generated patch must not enter the main branch directly. This case validates file scope, test gates, and human approval in a temporary repository without touching a real repository.
Design
Store the task input, baseline commit, patch hash, and test result as one release-candidate record. The gate reads those results; it does not ask the model to interpret whether tests passed.
Minimal implementation
from dataclasses import dataclass
@dataclass(frozen=True)
class Candidate:
patch_hash: str
files: tuple[str, ...]
tests_passed: bool
approved: bool
def gate(candidate: Candidate) -> bool:
return candidate.files and all(path.startswith('src/') for path in candidate.files) and candidate.tests_passed and candidate.approved
assert gate(Candidate('sha256:demo', ('src/app.py',), True, True))
Reproduction steps
- Copy an isolated repository and pin its commit; keep real secrets out of the directory.
- Generate or write a patch that changes only src/, calculate its hash, and save it.
- Run formatting checks and tests. Deliberately add one failing result and confirm that the gate rejects it.
- Add the approval marker and run again, confirming that the same patch is replayable without repeating side effects.
Verification checklist
Verify allowed directories, failed tests, missing approval, changed patch hashes, and duplicate replay.
Common questions
Does this prove that the model is always correct? No. It verifies delivery boundaries and replayability only.
Can this be copied directly into production CI? Start with read-only checks, then expand according to repository permissions and audit requirements.
Further reading
Official references
OpenAI Codex documentation; GitHub Actions security hardening.
Last checked: 2026-11-17. Follow the linked sources for current interfaces, model behavior, and pricing; this case does not present one local run as a general performance result.
