Engineering Practice
An adversarial review loop that pipes domain knowledge into a second AI model, iterates until convergence, and stops before it gets nitpicky.
LLM-assisted coding has a review gap. The model that writes the code can't reliably review its own work — it self-ratifies, softening findings across rounds. Cross-model review (Claude writes, GPT reviews) solves this, but doing it manually means copy-pasting output between two chat sessions after every step. For high-stakes features — where a wrong formula silently produces wrong numbers for months — this manual loop is both essential and unsustainable.
The deeper issue is context. A generic code review catches data.table copy semantics and off-by-one errors. It does not catch that a US GDP deflator was applied to a Nigerian-naira delivery cost (a ~12% silent error), or that three budget rows sharing one institutional freight assumption were counted as three independent evidence points instead of one. Those are domain errors, and they require domain knowledge the reviewer doesn't have.
The gap isn't the mechanism — it's the context. A warm review thread that has done a literature review on your feature's domain catches things a cold code reviewer never will. The deep review system captures that warmth in a file.
Four components, designed to be portable across any project.
The runner pipes the current diff, a domain context file, and the project rubric into codex exec --sandbox read-only. Codex returns structured JSON with typed severity levels. The skill orchestrates the fix-and-recheck loop. Domain context files are version-controlled with the project.
The structured output schema enforces a two-level severity split, with the definition baked into the codex prompt:
The anti-nitpicking rule. "Must name the specific wrong output" is the single most effective filter. It forces the reviewer to construct a concrete failure scenario — these inputs produce this wrong output — rather than raising vague concerns. Research on LLM code review finds ~80% of findings are noise; this constraint filters most of it.
The gate never silently degrades to "no review." Missing codex, missing context files, invalid refs, empty output, and parse failures all produce structured STOPPED JSON and exit code 2. The runner derives its verdict from the actual findings list, never trusting the model's own verdict field (which can say PASS while listing blocking findings).
The fix-review loop is autonomous but bounded. Six guards, checked in order after each round, prevent infinite iteration:
The 3-round cap is based on the quality-stagnation model from Yang et al.: with typical LLM parameters, the first two rounds capture ~87% of achievable improvement. Rounds 3–5 add 6%, 3%, 1.5% respectively. Semantic early-stopping research (arxiv 2606.27009) confirmed that simpler convergence guards outperform quality-judge loops, which consumed 129% more tokens at statistically indistinguishable output quality.
The skill tracks three sets across rounds: attempted_fixes (findings Claude fixed), challenged_ids (findings Claude rejected as false positives), and prev_blocking_ids (last round's findings). State is snapshotted before any mutations to prevent a round from comparing against itself. A false positive that reappears triggers the stuck-finding guard, just like a failed fix.
Regression tests are allowed. When Claude fixes a verified blocking finding with a concrete failure scenario, it adds the smallest test that reproduces that scenario. This is the one exception to "don't change anything except the fix" — a verified bug with a known input-output pair is exactly when a regression test has value.
The context file is the central artifact. It captures the domain knowledge a code-only reviewer would lack, turning a cold codex invocation into a warm one. Each high-stakes feature gets its own context file, version-controlled with the project.
The recommended workflow for seeding a context file: open a Codex thread, have it do a literature review on the feature's domain (15–30 minutes), then ask it to export its accumulated knowledge into the template. The context file is a snapshot of that thread's expertise — not a document you author from scratch.
The system was built for and first deployed on the malaria Standard-of-Care cost evidence pipeline — a feature that replaces ad-hoc cost assumptions with a structured, auditable, multi-layer evidence system covering procurement transactions (Global Fund PQR), reference prices, and delivery cost studies (WHO IDCC, VectorLink IRS, Gilmartin SMC, GiveWell budgets).
This feature has exactly the properties that make domain-aware review essential: cost evidence spans multiple currencies and price years, unit definitions vary by product and source, and the distinction between "financial" and "economic" perspective silently changes values by up to 3×.
The first deep review ran codex (gpt-5.6-sol) against a 482-line diff with the full domain context file (~180 lines of domain knowledge). Codex found 9 findings in the initial pass:
One finding was a bug in the review runner itself (the diff wasn't being piped to codex — caught on the first invocation). Five were domain-validity findings that required the context file to detect. Three were data-contract violations.
These are the findings a generic code review would miss entirely:
cost_delivery by units²/quant — quadratic where it should be linear. For trt_3 with quant=1, delivery cost scales as units². The context file documents the exact arithmetic at rollup_fx.R:539-545.central_eligible=FALSE, but the cost function aggregates all rows. One eligible oral ACT at $1 and one excluded injectable at $10 produces a mean of $5.50 instead of $1.selected_scope='full' with no conversion.usd_commodity and deflated with the US GDP deflator — but Gilmartin's drug/supply and service-delivery categories have different economic content. The context file's protocol rules caught this.The subsequent review rounds on the normalization implementation showed the convergence guards working as designed. Round 2 found 7 remaining findings after fixes. The stuck-finding guard correctly stopped the loop when Claude and codex disagreed about whether the Gilmartin US-deflator-on-delivery-costs issue was a blocking bug (codex: fail closed until local CPI is available) or a documented approximation (Claude: the rows are sensitivity-only, the protocol label is explicit, and blocking them discards the only published multi-country SMC delivery cost evidence).
This is the kind of dispute that should go to a human. The system escalated it correctly.
usd_reported_delivery_proxy protocol was designed for exactly this case. These rows are all role = "sensitivity", never central.
The human (correctly) sided with Claude: the protocol label is explicit about the limitation, the rows are sensitivity-only, and discarding them would lose the only published multi-country SMC delivery cost evidence that exists.
Seven steps for each new high-stakes feature. Steps 2 and 3 are the ones that matter most.
| Step | Action | Time |
|---|---|---|
| 1 | Decide. Does this feature need deep review? Touches money, units, or formulas? Combines data from multiple sources? Domain-specific correctness rules? | 1 min |
| 2 | Literature review. Open a Codex thread. Have it research the domain — key papers, standards, known pitfalls. Let the thread accumulate expertise. | 15–30 min |
| 3 | Export. Ask the thread to fill in the context template. Save as .claude/review-contexts/<feature>.md. | 5 min |
| 4 | Register. Add one line to CLAUDE.md listing the active review context. | 30 sec |
| 5 | Implement. Work normally. The review runs after each step, autonomously. Most steps pass. | — |
| 6 | Final review. /deep-review <feature>.md on the full diff before committing. | 3–5 min |
| 7 | Update. Add new pitfalls the review caught to the context file. It gets smarter over time. | 2 min |
The sleep-deprived shortcut. If you only have energy for one thing, it's the context file. Without it, /deep-review is just a generic code review — no better than a cold invocation. The context file is the whole point.
The system requires Claude Code (the CLI) and the Codex CLI. Both must be installed and authenticated.
Install the canonical runner and portable files:
# The runner
mkdir -p ~/bin
curl -o ~/bin/deep_review.sh https://raw.githubusercontent.com/... # or copy from your repo
chmod +x ~/bin/deep_review.sh
# The skill and supporting files
mkdir -p ~/.claude/commands ~/.claude/review-contexts
# Copy deep-review.md to ~/.claude/commands/
# Copy review-finding-schema.json and review-rubric.md to ~/.claude/
# Copy TEMPLATE.md to ~/.claude/review-contexts/# Create a thin forwarder (optional — the skill finds ~/bin/ directly)
cat > scripts/deep_review.sh <<'EOF'
#!/usr/bin/env bash
exec "$HOME/bin/deep_review.sh" "$@"
EOF
chmod +x scripts/deep_review.sh
# Create a context file for your first feature
cp ~/.claude/review-contexts/TEMPLATE.md .claude/review-contexts/my-feature.md
# Add the gate to CLAUDE.md
echo '- `my-feature.md` — description' >> CLAUDE.md # under "Active review contexts"/deep-review my-feature.md # full domain-aware review
/deep-review my-feature.md join cardinality # with a focus area
/deep-review none # generic rubric only (no domain context)--base <ref>--paths <glob>--fail-open--allow-skipped