Reviewing AI-Generated Code: Why the Checklist Is Not Enough

The Short Answer

Conventional code review assumes a human author who had a reason for every decision, so the reviewer looks for mistakes. AI-generated code rarely contains mistakes in that sense. It contains code that is locally correct and globally incoherent: the same logic solved four slightly different ways in four files, errors caught and discarded rather than handled, abstractions that stop holding two sprints after they were introduced.

None of that is visible in a diff. Each individual change looks fine, because each individual change is fine. The damage is cumulative and cross-file, which means the instrument has to be too. Keep the checklist for what it catches, and add four repository-level measurements for what it cannot.

Why Per-Diff Review Cannot See It

This is not a claim about reviewer diligence. It is structural. A reviewer looking at a pull request has one file tree open and a few hundred changed lines in front of them. To notice that this retry helper is the fourth of its kind, they would need to already know about the other three, which live in modules they have no reason to open.

The consequence is measurable at repository scale. GitClear’s 2026 maintainability research found duplicated code blocks running at 73.0 per million changed lines so far in 2026 - an 81% increase over 2023 and the highest level on record - along with a 41% rise in within-commit copy/paste and a 47% rise in error-masking constructs. Those three numbers describe exactly the failure I have described above, and not one of them can be derived from a single pull request.

The perception problem compounds it. In METR’s randomised controlled trial, 16 experienced developers completed 246 tasks in repositories they averaged five years of familiarity with. Allowing AI tools made them 19% slower. Afterwards, those same developers estimated AI had made them 20% faster. They had forecast 24% faster going in.

A 39-point gap between measured and perceived performance, among experts, on their own code. Whatever your team believes about its AI-assisted velocity, that belief is not evidence.

What The Checklist Genuinely Catches

I want to be fair to the standard advice, because it is not wrong - it is just bounded. Per-diff review is the right instrument for a real class of defect, and you should keep doing it:

• Invented dependencies. Packages, methods and version behaviours the model produced from pattern rather than from your lockfile. Cheap to check, embarrassing to miss.
• Hardcoded secrets. Still one of the most common findings in AI-generated code, and trivially detectable.
• Happy-path bias. The success case is usually well written. The retry, the timeout and the malformed-input branch are where quality drops, because that is where the training data thins out.
• Tests that assert nothing. Test names describing the method called rather than the behaviour verified. Coverage percentage stays green; regressions sail through.

All four belong in review. None of them tells you whether the codebase is getting harder to change.

The Four Measurements That Do

These are repository-wide counts, tracked over time. Individually each is a weak signal. Moving together while shipping velocity appears steady, they are the clearest early indicator I know of that a codebase is being quietly spent.

1. Duplicate blocks per million changed lines

The single most diagnostic number, because duplication is what a model does instead of understanding your abstraction. It cannot see the helper you already wrote, so it writes another. Track the count, not a percentage - percentages hide growth in a growing repository.

2. Error-masking constructs

Empty catch blocks, bare excepts, errors logged and swallowed, promises with no rejection path. Each one converts a loud failure into a silent one. A rising count is a system becoming undebuggable in production, which you will discover at the worst possible moment.

3. Two-week churn

The proportion of lines rewritten within a fortnight of being committed. High churn means code is being shipped before it is understood, then repaired. It is the metric that most directly contradicts a velocity narrative, which is why it is worth having before the conversation rather than during it.

4. Features cross-referenced against tests that assert behaviour

Not coverage percentage. Coverage tells you a line executed during a test run; it does not tell you that anything would fail if the line were wrong. Map features to the tests that would actually break, and the gap between reported coverage and real coverage is usually the most alarming number in the whole exercise.

Running The Analysis Without Hallucinating It

The obvious way to gather these is to point a language model at the repository and ask. That fails, and it fails in the most expensive way available: it returns a confident, readable report describing vulnerabilities that are not in your code while missing ones that are. The model is recalling what code like yours usually looks like.

The fix is ordering, not prompting. This is the architecture I built into SystemAudit, and it is the reason its reports cite file and line numbers rather than adjectives:

1. Deterministic analysis first. Static analysis produces the hard evidence - vulnerability patterns with exact locations, the real import graph including proven circular dependencies, dependency health, structure metrics. These are facts, and they are reproducible by anyone who runs the same pass.
2. Interpretation second, and constrained. The model receives those findings as immutable inputs it is not permitted to contradict. Its job is judgement - which findings matter for this business, how the architecture behaves under change, what a non-engineer needs to understand - anchored to evidence it did not generate.

Reverse that order and you get fluent fiction. Keep it and you get a report someone can act on without having to trust you.

What To Do On Monday

Take the four measurements once, on your main branch, and write them down. That baseline is the entire point - a single reading tells you very little, and a reading you can compare against in three months tells you whether the trend is real. Then keep the existing review checklist exactly as it is, because it is catching a different class of problem.

If you would rather have an outside reading than build the pipeline yourself, that is what an AI code audit is: the same four measurements, plus security findings with file and line evidence and a fix plan ordered by risk. And if what you are actually worried about is an AI feature behaving unpredictably rather than a codebase becoming unmaintainable, that is a different problem with a different instrument.

Frequently Asked Questions