Back to PortfolioCode Audit · Evidence First

An AI code audit that shows you the evidence, not an opinion about your code.

Your team shipped fast with AI coding tools and it works. The question nobody can answer is whether it will still be maintainable in a year, and whether anything dangerous went in along the way.

I audit codebases and report what is actually there: vulnerabilities with the file and line, the real dependency graph, which features are genuinely covered by tests, and what to fix in what order.

I built the analysis engine this audit runs on, and its scanner is open source. SystemAudit is a live platform that turns a repository into a system health report, and it is continuously tested against a corpus of ten real public codebases ranging from 800 to over 600,000 lines. You can read how it works rather than take a methodology claim on trust.

Why AI-generated code needs auditing at all

Adoption of AI coding assistants reached 84% in the Stack Overflow Developer Survey, while trust in what those tools produce fell to 29% from 40% the previous year, and 45% of developers said debugging AI-generated code takes longer than writing it themselves. The most-cited frustration, at 66%, was solutions that are almost right, but not quite.

That phrase is the whole problem. Almost-right code passes review, passes tests that were generated alongside it, and ships. The cost arrives later, and it is measurable: GitClear’s 2026 analysis found duplicated code blocks at 73.0 per million changed lines so far in 2026 — an 81% rise over 2023 and the highest level on record — alongside a 41% increase in within-commit copy/paste and a 47% rise in error-masking constructs.

None of that shows up as a bug report. It shows up as a team that gets slower every quarter for reasons nobody can name, and it is invisible until somebody measures the repository as a whole.

The hard part: you cannot audit AI-written code with AI alone

The obvious approach is to point a language model at the repository and ask what is wrong. It does not work, and it fails in the most dangerous way available: it produces a confident, well-written report containing vulnerabilities that are not in your code and missing ones that are. The model is pattern-matching against everything it has seen rather than reading what you actually shipped.

The fix is not a better prompt. It is ordering. Deterministic analysis runs first and produces facts; the model runs second and is not permitted to contradict them.

Two layers, in order

This is the architecture behind SystemAudit, and it is why its reports cite line numbers instead of adjectives.

Layer 1 — evidence

A static analysis pass runs before any model sees the code. It produces exact vulnerability patterns with file and line references, the full import dependency graph including proven circular dependencies, dependency health flags, structure metrics and verified configuration state. These are facts. They are reproducible, and they cost nothing to check.

Layer 2 — interpretation

The model receives the source alongside every Layer 1 finding as an immutable constraint. Its job is judgement, not discovery: which findings matter for this business, how the architecture will behave under change, and what a non-engineer needs to understand. It is anchored to evidence it did not generate.

What the audit covers

AreaWhat you getDetermined by
SecurityHardcoded credentials, injection patterns, unsafe DOM writes and exposed sensitive data — each with the exact file and lineStatic analysis
ArchitectureA map of how components actually connect, rather than how the documentation says they doBoth layers
DependenciesCircular dependencies, single points of failure and dead code, proven rather than estimatedStatic analysis
Test coverageFeatures cross-referenced against test files, so you see what is genuinely covered and what only appears to beStatic analysis
AI readinessFive dimensions — code clarity, test coverage, modularity, documentation and type safety — graded A to FBoth layers
RemediationA fix plan ordered by risk, stating what each item costs to fix and what it costs to ignoreInterpretation

Languages and stacks

The scanner covers more than 50 languages across nine major ecosystems — JavaScript and TypeScript, Python, Java and Kotlin, Go, Rust, C# and .NET, PHP, Ruby — plus Docker, CI/CD configuration, serverless and monorepo layouts. If your stack is unusual, say so on the call and I will tell you honestly whether the automated pass adds anything or whether the value is in the manual review.

Who this is for

  • Founders who cannot read the code they own. You paid for it, or generated it, and need to know what you have before hiring against it.
  • Investors and acquirers. Technical due diligence on a target, where the dependency graph and the test coverage tell you more than the pitch deck.
  • CTOs inheriting a codebase. A baseline in week one instead of a guess in month three.
  • Teams that adopted AI coding tools quickly. Velocity went up and nobody has measured what it cost.

How engagements work

Every audit starts with a short call to agree scope, because the useful question is never “is this code good” but “is this code good enough for what you are about to do with it”. A pre-acquisition review and a pre-hiring baseline look at the same repository and report different things.

From there the analysis pass runs, I interpret the findings against your situation, and you get a written report you can hand to a board, an investor or the team that has to act on it. Where the work continues past the audit, my consulting engagements cover the remediation itself, and my company Bitto Tech handles full delivery where a team is needed.

If the underlying problem is an AI system that is unreliable rather than a codebase that is unmaintainable, that is a different page — and if you are building retrieval into the product, RAG implementation is where that starts.

Frequently asked questions

What is an AI code audit?

An independent review of a codebase to establish what it actually contains: security vulnerabilities with exact file and line references, the real dependency graph, which features are covered by tests, and where the structural risk sits. For AI-generated code specifically it also answers whether the codebase can be maintained and extended by a team, which is the failure mode that shows up months after the code ships and appears to work.

How is auditing AI-generated code different from a normal code review?

Conventional review assumes a human author who had a reason for each decision, so the reviewer looks for mistakes. AI-generated code fails differently: it is usually locally correct and globally incoherent. The same logic is reimplemented in four places with slight variations, errors are swallowed rather than handled, and abstractions do not hold across the codebase. Those are structural properties you find by measuring the whole repository, not by reading diffs.

Can you not just point another AI at the code?

Not on its own, and this is the central problem. A language model asked to review code will confidently report vulnerabilities that do not exist and miss ones that do, because it is pattern-matching against training data rather than reading your repository. The fix is ordering: run deterministic static analysis first, then give the model those findings as constraints it is not permitted to contradict. Evidence first, interpretation second.

Do you need access to our private repository?

For the full audit, yes - read access is enough, and it can be scoped to a single branch or a mirror. Where access cannot be granted, a scoped review of exported source, architecture documentation and dependency manifests still produces useful findings, though feature-to-test verification requires the test suite.

What do we actually receive at the end?

A written report: architecture map, security findings with file and line evidence, dependency and circular-dependency analysis, feature-to-test coverage verification, an AI readiness assessment across code clarity, test coverage, modularity, documentation and type safety, and a prioritised remediation plan ordered by risk rather than by ease. Each finding states what it costs to fix and what it costs to leave.

How long does an audit take?

The automated analysis pass completes in minutes. The part that takes real time is interpretation - reading the findings against what the business is trying to do, which decides whether a circular dependency is a genuine risk or an acceptable trade-off. Scope and timeline are agreed on a short call before anything starts.

Find out what is actually in your codebase

Bring a repository and the decision you are trying to make with it. I will tell you on the call whether an audit is the right instrument, and say so if it is not.

Last updated