AI security and architecture review, before your AI system meets real customers.
Your LLM, RAG or agent system works. Before it goes to paying customers, an enterprise pilot or a regulated buyer, someone should check what it can be talked into, what it can leak, whether one customer can ever see another’s data, and whether the design will hold when usage grows.
I review AI systems the way I build them: security and architecture together, with evidence for every finding and a fix order your team can act on. It is an architect’s review from the inside, not a penetration test, and the report says when you need one of those as well.
Or describe your project in the chat, or email nic.chin@nicchin.com
What an AI security and architecture review checks
Six areas, weighted by what your system can reach and who uses it. To run a first pass yourself, the LLM security checklist lists the checks and the evidence for each, and what to log in an LLM audit trail covers the area most teams have not built.
Prompt injection and jailbreaks
Direct attacks in the chat box and indirect ones planted in documents your system retrieves. Input guardrails for injection, role manipulation, jailbreaks and data-exfiltration patterns, tested with real attack cases rather than assumed.
Data leakage in answers
Personal data, credentials and secrets that the model can repeat back. Output checks for PII and secrets, and whether the model was ever given data the asking user should not see.
Tenant isolation, end to end
Whether one customer can read another customer’s data through any API path, and through the vector store, which is where isolation is most often forgotten.
Sessions, secrets and platform
Token lifetimes, admin access, secrets in code and configuration, exposed API documentation, security headers, container and infrastructure hardening.
Audit trail and traceability
Who asked what, which documents were used, what the model answered and what it cost, recorded in a way an enterprise buyer or regulator can rely on.
Architecture under load
Retrieval design, evaluation, model fallbacks, cost per request, one user running up the bill, load behaviour, backup and rollback. The things that decide whether the system survives its first real customers.
Example: an AI platform for pharmaceutical manufacturers
A multi-tenant platform that analyses manufacturing and quality documents with a RAG chatbot was heading into pilots with large pharmaceutical companies. It worked. It was not ready. The client is anonymised; the figures are from the engagement record.
26
pass/fail security checks, finding 2 critical, 4 high and 3 medium issues
15+
guardrail test cases for injection, jailbreaks, PII and secrets
24/24
acceptance criteria passed across the engagement
What the review found and what was fixed:
- Cross-tenant access. One document endpoint had no tenant filter, so one customer could read another’s documents. Fixed with tenant-scoped queries, then isolation was tested across every API path. A separate isolation gap in the vector store was found and documented.
- Session lifetime. Access tokens lasted seven days, so a stolen token meant a week of access to sensitive manufacturing data. Cut to eight hours, one work shift.
- AI guardrails. Input checks for prompt injection, role manipulation, jailbreaks and data-exfiltration patterns, and output checks for personal data and secrets, with a switch between blocking and logging so sensitivity could be tuned in production.
- Audit trail. Critical actions, including AI queries, logged with request tracing to append-only storage kept for 365 days, the foundation a regulated buyer asks for.
- Platform. Six container deployments hardened to run without root privileges, public API documentation removed from production, security headers and alarms added, and a compliance gap memo written against SOC 2, ISO 27001 and FDA 21 CFR Part 11.
Over seven weeks the platform went from functional to approved for enterprise pilot deployment with pharmaceutical clients, with every claim backed by evidence. The same discipline sits behind my multi-tenant RAG isolation and 21 CFR Part 11 guides.
The same controls, built rather than reviewed
A reviewer should have shipped the controls they check. On SureCiteAI, a live multi-tenant RAG product I built as sole architect, tenant isolation is enforced in four independent layers: Supabase row-level security in the database, a separate Pinecone namespace per tenant in the vector store, session tokens that carry the tenant and are checked on every request, and tenant resolution at the edge. A three-tier model failover with a circuit breaker keeps it answering through a provider outage, and every answer is checked against its sources, with 0 hallucinated citations across 297 cases in the published benchmark runs.
Security review, penetration test or compliance audit?
They answer different questions. A penetration test asks whether an outsider can get in. A compliance audit asks whether your controls meet a standard, and ends in a certificate only an accredited auditor can give. An AI security and architecture review asks whether the system is designed to be safe: what the model can reach, what enforces each boundary, and what happens when a control fails. It is the one to do first, because it finds the design problems that make the other two expensive.
Architecture: will it survive real customers?
Security is half of the review. The other half is whether the design holds: how retrieval is built and evaluated, what happens when a model provider slows down or fails, what each request costs and who can run that cost up, how the system behaves under realistic load, and whether you can restore it and roll it back. The report separates what must change before customers arrive from what can wait.
What you get
- A written report with evidence behind every finding, ranked by severity and exposure.
- A fix order that makes each change cheaper than the last, and a clear line between before-launch and later.
- A walkthrough with your engineers, remotely or on site.
- Optional retest of the fixes, so you can show a customer or investor that the findings are closed.
Each review is quoted to the system after a short call, because the scope is set by what the AI can reach, the number of data stores and the compliance context rather than by hours. The cost guide explains what drives that.
Which review do you need?
Frequently asked questions
What does an AI security review cover?
Everything an attacker, a careless user or an auditor could reach through the AI parts of your system and the platform around them: prompt injection and jailbreaks, including instructions hidden in documents your system retrieves; what the model can see, call and send; personal data and secrets leaking in answers; whether one customer can ever retrieve another customer’s data, in the database and in the vector store; session and token handling; secrets in code and configuration; the audit trail an enterprise or regulated buyer will ask for; and whether one user can run up your model bill. Each finding comes with evidence, a severity and a fix.
Is this a penetration test?
No. A penetration test asks whether an outsider can break in from the outside, and it is often something an enterprise customer requires from an accredited tester. This review reads the design and the code from the inside and asks whether the system is built to be safe: where the boundaries are, what enforces them and what happens when a control fails. It usually finds the problems a pentest would find later, earlier and cheaper, and the report says plainly when you also need a pentest and what to ask the tester to cover.
How is this different from an AI code audit?
An AI code audit measures the health of a codebase, often one written with AI tools: duplication, dead ends, error handling that hides failures. This review is about the AI system as a system: how retrieval, the model, tools, permissions and data stores fit together, and whether that design is secure and will survive real customers and scale. If the main worry is an app you built yourself with Lovable, Cursor or Claude Code, the lighter AI-built app review is the better fit.
Which AI stacks can you review?
Systems built on OpenAI, Anthropic, Google Gemini, Azure OpenAI or AWS Bedrock, with retrieval on pgvector, Pinecone or similar stores, orchestrated with LangGraph, LangChain, custom code or managed agent platforms. The questions are the same whatever the stack: what the model can reach, what enforces the boundaries, and how you would know if one failed.
Do you need access to production?
No. Read access to the repository, the infrastructure configuration and a staging environment is normally enough, and I will sign your NDA before seeing anything. Where a finding needs to be proven, it is proven in staging with test data.
Can the review happen on site?
Reviews run remotely by default, across the UK, US and Europe. On-site sessions are available in the UK, and elsewhere in Europe by arrangement. An on-site walkthrough is often the fastest way to cover the architecture with your engineers.
Going to paying customers, an enterprise pilot or a regulated buyer?
Tell me what the system does, what it is built on and who will use it. If a security review is not what you need yet, I will say so and tell you what is.
Or describe your project in the chat, or email nic.chin@nicchin.com
Last updated