Multi-Tenant RAG Isolation: Four Layers, Not One Filter

The Short Answer

A metadata filter is not tenant isolation. It is a correctness mechanism doing a security job, and it fails the way correctness mechanisms fail: silently, in the one code path nobody remembered.

Vector databases generally ship no native access-control model. That means isolation is whatever your application enforces - and if a single filter in a single query builder is the whole control, then a new export endpoint, a background job or an internal admin tool that omits it will quietly serve one customer another customer’s documents. Nothing will error. The answer will simply be wrong, and wrong in the most expensive possible way.

The alternative is four independent layers, each failing closed on its own, so no single omission is sufficient to expose data.

The Four Layers

This is the architecture behind SureCiteAI, a multi-tenant document-intelligence platform running in production. The layers are described here in the order a request passes through them.

The value is not in any individual layer - each is fairly ordinary engineering. It is in their independence. Layer 2 catches the mistake in layer 3’s code. Layer 3 makes layer 2’s failure non-fatal at the retrieval step. A design where all four are enforced by the same query builder has one layer wearing four hats.

1. Identity that cannot be asserted by the caller

The tenant identifier must arrive from a verified token and never from a parameter the client controls. This sounds obvious and is violated constantly, usually by internal tooling that passes a tenant ID explicitly “because it is only used by staff” - and then by a feature that reuses the same helper.

2. Row-level security at the database

PostgreSQL row-level security, or its equivalent, makes the storage layer refuse cross-tenant reads regardless of what the application asks for. This is the layer that saves you, because it is enforced somewhere other than the code most likely to contain the bug. Every new endpoint inherits it by default rather than by remembering.

3. Namespaces in the vector store

Namespaces, collections or per-tenant indexes give query-time separation inside the vector store, so a similarity search executes against one tenant’s vectors rather than filtering another tenant’s out afterwards. The distinction matters: filtering after retrieval means the data was reachable and something chose to exclude it.

Namespaces are the right default. Reserve dedicated indexes for tenants with a regulatory or contractual requirement, and decide that by policy rather than per customer. Which store you are on affects the mechanics here - see vector databases for enterprise RAG for how partitioning differs between them.

4. Routing and session boundaries

The subtlest layer. In pooled connections, long-lived worker processes, or anything caching per-process state, a tenant context set for one request can survive into the next. The fix is that tenant context is established per request and never inherited - and that caches are keyed by tenant, always.

The Model Is Not an Access-Control Layer

The most consequential mistake in this category is asking the model to enforce the boundary. A system prompt instructing it not to reveal other tenants’ data is a request, not a control, and prompt injection exists precisely to turn requests into suggestions.

The correct framing: if the wrong document was retrieved, the failure has already happened. What the model says next is irrelevant to whether you had a breach - the data was in the context window. So the boundary belongs at retrieval, before anything is assembled into a prompt, and the model should be treated as an untrusted consumer of whatever you hand it.

This is the same principle as the deterministic control layer in stopping hallucinations: controls the model cannot override, enforced in code, inspectable by a reviewer.

How Leakage Actually Happens

Cross-tenant exposure is rarely dramatic. In production the causes are mundane, which is exactly why layering beats a well-written filter:

• A new code path that forgot the filter. An export feature, an admin tool, a scheduled summarisation job. The main chat endpoint is well tested; the CSV export written in a hurry is not.
• A cache keyed without the tenant. Two tenants ask a similar question; the second is served the first’s cached answer. The retrieval layer was correct and the cache defeated it.
• Evaluation and debugging datasets. Built by copying production data across tenants, then living outside the access-control model entirely. Also the artefact most often missed during erasure requests.
• Client-supplied tenant identity. A parameter that should have been a token claim.

Testing It, Because It Regresses Silently

Isolation is the property most likely to break without anyone noticing, because nothing errors when it does. It needs adversarial tests in CI, not a design review.

The minimum useful test suite:

• Seed two tenants with deliberately distinctive content - a unique token string in each makes assertions trivial.
• Assert tenant A never retrieves tenant B content across every retrieval path: chat, search, export, background jobs, admin tooling, evaluation harness.
• Include prompt-injection attempts that instruct the system to ignore its constraints, and assert the retrieved set is unchanged - you are testing retrieval, not the model’s politeness.
• Assert cache keys include the tenant identifier, and that a warm cache for one tenant does not serve another.

On a pharmaceutical engagement, closing exactly this class of gap - cross-tenant isolation plus prompt-injection guardrails - was what moved a blocked platform to 24 of 24 acceptance criteria passed and FDA 21 CFR Part 11 readiness. The isolation was not the whole review, but it was the item that had to be evidenced rather than asserted, which is covered further in validating LLM systems under 21 CFR Part 11.

Frequently Asked Questions

Is a metadata filter enough for multi-tenant RAG isolation?

No. A metadata filter is a correctness mechanism, not a security boundary. It lives in application code, so one missing filter in one code path exposes every tenant, and nothing outside that code enforces it. Vector databases generally have no native access-control model, so if the filter is your only control then a bug, a new endpoint, a background job or an internal tool that forgets it produces silent cross-tenant retrieval. Keep the filter - it is necessary - but it should be the layer you least depend on.

What are the layers of a multi-tenant RAG isolation architecture?

Four independent layers, each failing closed on its own: verified identity, where the tenant ID comes from a signed token and never a request parameter; storage-level isolation such as PostgreSQL row-level security, so the database refuses cross-tenant reads even when application code asks; vector-store partitioning via namespaces, collections or per-tenant indexes, so similarity search cannot physically reach another tenant’s vectors; and routing and session isolation, so tenant context cannot be inherited or spoofed across requests. The property that matters is independence.

Should each tenant get its own namespace or its own index?

Namespaces are the default and sufficient for most tenants, giving physical query-time separation within one index at far lower operational cost. Move a tenant to a dedicated index when there is a regulatory or contractual reason, or scale that justifies isolating noisy-neighbour risk. It does not have to be uniform: namespaces by default, per-index isolation available for tenants that need it, decided by policy rather than case by case.

Can the LLM be trusted to enforce tenant access control?

No, and this is the most consequential mistake in the category. Instructing a model not to reveal other tenants’ data is a request, not access control. Isolation has to happen at retrieval, before anything reaches the model, so the context window physically cannot contain another tenant’s records. If the wrong document was retrieved, the failure has already occurred regardless of what the model then says.

How do you test that tenant isolation actually works?

With adversarial tests in CI, because this property regresses silently. Seed at least two tenants with deliberately distinctive documents, then assert tenant A queries never return tenant B content across every retrieval path - not just the main chat endpoint, but search, exports, background jobs, admin tooling and any evaluation harness. Include prompt-injection attempts. A test covering only the happy path through the primary endpoint proves very little.

What usually causes cross-tenant leakage in production RAG?

Rarely a dramatic breach. In practice: a new code path that forgot the filter, such as an export feature or scheduled job; a cache keyed without the tenant identifier, so one tenant is served another’s cached answer; an evaluation or debugging dataset built from real production data across tenants; or tenant identity taken from a client-supplied value rather than a verified token. Each is mundane, which is why layered enforcement beats a carefully written filter.

Independence Is the Whole Design

If you take one thing from this: the number of layers matters less than whether they fail independently. Four controls enforced by the same query builder is one control. Two controls in genuinely different places - the database and the vector store - is meaningfully safer than four in the same file.

Test it adversarially, in CI, across every retrieval path, because this is the failure that does not announce itself. How I build production RAG covers the pipeline this sits inside, and the SureCiteAI case study documents the system these four layers come from.