How to Choose a RAG Development Partner: A US Buyer’s Checklist

There is a specific conversation I have had more times than any other with US buyers in 2026 - in New York, San Francisco, Chicago and a dozen remote-first companies in between. A company has a corpus - contracts, policies, claims files, clinical protocols, SEC filings, a decade of support tickets - and someone has demonstrated that a language model can answer questions about it. The demo worked. Two vendors have now quoted the “RAG implementation,” and the quotes differ by a factor of five. The buyer wants to know which one is padded.

Usually neither is. They are quoting different systems. The cheap quote is for the three components that made the demo work. The expensive quote is for the nine components that keep it working when a compliance officer, a lawyer, and four hundred employees start using it. This article is the scope map I use to make that difference legible before anyone signs anything.

What Are You Actually Buying in a RAG Implementation?

Retrieval-augmented generation is not a product you install. It is an information retrieval pipeline with a language model at the end of it, and the language model is the least interesting part. Buying a RAG implementation means buying a decision on each of the following nine components. Every one of them has a cheap answer that demos well and an expensive answer that survives production.

Take that table into your next vendor call and ask which column each line item lands in. The conversation changes immediately, because a vendor who has shipped production RAG has opinions about all nine, and a vendor who has shipped a demo has opinions about three.

Why Do So Many US RAG Pilots Stall?

The pattern is consistent enough to be predictable. A pilot succeeds on a curated corpus with a friendly test group, then dies somewhere between pilot and rollout - not because the technology failed, but because three things arrive at once: messy real documents, users who ask questions nobody anticipated, and a security review.

The messy documents break ingestion. The unanticipated questions expose that nobody built an evaluation set, so there is no way to tell whether a fix is a fix. And the security review asks the question that ends more pilots than any other: can this system return a document the person asking is not cleared to see? If access control was implemented as UI filtering rather than enforced inside the retrieval query, the honest answer is yes, and the project goes back to the drawing board with its credibility spent.

This is a scoping failure, not an engineering failure. All three problems are cheap to prevent and expensive to retrofit, which is precisely the shape of the architecture mistakes that kill enterprise AI projects.

How Do SOC 2, HIPAA, and State Privacy Law Change the Architecture?

US buyers ask whether a RAG system is “SOC 2 compliant.” The system is not the thing that gets certified - your organisation is. But the architecture either supports your controls or quietly undermines them, and the differences are concrete:

• SOC 2 pushes three requirements into the design: authorisation enforced at retrieval time (a filter inside the query, not a filter over the results), an audit trail that records who asked what and which documents were surfaced, and change management over prompts, indexes, and model versions - because in a RAG system those are production configuration, not content.
• HIPAA changes the data path. PHI in embeddings is still PHI. That means a business associate agreement with any model or vector provider in scope, de-identified or in-boundary embedding where the BAA does not exist, and retention limits on the query logs that observability requires - because those logs contain user questions, and user questions contain patient details.
• State privacy law (CCPA/CPRA and the wave of statutes that followed it) makes deletion an architectural requirement. When a subject requests erasure, you must be able to remove their data from the source system, the chunk store, the vector index, and the caches. Systems that treat the index as a derived artefact nobody tracks cannot honour that request without a full rebuild.
• Regulated-industry review - financial services under SEC/FINRA expectations, life sciences under FDA scrutiny - adds one more: every answer needs a provenance chain a human reviewer can follow back to a source document, which is a retrieval and citation design problem long before it is a policy problem.

None of this is exotic. It is roughly two to three weeks of design and hardening work when planned, and a re-architecture when discovered in month five.

What Does a Production RAG System Look Like When It Works?

I will use my own, because I can show you the internals. SureCiteAI is a multi-tenant document intelligence platform I designed and built as sole architect - a twelve-component RAG pipeline running at 96.8% retrieval accuracy in production, with hybrid search, hierarchical retrieval, tenant isolation enforced at the database layer, and citation verification that checks every reference against the retrieved span before the answer is returned. The code is open source and the product is live.

The number I would actually judge a vendor on, though, is not accuracy. It is the hallucinated-citation count. On the most recent published evaluation run - against public suites including PatronusAI FinanceBench, CUAD, openFDA, and SEC EDGAR - the system returned 0 hallucinated citations across 297 cases, with a 221/297 aggregate retrieval hit rate. Both numbers matter and the second one is the honest one: 74% is a real hit rate on hard public questions, not a marketing figure. The methodology and the raw run artefacts are on the proof page so you can reproduce them rather than take my word for it.

A domain example of the same discipline: a legal document analysis platform I built for an investment fund law practice processes 150 to 200 page Limited Partnership Agreements, extracting six critical clause categories with risk analysis, delivered through a Microsoft Word add-in so lawyers never leave the document they are already in. It eliminated over 70% of manual LPA review time - and the reason it was adopted rather than shelved is that every extracted clause points back to its exact location in the agreement. Reviewers trust what they can check.

What Does a Realistic Delivery Sequence Look Like?

A focused single-corpus production build runs six to ten weeks. The sequence matters more than the total, because it determines where you can walk away:

1. Weeks 1-2 - discovery and the evaluation set. Sample the real corpus, not the clean one. Build a labelled question set with your subject-matter experts - a hundred questions with known correct sources is enough to change everything that follows. Deliverable: architecture document, evaluation set, delivery estimate. You keep all three whether or not you continue.
2. Weeks 3-5 - retrieval. Ingestion, chunking, index design, hybrid retrieval, reranking, all measured against the evaluation set from day one. This is where the quality of the final system is actually determined. Weekly demos should show the hit rate moving.
3. Weeks 5-7 - generation and grounding. Answer synthesis, citation verification, refusal behaviour when retrieval returns nothing good. A system that says “I do not have a source for that” is worth more than one that always answers.
4. Weeks 7-10 - hardening. Access control at the retrieval layer, audit logging, model failover, load behaviour, cost controls, and the compliance items above. Unglamorous, non-optional, and the phase most often cut when a deadline slips.

Multi-tenant or multi-corpus systems extend the hardening phase rather than the build phase, which surprises people. Isolation and per-tenant configuration are where that complexity lands.

How Do You Choose a RAG Development Partner in One Call?

Seven questions, and you can run them in thirty minutes. What you are listening for is whether the answers are specific:

1. “What does your evaluation set look like?” Anyone who has shipped RAG has one. The answer should include how it was built, how big it is, and what the current hit rate is. “We test it manually” means no production experience.
2. “What are your retrieval failure modes?” Real practitioners answer instantly - acronyms, negations, cross-document questions, tables. A vendor with no failure modes has not looked.
3. “How is a citation verified?” The answer you want is a mechanical check against the retrieved span. The answer you do not want is “we instruct the model to cite.”
4. “Where is authorisation enforced?” Inside the retrieval query. Any other answer means the retrieval layer can see more than the user can.
5. “What happens when the model provider has an outage?” Failover and circuit breakers, or an honest “the system is down.” Both are acceptable answers; not having considered it is not.
6. “Whose accounts does this live in?” Your cloud, your GitHub org, your vector store, from day one. No handoff cliff at the end of the engagement.
7. “What would you refuse to build?” The most revealing question on the list. Consultants who have shipped enough systems have a list of things RAG is bad at. Consultants who will build anything you describe are selling capacity, not judgement.

For the broader hiring frame - independents versus boutiques versus Big 4 practices - see top AI consultants in the US and how to hire an AI consultant.

When Is RAG the Wrong Thing to Buy?

Three cases come up repeatedly, and in all three the honest recommendation is to spend the money elsewhere:

• Your corpus is small and stable. A few hundred pages that rarely change may not need retrieval infrastructure at all - long-context prompting with a curated set of documents can be sufficient and dramatically simpler to operate.
• The real problem is format, not knowledge. If you need the model to write in a house style, follow a rigid schema, or handle a specialised syntax, that is a fine-tuning or structured-output problem. The distinction is worked through in RAG vs fine-tuning.
• The task is a workflow, not a question. “Answer questions about our contracts” is RAG. “Route the contract, extract the terms, check them against policy, and open a ticket” is human-approved AI automation with retrieval as one step inside it - and if the sequence of steps changes case by case, it may be an agent build instead, which I scope in AI agent development services. Buying the wrong one costs you a rebuild.

A vendor whose answer to all three is still “you need RAG” is answering from their capability, not your problem. The build versus buy framework is the right next read if you are still deciding whether to own the system at all.

Frequently Asked Questions

How do I choose a RAG development partner?

Compare candidates on six things rather than on their tech stack. First, the evaluation set — any partner who has shipped RAG into production has a labelled question set and can quote their current hit rate. Second, verifiable accuracy: reproducible public benchmark runs beat self-reported percentages. Third, compliance fluency for your industry, specifically how authorisation is enforced at retrieval time. Fourth, whether they will start with a paid two-week discovery sprint whose deliverables you keep either way. Fifth, whether code and infrastructure live in your accounts from day one. Sixth, whether they can name something they would refuse to build — partners who will build anything are selling capacity, not judgement. The seven-question call script above operationalises all six.

What is included in a RAG implementation service?

Nine components, not one: document ingestion and parsing, chunking strategy, embedding and index design, retrieval (usually hybrid vector plus keyword), reranking, grounded generation with verified source attribution, an evaluation harness, access control and tenant isolation, and observability. Vendors who scope only ingestion, embeddings and a chat endpoint are quoting a prototype. The components that decide whether the system survives contact with real users are evaluation, retrieval quality and access control - and those are the ones most often missing from the statement of work.

How long does a RAG implementation take for a US company?

A focused production build on a single corpus typically runs six to ten weeks: roughly two weeks of discovery and evaluation-set construction, three to five weeks of retrieval and generation work measured against that evaluation set, and two to three weeks of hardening - access control, audit logging, failover and load behaviour. Multi-corpus and multi-tenant systems extend the hardening phase rather than the build phase. Anyone promising a production RAG system in two weeks is describing a demo.

Does a RAG system need to be SOC 2 or HIPAA compliant?

The system does not hold a certification - your company does - but the architecture either supports your controls or quietly breaks them. For SOC 2 that means authorisation enforced at retrieval time rather than in the UI, audit logs covering every query and every document surfaced, and change management over prompts, indexes and model versions. For HIPAA it means PHI kept out of embedding paths that leave your boundary, a business associate agreement with any model or vector provider in scope, and retention limits on query logs. Retrofitting these after launch costs substantially more than designing for them.

How do I know whether a RAG vendor can actually build production systems?

Ask for the evaluation set. Any vendor who has shipped RAG into production has a labelled question-and-answer set they test retrieval against, and can state their current hit rate and their failure modes without hesitating. Vendors who answer with model names, framework names or a demo video have built a prototype. The second-strongest signal is reproducibility: published benchmark runs against public suites, with raw artefacts, beat self-reported accuracy percentages by a wide margin.

Can we start with an off-the-shelf RAG product and replace it later?

Often yes, and for a first internal use case it is frequently the right call. Switching cost is low while the corpus is small and the users are internal. It rises sharply once the system is customer-facing, once retrieval quality becomes a differentiator, or once compliance requires controls the vendor does not expose. Decide deliberately rather than by drift - the trap is discovering in month nine that the product you can no longer live without is the one you cannot audit.

Do we need a vector database, or can we use the infrastructure we already have?

Postgres with pgvector is genuinely sufficient for a large share of corpora, and using infrastructure your team already operates is worth real money in reliability. Dedicated vector stores earn their place at scale, with heavy metadata filtering, or when you need managed hybrid search. A vendor who insists on a specific product before seeing your corpus is pattern-matching rather than designing.

Who from our team needs to be involved in a RAG project?

One subject-matter expert for roughly a day a week during the first month - they are the only people who can say whether an answer is correct, and no engineer substitutes for that. Plus whoever owns access control in your source systems, involved before week three rather than during the security review, and a named decision-maker who can approve the autonomy and retention choices without a committee.

What does an engagement with me look like?

A free discovery call to determine whether retrieval is even the right purchase, then a paid two-week discovery sprint producing the architecture document, the evaluation set, and a delivery estimate - yours to keep, and to take to another builder if you prefer. Builds run in your accounts with weekly demos against the evaluation set. See RAG implementation services for how the build itself is structured, AI consulting for US companies or custom AI development in the US for the fuller picture, and fractional AI CTO if what you actually need is ongoing technical leadership rather than a single build.