A practical RAG checklist before you go to production

Retrieval can be a useful capability inside an AI application. Before release, work through this focused checklist.

1. Retrieval that matches real questions

• Chunking strategy matches how users ask questions (not arbitrary page splits).
• Hybrid search (keyword + vector) when acronyms and exact phrases matter.
• Re-ranking when top-k isn’t enough.

2. Grounding and citations

• Answers cite sources users can verify.
• “I don’t know” is a valid answer when context is thin.

3. Evaluation beyond “vibes”

• Golden set of Q/A pairs from real or realistic queries.
• Regression checks when you change embeddings, chunking, or models.

4. Latency and cost

• p95 latency budget agreed with the product team.
• Caching and batching where it helps without stale answers.

5. Safety and access

• Access control on documents and namespaces.
• Consider what request history and access information the operating team needs for sensitive workflows.

This isn’t exhaustive — but skipping these is how RAG demos turn into support tickets. If you want help pressure-testing your stack, we’re happy to book a short call.