RAG (retrieval-augmented generation) — embeddings plus LLM for document search
RAG is the default architecture for Q&A over your own documents: an embedding model turns chunks and queries into vectors, a vector DB finds similar chunks, and a generative model answers with the retrieved passages as context. You need two models — one for embeddings, one for generation.
Is this your use case?
- ·You have a knowledge base (docs, wiki, support articles) and want AI answers grounded in it.
- ·Answers need to cite sources — no hallucinating.
- ·Data changes, so no fine-tuning — retrieval instead.
What to look for
- →Embedding model: strong semantics across your language mix, Matryoshka dimensions for storage trade-off.
- →Generative model: enough context for top-k chunks plus question plus system prompt.
- →JSON mode or structured output on the generator if you want citations as fields.
Recommended models
Best-of list, ordered by fit — not alphabetically.
- 1openai/text-embedding-3-largeTop pickOpenAI's large embedding model — 3072-dim, very strong semantic search.
Top MTEB scores for cross-language retrieval, API-shortenable dimensions. The workhorse embedding for RAG.
Model details - 2openai/text-embedding-3-smallBest budgetOpenAI's small embedding model — 1536-dim, very cheap, broadly applicable.
Clearly better than ada-002, much cheaper than 3-large. Default pick for mid-sized corpora and prototypes.
Model details - 3anthropic/claude-sonnet-4.6Anthropic's workhorse: top-tier coding and agentic, fair pricing.
As the generator: 200k context for plenty of top-k chunks, clean source citing, low hallucination.
Model details - 4google/gemini-3-proLongest contextGoogle's flagship: 2M token context, native video and audio input.
2M token context — fits a whole book library in one call. If your RAG recall is weak, this is the brute-force option.
Model details