RAG
Retrieval-Augmented Generation: improve factuality with retrieval
General-purpose language models can handle common tasks like sentiment analysis and named entity recognition with fine-tuning. These tasks don't need extra background knowledge.
But for more complex, knowledge-intensive tasks, you can build a system on top of a language model that accesses external knowledge sources. This makes outputs more factually consistent, more reliable, and helps reduce hallucinations.
Meta AI researchers introduced Retrieval Augmented Generation (RAG) for exactly these kinds of tasks. RAG combines an information retrieval component with a text generation model. It can be fine-tuned, and its internal knowledge can be updated efficiently without retraining the whole model.
RAG takes an input, retrieves a set of relevant/supporting documents (from a source like Wikipedia), and combines those documents as context with the original prompt before feeding everything to the text generator. This makes RAG much better at handling facts that change over time. And that matters -- LLM parametric knowledge is static. RAG lets the model access the latest information without retraining, producing reliable outputs based on retrieval.
Lewis et al. (2021) proposed a general RAG fine-tuning recipe. It uses a pre-trained seq2seq model as parametric memory and a dense vector index of Wikipedia as non-parametric memory (accessed via a neural pre-trained retriever). Here's how it works:
Image source: Lewis et al. (2021)
RAG performed strongly on benchmarks like Natural Questions, WebQuestions, and CuratedTrec. On MS-MARCO and Jeopardy questions, RAG generated answers that were more factual, more specific, and more diverse. FEVER fact verification also improved with RAG.
This shows RAG is a viable approach for boosting language model output on knowledge-intensive tasks.
Recently, retriever-based methods have become increasingly popular, often combined with popular LLMs like ChatGPT to improve their capabilities and factual consistency.
You can find a simple example of using a retriever and LLM for question answering with source citations in the LangChain docs.
📚 Related resources
❓ Common questions
Open a question to review the practical answer.
RAG was introduced by Meta AI — what problem does it actually solve?
RAG (Retrieval Augmented Generation, Lewis et al. 2021) bolts a retrieval component onto a text generator to tackle knowledge-intensive tasks. On every query it pulls supporting documents from an external store (e.g. Wikipedia), then feeds them into the prompt alongside the user input. Payoff: less hallucination, citable sources, and knowledge updates without retraining the whole model.
RAG vs fine-tuning — how do I choose?
If facts change frequently — product docs, regulations, internal wiki — use RAG, since updating the vector store leaves the model untouched. If you need style, domain tone or a fixed output format, fine-tune. They stack well: fine-tune for voice + RAG for live facts is the standard enterprise QA recipe.
Which benchmarks did Lewis 2021's RAG actually beat?
The paper reports strong results on Natural Questions, WebQuestions and CuratedTrec; on MS-MARCO and Jeopardy, RAG's answers were more factual, more specific and more diverse than seq2seq baselines; FEVER fact verification also improved. Across the knowledge-intensive QA + fact-checking band, RAG was a real win.
What does RAG use for the retriever and generator?
Lewis 2021's recipe pairs a pretrained seq2seq model (parametric memory = generator) with a dense vector index over Wikipedia (non-parametric memory, accessed by a neural pretrained retriever). Modern stacks swap in dense embeddings (OpenAI, BGE) for retrieval, GPT-4 / Claude for generation, and Pinecone / Weaviate / pgvector for the index.
ChatGPT is already strong — do I still need RAG?
Yes. An LLM's parametric knowledge is frozen at training cutoff — anything after that, plus your internal docs, contracts and policies, is invisible to it. RAG gives the model fresh, citable information without retraining, which is why retriever-based methods now sit on top of ChatGPT-class LLMs by default to keep answers factually consistent.