Retrieval-Augmented Generation (RAG)

What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation (RAG) is a method where an AI application first searches a knowledge source, such as documents in a vector database, and adds the most relevant passages to the prompt before the large language model (LLM) answers. It grounds answers in current, company-specific data without retraining the model.

For example, an HR team builds an assistant over its policy documents and case files. An employee asks about the parental leave policy. The retriever finds the right policy page, but it also pulls a past case file that mentions a named colleague’s medical leave and salary, because the words matched. The model now has that colleague’s data in its context and may repeat it in the answer.

How RAG works

  1. Ingest. Documents are collected and split into chunks of a few hundred words.
  2. Embed. An embedding model turns each chunk into a vector, a list of numbers that captures its meaning.
  3. Store. Vectors and the chunk text go into a vector database or search index.
  4. Retrieve. A user’s question is embedded and matched against the stored chunks.
  5. Generate. The top chunks are added to the prompt, and the LLM writes an answer from them.

The term comes from a 2020 research paper from Facebook AI Research. RAG is now a standard way to connect LLMs to company documents, from internal help desks and policy assistants to customer support bots and AI agents that look up records before they act.

RAG vs fine-tuning

RAG Fine-tuning
How it adds knowledge Looks up documents at question time Changes the model’s weights with new training data
Updating content Re-index the changed documents Train again
Where sensitive data sits In the index, the prompts and the answers Inside the model, where it is hard to remove

Where sensitive data leaks in RAG

  • Ingestion copies raw PII and PHI into chunks and embeddings.
  • Retrieval often ignores who is asking, so any user can pull any chunk.
  • Retrieved text goes to the model provider inside the prompt.
  • Answers can repeat sensitive values word for word.
  • Logs and caches keep copies of prompts and answers.

A safer RAG design adds two controls. Permission-aware retrieval filters chunks by what the asking user may read. Masking at ingestion keeps raw values out of the index in the first place. The first decides which documents a user gets. The second decides which values inside those documents ever appear in clear text, to the model or to the user. The guide on how to keep PII out of a vector database covers the ingestion side in detail.

How Protecto helps

Protecto Secure RAG tokenizes sensitive values when documents are ingested, so chunks and embeddings hold consistent tokens instead of raw PII. At answer time, each reader sees only the real values their role and context allow, and the rest stay masked. Because the same value always gets the same token, retrieval and answers still connect the right records.

Frequently asked questions

What does RAG stand for in AI?

RAG stands for retrieval-augmented generation. The AI application retrieves relevant documents and adds them to the prompt so the model can answer from them.

Does RAG keep my data private?

Not by itself. RAG copies document text into an index and into prompts, so any sensitive data in the source documents can reach the model, the logs and users who ask the right question.

Should I mask data before or after embedding?

Before. Masking at ingestion keeps raw values out of chunks, embeddings and the index. Masking only the final answer leaves copies in the store and in the prompts sent to the model.

Related terms

Vector database · Tokenization · Controlled unmasking · LLM data leakage · Data minimization · AI agent

Book a demo

See Protecto on your own data

Book a demo to see how Protecto finds and masks sensitive data before it reaches any AI model.

New: Protecto now runs inside LiteLLM. Control what sensitive data your models and tools see.
Read the launch blog