recallA retrieval
field guide

Building a Naïve RAG Pipeline

Build this skill

Build a naïve RAG pipeline end to end and predict where it breaks first.

The simplest system that works

A naïve RAG pipeline is the simplest possible retrieval-augmented generation system: split documents into chunks, embed them, retrieve the closest chunks for a question, and ask a language model to answer from them. "Naïve" does not mean bad. This basic design answers a surprising share of real questions correctly, and every advanced technique is an improvement on one of its steps. Building it first gives you a baseline to measure everything else against.

The whole pipeline is four steps:

Chunksplit documentsEmbedchunks → vectorsRetrievetop-K for thequestionGenerateanswer from thechunks

Chunk and embed once ahead of time; retrieve and generate for every question.

The prompt that ties it together

The language model never sees your index. It only sees the prompt. So the generation step comes down to a prompt with three parts: instructions, the retrieved chunks and the question. A typical prompt looks like this:

text
You are the Kaveri Institute student helpdesk.
Answer the question using ONLY the sources below.
Cite the sources you use as [1], [2].
If the sources do not contain the answer, say "I could not find this in the
institute's documents" and suggest who to contact.

Sources:
[1] Academic Regulations 2026, Section 4: Students may take up to 10 days of
    medical leave in a semester. More than 10 days needs approval from the
    Dean of Academics.
[2] Academic Regulations 2026, Section 4: A certificate from a registered
    doctor must reach the department office within 7 days of rejoining.

Question: Can I take 12 days of medical leave?

The instruction to answer only from the sources, the numbered sources, and the instruction for what to do when the answer is missing are what turn a general chatbot into a grounded one.

A complete pipeline you can run

The program below is a full naïve RAG pipeline. It retrieves with simple word overlap instead of embeddings (embedding models cannot run in this page), and the "language model" is a stand-in function that prints the prompt it would send. Everything else, the chunking, retrieval, prompt assembly and citations, is exactly how a real pipeline is structured. With a real system, you would swap in an embedding model and a call to a language model API.

python · runs in your browser

Expected output:

text
Answer using ONLY the sources below and cite them as [1], [2]. If they do not contain the answer, say so.

Sources:
[1] Academic Regulations 2026, Section 4: Students may take up to 10 days of medical leave in a semester.
[2] Academic Regulations 2026, Section 4: More than 10 days needs approval from the Dean of Academics.
[3] Academic Regulations 2026, Section 4: A certificate from a registered doctor must reach the department office within 7 days of rejoining.

Question: How many days of medical leave can I take, and who approves extra days?
Step What the code does
Chunk re.split(r"(?<=\.)\s+", text) cuts each document after every full stop, and each sentence is stored with the name of its source.
"Embed" words() turns a text into a set of its meaningful words. A real system would call an embedding model here.
Retrieve len(q & words(c[1])) counts the words a chunk shares with the question. Chunks are sorted by that count and the top k with any overlap are kept.
Generate build_prompt numbers the retrieved chunks, puts the rules above them and the question below.

The retrieved chunks contain everything needed for a correct answer, and the prompt asks for citations. A language model reading this prompt would answer: "You can take up to 10 days [1]; more than 10 days needs approval from the Dean of Academics [2]."

The lab runs the same four steps over the 13 pages of the Kaveri Institute knowledge base, with real keyword ranking.

Where naïve RAG breaks first

Naïve RAG works well when the question is clear, the answer sits in one or two chunks, and those chunks use similar words or meaning to the question. It breaks in predictable ways:

Symptom Likely cause
Answer misses half the question The second fact was in a chunk ranked just below K
Correct rule, wrong year Old and new versions both retrieved, and nothing preferred the current one
"I could not find this" for a vague follow-up question like "and for hostel students?" The question makes no sense without the previous turn
Right passage at rank 9, not used Retrieval order is rough; nothing re-checks relevance
Long, unfocused answer Too many loosely related chunks in the prompt

Each row points to a technique that improves one step: better context building, query rewriting, reranking, conversation handling and more. That is why it is worth building the naïve version first, keeping its test results, and adding one improvement at a time.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close