Building a Naïve RAG Pipeline
Build a naïve RAG pipeline end to end and predict where it breaks first.
The simplest system that works
A naïve RAG pipeline is the simplest possible retrieval-augmented generation system: split documents into chunks, embed them, retrieve the closest chunks for a question, and ask a language model to answer from them. "Naïve" does not mean bad. This basic design answers a surprising share of real questions correctly, and every advanced technique is an improvement on one of its steps. Building it first gives you a baseline to measure everything else against.
The whole pipeline is four steps:
Chunk and embed once ahead of time; retrieve and generate for every question.
The prompt that ties it together
The language model never sees your index. It only sees the prompt. So the generation step comes down to a prompt with three parts: instructions, the retrieved chunks and the question. A typical prompt looks like this:
You are the Kaveri Institute student helpdesk.
Answer the question using ONLY the sources below.
Cite the sources you use as [1], [2].
If the sources do not contain the answer, say "I could not find this in the
institute's documents" and suggest who to contact.
Sources:
[1] Academic Regulations 2026, Section 4: Students may take up to 10 days of
medical leave in a semester. More than 10 days needs approval from the
Dean of Academics.
[2] Academic Regulations 2026, Section 4: A certificate from a registered
doctor must reach the department office within 7 days of rejoining.
Question: Can I take 12 days of medical leave?
The instruction to answer only from the sources, the numbered sources, and the instruction for what to do when the answer is missing are what turn a general chatbot into a grounded one.
A complete pipeline you can run
The program below is a full naïve RAG pipeline. It retrieves with simple word overlap instead of embeddings (embedding models cannot run in this page), and the "language model" is a stand-in function that prints the prompt it would send. Everything else, the chunking, retrieval, prompt assembly and citations, is exactly how a real pipeline is structured. With a real system, you would swap in an embedding model and a call to a language model API.
Expected output:
Answer using ONLY the sources below and cite them as [1], [2]. If they do not contain the answer, say so.
Sources:
[1] Academic Regulations 2026, Section 4: Students may take up to 10 days of medical leave in a semester.
[2] Academic Regulations 2026, Section 4: More than 10 days needs approval from the Dean of Academics.
[3] Academic Regulations 2026, Section 4: A certificate from a registered doctor must reach the department office within 7 days of rejoining.
Question: How many days of medical leave can I take, and who approves extra days?
| Step | What the code does |
|---|---|
| Chunk | re.split(r"(?<=\.)\s+", text) cuts each document after every full stop, and each sentence is stored with the name of its source. |
| "Embed" | words() turns a text into a set of its meaningful words. A real system would call an embedding model here. |
| Retrieve | len(q & words(c[1])) counts the words a chunk shares with the question. Chunks are sorted by that count and the top k with any overlap are kept. |
| Generate | build_prompt numbers the retrieved chunks, puts the rules above them and the question below. |
The retrieved chunks contain everything needed for a correct answer, and the prompt asks for citations. A language model reading this prompt would answer: "You can take up to 10 days [1]; more than 10 days needs approval from the Dean of Academics [2]."
The lab runs the same four steps over the 13 pages of the Kaveri Institute knowledge base, with real keyword ranking.
Where naïve RAG breaks first
Naïve RAG works well when the question is clear, the answer sits in one or two chunks, and those chunks use similar words or meaning to the question. It breaks in predictable ways:
| Symptom | Likely cause |
|---|---|
| Answer misses half the question | The second fact was in a chunk ranked just below K |
| Correct rule, wrong year | Old and new versions both retrieved, and nothing preferred the current one |
| "I could not find this" for a vague follow-up question like "and for hostel students?" | The question makes no sense without the previous turn |
| Right passage at rank 9, not used | Retrieval order is rough; nothing re-checks relevance |
| Long, unfocused answer | Too many loosely related chunks in the prompt |
Each row points to a technique that improves one step: better context building, query rewriting, reranking, conversation handling and more. That is why it is worth building the naïve version first, keeping its test results, and adding one improvement at a time.