Why LLMs Need External Knowledge
Explain why language models hallucinate on private and changing facts, and how grounding with retrieved evidence reduces the risk.
Sounding right is not the same as being right
Ask a general chatbot "Can I take 12 days of medical leave this semester at Kaveri Institute of Technology?" and you will probably get a friendly, confident reply. It might say most colleges allow about 15 days, remind you to get a medical certificate and wish you a quick recovery. It sounds like an answer, but it is a guess. The model has never seen the institute's rules, and the real limit is 10 days.
To see why this happens, it helps to know roughly how a large language model (LLM) works. An LLM writes text one small piece at a time, each time predicting a likely next word based on everything before it. It learned what is "likely" by training on a huge amount of text. Along the way it picked up a lot of general knowledge: what a semester is, what medical leave usually involves, how Indian colleges tend to word their rules. That knowledge is stored inside the model's parameters, the billions of numbers adjusted during training, so it is called parametric knowledge.
Parametric knowledge is broad, but it has limits, and the model does not hesitate at those limits. A guess about something it has never seen looks exactly as confident as a fact it knows well.
Three things a model cannot know
| Gap | What it means | Example at the Kaveri Institute |
|---|---|---|
| Knowledge cutoff | Training stops at some date; anything newer was never seen | The new fee structure announced last month |
| Private data | Documents that were never public were never in the training data | The academic regulations PDF on the student portal |
| Changing data | Facts that change faster than a model can be retrained | Hostel closing dates, exam timetables, the current warden's name |
The knowledge cutoff is the date the training data ends. Models take months to train and are then used for months or years, so they are out of date even on release day. Private data, such as a college's internal rules or a company's customer records, was never available for training, and should not be. Changing data is the trickiest: even if the model saw last year's circular, it has no way of knowing that the circular has been replaced. Retraining a model every time a document changes is far too slow and expensive.
Why models make things up
When a model produces fluent text that is not true, or not supported by its sources, it is called a hallucination. The word suggests a malfunction, but it is really the model doing what it was trained to do: produce likely-sounding text. Most of the text it learned from contains confident, specific answers. Sentences like "I don't know the medical leave limit at this particular college" are rare by comparison. So when the model lacks a fact, the most likely-sounding reply is often a confident answer built from patterns: a typical number, a typical rule, a reference that looks real.
Hallucinations come in a few forms: made-up facts (the 15-day limit), two real facts blended into a false one, and invented sources such as book titles, court cases or web links that look right but do not exist. Newer models hallucinate less often, but the root cause stays, because a model cannot contain information that was never in its training data.
Grounding: give the model the evidence
Think about two kinds of exam. In a closed-book exam, students answer from memory, and when memory fails some write a confident guess. In an open-book exam, they find the right page and answer from it. Open-book answers about specific facts are far more reliable.
Grounding turns the model's job into an open-book exam. Before the model answers, the system finds passages related to the question and puts them into the prompt, along with an instruction to answer from those passages. The model's task changes from "remember this fact" to "read these passages and answer from them", which language models are very good at. There is a second benefit: because the answer comes from specific passages, the system can show them, so the reader can check the answer instead of just trusting it.
Try the same question in four set-ups below. Before clicking, predict which ones will give a correct answer and which a confident wrong one. Pay attention to the third.
In code, grounding is nothing more than building a longer prompt. The program below puts two passages and an instruction in front of a question.
Expected output:
Answer the question using only the passages below. If the answer is not in them, say you do not know.
[1] Students may take up to 10 days of medical leave in a semester.
[2] Medical leave of more than 10 days needs approval from the Dean of Academics.
Question: How many days of medical leave can I take?
enumerate(passages, start=1) numbers the passages, so the model can point to [1] or [2] in its answer. Everything in prompt is sent to the model as one piece of text. The model itself has not changed; only what it is given to read has.
Where the name RAG comes from
The name retrieval-augmented generation (RAG) comes from a 2020 research paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University. "Retrieval" means finding documents, "augmented" means added to, and "generation" means writing the answer. Their system had two parts. A retriever searched about 21 million short passages from Wikipedia and picked the ones closest in meaning to the question. A generator, a language model, then wrote an answer using both the question and those passages.
The paper described two kinds of memory. The model's parameters were its parametric memory: knowledge baked in during training. The passage collection was its non-parametric memory: knowledge kept outside the model, where it can be read, checked and replaced.
RAG keeps general language skills in the model and puts specific facts in documents that can be read, checked and replaced.
One experiment in the paper showed the practical value. The authors swapped the 2016 Wikipedia collection for a 2018 one and asked about world leaders who had changed in between. The answers updated to match the new documents, with no retraining at all. Updating what the system knows became as simple as updating the documents it searches.
Most systems today are simpler than the original: they connect an existing language model to an existing search system through the prompt. The principle is the same: search first, then write.
Search first, then write: the model only sees the question once the evidence is next to it.
Grounded does not mean always correct
Grounding cuts down hallucinations but does not remove them. Three problems remain. The search can miss the right passage or return an old one, and the model will faithfully answer from the wrong evidence, as the old circular showed. The model can misread or ignore the passages, especially long or conflicting ones. And the model can add details that are not in the passages, mixing what it was given with what it "remembers". A claim that no passage supports is called unsupported or ungrounded, and catching such claims is one of the main reasons RAG systems show their sources.
Practise catching them. The answer below was written from one retrieved passage. Click every sentence the passage does not support, then press Check.