recallA retrieval
field guide

Contextual Retrieval

Build this skill

Add generated context to chunks before indexing, and weigh its benefit against indexing cost.

Chunks that don't know what they are about

Imagine the hostel handbook says, in Section 6 on Block C: "The block has 240 rooms. Its gates close at 10:00 PM. Visitors are allowed until 7:00 PM." After chunking, one chunk reads simply: "Its gates close at 10:00 PM. Visitors are allowed until 7:00 PM." Neither "Block C", nor "first-year", nor even "hostel" appears in it. A student who asks "when does the first-year hostel close?" may never see it, because neither keyword search nor embedding search can tell what "its" refers to.

Contextual retrieval fixes this by adding a short explanation of each chunk's context to the chunk itself before it is indexed. The approach was described by Anthropic in a September 2024 engineering article, which reported that adding context reduced the rate of failed retrievals (the right chunk missing from the top 20) by 49 percent, and by 67 percent when combined with reranking, on the datasets they tested.

How it works

For each chunk, a language model is given the whole document and the chunk, and asked to write one or two sentences that situate the chunk within the document. That sentence is put in front of the chunk, and the combined text is what gets embedded and indexed.

Original chunk Generated context added in front
"Its gates close at 10:00 PM. Visitors are allowed until 7:00 PM." "This chunk is from the KIT Hostel Handbook, Section 6, describing timings for Block C, the first-year boys' hostel."
"The limit is 10 days per semester." "This chunk is from the Academic Regulations 2026, Section 4, on medical leave for students."
Wholedocument+ one chunkLanguagemodelwrite 1–2 linesof contextContext +chunkcombined textEmbed andBM25index thecombined textSearchas usual

Context is added once, when indexing. Searching works exactly as before, but every chunk now carries its own explanation.

Contextual embeddings and contextual BM25

The context helps both kinds of search. Contextual embeddings are embeddings of the context-plus-chunk text, so their meaning now includes "first-year hostel" and "Block C". Contextual BM25 is a keyword index built over the same combined text, so a search for "Block C" or "first-year hostel" finds the chunk by keyword too. Combining both in a hybrid search, then reranking, gave the largest improvement in Anthropic's experiments.

The program below shows the keyword side of the effect: the overlap between a question and a chunk, with and without the added context.

python · runs in your browser

Expected output:

text
shared words without context: ['close']
shared words with context:    ['close', 'first', 'hostel', 'year']

words() turns a text into a set of its meaningful words, and & keeps the words two sets have in common. The second print joins the context sentence to the front of the chunk before comparing, which is all contextual retrieval does to the text.

Without context, the only shared word is "close". With context, the chunk now shares "first", "year", "hostel" and "close" with the question, so it ranks far higher.

What it costs

Contextual retrieval adds one language model call per chunk at indexing time, and each call includes the whole document. For a large collection that sounds expensive, but two things help. The calls happen once, ahead of time, not on every question. And prompt caching, offered by several model providers, stores the repeated part of a prompt (here, the document) so that many calls about chunks of the same document pay full price for it only once. The added context also makes each chunk slightly longer to store and embed.

When to use it

Contextual retrieval helps most when chunks rely heavily on their surroundings: long reports, manuals and handbooks full of "this section", "the above" and "its". It helps least for collections of short, self-contained items, such as FAQ entries, which already say what they are about.

Before reaching for it, try the free version: prepend each chunk's document title and heading path ("KIT Hostel Handbook › Section 6: Block C"). This often recovers much of the benefit at no cost. Then measure whether model-written context improves your Recall@K enough to justify the indexing cost.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close