recallA retrieval
field guide

Tuning Retrieval Parameters

Build this skill

Tune top-K, similarity thresholds, MMR and index search settings on your own test queries.

Small settings, big effects

The same embedding model and the same index can give noticeably different results depending on a handful of retrieval parameters: how many results to return, whether to drop weak matches, whether to favour variety, and how hard the index should look. They cost nothing to change, which makes tuning them one of the quickest ways to improve a retrieval system.

Top-K: how many results to return

Top-K is the number of results the search returns. Choosing it is like deciding how many pages of notes to carry into an open-book exam. Too few, and the page you need may be the one left behind. Too many, and you waste time on pages that have nothing to do with the question. For a language model, every extra passage costs tokens and is one more thing that can distract it.

To set K, look at your test questions and find the K at which Recall@K (the share of questions whose answer passage is in the top K) stops improving. If the right passage is almost always in the top 4, returning 20 adds cost and no benefit.

Similarity threshold: drop weak matches

A vector search always returns K results, even when nothing in the collection is relevant. Ask a campus helpdesk about cricket scores and it will still hand over its five "closest" passages, and a model given those is more likely to invent an answer than one given nothing.

A similarity threshold removes results whose score falls below a cut-off, even if that leaves fewer than K. Scores depend on the embedding model and the data, so a threshold of 0.75 may suit one model and remove everything for another. Set it from the scores of known good and known bad matches in your own test set.

Explore K and the threshold together on one query.

Fetch-K and MMR: variety in the results

Sometimes the top results are near-copies of each other: three overlapping chunks of one paragraph, or the same rule from three editions of a handbook. Each copy takes a place that a different, useful passage could have filled. It is like a thali with three bowls of the same dal and no sabzi.

Maximal Marginal Relevance (MMR) fixes this. It first fetches a larger candidate list, whose size is called fetch-K (for example 20), and then builds the final K one pick at a time. Each candidate is scored on two things: how relevant it is to the query, and how similar it is to what has already been picked.

\[\text{MMR score} = \lambda \times \text{relevance} - (1 - \lambda) \times \text{similarity to the closest pick so far}\]

The setting lambda (λ) balances the two. With λ = 1 the second part vanishes and MMR is plain top-K. Lower values punish repetition more heavily.

Worked example: the second pick with λ = 0.5

The first pick is the overlapping leave-limit chunk. Two of the candidates competing for second place:

Chunk 4.1, another copy of the leave limit, has relevance 0.995 and similarity 1.00 to the first pick. Its score is 0.5 × 0.995 − 0.5 × 1.00 = −0.003.

The attendance rule has relevance 0.82 and similarity 0.78 to the first pick. Its score is 0.5 × 0.82 − 0.5 × 0.78 = 0.02.

The attendance rule wins, although it is less relevant, because it adds something the first pick did not already say.

Plain top-3Medical leave limit (chunk 4.1)Medical leave limit (overlap of 4.1)Medical leave limit (2025 copy)MMR top-3Medical leave limit (chunk 4.1)Doctor's certificate ruleHow leave counts for attendance

Plain top-K can fill every slot with near-duplicates. MMR trades a little relevance for variety, so more of the answer fits in the prompt.

Move λ in the lab and watch the three picks change.

MMR in code

The program below implements MMR and compares it with plain top-K on five candidates. Three of them are near-copies of the leave-limit chunk.

python · runs in your browser

Expected output:

text
plain top-3: ['Leave limit (overlapping chunk)', 'Leave limit (2025 copy)', 'Leave limit (chunk 4.1)']
MMR top-3:   ['Leave limit (overlapping chunk)', 'Leave counts for attendance', "Doctor's certificate rule"]
Part of the code What it does
relevance How similar this candidate is to the query.
repeats How similar it is to the closest passage already picked (0 for the first pick).
lam * relevance - (1 - lam) * repeats The MMR score from the formula.
picked.append(max(remaining, key=score)) Adds the best-scoring candidate not yet picked; the loop then scores everything again.

Plain top-K returns the same fact three times. MMR keeps one copy and uses the other two places for the certificate rule and the attendance rule. Call mmr(3, lam=1.0) to confirm that λ = 1 gives back the plain top-K.

Index search parameters

The index itself has settings that trade recall for speed on each query.

Index Parameter Higher value means
HNSW ef_search More candidates explored: better recall, slower queries
IVF nprobe / probes More clusters searched: better recall, slower queries

A good habit is to raise these until recall against exact search stops improving on your test queries, and then stop. Past that point you only pay in speed.

Different questions, different settings

No single setting suits every question. One with a single clear answer ("what is the hostel fee?") needs a small K and gains nothing from variety. A broad one ("what are all the rules about leave?") needs a larger K and benefits from MMR. Most systems start with one moderate setting, measured on a test set, and adjust from there.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close