Parent-Child Chunking
Search small child chunks and return their parent sections, sizing both levels sensibly.
Search small, read big
Chunk size forces an uncomfortable choice. Small chunks match questions precisely, because each one is about one specific thing. Large chunks give the language model enough context to answer well, because the surrounding rules, conditions and exceptions come along. You seem to have to pick one.
Parent-child chunking (also called small-to-big retrieval or parent document retrieval) avoids the choice by using two sizes at once. Documents are split into large parent chunks, such as a whole section, and each parent is split again into small child chunks, such as a sentence or two. The system searches over the small children, which match precisely, but returns the parent each matching child belongs to, which gives the model the full context.
Search finds the small child that matches best. The model receives its whole parent section.
Following a question through
A student asks: "Who approves medical leave beyond the limit?"
The search compares the question with the small child chunks. Child 4.2, "More than 10 days needs approval from the Dean of Academics", matches very closely, because it is short and about exactly this. Instead of sending only that one sentence to the model, the system looks up its parent, Section 4, and sends the whole section. The model now sees the 10-day limit, the Dean's role, the certificate deadline and the attendance rule, and can give a complete, accurate answer.
Compare what the model receives under three strategies below.
How it is stored
Parent-child retrieval needs two stores that are linked together:
- A vector index of children: each child chunk is embedded and stored with its parent's ID.
- A store of parents: the full parent text, looked up by ID. This does not need embeddings at all; a simple database table or key-value store is enough.
At search time, the system finds the top children, collects their parent IDs, removes duplicates (several children of the same parent often match together), and fetches those parents.
Expected output:
matched children: ['More than 10 days needs the Dean of Academics.', 'Up to 10 days per semester with department approval.', "Submit a doctor's certificate within 7 days."]
parents returned: ['S4']
sent to model: Section 4: Medical Leave. Up to 10 days per semester with department approval. More than 10 days needs the Dean of Academics. Submit a doctor's certificate within 7 days of rejoining.
sorted(children, key=lambda c: -c[2])[:3] keeps the three children with the highest similarity. The loop then walks through them and records each parent ID once, skipping repeats, and the last line looks the parent's full text up in the parents dictionary.
Three children matched, but they all belong to Section 4, so only one parent is sent, with no repetition.
Choosing sizes and avoiding pitfalls
Parents are usually sized to be a natural unit a person would read, such as a section or a page, often a few hundred to around two thousand tokens. Children are small, often one to three sentences or around 100 to 200 tokens.
Two things to watch. First, parents that are too big bring back the very problem you were avoiding: a prompt full of text the question did not need, and fewer distinct sources per prompt. If parents are huge, use the heading structure to make them smaller. Second, when several children from different parents match, the total size can exceed the prompt budget quickly, so cap the number of parents and keep the best ones.
A related, lighter-weight variant is sentence-window retrieval: search single sentences, but return each matched sentence together with a few sentences before and after it. It gives some of the benefit with no separate parent store.