recallA retrieval
field guide

Recursive and Structure-aware Chunking

Build this skill

Split documents recursively and by headings so that chunks follow the author's structure.

Cut where the writer paused

Cutting a document every 300 tokens is simple, but it ignores everything the author did to organise the text. Writers already divide documents into sections, paragraphs and sentences, and those divisions mark where one idea ends and the next begins. A chunk that follows them is much more likely to contain one complete idea.

Structure-aware chunking uses the document's own structure to decide where to cut. Recursive chunking is the most common way to do it when the structure is only partly visible: try to split at the biggest natural break first, and only fall back to smaller breaks when a piece is still too long.

How recursive chunking works

A recursive splitter is given a list of separators, from the strongest break to the weakest. For plain text a typical list is:

  1. A blank line (between paragraphs)
  2. A new line
  3. A sentence ending, such as ". "
  4. A space (between words)
  5. As a last resort, between characters

It first splits the text at blank lines. Any piece that fits within the size limit becomes a chunk. Any piece that is still too long is split again at the next separator down, and so on. "Recursive" just means the same process is repeated on the pieces that are too big.

Whole section (900tokens)blank lineParagraph 1 (250)fits → chunkblank lineParagraph 2 (520)too big → split againsentenceSentences 1–4 (280)fits → chunksentenceSentences 5–8 (240)fits → chunkblank lineParagraph 3 (130)fits → chunk

Split at the biggest break first. Only pieces that are still too long get split at a smaller break.

The result is that most chunks end at the end of a paragraph or sentence, rather than in the middle of a word or a rule.

The program below is a small recursive splitter written from scratch. Run it and look at where the cuts fall.

python · runs in your browser

Expected output:

text
[1] (4 words) Section 4: Medical Leave
[2] (24 words) Students may take up to 10 days of medical leave in a semester. More than 10 days needs approval from the Dean of Academics.
[3] (16 words) A certificate from a registered doctor must reach the department office within 7 days of rejoining.
Part of the code What it does
separators=("\n\n", "\n", ". ", " ") The cut points to try, coarsest first: paragraph breaks, line breaks, sentence ends, then spaces.
candidate = (current + sep + piece) ... Tries adding the next piece to the chunk being built. If it still fits, the loop carries on.
chunks.append(current.strip()) The next piece would not fit, so the chunk built so far is closed.
recursive_split(piece, max_words, rest) A piece that is too big by itself is split again with the next, finer separator. This is the recursive step.

Each chunk is a whole sentence or paragraph. None is cut mid-sentence. Notice, though, that the heading became a tiny chunk of its own, separated from the rules it introduces. That is a common weakness of plain recursive splitting, and the next idea fixes it.

Using real structure: headings, Markdown and code

When documents have been parsed into Markdown, the structure is visible: lines starting with # are headings. A heading-aware chunker splits at headings first, so each chunk covers one section, and it copies the heading path into every chunk, such as "Academic Regulations › Section 4: Medical Leave". Even a small chunk from deep inside the section then says what it is about.

Different content types have their own natural boundaries:

Content Natural split points
Markdown or HTML documents Headings, then paragraphs
FAQs One question and its answer per chunk
Legal documents and policies Numbered clauses and sub-clauses
Code Functions and classes
Tables Keep each table whole, or one row per chunk with its headers

Compare three splitting methods on the same text in the lab below. Notice where each one cuts, and whether any chunk holds the whole answer.

Doing it with a library

Frameworks provide ready-made splitters, so you rarely write your own. LangChain's text splitters package includes a recursive splitter and a Markdown header splitter. This example runs on your own computer after installing the package:

python
# pip install langchain-text-splitters
from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter

# 1. Split at headings, keeping the heading names as metadata
by_heading = MarkdownHeaderTextSplitter(headers_to_split_on=[("#", "section"), ("##", "subsection")])
sections = by_heading.split_text(markdown_text)

# 2. Split any long section further, at paragraphs, then sentences, then words
splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)
chunks = splitter.split_documents(sections)

The two-step pattern, headings first and then recursive splitting within long sections, is a strong default for most document collections.

When structure-aware chunking helps most

Structure-aware chunking helps most on long, well-organised documents such as regulations, manuals and technical documentation, where headings carry meaning. It helps least on text with no structure, such as a transcript of a meeting or a scanned letter with no headings, where you are back to sentences and sizes. Either way, always test on your own questions: the method that looks neatest is not always the one that retrieves best.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close