recallA retrieval
field guide

Extracting Tables, Forms and Figures

Build this skill

Keep tables, forms and figures usable for search by storing them as Markdown, label–value lines or descriptions.

The facts people ask about often live in tables

Look at where the answers to common student questions sit. Fee amounts are in a fee table. Exam dates are in a timetable grid. Leave limits are in a small table in Section 4. Placement statistics are in a chart. Forms have boxes with labels and filled-in values. These are exactly the parts of a document that simple text extraction handles worst.

Text is a line of words, read left to right. A table is two-dimensional: a value only makes sense together with its row heading and its column heading. The number "48,000" means nothing on its own. "Hostel fee, second instalment, January: ₹48,000" means everything. When a table is flattened into plain text, those connections are lost.

What happens to a table when it is flattened

Take a small fee table:

Fee July January
Tuition ₹75,000 ₹75,000
Hostel ₹48,000 ₹48,000
Exam ₹2,500 ₹2,500

Compare how three extraction methods represent it, and think about which version a search system and a language model could actually use.

Two good strategies come out of this. Keeping the table as Markdown preserves its shape and is compact. Linearising the table, turning each row or cell into a sentence that repeats its headers, makes every fact self-contained, which helps when tables are split into small chunks. Many systems store both: the Markdown table for the language model to read, and the sentences for search.

Extracting tables reliably

Getting from a PDF table to a clean Markdown table is a hard problem called table structure recognition. The extractor has to find the table on the page, work out its rows and columns (often without visible lines), handle cells that span several columns, and connect headers to values. Layout-aware tools such as Docling include dedicated table models for this, and dedicated libraries exist for born-digital PDFs.

Find thetablewhere on thepageFind rows andcolumnslines orspacingFill cellsincludingmerged onesAttachheadersrow and columnlabelsOutputMarkdown orsentences

A table only stays useful if every value keeps its row and column labels.

The program below takes a table that has already been extracted as rows and turns it into both formats.

python · runs in your browser

Expected output:

text
| Fee | July | January |
|---|---|---|
| Tuition | Rs 75,000 | Rs 75,000 |
| Hostel | Rs 48,000 | Rs 48,000 |
| Exam | Rs 2,500 | Rs 2,500 |

Tuition fee in July: Rs 75,000.
Tuition fee in January: Rs 75,000.
Hostel fee in July: Rs 48,000.
Hostel fee in January: Rs 48,000.
Exam fee in July: Rs 2,500.
Exam fee in January: Rs 2,500.

In the second format, zip(header[1:], r[1:]) pairs each month with its value, and r[0] repeats the row label in every sentence. Each sentence therefore carries its own row and column names, and still makes sense if it is retrieved without the rest of the table.

Forms: labels and values

Forms are a special kind of table: a set of key-value pairs, such as "Name: Meera Reddy", "Roll number: 22B81A0512", "Leave from: 3 March". When forms are scanned and OCR'd, the labels and values often come out in the wrong order or on different lines. Form extraction tools, including cloud services such as Azure AI Document Intelligence and Amazon Textract, are trained to pair each label with its value. For retrieval, the result is best stored as clean "label: value" lines, so that each fact stays connected to what it means.

Charts and images

A chart has no text to extract at all, apart from its title and axis labels. If a placement report shows "students placed per year" as a bar chart, a text-only pipeline sees only the title. There are three practical ways to deal with figures:

  • Keep the caption and nearby text, which often describes what the chart shows.
  • Describe the figure with a vision-language model, a model that can look at an image and write a description, such as "Bar chart: placements rose from 412 in 2023 to 538 in 2025".
  • Store the image itself and link it to the surrounding text, so the original can be shown with an answer.

Generated descriptions should be marked as generated, because a model can misread a chart just as OCR can misread text.

Decide how you would handle each element.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close