recallA retrieval
field guide

Agentic RAG

Build this skill

Treat retrieval as a tool in an agent loop, add self-correction, and judge when an agent is worth its cost.

From a fixed pipeline to a system that decides

Every RAG pipeline so far has followed a fixed path: take the question, search once, write the answer. That works for most questions. It struggles when one search is not enough: when the first results are poor, when the question has several parts that depend on each other, or when the answer needs information from different kinds of sources.

Agentic RAG gives a language model control over retrieval. Instead of searching exactly once, the model decides whether to search, what to search for, which source to use, and when it has enough to answer. Retrieval becomes a tool the model can call, possibly several times, rather than a fixed step.

Retrieval as a tool

In an agent, a tool is a function the language model can ask the system to run, such as search_regulations(query), search_placement_notices(query) or get_my_attendance(course). The model sees a description of each tool. At each step it either calls a tool, reads the result, and thinks again, or decides it has enough and writes the final answer. This repeated cycle is often called an agent loop.

Thinkwhat do I still need?Call a toolsearch or queryRead resultsenough? relevant?Repeat until there isenough to answer

The model decides what to search for next, based on what it has found so far.

Following an agent through a hard question

Consider: "Am I eligible for campus placements if I took 12 days of medical leave and my attendance in one subject is 70 percent?" Answering needs the placement eligibility rules, the attendance rules (do low attendance and detention affect eligibility?) and the medical leave rules (is leave added back to attendance?). A single search will not find all three. Step through how an agent handles it.

Self-correction: checking retrieval before answering

Several research ideas make agents check their own work. Corrective RAG (CRAG, 2024) adds a step that grades the retrieved passages: if they look irrelevant, the system rewrites the query or tries another source instead of answering from bad evidence. Self-RAG (2023) trains a model to decide when to retrieve and to critique whether its own answer is supported by the passages. The common idea is simple and practical even without special models: after retrieving, ask "are these passages actually relevant and sufficient?", and if not, search again differently.

The costs of letting the model decide

Agentic RAG is powerful, but it is not free:

Benefit Cost
Handles multi-part and multi-source questions Several model calls and searches per question
Can recover from poor first results Slower answers (often several seconds)
Can combine documents with databases and APIs Harder to test: the path changes from question to question
Adapts to the question Can loop, over-search or call the wrong tool

Good agent designs add limits: a maximum number of steps, a time budget, logging of every tool call, and clear permissions on what each tool may access. Tools that only read information are much safer than tools that change anything.

The program below is a tiny agent loop with a scripted "model" that decides which tool to call next. It shows the control flow: decide, call, read, repeat, stop.

python · runs in your browser

Expected output:

text
step 1: search_placements('placement eligibility') -> No active backlogs and CGPA of at least 6.5.
step 2: search_regulations('attendance rules and medical leave') -> At least 75% attendance is needed; approved medical leave is added back.
step 3: search_regulations('medical leave beyond 10 days') -> More than 10 days of medical leave needs the Dean.
step 4: enough information, writing the answer from 3 findings
Part of the code What it does
tools The actions the agent may take, each an ordinary function that takes a query and returns text.
script Stands in for the model's decisions: which tool to call next, or answer when it has enough.
notes.append(result) Each tool result is kept, so later decisions and the final answer can use it.
script[:MAX_STEPS] A hard limit on the number of steps, so the loop cannot run for ever.

When to use agentic RAG

Start with a fixed pipeline. Move to an agentic design when your test questions show failures that a fixed pipeline cannot fix: questions that need several dependent searches, several different sources, or a decision about whether to search at all. Many systems end up with a hybrid: a fast fixed pipeline for simple questions, and an agent for the hard ones, chosen by a router.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close