RAG with LangChain
Assemble loaders, splitters, embeddings, vector stores, retrievers and models into a LangChain RAG chain.
Why use a framework?
You can build a RAG pipeline from scratch in a page of Python, and doing so once is the best way to understand it. Real projects, though, quickly need more: loaders for PDFs, Word files and web pages; several text splitters; connections to different embedding models, vector stores and language models; and a way to swap any of these without rewriting everything.
LangChain is an open-source framework that provides these pieces as reusable components with common interfaces. Because every vector store looks the same to the rest of your code, you can move from a local store to pgvector, or from one embedding model to another, by changing a line or two. LangChain does not hide what RAG does; it gives names and standard shapes to the steps you already know.
The building blocks
| Component | Job | Example classes |
|---|---|---|
| Document loader | Read files into Document objects (text + metadata) |
PyPDFLoader, WebBaseLoader |
| Text splitter | Split documents into chunks | RecursiveCharacterTextSplitter |
| Embeddings | Turn text into vectors | HuggingFaceEmbeddings, OpenAIEmbeddings |
| Vector store | Store and search embeddings | PGVector, Chroma, FAISS |
| Retriever | "Given a question, return relevant documents" | vector_store.as_retriever() |
| Prompt template | A prompt with slots for context and question | ChatPromptTemplate |
| Chat model | The language model | ChatOpenAI, ChatOllama |
| Output parser | Turn the model's reply into a string or structure | StrOutputParser |
Every chunk travels through these components as a Document: an object with page_content (the text) and metadata (a dictionary such as source, page and section). Keeping metadata on every Document is what later makes citations and filters possible.
Each step of the pipeline is a swappable component with a standard interface.
Indexing: load, split, embed, store
The code below indexes the institute's regulations into PostgreSQL with pgvector. It runs on your own computer once the packages are installed and a database is available; package names and exact class locations change between LangChain versions, so check the current documentation when you install.
# pip install langchain-community langchain-text-splitters langchain-huggingface langchain-postgres pypdf
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_postgres import PGVector
docs = PyPDFLoader("Academic_Regulations_2026.pdf").load() # one Document per page
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100).split_documents(docs)
embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-m3")
store = PGVector(embeddings=embeddings, collection_name="kit_regulations",
connection="postgresql+psycopg://user:pass@localhost/kit_kb")
store.add_documents(chunks)
Answering: retriever, prompt, model
LangChain connects components with the | operator, building a chain in which each component's output becomes the next one's input. This style is called the LangChain Expression Language (LCEL).
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI # or ChatOllama for a local model
retriever = store.as_retriever(search_kwargs={"k": 4})
prompt = ChatPromptTemplate.from_template(
"Answer using ONLY the sources below. Cite them as [1], [2]. "
"If they do not contain the answer, say so.\n\nSources:\n{context}\n\nQuestion: {question}"
)
def format_docs(docs):
return "\n".join(f"[{i}] ({d.metadata.get('source')}, p.{d.metadata.get('page')}) {d.page_content}"
for i, d in enumerate(docs, 1))
chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| ChatOpenAI(model="gpt-4o-mini", temperature=0)
| StrOutputParser()
)
print(chain.invoke("Can I take 12 days of medical leave?"))
Read the chain from top to bottom: the question goes to the retriever, whose documents are formatted into numbered sources; the question also passes through unchanged; both fill the prompt; the prompt goes to the model; the parser returns plain text.
What the chain is doing, without the framework
It helps to see that a chain is nothing magical: just functions called in order. The program below mimics the same chain with plain Python functions and the |-style composition written out step by step.
Expected output:
Answer using ONLY the sources.
Sources:
[1] (Regulations, p.11) Up to 10 days of medical leave per semester.
[2] (Regulations, p.11) More than 10 days needs the Dean of Academics.
Question: Can I take 12 days of medical leave?
ANSWER: You can take up to 10 days; more than 10 needs the Dean of Academics [1][2].
Each function's output is the next one's input. step1 is a dictionary holding the formatted sources and the question, prompt turns it into one piece of text, and model turns that text into an answer. LangChain's | operator wires the same steps together so that you do not have to name the values in between.
Swapping components
The real benefit of a framework shows when you change something. Moving from OpenAI to a local model served by Ollama changes one line (ChatOllama(model="llama3.1")). Moving from pgvector to Chroma for a laptop demo changes the vector store line. Changing the retriever to return 6 chunks with MMR changes search_kwargs. The rest of the code, and your tests, stay the same.
Match each need to the component you would change.
A word of caution: frameworks change quickly, and their abstractions can hide what is happening when something goes wrong. Keep the plain-Python picture in mind, log the actual prompt sent to the model, and you will always be able to see through the framework to the steps underneath.