recallA retrieval
field guide

Vector Database Operations

Build this skill

Keep a vector store correct with upserts, metadata updates, replace-on-edit and deletes.

A vector store is a living database

It is easy to picture a vector index as something you build once and then search. In a real knowledge base, documents arrive, change and disappear every day. The store behind a retrieval system has to carry out four operations correctly, all the time and while people are searching: insert, search, update and delete. In database language these are often shortened to CRUD (create, read, update, delete).

When a search returns a withdrawn circular, or both the old and the new version of a rule, the language model has done nothing wrong. The data it was given was wrong.

Insertnew chunksSearchtop KUpdatemetadata or contentDeletewithdrawn docsOne knowledge base,changing every day

Insert, search, update and delete all run on the same store, every day, while people are searching.

Insert and upsert

Inserting adds new chunks with their embeddings and metadata. Chunks are normally inserted in batches of hundreds or thousands, which is much faster than sending them one at a time.

Ingestion pipelines get re-run: a job fails halfway and is restarted, or someone processes the same folder twice. With plain inserts, each re-run adds every chunk again and the store fills with duplicates.

Upsert (update or insert) means "insert this record, or replace it if a record with the same ID already exists". It only works if each chunk has a stable ID, one that comes out the same every time the same chunk is produced. Compare a roll number with a token from a queue machine. Your roll number is the same on every visit to the exam office, so the office updates your existing record. A token is new each time, so the office would open a fresh file for you on every visit. A chunk ID built from the document ID and the chunk's position, such as leave-rules#0, behaves like a roll number. A random ID behaves like a token.

Update

Updates come in two kinds, and they need different handling.

Metadata update Content update
What changed Facts about the document: owner, access level, status The document's text
Example A handbook is opened to students as well as staff The leave limit is revised from 15 days to 10
New embeddings needed? No Yes
Safe way to do it Change the metadata fields in place Delete all of the document's old chunks, then insert the new ones

When the text changes, the chunker may cut it differently: a document that produced three chunks may now produce two. Upserting the two new chunks replaces #0 and #1, but the old #2 stays in the store, holding text that no longer exists. Deleting everything that belongs to the document first, and then inserting, leaves nothing behind.

Try the events in the lab, first with stable IDs and then with random ones.

Delete

Deleting removes chunks when a document is withdrawn, or when the right to keep someone's data ends. Deleting by document ID, which removes every chunk of that document, is the most common case.

In some indexes a delete is not immediate. An HNSW graph often only marks a deleted vector as removed. Searches skip it, but it still occupies memory, and many marked vectors slowly lower search quality. Databases clear them with periodic clean-up, called VACUUM in PostgreSQL and compaction or re-indexing elsewhere.

The four operations in code

The program below is a tiny vector store with all four operations. Its rows live in a Python dictionary keyed by chunk ID, so writing to an existing ID replaces the row. That one property is what makes upsert work.

python · runs in your browser

Expected output:

text
rows: 2
student search: [(0.99, 'Up to 10 days of medical leave.')]
after access change: [(0.99, 'Up to 10 days of medical leave.'), (0.97, 'Staff get 20 days of medical leave.')]
after delete: [(0.99, 'Up to 10 days of medical leave.')]
Method What it does
upsert Stores the row under its chunk ID, overwriting any row already there.
update_meta Merges the changed fields into the metadata of every row of the document. Text and vector are untouched, so nothing is re-embedded.
delete_document Collects the IDs of the document's rows, then removes them.
search Keeps the rows whose metadata matches every filter and returns the best k by similarity.

The second upsert call used the same ID as the first, so the store holds two rows, not three. Change the second call's ID to "reg#1" and run it again: the store now holds the same sentence twice, and a search returns it twice.

Operations as an API

In a working application, these operations are usually offered by one small web service with endpoints such as POST /documents (ingest or replace a document), PATCH /documents/{id} (change metadata), DELETE /documents/{id} and POST /search. The ingestion pipeline calls the first three and the RAG application calls the last. Keeping one service in charge of every write stops two parts of the system from disagreeing about what the store contains.

Follow your curiosity.

↑ ↓ to explore · Enter to open · Escape to close