What Embeddings Are
Explain what an embedding is and why texts with similar meanings end up close together.
Turning meaning into numbers
Keyword search compares words. It cannot tell that "sick days" and "medical leave" mean nearly the same thing, because they share no letters. To search by meaning, a computer needs a way to represent what a piece of text is about, in a form it can compare quickly. Computers are very good at comparing numbers, so the idea is to turn text into numbers in a way that keeps its meaning.
An embedding is exactly that: a list of numbers that represents the meaning of a piece of text. A word, a sentence, a paragraph or a whole page can each be turned into one embedding. The list has a fixed length, often between 384 and 3,072 numbers, depending on the model that produced it. The model that does the conversion is called an embedding model.
The key property is simple: texts with similar meanings get similar lists of numbers. "Sick days" and "medical leave" end up with lists that are close to each other, while "library fine" ends up far away from both. Search by meaning then becomes a question of finding the lists of numbers closest to the query's list.
A map of meaning
The easiest way to picture an embedding is as a point on a map. Imagine every sentence placed on a large sheet of paper, with similar sentences placed close together. Leave policies would gather in one corner, hostel rules in another, exam rules in a third. A query is placed on the same map, and the nearest points are the best matches.
A point on a flat map needs two numbers, its x and y position. An embedding with 1,024 numbers is a point in a space with 1,024 directions instead of two. Nobody can picture 1,024 dimensions, but the maths of "near" and "far" works the same way. The extra directions give the model room to capture many separate aspects of meaning at once: topic, tone, whether something is a question or a rule, and much more.
The individual numbers do not have human-readable meanings. You cannot point at position 37 and say "this one measures how much the text is about hostels". Meaning is spread across all the numbers together. What matters is not any single number but how close two whole lists are.
An embedding model turns any piece of text into a fixed-length list of numbers. Similar meanings produce similar lists.
Exploring a tiny embedding space
Real embeddings have hundreds of dimensions, which makes them impossible to draw. The lab below uses a toy version with only two dimensions, where the positions have been placed by hand to behave the way real embeddings do. Pick a query and see which sentences sit closest to it. Notice that "sick days" finds the medical leave sentences without sharing a single word with them.
The last query, "late fee", is the interesting one. It sits between the hostel fee and the library fine, because both involve paying for being late. An embedding captures a blend of meanings, which is often useful and sometimes surprising.
Measuring "close"
To search with embeddings, you need a number that says how close two embeddings are. The most common choice is cosine similarity, which looks at the direction each list of numbers points in. Two embeddings pointing the same way score close to 1; unrelated ones score near 0. You will often see search results ranked by this score.
The code below uses tiny four-number embeddings, written by hand to keep the example readable, and ranks three campus sentences against a query. Run it and compare the scores.
Expected output:
0.98 Medical leave: up to 10 days
0.15 Library fine is Rs 5 per day
0.11 Hostel gates close at 10 PM
In cosine, a @ b multiplies the two lists position by position and adds up the results. Dividing by the two lengths (np.linalg.norm) removes the effect of length and leaves only direction. The sorted(...) line orders the sentences from the highest score to the lowest.
Why embeddings changed search
Embeddings fix the two failures of keyword search at once. Synonyms and paraphrases land close together, so "sick days" finds "medical leave". Different meanings of the same word can land far apart, because the model looks at the whole sentence: "my internet is not working" and "issued within 3 working days" get very different embeddings even though both contain "working".
They also work across languages and styles. A good multilingual embedding model places "छुट्टी के नियम" (Hindi for "leave rules") near "leave policy", and a casual question near a formal rule written in legal language.
Embeddings have limits too, and it is worth knowing them early. They capture general closeness of meaning, not exact facts, so "₹96,000" and "₹69,000" can look almost identical to an embedding model. They can be weak on rare codes, roll numbers and product IDs, where the exact characters matter more than the meaning. And the result depends heavily on the model: a model trained mostly on English web pages may place Indian legal terms or Telugu queries poorly. These limits are why many real systems combine embeddings with keyword search rather than replacing it.
Embeddings capture meaning, not exact characters, which is a strength for natural questions and a weakness for codes and numbers.