Choosing an Embedding Model
Choose an embedding model for a scenario by weighing data rules, languages, a small test, cost and storage.
There is no single best model
Picking an embedding model feels like it should have one right answer: find the top of the leaderboard and use that. In practice, the best model depends on your documents, your users, your budget and your rules about data. A model that is excellent for English legal contracts may be poor for Telugu student questions, and a model that is perfect for a pilot with 2,000 pages may be too slow or too expensive at two million.
A sensible choice weighs five things: quality on your own data, languages, cost, speed and storage, and where your data is allowed to go. This topic walks through each one, then turns them into a simple selection process.
Quality: benchmarks first, your own test second
The MTEB leaderboard is a good way to build a shortlist. Look at the retrieval scores rather than the overall average, and check the scores for the languages you need. Treat small gaps as ties: a model one point higher on a benchmark is not reliably better on your data.
Then test the shortlist yourself. Write 30 to 50 real questions, mark the passages that answer each one, embed your documents with each candidate, and measure how often the right passage appears in the top 5 (Recall@5). This takes an afternoon and is far more reliable than any public leaderboard, because your documents have their own vocabulary, such as "bonafide", "backlog" or "re-evaluation", that general benchmarks never test.
Dimensions and Matryoshka embeddings
More dimensions can hold more detail, but every extra dimension costs storage, memory and search time. A collection of one million passages needs about 1.5 GB for 384-dimension embeddings and about 12 GB for 3,072-dimension embeddings.
Some newer models offer a way to have both. Matryoshka embeddings are named after Russian nesting dolls, where each doll contains a smaller one. The model is trained so that the first part of its embedding is already a useful embedding on its own: the first 256 numbers form a decent small embedding, the first 512 a better one, and so on up to the full length. You can store a shortened version and give up only a little quality. OpenAI's text-embedding-3 models and several Jina and Nomic models support this.
A Matryoshka model packs the most important information into the first numbers, so a shortened embedding still works well.
The program below imitates this. It builds 500 embeddings in which the early numbers carry the most information, as Matryoshka training arranges, and then checks how many of the true top 10 results survive when only the first part of each embedding is kept.
Expected output:
first 512 numbers: 10 of the top 10 unchanged, 2x less storage
first 256 numbers: 9 of the top 10 unchanged, 4x less storage
first 64 numbers: 9 of the top 10 unchanged, 16x less storage
first 16 numbers: 7 of the top 10 unchanged, 64x less storage
passages[:, :dims] is the whole trick: it slices off the first dims columns and throws the rest away. The shortened vectors are normalised again, because cutting a vector changes its length. A quarter of the numbers keeps nine of the ten results here, and quality falls away only when the cut becomes severe. Real models behave the same way in outline; the exact figures depend on the model.
Cost, speed and where the data goes
For API models, you pay per token embedded. Embedding a collection once is usually cheap, but you also pay again for every query, and again for the whole collection whenever you change models or re-split documents. For open models, there are no per-token fees, but you need hardware: a GPU makes bulk embedding far faster, while small models can run on an ordinary CPU.
Speed matters at question time. Every search waits for the query to be embedded, so a model that takes 300 milliseconds per query adds that delay to every answer.
Data rules can decide the matter on their own. If a college's student records or a hospital's patient notes may not leave its own servers, API models are ruled out regardless of quality, and the choice is among open models that can run in-house. In India, the Digital Personal Data Protection Act makes organisations think carefully about where personal data is sent and processed.
Use the calculator below to see how collection size, embedding length and price combine. The price is an illustrative figure, not a quote from any provider.
A simple way to choose
Put the factors in order and most decisions become straightforward.
Data rules and languages narrow the field. A small test on your own questions makes the final call.
Now practise. Each scenario below has one sensible starting choice.
Changing your mind later is expensive
One last point makes this decision more important than it looks. Embeddings from different models cannot be mixed, so switching models means re-embedding every passage in the collection and rebuilding the index. For a few thousand pages that takes minutes; for tens of millions it can take days and real money. Record which model and version produced each embedding, so you always know what needs re-embedding when something changes.