How much memory do embedding models need? Model sizes, what the working memory depends on, and why the index is the bigger bill.
Short answer. Very little. nomic-embed-text-v1.5 has 137 million parameters and a 0.55 GB file at 32-bit; BGE-M3 is a 2.27 GB file, which is about 570 million parameters by my arithmetic. At 16-bit each takes half that, so both together fit in under 1.5 GB beside a chat model. An embedding model reads a passage once and returns a fixed-size vector, so it has no growing generation cache; its memory follows the batch and the passage length. The larger cost is the vector index: about 4.1 GB per million passages at 1,024 dimensions.
01
Sizes of two common models
Both are on Hugging Face. The nomic card describes Matryoshka training, which lets you cut the vector to a shorter length at a small cost in quality, with 512 and 256 shown in its examples. BGE-M3 is built from XLM-RoBERTa, supports dense, sparse and multi-vector retrieval, and its card says it supports more than 100 working languages. Check how well it handles Hindi and Hinglish on your own documents.
Model
Parameters
Published file
At 16-bit (estimate)
Vector size
Longest input
nomic-embed-text-v1.5
137 million
0.55 GB (32-bit)
0.27 GB
768, reducible
8,192 tokens
BGE-M3
About 570 million (my arithmetic: 2.27 GB ÷ 4 bytes)
2.27 GB
1.14 GB
1,024
8,192 tokens
02
What the working memory depends on
A chat model keeps a cache that grows with the conversation. An embedding model is different: it reads a passage once, turns it into one vector and moves on. Its working memory grows with the batch size and the length of the passages, not with a conversation. BGE-M3’s card shows a maximum length of 8,192 tokens and advises setting a smaller value if you do not need that length, which also speeds up encoding. Passages of a few hundred tokens in modest batches keep the working memory small.
This page measures neither memory nor speed on a particular machine. The models are small enough that memory is rarely the limit.
03
The index is the larger cost
Size = passages × dimensions × 4 bytes, before any search structure. The storage question works through a full sizing worksheet.
Vector size
Per million passages (32-bit numbers)
Typical models
256 dimensions
1.0 GB
nomic-embed-text-v1.5, shortened
768 dimensions
3.1 GB
nomic-embed-text-v1.5, full length
1,024 dimensions
4.1 GB
BGE-M3
04
In an office stack
The embedding model runs when documents are added, which is a batch job you can schedule, and once for each question to embed the question. It can stay loaded all day for about a gigabyte. Changing the embedding model later means re-embedding every document, because the vectors from two models are not comparable and may differ in size. ANVE answers questions over the office’s own documents and shows the source passage; this page does not say which embedding model ANVE uses.
05
Questions people ask
Can an embedding model run without a GPU?
These models are small enough that memory is not the issue on a CPU. How fast a CPU encodes depends on the chip, which this page does not measure; indexing a large set is a job to schedule.
Do I need a separate embedding model from the chat model?
Usually yes. Chat models write text; embedding models turn text into vectors for search. They are separate, small models that run alongside the chat model.
Can I swap the embedding model later?
Yes, but you must re-embed every document, because vectors from different models are not comparable and may have different sizes (768 for nomic-embed-text-v1.5, 1,024 for BGE-M3).
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
