How much storage does a private AI setup need? A worksheet: models, search index, documents, logs and backups.
Short answer. Models and the search index together come to under 100 GB in the worksheet below, so your own documents and backups usually decide the total. A 27B model is a 16.5 GB file at 4-bit and 29.1 GB at 8-bit, Whisper large-v3 is 3.1 GB and an SDXL image model about 7 GB, which gives about 61 GB for the model set below. The document index adds about 4.1 GB per million passages at 1,024 dimensions. Fast NVMe storage shortens model loading. All figures are estimates; the worksheet below shows how to size your own.
01
Models
llama.cpp’s documentation notes that a model is loaded fully into memory, so memory and disk requirements are the same: a file you store is the size you load. Keep only the models and precisions you use, plus the previous one while you test a switch.
Item
Size
Basis
27B model at Q4_K_M (Qwen3.8-27B)
16.5 GB
Published file
The same model at Q8_0
29.1 GB
Published file
Vision projector for that model
0.9 GB
Published file
4B model at Q4_K_M
2.4 GB
Estimate: 4 × 4.8 ÷ 8
Whisper large-v3 speech model
3.1 GB
Published 16-bit file
Embedding model (BGE-M3)
2.3 GB
Published file
Image model (SDXL base, 16-bit)
7.1 GB
Published files
Total for models
61.3 GB
Sum of the rows above
02
The search index
Each passage is stored as a vector with one 32-bit number (4 bytes) per dimension, so size = passages × dimensions × 4 bytes; a search structure on top adds some more. As a rough conversion, I assume a 500-word page is about 670 tokens using OpenAI’s rule of thumb of 0.75 English words per token, and a passage is 512 tokens, so a page makes about 1.3 passages and a million passages is roughly 770,000 pages. Those conversions are my assumptions, not measurements.
Passages indexed
1,024 dimensions (BGE-M3)
768 dimensions (nomic-embed-text-v1.5)
100,000
0.41 GB
0.31 GB
1,000,000
4.1 GB
3.1 GB
10,000,000
41 GB
31 GB
03
Everything else
Your original documents: the largest item for most offices, and the one only you can measure. Count the files you intend to index.
Logs and the audit trail: text records that are small beside the models. Size them as calls per day × a few kilobytes × the days you keep them.
Transcripts, generated files and exports: they accumulate with use.
Backups: every copy multiplies the total, and a backup on the same drive is not a backup. See the question on backing up a private AI.
04
Speed of the drive
Loading time is file size ÷ read speed. With an assumed 3 GB/s NVMe drive, a 29 GB model loads in about 10 seconds; at an assumed 0.5 GB/s from an older drive it takes about 58 seconds. The speeds are examples, not benchmarks, so check your drive’s rating. A model that stays loaded is read once, but a server that swaps between several models reads files often, which favours fast storage.
ANVE’s published position (see the security page) is that everything sits in one database on the box. It does not describe the backup design or deletion details, so ask for them in writing before a paid pilot. ANVE has not published the storage fitted to its launch units, because no sealed unit exists yet.
05
Questions people ask
How much disk space does one model need?
The file size. A 27B model is about 16.5 GB at 4-bit and 29.1 GB at 8-bit; an 8B model at 4-bit is about 5 GB; Whisper large-v3 is about 3 GB.
Does the search index take much space?
Less than most people expect: about 4.1 GB per million passages at 1,024 dimensions by the formula above, plus the search structure. The documents themselves are usually larger.
Do I need an NVMe drive?
It is not required, but a fast drive cuts the time to load large models and helps if the server swaps between models. Size the capacity first and the speed second.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
