Parameters (model size) What the “27B” in a model name means for memory and quality.
Short answer. Parameters are the numbers a model learns during training, also called weights; a model named 27B has about 27 billion of them. Parameter count mostly decides how much memory the model needs: multiply it by the bytes stored for each weight. At 16-bit precision that is 2 bytes, so a 27B model needs about 54 GB for its weights, about 27 GB at 8-bit and about 17 GB in the common 4-bit format. More parameters usually mean a more capable model, but training data, design and tuning matter too, and a mixture-of-experts model uses only part of its parameters for each token.
01
What a parameter is
Inside a model, every connection between layers carries a number, a weight. Training adjusts these numbers until the model predicts text well. A count such as 8B, 27B or 70B means 8, 27 or 70 billion of them. They are stored in a file, and the file’s size is the parameter count times the bytes used for each number.
Two terms appear on model cards. “Parameters” is the total. “Active parameters” is the part used for each token in a mixture-of-experts model. Qwen3-30B-A3B, for example, has 30.5 billion parameters in total and 3.3 billion active.
02
Memory for the weights
The 4-bit column uses 4.89 bits per weight, the average of the common Q4_K_M format in llama.cpp, which keeps some tensors at higher precision. It matches published downloads: the Ollama file for Gemma 3 27B is 17 GB, and llama.cpp lists 43.1 GB for Llama 3.1 70B at Q4_K_M, a little above the table because real parameter counts run slightly above the round names. Common 8-bit files such as Q8_0 use about 8.5 bits per weight, so add around 6% to the 8-bit column.
Model size
16-bit (2 bytes)
8-bit (1 byte)
4-bit (about 4.9 bits)
4B
8 GB
4 GB
about 2.4 GB
8B
16 GB
8 GB
about 4.9 GB
27B
54 GB
27 GB
about 16.5 GB
32B
64 GB
32 GB
about 19.6 GB
70B
140 GB
70 GB
about 42.8 GB
03
The weights are only part of the memory bill
Add the KV cache, which grows with context: about 8.6 GB for one 32,768-token conversation on a model like Qwen3-32B.
Add working space: Hugging Face’s documentation suggests allowing up to 20% on top of the weights for inference.
Add the operating system and other programs when memory is shared with the rest of the machine.
Worked example: Qwen3-32B has 32.8 billion parameters, so about 20 GB of weights at 4-bit, plus 8.6 GB of cache for one 32K-token chat, plus up to 4 GB of working space. That is roughly 33 GB for a single user.
04
Bigger is not always better
Training matters: DeepMind’s Chinchilla, with 70 billion parameters trained on four times more data, outperformed Gopher (280B), GPT-3 (175B) and Megatron-Turing NLG (530B) on the paper’s tests.
Design matters: a mixture-of-experts model keeps most parameters idle for each token, so its total count overstates the work it does.
The task matters: in ANVE’s own test harness a 4B-class model handled meeting minutes and a 3B-class model failed, so ANVE does not use anything smaller than 4B-class for minutes.
Always test on your own documents. Benchmark rankings rarely match a particular office’s work.
05
Why it matters for an office
Parameter count is the quickest way to size hardware and set expectations. Small models of 4B to 8B suit extraction and short summaries. Models of 27B to 35B suit general drafting and questions over documents. A 70B model needs over 40 GB just for weights at 4-bit. ANVE’s launch tiers are specified in these terms: ANVE One pairs a 27B-class default with a fast 4B model, and ANVE Office serves a 27B-class default model from a 24 GB graphics card.
06
Questions people ask
Does 27B mean 27 gigabytes?
No. It means 27 billion parameters. The file is about 54 GB at 16-bit and about 17 GB at 4-bit.
Is a 70B model always better than a 27B model?
It usually knows more and reasons better, but not always. Training data and tuning matter, and a better-trained smaller model can beat a larger one, as Chinchilla did. Test candidates on your own tasks.
What are active parameters?
In a mixture-of-experts model, only some experts work on each token. The active count is the parameters actually used per token. Qwen3-30B-A3B has 3.3 billion active out of 30.5 billion.
How many parameters do I need for meeting minutes?
ANVE’s published harness found a 4B-class model sufficient and a 3B-class model not. Larger models give more headroom for long or messy meetings.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
