How much memory does a 27B model need? The size class ANVE plans to use by default, worked out in gigabytes.
Short answer. A 27B model needs about 16 GB for its weights at 4-bit (Q4_K_M), 29 GB at 8-bit (Q8_0) and 54 GB at BF16, plus the KV cache and 1 to 2 GB of overhead. The cache is where 27B models differ. Qwen3.8-27B keeps a cache for only 16 of its 64 layers and needs about 0.5 GB per conversation at 8,192 tokens; a design that caches every layer, counted that way for Gemma 3 27B, needs about 4 GB. So a 4-bit 27B model fits a 24 GB graphics card for one or two people at moderate context and sits comfortably in 64 GB of unified memory. All figures are estimates.
01
Weights: the same method for every size
Weights (GB) ≈ parameters in billions × bits per weight ÷ 8. Real files run a little above the estimate because some tensors stay at higher precision. GB here means 10^9 bytes.
Precision
Bits per weight
Weights for 27B (estimate)
Real files for comparison
BF16
16
54.0 GB
54.0 GB in Google’s Gemma 3 report; 54.0 GB (Gemma 3 27B) and 54.7 GB (Qwen3.8-27B, includes its vision part) as community GGUF files
Q8_0
about 8.5
28.7 GB
28.7 GB (Gemma 3 27B) and 29.0 GB (Qwen3.8-27B), community quants
Q4_K_M
about 4.8
16.2 GB
16.6 GB (Gemma 3 27B) and 16.5 GB (Qwen3.8-27B), community quants; 14.1 GB for Google’s own 4-bit build
02
The KV cache differs by design
NVIDIA’s formula is 2 × layers × key/value heads × head size × bytes per value, per token. Qwen3.8-27B has 64 layers, but only every fourth one (16 layers) uses ordinary attention with a cache, with 4 key/value heads of 256 values; the other 48 use a linear-attention design with a fixed-size state instead. Gemma 3 27B has 62 layers with 16 key/value heads of 128 values. Google’s report says Gemma 3 mixes in sliding-window layers to shrink its cache, so the Gemma row below, which counts every layer as caching every token, is an upper bound.
Model
Bytes per token
8,192 tokens
32,768 tokens
131,072 tokens
Qwen3.8-27B (16 of 64 layers cached)
65,536
0.5 GB
2.1 GB
8.6 GB
Gemma 3 27B (every layer counted)
507,904
4.2 GB
16.6 GB
66.6 GB
03
Will it fit?
Qwen3.8-27B’s linear-attention layers also keep a fixed state for each conversation. From its configuration I estimate about 0.15 GB, which is included below as 2.30 GB per conversation at 32,768 tokens.
Hardware and case
Weights + caches + 2 GB overhead
Total
Fits?
24 GB card: Qwen3.8-27B Q4_K_M, 2 conversations at 32,768 tokens
64 GB unified memory (48 GB usable, per AMD): Qwen3.8-27B Q8_0, 4 conversations at 32,768 tokens
28.7 + 9.2 + 2.0
39.9 GB
Yes
128 GB unified memory (96 GB usable, per AMD): Qwen3.8-27B BF16, 8 conversations at 32,768 tokens
54.0 + 18.4 + 2.0
74.4 GB
Yes
04
What it means for an office
A 27B model reads about 16 GB of weights for each word it writes at 4-bit, against about 42 GB for a 70B model, which is the main reason this class suits a shared box. The vision part of Qwen3.8-27B is a separate file of about 0.9 GB in the community GGUF release.
ANVE One is specified with 64 GB of unified memory, a 27B-class default model (Qwen 3.8 27B) and a fast 4B model for 1 to 3 named users. ANVE Office is specified with a 24 GB NVIDIA RTX PRO 4000 Blackwell SFF Edition and 64 GB of system memory, for up to 25 named users, with a 27B-class default model. These are launch specifications with indicative prices; the software runs today on a development machine and no sealed unit exists yet.
05
Questions people ask
Is a 27B model enough for office work?
For summaries, extraction and questions about your own documents, models of this size are what ANVE plans to use by default on ANVE One. Test any model on your own files first, because size is only one factor.
Why are real files bigger than the estimate?
Q4_K_M is a mixture: llama.cpp keeps some tensors at higher precision than the rest, and its quantize tool has a --pure option that turns the mixing off. The community 4-bit files above are 16.5 GB and 16.6 GB against the 16.2 GB estimate.
Does a long context make a 27B model too big?
It can. For a design that caches every layer, 131,072 tokens would add about 67 GB. Qwen3.8-27B’s cache is about 8.6 GB for the same length, which is why newer hybrid designs fit long documents better.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
