VRAM The memory on a graphics card, and why AI models need so much of it.
Short answer. VRAM (video random-access memory) is the memory attached to a graphics card, where the GPU keeps the data it is working on. For AI it must hold the model’s weights plus the working memory for the conversation, so its size decides which models a card can run at all. A model with 7 billion parameters stored in 16-bit form needs about 14 GB for its weights alone, and about a quarter of that in 4-bit form. NVIDIA’s RTX PRO 4000 Blackwell has 24 GB of GDDR7. Chips that share one memory pool, such as AMD’s Ryzen AI Max+ 395, let the GPU use up to 112 GB of a 128 GB pool.
01
What VRAM does
A GPU works fastest on data held in its own memory. If a model does not fit in VRAM, the software has to keep part of it in ordinary system memory and move it across the PCIe link as needed, which is much slower. That is why VRAM size, more than a GPU’s headline speed, decides which models run well.
VRAM is also built for bandwidth, the rate at which data can be read. NVIDIA lists 672 GB/s for the 24 GB of GDDR7 on the RTX PRO 4000 Blackwell. JEDEC, the memory standards body, says its GDDR7 standard doubles the bandwidth of GDDR6 and reaches up to 192 GB/s per memory device.
02
What fills VRAM
Three things compete for the space. The weights are the largest and are fixed for a given model and precision. The KV cache grows as the conversation gets longer and as more people use the model at once. The runtime and other programs take the rest.
What uses memory
How it scales
Example
Model weights
Parameters multiplied by bytes per weight
A 7-billion-parameter model in 16-bit form is about 14 GB (NVIDIA)
KV cache
Grows with every token and every concurrent user
About 2 GB for a 7-billion-parameter model at 16-bit with a 4,096-token context and one user (NVIDIA)
Runtime and other programs
Fixed overhead plus anything else using the GPU
The display, a second model, other applications
03
Weights by model size and precision
These figures are bare arithmetic, parameters multiplied by bytes per weight, so treat them as floors. Real quantised files are larger. AMD lists a 9-billion-parameter model at the 4-bit Q4_K_M setting as a 6.2 GB file, against 4.5 GB from the arithmetic, and a 35-billion-parameter model as 20.5 GB, against 17.5 GB. Add the KV cache and some headroom on top.
Model size
16-bit weights
8-bit weights
4-bit weights
7 billion parameters
14 GB
7 GB
3.5 GB
27 billion parameters
54 GB
27 GB
13.5 GB
70 billion parameters
140 GB
70 GB
35 GB
04
Dedicated VRAM and shared memory
AMD’s ROCm team says discrete graphics cards typically offer 8 to 24 GB of dedicated VRAM. A shared-memory chip can offer far more, because the CPU and GPU draw on one pool.
Point
Discrete graphics card
Shared-memory chip
Where the memory sits
On the card, fixed when you buy it
One pool used by CPU and GPU
Example
NVIDIA RTX PRO 4000 Blackwell: 24 GB GDDR7, 672 GB/s
AMD Ryzen AI Max+ 395: up to 128 GB of 256-bit LPDDR5x
What the GPU can use
All of the card’s VRAM
AMD says up to 112 GB of 128 GB; the share is set in the BIOS (64 GB in AMD’s own test)
Trade-off
Fast, but a model larger than the card spills over PCIe
Larger models fit, but bandwidth is lower: 256 GB/s from AMD’s published bus width and speed
05
What to check before you buy
Write down the largest model you want to run and its precision, then add KV cache for the number of people using it at once.
Leave headroom. A model that fills every byte of VRAM leaves nothing for context.
On a shared-memory machine, find out how much of the pool the GPU can address, and whether that is set in the BIOS or the operating system.
On a machine with a separate graphics card, do not count system RAM as VRAM.
If the card uses ECC, expect slightly less usable memory: NVIDIA’s documentation says reported memory can drop by several percent when ECC is on, because of the parity bits.
Ask whether a vendor’s “runs a 70B model” claim assumes 4-bit or 16-bit weights, and how long a context.
06
Where ANVE stands
ANVE’s Office Pro unit is specified with an NVIDIA RTX PRO 4000 Blackwell (24 GB) and 128 GB of system memory. At launch, ANVE One uses an AMD Strix Halo class chip with 64 GB of unified memory, and ANVE Office an NVIDIA RTX PRO 4000 Blackwell SFF Edition with 24 GB of GDDR7 and 64 GB of system memory. These are launch specifications: no sealed unit exists yet, and ANVE has published no speed figures.
07
Questions people ask
How much VRAM do I need for a 27-billion-parameter model?
The weights alone are about 13.5 GB at 4-bit and 54 GB at 16-bit, before the KV cache. In AMD’s examples real 4-bit files ran 15 to 40% larger than the bare arithmetic, so plan for roughly 16 to 19 GB at 4-bit, plus room for context.
Is VRAM the same as RAM?
No. VRAM sits on the graphics card and is built for bandwidth. System RAM serves the processor. On a shared-memory chip such as the Ryzen AI Max+ 395, one pool serves both.
Does more VRAM make a model run faster?
Not by itself. More VRAM lets a bigger model fit. Speed depends on memory bandwidth and compute: NVIDIA describes the word-by-word generation step as memory-bound.
What happens when a model does not fit in VRAM?
Runtimes can split it, keeping some layers on the GPU and the rest in system memory. In AMD’s test a 76 GB model exceeded the 64 GB given to the GPU and ran 61% on the GPU and 39% on the CPU. It works, but more of the model on the GPU generally means better speed.
08
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
