Mixture of experts (MoE) Big-model memory, small-model speed: how sparse models work.
Short answer. A mixture-of-experts (MoE) model splits parts of its network into many specialist sub-networks, called experts, and a small router picks only a few of them for each token. Total parameters set how much memory the model needs; active parameters set how much work it does per token. Qwen3-30B-A3B has 30.5 billion parameters in total and uses 3.3 billion for each token, drawn from 8 of its 128 experts. It must still be loaded in full, about 19 GB at 4-bit, but it reads far fewer bytes per token than a dense 32B model, so for one user on the same machine it can generate several times faster.
01
How it works
In an ordinary dense Transformer, each layer ends with one large feed-forward block. In an MoE model that block is replaced by many smaller ones, the experts, plus a router: a small network that scores the experts for each token. Only the top few are run and their outputs are combined. Hugging Face calls this conditional computation, since only parts of the system run for each input. The sparsely-gated layer used in modern models was introduced in a 2017 paper, which reached up to 137 billion parameters.
Despite the name, experts are not departments. Mixtral’s authors found no obvious pattern in which expert handled which topic. They saw some syntactic patterns instead, and consecutive tokens were often sent to the same experts. The router chooses separately for every token and every layer.
02
Examples in numbers
Model
Total parameters
Active per token
Experts used
Memory to load the weights
Qwen3-30B-A3B
30.5B
3.3B
8 of 128
about 19 GB at 4-bit
gpt-oss-20b
21B
3.6B
4 of 32
within 16 GB (maker’s figure)
gpt-oss-120b
117B
5.1B
4 of 128
one 80 GB GPU (maker’s figure)
Mixtral 8x7B
47B
13B
2 of 8
about 29 GB at 4-bit
Qwen3-32B (dense, for comparison)
32.8B
32.8B
All
about 20 GB at 4-bit
03
Memory versus speed
Two numbers matter. Total parameters set memory: Mixtral’s authors state that its memory cost is proportional to the full 47B, because every expert must be loaded, and Hugging Face likewise notes that MoE models need high VRAM since all experts sit in memory. Active parameters set speed: each token reads and computes with only the chosen experts.
The table compares two Qwen3 models for one user. The ceilings divide memory bandwidth by the bytes read per token. They are upper bounds, not measurements. When many users are served at once, different tokens pick different experts, so a busy server reads most experts at every step and the gap narrows.
One user, 4-bit weights
Qwen3-32B (dense)
Qwen3-30B-A3B (MoE)
Weights held in memory
about 20 GB
about 19 GB
Bytes read per token
about 20 GB
about 2 GB
Ceiling on a 200 GB/s machine
about 10 tokens per second
about 100 tokens per second
KV cache per token (16-bit)
262 KB
98 KB
04
Why it matters for an office
Single-user speed: on a machine limited by memory bandwidth, an MoE model answers several times faster than a dense model of similar file size.
Memory is not saved: every expert must fit, so MoE does not make a big model run on a small machine.
Quality varies: Mixtral 8x7B, with 13B active parameters, matched or beat Llama 2 70B on its paper’s benchmarks, and Qwen reports that its 30B-A3B outcompetes QwQ-32B, which has ten times as many activated parameters. Test on your own tasks rather than trust a parameter count.
Tuning is harder: Hugging Face notes that MoE models face challenges in fine-tuning and are more prone to overfitting.
Read names carefully: 30B-A3B means about 30 billion in total and about 3 billion active.
05
Questions people ask
What does A3B mean in Qwen3-30B-A3B?
The model has about 30 billion parameters in total (30.5 billion exactly), and about 3 billion are active for each token (3.3 billion exactly).
Do all the experts have to be in memory?
Yes. The router can send any token to any expert, so all must be loaded. Mixtral’s authors put its memory cost at the full 47 billion parameters.
Is an MoE model faster than a dense model?
For one user, usually yes, because each token reads and computes with fewer parameters. With many simultaneous users the advantage shrinks.
Are the experts specialists in law, medicine or code?
Not in a way you can use. Mixtral’s routing analysis found no obvious topic patterns in how experts were chosen.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
