Is Strix Halo good enough for a 27B model?

Memory-wise, easily: a 4-bit 27B model is about 16 GB and AMD says a 64 GB machine can dedicate up to 48 GB to graphics. How fast it answers depends on the 256 GB/s of bandwidth and the software, which this page does not measure.

Can an RTX card use system memory when a model is too big?

Yes. llama.cpp can split a model between the card and system memory, but it describes this as partly accelerating the model, so only the part on the card runs at full speed.

Does the NPU change the choice?

Not much for large language models. AMD lists the NPU at up to 50 TOPS, but its LLM software runs NPU-only models with contexts of 4,096 to 16,000 tokens, and llama.cpp uses the graphics engine. See the NPU question.

ANVE: the private AI office, on the record. See what that means

ANVE

Private AI, answered · updated 2026-10-16

AMD Strix Halo or an NVIDIA RTX card for local AI?
Capacity against bandwidth, with the vendors’ own numbers.

Short answer. Choose by what the model needs: capacity or bandwidth. AMD’s Ryzen AI Max+ 395 (Strix Halo) offers up to 128 GB of unified memory at 256 GB/s, so it holds models that a 24 GB card cannot. An NVIDIA RTX PRO 4000 Blackwell has 24 GB of GDDR7 at 672 GB/s, 2.6 times the bandwidth, so a model that fits is read faster, but a model that does not fit spills into slower memory. ANVE’s launch line-up includes both: Strix Halo class for ANVE One, and an RTX PRO 4000 Blackwell for ANVE Office (the SFF Edition) and ANVE Office Pro. The right pick depends on model size and on how many people use it at once.

Side by side

Specification	AMD Ryzen AI Max+ 395 (Strix Halo)	NVIDIA RTX PRO 4000 Blackwell
What it is	A processor: 16 Zen 5 cores, a Radeon 8060S graphics engine with 40 graphics cores and an NPU in one package	A single-slot graphics card that needs a host computer
Memory	Up to 128 GB LPDDR5x-8000 on a 256-bit bus, shared with the processor	24 GB GDDR7 with error-correcting code (ECC), dedicated to the card
Memory bandwidth	256 GB/s (256 bits × 8,000 MT/s ÷ 8, matching AMD’s FAQ)	672 GB/s
Power	Default 55 W, configurable 45 to 120 W (chip only)	145 W maximum (card only)
AI figures the vendor publishes	FP16 AI compute up to 60 TFLOPS; overall up to 126 TOPS; NPU up to 50 TOPS	Blackwell architecture with 5th-generation Tensor Cores; the page gives no TOPS figure
Software routes in Ollama	ROCm 7 on Linux (listed for Ryzen AI Max+ 395) or Vulkan	NVIDIA CUDA (RTX PRO 4000 Blackwell is listed)

Capacity against bandwidth

Generating a word means reading the model’s active weights from memory once, so for one conversation bandwidth sets the pace; NVIDIA describes this decode phase as memory-bound. AMD’s own FAQ says that doubling bandwidth to 256 GB/s roughly doubles tokens per second against its lower tier. The card’s 672 GB/s is 2.6 times the 256 GB/s of Strix Halo (672 ÷ 256), so a model that fits in 24 GB is usually read faster on the card.

Capacity cuts the other way. A 4-bit 27B model is about 16 GB and fits both. A 4-bit 70B model is about 42 GB and fits only the Strix Halo machine, where AMD says 96 GB of a 128 GB machine can be graphics memory. A model that does not fit on the card is split with system memory, which llama.cpp describes as partly accelerating it.

What each does well

Strix Halo class: large models in a compact, low-power box; room for several models and long conversations at once; one pool of memory for the whole stack. One reviewed Framework Desktop with this chip drew 10 to 20 W at idle and 140 to 170 W under load in ServeTheHome’s test, at 36 to 41 dBA.
RTX card: faster reading of any model that fits in 24 GB; mature software support, since llama.cpp, Ollama and vLLM all have NVIDIA paths; ECC memory on the card. Its limit is 24 GB for the model plus every conversation’s cache.
Both: holding a 27B-class model in memory. The difference shows up when you ask for a bigger model, longer documents or more simultaneous conversations.

How ANVE’s plan uses each

ANVE One (1 to 3 named users, 64 GB) is specified on Strix Halo-class hardware with unified memory. ANVE Office (up to 25 named users) is specified on an NVIDIA RTX PRO 4000 Blackwell SFF Edition with 24 GB and 64 GB of system memory, and ANVE Office Pro (up to 100 named users) on an NVIDIA RTX PRO 4000 Blackwell with 24 GB and 128 GB of system memory. These are launch specifications with indicative prices, no sealed unit exists yet, and ANVE has not published speed or concurrency figures for either design.

Questions people ask

Is Strix Halo good enough for a 27B model?: Memory-wise, easily: a 4-bit 27B model is about 16 GB and AMD says a 64 GB machine can dedicate up to 48 GB to graphics. How fast it answers depends on the 256 GB/s of bandwidth and the software, which this page does not measure.
Can an RTX card use system memory when a model is too big?: Yes. llama.cpp can split a model between the card and system memory, but it describes this as partly accelerating the model, so only the part on the card runs at full speed.
Does the NPU change the choice?: Not much for large language models. AMD lists the NPU at up to 50 TOPS, but its LLM software runs NPU-only models with contexts of 4,096 to 16,000 tokens, and llama.cpp uses the graphics engine. See the NPU question.

What is not real yet

No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.

Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.

AMD Strix Halo or an NVIDIA RTX card for local AI?Capacity against bandwidth, with the vendors’ own numbers.