Can an NPU run a large language model? What an NPU is good for, what AMD’s software supports, and why TOPS is not the deciding number.
Short answer. Partly. An NPU can run small to mid-size language models, but through the vendor’s own software and with limits, while large models are still run on the graphics engine and unified memory. AMD rates the NPU in the Ryzen AI Max+ 395 at up to 50 TOPS out of up to 126 TOPS for the whole chip, and Microsoft sets 40+ TOPS for Copilot+ PCs. AMD’s Ryzen AI software runs NPU-only models with contexts of 4,096 or up to 16,000 tokens, and llama.cpp on these chips uses the graphics engine. For a 27B model, memory size and bandwidth matter more than TOPS.
01
The numbers
TOPS counts trillions of low-precision operations per second at the chip’s peak. Microsoft describes an NPU as a specialised chip for AI-intensive processes such as real-time translation and image generation.
Item
Figure
Source
NPU required for Copilot+ PCs
More than 40 TOPS
Microsoft Learn
NPU in the Ryzen AI Max+ 395
Up to 50 TOPS
AMD specifications
Whole-chip AI engine, same processor
Up to 126 TOPS; FP16 AI compute up to 60 TFLOPS
AMD specifications
Memory bandwidth, same processor
256 GB/s (256-bit LPDDR5x-8000)
AMD specifications and FAQ
02
What AMD’s software runs on the NPU
As of October 2026, AMD provides pre-optimised models in its NPU and hybrid modes for families including Llama, Mistral, Qwen, Gemma, GPT-OSS and Phi. The page lists the Ryzen AI 300 series (Strix and Krackan Point) for NPU-only and hybrid modes; we did not find the Ryzen AI Max+ 395 listed separately there, so confirm support for your exact chip with AMD.
Mode
Runs on
Context limit
Notes
NPU-only, Full Fusion
NPU
Up to 4,096 tokens (input plus output)
Higher throughput on short sequences
NPU-only, Token Fusion
NPU
Up to 16K tokens
For longer contexts
Hybrid
NPU and integrated GPU
Longer contexts need extra configuration beyond 4,096 tokens
NPU and GPU share the work of reading the prompt and writing the answer
GPU-only
Integrated GPU, through llama.cpp
Set by the model and memory
The route that open runtimes such as llama.cpp use
03
Why TOPS does not decide LLM speed
Writing each word means reading the model’s weights from memory, and NVIDIA describes that phase as memory-bound. The NPU shares the same memory as the rest of the chip, so a bigger TOPS figure does not raise the 256 GB/s ceiling. For a 27B model at 4-bit, about 16 GB of weights, capacity and bandwidth decide what is practical, which is why such models run on the graphics engine with unified memory.
04
What this means for an office
AMD describes its NPU-only mode as keeping the integrated GPU free for parallel work, which suits a small model running alongside others. A shared office assistant based on a 27B-class model needs memory capacity instead, and the unified-memory machines are built for that. The Strix Halo-class chip specified for ANVE One includes this NPU, and its 64 GB of unified memory is what lets it hold 27B-class models. These are launch specifications; ANVE has not published speed figures.
05
Questions people ask
Can an NPU run a 27B model?
AMD’s pages describe pre-optimised models and short contexts for the NPU and do not show a 27B-class model there. Large open models in llama.cpp run on the graphics engine.
Do I need a Copilot+ PC to run AI locally?
No. Copilot+ is Microsoft’s label for Windows PCs with a 40+ TOPS NPU. Ollama and llama.cpp run on CPUs and graphics hardware without one.
Does a higher TOPS figure mean faster answers?
Not for large language models on one conversation. Memory bandwidth sets the pace for generating text; TOPS is a peak arithmetic figure at low precision.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
