Top-p sampling Cutting off the long tail of unlikely words.
Short answer. Top-p sampling, also called nucleus sampling, limits a model’s next-word choice to the smallest group of tokens whose probabilities add up to p, then picks among them at random. With top-p at 0.9, the model ignores the unlikely tail that makes up the last 10% of probability, which avoids odd or nonsense words while still allowing variety. The method comes from the 2019 paper “The Curious Case of Neural Text Degeneration”. Ollama’s default is top-p 0.9 with top-k 40, and Qwen3 recommends 0.95 for thinking mode and 0.8 for non-thinking mode, both with top-k 20.
01
How it works
After the model scores every candidate token, top-p sorts them from most to least likely and adds up their probabilities from the top until the total reaches p. Everything below that point is dropped, the remaining probabilities are rescaled to add up to 100%, and one token is drawn at random. The group kept is called the nucleus. Its size changes with each step: when the model is sure, one or two tokens may be enough; when it is unsure, many are kept. That adaptive behaviour is the method’s main advantage over a fixed cut-off.
Example: six candidates with probabilities 50%, 25%, 12%, 7%, 4% and 2%.
Top-p setting
Tokens kept
Chance of the likeliest after rescaling
0.5
1
100%
0.8
3
57.5%
0.9
4
53.2%
0.95
5
51.0%
1.0 (off)
6
50.0%
02
Top-p, top-k and min-p
Three truncation settings are common, and they combine with temperature. Top-k keeps a fixed number of the likeliest tokens. Top-p keeps a share of the probability. Min-p, described in Ollama’s documentation as an alternative to top-p, keeps tokens whose probability is at least a set fraction of the likeliest token’s.
Setting
Ollama default
Qwen3 thinking mode
Qwen3 non-thinking mode
Temperature
0.8
0.6
0.7
Top-k
40
20
20
Top-p
0.9
0.95
0.8
Min-p
0.0
0
0
03
Practical settings
Start from the model maker’s recommended settings, which the model card usually lists, and change one setting at a time.
At a very low temperature, top-p has little effect, since one token already holds almost all the probability. At temperature 0, it has none.
A top-p of 1.0 switches the cut-off off. Ollama’s default of 0.9 and Qwen3’s 0.8 and 0.95 show the usual range for general use.
For structured output such as JSON, low temperature and clear instructions matter more than top-p.
Write down the settings used for any result that goes into a record, since different tools ship different defaults.
04
Why it matters for an office
If the same model gives different-looking answers in two tools, sampling defaults are a common cause: the tools may use different temperatures, top-p or top-k values. Matching these settings, and recording them next to the model name, makes results comparable. The Holtzman paper that introduced the method observed that always choosing the likeliest word produces bland, repetitive text, while sampling from the full distribution lets in the unreliable tail. Top-p is the middle path, and for office work the aim is usually to stay close to the source, so keep the settings modest.
05
Questions people ask
What top-p value should I use?
Use the one the model’s maker recommends. Without guidance, 0.9 is a common default. Lower values such as 0.5 give more focused text and higher values such as 0.95 give more variety.
Is top-p better than top-k?
Top-p adapts to the model’s confidence, while top-k keeps a fixed number of tokens whatever the situation. Many setups use both, as Ollama does by default with top-k 40 and top-p 0.9.
Should I change top-p and temperature together?
It is easier to reason about one at a time. Both narrow or widen the choice, so changing both can hide which one made the difference. Test changes on the same set of prompts.
Does top-p stop hallucination?
No. It trims unlikely words, but a confident wrong answer is usually a likely one. Grounding the answer in sources and checking it are what reduce errors.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
