Temperature The dial between predictable and varied answers.
Short answer. Temperature is a setting that controls how random a language model’s word choices are. At each step the model gives every possible next token a probability. A low temperature sharpens that distribution so the likeliest token nearly always wins; a high temperature flattens it so unlikely tokens are chosen more often. Ollama’s default is 0.8, Qwen3 recommends 0.6 for its thinking mode and 0.7 for non-thinking, and ANVE’s published meeting-minutes test ran at 0.1. Even at 0, repeated runs can differ: one study of 1,000 runs at temperature 0 produced 80 different outputs.
01
How it works
The model produces a score for each candidate token. Temperature T divides those scores before they are turned into probabilities, the softmax step described in Hinton’s distillation paper. T below 1 widens the gaps between candidates; T above 1 narrows them. As T approaches 0, the choice becomes greedy: always the single likeliest token.
A worked example with three candidates whose probabilities at T = 1 are 60%, 30% and 10% shows the effect.
Temperature
Likeliest token
Second
Third
0.2
97.0%
3.0%
0.01%
0.5
78.3%
19.6%
2.2%
1.0
60.0%
30.0%
10.0%
2.0
47.3%
33.4%
19.3%
02
Starting points by task
These are starting points to test, not rules. Where a maker publishes recommended settings for a model, use those first.
Task
Temperature to try
Basis
Extraction, classification, meeting minutes
0 to 0.2
ANVE’s published minutes test ran at 0.1
Questions answered from documents
0.1 to 0.3
Low randomness helps the answer stay close to the source
General assistant and drafting
0.6 to 0.8
Qwen3 recommends 0.7 in non-thinking mode; Ollama defaults to 0.8
Reasoning models in thinking mode
0.5 to 0.7
Qwen3 recommends 0.6; DeepSeek advises 0.5 to 0.7. Qwen3 warns against greedy decoding in thinking mode
Brainstorming names or ideas
0.8 to 1.0
More variety is the point
03
Why temperature 0 is not always repeatable
Researchers at Thinking Machines sampled 1,000 completions at temperature 0 from Qwen3-235B-A22B-Instruct-2507 for the same prompt and got 80 unique completions. The outputs matched for the first 102 tokens and then diverged. They traced the cause to computations whose results depend on the batch size, which varies with server load, rather than, as often assumed, to concurrent floating-point arithmetic alone. With batch-invariant kernels, all 1,000 completions were identical. Ollama’s seed setting is meant to make output repeatable for the same prompt, but a busy server shared by many users can still vary.
04
Why it matters for an office
For records such as minutes, extracted amounts or a reading of a GST bill, use a low temperature and check the output; do not rely on identical wording from run to run. Low temperature reduces random slips but does not stop a model inventing a fact it does not know, so grounding and review still matter. Record the model, version and settings used for any output that goes into a file, so a result can be explained later.
05
Questions people ask
What temperature should I use for summaries?
A low to moderate value, about 0.1 to 0.5, keeps a summary close to the source. Try a few settings on real documents and have a person compare the results.
What does temperature 0 mean?
The model always picks its likeliest next token (greedy decoding). The result is usually, though not always, the same on every run.
Is a higher temperature more creative or just more wrong?
Both are possible. Higher temperature gives more varied wording and ideas, and a higher chance of odd or incorrect choices. Raise it only where variety is wanted.
Does lowering temperature stop hallucination?
No. It reduces random slips, but a model that lacks the fact can still state a wrong one with high confidence. Supplying sources and checking the answer matters more.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
