Large language model (LLM) The engine inside every AI assistant, in plain words.
Short answer. A large language model (LLM) is a computer program, built as a neural network with billions of adjustable numbers called parameters, that has learned from huge amounts of text to predict the next piece of a sentence. Repeating that one step lets it answer questions, summarise documents, translate and draft letters. Meta’s Llama 3 models were trained on about 15 trillion tokens of text, and the largest has 405 billion parameters. An LLM produces likely text rather than looking facts up, which is why it can sound sure and still be wrong, and why offices pair it with their own documents and a human check.
01
How it works
The text you type is cut into tokens, small pieces of words, and each token becomes a list of numbers. Those numbers pass through a stack of layers built on the Transformer design, introduced in 2017 in the paper “Attention Is All You Need”. In each layer, a mechanism called attention lets every token look at the others and weigh which ones matter. At the end the model scores every possible next token, one is chosen and added to the text, and the cycle repeats until the answer is complete.
Training happens in stages. Pre-training on trillions of tokens teaches language and a wide range of facts: Meta trained its 405B model on up to 16,000 H100 GPUs. Later stages teach the model to follow instructions and to decline some requests. The result is a large file of learned numbers, the weights. Using that file to answer a request is called inference, and it does not change the weights.
02
Sizes in numbers
Each parameter takes 2 bytes at 16-bit precision, so the file is roughly the parameter count times two. The 4-bit column is the size of the common Q4_K_M format, as listed in llama.cpp’s documentation for Llama 3.1. All three sizes can read up to 128K tokens at once.
Llama 3 model
Parameters
Layers
Weights at 16-bit
Weights at 4-bit
8B
8 billion
32
about 16 GB
about 5 GB
70B
70 billion
80
about 140 GB
about 43 GB
405B
405 billion
126
about 810 GB
about 249 GB
03
What an LLM does well and badly
Does well: drafting, summarising, pulling out names, dates and amounts, sorting and labelling text, translating, and answering questions about text you give it.
Does less well: exact arithmetic, facts from after its training cut-off, rare facts, and knowing when it does not know.
Fluent is not the same as correct: an LLM can state an invented figure in the same calm tone as a true one. This is called hallucination.
It sees only what is in its context window. It does not remember earlier conversations unless software feeds them back in.
04
Why it matters for an office
Three consequences follow. First, size drives hardware: memory is roughly the parameter count times the bytes per parameter, so a 27B model at 4-bit needs about 17 GB, while a 405B model needs about 249 GB even at 4-bit. Second, where the model runs decides where your text goes: a cloud service receives every prompt, while an open-weight model can run on a machine inside the building. Third, the model is one part of a system. The same LLM behaves very differently with and without your documents, a system prompt, tools and a person who confirms the result.
05
Where ANVE stands
ANVE is a private AI appliance designed to run open models on a device in the office. ANVE One, as specified for launch, uses a 27B-class model by default plus a fast 4B model, and in the development build every model call is logged with the user, the app and the model. No sealed unit exists yet; the software runs today on a development machine.
06
Questions people ask
Is an LLM the same as ChatGPT?
No. ChatGPT is a product built on LLMs. The LLM is the model inside; the product adds the chat screen, accounts, memory, tools and rules about what it may say.
How large is “large”?
There is no official line. In this glossary, small models have about 1 to 8 billion parameters, mid-size ones about 20 to 35 billion, and large ones 70 billion or more. The biggest openly released Llama 3 model has 405 billion.
Does an LLM learn from my documents when I use it?
Not by itself. Using a model is inference, which reads the weights and does not change them. Whether your text is stored or used to train later models depends on the service around it, so check a cloud service’s terms and a local system’s logging.
Why does an LLM sometimes make things up?
It is built to produce likely next tokens, not to check facts. When the answer is missing from its training and from its context, it can still produce fluent text. Supplying your documents and requiring quotes reduces this but does not remove it.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
