The software runs on your machine and its local API listens only on that machine by default. What leaves the building depends on what you configure and which models you call, since Ollama also has a cloud service. Use local models and verify with the network disconnected.

Can Ollama serve a whole office?

Its defaults suit one user: one parallel request per loaded model and a queue of 512. You can raise them, but you would also need to add sign-in, roles and logging.

What is the difference between Ollama and llama.cpp?

llama.cpp is the inference engine. Ollama wraps it with a model library, a one-command installer, a Modelfile format and its own API.

Does Ollama support the OpenAI API?

Partly. It supports a subset, including chat completions, embeddings, streaming, tools and vision, and the base URL is http://localhost:11434/v1/. Some fields, such as logprobs, are not supported.

ANVE: the private AI office, on the record. See what that means

ANVE

Private AI glossary · updated 2026-10-16

Ollama
The simplest way to run an open model on your own machine.

Short answer. Ollama is software that downloads and runs open language models on your own computer with one command, and exposes them through a local API on port 11434, including an OpenAI-compatible endpoint. It uses llama.cpp as its supported back end, keeps a model in memory for 5 minutes after the last request by default, and listens only on the machine itself unless you change the OLLAMA_HOST setting. It is a good fit for one technical user. An office that needs named users, roles, an audit trail and approved connectors has to add them around it: the API key in its OpenAI-compatible mode is a placeholder that the server ignores.

What it does

You install it on macOS, Windows or Linux, or run the official Docker image, then start a model with a command such as ollama run followed by a model name from its library. The same models are then available through a REST API at http://localhost:11434, and through an OpenAI-compatible API at /v1/ for chat completions, completions, models, embeddings and non-stateful responses. Ollama’s notes describe this as a subset of the OpenAI API, with gaps such as logprobs.

A Modelfile lets you build a variant: it can start from an existing model, from Safetensors weights or from a GGUF file, and set parameters, a prompt template and a system message. Thinking models can be switched on or off with a think setting. Models are kept in a folder on disk, which on macOS is ~/.ollama/models.

Settings that matter for shared use, as of October 2026

Setting	What it controls	Default in the documentation
OLLAMA_HOST	Address and port the server listens on	127.0.0.1, port 11434
OLLAMA_NUM_PARALLEL	Parallel requests each loaded model handles	1
OLLAMA_MAX_LOADED_MODELS	Models held in memory at once	3 times the number of GPUs, or 3 for CPU
OLLAMA_MAX_QUEUE	Requests queued before new ones are rejected	512
OLLAMA_CONTEXT_LENGTH	Context window	4k, 32k or 256k tokens depending on video memory
keep_alive	How long a model stays loaded after a request	5 minutes
OLLAMA_KV_CACHE_TYPE	Precision of the KV cache	f16; q8_0 uses about half the memory and q4_0 about a quarter

What an office has to add

Named users and authentication. In the FAQ and compatibility notes we reviewed, the API key is a placeholder and no per-user accounts are described.
Roles and permissions: who may use which model, which documents and which tools.
An audit trail: who asked what, which model answered, and whether it ran locally or in the cloud.
Safe exposure. If you set OLLAMA_HOST to share the server on the network, anyone who can reach the port can use it. Ollama’s FAQ shows putting it behind a proxy such as Nginx, and sign-in has to be added there.
Document search with sources, approved connectors, backups and update management.
A clear line between local and cloud. Ollama’s own notes also refer to a cloud API, so check which one a tool is calling.

Where ANVE stands

ANVE’s models are reachable today through the runtime’s own OpenAI-compatible endpoint, which is Ollama or vLLM, and ANVE’s published 4B test ran on Ollama on an Apple M3 Pro laptop. On top of the runtime, roles today are administrator and user, with teams, finer roles and single sign-on not on ANVE yet. Cloud models, connectors and agent tools all need an administrator’s approval, and every call is logged. The governed ANVE API, with a key per user or agent and every request in the audit trail, is not on ANVE yet. Our security page sets out these positions, and our comparison sets ANVE beside a mini PC running Ollama.

Questions people ask

Is Ollama private?: The software runs on your machine and its local API listens only on that machine by default. What leaves the building depends on what you configure and which models you call, since Ollama also has a cloud service. Use local models and verify with the network disconnected.
Can Ollama serve a whole office?: Its defaults suit one user: one parallel request per loaded model and a queue of 512. You can raise them, but you would also need to add sign-in, roles and logging.
What is the difference between Ollama and llama.cpp?: llama.cpp is the inference engine. Ollama wraps it with a model library, a one-command installer, a Modelfile format and its own API.
Does Ollama support the OpenAI API?: Partly. It supports a subset, including chat completions, embeddings, streaming, tools and vision, and the base URL is http://localhost:11434/v1/. Some fields, such as logprobs, are not supported.

What is not real yet

No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.

Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.

OllamaThe simplest way to run an open model on your own machine.