How are models updated on an offline AI box? Files go in on checked media, and nothing switches until your own tests pass.
Short answer. A new model reaches an offline box as files. Someone downloads them on a computer that has internet, checks them against the publisher’s checksum, carries them in on a USB drive and imports them with the runtime’s own import command. Nothing should change for staff until the new model has passed your own set of test questions, and the old model should stay on disk so you can roll back. Two checks matter more than speed: the integrity of the file, because model repositories give weak guarantees about where a model came from, and the licence, because open models carry different terms.
01
The seven steps
Decide why you are changing: a fault you hit, better quality on your own tasks, or lower memory use. A newer model is not automatically better for your documents.
Download on a connected computer from the publisher’s official repository, and record the exact version.
Check integrity. OWASP advises using models from verifiable sources and applying integrity checks with signing and file hashes, because published models have no strong provenance.
Read the licence and the model card before importing. Terms differ between open models.
Carry the files in on media that is scanned first and kept for this one purpose.
Import with the runtime. In Ollama, create a Modelfile whose FROM line points at the GGUF file or the safetensors folder, then run the create command. Ollama does not quantise GGUF files during import, so prepare the quantised file first.
Test with your own questions, switch one group first, and keep the previous model so you can go back.
02
What to test before switching
Check
How to do it
Why it matters
Your question set
Run 20 to 30 real questions with known answers on the old and new model and compare.
Public benchmarks do not use your documents or your house style.
Citations
Confirm each answer still points to the right source passage.
A fluent answer with the wrong source is worse than a plain one with the right source.
Languages
Include samples in every language your staff use, including mixed Hindi and English.
Quality can differ between languages.
Memory and speed
Watch memory use while several people ask at once.
A larger model can crowd out other work on the same box.
Licence
Read the licence and model card for permitted use.
Commercial terms are not the same for every open model.
03
The embedding model is a special case
Most model changes touch only the chat model. If you change the embedding model, the one that turns documents into searchable vectors, every document has to be embedded again. Open WebUI’s documentation says changing it requires a re-index of all knowledge-base documents, because embeddings from different models are not compatible. Plan the time, and run it outside working hours.
04
Why integrity checks matter
OWASP’s supply-chain entry for language-model applications points out that model cards give no guarantee of a model’s origin, and that an attacker can compromise a supplier’s account on a model repository, or set up a look-alike one, to attack an application’s supply chain. The same logic applies to carried-in media. NIST’s patching guide makes a related point about updates in general: a patch can be flawed or even compromised, so try it on a small number of machines first.
Offline mode in the Hugging Face libraries works from files that were downloaded and cached beforehand, with no calls to the Hub, which suits this routine.
05
What ANVE says
ANVE’s meeting-minutes maker never uses a model smaller than 4B class, because a 3B-class model failed ANVE’s test harness (see the note on the 4B question). Setting a floor from a fixed set of tests is the kind of rule to ask any vendor about before a model is swapped.
Today ANVEAI installs updates on site. Signed update packages applied offline by USB are not on ANVE yet, and ANVE does not update itself over the internet (see /security). Whether an administrator can add their own models is not stated here. ANVEAI confirms it in writing before a paid pilot.
06
Questions people ask
Where does Ollama keep its model files?
On Linux with the standard installer, under /usr/share/ollama/.ollama/models; on macOS in ~/.ollama/models; on Windows under the user profile. Set OLLAMA_MODELS to use another folder.
Do we have to retrain the AI on our documents when the model changes?
No. Documents are searched at question time, not baked into the model. Only a change of embedding model forces the documents to be embedded again.
Can we skip updates and keep the model we have?
Yes. A model that passes your tests does not expire. Updates matter for fixes you need and for security patches in the software around the model, so keep the version you tested and change it on purpose.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
