How do you secure an AI box on the office network? Ten checks, from patching to physical access, with the reasons behind each.
Short answer. Treat the box like any server that holds client records, and add one AI-specific check: the model service must never be reachable from the internet. Patch on a schedule. Give administrators separate accounts with multi-factor sign-in. Let staff reach only the web interface, and use HTTPS. Put the box on its own network segment, keep logs with correct time, and back up and test restores. Encrypt the disk and lock the room. The AI-specific risk is real: Cisco researchers found 1,139 exposed, unauthenticated Ollama servers, and the vLLM project warns that its API key does not protect every endpoint.
01
Ten checks
Patch. Define routine and emergency patching, try updates on a copy first and keep a log (NIST SP 800-40 Rev. 4).
Accounts. Named users, a separate account for administrators, multi-factor sign-in and no default passwords.
Least privilege. Give each person, and each AI tool or connector, only the access the job needs. OWASP names excessive functionality, permissions and autonomy as the root causes of harm from AI agents.
Ports. Expose only the web interface to staff. Keep the model runtime on the local machine or a private interface.
Encryption in transit. HTTPS in front of the application.
Segmentation. Put the box on a server segment behind firewall rules, not on guest or flat Wi-Fi.
Logs and time. Record logins and administrator actions, successful and unsuccessful, and keep the logs. CERT-In’s directions call for logs kept for a rolling 180 days and clocks synchronised to NIC or NPL time.
Backups. Three copies, one offline, restore tested.
Disk encryption and secure boot. Protect the data if the box leaves the building, and decide how the key is released after a restart.
Physical. A lockable room, a cable lock, an asset tag, and a list of who holds keys.
02
The AI-specific risk: the model port
Model runtimes are built for trusted machines. Ollama binds to 127.0.0.1 by default. The vLLM security guide says its --api-key option covers only the /v1, /v2, /inference and /cohere path prefixes and leaves other endpoints on the same server unauthenticated, so it advises a reverse proxy that allows only the endpoints you want to expose. Open WebUI’s hardening guide says it is built for private, trusted networks and belongs behind a VPN, a zero-trust access proxy or an authenticated reverse proxy.
Surveys show what happens otherwise. Cisco found 1,139 exposed Ollama servers answering prompts without authentication. SentinelLABS and Censys counted 175,108 Ollama hosts that were publicly accessible at some point during 293 days.
03
Where each control is written down
Check
Why
Where to read it
Patching
Delay gives attackers more time, and makes emergencies harder
NIST SP 800-40 Rev. 4
Firewalls, secure configuration, update management, user access control, malware protection
Five basic controls against common attacks
NCSC Cyber Essentials
Server hardening and planning
Minimum services, least privilege, logs, backups
NIST SP 800-123
Limits on AI tools
Restrict functions, permissions and autonomy
OWASP Top 10 for LLM Applications 2025, LLM06
Logs and time
Evidence for investigating incidents
CERT-In directions of 28 April 2022
04
Reporting: who to tell if it goes wrong
CERT-In’s directions of 28 April 2022 require service providers, intermediaries, data centres, body corporate and government organisations to report listed cyber incidents within six hours of noticing them. The list includes unauthorised access to IT systems or data, data breach, data leak and attacks on systems related to artificial intelligence and machine learning. CERT-In’s FAQ explains that “body corporate” includes a firm, sole proprietorship or other association engaged in commercial or professional activities.
Whether a given incident must be reported is a question for the office’s own advisers. This is general information, not legal advice. Write the reporting contact into your plan before you need it.
05
What ANVE says
ANVE keeps the office’s data on the box in the office, and the legal duties for that data remain the office’s. ANVE has no security certifications, and none is claimed. Today its models are reached through the runtime’s own OpenAI-compatible endpoint (Ollama or vLLM); the governed ANVE API, with a key per user or agent and every request in the audit trail, is not on ANVE yet.
The data-flow sheet at /security publishes several positions. Full-disk encryption and secure boot are part of the appliance image, which is not on ANVE yet. Today ANVEAI installs updates on site; signed update packages applied offline by USB are not on ANVE yet, and ANVE does not update itself over the internet. Roles today are administrator and user, with single sign-on not on ANVE yet. Connectors work only if an administrator approves one, and every call is logged on the box. The tools an agent can use must be approved by an administrator, and every tool call is logged.
Default ports and remote-support paths are not stated here. Ask for a written security sheet that covers them before a paid pilot.
06
Questions people ask
Do we need antivirus on an AI box?
Ask what malware protection the vendor ships. The NCSC lists malware protection among five basic controls. On an appliance the vendor may use other methods, so ask what they are and how they are updated.
Should the box be reachable from the internet so the vendor can help?
Not by default. If support needs access, use a session your office starts, approves, logs and ends. NIST’s nonlocal-maintenance control (MA-4) asks for approval, strong authentication, records and termination of remote sessions.
How often should we review the settings?
At every update and on a fixed review date. Open WebUI’s hardening guide advises reviewing accounts and permissions periodically and removing inactive accounts.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
