ANVE: the private AI office, on the record. See what that means

ANVE

Private AI, answered · updated 2026-10-16

Can local AI models use tools and APIs?
How tool calling works on your own hardware, and how to keep it safe.

Short answer. Yes. Open models such as Qwen, Llama 3.3 and gpt-oss are trained to ask for a tool call, and local servers pass that request to your code through OpenAI-style function calling in Ollama, in llama.cpp (with the --jinja flag) and in vLLM (with a tool parser). The model never runs the tool itself: your software does, then returns the result for the model to use. Reliability varies by model, so test your own tools, cap the number of steps and log every call. ANVE’s agent loop allows up to seven tool steps per request and challenges any claim made without a successful tool call.

01

How a tool call works

  • Your application sends the question together with a list of tools: a name, a description and the arguments each accepts.
  • The model replies with a structured request, such as “call get_temperature with city = Kanpur”, instead of a normal answer.
  • Your application runs the tool, which could be a database query, a spreadsheet lookup or a web search, and sends the result back.
  • The model reads the result and either asks for another tool or writes the final answer.
02

What the local runtimes support

All three expose an OpenAI-compatible endpoint, so tools written against OpenAI’s function-calling format usually move across by changing the base address.

RuntimeHow tool calling is switched onNotes from its documentation
OllamaA tools list in the chat requestDocuments single calls, parallel calls and multi-turn agent loops; Qwen3 appears in its examples
llama.cpp (llama-server)The --jinja flagNative formats for Llama 3.1 and 3.3 (including built-in tools), Qwen 2.5, Hermes and GPT-OSS (Harmony); other templates fall back to a generic format that may use more tokens and be less efficient
vLLM--enable-auto-tool-choice with a --tool-call-parserSupports named functions and the auto, required and none options of the tool_choice field
03

Which models are trained for it

OpenAI’s gpt-oss card lists native function calling, web browsing, Python code execution and structured outputs, and its model paper reports higher maths scores when the Python tool is allowed. Meta’s Llama 3.3 is supported with built-in tool names in llama.cpp. The Qwen3.8-27B card describes agent execution and compatibility with popular harnesses as headline improvements. Tool-calling quality still varies, and the Berkeley Function Calling Leaderboard (version 4, last updated 12 April 2026) ranks models on it, which is the place to compare before you test your own.

04

Keeping it safe

  • Start with read-only tools. Add any tool that sends, pays, deletes or changes data only with a person approving each use.
  • Cap the steps. A model can loop or chase a wrong lead, so set a hard limit on tool steps per request.
  • Check claims against calls. Challenge any answer that says an action was done when there is no matching successful tool call.
  • Log every call with the user, the tool and the result, and let an administrator decide which tools exist at all.
  • Treat tool output as untrusted text. A web page or file can contain instructions aimed at the model, which is called prompt injection.
  • For comparison, ANVE’s published positions: its agent loop allows up to seven tool steps per request and challenges a claim made without a successful tool call; its model gateway logs each model call with the user, app and model; and, per its security page, the tools an agent can use must be approved by an administrator, a tool that changes is switched off, and every tool call is logged.
05

Connecting real tools

The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems, so one connector can serve many applications. On ANVE, tools such as Zoho and others can connect through MCP connectors that an administrator approves; a connector sends that service only what it is asked to fetch or send, and every call is logged on the box. Ready-made Google Workspace and Zoho connectors are not on ANVE yet. The governed ANVE API, with a key per user or agent and every request in the audit trail, is not on ANVE yet; today the models are reachable through the runtime’s own OpenAI-compatible endpoint.

06

Questions people ask

Does a local model need the internet to use tools?
The model runs locally, so it does not. A tool needs whatever it needs: a web search tool needs the internet, while a query against your own database does not.
Which local model calls tools most reliably?
Check the Berkeley Function Calling Leaderboard for current scores, then test two or three on your own tools. Scores depend on the tool set and the prompt format, and models change quickly.
Is function calling the same as MCP?
No. Function calling is how a model asks for a tool. MCP is a standard way to describe and connect tools so many applications can share them.
07

What is not real yet

No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.

Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.

08

Sources