Can a local AI model read images and diagrams? How a language model sees a picture, and what it costs in memory.
Short answer. Yes. Vision-language models read photos, scanned pages, charts and diagrams on local hardware. Qwen3.8-27B is a native vision-language model that understands images and video, and Gemma 3 turns each image into 256 tokens with a SigLIP vision encoder. The vision part is a separate file of under 1 GB in the community GGUF releases, and each image uses context, so the memory cost is modest. Results vary by document: handwriting, small print and complex tables are still hard, so test your own scans. ANVE’s vision models read scanned documents, photos and diagrams, and it reads GST bills.
01
How a model reads a picture
A vision-language model has a vision encoder that turns an image into numbers, followed by the language model that reads those numbers as if they were words. Google’s Gemma 3 report says its models treat images as a sequence of soft tokens from a SigLIP encoder, condensed to a fixed 256 vectors per image at 896 × 896 pixels, with a Pan and Scan option for other shapes. The Qwen3.8-27B card describes the model as a causal language model with a vision encoder and says it understands images and video, from STEM diagrams and documents to hour-scale videos.
In llama.cpp the vision part is a separate file called an mmproj. Its documentation advises keeping these multimodal components at a high-quality format such as bf16 or q8, because the quality of the encoder feeds everything the language model sees.
02
What it costs in memory
Images also take time to encode. The cache figures below are memory only, using NVIDIA’s per-token cache formula; Qwen3.8-27B’s cache is smaller per token because only a quarter of its layers keep one.
Item
Size
Source or method
Vision file, Gemma 3 27B (mmproj, BF16)
0.86 GB
Community GGUF release
Vision file, Qwen3.8-27B (mmproj, BF16)
0.93 GB
Community GGUF release
Tokens per image, Gemma 3
256
Google’s Gemma 3 report
Cache for one image, Gemma 3 27B, every layer counted
0.13 GB
256 tokens × 507,904 bytes per token (upper bound, estimate)
Cache for a 20-page scan, Gemma 3 27B, every layer counted
2.6 GB
20 images × 256 tokens (upper bound, estimate)
03
What they read well and badly
Typed pages, printed tables, charts and slides: model cards report strong scores on document and chart benchmarks, but those are the makers’ own figures on public test sets, not on your scans.
Handwriting, faint photocopies, stamps over text and small print remain hard; a wrong digit in a total looks as confident as a right one.
Tables that run across pages and forms with boxes can be read out of order.
Always compare totals and key fields with the printed originals on a sample before trusting a pipeline.
04
A test before you rely on it
Take 30 scans from your real files, including the worst. Ask the model to transcribe each page first, then to answer your question from the transcript. Count the digits, dates and names that differ from the original. ANVE’s vision models read scanned documents (OCR), photos and diagrams, and the Bills maker reads GST bills; ANVE’s document answers show the source passage so a person can check, and ANVE has not published read-accuracy figures.
05
Questions people ask
Can a local model read handwriting?
Sometimes, with more errors than for print. Test on your own handwriting samples; see the question on handwritten documents.
Does reading images need a bigger GPU?
Slightly. The vision encoder is under 1 GB in the releases checked here, and each image adds tokens to the conversation, which uses cache memory, so the main cost is the language model itself.
Can it understand a chart or a diagram?
Vision-language models are built for it, and makers publish chart and diagram scores. The result depends on image quality and the question, so ask for the numbers it read, not just its conclusion.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
