The 4B question. Can a 4B-class model on a 50 TOPS NPU be trusted with your meeting's action items? We wrote the test before we wrote anything else.
On 24 August, on one transcript, the model we want in the entry-tier box assigned a task to the person who gave the instruction instead of the person who had to carry it out. For a minutes product that is the bug, not a bug.
So before choosing hardware we built a harness with the pass criteria written down first. This note is what the harness checks, what two small models scored on 22 September, and what that does and does not prove.
01
What the harness does
Five synthetic office meetings, in the language our buyers speak:
A CA firm's GST week.
A clinic's stock and NABH prep.
A college examination office.
A district office with a flood-relief list.
A two-partner law firm.
Each carries seven facts a summary must keep, four action items with a correct owner, one question the office policy document answers, and one it does not.
Rules fixed before the run:
The owner is the person who must do the task, never the person who asked.
Any number in the summary absent from the transcript counts as invented.
The model must answer "not in document" when the document does not answer.
One run per model, temperature 0.1, thinking off, on a laptop (Apple M3 Pro, via Ollama).
02
Results
Two models. Every number is from our own harness run on 22 September 2026.
Check
gemma3:4b
qwen2.5:3b-instruct
Facts recalled
34 of 35
32 of 35
Owners correct
19 of 20
18 of 20
Misassigned owners
0
1
Items missed
1
1
Valid JSON
5 of 5
5 of 5
Grounded answers
5 of 5
4 of 5
Trick questions declined
5 of 5
5 of 5
Invented numbers
0
0
Tokens per second (laptop)
40.9
47.6
The one gemma3:4b miss was an item not extracted at all: a bar-council fee the person said she would pay. It was not a wrong owner.
The qwen2.5:3b misassignment put a question-paper task on the office manager instead of the professor.
03
What changed
The 24 August defect did not reproduce across 20 items once the prompt states that the owner is the doer.
The fix is one sentence in the prompt plus a verification pass we now run on every extraction.
The 3B model still gets it wrong with the same sentence. That is why the specification says 4B-class minimum for the entry tier, and why we will not save a few thousand rupees of memory by going smaller.
04
What this does not prove
Five transcripts we wrote ourselves.
One run each.
Facts matched by token overlap, not by a person reading.
Laptop speed says nothing about the NPU in the box. That number comes from the prototype.
n=5 is a smoke test, not a benchmark.
We will re-run with real meetings from design partners, with their permission, and publish the harness so anyone can run it.
05
Why it matters for the product
The entry tier's whole promise is a meeting that becomes minutes with owners and dates, on a device that never sends the recording out.
If owners are wrong, the product tells the wrong person to act. That is worse than no minutes.
So this check gates the entry tier: zero misassigned owners on the release harness, or it does not ship.
06
What is not real yet
No sealed unit exists. Every image of the appliance on this site is a labelled render.
The software runs on a development machine and has been used in one real working setting.
No customers, no pre-orders confirmed. Selling has not started.
No certifications. No delivery date.
Final price not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing. The terms are two screens long and say so. If your office would let us test on two real meetings, we are taking ten design partners.
