Whisper An open speech-to-text model: what it does on office hardware, and where it fails.
Short answer. Whisper is an open speech-recognition model from OpenAI, trained on 680,000 hours of labelled audio, of which 117,000 hours cover 96 languages other than English. It comes in sizes from 39 million to 1.55 billion parameters; the GitHub README lists about 1 GB of VRAM for the smallest and about 10 GB for the large model, and about 6 GB for the 809-million-parameter turbo model. Whisper works on 30-second windows, so long recordings are processed in sequence. It can invent text that was not spoken, it does not label speakers, and its accuracy in a language tracks the amount of training data that language had.
01
Sizes and memory
OpenAI’s README gives these figures. Relative speed was measured on an A100 transcribing English and varies with language, speaker and hardware.
Size
Parameters
VRAM
Relative speed
tiny
39 million
About 1 GB
About 10 times
base
74 million
About 1 GB
About 7 times
small
244 million
About 2 GB
About 4 times
medium
769 million
About 5 GB
About 2 times
large
1,550 million
About 10 GB
1 time
turbo
809 million
About 6 GB
About 8 times
02
How it works
Whisper is an encoder-decoder transformer. Audio is resampled to 16,000 Hz and turned into a log-Mel spectrogram (80 channels, or 128 in large-v3). One model handles transcription, translation to English, language identification and voice activity detection, steered by special tokens at the start of its output. It reads 30-second windows, so for long audio it shifts the window using the timestamps it predicts.
OpenAI’s paper says that long-form transcription needed extra heuristics: beam search with 5 beams, raising the temperature when the average log probability falls below −1 or the text compresses too well (a sign of repetition), and a no-speech threshold of 0.6. It calls these a workaround for noisy predictions. The turbo model is large-v3 with the decoder cut from 32 layers to 4: faster, with a minor quality loss, and not trained for translation.
03
Hindi, Urdu, Tamil and English: what the paper reports
The 2022 paper lists the hours of speech-recognition training data for each language and reports word error rates for the large-v2 model on FLEURS, a read-speech test of about 12 hours per language, not meetings. The paper says that performance in a language is very well predicted by its amount of training data.
Language
Training hours in the 2022 paper
FLEURS word error rate, large-v2
English
438,218
4.2%
Hindi
12
21.5%
Urdu
104
22.6%
Tamil
136
17.5%
Marathi
0.6
38.3%
Telugu
4.3
99.0%
Gujarati
0.3
102.7%
04
Running it in an office
The faster-whisper project, a faster reimplementation, publishes a benchmark: 13 minutes of audio with large-v2 on an NVIDIA RTX 3070 Ti with 8 GB. OpenAI’s own code took 2 minutes 23 seconds and 4,708 MB of VRAM. faster-whisper took 1 minute 3 seconds and 4,525 MB at 16-bit, 59 seconds and 2,926 MB at 8-bit, and 17 seconds with a batch size of 8. Scaled to a one-hour meeting, the 16-bit run is roughly five minutes; that is an estimate from the benchmark, not a measurement. The same project’s small model took 1 minute 42 seconds at 8-bit on an Intel Core i7-12700K processor with no GPU.
faster-whisper can integrate the Silero VAD model to skip silence, and the project points to WhisperX for speaker diarization and word-level timestamps. These benchmarks use a gaming card and a desktop CPU, not any ANVE hardware.
05
What to check
Consent for recording, and where the audio and transcripts are kept.
Your language mix. Neither the paper nor the model cards report a test on code-switched speech.
Hallucination: the model card warns that predictions may include text not spoken in the audio, and that the model is prone to repeating itself.
Names, numbers and dates, which a person should check.
The licence of the exact checkpoint: the README says MIT for code and weights, the large-v3 card lists Apache 2.0, and the turbo card lists MIT.
Which model you need: turbo cannot translate; use medium or large for that.
06
Where ANVE stands
ANVE’s minutes maker produces transcripts in Hindi, English and Hinglish on a device in the office. Which speech model and settings it uses is a detail to get in writing before a paid pilot, and ANVE has published no speed figures. Our open-model pages cover Whisper large-v3 and large-v3-turbo.
07
Questions people ask
Is Whisper free for commercial use?
OpenAI’s README says Whisper’s code and weights are under the MIT licence, the large-v3-turbo card lists MIT, and the large-v3 card lists Apache 2.0. Read the licence file of the checkpoint you download. This is general information, not legal advice.
Can Whisper tell who is speaking?
No. The model card says Whisper may show some ability on tasks such as speaker diarization if fine-tuned, but it has not been robustly evaluated for them. Use a separate diarization tool.
Does Whisper handle Hinglish?
Neither the paper nor the model cards report a test on code-switched speech, and the model predicts a language token at the start of its output. Test it on your own recordings.
What is the difference between large-v3 and turbo?
Turbo has 809 million parameters against 1.55 billion, because its decoder was cut from 32 layers to 4. The README says it is faster with a minimal drop in accuracy and is not trained for translation.
08
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.