Can AI tell who said what in a meeting? It can tell voices apart. Putting names on them is still your job.
Short answer. Partly. A step called speaker diarization splits a recording into turns labelled speaker 1, speaker 2 and so on, and open tools do it offline. They label voices, not people, and they slip when people talk over one another. The open pyannote 3.1 pipeline had a diarization error rate of 18.8 per cent on AMI, a set of recorded meetings with headset microphones, and 24.4 per cent on AliMeeting. In plain terms, about a fifth to a quarter of speech time was missed, falsely detected or given to the wrong speaker. Putting names on the labels needs a person, and nobody should be assigned work on the strength of a label alone.
01
How speaker separation works
Diarization answers who spoke when, without knowing anyone’s name. The open toolkit pyannote.audio breaks the job into speech activity detection, speaker change detection, overlapped speech detection and speaker embeddings, which are numerical voice fingerprints. The output is anonymous labels such as SPEAKER_00 and SPEAKER_01.
WhisperX joins this to transcription. It runs Whisper, aligns each word to the audio with a wav2vec2 model, and uses pyannote to attach a speaker label to every word. pyannote’s pipelines are released under the MIT licence but gated: you accept the conditions and use a Hugging Face token once to download them, and they then run locally. They expect 16 kHz mono audio and convert other formats on their own.
NVIDIA’s streaming Sortformer model takes another route. Its card says it detects at most four speakers and gets worse on recordings with five or more.
02
How accurate it is
Diarization error rate adds up three things as a share of all the speech in the recording: speech the system missed, non-speech it called speech, and speech given to the wrong speaker. pyannote’s published scores count overlapping speech and allow no forgiveness margin around turn boundaries, which is the strict way to score. WhisperX’s own README is blunt about the limits: overlapping speech is not handled particularly well, and diarization is far from perfect.
Test set (pyannote 3.1)
Total error
Speech missed
Wrong speaker
AMI, headset microphones
18.8%
9.5%
5.7%
DIHARD 3
21.7%
8.1%
7.3%
AliMeeting
24.4%
10.0%
10.0%
VoxConverse
11.3%
3.4%
3.8%
AISHELL-4
12.2%
4.4%
4.0%
03
Limits, and what a person must check
Overlap. The error rate counts people talking over each other, and the pyannote.metrics documentation notes that a system without overlap detection can show more missed speech.
Speaker count. Compare the number of labels with the attendance list. One person can be split into two labels, or two people merged into one.
Names. Labels are anonymous. Mapping them to names takes a person or a stored voice sample, and the DPDP Act defines personal data as any data about an individual who is identifiable by or in relation to such data, which on its face covers a recording of an identifiable person’s voice. Tell participants first. This is general information, not legal advice.
Quotes and decisions. Before a statement is attributed to a named person in minutes, confirm who said it against the audio.
Your own room. The AMI figure above is for headset microphones. Test with the microphone set-up you will really use.
04
How an office would set this up
Record with the best microphones you can: one per person or a conference array rather than one phone on the table.
Note the attendance list with the recording. The number of speaker labels should match the number of people who spoke.
Run transcription and diarization together on the office machine, for example with WhisperX. The Whisper and pyannote models are downloaded once.
Show the transcript with labels, then let a person rename Speaker 1 to a real name once for the whole file.
Do not let any system assign tasks or quote people by label alone. Keep a link from every quote back to its transcript line.
Test on three of your own meetings and count the labelling mistakes before you rely on it.
05
What ANVE does today
ANVE does not list speaker identification as a function today. It does list a rule that covers the same risk: in the meeting minutes maker, every action item must quote the transcript line it came from, and a person has to confirm its owner before the item is approved. Each confirmation is logged.
The minutes maker works with transcripts in Hindi, English and Hinglish and runs in ANVE’s development build. No sealed unit exists yet.
06
Questions people ask
What is speaker diarization?
The step that decides who spoke when in a recording and labels each turn with an anonymous speaker label. It is separate from speech-to-text, and separate from recognising a named person. The glossary has a short entry.
Can AI recognise a named person’s voice?
Yes, if you supply a voice sample and use speaker verification or recognition models, which toolkits such as pyannote.audio include. That stores a voice template of an identifiable person, so get consent and keep it on your own machine.
How many speakers can it handle?
It depends on the tool. NVIDIA’s streaming Sortformer is limited to four speakers and gets worse with five or more. pyannote’s card states no hard limit, so test it with your usual meeting size.
Is a 20 per cent diarization error rate good enough?
For a searchable transcript that someone reviews, often yes. For attributing statements or tasks to named people, no: that needs a person checking the audio.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
