Twilio’s documentation says its media streams carry audio that is always 8,000 Hz mu-law, one channel, which is lower than the 16 kHz audio that Whisper, for one, resamples everything to. The Gram Vaani challenge, published at Interspeech 2022, is built on 1,108 hours of real, spontaneous telephone speech in regional variations of Hindi. Its baseline systems, a conventional model trained on 100 labelled hours and a neural conformer model, reached word error rates of 29.7 and 32.9 per cent. Those are 2022 baselines, not today’s best systems, so test current models on your own calls.
Callers also mix Hindi and English, give names and numbers, interrupt, and call from noisy places. Each adds errors, and the delay of three steps in a row adds up. An agent that replies late or talks over the caller feels broken even when its answers are right.