| Speech to text: Whisper | Whisper large-v2, large-v3 | Arabic (ar) is one code in Whisper’s tokenizer. The paper counts 739 hours of Arabic speech with Arabic transcripts and 2,286 hours paired with English text. Large-v2 scored 16.0% word error rate on FLEURS and 53.8% on Common Voice 9, a spread that shows how much the test set matters. The large-v3 card warns of uneven accuracy across accents and dialects. |
|---|
| Speech to text: Gulf and other dialects | Whisper large-v3, Meta MMS | Casablanca study, zero-shot test sets: Whisper large-v3 scored 62.3% word error rate on Emirati speech (52.9% after the paper’s text clean-up), 69.9% on Yemeni, 48.4% on Jordanian and 87.2% on Moroccan; the average over its eight dialects was 69.5%. Meta’s MMS scored 85.4% on Emirati. On Saudi television speech (SADA), the best result in a 2025 study was MMS 1B fine-tuned on that data with a language model: 40.9%. |
|---|
| Speech to text: Arabic mixed with English | Whisper large-v3 | Casablanca’s segments with English words in Latin letters inside Arabic speech: 90.9% word error rate with automatic language detection, 131.5% when told the language is English and 103.6% when told it is Arabic. |
|---|
| Chat, summaries, drafting: general models | Llama 4, Qwen3, gpt-oss, Gemma | Llama 4’s card lists Arabic; Llama 3.3’s does not. Qwen3 lists eight Arabic varieties (Standard, Najdi, Levantine, Egyptian, Moroccan, Mesopotamian, Ta’izzi-Adeni, Tunisian). gpt-oss reports Arabic MMMLU of 82.7% (120b) and 76.3% (20b) at high reasoning effort. Gemma’s cards say over 140 languages without naming them. |
|---|
| Chat, summaries, drafting: models led from the Gulf | Fanar-2, Jais 2, ALLaM | Fanar-2-27B-Instruct (Qatar, Apache 2.0, built on Gemma 3) says it supports Modern Standard Arabic and Gulf, Levantine and Egyptian dialects. Its own card reports ArabicMMLU of 74.67% against 73.08% for Qwen3-32B and 72.21% for Gemma 3 27B, and dialectal-Arabic reading comprehension (Belebele) of 86.81% against 85.98% and 85.54%, with no breakdown by dialect; these are the vendor’s figures. Jais 2 (MBZUAI and Inception in the UAE, with Cerebras; Apache 2.0; 8B and 70B chat sizes; gated download) says it covers Modern Standard Arabic, regional dialects and Arabic-English mixing. ALLaM 7B (Saudi Arabia, listed as Apache 2.0) has a 4,096-token context. |
|---|
| Translation | NLLB-200 | FLORES-200 and NLLB-200 list ten Arabic entries covering nine varieties: Modern Standard Arabic (in Arabic script and romanised), Egyptian, Moroccan, Najdi, Mesopotamian, North and South Levantine, Tunisian and Ta’izzi-Adeni, but none labelled Gulf. NLLB-200’s card gives a CC-BY-NC licence, calls it a research model not released for production, and says its translations cannot be used as certified translations. |
|---|