Dialectal Coverage And Generalization in Arabic Speech Recognition
Authors: Amirbek Djanibekov, Hawau Olamide Toyin, Raghad Alshalan, Abdullah Alitr, Hanan Aldarmaki
Accepted at: acl25

Dialectal Coverage And Generalization in Arabic Speech Recognition

Amirbek Djanibekov1*, Hawau Olamide Toyin1*
Raghad Alshalan2, Abdullah Alitr2, Hanan Aldarmaki1

1Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
2STC, Riyadh, Saudi Arabia
{amirbek.djanibekov,hawau.toyin,hanan.aldarmaki}@mbzuai.ac.ae
{rsalshalan,aalatir}@stc.com.sa

Abstract
Developing robust automatic speech recognition (ASR) systems for Arabic requires effective strategies to manage its diversity. Existing ASR systems mainly cover the modern standard Arabic (MSA) variety and few high-resource dialects, but fall short in coverage and generalization across the multitude of spoken variants. Code-switching with English and French is also common in different regions of the Arab world, which challenges the performance of monolingual Arabic models. In this work, we introduce a suite of ASR models optimized to effectively recognize multiple variants of spoken Arabic, including MSA, various dialects, and code-switching. We provide open-source pre-trained models that cover data from 17 Arabic-speaking countries, and fine-tuned MSA and dialectal ASR models that include at least 11 variants, as well as multi-lingual ASR models covering embedded languages in code-switched utterances. We evaluate ASR performance across these spoken varieties and demonstrate both coverage and performance gains compared to prior models.

1 Introduction
The advent of large self-supervised acoustic models has transformed speech technology, enabling transfer learning and improving performance for both high-resource and low-resource languages. Prominent examples of such models include various versions of wav2vec (Schneider et al., 2019; Baevski et al., 2020), HuBERT (Hsu et al., 2021), and SpeechT5 (Ao et al., 2022), which have predominantly been trained on English datasets. Their multi-lingual variants, e.g. XLS-R (Babu et al., 2022) with 53 and 128 languages, in addition to models that include both self-supervised and supervised pre-training, such as Whisper (Radford et al., 2023) with approximately hundred supported languages, MMS (Pratap et al., 2024) with thousands of languages, and UniSpeech (Wang et al., 2021), illustrate the potential for cross-lingual transfer learning for more inclusive ASR. Yet, while these models indeed show great potential for transfer learning to new languages, even those unseen in training, they remain suboptimal for some target languages. A case in point is the Arabic Text and Speech Transformer (ArTST), a model pre-trained exclusively on Arabic, which has demonstrated superior performance for Modern Standard Arabic (MSA), surpassing larger multilingual models like Whisper and MMS in benchmark tests, in addition to establishing a new state-of-the-art performance compared to previous efforts for Arabic ASR. This highlights the advantage of monolingual pre-training when large amounts of unlabeled data for the target language are available. While the model showed some potential for dialectal coverage, it was trained and validated exclusively on MSA data, which questions its applicability for spoken dialectal variants of Arabic. Evaluations on code-switched data showed poor performance of ArTST compared to multilingual models (Kadaoui et al., 2024), demonstrating the delicate trade-off between monolingual and multilingual optimization. Arabic is a pluricentric language (Schuppler et al., 2024), diverse in regional variations, and models trained on MSA frequently struggle to adapt to these variations. This limitation is particularly acute given that many Arabic dialects are underrepresented and considered low-resource in speech technology research. Consequently, there is a need for optimized ASR systems that embrace, rather than overlook, the linguistic diversity of the Arabic-speaking world.

In light of these challenges, we conduct various investigations aimed at understanding and enhancing the dialectal diversity and performance of Arabic ASR systems. We focus on four inquiries aimed at optimizing potential strategies for integrating dialectal variation into ASR systems. First, we mea-
Figure 1: The architecture of SpeechT5/ArTST, which contains an encoder-decoder module and six modal specific pre/post-nets. During self-supervised pre-training (left), quantized tokens are shared across speech and text modalities. Hidden states and latent units are mixed up and used as the inputs of the cross-attention module in the decoder. The fine-tuning stage for ASR is shown on the right. Refer to Ao et al. (2022) for more details.

sure the impact of incorporating a broad collection of Arabic dialects during the model’s pre-training phase. We hypothesize that a wider dialectal foundation could improve the model’s performance across various dialects in the fine-tuning stage. Second, we quantify the comparative effectiveness of dialect-specific fine-tuning versus a more holistic, multi-dialectal fine-tuning strategy. The third question examines the model’s capacity for zero-shot transfer to dialects not explicitly included in fine-tuning. Finally, we evaluate the model on code-switched utterances, and examine the effect of multilingual pre-training and fine-tuning on both monolingual and code-switched datasets. Our key findings from experiments spanning over 17 variants of spoken Arabic are: (1) pre-training with more data and wider dialectal coverage improves performance across most dialectal variants, including MSA, (2) multi-dialectal fine-tuning improves performance for low-resource dialects, but may not be optimal for high-resource dialects, (3) multi-dialectal pre-training and fine-tuning has higher potential for zero-shot transfer to unseen dialects, and (4) multilingual pre-training and fine-tuning greatly boosts performance on code-switching, while negatively impacting monolingual performance due to language interference. Our pretraining checkpoints and joint models were trained exclusively on open-source data and are released as open-source, open-weights models. All scripts required to reproduce our results including model training, evaluation, and checkpoints, are publicly available ¹.

² https://github.com/mbzuai-nlp/ArTST

2 Related Work

Arabic speech recognition research has a long history, but the majority of this research has focused on Modern Standard Arabic (MSA), the formal variant predominant in news broadcasts and official communications. A review article covering peer-reviewed publications between 2011 and 2021 estimates that 89% of papers on Arabic ASR cover MSA, and only a quarter cover some dialectal variant of Arabic (Dhouib et al., 2022). Recent research in end-to-end ASR for Arabic, as presented in Hussein et al. (2022), demonstrates the potential of contemporary deep learning techniques for decoding spoken Arabic, but it was also limited to MSA. Large-scale multilingual ASR models, such as Whisper (Radford et al., 2023) and MMS (Pratap et al., 2024), cover many languages within their scope, including Arabic. They utilize language embeddings or adapters to enhance language coverage and performance within the same model, but their performance across languages vary considerably. For instance, while Whisper demonstrates strong zero-shot capabilities on MSA, its zero-shot accuracy drops substantially on dialects, and additional finetuning on dialectal data is needed to improve performance (Waheed et al., 2023). Recent work has shown that smaller, Arabic-specific student models distilled from large models like Whisper can achieve comparable or even superior performance, especially on dialectal data, with good generalization to unseen dialects (Waheed et al., 2024). Specialized Arabic models like ArTST (Toyin et al., 2023), primarily pre-trained on MSA, have shown
competitive results on MSA tasks and even out-performed larger multilingual models on some mixed-dialect datasets like QASR (Mubarak et al., 2021). However, due to its monolingual pretraining, the model was shown to perform poorly in code-switched Arabic-English speech (Al Ali and Aldarmaki, 2024). This illustrates that strong MSA performance is not a reliable predictor for dialectal or code-switching capabilities, with a substantial gap persisting between SOTA MSA and dialectal performance. The development of diverse datasets such as QASR (multi-dialectal broadcast news) (Mubarak et al., 2021), SADA (Saudi Arabic) (Alharbi et al., 2024), ArZen (Egyptian-English code-switching) (Al-Sabbagh, 2024), and Mixat (Emirati-English code-switching) (Al Ali and Aldarmaki, 2024), and other public datasets covering various dialects and code-switched instances presents an opportunity for improving the generalization of ASR systems to diverse spoken varieties.

3 Methodology

Based on prior work, we start with the premise that monolingual training is more suitable for maximizing performance in Arabic ASR. However, the current Arabic SOTA models have limited coverage of spoken varieties and struggle with code-switching due to their monolingual training. Our objective is to maximize performance while also widening the coverage to include MSA, regional dialects, and instances of code-switching. To that end, we start with an Arabic-centric ASR model, ArTST (Toyin et al., 2023), as the foundation for our investigation. Figure 1 illustrates the high-level architecture of ArTST for self-supervised pre-training and fine-tuning. This model is based on the SpeechT5 approach (Ao et al., 2022), and supports multi-modal fine-tuning. The first version of the model was pre-trained on the MGB2 (Ali et al., 2016) dataset, which consists of newswire data, mainly in MSA, with a small subset of dialectal variants. In this work, we attempt to understand the factors that enable both high performance and wide coverage; we explore the following questions:

1. Is pretraining on dialectal data beneficial for improving down-stream dialectal performance, and would it negatively impact MSA performance?
2. Is it better to finetune ASR models jointly on multiple dialects or fine-tune on a specific target dialect?
3. Can we achieve reasonable zero-shot performance on unseen dialects?
4. Can we optimize performance in code-switched utterances using multilingual pre-training?
5. What is the effect of multilingual pretraining and finetuning on monolingual Arabic performance? (i.e. language interference).

The remaining sections detail our experimental settings and findings of these questions.

3.1 Terminology

For the rest of the paper, we will refer to Arabic variants using abbreviations. The categories below are based on regions and countries, and do not reflect any official classification of dialectal families:

MSA: Modern Standard Arabic. This is a common official variant of Arabic used in news, books, and education. CA: Classical Arabic. This is an old variant of Arabic found on religious texts and old books. It resembles MSA, but also contains outdated lexical items and structures.

GLF: A broad category of dialects spoken in the Arabian Peninsula, in particular the Gulf region, which, in our data sources, include SAU: Saudi, KUW: Kuwait, UAE, OMA: Oman, QAT: Qatar, IRA: Iraq, and YEM: Yemen.

LEV: Levantine dialects, which, in our data sources, include SYR: Syria, JOR: Jordan, LEB: Lebanon, and PAL: Palestine.

NOR: North African dialects, including EGY: Egypt, TUN: Tunisia, MOR: Morocco, ALG: Algeria, MAU: Mauritania, and SUD: Sudanese.

3.2 Pre-Training Data & Settings

To examine the effect of pre-training data coverage on downstream performance, we pre-trained ArTST from scratch\(^2\) on both MSA and dialectal data. We sourced our data from various datasets, including: MGB2 (Ali et al., 2016), QASR (Mubarak et al., 2021) MGB3 (Ali et al., 2017), MGB5 (Ali et al., 2019), CIARITS (Kulkarni et al., 2023), ASC (Halabi et al., 2016), and Common Voice (Ardila et al., 2020), SADA (Alharbi et al., 2024), and others. We also used MADAR (Bouamor et al., 2018) and NADI (Abdul-Mageed et al., 2023) text

\(^2\)We used the scripts and configurations provided in github.com/mbzua1-nlp/ArTST
<table>
  <tr>
    <th>Dataset</th>
    <th>Dialect</th>
    <th>Hours</th>
    <th>Words</th>
  </tr>
  <tr>
    <td>QASR</td>
    <td>MSA</td>
    <td>2000 hrs</td>
    <td>13.33 M</td>
  </tr>
  <tr>
    <td>MGB2</td>
    <td>MSA</td>
    <td>1000 hrs</td>
    <td>7.31 M</td>
  </tr>
  <tr>
    <td>MGB3[ASR]</td>
    <td>EGY</td>
    <td>2.83 hrs</td>
    <td>18.93 K</td>
  </tr>
  <tr>
    <td>MGB5[ASR]</td>
    <td>MOR</td>
    <td>6.74 hrs</td>
    <td>56.97 K</td>
  </tr>
  <tr>
    <td>SADA (Alharbi et al., 2024)</td>
    <td>SAU</td>
    <td>418 hrs</td>
    <td>3.25 M</td>
  </tr>
  <tr>
    <td>Mixat (Al Ali and Aldarmaki, 2024)</td>
    <td>UAE</td>
    <td>15 hrs</td>
    <td>57.94 K</td>
  </tr>
  <tr>
    <td>TARIC-SLU (Mdhaifar et al., 2024)</td>
    <td>TUN</td>
    <td>8 hrs</td>
    <td>72.00 K</td>
  </tr>
  <tr>
    <td rowspan="4">ParallelCorp (Almeman et al., 2013)</td>
    <td>MSA</td>
    <td>32 hrs</td>
    <td>30.66K</td>
  </tr>
  <tr>
    <td>GLF</td>
    <td>32 hrs</td>
    <td>27.26K</td>
  </tr>
  <tr>
    <td>LEV</td>
    <td>32 hrs</td>
    <td>18.43K</td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>32 hrs</td>
    <td>48.56K</td>
  </tr>
  <tr>
    <td rowspan="10">MASC (Al-Fetyani et al., 2021)</td>
    <td>MSA</td>
    <td>612.28 hrs</td>
    <td>3.80 M</td>
  </tr>
  <tr>
    <td>SAU</td>
    <td>452.24 hrs</td>
    <td>301.92 K</td>
  </tr>
  <tr>
    <td>SYR</td>
    <td>211.33 hrs</td>
    <td>1.06 M</td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>175.36 hrs</td>
    <td>1.03 M</td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>42.21 hrs</td>
    <td>330.83 K</td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>25.20 hrs</td>
    <td>155.76 K</td>
  </tr>
  <tr>
    <td>IRA</td>
    <td>17.37 hrs</td>
    <td>121.12 K</td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>12.17 hrs</td>
    <td>34.34 K</td>
  </tr>
  <tr>
    <td>Multiple</td>
    <td>10.57 hrs</td>
    <td>80.08 K</td>
  </tr>
  <tr>
    <td>UAE</td>
    <td>9.87 hrs</td>
    <td>6.42 K</td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>8.60 hrs</td>
    <td>58.38 K</td>
  </tr>
  <tr>
    <td>PAL</td>
    <td>6.17 hrs</td>
    <td>45.35 K</td>
  </tr>
  <tr>
    <td>KUW</td>
    <td>4.04 hrs</td>
    <td>32.37 K</td>
  </tr>
</table>

Table 1: Summary of Dataset Statistics for Fine-Tuning. Hours of Audio, Word Counts, and Associated Dialects. Multiple is mix of several dialects not neccessary from the listed dialects (no information from the source).

datasets for pretraining. In our experiments, we compare the following:

• v1: This variant is as described in Toyin et al. (2023), pretrained only on MSA.

• v2: In this variant, we use a mixture of MSA and dialectal data in pretraining.

• v3: In this variant, we use a mixture of MSA, dialectal, and multilingual data in pretraining.

See Table 12 in Appendix A for details of all the datasets used in pre-training.

3.3 Dialectal Fine-Tuning

The datasets we use for dialectal fine-tuning are shown in Table 1. We collected as many open-source data as needed to maximize coverage of dialects. Note that, for MGB5 and MGB3, as the data is based on YouTube videos, many of the originally referenced videos are no longer available, so at the time of our experiments, only 2.5 hours of training were available for MGB3 and 2 hours for MGB5. Furthermore, multi dialectal datasets, such as MASC (Al-Fetyani et al., 2021), have unbalanced representation of dialects. The high-resource dialects in our collection include SAU, SYR, EGY, and MSA; each has at least 200 hours of transcribed ASR data. UAE, MOR, JOR, LEB, IRA, and TUN have a medium amount of fine-tuning data between 10 and 50 hours. KUW and PAL are low-resource dialects with less than 10 hours of transcribed data in total. Finally, we left ALG, YEM, and SUD from the MASC dataset for zero-shot testing.

Figure 2 illustrates the distribution of dialectal data we use for pre-training and fine-tuning our dialectal model. We exclude MSA from the figure as it has disproportionally more data than all dialects.

![Bar chart showing distribution of dialectal speech data in pre-training and fine-tuning](page_1012_186_388_246.png)

Figure 2: Distribution of dialectal speech data in pre-training and fine-tuning. MSA data are not shown.

3.4 Multi-Lingual Fine-Tuning

In addition to the above dialectal data, we used English, French, and Spanish sets for multi-lingual fine-tuning and code-switching experiments described in section 8. English and French are commonly spoken in various Arabic-speaking countries, and to a lesser extent, Spanish is spoken in some parts of North Africa. More details about the datasets used in these experiments are provided in section 8.

3.5 Experimental Settings

For partitioning the data into training, development, and test sets, we adhered to the official splits provided with each dataset. We followed the data preparation and training methodology established in the original ArTST implementation. For comprehensive details regarding the model architecture and data preprocessing, readers are directed to Toyin et al. (2023).

Computational Details The pre-training was executed on a cluster of 4 A100 GPUs over a duration of 14 to 21 days for each model. We used Adam optimizer with a learning rate of \( 2 \times 10^{-4} \), spanning 335K updates, and a warm-up phase of 64K updates. The maximum speech token length was set at 250K (equivalent to 15.625 seconds). Each
fine-tuning experiment was run on one A100 GPU over a duration of 7 days (MGB2, QASR, MASC, SADA) or 2 days for smaller sets (MGB3, MGB5, etc.). We used Adam optimizer with a tri-stage scheduler with learning rate of \(6 \times 10^{-5}\). The total computational budget for all experiments is estimated to be ~6000 GPU-hours.

Normalization Prior to model training, we implemented the same data normalization steps outlined in Toyin et al. (2023). In addition, we applied post-prediction normalization steps before calculating Word Error Rates (WER), following standard practices in Arabic ASR. All reported results reflect post-normalization performance. The normalization script, sourced from a publicly available GitHub repository\footnote{github.com/iamjazzar/arabic-nlp/blob/master/normalization/orthographic_normalization.py}, performs orthographic standardization of *Alef*, *Yaa*, and *Taa* characters.

4 Effect of Pre-Training Data

We first examine the effect of pre-training on downstream ASR performance. As described in section 3.2, we compare a model pre-trained mainly on MSA (ArTST v1), and our multi-dialectal version (henceforth v2). Note that pre-training does not utilize aligned speech and text; it incorporates unaligned speech and text data for self-supervised learning. For these experiments, we use the same finetuning data, and only vary the pretraining sets.

4.1 Benchmarking MSA Performance

We first report results on benchmark datasets to compare the performance of both models against the state-of-the-art. MGB2 is the main benchmark for MSA speech recognition. We evaluated the performance of ArTST v1 and v2 fine-tuned in MGB2, compared to existing SOTA models, in Table 2. The results show that incorporating dialectal data in pretraining does not negatively affect MSA performance, as we achieve the best WER of 12.48% and 12.39%, with and without LM fusion.

<table>
  <tr>
    <th>System</th>
    <th>WER(%)</th>
    <th>CER(%)</th>
  </tr>
  <tr>
    <td colspan="3">From (Hussein et al., 2022):</td>
  </tr>
  <tr>
    <td>HMM-DNN</td>
    <td>15.80</td>
    <td>—</td>
  </tr>
  <tr>
    <td>E2E, CTC + LM</td>
    <td>16.90</td>
    <td>—</td>
  </tr>
  <tr>
    <td>E2E, Attention + LM</td>
    <td>13.40</td>
    <td>—</td>
  </tr>
  <tr>
    <td>E2E, CTC , Attention + LM</td>
    <td>12.50</td>
    <td>—</td>
  </tr>
  <tr>
    <td>ArTST v1 + LM (Toyin et al., 2023)</td>
    <td>12.78</td>
    <td>6.33</td>
  </tr>
  <tr>
    <td>v2</td>
    <td>12.49</td>
    <td>6.44</td>
  </tr>
  <tr>
    <td>v2 + LM</td>
    <td><b>12.39</b></td>
    <td>6.51</td>
  </tr>
</table>

Table 2: Comparing our performance against models reported in Hussein et al. (2022) and Toyin et al. (2023), which include the best performing models previously reported on MGB2.

<table>
  <tr>
    <th>System</th>
    <th>Adaptation</th>
    <th>Fine-Tuning</th>
    <th>AV-WER</th>
    <th>MR-WER</th>
  </tr>
  <tr>
    <td>Aalto</td>
    <td>MGB2</td>
    <td>MGB3</td>
    <td>37.50</td>
    <td>29.30</td>
  </tr>
  <tr>
    <td rowspan="2">Whisper</td>
    <td>ComVoice</td>
    <td rowspan="2">MGB3</td>
    <td rowspan="2">39.04</td>
    <td rowspan="2">24.92</td>
  </tr>
  <tr>
    <td>Fleurs Covost2</td>
  </tr>
  <tr>
    <td rowspan="2">MMS</td>
    <td>BibleTrans NewTestamentRec</td>
    <td>MGB3</td>
    <td>100.04</td>
    <td>99.92</td>
  </tr>
  <tr>
    <td>v1</td>
    <td>MGB2</td>
    <td>MGB3</td>
    <td>37.08</td>
    <td>29.39</td>
  </tr>
  <tr>
    <td>v2</td>
    <td>MGB2</td>
    <td>MGB3</td>
    <td><b>33.20</b></td>
    <td><b>25.28</b></td>
  </tr>
</table>

Table 3: WER(%) on MGB3 Egyptian ASR. Aalto is the best system in the MGB3 challenge (Ali et al., 2017)

<table>
  <tr>
    <th>System</th>
    <th>Adaptation</th>
    <th>Fine-Tuning</th>
    <th>AV-WER</th>
    <th>MR-WER</th>
  </tr>
  <tr>
    <td>RDI-CU</td>
    <td>MGB2</td>
    <td>MGB5</td>
    <td>59.40</td>
    <td>37.60</td>
  </tr>
  <tr>
    <td rowspan="2">Whisper</td>
    <td>ComVoice</td>
    <td rowspan="2">MGB5</td>
    <td rowspan="2">164.13</td>
    <td rowspan="2">227.34</td>
  </tr>
  <tr>
    <td>Fleurs Covost2</td>
  </tr>
  <tr>
    <td rowspan="2">MMS</td>
    <td>BibleTrans NewTestamentRec</td>
    <td>MGB5</td>
    <td>111.89</td>
    <td>102.30</td>
  </tr>
  <tr>
    <td>v1</td>
    <td>MGB2</td>
    <td>MGB5</td>
    <td>49.39</td>
    <td><b>27.95</b></td>
  </tr>
  <tr>
    <td>v2</td>
    <td>MGB2</td>
    <td>MGB5</td>
    <td><b>48.91</b></td>
    <td>28.02</td>
  </tr>
</table>

Table 4: WER(%) on Moroccan ASR. RDI-CU is the best system in the MGB5 challenge (Ali et al., 2019)

as well as the SOTA model in each benchmark. Each model is first fine-tuned on MSA, then fine-tuned again on the target MGB train sets. We also report the results of the large multilingual models: Whisper (Radford et al., 2023) and MMS (Pratap et al., 2024), fine-tuned on the same set. We refer to the Arabic data the models are previously fine-tuned on as 'Adaptation' data. Starting with MSA data before fine-tuning on the target dialect has previously been established as an effective strategy for dialectal ASR (Ali et al., 2017).

In MGB3, dialectal pretraining (v2) results in about 4% absolute reduction in WER, establishing a new SOTA result on this benchmark. Smaller improvement in terms of Average WER is observed
<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th colspan="4">Zero-Shot</th>
    <th colspan="2">Fine-Tuning</th>
  </tr>
  <tr>
    <th>Whisper</th>
    <th>MMS</th>
    <th>v1</th>
    <th>v2</th>
    <th>v1</th>
    <th>v2</th>
  </tr>
  <tr>
    <td>TARIC-SLU (TUN)</td>
    <td>138.14</td>
    <td>93.54</td>
    <td>107.56</td>
    <td>106.46</td>
    <td><b>14.70</b></td>
    <td>14.80</td>
  </tr>
  <tr>
    <td>ParallelCorp (MULT)</td>
    <td>99.17</td>
    <td>83.16</td>
    <td>128.72</td>
    <td>141.92</td>
    <td>9.57</td>
    <td><b>9.31</b></td>
  </tr>
  <tr>
    <td>SADA (SAU)</td>
    <td>82.16</td>
    <td>78.28</td>
    <td>39.41</td>
    <td><b>29.77</b></td>
    <td>39.24</td>
    <td>29.91</td>
  </tr>
  <tr>
    <th colspan="7">MASC</th>
  </tr>
  <tr>
    <td>SAU</td>
    <td>48.39</td>
    <td>65.30</td>
    <td>61.23</td>
    <td>58.72</td>
    <td>27.40</td>
    <td><b>27.33</b></td>
  </tr>
  <tr>
    <td>SYR</td>
    <td>26.65</td>
    <td>33.21</td>
    <td>21.99</td>
    <td>18.37</td>
    <td>18.64</td>
    <td><b>17.42</b></td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>41.73</td>
    <td>66.04</td>
    <td>50.87</td>
    <td>47.17</td>
    <td>38.47</td>
    <td><b>36.43</b></td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>28.65</td>
    <td>54.63</td>
    <td>61.23</td>
    <td>34.97</td>
    <td><b>19.72</b></td>
    <td>21.08</td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>40.95</td>
    <td>64.58</td>
    <td>35.65</td>
    <td>42.66</td>
    <td>30.01</td>
    <td><b>28.05</b></td>
  </tr>
  <tr>
    <td>IRA</td>
    <td>41.69</td>
    <td>59.33</td>
    <td>50.46</td>
    <td>48.03</td>
    <td><b>31.10</b></td>
    <td>34.64</td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>47.45</td>
    <td>60.58</td>
    <td>50.37</td>
    <td>46.67</td>
    <td>19.26</td>
    <td><b>18.52</b></td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>65.87</td>
    <td>80.84</td>
    <td>78.92</td>
    <td>66.87</td>
    <td><b>47.59</b></td>
    <td>49.40</td>
  </tr>
  <tr>
    <td>PAL</td>
    <td><b>53.20</b></td>
    <td>83.72</td>
    <td>77.94</td>
    <td>73.53</td>
    <td>55.88</td>
    <td>53.53</td>
  </tr>
  <tr>
    <td>KUW</td>
    <td><b>36.00</b></td>
    <td>81.71</td>
    <td>64.74</td>
    <td>52.02</td>
    <td>50.29</td>
    <td>46.24</td>
  </tr>
</table>

Table 5: WER (%) in zero-shot and fine-tuning settings. We compare zero-shot performance of Whisper, MMS, ArTST v1, and Our dialectal pretraining (v2). ArTST v1 and v2 are finetuned on MGB2 (MSA), whereas Whisper and MMS are finetuned/pretrained with multi-lingual data, including Arabic.

for MGB5, where there is no clear advantage observed using our dialectal version. This difference is likely a result of our pre-training having a lot more Egyptian than Moroccan data (see Figure 2).

4.3 Zero-Shot & Fine-Tuning Results

To further quantify the effect of dialectal pretraining, we evaluate the performance of our model across different datasets. We first fine-tune models on MSA using MGB2 dataset. We test the model performance on dialects directly (zero-shot) and with dialectal fine-tuning. The results are shown in Table 5. On average, we see improvements in performance in both zero-shot and fine-tuning experiments using dialectal pretraining (v2) compared to MSA-centric pretraining (v1). We also see that both models perform better than Whisper and MMS in zero-shot settings in most cases. There are some exceptions, such as in KUW, where Whisper performs better than all other models, including the fine-tuned models, but in most cases v2 performs best. This underscores the advantage of monolingual models compared to multilingual performance, as observed in Toyin et al. (2023) and Radford et al. (2023). In addition, the results underscore the importance of dialectal coverage in pretraining: the cases where v2 performs worse than v1 are all dialects for which pretraining data are limited, such as TUN (no pretraining data) and JOR (smallest dialect size in pretraining).

5 Joint Models & Dialect ID

So far, models were first fine-tuned on MSA, followed by additional fine-tuning on each target dialect. This results in a separate model per dialect, which incurs memory costs and may have practical limitations as it requires advance knowledge of the dialect ID for deploying the correct model.

In this section, we assess the relative effectiveness of individual dialectal fine-tuning compared with joint dialect fine-tuning, where we train a single model for all dialects. To that end, we joined multiple dialectal train sets from MASC, as shown in Table 6. We excluded ALG, YEM, SUD for zero-shot evaluation. The resulting joint corpus consists of 12 dialects including MSA, with approximately 1,501 hours in total. We fine-tuned a single joint model using this data.

<table>
  <tr>
    <th>Dialect</th>
    <th>Hours</th>
    <th>Words</th>
    <th>Source</th>
  </tr>
  <tr>
    <td>MSA</td>
    <td>612.28 hrs</td>
    <td>3.80 M</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>SAU</td>
    <td>452.24 hrs</td>
    <td>301.92 K</td>
    <td>SADA, MASC</td>
  </tr>
  <tr>
    <td>SYR</td>
    <td>211.33 hrs</td>
    <td>1.06 M</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>175.36 hrs</td>
    <td>1.03 M</td>
    <td>MGB3, MASC</td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>42.21 hrs</td>
    <td>330.83 K</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>25.20 hrs</td>
    <td>155.76 K</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>IRA</td>
    <td>17.37 hrs</td>
    <td>121.12 K</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>12.17 hrs</td>
    <td>34.34 K</td>
    <td>TARIC-SLU, MASC</td>
  </tr>
  <tr>
    <td>UAE</td>
    <td>9.87 hrs</td>
    <td>6.42 K</td>
    <td>Mixat, MASC</td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>8.60 hrs</td>
    <td>58.38 K</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>PAL</td>
    <td>6.17 hrs</td>
    <td>45.35 K</td>
    <td>MASC</td>
  </tr>
  <tr>
    <td>KUW</td>
    <td>4.04 hrs</td>
    <td>32.37 K</td>
    <td>MASC</td>
  </tr>
</table>

Table 6: Datasets used to train the joint models.
<table>
  <tr>
    <th rowspan="2">Approach</th>
    <th colspan="2">Zero-Shot</th>
    <th colspan="2">Fine-Tuning</th>
    <th colspan="3">Joint Multi-Dialectal Set (Table 6)</th>
  </tr>
  <tr>
    <th>MG2B</th>
    <th>QASR</th>
    <th>MG2B→Target</th>
    <th>No Dialect ID</th>
    <th>Dialect Forcing</th>
    <th>Dialect Inference</th>
  </tr>
  <tr>
    <td>SAU</td>
    <td>58.72</td>
    <td>43.41</td>
    <td><b>27.33</b></td>
    <td>29.41</td>
    <td>30.56</td>
    <td>29.94</td>
  </tr>
  <tr>
    <td>SYR</td>
    <td>18.37</td>
    <td><b>16.20</b></td>
    <td>17.42</td>
    <td>19.20</td>
    <td>22.41</td>
    <td>20.30</td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>47.17</td>
    <td><b>38.78</b></td>
    <td>36.43</td>
    <td>45.17</td>
    <td>61.06</td>
    <td>46.79</td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>34.97</td>
    <td>25.42</td>
    <td>21.08</td>
    <td><b>19.63</b></td>
    <td>21.49</td>
    <td>20.11</td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>42.66</td>
    <td>40.51</td>
    <td>28.05</td>
    <td>28.22</td>
    <td>29.43</td>
    <td><b>26.89</b></td>
  </tr>
  <tr>
    <td>IRA</td>
    <td>48.03</td>
    <td>40.27</td>
    <td>36.10</td>
    <td><b>29.33</b></td>
    <td>31.75</td>
    <td>30.83</td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>46.67</td>
    <td>45.93</td>
    <td><b>26.67</b></td>
    <td>37.23</td>
    <td>28.47</td>
    <td>27.74</td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>66.87</td>
    <td>55.42</td>
    <td>56.63</td>
    <td>57.49</td>
    <td>53.89</td>
    <td><b>49.10</b></td>
  </tr>
  <tr>
    <td>PAL</td>
    <td>73.53</td>
    <td>45.59</td>
    <td>53.53</td>
    <td>46.22</td>
    <td><b>43.90</b></td>
    <td>44.48</td>
  </tr>
  <tr>
    <td>KUW</td>
    <td>52.02</td>
    <td>45.09</td>
    <td>46.24</td>
    <td><b>35.43</b></td>
    <td>39.43</td>
    <td>37.71</td>
  </tr>
  <tr>
    <td>MSA</td>
    <td>21.09</td>
    <td>16.78</td>
    <td>15.34</td>
    <td><b>11.59</b></td>
    <td>12.66</td>
    <td>12.09</td>
  </tr>
  <tr>
    <td>Macro Average</td>
    <td>46.37</td>
    <td>37.58</td>
    <td>33.17</td>
    <td>32.63</td>
    <td>34.09</td>
    <td><b>31.45</b></td>
  </tr>
</table>

Table 7: WER (%) of various models compared with joint dialectal fine-tuning with different dialect ID strategies.

5.1 Dialect ID

We trained another model with the aforementioned joint dataset, but with the inclusion of explicit dialect identifiers. We augmented the dictionary with special tokens for dialect IDs, and used them to prepend the decoding string:

<ss> DIALECT T_1 T_2 ... T_n </ss>

For inference, we experimented with two strategies: (1) Transcribing with dialect forcing, where we manually add the dialect ID to condition the decoder output; the decoder is forced to start predictions with the tokens <ss> DIALECT . (2) Transcribing with dialect inference, where we let the model predict the dialect token. We use this approach for zero-shot ASR on unseen dialects (Table 9).

The results of the models trained with joint dialects compared to models trained on MG2B and QASR are shown in Table 7. Note that both MG2B and QASR contain mostly MSA, but also a small amounts of various dialects, but their exact distribution is unknown. We also reproduce the fine-tuning results from Table 5 for easy comparison. We see that joint modeling results in improvement for low-resource dialects, including: JOR, TUN, and KUW, but degrades performance of the high-resource SYR and EGY dialects. Interestingly, dialect forcing was worse on average than joint modeling with no dialect ID, while dialect inference resulted in the best performance overall. We surmise that the model learns dialectal patterns that do not perfectly align with the dialect ID as indicated in the training data. Since the dialect IDs are coarse country-level approximations, letting the model infer the dialect based on the speech is the best approach for most cases. Many dialectal sets, such as SYR and SAU, contain a lot of MSA utterances that are incorrectly identified as dialectal.

Figure 3 illustrates dialect inference errors. Note that the number of errors is proportional to the test data size. The overall dialect identification performance is around 90%. Some low-resource dialects, such as KUW, are predicted as their closest high-resource variant, such as SAU, resulting in worse performance compared to joint models without a dialect ID, but on average, dialect forcing leads to the lowest WER.

![Dialect identification performance: true (left), predicted (right). All lines are proportional to their ratio over the total errors except for SAU→MSA, which is reduced 5 times for clarity.](page_1012_1342_420_312.png)

Figure 3: Dialect identification performance: true (left), predicted (right). All lines are proportional to their ratio over the total errors except for SAU→MSA, which is reduced 5 times for clarity.

6 Effect of Data Adaptation

In the above experiments, we followed the strategy of initializing the models by first fine-tuning on MSA data. In most cases, we used MG2B as the base model, following previously established results on Egyptian ASR (Ali et al., 2017). This adaptation approach is meant to enhance the performance on low-resource dialects, facilitating faster
<table>
  <tr>
    <th rowspan="2">Fine-Tuning</th>
    <th colspan="3">Adaptation</th>
  </tr>
  <tr>
    <th>—</th>
    <th>MGB2</th>
    <th>QASR</th>
  </tr>
  <tr>
    <td>MGB3</td>
    <td>136.52</td>
    <td>25.28</td>
    <td><b>19.53</b></td>
  </tr>
  <tr>
    <td>MGB5</td>
    <td>94.84</td>
    <td>28.02</td>
    <td><b>27.58</b></td>
  </tr>
  <tr>
    <td>TARIC-SLU</td>
    <td>30.46</td>
    <td>14.80</td>
    <td><b>14.48</b></td>
  </tr>
  <tr>
    <td>ParallelCorp</td>
    <td>28.97</td>
    <td>9.31</td>
    <td><b>9.08</b></td>
  </tr>
  <tr>
    <td>SADA</td>
    <td>29.77</td>
    <td>29.91</td>
    <td><b>29.55</b></td>
  </tr>
  <tr>
    <td>Mixat</td>
    <td>100.0</td>
    <td><b>33.40</b></td>
    <td>35.21</td>
  </tr>
</table>

Table 8: WER (%) of fine-tuned models on various datasets with different adaptation methods: —: no adaptation, MGB2, or QASR.

convergence with limited training samples. However, as pre-training on more diverse sets proved to be effective, adaptation on more diverse data is also likely to be fruitful. As observed in Table 7, models trained on QASR resulted in far better zero-shot performance, approaching the performance of joint-dialects models. This is attributed to the fact that QASR is both larger in size and known to have more dialectal data compared to MGB2 (but both have no documented statistics of dialectal coverage). To validate this observation, we experimented with dialectal fine-tuning adapted from two variants: one based on MGB2 and one based on QASR (Mubarak et al., 2021), followed by dialect-specific fine-tuning. Table 4 shows fine-tuning results with no adaptation (directly fine-tuning on the target dialect), compared with starting from MGB2 or QASR. First, our results corroborate the previous findings that adapting models from MSA results in large reduction in error rates. In all except the Mixat dataset, starting from QASR results in better performance compared to MGB2. However, the difference is negligible except on MGB3 Egyptian set (around 6% absolute WER reduction). There are two factors that we speculate underline this result: The small size of the MGB3 set, and the existence of Egyptian dialect in the QASR corpus more substantially than the other dialects. Overall, using the QASR dataset as a basis for adapting dialectal models is recommended as it improves or maintains performance.

7 Zero-Shot Performance

We show the zero-shot performance on the three held-out sets: ALG, SUD, and YEM. We compare the baseline, v1, with multi-dialectal pretraining (v2). We also compare models fine-tuned on MGB2, QASR, or our joint dialectal set. The results are shown in Table 9. Our model achieves slightly lower error rates compared to v1, even when fine-tuned on the same MGB2 set. Better performance is achieved with QASR, which includes some dialectal data. The joint dialectal fine-tuning achieves the best performance on the held-out dialects. In general, performance in held-out sets is on a par with low-resource dialects, with WER above 30%. Table 13 in the Appendix shows the zero-shot performance after fine-tuning on a single target dialect.

<table>
  <tr>
    <th rowspan="2">System</th>
    <th colspan="3">Dialect</th>
  </tr>
  <tr>
    <th>ALG</th>
    <th>SUD</th>
    <th>YEM</th>
  </tr>
  <tr>
    <td>ArTST v1→MGB2</td>
    <td>73.18</td>
    <td>69.20</td>
    <td>41.64</td>
  </tr>
  <tr>
    <td>v2→MGB2</td>
    <td>70.82</td>
    <td>69.31</td>
    <td>39.45</td>
  </tr>
  <tr>
    <td>v2→QASR</td>
    <td>51.72</td>
    <td>46.64</td>
    <td>34.78</td>
  </tr>
  <tr>
    <td>v2→Joint</td>
    <td><b>45.20</b></td>
    <td>40.69</td>
    <td>33.08</td>
  </tr>
  <tr>
    <td>v2→Joint (w. dialect inference)</td>
    <td>47.12</td>
    <td><b>40.15</b></td>
    <td><b>31.84</b></td>
  </tr>
</table>

Table 9: WER% of various models on held-out dialects.

8 Code-Switching Performance

The models analyzed so far were trained exclusively on Arabic data. While small amounts of code-switching (CS) exist in these sources, they are insufficient to learn the characteristics of the embedded languages. Large multi-lingual models are generally more effective on CS data (Kadaoui et al., 2024), even if they are less competent on monolingual Arabic. To make our models more inclusive, improving performance in the presence of code-switching is necessary. To that effect, we train a multilingual version of the model (we will refer to this as v3). The pre-training data for this version are listed in Table 12 in the Appendix. We test v3 against v1 and v2 on available CS data for Arabic: ArZN (Al-Sabbagh, 2024) for Egyptian-English speech, Mixat (Al Ali and Aldarmaki, 2024) for Emirati-English speech, and TunSwitch (Abdallah et al., 2024) for Tunisian-French speech. We also train a joint multi-lingual model (without dialect or language ID). In addition to the datasets described in Table 6, we add the multi-lingual and code-switching data shown in Table 10.

In Table 11, we show the performance of models finetuned directly from the pretrained checkpoints, or finetuned from existing ASR checkpoints (MGB2 checkpoint for v1, joint multi-dialectal checkpoint for v2, and joint multi-lingual checkpoint for v3). First, for models fine-tuned directly on the target set, we observe that multilingual pretraining significantly improves performance across all CS test sets, resulting in more than 10% ab-
<table>
  <tr>
    <th>Languages</th>
    <th>Hours</th>
    <th>Words</th>
    <th>Source</th>
  </tr>
  <tr>
    <td>EN</td>
    <td>1601.92 hrs</td>
    <td>10.35 M</td>
    <td>CommonVoice</td>
  </tr>
  <tr>
    <td>FR</td>
    <td>732.02 hrs</td>
    <td>5.03 M</td>
    <td>CommonVoice</td>
  </tr>
  <tr>
    <td>SP</td>
    <td>408.34 hrs</td>
    <td>2.79 M</td>
    <td>CommonVoice</td>
  </tr>
  <tr>
    <td>TUN-FR</td>
    <td>10.89 hrs</td>
    <td>70.86 K</td>
    <td>TunSwitch</td>
  </tr>
  <tr>
    <td>UAE-EN</td>
    <td>8.97 hrs</td>
    <td>57.82 K</td>
    <td>Mixat</td>
  </tr>
  <tr>
    <td>EGY-EN</td>
    <td>5.61 hrs</td>
    <td>52.00 K</td>
    <td>ArzEn</td>
  </tr>
</table>

Table 10: Additional datasets used to train the joint multilingual model.

<table>
  <tr>
    <th rowspan="2">Adaptation data</th>
    <th colspan="3">MSA</th>
    <th colspan="3">Dialectal</th>
    <th colspan="3">Multilingual</th>
  </tr>
  <tr>
    <th>v1</th>
    <th>v2</th>
    <th>v3</th>
    <th>v1</th>
    <th>v2</th>
    <th>v3</th>
  </tr>
  <tr>
    <td>Test Set</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MGB2</td>
    <td>13.42</td>
    <td>12.5</td>
    <td>13.0</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ArzEn</td>
    <td>43.21</td>
    <td>77.59</td>
    <td>35.26</td>
    <td>34.85</td>
    <td>33.71</td>
    <td>27.43</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>TunSwitch</td>
    <td>53.85</td>
    <td>101.94</td>
    <td>40.68</td>
    <td>43.87</td>
    <td>43.59</td>
    <td>36.66</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Mixat</td>
    <td>42.50</td>
    <td>92.41</td>
    <td>34.27</td>
    <td>27.07</td>
    <td>25.73</td>
    <td>21.66</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 11: ASR Results using the various checkpoints: v1, v2 and v3. We compare models trained directly from the pretrained checkpoint vs. starting with an ASR checkpoint trained with the specified adaptation data: MSA adaptation using the MGB2 dataset; Dialectal adaptation using data listed in Table 6; Multilingual adaptation using data from Table 6 and Table 10.

solute reductions in WER for all test sets. This clearly illustrates the advantage of multi-lingual pretraining in code-switching scenarios. We also evaluated models initialized from the joint models followed by target fine-tuning on the CS train sets, and this reduced error rates further. The best performing model is the joint multilingual v3 mode, with 4 to 7% absolute reduction in WER compared to the second best model. We show examples of ASR outputs from the three CS datasets using the various models in Figure 6 in the Appendix.

Language Interference: We test the effect of multi-lingual pre-training on MSA performance. Language interference is known to negatively affect monolingual performance (Toyin et al., 2023) so we test our multilingual model on the MGB2 benchmark to quantify this effect (see Table 11). The model achieves 13.0% WER, which is indeed worse than the SOTA result we achieve with the Arabic-only model (see Table 2), but the difference at 0.5% absolute WER is rather small. When it comes to dialects, however, we find that language interference has a significant negative effect, resulting in 4% to 16% absolute increase in error rates, as shown in Figure 4.

![WER (%) and absolute difference on a subset of dialects, comparing our joint dialectal fine-tuning vs. joint multi-lingual fine-tuning on Arabic dialects.](page_1012_186_410_312.png)

Figure 4: WER (%) and absolute difference on a subset of dialects, comparing our joint dialectal fine-tuning vs. joint multi-lingual fine-tuning on Arabic dialects.

various training paradigms on ASR performance. We compared models pre-trained with and without dialects, in high, low, and medium-resource settings, in addition to zero-shot. We find that overall, dialectal pre-training improves performance in zero-shot and low-resource cases, and mostly maintains performance on MSA and high-resource dialects. We also find that all dialects benefit from adaptation of models pre-fine-tuned on MSA, and this effect is most noticeable for low and medium-resource dialects. We experimented with multi-dialectal fine-tuning, where we joined the train sets of 12 dialects. We observe performance improvements on average, and at least the same performance as the target-dialect fine-tuning setting, and the best performance on held-out dialects. Interestingly, while using dialect ID in decoding is effective, forcing the dialect ID results in worse performance compared to dialect inference. While joint training results in improved performance for the medium and low-resource dialects, target-dialect fine-tuning is more effective for high-resource dialects. Finally, we experimented with multi-lingual pre-training and fine-tuning for improving performance on code-switched utterances, and achieved significant reductions in error rates on all available test sets. However, reductions in monolingual performance were also observed due to language interference, particularly for dialects, were WER increased by 4% to 16% for some dialects. To enable easier adoption and further experiments, we released the pretrained dialectal and multilingual checkpoints, the fine-tuned MGB2 models, and the joint dialectal and multilingual ASR models.

9 Conclusions

We presented the largest study on dialectal Arabic ASR to empirically demonstrate the effect of
Limitations

One of the limitations in dialect-related work is the coarse classification of dialect IDs; dialects in our datasets are classified by regions or countries, whereas actual dialectal variations are far more fine-grained. For example, the Saudi dataset, SADA, covers a large geographical area and many dialects, but it is considered as one dialect based on our classification. Moreover, the way the datasets are collected do not guarantee that the data are indeed dialectal. For instance, with manual inspection of the Syrian test and dev sets from MASC, we observed that all instances are in MSA rather than Syrian dialects. In addition, Arabic dialects are spoken varieties that do not have a standard spelling system. This results in large variations in transcriptions, but standard WER does not account for these variations, resulting in more pessimistic results. With the exception of the MGB3 and MGB5 benchmarks where we report average and multi-reference WER across 4 references, all datasets have only a single reference.

References

Ahmed Amine Ben Abdallah, Ata Kabboudi, Amir Kanoun, and Salah Zaiem. 2024. Leveraging data collection and unsupervised learning for code-switched tunisian arabic automatic speech recognition. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12607–12611. IEEE.

Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The fourth nuanced Arabic dialect identification shared task. In Proceedings of ArabicNLP 2023, Singapore (Hybrid). Association for Computational Linguistics.

Maryam Khalifa Al Ali and Hanan Aldarmaki. 2024. Mixat: A data set of bilingual emirati-English speech. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024.

Mohammad Al-Fetyani, Muhammad Al-Barham, Gheith Abandah, Adham Alsharkawi, and Maha Dawas. 2021. Masc: Massive arabic speech corpus.

Rania Al-Sabbagh. 2024. Arzen-multigenre: An aligned parallel dataset of egyptian arabic song lyrics, novels, and subtitles, with english translations. Data in Brief.

Sadeen Alharbi, Areeb Alowisheq, Zoltán Tüske, Ka-reem Darwish, Abdullah Alrajeh, Abdulkmajed Al-rowwithi, Aljawharah Bin Tamran, Asma Ibrahim, Raghad Aloraini, Raneeam Almajim, Ranya Alkahtani, Renad Almusasad, Sara Alrashheed, Shaykhah Alsubaie, and Yaser Alonaizan. 2024. Sada: Saudi audio dataset for arabic. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10286–10290.

Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. The mgb-2 challenge: Arabic multi-dialect broadcast media recognition. In 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE.

Ahmed Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James Glass, Steve Renals, and Khalid Choukri. 2019. The mgb-5 challenge: Recognition and dialect identification of dialectal arabic speech. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE.

Ahmed Ali, Stephan Vogel, and Steve Renals. 2017. Speech recognition challenge in the wild: Arabic mgb-3. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE.

Khalid Almemon, Mark Lee, and Ali Abdulrahman Alimiman. 2013. Multi dialect arabic speech parallel corpora. In 2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA).

Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2022. SpeechT5: Unified-modal encoder-decoder pre-training for spoken language processing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5723–5738, Dublin, Ireland. Association for Computational Linguistics.

Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4218–4222.

Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al. 2022. Xls-r: Self-supervised cross-lingual speech representation learning at scale. In Proc. Interspeech 2022, pages 2278–2282.

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems.

Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhil Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The
MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).

Amira Dhoubi, Achraf Othman, Oussama El Ghoul, Mohamed Koutheair Khribi, and Aisha Al Sinani. 2022. Arabic automatic speech recognition: a systematic literature review. Applied Sciences, 12(17):8898.

Nawar Halabi et al. 2016. Arabic speech corpus. Oxford Text Archive Core Collection.

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing.

Amir Hussein, Shinji Watanabe, and Ahmed Ali. 2022. Arabic speech recognition by end-to-end, modular systems and human. Computer Speech & Language, 71:101272.

Karima Kadaoui, Maryam Al Ali, Hawau Olamide Toyin, Ibrahim Mohammed, and Hanan Aldarmaki. 2024. PolyWER: A holistic evaluation framework for code-switched speech recognition. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6144–6153, Miami, Florida, USA. Association for Computational Linguistics.

Ajinkya Kulkarni, Atharva Kulkarni, Sara Abedalmonem Mohammad Shatnawi, and Hanan Aldarmaki. 2023. Clartts: An open-source classical arabic text-to-speech corpus. In 2023 INTERSPEECH.

Salima Mdhaﬀar, Fethi Bougares, Renato de Mori, Salah Zaiem, Mirco Ravanelli, and Yannick Estève. 2024. TARIC-SLU: A Tunisian benchmark dataset for spoken language understanding. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15606–15616, Torino, Italia. ELRA and ICCL.

Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. QASR: QCRI aljazeera speech resource a large scale annotated Arabic speech corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2274–2285, Online. Association for Computational Linguistics.

Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. 2024. Scaling speech technology to 1,000+ languages. Journal of Machine Learning Research.

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR.

Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. In Proc. Interspeech 2019, pages 3465–3469.

Barbara Schuppler, Martine Adda-Decker, Catia Cucchiari, and Rudolf Muhr. 2024. An introduction to pluricentric languages in speech science and technology. Speech Communication, 156:103007.

Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, and Hanan Aldarmaki. 2023. ArTST: Arabic text and speech transformer. In Proceedings of ArabicNLP 2023, pages 41–51, Singapore (Hybrid). Association for Computational Linguistics.

Abdul Waheed, Karima Kadaoui, and Muhammad Abdul-Mageed. 2024. To distill or not to distill? on the robustness of robust knowledge distillation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12603–12621.

Abdul Waheed, Bashar Talafha, Peter Sullivan, Abdel-Rahim Elmadany, and Muhammad Abdul-Mageed. 2023. VoxArabica: A robust dialect-aware Arabic speech recognition system. In Proceedings of ArabicNLP 2023, pages 441–449, Singapore (Hybrid). Association for Computational Linguistics.

Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, and Xuedong Huang. 2021. Unispeech: Unified speech representation learning with labeled and unlabeled data. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, Proceedings of Machine Learning Research. PMLR.
A  Pre-Training Dataset Statistics

Table 12 shows the complete list of datasets used for pre-training v1, v2, and v3.

<table>
  <tr>
    <th>Dataset</th>
    <th>Dialect</th>
    <th>Hours</th>
    <th>Words</th>
    <th>v1</th>
    <th>v2</th>
    <th>v3</th>
  </tr>
  <tr>
    <td>ASC</td>
    <td>MSA</td>
    <td>3.7 hrs</td>
    <td>20,58 K</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>ArzEn[cs]</td>
    <td>EGY</td>
    <td>5.61 hrs</td>
    <td>52.00 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>CommonVoice</td>
    <td>Dialect Mix</td>
    <td>133.24 hrs</td>
    <td>494.83 K</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td rowspan="3">CommonVoice</td>
    <td>ENG</td>
    <td>1601.92</td>
    <td>10.35 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>FR</td>
    <td>732.02</td>
    <td>5.03 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ES</td>
    <td>408.34</td>
    <td>2.79 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>CIAR-TTS</td>
    <td>CA</td>
    <td>12 hrs</td>
    <td>76.31 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="10">MASC</td>
    <td>EGY</td>
    <td>175.36 hrs</td>
    <td>1.03 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>IRA</td>
    <td>17.37 hrs</td>
    <td>121.12 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>42.21 hrs</td>
    <td>330.83 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>KUW</td>
    <td>4.04 hrs</td>
    <td>32.37 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>23.15 hrs</td>
    <td>176.76 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>8.60 hrs</td>
    <td>58.38 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MSA</td>
    <td>612.28 hrs</td>
    <td>3.80 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>PAL</td>
    <td>6.17 hrs</td>
    <td>45.35 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>SAU</td>
    <td>452.24 hrs</td>
    <td>301.92 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>SYR</td>
    <td>211.33 hrs</td>
    <td>1.06 M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>12.17 hrs</td>
    <td>34.34 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>UAE</td>
    <td>9.87 hrs</td>
    <td>6.42 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MGB2</td>
    <td>Mostly MSA</td>
    <td>1000 hrs</td>
    <td>7.31 M</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>QASR</td>
    <td>Mostly MSA</td>
    <td>2000 hrs</td>
    <td>13.33 M</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>MGB3[ASR]</td>
    <td>EGY</td>
    <td>2.83 hrs</td>
    <td>18.93 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MGB3[AD1]</td>
    <td>EGY</td>
    <td>11.15 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>GLF</td>
    <td>8.92 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>LAV</td>
    <td>9.27 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>MISA</td>
    <td>9.39 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>NOR</td>
    <td>9.99 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MGB5[ASR]</td>
    <td>MOR</td>
    <td>115.7 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>ALG</td>
    <td>115.7hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>EGY</td>
    <td>451.1 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>IRA</td>
    <td>815.8 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>JOR</td>
    <td>25.9 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>KSA</td>
    <td>186.1 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>KUW</td>
    <td>108.2 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>LEB</td>
    <td>116.8 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>LIB</td>
    <td>127.4 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>MAU</td>
    <td>456.4 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>MOR</td>
    <td>57.8 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>OMA</td>
    <td>58.5 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>PAL</td>
    <td>121.4 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>QAF</td>
    <td>62.3 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>SUD</td>
    <td>47.7 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>SYR</td>
    <td>119.5 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>UAE</td>
    <td>108.4 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>YEM</td>
    <td>53.4 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Mixat[cs]</td>
    <td>UAE</td>
    <td>9.97 hrs</td>
    <td>57.82 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ParallelCorp</td>
    <td>EGY</td>
    <td>32 hrs</td>
    <td>48.56 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>GLF</td>
    <td>32 hrs</td>
    <td>27.26 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>LEV</td>
    <td>32 hrs</td>
    <td>18.43 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>MSA</td>
    <td>32 hrs</td>
    <td>30.66 K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>MADAR</td>
    <td>MOR ALG TUN LIB EGY LEV IRA GLF YEM</td>
    <td>—</td>
    <td>532.37K</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>NADI</td>
    <td>ALG BAH EGY IRA JOR KUW LEB LIB MOR OMN PAL QAT SAU SUD SYR TUN UAE YEM</td>
    <td>—</td>
    <td>702.67K</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>SADA</td>
    <td>SAU</td>
    <td>417.63 hrs</td>
    <td>3.26M</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>TARIC-SLU</td>
    <td>TUN</td>
    <td>6.81 hrs</td>
    <td>53.48K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>TunSwitch[cs]</td>
    <td>TUN</td>
    <td>10.89 hrs</td>
    <td>70.86K</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>VoxBlink</td>
    <td>Dialect Mix</td>
    <td>19.92 hrs</td>
    <td>—</td>
    <td>✓</td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 12: Summary of Dataset Statistics for Pre-Training: Hours of Audio, Word Counts, and Associated Dialects. *[cs] refers to Code Switching datasets. *[txt] refers to textual datasets.

B  Inference examples

Figure 5 lists examples of ASR outputs using the dialect-specific fine-tuned models. Note that the ‘errors’ in SAU, EGY, and JOR examples are in fact alternative spellings.

<table>
  <tr>
    <th colspan="2">(SAU)</th>
  </tr>
  <tr>
    <td>فقطن كذا راج احليكم تكلمون ذلك الملفوق ورجولكم يدعوا</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(SYR)</th>
  </tr>
  <tr>
    <td>وصف المجامع الأسلمي من بعده في آخر الزمان بالمجامع الكافر، ووصف المجامع الإسلامي من بعده في آخر الزمان بالمجامع الكافر</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(EGY)</th>
  </tr>
  <tr>
    <td>و بارث الناس اللي بتكتب اسماء الشهور بتكتب اسماء سهل ان تحفظ</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(JOR)</th>
  </tr>
  <tr>
    <td>و لاندا االحاجات الخاصة بهم وقت اطول بالله اهل الاشياء</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(LEB)</th>
  </tr>
  <tr>
    <td>و كل واحد بابا فيه يبقى اسمه محفوظ على اخو</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(IRA)</th>
  </tr>
  <tr>
    <td>بعدين احسن نوح قوي محفوظ</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(TUN)</th>
  </tr>
  <tr>
    <td>وعلى جابر اور و وايس اني اكيد يديو في تحميلهم اداري برتبهم</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(MOR)</th>
  </tr>
  <tr>
    <td>والضبط بعد بداية الاستشارة لدربوا</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(PAL)</th>
  </tr>
  <tr>
    <td>بهم نقص الله كيف ما يتقعد مش على الأطفال التلاميذ او يمكن يقدم على الشخص التلاميذ</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="2">(KUW)</th>
  </tr>
  <tr>
    <td>لكن البيض الاخير جدا قوي در يدير وجهه بكرم على</td>
    <td></td>
  </tr>
</table>

Figure 5: Examples of dialectal recognition after targeted fine-tuning, following MGB2 adaptation.

Figure 6 shows example outputs from each model on the code-switching datasets: ArzEn (Egyptian-English), TunSwitch (Tunisian-French), and Mixat (Emirati-English).

C  Cross-Dialectal Performance

Table 13 shows the cross-dialectal performance, where models trained on a single target dialect are tested on other dialects, including the three held-out sets: ALG, SUD, and YEM. In most cases, the best performance is achieved by the model trained on the same target dialect (the diagonal in Table 13). However, for low-resource dialects, like KUW and PAL, the model trained on SAU achieved the lowest WER. This is likely a result of the large size of the SAU train set and the wide geographical area and dialectal variants it covers. Curiously, all models perform well on the SYR test set; upon close inspection, we found that the set consists mostly of MSA utterances, which explains the result since all models are adapted from MSA.
ArzEn

pre-training
ArtSTV1: كثير فايداهم هيفيا عارين انهم هيفيوا ما يبعتوا interviews
ArtSTV2: يعروفو الناس اللي يبقاهمو و يبيتشو معامله ازى
ArtSTV3: كثير فايداهم هيفيا عارين انهم هيفيوا ما يبعتوا personality

adaptation
ArtSTV1: كثير فايداهم هيفيا عارين انهم هيفيوا ما يبعتوا interviews
ArtSTV2: يعروفو الناس اللي يبقاهمو و يبيتشو معامله ازى
ArtSTV3: كثير فايداهم هيفيا عارين انهم هيفيوا ما يبعتوا personality

Reference: كثير فايداهم هيفيا عارين انهم هيفيوا ما يبعتوا interviews
personality
يعروفو الناس اللي يبقاهمو و يبيتشو معامله ازى

TunSwitch

pre-training
ArtSTV1: لي هوما موش <unk>forme a <unk>aire
ArtSTV2: الجوال لي عملت قفست، ولا دارو معا
ArtSTV3: لي هوما موش des formes a caire

adaptation
ArtSTV1: لي هوما موش des formes a desire
ArtSTV2: لي هوما موش des formes a desire
ArtSTV3: لي هوما موش des formes d’ailleur

Reference: لي هوما موش des formes d’ailleur

Mixat

pre-training
ArtSTV1: وهـالـه يعطينا الطرق عشان نقدر نزكي طبق عن طريق السيرjevel
ArtSTV2: أشكال في طرقي برج 3 في ال one tool في ال one
ArtSTV3: بس لاأأقل خط من الشكلة لحن نتعلم عن طرق هذا الكافر كات

adaptation
ArtSTV1: والله يعطينا الطرق عشان نقدر نزكي طبق عن طريق السيرlevel four
ArtSTV2: والله يعطينا الطرق عشان نقدر نزكي طبق عن طريق السيرlevel three level four level one
ArtSTV3: والله يعطينا الطرق عشان نقدر نزكي طبق عن طريق السيرlevel two level three level four level one level four level three

Reference: والله يعطينا الطرق عشان نقدر نزكي طبق عن طريق السيرlevel two level three level four level one level four level three level four level one level four level three

Figure 6: Examples of code-switching samples from each of the code-switching dataset using (a) pre-training checkpoints fine-tuned on CS data directly, and (2) adapted models after fine-tuning on general data: v1 (MGB2), v2 (Dialectal data from Table 6), and v3 (dialectal data from Table 6 + Multilingual and CS data from Table 10).

<table>
  <tr>
    <th rowspan="2">Dialect</th>
    <th colspan="9">Fine-Tuning Set</th>
  </tr>
  <tr>
    <th>SAU(%)</th>
    <th>SYR(%)</th>
    <th>EGY(%)</th>
    <th>JOR(%)</th>
    <th>LEB(%)</th>
    <th>IRA(%)</th>
    <th>TUN(%)</th>
    <th>MOR(%)</th>
    <th>PAL(%)</th>
    <th>KUW(%)</th>
    <th>UAE(%)</th>
  </tr>
  <tr>
    <td>SAU</td>
    <td>27.33</td>
    <td>53.88</td>
    <td>63.24</td>
    <td>43.95</td>
    <td>56.64</td>
    <td>46.10</td>
    <td>49.36</td>
    <td>50.02</td>
    <td>45.48</td>
    <td>46.85</td>
    <td>46.14</td>
  </tr>
  <tr>
    <td>SVR</td>
    <td>19.54</td>
    <td>17.42</td>
    <td>23.91</td>
    <td>16.88</td>
    <td>25.71</td>
    <td><b>16.85</b></td>
    <td>24.36</td>
    <td>17.58</td>
    <td>16.86</td>
    <td>17.38</td>
    <td>17.53</td>
  </tr>
  <tr>
    <td>EGY</td>
    <td>38.07</td>
    <td>53.43</td>
    <td><b>36.43</b></td>
    <td>40.08</td>
    <td>51.61</td>
    <td>41.37</td>
    <td>45.58</td>
    <td>43.88</td>
    <td>40.41</td>
    <td>41.88</td>
    <td>40.38</td>
  </tr>
  <tr>
    <td>JOR</td>
    <td>22.88</td>
    <td>30.37</td>
    <td>34.14</td>
    <td><b>21.08</b></td>
    <td>28.11</td>
    <td>28.68</td>
    <td>32.96</td>
    <td>30.50</td>
    <td>25.25</td>
    <td>27.88</td>
    <td>28.05</td>
  </tr>
  <tr>
    <td>LEB</td>
    <td>41.03</td>
    <td>42.53</td>
    <td>53.23</td>
    <td>39.07</td>
    <td><b>28.05</b></td>
    <td>41.23</td>
    <td>48.53</td>
    <td>42.34</td>
    <td>41.55</td>
    <td>42.14</td>
    <td>40.97</td>
  </tr>
  <tr>
    <td>IRA</td>
    <td><b>35.18</b></td>
    <td>49.10</td>
    <td>56.91</td>
    <td>40.47</td>
    <td>46.72</td>
    <td>36.10</td>
    <td>47.84</td>
    <td>42.55</td>
    <td>40.76</td>
    <td>41.00</td>
    <td>42.16</td>
  </tr>
  <tr>
    <td>TUN</td>
    <td>44.44</td>
    <td>57.78</td>
    <td>51.85</td>
    <td>46.67</td>
    <td>45.19</td>
    <td><b>45.19</b></td>
    <td>26.67</td>
    <td>44.44</td>
    <td>47.41</td>
    <td>46.67</td>
    <td>47.41</td>
  </tr>
  <tr>
    <td>MOR</td>
    <td>59.04</td>
    <td>71.69</td>
    <td>74.70</td>
    <td>54.82</td>
    <td>69.88</td>
    <td>57.23</td>
    <td>74.70</td>
    <td><b>56.02</b></td>
    <td>59.64</td>
    <td>57.83</td>
    <td>59.64</td>
  </tr>
  <tr>
    <td>PAL</td>
    <td><b>48.53</b></td>
    <td>66.18</td>
    <td>62.65</td>
    <td>52.35</td>
    <td>60.59</td>
    <td>58.53</td>
    <td>60.00</td>
    <td>63.24</td>
    <td>53.53</td>
    <td>58.53</td>
    <td>60.59</td>
  </tr>
  <tr>
    <td>KUW</td>
    <td><b>26.59</b></td>
    <td>59.54</td>
    <td>78.03</td>
    <td>43.93</td>
    <td>67.63</td>
    <td>46.82</td>
    <td>53.18</td>
    <td>51.45</td>
    <td>44.51</td>
    <td><b>46.24</b></td>
    <td>50.87</td>
  </tr>
  <tr>
    <td>held-out</td>
    <td>ALG</td>
    <td><b>50.21</b></td>
    <td>60.09</td>
    <td>73.61</td>
    <td>52.58</td>
    <td>62.02</td>
    <td>57.51</td>
    <td>57.51</td>
    <td>59.01</td>
    <td>52.79</td>
    <td>58.37</td>
    <td>57.30</td>
  </tr>
    <td>SUD</td>
    <td><b>40.89</b></td>
    <td>64.97</td>
    <td>64.32</td>
    <td>53.15</td>
    <td>64.86</td>
    <td>54.12</td>
    <td>56.51</td>
    <td>58.79</td>
    <td>52.39</td>
    <td>56.18</td>
    <td>55.53</td>
  </tr>
    <td>YEM</td>
    <td>38.40</td>
    <td>42.16</td>
    <td>38.49</td>
    <td>34.68</td>
    <td>39.69</td>
    <td>34.92</td>
    <td>30.68</td>
    <td>35.73</td>
    <td><b>33.06</b></td>
    <td>34.68</td>
    <td>34.11</td>
  </tr>
</table>

Table 13: WER(%) for various models on unseen dialects. All models are adapted from v2→MGB2.

Metareview score: 3.5
Metareview: This paper presents a comprehensive study on the generalization of ASR model for Arabic. Since Arabic is a family of spoken dialects that live in diglossia with other varieties of Arabic, ASR is often challenging. The evaluation is across various country dialects, pre-trained MSA and dialectal models, training methods and pipelines, and multi-lingual models.

- For zero-shot and low-resource setups, dialectal pre-training was effective.
- For high-resource setup, target dialect fine-tuning was more effective.
Summary of reasons to publish: - Comperhensive evaluation of the various models and training pipelines with seemingly insightful conclusions. This is needed for Arabic ASR.
- Evaluating and improving upon ASR for code-switched Arabic speech.