Exploiting Dialect Identification in Automatic Dialectal Text Normalization (2024)

Abstract: Dialectal Arabic is the primary spoken language used by native Arabic speakers in daily communication. The rise of social media platforms has notably expanded its use as a written language. However, Arabic dialects do not have standard orthographies. This, combined with the inherent noise in user-generated content on social media, presents a major challenge to NLP applications dealing with Dialectal Arabic. In this paper, we explore and report on the task of CODAfication, which aims to normalize Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA). We work with a unique parallel corpus of multiple Arabic dialects focusing on five major city dialects. We benchmark newly developed pretrained sequence-to-sequence models on the task of CODAfication. We further show that using dialect identification information improves the performance across all dialects. We make our code, data, and pretrained models publicly available.1

heading: 1 Introduction
text: Arabic exhibits a diglossic (Ferguson, 1959) linguistic situation where a non-standard variety, Dialectal Arabic (DA), coexists with Modern Standard Arabic (MSA), the standard form of the language. Complicating matters, there are multiple DA varieties, each differing from both other dialects and MSA in phonology, morphology, and lexicon. Arabic dialects are typically classified regionally, e.g., Egyptian, North African, Levantine, and Gulf. These dialects are the true native languages historically connected to Classical Arabic and other regional languages. While Arabic dialects are primarily spoken, they are increasingly used in written form on social media. Since Arabic dialects lack standard orthographies (Habash et al., 2018), DA text tends to be highly varied and noisy.
1https://github.com/CAMeL-Lab/codafication
This high degree of noise poses major challenges for NLP systems as it increases the degree of sparsity in the data. Such noise can be handled using modeling techniques that normalize DA if it is used as an input to the system, e.g., in machine translation from dialects to other languages. However, challenges arise when the dialect itself is the desired output, for example, in automatic speech recognition systems (Ali et al., 2019; Sahyoun and Shehata, 2023). Consequently, evaluating and optimizing these systems can become problematic.
To mitigate the lack of orthographic standards for DA, several efforts in Arabic NLP introduced a common convention for DA spelling, named Conventional Orthography for Dialectal Arabic (CODA) (Habash et al., 2018). However, the majority of approaches involving CODA consider it a side task to efforts like morphological disambiguation, diacritization, and lemmatization, as opposed to being the main target task.
Our contributions in this paper are as follows:
• We explore and report on the task of CODAfication (Eskander et al., 2013), normalizing DA text into the CODA convention. We work with a unique parallel corpus of multiple Arabic dialects (Eryani et al., 2020), focusing on five cities: Beirut, Cairo, Doha, Rabat, and Tunis.
• We benchmark newly developed pretrained sequence-to-sequence (Seq2Seq) models on the task of CODAfication.
• We demonstrate that using dialect identification information improves the performance across all dialects.
Next, we discuss some related work (§2) and then give a background on Arabic linguistic facts, CODA, and the data we use to train and test our models (§3). We describe our approach in §4 and present our experimental setup and results in §5.
42

heading: 2 Related Work

heading: 2.1 Dialectal Arabic Text Normalization
text: DA NLP research has been receiving a considerable amount of attention, mainly due to the availability of monolingual and multilingual DA corpora (McNeil and Faiza, 2011; Zaidan and Callison-Burch, 2011; Zbib et al., 2012; Cotterell and Callison-Burch, 2014; Salama et al., 2014; Jeblee et al., 2014; Al-Badrashiny and Diab, 2016; Zaghouani and Charfi, 2018; Abdul-Mageed et al., 2018; Bouamor et al., 2019). While MSA has welldefined orthographic standards, none of the Arabic dialects do today. As a result, almost all DA corpora were created without following any spelling conventions or standards, which are necessary for building robust DA NLP applications, e.g., machine translation (Erdmann et al., 2017). To mitigate this problem, several efforts have been introduced to standardize and develop orthographic conventions for Arabic dialects. Habash et al. (2012a) introduced the Conventional Orthography for Dialectal Arabic (CODA), the very first attempt to create guidelines and spelling conventions for Egyptian Arabic orthography. The convenience CODA offered by providing a standardized orthography led to the creation of many CODA extensions covering various dialects including Tunisian, Algerian, Palestinian, Moroccan, Yemeni, and Gulf Arabic (Zribi et al., 2014; Saadane and Habash, 2015; Jarrar et al., 2016; Turki et al., 2016; Khalifa et al., 2018). Each of these extensions tended to curate its own list of exceptional spellings for closed class words. Habash et al. (2018) introduced a unified set of guidelines for Arabic Dialect orthography – dubbed CODA* (CODA Star).
CODA has been used in the creation of a number of resources for DA NLP (Habash et al., 2012b; Eskander et al., 2013; Maamouri et al., 2014; Diab et al., 2014; Pasha et al., 2014b; Jarrar et al., 2016; Khalifa et al., 2018; Eryani et al., 2020). Most relevant to this paper is the work of Eryani et al. (2020) who extended a portion of the MADAR Corpus (Bouamor et al., 2018) to create the MADAR CODA Corpus, a collection of 10,000 sentences from five Arabic city dialects (Beirut, Cairo, Doha, Rabat, and Tunis) represented in the CODA standard in parallel with their original raw form. We use this corpus to train and test our models.
In terms of modeling approaches to CODAfication, the first work was proposed by Eskander et al. (2013) where they introduced CODAFY, a
feature-based machine learning classifier to normalize Egyptian Arabic into CODA. Al-Badrashiny et al. (2014) and Shazal et al. (2020) targeted CODA output for dialectal Arabizi (Romanized Arabic) input. Most other approaches attempted to normalize DA texts into CODA as part of morphological analysis and disambiguation (Pasha et al., 2014a; Zalmout et al., 2018; Khalifa et al., 2020; Zalmout and Habash, 2020; Obeid et al., 2022). Our work is most similar to the one of Eskander et al. (2013) where we consider the task of CODAfication as a standalone text normalization task.
There has been some work on normalizing DA into MSA (Shaalan et al., 2007; Salloum and Habash, 2011, 2012; Alnajjar and Hämäläinen, 2024). While all this work is similar to ours in that dialectal input is processed, our output is still dialectal and not in MSA. Moreover, our proposed work has some similarities to grammatical error correction (GEC) for MSA (Zaghouani et al., 2014, 2015; Mohit et al., 2014; Rozovskaya et al., 2015; Watson et al., 2018; Habash and Palfreyman, 2022; Kwon et al., 2023; Alhafni et al., 2023). However, our task is different from GEC for MSA since GEC assumes a standard orthography that the writer is also assumed to aim for.

heading: 2.2 Dialect Identification
text: Dialect Identification (DID) is the task of identifying the dialect of a given speech or text fragment (Etman and Beex, 2015). Since informal conversations in real-world and online settings are typically conducted in DA, there has been a growing interest in developing and scaling automatic Arabic DID systems. This can be observed in the organization of multiple shared tasks (Bouamor et al., 2019; Abdul-Mageed et al., 2021, 2022, 2023, 2024) and the existence of various datasets and tools (Zaidan and Callison-Burch, 2011; Bouamor et al., 2014; Salama et al., 2014; Alsarsour et al., 2018; Abu Kwaik et al., 2018; Zaghouani and Charfi, 2018; Salameh et al., 2018; Bouamor et al., 2019; Abdelali et al., 2021; Baimukan et al., 2022). Besides its obvious use for profiling (Rangel et al., 2019), DA identification has already proved to be helpful for system selection in NLP tasks such as machine translation (Salloum et al., 2014), and morphological tagging (Obeid et al., 2022). In our work, we explore using text DID at the sentencelevel in aiding CODAfication. For this, we use the CAMeL Tools (Obeid et al., 2020) DID implementation of Salameh et al. (2018).

heading: 3 Background

heading: 3.1 Arabic Linguistic Facts
text: Arabic encompasses a wide range of dialectal varieties, with Modern Standard Arabic (MSA) serving as the common language of culture, media, and education across the Arab world. However, MSA is not the native language of any Arabic speaker, as dialectal Arabic dominates daily conversations. When native speakers write or speak (e.g., TV shows) in MSA, there is frequent code-mixing with the dialects in terms of phonological, morphological, and lexical choices (Abu-Melhim, 1991; Habash et al., 2008; Bassiouney, 2009). While Arabic dialects are typically classified regionally, e.g., Egyptian, North African, Levantine, and Gulf (Habash, 2010), hierarchical labels have been proposed to include countries, provinces, and cities (Baimukan et al., 2022). In this work, we focus on five city dialects: Beirut, Cairo, Doha, Rabat, and Tunis.
Despite their similarities, DA and MSA have many differences that prevent MSA tools from being effectively utilized for dialectal text. Arabic dialects vary phonologically, lexically, and morphologically from MSA and from each other; and they vary from region to region and, to a lesser extent, from city to city in each region (Watson, 2007). While MSA has a well-defined standard orthography, none of the Arabic dialects do today. When Arabic speakers write in DA, they typically write in a way that reflects the phonology or the etymology of the words. Therefore, apart from unintentional typographical errors, no spelling of a dialectal word can be deemed truly “incorrect”. This phenomenon is referred to as
spontaneous orthography (Eskander et al., 2013; Eryani et al., 2020). For instance, the word for ‘small [feminine singular]’ in the Beirut dialect, /zKi:ri/, can be written in a range of spontaneous Arabic spellings, some of which highlighting its phonology and others its etymological connections to MSA èQ ª Sγyrh̄ /sQaKi:ra[t]/. These include: ø Q
« P zγyry, èQ « P zγyrh, èQ « P zγyrh̄, ø Q ª Sγyry, èQ ª Sγyrh, and èQ ª Sγyrh̄.

heading: 3.2 CODA
text: CODA*2 (Habash et al., 2018) consolidates and standardizes several prior dialect-specific CODA conventions (Habash et al., 2012a; Saadane and Habash, 2015; Turki et al., 2016; Khalifa et al., 2016; Jarrar et al., 2016). CODA*, henceforth CODA, is an internally consistent and coherent convention for writing all DA varieties using the Arabic script aiming to balance dialectal uniqueness with MSA-DA similarities. CODA ensures consistency by controlling the natural spelling tendencies in spontaneous orthography that arise from writers considering etymological or phonological references of words. And while it is created for computational purposes, it is designed to be easily learnable and readable.
In the example mentioned above, the Beirut dialect word /zKi:ri/ ‘small [feminine singular]’ is written in a form reflective of MSA etymology: èQ ª Sγyrh̄. Other examples of CODA from the MADAR CODA Corpus (Eryani et al., 2020) appear in Table 1. Note that foreign words pose a particular challenge to CODA due to the ambiguous phonological signals in the Arabic raw text. Conse-
2Pronounced CODA Star, as in, for any dialect.
BEI CAI DOH RAB TUN RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ <SPC> 863 <SPC> 1166 ا 150 ا 548 ا 458 <SPC> أ ا 409 أ ا 608 أ ا 124 ا 352 أ ا 288 أ Â إ ا 405 ه ة 323 ه 82 <SPC> 324 إ ا 189 إ Ă ا 324 ت ث 257 ذ ذا 62 أ ا 256 <SPC> 175 ت ث 294 إ ا 146 ج شت 33 ت ث 190 <SPC> 148 ت t ه 173 <SPC> 142 ج ك 31 <SPC> 168 ا ة 115 و w و ه 138 د ذ 95 <SPC>ا 28 ا ة 160 ا 109 د d أ ق 129 ى ي 80 ا آ 25 إ ا 84 ل 100 ا A د ذ 119 ا 73 إ ا 23 د ذ 68 و ه 97 ث θ <SPC> 106 ا 68 ي ج 20 ل 67 ن 85 ه h ق q ذ ð BEI CAI DOH RAB TUN ى ý RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ RAW CODA FREQ ة ħ <SPC> 863 <SPC> 1166 A 150 548 A 458 ي y Â A 409 Â A 608 Â A 124 352 Â A 288 ج j Ă A 405 h ħ 323 h 82 <SPC> 324 Ă A 189 اذ Að A 324 t θ 257 ð Að 62 256 <SPC> 175 تش tš t θ 294 Ă A 146 j tš 33 190 <SPC> 148 ك k
h 173 <SPC> 142 j k 31 <SPC> 168 A ħ 115 <SPC>ا <SPC>A w h 138 d ð 95 <SPC>A 28 160 A 109 آ Ā Â q 129 ý y 80 A Ā 25 84 l 100 ل l d ð 119 A 73 Ă A 23 68 w h 97 ن n
<SPC> 106 A 68 y j 20 67 n 85
quently, Eryani et al. (2020) adopted a minimalistic strategy for CODAfying these words, resulting in some plausible but inconsistent variants. For example, the word for ‘hamburger’ in Table 1 appears as both Q «Q .Òë hmbrγr and Qk. Q .Òë hmbrjr.

heading: 3.3 MADAR CODA Corpus
text: We use the manually annotated MADAR CODA Corpus (Eryani et al., 2020), a collection of 10,000 sentences from five Arabic city dialects (Beirut, Cairo, Doha, Rabat, and Tunis) represented in the CODA standard in parallel with their original raw form. The sentences come from the Multi-Arabic Dialect Applications and Resources (MADAR) Project (Bouamor et al., 2018) and are in parallel across the cities (2,000 sentences from each city).
The corpus is originally split into train and test, with each split consisting of 5,000 parallel sentences (1,000 per dialect). In our setup, we combine the original train and test splits and then divide the data randomly into separate training (Train), development (Dev), and testing (Test) sets. We use a 70/15/15 split, resulting in 1400, 300, and 300 sentences, respectively, per dialect. In total, we end up with 7,000 sentences for Train, 1,500 for Dev, and 1,500 for Test. Table 1 shows an example of a sentence from the corpus in its raw and CODA parallel forms across the five city dialects.
Table 2 presents the top 10 character-level edit changes from raw text to CODA in the five city dialects. It is noteworthy that while there are many shared transformations, they appear with different distributions. This suggests that a model making use of DID could learn dialect-specific preferences. At the same time, the shared phenomena can aid in learning dialect-independent general patterns.

heading: 4 Approach
text: We frame the CODAfication task as a controlled text generation problem. Formally, given a dialectal input sentence X and its dialect D, the goal is to generate the CODAfied sentence Y according to P (Y |X,D). One way to condition text generation models on the desired dialect, D, is to represent it as a special “control” token appended to the input sequence [D;X], which acts as a side constraint (Sennrich et al., 2016a). In Seq2Seq models, this allows the encoder to learn a representation for this token as any other token in its vocabulary, and the decoder attends to this representation to guide the generation of the output sequence.
This simple strategy has been used in various controlled text generation tasks such as machine translation (Sennrich et al., 2016b; Sennrich and Haddow, 2016; Johnson et al., 2016; Agrawal and Carpuat, 2019), style transfer (Niu et al., 2017, 2018), text simplification (Yanamoto et al., 2022; Agrawal and Carpuat, 2023), and Arabic gender rewriting (Alhafni et al., 2022).
We experiment with two recently developed pretrained Arabic Transformer-based Seq2Seq models: AraBART (Kamal Eddine et al., 2022), which was pretrained on 24GB of MSA data primarily from the news domain, and AraT5-v2 (Nagoudi et al., 2022; Elmadany et al., 2023), which was pretrained on a larger dataset of 250GB covering MSA, DA, and Classical Arabic (CA) data.
We explore using four different control tokens to pass the dialect information to the models. Table 3 presents the control tokens we considered in our experiments:
• City: The name of the city where the Arabic dialect is spoken.
• MSA Phrase: An MSA phrase that follows the template Èñ ® K <city> ú
̄ ‘in <city> we say’, where <city> represents one of the five cities whose dialects we are modeling.
• DA Phrase: A DA phrase that follows the template <we-say> <city> ú
̄ ‘in <city> we say’, where <city> represents one of the five dialects we are modeling, and <we-say> represents a spontaneous orthography of the dialectal version of the phrase ‘we say’.
• Digit: An ad hoc unique numerical value for each dialect.
During training, we use the gold dialect for each sentence to induce its control tokens. To obtain the dialect during inference, we use the DID system that is available in CAMeL Tools (Obeid et al., 2020). The system is an implementation of Salameh et al. (2018)’s best-performing model on the MADAR shared task on DID (Bouamor et al., 2019). The system models DID for the five city dialects and MSA. We fine-tune the Seq2Seq models on a single GPU for 10 epochs, a batch size of 16, and a maximum sequence length of 200 using Hugging Face’s Transformers (Wolf et al., 2019). We use learning rates of 5e-5 and 1e-4, for AraBART and AraT5, respectively. During inference, we use beam search with a beam width of 5.

heading: 5 Experimental Setup and Results

heading: 5.1 Metrics
text: We use the MaxMatch (M2) scorer (Dahlmeier and Ng, 2012), which is predominantly used to evaluate grammatical error correction systems. The M2 scorer assesses the edits made by the system against
the ‘gold standard’ edits in the target CODA, calculating precision (P), recall (R), F1, and F0.5 scores. F0.5 weighs precision twice as much as recall, to prioritize the accuracy of edits relative to all edits made by the system. To obtain the gold edits, we use the alignment algorithm that was proposed by Alhafni et al. (2023). We also use their optimized version of the M2 scorer that deals with the extreme running times of the original release in cases where the generated outputs differ significantly from the input.
Moreover, and to be consistent with previous work, we report the Word Error Rate (WER). However, we believe that WER is not a suitable metric for the task of CODAfication due to the high similarity between the input and output sentences.

heading: 5.2 Models: Baselines and Systems
text: Do Nothing Our first baseline simply copies the input sentences to the output. This baseline highlights the level of similarity between the inputs and outputs.
Maximum Likelihood Estimation (MLE) For the second baseline, we build a simple wordlevel lookup model to map input words to their CODAfied versions. We first obtain word-level alignments over all the training data from all the dialects (Joint) by using the algorithm developed by Alhafni et al. (2023). We then exploit the alignments to implement the lookup model as a bigram maximum likelihood estimator: given an input word with its bigram surrounding context (wi, wi−1), and a CODAfied target word (yi), the model is built by computing P (yi|wi, wi−1) over the training examples. During inference, we generate all possible alternatives for the given input word
Model Training Control Token P R F1 F0.5 WER
AraT5 Joint
- 87.26 77.98 82.36 85.23 0.0622 City 87.99 78.25 82.83 85.85 0.0601 MSA Phrase 88.17 78.85 83.25 86.13 0.0583 DA Phrase 88.35 78.95 83.39 86.29 0.0573
Digit 87.67 78.25 82.69 85.61 0.0610
(wi). If the bigram context (wi, wi−1) was not observed in the training data, we backoff to a unigram context. If the input word was not observed during training, we pass it to the output as it is.
Seq2Seq We train both AraBART and AraT5 on all the dialects’ training data jointly with and without using DID information. We refer to this modeling setup as Joint. Moreover, to examine the effect of the joint dialectal training, we train five separate models, one for each dialect. During inference, we combine the separate models in an ensemble setup where we use the DID predictions for each sentence to select the appropriate model. We refer to this setup as Ensemble.

heading: 5.3 Results
text: Overall Results Table 4 presents the results on the Dev set. Among the baselines, both AraBART and AraT5 demonstrate superior performance compared to the MLE model. In terms of training setups, Joint training outperforms Ensemble models
for both AraBART and AraT5, with AraT5 being the better performer achieving 84.72 F0.5.
When we train the AraBART Joint variants with DID control tokens, the performance increases compared to the AraBART Joint baseline, except when training with the DA Phrase DID control token. All the AraT5 Joint variants benefit from training with DID control tokens compared to the AraT5 baseline, with the City control token being the best performer with 85.80 F0.5 (1.08 increase over the AraT5 baseline and statistically significant at p < 0.05).3 It is noteworthy that the AraT5 variants perform better compared to their AraBART counterparts across all experiments. We suspect this is due to the fact the data used to pretrain AraT5 consisted of a mix of MSA, DA, and CA compared to only MSA in the case of AraBART’s pretraining.
Since AraT5 performed better than AraBART across all experiments, we present the results on
3Statistical significance was done using a two-sided approximate randomization test.
Model Training Control Token P R F1 F0.5 WER Do Nothing - - 100 0 0 0 0.2677 MLE Joint - 66.81 44.62 53.51 60.77 0.1456 AraT5 Joint - 86.76 77.44 81.83 84.72 0.0620 City 87.58 79.34 83.25 85.80 0.0566 87.52 79.34 83.23 85.75 0.0566 MSA Phrase 87.43 79.06 83.03 85.62 0.0573 87.43 79.06 83.03 85.62 0.0572 DA Phrase 87.25 78.56 82.68 85.36 0.0709 87.26 78.61 82.71 85.38 0.0710 Digit 87.37 79.00 82.98 85.56 0.0588 87.37 79.00 82.98 85.56 0.0588 Ensemble - 85.65 72.84 78.73 82.74 0.0739 85.52 73.07 78.80 82.70 0.0730 AraBART Joint - 85.35 74.36 79.47 82.90 0.0728 City 85.69 74.41 79.65 83.17 0.0715 85.64 74.47 79.66 83.15 0.0715 MSA Phrase 85.48 74.47 79.59 83.02 0.0716 85.48 74.47 79.59 83.02 0.0716 DA Phrase 85.00 74.58 79.45 82.69 0.0721 85.00 74.58 79.45 82.69 0.0721
Digit 86.11 73.96 79.58 83.38 0.0732 86.11 73.96 79.58 83.38 0.0731
Ensemble - 84.59 67.92 75.34 80.63 0.0843 84.40 68.48 75.61 80.65 0.0829
AraT5 (Baseline) AraT5 + City # edits (raw-coda) # edits (gen-coda) Dialect P R F1 F0.5 WER P R F1 F0.5 WER 527 100 Beirut 86.07 79.70 82.76 84.71 0.0829 89.30 82.35 85.69 87.82 0.0673 3.11 480 88 Cairo 89.52 85.42 87.42 88.67 0.0582 89.13 85.42 87.23 88.36 0.0588 -0.31 112 40 Doha 83.52 67.86 74.88 79.83 0.0302 85.26 72.32 78.26 82.32 0.0277 2.49 333 85 Rabat 85.21 72.67 78.44 82.37 0.0557 86.35 75.98 80.83 84.05 0.0493 1.68 334 112 Tunis 86.08 70.36 77.43 82.40 0.0821 84.15 71.56 77.35 81.29 0.0792 -1.11
AraT5 (Baseline) AraT5 + DA Phrase # edits (raw-coda) # edits (gen-coda) Dialect P R F1 F0.5 WER P R F1 F0.5 WER 536 116 Beirut 85.57 78.54 81.91 84.07 0.0847 87.02 80.04 83.38 85.53 0.0760 1.46 519 95 Cairo 89.65 83.43 86.43 88.33 0.0609 89.57 84.39 86.90 88.48 0.0592 0.15 120 39 Doha 85.86 70.83 77.63 82.36 0.0295 89.80 73.33 80.73 85.94 0.0245 3.58 375 109 Rabat 87.11 73.87 79.94 84.09 0.0685 87.34 73.60 79.88 84.20 0.0674 0.11 303 81 Tunis 86.74 75.58 80.78 84.25 0.0661 89.23 76.57 82.42 86.37 0.0577 2.12
Table 6: Dialect-specific results of the best system (AratT5 + City) against the baseline (AraT5) on the Dev set.
. .
. . . . . . . . . . i . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . i . . . . . . . . . .
( li ) i l t . .
i t . . . . . . . . . . i . . . . . . . . . .
. . . . . . . . . . t . . . . . . . . . . is . . . . . . . . . .
Table 7: Dialect-specific results of the best system (AratT5 + DA Phrase) against the baseline (AraT5) on the Test set.
the Test set using AraT5 and its variants in Table 5. Training AraT5 with the DA Phrase control token yields the best performance on the Test set with 86.29 F0.5 (1.06 increase over the AraT5 baseline and statistically significant at p < 0.05).
DID Efficacy We estimate an oracle upper bound by using gold DID labels during inference on the Dev set (Table 4). We do not notice significant improvements across all variants compared to the models that use predicted DID labels. In some cases, using gold DID labels results in identical performance to models using predicted labels. This can be attributed to the robustness of our CODAfication models and the reliability of the DID system we are using, which achieves a high accuracy of 92.1% on the Dev set.
Most of the prediction errors made by the DID system occur in sentences lacking distinctive cues that would allow clear assignment to a specific dialect. Therefore, these errors cannot be considered true errors, but rather stem from the MADAR
dataset’s limitation of not having multi-dialectal labels. This is consistent with the findings of Keleg and Magdy (2023) where they manually analyzed the errors of a single-label DID system and found that ∼66% of the errors are not true errors and could be resolved with multi-dialect labels.
Dialect-Specific Results We present the dialectspecific results on the Dev and Test sets in Tables 6 and 7, respectively. Our best system on the Dev set, AraT5 trained with the City DID control token, improves the results over the AraT5 baseline for all dialects (with the largest increase seen for Beirut at 3.11 F0.5), except for Cairo and Tunis, where the performance drop is attributed to decreased precision rather than recall. This suggests that our best system may be making unnecessary extra rewrites. On the Test set, our best system, AraT5 trained with the DA Phrase DID control token, improves the results over the AraT5 baseline across all dialects, with the largest increase for Doha at 3.58 F0.5.

heading: 5.4 Error Analysis
text: To gain insights into the errors present in our best performing system on the Dev set, we conducted an error analysis on a sample of 100 cases, which accounted for 21% of the total 471 erroneous instances in the generated output. We classified these errors into specific categories, with results and examples provided in Table 8:
• Non-CODA: These are cases characterized by having plausible spontaneous spelling but incorrect CODA. This is the largest group of errors.
• Hallucination and Related Hallucination: Hallucinations refer to word rewrites that are implausible under any circumstance as a CODA correction or non-CODA spelling. We distinguish cases that seem morphologically related to the input but are actually unrelated forms. We observe that 2/3 of the cases were largely unrelated to the reference.
• Valid: This category encompasses valid alternative spellings, particularly those associated with proper nouns and foreign words.
• Deletion: Deletions refer to omitted words. 55.6% of these are non-CODA spellings, e.g., a missed split (Table 8 example), while the rest are divided between gold errors and hallucinations.
• Punctuation: Punctuation generation errors.
The error analysis highlights that CODA issues constitute a significant portion of the remaining errors, potentially accounting for half of the cases between non-CODA words and deletions. Hallucinations, whether minor or severe, make up nearly
a third of the errors. These findings suggest the need for more training data and improved models to address these problems. The presence of valid variants, which represent one-eighth of the errors, indicates the need to adopt a multi-reference approach for text normalization evaluation.

heading: 6 Conclusion and Future Work
text: We explored and reported on the task of CODAfication, i.e., normalizing Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA). We benchmarked newly developed pretrained Seq2Seq models on the task of CODAfication. We further showed, for the first time to our knowledge, that using dialect identification information improves Arabic text normalization.
In future work, we plan to explore other modeling approaches, including multitask learning models for both DID and CODAficiation, as well as text editing models (Omelianchuk et al., 2020, inter alia). We also plan to extend our work to CODA data sets for other dialects (Jarrar et al., 2016; Khalifa et al., 2016), evaluate the added value of improved CODAfication on downstream NLP tasks, and develop models of CODA error type classification (Belkebir and Habash, 2021).
Limitations
Although we benchmarked pretrained Seq2Seq models on the task of CODAfication and demonstrated the added improvements of using dialect identification information, we did not conduct experiments to showcase the added value of the task of CODAfication on downstream NLP tasks such as sentiment analysis and machine translation. The efficacy of CODAfication in enhancing these downstream applications, particularly with newer models, remains an area for future exploration.
Our work is based on a unique curated parallel corpus encompassing multiple Arabic dialects from five cities. While this dataset provides valuable insights into CODAfication performance across diverse dialectal variations, it also introduces limitations in generalizing our findings to a broader spectrum of Arabic dialects beyond our specific dataset. Future research should aim to extend the evaluation of CODAfication models across a more extensive range of dialectal datasets to ensure robustness and applicability across diverse linguistic contexts.

heading: Acknowledgements
text: We acknowledge the support of the High Performance Computing Center at New York University Abu Dhabi. We thank the anonymous reviewers for their feedback and suggestions.

heading: Future Work
text: In future work, we plan to explore other modeling approaches, including multitask learning models for both DID and CODAficiation, as well as text editing models (Omelianchuk et al. , 2020, inter alia). We also plan to extend our work to CODA data sets for other dialects (Jarrar et al. , 2016; Khalifa et al. , 2016), evaluate the added value of improved CODAfication on downstream NLP tasks, and develop models of CODA error type classification (Belkebir and Habash, 2021). Limitations
Although we benchmarked pretrained Seq2Seq models on the task of CODAfication and demonstrated the added improvements of using dialect identification information, we did not conduct experiments to showcase the added value of the task of CODAfication on downstream NLP tasks such as sentiment analysis and machine translation. The efficacy of CODAfication in enhancing these downstream applications, particularly with newer models, remains an area for future exploration. Our work is based on a unique curated parallel corpus encompassing multiple Arabic dialects from five cities. While this dataset provides valuable insights into CODAfication performance across diverse dialectal variations, it also introduces limitations in generalizing our findings to a broader spectrum of Arabic dialects beyond our specific dataset. Future research should aim to extend the evaluation of CODAfication models across a more extensive range of dialectal datasets to ensure robustness and applicability across diverse linguistic contexts