Exploiting Dialect Identification in Automatic Dialectal Text Normalization (2024)

Abstract: Dialectal Arabic is the primary spoken language used by native Arabic speakers in daily communication. The rise of social media platforms has notably expanded its use as a written language. However, Arabic dialects do not have standard orthographies. This, combined with the inherent noise in user-generated content on social media, presents a major challenge to NLP applications dealing with Dialectal Arabic. In this paper, we explore and report on the task of CODAfication, which aims to normalize Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA). We work with a unique parallel corpus of multiple Arabic dialects focusing on five major city dialects. We benchmark newly developed pretrained sequence-to-sequence models on the task of CODAfication. We further show that using dialect identification information improves the performance across all dialects. We make our code, data, andpretrained models publicly available.

## introduction arabic exhibits a diglossic @xcite linguistic situation where a non-standard variety, dialectal arabic (da), coexists with modern standard arabic (msa), the standard form of the language. complicating matters, there are multiple da varieties, each differing from both other dialects and msa in phonology, morphology, and lexicon. arabic dialects are typically classified regionally, e.g., egyptian, north african, levantine, and gulf. these dialects are the true native languages historically connected to classical arabic and other regional languages. while arabic dialects are primarily spoken, they are increasingly used in written form on social media. since arabic dialects lack standard orthographies @xcite , da text tends to be highly varied and noisy. to mitigate the lack of orthographic standards for da, several efforts in arabic nlp introduced a common convention for da spelling, named conventional orthography for dialectal arabic (coda) @xcite . however, the majority of approaches involving coda consider it a side task to efforts like morphological disambiguation, diacritization, and lemmatization, as opposed to being the main target task. our contributions in this paper are as follows: next, we discuss some related work ( §2) and then give a background on arabic linguistic facts, coda, and the data we use to train and test our models ( §3). we describe our approach in §4 and present our experimental setup and results in §5. ## approach we frame the codafication task as a controlled text generation problem. formally, given a dialectal input sentence x and its dialect d, the goal is to generate the codafied sentence y according to p (y |x, d). one way to condition text generation models on the desired dialect, d, is to represent it as a special "control" token appended to the input sequence [d; x], which acts as a side constraint @xcite . in seq2seq models, this allows the encoder to learn a representation for this token as any other token in its vocabulary, and the decoder attends to this representation to guide the generation of the output sequence. this simple strategy has been used in various controlled text generation tasks such as machine translation @xcite @xcite , style transfer @xcite ), text simplification @xcite , and arabic gender rewriting @xcite . we experiment with two recently developed pretrained arabic transformer-based seq2seq models: arabart (kamal eddine et al., 2022), which was pretrained on 24gb of msa data primarily from the news domain, and @xcite , which was pretrained on a larger dataset of 250gb covering msa, da, and classical arabic (ca) data. we explore using four different control tokens to pass the dialect information to the models. during training, we use the gold dialect for each sentence to induce its control tokens. to obtain the dialect during inference, we use the did system that is available in camel tools @xcite . the system is an implementation of @xcite 's best-performing model on the madar shared task on did @xcite . the system models did for the five city dialects and msa. we fine-tune the seq2seq models on a single gpu for 10 epochs, a batch size of 16, and a maximum sequence length of 200 using hugging face's transformers @xcite . we use learning rates of 5e-5 and 1e-4, for arabart and arat5, respectively. during inference, we use beam search with a beam width of 5. ## conclusion and future work we explored and reported on the task of codafication, i.e., normalizing dialectal arabic into the conventional orthography for dialectal arabic (coda). we benchmarked newly developed pretrained seq2seq models on the task of codafication. we further showed, for the first time to our knowledge, that using dialect identification information improves arabic text normalization. in future work, we plan to explore other modeling approaches, including multitask learning models for both did and codaficiation, as well as text editing models @xcite . we also plan to extend our work to coda data sets for other dialects @xcite , evaluate the added value of improved codafication on downstream nlp tasks, and develop models of coda error type classification @xcite .