i C ompass Working Notes for the Nuanced A rabic Dialect Identification Shared task (2022)

Abstract: We describe our submitted system to the Nuanced Arabic Dialect Identification (NADI) shared task. We tackled only the first subtask (Subtask 1). We used state-of-the-art Deep Learning models and pre-trained contextualized text representation models that we finetuned according to the downstream task in hand. As a first approach, we used BERT Arabic variants: MARBERT with its two versions MARBERT v1 and MARBERT v2, we combined MARBERT embeddings with a CNN classifier, and finally, we tested the Quasi-Recurrent Neural Networks (QRNN) model. The results found show that version 2 of MARBERT outperforms all of the previously mentioned models on Subtask 1.

## introduction nowadays, social media is spread all over arabic countries where people tend to express themselves in their own local dialect. since it has different variants and dialects across the world, arabic dialect identification presents a challenging task. even if some dialects share some vocabulary, they still differ according to countries, where each dialect has its own specifications. because of the massive amount of such content, automatic identification of arabic dialects becomes crucial. following the first @xcite and second @xcite nuanced arabic dialect identification (nadi 2020 and nadi 2021), nadi 2022 subtask 1 focuses on identifying the arabic dialect of a given text, especially on social media sources where there is no established standard orthography like modern standard arabic (msa) @xcite . the first attempts to tackle this challenge identified different arabic dialects categories in addition to msa: maghrebi, egyptian, levantine, gulf, and iraqi @xcite . in @xcite authors proposed 4 arabic dialects categories by merging the iraqi with the gulf. the paper is structured as follows: section 2 provides a concise description of the used dataset, its statistics, and pre-processing techniques. section 3 describes the used systems and the experimental setup to build models for country-level dialect identification. section 4 presents and discusses the obtained results. finally, section 5 concludes and points to possible directions for future work. ## data description the provided training dataset of the competition (abdul-mageed et al., 2022) dedicated for the first subtask consists of around 25k tweets written in eighteen arabic dialects including: egypt, iraq, ksa, algeria, oman, syria, libya, tunisia, morocco, lebanon, uae, jordan, kuwait, yemen, palestine, bahrain, qatar, and sudan. figure 1 presents the distribution of the tweets over the eighteen labels. in fact, the training dataset is imbalanced and presents skewed class proportions. we notice the domination of egypt and iraq tweets compared to the other countries. ## system description different deep learning architectures and pretrained language models were used in order to achieve the best results. ## conclusion in this work, marbert @xcite in its second version was used to identify country-level dialect. the best results were obtained by marbert v2 with specific hyperparameters, which was selected for the final submission. future work would involve building a multi-script arabic dialects language model including arabic script and latin script based characters. taking as example, tunisians, who tend to express themselves using an informal way called tunizi @xcite that represents the tunisian text written using latin characters and numbers instead of arabic letters.