A rabic dialect identification using machine learning and transformer-based models: Submission to the NADI 2022 Shared Task (2022)

Abstract: Arabic has a wide range of dialects. Dialect is the language variation of a specific community. In this paper, we show the models we created to participate in the third Nuanced Arabic Dialect Identification (NADI) shared task (Subtask 1) that involves developing a system to classify a tweet into a country-level dialect. We utilized a number of machine learning techniques as well as deep learning transformer-based models. For the machine learning approach, we build an ensemble classifier of various machine learning models. In our deep learning approach, we consider bidirectional LSTM model and AraBERT pretrained model. The results demonstrate that the deep learning approach performs noticeably better than the other machine learning approaches with 68.7% accuracy on the development set.

## introduction dialect identification is the task of automatically identifying the dialect of a particular part of the text @xcite . arabic dialects differ by region, and there are no available dictionaries for their vocabulary or written rules for the words that are specific to those dialects. developing an arabic dialect identification system experimenting with different corpora and working at different levels of representation has attracted increasing attention in recent years @xcite . in this paper, we present our work to tackle the third nuanced arabic dialect identification (nadi) shared task that targets country-level dialects. we built multiple classifiers based on machine learning and deep learning techniques. we experimented with an approach of combining different machine learning models using a combination of n-grams and tf-idf as features to enhance the performance. another method applied in this study is a deep learning approach including bidirectional long-short term memory (bilstm) model and pre-trained arabert model. this paper is organized as follows: section 2 details the used dataset. section 3 presents the applied preprocessing steps and the proposed approach for arabic dialect identification. in section 4, we discuss the obtained results. section 5 contains the conclusion. ## datasets the dataset used in nadi 2022 shared task (subtask 1) is the same as the prior nadi shared task (abdul-mageed et al. 2020) @xcite . it consists of 20k labeled tweets for training, 4,871 for development that covers 18 arabic dialects. for testing, two test sets were provided test-a and test-b. test-a includes 18 country-level dialects. in the second test (test-b), k country-level dialects are covered where k is kept unknown. the training data which consists of 20k tweets is unbalanced as you can see in figure 1 . figure 1 displays how tweets are distributed among arab countries. most of the tweets belong to egypt (4283 tweets) and only 215 belong to bahrain, qatar, and sudan. the provided data is normalized in which all urls are replaced with the word 'url' and mentions replaced with the word 'user'. around 10m unlabeled tweets were also provided to participating teams by the nadi shared task organizers. ## methodology this section shows the models we used in our experiments starting with machine learning methods and moving on to deep learning and transformer-based approaches. ## results & discussion in our experiments, we have reported the result of multiple models starting with machine learning approaches and moving on to transformer-based methods. the evaluation measures include f-score, accuracy, precision, and recall. however, the macro averaged f-score is the official metric of evaluation. models dev test-a test-b acc f1 acc f1 acc f1 logistic regressio n 36.9 19.1 7.8 5.5 17.6 7.9 svm 40.9 19.4 36.9 16. 1 21.3 7.4 ensemble classifier 46.2 24.4 39.1 18. 8 24.4 9.1 bidirectio nal lstm 68.7 50.5 39.9 22. 4 23.7 9.3 finetuning arabert transfor mer 68.7 50.5 38.2 21. 9 23.5 9.1 5 conclusion in this paper, we present our submitted models to the third nuanced arabic dialect identification shared task. we conducted different experiments in which we tried different preprocessing procedures and several feature combinations for model training. we combined different machine learning approach such as (logistic regression, support vector machine, and multinomial naive bayes) to build a strong arabic dialect identification system. we further developed a transformer-based model using embedding layer with a bidirectional lstm model and fine-tuning arabert transformer. the obtained results have shown that our transformerbased model outperforms all machine learning model on macro-f1 evaluation measure.