Word Representation Models for A rabic Dialect Identification (2022)

Abstract: This paper describes the systems submitted by BFCAI team to Nuanced Arabic Dialect Identification (NADI) shared task 2022. Dialect identification task aims at detecting the source variant of a given text or speech segment automatically. There are two subtasks in NADI 2022, the first subtask for country-level identification and the second subtask for sentiment analysis. Our team participated in the first subtask. The proposed systems use Term Frequency Inverse/Document Frequency and word embeddings as vectorization models. Different machine learning algorithms have been used as classifiers. The proposed systems have been tested on two test sets: Test-A and Test-B. The proposed models achieved Macro-f1 score of 21.25% and 9.71% for Test-A and Test-B set respectively. On other hand, the best-performed submitted system achieved Macro-f1 score of 36.48% and 18.95% for Test-A and Test-B set respectively.

## introduction social media's widespread use has made it easy to collect user data in surpassing ways. these data can include behaviour and usage, content, and network @xcite . this work focuses on predicting social media user dialect based on language of his/her post. dialect identification task comprises of some challenges such as finding the differences in writing style between men and women on social networks, ages of authors, or location @xcite . the solutions to these questions are very important for new problems in the era of social networks such as fake news analysis, plagiarism detection, privacy and security issues. the author profiling task aims at examining the written documents to extract pertinent demographic information from their authors @xcite . lately, the research community concerning arabic natural language processing started to pay attention to dialect identification. nuanced arabic dialect identification shared task (nadi 2021) aimed at predicting the dialect in arabic tweets @xcite . this work explores different vectorization techniques integrated with the various machine learning approaches. term frequency/inverse document frequency (tf/idf) and word embeddings have been used as vectorization models. multinomial naïve bayes (mnb), complement naïve bayes (cnb), support vector machine (svm), k-nearest neighbor (knn), decision tree (dt), random forest (rf), and multi-layer perceptron (mlp) have been used as classifiers due to their ability to deal with multi-class classification problems. the rest of the paper is organized as follows: section 2 presents the dataset; section 3 describes the system architecture. experimental settings and results are given in section 4. finally, conclusion and future work are presented in section 5. ## data the nadi 2022 datasets that we used for building, developing, and evaluating the submitted systems were distributed by the task organizers @xcite . the dataset targets nuanced arabic dialect identification at the country level for arabic tweets. it comprises training, development, and test sets. it covers 18 dialects (a total of approximately 20k tweets). the evaluation depends on two test sets, test-a covers 18 country-level dialects, whereas the second test set (test-b) covers k country-level dialects. the value of k was kept unknown by the task organizers ## system architecture the general framework of our model is shown in figure 1 . the model consists of three main phases. the first phase is preprocessing where the raw data was prepared to further steps. the second phase is feature extraction and the third phase is training the model. after model construction, test set was fed to the model for model evaluation. the following are details of each phase. ## experiments and results we proposed seven classifiers with tf/idf, and five classifiers with word2vec. all algorithms were implemented on nadi 2022 shared task dataset for subtask1. we calculated four evaluation metrics, accuracy (acc), precision (p), recall (r), and f1-score to measure the performance of our models. the macro-averaged f1-score is the official metric for subtask1. we implemented svm with two different kernels, linear kernel and radial bases function (rbf). different numbers of hidden layers (@xmath0, 20, 30 and 40) have been implemented in mlp. from ## conclusion and future work in this paper, a simple framework for dialect identification has been introduced. two main vectorization approaches (tf/idf and word embeddings) have been compared. it is clear from results that word embeddings outperforms tf/id. from this study, we can conclude that dialect identification of arabic text is one of the most challenging tasks. the results of training using mlp (@xmath2. cnb with tf/idf comes in the second as it can deal with unbalanced text data. in future work, pre-trained models could be used to improve the performance of classification, such as bert @xcite , @xcite , marbert model @xcite . transfer learning can be applied that knowledge from one domain can be transferred to another domain.