Dialect Identification in Nuanced A rabic Tweets Using Farasa Segmentation and A ra BERT (2021)

Abstract: This paper presents our approach to address the EACL WANLP-2021 Shared Task 1: Nuanced Arabic Dialect Identification (NADI). The task is aimed at developing a system that identifies the geographical location(country/province) from where an Arabic tweet in the form of modern standard Arabic or dialect comes from. We solve the task in two parts. The first part involves pre-processing the provided dataset by cleaning, adding and segmenting various parts of the text. This is followed by carrying out experiments with different versions of two Transformer based models, AraBERT and AraELECTRA. Our final approach achieved macro F1-scores of 0.216, 0.235, 0.054, and 0.043 in the four subtasks, and we were ranked second in MSA identification subtasks and fourth in DA identification subtasks.

## introduction spoken by about 500 million people around the world, arabic is the biggest part of the semitic language family. being the official language of almost 22 countries belonging to the middle-east north africa (mena) region, it is not only an integral member of the six official un languages, but also fourth most used language on the internet @xcite . middle east contributes to 164 million internet users and north africa contributes to 121 million internet users. comparing with other languages, arabic language has received little attention in modern computational linguistics, despite its religious, political and cultural significance. however, with rapid development of tools and techniques delivering state-of-the-art performance in many language processing tasks, this negligence is being taken care of. the presence of various dialects and complex morphology are some of the distinguishing factors prominent in the arabic language. also, the informal nature of conversations on social media and the differences in modern standard arabic (msa) and dialectical arabic (da), both significantly increase this complexity. while da is used for informal daily communication, msa is used for formal writing. social media is the home for both of these forms, with the former being the most common form. lack of data is the primary reason why many of the arabic dialects remain understudied. with the availability of diverse data belonging to 21 arab countries, this bottleneck can be diminished. the nuanced arabic dialect identification (nadi), with this goal, is the task of automatic detection of the source variety of a given text or speech segment. previously, on the lines of arabic dialect identification, there have been approaches focusing on coarse-grained regional varieties such as levantine or gulf @xcite @xcite or country level varieties @xcite . there have been tasks that involved city level classification on human translated data @xcite . some tasks have focused on country and province level classification simultaneously @xcite . in this paper, we present our process to tackle the wanlp-2021 shared task 1. the paper is organised in the following way: section 2 presents the problem statement and details of the provided dataset. section 3 describes a modularised process that we inculcate as part of methodology. section 4 describes the experiments that were conducted, with detailed statistics about the dataset, system settings and results of these experiments. a brief conclusion of the paper with the potential prospects of our study are presented in section 5. ## task definition the wanlp-2021 shared task 1 @xcite is based on a multi-class classification problem where the aim is to recognize which country or province an arabic tweet in the form of modern standard arabic or dialect belongs to. the task targets dialects at the province-level, and also focuses on naturally-occurring fine-grained dialects at the sub-country level. the nadi 2021 task promotes efforts made towards distinguishing both modern standard arabic (msa) and dialects (da) according to their geographical origin, focusing on fine-grained dialects with new datasets. the provided data comes from the domain of twitter and covers 100 provinces from 21 arab countries. the task is divided into 4 subtasks as described below: subtask 1. the training dataset has a total of 21,000 tweet, validation and test datasets have 5,000 tweets each. every example belongs to one of 100 provinces of 21 arab countries. additional 10m unlabeled tweets are provided that can be used in developing the systems for either or both of the tasks. f-score, accuracy, precision and recall are the evaluation metrics. however, the official metric of evaluation is the macro averaged f-score. ## methodology we present our methodology in two parts. the first part in the methodology is data preprocessing. this is followed by experimenting with different transformer based models for the task at hand. both these parts have been described in detail in the following sub sections. ## experiments we experiment with eight transformer based models using the given training and validation sets. we calculate the final test predictions by fine tuning the most efficient model, which is decided by the scores produced above, with the concatenated labeled training and validation splits. this is followed by evaluating the test set on this fine tuned model. this section presents the country-level dataset distribution, system settings, results of our research followed by a descriptive analysis of our system. from the above results, we choose arabertv2large for subtasks 1.1, 1.2 and arabertv0.2-base for subtasks 2.1, 2.2 to be the primary models to fine tune on the concatenated training and validation set as well as carry out inferences on the unseen dataset. the final test set results in terms of macro f1 score(m-f1), recall(r), accuracy(a) and precision(p) are specified in ## conclusion and future work in this paper, we present a comprehensive overview of the approach that we employed to solve the eacl wanlp-2021 shared task 1. we tackle the given problem in two parts. the first part involves pre processing the given data by modifying various parts of the text. the second part involves experimenting with different versions of two transformer based networks, arabert and araelectra, all pre-trained on arabic text. our final submissions for the four subtasks are based on the best performing version of arabert model. with macro averaged f1-score as the final evaluation criteria, our approach fetches a private leaderboard rank of 2 for msa identification and 4 for da identification. in the future, we aim to utilise other features relevant for classification tasks like urls, emoticons, and experiment with ensembles of transformer based and word vector based input representations.