Faheem at NADI shared task: Identifying the dialect of A rabic tweet (2020)

Abstract: This paper describes Faheem (adj. of understand), our submission to NADI (Nuanced Arabic Dialect Identification) shared task. With so many Arabic dialects being under-studied due to the scarcity of the resources, the objective is to identify the Arabic dialect used in the tweet, country wise. We propose a machine learning approach where we utilize word-level n-gram (n = 1 to 3) and tf-idf features and feed them to six different classifiers. We train the system using a data set of 21,000 tweets—provided by the organizers—covering twenty-one Arab countries. Our top performing classifiers are: Logistic Regression, Support Vector Machines, and Multinomial Na ̈ıve Bayes.

## introduction arabic language users constitute the fastest-growing language group when it comes to the number of internet users. according to ( www.internetworldstats.com/stats7.htm ), arabic language internet users have grown by 9,348% followed by russian with a growth of 3,653% during the last twenty years. there is a serious lack of arabic contents in the internet. as one author put it, searching for "how to make ice cream?" in arabic yields 99,100 results, while the same search in english yields 37,600,000 hits. contrast this with twenty-two arab countries, holding 420m speakers, vs 260m native english speakers worldwide @xcite . the majority of the arabic content in the web is in modern standard arabic (msa), and though it is understandable to most of the arabs, the mass majority do not use it in their daily communication. rather they switch to colloquial or informal arabic, which was originally confined to the domain of verbal communication @xcite . social media gave masses the opportunity to write in whatever language suites them, and this led to a rise in writing informally. the dialectical arabic differs from region to another, and the vocabulary of some dialects overlaps with msa by as much as 90% @xcite ; however, differences include some very common words such as those meaning "see", "go", and "not", as well as phonology, syntax and morphology rules @xcite . dialect identification is the task of automatically identifying the dialect of a particular part of text @xcite . arabic dialects differ from one region to another, and there are no available dictionaries for their vocabulary nor written rules. the range of arabic dialects is much more varied than the range of dialects that are typically considered to comprise european languages such as @xcite . even with the lack of written rules there are common set of conventional orthography guidelines for arabic dialect @xcite . there guidelines may give clue to the dialect. the nuanced arabic dialect identification (nadi) shared task is associated with the fifth arabic natural language processing workshop (wanlp 2020) @xcite . the shared task follows the efforts of arabic natural language processing (nlp) workshops and conferences that have taken place in the recent years @xcite . @xcite describes the first effort to collect a large-scale corpus and lexicon for dialectal arabic. the madar corpus is what was used to build the madar shared task. @xcite is another important work where the authors worked on city-level dialect identification. the nadi shared task consists of two subtasks. the goal of the first subtask is to identify the origin country of the tweet based on the dialect. while the second subtask focused on province-level dialect, where the goal is to predict the country province of the tweet. the organizers provided a data set of 21,000 labeled tweets covering 21 arab countries with country and province labels. in addition, they provided 10m unlabeled tweets covering different arab countries. we briefly summarize the challenges as: (a) the maximum length of a tweet is 280 characters which includes hashtags, special characters, urls, etc. this does not leave enough information-in term of words-to accurately predict the dialects; (b) some of the tweets are written in msa rather than country dialect which complicates the classification task; (c) certain similarity between dialects and the mixing between msa and dialect in a given text @xcite ; and (d) some of the tweets are re-tweets possibly from another different country with a different country dialect. in this paper, we develop a machine learning model to identify and predict the dialect of a tweet over 21 different arabic dialects (nadi shared task subtask 1). dialect identification can be seen as a multiclass text classification task, where we predict the probability of a dialect given a tweet (set of words) @xcite . our proposed system use word n-grams and tf-idf as features, and six different classifiers (e.g., multinomial naïve bayes, logistic regression). in developing the system we relied solely on the data provided by the shared task organizers, no external data was involved. we assume the data is free of spelling error, and/or context-sensitive error. the latter error occurs when a user types a correctly spelled word when another is intended @xcite , e.g. "i want a peace (piece) of cake". the rest of the paper is organized as follows. in section 2, we present the data set used in building our machine learning models. we describe the proposed system in section 3. section 4 discusses system performance and the results. we conclude with suggestions for future research in section 5. ## dataset and evaluation all the registered participants to the nadi shared task received the data set. the data set is divided into train, development (dev), and test sets (see the evaluation metrics includes precision (p ), recall (r), f -score, and accuracy. the official metric is the macro-averaged f -score, which computes the metric independently for each country and then take the average (hence treating all countries equally). let @xmath0, iraq, saudi, . . .} be the list of countries. if we denote f -score of classifying the tweet into country c by f 1 (c), then we can define the macro-average f -score as, ## system overview our goal is to accurately classify the tweets to one dialect out of 21 arabic dialects (corresponding to twenty-one arab countries). to develop our system, we used python, the machine learning library scikit-learn foot_0 @xcite , and nltk (natural language toolkit) library @xcite . there are three main stages in our system: pre-processing the data, feature extraction, and finally classification. ## conclusion in this paper we described our participation in the 2020 nuanced arabic dialect identification (nadi) shared task, where the goal is to build a system that can predict which of the 21 arabic dialects the tweet is written in. we submitted the result of executing our system on a standard test data using three different classifiers: svm, multinomial naïve bayes, and logistic regression. our official performance of macro-f 1 ≈ 0.151 using the multinomial naïve bayes classifier is not in par with our aspiration.