A rabic Dialect Identification based on a Weighted Concatenation of TF - IDF Features (2021)

Abstract: In this paper, we analyze the impact of the weighted concatenation of TF-IDF features for the Arabic Dialect Identification task while we participated in the NADI2021 shared task. This study is performed for two subtasks: subtask 1.1 (country-level MSA) and subtask 1.2 (country-level DA) identification. The classifiers supporting our comparative study are Linear Support Vector Classification (LSVC), Linear Regression (LR), Perceptron, Stochastic Gradient Descent (SGD), Passive Aggressive (PA), Complement Naive Bayes (CNB), MutliLayer Perceptron (MLP), and RidgeClassifier. In the evaluation phase, our system gives F1 scores of 14.87% and 21.49%, for country-level MSA and DA identification respectively, which is very close to the average F1 scores achieved by the submitted systems and recorded for both subtasks (18.70% and 24.23%).

## introduction nowadays, the study of arabic dialects has become the focus of many researches, and many workshops have been dedicated for this interesting topic. one of the noteworthy works are those featured in madar 2019 @xcite , we can mention @xcite @xcite where authors used machine learning techniques combined with resemble classifier or major voting rules to enhance the performance of the used models. the madar 2019 has been followed by nadi 2020 in which more dialects from 100 arab provinces have been covered @xcite . @xcite ; el @xcite ; @xcite , authors adopted the combination of ngrams and tf-idf features. an analysis of the impact of preprocessing and n-grams on automatic classification of tweets that mention medications has been presented in @xcite . in @xcite , the authors studied the effect of tf-idf features and morphological process on profiling fake news spreaders on twitter. in some works, because of the imbalanced nature of the used dataset, there is a need to apply sampling techniques: upsampling the minority class as presented in @xcite using the scikit-learn library @xcite , or the use of oversampling techniques from imblearn toolbox @xcite like: random over-sampler (ros), synthetic minority oversampling technique (smote) and adaptive synthetic (adasyn) as described in @xcite . in this work, we combined multiple classifiers, using a combination of ngrams and tf-idf as features @xcite with the same concept as suggested in abu @xcite ; touileb (2020) by using a weighted concatenation of these features. we also readapted the oversampling approach proposed in @xcite , in addition to the morphological preprocessing steps that we applied like stemming, lemmatization and pos tagging. this paper is structured as follows: in section 2, a description of the used dataset is presented. the applied cleaning steps and preprocessing are explained in section 3. in section 4, we present the proposed approach and discuss the findings. finally, we conclude in section 5. ## description of the dataset the used dataset in this shared task of nadi 2021 @xcite is an extension of the previous shared task @xcite . it is composed of two parts, the first one includes tweets written in modern standard arabic (msa), and the second one in the local arabic dialects (da). these two subsets (msa and da) include tweets from 21 arab countries and 100 provinces. we divided these subsets into three parts: train, dev and test, for which we addressed some statistics in ## data cleaning and preprocessing in most nlp tasks, dealing with tweets necessitates cleaning before analysis and processing. hence we applied some simple surface cleaning steps including arabic letter normalizer (aln) and the removal of: punctuation (rp), emojis (i.e., emoticons, symbols & pictographs, transport & map symbols and flags (ios)) (re), stop words (rsw), arabic diacritics (rad), latin letter (rll) and words (rlw), repeated words (rrw) and chars (rrc). we used also three morphological preprocessing steps: wordnetlemmatizer (lem), isri arabic stemmer (stem) and postagger (postag) of nltk foot_0 . after a series of experiments in order to investigate the impact of all the possible combinations of the different aforementioned cleaning processes, we kept the best configuration which is composed of emojis removal, diacritics removal, repeated words removal, in addition to the stemming process. furthermore, we selected the features which yielded the best performance after testing a huge number of n-grams combinations (200 experiments). these features have been transformed using the tf-idf vectorizer and concatenated with (see ## proposed system it should be noted from figure 1 , that we first combined all the aforementioned preprocessing steps to generate all the possible textual presentations, after that we applied a second combination between n-grams (@xmath0, char, char with boundary, union of the three) to generate all the possible vector features. then we fed these features to the best oversampling procedures (i.e., adasyn) which were applied before training our three best classifiers (lsvc, rdg, sgd). finally we used an resemble classifier (i.e., majority voting rule) which takes as input the three predictions and applies a majority voting rule to predict the dialect. we divided our work into three setups. in the first setup, we conducted an empirical comparison of some classifiers @xcite , namely: linear support vector classification (lsvc), linear regression (lr), perceptron, stochastic gradient descent (sgd), passive aggressive (pa), complement naive bayes (cnb), mutlilayer perceptron (mlp), and ridgeclassifier. we used these classifiers in their default setup except for the sgd classifier for which we examined the performance in function of runs number. in the first setup, we used three types of features independently, it is about the tf-idf features of ngrams for words, chars and char wb. using different values of n (ngrams) resulted in diverse feature vectors using the eight aforementioned classifiers. we then ranked the best "features+classifiers" combinations according to the obtained f 1 scores, in order to use them in the following setups. the second setup is based on the first one, since we concatenate the best "features+classifiers", mentioned above, without weighting. the best performance is obtained by three classifiers, as shown in according to the results shown in in the third setup, we applied weighted concatenation of the features used in the second setup. we used multiple values of the weights: w1, w2 and w3 for word, char, and char wb analyzers, respectively. the best performance is recorded for both msa and da subtasks using a majority voting rule on the three classifiers svc, sgd, and rdg with an f 1 -score of 14.90% and 20.09%, respectively. the best performance for the second subtask is achieved using the svc classifier with an f 1 -score of 21.25%. we summarize the results in for the first subtask (msa), we obtained these performances by applying a set of surface and morphological preprocessing followed by a union of (word 5-grams, char (2,8) grams and char wb (2,10)) multiplied by a vector weights (1.3, 1.3, 1), in addition to the intern parameters of sgd classifier(l1 @xmath1.25, @xmath2,@xmath3. we obtained f 1 =16.30% (dev) f 1 =14.50% (test). for the second subtask (da), we included the adasyn oversampling technique to the configuration used in the first subtask while changing the preprocessing, ngrams range used by union-vect, vector weight as well as the parameters of configs preprocess union weights oversampling sgd conf1 rsw, rad, aln, rrw, rll, stem, lem 1, 5, 2, 8, 2, 10 1.3, 1.3, 0.9 n/a l1 @xmath4.25, @xmath5, @xmath6.3, 1.3, 1.0 conf3 1.3, 1.3, 1.9 conf4 re, rad, rrw, stem 1, 3, 1, 4, 2, 4 1.8, 0.7, 1.8 @xmath7, @xmath8, random @xmath9, random @xmath10: performance (f1) achieved using weighted concatenation and oversampling techniques for both subtasks. the sgd classifier. we obtained performance equal to 23.01% and 21.13% for development and test phases, respectively. ## conclusion in this paper, we described our two submitted systems for identifying arabic dialects, using twitter texts collected from 21 arabic countries. we carried out an empirical comparison study by conducting 200 experiments in which we used several combinations of features, and preprocessing steps as stemming, lemmatization and part of speech tagging. we used a simple classification technique comprising three classifiers: lsvc, rdg and sgd. we addressed the issue of imbalanced data using a set of oversampling methods. furthermore, we tried to investigate the impact of concatenating tf-idf features with and without weight vector.