Semi-supervised Fine-grained Approach for A rabic dialect detection task (2020)

Abstract: Arabic being a language with numerous different dialects, it becomes extremely important to device a technique to distinguish each dialect efficiently. This paper focuses on the fine-grained country level and province level classification of Arabic dialects. The experiments in this paper are submissions done to the NADI 2020 shared Dialect detection task. Various text feature extraction techniques such as TF-IDF, AraVec, multilingual BERT and Fasttext embedding models are studied. We thereby, propose an approach of text embedding based model with macro average F1 score of 0.2232 for task1 and 0.0483 for task2, with the help of semi supervised learning approach.

## introduction as arabic has a diverse collection of dialects and as dialect arabic varies phonologically, lexically, and morphologically from modern standard arabic @xcite , fine-grained dialect identification becomes an important task. the classification helps for studying various indigenous properties among different dialect speakers. previously, work on fine-grained city-level classification has been done @xcite , for madar tasks @xcite . nadi 2020 task consist of 21 class country-level dialect labels and 100 class province-level fine-grained dialect labels. with recent development in text classification, from emerging techniques like embedding models, not only for english, but trained on multiple languages have opened opportunities for arabic text classification. as models like word2vec, fasttext and bert perform excellent in text classification of english twitter data, these can be extended into the usage of arabic languages. features extracted from these models are then further fine tuned using neural network layers, to fit the required classification. in this paper we propose a model giving extremely good scores, using a combination of labelled and unlabelled data, by semi-supervised learning technique. unlabelled 10 million twitter data is collected with their id provided for the challenge and the top confidence predictions are added to extend the main labelled data set to increase the training set. repeated training with the added data in each step, increased the f1 score significantly. the remainder of the paper is organized as follows. section 2 discusses the description of the data set and preprocessing methods used. section 3 outlines the features used for the experiments for both subtask 1 and subtask 2. results are discussed in section 4. section 5 concludes the paper. ## data description and processing the data provided in the nadi 2020 task consist of 21,000 labelled and 10 million unlabelled data set for training, 4,957 tweet dev set and 5,000 tweet test set. the data consist of 21 different country-level labels and 100 different province-level labels with unequal distribution of tweets in various classes. the details about the shared task is described in @xcite this work is licensed under a creative commons attribution 4.0 international license. license details: http:// creativecommons.org/licenses/by/4.0/ . additionally 10 million tweet ids were given without labels. these unlabelled tweets were extracted by using the twitter api. out of the 10 million tweets approximately 92% of tweets are available and is considered for semi-supervised learning. as the texts are direct tweet messages, it contains links, hashtags and numbers, which is against the context of our study. so tweets are cleaned by removing the unwanted links, english and special characters, considering just the arabic text to maintain generality. ## experimental setup for both subtask1 and subtask2, tf-idf, aravec, bert and fasttext features are extracted and these feature vectors are fine-tuned to classify dialect classes by appending an extra layer. by this method of transfer learning we have established an effective and faster way of training the model. due to the large number of classes and great class imbalance, macro average f1 score is suitable and is used in the nadi task. also accuracy, precision and recall are observed. ## results the pre-trained models for arabic corpus is hard to find, making models like bert perform slightly lesser than expected. even though normalized count based model is basic, tf-idf method showed relatively better results than the aravec model, as tf-idf transform is fitted on the training corpus, and not using different corpus. out of all models trained on only the labelled training data, the fasttext model performs the best as its implementation with the magnitude library @xcite , as it assigns the same id for repeating words that doesn't exist in the pre-trained corpus. by this model we reach an excellent f1 score of 0.1890 for country-level classification and 0.0437 for province-level classification, on the dev set. the results of each feature extraction technique are observed and tabulated in in this paper, we explore various models such as tf-idf, aravec, multi-bert and ar-fasttext. we could see that the fasttext model performed relatively better than the other models for such small training set. and also we could see the improvement of performance, by semi-supervised learning technique using the unlabelled data. by this strategy we achieve 0.2232 and 0.0483 macro avg f1 for subtask 1 and subtask 2 respectively on dev data set. the model showed 0.2004 and 0.0403 macro avg f1 for subtask 1 and subtask 2 on test data set.