The QMUL / HRBDT contribution to the NADI A rabic Dialect Identification Shared Task (2020)

Abstract: We present the Arabic dialect identification system that we used for the country-level subtask of the NADI challenge. Our model consists of three components: BiLSTM-CNN, character-level TF-IDF, and topic modeling features. We represent each tweet using these features and feed them into a deep neural network. We then add an effective heuristic that improves the overall performance. We achieved an F1-Macro score of 20.77% and an accuracy of 34.32% on the test set. The model was also evaluated on the Arabic Online Commentary dataset, achieving results better than the state-of-the-art.

## introduction arabic is widely spoken in the middle east and certain parts of africa-21 countries-and the total number of its speakers is approximately 420 million. it is also one of the six official languages of the united nations. but arabic is a general term that can refer to classical arabic (ca), modern standard arabic (msa), or several arabic dialects (ads). both classical arabic and msa are standardized, while arabic dialects are not. so identifying different arabic dialect varieties is quite a challenging task. thus, many studies have been devoted to arabic dialect identification, because it benefits automatic speech recognition, remote access, e-health, and other applications @xcite . the main challenge for arabic dialect identification is the lack of large dataset that can be exploited in computational models. among the studies addressing this issue, zaidan and callison-burch (2014) created a large dataset for this purpose, called arabic online commentary @xcite . the data consists of texts in msa and arabic regional dialects collected from arabic news sites. @xcite collected and annotated twitter data from 11 regions and 16 countries in the arab world. @xcite also collected twitter data and annotated them at the city-level i.e 29 cities of 11 countries. @xcite translated the basic traveling expression corpus (btec) @xcite into 25 city dialects of arab countries; this corpus is referred to as corpus-25. anong the models, some proposals only aim to separating one dialect (e.g., egyptian) from msa on the aoc dataset @xcite . @xcite also distinguished between egyptian and msa, but focusing on twitter data. zaidan and callison-burch (2014) proposed a model to identify msa and regional dialects including egyptian, levantine, and gulf. @xcite applied a word-level n-gram model to identify msa and regional dialects, and also considered facebook posts . @xcite used the aoc dataset to evaluate several machine learning and deep learning models. @xcite proposed a fine-grained dialect identification module for corpus-25 by applying a multinomial naive bayes classifier with a large set of features. the nuanced arabic dialect identification (nadi) shared task aims to incentivise research on identifying different arabic dialects in every arab country @xcite . the nadi dataset was collected from arabic twitter, and each tweet is labeled with two labels, country-label and provincelabel, for a total of 21 countries and 100 of their provinces. there are two subtasks in nadi: the first is targeting these country-level labels and the second one is targeting the provinces. in the rest of the paper, we firt introduce the datasets in section 2. our arabic identification model is discussed in section 3. we show the results on nadi subtask 1 and aoc in section 4 and discuss them in section 5. we conclude with section 6. ## data the nadi dataset was the main corpus used for training and testing the system. the dataset is partitioned into train, development, and test sets, containing 21,000, 4,957, and 5,000 tweets respectively. the test set was published unlabeled, and the system output was evaluated by the nadi shared task team. the training set was used to train our model while the development set was used to optimize model parameters. in addition, nadi also provided 10 million unlabeled tweets which we used to train a word embedding model for arabic tweets. our model was also evaluated against another dataset, the arabic online commentary (aoc). a subset of the data was annotated using crowd-sourcing and has been used in previous work @xcite . the aoc dataset classifies arabic dialects into three dialects in addition to the msa, so spoken dialects in different countries may be grouped together as one dialect. for example, the "gulf" dialect label may includes all spoken dialects in saudi arabia, united arab emirates, kuwait, bahrain, iraq, etc. (elaraby and abdul-mageed, 2018) benchmarked a portion of aoc, and applied various machine learning algorithms to identify msa and dialects based on three settings: ## discussion our model was evaluated on two datasets: aoc and nadi. the results on aoc are very high compared to the results on nadi because nadi data contains many difficult cases. in nadi, there are many figure 2: msa ratios of train and development sets in nadi 1 following the arabic transliteration scheme in @xcite ## conclusion we presented a model to identify arabic dialects based on three components: bilstm-cnn, characterlevel tf-idf, and topic modeling. we evaluated the model on the country-level subtask of nadi, and also on the aoc dataset. we showed their results and discussed the challenges of nadi dataset.