Country-level A rabic Dialect Identification using RNN s with and without Linguistic Features (2021)

Abstract: This work investigates the value of augmenting recurrent neural networks with feature engineering for the Second Nuanced Arabic Dialect Identification (NADI) Subtask 1.2: Country-level DA identification. We compare the performance of a simple word-level LSTM using pretrained embeddings with one enhanced using feature embeddings for engineered linguistic features. Our results show that the addition of explicit features to the LSTM is detrimental to performance. We attribute this performance loss to the bivalency of some linguistic items in some text, ubiquity of topics, and participant mobility.

## introduction arabic exhibits diglossia-the existence of two spoken varieties of a language side by side in a community @xcite ; while there are a multitude of informal regional varieties, modern standard arabic (msa) serves as the chief formal variety. not only the existence of the two spoken varieties is a complex situation for linguists to investigate @xcite , but it is more complex for data scientists to classify text data of such a language. while phonological differences are apparent in speech, the distinction is lost in writing, as all varieties use the same orthographic system. additionally, short vowels in the orthographic system are represented by a diacritic above each phoneme as " " meaning "he loved"; recently, however, these vocalic diacritics are dropped from any word as in " over the years, there have been several attempts at classifying arabic dialects, starting from classical natural language processing methods to deep learning whether throughout individual work or shared tasks such as madar series @xcite , which continued to enhance arabic dialect identification followed by the nadi series starting in 2019 @xcite . in 2013, @xcite implemented a supervised system for identifying msa and egyptian arabic at the sentence level, by predicting the level of formality of a sentence harvested from the web. observing the lack of the other arabic dialects' representation in previous work, zaidan and callison-burch (2014) constructed a corpus focused on including other arabic varieties. using n-gram and word character models, they were able to evaluate annotators' biases towards labeling text written in their own dialects. deep learning (dl) methods have revolutionized tasks such as large-scale language modeling @xcite @xcite , language identification @xcite , and sentiment analysis @xcite @xcite . the orthographic overlap between msa and regional dialects has posed a serious challenge to past work on fine-grained dialect identification. @xcite demonstrated that both recurrent and convolutional neural networks can surpass linear models such as logistic regression, multinomial naive bayes, and linear kernel support vector machines (svm) classifiers. other methods such as word vector modeling are able to identify some linguistic features of arabic tweet corpus @xcite . ## data in our experiments, we restricted ourselves to using only the official twitter corpus provided by the second nadi shared task @xcite . as a preprocessing step, we normalized all partitions of the data by removing non-arabic words, emojis, links, and excess white space. after normalization, we tokenized the tweets using keras @xcite . 10% of the training partition was set aside for monitoring validation loss in an effort to avoid overfitting through early stopping. foot_0 ## experiments in this work, we explored two approaches foot_1 to finegrained dialect classification. the first one involved using pretrained word embeddings as the input to an lstm @xcite used to encode each tweet. in experiment 2, we combined the lstm from experiment 1 with a feed-forward neural network that encodes a concatenation of low-dimensional dense embeddings representing explicit linguistic features. these linguistic features were used side-by-side with the word-level rnn from experiment 1 with the aim of supplementing our input with features deemed salient to dialect classification. ## results & discussion we evaluated the architectures from both experiments on development data provided for the task. based on the performance of the two systems, our submission for the shared task uses the architecture from experiment 1 which does not incorporate any engineered linguistic features. from a linguistic perspective, we believe that explicitly modeling salient features is a promising direction for improving our model; however, there are are a number of reasons this approach was unsuccessful here. sparse features our system has few features relative to the number of classes, and the frequency feature in the corpus (and thus their coverage) is low. that is, these features are insufficient to cover the set of documents available for each dialect. genre much of the data is characterized by what we call global genre-meaning that the content of the text contains global shared topics such as sports and popular culture. for instance, examples 3, 4, and 5 in no label tweet gloss type 1. oman hahaha stale joke hahaha bivalent linguistic units 2. syria haha don't believe but you deserve bivalent linguistic units 3. syria messi ... the state of being dead global genre 4. iraq more time to messi & nymear, barca will be dynamite global genre 5. algeria oh my dear, messi & nymar are demands global genre 6. algeria how will you win the europe champion while the coach is valverde? participant mobility 7. djibouti we want a trend everywhere alissa in beriut festival participant mobility 8. iraq oh the hassawi people are not likable shared gulf lexical items 9. oman a guy with good-looking face that i like to stare at cross-dialect linguistic units location granularity a number of the features included in this work are regional, rather than unique to a single dialect. an example of such a situation is example 8 in dataset design this is a challenging dataset. consider, for instance, the topic of participant mobility. examples 6 and 7 show some linguistic items that belong to different dialects compared to the location of the tweets. in this sense, the word " " in example 7 is a levantine lexicon, rather than a djibouti one. while examining the training data, we came across tweets that, to a native speaker, could just as easily be classified into any dialect (see examples 1 & 2). sociolinguists characterize these samples as bivalent linguistic units @xcite . such datapoints may provide very little signal for their assigned class. impoverished features rather than incorporating general linguistic features, our approach focused on devising distinctive word-level features that signify each dialect. considering the complexity of the task, our preliminary features are inadequate both in terms of their number (56) and distribution across linguistic levels (morphology, syntax, etc.). there are several arabic morphological analysis tools that could be used to greatly enrich and expand our set of features @xcite . in the same respect, @xcite presented a large parallel corpus of 25 arabic city dialects in the travel domain and a lexicon of 1,045 concepts with an average of 45 words from 25 cities per concept. while the approach presented here does not make use of these resources, our architecture was designed with extensibility in mind. we hope to explore this avenue in future work. ## conclusion in this work, we report results for two experiments: one that uses pretrained cbow word embeddings with an lstm, and another that integrates linguistic features as low-dimensional feature embeddings fed through a simple feed-forward network. the two experiments are used to classify arabic tweets for the second nuanced arabic dialect identification (nadi) subtask 1.2.: country-level da identification. the results show that rare linguistic features do not enhance the performance of an lstm with pretrained cbow word embeddings. these results emphasize the importance of pretrained models in nlp tasks related to arabic dialects.