ISL - AAST at NADI 2023 shared task: Enhancing A rabic Dialect Identification in the Era of Globalization and Technological Progress (2023)

Abstract: Arabic dialects have extensive global usage owing to their significance and the vast number of Arabic speakers. However, technological progress and globalization are leading to significant transformations within Arabic dialects. They are acquiring new characteristics involving novel vocabulary and integrating of linguistic elements from diverse dialects. Consequently, sentiment analysis of these dialects is becoming more challenging. This study categorizes dialects among 18 countries, as introduced by the Nuanced Arabic Dialect Identification (NADI) shared task competition. Our approach incorporates the utilization of the MARABERT and MARABERT v2 models with a range of methodologies, including a feature extraction process. Our findings reveal that the most effective model is achieved by applying averaging and concatenation to the hidden layers of MARABERT v2, followed by feeding the resulting output into convolutional layers. Furthermore, employing the ensemble method on various methods enhances the model’s performance. Our system secures the 6th position among the top performers in the First subtask, achieving an F1 score of 83.73%.

## introduction the arabic region encompasses numerous cultures, each characterized by dialectal variations influenced by historical, geographical, and sociopolitical factors @xcite . while this variety showcases the region's cultural wealth, it creates difficulties when analyzing arabic information, especially on social media networks. moreover, the rapid evolution of the language in the digital age and the widespread use of social media are presenting a new era for the arabic language. modern communication tools are enabling speakers of various arabic dialects to interact globally. this interaction is leading to a dynamic evolution of the language, characterized by the emergence of new vocabulary, slang, and expressions @xcite . the continuous generation of new words and language adaptations is presenting a unique challenge for linguistic analysis. therefore, modern standard arabic has a disequilibrium between preserving tradition and adjusting to the demands of modern communication. moreover, arabic dialect identification plays a pivotal role in understanding regional language variations on social media. improving this task has implications for cultural preservation, social analysis, and natural language processing technology. however, the presence of diverse arabic dialects with distinct linguistic traits can pose challenges in analyzing and interpreting social media content @xcite . people from different regions might use completely different words to express the same concepts. recent advancements in arabic dialect identification research have been notable, with various studies addressing the intricate nuances of arabic dialects. the madar shared task on fine-grained dialect identification @xcite delved into sub-dialect distinctions, highlighting the complexity of arabic language variations. machine translation of arabic dialects @xcite focused on adapting translation models to handle dialect-specific expressions, facilitating communication across dialect differences. moreover, efforts in the automatic identification of arabic dialects in social media @xcite utilized natural language processing and machine learning to automate dialect recognition, revealing regional language trends online. various methods, including feature extraction and machine learning algorithms @xcite , have contributed to improving automated dialect identification accuracy and uncovering the rich diversity of arabic dialects. in the recent nadi shared task series @xcite @xcite , teams employed a range of approaches, including traditional methods like svm with tf-idf @xcite , customized bert-based models @xcite , and deep learning techniques with models like marbert and arabert @xcite @xcite . these efforts collectively contribute to the advancement of arabic dialect identification, showcasing diverse methodologies and approaches in the field. in this research, we aim to enhance the f1 score of arabic dialect identification, provided by nadi shared task 2023 @xcite , by investigating the impact of various model enhancements. our study conducts a series of experiments using marbert and marbert v2 models @xcite , involving various techniques. this approach includes concatenating hidden layers @xcite and processing the resulting outputs using cnn layers @xcite , bilstm models @xcite , or a combination of bilstm and cnn. additionally, experiments involve adapters with the marbert model @xcite . finally, to maximize our results, our work utilizes ensemble methods that combine the outcomes of the majority of these experiments @xcite . the rest of the paper is organized as follows: providing the dataset and its preparation are presented in section 2. in section 3, we explain the methodologies employed for arabic dialect identification. subsequently, section 4 presents the results of our model's performance, including an analysis of our findings. in section 6, we summarize and conclude our findings. 2 data 2.1 dataset description the presented approach utilized the training and validation data provided by the task organizers @xcite . the training set for subtask 1 consists of around 18,000 tweets with 18 different labels representing 18 country dialects. while the development set consists of 1,800 labeled tweets. the submitted results were evaluated on a test set consisting of 3,600 tweets covering 18 country-level dialects. ## results and discussion we present a summary of our experimentation and evaluation of various model enhancements, as reported in in our extensive series of experiments, we conduct an exploration of various model enhancements with a primary focus on optimizing f1 scores. among these experiments, exp.14 showcases the best results as a standalone model. as shown in figure 2 , this model is built upon the foundation of exp.7 with the addition of three convolution filters (kernel sizes: 10, 8, 6), followed by max-pooling, and also demonstrates robustness with an impressive f1-score of 81.67% on the test dataset. these results emphasize the significance of spatial feature extraction in text classification tasks. regarding our methodological approach, exp.17 represents the most effective method. it serves as our submission and leverages ensemble techniques to combine the predictions of multiple models. this ensemble method significantly outperforms individual models, achieving outstanding f1-scores of 85.27% for the dev dataset and 83.73% for the test dataset. notably, we observe instances of misclassification between the two classes, notably between jordan and palestine, as well as between kuwait and bahrain, as illustrated in figure 3 . these misclassifications can be attributed to several factors, including historical, cultural, and linguistic nuances that may pose challenges for natural language processing models. the misclassification of content related to kuwait and bahrain is a result of shared geographical proximity and cultural ties, leading to overlapping themes and terminology in text data. these overlapping characteristics can cause our models to occasionally struggle in correctly differentiating between the two, resulting in fluctuations in classification performance. these observed misclassifications underscore the need for continued research and model refinement, especially when dealing with regions or topics characterized by subtle distinctions. addressing such complexities will contribute to improving the accuracy and robustness of models in handling cases with inherent challenges like those presented by jordan vs. palestine and kuwait vs. bahrain. with more time available, we will delve into training on larger datasets. additionally, our study will explore the use of different loss functions for various hyperparameters and incorporate additional ensemble methods such as stacking, bagging, boosting, random forests, adaboost, and gradient boosting. ## conclusion overall, this paper outlines our methods for solving nuanced arabic dialect identification (nadi) shared task 2023 subtask-1. the extensive experimentation and analysis highlighted the nuanced nature of model enhancements and adaptations. some modifications, like layer concatenation and the addition of convolution layers, exhibited clear benefits. on the other hand, adapters had more limited impacts. additionally, ensemble methods emerged as a powerful tool for boosting the score. these findings emphasize the need for a thoughtful and data-driven approach when fine-tuning models for specific tasks and domains in natural language processing. our system ranks in the 6th best spots of the leaderboards of the first subtask with an f1-score of 83.73%. future research directions include investigating the impact of larger training datasets on model performance.