Frank at NADI 2023 Shared Task: Trio-Based Ensemble Approach for A rabic Dialect Identification (2023)

Abstract: We present our system designed for Subtask 1 in the shared task NADI on Arabic Dialect Identification, which is part of ArabicNLP 2023. In our approach, we utilized models such as: MARBERT, MARBERTv2 (A) and MARBERTv2 (B). Subsequently, we created a majority voting ensemble of these models. We used MARBERTv2 with different hyperparameters, which significantly improved the overall performance of the ensemble model. In terms of performance, our systems achieved a competitive an F1 score of 84.76 . Overall, our system secured the 5^\text{th} position out of 16 participating teams.

## introduction the arabic language, with its vast and varied tapestry of dialects, offers a mesmerizing blend of history, culture and linguistic evolution. each dialect, from the mellifluous notes of levantine to the rhythmic cadences of maghrebi, narrates a unique story of its people, their journeys, and their experiences. however, such linguistic richness often goes unnoticed, overshadowed by mainstream dialects and a lack of comprehensive research tools. the persistent gaps in our understanding, exacerbated by limited resources, such as datasets, have made the exploration of these dialects both a challenge and a treasure hunt for researchers @xcite . in response to this, the series of nuanced arabic dialect identification (nadi) shared tasks, initiated by @xcite , emerged as a beacon of hope, spotlighting lesser studied dialects. over the years 2020 @xcite @xcite , and 2022 @xcite , nadi provided invaluable datasets and created a vibrant platform where scholars and enthusiasts could exchange insights, challenge conventional methodologies, and ignite renewed interest in dialect identification. this discipline, which is based on determining the variety of sources of textual or spoken content, has now become central to understanding the rich fabric of the arabic linguistic diversity. the subtask can be formulated as follows: identify the specific country-level dialect of a given arabic tweet. this task is armed with the novel twt-2023 dataset, which covers 18 mesmerizing dialects, and is supplemented by external datasets such as nadi-2020-twt, nadi-2021-twt and madar-2018 @xcite . our contributions are as follows: in section 2, we outline previous and more recent studies on dialect identification. in section 3, we illustrate a thorough examination of the dataset. in section 4 we describe the system and the results. lastly, section 5 presents our conclusion and proposes potential avenues for future research. ## related work arabic exists in three main forms: classical arabic (ca), modern standard arabic (msa), and dialectal arabic (da). although ca and msa have been thoroughly explored in previous research, interest in da has recently risen due to limited resources @xcite . the initial research on da was regional @xcite , later expanding to multi-dialectal studies @xcite @xcite figure 1 : statistics about tweets distribution in train and development sets. @xcite . the vardial workshop highlighted the identification of dialects using acoustic and phonetic traits @xcite . madar @xcite provided enriched dialect data, but faced authenticity questions in online contexts. recent work has taken advantage of the vast twitter datasets @xcite , with althobaiti (2022) introducing an unsupervised dialect-tagging approach. further, abdul-mageed et al. (2020b) investigated city-specific dialect variations. nadi's initiatives produced notable datasets on arabic dialect identification, including a detailed review by althobaiti (2020). nadi 2020 collaborated with wanlp 2020, leading to the categorization of dialects from 21 arab countries via twitter. nadi 2021, in association with wanlp 2021, improved its dataset, distinguishing between msa and da. this led to the development of four specific subtasks @xcite . in nadi 2022, the focus had shifted to sentiment analysis of data tagged with dialects. in particular, alsudais et al. (2022) integrated the madar and nadi datasets into their research. lastly, nadi 2023 introduced three subtasks: country-level dialect identification and closed-and open-speech machine translation from four dialects to msa. ## data this section provides a detailed explanation of the dataset made available by the nadi shared task organizers. ## conclusion and future work in this paper, we discussed our approach for subtask 1 of the shared task nadi in arabic dialect identification. we used the majority-voting ensemble with the marbert and marbertv2 (a) and marbertv2 (b) models and according to the official leaderboard results, our system achieved an f1 score of 84.76 outperforming two-thirds of the participating teams. we also detailed a series of experiments and made comparisons of our models with a majority-voting ensemble. in future work, we plan to enhance our ensemble approach with advanced transformer architectures (e.g., mbert and xlm-roberta) and data augmentation specific to arabic dialects (e.g., backtranslation or dialectical synonym replacement). moreover, we would like to investigate classifier calibration and soft voting.