NADI 2022: The Third Nuanced A rabic Dialect Identification Shared Task (2022)

Abstract: We describe the findings of the third Nuanced Arabic Dialect Identification Shared Task (NADI 2022). NADI aims at advancing state-of-the-art Arabic NLP, including Arabic dialects. It does so by affording diverse datasets and modeling opportunities in a standardized context where meaningful comparisons between models and approaches are possible. NADI 2022 targeted both dialect identification (Subtask 1) and dialectal sentiment analysis (Subtask 2) at the country level. A total of 41 unique teams registered for the shared task, of whom 21 teams have participated (with 105 valid submissions). Among these, 19 teams participated in Subtask 1, and 10 participated in Subtask 2. The winning team achieved F1=27.06 on Subtask 1 and F1=75.16 on Subtask 2, reflecting that both subtasks remain challenging and motivating future work in this area. We describe the methods employed by the participating teams and offer an outlook for NADI.

## introduction arabic is a collection of languages and language varieties some of which are not mutually intelligible, although it is sometimes conflated as a single language. classical arabic (ca) is the variety used in old arabic poetry and the qur'an, the holy book of islam. ca continues to be used to date, side by side with other varieties, especially in religious and literary discourses. ca is also involved in code-switching contexts with modern standard arabic (msa) @xcite . in contrast, as its name suggests, msa is a more modern variety @xcite of arabic. msa is usually employed in pan-arab media such as aljazeera network and in government communication across the arab world. 1 dialectal arabic (da) is the term used to collectively refer to 1 https://www.aljazeera.com/ arabic dialects. da is sometimes defined regionally into categories such as gulf, levantine, nile basin, and north african @xcite . more recent treatments of da focus on more nuanced variation at the country or even sub-country levels @xcite . many of the works on arabic dialects thus far have focused on dialect identification, the task of automatically detecting the source variety of a given text or speech segment. in this paper, we introduce the findings and results of the third nuanced arabic dialect identification shared task (nadi 2022). nadi aims at encouraging research work on arabic dialect processing by providing datasets and diverse modeling opportunities under a common evaluation setup. the first instance of the shared task, nadi 2020 @xcite , focused on province-level dialects. nadi 2021 @xcite , the second iteration of nadi, focused on distinguishing both msa and da according to their geographical origin at the country level. nadi 2022 extends on both editions and offers a richer context as it targets both arabic dialect identification and and dialectal sentiment analysis. nadi 2022 shared tasks proposes two subtasks: subtask 1 on dialect identification, and subtask 2 on dialect sentiment analysis. while we invited participation in either of the two subtasks, we encouraged teams to submit systems to both subtasks. by offering two subtasks, our hope was to receive systems that exploit diverse machine learning and other methods and architectures such as multi-task learning systems, ensemble methods, sequence-tosequence architectures in single models such as the text-to-text transformer, etc. many of the submitted systems investigated diverse approaches, thus fulfilling our objective. a total of 41 unique teams registered for nadi 2022. of these, 21 unique teams actually made submissions to our leaderboard (@xmath0. we received 16 papers from 15 teams, of which we accepted 15 for publication. results from participating teams show that both dialect identification at the country level and dialectal sentiment analysis from short sequences of text remain challenging even to complex neural methods. these findings clearly motivate future work on both tasks. the rest of the paper is organized as follows: section 2 provides a brief overview of arabic dialect identification and sentiment analysis. we describe the two subtasks and nadi 2022 restrictions in section 3. section 4 introduces shared task datasets and evaluation setup. we present participating teams and shared task results and provide a high-level description of submitted systems in section 5. we conclude in section 6. ## conclusion and future work we presented the findings and results of the third nuanced arabic dialect identification shared task, nadi 2022. the shared task has two subtasks: subtask 1 on country-level dialect identification (including 18 countries) and subtask 2 on dialectal sentiment analysis (including 10 countries). nadi continues to be an attractive shared task, as reflected by the wide participation: 41 registered teams, 21 submitting teams scoring 105 valid models, and 15 published papers. results obtained by the various teams show that both dialect identification and dialectal sentiment analysis of short text sequences remain challenging tasks. this motivates further work on arabic dialects, and so we plan to run future iterations of nadi. our experience from nadi 2022 shows that inclusion of additional subtasks, along with dialect identification, provides a rich context for modeling. hence, we intend to continue adding at least one subtask (e.g., sentiment analysis covering more countries, emotion detection) to our main focus of dialect identification. we will also consider adding a data contribution track to nadi. in that track, teams may collect and label new datasets for public release.