NADI 2021: The Second Nuanced A rabic Dialect Identification Shared Task (2021)

Abstract: We present the findings and results of theSecond Nuanced Arabic Dialect IdentificationShared Task (NADI 2021). This Shared Taskincludes four subtasks: country-level ModernStandard Arabic (MSA) identification (Subtask1.1), country-level dialect identification (Subtask1.2), province-level MSA identification (Subtask2.1), and province-level sub-dialect identifica-tion (Subtask 2.2). The shared task dataset cov-ers a total of 100 provinces from 21 Arab coun-tries, collected from the Twitter domain. A totalof 53 teams from 23 countries registered to par-ticipate in the tasks, thus reflecting the interestof the community in this area. We received 16submissions for Subtask 1.1 from five teams, 27submissions for Subtask 1.2 from eight teams,12 submissions for Subtask 2.1 from four teams,and 13 Submissions for subtask 2.2 from fourteams.

## introduction arabic is the native tongue of ∼ 400 million people living the arab world, a vast geographical region across africa and asia. far from a single monolithic language, arabic has a wide number of varieties. in general, arabic could be classified into three main categories: (1) classical arabic, the language of the qur'an and early literature; (2) modern standard arabic (msa), which is usually used in education and formal and pan-arab media; and (3) dialectal arabic (da), a collection of geopolitically defined variants. modern day arabic is usually referred to as diglossic with a so-called 'high' variety used in formal settings (msa), and a 'low' variety used in everyday communication @xcite . da, the presumably 'low' variety, is itself a host of variants. for the current work, we focus on geography as an axis of variation where people from various sub-regions, countries, or even provinces within the same country, may be using arabic differently. the nuanced arabic dialect identification (nadi) series of shared tasks aim at furthering the study and analysis of arabic variants by providing resources and organizing classification competitions under standardized settings. the first nuanced arabic dialect identification (nadi 2020) shared task targeted 21 arab countries and a total of 100 provinces across these countries. nadi 2020 consisted of two subtasks: country-level dialect identification (subtask 1) and province-level detection (subtask 2). the two subtasks depended on twitter data, making it the first shared task to target naturally-occurring fine-grained dialectal text at the sub-country level. the second nuanced arabic dialect identification (nadi 2021) is similar to nadi 2020 in that it also targets the same 21 arab countries and 100 corresponding provinces and is based on twitter data. however, nadi 2021 has four subtasks, organized into country level and province level. for each classification level, we afford both msa and da datasets as ## related work as we explained in section 1, arabic has three main categories: ca, msa, and da. while ca and msa have been studied extensively @xcite @xcite @xcite , da has received more attention only in recent years. one major challenge with studying da has been rarity of resources. for this reason, most pioneering da works focused on creating resources, usually for only a small number of regions or countries @xcite @xcite @xcite @xcite @xcite . a number of works introducing multi-dialectal data sets and regional level detection models followed @xcite @xcite . arabic dialect identification work as further sparked by a series of shared tasks offered as part of the vardial workshop. these shared tasks used speech broadcast transcriptions @xcite , and integrated acoustic features @xcite and phonetic features @xcite extracted from raw audio. althobaiti (2020) is a recent survey of computational work on arabic dialects. the multi arabic dialects application and resources (madar) project @xcite introduced finer-grained dialectal data and a lexicon. the madar data were used for dialect identification at the city level @xcite of 25 arab cities. an issue with the madar data, in the context of da identification, is that it was commissioned and not naturally occurring. several larger datasets covering 10-21 countries were also introduced @xcite @xcite . these datasets come from the twitter domain, and hence are naturally-occurring. several works have also focused on sociopragmatics meaning exploiting dialectal data. these include sentiment analysis @xcite , emotion @xcite , age and gender @xcite , offensive language @xcite , and sarcasm @xcite . concurrent with our work, @xcite ) also describe data and models at country, province, and city levels. the first nadi shared task, nadi 2020 @xcite , comprised two subtasks, one focusing on 21 arab countries exploiting twitter data, and another on 100 arab provinces from the same 21 countries. as is explained in @xcite , the nadi 2020 datasets included a small amount of non-arabic and also a mixture of msa and da. for nadi 2021, we continue to focus on 21 countries and 100 provinces. however, we breakdown the data into msa and da for a stronger signal. this also gives us the opportunity to study each of these two main categories independently. in other words, in addition to dialect and sub-dialect identification, it allows us to investigate the extent to which msa itself can be teased apart at the country and province levels. our hope is that nadi 2021 will support exploring variation in geographical regions that have not been studied before. ## task description the nadi shared task consists of four subtasks, comprising two levels of classification-country and province. each level of classification is carried out for both msa and da. we explain the different subtasks across each classification level next. ## shared task datasets we distributed two twitter datasets, one in msa and another in da. each tweet in each of these two datasets has two labels, one label for country level and another label for province level. for example, for the msa dataset, the same tweet is assigned one out of 21 country labels (subtask 1.1) and one out of 100 province labels (subtask 2.1). the same applies to da data, where each tweet is assigned a country label (subtask 1.2) and a province label (subtask 2.2). similar to msa, the tagset for da data has 21 country labels and 100 province labels. in addition, as mentioned before, we made available an unlabeled dataset for optional use in any of the four subtasks. we now provide more details about both the labeled and unlabeled data. ## conclusion and future work we presented the findings and results of the nadi 2021 shared task. we described our datasets across the four subtasks and the logistics of running the shared task. we also provided a panoramic description of the methods used by all participating teams. the results show that distinguishing the language variety of short texts based on small geographical regions of origin is possible, yet challenging. the total number of submissions during official evaluation (@xmath0, as well as the number of teams who registered and acquired our datasets (@xmath1. in the future, we plan to host a third iteration of the nadi shared task that will use new datasets and encourage novel solutions to the set of problems introduced in nadi 2021. as results show all the fours subtasks remain very challenging, and we hope that encouraging further solutions will help advance work in this area. anshul wadhawan. 2021. dialect identification in nuanced arabic tweets using farasa segmentation and arabert. in proceedings of the sixth arabic natural language processing workshop (wanlp 2021). wajdi zaghouani and anis charfi. 2018. araptweet: a large multi-dialect twitter corpus for gender, age and language variety identification. in proceedings of the language resources and evaluation conference (lrec), miyazaki, japan. omar f zaidan and chris callison-burch. 2011. the arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. in proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies: short papers-volume 2, pages 37-41. association for computational linguistics. marcos zampieri, shervin malmasi, nikola ljubešić, preslav nakov, ahmed ali, jörg tiedemann, yves scherrer, and noëmi aepli. 2017. findings of the vardial evaluation campaign 2017. marcos zampieri, shervin malmasi, preslav nakov, ahmed ali, suwon shon, james glass, yves scherrer, tanja samardzic, nikola ljubešić, jörg tiedemann, et al. 2018. language identification and morphosyntactic tagging: the second vardial evaluation campaign. in proceedings of the fifth workshop on nlp for similar languages, varieties and dialects (vardial 2018), pages 1-17.