NADI 2020: The First Nuanced A rabic Dialect Identification Shared Task (2020)

Abstract: We present the results and findings of the First Nuanced Arabic Dialect Identification Shared Task (NADI). This Shared Task includes two subtasks: country-level dialect identification (Subtask 1) and province-level sub-dialect identification (Subtask 2). The data for the shared task covers a total of 100 provinces from 21 Arab countries and is collected from the Twitter domain. As such, NADI is the first shared task to target naturally-occurring fine-grained dialectal text at the sub-country level. A total of 61 teams from 25 countries registered to participate in the tasks, thus reflecting the interest of the community in this area. We received 47 submissions for Subtask 1 from 18 teams and 9 submissions for Subtask 2 from 9 teams.

## related work as we explained in section 1, arabic could be viewed as comprised of 3 main types: ca, msa, and da. while ca and msa have been studied and taught extensively, da has only received more attention relatively recently @xcite @xcite @xcite . a majority of da computational efforts have targeted creating resources for country or regionally specific dialects @xcite @xcite @xcite @xcite @xcite . the expansion into multi-dialectal data sets and models to identify them was initially done at the regional level @xcite @xcite . a number of arabic dialect identification shared tasks were organized as part of the vardial workshop. these focused on regional varieties such as egyptian, gulf, levantine, and north african based on speech broadcast transcriptions @xcite but also acoustic features @xcite and phonetic features @xcite extracted from raw audio. althobaiti (2020) presents a recent survey of computational work on arabic dialects. an early effort for creating finer grained parallel dialectal corpus and lexicon was done under the multi arabic dialects application and resources (madar) project @xcite . the parallel data was created by commission under controlled settings to maximize its use for cross-dialectal comparisons and machine translation. their data was also used for dialectal identification at the city level @xcite of 25 arab cities. one issue with the madar data in the context of identification is that it was commissioned and not naturally occurring. concurrently, larger twitter-based datasets covering 10-21 countries were also introduced @xcite @xcite . researchers are also starting to introduce da datasets labeled for socio-pragmatics, e.g., @xcite . the madar shared task @xcite comprised two subtasks, one focusing on 21 arab countries exploiting twitter data manually labeled at the user level, and another on 25 arab cities mentioned above. during the same time as nadi, @xcite describe data and models at country, province, and city levels. the nadi shared task follows these pioneering works by availing data to the (arabic) nlp community, and encouraging work on arabic dialects. similar to the madar shared task, we include a country-level dialect identification task (subtask 1), and a sub-country dialect identification task (subtask 2). however, our sub-country task is a province-level identification task with a much larger label set than madar's city-level task, and is based on naturally occurring data. we hope that our work will be setting the stage for exploring variation in geographical regions that have not been studied before. ## task description the nadi shared task consists of two subtasks for country-level and province-level classification. ## shared task datasets we distributed a single dataset with two sets of labels, one for subtask 1 and another for subtask 2. in other words, the same tweet occurs in each of the two subtasks but with different subtask-specific labels. additionally, we made available an unlabeled dataset for optional use in any of the two subtasks. we now provide more details about both the labeled and unlabeled data. ## conclusion and future work we presented an overview of the nadi 2020 shared task. we described the dataset and identified areas of improvement especially related to its collection. we also provided a high-level description of participating teams. the number of submissions to the shared task reflects an interest in the community and calls for further work in the area of arabic dialect detection, but also more generally arabic dialect processing. in the future, we plan to host a second iteration of the nadi shared task that will use new datasets and pursue a number of novel questions inspired by the issues discovered in this year's task. for example, in addition to da classification, we will propose msa regional use classification as a subtask. since msa is shared across the arab world, we hypothesize this will be a more challenging task than da classification. we will also encourage teams to experiment with studying the interaction between msa and da in