Findings of the V ar D ial Evaluation Campaign 2022 (2022)

Abstract: This report presents the results of the shared tasks organized as part of the VarDial Evaluation Campaign 2022. The campaign is part of the ninth workshop on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects (VarDial), co-located with COLING 2022. Three separate shared tasks were included this year: Identification of Languages and Dialects of Italy (ITDI), French Cross-Domain Dialect Identification (FDI), and Dialectal Extractive Question Answering (DialQA). All three tasks were organized for the first time this year.

## introduction the workshop series on nlp for similar languages, varieties and dialects (vardial), traditionally co-located with international conferences, has reached its ninth edition. since the first edition, vardial has hosted shared tasks on various topics such as language and dialect identification, morphosyntactic tagging, question answering, and cross-lingual dependency parsing. the shared tasks have featured many languages and dialects from different families and data from various sources, genres, and domains @xcite @xcite @xcite @xcite . we offered three shared tasks as part of the var-dial evaluation campaign 2022, which we present in this paper: identification of languages and dialects of italy (itdi), french cross-domain dialect identification (fdi), and dialectal extractive question answering (dialqa). this overview paper is structured as follows: in section 2, we briefly introduce the three shared tasks. section 3 presents the teams that submitted systems to the shared tasks. each task is then discussed in detail, focusing on the data, the participants' approaches, and the obtained results. sec-figure 1 : rough regions where the eleven considered languages and dialects of italy are spoken. magenta: italo-dalmatian; turquoise: gallo-italian; yellow: gallo-rhaetian; red: sardinian. the map is vague; the situation is more complex. however, it gives an idea of where in italy to locate the varieties. tion 4 is dedicated to itdi, section 5 to fdi, and section 6 to dialqa. ## participating teams a total of five teams submitted runs to the itdi and fdi shared tasks. unfortunately, we did not receive any submissions for dialqa. in ## conclusion this paper presented an overview of the three shared tasks organized as part of the vardial evaluation campaign 2022: identification of languages and dialects of italy (itdi), french cross-domain dialect identification (fdi), and dialect extractive question answering (dialqa). participants of these shared tasks were provided with existing or new data sets made available to the community, which were discussed in detail in the respective sections. we furthermore included short descriptions of each team's systems, along with references to all system description papers published in the vardial workshop proceedings ( for the itdi task, we observed that shallow machine learning models outperformed deep learning models -even when using pre-trained language models for italian. in contrast, pre-trained french language models provided much better performances than shallow models in the fdi task. it seems therefore that the optimal model choice for language and dialect identification tasks is largely task-dependent. this confirms the findings of previous editions of the vardial campaign @xcite , where similar diverging trends were observed.