U ralic Language Identification ( ULI ) 2020 shared task dataset and the Wanca 2017 corpora (2020)

Abstract: This article introduces the Wanca 2017 web corpora from which the sentences written in minor Uralic languages were collected for the test set of the Uralic Language Identification (ULI) 2020 shared task. We describe the ULI shared task and how the test set was constructed using the Wanca 2017 corpora and texts in different languages from the Leipzig corpora collection. We also provide the results of a baseline language identification experiment conducted using the ULI 2020 dataset.

## introduction as part of the finno-ugric languages and the internet project, (suki) foot_0 we have collected textual material for some of the more endangered uralic languages from the internet @xcite . in this paper, we introduce the wanca 2017 corpora which will be published in the language bank of finland foot_1 as a downloadable package as well as through the korp foot_2 concordance service. the uralic language identification (uli) 2020 shared task foot_3 was organized as part of the vardial 2020 evaluation campaign. 5 in order to create a training dataset for the shared task, we used the earlier version of the corpora, wanca 2016 foot_5 @xcite , together with corpora available from the leipzig corpora collection foot_6 @xcite . different corpora from the leipzig corpora collection and a manually verified subset of the wanca 2017 corpora were used to create the test set for the shared task. we also performed a baseline language identification experiment for the uli dataset using the heli method described by @xcite . in this paper, we first introduce some related work and resources for language identification and the uralic languages in section 2. we then describe the wanca 2017 corpora and its creation in section 3. in section 4, we give a detailed description of the creation of the dataset for the uli 2020 shared task as well as the information about the baseline language identification experiments using the dataset. finally, we provide some error analysis for the results of those experiments. ## related work in this section, we first introduce some previous work on language identification of texts, then we give a short introduction to the uralic languages and present some of the text corpora already available for those languages. ## the uli 2020 shared task the uli 2020 shared task was organized as a part of the vardial 2020 evaluation campaign. the evaluation campaign was the 7th incarnation of a series of shared tasks concentrating on close languages which have always incorporated some form of language identification tasks @xcite @xcite @xcite . ## conclusions and future work in the beginning, we were worried about not getting enough new sentences from a simple re-crawl of the old addresses. in the end, the new sentences created an interesting setting for a language identification shared task. the three tracks highlighted different aspects of the problem of language identification. the next edition of the uli shared task will incorporate new sentences from the 2018 crawl performed by the suki project. before processing the crawled material, we aim to improve our sentence extraction algorithm in such a way that it could allow sentences to span line-breaks. also, as pointed out by one of the reviewers, it would be a good idea to manually inspect at least a random subset of the wanca 2017 corpora in order to objectively assess the reliability of the language annotation process. unlike the 2017 re-crawl, the 2018 crawl was a real crawl going beyond the addresses stored in the wance service. thus, we expect to find more new sentences after we process the material using the improved process.