The SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion (2020)

Abstract: In this paper, we describe the findings of the SIGMORPHON 2020 shared task on unsupervised morphological paradigm completion (SIGMORPHON 2020 Task 2), a novel task in the field of inflectional morphology. Participants were asked to submit systems which take raw text and a list of lemmas as input, and output all inflected forms, i.e., the entire morphological paradigm, of each lemma. In order to simulate a realistic use case, we first released data for 5 development languages. However, systems were officially evaluated on 9 surprise languages, which were only revealed a few days before the submission deadline. We provided a modular baseline system, which is a pipeline of 4 components. 3 teams submitted a total of 7 systems, but, surprisingly, none of the submitted systems was able to improve over the baseline on average over all 9 test languages. Only on 3 languages did a submitted system obtain the best results. This shows that unsupervised morphological paradigm completion is still largely unsolved. We present an analysis here, so that this shared task will ground further research on the topic.

## introduction in morphologically rich languages, words inflect: grammatical information like person, number, tense, and case are incorporated into the word itself, rather than expressed via function words. not all languages mark the same properties: german nouns, for instance, have more inflected forms than their english counterparts. when acquiring a language, humans usually learn to inflect words without explicit instruction. thus, most native speakers are capable of generating inflected forms even of artificial lemmas @xcite . however, models that can generate paradigms without explicit morphological train- * equal contribution. figure 1 : the task of unsupervised morphological paradigm completion @xcite ) consists of generating complete inflectional paradigms for given lemmas, with the only additional available information being a corpus without annotations. in this paper, we present the sigmorphon 2020 shared task on unsupervised morphological paradigm completion (sigmorphon 2020 task 2). we asked participants to produce systems that can learn to inflect in an unsupervised fashion: given a small corpus (the bible) together with a list of lemmas for each language, systems for the shared task should output all corresponding inflected forms. in their output, systems had to mark which forms expressed the same morphosyntactic features, e.g., demonstrate knowledge of the fact that walks is to walk as listens is to listen, despite not recognizing the morphological features explic-itly. we show a visualization of our shared task setup in figure 1 . unsupervised morphological paradigm completion requires solving multiple subproblems either explicitly or implicitly. first, a system needs to figure out which words in the corpus belong to the same paradigm. this can, for instance, be done via string similarity: walks is similar to walk, but less so to listen. second, it needs to figure out the shape of the paradigm. this requires detecting which forms of different lemmas express the same morphosyntactic features, even if they are not constructed from their respective lemmas in the exact same way. third, a system needs to generate all forms not attested in the provided corpus. using the collected inflected forms as training data, this can be reduced to the supervised morphological inflection task @xcite . this year's submitted systems can be split into two categories: those that built on the baseline (retrieval+x) and those that did not (segment+conquer). the baseline system is set up as a pipeline which performs the following steps: edit tree retrieval, additional lemma retrieval, paradigm size discovery, and inflection generation @xcite . as it is highly modular, we provided two versions that employ different inflection models. 1 all systems built on the baseline substituted the morphological inflection component. no system outperformed the baseline overall. however, two retrieval+x models slightly improved over the baseline on three individual languages. we conclude that the task of unsupervised morphological paradigm completion is still an open challenge, and we hope that this shared task will inspire future research in this area. ## systems in this section, we first review the baseline before describing the submitted systems. an additional overview of the submissions is shown in ## conclusion we presented the findings of the sigmorphon 2020 shared task on unsupervised morphological paradigm completion (sigmorphon 2020 task 2), in which participants were asked to generate paradigms without explicit supervision. surprisingly, no team was able to outperform the provided baseline, a pipeline system, on average over all test languages. even though 2 submitted systems were better on 3 individual languages, this highlights that the task is still an open challenge for the nlp community. we argue that it is an important one: systems obtaining high performance will be able to aid the development of human language technologies for low-resource languages. all teams that participated in the shared task devised modular approaches. thus, it will be easy to include improved components in the future as, for instance, systems for morphological inflection improve. we released all data, the baseline, the evaluation script, and the system outputs in the official repository, foot_2 in the hope that this shared task will lay the foundation for future research on unsupervised morphological paradigm completion.