Morphological Processing of Low-Resource Languages: Where We Are and What’s Next (2022)

Abstract: Automatic morphological processing can aid downstream natural language processing applications, especially for low-resource languages, and assist language documentation efforts for endangered languages. Having long been multilingual, the field of computational morphology is increasingly moving towards approaches suitable for languages with minimal or no annotated resources. First, we survey recent developments in computational morphology with a focus on low-resource languages. Second, we argue that the field is ready to tackle the logical next challenge: understanding a language’s morphology from raw text alone. We perform an empirical study on a truly unsupervised version of the paradigm completion task and show that, while existing state-of-the-art models bridged by two newly proposed models we devise perform reasonably, there is still much room for improvement. The stakes are high: solving this task will increase the language coverage of morphological resources by a number of magnitudes.

## introduction automatic morphological processing tools have the potential to drastically speed up language documentation @xcite and thereby help combat the language endangerment crisis @xcite . explicit morphological information also benefits myriad nlp tasks, such as parsing @xcite , language modeling @xcite @xcite , and machine translation @xcite . for low-resource languages, valuable morphological resources are typically small or non-existent. of late, the field of computational morphology has increased its efforts to extend the coverage of multilingual morphological resources @xcite @xcite . simultaneously, there has been a revival of minimally supervised and unsupervised models for morphological tasks, such as segmentation @xcite , inflection @xcite , and lemmatization @xcite . given the speed of recent developments, it is important to reflect on where we are as a field and what future challenges lie ahead. to this end, we survey recent computational morphology: we review existing multilingual resources ( §2) and tasks and systems ( §3), with a focus on low-resource languages. given recent developments in unsupervised segmentation, low-resource morphological inflection, and unsupervised morphological paradigm completion @xcite )-which we argue is not fully unsupervised-we believe the community is poised for the next logical step: inferring a language's morphology purely from raw text. in §4, we formalize a new task: truly unsupervised morphological paradigm completion (tumpc). we then introduce a pipeline with two novel components ( §4.3): one model for aligning paradigm slots across lexemes and another for predicting the slots of observed forms. with these, we assess several state-of-the-art models and the influence of different types of unlabeled corpora within the framework of tumpc. while existing methods leave room for improvement, they perform reasonably enough to support our argument that inferring a language's morphology from raw text is within reach and worthy of community efforts. to summarize, we present the following contributions: (i) a survey of tasks and systems in computational morphology with a focus on lowresource languages; (ii) models for the tasks of paradigm slot alignment and slot prediction, (iii) a formalization of the task of truly unsupervised morphological paradigm completion and (iv) an evaluation of state-of-the-art approaches and differ-ent corpora within the framework of this task. our code and data are publicly available. 1 ## morphological resources manually created resources are necessary for developing and evaluating nlp systems. they also serve as a basis for research questions in a multilingual context @xcite . 2 below, we review the two largest active multilingual resources for morphology and a number of language-specific resources. background and notation the canonical form of a word is called its lemma, and the set of all surface forms of a lemma is referred to as that lemma's paradigm. as is common, we formally write the paradigm of a lemma ℓ as: with f : σ * × @xmath0. σ is an alphabet of discrete symbols: the characters used in the language of lemma ℓ. γ(ℓ) is the set of slots in ℓ's paradigm. ## conclusion thanks to strong systems for inflection, segmentation, and paradigm completion, computational morphology is ripe to contribute to the large number of the world's languages with very few digital resources. we explore this through the novel task tumpc-which presents several challenges. we believe that truly unsupervised morphology is an important direction, and it can have a large impact on language technology for thousands of languages. with the goal of preserving endangered languages, we note that more than half the world's languages have no writing system @xcite . a frontier for this task would process speech as a strategy for language documentation in unwritten languages. judit ács. 2018. bme-has system for conll-sigmorphon 2018 shared task: universal morphological reinflection. in proceedings of the conll-sigmorphon 2018 shared task: universal morphological reinflection, pages 121-126, brussels. association for computational linguistics. željko agić and ivan vulić. 2019. jw300: a widecoverage parallel corpus for low-resource languages. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 3204-3210, florence, italy. association for computational linguistics. roee aharoni and yoav goldberg. 2017. morphological inflection generation with hard monotonic attention. in proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pages 2004-2015, vancouver, canada. association for computational linguistics. antonios anastasopoulos and graham neubig. 2019. pushing the limits of low-resource morphological inflection. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 984-996, hong kong, china. association for computational linguistics. antti arppe, jeff good, atticus harrigan, mans hulden, jordan lachler, sarah moeller, alexis palmer, miikka silfverberg, and lane schwartz, editors. 2021. proceedings of the 4th workshop on the use of computational methods in the study of endangered languages volume 1 (papers). association for computational linguistics, online. giusepppe attardi. 2015. wikiextractor. https:// github.com/attardi/wikiextractor. p. austin and j. sallabank, editors. 2011. the cambridge handbook of endangered languages. cambridge handbooks in language and linguistics. cambridge university press. r. harald baayen, richard piepenbrock, and leon gulikers. 1996. the celex lexical database (cd-rom). dzmitry bahdanau, kyunghyun cho, and yoshua bengio. 2015. neural machine translation by jointly learning to align and translate. corr, abs/1409.0473. toms bergmanis and sharon goldwater. 2017. from segmentation to analyses: a probabilistic model for unsupervised morphology induction. in proceedings of the 15th conference of the european chapter of the association for computational linguistics: volume 1, long papers, pages 337-346, valencia, spain. association for computational linguistics. toms bergmanis and sharon goldwater. 2018. context sensitive neural lemmatization with lematus. in proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), pages 1391-1400, new orleans, louisiana. association for computational linguistics. toms bergmanis and sharon goldwater. 2019. data augmentation for context-sensitive neural lemmatization using inflection tables and raw text. in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4119-4128, minneapolis, minnesota. association for computational linguistics. toms bergmanis, katharina kann, hinrich schütze, and sharon goldwater. 2017. training data augmentation for low-resource morphological inflection. in proceedings of the conll sigmorphon 2017 shared task: universal morphological reinflection, pages 31-39, vancouver. association for computational linguistics. terra blevins and luke zettlemoyer. 2019. better character language modeling through morphology. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 1606-1613, florence, italy. association for computational linguistics. piotr bojanowski, edouard grave, armand joulin, and tomas mikolov. 2017. enriching word vectors with subword information. transactions of the association for computational linguistics, 5:135-146. jan buys and jan a. botha. 2016. cross-lingual morphological tagging for low-resource languages. in proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1954-1964, berlin, germany. association for computational linguistics. nicoletta calzolari, frédéric béchet, philippe blache, khalid choukri, christopher cieri, thierry declerck, sara goggi, hitoshi isahara, bente maegaard, joseph mariani, et al., editors. 2020. proceedings of the 12th language resources and evaluation conference. european language resources association, marseille, france. erwin chan. 2006. learning probabilistic paradigms for morphology in a latent class model. in proceedings of the eighth meeting of the acl special interest group on computational phonology and morphology at hlt-naacl 2006, pages 69-78, new matt hohensee and emily m. bender. 2012. getting more from morphology in multilingual dependency parsing. in proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 315-326, montréal, canada. association for computational linguistics. hoifung poon, colin cherry, and kristina toutanova. 2009. unsupervised morphological segmentation with log-linear models. in proceedings of human language technologies: the 2009 annual conference of the north american chapter of the association for computational linguistics, pages 209-217, boulder, colorado. association for computational linguistics. peng qi, yuhao zhang, yuhui zhang, jason bolton, and christopher d. manning. 2020. stanza: a python natural language processing toolkit for many human languages. in proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pages 101-108, online. association for computational linguistics. mohammad sadegh rasooli and michael collins. 2017. cross-lingual syntactic transfer with limited resources. transactions of the association for computational linguistics, 5:279-293. teemu ruokolainen, oskar kohonen, sami virpioja, and mikko kurimo. 2013. supervised morphological segmentation in a low-resource learning setting using conditional random fields. in proceedings of the seventeenth conference on computational natural language learning, pages 29-37, sofia, bulgaria. association for computational linguistics. teemu ruokolainen, oskar kohonen, sami virpioja, and mikko kurimo. 2014. painless semi-supervised morphological segmentation using conditional random fields. in proceedings of the 14th conference of the european chapter of the association for computational linguistics, volume 2: short papers, pages 84-89, gothenburg, sweden. association for computational linguistics. tatyana ruzsics and tanja samardžić. 2017. neural sequence-to-sequence learning of internal word structure. in proceedings of the 21st conference on computational natural language learning (conll 2017), pages 184-194, vancouver, canada. association for computational linguistics. patrick schone and daniel jurafsky. 2001. knowledgefree induction of inflectional morphologies. in second meeting of the north american chapter of the association for computational linguistics. wolfgang seeker and özlem c ¸etinoglu. 2015. a graphbased lattice dependency parser for joint morpholog-