Unsupervised Morphological Paradigm Completion (2020)

Abstract: We propose the task of unsupervised morphological paradigm completion. Given only raw text and a lemma list, the task consists of generating the morphological paradigms, i.e., all inflected forms, of the lemmas. From a natural language processing (NLP) perspective, this is a challenging unsupervised task, and high-performing systems have the potential to improve tools for low-resource languages or to assist linguistic annotators. From a cognitive science perspective, this can shed light on how children acquire morphological knowledge. We further introduce a system for the task, which generates morphological paradigms via the following steps: (i) EDIT TREE retrieval, (ii) additional lemma retrieval, (iii) paradigm size discovery, and (iv) inflection generation. We perform an evaluation on 14 typologically diverse languages. Our system outperforms trivial baselines with ease and, for some languages, even obtains a higher accuracy than minimally supervised systems.

## introduction morphologically rich languages express syntactic and semantic properties-like tense or caseof words through inflection, i.e., changes to the surface forms of the words. the set of all inflected forms of a lemma-the canonical form-is called its paradigm. while english does not manifest a rich inflectional morphology, polish verbs have around a hundred different forms @xcite , and archi paradigms, an extreme example, can have over 1.5 million slots @xcite . morphologically rich languages constitute a challenge for natural language processing (nlp) systems: because each lemma can take on a variety of surface forms, the frequency of each individual inflected word decreases drastically. this yields problems for speech recognition (creutz 1 our implementation is available under https://github. com/cai-lw/morpho-baseline . sé vigilante y confirma las otras cosas que están para morir , porque no he hallado tus obras bien acabadas delante de dios . acuérdate , pues , de lo que has recibido y oído ; guárdalo y arrepiéntete , pues si no velas vendré sobre ti como ladrón y no sabrás a qué hora vendré sobre ti . el vencedor será vestido de vestiduras blancas , y no borraré su nombre del libro de la vida , y confesaré su nombre delante de mi padre y delante de sus ángeles . el que tiene oído , oiga lo que el espíritu dice a las iglesias . ' " » escribe al ángel de la iglesia en filadelfia : » " esto dice el santo , el verdadero , el que tiene la llave de david , el que abre y ninguno cierra , y cierra y ninguno abre : ... estar tener empezar pasar ... we describe it in detail in §4. @xcite , parsing @xcite , and keyword spotting @xcite , inter alia. for unsupervised machine translation, @xcite encounter difficulties when translating into the morphologically rich languages nepalese and sinhalese. children acquire morphological knowledge from raw utterances and, in particular, without access to explicit morphological information @xcite . do they have an innate capacity that enables them to learn a language's morphology? or can morphology be learned in an unsupervised fashion? this question-in addition to practical considerations like benefits for the aforementioned nlp taskshas motivated work on unsupervised morphological analyses @xcite . to the best of our knowledge, no previous work has considered unsupervised morphological generation. 2 however, over the last few years, there has been a lot of progress on morphological generation tasks with limited amounts of supervision, in particular on morphological inflection @xcite and paradigm completion @xcite , which can potentially be leveraged for unsupervised solutions. here, we fill the gap between unsupervised morphological analysis and morphological generation with limited training data by proposing the task of unsupervised morphological paradigm completion. that is, we aim to construct and fill inflection tables exclusively from raw text and a lemma list for a known part of speech (pos), in a situation similar to those encountered by field linguists. we further present a system for the task (see figure 1 ) which employs state-of-the-art methods common in nlp and computational morphology. it performs the following four steps: (i) edit tree @xcite retrieval ( §4.1), (ii) additional lemma retrieval ( §4.2), (iii) paradigm size discovery using distributional information ( §4.3), and (iv) inflection generation ( §4.4). to evaluate our approach, we design a metric for unsupervised paradigm completion, best-match accuracy ( §5.4), and experiment on 14 languages from 7 families. as we are tackling a novel task with no baselines in the nlp literature, we perform an extensive ablation study to demonstrate the importance of all steps in our pipeline. we further show that our system outperforms trivial baselines and, for some languages, even obtains higher accuracy than a minimally supervised system. ## formal task description given a corpus @xmath0, . . . , w |d| with a vocabulary v of word types {w i } and a lexicon @xmath1, the task of unsupervised morphological paradigm completion consists of generating the paradigms {π(ℓ)} ℓ∈l of the entries in the lexicon. following @xcite and @xcite , we treat a paradigm as a vector of inflected forms belonging to a lemma ℓ. paradigm completion consists of predicting missing slots in the paradigm π(ℓ): remarks on task design in general, not all paradigm entries will be present in the corpus d. thus, the task requires more than a keyword search. on another note, it is not necessary to predict the features t γ corresponding to each slot γ; as the exact denotation of features is up to human annotators, they cannot be inferred by unsupervised systems. for our task, it is enough to predict the ordered vector π(ℓ) of inflected forms. ## methodology our system implicitly solves two subtasks: (i) determining the number of paradigm slots; and (ii) generating the inflected form corresponding to each paradigm slot for each lemma. it is organized as a @xcite . ## conclusion we proposed unsupervised morphological paradigm completion, a novel morphological generation task. we further developed a system for the task, which performs the following steps: (i) edit tree retrieval, (ii) additional lemma retrieval, (iii) paradigm size discovery, and (iv) inflection generation. introducing best-match accuracy, a metric for the task, we evaluated our system on a typologically diverse set of 14 languages. our system obtained promising results for most of our languages and even outperformed a minimally supervised baseline on basque, english, and navajo. further analysis showed the importance of our individual components and detected possible sources of errors, like wrongly identified edit trees early in the pipeline or syncretism. in the future, we will explore the following directions: (i) a difficult challenge for our proposed system is to correctly determine the paradigm size. since transfer across related languages has shown to be beneficial for morphological tasks @xcite @xcite , inter alia), future work could use typologically aware priors to guide the number of paradigm slots based on the relationships between languages. (ii) we plan to explore other methods, like word embeddings, to incorporate context information into our feature function. (iii) we aim at developing better performing string transduction models for the morphological inflection step. by substituting the current transducers in our pipeline, we expect that we will be able to improve the overall performance of our system.