Morphologically Aware Word-Level Translation (2020)

Abstract: We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and inflectional morphology in a structured way. Our model exploits the basic linguistic intuition that the lexeme is the key lexical unit of meaning, while inflectional morphology provides additional syntactic information. This approach leads to substantial performance improvements—19% average improvement in accuracy across 6 language pairs over the state of the art in the supervised setting and 16% in the weakly supervised setting. As another contribution, we highlight issues associated with modern BLI that stem from ignoring inflectional morphology, and propose three suggestions for improving the task.

## introduction the ability to generalize to rare and unseen morphological variants of a known word lies at the heart of translation. for instance, a capable human spanish-english translator would find translating the exceptionally rare form tosed (2 nd person plural imperative form of 'to cough') as straightforward as translating the infinitive toser-despite the fact that tosed is so infrequent that many native spanish speakers may have never encountered the form themselves. given how basic this generalization ability is for humans, one should expect a good bilingual lexicon inducer to exhibit a similar capacity to generalize. in other words, the model should translate infrequent, regular forms of a lexeme as accurately as it translates a lexeme's most common forms. nevertheless, current approaches to bilingual lexicon induction (bli) fall short of this desideratum. in a recent study, @xcite reveal that the performance of state-of-the-art bilingual lexicon inducers degrades severely when translating less frequent inflected forms-even for the most common lexemes. the problem is severe: in the case of inducing a french-spanish bilingual lexicon, the model of @xcite correctly translates infinitives 50.6% of the time, but correctly translates the 2 nd person plural imperative forms only 1.5% of the time. motivated by this disparity, this work introduces a novel morphologically aware probability model for bli that jointly models lexeme translation and inflectional morphology in a structured way. our model exploits the basic linguistic intuition that the lexeme is the core lexical unit of meaning, while inflectional morphology provides additional syntactic information on top of it @xcite . it follows that we should ignore this syntactic information when translating at the word level and handle morphological inflection with a different component of the model. the empirical portion of our paper describes experiments on french, italian and spanish. we find our joint model substantially improves over several strong baselines on the bli task. when evaluating on held-out lexemes, foot_0 we observe an average performance improvement of 19% and 16% over the previous state of the art @xcite in the supervised and weakly supervised settings, respectively. in addition, we propose a simple heuristic to further boost performance: inspired by the dual-route hypothesis @xcite and the network model of morphological processing @xcite , we translate high-frequency forms, which are most likely to exhibit irregularity, directly (without going through the lexeme) and reserve our morphologically aware model for low-frequency forms. this heuristic gives us a further 2% improvement. ## a joint model for morphologically aware word-level translation the primary contribution of this work is a morphologically aware probabilistic model for word-level translation. our model exploits a simple intuition: because the core unit of meaning is the lexeme, one should translate through the lexeme and then inflect the word according to the target language's morphology. notation. in the task of bli, we consider a source language s and a target language t. the goal is to translate inflected forms in a source language ι @xmath0, where l s and l t are the source-side and the target-side lexicons, respectively. we denote source-side lemmata as λ @xmath1. we use τ @xmath2, where t s and t t are the sets of possible tags in the source and the target language. finally, let e : @xmath3. model. we construct a joint probability model for morphologically aware word-level translation. using the notation defined in the previous paragraph, we formally define the model as ## a frequency-based heuristic as defined in §3, the model handles regular and irregular word forms in the same manner. this can be problematic, as the analysis and synthesis modules have no means of handling irregular morphology, beyond the irregular forms they have been exposed to during training. guided by this insight, we propose an alternative version of our model, which employs a special treatment for forms likely to have irregular morphology-the most frequent forms @xcite @xcite . we term this extension the hybrid model. it employs a frequency-based heuristic and translates the source form through its lemma only if the lemma is more frequent. 6 otherwise, it translates the inflected form directly, using only the translator component. for example, the spanish form pide-an irregular inflected form of pedir ('to ask for')-would be translated directly, since pedir is less frequent. in a broader context of morphological processing, different handling of a form depending on its frequency or regularity can be linked to the dual-route hypothesis @xcite , which posits that regular and irregular inflection are handled by different cognitive mechanisms, or the works of @xcite , @xcite and @xcite , which have loosely inspired this heuristic. foot_6 ## experimental setup our evaluation involves 3 romance languages which exhibit a higher degree of inflection and are commonly experimented on within bli-french, spanish and italian. because these languages come from the same branch of the indo-european family, the results serve as an empirical upper-bound on bli. ## results we consider the fully supervised and the weakly supervised setting. weak supervision in the case of bli refers to populating the seed dictionary with identically spelled strings @xcite . in in the weakly supervised setting, the performance gains remain similarly high-for our base model they reach 2% and 16%, respectively, compared to the best baseline @xcite , while for the hybrid model we observe a 6% and 18% improvement, respectively. our experiments with an oracle analyzer demonstrate that even larger gains are possible: the oracle consistently outperforms every other model across all evaluation conditions. when evaluated on muse @xcite , the hybrid model is competitive with the baselines, while our base model performs worse. this is not surprising given that muse contains only the most frequent inflected forms which, as discussed in §4 and as we demonstrate later in this section, are the weakness of our base model. the hybrid model can also suffer from misanalyzing frequent irregular forms of even more frequent lemmata, such as puede or pudo-both forms of spanish poder ('to be able to'). ## a critique of modern bli the model we develop in §3 stems from a desire to better integrate inflectional morphology into current state-of-the-art models for bli. on one hand, the empirical findings we discuss in §6 indicate this attempt more fine-grained lexicons. our first proposal is to create more elaborate morphological dictionaries, in the style of those by @xcite . the primary draw-back of this suggestion is that such resources are tedious to create. @xcite focus on genetically related languages, for which inflectional morphological systems are easily compatible. however, this is often not the case. every morphological system carves up the semantic space in its own way, e.g. there is no good german equivalent of polish verbal aspect. thus, such elaborate resources should be carefully crafted and specify lemmata, inflected forms and tags for both the source and the target language. it follows, modulo polysemy and other lexical ambiguity, that the task would be well defined. ## conclusion we propose a novel model for bilingual lexicon induction which jointly models lexeme translation and inflectional morphology in a structured way. our model improves handling of less frequent, morphologically complex forms, especially if they belong to large inflectional paradigms. in our experiments on morphologically enriched dictionaries, we observe substantial performance improvements over the state of the art, particularly prominent for rare verbal paradigm slots.