Rule-based Morphological Inflection Improves Neural Terminology Translation (2021)

Abstract: Current approaches to incorporating terminology constraints in machine translation (MT) typically assume that the constraint terms are provided in their correct morphological forms. This limits their application to real-world scenarios where constraint terms are provided as lemmas. In this paper, we introduce a modular framework for incorporating lemma constraints in neural MT (NMT) in which linguistic knowledge and diverse types of NMT models can be flexibly applied. It is based on a novel cross-lingual inflection module that inflects the target lemma constraints based on the source context. We explore linguistically motivated rule-based and data-driven neural-based inflection modules and design English-German health and English-Lithuanian news test suites to evaluate them in domain adaptation and low-resource MT settings. Results show that our rule-based inflection module helps NMT models incorporate lemma constraints more accurately than a neural module and outperforms the existing end-to-end approach with lower training costs.

## introduction incorporating terminology constraints in machine translation (mt) has proven useful to adapt translation lexical choice to new domains @xcite and to improve its consistency in a document @xcite . in neural mt (nmt), most prior work focuses on incorporating terms in the output exactly as given, using soft @xcite @xcite or hard constraints @xcite . these approaches are problematic when translating into morphologically rich languages where terminology should be adequately inflected in the output, while it is more natural and flexible to provide constraints as lemmas as in a dictionary. to the best of our knowledge, only one paper has directly addressed this problem for neural mt: @xcite design an nmt model trained to copy-and-inflect the terminology constraints using target lemma annotations (tla) -tla are synthetic training samples where the source sentence is tagged with automatically generated lemma constraints. while this approach improves translation quality, the end-to-end training set-up prevents fast adaptation to lemmas and inflected forms that are rare or unseen at training time. its impact is also limited to a specific neural architecture, and it is unclear whether its benefits port to more generic sequence-to-sequence models. in this paper, we introduce a modular framework for inflecting terminology constraints in nmt. it relies on a cross-lingual inflection module that predicts the inflected form of each lemma constraint based on the source context only. the inflected lemmas can then be incorporated into nmt using any of the aforementioned constrained nmt techniques. compared with tla, this framework is more flexible, as it can be applied to diverse types of nmt architectures and inflection modules, and facilitates fast adaptation to new terminologies without retraining the base nmt model from scratch. this flexibility is enabled by the cross-lingual nature of the inflection module, which predicts the inflected form of each target lemma based on the source context only. this differs from traditional inflection models that predict the inflected forms based on pre-specified morphological tags or monolingual target context. based on this framework, this paper makes the following contributions: ## inflecting target lemmas given the source context we introduce a modular framework for inflecting terminology constraints for nmt, where we first build an inflection module that predicts the inflected form of each target lemma term based on the source sentence and then incorporate the inflected constraints in nmt using any of the aforementioned techniques. by framing the problem this way, we assume that the inflected forms can be inferred based only on the source context and integrated in a fluent translation by nmt models. in cases where there are multiple possible inflected forms corresponding to different ways of translating the source, the inflection module can predict one of the possible forms, and the nmt model can generate a translation conditioned on the predicted forms of the constraints. compared with @xcite , our framework is more flexible -it can be combined with any nmt model that enables translation with constraints and can leverage diverse types of morphological inflection modules in which linguistic knowledge can be easily incorporated. formally, given a source sequence x and k target lemma words @xmath0, z2 , ..., zk ) that need to be inflected, the inflection module θ predicts the inflected form of each target lemma @xmath1, z 2 , ..., z k ) independently: ## evaluation test suites to evaluate the models' ability to incorporate diverse types of lemma constraints in different context, we choose the two morphologically complex languages -german and lithuanian -as the target languages, both of which are fusional languages with strong suffixing. we create two test suitesthe @xmath2. different from the automotive test suite of @xcite , which contains short sentences (15 tokens per source sentence on average) annotated with limited types of constraints (mostly nouns and proper nouns), our test suites contains longer sentences (20 and 25 tokens per source sentence on average) and diverse types of constraints including adjectives, nouns, proper nouns, and verbs. different from the upcoming wmt21 terminology task 9 where 7 so for instance, given the input "freeze [plh] [plh]", the model could predict the output "fro@@ zen [plh]". 8 to the best of our knowledge, there is no public health (or any non-news) domain mt test set for @xmath3. ## experimental settings training data for @xmath4, we use the training corpora from wmt14 @xcite and newstest2013 for validation. for @xmath5, we use the training data from wmt19 @xcite and newsdev2019 as the validation set. for preprocessing, we apply normalization, tokenization, true-casing, and bpe @xcite . 17 baselines we compare our model with the following baselines: ## conclusion we introduced a modular framework for leveraging terminology constraints provided in lemma forms in neural machine translation. the framework is based on a novel cross-lingual inflection module that inflects the target lemma constraints given source context and an nmt model that integrates the inflected constraints in the output. we showed that our framework can be flexibly applied to different types of inflection modules, including rule-based and neural-based ones, and different nmt models, including autoregressive and non-autoregressive ones, with minimal training costs. results on the english-german health and english-lithuanian test suites showed that the linguistically motivated rule-based inflection module helps nmt models incorporate terminology constraints more accurately than both neural-based inflection and the existing end-to-end approach to incorporating lemma constraints. this work opens future avenues for further improving the inflection module by combining linguistic knowledge with data-driven approaches. future work is needed to explore the strengths and weaknesses of this framework for languages with a broader range of morphological properties.