End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages (2021)

Abstract: Lexically constrained machine translation allows the user to manipulate the output sentence by enforcing the presence or absence of certain words and phrases. Although current approaches can enforce terms to appear in the translation, they often struggle to make the constraint word form agree with the rest of the generated output. Our manual analysis shows that 46% of the errors in the output of a baseline constrained model for English to Czech translation are related to agreement. We investigate mechanisms to allow neural machine translation to infer the correct word inflection given lemmatized constraints. In particular, we focus on methods based on training the model with constraints provided as part of the input sequence. Our experiments on English-Czech language pair show that this approach improves translation of constrained terms in both automatic and manual evaluation by reducing errors in agreement. Our approach thus eliminates inflection errors, without introducing new errors or decreasing overall quality of the translation.

## introduction in neural machine translation (nmt), lexical constraining @xcite @xcite involves changing the translation process in a way that desired terms appear in the model's output. translation constraints are useful in domain adaptation, interactive machine translation or named entities translation. current approaches focus either on manipulating beam search decoding @xcite @xcite or training an nmt model using constraints alongside the input @xcite @xcite . in inflected languages, constraints from both source and target sides may appear in numerous surface forms, which may result in errors during translation. by enforcing the presence of a certain exact term on the target side, existing approaches fail to deal with word inflections. as we show, they preserve the surface form of the word provided as constraint regardless of the context. morphologically rich languages have multiple forms of each word, e.g. inflections to nouns. for satisfactory results in these languages, the constraint processing method needs to be capable of detecting any surface form on the source side and generating the correct surface form on the target side. to illustrate the problem, figure 1 shows a sentence translation from english to czech with outputs from three methods. the first one is a noconstraint translation where "hawkish" is translated as "jestřábím" (literally "hawkish", no figurative meaning; followed by a further mis-translation of "lot"). the second is a constrained model requested to use the word form "radikální" ("radical") in the output. the constraint was satisfied but the adjective should have taken the comparative degree to match the rest of the translation. the third output is the result of a model that processes the input along with the canonical form constraint ("radikální") and modifies the constraint inflection in the final translation ("radikálnější") to correctly express the comparative form (although the translation of "lot" is worse than in previous case). we evaluate different methods of lexically constrained machine translation on the czech language. we propose an approach to deal with word inflection in lexically constrained translation. by training a model that receives lemmatized target constraints as the input alongside the source sentence, we improve the generation of constraints in forms matching the output context. we run experiments on both synthetic and real-world test scenarios. ## related work in mt, there are scenarios where words that should or should not appear in the output are known upfront. common use cases include integration of domain-specific terminology and translation of named entities or rare words using a dictionary. such functionality was previously implemented in phrase-based systems @xcite , like moses @xcite . in nmt, this task is not yet definitely solved, since the translation process is hard to interpret and influence. ## proposed methods building upon the described techniques, we focus on allowing the model to choose the correct word form. our approaches are based on learned constraining, where the constraints are lemmatized during both training and test time. ## manual analysis in this section, we analyse examples marked as errors by automatic evaluation. in appendix a.1, we analyse the position of constraints in translation outputs, showing that they are placed correctly. in appendix a.2, we look closely at the constrained translation of an out-of-domain document. ## conclusion we described the problem of word inflection in lexically constrained machine translation. our solution capitalizes on the ability of nmt models to generate correct word forms in the output translation. we train a transformer model using lemmatized constraints supplied alongside the input sentences, and correct surface forms of the constraints in the reference. this training leads to a model producing the constraints in the output with high coverage, correct placement, and in a correct surface form. we compare several methods of obtaining constraints and integrating them into the input. in the realistic use case of terminology integration, we evaluated our methods and show that without lemmatizing the training constraints, the chosen approach of integrating constraints into nmt does not work well for czech. we effectively solve the issue of inflection errors by lemmatizing constraints, taking advantage of the transformer's language modelling capacity with no additional inference costs. this has been proven by both automatic and manual evaluation. we show our method is also effective in translating general domain rare words using a bilingual dictionary and we plan future work in solving the problem of choosing correct translation term from number of variants.