CUNI Systems for WMT 21: Terminology Translation Shared Task (2021)

Abstract: This paper describes Charles University sub-mission for Terminology translation Shared Task at WMT21. The objective of this task is to design a system which translates certain terms based on a provided terminology database, while preserving high overall translation quality. We competed in English-French language pair. Our approach is based on providing the desired translations alongside the input sentence and training the model to use these provided terms. We lemmatize the terms both during the training and inference, to allow the model to learn how to produce correct surface forms of the words, when they differ from the forms provided in the terminology database. Our submission ranked second in Exact Match metric which evaluates the ability of the model to produce desired terms in the translation.

## introduction terminology integration, or, more generally, constrained translation in nmt was extensively studied in recent years. lexically constrained translation means that aside from the source sentence, we have available some additional knowledge of what tokens or expressions should appear in the translation and we want to force the system to include them in the generated output. three main ways of enforcing these constraints have been studied. first, replacing the source part of the constraint that is found in the source sentence with a placeholder which is then copied by the model into the output. there it gets replaced by the target part of the constraint @xcite ; @xcite ). second way is to modify the decoding search algorithm in a way that only allows hypotheses containing the constraints to be marked as finished @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ) finally, some works focus on providing the constraints directly to the model as part of the input sequence. the model is trained to incorporate these constraints into the output, for example @xcite ; @xcite ; @xcite or @xcite . as apparent from previous paragraphs, the problem of integrating lexical constraints into nmt is well studied, but one issue was largely ignored. in inflected languages, the surface form of the constraint in the output cannot be known beforehand, as there are usually many possible ways to translate a sentence and many of them need different surface forms of the constraint to be fluent and grammatically correct. for example, let's say we have a terminology database containing term pair influenza -> grippe and this source sentence: during the 2018-2019 influenza season. possible correct translation is: pendant la saison grippale 2018-2019. where the term base noun form grippe is inflected into adjective grippale. traditional constraint integration methods will try to enforce the term db form grippe instead. we have studied this problem in our recent work @xcite concurrently with @xcite , who used a very similar approach. both works use different languages and evaluation pipelines and both show that the proposed approach is feasible. ## method nmt models are known to produce fluent, consistent and grammatically correct outputs @xcite . thus, it makes sense to utilize this ability of the model to inflect the constraint into correct form, instead of trying to disambiguate the form externally. our approach is based on annotating source sentences with the desired target constraints and training the model to incorporate these constraints into the output. we publish our preprocessing scripts at https://github.com/ufal/ bergamot/wmt21-terminology ## conclusion we describe our submission to terminology translation shared task at wmt21. we show our method can effectively incorporate the terminology without negative effects on overall translation quality. we analysed all ten examples in the validation set where our model did not cover the desired term constraint and we show that most of them can be explained by preprocessing issues. sentations in multi-modal and multi-lingual modelling, 19-26934x (riv: gx19-26934x)) and by svv 260 575 grant. the work described herein has also been using data provided by the lindat/clariah-cz research infrastructure, supported by the ministry of education, youth and sports of the czech republic (project no. lm2018101).