e T ranslation’s Submissions to the WMT 2021 News Translation Task (2021)

Abstract: The paper describes the 3 NMT models submitted by the eTranslation team to the WMT 2021 news translation shared task. We developed systems in language pairs that are actively used in the European Commission’s eTranslation service. In the WMT news task, recent years have seen a steady increase in the need for computational resources to train deep and complex architectures to produce competitive systems. We took a different approach and explored alternative strategies focusing on data selection and filtering to improve the performance of baseline systems. In the domain constrained task for the French–German language pair our approach resulted in the best system by a significant margin in BLEU. For the other two systems (English–German and English-Czech) we tried to build competitive models using standard best practices.

## introduction the etranslation team is behind the translation services of the european commission's etranslation project foot_1 . this is a building block of the connecting europe facility (cef), with the aim of supporting european and national public administrations' information exchange across language barriers in the eu. the project is described in more details in @xcite . the team's participation in the wmt shared tasks has provided valuable insights to improve the quality of our production systems and allowed us to explore languages and domains beyond the formal language of eu institutions, leading to a continuous extension of the etranslation service and helping in the search for the right balance between the use of resources in production environments and the best possible performance of models. this year the team participated in the news translation shared task with 3 different language pairs: @xmath0, @xmath1. the selection was motivated by the fact that these language pairs can all be considered as high or medium resource, which is the main scenario in the etranslation service, while the constrained domain in @xmath2, which is a more viable alternative in our environment than the resource demanding (brute-force) increase in model complexity. ## data preparation here we briefly describe the base data sets, the general selection and filtering methods we applied to prepare these initial data sets used to train the first models. further data selection and augmentation methods to improve the quality of baseline models are described in section 3.2. for all models we only used the provided parallel and monolingual data, so our 3 submissions fall into the constrained category. ## trainings in competitive systems big transformer architectures have become the norm in recent years @xcite . we can in general see a significant increase in the need for computational resources to train deeper and more complex architectures up to 40-50 encoder layers @xcite @xcite . our resource environment does not allow us to fully follow this trend, limiting the complexity of the models as well as the scope of the experiments. similarly to previous years, in all experiments we used marian, as the core tool of our standard nmt framework in the etranslation service. all trainings were run as multi-gpu trainings on 2 or 4 nvidia v100 gpus with 16gb ram, while for one training we were able to use a server with 8 32gb v100 gpus. 7 base transformers were typically trained for 20-30 epochs, whereas big transfomers were generally trained for 4-9 epochs for very high resource setups (>400m segments) and 20-25 epochs for medium resource. ## results we submitted a constrained system for each of the 3 language pairs. for @xmath3, we ran out of time and had to reuse our last year submission. for the other language pairs, we provide the evaluation scores for models at important stages in the development, which reflect how the models got better as we tried various methods for improvement. all results are reported in detokenized bleu. ## conclusion we presented the submissions of the etranslation team to the wmt 2021 news translation shared task on 3 language pairs: english-german, french-german and english-czech. unlike in previous years, we had to face a few unexpected challenges with respect to resource availability, which inevitably affected some experiments we planned to carry out. we tried to put more emphasis on data selection, filtering and domain specific evaluation with custom test sets in the task where it seemed to be most rewarding and automatic evaluation results justified this approach.