e T ranslation’s Submissions to the WMT 2020 News Translation Task (2020)

Abstract: The paper describes the submissions of the eTranslation team to the WMT 2020 news translation shared task. Leveraging the experience from the team’s participation last year we developed systems for 5 language pairs with various strategies. Compared to last year, for some language pairs we dedicated a lot more resources to training, and tried to follow standard best practices to build competitive systems which can achieve good results in the rankings. By using deep and complex architectures we sacrificed direct re-usability of our systems in production environments but evaluation showed that this approach could result in better models that significantly outperform baseline architectures. We submitted two systems to the zero shot robustness task. These submissions are described briefly in this paper as well.

## introduction the european commission's etranslation project foot_0 , a building block of the connecting europe facility (cef), has been set up to help european and national public administrations exchange information across language barriers in the eu. more details about the project can be found in @xcite . our participation in last year's wmt shared task marked an important step towards opening the service to the coverage of additional, non-eu languages and to domains beyond the formal language of eu institutions. due to the encouragement and insights we received from wmt 2019, a complete set of general domain mt engines has meanwhile been implemented and incorporated into the etranslation service. this year the team participated in the news translation shared task with five different language pairs: @xmath0, @xmath1, @xmath2, @xmath3. the varying performance of these systems reflects the amount of resources dedicated to their developments. ## data preparation this section briefly describes the data sets, the selection, and filtering methods applied to the provided parallel and monolingual data in order to increase the quality of trained models. we primarily focused on constrained submissions, but due to the low quality of our first @xmath4.3). ## trainings our access to computing resources is not unlimited. therefore, we did not have much room for large scale experiments with either a wide range of scenarios or extensive tuning of hyperparameters. nevertheless, as opposed to last year, where we decided to stick only to simple setups and training procedures, this year we tried more complex models and utilized significantly more data where it was possible. in all experiments we used marian, which is the core tool of our standard nmt framework in the etranslation service. all trainings were run as multi-gpu trainings on 2 or 4 nvidia v100 gpus with 16gb ram. base transformers were typically trained for 7 epochs for high resource and 11 epochs for lower resource language pairs, whereas big transfomers were generally trained for 12 epochs for high and 30 epochs for lower resource. ## results we submitted one system for each of the five language pairs. in this section we provide evaluation scores for models at important stages in the experiments, which reflect how the models got better as we tried various methods for improvement. all results are reported in detokenized bleu. 11 in ## conclusion we described the submissions of the etranslation team to the wmt 2020 news translation shared task on 5 language pairs: english-german, japanese-english, english-polish, russian-english, and english-czech. like last year, we tried to build the best possible systems in a relatively lowresource production environment. but in contrast to last year, we dedicated more resources to certain language pairs, and tried more complex models and utilized significantly more data where possible. in particular, we experimented with various techniques (big transformer models, synthetic data obtained from tagged back-translation, two stage continued training process, ensembling up to 4 models) and obtained significant improvements over baseline models: from 4 to 7 bleu points depending on the language pair on the 2020 test sets. we ranked competitively in all language pairs, reducing the gap from the best systems significantly from last year. 13 however, the submitted setups cannot be reused in our production environment due to their excessive demands on resources, but lessons learnt from those experiments shall provide valuable insights to improve the etranslation system