Tilde at WMT 2020: News Task Systems (2020)

Abstract: This paper describes Tilde’s submission to the WMT2020 shared task on news translation for both directions of the English-Polish language pair in both the constrained and the unconstrained tracks. We follow our submissions form the previous years and build our baseline systems to be morphologically motivated sub-word unit-based Transformer base models that we train using the Marian machine translation toolkit. Additionally, we experiment with different parallel and monolingual data selection schemes, as well as sampled back-translation. Our final models are ensembles of Transformer base and Transformer big models which feature right-to-left re-ranking.

## introduction this year, we developed both constrained and unconstrained nmt systems for the english↔polish language pair. we base our methods on the submissions of the previous years @xcite @xcite including methods for parallel data filtering from @xcite . specifically, we lean on @xcite and @xcite for data selection and filtering, @xcite for morphologically motivated sub-word units and synthetic data generation, @xcite for sampled back-translation and finally @xcite for re-ranking with right-to-left models. we use the marian toolkit @xcite to train models of transformer architecture @xcite . although document level nmt as showcased by (junczys-dowmunt, 2019) have yielded promising results for the english-german language pair, we were not able to collect sufficient document level data for the english-polish language pair. as a result, all our systems this year translate individual sentences. the paper is further structured as follows: section 2 describes the data used to train our nmt systems, section 3 describes our efforts to identify the best-performing recipes for training of our final systems, section 5 summarises the results of our final systems, and section 6 concludes the paper. ## data for training of the constrained nmt systems, we used data from the wmt 2020 shared task on news translation foot_0 . for unconstrained systems, we used data from the tilde data library foot_1 . the 10 largest publicly available datasets that were used to train the unconstrained systems were open subtitles from the opus corpus @xcite , paracrawl @xcite (although it was discarded due to noise found in the corpus), dgt translation memories @xcite , microsoft translation and user interface strings glossaries 3 from multiple releases up to 2018, the tilde model corpus @xcite , wikimatrix @xcite , digital corpus of the european parliament @xcite , jrc-acquis @xcite , europarl @xcite , and the qcri educational domain corpus @xcite . ## experiments in this section, we describe the details of the methods used and experiments performed to identify the best-performing recipes for training of tilde's nmt systems for the wmt 2020 shared task on news translation. all experiments that are described in this section were carried out on the constrained datasets unless specifically indicated that also unconstrained datasets were used. ## final systems we chose the best performing system variants from section 3 to serve as a base for the final submission for the news translation task. for the constrained scenario, we trained final systems using parallel data that were filtered with tilde's filtering methods and dccef, back-translated monolingual data using a ratio of 1:2 (different data selection methods were applied for both translation directions), and synthetic data featuring unknown phenomena. for the unconstrained scenario, we trained final systems using parallel data that were filtered only with tilde's filtering methods, back-translated monolingual data that were selected using the combined data selection strategy using a ratio of 1:1, and synthetic data featuring unknown phenomena. all models were trained using the adam optimiser. when preparing the final systems, we also employed r2l re-ranking (see section 3.6), ensembling of the best three models, and trained transformer models using the big model configuration. ## results the bleu scores for the systems that were evaluated for the final submission are shown in in overall, the results differ from what we have observed in previous years. back-translation for @xmath2, which raises a question of a possible domain mismatch between the monolingual data we back-translated and the development data. unconstrained systems are only slightly better than constrained systems for @xmath3, which shows that current nmt methods are not able to benefit from larger datasets. hence, having in-domain data is more important. ## conclusion in this paper, we described tilde's nmt systems for the wmt shared task on news translation. this year, we trained constrained and unconstrained systems for the english↔polish language pair. we de-tailed the methods applied and the training recipes. during our experiments, we identified several avenues of possible further research. we saw that larger datasets even after applying data selection methods did not allow improving translation quality (at least not significantly). we made a similar observation also previous years when participating in wmt. we saw in our results also that back-trannslation did not yield positive results for @xmath4. we hypothesise that there may be a domain mismatch between the data we used for training and the newsdev2020 dataset. however, this requires further investigation.