IIE ’s Neural Machine Translation Systems for WMT 20 (2020)

Abstract: In this paper we introduce the systems IIE submitted for the WMT20 shared task on German-French news translation. Our systems are based on the Transformer architecture with some effective improvements. Multiscale collaborative deep architecture, data selection, back translation, knowledge distillation, domain adaptation, model ensemble and re-ranking are employed and proven effective in our experiments. Our German-to-French system achieved 35.0 BLEU and ranked the second among all anonymous submissions, and our French-to-German system achieved 36.6 BLEU and ranked the fourth in all anonymous submissions.

## introduction we participate in the wmt20 shared news translation task in one language pair and two language directions, @xmath0. our methods are based on techniques and approaches used in submissions from past years @xcite @xcite @xcite , including the use of subword models @xcite , iterative back-translation, knowledge distillation, model ensembling and several techniques we proposed recently @xcite . for our submissions of two language directions, we adopt the deep transformer architectures (48layer) based on multiscale collaboration mechanism @xcite as our baseline, which outperformed the standard transformer-big as well as shallower models significantly in terms of translation quality. we also use an iterative back-translation approach @xcite ) with the controllable sampling to extend the back translation method by jointly training source-to-target and target-to-source nmt models. moreover, the knowledge distillation @xcite is employed to leverage the source-side monolingual data. for our final models, we apply a domainspecific fine-tuning process and model ensembling, and decode using noisy channel model re-ranking. the paper is structured as follows: section 2 describes the techniques we used, then section 3 shows the experimental settings and results. finally, we conclude our work in section 4. ## system overview we submit constrained systems to both german to french and french to german translations, with the same techniques. ## conclusion this paper describes cas iie's submission to the wmt20 german↔french news translation task. we investigate extremely deep models (with 48 layers) and exploit effective strategies to better utilize parallel data as well as monolingual data. finally, our @xmath1.0 bleu and ranked the second among all anonymous submissions, and our @xmath2.6 bleu and ranked the fourth in all anonymous submissions.