Tencent Neural Machine Translation Systems for the WMT 20 News Translation Task (2020)

Abstract: This paper describes Tencent Neural Machine Translation systems for the WMT 2020 news translation tasks. We participate in the shared news translation task on English \leftrightarrow Chinese and English \rightarrow German language pairs. Our systems are built on deep Transformer and several data augmentation methods. We propose a boosted in-domain finetuning method to improve single models. Ensemble is used to combine single models and we propose an iterative transductive ensemble method which can further improve the translation performance based on the ensemble results. We achieve a BLEU score of 36.8 and the highest chrF score of 0.648 on Chinese \rightarrow English task.

## introduction recently, transformer @xcite , that depends on self-attention mechanism , has significantly improved the translation quality. it is widely used as basic neural machine translation (nmt) models in previous wmt translation tasks @xcite @xcite . in this year's translation task, our tencent translation team participated in three wmt2020 shared news translation tasks, including @xmath0, @xmath1. for the three tasks, we use similar model architectures and training strategies. four structures are used and all of them are based on deep transformer which are proven more effective than the standard transformer-big models @xcite . in terms of data augmentation, we adopt r2l training @xcite to all the tasks. monolingual data is only used in @xmath2. different from the standard backtranslation, we add noise to the synthetic source sentence in order to take advantage of large-scale monolingual text. in addition, we add a special token to the synthetic source sentence to help the model better distinguish the bilingual data and synthetic data. the in-domain finetuning @xcite is very effective in our three experiments and specially, we propose a boosted finetuning method for english ↔ chinese tasks. we also take advantage of the combination methods to further improve the translation quality. the "greedy search ensemble algorithm" @xcite is used to select the best combinations from single models. then for english ↔ chinese tasks we propose an iterative transductive ensemble (ite) method based on the translation results of the ensemble models. for @xmath3, we apply the noise channel model for re-ranking @xcite . this paper was structured as follows: section 2 describes the dataset. we present the detailed overview of our system in section 3. the experiment settings and main results are shown in section 4. finally, we conclude our work in section 5. ## conclusion this paper presents the tencent translation systems for wmt2020 @xmath4. we investigate various deep architectures to build strong baseline systems. then popular data augmentation methods such as backtranslation and r2l training are used to improve the baselines. we also prove that in-domain finetuning is very effective for news translation tasks especially on @xmath5. finally, we adopt the greed-based ensemble algorithm and propose an iterative transductive ensemble method for further improvement. it is worth mentioning a number of advanced technologies reported in this paper are also adapted to our systems for biomedical translation @xcite and chat translation @xcite tasks, which respectively achieve up to 1st and 2nd ranks in terms of bleu scores.