W e C hat Neural Machine Translation Systems for WMT 20 (2020)

Abstract: We participate in the WMT 2020 shared newstranslation task on Chinese→English. Our system is based on the Transformer (Vaswaniet al., 2017a) with effective variants and the DTMT (Meng and Zhang, 2019) architecture. In our experiments, we employ data selection, several synthetic data generation approaches (i.e., back-translation, knowledge distillation, and iterative in-domain knowledge transfer), advanced finetuning approaches and self-bleu based model ensemble. Our constrained Chinese→English system achieves 36.9 case-sensitive BLEU score, which is thehighest among all submissions.

## introduction our wechat ai team participates in the wmt 2020 shared news translation task on @xmath0. in this year's translation task, we mainly focus on exploiting several effective model architectures, better data augmentation, training and model ensemble strategies. for model architectures, we mainly exploit two different architectures in our approaches, namely transformers and rnmt. for transformers, we implement the deeper transformer with pre-norm, the wider transformer with a larger filter-size and the average attention based transformer @xcite . for the rnmt, we use the deep transition based dtmt @xcite model. we finally ensemble four kinds of models in our system. for synthetic data generation, we explore various methods for out-of-domain and in-domain data generation. for out-of-domain data generation, we explore the back-translation method @xcite to leverage the target side monolingual data and the knowledge distillation method @xcite to leverage the source side of golden parallel data. for in-domain data generation, we employ iterative in-domain knowledge transfer to leverage the source-side monolingual data and golden parallel data. furthermore, data augmentation methods, including noisy fake data @xcite and sampling @xcite , are used for training more robust nmt models. for training strategies, we mainly focus on the parallel scheduled sampling @xcite , the target denoising and minimum risk training @xcite algorithm for indomain finetuning. we also exploit a self-bleu @xcite ) based model ensemble approach to enhance our system. as a result, our constrained @xmath1. in the remainder of this paper, we start with an overview of model architectures in section 2. section 3 describes the details of our systems and training strategies. then section 4 shows the experimental settings and results. finally, we conclude our work in section 5. ## model architectures in this section, we first describe the model architectures we use in the chinese-→english shared task, including the transformer-based @xcite models and rnn-based @xcite models. ## system overview in this section, we describe our system used in the wmt 2020 news shared task. figure 1 depicts the overview of our wechat nmt. our system can be divided into four parts, namely data filtering, synthetic data generation, indomain finetuning, and ensemble. the synthetic generation part further includes the generation of out-of-domain and in-domain data. next, we will illustrate these four parts. ## conclusion in this paper, we introduce the system wechat submitted for the wmt 2020 shared task on @xmath2. our system is based on the transformer @xcite with different variants and the dtmt @xcite architecture. data selection, several effective synthetic data generation approaches (i.e., back-translation, knowledge distillation, and iterative in-domain knowledge transfer), advanced finetuning approaches (i.e., parallel scheduled sampling, target denoising, and minimum risk training) and self-bleu based model ensemble are employed and proven effective in our experiments. our constrained @xmath3.9 case-sensitive bleu score which is the highest among all submissions.