HW - TSC ’s Participation in the WMT 2021 News Translation Shared Task (2021)

Abstract: This paper presents the submission of Huawei Translate Services Center (HW-TSC) to the WMT 2021 News Translation Shared Task. We participate in 7 language pairs, including Zh/En, De/En, Ja/En, Ha/En, Is/En, Hi/Bn, and Xh/Zu in both directions under the constrained condition. We use Transformer architecture and obtain the best performance via multiple variants with larger parameter sizes. We perform detailed pre-processing and filtering on the provided large-scale bilingual and monolingual datasets. Several commonly used strategies are used to train our models, such as Back Translation, Forward Translation, Multilingual Translation, Ensemble Knowledge Distillation, etc. Our submission obtains competitive results in the final evaluation.

## introduction this paper introduces our submission to the wmt 2021 news translation shared task. we participate in seven language pairs including chinese/english (zh/en), german/english (de/en), japanese/english (ja/en), hausa/english (ha/en), icelandic/english (is/en), hindi/bengali (hi/bn), and xhosa/zulu (xh/zu) in both directions. we consider that the officially provided dataset has the acceptable size and quality and therefore only participate in the constrained evaluation. our method is mainly based on previous works but with finegrained data cleansing techniques and languagespecific optimizations. for each language pair, we perform multi-step data cleansing on the provided dataset and only keep a high-quality subset for training. at the same time, several strategies are tested in a pipeline, including backward @xcite and forward @xcite translation, multilingual translation @xcite , right-to-left models @xcite , iterative joint training @xcite , ensemble knowledge distillation @xcite , fine-tuning @xcite , ensemble @xcite , and postprocess. we combined all the techniques mentioned above and the overall training process is shown in figure 1 . section 2 focuses on our data processing strategies while section 3 describes our training techniques, including model architecture and iterative training, etc. section 4 explains our experiment settings and training processes and section 5 presents our experiment results. ## conclusion this paper presents the submissions of hw-tsc to the wmt 2021 news translation task. for each direction in all pairs, we perform experiments with a series of pre-processing and training strategies. the effectiveness of each strategy is demonstrated. our experiments show that in low-resource scenarios, multilingual model that utilizing data from other languages can improve system performance to a large extent. data augmentation strategy is still effective for multilingual models. our submissions finally achieves competitive results in the evaluation.