HW - TSC ’s Participation in the WMT 2020 News Translation Shared Task (2020)

Abstract: This paper presents our work in the WMT 2020 News Translation Shared Task. We participate in 3 language pairs including Zh/En, Km/En, and Ps/En and in both directions under the constrained condition. We use the standard Transformer-Big model as the baseline and obtain the best performance via two variants with larger parameter sizes. We perform detailed pre-processing and filtering on the provided large-scale bilingual and monolingual dataset. Several commonly used strategies are used to train our models such as Back Translation, Ensemble Knowledge Distillation, etc. We also conduct experiment with similar language augmentation, which lead to positive results, although not used in our submission. Our submission obtains remarkable results in the final evaluation.

## introduction this paper introduces our work for the wmt 2020 news translation shared task. we participate in three language pairs including chinese/english (zh/en), khmer/english (km/en), pashto/english (ps/en) and in both directions. after observation, we consider that the officially provided dataset has the acceptable size and quality therefore only participate in the constrained evaluation. our method is mainly based on previous works but with finegrained data cleaning techniques and language pair specific optimizations. for each language pair, we perform careful multistep cleaning on the provided dataset and only keep a high-quality subset for training. at the same time, several strategies are tested in a pipeline including back-translation @xcite , ensemble knowledge distillation @xcite , forward translation @xcite , fine-tuning @xcite , and ensemble and re-ranking @xcite . due to the page limitation, we mainly introduce our methods and experiments on the zh-en and en-zh language pairs. most of these methods are also employed on the km/en and ps/en pairs. special optimizations regarding different language will be introduced separately. ## data in this section, we describe the size and source of the dataset as well as our cleaning and filtering techniques. ## system overview this section describes the model and techniques of our work. we basically perform such strategies sequentially. our experimental result will be presented on each part. ## results this section presents the experimental results for each direction of all three language pairs in ## analysis here are several findings worthy of sharing during our experiments: ## conclusion this paper presents the submissions by hw-tsc on the wmt 2020 news translation task. for each direction in three language pairs, we perform experiments with a series of pre-processing and training strategies. the effectiveness of each strategy is demonstrated. our experiments on similar language augmentation shows that corpora with similar languages can be used for performance improvements in low resource scenarios. our submission finally achieves competitive result in the evaluation.