Xiaomi’s Submissions for IWSLT 2020 Open Domain Translation Task (2020)

Abstract: This paper describes the Xiaomi’s submissions to the IWSLT20 shared open domain translation task for Chinese<->Japanese language pair. We explore different model ensembling strategies based on recent Transformer variants. We also further strengthen our systems via some effective techniques, such as data filtering, data selection, tagged back translation, domain adaptation, knowledge distillation, and re-ranking. Our resulting Chinese->Japanese primary system ranked second in terms of character-level BLEU score among all submissions. Our resulting Japanese->Chinese primary system also achieved a competitive performance.

## introduction in this paper, we describe the xiaomi's neural machine translation (nmt) systems evaluated at iwslt 2020 @xcite shared open domain translation task in two directions, @xmath0. the accuracy of nmt systems relies on the quality of training data, we first consider careful preprocessing and discard the corrupted data from the existing bilingual sentences according to rule-based filtering and model-based scoring. in the aspect of nmt architecture, we exploit some recent transformer variants, including different transformer models with deeper layers or wider inner dimension of feed-forward layers than the standard transformer-big model, transformer with a dynamic linear combination of layers (dlcl) @xcite and neural architecture search (nas) based transformer-evolved @xcite , to increase the diversity of the system. we further strengthen our systems by diversifying the training data via some effective methods, including back-translation (bt) @xcite , knowledge distillation (kd) @xcite and right-to-left (r2l) nmt model. finally, we also explore re-rank the n-best translation candidates generated by models ensembling with some effective features, including target-to-source (t2s) nmt model, left-to-right (l2r) nmt model, r2l nmt model @xcite , bilingual sentence bert and language model (lm). through experiments, we evaluate how each system feature affects the accuracy of nmt. our resulting @xmath1. our resulting @xmath2. ## experiments and results in this section, we introduce the experimental and data setup used in our experiments and then evaluate each of the systems introduced in section 3. ## conclusions we present the xiaomi's nmt systems for iwslt 2020 chinese↔japanese open domain translation tasks. for both translation tasks, our final systems achieved substantial improvements up by about 9 bleu score over baseline systems by integrating careful data filtering, data augmentation, and other effective nmt techniques. as a result, our submitted @xmath3.