The Mininglamp Machine Translation System for WMT 21 (2021)

Abstract: This paper describes Mininglamp neural machine translation systems of the WMT2021 news translation tasks. We have participated in eight directions translation tasks for news text including Chinese to/from English, Hausa to/from English, German to/from English and French to/from German. Our fundamental system was based on Transformer architecture, with wider or smaller construction for different news translation tasks. We mainly utilized the method of back-translation, knowledge distillation and fine-tuning to boost single model, while the ensemble was used to combine single models. Our final submission has ranked first for the English to/from Hausa task.

## introduction this paper describes the mininglamp submissions to the wmt2021 news translation tasks for eight directions including four highresource chinese↔english, german↔english, two medium-resource french↔german and two low-resource hausa↔english. furthermore, all of our systems were built with constrained data sets. for this participation, we experimented with some smaller or wider transformer @xcite architectures to reach a reliable baseline based on different resource scales, sampling or beam search in back-translation to generate more suitable pseudo bilingual sentences. particularly in the low-resource tasks, hausa↔english, the transformer-small neural machine translation was built for the baseline, we presented iterative between back-translation and fine-tuning pattern which significantly improve the bleu score on the validation set, and it worked well on @xmath0. due to time constraints, we did not experiment on @xmath1. this path could be an experiment in the future work. as for the data augmentation aspect, we experimented with several back-translation methods @xcite , including the beam search, un-restricted sampling and sampling-topk @xcite , to leverage the target-side monolingual data. we also applied knowledge distillation @xcite to leverage the source-side monolingual data. our systems followed four main steps:1) data filtering and preprocessing, 2) back-translation to generate pseudo bilingual data, 3) knowledge distillation by monolingual data, 4) fine-tuning with in-domain. it should be emphasized that we used marian foot_0 @xcite to implement only for hausa↔english baseline systems, and fairseq foot_1 @xcite for the rest, include hausa↔english back-translation and knowledge distillation models. ## conclusions this paper described the mininglamp submissions to the wmt2021 eight news translation tasks, and our main exploration was using more diversified architectures, back-translation, fine-tuning and ensemble. we used a similar data preprocess and filtering strategy for all the tasks, containing statistical information-based rules. and we experimented with back-translation by different decoding strategies, using the transformer-small model and iterative between back-translation and fine-tuning for low-resource.