N oah NMT at WMT 2021: Dual Transfer for Very Low Resource Supervised Machine Translation (2021)

Abstract: This paper describes the NoahNMT system submitted to the WMT 2021 shared task of Very Low Resource Supervised Machine Translation. The system is a standard Transformer model equipped with our recent technique of dual transfer. It also employs widely used techniques that are known to be helpful for neural machine translation, including iterative back-translation, selected finetuning, and ensemble. The final submission achieves the top BLEU for three translation directions.

## introduction in this paper, we describe the noahnmt system submitted to one of the wmt 2021 shared tasks. the shared task features both unsupervised machine translation and very low resource supervised machine translation. as our core technique is mainly suitable for low resource supervised machine translation, we participated in four translation directions between chuvash-russian (chv-ru) and upper sorbian-german (hsb-de). our core technique is called dual transfer @xcite , which belongs to the family of transfer learning. it transfers from both high resource neural machine translation model and pretrained language model to improve the quality of low resource machine translation. during the preparation for the shared task, we conducted additional experiments that supplement the original paper, including the choice of parent language, the validation of transformer big model, and the usage of dual transfer along with iterative back-translation. in addition, we also applied proven techniques to strengthen the quality of our system, including selected finetuning and ensemble. our final submission achieves the top bleu on the blind test sets for three translation directions: @xmath0, @xmath1, and @xmath2. ## approach in this section, we describe the techniques used in our system. interested readers are encouraged to check out the original papers for further details. ## conclusion in this paper, we describe a series of experiments that contribute to our submission to the wmt 2021 shared task of very low resource supervised machine translation. these experiments, as well as the good results of the final submission, show that dual transfer can work in synergy with several widely used techniques in realistic scenarios.