T en T rans Multilingual Low-Resource Translation System for WMT 21 I ndo- E uropean Languages Task (2021)

Abstract: This paper describes TenTrans’ submission to WMT21 Multilingual Low-Resource Translation shared task for the Romance language pairs. This task focuses on improving translation quality from Catalan to Occitan, Romanian and Italian, with the assistance of related high-resource languages. We mainly utilize back-translation, pivot-based methods, multilingual models, pre-trained model fine-tuning, and in-domain knowledge transfer to improve the translation quality. On the test set, our best-submitted system achieves an average of 43.45 case-sensitive BLEU scores across all low-resource pairs. Our data, code, and pre-trained models used in this work are available in TenTrans evaluation examples.

## introduction we participate in the wmt21 multilingual low-resource translation shared task. this task focuses on the multilinguality in the cultural heritage domain for two indo-european language families: north-germanic and romance. we devote the research into translations among romance languages, including @xmath0, @xmath1, @xmath2. additionally, this task explicitly encourages the use of data of four related highresource languages (spanish, french, portuguese and english) in the same linguistic family. for the model architecture, we adopt a universal encoder-decoder architecture that shares parameters across all languages @xcite . and almost all of the subsequent experiments are based on transformer base model @xcite . to effectively exploit low and high resource data in the multilingual low-resource scenario, we explore several approaches, and each approach shows effectiveness. we employ back-translation @xcite and pivot-based methods to augment the training corpus. in terms of knowledge transfer, we explore the pre-trained model and the multilingual model that trained with both low and high resource language pairs. moreover, we extract in-domain corpus by a domain classifier and adapt the model to the target domain by in-domain fine-tuning. this paper is structured as follows: section 2 introduces used datasets, data statistic and preprocessing pipeline. section 3 describes the details of different approaches. in section 4 we present experimental settings and results. section 5 draws a brief conclusion of our work in the wmt21.