IOL Research Machine Translation Systems for WMT 23 Low-Resource I ndic Language Translation Shared Task (2023)

Abstract: This paper describes the IOL Research team’s submission systems for the WMT23 low-resource Indic language translation shared task. We participated in 4 language pairs, including en-as, en-mz, en-kha, en-mn. We use transformer based neural network architecture to train our machine translation models. Overall, the core of our system is to improve the quality of low resource translation by utilizing monolingual data through pre-training and data augmentation. We first trained two denoising language models similar to T5 and BART using monolingual data, and then used parallel data to fine-tune the pretrained language models to obtain two multilingual machine translation models. The multilingual machine translation models can be used to translate English monolingual data into other multilingual data, forming multilingual parallel data as augmented data. We trained multiple translation models from scratch using augmented data and real parallel data to build the final submission systems by model ensemble. Experimental results show that our method greatly improves the BLEU scores for translation of these four language pairs.

## introduction this paper describes our submissions for the wmt23 low-resource indic language translation shared task. we participated in 4 language pairs, including english↔assamese (en↔as), english↔mizo (en↔mz), english↔khasi (en↔kha), and english↔manipuri (en↔mn). our core approach is based on denoising language model pre-training @xcite @xcite @xcite and backtranslation @xcite based data augmentation. neural machine translation methods are almost the first choice for implementing translation systems at present, but they have certain requirements on the amount of parallel corpora. low-resource or even zero-resource neural machine translation has been a daunting challenge due to the lack of adequate parallel corpora. pretraining methods are popular solutions for lowresource cases. when the model parameter scale is large enough and there is enough training data, this method can even perform well in zero resource situations. for the machine translation task, as early as around 2019, xlm @xcite and mass @xcite were able to build unsupervised machine translation systems with near-supervised effects using only monolingual data. now, more advanced pre-training methods like bart @xcite and t5 @xcite are popular choices for training machine translation models in low-resource situations. therefore, in this paper, referring to the training methods of bart and t5, we trained a t5-style pre-training model and a bart-style pre-training model from scratch using monolingual data. backtranslation is a commonly used method in the field of machine translation. whether it is low-resource, medium-resource or high-resource, this approach can almost help the model to obtain further improvements on the original basis. therefore, we also use back-translation to help us further improve the translation quality. the layout of the subsequent paper is as follows: in section 2 we introduce the data source and processing strategy; in section 3 we describes the implementation process of our translation systems; in section 4 we describe the experimental settings and results; finally, the conclusion is drawn in section 5. ## system overview we chose transformer @xcite with pre-norm as our base translation model. in general, our procedure for improving the quality of lowresource translations is divided into two phases, an improvement phase based on pre-training methods and an improvement phase based on data augmentation. instead of using the pre-trained model to initialize the parameters of the translation model, the pre-training phase merely provides synthetic data for the data augmentation phase, which means that the translation model in the data augmentation phase is trained from scratch. in addition to this, we also used model ensemble in the final submissions. ## conclusion in this paper, we describe iol research's submission to the wmt2023 low-resource indic language translation shared task. we participated in four sub-tasks with a total of eight translation directions. our system mainly improves the translation quality of these languages in the low-resource case through pre-training and data augmentation. experimental results show that we achieved large improvements in all directions.