NICT - AI 4 B ’s Submission to the I ndic MT Shared Task in WMT 2023 (2023)

Abstract: In this paper, we (Team NICT-AI4B) describe our MT systems that we submit to the Indic MT task in WMT 2023. Our primary system consists of 3 stages: Joint denoising and MT training using officially approved monolingual and parallel corpora, backtranslation and, MT training on original and backtranslated parallel corpora. We observe that backtranslation leads to substantial improvements in translation quality up to 4 BLEU points. We also develop 2 contrastive systems on unconstrained settings, where the first system involves fine-tuning of IndicTrans2 DA models on official parallel corpora and seed data used in AI4Bharat et al, (2023), and the second system involves a system combination of the primary and the aforementioned system. Overall, we manage to obtain high-quality translation systems for the 4 low-resource North-East Indian languages of focus.

## introduction the increasing online presence foot_0 of the indian population along with the economic growth foot_1 of india has necessitated the development of translation systems for indian languages. there have been substantial efforts towards collecting monolingual and parallel corpora, as well as developing machine translation systems using them @xcite . most recently, indictrans2 @xcite , an mt system, and its accompanying parallel corpus bpcc, were released. this corpus covers all 22 indian languages covered in the 8th schedule foot_2 of the constitution of india. while indictrans2 has achieved comparable or better results compared to existing systems like nllb @xcite , a major limitation is that there is no specific focus on language subgroups. one of such subgroups is the north-east indian languages, which this shared task focuses on. the task focuses on translation to/from english and the following 4 north-east indian languages: assamese, manipuri, mizo and khasi. we submit constrained as well as unconstrained mt systems for the 8 translation directions in this task. for further details on the shared task, kindly refer to @xcite . we leveraged ideas such as joint multilingual denoising and mt training followed by backtranslation at scale. first, due to the small size of the official parallel corpora, we utilized available and permitted monolingual corpora for all languages involved and trained on a combination of text-infilling and mt objectives to train an initial mt system. we used this system to generate large back-translated corpora, which were combined with the official parallel corpora to train the final primary system. the back-translated corpora, due to their scale, led to improvements up to 4 bleu as measured on the development set. we also submitted two contrastive systems: the first one was obtained via fine-tuning indictrans2 da models @xcite and the second one was a system combination of the primary and the aforementioned contrastive system. we observed that our first contrastive system outperformed the primary for some language pairs due to the utilization of a strong pretrained mt model as initialization and additional high-quality data being used to finetune them. as for our second contrastive system, we observed improvements for directions where there was a small performance gap between the primary and the first contrastive system. ## related work our submissions leverage ideas from topics such as multilingualism @xcite , denoising pre-training @xcite , backtranslation @xcite , transfer learning @xcite and system combination @xcite . the north-east indian languages of focus in this shared task are all low-resource languages, and transfer learning via multilingualism is a reliable solution in this case. transfer learning can be achieved by fine-tuning a pre-trained model @xcite but this involves two stages. on the other hand, multilingual training @xcite involves implicit transfer via joint training. we explore both strategies when developing our systems. backtranslation @xcite involves taking intermediate translation systems and translating monolingual corpora into another language. the synthetic-source and original target parallel corpora, can typically be orders of magnitude larger than the parallel corpora used to train the intermediate systems and when used at scale, such backtranslated corpora are known to help improve translation quality @xcite and therefore we attempt to use as much monolingual corpora as possible for backtranslation. while there are iterative backtranslation @xcite strategies where the process of model training and backtranslation is performed repeatedly, their computational complexity makes them a less attractive solution to us. an alternative to backtranslation is denoising pretraining using monolingual corpora @xcite and when combined with mt training as a joint objective @xcite is known to significantly improve mt quality. since backtranslation and denoising pre-training are known to be orthogonal @xcite , we leverage the joint denoising and mt training approach only for intermediate models which are used for backtranslation. ## our systems we submit 3 systems, one primary (constrained) and two contrastive (unconstrained). ## experiments in this section, we describe the datasets, implementation and evaluation settings. ## results in this section, we describe the results we obtained on the test sets. ## conclusion in this paper, we have described our systems submitted to the wmt 2023 indic translation task. we leveraged ideas ranging from joint denoising and mt training, backtranslation, fine-tuning pretrained models, and system combination. we reported our results, which show the benefits of the various ideas we explored. finally, we recommend best practices.