NITS - CNLP Low-Resource Neural Machine Translation Systems of E nglish- M anipuri Language Pair (2023)

Abstract: This paper describes the transformer-based Neural Machine translation (NMT) system for the Low-Resource Indic Language Translation task for the English-Manipuri language pair submitted by the Centre for Natural Language Processing in National Institute of Technology Silchar, India (NITS-CNLP) in the WMT 2023 shared task. The model attained an overall BLEU score of 22.75 and 26.92 for the English to Manipuri and Manipuri to English translations respectively. Experimental results for English to Manipuri and Manipuri to English models for character level n-gram F-score (chrF) of 48.35 and 48.64, RIBES of 0.61 and 0.65, TER of 70.02 and 67.62, as well as COMET of 0.70 and 0.66 respectively are reported.

## introduction our team from centre for natural language processing at national institute of technology silchar, india (nits-cnlp) participated in low-resource indic language translation task for english-manipuri language pair in the wmt 2023 shared task @xcite . the shared task involves developing machine translation (mt) systems with relatively small parallel datasets. neural machine translation (nmt) has been a trending topic for the last few years for translating human languages. manipuri's mt task is still in its infancy because of the limited resources. singh and bandyopadhyay (2011a) conducted a study on supervised statistical methods in which the authors present a convincing study of the impact of morphosyntactic information and dependencies in the context of statistical machine translation. in another work, @xcite showed that the expression grounded statistical machine translation (smt) system improves by incorporating verbal features including named entities and reduplicated multiword expressions. de-spite the advancement in mt tasks, its investigation in low-resource languages is limited. mt researchers have introduced several approaches to overcome this bottleneck such as data augmentation using back-translation @xcite , multilingual approach (singh and singh, 2022a), semi-supervised approach @xcite and exploiting cues from multiple modalities @xcite . there are also reports of a comparative study of mt systems on the low resource machine translation focusing on indian languages such as assamese @xcite and @xcite . driven by the benefits of nmt over traditional mt systems and the encouraging outcomes achieved by nmt in recent times, a study to assess its efficacy in the domain of indian languages is conducted. specifically, we have developed and assessed nmt models for translating english to manipuri and manipuri to english. the predicted translations are evaluated using automatic evaluation metric and qualitative analysis. ## results and discussion in this section, we discuss the experimental results and the performance of models. the reported bleu score is calculated upon the de-tokenized text using sacrebleu @xcite while remaining score such as chrf, ribes, ter and comet are calculated using tool provided by the organizer foot_2 . the scores of the system are given in ## conclusion enabling mt for low-resource languages poses several challenges due to the lack of parallel resources available for training. in this work, we report the performance of the mt systems trained on low resource setting for english to manipuri and manipuri to english using a transformer-based encoder and decoder architecture. the automatic evaluation shows that english to manipuri mt system achieved 22.75 bleu and manipuri to english mt system achieved 26.92 bleu. the automated scoring mechanism is inadequate in capturing the linguistic nuances of the morphologically complex manipuri language which requires the use of multiple references. based on the subjective evaluation, we observed that the translation qual-ity is deemed satisfactory and fluent in some cases, given the relatively small size of the dataset and the utilization of a single test reference.