Neural Machine Translation for E nglish - M anipuri and E nglish - A ssamese (2023)

Abstract: The internet is a vast repository of valuable information available in English, but for many people who are more comfortable with their regional languages, accessing this knowledge can be a challenge. Manually translating this kind of text, is a laborious, expensive, and time-consuming operation. This makes machine translation an effective method for translating texts without the need for human intervention. One of the newest and most efficient translation methods among the current machine translation systems is neural machine translation (NMT). In this WMT23 shared task: low resource indic language translation challenge, our team named ATULYA-NITS used the NMT transformer model for the English to/from Assamese and English to/from Manipuri language translation. Our systems achieved the BLEU score of 15.02 for English to Manipuri, 18.7 for Manipuri to English, 5.47 for English to Assamese, and 8.5 for Assamese to English.

## introduction in countries like india, linguistic diversity is a significant aspect, with a multitude of languages varying across different regions. india officially recognizes 23 languages @xcite ) (e.g., hindi, sanskrit, assamese, odia, etc.), and alongside these, there are several hundred unofficial local languages spoken by communities. despite india's vast population of approximately 1.4 billion, only about 11% of the population is proficient in english @xcite . this language barrier becomes crucial when considering the abundance of valuable resources available on the internet, mostly in english, as a significant proportion of people in india cannot fully comprehend this content. consequently, there arises a pressing need to translate such valuable information into local languages to facilitate knowledge sharing among the population. such knowledge dissemination is crucial not just for business purposes but also for enabling the exchange of feelings, opinions, and actions, thereby fostering better communication and understanding among people from diverse linguistic backgrounds. manual translation of such copious amounts of content would be extremely laborious and timeconsuming, making automatic machine translation an indispensable solution. however, machine translation for indian languages presents its own set of challenges @xcite . one key challenge is the scarcity of parallel corpora, as there are fewer resources available for indian languages compared to more widely spoken foreign languages. moreover, the structural differences between indian languages and english, particularly in terms of morphological richness and word order, pose significant obstacles to accurate translation. for instance, english follows a subject-verb-object (svo) word order, whereas indian languages like assamese and manipuri, follow a subject-object-verb (sov) word order @xcite . furthermore, english is a fusional language, while assamese and manipuri are agglutinative languages @xcite ; , leading to distinct syntactic and morphological complexities that further complicate the translation process. we participated in the low-resource indic language translation task on translating two language pairs i.e. english to/from assamese, and english to/from mizo. we did the preprocessing of the given dataset and applied a neural machine translation technique i.e. transformer model. the performance was evaluated using the widely used evaluation metric bleu. the rest of the paper is organized as follows: section 2 discusses the existing machine translation systems and techniques tailored to indian languages. in section 3, we present details about the dataset, preprocessing of the dataset, and transformer model. in section 4, we discussed about the result. finally, in section 5, we conclude with a discussion of the future prospects. ## literature survey over the past few decades, machine translation (mt) has been the subject of extensive research. researchers have explored various approaches in this field, including rule-based mt @xcite , corpus-based mt, also known as data-driven @xcite @xcite , and hybrid-based mt @xcite . each of these approaches has its own advantages and disadvantages. in rule-based mt, systems analyze the source text to create an intermediate representation, and depending on this representation, it can be further categorized into transfer-based (tba) and interlingua-based (iba) approaches. the corpusbased approach, on the other hand, relies on large parallel corpora consisting of text and their translations to acquire translation knowledge and is sub-divided into two sub-types, i.e. statistical machine translation (smt) and example-based machine translation (ebmt). smt generates translations using statistical models that combine language models and translation models with decoding algorithms. in contrast, ebmt uses existing translation examples to generate new translations. hybrid-based machine translation combines aspects of both rule-based and corpus-based approaches to address their respective limitations. the machine translation performance for indian language pairs (e.g., hindi, bengali, tamil, punjabi, gujarati, and urdu) into english achieves an average accuracy of only 10%, @xcite highlighting the need for improved machine translation systems for these languages. neural machine translation (nmt) has emerged as a novel and promising technique for various languages, exhibiting remarkable results @xcite @xcite . in this paper, we have applied the transformer model to the english-assamese and english-manipuri language pair @xcite 3 methodology and evaluation ## conclusion in this paper, we applied nmt to the two most difficult language pairs (english-assamese and english-manipuri). we showed that the transformer model performs better for indian languages. we achieved a fairly good bleu score for the english-manipuri language pair. so, this model can be used for domains such as tourism and education. moreover, this transformer model is useful for various english-indian language pair translations.