E n A s C orp1.0: E nglish- A ssamese Corpus (2020)

Abstract: The corpus preparation is one of the important challenging task for the domain of machine translation especially in low resource language scenarios. Country like India where multiple languages exists, machine translation attempts to minimize the communication gap among people with different linguistic backgrounds. Although Google Translation covers automatic translation of various languages all over the world but it lags in some languages including Assamese. In this paper, we have developed EnAsCorp1.0, corpus of English-Assamese low resource pair where parallel and monolingual data are collected from various online sources. We have also implemented baseline systems with statistical machine translation and neural machine translation approaches for the same corpus.

## introduction assamese (also called as axomiya) is the predominant language of the indian state of assam. it is one of the 23 official languages (22 indian languages and additional language english) recognized by the republic of india 1 . it is the most widely used language of the entire north-eastern region (also known as the seven sister states) of india. assamese is spoken by about 14 million speakers @xcite , they are known as assamese or axomiya/asomiya people. it belongs to the indo-aryan language family, and its script is evolved from the gupta script @xcite , also known as assamese-bengali script @xcite . in today's digital world, there is a demand for automatic translation of english to assamese and vice-versa. in a multilingual country like india, machine translation (mt) plays a vi-1 https://www.mhrd.gov.in/sites/upload_ files/mhrd/files/upload_document/languagebr. pdf tal role to minimize the language barrier via automatic translation among human spoken languages (also known as natural languages). for all official modes of communication, hindi and english languages are used by the government of india. therefore, to facilitate greater access among the local community, it is necessary to translate the official documents like orders, notices, messages released by the government of india into the regional language of assam. moreover, english is a high resource and widely accepted language all over the globe. to establish a better communication in the native languages at the national or international level, mt system for the english-assamese pair is very much essential. but due to the non availability of a suitable corpus, the mt system for this language pair still remains in beginning stage @xcite . in mt, based on the availability of corpus resource, there are two categories of natural languages: high and low resource languages. high resource languages are those languages which are resource-rich languages like english, german, french, and hindi. on the other hand, low resource languages are resource-poor like many indian languages especially found in the north-eastern region of india like assamese, boro, khasi, manipuri, kokborok, and mizo. data scarcity is one of the major problems in mt for low resource language scenarios. for english-assamese, there is a lack of standard parallel data and monolingual assamese data. because of this reason, english-assamese can be categorized under low resource pair. the word order and script of both languages are completely different. unlike english, assamese follows the subject-objectverb (sov) word order. figure 1 depicts an example sentence, where an assamese sentence is shown with its english translation and transliteration. moreover, assamese is known as mor-he lives near the church subject verb object subject object verb (teketh girjagoror uchoroth thake) ## baseline systems for the baseline systems, we have trained two models: phrase-based smt @xcite and recurrent neural network (rnn) based nmt with attention @xcite to estimate benchmark translation accuracy for english to assamese and assamese to english translation respectively. we have used the developed dataset enascorp1.0 and english monolingual data about 3 million sentences from wmt16 foot_4 . ## conclusion and future work in this paper, we have presented enascorp1.0 for various tasks of nlp, mainly mt. we have developed both parallel and monolingual corpus for low resource english-assamese pair. the dataset will be available here: https: //github.com/cnlp-nits/enascorp1.0 . we have evaluated baseline systems using smt and nmt. it is observed that the performance accuracy of nmt is better than smt. in the future, we will increase the size of the corpus and develop mt system by utilizing advanced deep learning techniques for the same pair. the proposed dataset will be available for the benefit of the research community.