GUIT - NLP ’s Submission to Shared Task: Low Resource I ndic Language Translation (2023)

Abstract: This paper describes the submission of the GUIT-NLP team in the “Shared Task: Low Resource Indic Language Translation” focusing on three low-resource language pairs: English-Mizo, English-Khasi, and English-Assamese. The initial phase involves an in-depth exploration of Neural Machine Translation (NMT) techniques tailored to the available data. Within this investigation, various Subword Tokenization approaches, model configurations (exploring differnt hyper-parameters etc.) of the general NMT pipeline are tested to identify the most effective method. Subsequently, we address the challenge of low-resource languages by leveraging monolingual data through an innovative and systematic application of the Back Translation technique for English-Mizo. During model training, the monolingual data is progressively integrated into the original bilingual dataset, with each iteration yielding higher-quality back translations. This iterative approach significantly enhances the model’s performance, resulting in a notable increase of +3.65 in BLEU scores. Further improvements of +5.59 are achieved through fine-tuning using authentic parallel data.

## introduction work on machine translation (mt) involving indigenous languages is on the rise to provide such languages a global existence rather than limiting its scope to regional geographical boundaries. but such a work is quite challenging owing to its typical characteristic being limited (low) resourced as nmt models are data hungry which tend to degrade with limited data input. established methods like back translation @xcite , transfer learning @xcite , multilingual neural translation (mnt) @xcite , dual learning @xcite and such do exist to tackle the low-resource challenge. with the monolingual and limited parallel data provided to the teams to work with, back translation (bt) seemed to be an appropriate choice. in bt, a target to source model translates the target side monolingual data to generate a substantial amount of synthetic parallel data which could be augmented with the limited authentic parallel data to increase the volume of training data. previous experiments @xcite , @xcite have shown improved results in such scenarios. the general nmt pipeline comprises of various stages like tokenization, subword tokenization, nmt model training, inference and post-editing. it should be noted that several methods are available for every stage making it difficult for the researcher to select the one that would suit the data best as each method has its own influence on the model performance. we, therefore, perform an initial investigation on two popular subword tokenization methods to find the best choice. the rest of the paper is organized as follows: section 2 describes the methods applied for the task, section 3 presents the experimental setup and the results obtained for the three language pairs: english-mizo, english-khasi and english-assamese. section 4 concludes the paper. ## methodology the following section describes the methodology used for the task for each of the language pairs. ## conclusion in this study, we have provided a comprehensive overview of our neural machine translation (nmt) system developed for three language pairs: english-assamese, english-khasi, and english-mizo, encompassing both translation directions. our research delved into the intricacies of model configurations (transformer layers, heads, batch sizes, etc.) and subword tokenization schemes (byte pair encoding and sentencepiece and its variants). through rigorous experimentation, we identified and adopted the optimal configurations for each language pair. challenged by the inherent scarcity of data in these low-resourced language pairs, we innova-