Neural Machine Translation for A ssamese- B odo, a Low Resourced I ndian Language Pair (2023)

Abstract: Impressive results have been reported in various works related to low resource languages, using Neural Machine Translation (NMT), where size of parallel dataset is relatively low. This work presents the experiment of Machine Translation in the low resource Indian language pair AssameseBodo, with a relatively low amount of parallel data. Tokenization of raw data is done with IndicNLP tool. NMT model is trained with preprocessed dataset, and model performances have been observed with varying hyper parameters. Experiments have been completed with Vocab Size 8000 and 16000. Significant increase in BLEU score has been observed in doubling the Vocab size. Also data size increase has contributed to enhanced overall performances. BLEU scores have been recorded with training on a data set of 70000 parallel sentences, and the results are compared with another round of training with a data set enhanced with 11500 Wordnet parallel data. A gold standard test data set of 500 sentence size has been used for recording BLEU. First round reported an overall BLEU of 4.0, with vocab size of 8000. With same vocab size, and Wordnet enhanced dataset, BLEU score of 4.33 was recorded. Significant increase of BLEU score (6.94) has been observed with vocab size of 16000. Next round of experiment was done with additional 7000 new data, and filtering the entire dataset. New BLEU recorded was 9.68, with 16000 vocab size. Cross validation has also been designed and performed with an experiment with 8-fold data chunks prepared on 80K total dataset. Impressive BLEU scores of (Fold-1 through fold-8) 18.12, 16.28, 18.90, 19.25, 19.60, 18.43, 16.28, and 7.70 have been recorded. The 8th fold BLEU deviated from the trend, might be because of nonhomogeneous last fold data.

## introduction assamese and bodo both are official languages spoken in the north eastern part of india. approximate 16 million speakers use assamese language, while around 3 million native speakers in the state of assam, particularly in the bodoland area uses bodo language. different natural language processing (nlp) works are reported for both the languages, including corpus creation, tagging, wordnet development etc. but no attempt has been seen for machine translation in this language pair. a neural machine translation platform has been customized for assamese-bodo machine translation experiments. preprocessing pipeline includes subword tokenization using byte pair encoding (bpe). indicnlp tool has been reported to work better for indian language preprocessing, and hence for tokenization, we have adopted indicnlp. a bilingual parallel corpus of 70000 parallel sentences (assamese-bodo) has been sourced from nltm 1 . 11000 parallel sentences have been extracted from the assamese and bodo wordnet data. as nmt demands significant amount of parallel corpus at sentences level, and as there is a scarcity of such resources, in the current work, we have tried to integrate resources obtaining from different sources, including available open sources data, and generating in-house using linguistic experts. details of series of performances under different training parameters are reported here. ## related works nmt has nowadays become and widely used and acceptable machine translation platform, as it outperformed the earlier rule based and smt approaches. open nmt contributes to the efficiency, modularity and extensibility (guillaume @xcite . faster training and efficient test strategies, modular structure, and scopes for incorporating new and innovative research directions are important features of opennmt. this also gives an exhaustive library for training and deployment of neural machine translation. other nmt platform includes joeynmt which is a pytorch based toolkit particularly designed for novices (julia @xcite . in a simple code base, joey nmt provides many important nmt features, and it supports rnn and transformer models. it is reported that nmt shows very impressive translation results in short sentences, but the performance decreases as the number of tokens in a sentence increases. @xcite . we also observed this performance bottleneck in our experiments, but have not included, within our scope in this paper, analysing the results based on length of sentences. nmt systems are also reported to offer better performance in many language pairs of broad domains, however for narrow domain results still smt are reported to perform better (lucia @xcite . despite various limitations, and varying performances over range of parameters and characteristics of resources, nmt is the contemporary dominant approach for machine translation tasks for both research and applications. for low resource language pairs, nmt becomes challenging. there are many nmt approaches proposed and experimented for low resource language pairs. a novel approach with multihead self attention has been experimented for english-tamil and english malayam @xcite . here they have collected corpus from multiple sources, addressed the issues related with the resources, and then integrated and refined. comparatively better bleu scores were reported as 24.34 and 9.78 for the pairs. another work reported nmt experimentations for english-tamil, english-hindi, and english-punjabi machine translations, where they have evaluated the nmt performances using bleu as well as human evaluators for assessing quality of translation in terms of fluency adequacy ( ## nmt model experiments we trained the model initially with a dataset of 30000 parallel sentences. this was not to observe and record results and system performances, rather we used this phase for having a working test of the customized installed pipeline and model. then a series of experiments has been designed and performed. we have adopted evolutionary model, so that as we travel through dynamics of the data and parameters, system continuously evolves and performs better and better. in fact, from our experimental results, it has been evident, the details of which is elaborated in the results and analysis section. we used opennmt for our nmt experiments. our model building is on the pytorch version of opennmt and based on encoder decoder architecture. both the encoder and decoder parts consists of 3 layers of transformer blocks. the first set of experiment is with 70000 parallel sentences dataset. the model is trained with 8000 vocab size, and an overall bleu of 4.0 was recorded. domain specific bleu scores have been recorded, and presented in the next section. with same vocab size, we now enhanced the dataset with the wordnet data of 11500. experiment has been performed in the same pipeline and with the same model, and a slight increase of over bule score of 4.33 has been recorded. then the next level of experiment was performed with doubling the vocab size to 16000. experiment in the same pipeline and with the same model has been performed on the wordnet enhanced dataset, to record significant increase in bleu score to 6.94. then we moved to the next round of experiment. this time the dataset is enhanced with gold standard additional dataset of 7000, created inhouse with expert linguists. data filtering using python based code set was operated on the integrated dataset, and model was training with 16000 vocab size. new bleu recorded was 9.68. as part of the experiments, we also designed cross validation of the system. cross validation has been designed and performed with an experiment with 8-fold data chunks prepared on 80k total dataset. impressive average bleu scores of around 17 has been observed, which signifies model performance as an acceptable machine translation model for practice. ## results and analysis testing was done with the gold standard 500 sentences test dataset. testing has been performed domain wise, as well as for overall dataset. bleu scores have been recorded under varying vocab size as well as evolving dataset.