Investigating the Effect of Backtranslation for I ndic Languages (2025)

Abstract: Neural machine translation (NMT) is becoming increasingly popular as an effective method of automated language translation. However, due to a scarcity of training datasets, its effectiveness is limited when used with low-resource languages, such as Indian Languages (ILs). The lack of parallel datasets in Natural Language Processing (NLP) makes it difficult to investigate many ILs for Machine Translation (MT). A data augmentation approach such as Backtranslation (BT) can be used to enhance the size of the training dataset. This paper presents the development of a NMT model for ILs within the context of a MT system. To address the issue of data scarcity, the paper examines the effectiveness of a BT approach for ILs that uses both monolingual and parallel datasets. Experimental results reveal that while the BT has improved the model’s performance, however, it is not as significant as expected. It has also been observed that, even though the English-ILs and ILs-English models are trained on the same dataset, the ILs-English models perform better in all evaluation metrics. The reason for this is that ILs frequently differ in sentence structure, word order, and morphological richness from English. The paper also includes error analysis for translations between languages that were utilized in experiments utilizing the Multidimensional Quality Metrics (MQM) framework.

## introduction an automated system that converts a source language into a target language is known as machine translation @xcite @xcite . it has made significant strides recently in translating high-resource languages like spanish, french, and english @xcite . but as ils present a unique combination of challenges and opportunities, it is still difficult to get a good translator. this linguistic diversity, a testament to india's cultural heritage, poses distinct translation challenges when translating from english to ils and vice versa. despite their tremendous linguistic richness, ils are characterized as low-resource due to a lack of training data available for language models @xcite . compared to widely spoken languages such as english, 'low-resource languages' like ils possess a restricted range of linguistic resources such as parallel corpora, dictionaries, grammar, and trained models @xcite . in order to address the scarcity of resources, translation faces unique challenges, necessitating the utilization of efficient mt as a valuable tool @xcite . in fact, developing reliable and accurate mt systems for ils is very challenging. in this regard, backtranslation (bt) comes as an effective method for dealing with limited data and synthetically increasing the amount of data used for training for mt models @xcite . in different scenarios, nmt systems have shown to benefit from using bt, especially in most low-resource environments where it can be challenging to acquire high-quality corpora @xcite . its potential to improve translation model performance in this linguistic domain is the driving force behind the investigation of its efficacy in the context of ils. this motivates us to investigate backtranslation methods for ils. in this paper, first, a baseline nmt model for english-ils and ils-english using @xcite transformer architecture is developed. baseline models (nmt models which are generated) are trained using the samanatar dataset @xcite for experiments. the impact of the back translation for nmt models on ils is examined. all the generated translation outputs are examined using evaluation metrics, and the generated model output's error analysis is also done. the rest of this work is structured as follows: section 2 contains a thorough overview of literature of nmt. in section 3, a short description of the languages utilized is described. section 4 discusses the model utilized for our experiment. section 5 contains all results obtained after our experiments. in section 6, we summaries our study and suggest some future research directives. ## experimental setup this section describes the dataset, preprocessing method, steps before training, and evaluation metrics. ## results and discussion similarly, hindi despite having a better result with nmt, failed to produce substantial improvements through backtranslation. ta, kn, te, and ml are agglutinative, which means that words are often created through the combination of smaller units (morphemes) having particular meanings. hence, these languages benefit from word formation while using bt because their learned patterns can be utilized continuously during backtranslation. however, in en-ml, kn-en a small decrease in evaluation metrics is noticed. the findings show a slight decrease in evaluations in some ils when bt is used. particularly, variations to this decrement exist, especially in translations from english to ils. the limited effect may be caused by a number of factors, including the inherent characteristics of the language pair being translated, potential domain inconsistencies, and the quality and diversity of the language. ## conclusion in this paper, a baseline nmt model on the samanantar dataset utilizing transformer architecture is developed. in terms of bleu, ribes, meteor, chrf, and comet, hindi excels when compared to other languages using nmt. from the result, it has been observed that the ils-english nmt model outperforms and achieves higher bleu scores than the english-ils nmt model. for en-il translation using the nmt model, pa, gu, bn, and hi perform better than other languages while for il-en translation pa, te, bn, and hi perform better than other languages. the paper also discusses and investigates the effectiveness of backtranslation (bt) for ils and checks its performance in mt model. the results show that although bt enhanced the model's performance, however, this improvement was not as large as anticipated, and the model did not significantly outperform the baseline nmt models. one reason for the lack of noticeable improvements could be that the baseline nmt models' performance is subpar. an analysis of the experiment shows that while nmt models perform substantially better in some cases, they generally produce disappointing results over a wide range of languages. even in these circumstances, their performance is below expectations. since bt uses nmt models to produce data, its shortcomings affect its capacity to produce high-quality data. another factor could be bt performs best when for experiments high-quality data in both languages are available. however, even after filtration, the data obtained from experiments with ils is not particularly clean or reliable. this means that the models that are used to create bt data aren't very good. hence, there is not much effect of bt being noticed using ils. in future work, our findings can be expanded by examining monolingual datasets of varying sizes and domains to precisely determine the different levels of saturation for backtranslation.