Exploring Low-resource Neural Machine Translation for S inhala- T amil Language Pair (2023)

Abstract: At present, Neural Machine Translation is a promising approach for machine translation. Transformer-based deep learning architectures in particular show a substantial performance increase in translating between various language pairs. However, many low-resource language pairs still struggle to lend themselves to Neural Machine Translation due to their data-hungry nature. In this article, we investigate methods of expanding the parallel corpus to enhance translation quality within a model training pipeline, starting from the initial collection of parallel data to the training process of baseline models. Grounded on state-of-the-art Neural Machine Translation approaches such as hyper-parameter tuning, and data augmentation with forward and backward translation, we define a set of best practices for improving Tamil-to-Sinhala machine translation and empirically validate our methods using standard evaluation metrics. Our results demonstrate that the Neural Machine Translation models trained on larger amounts of back-translated data outperform other synthetic data generation approaches in Transformer base training settings. We further demonstrate that, even for language pairs with limited resources, Transformer models are able to tune to outperform existing state-of-the-art Statistical Machine Translation models by as much as 3.28 BLEU points in the Tamil to Sinhala translation scenarios.

## introduction since 1949, when machine translation was initially proposed @xcite , statistical machine translation (smt) models dominated the machine translation field for decades. however, the advent of neural machine translation (nmt) using deep learning @xcite @xcite has revolutionized the field, enabling superior performance in translation tasks. recently, nmt tends to employ trans-former @xcite architecture which is a novel architecture grounded only on attention mechanisms. while it has shown remarkable results for high-resource languages, such as english, it struggles with low-resource languages like sinhala and tamil, which are morphologically rich and low-resourced languages. despite the existence of the best open-source sinhala-tamil translator using smt, nmt has not been widely experimented on in an open-domain setting. hence, improving nmt for low-resourced languages remains an open research problem with proven success. in this paper we aims to investigate the performance of transformer models on tamil and sinhala machine translation, with the goal of establishing best practices for low-resource nmt. we explore various model architectures and hyperparameter tuning methods to develop an accurate model for these languages. to address the issue of insufficient parallel data, we expand the corpus size and evaluate the impact of data size on nmt for lowresource languages. we also examine the effects of back translation and forward translation mechanisms for machine translation. finally, we compare the performance of our transformer models with smt. our research represents a novel contribution, as there is currently an absence of exploration into the best practices for transformer-based models in sinhala and tamil neural machine translation (nmt) within low-resource contexts. the rest of the paper is structured as follows: the state-of-the-art studies are critically analyzed in section 2, section 3 describes the methodology, and section 4 presents the detailed experimental settings, including the utilised data sets, tools, and training protocol of mt. in section 5 we present the experimental results. finally, section 6 presents the future works and concludes the paper. ## methodology in this article, we concentrate on two primary research directions to address the low-resource problem: (1) exploring hyper parameters with available parallel data (25k), and (2) devise methods to exploit additional opportunistic data sources. ## results this section presents the results obtained from the experiments conducted on the tamil and sinhala language pairs. ## conclusion we performed an empirical comparison of smt and nmt in low-resource settings: tamil-to-sinhala. benchmarking common models and establishing best practises are our objectives. this study has demonstrated that, for low-resource data sizes, a proper combination of transformer configurations together with regularization techniques @xcite and also with proper vocabulary selection, results in significant improvements when compared with the transformer system with default settings. moreover, this research proved the fact suggested by @xcite , in low-resource scenarios, both statistical machine translation (smt) and neural machine translation (nmt) can work similarly, but in order to get better performance, the neural systems require more careful tuning. developing machine translation models for lowresource languages with limited online presence can be challenging, but our preliminary results sug-gest that even a small amount of parallel data (a few hundred thousand example translations) can make a significant difference when using current neural architectures. therefore, we believe it is essential to continue pushing the boundaries of finding and curating exploitable parallel text for low-resource languages. in future, nmt system can be improved for low-resource scenarios by experimenting with transfer-learning approaches.