Transformers: Leveraging O pen NMT and Transfer Learning for Low-Resource I ndian Language Translation (2025)

Abstract: This paper describes our submission to the WMT 2025 (Pakray et al, 2025) Shared Task on Low-Resource Machine Translation for Indic languages. This task is an extension of the efforts which was originally initiated in WMT 2023 (Pal et al., 2023), and further continued to WMT 2024 (Pakray et al, 2024), received significant participation from the global community. We address English ↔ {Assamese, Bodo, Manipuri} translation, leveraging Hindi and Bengali as high-resource bridge languages. Our approach employs Transformer-based Neural Machine Translation (NMT) models, initialized through multilingual pre-training on high-resource Indic languages, followed by fine-tuning on limited parallel data for the target low-resource languages. The pre-training stage provides a strong multilingual representation space, while fine-tuning enables adaptation to specific linguistic characteristics of the target languages. We also apply consistent preprocessing, including tokenization, true casing, and subword segmentation (Sennrich et al., 2016) with Byte-Pair Encoding (BPE), to handle the morphological complexity of Indic languages. Evaluation on the shared task test sets demonstrates that pre-training followed by fine-tuning yields notable improvements over models trained solely on the target language data.

## introduction india is home to an extraordinary linguistic diversity, with 22 scheduled languages, languages written using many scripts and hundreds of regional and tribal languages spoken across its vast geography. while this richness offers immense cultural value, it presents significant challenges for computational linguistics and natural language processing (nlp). many of these languages are classified as low resource, meaning that the quantity and quality of available digital text, speech, and annotated corpora are insufficient to support the development of robust nlp tools and machine translation systems. the scarcity of datasets for low-resource indian languages arises from multiple factors: historical underrepresentation in digital media, limited digitization of printed and oral resources, and the absence of standardized orthographies and lexical resources for certain languages. moreover, much of the available data is random, noisy, or inconsistently encoded, making it unsuitable for large-scale model training without extensive preprocessing. this lack of data creates a bottleneck for building accurate and inclusive ai systems that can serve speakers of these languages. efforts to address these challenges are further complicated by the prevalence of code-mixing with english and other indian languages in everyday communication. consequently, the digital divide in language technology is widening, with highresource languages benefiting from rapid advances in ai, while low-resource languages risk further marginalization. in this paper, we examine the specific dataset challenges faced by low-resource indian languages, explore their impact on model performance, and results for low resource languages. ## data source low-resource languages such as assamese, bodo, and manipuri face significant challenges in neural machine translation (nmt) due to limited parallel corpora. recent advances in transfer learning have shown that pretrained models on large multilingual datasets can be effectively adapted to such languages, significantly improving translation quality. in this paper, we describe our submission to the wmt 2025 shared task, which combines the opennmt-py 4 framework, large-scale pretrained models, and fine-tuning on target language pairs. we used a combination of publicly available datasets, including: ## results training from scratch for low-resource languages like bodo yields moderate performance but transfer learning from high-resource related languages provides significant gains. using pretrained models trained on bppcc as a base, we achieved bleu improvements of over 12 points for bodo and similar gains for assamese and manipuri. future work will explore multilingual joint fine-tuning and domain adaptation. the evaluation results of three language pair directions nmt system on flores dev set is shown in table ## conclusion we described the team submission to the wmt 2025 shared task on low-resource indic language translation. by combining opennmtpy with transfer learning from bpcc @xcite , we achieved competitive results for english-assamese, english-bodo, and english-manipuri, and vice-versa. future work will explore back-translation, domain adaptation, and multilingual pre-training with additional indic languages to further enhance low-resource translation performance