Findings of WMT 2025 Shared Task on Low-resource I ndic Languages Translation (2025)

Abstract: This study proposes the results of the lowresource Indic language translation task organized in collaboration with the Tenth Conference on Machine Translation (WMT) 2025. In this workshop, participants were required to build and develop machine translation models for the seven language pairs, which were categorized into two categories. Category 1 is moderate training data available in languages i.e English–Assamese, English–Mizo, English-Khasi, English–Manipuri and English– Nyishi. Category 2 has very limited training data available in languages, i.e English–Bodo and English–Kokborok. This task leverages the enriched IndicNE-corp1.0 dataset, which consists of an extensive collection of parallel and monilingual corpora for north eastern Indic languages. The participant results were evaluated using automatic machine translation metrics, including BLEU, TER, ROUGE-L, ChrF, and METEOR. Along with those metrics, this year’s work also includes Cosine similarity for evaluation, which captures the semantic representation of the sentence to measure the performance and accuracy of the models. This work aims to promote innovation and advancements in low-resource Indic languages.

## introduction the indic mt shared task, first organized alongside the eighth conference on machine translation (wmt) 2023 1 @xcite , demonstrated the critical need for sustained research attention toward low-resourced languages. that inaugural effort not only revealed the untapped potential for advancing machine translation in these linguistic contexts but 1 https://www2.statmt.org/wmt23/indic-mt-task. html also provided a robust methodological and collaborative foundation for future work. motivated by its impact, the task was expanded and refined in the ninth conference on machine translation (wmt) 2024 foot_0 @xcite , drawing broader participation and richer system diversity. building upon these successive advancements, the 2025 edition foot_1 has emerged as the most successful iteration to date-surpassing previous years in both scale and the quality of contributions-further cementing the task's role as a driving force in low-resource mt research. india's linguistic landscape is remarkably diverse, encompassing hundreds of languages spoken across its regions. while 22 languages are officially recognized under the eighth schedule of the indian constitution and receive governmental support in terms of infrastructure, research, and funding, many others-often spoken by indigenous and minority communities-remain excluded from such provisions. these low-resource languages frequently lack standardized orthographies, adequate lexical resources (e.g., corpora), and formal linguistic documentation. limited institutional support, declining intergenerational transmission, and minimal access to modern technologies further threaten their preservation and vitality. to address these challenges, our initiative is dedicated to revitalizing and documenting low-resource indic languages through targeted, technologydriven solutions. building upon the success of the indic mt shared tasks at wmt 2023 and wmt 2024, which focused on four language pairs (en-glish-assamese, english-mizo, english-khasi, and english-manipuri) using the enriched indicne-corp1.0 dataset, we have expanded the scope in 2025 to seven language pairs. these are divided into two categories: (1) languages with moderate amounts of training data: english-assamese, english-mizo, english-khasi, english-manipuri, and english-nyishi. (2) languages with very limited training data: english-bodo and english-kokborok. this year's task, which has already surpassed prior editions in scale and participation, continues to drive innovation in machine translation and nlp, developing solutions specifically adapted to the unique linguistic and resource constraints of lowresource indic languages. the indic mt shared task initiative is also committed to safeguarding india's rich linguistic diversity and cultural heritage by strengthening the rights and identities of minority language communities. leveraging state-of-theart technologies, it seeks to advance the capabilities of low-resource indic languages, enabling them not only to survive but to flourish within today's increasingly digital and interconnected landscape. the task is evaluated using a wide range of metrics, integrating automatic lexical evaluation metrics such as bleu @xcite , ter @xcite , rouge-l @xcite , meteor @xcite , and chrf @xcite . in addition to the standard evaluation metrics, we used cosine similarity using sentence-bert(all-mpnet-base-v2) @xcite to compute the semantic similarity between the predicted and reference sentences in the english language direction. by combining traditional and semantic evaluation methods, this approach provides a thorough assessment of the performance and accuracy of translation systems. ## languages this section is divided into two parts based on the availability of training data: category 1 includes languages with moderate resources, and category 2 includes languages with very limited resources. in the wmt 2024 edition of the shared task, the focus was limited to four languages: assamese, mizo, khasi, and manipuri. building upon that foundation, the wmt 2025 task expanded the linguistic coverage by including three additional languages: nyishi, bodo, and kokborok. this expansion reflects our ongoing commitment to improving the representation of under-resourced languages in machine translation research and to broadening the scope of technological inclusion for more linguistically marginalised communities. ## participants and system descriptions in this wmt 2025 indic mt shared task, a total of 17 teams, as illustrated in the a3-108: this team (yadav and shrivastava, 2025) system focused on translation for lowresource language pairs, combining a phrasebased smt framework with subword segmentation through multiple bpe merge operations (500-3000 merges). their approach involved concatenating and deduplicating segmented bitext to enhance vocabulary coverage and reduce sparsity, supported by kenlm-trained target-side language models. they submitted results for four english-x pairs: nyishi, manipuri, khasi, and assamese. ## results and discussion the results of the wmt 2025 indic machine translation (mt) shared task are illustrated in the tables below. for clarity, results are reported separately for each language pair and direction: assamese-english (as-en) in an evaluation of the quantitative results was performed using metrics like bleu, meteor, rouge-l, chrf, and ter. bleu measures the precision of n-grams in candidate translations relative to reference translations. ter quantifies the number of edits required to transform the candidate translation into the reference. rouge-l evaluates the longest common subsequence between the candidate and reference, emphasizing recall-oriented aspects of translation quality. chrf computes the character n-gram f-score, providing sensitivity to morphological variations. meteor combines precision, recall, and synonym matching to capture translation adequacy and fluency. in addition to the traditional statistical metrics this year's evaluation also incorporates semantic similarity based on cosine similarity between sentence embeddings generated by the all-mpnet-base model for the indic language to english direction. ## analysis the evaluation results across multiple translation directions reveal a competitive landscape with significant variations in performance. a key finding is the direct correlation between the size of the parallel training data and the translation quality, although some notable exceptions exist. the figures (figure 1 , 2 , and 3 ) illustrate these findings, providing a visual context for the observations. figure 1 shows the best primary bleu score for each language direction (the highest bleu among primary submissions for that direction). figure 2 visualizes bleu vs. chrf for the set of all primary submissions.