GenAI4ELab/papercli-papers huggingface dataset:
DLUT and GTCOM’s Large Language Model Based Translation System for WMT25
Authors: Hao Zong; Chao Bei; Wentao Chen; Conghu Yuan; Huan Liu; Degen Huang
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper presents the submission from Dalian University of Technology (DLUT) and Global Tone Communication Technology Co., Ltd. (GTCOM) to the WMT25 General Machine Translation Task. Amidst the paradigm shift from specialized encoder-decoder models to general-purpose Large Language Models (LLMs), this work conducts a systematic comparison of both approaches across five language pairs. For traditional Neural Machine Translation (NMT), we build strong baselines using deep Transformer architectures enhanced with data augmentation. For the LLM paradigm, we explore zero-shot performance and two distinct supervised fine-tuning (SFT) strategies: direct translation and translation refinement. Our key findings reveal a significant discrepancy between lexical and semantic evaluation metrics: while strong NMT systems remain competitive in BLEU scores, fine-tuned LLMs demonstrate marked superiority in semantic fidelity as measured by COMET. Furthermore, we find that fine-tuning LLMs for direct translation is more effective than for refinement, suggesting that teaching the core task directly is preferable to correcting baseline outputs.
Yandex Submission to the WMT25 General Machine Translation Task
Authors: Nikolay Karpachev; Ekaterina Enikeeva; Dmitry Popov; Arsenii Bulgakov; Daniil Panteleev; Dmitrii Ulianov; Artem Kryukov; Artem Mekhraliev
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper describes Yandex submission to the WMT25 General Machine Translation task. We participate in English-to-Russian translation direction and propose a purely LLM-based translation model. Our training procedure comprises a training pipeline of several stages built upon YandexGPT, an in-house general-purpose LLM. In particular, firstly, we employ continual pretraining (post-pretrain) for MT task for initial adaptation to multilinguality and translation. Subsequently, we use SFT on parallel document-level corpus in the form of P-Tuning. Following SFT, we propose a novel alignment scheme of two stages, the first one being a curriculum learning with difficulty schedule and a second one - training the model for tag preservation and error correction with human post-edits as training samples. Our model achieves results comparable to human reference translations on multiple domains.
IRB-MT at WMT25 Translation Task: A Simple Agentic System Using an Off-the-Shelf LLM
Authors: Ivan Grubišić; Damir Korencic
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: Large Language Models (LLMs) have been demonstrated to achieve state-of-art results on machine translation. LLM-based translation systems usually rely on model adaptation and fine-tuning, requiring datasets and compute. The goal of our team’s participation in the “General Machine Translation” and “Multilingual” tasks of WMT25 was to evaluate the translation effectiveness of a resource-efficient solution consisting of a smaller off-the-shelf LLM coupled with a self-refine agentic workflow. Our approach requires a high-quality multilingual LLM capable of instruction following. We select Gemma3-12B among several candidates using the pretrained translation metric MetricX-24 and a small development dataset. WMT25 automatic evaluations place our solution in the mid tier of all WMT25 systems, and also demonstrate that it can perform competitively for approximately 16% of language pairs.
Improving Low-Resource Japanese Translation with Fine-Tuning and Backtranslation for the WMT 25 General Translation Task
Authors: Felipe Fujita; Hideyuki Takada
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: In this paper, we explore the effectiveness of combining fine-tuning and backtranslation on a small Japanese corpus for neural machine translation. Starting from a baseline English→Japanese model (COMET = 0.460), we first apply backtranslation (BT) using synthetic data generated from monolingual Japanese corpora, yielding a modest increase (COMET = 0.468). Next, we fine-tune (FT) the model on a genuine small parallel dataset drawn from diverse Japanese news and literary corpora, achieving a substantial jump to COMET = 0.589 when using Mistral 7B. Finally, we integrate both backtranslation and fine-tuning—first augmenting the small dataset with BT generated examples, then adapting via FT—which further boosts performance to COMET = 0.597. These results demonstrate that, even with limited training data, the synergistic use of backtranslation and targeted fine-tuning on Japanese corpora can significantly enhance translation quality, outperforming each technique in isolation. This approach offers a lightweight yet powerful strategy for improving low-resource language pairs.
Multi-agentMT: Deploying AI Agent in the WMT25 Shared Task
Authors: Ahrii Kim
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We present Multi-agentMT, our system for the WMT25 General Shared Task. The model adopts Prompt Chaining, a multi-agent workflow combined with Rubric-MQM, an automatic MQM-based error annotation metric. Our primary submission follows a Translate–Postedit–Proofread pipeline, in which error positions are explicitly marked and iteratively refined. Results suggest that a semi-autonomous agent scheme for machine translation is feasible with a smaller, earlier-generation model in low-resource settings, achieving comparable quality at roughly half the cost of larger systems.
Laniqo at WMT25 General Translation Task: Self-Improved and Retrieval-Augmented Translation
Authors: Kamil Guttmann; Zofia Rostek; Adrian Charkiewicz; Antoni Solarski; Mikołaj Pokrywka; Artur Nowakowski
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This work describes Laniqo’s submission to the constrained track of the WMT25 General MT Task. We participated in 11 translation directions. Our approach combines several techniques: fine-tuning the EuroLLM-9B-Instruct model using Contrastive Preference Optimization on a synthetic dataset, applying Retrieval-Augmented Translation with human-translated data, implementing Quality-Aware Decoding, and performing postprocessing of translations with a rule-based algorithm. We analyze the contribution of each method and report improvements at every stage of our pipeline.
Command-A-Translate: Raising the Bar of Machine Translation with Difficulty Filtering
Authors: Tom Kocmi; Arkady Arkhangorodsky; Alexandre Berard; Phil Blunsom; Samuel Cahyawijaya; Théo Dehaze; Marzieh Fadaee; Nicholas Frosst; Matthias Galle; Aidan Gomez; Nithya Govindarajan; Wei-Yin Ko; Julia Kreutzer; Kelly Marchisio; Ahmet Üstün; Sebastian Vincent; Ivan Zhang
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We present Command A Translate, an LLMbased machine translation model built off Cohere’s Command A. It reaches state-of-the-art machine translation quality via direct preference optimization. Our meticulously designed data preparation pipeline emphasizes robust quality control and a novel difficulty filtering – a key innovation that distinguishes Command A Translate. Furthermore, we extend our model and participate at WMT with a system (CommandA-WMT) that uses two models and post-editing steps of step-by-step reasoning and limited Minimum Bayes Risk decoding.
GENDER1PERSON: Test Suite for Estimating Gender Bias of First-person Singular Forms
Authors: Maja Popović; Ekaterina Lapshinova-Koltunski
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: The gender1person test suite is designed to measure gender bias in translating singular first-person forms from English into two Slavic languages, Russian and Serbian. The test suite consists of 1,000 Amazon product reviews, uniformly distributed over 10 different product categories. Bias is measured through a gender score ranging from -100 (all reviews are feminine) to 100 (all reviews are masculine). The test suite shows that the majority of the systems participating in the WMT-2025 task for these two target languages prefer the masculine writer’s gender. There is no single system which is biased towards the feminine variant. Furthermore, for each language pair, there are seven systems that are considered balanced, having the gender scores between -10 and 10.Finally, the analysis of different products showed that the choice of the writer’s gender depends to a large extent on the product. Moreover, it is demonstrated that even the systems with overall balanced scores are actually biased, but in different ways for different product categories.
Evaluation of LLM for English to Hindi Legal Domain Machine Translation Systems
Authors: Kshetrimayum Boynao Singh; Deepak Kumar; Asif Ekbal
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: The study critically examines various Machine Translation systems, particularly focusing on Large Language Models, using the WMT25 Legal Domain Test Suite for translating English into Hindi. It utilizes a dataset of 5,000 sentences designed to capture the complexity of legal texts, based on word frequency ranges from 5 to 54. Each frequency range contains 100 sentences, collectively forming a corpus that spans from simple legal terms to intricate legal provisions. Six metrics were used to evaluate the performance of the system: BLEU, METEOR, TER, CHRF++, BERTScore and COMET. The findings reveal diverse capabilities and limitations of LLM architectures in handling complex legal texts. Notably, Gemini-2.5-Pro, Claude-4 and ONLINE-B topped the performance charts in terms fo human evaluation, showcasing the potential of LLMs for nuanced trans- lation. Despite these advances, the study identified areas for further research, especially in improving robustness, reliability, and explainability for use in critical legal contexts. The study also supports the WMT25 subtask focused on evaluating weaknesses of large language models (LLMs). The dataset and related resources are publicly available at https://github.com/helloboyn/WMT25-TS.
RoCS-MT v2 at WMT 2025: Robust Challenge Set for Machine Translation
Authors: Rachel Bawden; Benoît Sagot
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: RoCS-MT (Robust Challenge Set for Machine Translation) was initially proposed at the test suites track of WMT 2023. Designed to challenge MT systems’ translation performance on user-generated content (UGC), it contains examples sourced from English Reddit, with manually normalised versions, aligned labelled annotation spans and reference translations in five languages. In this article, we describe version 2 of RoCS-MT in the context of the 2025 WMT test suites track. This new version contains several improvements on the initial version including (i) minor corrections of normalisation, (ii) corrections to reference translations and addition of alternative references to accommodate for different possible genders (e.g. of speakers) and (iii) a redesign and re-annotation of normalisation spans for further analysis of different non-standard UGC phenomena. We describe these changes and provide results and preliminary analysis of the MT submissions to the 2025 general translation task.
Automated Evaluation for Terminology Translation Related to the EEA Agreement
Authors: Selma Dis Hauksdottir; Steinthor Steingrimsson
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper presents a submission to the WMT25 test suite subtask, focusing on terminology in official documents related to the EEA Agreement. The test suite evaluates the accuracy of MT systems in translating terminology for the English-Icelandic translation direction when applied to the EEA documents. We focus on the use of terminology in four domains of the agreement; science, technology, economics, and society. We find that the manual evaluation confirms that our test suite can be helpful in selecting the best MT system for working with these domains. Surprisingly, an online system that does not achieve very high scores on the general translation task, according to preliminary results, is most adept at translating the terminology our test suite evaluates.
Up to Par? MT Systems Take a Shot at Sports Terminology
Authors: Einar Sigurdsson; Magnús Magnússon; Atli Jasonarson; Steinthor Steingrimsson
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We present a submission to the WMT25 test suite subtask, focusing on the capabilities of MT systems to translate sports-related language. Although many sports attract extensive media attention and feature a rich, polysemous language, often shaped by active neologism and community-driven translations, the sports domain has received relatively little focus in MT research. In English-Icelandic automatic translations, sports-specific vocabulary often appears to be mistranslated. Our test suite is designed to test whether this observation holds merit. We evaluate 34 systems, both automatically and manually, and find that sports language poses challenges to a varying degree for all the systems.
Fine-Grained Evaluation of English-Russian MT in 2025: Linguistic Challenges Mirroring Human Translator Training
Authors: Shushen Manakhimova; Maria Kunilovskaya; Ekaterina Lapshinova-Koltunski; Eleftherios Avramidis
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We analyze how English–Russian machine translation (MT) systems submitted to WMT25 perform on linguistically challenging translation tasks, similar to problems used in university professional translator training. We assessed the ten top-performing systems using a fine-grained test suite containing 465 manually devised test items, which cover 55 lexical, grammatical, and discourse phenomena, in 13 categories. By applying pass/fail rules with human adjudication and micro/macro aggregates, we observe three performance tiers. Compared with the official WMT25 ranking, our ranking broadly aligns but reveals notable shifts.Our findings show that in 2025, even top-performing MT systems still struggle with translation problems that require deep understanding and rephrasing, much like human novices do. The best systems exhibit creativity and can be very good at handling such challenges, often producing more natural translations rather than producing word-for-word renditions. However, persistent structural and lexical problems remain: literal word order carry-overs, misused verb forms, and rigid phrase translations were common, mirroring errors typically seen in beginner translator assignments.
Tagged Span Annotation for Detecting Translation Errors in Reasoning LLMs
Authors: Taemin Yeom; Yonghyun Ryu; Yoonjung Choi; Jinyeong Bak
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We present the AIP team’s submission to the WMT 2025 Unified MT Evaluation SharedTask, focusing on the span-level error detection subtask. Our system emphasizes response format design to better harness the capabilities of OpenAI’s o3, the state-of-the-art reasoning LLM. To this end, we introduce Tagged SpanAnnotation (TSA), an annotation scheme designed to more accurately extract span-level information from the LLM. On our refined version of WMT24 ESA dataset, our reference-free method achieves an F1 score of approximately 27 for character-level label prediction, outperforming the reference-based XCOMET-XXL at approximately 17.
COMET-poly: Machine Translation Metric Grounded in Other Candidates
Authors: Maike Züfle; Vilém Zouhar; Tu Anh Dinh; Felipe Maia Polo; Jan Niehues; Mrinmaya Sachan
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: Automated metrics for machine translation attempt to replicate human judgment. Unlike humans, who often assess a translation in the context of multiple alternatives, these metrics typically consider only the source sentence and a single translation. This discrepancy in the evaluation setup may negatively impact the performance of automated metrics. We propose two automated metrics that incorporate additional information beyond the single translation. COMET-polycand uses alternative translations of the same source sentence to compare and contrast with the translation at hand, thereby providing a more informed assessment of its quality. COMET-polyic, inspired by retrieval-based in-context learning, takes in translations of similar source texts along with their human-labeled quality scores to guide the evaluation. We find that including a single additional translation in COMET-polycand improves the segment-level metric performance (0.079 to 0.118 Kendall’s tau-b correlation), with further gains when more translations are added. Incorporating retrieved examples in COMET-polyic yields similar improvements (0.079 to 0.116 Kendall’s tau-b correlation). We release our models publicly.
Long-context Reference-based MT Quality Estimation
Authors: Sami Haq; Chinonso Osuji; Sheila Castilho; Brian Davis; Thiago Castro Ferreira
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: In this paper, we present our submission to the Tenth Conference on Machine Translation (WMT25) Shared Task on Automated Translation Quality Evaluation. Our systems are built upon the COMET framework and trained to predict segment-level ESA scores using augmented long-context data. To construct long-context training examples, we concatenate multiple in-domain sentences and compute a weighted average of their scores. We further integrate human judgment datasets MQM, SQM, and DA) through score normalisation and train multilingual models on the source, hypothesis, and reference translations. Experimental results demonstrate that incorporating long-context information yields higher correlations with human judgments compared to models trained exclusively on short segments.
Evaluating WMT 2025 Metrics Shared Task Submissions on the SSA-MTE African Challenge Set
Authors: Senyu Li; Felermino Dario Mario Ali; Jiayi Wang; Rui Sousa-Silva; Henrique Lopes Cardoso; Pontus Stenetorp; Colin Cherry; David Ifeoluwa Adelani
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper presents the evaluation of submissions to the WMT 2025 Metrics Shared Task on the SSA-MTE challenge set, a large-scale benchmark for machine translation evaluation (MTE) in Sub-Saharan African languages. The SSA-MTE test sets contains over 12,768 human-annotated adequacy scores across 11 language pairs sourced from English, French, and Portuguese, spanning 6 commercial and open-source MT systems. Results show that correlations with human judgments remain generally low, with most systems falling below the 0.4 Spearman threshold for medium-level agreement. Performance varies widely across language pairs, with most correlations under 0.4; in some extremely low-resource cases, such as Portuguese–Emakhuwa, correlations drop to around 0.1, underscoring the difficulty of evaluating MT for very low-resource African languages. These findings highlight the urgent need for more research on robust, generalizable MT evaluation methods tailored for African languages.
Nvidia-Nemo’s WMT 2025 Metrics Shared Task Submission
Authors: Brian Yan; Shuoyang Ding; Kuang-Da Wang; Siqi Ouyang; Oleksii Hrinchuk; Vitaly Lavrukhin; Boris Ginsburg
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper describes Nvidia-Nemo’s WMT 2025 Metrics Shared Task submission. We investigated two strategies for extending Machine Translation (MT) evaluation to unsegmented documents: 1) first segmenting into sentences and then applying regression-based metrics and 2) directly utilizing the long-context capabilities of LLMs. The base comparison of the segmentation-based and LLM-based metrics on the WMT 2023-24 evaluation sets indicated that the former performs more robustly across language pairs.Thus we sought to improve the LLM-based approach by incorporating relative evaluation - this setting jointly evaluates all candidate translations at once and relative to each other, rather than evaluating each separately. Our experiments using the open-source Qwen3 LLM show that relative evaluation improves score correlations with human judgment, but only if the task is structured as a 2-stage evaluate-then-refine problem.
GEMBA V2: Ten Judgments Are Better Than One
Authors: Marcin Junczys-Dowmunt
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: We introduce GEMBA-MQM V2, an MQM-inspired, reference-free LLM evaluation metric for the WMT25 Metrics Shared Task (Subtask 1). Building on GEMBA/GEMBA-MQM, we prompt GPT-4.1-mini to produce structured MQM error annotations per segment. We map annotations to scores with 25/5/1 severity weights (minor punctuation = 0.1). To reduce stochastic variance, each segment is scored ten times and aggregated with a reciprocal-rank weighted average (RRWA) after removing outliers beyond 2𝜎. On the WMT24 MQM test sets, GEMBA-MQM V2 ranks first by average correlation, with strong results across languages and evaluation levels; WMT23 results show comparable performance.
CUNI and Phrase at WMT25 MT Evaluation Task
Authors: Miroslav Hrabal; Ondrej Glembek; Aleš Tamchyna; Almut Silja Hildebrand; Alan Eckhard; Miroslav Štola; Sergio Penkale; Zuzana Šimečková; Ondřej Bojar; Alon Lavie; Craig Stewart
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: This paper describes the joint effort of Phrase a.s. and Charles University’sInstitute of Formal and Applied Linguistics (CUNI/UFAL) on the WMT25Automated Translation Quality Evaluation Systems Shared Task. Both teamsparticipated both in a collaborative and competitive manner, i.e. they eachsubmitted a system of their own as well as a contrastive joint system ensemble.In Task~1, we show that such an ensembling—if chosen in a clever way—canlead to a performance boost. We present the analysis of various kinds ofsystems comprising both “traditional” NN-based approach, as well as differentflavours of LLMs—off-the-shelf commercial models, their fine-tuned versions,but also in-house, custom-trained alternative models. In Tasks~2 and~3 we showPhrase’s approach to tackling the tasks via various GPT models: Error SpanAnnotation via the complete MQM solution using non-reasoning models (includingfine-tuned versions) in Task~2, and using reasoning models in Task~3.
MSLC25: Metric Performance on Low-Quality Machine Translation, Empty Strings, and Language Variants
Authors: Rebecca Knowles; Samuel Larkin; Chi-Kiu Lo
Venue: EMNLP 2025 (Proceedings of the Tenth Conference on Machine Translation)

Abstract: In this challenge set, we examine how automatic metrics for machine translation perform on a wide variety of machine translation output, covering a wider range of quality than the WMT submissions. We also explore metric results on specific types of corner cases, such as empty strings, wrong- or mixed-language text, and more. We primarily focus on Japanese–Chinese data, with some work on English and Czech.