U v A - MT ’s Participation in the WMT 25 General Translation Shared Task (2025)

Abstract: This paper presents UvA-MT’s submission to the WMT 2025 shared task on general machine translation, competing in the unconstrained track across all 16 translation directions. Unusually, this year we use only WMT25’s blind test set (source sentences only) to generate synthetic data for LLM training, and translations are produced using pure beam search for submission. Overall, our approach can be seen as a special variant of data distillation, motivated by two key considerations: (1) perfect domain alignment, where the training and test domains are distributionally identical; and (2) the strong teacher model, GPT-4o-mini, offers high-quality outputs as both a reliable reference and a fallback in case of mere memorization.Interestingly, the outputs of the resulting model, trained on Gemma3-12B using Best-of-N (BoN) outputs from GPT-4o-mini, outperform both original BoN outputs from GPT-4o-mini and Gemma3-12B in some high-resource languages across various metrics. We attribute this to a successful model ensemble, where the student model (Gemma3-12B) retains the strengths of the teacher (GPT-4o-mini) while implicitly avoiding their flaws.

## introduction in this paper, we describe the details of our submission to the wmt 2025 shared task on the general machine translation (unconstrained track), which includes 16 translation directions. with recent advances in large language models (llms), particularly the emergence of stronger multilingual models, our focus in this paper is on effectively and efficiently adapting a general-purpose llm for translation-specific tasks with limited training. unusually, this year we use only the test set to build synthetic data for model training and generate translations again based on the test set using pure beam search for submission, as shown in figure 1 . this is based on several considerations: figure 1 : we use gpt-4o-mini to generate 16 hypotheses per sample on the wmt25 test set (at the paragraph level) using nucleus sampling. the resulting hypotheses are then used to train gemma3-12b with the calibration method of @xcite . finally, the calibrated gemma model is used to translate the wmt25 test set for submission using pure beam search. in the following sections, we test whether the student model (gemma-3-12b) retains the strengths of gpt-4o-mini's outputs while implicitly avoiding certain flaws-effectively acting as a model ensemble. more specifically, our strategy consists of two main steps: synthetic data building. we feed the wmt25 test set into gpt-4o-mini @xcite , using the prompts provided with the offical test set 2 , to generate 16 hypotheses per sample. the hypotheses are decoded using nucleus sampling with a topp of 0.98 3 and a temperature of 1.0. each sample is at the paragraph level, where "\n" remains in the original data as a separator. post-training. we apply the calibration method @xcite only to post-train gemma-3-12b, which has been shown to be more effective than supervised fine-tuning or recent preference optimization methods, like cpo @xcite . the calibration method aims to improve the correlation between translation likelihood and quality scores as measured by a reference metric model, enhancing the effectiveness of beam search decoding. following @xcite , we use cometkiwi-xxl to score each one-to-many translation pair in our synthetic dataset. finally, the resulting model, trained on synthetic data derived from wmt25 test set, is used to again translate the wmt25 test set. we observe that for some high-resource languages, the resulting model's outputs even surpass the best hypotheses in the synthetic data-demonstrating a successful form of model ensemble. in our next version, we provide detailed experimental settings and results, including: (1) offline experiments demonstrating the effectiveness of the calibration method; (2) offline experiments evaluating this test-time model ensemble strategy; and (3) our evaluation results for the final submission. ## calibration method we now briefly describe our post-training method, namely calibration @xcite . this method addresses the miscalibration problem in machine translation, where translation quality deteriorates as search approximations improve and higherprobability hypotheses are potentially worse translations. prior studies have tried to mitigate this miscalibration issue by introducing an additional optimization step during inference time, known as qualityaware decoding (qad) @xcite . these approaches typically involve generating multiple candidate translations through sampling, followed by reranking or voting using reference-free and/or reference-based machine translation metrics, such as best-of-n (bon) sampling @xcite and minimum bayes risk (mbr) decoding @xcite . the calibration approach mitigates this issue by optimizing the pearson correlation between translation likelihood and quality during training time. extensive experiments from @xcite show several key advantages of this method, including: in this shared task, we employ calibration as our only post-training method. for further technical details, please refer to @xcite . ## online evaluation we thoroughly evaluate our system's outputs and compare them with those of several highperformance open-source and closed-source llm-based translation systems, including gpt-4.1, claude-4, command-r+, deepseek-v3, towerplus-9b, towerplus-72b, qwen2.5-7b, qwen3-235b, and ayaexpanse-32b. we access these systems' results from the wmt25 mt evaluation test set foot_4 , which was released a few weeks before the submission deadline of this paper. we report results using three metrics: cometkiwi da 23 -xl, cometkiwi da 23 -xxl @xcite , and comet da 22 @xcite . in addition, we conduct a light human evaluation for the english-chinese track, comparing our system (uva-mt) with our base model, gemma-3-12b. ## discussion and conclusion beyond metric hacking. we acknowledge that some degree of metric gaming is present in the results above, although its extent is difficult to quantify. our focus in this competition, however, is to demonstrate gains that go beyond mere metric hacking. in the extreme case where uva-mt simply memorized the best outputs from gpt-4o-mini (maximizing hacking), the latter's score would represent the upper bound of the former. therefore, a direct comparison between uva-mt and gpt-4o-mini (best) remains realistic, and any gain over gpt-4o-mini (best) would reflect genuine enhancements. interestingly, we observe them in most of the language directions. we attribute these gains to a form of successful model ensemble, in which the student llms integrate the strengths of the teacher model's outputs while discarding some of their shortcomings. regarding the role of the post-training method applied here, including whether it is a critical component for this ensemble, we leave for future investigation. practical significance. our setting is not wellsuited for real-time translation systems, as training a student model is required for each group of new inputs. nevertheless, our findings point to a promising direction for ensembling the strengths of two models when the target domain is established in advance. this is particularly relevant in practical scenarios such as customized translation, where latency is secondary and effectiveness is the foremost priority.