G oogle T ranslate’s Research Submission to WMT 2025 (2025)

Abstract: Large Language Models have shown impressive multilingual capabilities, where translation is one among many tasks. Google Translate’s submission to the 2025 WMT evaluation tries to research how these models behave when pushing their translation performance to the limit. Starting with the strong Gemma 3 model, we carry out supervised fine tuning on high quality, synthetically generated parallel data. Afterwards we perform an additional reinforcement learning step, with reward models based on translation metrics to push the translation capabilities even further. Controlling the combination of reward models, including reference-based and quality estimation metrics, we found that the behaviour of the model could be tailored towards a more literal or more creative translation style. Our two submissions correspond to those two models. We chose the more creative system as our primary submission, targetting a human preference for better sounding, more naturally flowing text, although at the risk of losing on the accuracy of the translation. It is an open question to find the sweet spot between these two dimensions, which certainly will depend on the specific domain to handle and user preferences.

## introduction in this paper we present google translate's research submission to the general mt track for the wmt 2025 shared task. starting with gemma 3 (gemma team, 2025), a strong multilingual llm, we focus on improving its translation capabilities through supervised fine-tuning (section 2) and reinforcement learning (rl) (section 3). we use a mix of human-and synthetically-generated parallel data for boosting translation performance, as well as general domain post-training data in order to mostly retain the general capabilities of the original model. through combinations of different reward models in the reinforcement learning step, we were able to generate two candidates: one more targeted towards fluent translations, the other towards more literal but sometimes slightly unnatural translations. in the end we chose to submit the more fluent system as our primary submission. ## reinforcement learning we performed reinforcement learning on top of the sft checkpoint, using an ensemble of metrics as reward models, to further boost translation quality. ## automatic post-editing in order to minimize formatting errors, we run an additional post-editing pass on the resulting translations. each model post-edits its own translations, i.e. they serve both as translation and post-edit systems. we prompt the model to just fix the formatting, without altering the text of the translation, using the prompt shown in appendix a. the examples were chosen from errors we spotted during the development process of the model. on automatic metrics we saw small but consistent improvements for both gemtrans1 and gemtrans2 systems. ## final submission as described in section 3.3, we ended up with two main candidates for submission. metricx showed a clear preference for gemtrans1, although there was a clear drop in chrf. 4 this might be a signal of the model overfitting to metricx, so in order to get a better picture, we prompted gemini to compare a subset of the translations of the wmt25 test sets, assigning a score of +1 if gemtrans1 was preferred, and -1 otherwise. the results can be found in additionally we also prompted gemini to perform an mqm evaluation with additional quality scoring (similar to this year's setup in the shared task evaluation). in this case, the system preferred gemtrans2, both in terms of the quality score as well as mqm. when looking into the decomposition into accuracy and fluency scores, the mqm analysis shows better accuracy for gemtrans2, but at the cost of fluency. language pair mean @xmath0.22 @xmath1.07 @xmath2.24 @xmath3.23 @xmath4.07 @xmath5.15 @xmath6.26 @xmath7.26 @xmath8.31 @xmath9.31 @xmath10.26 @xmath11.20 @xmath12.31 @xmath13.11 @xmath14.12 average +0.17 lastly we manually spot checked the translations (focusing on german outputs). we can confirm the conclusions of the mqm analysis, as we found the output of gemtrans1 to be more fluent than that of gemtrans2. of course all that glitters is not gold, and we also found examples where the added text from gem-trans1 was incorrect or even hallucinated. as for the question of overfitting to metricx, we can- source so that's it for today's ikea hacks video. i hope that you enjoy this one and give it a big thumbs up if you did and let me know in the comment section below which one of these projects was your favorite. [...] gemtrans1 das war's für heute mit unserem video zu ikea-hacks. ich hoffe, es hat euch gefallen, und wenn ja, gebt uns gerne ein "daumen hoch"! schreibt uns auch in den kommentaren, welches dieser projekte euch am besten gefallen hat. [...] gemtrans2 das war's für heute mit diesem video zu ikea-hacks. ich hoffe, es hat ihnen gefallen, und geben sie ihm einen daumen hoch, wenn das der fall war, und lassen sie mich im kommentarbereich unten wissen, welches dieser projekte ihr favorit war. [...] source game 17 of #universitychallenge is the first of eight knockout games in the second round. this was ucl and lse, and ucl gave an outstanding performance: [...] gemtrans1 das 17. spiel der #universitychallenge ist das erste von acht k.o.-spielen in der zweiten runde. es wurde zwischen der ucl (university college london) und der lse (london school of economics) gespielt, und die ucl zeigte eine herausragende leistung: [...] gemtrans2 spiel 17 von #universitychallenge ist das erste von acht k.o.-spielen in der zweiten runde. es handelte sich um ucl und lse, und ucl zeigte eine herausragende leistung: [...] not completely discard this hypothesis, but we did not see any evidence of pathological outputs that might be gaming the metric. we decided to move forward with this system, as the general quality indeed seemed to be superior to that of gemtrans2, and this constituted our primary submission for the shared task. ## findings of the human evaluation the organizers of wmt shared the results of the human evaluation. the performance of gemtrans1 the ranks show a wider variance, due to the highly competitive landscape of this year's shared task and the big number of participating systems. for english to arabic gemtrans1 obtained a very low human score, along with all other systems in its cluster. this was the result of the system failing to produce the correct arabic dialect (egyptian). in a related fashion, bhojpuri showed low automatic scores (thus gemtrans1 was not included in the human evaluation) and we suspect that gem-trans1 failed to generate bhojpuri, falling back to hindi instead. in the report about the shared task, in the additional analysis of the serbian translations, the organizers explicitly highlight the gemtrans1 system for a "notable amount of idiomatic translations, even more than humans" @xcite , although they also point out a "relatively high number of errors". these findings agree with our own observations about the system. ## conclusions we have presented the google translate research submission to the wmt25 evaluation campaign. starting from gemma 3 we used supervised finetuning and rl to boost translation performance, in addition to a small automatic post-editing step to improve the formatting of the translations. out of the two final candidate systems, we found that one was more tailored towards fluency while the other one was more tailored towards accuracy (possibly illustrating the tradeoff discussed by e.g. @xcite ; schleiermacher (1816); dryden (1685)). after evaluation with automatic metrics as well as manual inspection, we decided to move forward with the more fluent system, as it seemed to produce generally higher quality translations. whether that was the correct decision may largely depend on the criterion of the human evaluators. quality of machine translation is indeed in the eye of the beholder, and the more "free-style" translations produced by the system may not be the preferred ones if more literal translation are desired, closer to the source sentence. this dichotomy again highlights the difficulties of machine translation evaluation, and may be indeed point towards new research directions (for both mt generation and evaluation), where the intent of the translation can play a more relevant role.