SH at WMT 25 General Machine Translation Task (2025)

Abstract: I participated in the unconstrained track of the English-to-Japanese translation task at the WMT 2025 General Machine Translation Task.My submission leverages several large language models, all of which are trained with supervised fine-tuning, and some further optimized via preference learning.To enhance translation quality, I introduce an automatic post-editing model and perform automatic post-editing.In addition, I select the best translation from multiple candidates using Minimum Bayes Risk (MBR) decoding.For MBR decoding, I use COMET-22 and LaBSE-based cosine similarity as evaluation metrics.

## introduction in this paper, we describe the system submitted by team sh to wmt2025. we participated in the unconstrained track of the general machine translation task for englishto-japanese (en-ja) translation. our submission leverages several large language models (llms) trained with supervised fine-tuning, and some further optimized via preference learning. to enhance translation quality, we introduce an automatic post-editing model that performs automatic post-editing. additionally, we select the best translation from multiple candidates using minimum bayes risk (mbr) decoding @xcite . for mbr decoding, we use comet-22 @xcite and labse-based cosine similarity @xcite as evaluation metrics. our system is designed to translate text on a sentence-by-sentence basis, with each sentence separated by newlines. our system is based on two hypotheses. first, we hypothesize that preference learning contributes to improving translation quality, as it can consider both positive and negative examples, encouraging the model to generate better translations. second, we hypothesize that the combination of automatic post-editing and mbr decoding contributes to improving translation quality. while automatic post-editing can sometimes degrade translations, using mbr decoding allows us to select the best translation from multiple candidates, thereby mitigating the risk of degradation and improving overall translation quality. ## system overview our system consists of three components: an initial translation model (section 3), an automatic post-editing model (section 4), and a reranking step (section 5). the overall architecture of the system is shown in figure 1 . the initial translation model produces a translation given the source text as input. the automatic post-editing model takes both the source text and the initial translation as input and generates an improved translation, thereby enhancing the output of the initial translation model. the initial translation model is trained using supervised fine-tuning (sft) and preference learning, while the automatic post-editing model is trained using sft. we denote the model trained with only sft as init sft and the model trained with both sft and preference learning as init simpo . their corresponding automatic postediting models are denoted as pedit sft and pedit simpo , respectively. during inference, we generate multiple translations from these models and select the best translation using mbr decoding. specifically, we construct a candidate set from four pipelines: init sft alone, init simpo alone, pedit sft applied to the output of init sft , and pedit simpo applied to the output of init simpo . 3 initial translation model 3.1 datasets we used the following datasets for supervised finetuning: news commentary @xcite , the kyoto free translation corpus (kftt) (neubig, 2011), ted talks @xcite , and past wmt general machine translation task test data (wmt20, wmt21, wmt22, wmt23). for preference learning, we used the same datasets as for supervised fine-tuning, but reformatted them to meet the requirements. preference learning requires a triplet consisting of source text, preferred translation, and non-preferred translation. however, the original datasets contain only the source text and the reference translation. therefore, we translated the source text using sft model and used the output as the nonpreferred translation. ## experiment and analysis we used the wmt24++ @xcite as the evaluation dataset. we evaluated the system using automatic metrics, specifically comet-22 and bleu @xcite . we used sacre-blue @xcite for bleu calculation. we evaluated zero-shot performance of the base model as a baseline. in addition to the submitted system, we compared the following four model configurations: for baseline, init sft and init simpo , we used the same prompt as the one used during the training of the initial translation model. for pedit sft and pedit simpo , we used the same prompt as the one used during the training of the automatic post-editing model. since the baseline outputs think tokens, we removed the text enclosed @xmath0. except for the submitted system, we used the default decoding strategy of the base model: temperature 0.6 and top-p 0.95. the baseline achieved a significantly lower bleu score, possibly because its outputs often contained extraneous information in addition to the translation (see also, when automatic post-editing was applied, the bleu score decreased, but the comet-22 score improved. this may be because there was only one reference sentence when calculating the bleu score this time, and the automatic postediting, which generates diverse expressions, led to a decrease in the bleu score. on the other hand, since comet-22 is tolerant of diverse expressions, it is thought that the comet-22 score improved when automatic post-editing was applied. in fact, when we checked some translation examples, we found that there were few cases where the quality significantly deteriorated when automatic post-editing was applied. the submitted system achieved a highest score in comet-22. this is thought to be due to mbr decoding, which can suppress quality degradation caused by automatic post-editing. based on these results, we confirm that supervised fine-tuning, preference learning, and the introduction of the automatic post-editing model are effective. ## conclusion in this paper, we described the system submitted by team sh to the wmt2025 general machine translation task. we developed the initial translation model through supervised fine-tuning and preference learning, and the automatic postediting model through supervised fine-tuning. we generated multiple translations from these models and selected the best translation using mbr decoding. the submitted system demonstrated superior performance compared to the baseline in both automatic evaluation metrics, namely bleu and comet-22.