Calibrating Translation Decoding with Quality Estimation on LLMs
Authors: Di Wu
Yibin Lei
Christof Monz
Venue: NeurIPS 2025 poster ()
Keywords: Translation
Quality Estimation
LLMs
Calibration
Primary area: applications
TL;DR: We propose a shared calibration objective for both quality optimization and quality estimation in translation, offering a unified perspective on these two tasks.

Abstract: Neural machine translation (NMT) systems typically employ maximum *a&nbsp;posteriori* (MAP) decoding to select the highest-scoring translation from the distribution. However, recent evidence highlights the inadequacy of MAP decoding, often resulting in low-quality or even pathological hypotheses as the decoding objective is only weakly aligned with real-world translation quality. This paper proposes to directly calibrate hypothesis likelihood with translation quality from a distributional view by directly optimizing their Pearson correlation, thereby enhancing decoding effectiveness. With our method, translation with large language models (LLMs) improves substantially after limited training (2K instances per direction). This improvement is orthogonal to those achieved through supervised fine-tuning, leading to substantial gains across a broad range of metrics and human evaluations. This holds even when applied to top-performing translation-specialized LLMs fine-tuned on high-quality translation data, such as Tower, or when compared to recent preference optimization methods, like CPO. Moreover, the calibrated translation likelihood can directly serve as a strong proxy for translation quality, closely approximating or even surpassing some state-of-the-art translation quality estimation models, like CometKiwi. 
Lastly, our in-depth analysis demonstrates that calibration enhances the effectiveness of MAP decoding, thereby enabling greater efficiency in real-world deployment. The resulting state-of-the-art translation model, which covers 10 languages, along with the accompanying code and human evaluation data, has been released: https://github.com/moore3930/calibrating-llm-mt.

Calibrating Translation Decoding with Quality Estimation on LLMs
Di Wu
Yibin Lei University of Amsterdam {d. wu, y. rel,, c.monz)@uva.nn nl
Christof Monz
wu,
Abstract
Neural machine translation (NMT) systems typically employ maximum a posteriori (MAP) decoding to select the highest-scoring translation from the distribution. However, recent evidence highlights the inadequacy of MAP decoding, often result- ing in low-quality or even pathological hypotheses as the decoding objective is only weakly aligned with real-world translation quality. This paper proposes to calibrate hypothesis likelihood with translation quality from distributional view by directly optimizing their Pearson correlation, thereby enhancing decoding effectiveness. With method, translation with large language models (LLMs) improves sub- stantially after limited training (2K instances per direction). This improvement is orthogonal to those achieved through supervised fine-tuning, leading to substantial gains across a broad range of metrics and human evaluations. This holds even when applied to top- oo-performing translation- rransatton-sseccialzeed LLMs fine-tuned on high- quality translation data, such as Tower, or when compared to recent preference optimization methods, like CPO. Moreover, the calibrated translation likelihood can directly serve as a strong proxy for translation quality, closely approximating even surpassing some state-of-the-art translation quality estimation models, like CometKiwi. Lastly, our in-depth analysis demonstrates that calibration enhances the effectiveness of MAP decoding, thereby enabling greater efficiency in real- world deployment. The resulting state-of-the-art translation model, which covers 10 languages, along with the accompanying code and human evaluation data, has http:hhuw.oom/mooooe9///////aaae -lmm-m
been released:
Introduction
The training of neural machine translation (NMT) has long been formulated as maximum likelihood estimation (MLE) problem, and maximum a posteriori decoding (MAP) a common decision rule, aiming to identify the highest-scoring translation. However, recent findings in NMT suggest that such a system has serious flaws. One particularly counterintuitive issue the beam search curse [Koehn and Knowles, 2017, Murray and Chiang, 2018, Ott et al., 201 8, Kumar and Sarawagi. 2019], where translation quality deteriorates as search approximations improve and higher- probbbiiry hypotheses possibly worse translations.
Ott et al. [2018] explain this as calibration issue. There is generally low correlation between hypothesis likelihood and quality beyond certain likelihood value. The flaws of poorly calibrated translation models can be categorized into two main aspects: First, their performance tends to be suboptimal, as better hypotheses may remain hidden within the distribution mass, inaccessible to MAP decoding [Otr et al., 2018]. Second, likelihood, as key measure of uncertainty, can serve as a valuable indicator of translation errors within the output at test time [Wang et al., 2020, Fomicheva al., 2020]; however, poorly calibrated models cannot effectively support this use.
Prior studies have tried to mitigate this miscalibration issue by introducing an additional optimization step during inference time, shown in Figure (top), known as quality-aware decoding (QAD) [Fer-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Testing Time Optimization
Sampling
Reranking / Voting
Highest-scoring Output
Trandlae the following toxt from Chinese.
Quality
English: Amsterdam famous its hundreds of beautiful canals, millions bicycles, stunning summers, and disgusting winters.
Training Time Optimization
Chinese:
Sampling
Calibration
NMT
Libelthood
Likelihood Quality
Figure I: The illustration of quale decoding (top) and our calibration method (bottom). The former explores the decoding space by adding extra step during test-time decoding, which involves multiple rounds sampling followed by reranking or voting to select the highest-scoring output. Our method focuses on training-tinn time optimization, aiming to calibrate the likelihoods of hypotheses to their corresponding quality scores, enabling effective approximated MAP decoding.
nandes al., 2022]. These approaches typically involve generating multiple candidate translations through sampling. followed by reranking or voting using reference-free and/or reference-based machine translation metries, such as Best-of-N (BoN) sampling [Rei al., 2024a, Faria et al., 2024, Brown et al., 2024, Ichihara et al., 2025] and Minimum Bayes Risk (MBR) decoding [Kumar and Byrne, 2004, Freitag et al., 2022a]. While effective, this line of methods inevitably incurs significantly higher latency. For example, the current top- oopperforming system in the WMT24 competition [Koemi et al., 2024a], Tower [Rei et al., 2024a], requires sampling 100 times for each prompt from a 70B LLM-based translation model, which is impractical, particularly for online environments.
This paper proposes simple yet highly effective method to directly optimize the calibration during training, where we encourage the likelihood of translation hypotheses to align more closely with their quality by directly optimizing Pearson correlations between them. More specifically, as shown in Figure (bottom), given translation prompt we sample multiple hypotheses from an NMT model during training time and compute both their likelihoods and quality scores, as measured by any external metric, such as COMET. We then define the loss the negative Pearson correlation between the two sets data points and minimize directly using a standard gradient-basee optimizer. While simple, approach yields substantial performance improvements with limited training- only 2K instances per language even when applied to state-of-the-an LLM-based MT systems such Tower. Moreover, due to clear calibration improvements, the resulting models' likelihood can directly serve as a strong proxy for translation quality, even surpassing some state-of-the- translation quality estimation (QE) models, like Comet- Kiwi [Rei et al., 2022a].
Our main contributions are summarized as follows:
We introduce simple yet effective method to mitigate the well-known miscalibration issue in translation during training time, where we calibrate translation decoding for quality from holistic view by directly optimizing the Pearson correlation objective. With limited training, our models consistently outperform the strongest LLM--ased MT systems, such as ALMA [Xu et al., 2023, 2024] and Tower [Rei al., 2024a, Alves et al., 2024] series as well as the GPT-4o model, by substantial margin across multiple automatic and human evaluation metrics. Extensive experiments demonstrate effectiveness over recent preference optimization methods, such CPO [Xu et al., 2024], for translation.

---

Our method, to some extent, unifies quality optimization and quality estimation (QE) in translation by sharing one single objective. Specifically, the uncertainty quantification of our model's output, measured by average log-likelihood, exhibits strong correlation with human judgments, which even outperforms some state-of-the-arr QE models in the community that were explicitly trained with human-annotater data, such as CometKiwi (Rei et al., 2022b]. This finding supports the view that a well-performing model should inherently "know" what constitutes good translation.
Our in-depth analysis demonstrates that improved likelihood-quality calibration-achievell through our calibration wrthod-enhances the effectiveness of maximum a posteriori decoding, offering greater efficiency potential for real-world deployment.
2 Background
At practical level, this study aims to optimize both translation quality as well as quality estimation, simultaneously a single model. To this end, we briefly introduce the relevant foundational concepts.
Translation metric meta-evaluation. To automatically assess translation quality, various metrics have been proposed over the past decades, such as BLEU [Papineni et al., 2002], ChrF [Popović, 2015], COMET [Rei et al., 2020, 2022b]. MetricX [Juraska et al., 2023, 2024]. Metric meta-evaluation assesses how well these metrics correlate with human judgments, where few well-known statisties are applied, such as Pearson's r, Spearman's 's p. and Kendall's т. In this paper, we focus on calibrating hypothesis likelihood with translation quality by directly optimizing Pearson correlation, enabling simultaneous quality optimization and estimation within a single model; see $3. To ensure robust cross-metric evaluation, we report Spearman's 's and Kendall's s scores for quality estimation; see $5.2.
Translation quality optimization. formance using signals from external metric models. We briefly summarize them as follows:
few different themes investigate optimizing translation per-
(1) Test-time quality optimization. To alleviate the gap between decoding objective and translation quality, Minimum Bayes Risk (MBR) decoding [Kumar and Byrne, 2004, Freitag et al., 2022a, Yang al., 2024] suggests basing decoding decisions on statistics gathered from the distribution as a whole, As reference-free quality estimation (QE) progresses, Best-of-N sampling [Rei et al., 2024b, Faria et al., 2024, Brown al., 2024, Ichihara et al., 2025] with a QE model becomes a straightforward strategy. However, both of these methods result in significant amount of additional decoding time. Moreover, the risk of metric bias increases as translation improvements may stem from metric hacking rather than genuine quality enhancements [Skalse et al., 2022, Kovacs et al., 2024].
(2) Preference optimization. Recently, direct preference optimization (DPO) and its variants [Rafailov et al.. 2023, Xu et al., 2024, Meng et al., 2024, Ethayarajh et al., 2024] have emerged as promising approaches for post-training LLMs, providing efficient alternatives reinforcement learning from human feedback (RLHF). This line of methods often employs pairwise preferences under the Bradley- Terry framework [Bradley and Terry, 1952] to provide rewards, or a more general Plackett-Luce ranking framework [Plackett, 1975] when multiple ranked hypotheses are available and few studies [Xu et al., 2024, Zhu et al., 2024] in translation also follow them. Additionally, some works [Guo et al., 2024, Lambert et al., 2024] employ these methods in an on-policy training paradigm for online LLM alignments. Notably, regardless of employing pairwise/liswise data or on-/off-policy training, the objective is generally to maximize expected rewards. A key distinction of our method, as detailed in Scction 3, is that it optimizes correlation rather than expected rewards, forming the basis for unifying quality optimization and estimation. Therefore, we follow prior research [Ott et al., 2018, Zhao et al., 2022] and use the term "calibration" to highlight the difference. More detailed differences are discussed in the next section.
3 Translation Calibration
Formally, given a parameterized regr-segressive language model py and translation instruction æ, the log- bogllikel of a translation hypothesis y; is denoted as =0(ya(a)) (xx) log (B) Mcanwhile, the quality of this translation can defined as q(y; (ar) where g represents any external quality evaluation model. When sampling hypotheses y from po conditioned on both =)((x) and g(ylar) can be viewed as random variables defined over the output space. Our goal is to calibrate the likelihoods of generated hypotheses with their quality to maximize the correlation between 24 (y) ()) and g(vlz).
3

---

Here, we use the statistic, i.e., Pearson correlation coefficient p(a. b), to quantify the correlation. Let a, b y RR be two real-values functions defined over domain V. Their covariance with respect to data distribution p(y) is defined as
ca b) Eyxp [(a(y) y (a())) (b(y) Ey--bb(y)))]
(I)
The corresponding Pearson score between and 6 is given by
cov(a,
E Eypp [(((y) - (a) (b(y) -b0]]
p(a, b) /cov(a, coy
(2)
Ja
where Max H4 and oa. Og denote expectations and standard deviations, respectively. This formulation computes the correlation by normalizing the expected product of their centered values. Due to its scale-invariance and ability to capture trend consistency, the Pearson correlation coefficient is widely used translation metric meta-evaluation.
In this study, we calculate and optimize with respect to the likelihood of hypotheses (v((ar) and the quality score ((ylz)) Practically, given the intractably large decoding space, we employ Monte Carlo sampling for approximation. For each source sentence z, generate k hypotheses by repeatedly prompting large language model 0 with nucleus sampling, and compute the corresponding 20 20(3a (z) and 9(v (z), and estimate the corresponding Pz, My and d2 Tq.
y/ (i E {I...., k}) .......,k)
Notably, the standard definition of correlation assumes expectations under sampling from the full distribution, i.e., unweighted correlation. Here, however, we use nucleus sampling, which introduces a biased estimate by restricting sampling to high-probabillty regions. This approach motivated by two factors: (1) uniform sampling over the output space is infeasible due to its vast size, and (2) following Ott al. [2018], we focus on correlations among likely hypotheses. Accordingly, we define the Pearson-basee loss using estimates under the nucleus-induced distribution p as follows:
20(Ui x) Mh. a(vi xz) - My 14)
Cpearson
k
0g
We additionally introduce supervised fine-tuning (SFT) term the highest-scoring samples as a regularizer to ensure that the model's likelihood distribution remains grounded in high-quality translations, since the Pearson objective alone enforces correlation but does not constrain the absolute scale. The final loss for calibration is formulated as Leal
Lparsson + Lan-
An off off-pylicy formulation can be obtained by trivially replacing the current model pe with an external model pg- for sampling. Overall, by minimizing Leal. we encourage the Pearson score between and q to increase. In practice, we use gradient-based optimizer, Adam, to optimize 0 for this goal, with gradients propagated through 2p, Mzz, and σ.. Despite its simplicity, several important characteristics are captured in this formulation:
It models hypothesis qualities from a holistic view, enabling the model to make finer-grained distinctions in translation quality within the decoding space.
It considers the value of translation quality by the metric function g(-1z), which ignored in virtually all existing methods based on Bradley-Terry and Plackett-Luce, such as CPO.
Pearson's 's correlation inherently applies normalization quality points. This normalization makes the objective invariant to scale and shift, thereby promoting stable and robust optimization across diverse input distributions.
group of both likelihood and
The objective, i.e., the Pearson's 's score itself, is inherently shared with that of translation metric meta-evaluation, offering unified perspective for both quality optimization and estimation. Mcanwhile, unlike other statistics like Spearman's 's or Kendall's scores, Pearson's 's coefficient is differentiable and thus suitable for gradient-basee optimization frameworks.
Overall, the Pearson loss can be reduced to a mere dot product between two sets of normalized points, see Appendix 1. Despite extreme simplicity, we show the next sections that it is highly effective in simultaneously optimizing translation quality and quality estimation within a single model.

---

Experimental Settings
4.1 Base Models, Data, and Evaluation
Base Models. We base our experiments on two strong translation-specinee LLMs, ie.. ALMA-Bas [Xu et al., 2023] and Tower-Base [Alves et al., 2024], in their 7B and 13B variants. Both models achieve remarkable translation performance after post-training. For instance, the Tower series models [Rei fi--hhe-rtt art results across all translation tasks in the WMT24 general translation tracks [Koemi et al., 2024a], surpassing both Google Translate and GPT-4
et al., 2024b] achieved state-of- ofeee-of-hee--eee
Translation evaluation dataset. For fair comparison with the ALMA and Tower model series, we evaluate translation performance the WMT22 and WMT24 datasets [Zerva al., 2022, Koemi et al., 2024a], respectively. Detailed dataset descriptions can be found in Appendix D. Following current. best practice [Freitag al., 2022b, 2023], all results in this paper are measured by widely used neural metrics, involving reference-based metric COMET, and the reference-free metrics XCOMET, CometKiwi-XXL and CometKiwi-XXI'.
Metric mnta-evaluation dataset. To assess how well translation likelihoods correlate with human judgments,
the sentence-level quality estimation (QE) datasets from WMT-22 [Zerva et al., 2022], where sentence pairs are annotated in multiple ways to reflect translation quality, e.g., direct assessments (DA), post-edits (PE), and multidimensional quality metrics (MQM) [Freitag al., 2021]. In this paper, we use MQM scores as ground-truth human annotations against which the metrics' scores are evaluated, MQM scores come from expert translators and are than the crowd-sourced DA scores. A detailed description of the dataset can be found in Appendix D.
reliable
Calibration dataset. For the training set, we merge all English sentences from the Flores-200 dataset [Costa-Jussà et al., 2022] in dev and devtest splits and use them as the source, consisting of 2,009 samples. For both on- and off-policy experiments, we use these sentences to construct translation prompts for each direction. The prompt templates can be found in Appendix E. For the off- policy setting, we query gpt-4o-ma temperature of 1.0 and a top-p of 0.98. The resulting bitexts are evaluated using varying metrics to reflect corresponding quality scores. For on-policy experiments, all settings remain the same, except that a top-k value of 5 is used for each sampling step, and the model itself is queried directly.
16 times per prompt, employing nucleus sampling with
4.2 Training Setups
For all experiments, we train models using LoRA [Hu et al., 2022] with rank 8, setting a to 32 and dropout to 0.05. Training uses batch size of 32. gradient accumulation of 8 steps, and sequences capped at 512 tokens. We train each model for 3 epochs, selecting checkpoints based on the best validation performance measured by XCOMET on NTREX [Federmann et al., 2022], which contains 1,997 samples per direction. To ensure robust results, we experiment with learning rates ranging from le-5 to Ie-4, reporting the best results for all settings. Adam [Kingma and Ba, 2014] is used as the optimizer. Unless otherwise specified (e.g.. $ 611,, we use CometKiwi-XX as signal during training and report results in XCOMET, COMET, and CometKiwi-XL. All experiments use H100 GPUs, with 7B models trained on one GPU and 13B models trained on two GPUs.
5 Results
5.1
Calibration Leads to Clear Quality Improvements
In this section, we demonstrate the effectiveness of our calibration approach by applying it to some stae-off-the-an translation systems, e.g., Tower and ALMA. Table 1 presents the results for the Tower series under an off- off-policy setting (see $4.2 for the training data), measured by CometKiwi- XI, the official reference- metric for WMT24, and XCOMET, the best performing metric as evaluated by Freitag et al. [2023]. Except for closed-source models, all results are decoded by beam search with beam size of 5. Nooeerlntrrnet7BB/11333 and Inserrrntttt--ssttaa implementations [Rei et al., 2024a], supervised fine-tuned (SFT) on the corresponding base models using TowerBlock, set of high-quality translation instructions constructed through careful data selection and filtering from human-annotated translation datasets, consisting of 637k instructions.
are official
'The corresponding metric model versions are 2ttpbbeellmmt22-comeee Unbabel/vmt23 http:///mmmmmmmmeeeiiieeemmm cometkiwi da-xl, and Unbabel/vmt223 www.tkwwi-eeeiio. xx1,, respectively.
Rababel/XCOMETTBBE
5

---

Table I: Evaluation of en en-xxx translation on WMT24 using CometKiwi-XL and XCOMET. Results are reported for all languages covered during Tower pretraining. Note that the Tower-v2 models, including Tower-70B-v2, have not been publicly released. We report their best results as published by Rei et al. [2024a]. For 4PT-4o and GPT-4o-mini, we use the prompts following Hendy et al. [2023]. Results in other metrics can be found in Appendix F. Notably, according to Koemi et al. [2024b], improvements of 2 1.99 in XCOMET or 2 0.94 in COMET scores correspond to at least 90% estimated accuracy in human judgment- both of which are achieved by our method. The best results across each model variant are bolded.
e0 ide
40 140
C6 126
C6 12h
CL -Ix
Models
KIWI-XL
XCOMET
KIWI-XL
XCOMET
KIWI-XL
XCOMET
KIWT-XL
XCOMET
KIWI-XL
XCOMET
Olocdd
GPT do mini CPT-4u
68.6
68.3
92.6
91.7
70.6
70.2
$7.7
87.01
69.1
68.1
85.4
$1.6
69.9
69.0
81.3
79.7
65.0
65.6
83.9
83,0
Tower 70B x2
Tomex-70B--2-- MBRR/RR
72.3
74.5
74.2
72,6
69.0
91.7
70.8
69.0
81.5
68.5
76.7
65.0
84.1
SPT BoN data
7U0
92.0
7U.
36.5
09.6
$1.6
05.4
71.9
05.0
15.7
CCO
71.1
03.1
72.0
$7.6
71.6
85.8
70.4
B0LD
69.3
85.8
Callteaton (ours)
71A
93.
73.5
72.8
86.8
78.8
N1.0
70.0
M6.8
Toweriastract 13B
92.5
71.
$7.7
70.6
55.5
70.1
90.5
65.1
85.1
TrwerBBau--13 data
70L1
.CP
70.5
92.2
72.0
$7.7
$4.0
65.6
85.5
Caliitation (ours)
72.5
94.2
758
9000
75.6
86.8
72,1
85.6
7888
87.5
Towerlastract Mistral 7B + SFT en BuN datu
7000
92.6
71.9
87.5
70.5
85.3
69.5
80.4
65.3
94.7
70.7
02.7
71.8
$7.1
70.8
82.9
70.5
80.4
65.5
84.4
712
93.0
73.1
$9.0
72.3
85.1
71.8
83.6
7000
86.9
Calibcatin (ours)
72.4
73M
73.6
86.1
72.6
85.7
78.6
N7.4
191
196
-No
Madels
KIWI-XI
XCUMPF
KTW1-XI
XCCIMI
KIW1-I
XCOMIT
KIWS-XI
XOOMITT
KIWSXI
XCOMET
GPP1-4
69.3
88.9
16.1
83.7
71.2
87.6
75.2
84.3
09.1
85.1
GPT-4u
70.6
90.5
68.7
85.7
71.5
88.5
73.7
85.6
69.6
86.5
12 MBR/TRR
Toweriastrnet- 7B
71.5
90M
71.1
36.1
71.1
86.8
75.6
82.8
70.3
85.5
TowerBase-7B
SPY on BoN dats
71.5
39.6
8
85.6
72.5
87.6
75.7
1
20.8
85.4
719
72.2
$6.7
$5.7
76.1
87.2
(mum)
Towerisstrat-11 138
71.7
91.0
71.1
87.5
72.1
56.2
75.4
34.8
73.2
W6.7
TowerBase- 13B SFT en BuN data
71.7
90.4
71.6
$6.1
73.0
$5.1
76.2
85.2
71.6
85.4
72.5
508
72.5
87.
93.3
86.9
76.9
81.9
71.9
87.1
Culibcaton (ouss)
739
73.9
89.3
75.2
90.4
75.0
89.5
735
89.3
Towerlastuct Mistral--7
71.9
91.1
71.6
87.2
72.1
85.0
74.2
85.6
71.1
85.7
SPT BoN cata
22.5
50.7
71.6
86.3
12.7
879
76.2
86.0
71.7
85.5
CPO
23.3
92.3
73.1
88.5
74.0
89.7
27.4
89.3
72.9
85.5
.Clibeatio (18m))
74.2
93.2
76.1
89.6
95.1
90.7
7K.1
89.7
23.9
W9.4
We also conducted SFT on the TowerBase series using 2K Best-of-N samples per direction, selected from our calibration dataset ($4.2) based on the highest CometKiwi-XXL. scores. The resulting performance is comparable to the official instruction models. When fine-tuning on the full calibration set, performance is expected to degrade, as some sampled bitext examples are of low quality.
When applying our calibration approach, very strong improvements can be observed across all directions, metrics, and base models. First, it leads an average improvement of +2.8 points in Additionally, Table 4 shows
KIWI-XL and +2.7 points in XCOMET over Meeerrnsttutt-iistaalll gains of +3.6 points in KIWI-XXL and +1.2 points in COMET, respectively. Second, this performance is comparable to that of the current top-performing system, that is Tower-70B-V equipped with 100-time-san sampliigg MBR/TRR2, while being approximately 200 times faster?.
We also compare with CPO [Xu et al., 2024], a widely- preference optimization method for translation, following its original setting by selecting the highest- and lowest- scoring candidates as accepted and rejected samples, respectively, achieving consistent and substantial improvements over CPO. Additionally, Appendix F.2 reports results based on the ALMA base model, showing gains of +2.0 XCOMET and +1.4 KIWI-XXL over CPO for ouf-of-English translation.
Note that experimental results for on- on-policy settings are also provided in Appendix F.3, where sub- stantial improvements can also be consistently observed across metrics, demonstrating effectiveness under different training dynamics. Unless otherwise specified, the following analysis focuses on the off- off-policy setting to simplify the investigation of the properties of our calibration method.
'TRR [Rei al., 2024a] denotes an ensemble strategy that applies reranking based on multiple metric model select the best candidate from multiple sampled hypotheses. They report TRR results when surpasses MBR. 'We roughly estimate the latency of the Tower-70B-v model to be 10 times that of the Tower-Mistral Mi model. Meanwhile, the former employs 100x sampling, while the latter uses beam search with beam size of 5.

---

73.0
Tower-inst-78
Tower-Base-78 SFT
72.5
Tower-Base-78 + Callbration
Tower-708Bve MBR/TRR
Tower-inst-13
72.0
Tower Base-138 + SFT
Tower-Base-1388 + Calibration
CometKiwi- XL
71.5
71.0
CometKiwi
70.5
70.0
69.0
49.0
50.0
51.0
52.0
53.0
54.0
55.0
56.0
57.0
58.0
59.0
Spearman Score
Figure The Spearman coefficient and the corresponding translation performance in en en->de direction under different settings for the Tower series models. The color gradients of v and e, from lighter to darker shades, indicate the results of fine-tuning with varying amounts of Best-of-N data, from 400 to 2000 samples. * denotes the application of our calibration method, which simultaneously surpasses both the state-of-the-art translation system and the widely used quality estimation model. Results for other languages and statistics can be found in Appendix G.
5.2 Calibration Leads Direct Quality Estimation
As detailed in $3, we shared the objective for translation quality optimization and estimation, although supervisions are from machine annotations instead of human annotations. If optimized effectively, the resulting model should inherently acquire the ability to assess translation quality using hypothesis log-likelihood as a metric. This section evaluates how effectively calibration can elicit this capability,
We use the WMT22 metric meta-evaluation dataset [eerva et al., 2022] and follow the official practice, see $ 4.2, to assess quality estimation ability using Spearman's is and Kendall's correlation. We evaluate all training directions Tower that overlap with the WMT dataset, namely, en en-de and en-ru.
Figure depicts the Spearman score (metric performance) and the corresponding translation perfor- mance under different settings for Tower-7B and Tower- 13B, including: (1) supervised fine-tuning using varying amounts of best-of-N samples (400/800/1200) 20000/1110111100 1000//00002000 samples per direction), (2) scaling the base model size from 7B to 13B, and (3) applying our calibration method. It shows that:
(1) As more Best-of-N samples are included in SFT, translation performance progressively improves. Interestingly, the quality estimation ability (Spearman scores) increases from around 51.5 to 54.0 points. We attribute this the fact that the model assigns higher likelihoods to better hypotheses. However, these improvements are limited and not general across languages, see Appendix G.
(2) Examining the effects of scaling, observe that: (i) scaling up from 7B to 13B generally improves translation performance for both the original TowerInstruct models and the fine-tuned models; (ii) however, its impact calibration, i.e., quality estimation ability, remains minimal.
(3) Our calibration method manifests very strong improvements in both translation and quality estimation. For example, when applying our method to TowerBase- 13B, the resulting model surpasses some otate-of-the-art systems both translation performance and quality estimation ability, i.e., Tower-70B-v2+MBRTIII and CometKiwi, at the same time.
Results for other statistics are provided in Appendix G. Overall, we observe a clear, albeit sometimes non-linear, correlation between the models' translation performance and their quality estimation ability. These results suggest-to some extent-a unified perspective: well-performing translation system should inherently "know" what constitutes good translation. In turn, we also suggest optimizing translation quality by improving calibration on LLMs, rather than relying solely on extreme scaling or supervised fine-tuning, as the latter approaches show relatively limited effectiveness.
7

---

5.3 Calibration Enhances Effectiveness for Maximum a Posteriori Decoding
Compared test-time optimization, such as Best-of-N (BoN) sampling, our approach performs sampling and directly calibrates the likelihoods of hypotheses with their translation qualities during training. Ideally, for well-calibrated model, the potential of maximum a posteriori (MAP) decoding, such as beam search, should be realized more effectively.
Figure 3 compares the effectiveness of beam search (with beam size of 5) to that of BoN sampling with varying sampling sizes, for both the baseline (Tower cesssruct-Mistraal 7B) and our calibrated models. To ensure fairness, we adopt a cross-metric evaluation for BoN sampling: Candidates are reranked using KIWI- XXI., while the best-scoring results are evalu- ated using COMET, Two key findings are clearly illustrated: (1) As the BoN sampling size in- creases, translation performance improves, with model consistently outperforming the base- line across all settings. (2) Under the MAP de- coding strategy, i.e., beam search, the baseline model (dashed blue line) underperforms com- pared BoN sampling with sampling size small as 10. However, after calibration, beam search achieves performance (dashed red line) comparable to that of BoN sampling with a size of 100, showing much stronger realizations of MAP capabilities. Note that beam search here is dozens of times more efficient than BoN sampling.
84.5
COMET Score
83.5
83.0
82.5
ln Brstion 3009 Bration (kaar)))
82.0
30
65
60
100
Best Sampling Size
Figure
The average performance ,,,, ru, zh) measured by COMET score for both TonerInstruct-Mstrrae calibrated model, when varying sampling size for Best-of-N (BoN) sampling and applying beam search with beam size of 5.
(en-tde es,
and
Overall, compared to the state-of-the-art baseline translation system, calibration method yields clear performance improvements and offers greater efficiency potential for real-world deployment. Unlike computation intensive inntensiiionnneensivee test-time optimization methods like Best-of-N sampling, it inherently improves effectiveness by calibrating decoding to the real-world quality objective, unlocking the potential of efficient MAP decoding rules, such beam search.
6 Analysis
6.1
Cross-Metric Evaluation
Table 2: Average score differences on WMT24 over those of Tooerrnstrruct-isstaal across metrics, when applying (a) supervised fine-tuning and (b) our calibration method under different settings.
(a) SFT Over Ceerrlastract-iistaal.ht
Calibration Over TowerInstruct-Nisttall
Objective
KXWIXXL
XCOMET
COMET
Objective
XXW
XCOMET
COMET
XXII
+0.3
0.2
-0.3
KIWI-XXL
+3.5
+2.7
+1.2
XCOMET
-0.3
+0.1
-0.3
XCOMET
+2.4
+2.8
+1.0
COMET
.05
.0.
-0.4
COMET
+2.1
+1.4
+0.8
potential concern with directly optimizing towards translation quality metric gaming [Casper et al., 2023, Kovacs et al., 2024]. To address this, we conduct experiments with TowerInstruct- Mistral-7B, using various metrics to assess hypothesis quality during training and to examine whether improvements consistent across metrics. To maximize the contrast between metrics, we focus on three representative ones our setting: (1) COMET, a strong widely used reference-
metrie; (2) KIWI-XXL, the strongest reference-free metric in the KIWI series; and (3) the reference-free free version of XCOMET which combines sentence- level evaluation with error span detection.
Table 2 presents the average improvements for WMT24 when applying (a) supervised fine-tuning and (b) our calibration method, using these three different metrics. It shows that: (1) When fine-tuning on our Best-of-N dataset, we generally observe slight performance degradation for all settings, except when KIWI-XXI. and XCOMET are used simultaneously as both the training objective and the evaluation metric. We attribute the general degradation to the fact that weteerrnnrtut--Mistalllla

---

is already strong, and the improvements observed in KIWI-XXL and XCOMET to the existence of slight metric gaming. (2) When applying our calibration methods, we observe consistently
strong improvements across all training objectives and evaluation metries, highlighting cross-metric effectiveness. (3) The reference-based metric, COMET, demonstrates relatively low effectiveness during training in both settings. We attribute this to the fact that reference-basee metrics constrain the feasible space of positive hypotheses to be similar to the reference, thereby weakening the diversity of hypotheses recognized as valid high-quality translations.
Tewerinstruc-Missran
Callbration-78
150
Number of Samples
Human Evaluation Summary
100
TScore
CScore
Win
Loss
Tie
4.77
5.49
106
32
62
Annotation Score
Figure 4: Human evaluation results in en en-zh direction: score distribution and an aggregated summary. TScore and CScore denote the average scores over 200 annotations for TowerInstruct- Mistral-7B and our calibrated model, respectively. Win, Loss, and Tie indicate the number of samples for which our model outperformed, underperformed, or tied with Sorerlnsttttt-iistraae
6.2 Human Study
The preceding results demonstrate the effectiveness of method across different settings and metrics. Here, we incorporate human evaluation as direct evidence for comprehensive improvements.
We randomly sample 200 translation outputs in the directions of en en-tz/ruu for both the baseline model TTowerlnstruct-Misttaal
and our calibrated model from the WMT24 dataset. Several bilingual speakers, native in the respective target languages, were recruited to annotate each translation on a 0-6 scale, following the criteria outlined in Koemi al. [2022], see Appendix H.1.
Figure 4 shows the results for en en- annotations, covering both fine-grained score distributions and an aggregated summary. shows that our calibrated model yields significantly more near perfepe translations (137 vs. 67 for scores of 6), with the main gains over the baseline observed in improved handling of minor-to-major errors (denoted as score of 3/4/5) in grammar, wording, or tone, see Appendix H.2. Overall, the average score for the calibrated model is 5.49 out of 6, showing very strong system effectivvness. Compared the otate-of-the-ee baseline, Soverlnstruct-Misttaa--- overwhelming win rate (106 vs. 32) can be observed. Summaries of the annotation results for other directions are in Appendix H.3. We also release all detailed human annotations to the community"
an
6.3 Sensitivity to Sampling Size
As $3 mentions, we approximately characterize the decoding space through sampling. Therefore, increasing the sampling size for each source sentence is expected to improve effectiveness. To validate this, we conduct experiments on SowerInstruct-Mistrlll training. Also, compared our approach with CPO and selected the highest- and lowest-scoring hypotheses as the accepted and rejected hypotheses, respectively.
varying the sampling size during
Figure 5 provides the results measured by both reference-free and reference-basee metrics, i.e., CometKiwi-XI. and COMET, different settings when varying the sampling size for each source sentence. It elearly shows that: (1) SFT with the best-scoring hypothesis has limited impact on translation performances; (2) CPO yields clear improvements over supervised fine-tuning, but these improvements platcau as the sample size exceeds 8; (3) our approach manifests substantial improve- ments over both CPO and SFT. Moreover, as the sampling size increases, continued improvements can be observed, demonstrating greater potential for further advancements. Also, the results support the underlying hypothesis that more accurate modeling of the decoding space facilitates better calibration, which in turn yields greater performance improvements.
'Detailed human annotations are available at https http://ggggaa.....//aattst////// nsla attitnoooo MalaaatoonCaaaaai /aal Micaabbree bration. ion- translation. human- eval.
co/ wttpe//tttttttt/aaaaatteaaaa cattraeeettte/aibbiiiooooo -ra

---

13.5
COME
81.2
828
Sampling
(a) KIWI-XI. scores varying sampling size
(b) COMET scores varying sampling size
Figure 5: Average performance variation on the WMT24 dataset for Swwerrnntruct-Mistta.... across
different sampling sizes during training, measured by (a) CometKiwi-XL and (b) COMET.
6.4 Learning from Human Feedback
We also examine the applicability of our calibration method to aanman-n data. Specifically, use the MAPLE dataset [Zhu et al., 2024], which in- cludes four translation directions, as the calibration set. Each source sen- tence is paired with five candidate translations rated on 1-6 scale by human experts. We compare method against (1) SFT using the top- rated data and (2) the PL-based pref- erence optimization approach [Zhu al., 2024], which designed for hu- man preference data. Table shows
Table 3: Performance comparison across SFT, PL, and cali-
bration method on WMT22 dataset, measured by COMET.
Model
Mistral-Ins. SFT
81.8
83.0
83.3
78.3
81.6
Mistral-Ins. PL
82,9
83.4
84.7
79.3
82.6
Mistral-Ins. + Calibration
83.5
84.4
85.4
80.1
83.4
TowerMistral + SFT
88.9
85.7
89.3
83.3
86.8
TowerMistral Calibration
89.9
86.6
90.1
84,7
87.8
results for the 7B Mistral-Instruct and TowerMistral Base models. Notably, our calibration method consistently improves over both SFT and PL baselines. Although TowerMistral -Bse with SFT already substantially outperforms Mistral-Instruck our calibration approach can further improves its performance across all directions, showing its applicability to human-annotated data.
7
Conclusions
This paper addresses the well-known miscalibration problem in machine translation by introducing simple yet effective training-time time method that directly optimizes the Pearson correlation objective to improve saelihood-quality calibration. Extensive experiments demonstrate several key advantages:
Very clear and consistent performance improvements. Our calibration method yields substantial improvements across wide range of automatic metrics and human evaluations. For example, a calibrated Tower-7B model with beam search achieves state-of-the-an translation performance comparable to to-if not exceeding that of the much larger Tower-70B-v2 model equipped with test-time optimizations technology such as MBR decoding, all while maintaining high efficiency.
A unified framework for quality optimization and estimation. By using shared objective for both tasks, our method offers a unified perspective on quality optimization and quality estimation (QE) in translation. Our empirical analysis shows strong correlation between calibration and translation quality. Experimentally, our calibrated method achieves both ooppprfforming translation performance and accurate quality estimation within a single model, with the latter even surpassing widely used QE models such as CometKiwi all without relying on human-amme annotated data.
Enhanced realization for maximum posterior decoding. Experimentally, we show that calibration clearly improves the effectiveness of efficient MAP decoding methods like beam search. Originally seen as less effective than Best-of-N sampling, beam search-whie properly calibrated-matchhe the performance of Best-of- sampling with a beam size of just 5, significantly cutting inference cost without sacrificing quality and showing strong promise for real-world use.
10

---

Acknowledgements
This work was funded in part by the Netherlands Organization for Scientific Research (NWO) under project number VI.C. We thank our colleagues at the University of Amsterdam, especially Sergey Troshin, Evgeniia Tokarchuk and Maya Nachesa, for their insightful discussion. D.W. thanks Chongyang for its invaluable spiritual support. The authors thank the anonymous reviewers for their constructive efforts to improve this research.
References
Duarte M Alves, José Pombal, Numo M Guerreiro, Pedro Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, al. Tower: An open multilingual large language model for transtation-related tasks. arXiv preprint arXiv:2402. 17733, 2024.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4)3324-35) 1952.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Sealing inference compute with repeated sampling. arXiv preprint arXiv:2407. 1787, 2024.
Stephen Casper, Xander Davies, Claudia Shi. Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307. 152 2023.
Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hefferaan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centered machine translation. arXiv preprint 2tt.:2227704772 2022.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024.
Gonçalo Faria. Sweta Agrawal, António Farinhas, Ricardo Rei, José de Souza, and André Martins. Quest: Quality- aware metropolis caatoiiis-httissssso sampling for machine translation. Advances in Neural Information Processing Systems, 17:89042-89066 2024.
Christian Federmann, Tom Koemi, and Ying Xin. NTREX-122 128 news test references for MT evaluation of 128 languages. Kabir Ahuja, Antonios Anastasopoulos, Barun Patra, Graham Neubig, Monojit Choudhury, Sandipan Dandapat, Sunayana Sitaram, and Vishrav Chaudhary, editors, Proceedings of the First Workshop on Scaling Up Multilingual Evaluation, pages 21-24, Online, November 2022. Association for Computational Linguistics. doi: 10. 10.11853/ eval-1.4 URL https::pp/. / /ar//acllanttt acw.acoollltoy.oc..c .rw.orgggooo.o.....e 2022. sumeval-1.4/.
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, and Andre Martins. Quality-aware decoding for neural machine translation. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1396- 1396-1412, Seattle, United States, July 2022. Association for Computational Linguistics. doi: (0.18653/v1/200 1/w.22..2aaaaaa mac.ll-acl--ooo attps://aclanthin
main. www.aaaalanannle1
URL
org 220g/202..coo maacl 100/.
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8:539-555, 2020. doi: 10. 16.1/6// 00330. URL https: http:////caaaahoooyy.y.o org /020
Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9: 9:1460-1474, 2021. doi: 10. 1162/ tacl 00437. URL https://aat http::://aclannhhllgg. org/ org/2021.20 tac1-1.87/.
11

---

Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. High quality rather than high model probability: Minimum Bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics, 10:811-825 1-825, 2022a. doi: 10.1162/tac_aa
URL
attpss//acllamthology http:/hoowog.....22.22.... tac1-1147/.1
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André F. T. Martins. Results of WMT22 metrics shared task: Stop using BLEU - neural metrics are better and more robust. In Philipp Koehn, Loic Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Kocmi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, and Marcos Zampieri, editors, Proceedings of the Seventh Conference on Machine Translation (WMT), pages 46-68, Abu Dhabi, United Arab Emirates (Hybrid), December 2022b. Association for Computational Linguistics. URL https://aaaaanthlllggoooo
2022/2222.c2 wmt 1.2/.
Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, and George Foster. Results of WMT23 metrics shared task: Metrics might be guilty but references are not innocent. In Philipp Koehn, Barry Haddow, Tom Koemi, and Christof Monz, editors, Proceedings of the Eighth Conference Machine Translation, pages 378-628, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 11.8666//1//223../wv 1.51. URL httpst http:::::///aaaaaloolooooo/o anthology org/2023, wmt mmtt-151/ 1.51/.
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, et al. Direct language model alignment from online ai feedback. arXiv preprint arXiv:2402.04799 2024.
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Mat- sushita. Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. How good are gpt models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.092 09210, 2023.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Yuki Ichihara, Yuu Jinnai, Tetsuro Morimura, Kaito Ariu, Kenshi Abe, Mitsuki Sakamoto, and Eiji Uchibe. Evaluation of best-of-n sampling strategies for language model alignment. arXiv preprint arXiv:2502. 1 /2668, 2025.
Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, and Markus Freitag. MetricX-23: The Google submission the WMT 2023 metries shared task. In Philipp Koehn, Barry Haddow, Tom Koemi, and Christof Monz, editors, Proceedings of the Eighth Conference on Machine Translation, pages 756-767, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/1/20023.vo wwt- 1.63. URL https: /aacll//clllec aal aaclanhoooogy... rg/2023. 1m6/1.
Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. Metricx-24: The google submission to the wmt 2024 metrics shared task. arXiv preprini urXi::2410.03983, 2024,
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412:91412.6990, 2014.
Tom Kocmi, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz. Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel. and Maja Popović. Findings of the 2022 conference on machine translation (WMT22). In Philipp Koehn, Loic Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussă, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Koemi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi,
12

---

and Marcos Zampieri, editors, Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1-45, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL. https: httpa////allaallaaaaaaee org/ 2022.
Tom Koemi, Eleftherios Avramidis, Rachel Bawden, Ondfej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popović, Mariya Shmatova, Steinthór Steingrímsson, and Vilém Zouhar. Findings of the WMT24 general machine translation shared task: The LLM era is here but MT is not solved yet. In Barry Haddow, Tom Koemi, Philipp Kochn, and Christof Monz, editors, Proceedings of the Ninth Conference on Machine Translation, pages 1-46, Miami. Florida, USA, November 2024a. Association for Computational Linguistics. doi: 10.18653/v1/2024 wmt- 1.1. 1/. 1.1/.
URL https://aclaneeee http://ccaaantohogoo/oo org org/2024 2024. wmt
Tom Kocmi, Vilém Zouhar, Christian Federmann, and Matt Post. Navigating the metrics maze: Reconciling score magnitudes and accuracies. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics Bangkok, Thailand, August 2024b. Association for
(Volume I: Long Papers), pages Computational Linguistics. doi: 1/0653//1/2224.aal-oa .org/2024 aac--long 110/.
10. URL https: / //c//acaaatttt acclanthology
Philipp Koehn and Rebecca Knowles. Six challenges for neural machine translation. In Thang Luong, Alexandra Birch, Graham Neubig, and Andrew Finch, editors, Proceedings of the First Workshop on Neural Machine Translation, pages 28-39, Vancouver, August 2017. Association for Computational Linguisties. doi: 10. 10.18653/v 1/tp:////111133000 URL https://acllantthoooog wtt.:/tooooooooooo///----01
Geza Kovacs, Daniel Deutsch, and Markus Freitag. Mitigating metric bias in minimum Bayes risk decoding. In Barry Haddow. Tom Koemi, Philipp Koehn, and Christof Monz, editors, Proceedings of the Ninth Conference on Machine Translation, pages 1063-1094, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 10.18653/w1/2224.mwm .109. URL https: http//aaaaaatttgg..aoooooo
wtt./ttsss//cccaaaaaolgoogooo hwwphhhooogy.ogg//2200 2r2/2024. wmt 1. 109/.
Aviral Kumar and Sunita Sarawagi. Calibration of encoder decoder models for neural machine translation. arXiv preprint 1oXiv:9003.00 000000 2019.
Shankar Kumar and William Byme. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguisties: HI.T-NACC. 2004, pages 169-176, Boston, Massachusetts, USA, May 2 - May 7 2004. Association for Computational Linguistics. URL org/ 1022/1
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, al. TV," ulu 3: Pushing frontiers in open language model post-training arXiv preprint arXiv:2411.551 2024.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference- free reward. Advances in Neural Information Processing Systems, 37:124198-124233 2024.
Kenton Murray and David Chiang. Correcting length bias in neural machine translation. In Ondfej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor, editors, Proceedings of the Third Conference on Machine Translation: Research Papers, pages 212-223, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 18.13/3//v1/ W18-6322 118-6322. URL
ht/p///caannhooogyyyyoooo 2Pgg/18-6322/....
Myle Ott, Michael Auli, David Grangier, and Marc Aurelio Ranzato. Analyzing uncertainty in neural machine translation. In International Conference on Machine Learning pages 3956-3965. PMI.R, 2018.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin, editors,
13

---

Proceedings of the 40th Annual Meeting of the Association for Computational Linguisties, pages 311-318, Philadelphia. Pennsylvania, USA. July 2002. Association for Computational Linguistics. doi: 10.31 1,.111//0773883100333111 htt /tattpss///cllnnhoooogyy wttph/oooooooooo/22-111111 .rg/P02-1040//
URI.https
Robin L Plackett. The analysis of permutations. Journal of the Royal Statistical Society Series C: Applied Statistics, 24(2): 1975.
Maja Popović. chrF: character n-gram F-score for automatic MT evaluation. In Ondfej Bojar, Rajan Chatterjee, Christian Federmann, Barry Haddow, Chris Hokamp, Matthias Huck, Varvara Logacheva, and Pavel Pecina, editors, Proceedings of the Tenth Workshop on Statistical Machine Trunslation, pages 392-395, Lisbon, Portugal, September 2015. Association for Computational Linguisties. doi: 10.11865//1//1155- httpt
URL https: /tap:p:///caaaaaa hataaaaaaaattoooggygooooo org//15- 3049/.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-53777 2023.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: A neural framework for MT evaluation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 2685 27022 Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v 1/2020.emnlp- 213. URL https: /////llaaata aal.aaaaaanoooooyyoooooooooooool org/2020. emnlp- main.. 213/.
Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and André F. T. Martins. CometKiwi: IST-unbabel 2022 submission for the quality estimation shared task. In Philipp Kochn, Loic Barrault, Ondicj Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Koemi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, and Marcos Zampieri, editors, Proceedings of the Seventh Conference on Machine Translation ( (WMT), pages 634-645, Abu Dhabi, United Arab Emirates (Hybrid), December 2022a. Association for Computational Linguisties. URL https: / h/tp///clllaaa acl anthology org 2022. wmt- 1. 1.60/.
Ricardo Rei, Marcos Treviso, Nuno M Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José GC De Souza, Taisiya Glushkova, Duarte M Alves, Alon Lavie, et al. Cometkiwi: lst-unbabel 2022 submission for the quality estimation shared task. arXiv preprint arXiv:2209.06.: 06243, 2022b.
Ricardo Rei, Jose Pombal, Nuno M. Guerreiro, João Alves, Pedro Henrique Martins, Patrick Fer- nandes, Helena Wu, Tania Vaz, Duarte Alves, Amin Farajian, Sweta Agrawal, Antonio Farinhas, José G. C. De Souza, and André Martins. Tower v2: Unbabel-IST 2024 submission for the general MT shared task. In Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz, editors, Pro- ceedings of the Ninth Conference Machine Translation, pages 185-204, Miami, Florida, USA, November 2024a. Association for Computational Linguistics. doi: 10. 18853/v1/20222 1/224w///00. 1.1 URL. https:tp/........m //t: htt///aaaaaattooooeoe/ org/2024/ wmt 1mmt-114 1.12/
Ricardo Rei. José Pombal, Nuno M Guerreiro, João Alves, Pedro Henrique Martins, Patrick Fernandes, Helena Wu, Tania Vaz, Duarte Alves, Amin Farajian, et al. Tower v2: Unbabel-ist 2024 submission for the general mt shared task. In Proceedings of the Ninth Conference Machine Translation, pages 185-204, 2024b.
Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460-9477 2022.
Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. On the inference calibration of neural machine translation. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3070-3079, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/v1/2200 m78. URL https:/ /tappp:://caaaaaaaaaaa aallaaahlllooy...... org/2020 aacl-maii mmain.278/.
14

---

Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in ma- chine translation: Boosting translation performance of large language models. arXiv preprint arXiv:2309. / 2023.
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries of Ilm 2024.
performance in machine translation. arXiv preprint
Guangyu Yang. Jinghong Chen, Weizhe Lin, and Bill Byme. Direct preference optimization for neural machine translation with minimum Bayes risk decoding. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference the North American Chupter of the Association for Computational Linguistics: Human Language Technologies ( (Volume 2: Short Papers), pages 391-398, Mexico City, Mexico, June 2024, Association for Computational Linguistics. doi: 10.18653/w1/2224 wwwp2222..aaallliiii. maacl-short.34, URL https /at///aalaaaalooo/a//eee 0024 naacl-short.
Chrysoula Zerva, Frédéric Blain, Ricardo Rei, Piyawat Lertvitayakunjorn, José G. Souza, Steffen Eger, Diptesh Kanojia, Duarte Alves, Constantin Oräsan, Marina Fomicheva, André F. T. Martins, and Lucia Specia. Findings of the WMT 2022 shared task on quality estimation. In Philipp Koehn, Loic Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa- jusså, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Koemi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, and Marcos Zampieri, editors, Proceedings of the Seventh Conference on Machine Translation (WMT), pages 69-99, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL htw.h:tt////. http://ccaaaattooooo /ataa///sttttttttt/////aaaattooo
.rg. wtwgg2222..co wmt-1.3/.
Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. Calibrating sequence likelihood improves conditional language generation. arXiv preprint arXiv:2210.00045, 2022.
Dawei Zhu, Sony Trenous, Xiaoyu Shen, Dietrich Klakow, Bill Byrne, and Eva Hasler. A preference-driven paradigm for enhanced translation with large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pupers), pages 3385-3403, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10. 10.18653/v 1/2024. naacl-long.186 URL https: http:://aacaattho/e http//lcoantooooo....ooo org 2024 maacl-long 186/.
15

---

A Limitations
In this paper, we examined the impact of our calibration method under an approximate maximum posteriori (MAP) decoding (i.e., beam search) across various settings. However, we did not explore decoding with the exact highest-probability output, which is computationally intractable due the exponential search space. We also did not investigate substantially larger beam sizes (e.g., 200), which are technically feasible but incur prohibitive computational costs. We leave these directions for future work.
B Broader Impacts
Due to resource constraints, we focus on calibrating open-source pretrained LLMs, whose language coverage is limited to that of the base models. The impact on languages beyond this scope is left for future work. Moreover, our method inherits the limitations of the underlying models. particular, translation quality may be consistent across languages demographic groups, potentially raising fairness concerns. This includes the risk of amplifying societal biases, such as gender or racial bias, that may be present in the training data.
C Supplementary Details on the Methodology
C.1 Pearson Correlation Coeflicient
The Pearson correlation coefficient between two sets of values x, y E R2 defined as:
[x= x (1) 11
Pearson(x,
(z, 2)² V. (ys 7)
where ह and ÿ denote the means x and y. respectively. Let us define the mean-centered and fz-normalized versions of and y
y
(5)
X
ly 7l
Then, the Pearson correlation simplifies to the dot product:
Pearson(x, = x T
(6)
This shows that the Pearson loss can be computed using only mean-centering, normalization, and single dot product-operation that are both differentiable and computationally efficient.
D Detailed Dataset Description
WMT22 and WMT24 translation datasets. In this paper, we use the WMT22 and WMT24 datasets in the general translation track. WMT24 covers 11 language directions. In this paper, focus on the 9 out-of-English translation directions (en- es ru zh fr mnl lit pt ko) and use them conduct experiments on Tower series models. All testsets in WMT24 are paragraph-level and share the same English parts, consisting of 960 samples for each direction. WMT22 consists of 22 language directions, covering out-, into-, and non-English translations. In this paper, for fair comparison, we focus on 10 directions that are covered by ALMA series models, i.e., both out- cs /zh/ru) translations. Each
of of-Fnglish (en->del cs is zzh ru) and Into-nnllisheens direction contains 2037 sentence pairs.
WMT22 quality estimation dataset.
To evaluate the effectiveness of using our model's likelihood as a quality estimation metric, we use the WMT 2022 Quality Estimation (QE) dataset, which provides source sentences, machine-translated outputs, and corresponding human annotations at both the sentence and word levels. The dataset includes direct assessment scores (DA), post-editing effort indicators such as HTER, word-level quality tags (OK/BAD), and multidimensional quality annotation (MQM). In this paper, use MQM scores the ground-trute truth human annotations against
16

---

which the metrics' scores are evaluated, as MQM scores come from expert translators and are more reliable than the crowdsoureed DA scores. WMT22 QE dataset covers three language directions, ie.. en en-ade, en en-ruu, zh zn. We focus on the first two they overlap with those during the Tower base model pretraining, amounting to approximately ,,000 segments per language pair. More detailed dataset descriptions can be found in Zerva et al. [2022].
E Prompt Templates
The prompts used in our experiments are presented in Sections E. E.2, and E.3. To ensure fair comparison, the prompts strictly follow the default settings the original models [Xu et al., 2023, Rei et al., 2024a] and prior work [Xu et al., 2024].
E.1
GPT Prompt
System:
You helpful translator and only output the result.
User: ### Translate this sentence from <source language> to <target language>, <source language>:
<source sentence>
### <target languages:
E,2
Tower Prompt
Translate the following text from <source language> into <target language>.
<source language>: <source sentence>
<target language>:
E.3
ALMA Prompt
Translate this from <source language> into <target language>: <source language>: <source sentence>
<target language>:
More Results for Section 5.1
In this appendix, we provide additional results from the experiments presented in the main text, including (1) a broader range of translation evaluation metrics in Section F.1, (2) the experimental results based on the ALMA model in Section F.2, and (3) the experimental results under on-policy training dynamics in Section F.3.
F.1 Off-Policy Results based on Tower in KIWI-XXL and COMET
Table 4 shows off-policy results measured by two other metrics, i.e., CometKiwi XXL (abbreviated KIWI-XXL) and COMET-22 (abbreviated COMET). Very strong average performance improve- ments can be observed. For instance, +3.6 and +1.1 points of KIWI-XXI, and COMET average gains shown over Towverinstrutt-iistra1777
F.2 Off-Policy Results based on ALMA
Tables 5 and 6 present the experimental results based on the ALMA model under off-policy set- tings. Consistently substantial improvements across metrics COMET-22 (abbreviated as COMET), CometKiwi-XXL (abbreviated as KIWI-XXL), and XCOMET-XXL (abbreviated as XCOMET)- -are
17

---

observed when applying our calibration method. The resulting performance surpasses all base- lines across nearly all language directions, including the strongest ALMA variant, ALMA-7B-R. For example, we observe gains of +0.5 COMET, +1.4 KIWI-XXI, and +2.0 XCOMET points on out-of-English translation compared to ALMA-7B-R.
F.3 On-Policy Results based on Tower
Table 7 and Table 8 present on-policy results measured by KIWI-XL and COMET, and by KIWI-XXL and XCOMET, respectively. We found that the on-policy setting, our calibration method performs better with relatively smaller learning rate (lc-5), whereas in the off-policy setting, larger learning rate (5e-5) yields the best performance. All other training settings are the same as those for off off-policy, except for sampling from the model itself instead of external model like GPT-4o-mini.
G More Results for Section 5.2
In this appendix, we provide additional results complementing Section 5.2 of the main text, including (1) an additional evaluation direction, en en-ru,, and (2) an alternative statistic for metric meta- evaluation, Kendall's T.
Figure 8 shows the Spearman coefficient and the corresponding translation performance in the en en-rr direction. Meanwhile, Figures and 10 present the results using Kendall's r for the en en-dde and en->r directions. respectively. clear that the main findings, mentioned in Section 5.2, hold across language directions and statistics.
H Human Study
In this appendix, we present the detailed human evaluation results for three additional target languages: German, Russian, and Dutch. Section H.1 outlines the annotation criteria used, and Section H.3 reports the score distributions and aggregated summaries for these language directions.
H.1 Annotation Criteria
We follow the standard of the official assessment system during WMT22 [Koemi et al., 2022], Annotators are asked to rate pair of source sentences and the corresponding hypothesis with scores ranging from 0 to 6. The descriptions for each quality level follows:
0: Nonsense/No meaning preserved: Nearly all information is lost between the translation and the source. Grammar is irrelevant.
2: Some meaning preserved: The translation preserves some of the meaning of the source but misses significant parts. The narrative is hard follow due fundamental errors. Grammar may be poor.
4: Most mcaning preserved and few grammar mistakes: The translation retains most of the meaning of the source. may have some grammar mistakes minor contextual inconsistencies.
6: Perfect meaning and grammar: The meaning of the translation is completely consistent with the source and the surrounding context (if applicable). The grammar is also correct.
Note that for WMT, continuous score is allowed for annotators. However, require annotators score with integers. For example, a score of 5 means in between of minor error and perfection, which may be reflected in tone, grammar, some subtle wording.
H.2 English-to-Chinese Translation Annotation Analysis
Figure 6 presents detailed annotation results, illustrating the changes in en en- translations before (TowerInstruct-Mistrl and after applying our calibration method. We can see that:
Both systems perform well, only few translations received scores below 3.
18

---

Calibrated Model
en-zh
2
0
0
0
3
Towerinstruct Sar---Mistt Mistral-7B
0
0
0
0
0
0
4
13
0
0
6
29
0
0
5
18
44
Figure 6: Detailed annotation results illustrate the number of changes in en-tzh translations. Each data point represents the number of samples corresponding to specific pair of scores before and after calibration (vertical and horizontal axes), respectively.
The primary improvements over the baseline are observed in better handling of minor to major errors. For instance, 29 and 47 translations previously rated and (minor errors) were upgraded to 6 (nearly perfect), respectively.
Note that annotators were blinded the source system of each translation by shuffling the order of each translation pair to prevent systematic (order) bias.
H.3 Other Annotation Results
Towerinstruct Mistral--
Number Samples
Human Evaluation Summary
TScore
CScore
Win
Loss
Tie
28
4.79
5.21
93
44
63
2
5
Annolation Score
Figure 7: Human evaluation results in en-tu direction: score distribution and an aggregated summary. TScore and CScore denote the average scores over 200 annotations for Towerlnstruct- Mistral-7B and our calibrated model, respectively. Win, Loss, and Tie indicate the number of samples for which our model outperformed, underperformed, tied with wwverlnstruct-Mistaal-777
We will release all detailed human annotation results other translation directions to the community".
'Detailed human annotations are available at http:////ggggggge..o///tttttt//e Tra Ceaeeiionnnaabbraaiaeeeee bration- translation. comtiiionuua----- human- eval.
19

---

Table 4: Evaluation of en-xxx translation on WMT24 using CometKiwi-XXX and COMET. Results are reported for all languages covered during Tower-vi pretraining. Note that the Tower-v2 models, have not been publicly released. For 4o and GPT-4o-mini, we use [2024b], improvements of 2 1.99 in XCOMET or 0.94 in COMET scores correspond to at least 90% estimated accuracy human judgment- both of which are achieved by our method.
including Tower-
the prompts following Hendy et al. [2023]. Notably, according to Koemi
KIWI-XXL.
COMET
KIWI-XXL
COMET
KIWI-XXL
COMET
COMET
Cloved
Tower
Tower 708-22-4
Towerfistrae--88
76.5
81
TowerBase 7B
+ 3TT BoN data
77.2
81.3
75.8
* CCO
78.9
82.2
78.0
83.2
78.
82.2
77.8
K3.4
7877
81.2
Calihottion
79.5
82.8
70.8
H3.7
80.4
82.9
78.0
R3.2
R0.2
81.7
Towertnstrass-13
78.1
82,3
77.6
M3.5
76.2
82.1
76.9
83.8
77.4
81.6
TowerRase- 13R
SPT BoN data
79.0
82.3
77.0
83.1
76.4
82.0
768
83.8
77.2
81.5
CPO
19.1
78.6
82.5
80.3
82.6
78.0
79.3
Calibration
Towerlnstruct-
w0--11
wh-
wh-skn
Avg
XTWI-XXI,
COMFT
KIWI-XXI.
COMBT
KIWI-XXI
KIWI-XXI
KIWI-XXL
GPT-4o-mimi GPT-44
78.3
84,6
Clooed
90.7
84.6
76.0
83.8
79.1
81.9
82.3
86.2
78.3
83.5
Tower- 70B-2-708--2
Towes- 70B-42 MBR/TRR
TowerInsracs78
81.1
84.4
27.7
83.7
77.9
81
84.7
77.4
82.7
TowerBase- 70
SFT on BoN data
80.5
83.5
76.9
83.4
78.6
H1.5
H2.3
K5.3
T7.6
82.4
CPP
81.9
83.8
70.4
83.7
80.5
81
B3.7
85.8
70.7
83.0
Calibottonn
83.6
84.8
81.0
84.3
81.9
82.7
84.6
86L1
81.0
83.6
Towerlnstrues
TowerBasc-
Calibration
79.0
79.3
+ SET on BoN data
81.4
84.2
78.4
83.7
79.6
81.7
83.8
86.1
79.1
83.0
CCO
83.9
84.8
80.7
84.0
81.9
82.2
85.9
86.9
81.3
83.6
Calibration
84.6
65.2
82.3
84.8
83.2
83.0
86.9
87.3
82.4
81.2
Table 5: Out-of-English translation evaluation results for ALMA models on the WMT22 dataset, measured by COMET, KIWI-XXL, and XCOMET. When applying calibration method on ALMA-base, consistent substantial improvements across all metrics and directions can be observed.
en en-ide
en-+08
COMET
KIWI-XXI.
XCOMET
COMET
KIWI-XXI.
XCOMET
ALMA-7B-Loud
85.45
80.70
96.49
89.05
82.06
90.82
+ on preferred data
89.11
81.28
DPO
85.19
80.02
96.22
88.78
81.03
90.12
CPO (ALMA-7B-R)
86.06
82.77
97.11
89.61
84.81
91.91
Calibration (ours)
86.22
83.83
97.25
90.24
86.25
93.06
COMET
KIWI-XXL
XCOMET
COMET
KIWI-XXL
XCOMET
ALMA-7B-LORA
85.44
81.51
89.94
84.87
77.14
88.11
+ SFT on preferred data
85.19
80.25
89.15
85.36
78.16
88.34
+ DPO
85,20
80.42
88.97
84.73
76.96
87.72
+ CPO (ALMA-7B-R)
82.35
89.63
85.89
81.79
89.55
+ Calibration (ours)
86.29
84.11
91.62
86.47
83.18
90.75
en-Fru
Avg.
COMET
KIWI-XXI
XCOMET
COMET
KIWI-XXL
XCOMET
ALMA--B-LLORA
87.05
82.60
86.37
80.80
91.67
SFT preferred data
86.88
81.79
92.57
86.39
80.38
91.32
DPO
86.70
81.61
92.53
86.12
80.01
91.11
+ CPO (ALMA-7B-R)
87.86
84.97
94.15
87.04
83.34
92,47
+ Calibration (ours)
88.41
86.30
94.46
87.53
84.73
94.43
20

---

Table 6: Into-English translation evaluation results for ALMA models on the WMT22 dataset, measured by COMET, KIWI-XXL, and XCOMET. When applying calibration method ALMA-base, consistent substantial improvements across all metrics and directions can be observed.
-en
CB-te
COMET
KIWI-XXL
XCOMET
COMET
KIWI-XXL
XCOMET
HLMA-7B-LoRA.
83.95
82.58
92.35
85.93
81.42
81.34
SFT preferred data
84.39
82.72
93.19
86.17
81.95
84.55
DPO
84.02
82,47
92.26
85.87
81.30
81.10
CPO (ALMA-7B-R)
84.61
83.11
93.85
86,29
82.29
85.76
Calibration (ours)
85.03
84.30
94.33
86.52
84.13
84.79
is-+en
zh-ten
COMET
KIWI-XXL
XCOMET
COMET
KIWI-XXL
XCOMET
ALMA-7B-LORA
86.09
84.65
75.02
79.78
73.65
83.94
SFT on preferred data
86.47
85,23
78.87
80.50
74.91
89.81
+ DPO
85.96
84.44
75.19
79.91
73.51
89.22
(ALMA-7B-R)
85.13
79.14
80.95
75.72
90.74
Calibration (ours)
87.29
86.37
79.07
81.33
76.88
92.61
ru-+en
Avg.
COMET
KIWI-XXL
XCOMET
COMET-22
KIWI-XXL
XCOMET
ALMA-7B-LoRA
84,84
80.19
88,50
84.12
80.50
84.23
SFT on preferred data
85.00
80.47
89.54
84.51
81.06
87.19
DPO
84.71
80.04
88.34
84.09
80.35
85.22
CPO (ALMA-7B-R)
85.11
80.69
90.10
84.72
81.39
87.92
Calibration (ours)
85.12
81.87
90.56
85.06
82.71
88.27
Table 7: On-policy results of en-xxx translation WMT24 using CometKiwi-XX XI. and XCOMET. Results are reported for all languages covered during Tower-vi pretraining.
KIWI-XI.
XCOMET
KIWI-XI.
XOOMET
KIWI-XI.
XOOMET
XCOMET
XCOMET
Towes
72.6
Towestostree-788
70.8
TowerBase-70
* SFT on BoN data
69.8
92.0
71.6
87.2
05.2
81.3
08.6
78.6
68.7
MLD
+ SFT on CPO
70.2
92.1
73.5
80.0
71.8
84.6
70.0
80.4
69.4
86
Calibntian
71.6
93.6
74.0
91.1
73.1
85.7
71.6
$1.5
70.9
N7.5
Towerinstrac: 13B
69.9
92.5
71.8
87.7
70.6
83.3
70.1
80.8
68.1
85.1
TowerBase- 138
SPT BoN data
70.6
92.9
71.9
$8.8
71.2
84.3
70.3
$1.6
68.3
85.0
CPO
92.4
72.0
70.6
Calibration
71.5
94.2
74.3
90.3
73.3
86.2
72,0
83.9
70.3
87.3
72.1
Calibnation
71.8
93.9
74.1
90.0
73.5
86.3
72.5
84.0
70.5
87.0
60-151
00-11t
00-
65-1ko
Avg
Models
XIWIXL
XCOMET
KIWEXL
XCOMET
KIWE-L
XCOMET
KIWL-XL
XCOMET
KIWI-LL
XCOMET
Olood
GFT mini GPT-4o
69.4
70.6
90.5
$8.9
68.1
68.7
85.7
83.7
71.2
71.5
87.6
88.5
73.2
73.7
86.2
85.6
69.2
69.3
85.3
85.6
Tower- Towes-708--y2 708-v2
Towee-70B--22 MBR/TRR
TBwerinstract-18
71.5
90.9
71.1
$6.1
71.1
86.8
73.6
82.8
70.3
85.5
TowerBase- 7B
SPT on BoN data
71.8
90.9
71.5
86.3
71.2
87.0
76.5
82.8
70.8
85.6
CPO
72.2
91.2
72.5
87.7
73.7
$0.9
75.3
86.
71.8
87.4
Calibnatio
73.7
92.3
73.9
$8.7
75.0
90.5
76.77
87.5
73.4
88.6
Towsststrues-3B
71.1
71.2
TowerBase- BaN
73.0
Calibittinn
Towerinstrurt Mistral
71.9
91.1
71.6
87.2
72.1
88.0
76.2
85.6
71.1
86.7
a SFT on BoN data
72.5
91.7
72.5
87.7
72.8
88.6
75.5
86.99
71.5
N7.5
CRU
73.4
92.3
73.0
87.8
72.9
SNL3
77.4
72.5
N7.3
Calibantin
73.9
93.2
7400
$9.3
74.7
90.2
77.7
80.4
73.6
80.3
21

---

Table 8: On-policy results of en- translation WMT24 using CometKiwi-Xe XXL and COMET. Results are reported for all languages covered during Tower-vi pretraining.
KIWI-XXI
KIWI-XXI
KIWI-XXI
Clovel
Towes-708-a22
Towes 708--224
[owerlostrrc: 7D
16.5
81.2
76.3
82.8
75.9
81.1
83.1
76.7
81.2
TiwerBaase
+ SFT on BoN data
76.7
81.4
76.6
M2.4
76.1
H1.7
75.4
H2.2
77.3
81.2
+ -CPO
78.3
81.1
79.7
82.7
80.0
82.
77.8
82.9
79.8
80.9
Calihotian
70.8
82.2
80.8
83.4
M1.2
78.4
R3R
81.4
8L8
Towerfnstraii133 13B
78.1
82.3
77.6
83.5
76.2
82.1
76.9
83.8
77.4
81.6
TowerBase- 138
STT BoN
18.7
78.5
79.6
78.4
78.7
CPU
Calibration
Iowertastres
79.1
+ Calibration
80.7
83.0
81.3
83.6
82.0
83.6
81.2
00--11
00-11t
00-ppt
AVE
Models
KIWI XXL
COMET
KIWI XXL
COMET
KIWEXX XXL
COMET
KIWLXXL
COMET
KIWI-XLL
COMET
Clonn
GPT-4o mini GPT-4o
78.3
90.7
84.6
84.5
24.1
76.0
83.6
83.8
27.9
79.1
81.9
81.9
81.2
82.3
86.2
86.1
78.3
76.7
83.5
Towes-70B-v2
Tower 700-v2 MBR/TRR
Towerlnstract-10 -10
81.1
84.4
77.7
83-7
77.9
81.8
80.0
B4.7
77.4
82.7
TowerBase-7B 7B
SPT on BoN data
81.4
84.9
76.0
84.5
76.7
H2.5
80.2
85.0
77.8
82.9
CPO
82.6
83.9
80.9
83.5
81.2
82.3
83.3
85.5
80.4
82.8
Calibration
84.1
82.9
83.5
86.2
Towerfsstrus-188
TowerBase-
Caliboation
Towerlastreet Mistral
81.5
84.5
79.0
84.0
79.3
82.2
81.7
85.3
78.8
83.1
+ SET on BaN data
82.2
84.7
79.8
84.6
80.2
82.6
83.3
86.
73.8
83.6
CCO
83.6
84.6
80.5
84.0
80.
82.3
85.6
809
K3.3
Calibration
84.8
84.9
82.1
84.7
82.7
83.1
86.1
87.2
82.4
84.1
75
Towerinst-78 78
Tow-r-708-v2
MBR/TRA
74
Tower-llasee// 70 Calibration
Towerinst:133 138
Inwer-iase-1 30 Calibration
CometKiwi-Xi
72
71
69
44.0
45.0
46.0
47.0
48.0
49.0
50.0
51.0
52.0
53.0
54.0
Spearman Score
Figure The Spearman coefficient and the corresponding translation performance in en-tru direction under different settings for the Tower series models. The color gradients of and e, from lighter to darker shades, indicate the results of fine-tuning with varying amounts of Best-of-N data, from 400 to 2000 samples. * and * denote the application of our calibration method on 13B and 7B models, respectively.
22

---

73.0
72.5
Tower 708-v2 MBR/TRR
Towerins-78
TowrdBase78 + SFT
CometKiwi-X
71.5
Towerins-1338
lower-lase-78 Calbration
Tower Deve--aae-133 SFT
71.0
238 Calibrttonn
CometKlvi
70.5
69.5
69.0
37.0
38.0
39.0
40.0
41.0
42.0
43.0
44.0
45.0
46.0
47.0
Kendall Score
Figure 9: The Kendall coefficient and the corresponding translation performance in en-dde direction under different settings for the Tower series models. The color gradients of v and from lighter to darker shades, indicate the results of fine-tuning with varying amounts of Best-of-N data, from 400 to 2000 samples. * and * denote the application of our calibration method 13B and 7B models, respectively.
75
Tower 708-v2 MBRTTRR
74
Towerinst-78
Tower-ase-78 SFT
CometKiwi-Xi
7B Calibration
Towesinst-138
72
Inwer-tkse-1300 38 SFT
TowerBase Calibration
CometKiwi
71
69
32.0
32.8
33.6
34.4
35.2
36.0
36.8
37.6
38.4
39.2
40.0
Kendall Score
Figure The Kendall coefficient and corresponding translation performance en-tru direction under different settings for the Tower series models. The color gradients of v and from lighter to darker shades, indicate the results of fine-tuning with varying amounts of Best-of-N data, from 400 to 2000 samples. * and * denote the application of our calibration method on 13B and 7B models, respectively.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction accurately reflect the ideas, findings, and impli- cations of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We acknowledge assumptions and limitations in our paper where applicable. We also discuss the limitations of our analysis and point to future work Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [NA]
Justification:
Our paper does not include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide complete description of the dataset and hyperparameters in Section 4. We also release the code an anonymous repository for reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., .., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
25

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide complete description of the dataset and hyperparameters in Section We also release the code an anonymous repository for reproduction, along with part of the human evaluation data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nips cc/pu
blic/ Sunides/CooeSuuleee
irsiiinnooccyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ucee//ddddeeeoeeuummmiii / //nips.ccac wttpccccpppppuuueeeeeeeeee wwwiiccgggeee//eeeeeeeee /CodeSubdeieee SetdeSSumisssioo ssssssssssssoooooeeeo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide a complete description of the dataset, including training and test details, and hyperparameters in Section 4. We also provide more detailed dataset description, along with the prompt formats we used Appendix D, E.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We did not provide statistical significance of the experiments because 1) it would be too computationally expensive, and 2) our improvements over the baseline are very clear and consistent across different metrics, model types, and language directions, indicating predictably low statistical risk.
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the GPU usage for each training setting in Section 4.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //tpr//////...//eeeeeeeee cc cetblic/Ethicsshideell.
Answer: [Yes]
Justification: The paper strictly follows the full Code of Ethics from NeurIPS.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss possible impacts of our work in Appendix B.
Guidelines:
27

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do work with any high-risk datasets models in this work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets and related work are properly cited in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: In our experiments, we largely repurpose publicly available datasets. For human evaluation data along with the machine-gennrated dataset involved in this paper, provide comprehensive documentation, including the annotation guidelines and data format. These materials are included in Appendix H.1 and the anonymous repository submitted with the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: For the human evaluation, we provide comprehensive documentation, including the annotation guidelines and data format in Appendix H.1.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our experiments do not involve potential ethical risks.
Guidelines:
29

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Our experiments are based on open-source LLMs, e-g.. Tower and ALMA. Training and decoding details as present in Section 3, 4. We also provide off off-policy data generation details in Section 4.1, where the use of GPT-4o-mini is involved.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /atpss://euuipss// eee//f/feeennce22222 /LLM) for what should or should be described.
30


Reviews:
[{"review_id": "X7AJdNmiF6", "reviewer": "Reviewer_jW1Y", "summary": "This paper addressed a well-known gap in translation decoding, where employing maximum a posteriori (MAP) search often generates lower-quality outputs, as model probability doesn't align well with human judgements. The issue is known as miscalibration issue. To resolve it, the authors proposed calibrating translation likelihoods with quality estimates, specifically, they train a lightweight calibration layer on top of LLMs, optimizing the Pearson correlation between predicted likelihood and reference quality. With just 2k instances per language direction, this quality-calibrated likelihood consistently enhances translation quality that are detected both in automatic evaluation metrics and human evaluations. Tested across 10 languages and on translation-specialized LLMs like TowerMistral, the authors confirmed the consistent gains by using the proposed approach. \n\n- the paper is mostly easy to follow.\n- looks like 2k samples suffice, but I am curious how smaller or larger sample sizes impact performance. If possible, additional experiments in mid/low-resourced directions would be interesting if the page allows.\n- do you have any insights into search dynamics of the proposed method? Compared to MAP decoding, does calibration impact beam search diversity or risk repetitive patterns or hallucination unnecessarily?", "questions": "please see the summary.", "limitations": "n/a", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strengths\n- the proposed method works well with minimal data of 2k examples per direction and computational resources of 1-2 H100 GPUs, making it practical and efficient.\n- Experimental results across multiple languages, metrics, and human evaluations demonstrate improvements\n- the authors will likely make available code and model checkpoints, enabling reproducibity\n\nweaknesses\n- unclear to me how well the calibration holds under domain shifts or when scaling to low-resource languages with low-quality noisier signals. It'd be interesting to report the results for mid/low-resourced language directions.\n- this work would benefit from deeper comparisons to related methods like quality-aware decoding [Fernandes et al., 2022]. Have you ever done such experiments?", "was_revised": true, "final_justification": "In response from the authors, my concern and questions are well addressed. I raise my score 4->5.", "rebuttal": "Thank you for carefully reading our paper and highlighting its contributions. We address your concerns as follows:\n\n> \"Unclear to me how well the calibration holds under domain shifts or when scaling to low-resource languages with low-quality, noisier signals. It'd be interesting to report the results for mid/low-resourced language directions.\"\n\nIn fact, our experiments with ALMA already include a low-resource language: Icelandic. As shown in Tables 4 and 5, ALMA achieves notable improvements for **EN→IS (+0.9 COMET, +2.6 COMETKIWI, +1.7 XCOMET)** and **IS→EN (+1.2 COMET, +1.7 COMETKIWI, +4.0 XCOMET)**, consistently outperforming both SFT and CPO baselines.\n\nTo further demonstrate the effectiveness of calibration in low-resource settings, we conducted one additional experiment. Since Tower and ALMA primarily target high-resource languages, we selected **Gemma-3-12B** (google/gemma-3-12b-it) as our testbed, due to its strong multilingual base capabilities and broader language coverage.\n\nSettings: (1) We applied our calibration method directly to Gemma-3-12B across 10 languages directions from this year’s WMT25 competition (ongoing), covering low-resource languages (Icelandic, Estonian) and mid-resource languages (Serbian, Ukrainian), as well as six other 6 high-resource languages. (2) We used the same data generation pipeline described in our paper, originally based on ​Flores; (3) We reused the same training hyperparameters as those used for ​TowerMistral-7B. The results are as follows:\n\n|             | Metric  | en-zh | en-ar | en-cs | en-ja | en-ko | en-ru | en-sr (mid) | en-uk (mid) | en-et (low) | en-is (low) | Avg. |\n|-------------|---------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----------:|:-----------:|:-----------:|:-----------:|:----:|\n| SFT         | KIWI-XL |  70.8 |  65.7 |  68.9 |  75.1 |  76.0 |  71.4 |     71.3    |     69.3    |     71.9    |     66.8    | 70.7 |\n| CPO         | KIWI-XL |  72.1 |  67.0 |  70.2 |  76.5 |  77.4 |  72.4 |     71.4    |     70.3    |     73.4    |     68.7    | 71.9 |\n| **Calibration** | KIWI-XL |  **72.9** |  **67.9** |  **71.4** |  **77.1** |  **77.9** |  **73.6** |     **72.9**    |     **71.4**    |     **75.1**    |     **69.2**    | **72.9** |\n| SFT         | Comet22   |  80.2 |  73.4 |  81.3 |  83.9 |  83.8 |  80.8 |     82.6    |     80.4    |     82.1    |     75.4    | 80.4 |\n| CPO         | Comet22   |  80.7 |  74.1 |  81.3 |  84.3 |  84.1 |  80.9 |     82.3    |     80.4    |     82.4    |     76.1    | 80.7 |\n| **Calibration** | Comet22   |  **80.9** |  **74.7** |  **82.1** |  **84.6** |  **84.4** |  **81.5** |     **83.2**    |     **81.0**    |     **83.0**    |     **76.6**    | **81.2** |\n\nWe can see that in this setting, for mid- and low-resource languages (the last four columns), calibration yields clear improvements over both SFT and CPO across both reference-free and reference-based metrics, aligning with our paper. That said, we acknowledge that for certain low-resource languages lacking support from neural metric models, such as Bhojpuri, our method is likely to fail. Addressing these directions will require joint efforts from the broader community, particularly in developing better evaluation metrics, which fall beyond the scope of this paper.\n\n> \"This work would benefit from deeper comparisons to related methods like quality-aware decoding [Fernandes et al., 2022]. Have you ever done such experiments?\n\n**Yes, we do have such experiments**. Quality-aware decoding (QAD), particularly in the context of [Fernandes et al., 2022], typically involves reranking or voting strategies. A common instantiation of QAD is Best-of-N (BoN) sampling, where translations are scored and ranked using a reference-free metric model (e.g., COMETKiwi-XXL). In Section 5.3, we present direct results under settings of (1) base-model + beam-search, (2) base-model + BoN, (3) calibration-model + beam-search, and (4) calibration-model + BoN.\n\nTo ensure fairness, we adopt a cross-metric evaluation: Candidates are reranked using KIWI-XXL, while the results are measured using a reference-based model, Comet22. To be fair, for calibration, we also use KIWI-XXL as the training signal. We summarize them as follows:\n\n|       Settings                    | Comet22 |\n|---------------------------|---------|\n| Tower-Base (Beam=5)       | 82.6    |\n| Tower-Base (BoN=10)       | 82.8    |\n| Tower-Base (BoN=100)      | 83.4    |\n| Tower-Calibrate (Beam=5)  | 83.9    |\n| Tower-Calibrate (BoN=10)  | 83.3    |\n| Tower-Calibrate (BoN=100) | 84.0    |\n\nIn short, we can easily see that:\n\n1. Our calibrated model with beam search (beam=5) achieves clearly better performance than base model + BoN (100-time sampling);\n2. When controlling the model as Tower-Base,  beam search (beam=5) leg behind that of BoN, even with only 10-time sampling;\n3. When controlling the model as our calibrated model,  beam search (beam=5) achieves comparable results with that of BoN (100-time sampling).\n\n**These results form the basis of our conclusion in Section 5.3: ​Calibration enhances the effectiveness of maximum *a posteriori* decoding. Also, compared to QAD, it offers clear efficiency benefits.** Note that 100-time sampling is far from practical for online services. From another perspective, calibration can be seen as a training time optimization that captures the benefits of QAD.\n\n> \"Do you have any insights into search dynamics of the proposed method? Compared to MAP decoding, does calibration impact beam search diversity or risk repetitive patterns or hallucination unnecessarily?\"\n\nThanks for your insightful questions! In our paper, we did not dig deep into the search dynamics or changes in the decoding space. But we can share with you some observations during our experiments.\n\n1. **Hallucination**: We did not observe any hallucinations (lexicon/pattern repetition) during our experiments, at least not in Chinese, German, Dutch, and Russian, where we conducted detailed human analysis. In fact, even the base model (TowerMistral-7B) exhibited only minimal instances of hallucination. While not rigorously, we have a feeling that hallucinations are largely due to limited instruction-following ability, which can be effectively mitigated through SFT with a few translation examples, at least for high- and mid-resource languages. Notably, in our additional experiments with the instruction-tuned Gemma-12B model, we also found no hallucination issues, even without SFT stage.\n2. **Beam Search Diversity**: Broadly speaking, calibration is to reorder the hypothesis likelihood within decoding space, while its impact on the shape of the space (flatter or sharper) is hard to say. We believe it depends on multiple aspects, like the sampling strategy.\n   Additionally, we read your question about diversity as a concern that calibration might reduce the diversity of the decoding space, potentially diminishing the benefits of using a larger beam size. To investigate this, we conducted an experiment using the TowerMistral-7B model, selecting four language directions (EN-DE, EN-ES, EN-RU, EN-ZH) and varying the beam size to compare its impact on both the base and our calibrated model as follows. It is easy to see that: (1) When the beam size exceeds 5, we do not observe further gains. (2) From Beam-1 to Beam-5, the gains for the SFT model and Calibrated models are comparable, 0.7 vs. 0.8 in KIWI-XL and 0.4 vs. 0.6 in Comet22. **Thus, we can assume that calibration does NOT change the shape of the space to a degree that affects search effectiveness.**\n\n| Beam-Size             |  1 |  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 | 10 |\n|-----------------------|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n| SFT (KIWI-XL)         | **70.3** | 70.8 | 70.9 | 70.9 | **71.0** | 70.9 | 70.9 | 70.9 | 70.9 | 70.9 |\n| Calibration (KIWI-XL) | **72.3** | 72.8 | 73.1 | 73.1 | **73.1** | 73.2 | 73.2 | 73.2 | 73.2 | 73.2 |\n|                       |      |      |      |      |      |      |      |      |      |      |\n| SFT (Comet22)           | **82.2** | 82.4 | 82.6 | 82.6 | **82.6** | 82.6 | 82.6 | 82.6 | 82.6 | 82.6 |\n| Calibration (Comet22)   | **83.3** | 83.6 | 83.7 | 83.9 | **83.9** | 83.9 | 83.8 | 83.9 | 83.9 | 83.9 |\n\n---\n\nThank you again for your careful reading of our paper and for your insightful suggestions and questions. Hopefully, our response addresses your concerns. We will incorporate this discussion, particularly for the low-resource and decoding space aspects, in our next version. Also, we are more than happy to provide further explanation or discussion if needed."}, {"review_id": "AWbfL1EDk1", "reviewer": "Reviewer_FndS", "summary": "This paper introduces a method to enhance machine translation for LLMs by calibrating the model's hypothesis likelihood with translation quality. At small computational cost, the method substantially improves translation decoding quality. Experiments on two top-performing translation-specialized LLMs (ALMA and Tower, 7B/13B parameters) demonstrate its efficacy. Furthermore, analysis experiments indicate that the model also improves its translation assessment capability.", "questions": "1. The comparison between the MBR of the Tower-70B model and the additionally trained 7B model are not univariate. It is suggested to supplement with an SFT fine-tuned or fine-tuned with proposed method on 7B model to perform MBR, to better control variables and demonstrate the effect of the proposed method when superimposed with MBR.\n2. Supplement experiments or extra explanations for the points mentioned in the weakness part.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. This paper proposes a simple and effective method presented in clear writing style, which is easy to follow.\n2. The method is computationally economical compared with methods like MBR, and even achieves a higher performance.\n3. Thorough experiments demonstrate significant improvement at low computational cost, reflected in both COMET-based metrics and human evaluation.\n4. Analysis experiments indicate enhanced translation quality assessment ability, further illustrating the effectiveness of translation quality data calibration.\n\nWeaknesses：\n1. The experimental baselines are insufficient, only comparing the single baseline of CPO. Fine-tuning work based on translation quality preferences, such as the work [1] referenced by Section 2, should be considered for inclusion in the main experiments.\n2. The experimental setup mentions that the presented results show the best outcomes under different learning rate settings. Sensitivity analysis regarding learning rates should be added to provide more information about the method’s robustness.\n\n[1] A Preference-driven Paradigm for Enhanced Translation with Large Language Models", "was_revised": true, "final_justification": "Thank you for your reply. Considering your reply and other reviewers' comments, I tend to maintain my current positive score.", "rebuttal": "Thanks for your insightful suggestions on our paper and highlighting its contributions. We address your concerns as follows:\n\n> \"The experimental baselines are insufficient, only comparing the single baseline of CPO. Fine-tuning work based on translation quality preferences, such as the work [1] referenced by Section 2, should be considered for inclusion in the main experiments.\"\n\nThank you for your suggestion. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhu et al., 2024] did not release their codebase, and their implementation is based on weaker models such as LLaMA-1 or Mistral. To address your concern, we apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying calibration to the top-performing translation model, i.e., TowerMistral-7B [1], using their data.\n\nTo ensure comparability, we use the same beam search size of 4. The results are evaluated on the WMT22 test set using COMET as shown in the following table:\n\n|                            | EN-DE | DE-EN | EN-ZH | ZH-EN | Avg. |\n|----------------------------|:-----:|:-----:|:-----:|:-----:|:----:|\n| Mistral-Ins.               | 81.2  | 82.7  | 82.5  | 77.7  | 81.0 |\n| Mistral-Ins. + BEST        | 81.8  | 83.0  | 83.3  | 78.3  | 81.6 |\n| Mistral-Ins. + PL          | 82.9  | 83.4  | 84.7  | 79.3  | 82.6 |\n| **Mistral-Ins. + Calibration** | **83.5** | **84.4** | **85.4** | **80.1** | **83.4** |\n| TowerMistral + BEST        | 88.9  | 85.7  | 89.3  | 83.3  | 86.8 |\n| **TowerMistral + Calibration** | **89.9** | **86.6** | **90.1** | **84.7** | **87.8** |\n\n\\+ Best is supervised fine-tuning using best scoring samples, and \\+ PL is the result of their implementation of the PL model [2]. \\+ Calibration is applying our method. It is easy to see that\n\n1. On the weaker base models, such as Mistral-Ins, **Calibration yields clear improvements** over both SFT and PL. We **provisionally attribute** this gain to Calibration’s broader consideration of factors compared to PL. Specifically: (1) Calibration normalizes both likelihood and quality scores within a group, making the objective invariant to scale and shift. This promotes stable and robust optimization across diverse input distributions. (2) Calibration directly integrates translation quality via the metric function q(⋅∣x), and optimizes at the distribution level using the Pearson correlation. In contrast, [2] introduces reweighting for each pairwise comparison. In extreme cases, this approach may treat a comparison between ratings 2 (bad) and 1 (nonsense) the same as one between 4 (nearly perfect) and 3 (good), which could be overly heuristic.\n2. Even when applying calibration to the top-performing system, TowerMistral, using human-annotated data, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.\n\nWe will include these analyses in our next version.\n\n[1] Tower: An Open Multilingual Large Language Model for Translation-Related Tasks\n[2] A Preference-driven Paradigm for Enhanced Translation with Large Language Models\n\n> \"The experimental setup mentions that the presented results show the best outcomes under different learning rate settings. Sensitivity analysis regarding learning rates should be added to provide more information about the method’s robustness.\"\n\nThank you for your careful reading. Yes, as mentioned in lines 195–196, to ensure robust results, we experiment with learning rates ranging from 1e-5 to 1e-4, in steps of 1e-5, for each setting.\n\nDue to space limitations, we present a subset of these results here. The following table (aligned with experiments in Table-1) provides the details, using TowerMistral-7B as the base model and CometKIWI-XL as the evaluation metric.\n\n|                    | en-de | en-es | en-ru | en-zh | en-fr | en-nl | en-it | en-pt | en-ko | Avg. |\n|--------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|------|\n| SFT (1e-5)         |  70.7 |  71.6 |  70.6 |  70.3 |  68.5 |  72.2 |  71.6 |  72.8 |  76.3 | 71.6 |\n| SFT (3e-5)         |  70.8 |  71.7 |  70.7 |  70.4 |  68.4 |  72.3 |  71.6 |  72.6 |  76.2 | 71.6 |\n| **SFT (5e-5)**     |  **70.7** |  **71.8** |  **70.8** |  **70.5** |  **68.5** |  **72.3** |  **71.6** |  **72.7** |  **76.2** | **71.7** |\n| SFT (7e-5)         |  70.8 |  71.8 |  71.1 |  70.6 |  68.5 |  72.3 |  71.8 |  72.7 |  76.1 | 71.7 |\n| SFT (9e-5)         |  70.9 |  71.9 |  70.9 |  70.6 |  68.4 |  72.2 |  71.7 |  72.7 |  76.2 | 71.7 |\n| CPO (1e-5)         |  71.3 | 72.7 | 72.5 | 71.8 | 69.6 | 72.9 | 72.8 | 73.7 | 77.2 | 72.7 |\n| CPO (3e-5)         |  71.3 |  72.7 |  72.6 |  72.0 |  69.6 |  73.2 |  73.2 |  74.0 |  77.2 | 72.9 |\n| **CPO (5e-5)**     |  **71.2** |  **73.1** |  **72.3** |  **71.8** |  **70.0** |  **73.3** |  **73.1** |  **74.0** |  **77.4** | **72.9** |\n| CPO (7e-5)         |  71.2 |  73.0 |  72.7 |  71.9 |  69.6 |  73.1 |  72.9 |  74.1 |  77.1 | 72.8 |\n| CPO (9e-5)         |  71.1 |  72.8 |  72.4 |  71.6 |  69.5 |  72.8 |  72.6 |  73.7 |  76.9 | 72.6 |\n| Calibration (1e-5) |  72.3 |  73.9 |  73.2 |  72.4 |  70.5 |  74.0 |  73.9 |  74.7 |  78.1 | 73.7 |\n| Calibration (3e-5) |  72.3 |  73.6 |  73.5 |  72.6 |  70.4 |  74.1 |  73.8 |  74.8 |  78.0 | 73.7 |\n| **Calibration (5e-5)** |  **72.4** |  **73.9** |  **73.6** |  **72.6** |  **70.8** |  **74.2** |  **74.1** |  **75.1** |  **78.1** | **73.9** |\n| Calibration (7e-5) |  72.4 |  73.9 |  73.4 |  72.7 |  70.5 |  74.1 |  73.7 |  74.8 |  78.1 | 73.7 |\n| Calibration (9e-5) |  72.1 |  73.6 |  73.3 |  72.5 |  70.4 |  73.4 |  73.4 |  74.7 |  77.0 | 73.4 |\n\nThe average results for all 10 steps are as follows.\n\n| Learning Rate              | 1e-05 | 2e-05 | 3e-05 | 4e-05 | 5e-05 | 6e-05 | 7e-05 | 8e-05 | 9e-05 | 1e-04 |\n|----------------------------|---------:|---------:|---------:|---------:|---------:|---------:|---------:|---------:|---------:|---------:|\n| SFT (CometKIWI-XL)         |     71.6 |     71.6 |     71.6 |     71.6 |     **71.7** |     71.7 |     71.7 |     71.7 |     71.7 |     71.7 |\n| CPO (CometKIWI-XL)         |     72.7 |     72.8 |     72.9 |     72.9 |     **72.9** |     72.3 |     72.8 |     72.8 |     72.6 |     72.6 |\n| Calibration (CometKIWI-XL) |     73.7 |     73.6 |     73.7 |     73.6 |     **73.9** |     73.6 |     73.7 |     73.5 |     73.4 |     73.5 |\n\nDuring our experiments, we found that SFT is not particularly sensitive within the explored learning rate range, which we attribute to the extensive high-quality SFT already performed on TowerMistral-7B.\nFor CPO and Calibration, the performance tends to fluctuate, often degrading at relatively small or large learning rates. The optimal learning rate for both is generally around 5e-5 across different evaluation settings.\n\n> \"The comparison between the MBR of the Tower-70B model and the additionally trained 7B model is not univariate. It is suggested to supplement with an SFT fine-tuned or fine-tuned with the proposed method on 7B model to perform MBR, to better control variables and demonstrate the effect of the proposed method when superimposed with MBR.\"\n\nThank you for your suggestion. Indeed, directly conducting experiments on the base model with MBR applied would better demonstrate the effect of our method. However, applying the same magnitude of MBR (e.g., 100-fold sampling) to TowerMistral is challenging due to our limited computational resources. (That said, we are currently running such an experiment based on your suggestion, and we hope to provide updated results before the rebuttal period ends.)\n\nAlthough indirect, in Section 5.3, we provide a comparison of calibration with BoN (sampling and reranking using QE model), another key form of quality-aware decoding that is similar to MBR. We summarize them as follows:\n\nIn Section 5.3, we present direct results under settings of (1) base-model + beam-search, (2) base-model + BoN, (3) calibration-model + beam-search, and (4) calibration-model + BoN. To ensure fairness, we adopt a cross-metric evaluation: Candidates are reranked using KIWI-XXL, while the results are measured using a reference-based metric, Comet22: \n\n|       Settings                    | Comet22 |\n|---------------------------|---------|\n| Tower-Base (Beam=5)       | 82.6    |\n| Tower-Base (BoN=10)       | 82.8    |\n| Tower-Base (BoN=100)      | 83.4    |\n| Tower-Calibrate (Beam=5)  | 83.9    |\n| Tower-Calibrate (BoN=10)  | 83.3    |\n| Tower-Calibrate (BoN=100) | 84.0    |\n\nIn short, we can easily see that:\n\n1. Our calibrated model with beam search (beam=5) achieves clearly better performance than base model + BoN (100-time sampling);\n2. When controlling the model as Tower-Base,  beam search (beam=5) leg behind that of BoN, even with only 10-time sampling;\n3. When controlling the model as our calibrated model,  beam search (beam=5) achieves comparable results with that of BoN (100-time sampling).\n\n**These results form the basis of our conclusion in Section 5.3: calibration improves the effectiveness of maximum *a posteriori* decoding and offers clear efficiency advantages over QAD.**\n\n---\n\nThank you again for your careful reading and meaningful suggestions. We hope our additional explanation and experiments address your concern. We look forward to your further discussion!"}, {"review_id": "5YCBA6SEj1", "reviewer": "Reviewer_bA3p", "summary": "The paper propose a method to calibrate the likelihood of translation hypotheses with their actual translation quality by optimizing the Pearson correlation between these two metrics during training, which aims to enhance the effectiveness of decoding and improve translation quality.\nThis calibration method involves sampling multiple hypotheses, computing their likelihoods and quality scores using an external metric (e.g., COMET), and minimizing the negative Pearson correlation between these scores.\n\nThe experiments conducted demonstrate significant improvements in translation quality across various metrics and languages when using this calibration method. Additionally, the calibrated likelihoods serve as a capable proxy for translation quality, surpassing some existing quality estimation models.", "questions": "Shown in weakness part, 1 for the SLiC paper comparison, 2 for the discrepancy between automatic evaluation improvement and human evaluation improvement.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\nFirst, I think the idea of applying likelihood calibration with quality estimation for machine translation task is novel, the method is simple and clear, which is also orthogonal to SFT method.\n\nSecond, give credit to the author for carrying out extensive experiments across 10 languages, multiple base models (Tower, ALMA), and metrics (COMET, XCOMET, KIWI) demonstrate consistent, significant gains over strong baselines. Human evaluations (200 samples/direction) confirm quality improvements. \n\nWeakness:\n\nI think the paper bears some similarity with the paper \"Calibrating Sequence likelihood Improves Conditional Language Generation\" in ICLR2023 (which the author has already cited.) which proposed a sequence likelihood calibration (SLiC) stage after MLE to enhance the model quality.  Although the metric and task may differ a little bit (this paper focus on MT while the SLiC paper doesn't cover ), I think the core idea is very relevant. So it's kind of bring down the innovativeness a bit.  So I would like to see more analysis on this part, may be a direction comparison in experiment.\n\nAnother small concern is that the translation quality improvement of human evaluation shown in Figure 4 is far more significant than the automatic evaluation improvement in Table1. Does it mean that the automatic evaluation metric like XCOMET and KIWI-XL could not fully reflect the improvement of this method ?  If so, is it problematic (or can be improved) to use COMET as the translation quality score for likelihood calibration.", "was_revised": true, "final_justification": "My concern of the overlap with SLiC paper is properly addressed. \nit's nice to see Calibration yields clear improvements over both SFT and PL.", "rebuttal": "Thanks for your insightful suggestions and for noting our contribution. We summarize and address your concerns as follows:\n\n> The relevance and differences compared to the SLiC paper.\n\nThank you for your careful reading and valuable suggestions. Our paper indeed shares part of the motivation with SLiC [Zhao et al., 2023][1]. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhao et al., 2023] did not release their codebase, and the implementation complexity, along with a lack of certain details, currently prevents us from conducting a direct comparison.\n\nBroadly speaking, SLiC can be viewed as a preliminary variant of PL-based preference optimization. Although indirect, it operates in a similar manner to [Zhu et al., 2024][2], a PL-based PO method applied in the machine translation domain. **To address your concern, we provide a direct comparison with this approach to demonstrate the effectiveness of our method.**\n\nTo ensure comparability, (1) we follow their setting and apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying to the top-performing translation model, i.e., TowerMistral-7B [3], using their data. (2) We use the same beam search size of 4 in [Zhu et al., 2024]. The results are evaluated on the WMT22 test set using COMET as shown in the following table:\n\n|                            | EN-DE | DE-EN | EN-ZH | ZH-EN | Avg. |\n|----------------------------|:-----:|:-----:|:-----:|:-----:|:----:|\n| Mistral-Ins.               | 81.2  | 82.7  | 82.5  | 77.7  | 81.0 |\n| Mistral-Ins. + BEST        | 81.8  | 83.0  | 83.3  | 78.3  | 81.6 |\n| Mistral-Ins. + PL          | 82.9  | 83.4  | 84.7  | 79.3  | 82.6 |\n| **Mistral-Ins. + Calibration** | **83.5** | **84.4** | **85.4** | **80.1** | **83.4** |\n| TowerMistral + BEST        | 88.9  | 85.7  | 89.3  | 83.3  | 86.8 |\n| **TowerMistral + Calibration** | **89.9** | **86.6** | **90.1** | **84.7** | **87.8** |\n\n\\+ Best is supervised fine-tuning using best scoring samples, and \\+ PL is the result of their implementation of the PL model [2]. \\+ Calibration is applying our method. It is easy to see that\n\n1. On the weaker base models, such as Mistral-Ins, **Calibration yields clear improvements** over both SFT and PL.\n2. Even when applying calibration to the top-performing system, TowerMistral, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.\n\nFurther, our work encompasses broader underlying concepts compared to [Zhao et al., 2023] in motivation, modeling, and analysis. We summarize and discuss them as follows:\n\n1. [Zhao et al., 2023] aim to optimize candidate ranking based on their similarity to a target output, where similarity is computed using token or sentence representations. In contrast, our work proposes optimizing the Pearson correlation between translation likelihood and quality, rather than similarity. Moreover, the Pearson objective normalizes both likelihood and quality scores within a group, making it invariant to scale and shift. Our calibration approach directly incorporates translation quality via the metric function q(⋅∣x), optimizing at the distribution level using the Pearson correlation. By contrast, [Zhao et al., 2023] focus on pairwise comparisons, lacking a more direct utilization of absolute quality values at a distribution level.\n2. In Section 5.2, we show that our objective, Pearson correlation itself, shares the goal with translation meta-evaluation. It provides a theoretical foundation for unifying translation quality optimization and quality evaluation within a single model.\n3. In Section 5.3, we analyze the connection between calibration method and quality-aware decoding, and show that calibration captures the benefits of Best-of-N sampling (a key form of QAD) during training time. This, in turn, enhances the effectiveness of MAP decoding strategies such as beam search.\n\n[1] Calibrating Sequence Likelihood Improves Conditional Language Generation\n\n[2] A Preference-driven Paradigm for Enhanced Translation with Large Language Models\n\n[3] Tower: An Open Multilingual Large Language Model for Translation-Related Tasks\n\n> “Another small concern is that the translation quality improvement of human evaluation shown in Figure 4 is far more significant than the automatic evaluation improvement in Table 1. Does it mean that the automatic evaluation metrics like XCOMET and KIWI-XL could not fully reflect the improvement of this method? If so, is it problematic (or can be improved) to use COMET as the translation quality score for likelihood calibration.”\n\nThanks for these insightful questions. We try to address them as follows:\n\n1. Firstly, in Table 1, our method achieves +2.8 KIWI-XL and +2.7 XCOMET improvements. It is worth noting that these gains are **indeed big**. [Kocmi et al., 2024][4] provide a collective understanding of the meaning of differences in different metrics, emperically showing how big a metric difference is reflected in human annotator judgments. According to their finding, improvements of **≥ 1.99 in XCOMET or ≥ 0.94 in COMET** scores correspond to at least **90%** estimated agreement in human judgment—both of which are achieved by our method. To some extent, this aligns with our human annotation in EN-ZH and EN-RU, where the win rates are also high, e.g., 106 (win) / 32 (loss), and 93 (win) / 44 (loss).\n2. Secondly, each metric has its own downside; COMET series scores are the same. The training data for COMET models comes from human annotations of each year's WMT data, which means they may be effective only within a limited range of translation quality, as WMT submissions are generally not bad. For example, [Lo et al., 2023][5] show that the COMET series struggles to evaluate translations when quality falls below a certain level. Accordingly, we are not sure, but it might also lack of ability to measure a system that outperforms previous years' systems, e.g., our method's output. We are not sure whether it fully reflects our system improvement. That is also a motivation for us to conduct human annotation.\n3. Broadly speaking, although not perfect, using COMET-like neural metrics as quality signals for calibration is a successful practice, at least for mid- and high-resource languages that are well supported by current neural models. A potential improvement could be to combine multiple metrics to better exploit their complementary strengths and mitigate individual biases. Incorporating recent advanced metrics, such as LLM-based MQM, may also offer additional benefits. That said, we acknowledge that our method may not generalize well to certain low-resource languages, such as Bhojpuri, where neural metric support is lacking. Addressing these challenges will require collaborative efforts from the broader community, particularly in advancing evaluation metrics, which lie beyond the scope of this paper.\n\n[4] Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies\n\n[5] Metric Score Landscape Challenge (MSLC23): Understanding Metrics’ Performance on a Wider Landscape of Translation Quality\n\n---\n\nThank you again for your insightful suggestions on our paper. We hope that our additional experiments and explanations address your concerns. We will incorporate our discussion into the next version. Please don’t hesitate to let us know if further clarification or discussion is needed; we’d be happy to provide it."}, {"review_id": "i2CrsWuY3u", "reviewer": "Reviewer_RNS9", "summary": "The presented paper proposes an additional loss for machine translation fine-tuning of LLMs which directly optimizes the spearman correlation towards neural translation quality metrics. The authors demonstrate that this approach leads to a better calibrated model with superior translation quality that can also serve as a quality estimator based on its hypothesis likelihoods.", "questions": "N/A", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- Interesting finding that we can directly optimize Pearson correlation to not only get a better translation model but also a better QE metric.\n- Following best practices in machine translation e.g. multiple neural metrics and the experiments are comprehensive.\n- Interesting finding that scaling up the calibrated model improves translation quality but not quality estimation\n\n\n### Weaknesses\n\n- l. 150 the statement that the translation quality has not been used in previous works is not exactly accurate, see for example \"[A Preference-driven Paradigm for Enhanced Translation with Large Language Models](https://aclanthology.org/2024.naacl-long.186/)\" (Zhu et al., NAACL 2024).\n- If I understand correctly, the proposed approach uses listwise preference data through the proxy of the $L_{pearson}$ which is not really a fair comparison to e.g. CPO which only leverages a single rejected hypothesis. A more fair comparison would be towards \"[A Preference-driven Paradigm for Enhanced Translation with Large Language Models](https://aclanthology.org/2024.naacl-long.186/)\" (Zhu et al., NAACL 2024) where we leverage the metric scores as replacement for human annotations and optimize directly on listwise rejected hypothesis.\n- The presented approach is only evaluated on the sentence level and doesn't venture into paragraph or document level machine translation which may come with different learning dynamics.\n- l. 218ff. / l.333f.: I don't think it is justified to say that the quality is \"comparable to that of the current top-performing system, that is Tower-70B-v2 equipped with 100-time-sampling MBR/TRR\"as there are still significant gaps towards that from Table 1 which are as big (and sometimes even bigger) than the gain achieved through this approach compared to CPO.", "was_revised": true, "final_justification": "All my concerns have been addressed in the rebuttal specifically with additional experiments around PL, paragraph level results, and clarifications on the comparison.", "rebuttal": "Thanks for your careful reading and insightful suggestions, and for noting our contribution. We summarize and address your concerns as follows:\n\n> \"l. 150 the statement that the translation quality has not been used in previous works is not exactly accurate, see for example \"A Preference-driven Paradigm for Enhanced Translation with Large Language Models\" (Zhu et al., NAACL 2024).\"\n\nWe agree and appreciate the clarification. We had misread the reference, thinking it referred to the original PL model. In fact, they reweight each pairwise comparison by the score difference. Thank you for pointing this out. We will revise our wording accordingly and include a discussion in the background section.\n\n> \"If I understand correctly, the proposed approach uses listwise preference data through the proxy of the Pearson Loss, which is not really a fair comparison to e.g. CPO which only leverages a single rejected hypothesis. A fairer comparison would be towards A Preference-driven Paradigm for Enhanced Translation with Large Language Models (Zhu et al., NAACL 2024) where we leverage the metric scores as replacement for human annotations and optimize directly on listwise rejected hypothesis.\"\n\nThank you for your suggestion. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhu et al., 2024] did not release their codebase, and their implementation is based on weaker models such as LLaMA-1 or Mistral. To address your concern, we apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying calibration to the top-performing translation model, i.e., TowerMistral-7B [1], using their data.\n\nTo ensure comparability, we use the same beam search size of 4. The results are evaluated on the WMT22 test set using COMET as shown in the following table:\n\n|                            | EN-DE | DE-EN | EN-ZH | ZH-EN | Avg. |\n|----------------------------|:-----:|:-----:|:-----:|:-----:|:----:|\n| Mistral-Ins.               | 81.2  | 82.7  | 82.5  | 77.7  | 81.0 |\n| Mistral-Ins. + BEST        | 81.8  | 83.0  | 83.3  | 78.3  | 81.6 |\n| Mistral-Ins. + PL          | 82.9  | 83.4  | 84.7  | 79.3  | 82.6 |\n| **Mistral-Ins. + Calibration** | **83.5** | **84.4** | **85.4** | **80.1** | **83.4** |\n| TowerMistral + BEST        | 88.9  | 85.7  | 89.3  | 83.3  | 86.8 |\n| **TowerMistral + Calibration** | **89.9** | **86.6** | **90.1** | **84.7** | **87.8** |\n\n\\+ Best is supervised fine-tuning using best scoring samples, and \\+ PL is the result of their implementation of the PL model. \\+ Calibration is applying our method. It is easy to see that\n\n1. On the weaker base models, such as Mistral-Ins, **Calibration yields clear improvements** over both SFT and PL. We **provisionally attribute** this gain to Calibration’s broader consideration of factors compared to PL. Specifically: (1) Calibration normalizes both likelihood and quality scores within a group, making the objective invariant to scale and shift. This promotes stable and robust optimization across diverse input distributions. (2) Calibration directly integrates translation quality via the metric function q(⋅∣x), and optimizes at the distribution level using the Pearson correlation. In contrast, [Zhu et al.] introduce reweighting for each pairwise comparison in PL modeling. In extreme cases, this approach may treat a comparison between ratings 2 (bad) and 1 (nonsense) the same as one between 4 (nearly perfect) and 3 (good), which might be overly heuristic.\n2. Even when applying calibration to the top-performing system, TowerMistral, using human-annotated data, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.\n\n> \"The presented approach is only evaluated on the sentence level and doesn't venture into paragraph or document level machine translation which may come with different learning dynamics.\"\n\nTo address your concern, we collect paragraph-level data in four high-resource language pairs (en-de, en-fr, en-zh, and en-ru) from WMT24++ [1] for evaluation, following current representative practices for long-form data [2]. We merge sentence-level segments into paragraphs using the WMT24++ metadata. This process yields 221 paragraphs, each capped at 150 space-separated tokens in English to ensure compatibility with current evaluation metrics [3]. We follow all settings the same as those in Table 1 based on TowerMistral-7B. We report the results in three metrics, i.e., KIWI-XL, KIWI-XXL, and COMET, as follows:\n\n|             | en-de | en-fr | en-zh | en-ru | Avg. |\n|-------------|:-----:|:-----:|:-----:|:-----:|:----:|\n| SFT (KIWI-XL)    |  68.3 |  65.5 |  69.0 |  66.2 | 67.2 |\n| CPO (KIWI-XL)        |  67.9 |  66.1 |  69.8 |  66.6 | 67.6 |\n| **Calibration (KIWI-XL)** |  **69.4** |  **66.6** |  **70.2** |  **68.4** | **68.6** |\n|             |       |       |       |       |      |\n| SFT (KIWI-XXL)   |  77.9 |  77.4 |  76.2 |  75.1 | 76.6 |\n| CPO (KIWI-XXL)         |  77.0 |  78.6 |  77.6 |  76.0 | 77.3 |\n| **Calibration (KIWI-XXL)** |  **79.3** | **79.2** | **78.2** | **78.0** | **78.6** |\n|             |       |       |       |       |      |\n| SFT (COMET) |  80.1 |  79.5 |  82.3 |  80.7 | 80.7 |\n| CPO (COMET)         |  81.1 |  81.5 |  84.5 |  82.9 | 82.5 |\n| **Calibration (COMET)**  | **82.1** | **81.9** | **84.3** | **83.8** | **83.0** |\n\n\nIt is easy to see that our calibration method consistently outperforms both SFT and CPO across all metrics, demonstrating its effectiveness in paragraph-level evaluation. That said, document-level evaluation remains an open challenge in current MT research. Practical measures are still under development and need to account for broader aspects such as consistency. These issues are beyond the scope of this paper and are left for future work.\n\n[1] WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects\n\n[2] Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts\n\n[3] Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level\n\n> \"l. 218ff. / l.333f.: I don't think it is justified to say that the quality is \"comparable to that of the current top-performing system, that is Tower-70B-v2 equipped with 100-time-sampling MBR/TRR\"as there are still significant gaps towards that from Table 1 which are as big (and sometimes even bigger) than the gain achieved through this approach compared to CPO.\"\n\nLet us clarify the comparison. In Table 1, our model, ​**TowerInstruct-Mistral-7B + Calibration**​, compared to ​**Tower-70B-v2 + MBR/TRR**​, achieves scores of 72.4 vs. 72.3 in en-de, 73.9 vs. 74.5 in en-es, 73.6 vs. 74.2 in en-ru, and 72.6 vs. 72.6 in en-zh. **On average, it slightly lags behind Tower-70B-v2 + MBR/TRR (73.1 vs. 73.4). However, our model outperforms CPO across these four directions by +1.0 (73.1 vs. 72.1)**, and by +1.0 across all directions (73.9 vs. 72.9).\n\nIt is worth noting that Tower-70B+MBR/TRR is far from practical: (1) the base model is 10 times larger than our setting, and (2) MBR/TRR requires 100 times sampling followed by reranking. In contrast, our results rely solely on efficient MAP decoding, using beam search with a beam size of 5.\n\n---\n\nThank you again for your careful reading of our paper and for your insightful suggestions and questions. Hopefully, our response addresses your concerns. We will incorporate this discussion in our next version. Also, we are more than happy to provide further explanation or discussion if needed."}]

Author rebuttal:

Meta-review: The paper proposes to calibrate NMT systems’ hypotheses likelihood to translation quality by directly optimizing the correlation between the two during training.This results in improved translation quality across many settings and language pairs for translation-specialized LLMs..

Reviewers note that the paper is easy to follow and clearly written (jW1Y, jW1Y, bA3p), the approach has good sample efficiency (jW1Y, FndS) and the experiments are comprehensive and show clear improvement in multiple settings (jW1Y, FndS, bA3p, RNS9). Reviewer RNS9 also note that the paper provides interesting and novel observations (calibration improves quality and QE, scale improves quality but not QE).

During rebuttal, the authors provided several additional comparisons in response to the discussion about the lack of baseline comparisons to prior works (SliC, PO) raised by reviewers (bA3p, RNS9, FndS), sensitivity to hyperparameters (FndS),  generalization of approach in new settings (low-resource LPs and paragraph-level data) raised by jW1Y, RNS9, which have significantly strengthen the  submission.

Given the methodological clarity, the empirical thoroughness (both in the original and updated results), and the relevance of the problem addressed, the consensus among reviewers lean toward a positive assessment, which I agree with.
Decision: Accept (poster)