Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
Authors: Beier Luo
Shuoyuan Wang
Sharon Li
Hongxin Wei
Venue: NeurIPS 2025 poster ()
Keywords: Confidence Calibration
LLM
Primary area: general_machine_learning

Abstract: Post-training of large language models is essential for adapting pre-trained language models (PLMs) to align with human preferences and downstream tasks. 
While PLMs typically exhibit well-calibrated confidence, post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs, which can undermine reliability in critical applications.
A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks. 
To address this, we propose Disagreement-Aware Confidence Alignment (DACA), a novel unsupervised method to optimize the parameters (e.g., temperature $\tau$) in post-hoc confidence calibration. 
Our method is motivated by the under-confidence issue caused by prediction disagreement between the PLM and PoLM while aligning their confidence via temperature scaling. Theoretically, the PLM's confidence underestimates PoLM's prediction accuracy on disagreement examples, causing a larger $\tau$ and producing under-confident predictions. DACA mitigates this by selectively using only agreement examples for calibration, effectively decoupling the influence of disagreement.
In this manner, our method avoids an overly large $\tau$ in temperature scaling caused by disagreement examples, improving calibration performance.
Extensive experiments demonstrate the effectiveness of our method, improving the average ECE of open-sourced and API-based LLMs (e.g. GPT-4o) by up to 15.08$\%$ on common benchmarks.

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
Beier Luo', Shuoyuan Wang', Sharon Li,, Hongxin Wei!*
Department of Statistics and Data Science, Southern University of Science and Technology Department of Computer Sciences, University of Wisconsin-Madison
Abstract
Post-training of large language models essential for adapting pre-trained lan- guage models (PLMs) to align with human preferences and downstream tasks. While PLMs typically exhibit well-calibrated confidence, post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs, which can undermine reliability in critical applications. A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks. To address this, we propose Disagreement-Aware Confidence Alignment (DACA), novel unsupervised method to optimize the parameters (e.g., temperature T) in post-hoc confidence calibration. Our method is motivated by the under-confidence issue caused by prediction disagreement be- tween the PLM and PoLM while aligning their confidence via temperature scaling. Theoretically, the PLM's confidence underestimates PoLM's prediction accuracy on disagreement examples, causing a larger r and producing under-confident pre- dictions. DACA mitigates this by selectively using only agreement examples for calibration, effectively decoupling the influence of disagreement. In this manner, our method avoids overly large in temperature scaling caused by disagreement examples, improving calibration performance. Extensive experiments demonstrate the effectiveness of our method, improving the average ECE of open-sourced and API-based LLMs (e.g., GPT-4o) by up to 15.08% on common benchmarks.
Introduction
Post-training has been critical procedure to ensure large language models (LLMs) generate helpful, honest, and harmless responses [Weng et al., 2023, Kumar et al., 2025]. While post-trained language models (PoLMs) perform well on various downstream tasks [Achiam et al., 2023, DeepSeek-Al and et al., 2025], their reliability and trustworthiness still remain an open challenge. In principle, reliable LLM should not only demonstrate high confidence in its correct generations but also exercise caution in uncertain situations [Thirunavukarasu et al., 2023, Dahl et al., 2024]. Previous studies [Achiam et al., 2023, Zhu et al., 2023] show that post-training, especially RLHF [Christiano et al., 2017, Stiennon et al., 2020], compromises the well-calibrated confidence estimation of pre-trained language models (PLMs), resulting in over-confidence issues of PoL.s. This gives rise the importance of confidence calibration for PoLMs, ensuring the confidence score associated with the generation should reflect its ground truth correctness likelihood.
Compared to expensive training methods, post-hoc calibration methods such as temperature scaling [Guo et al., 2017] are more practical for LLMs due to their high efficiency [Shen et al., 2024, Xie et al., 2024]. However, primary challenge of post-hoc calibration methods is their dependence on labeled data. In practice, generating reliable labeled dataset for tasks such mathematics problem solving and medical diagnosis is particularly challenging and time-consuming due to the high level of
"Corresoondig author (weihx @ sustcchh.du.com
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

domain expertise required. Such difficulty is further compounded by the fact that temperature scaling cannot perform effectively given limited labeled data [Mozafari et al., 2018, Liang al., 2020]. In contrast, unlabeled data is ubiquitous in real-world deployment scenarios and easy to collect without requiring human intervention. This creates an underutilized resource: vast amounts of unlabeled data are already available during LLM operation, yet are not leveraged for calibration. Thus, this paper studies an unexplored and practical perspective: How can we achieve effective confidence calibration for PoLMs using unlabeled data in unsupervised manner?
To calibrate PoLMs without relying on labeled data, we introduce Disagreement- Confidence Aligmment (DACA) a simple and effective post-hoc method that leverages the well-calibrated confidence scores of PLMs. A natural starting point of our idea to align the confidence of PoLMs with that of PLMs on an unlabeled validation set, minimizing the divergence between the predictive distributions of the PLM and PoI.M over all samples. However, we find that this direct confidence alignment can paradoxically lead to under-confidence in the PoL-M-when the two models disagree on prediction, the PLM's confidence often underestimates the actual correctness of the PoLM's output. Our theoretical analysis reveals that such prediction disagreement can drive the optimization to increase the temperature parameter excessively, further exacerbating the under-confidence issue. Motivated by our theory, DACA mitigates this issue by decoupling the influence of disagreement examples from the confidence aligmment process. Specifically, it optimizes the temperature parameter using only agreement examples- those where the PLM and PoLM make identical predictions. This ensures that confidence alignment occurs only when the PLM's s scores are a trustworthy proxy for conrectness. As result, DACA yields more conservative and reliable temperature estimates, avoiding the calibration failures of naive alignment (see Figure 2b).
Extensive experiments with both open-sourced and API-based LLMs on common benchmarks demonstrate the effectiveness of the DACA method for confidence calibration. Notably, DACA achieves performance comparable to labeled temperature scaling, even in the absence of labeled data. For example, DACA improves the average Expected Calibration Error (ECE) of the Gemma-3-12BB Instruct model [Team et al., 2025] across 57 subjects of the MMLU dataset [Hendrycks et al., 2021] 1], reducing it from 23.68% to 8.60% In comparison, TS only reduces the ECE to 9. 9.75%. Importantly, DACA is applicable even in scenarios where post-trained and pre- traited models differ in architecture, making it more efficient for the calibration of large-scale PoLMs. For instance, DACA reduces the ECE of GPT-4o [Hurst et al., 2024] from 21.23% to 6.99% when calibrated using the pre-trained Gemma-3-12B model on the MedMCQA dataset [Pal et al., 2022]. Furthermore, our method can be applied to open-ended guestion-answeringg tasks and offers benefits for selective classification. Codes are publicly available at https://githubb.com www.wa..cco/mmmmaaao /ml hwwmmmmmsstttttteee Saadd-ssseccDDDsagreeeee Aware-
We summarize our contributions follows.
1. We show that the well-calibrated outputs of PLMs unlabeled data can be leveraged to calibrate PoLMs. Theoretically, we demonstrate that prediction disagreement can impair calibration performance when directly aligning the confidence of PLMs and PoLMs.
2. Our proposed post-hoe method DACA, formalizes the confidence calibration problem by harnessing the target-specific unlabeled data in the wild. This formulation offers strong practicality and flexibility for real-world applications.
3. We empirically show that DACA enhances the calibration of both open-sourced and API- based PoLMs across various datasets. Moreover, our method applies to open-ended QA tasks and enhances selective classification.
2 Preliminaries
2.1 Confidence Calibration for LLMS
In this work, we focus on the confidence calibration problem of question answering for the Post- trained Language Model (PoLM), denoted as f. Our method primarily targets Multiple-Choic Choicce QA (MCQA) and can be extended to open-ended QA. For MCQA with choices y (A, B, C. D) and prompt æ, let 2x)x) E IR >ll
be the logits of f. The predicted probabilities and confidence are
exp (z),) (x))
pf (y = i x)
P(x) max p jEy
(1)
Eyey exp (2 (x))

---

Pre-trained Version
Pre-trained Version
Pre-trained Version
Pre-tra ned Version
Quiputs
Ouipus
Gap
Gap
Gap
6/+
In structt Version
Instruct Version
Chat Version
Chat Version
003: 19, .000%%
ECE: 20 54%
ECE: 20.81%
ECE: 28.13%
Accurao
Accuraor
Confidence
Confidence
Confidence
Confidercc
(a) Llama-3- B-BB
(b) Qwen-2.5-7B
(c) DeepSeek Vi-LLie
(d) Yi-1.5-BB
Figure I: Reliability diagram evaluation for pre-tran pre-trained post-trained models across four modern
LLM architectures on MMLU [Hendrycks et al., 2021]. The post-trained models are trained by multiple post-training techniques, including SFT, RLHF, and DPO. More reliability diagrams of various post-training methods are provided in Appendix A.
with prediction (x) arg max, Py(y = x). We denote the n-th prompt in dataset by In = In,t, In,ty: Zn,t,, which is a sequence of in tokens, with its corresponding response denoted as yn. Formally, given prompt perfectly calibrated model satisfies,
Pr(Y Y P 3) = 3, vB E [0, 1].
(2)
where Ý = arg max, p(y (=) is the predicted response, and P = max, ply x) is the corresponding confidence score [Guo al., 2017].
To quantify the degree of miscalibration, expected calibration error (ECE) [Naeini et al., 2015] is defined as [E[ = r = 3) 31]. which measures the difference between confidence and accuracy. An empirical estimate of ECE is calculated by partitioning N samples into G bins (bi, b2....,ba according to the confidence predicted by the model. The ECE is then formulated as
ECE Σ 1byl N
Jaco(by) conf(b,)),
9=1
where acc(b,) and conf(b,,) denote the average accuracy and confidence within bin by. respectively. A smaller ECE indicates better calibration performance of the model.
Post-hoc calibration methods aim calibrate model after training. Among these approaches, Platt scaling [Platt et al., 1999] based approaches are commonly adopted due to their low complexity and efficiency, including temperature scaling (TS) [Guo et al., 2017] and its extensions [Mozafari et al., 2018, Kull et al., 2019]. In particular, given a miscalibrated model f. TS introduces a temperature parameter soften the model's predicted probability: p(y = i|ær, 7) (f()))7)) where (-)) denotes the softmax function and 7 > O for all classes. The optimal temperature value for the target dataset by minimizing the negative log-likelihood (NLL) on labeled calibration dataset
D* = {In, Dn V is given by:
arg min (-E [log((y|z,T)] r>0
Temperature scaling simplifies matrix (vector) scaling [Guo al., 2017], where single is applied to all classes, offering great calibration performance while maintaining minimal computational complexity [Guo al., 2017, Minderer et al., 2021].
2.2 The effects of LLM post-training
The success of large language models (L.MMs) has led to standardized training paradigm of pre- training followed by post-training. Post-training refines pre-trained language models (PLMs) for
3

---

specific tasks through techniques such as fine-tuning [Ziegler et al., 2019, Wei et al., 2022], alignment [Peng et al., 2023, Su et al., 2023, Bai et al., 2022], knowledge adaptation [Dong et al., 2022, Rubin et al., 2021], and reasoning enhancement [Yao et al., 2023]. While post-training improves task performance, often comes at the cost of degraded calibration-introdueee model's predictions. In contrast, PLMs typically exhibit more accurate confidence estimates [Achiam et al., 2023, Zhu et al., 2023]. Formally, in multiple-choice tasks, we denote the pre-trained LM as f + Rx, where k is the number of choices. Through post-training, we learn a post-trained language model (PoLM) g : X + R² We present the reliability diagram multiple PoLMs on the MMLU dataset in Figure 1. The diagram illustrates that PoLMs consistently exhibit wer-connfdence, with confidence scores notably higher than the true likelihood of correctness.
overconfidence in the
Post-hoc calibration techniques like temperature scaling mitigate overconfidence effectively but rely on labeled validation datasets. Generating a reliable labeled dataset for tasks like mathematical problem-solving and medical diagnosis is challenging and time-consuming due to the required domain expertise. However, under limited labeled data, the calibration performance of post-hoc methods cannot be guaranteed. Leveraging unlabeled data for confidence calibration offers promising solution for ensuring reliable model behavior in resource-constranned settings. Given the inherently well-calibrated property of PLMs, a natural question arises: Can we leverage the well-calibrated confidence scores of PLMs on unlabeled data to calibrate over-confident PoLMs?
3 Motivation and Method
To leverage the well-calibrated confidence scores from PLMs, an intuitive approach is to align the confidence levels of PoI.Ms with those of well-calibrated PLMs on an unlabeled validation set. A naive approach for confidence alignment is to modify the objective in traditional temperature scaling on an unlabeled validation set D {æx; ) Instead of minimizing the negative log-likelihood. we minimize the Kullback-eeibler (KL) divergence between the predictive distributions of the pre- trained and post-trained language models on D. Formally, given the post-trained model g, the optimal temperature T" on D is given by
& Σ (x) log
(x (,(o()/TT)
T" = arg min E EZED T>0
(5)
Here, a() denotes the softmax function, and (x)) is the i-th element of the softmax probability of (f()) of model f. For convenience, we refer to this approach as "naive confidence alignment".
Naive confidence alignment leads to under-confidence. In Figure 2a, we show that the naive confidence alignment can lead PoLMs to become significantly under-confident,, indicating that their predicted confidence underestimates the actual accuracy. In the following, we investigate why confidence alignment-scaled PoLMs tend give under-confident predictions. Our analysis suggests that the prediction disagreement introduced by post-training can be a culprit.
Prediction disagreement between two models and g refers arg max; fi(x) + arg max; (x(x)) on the same input prompt x. For convenience, we denote the examples with the existence of prediction disagreement disagreement examples. It is known that post-traiming training techniques frequently alter the PL'M's output distribution, resulting in prediction disagreement. Formally, the unlabeled data can be characterized by the Huber contamination model [Huber, 1992] follows:
Definition 3.1 (Unlabeled data distribution). We define the unlabeled data be the following mixture of distributions
P unlaabeeed (1 T) agree + TPP diss
(6)
where (0.1] denotes the disagreement ratio, IP Pagree and dis the marginal distributions of agreement examples and disagreement examples, respectively. In practice, т >> O, as post-training typically changes some PLMs' predictions.
With the above definition, we assume the unlabeled dataset D is i.i.d. sampled from the mixture distribution P Puntabeled- In the following, we analyze the limitations of naive confidence alignment in the presence of prediction disagreement.

---

Beeerat Peoperent
Bewwel
117 080
150
201 200
150
Camputa Socnity
Duligg: Chanslly
College Chemstry
(a)
(b)
Figure 2: Under-confidence issue of naive confidence alignment. (a): Reliability diagram for Yi-1.3-
9B-Chat the computer security and college chemistry subjects of MMLU [Hendrycks et al., 2021]. Results of more models are presented in Appendix D. (b): Temperature values of Yi-1.5-9B-CM under different training epochs when trained separately the disagreement and agreement sets and the whole dataset. The training process is performed on the computer security and the college chemistry subject of MMLU.
Proposition 3.2. Assume f()) be a perfectly calibrated predictor with ECE, = 0 and g(.) denote a predictor perfectly aligned the predictor f. Let I be the unknown label of sample The expected calibration error (ECE) of g over the unlabeled distribution IP uullabeled:
ECE, Pantoiee [farg f((x) '0} - 1 farrgmax gx))) 01]
The proposition's proof presented in Appendix The proposition illustrates that the ECE of PoLM cannot reach zero even in an ideal case where the PoLM is perfectly aligned with perfectly calibrated PLM in confidence, due the existence of prediction disagreement. Intuitively, PLM's confidence for disagreement examples reflects its own prediction's accuracy, instead of that of Pol prediction. Since post-training typically improves PoL.M' accuracy, PLM's confidence level will be lower than the prediction accuracy of PoLM, resulting in the under-confidence issue. In the following, we further analyze how prediction disagreement impacts the parameter r temperature scaling as an example.
Proposition 3.3. Given a sample x. let g(x:) denote the output logits of post-trained language model, and p(x) denote the softmax probability from the pre-trained language model. If arg max, gi( and (f (x))) < E. then the optimal temperature is given by:
arg min Dxr. (xr(p(()) ( (((())/r))=== 00.
The proof of this proposition is provided in Appendix B. Proposition 3.3 indicates that the gradient of the KL divergence w.r.t the temperature remains positive on the disagreement set, which increases the value of continuously during optimization. Consequently, the optimization will further exacerbate the under-confidence issue. To provide a straightforward view, Figure 2b shows the temperature dynamics during training exclusively on the disagreement set, revealing a gradual increase to a significantly high value.
Disagreement-Aware Confidence Alignment. In our previous analysis, we showed that disagree- ment examples tend to drive the temperature parameter to excessively high values, leading to an under-confidence issue. To address this problem, our key idea is to decouple the influence of dis- agreement examples from the confidence alignment process. We propose Disagreement-ANe Aware Confidence Alignment (DACA), which eliminates the gradient of the KL divergence with respect to the temperature on disagreement examples, thereby ensuring that temperature optimization is guided solely by agreement examples. Formally, the new loss function of DACA can be defined as:
L(T:0) = 1 = - -')- Σ px(ar) log (g(g())/7)
Pi(x)
where ý = arg max; ((():) and j = arg max; gi (2r) denote the predictions of the pre-trained model and the post-trained model g. respectively.
5

---

Minimizing the loss function in Equation (7) mitigates the under- coneidenneeecce issue effectively. We illustrate with an example in Figure 2b, which demonstrates that optimizing the temperature solely on the agreement set yields a more conservative estimate than optimizing on the whole dataset.
Extensions to other post-hoe calibration methods. Notably, our method is general and can be easily incorporated into other existing post-hoe calibration methods such as vector scaling and matrix scaling [Guo et al., 2017]. Formally, for any rescaling function dg with parameter 9, we can formulate the method as follows. First, we define the i-th softmax probability of the post-trained model after rescaling qi (z; 0) a ( f(x))): The corresponding probability of the pre-trained model is given by (x.). Then, the optimization objective can be formulated as:
Pi( q: (:(3;
0* arg min BEED
1 0') Epi(t) log
(8)
where ÿ = arg max; (z) and s' = arg max; 9: (2x) denote the predictions of the pre-trained model and the post-trained model, respectively. We present the calibration performance of our method with vector scaling and matrix scaling in Appendix D.3.
4 Experiments
4.1
Setup
Models. We conduct extensive experiments on diverse LLMs, including both open-source models and those accessible via online APIs. For open-sourced LLMs, include Llama-3 family [Grattafiori et al., 2024], Gemma-3 family [Team et al., 2025], Qwen-2.5 family [Yang et al., 2024], and Yi-1.5 family [Young et al., 2024]. Unless explicitly stated otherwise, we perform calibration using the pre-trained counterpart of each post-trained LLM. The above models are provided by Hugging Face. To scale up our findings, we also evaluate large-scale LLMs accessed through online APIs, such as GPT-4o [Hurst et al., 2024] and DeepSeek-V. [Liu et al., 2024a].
Datasets. To verify the effectiveness of our proposed methods, employ three common datasets for evaluations, including: MMLU [Hendrycks et al., 20 McdMCQA [Pal et al., 2022], and MathQA [Amini et al., 2019]. For MMLU, we learn specific temperature parameter for each subject using subject-specific validation set. The datasets are provided by Hugging Face. Due to limited space, detailed information about each dataset is presented in Appendix C.
Compared methods. Since our method is the first unlabeled post-hoc approach to calibrate LLMs without training auxiliary models, we exclude many existing calibration methods that rely on labeled data and additional training. To compare with other unlabeled calibration approaches, we select three prompt-based methods baselines, including CAPE [Jiang et al., 2023]: prompt- based method that calibrates next-token probabilities by permuting option order to mitigate LLM biases, Elicitation [Tian et al., 2023]: estimates confidence by prompting the model to generate verbalized probabilities, Fiecitation-Ensembll [Xiong et al., 2023a]: improves upon this by aggregating outputs from multiple prompts. Specifically, Vanilla represents the calibration performance of LLMs without any calibration techniques applied, and Temperature Scaling (TS) leverages labeled data from the test task to tune task-specific temperatures and is included supervised reference baseline.
Evaluation metrics. We evaluate the calibration performance using the following metrics: (1) Ex- pected Calibration Error (ECE) [Naeini et al., 2015]: measures the average error between prediction confidence and accuracy across different confidence intervals. For evaluation, we use 10 bins in evaluation. (2) Maximum Calibration Error (MCE) [Nacini et al., 2015]: measures the largest discrepancy between prediction confidence and accuracy across all confidence bins, reflecting the worst-case calibration scenario. (3) Adaptive ECE (AECE) [Nixon et al., 2019]: proposes new binning strategy that uses an adaptive scheme to space the bin intervals, ensuring that each bin contains equal number of examples. (4) Brier Score [Brier, 1950]: directly measures the distance between the model confidence and the binary correctness label of the generation.
Implementation details. For multiple-choice datasets, the model estimates the probability that the next token matches one of the options (e.g., A, B, C, or D), reflecting its confidence. Due to the space limitation, more details of implementation are provided in Appendix C.

---

Table I: Average calibration performance across 57 MMLU subjects for several contemporary PoLMs. "Vanilla" refers to the uncalibrated model. indicates calibration methods with access to labels. Best results are shown in bold, and the second-best results are presented italies. Detailed results for broader range of I LLMs are available in the Appendix D.2.
Metrics
Models
Methods
BCE %())
MCE MCE%((
ABCE (6())
Brier Seore
Vanilla
16.383+00433
18.190+1.547
24.90++D.667
0.179-0.003
CAPE
11.524
31.741.0.152
17.614.10.048
0.157
Qwen3-88
Elicitation
16,714+0:214
66.884+
27.568--2 897
Elieitaiion-Bseebble
16,475=0:407
44.991
20,515:22.55 394
Ours
4.393-0.222
-3.370
12.601#00.117
0.144:0.001
8.655=0.220
28 108::1.730
14.547::0666
0.140=0000
Vanilla
23.679=00522
48 5..50011..30 ..884
35 886-: 257
0.235=0.005
CAPE
13.906:+02209
32.8 020830+007000
19.278-+0.377
0.168=0.001
Blicitation
25.464==0:877
766 16.000+115 15
3.733
Gemma-3-1 Hostttttt
Elicitation- Easemble
25.417=20,244
42.011=: 10.256
32.221##1988
Ours
8.596-0.380
27.022++3.335
13.551+008004
0.154-0.002
TS
9.745=0.361
29. 29.80412.750
03.60V:10.339
0.159-2,003
Vanilla
16,200
33.819 452
20.353::0.666 6664
0.199=0.005
CAPE
10,251+0:2289
22.759++0.665
1212+00012
0.179=0.001
Elicitation
27.152=66.513
83,000+ 000
49,211
Vi-LS-33BB-Caat
Elicitation- Ensemble
Decitation-Eesemblle
23.954:7.4857
61.487111.487
39.259913.049
Ours
9,465=0.174
19.89811.082
11.700:00411
0.174=0.004
TS"
0.592=0.170
28.599111.77
12.553
0.073
Vanilla
12.870:20:483
36.873111415
23.837:10.760
0.143
CAPE
9.346=0.122 122
30.903++
0.125=0.001
Elicitation
11.227
60.00012 14,142
21.237=#1.08
Bumma-3-70B-Imm
Elicitation
Diicttatoon-Enssmblle Ensembbl
16.632.10.068
70.066.128.
21.790.16.979
Ours
7.844=0.418
24.275::1.285
03,158200488
0.120=00001
TS
8. 8.360=0.283
27. 1666::17775
14.928=0.686
0. 0.120000002
4.2
Main results
DACA significantly improves the calibration performance of PoLMs. Table presents the average calibration performance of the baselines and our method across 57 subjects of the MMLU datasets, with four contemporary LLMs. The validation set is the validation split of each subject in MMLU on Huggingface, where the size of the validation set is limited. A salient observation is that our method effectively mitigates the mis-calibration in various models across all metrics and even comparable to the labeled TS with limited validation data. For instance, our method improves the ECE of Llama-3-70B- 3-770-Insstuuc from 12.870% to 7.84%.. Similarly, it improves the ECE of the latter released Qwen3-8B from 16.383% to 8.566%. It is worth noting that the verbalization- such as Elicitation and Dlicitation-Ensemble based method, which is consistent with the results reported in previous work [Shen et al., 2024]. We further evaluate our method on additional datasets, including MedMCQA and MathQA, as shown in Appendix D.2. Our method can also be extended to vector and matrix scaling, with results shown in Appendix D.3, demonstrating improved calibration across these post-hoc methods.
method, performs significantly worse than the next-token logits-
DACA is effective across models of different sizes. We also verify the calibration performance of the baselines and our methods from models of different sizes. In Figure 3, our results indicate that our approach is effective with different-sized LLMs and achieves impressive performance across diverse architectures. Notably, the Vanilla ECE decreases monotonically with increasing model scale, trend that aligns with the conclusions drawn in previous research [Zhu et al., 2023].
DACA agnostic to the choice of PLMs. In practice, many closed-source, large-scale PoLMs (e.g., GPT-4o and DeepSeek-V3) are accessed via APIs. As such, calibrating these API-based models becomes essential. However, these models typically lack accessible pre-trained versions, and their large scale requires significant computational resources. Our method effectively calibrates both API-based and large-scale PoLMs, as well as smaller models. Specifically, we use three small-scale calibrate GPT-4o and DeepSeek- V3. As
LL.Ms-Llamaa--881 Qwen2.5-7B. and Gemma-3-122 shown Table 2, our method consistently improves the calibration performance of GPT-4o regardless of the PLM choice. For example, DACA reduces the ECE of GPT-4o from 21.231% 1% to 6.993% using
7

---

14
126
instruct Family
Fomily
instruct Family
instruct Familly
Figure 3: ECE comparison between our methods and baselines on MedMCQA across varying contemporary LLM families and parameter sizes.
Table 2: Calibration performance of DACA for GPT-4o using various pre-trained models on MedM- CQA. "Vanilla" refers to the uncalibrated model. ECE represents the original ECE of pre-trained models. Best results are shown in bold.
Methods
Pre-trained Models
Metrics
ECE %
ECE %6(4]
MCE %61)
AECE %(1)
Brier Score(
Vanilla
21.231
35.218::4:260
27.619:11.61
0.2162200003
Llama-3-88 3-8B
9.450+0,77 777
7.984:00.997
10.640=0.4133
6.879:50.737
0.150:10.001
Qwen2.5-78
10.467-+042
0.150+00001
Gemma-3-1226
4.424
6.993:00.900
10.057=0,115
6.115:007877
0.148:00.00
Gemma-3-122 3-BBB. While the calibration performance PoLMs is similar when scaled with the three PLMs, we find that better-calibrated PLMs yield lower ECEs after alignment. We provide the detailed calibration results for DeepSeek- V3 in Appendix D.4.
Is our method effective with different post-training strategies? To demonstrate that our proposed method is agnostic to the post-training strategy, we conduct experiments on a diverse set Llama- 3.1-8B models post-trained with different techniques and report the results in Table 3. We the models released by Ai2 on Hugging Face?. The results show that our method consistently improves calibration performance across all tested post-traine post-training strategies. For example, DACA reduces the calibration error of the model post-trained with SFT and DPO from 25.19 193% to 5.418%. Additional results post-trained models with different post-training techniques are provided in Appendix D.5.
Discussion
Can DACA be applied to open-ended QA tasks? Previous works estimate confidence scores in open-ended question answering (QA) tasks by reformulating the free-form QA problem into a multiple-choice format [Shen et al., 2024, Kapoor al., 2024]. Specifically, they pose binary "Yes" or "No" question to language model, asking whether its own generated answer is correct or incorrect. This approach, commonly referred to P(True) in the hallucination detection literature, serves as a well-known baseline. Following prior works, we also adopt the P(True) approach to obtain confidence scores for experiments. Formally, the confidence score of model f sample defined as p( Yes x. f). We then define the prediction disagreement between models ff and g in open-ended QA tasks arg max; (c f) A arg max; p; (a( g), where i (1,2).
Figure 4 illustrates the calibration performance of our method on the TruthfulQA datasets [Lin et al., 2021], evaluated across models of varying sizes from the Qwen2.5 and I.LaMA-3 families. Specifically, we use Qwen2.5-32D and LLaMA-3-70B as the pre-trained models to calibrate the corresponding post-trained models within each family, The results demonstrate that method consistently reduces calibration error across different models. For example, DACA reduces the vanilla ECE from 30.955% 5.244% Owen2.5-325 Instruett highlighting its applicability to open-ended QA tasks. Detailed results, including additional metrics, are provided Appendix D.6.
DACA benefit selective classification. Selective classification [Geifman and El-Yaniv, 2017] leverages model confidence to decide whether to make a prediction or abstain, thereby improving
Phtpss/ttuggingfaaccc/aaalel

---

Table 3: Calibration performance of DACA and baselines on MedMCQA across different post- post-training techniques applied to Llama-3. 3.-e-....BB.... "Vanilla" refers to the uncalibrated model, while "Oracle TS" represents a lower bound achieved by temperature scaling with access labeled data from the test task. Best results are shown in bold.
Metrics
Post-trinning Techniques
Methods
ECE %(4)
MCE %())
AECE %(()
Brier Score(
Vanilla
04.850=0.857
19.893=1.736
14.289 649
0.237:400004
SFT
CAPE
7,533-+0.334
12.323=1.268
7.898=0.224
0.210=20.00
Ours
4.573==0.410
10.000=0.000
4.812=0.800
0,213+0.001
Vanilla
25.120=00,953
29.381=1.534
22.413=#1.55 1.387
0.28240.004
SFT + DPO
CAPE
15:576-0.325
19.765+1.314 1.314
14.867-40.855
0.233++0.001
Ours
5.418:=0.354
10.000=0.000
4.961
0.212:40.01
Vanilla
25.193+
30.836-11 598
22.447-22 447-+2.532
0.282-40.00 0.05
SFT + DPO + RLVR
CAPE
15.729=0.363
20.621=110 1.03
14. 14.960-20.0 925
0.234 001
Ours
5.988-00448
+00.00
0.212-40001
Varill
Oun
40-
20-
30
240 241
330
3.0
Qween S-nsstact Family
Stme
Llemachiinetrde Family
Size
Meta: 3-70--rriiu
Owar2.5-728 vntruct
Figure 4: ECE of DACA with different LLMS for the open-ended TruthfulQA benchmark. The lower ECE indicates better calibration per- formance. Detailed results with more models are provided in Appendix D.
Figure 5: Selective classification accuracy MedMCQA across different models. Accuracy reported on subsets of examples with con- fidence scores above thresholds ranging from 0.55 0.95.
reliability by trading off coverage for higher accuracy on accepted examples. This is particularly important when using LLMs for decision-making, where unreliable predictions can lead significant downstream consequences. Although temperature scaling is accuracy eccuracy-preserving -preserving by design, cali- brated confidence scores can nonetheless enhance selective classification by enabling more reliable abstention decisions, thereby improving accuracy on the retained subset.
In Figure 5, present the accuracy comparison of baselines and our method under varying confidence thresholds ranging from 0.5 to 0.95, where predictions with confidence below the threshold are rejected. A salient observation is that confidence scores calibrated by method significantly exceed the original accuracy at every confidence threshold, demonstrating improved reliability in selective classification. Notably, the performance gains become increasingly pronounced as the confidence threshold rises. This is attributable to our method's ability to mitigate over-confidence issues, thereby improving the model's accuracy on high-confidence predictions.
Related work
Post-training in LLMs.. Post-training in Large Language Models (LLMs) is critical phase that refines models after their initial pre-training [Tiie et al., 2025, Kumar et al., 2025], where they learn general language patterns through next-token prediction on vast datasets. LLMs undergo structured enhancement process that typically follows a sequential order. Initially, fine-tuning is employed to adapt the pre-trained model specific tasks domains. This step involves updating the model's parameters using curated datasets, which significantly improves its performance on targeted tasks [Yue et al., 2023, Luo al., 2023]. To optimize resource efficiency, parameter- efficient fine-tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA) and Adapters [Hu al., 2022, Gao et al., 2023, Luong et al., 2024], are often utilized. These methods adjust only small subset of the model's parameters introduce a limited number of trainable parameters, achieving comparable performance to full fine-tuning while significantly reducing computational and memory requirements. Following this, reinforcement learning (RL) techniques applied to
the post-training phase,

---

further refine the model's behavior. Methods such as Reinforcement Learning from Human Feedback (RLHF) [Ouyang al., 2022] and Direct Preference Optimization (DPO) [Rafailov al.. 2023] incorporate dynamic feedback to optimize decision-makinn and align the model's outputs with user preferences. Together, these strategies transform L.MMs into versatile, user-aligned tools for diverse applications. In this work, we address the confidence calibration problem in Post-trained Language Models (PoLMs) by leveraging well-calibrated Pre-trained Language Models (PLMs). Our method aligns the confidence scores of PoLMs with PLMs samples where both models produce the same prediction.
Confidence Calibration. Confidence calibration has been widely studied to ensure that the confidence levels output by models accurately reflect their true performance. To achieve this, the state-of-the-an calibration methods can be categorized into two paradigms: post-hoc methods [Platt et al., 1999, Guo et al., 2017, Mozafari et al., 2018, Kull et al., 2019, Xiong et al., 2023b, Wang et al., 2024] and regularization methods [Müller et al., 2019, Mukhoti et al., 2020, Hebbalaguppe et al., 2022]. For post-hoc calibration, a representative method is temperature scaling [Guo et al., 2017], which learns a single scalar for rescaling the logit. Recently, several studies have investigated calibration in LLMs (Jiang et al., 2023, Xiao et al., 2022, Chen et al., 2022, Liu et al., 2024b], highlighting that post- training often leads to overconfidence. One line of work explores fine-tuning methods to encourage well-calibrated numerical and linguistic verbalized confidence [Lin et al., 2022, Kapoor et al., 2024, Tao et al., 2025], while another focuses on training auxiliary models to predict model confidence [Kadavath et al., 2022, Liu et al.. 2024b, Ulmer et al., 2024] or estimate temperature parameters for unseen tasks [Shen al., 2024]. However, these approaches typically require labeled data and, in some cases, are computationally expensive. Other works [Xie et al., 2024, Tian et al., 2023) examine post-trained LLMS and show that carefully designed prompts can elicit better-calibrated uncertainty estimates. Distinct from prior approaches, our work is the first to leverage unlabeled data for post-hoc confidence calibration, offering both efficiency and flexibility.
7 Conclusion
In this paper, we introduce Distance-Aware Confidence Alignment (DACA), an unsupervised post- hoc method designed to calibrate overconfident PoLMs. To the best of our knowledge, this is the first approach that uses unlabeled data for the post-hoc calibration of LLMs. The core idea behind DACA is to decouple the influence of prediction disagreement when aligning confidence between PoLMs and well-calibrated PLMs. Specifically, DACA optimizes the temperature parameter using only agreement examples- those in which the PLM and PoLM make identical predictions senfiction-eenuuinngg that confidence alignment occurs only when the PLM's scores serve as a trustworthy proxy for correctness. Extensive experiments demonstrate the effectiveness of DACA in calibrating PoLMs across a wide range of models and common datasets. This method be easily adopted in practical settings, it can be applied to both open- open-sourced and API-based LLMs and is computationally efficient.
Limitations. Our method involves an additional inference step using pre-trained models, leading to modest increase in computational cost. Additionally, filtering out disagreement examples may reduce the pool of unlabeled examples available for calibration. However, this trade-off is generally acceptable, given the wide availability of unlabeled data. Future work could explore how to leverage these disagreement examples to further improve calibration.
Acknowledgment
Beier Luo and Hongxin Wei are supported by the Shenzhen Fundamental Research Program (Grant No. JCYJ2023080709 1800000.. We gratefully acknowledge the support of the Center for Computational Science and Engineering at the Southern University of Science and Technology for our research.
References
Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. Findings of the Association for Computational Linguistics: EMNLP, 2023.
10

---

Komal Kumar, Tajamul Ashraf. Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Mina-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.23321, 2025.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
DeepSeek-Al and et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting, Large language models in medicine. Nature medicine, 29(8): 1930-1940, 2023.
Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1):64-93, 2024.
Chiwei Zhu, Benfeng Xu, Quan Wang, Yongdong Zhang, and Zhendong Mao. On the calibration of large language models and alignment. Findings of the Association for Computational Linguistics: EMNLP, 2023.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30, 2017.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3023 30208000212220000 2020.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017.
Maohao Shen, Subhro Das, Kristjan Greenewald, Prasanna Sattigeri, Gregory W Wornell, and Soumya Ghosh. Thermometer: Towards universal calibration for large language models. In International Conference on Machine Learning, 2024.
Johnathan Xie, Annie S Chen, Yoonho Lee, Eric Mitchell, and Chelsea Finn. Calibrating language models with adaptive temperature scaling. arXiv preprint 2rXi
2024.
Azadeh Sadat Mozafari, Hugo Siqueira Gomes, Wilson Leão, Steeven Janny, and Christian Gagné. Attended temperature scaling: a practical approach for calibrating deep neural networks. arXiv preprint arXiv: 11586, 2018
Gongbo Liang, Yu Zhang, Xiaoqin Wang, and Nathan Jacobs. Improved trainable calibration method for neural networks on medical imaging classification. arXiv preprint arXiv:2009 240557, 2020.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa : A large-scale multi-subject multi-choice dataset for medical domain question answering, 2022.
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015.
11

---

John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
Meelis Kull, Miquel Perello Nieto. Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in Neural Information Processing Systems, 32, 2019.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34:1 6682-15694, 2021.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv 2019.
preprint arXiv:I909.00
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du Andrew M Dai, and Quoe V Le. Finetuned language models are zero-shot learners, leamnrrs, In International Conference on Learning Representations, 2022.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4, arXiv preprint arXiv:2304.03277 2023.
Yixuan Su, Tian Lan, Huayang Li. Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305. 16355, 2023.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0580) 2022.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey in-context learning. arXiv preprint arXiv:2301.0022 2022.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning retrieve prompts for in-context learning. arXiv preprint arXiv:2112.0863: 2021.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36:1 1809-11822. 2023.
Peter J Huber. Robust estimation of location parameter. In Breakthroughs in statistics: Methodology and distribution, pages 492-518. Springer, 1992.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint arXiv:2407.21788 2024.
An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arX1v:2412.15115. 2024.
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, al. Yi: Open foundation models by 01. ai. arXiv preprint 04564 2024.
Aixin Liu, Bei Feng. Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation- based formalisms. arXiv preprint 1905 2019.
12

---

Mingjian Jiang, Yangjun Ruan, Sicong Huang, Saifei Liao, Silviu Pitis, Roger Baker Grosse, and Jimmy Ba. Calibrating language models via augmented prompt ensembles. ICML 2023 Workshop on Deployable Generative AI, 2023.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305. 14975, 2023.
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Ilms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306. 13063, 2023a.
Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR workshops, 2019.
Glenn W Brier. Verification of forecasts expressed terms of probability. Monthly weather review, 78(1):1-3, 1950.
Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, and Andrew Wilson. Calibration-tuning Teaching large language models know what they don't know. In Proceedings of the Ist Workshop on Uncertainty-Aware NLP (UncertaiNIP 2024), pages 1-14, 2024.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human 2021.
falsehoods. arXiv preprint
Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. Advances in Neural Information Processing Systems, 30, 2017.
Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, et al. A survey post-training of large language models. arXiv preprint arXiv:2503.06072. 2025.
Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, al. Disc-lawIlm: Fine-tuning large language models for intelligent legal services. arXiv preprint arXiv:2309, 11325, 2023.
Yun Luo. Zhen Yang, Fandong Meng. Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747, 2023.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, et al. lama-adapter v2: Parametee-effficient visual instruction model. arXiv preprint arXiv:2304.1501 2023.
Trung Quoe Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning, arXiv preprint arXiv:2401.08967,3 3, 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems,
2023.
Miao Xiong, Ailin Deng, Pang Wei W Koh, Jiaying Wu, Shen Li, Jianqing Xu, and Bryan Hooi. Proximity-informedd calibration for deep neural networks. Advances Neural Information Pro- cessing Systems, 36:68511 1-66588,, 2023b.
13

---

Shuoyuan Wang, Jindong Wang, Guoqing Wang, Bob Zhang, Kaiyang Zhou, and Hongxin Wei. Open-vocabulary calibration for fine-tuned clip. arXiv preprint arXivv2402.04665, 2024.
Rafael Müller, Simon Korbllihh, and Geoffrey Hinton. When does label smoothing help? Advances in Neural Information Processing Systems, 32, 2019.
Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss. Advances in Neural Information Processing Systems, 33:15288-153 15299, 2020.
Ramya Hebbalaguppe, Jatin Prakash, Neclabh Madan, and Chetan Arora. A stitch in time saves nine: A train-time regularizing loss for improved neural network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16081-16090, 2022.
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. arXiv preprint arXiv:2210.047144 2022.
Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, and Heng Ji. A close look into the calibration of pre-trained language models. arXiv preprint 2211022111.01111, 2022.
Xin Liu, Muhammad Khalifa, and Lu Wang. Liteab: Lightweight language model calibration over short- and long-form responses. In The Twelfth International Conference on Learning Representations, 2024b.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv: 14334, 2022.
Linwei Tao, Yi-Fan Yeh, Minjing Dong, Tao Huang, Philip Torr, and Chang Xu. Revisiting uncertainty estimation and calibration of large language models. arXiv preprint wxXv:22555.23854 2025.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint ar *xXxv:220700222) 2022.
Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. Calibrating large language models using their generations only. arXiv preprint XXiv:2403.00975 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our experimental results consistently support the claims presented in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of this work in the conclusion section 7.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
We provide the proof of our theoretical result in the Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide the detailed implementation details reproduce our results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We will release the code of our work.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nipp cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wwtppcccpppccceeccccccccccccccat /tneeeeeudeeeeeeeeeiiiiiiii ssssssooiccyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We provide the data split information in Appendix and other information in implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Our results are provided mean and standard error format.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are provided in Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https:
Answer: [Yes]
Justification: Yes, we affirm that our research adheres to the NeurIPS Code of Ethics in all aspects, including considerations related to data usage, transparency, and potential societal impact.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss about the societal impacts of confidence calibration the introduc- tion. What's more, we also discuss why we need to leverage the unlabeled data performe confidence calibration.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not involve the release of model and dataset. Our models and datasets are all from Hugging Face.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The dataset and model we are both open-sourced on Hugging Face. Our code is based on PyTorch and vLLM.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cow datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will upload our code in the supplement material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy btttp::/neeuips... htt.r/ppscc//oooffeeeee////2255 for what should or should not be described.
21

---

Table 4: Post-trained LLM summarization. "Source" refers the URL indicating the origin or provider of the post-trane tstintraind LLM.
Post-training Techniques
Chmm-3. 30mm--3--Tolo---8--5555
SFT
httgpe / httgr///gggggggeeeeeeooo co/aal lenat Liana --t------------------- 1-Tuto-3-88-----0 88----------11
Llama 3.1 3.T-Tul--3 3-----3-SD--- DPO
SFT+DPO
httepe: /hnggingt ace ht//aalletat/aaaeeee LLaana 3. T-Tuio-3 88-00---------0
8B-DPU
L.ams-3.--ullae L-Tuln-3-SB
SFT4DPO-RIVR
hetpe /hogg now. / /Lalam- 1.t.ma---------see 1-Tulu-3-88
hesiive-DO
httppe/ /hogg higfaee Open
ter-dpo-
Llam- Brse- 8B SFT-IPO
SFT-IPO
bttge htteee///ggggggge htt.//gggggggggg.. ace co/ corpptt noeson
laane SPF-
Llmma-3-8---ell Seiff--nssuss--00 100K
Self-Instract
hangee Puug lngface cay Magpi 3- 3-88-Set Instrrcct- 100K
Appendix
A Over-confidence issue with more post-trained models
In this section, present evidence that post- training lead overconfidence issues with additional PoL.Ms, as illustrated in Figures and 7. We summarize the PoL.Ms, along with their post-training technologies and source websites, in Table 4.
Meta-L lama-3.1-a-
1ita-Llama-3.1188
Meta-Llama-
Cutputs
Gap
4.50%
Pre- -re-trained
ECE: 4 4.50%
Accuracy
Llama-3 Llama-3.1-Tulu-3 3.1-Tulu-3-8B-57
Llama-3.1 3.1-T..----U-----100
Uama- 3.1-Tulu-3-8B
20.27%
Post-trained
Accuracy
Confidence
Confidence
0.2
0.4
1.0
Figure 6: Over-confidence issue of various post-trained Llama-3.1-8BB
22

---

Meta-Llama-3-8
Meta-Ulama-3-a
Meta- Lamma-3-8
Outsuts
Gap
ECE: 3.52%
ECE: 3.52%
Pre-tre -re-trained
Accuracy
Llama-3-8B-ttrr-PPO
Baaa---Base-a Llama-3-8B-Ser S-rttt-------tttucctttt 100K
ECE: 19.45%
ECE: 30.25%
ECE: 13.96%
Post-trained
Accuracy
Confidence
Confidence
0.0
0.2
0.4
Confidence
0.6
0.8
1.0
Figure 7: Over-confidence issue various post-trained Llama-3-8B.
B
Theoretical Proof
B.1 Proof of Theorem 3.2
Proof. First, we review that the ECE is defined as
ECE P
= rIp= 3) - |
Then given dataset 'D = {xi, Vi J. the ECE of f is given by
ECE, Fntrass ()((x) 1 (arg max fi(x) 81}
where (z is the confidence score of ff on sample æ. In the same way, the ECE of g is given by
ECE, = Eup--Fratteess (x)) farg max max (x()) = 0)
where (x(x) the confidence score of g on sample æ. Since the confidence level of g is aligned with f. we have that
Eg-Taatted [py(x) (2)())] = 0.
B.2 Proof of Proposition 3.3
Proof. The KL divergence between the distribution p(x) and the model distribution (g(x)/77) is given by:
& px(x)
Dkr(pl=))l(((())//) Σ (x) log
(gg(T)/
where
e
dgg(x//7)
Σ ea;(a)/7
Our goal is to show that DxL(o())) is minimized -
23

---

First, note that the KL divergence can be expressed as:
DkL(pl=)ll((0(()//)) = H
where
H(p(z)) = Σ logP:))
is the entropy of p(z), constant, and
o(ll)) =
Σ pı(z) log
the cross-entropy. Therefore, minimizing DRL (xz(())))lo(((()))) with respect to 7 is equivalent to minimizing the cross-entropy H(p(t), Next, we analyze the behavior of o(g(x)/) as varies:
As T + 0: Since c = arg max g(r), (g(x)/T), + 1 and (g(x)//)) - 0 for c. If
pi(x) > 0 for some + then - log (g(x)/T); co, implying (()()))((((()/7) 00.
As T + 0o: (g(()/T)) + for all i, since (x)/TT + O. Thus,
H((()))((9())/11) + Σ pr(o)) log
(1) log
Now, for finite r > 0, since ge(x)) > (x( for i * (assuming a strict maximum for simplicity), we Given that P=(x) < f. the model distribution (g(x)/TT assigns more probability to class c than the uniform distribution for finite т, while the true distribution p(x)) assigns less than uniform class
have (g(2)/T)) 1. with equality only
To see why the minimum occurs at r 0o, consider that as T increases, d(g(x//T) approaches the uniform distribution, which reduces the cross-entropy by making a (g(()/T), closer to E. Since
P=(x) L. and typically (.) for i # are such that the uniform distribution provides better approximation than a distribution concentrated on c, the cross-entropy decreases as T increases.
More formally, one can consider the derivative of Hp((x), ((g()/T)) ((p(a)))(g(())//)
with respect to r, but the limit behaviors suffice establish that (p(xx), a (g(x)/T) /T) is minimized as T co. Specifically, since 0 and H (p(x), a (g(x)/T)) + log k as
(p((), (g(x)/T) oo ( is continuous and decreasing in T, the infimum is achieved as T > oc.
and assuming
Therefore, the temperature parameter that minimizes the KL divergence is:
00.
C Implementation details
Experiment details. We experiments NVIDIA GeForce RTX 4090 and NVIDIA L40 GPU, and implement all methods by PyTorch and vLLM.
Optimizer details. For both TS and DACA, use the Adam optimizer with batch size of 256, learning rate of 0.05, and train for 400 epochs.
Datasets details. For the main experiments, we apply confidence calibration to each of the 57 subjects from MMLU and report the average of the calibration metrics. Specifically, we use the validation split of each subject as the validation set. For the MMLU datasets, we conduct five experiments with five different prompts to calculate the mean and standard deviation of the results, as the validation and test splits are predetermined. We provide the choices of the prompt in Table 5. For other datasets, we use first prompt and report the mean and standard deviation over five random splits of the validation and test sets, with
ratio of 7:3.

---

Table 5: Variants of multiple-choice question instructions.
ID
Prompts
The following multiple-choice questions. Give ONLY the correct option, other words explanation:
[Question] A: [Option 1] B: [Option 2] C: [Option 3] D: [Option 4] Answer: [Mask]
2
Answer the following multiple choice questions by selecting ONLY the option:
[Question] A: [Option 1| B: [Option 2] C: [Option 3] D: [Option 4] Answer: [Mask]
3
For each of the following multiple choice questions, provide just the correct letter:
[Question] A: [Option 1] B: [Option 2] C: [Option 3] D: [Option 4] Answer: [Mask]
Select correct answer each of the following questions:
[Question] A: [Option 1]
B: [Option 2] C: [Option 3] D: [Option 4] Answer: [Mask]
5
Chouse the right option each multiple-choice question below. Respond with letter only:
[Question] A: [Option 1| B: [Option 2] C: [Option 3] D: [Option 4] Answer: [Mask]
Meta-LLama-3.J. 1ataamm-3--88--nstrure
Mistral-7B-Instrucee 78innsrruutts
DeepSeek-V2 Lite-Chat
Cutputs
FCF: 11 .8%
FCF: 18, 90%
14.10%
Accuracy 8
Confidence
Confidence
Confidence
Figure 8: Under-confidence problems of naive confidence alignment with more L.MMs.
D Detailed results
D.1
More under-confidencce results of naive confidence alignment
We present the reliability diagram of more models scaled with naive confidence alignment on the MMLU dataset in Figure 8.
D.2 Extended results across diverse models and datasets
The performance our method on more datasets and models. We present the average calibra- tion performance with more models across 57 subjects of MMLU in Table 6. In addition, we compare the calibration results of our method and baseline approaches on MathQA and McdMCQA in Table 7 and Table 8, respectively. The results show that method significantly reduces the miscalibration of PoLMs and achieves performance comparable to TS, which has access to labels. For instance, our method reduces the ECE of Seappeeee----Liee---e on MedMCQA from 26.553% to 1.715%, while TS reduces to 1.800%.
D.3 Extension to vector scaling and matrix scaling
We present the results of applying the DACA extension with vector scaling (VS) and matrix scaling (MS) on MedMCQA Table Across all models, our method consistently reduces the calibration error, regardless of whether VS or MS is used. For example, reduces the ECE from 21, 21.700% to 4.1 133,, which is comparable to the oracle VS result of 4.558% Similarly, DACA+MS lowers the ECE to 4.407%, closely matching the oracle MS result of 4.201%.
DACA+VS
25

---

D.4 Results of additional large-scale PoLMs
We present the results of our method on DeepSeek-V3. V3 with various PLMs in Table 10. Across all PLMs. our method consistently reduces calibration error. A similar trend is observed, where the lower ECE of the pre-trained model leads to lower ECE the scaled post-trained model.
D.5 Results of additional post-training techniques
To evaluate the effectiveness of our method, we perform experiments with more post-trained models, each trained using different post-training techniques. The specific post-training methods applied to each model are listed in Table 4. We present the calibration performance results in Table 11.
D.6 Detailed results for open-ended tasks
For open-ended tasks, conduct experiments with Qwen2.5 family and Llama-3 family the TruthfulQA datasets. For the Qwen2.5 family, we choose Qwen2.5-32B as pre-trained models to calibrate all size post-trained models. And for the Llama-3 family, choose Llama-3-708 as pre-trained models to calibrate all size post-trained models. We present the detailed results in Table 12 verify the effectiveness of our method.
26

---

Table 6: Average calibration performance across 57 subjects of MMLU on several modern LLMs. "Vanilla" denotes the performance without any calibration applied. represents the calibration method with access labels. Best results are shown in bold, and the second-best results are presented in italics.
Metrics
Models
Methods
ECE %((L
MCE %(()
ABCE %(()
Brier Score
Vanilla
Elicitation
72.867
Qwen2.5-7B Instructt
ElicitationEnneebble
43.145
23.635
Ours
23.254
TS
8.738
26.747
13.720
0.157
Vanilla
17.810
36.636
22.848
0.211
CAPE
15.436
31,476
19.726
0.199
Flicitation
26.524
28.009
18.211
Llama-3-8U-Inae
Dlleitation-Fnsemblle
29.548
19.794
34.334
Ours
9.485
21,650
12.120
0,176
TS!
8.335
25. 260
12.245
0.174
Vanilla
37.794
24.717
Elicitation
38.6366
Llama-3.1 I-Tulu-3-8B
Elicitaioo Ensemble
38.636
TS!
8.475
22.336
11.914
0.170
Vanilla
24.717
41.613
28.256
0.259
CAPE
13.183
27.348
16.761
0.197
Elicitation
38.769
44,550
21.719
Yi-1.566B-Chat
Elicitation Ensemble
31.504
39.339
25.478
Ours
9.208
21.059
12.459
0.187
TS
8,998
34,684
13.084
0,188
Vanilla
22.010
40,400
28.689
0.228
CAPE
37.144
16.205
Elicitation
$7.500
Yi-L-5-9B-Cha
Elicitation Ensembblle
22.405
47.619
19.640
22.951
28.165
13.619
0./71
Vanilla
24,860
41,401
27.878
0.259
CAPE
13.473
26.200
16.899
0.198
Elicitation
39.840
43.549
26.308
Searral7B--Inttret-tee
Elleitation-Ensemble
34.754
50.000
29.318
Ours
9.260
18.385
11.554
0.186
TS!
8.634
31.25
12.100
0.185
Vanilla
20. 184
34,147
22.303
0.246
CAPE
10.219
22.348
13.745
0.197
Eliccitaionn
24.483
44.466
22.999
DeepSeek V2-Lite Chat
Eliciation-Easeebbà
27.773
34.314
23.342
Ours
26.590
12.605
39,242
0.207
27

---

Table 7: Calibration performance of MathQA on several modern LLMs. "Vanilla" denotes the performance without any calibration applied. represents the calibration method with access to labels. Best results are shown in bold.
Metris
Models
Methods
ECE %%()
MCE (
AECE %)(L
Brier Score(
Vanilla
35.825 1.0.26
40.369 40.369 10.571
29,209 10,716
0.21910.001
Qwen2.5-7B-Insuee
Ours
5.823-10.345
22.524 2,420
12.589 -07339
0.218 0.001
Oracle TS
5..11
20.243-:1881
12.611:10880
0.21910.001
Vanilla
32.849:0.255
36.634=0.535
28.628=2.630
0.339:0.002
@wen2.5.144 I
2.795:103341
10.000:10.00
4.527=0.166
0.220:0.001
Oracle
5.223+0.236
13.88 13.86100.669
8.048+0.311 316
0.213-00.01
Vanilla
22.239+00.720
28.376+1.181
21.232-+1.99 1.190
0.25740.004
Ours
3.659-+0.390
10.001+0.003
4.528=006188
0. 0.199+0.001
Qwen2.5:32B Instruct
Oracle TS
10,31
10.00010.000
4.655
0.19610.002
Vanilla
30.6641:0346
32.900=0.2274
24.656:1.3318
0.31810.003
Q
Ours
4.127:10.887
10.000
0.298
0.216.10.001
Oracle TS
3.472:10.275
10.000=0.000
4.4:4420.223
0.212:10.011
Vanilla
15.186110:475
26.623100.939
17.473:10.59
0.246:10.002
Qwen2.5-Mo 1aan2.55Math-BBBB 1--Mat--B--Istmee
Ours
7.491=0.757
19.40566.336
10.084=0.911
0.225+0.001
Omclle TS
3.094-00559
20.324-0.2113
8.892--5.444
0.219-0.001
Table 8: Calibration performance MedMCQA on several modern LLMss "Vanilla" denotes the performance without any calibration applied. represents the calibration method with access to labels. Best results are shown in bold, and the second-best results are presented italics.
Metrics
Models
Methods
ECB %[[)
MCE %6[)
ABCE %6[]]
Brier Score(
Vanilla
21.81420.325
26.232+0.915
26.030=3.116
0.237=0.002
CAPE
13.488+00.20
19.813+00889
15.006=1.677
087=000.001
Elicitation
69.021 +0.0014-0.004
70.262::0.449
53.556=1.88 1.814
Qwen2:5-77B-Instreee
Elicitation- Easemble
73.151=:0.000
79.505:20.08 303
11.361177772
Ours
3.938:2.2277
10.000:20.000
4.891:0.683
0.173=0.001
4.11350.266 267
10.000:200000
0.9999000 594
0. 0,174=0.00 00/
19.814+0.433
22.311++13118
103=1.355
0.217-0.003
14.272=0.161
1.07
38529220.356
0.18:00.001
Llama-3-70B Instructttt
Elicitation
63.62920.048
71.6782:0377
49.057=3.004
Elicitation Ensemblle
71.14710.300
88.88517.A88 8599
41.940-55.35
Ours
3.4241.0.229
10.000-0.0000
4.406
0.163
TS!
3.640110.341
10.00010.000
4. 4.48210.731
0.163-10.001
Vanilla
26.55310.389
35.517:10.120
23. 23.724=00 460
0.311=00001
22.41411.1176
20.567710.116
0.280-0.001
DeepSeek. Veee-eee-VV--ite......
Elicitaiion
18.19210.1182
75.173
44.333 1.003
Elicitation Ensemble
Ellictation-Ensemhhlle
63.350:--..435
91.2199:0.480
19..134::: 1.107
Ours
1.715-00357
33.521 1..211+2..009
5. 1.994-11.177
0.229-0.001
TS!
1.806-00366 332
11.506:33001
3.0944:0.755
1.229-00.00
28

---

Table 9: Average calibration performance of the DACA extension with vector scaling and matrix scaling on MedMCQA with various models. "Vanilla" denotes performance without any calibration. 1 denotes methods that are accessible to labels. Best results are shown in bold, and the second-best results are presented in italics.
Metrics
Models
Methods
ECE %1
MCE %6
AECE %1
Brier 1
Vanilla
19,399::0.522
22.564 1.356
19.574:10.79
0.215:100004
Ours+VS
3.838:1.3366
10.000=0.000
5.286=0.831
0.1641:0
Maama-3-70B-Insuueee
Ous+MMS
3.734:10.413
10.007=0.135
1.698:1.706
0.161100003
VST
0.9582
10,000 - 0.000
5.685--0.879
0.162 000022
MST
3.823++0.484
10.618- 1.236
6.022-+0.877
0.162-H0.002
Vanilla
21.720=:0,5 502
28.676=
03.4113:0.555 546
0,235++0000 004
Ous++VS
4.133:10.555
10.130=0.20
5,880
0.175:10.002
Owen2.5-72B-Inssee
Ous++MS
4,407=10.665
10.086=0.171
5.904=1.383
0.173-100002
VS
4.558:20.769
10.387=0.775
7,038:E 1.00 1.054
0.174-0.000 002
4.201=00.595
10.416=0.832
6.527=1.196
0.174+00 002
Vanilla
28.914++1.00
31.296-
24, 980-+11 767
0,303-H0
Ours+VS
4.833=#0.792
10.000=00000
5.55|==0.917
0,209=00.00 003
Gemma-3-221 3-27B-Inntructt I-Issrruct
Ours+MS
4.614:10.987
10.000=0.000
5.904
0.207
0-.592
10.142
5,203 1.046
0.202-+0.000
MSI
6.969-0.866
0.199-0.003
Table 10: Calibration performance comparison of DACA with different pre-trained LLMs on MedM- CQA for DeepSeck-V3. V3. "Vanilla" denotes the performance without any calibration applied. Oracle TS serves as the lower bound since it has access the labeled data for the testing task, and ECE" represents the original ECE of the pre-trained model. Best results are shown in bold.
Metrics
Methods
Pre-trained Models
ECE %
ECE %6[1
MCE %(()
AECE X(1
Brier Score())
Vanilla
20.473::0:449
29.668=1.5588
22.518=00.64
0.217
Llama-3-88 SB
9.45010,777
7.127:0.085
11.04720.131
6.098100.055
0.161
Qwen2.5:7B
6.990.10.100
6.071::0.056
Gemma-3-112
4.424+0.696
6.721+00078
10.722-0.074
5.835=00072
0.160=0.001
Table 11: Calibration performance of MedMCQA of Llama-3-88 post-trained with various techniques. "Vanilla" denotes the performance without any calibration applied. represents the calibration method with access to labels. Best results are shown in bold.
Metries
Post-training Techniques
Methods
ECE %(L)
MCE %()
AECE %(4)
Brier Score ())
Vanilla
16.22510.455
0.200100002
SFT
CAPE
14.286=0.131
18.219=0.472
14.001=0.913
0.227+:0002
Ours
6.969-10.255
0.000
6.849:10.532
0.218:10.001
Vanilla
23.332::0.261
28.756=0.591
1.5014:11.592
0.272-10.003
Ierattiv-DPPO
CAPE
19.719=0.167
24.740=1.129
19. 12.2933
0.247+0.002
Ours
6.925-10.200
10.000=0.000
6.701
0.214:10.001
Vanilla
16.98120.181
21.222=0.915
1.37
0.242-10.00
Self-Instruct
CAPE
16.379=0.304
18 18.927=0.855
06.228=008522
0.231+00001
Ours
7.209:10.366
10.466=0.536
7.211=0.914
0.214:1.001
29

---

Table 12: Calibration performance TruthfulQA with several contemporary LLMs. "Vanilla" denotes the performance without any calibration. Best results are shown bold.
Metries
Models
Methods
ECE %([)
Brier Score(
NLL(J)
Vanilla
29.870:11.017
0.348100007
1.155=0.024
Qwen2.5-7B-Instee Instrucct
Ours
6.245-+0.974
0.253++0.002
0.701=0.004
Vanilla
47.229=#08447
0.479+000007
5.333=00122
Qwen2.5-14B Instruct
Ours
25.423::0.833
0.331:00.006
0.918::0.016
Vanilla
30.955::0.814
0.359=0.0006
1.256=00.025
Qwen2.5-32B-Instruu
Ours
5.244:10800
0.252:1.002
0.698
Vanilla
66.189-11.05
0.464
2.889
Qwen2.5577B-Imstrruct
Ours
17.540-1.916
0.277
0.754=0.006
Vanilla
37.61510.966
0.39170.007
1.422=0.040
-lama-3-8B-Instruec
Ours
11.233-11.064
0.271-1.0033
0.739=0.000
Vanilla
42.495++1.160
0.430-10.013
3.363
Llama-3-70B Instructt
Ours
17.001++1.179
0.278+00000
0.761-0.014
30


Reviews:
[{"review_id": "CUkUdCmPCl", "reviewer": "Reviewer_8hAp", "summary": "This paper propose Disagreement-Aware Confidence Alignment (DACA), a novel unsupervised method to optimize temperature for post-hoc confidence calibration. They theoretically demonstrate that the confidence of pre-trained language models underestimates the prediction accuracy of post-trained language models on disagreement examples and therefore propose DACA to address this issue. The empirical results show that DACA enhances the calibration of both open-sourced and API-based PoLMs across various dataset.", "questions": "Please refer to Strengths And Weaknesses.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is well-written and well-structured. The related work section clearly outlines the relationship between existing approaches and the proposed method. The authors first theoretically demonstrate that the confidence of pre-trained language models underestimates the prediction accuracy of post-trained language models on disagreement examples. Then they propose a simple yet practical post-hoc calibration method that can be applied to most existing calibration techniques to enhance performance. Therefore, the proposed method is well-motivated. The authors showcase the superior performance of their proposed methods on multiple popular language models and use appropriate and sufficient evaluation metrics to validate the results. I appreciate how the authors highlight their key findings, such as \"DACA significantly improves the calibration performance of PoLMs\", \"DACA is effective across models of different sizes\", \"DACA is agnostic to the choice of PLMs\", and \"DACA can benefit selective classification.\" These concise summaries effectively emphasize the advantages of their proposed approach. Overall, the proposed approach is both theoretically sound and empirically significant. One potential weakness as noted by the authors is that the disagreement examples are filtered out, which reduces the pool of unlabeled examples available for calibration. It would be helpful if the authors could discuss more about how solve the issue.", "was_revised": true, "final_justification": "I maintain my positive score as the authors have addressed most of the concerns.", "rebuttal": "# Response to Reviewer 8hAp\n\nThanks for your positive and valuable suggestions. Please find our response below:\n\n## 1. [W1] The potential weakness of filtering examples\n\nThank you for the thorough review. The filtered examples could potentially enhance calibration performance, offering a promising avenue for future research. However, this limitation does not affect the contribution of this work, which achieves calibration performance comparable to, or surpassing, supervised temperature scaling. Notably, our method still significantly improve the calibration performance even in cases of high disagreement ratio. The table below present the results of DACA using Qwen2.5-7B-Instruct (PoLM) and Qwen2.5-7B (PLM) on the three MMLU subjects with the highest disagreement ratios. The results show that our method can outperform supervised TS, even when there are only four agreement samples. Additionally, a promising direction for utilizing filtered examples is to generate target confidences using specialized agents for confidence alignment, which could be particularly valuable in some extreme cases.\n\n| Subject             | $\\pi$ |Number of Agreement Samples| ECE (%) Vanilla | ECE (%) DACA | ECE (%) TS$^\\dagger$ |\n| :------------------ | :----------------- | :--------------| :--------------| :-------------- | :----------- | \n| Machine Learning    | 0.643     | 4         | 34.428      |   **6.943**    |             11.069  |\n| College Mathematics | 0.643      |   4     | 36.303       |   **9.302**    |         21.313 |\n| Abstract Algebra    | 0.452     |    6     | 24.224        |  13.471 | **10.464** |"}, {"review_id": "gpi7btuqbq", "reviewer": "Reviewer_APYD", "summary": "This paper introduces an unsupervised approach, Disagreement-Aware Confidence Alignment (DACA), to alleviate the overconfidence problem of post-trained language models. Motivated by the intrinsic weakness of temperature scaling, DACA ignores the disagreement samples when optimizing the temperature parameter, demonstrating competitive performance in extensive experiments and shedding light on efficient post-hoc calibration of large language models.", "questions": "1. In the main experiment, 57 different parameters $\\tau$ are learned for each subject in MMLU. What about learning a single $\\tau$ for all subjects? Is DACA robust under domain shift?\n2. Could you specify the values of $\\pi$ that were observed in your experiments? Furthermore, have you investigated whether the performance of DACA degrades significantly when this ratio is very high (e.g., on extremely difficult tasks), where the agreement examples may become insufficient or distributionally biased?\n3. The definition of `agreement` is ambiguous in real-world LLMs use scenarios. How to improve DACA to help calibrate the confidence of chatbots?\n4. Grammar and typos: L.309, You may add an \"is\" before \"computationally efficient\".", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n1. Unsupervised Calibration. As high-quality labeled data is usually expensive to obtain and scale up, DACA explores an unsupervised method for confidence calibration, by innovatively leveraging the well-calibrated confidence of PLMs.\n2. Sufficient Theoretical Analysis. This paper reveals that the failure of naive unsupervised calibration stems from disagreement examples between the two models, thereby providing strong theoretical support for the DACA method.\n3. Extensive Experiments on Classification. DACA’s effectiveness is grounded by sufficient experiments, and it also performs well on (reformatted) QA and selective classification tasks. \n\nWeaknesses:\n\n1. High dependency on the calibration of PLMs. The ceiling performance of DACA is limited to that of the selected PLM. The concept of \"well-calibrated\" PLMs is hard to define.\n2. Like many other post-hoc schemes, DACA learns a single scalar to adjust the model's output distribution, which does not touch on the underlying confidence mechanism of models, and is likely to cause overfitting. On one task, the learned parameter $\\tau$ might be inappropriate to apply to another.\n3. As mentioned by the authors, the disagreement samples are directly eliminated in DACA, which may be critical for better calibration.", "was_revised": true, "final_justification": "After careful consideration of their responses and alignment with fellow reviewers, I will maintain my original rating.", "rebuttal": "# Response to Reviewer APYD\n\nThank you for the constructive feedback. Please find our response below:\n\n## 1. [W1] Dependence on the selected PLMs\n\nYes, we agree that the calibration level of the chosen PLM may affect DACA's performance. However, prior studies [1,2] demonstrate that most PLMs exhibit significantly better calibration than PoLMs, as validated by Figure 1 of our manuscript. This makes it straightforward to choose an appropriate PLM for implementing DACA. Importantly, the excellent performance of our DACA is not tied to specific PLMs as shown in Table 2. Across various PLMs, including Llama-3-8B, Qwen2.5-7B, and Gemma-3-12B, DACA consistently reduces GPT-4o's ECE from 21.23% to below 8%. This highlights that the choice of PLM does not constrain DACA's practical applicability.\n\nRegarding the term \"well-calibrated PLMs\" (line 128), we clarify that it refers to the well-calibrated confidence scores of PLMs compared to those of PoLMs, not to restricting PLM selection (as most PLMs perform exceptionally well). In the final version, we will revise the phrase to \"align PoLMs' confidence levels with those of PLMs\" for greater clarity.\n\n\n## 2. [W2, Q1] Transferability of DACA\n\n\nThanks for raising the concern. To evaluate the transferability of the temperature parameter learned by DACA, we conducted experiments by training a temperature value on the MMLU dataset and applying it to the MedMCQA and MathQA datasets. The results in the table below demonstrate that DACA consistently reduces the ECE of several PoLMs on both MedMCQA and MathQA, which validates the practical transferability of the learned temperature $\\tau$ (despite it might be not the optimal).\n\n\n| Calibration Dataset | Test Dataset | Model               | ECE (%) Vanilla | ECE (%) DACA | Improvement (%) |\n| :------------------ | :----------- | :------------------ | :-------------- | :----------- | --------------- |\n| MMLU                | MedMCQA      | Llama-3-8B-Instruct | 22.943          | **7.908**    | 15.035          |\n|                     |              | Qwen2.5-7B-Instruct | 32.909          | **19.956**   | 12.953          |\n|                     |              | Yi-1.5-9B-Chat      | 34.250          | **17.775**   | 16.475          |\n|   MMLU                  | MathQA       | Llama-3-8B-Instruct | 42.545          | **20.790**   | 21.755          |\n|                     |              | Qwen2.5-7B-Instruct | 37.266          | **17.197**   | 20.069          |\n|                     |              | Yi-1.5-9B-Chat      | 35.977          | **16.697**   | 19.280          |\n\nSecondly, a more practical scenario is **learning a temperaute value on a few tasks and then generalize to other more tasks**, similar in spirit to the setting of OOD generalization [3]. We conducted experiments by randomly selecting 10 subjects from the MMLU dataset for calibration and testing on the remaining 47 subjects. The table below presents the mean results with standard deviations across 10 random trials. The results demonstrate that DACA consistently improves the calibration performance of three PoLMs across the 47 unseen subjects, indicating DACA's robust generalization to new tasks and robustness to domain shifts.\n\n| Model               | ECE (%) Vanilla | ECE (%) DACA   | Average Improvement (%) |\n| ------------------- | --------------- | -------------- | ----------------------- |\n| Qwen2.5-7B-Instruct | 19.02±0.44      | **10.04±0.28** | 8.98                    |\n| Llama-3-8B-Instruct | 20.26±0.36      | **9.39±0.27**  | 10.87                   |\n| Yi-1.5-9B-Chat      | 21.07±0.52      | **9.57±0.44**  | 11.50                   |\n\nAs for learning a single $\\tau$ for all 57 subjects in MMLU, we conduct new experiments by learning a single $\\tau$ on the validation set of all MMLU subjects and present the average results of test set in the table below. The results show that a single $\\tau$ indeed significantly improves the calibration performance of PoLMs on all subjects of MMLU.\n\n| Model               | ECE (%) Vanilla | ECE (%) DACA |Improvement (%) |\n| ------------------- | --------------- | ------------ | ----------------------- |\n| Qwen2.5-7B-Instruct | 18.97           | **9.97**     | 9.00                    |\n| Llama-3-8B-Instruct | 20.22           | **9.40**     | 10.82                   |\n| Yi-1.5-9B-Chat      | 21.18           | **9.85**     | 11.33                   |\n\n## 3. [W3,Q2] The potential limitation of filtering examples\n\nYes, the filtered examples could potentially enhance calibration performance, offering a promising avenue for future research. However, this limitation does not affect the contribution of this work, which achieves calibration performance comparable to, or surpassing, supervised temperature scaling. Notably, our method still significantly improve the calibration performance even in cases of high disagreement ratio. The table below presents the results of DACA using Qwen2.5-7B-Instruct (PoLM) and Qwen2.5-7B (PLM) on the three MMLU subjects with the highest disagreement ratios. The results show that our method can outperform supervised TS, even when there are only four agreement samples. \n\n\n\n| Subject             | $\\pi$ |Number of Agreement Samples| ECE (%) Vanilla | ECE (%) DACA | ECE (%) TS$^\\dagger$ |\n| :------------------ | :----------------- | :--------------| :--------------| :-------------- | :----------- | \n| Machine Learning    | 0.643     | 4         | 34.428      |   **6.943**    |             11.069  |\n| College Mathematics | 0.643      |   4     | 36.303       |   **9.302**    |         21.313 |\n| Abstract Algebra    | 0.452     |    6     | 24.224        |  13.471 | **10.464** |\n\n\nBesides, we report the statistics of disagreement sample ratios $\\pi$ across 57 MMLU subjects using three models in the following table.\n\n| Pre-trained Model | Post-trained Model  | Min | Max | Mean ± StD  |\n| :---------------- | :------------------ | :----- |:----- | :----- |\n| Qwen2.5-7B        | Qwen2.5-7B-Instruct | 0 | 0.6364 | 0.180±0.132 |\n| Llama-3-8B        | Llama-3-8B-Instruct | 0 | 0.5 | 0.218±0.127 |\n| Yi-1.5-9B         | Yi-1.5-9B-Chat      | 0.0952 | 0.6923 | 0.337±0.153 |\n         \n\n## 4. [Q3] Definition of `agreement` in open-ended QA\n\nThank you for the insightful question. In Section 5 (Discussion), we extend the proposed method to open-ended QA tasks. Specically, we employ the P(True) approach [4], where models first generate answers and then assess the probability, P(True), that their responses are correct. Here, \"Agreement\" occurs when both models give consistent \"Yes\" or \"No\" answers. Figure 4 illustrates the effectiveness of DACA in these tasks, notably reducing the Expected Calibration Error (ECE) of Qwen2.5-32B-Instruct on TruthfulQA from 30.955% to 5.244%. Additionally, a potential direction is to define a soft metrics for the `agreement` via measuring the semantic distance between the responses of PLM and PoLM, offering greater flexibility for chatbot applications. This will be included in the final version of our discussion. \n\n\n## 5. [Q4] Grammar typo\n\nThank you for pointing this out. We will revise this typo in our final version.\n\n[1] Achiam, Josh, et al. \"Gpt-4 technical report.\" (2023).\n\n[2] Zhu, Chiwei, et al. \"On the Calibration of Large Language Models and Alignment.\" EMNLP (2023).\n\n[3] Liu, Jiashuo, et al. \"Towards out-of-distribution generalization: A survey.\" (2021).\n\n[4] Kadavath, Saurav, et al. \"Language models (mostly) know what they know.\" (2022)."}, {"review_id": "bmou2PPaVe", "reviewer": "Reviewer_GoW5", "summary": "This paper introduces Disagreement-Aware Confidence Alignment (DACA), an unsupervised post-hoc calibration method for post-trained language models (PoLMs). The core insight is that while pre-trained language models (PLMs) are often well-calibrated, post-training (e.g., SFT, RLHF) introduces overconfidence. DACA proposes to align the confidence of a PoLM to that of a PLM using only agreement examples—inputs where both models make the same prediction. This strategy avoids calibration degradation due to disagreement examples, which tend to skew temperature scaling toward under-confidence. The method is simple, effective, and does not require labeled data. It significantly improves Expected Calibration Error (ECE) across multiple datasets and LLM families (including closed-source models like GPT-4o), and can generalize to other post-hoc techniques such as vector and matrix scaling.", "questions": "1. It can be seen that your results can outperform the TS with access to the ground truth label as shown in table 1, could you give some explanations.", "limitations": "Yes, the limitations are well included in the paper.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The performance is impressive.\n2. The table 2 experiment is interesting, it is a surprise to see such an increase by using other PLMs.\n3. Empirical results are comprehensive across models, datasets (MMLU, MedMCQA, MathQA), and post-training methods (SFT, DPO, RLVR).\n4. Clear experimental baselines and fair comparisons with both supervised and unsupervised methods.\n\n\nWeaknesses:\n1. How you define the confidence distribution of the LLM is important. Should be well defined at the beginning of the analysis. Or people will be lost, how llm expression confidence as traditional models do.\n2. It looks like there are some repeat in Eq6 and Eq7.\n3. To me, the research setting is strange, it is very limited to the white box and even require access to the pre and post training version. Which is hard to obtain in real practice, let alone the multiple inference of the pre trained model. This issue limited the applicability.\n4. The method is limited to multiple option QA.\n5. Table 2 is interesting, I wander if you can run some larger open sources PLMs to achieve even better performance.\n6. Linguistic Confidence is a trending way to express LLM uncertainty, need to be included and discuss [1,2,3]\n7. The method is very limited to the white box and even require access to the pre and post training version. Which is hard to obtain in real practice, let alone the multiple inference of the pre trained model\n\n[1] Revisiting Uncertainty Estimation and Calibration of Large Language Models\n\n[2] Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?", "was_revised": true, "final_justification": "The author has solved most of my concern. However, this work is still limited to limited settings, where training needs the availablility of both pre-trained model and post trained model. But overall, it is a good paper.", "rebuttal": "# Response to Reviewer GoW5\n\nWe appreciate the reviewer for the insightful and detailed comments. Please find our response below:\n\n## 1. [W1] Definition of LLM confidence distribution\n\nThank you for this valuable suggestion. In the main part, we focus on the multiple-choice QA task, where an LLM's confidence refers to the softmax probability of the token(s) corresponding to the chosen answer option (e.g., A, B, C, D) [1,2]. In Section 5 (discussion), we extend our method to open-ended QA tasks and employ the P(True) approach [3], where models first generate answers and then assess the probability, P(True), that their responses are correct. Following the suggestion, we will explicitly introduce the definition in Section 2 of the final version.\n\n\n## 2. [W2] Clarification on the repetition in Eq6 and Eq7\n\nThank you for the detailed review. We clarify that Eq. (6) defines the loss function for DACA's temperature scaling, while Eq. (7) presents the optimization objective for extending DACA to other post-hoc calibration methods, such as vector scaling (VS) and matrix scaling (MS). Specifically, Eq. (6) is a special case of Eq. (7), where the parameter $\\theta$ represents either a scalar temperature $\\tau$ for temperature scaling (TS) or vector/matrix parameters for VS and MS. Without the generalized objective in Eq. (7), the extension to various calibration methods might be ambigurous to readers.\n\n\n## 3. [W3, W7] Clarification on applicability\n\nThank you for raising this concern. Below, we address each point regarding the applicability of our method with clear explanations.\n\n* **\"white-box\"**. This work focuses on calibrating logit-based confidence [1,2,3,4], which is accessible in both open-source and many API-based LLMs (e.g., GPT-4o, Grok-4 and Llama-4-Maverick). Therefore, our method is broadly applicable to various LLMs (including APIs) with access to logits.\n* **\"require access to the pre and post training version\"**. We clarify that the effectiveness of DACA is agnostic to the choice of PLMs (See Tables 2 and 10), so **we do not require access to its pretraining version** for calibrating a PoLM. For example, we can leverage a small-scale, open-sourced PLMs (e.g., Qwen2.5-7B) to calibrate large-scale PoLMs (and APIs) such as GPT-4o and Deepseek-V3. This approach reduces the ECE of GPT-4o on MedMCQA from 21.23% to 7.82%, demonstrating that access to the pretrained version of the PoLM is not required.\n* **\"multiple inference of the pretrained model\"**. We clarify that post-hoc calibration methods, including TS and our DACA, perform optimization (Eq.(7)) during the calibration phase, using a small validation set to tune scaling parameters (e.g., $\\tau$). Our method only queries the pretrained model during this phase, not for every test instance. In deployment, DACA incurs no additional computational cost compared to standard post-hoc calibration methods like temperature scaling.\n\n\n## 4. [W4] Clarification on the limitation to multiple option QA\n\nIn Section 5 (Discussion), we extend the proposed method to open-ended QA tasks. Specically, we employ the P(True) approach [4], where models first generate answers and then assess the probability, P(True), that their responses are correct. Here, \"Agreement\" occurs when both models give consistent \"Yes\" or \"No\" answers. Figure 4 illustrates the effectiveness of DACA in these tasks, notably reducing the Expected Calibration Error (ECE) of Qwen2.5-32B-Instruct on TruthfulQA from 30.955% to 5.244%. \n\n## 5. [W5] Additional results with larger PLMs \n\nThank you for this valuable suggestion. We conduct new experiments by employing larger PLMs to calibrate GPT-4o, and present the results on MedMCQA in the table below (including the original results of Table 2 for convenience). The results indicate that leveraging more powerful PLMs can indeed further enhance the calibration of the PoLM.\n\n| Pre-trained Model | ECE (%)  Vanilla | ECE (%) DACA    |\n| :---------------- | :--------------- | :-------------- |\n| Gemma-3-12B       | 21.231±0.296     | 6.993±0.490     |\n| Gemma-3-27B       | 21.231±0.296     | **3.547±0.057** |\n| Llama-3-8B        | 21.231±0.296     | 7.984±0.397     |\n| Llama-3-70B       | 21.231±0.296     | **4.284±0.052** |\n| Qwen2.5-7B        | 21.231±0.296     | 7.816±0.215     |\n| Qwen2.5-72B       | 21.231±0.296     | **4.190±0.106** |\n\n## 6. [W6] Discussion of Linguistic Confidence\n\nThank you for introducing the related works. We agree that \"Linguistic Confidence\" is an interesting and promising approach to expressing LLM uncertainty. In this work, we focus on calibrating logit-based confidence and reveal the benefits of PLMs in confidence calibration. For future work, it will be an interesting direction to design calibration methods for verbalized or linguistic confidence. We will include a discussion of this concept with proper references in Related work of our final version.\n\n## 7. [Q1] Why DACA can outperform TS with labels?\n\nThank you for the insightful question. We conjecture that this pheonomenon might be related to the size of the calibration set. Previous studies [5,6] have shown that Temperature Scaling (TS) performs poorly with limited calibration data. Table 1 presents the average performance across 57 MMLU subjects, where TS’s scaling parameter $\\tau$ is separately tuned on each subject’s validation set. Many subjects have small validation sets (e.g., $\\leq 20$ samples), leading to degraded performance of TS. To validate this, we report the performance of Qwen2.5-7B-Instruct using TS and DACA across subjects with varying calibration set sizes in the table below. The results confirm that DACA outperforms TS in subjects with limited calibration samples, while TS excels when abundant data is available. This highlights DACA’s superior data efficiency, making it particularly valuable in data-scarce scenarios.\n\n\n\n| Subject                   | Size of Calibration Set| Agreement Samples | ECE (%) Vanilla | ECE (%) TS$^\\dagger$ | ECE (%) DACA |\n| ------------------------- | ----------------------- | ----------------------- | --------------- | ---------- | ------------ |\n| Machine Learning          | 11      |         4       | 34.428          | 11.859     | **6.943**    |\n| College Mathematics       | 11       |           4    | 36.303          | 17.051     | **9.302**    |\n| High School World History | 26        |      25        | 12.114          | **5.404**  | 5.851        |\n|Professional Law          | 170        |      126       | 38.764          | **6.63**   | 7.926        |\n\n[1] Shen, Maohao, et al. \"Thermometer: Towards Universal Calibration for Large Language Models.\" ICML (2024). \n\n[2] Achiam, Josh, et al. \"Gpt-4 technical report.\" (2023).\n\n[3] Zhu, Chiwei, et al. \"On the Calibration of Large Language Models and Alignment.\" EMNLP (2023).\n\n[4] Kadavath, Saurav, et al. \"Language models (mostly) know what they know.\" (2022).\n\n[5] Mozafari, Azadeh Sadat, et al. \"Attended temperature scaling: a practical approach for calibrating deep neural networks.\" (2018).\n\n[6] Liang, Gongbo, et al. \"Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification.\" BMVC (2020)."}, {"review_id": "vBzVlugUxH", "reviewer": "Reviewer_uRXE", "summary": "Post-trained large language models (LLMs) are known to suffer from miscalibration after post-training stages such as supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). This paper proposes leveraging a well-calibrated pre-trained LLM to improve the confidence calibration of post-trained LLMs. Rather than optimizing the negative log-likelihood (NLL) loss on a labeled dataset, a standard approach in existing calibration methods, this work proposes minimizing the KL divergence between the predictive distributions of the post-trained and pre-trained LLMs. The authors further argue that this optimization should be restricted to samples on which both models agree in their predictions. Extensive experiments are conducted, demonstrating the effectiveness of the proposed approach across various settings.", "questions": "- Instead of performing post-hoc confidence calibration, is it possible to incorporate the KL loss as a constraint during post-training? This could potentially lead to naturally calibrated predictions without relying on post-hoc temperature scaling.\n\n\n- How well does the proposed method generalize across different tasks? The current approach involves optimizing a temperature parameter for a specific downstream task or domain. Given that different tasks may require different temperature settings, can a **single learned temperature** value effectively calibrate predictions across a diverse set of tasks?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper explores a novel and under-explored setting: unsupervised confidence calibration for LLMs.\n- The proposed method is simple yet effective, with well-justified motivation.\n- Empirical results are convincing and show consistent improvements across various LLMs.\n- The paper is clearly written and easy to follow.\n\n## Weaknesses\n- While the problem setting is crucial, the proposed solution may be impractical in real-world scenarios. It requires deploying both the pre-trained and post-trained LLMs and performing inference with both, which could be computationally expensive.\n- The method is not universally applicable: the temperature parameter is learned specifically for each downstream task. However, the behavior of post-trained LLMs may vary significantly across tasks, potentially requiring different temperature values.", "was_revised": true, "final_justification": "I recommend accepting this paper with rating-4, and the reasons have been provided in my comment to the rebuttal.", "rebuttal": "# Response to Reviewer uRXE\n\nThank you for the constructive feedback. Please find our response below:\n\n## 1. [W1] Clarification on the inference costs\n\nThank you for highlighting this concern. We clarify that post-hoc calibration methods, including TS and our DACA, perform optimization (Eq.(7)) during the calibration phase, using a small validation set to tune scaling parameters (e.g., $\\tau$). Our method only queries the pretrained model during this phase, not for every test instance. In deployment, DACA incurs no additional computational cost compared to standard post-hoc calibration methods like temperature scaling. Therefore, we can perform inference on test instances using only the post-trained LLM, without needing to deploy or run the pretrained model alongside it. As for the calibration phase, practitioner can first generate the target confidences with (small-scale) PLMs and then use these to tune the scaling parameters for PoLMs, without need to load both models simultaneously. In summary, our solution is practical in real-world scenorios without introducing heavy computational costs.\n\n\n## 2. [W2, Q2] Generalization of DACA across tasks\n\nThanks for raising the concern. To evaluate the transferability of the temperature parameter learned by DACA, we conducted experiments by training a temperature value on the MMLU dataset and applying it to the MedMCQA and MathQA datasets. The results in the table below demonstrate that DACA consistently reduces the ECE of several PoLMs on both MedMCQA and MathQA, which validates the practical transferability of the learned temperature $\\tau$ (despite it might be not the optimal).\n\n\n| Calibration Dataset | Test Dataset | Model               | ECE (%) Vanilla | ECE (%) DACA | Improvement (%) |\n| :------------------ | :----------- | :------------------ | :-------------- | :----------- | --------------- |\n| MMLU                | MedMCQA      | Llama-3-8B-Instruct | 22.943          | **7.908**    | 15.035          |\n|                     |              | Qwen2.5-7B-Instruct | 32.909          | **19.956**   | 12.953          |\n|                     |              | Yi-1.5-9B-Chat      | 34.250          | **17.775**   | 16.475          |\n|   MMLU                  | MathQA       | Llama-3-8B-Instruct | 42.545          | **20.790**   | 21.755          |\n|                     |              | Qwen2.5-7B-Instruct | 37.266          | **17.197**   | 20.069          |\n|                     |              | Yi-1.5-9B-Chat      | 35.977          | **16.697**   | 19.280          |\n\nSecondly, a more practical scenario is **learning a temperaute value on a few tasks and then generalize to other more tasks**, similar in spirit to the setting of OOD generalization [3]. We conducted experiments by randomly selecting 10 subjects from the MMLU dataset for calibration and testing on the remaining 47 subjects. The table below presents the mean results with standard deviations across 10 random trials. The results demonstrate that DACA consistently improves the calibration performance of three PoLMs across the 47 unseen subjects, indicating DACA's robust generalization to new tasks and robustness to domain shifts.\n\n| Model               | ECE (%) Vanilla | ECE (%) DACA   | Average Improvement (%) |\n| ------------------- | --------------- | -------------- | ----------------------- |\n| Qwen2.5-7B-Instruct | 19.02±0.44      | **10.04±0.28** | 8.98                    |\n| Llama-3-8B-Instruct | 20.26±0.36      | **9.39±0.27**  | 10.87                   |\n| Yi-1.5-9B-Chat      | 21.07±0.52      | **9.57±0.44**  | 11.50                   |\n\nAs for learning a single $\\tau$ for all 57 subjects in MMLU, we conduct new experiments by learning a single $\\tau$ on the validation set of all MMLU subjects and present the average results of test set in the table below. The results show that a single $\\tau$ indeed significantly improves the calibration performance of PoLMs on all subjects of MMLU.\n\n| Model               | ECE (%) Vanilla | ECE (%) DACA |Improvement (%) |\n| ------------------- | --------------- | ------------ | ----------------------- |\n| Qwen2.5-7B-Instruct | 18.97           | **9.97**     | 9.00                    |\n| Llama-3-8B-Instruct | 20.22           | **9.40**     | 10.82                   |\n| Yi-1.5-9B-Chat      | 21.18           | **9.85**     | 11.33                   |\n\n\n## 3. [Q1] Incorporating KL Loss as a post-training constraint\n\nThank you for this insightful suggestion. We agree that it might be useful to incorporate the KL loss into the post-training process as a regularization. It is worth noting that most popular post-training methods —such as PPO [1] and DPO [2]— indeed employ an explicit or implicit KL-divergence constraint between PoLM and PLM. Their purpose is to improve the training stability by limiting its deviation from the PLM. However, it may introduce a tradeoff: a weaker KL regularization allows the model to better align with preference data, but often at the cost of its calibration performance. Conversely, strong regularization may compromise the performance of preference alignment. As a result, those post-trained models are still poor calibrated (as shown in Table 3), although they are trained with the KL-divergence constraint. \n\n[1] Schulman, John, et al. \"Proximal policy optimization algorithms.\" (2017).\n\n[2] Rafailov, Rafael, et al. \"Direct preference optimization: Your language model is secretly a reward model.\" NeurIPS (2023)."}]

Author rebuttal:

Meta-review: The authors propose a novel framework leveraging the pretrained model to calibrate the posttrained model. Reviewers unanimously agree that the paper is above the acceptance bar, though they point out several significant limitations to the method. I recommend acceptance, but the authors will need to incorporate all promised edits to the paper and make sure to thoroughly, carefully, and prominently present the limitations noted by the reviewers in the camera ready.
Decision: Accept (poster)