Title: Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
Venue: NeurIPS 2025
Accepted: yes
Keywords: Confidence Calibration

LLM
Abstract: Post-training of large language models is essential for adapting pre-trained language models (PLMs) to align with human preferences and downstream tasks. 
While PLMs typically exhibit well-calibrated confidence, post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs, which can undermine reliability in critical applications.
A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks. 
To address this, we propose Disagreement-Aware Confidence Alignment (DACA), a novel unsupervised method to optimize the parameters (e.g., temperature $\tau$) in post-hoc confidence calibration. 
Our method is motivated by the under-confidence issue caused by prediction disagreement between the PLM and PoLM while aligning their confidence via temperature scaling. Theoretically, the PLM's confidence underestimates PoLM's prediction accuracy on disagreement examples, causing a larger $\tau$ and producing under-confident predictions. DACA mitigates this by selectively using only agreement examples for calibration, effectively decoupling the influence of disagreement.
In this manner, our method avoids an overly large $\tau$ in temperature scaling caused by disagreement examples, improving calibration performance.
Extensive experiments demonstrate the effectiveness of our method, improving the average ECE of open-sourced and API-based LLMs (e.g. GPT-4o) by up to 15.08$\%$ on common benchmarks.

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

Abstract

Post-training of large language models is essential for adapting pre-trained language models (PLMs) to align with human preferences and downstream tasks. While PLMs typically exhibit well-calibrated confidence, post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs, which can undermine reliability in critical applications. A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks. To address this, we propose Disagreement-Aware Confidence Alignment (DACA), a novel unsupervised method to optimize the parameters (e.g., temperature τ ) in post-hoc confidence calibration. Our method is motivated by the under-confidence issue caused by prediction disagreement between the PLM and PoLM while aligning their confidence via temperature scaling. Theoretically, the PLM's confidence underestimates PoLM's prediction accuracy on disagreement examples, causing a larger τ and producing under-confident predictions. DACA mitigates this by selectively using only agreement examples for calibration, effectively decoupling the influence of disagreement. In this manner, our method avoids an overly large τ in temperature scaling caused by disagreement examples, improving calibration performance. Extensive experiments demonstrate the effectiveness of our method, improving the average ECE of open-sourced and API-based LLMs (e.g., GPT-4o) by up to 15.08 % on common benchmarks.

1 Introduction

Post-training has been a critical procedure to ensure large language models (LLMs) generate helpful, honest, and harmless responses [Weng et al., 2023, Kumar et al., 2025]. While post-trained language models (PoLMs) perform well on various downstream tasks [Achiam et al., 2023, DeepSeek-AI and et al., 2025], their reliability and trustworthiness still remain an open challenge. In principle, a reliable LLM should not only demonstrate high confidence in its correct generations but also exercise caution in uncertain situations [Thirunavukarasu et al., 2023, Dahl et al., 2024]. Previous studies [Achiam et al., 2023, Zhu et al., 2023] show that post-training, especially RLHF [Christiano et al., 2017, Stiennon et al., 2020], compromises the well-calibrated confidence estimation of pre-trained language models (PLMs), resulting in over-confidence issues of PoLMs. This gives rise to the importance of confidence calibration for PoLMs, ensuring the confidence score associated with the generation should reflect its ground truth correctness likelihood.

Compared to expensive training methods, post-hoc calibration methods such as temperature scaling [Guo et al., 2017] are more practical for LLMs due to their high efficiency [Shen et al., 2024, Xie et al., 2024]. However, a primary challenge of post-hoc calibration methods is their dependence on labeled data. In practice, generating a reliable labeled dataset for tasks such as mathematics problem solving and medical diagnosis is particularly challenging and time-consuming due to the high level of

domain expertise required. Such difficulty is further compounded by the fact that temperature scaling cannot perform effectively given limited labeled data [Mozafari et al., 2018, Liang et al., 2020]. In contrast, unlabeled data is ubiquitous in real-world deployment scenarios and easy to collect without requiring human intervention. This creates an underutilized resource: vast amounts of unlabeled data are already available during LLM operation, yet are not leveraged for calibration. Thus, this paper studies an unexplored and practical perspective: How can we achieve effective confidence calibration for PoLMs using unlabeled data in an unsupervised manner?

To calibrate PoLMs without relying on labeled data, we introduce Disagreement-Aware Confidence Alignment ( DACA )-a simple and effective post-hoc method that leverages the well-calibrated confidence scores of PLMs. A natural starting point of our idea is to align the confidence of PoLMs with that of PLMs on an unlabeled validation set, minimizing the divergence between the predictive distributions of the PLM and PoLM over all samples. However, we find that this direct confidence alignment can paradoxically lead to under-confidence in the PoLM-when the two models disagree on a prediction, the PLM's confidence often underestimates the actual correctness of the PoLM's output. Our theoretical analysis reveals that such prediction disagreement can drive the optimization to increase the temperature parameter excessively, further exacerbating the under-confidence issue. Motivated by our theory, DACA mitigates this issue by decoupling the influence of disagreement examples from the confidence alignment process. Specifically, it optimizes the temperature parameter using only agreement examples-those where the PLM and PoLM make identical predictions. This ensures that confidence alignment occurs only when the PLM's scores are a trustworthy proxy for correctness. As a result, DACA yields more conservative and reliable temperature estimates, avoiding the calibration failures of naive alignment (see Figure 2b).

Extensive experiments with both open-sourced and API-based LLMs on common benchmarks demonstrate the effectiveness of the DACA method for confidence calibration. Notably, DACA achieves performance comparable to labeled temperature scaling, even in the absence of labeled data. For example, DACA improves the average Expected Calibration Error (ECE) of the Gemma-3-12BInstruct model [Team et al., 2025] across 57 subjects of the MMLU dataset [Hendrycks et al., 2021], reducing it from 23.68 % to 8.60 % . In comparison, TS only reduces the ECE to 9.75 % . Importantly, DACAis applicable even in scenarios where post-trained and pre-trained models differ in architecture, making it more efficient for the calibration of large-scale PoLMs. For instance, DACA reduces the ECE of GPT-4o [Hurst et al., 2024] from 21.23 % to 6.99 % when calibrated using the pre-trained Gemma-3-12B model on the MedMCQA dataset [Pal et al., 2022]. Furthermore, our method can be applied to open-ended question-answering tasks and offers benefits for selective classification. Codes are publicly available at https://github.com/ml-stat-Sustech/Disagreement-Aware-Calibration.

We summarize our contributions as follows.

We show that the well-calibrated outputs of PLMs on unlabeled data can be leveraged to calibrate PoLMs. Theoretically, we demonstrate that prediction disagreement can impair calibration performance when directly aligning the confidence of PLMs and PoLMs.

Our proposed post-hoc method DACA, formalizes the confidence calibration problem by harnessing the target-specific unlabeled data in the wild. This formulation offers strong practicality and flexibility for real-world applications.

We empirically show that DACA enhances the calibration of both open-sourced and APIbased PoLMs across various datasets. Moreover, our method applies to open-ended QA tasks and enhances selective classification.

2 Preliminaries

2.1 Confidence Calibration for LLMs

In this work, we focus on the confidence calibration problem of question answering for the Posttrained Language Model (PoLM), denoted as f . Our method primarily targets Multiple-Choice QA (MCQA) and can be extended to open-ended QA. For MCQA with choices Y = { A,B,C,D } and prompt x , let z f ( x ) ∈ R |Y| be the logits of f . The predicted probabilities and confidence are

0.0

0.2

0.4

0.6

0.8

1.0

Pre-trained Version

Outputs

Gap

ECE: 3.52%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Instruct Version

ECE: 19.00%

(a) Llama-3-8B

0.0

0.2

0.4

0.6

0.8

1.0

Pre-trained Version

Outputs

Gap

ECE: 5.41%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Instruct Version

ECE: 20.84%

(b) Qwen-2.5-7B

0.0

0.2

0.4

0.6

0.8

1.0

Pre-trained Version

Outputs

Gap

ECE: 3.39%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Chat Version

ECE: 20.81%

(c) DeepSeek-V2-Lite

0.0

0.2

0.4

0.6

0.8

1.0

Pre-trained Version

Outputs

Gap

ECE: 6.85%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Chat Version

ECE: 24.15%

(d) Yi-1.5-6B

with prediction ˆ Y ( x ) = arg max j p f ( y = j | x ) . We denote the n -th prompt in a dataset by x n = x n,t n , ..., x n,t 2 , x n,t 1 , which is a sequence of t n tokens, with its corresponding response denoted as y n . Formally, given a prompt x , a perfectly calibrated model satisfies,

where ˆ Y = arg max y p ( y | x ) is the predicted response, and ˆ P = max y p ( y | x ) is the corresponding confidence score [Guo et al., 2017].

To quantify the degree of miscalibration, expected calibration error (ECE) [Naeini et al., 2015] is defined as E [ | Pr( Y = ˆ Y | ˆ P = β ) -β | ] , which measures the difference between confidence and accuracy. An empirical estimate of ECE is calculated by partitioning N samples into G bins { b 1 , b 2 , . . . , b G } according to the confidence predicted by the model. The ECE is then formulated as

where acc ( b g ) and conf ( b g ) denote the average accuracy and confidence within bin b g , respectively. A smaller ECE indicates better calibration performance of the model.

Post-hoc calibration methods aim to calibrate a model after training. Among these approaches, Platt scaling [Platt et al., 1999] based approaches are commonly adopted due to their low complexity and efficiency, including temperature scaling (TS) [Guo et al., 2017] and its extensions [Mozafari et al., 2018, Kull et al., 2019]. In particular, given a miscalibrated model f , TS introduces a temperature parameter τ to soften the model's predicted probability: p ( y = i | x , τ ) = σ i ( f ( x ) /τ ) , where σ ( · ) denotes the softmax function and τ > 0 for all classes. The optimal temperature value for the target dataset by minimizing the negative log-likelihood (NLL) on a labeled calibration dataset D ∗ = { x n , y n } N n =1 is given by:

Temperature scaling simplifies matrix (vector) scaling [Guo et al., 2017], where a single τ is applied to all classes, offering great calibration performance while maintaining minimal computational complexity [Guo et al., 2017, Minderer et al., 2021].

2.2 The effects of LLM post-training

The success of large language models (LLMs) has led to a standardized training paradigm of pretraining followed by post-training. Post-training refines pre-trained language models (PLMs) for

specific tasks through techniques such as fine-tuning [Ziegler et al., 2019, Wei et al., 2022], alignment [Peng et al., 2023, Su et al., 2023, Bai et al., 2022], knowledge adaptation [Dong et al., 2022, Rubin et al., 2021], and reasoning enhancement [Yao et al., 2023]. While post-training improves task performance, it often comes at the cost of degraded calibration-introducing overconfidence in the model's predictions. In contrast, PLMs typically exhibit more accurate confidence estimates [Achiam et al., 2023, Zhu et al., 2023]. Formally, in multiple-choice tasks, we denote the pre-trained LM as f : X → R k , where k is the number of choices. Through post-training, we learn a post-trained language model (PoLM) g : X → R k . We present the reliability diagram of multiple PoLMs on the MMLUdataset in Figure 1. The diagram illustrates that PoLMs consistently exhibit over-confidence, with confidence scores notably higher than the true likelihood of correctness.

Post-hoc calibration techniques like temperature scaling mitigate overconfidence effectively but rely on labeled validation datasets. Generating a reliable labeled dataset for tasks like mathematical problem-solving and medical diagnosis is challenging and time-consuming due to the required domain expertise. However, under limited labeled data, the calibration performance of post-hoc methods cannot be guaranteed. Leveraging unlabeled data for confidence calibration offers a promising solution for ensuring reliable model behavior in resource-constrained settings. Given the inherently well-calibrated property of PLMs, a natural question arises: Can we leverage the well-calibrated confidence scores of PLMs on unlabeled data to calibrate over-confident PoLMs ?

3 Motivation and Method

To leverage the well-calibrated confidence scores from PLMs, an intuitive approach is to align the confidence levels of PoLMs with those of well-calibrated PLMs on an unlabeled validation set. A naive approach for confidence alignment is to modify the objective in traditional temperature scaling on an unlabeled validation set D = { x i } N i =1 . Instead of minimizing the negative log-likelihood, we minimize the Kullback-Leibler (KL) divergence between the predictive distributions of the pretrained and post-trained language models on D . Formally, given the post-trained model g , the optimal temperature τ ∗ on D is given by

Here, σ ( · ) denotes the softmax function, and p i ( x ) is the i -th element of the softmax probability σ ( f ( x )) of model f . For convenience, we refer to this approach as "naive confidence alignment".

Naive confidence alignment leads to under-confidence. In Figure 2a, we show that the naive confidence alignment can lead PoLMs to become significantly under-confident, indicating that their predicted confidence underestimates the actual accuracy. In the following, we investigate why confidence alignment-scaled PoLMs tend to give under-confident predictions. Our analysis suggests that the prediction disagreement introduced by post-training can be a culprit.

̸

Prediction disagreement between two models f and g refers to arg max i f i ( x ) = arg max i g i ( x ) on the same input prompt x . For convenience, we denote the examples with the existence of prediction disagreement as disagreement examples. It is known that post-training techniques frequently alter the PLM's output distribution, resulting in prediction disagreement. Formally, the unlabeled data can be characterized by the Huber contamination model [Huber, 1992] as follows:

Definition 3.1 (Unlabeled data distribution) . We define the unlabeled data be the following mixture of distributions

where π ∈ (0 , 1] denotes the disagreement ratio, P agree and P dis are the marginal distributions of agreement examples and disagreement examples, respectively. In practice, π > 0 , as post-training typically changes some PLMs' predictions.

With the above definition, we assume the unlabeled dataset D is i.i.d. sampled from the mixture distribution P unlabeled. In the following, we analyze the limitations of naive confidence alignment in the presence of prediction disagreement.

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Computer Security

Outputs

Gap

ECE: 18.67%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

College Chemistry

ECE: 21.23%

(a)

0

250

500

750

1000

1250

1500

1750

2000

Epoch

2

4

6

8

10

12

Temperature (t)

Computer Security

0

250

500

750

1000

1250

1500

1750

2000

Epoch

2

4

6

8

10

12

Temperature (t)

College Chemistry

Temperature (Disagreement Set)

Temperature (Agreement Set)

Temperature (Whole Dataset)

(b)

Proposition 3.2. Assume f ( · ) be a perfectly calibrated predictor with ECE f = 0 and g ( · ) denote a predictor perfectly aligned to the predictor f . Let ˜ y be the unknown label of sample x . The expected calibration error (ECE) of g over the unlabeled distribution P unlabeled :

The proposition's proof is presented in Appendix B. The proposition illustrates that the ECE of a PoLM cannot reach zero even in an ideal case where the PoLM is perfectly aligned with a perfectly calibrated PLM in confidence, due to the existence of prediction disagreement. Intuitively, PLM's confidence for disagreement examples reflects its own prediction's accuracy, instead of that of PoLM's prediction. Since post-training typically improves PoLM's accuracy, PLM's confidence level will be lower than the prediction accuracy of PoLM, resulting in the under-confidence issue. In the following, we further analyze how prediction disagreement impacts the parameter τ of temperature scaling as an example.

Proposition 3.3. Given a sample x , let g ( x ) denote the output logits of a post-trained language model, and p ( x ) denote the softmax probability from the pre-trained language model. If arg max i g i ( x ) = c and σ c ( f ( x )) < 1 k , then the optimal temperature is given by:

The proof of this proposition is provided in Appendix B. Proposition 3.3 indicates that the gradient of the KL divergence w.r.t the temperature τ remains positive on the disagreement set, which increases the value of τ continuously during optimization. Consequently, the optimization will further exacerbate the under-confidence issue. To provide a straightforward view, Figure 2b shows the temperature dynamics during training exclusively on the disagreement set, revealing a gradual increase to a significantly high value.

Disagreement-Aware Confidence Alignment. In our previous analysis, we showed that disagreement examples tend to drive the temperature parameter to excessively high values, leading to an under-confidence issue. To address this problem, our key idea is to decouple the influence of disagreement examples from the confidence alignment process. We propose Disagreement-Aware Confidence Alignment (DACA), which eliminates the gradient of the KL divergence with respect to the temperature on disagreement examples, thereby ensuring that temperature optimization is guided solely by agreement examples. Formally, the new loss function of DACA can be defined as:

where ˆ y = arg max i f i ( x ) and ˆ y ′ = arg max i g i ( x ) denote the predictions of the pre-trained model f and the post-trained model g , respectively.

Minimizing the loss function in Equation (7) mitigates the under-confidence issue effectively. We illustrate with an example in Figure 2b, which demonstrates that optimizing the temperature solely on the agreement set yields a more conservative estimate than optimizing on the whole dataset.

Extensions to other post-hoc calibration methods. Notably, our method is general and can be easily incorporated into other existing post-hoc calibration methods such as vector scaling and matrix scaling [Guo et al., 2017]. Formally, for any rescaling function ϕ θ with parameter θ , we can formulate the method as follows. First, we define the i -th softmax probability of the post-trained model after rescaling as q i ( x ; θ ) = σ ( ϕ θ · f ( x )) i . The corresponding probability of the pre-trained model is given by p i ( x ) . Then, the optimization objective can be formulated as:

where ˆ y = arg max i p i ( x ) and ˆ y ′ = arg max i q i ( x ) denote the predictions of the pre-trained model and the post-trained model, respectively. We present the calibration performance of our method with vector scaling and matrix scaling in Appendix D.3.

4 Experiments

4.1 Setup

Models. We conduct extensive experiments on diverse LLMs, including both open-source models and those accessible via online APIs. For open-sourced LLMs, we include Llama-3 family [Grattafiori et al., 2024], Gemma-3 family [Team et al., 2025], Qwen-2.5 family [Yang et al., 2024], and Yi-1.5 family [Young et al., 2024]. Unless explicitly stated otherwise, we perform calibration using the pre-trained counterpart of each post-trained LLM. The above models are provided by Hugging Face. To scale up our findings, we also evaluate large-scale LLMs accessed through online APIs, such as GPT-4o [Hurst et al., 2024] and DeepSeek-V3 [Liu et al., 2024a].

Datasets. To verify the effectiveness of our proposed methods, we employ three common datasets for evaluations, including: MMLU [Hendrycks et al., 2021], MedMCQA [Pal et al., 2022], and MathQA [Amini et al., 2019]. For MMLU, we learn a specific temperature parameter for each subject using a subject-specific validation set. The datasets are provided by Hugging Face. Due to limited space, detailed information about each dataset is presented in Appendix C.

Compared methods. Since our method is the first unlabeled post-hoc approach to calibrate LLMs without training auxiliary models, we exclude many existing calibration methods that rely on labeled data and additional training. To compare with other unlabeled calibration approaches, we select three prompt-based methods as baselines, including CAPE [Jiang et al., 2023]: a prompt-based method that calibrates next-token probabilities by permuting option order to mitigate LLM biases, Elicitation [Tian et al., 2023]: estimates confidence by prompting the model to generate verbalized probabilities, Elicitation-Ensemble [Xiong et al., 2023a]: improves upon this by aggregating outputs from multiple prompts. Specifically, Vanilla represents the calibration performance of LLMs without any calibration techniques applied, and Temperature Scaling (TS) leverages labeled data from the test task to tune task-specific temperatures and is included as a supervised reference baseline.

Evaluation metrics. We evaluate the calibration performance using the following metrics: (1) Expected Calibration Error ( ECE ) [Naeini et al., 2015]: measures the average error between prediction confidence and accuracy across different confidence intervals. For evaluation, we use 10 bins in our evaluation. (2) Maximum Calibration Error ( MCE ) [Naeini et al., 2015]: measures the largest discrepancy between prediction confidence and accuracy across all confidence bins, reflecting the worst-case calibration scenario. (3) Adaptive ECE ( AECE ) [Nixon et al., 2019]: proposes a new binning strategy that uses an adaptive scheme to space the bin intervals, ensuring that each bin contains an equal number of examples. (4) Brier Score [Brier, 1950]: directly measures the distance between the model confidence and the binary correctness label of the generation.

Implementation details. For multiple-choice datasets, the model estimates the probability that the next token matches one of the options (e.g., A, B, C, or D), reflecting its confidence. Due to the space limitation, more details of implementation are provided in Appendix C.

4.2 Main results

DACA significantly improves the calibration performance of PoLMs. Table 1 presents the average calibration performance of the baselines and our method across 57 subjects of the MMLU datasets, with four contemporary LLMs. The validation set is the validation split of each subject in MMLUon Huggingface, where the size of the validation set is limited. A salient observation is that our method effectively mitigates the mis-calibration in various models across all metrics and is even comparable to the labeled TS with limited validation data. For instance, our method improves the ECE of Llama-3-70B-Instruct from 12 . 870% to 7 . 844% . Similarly, it improves the ECE of the latter released Qwen3-8B from 16 . 383% to 8 . 566% . It is worth noting that the verbalization-based method, such as Elicitation and Elicitation-Ensemble, performs significantly worse than the next-token logitsbased method, which is consistent with the results reported in previous work [Shen et al., 2024]. We further evaluate our method on additional datasets, including MedMCQA and MathQA, as shown in Appendix D.2. Our method can also be extended to vector and matrix scaling, with results shown in Appendix D.3, demonstrating improved calibration across these post-hoc methods.

DACA is effective across models of different sizes. We also verify the calibration performance of the baselines and our methods from models of different sizes. In Figure 3, our results indicate that our approach is effective with different-sized LLMs and achieves impressive performance across diverse architectures. Notably, the Vanilla ECE decreases monotonically with increasing model scale, a trend that aligns with the conclusions drawn in previous research [Zhu et al., 2023].

DACA is agnostic to the choice of PLMs. In practice, many closed-source, large-scale PoLMs (e.g., GPT-4o and DeepSeek-V3) are accessed via APIs. As such, calibrating these API-based models becomes essential. However, these models typically lack accessible pre-trained versions, and their large scale requires significant computational resources. Our method effectively calibrates both API-based and large-scale PoLMs, as well as smaller models. Specifically, we use three small-scale PLMs-Llama-3-8B, Qwen2.5-7B, and Gemma-3-12B-to calibrate GPT-4o and DeepSeek-V3. As shown in Table 2, our method consistently improves the calibration performance of GPT-4o regardless of the PLM choice. For example, DACA reduces the ECE of GPT-4o from 21.231 % to 6.993 % using

4B

12B

27B

Size

0

10

20

30

40

50

ECE

(a) Gemma-3-Instruct Family

6B

9B

34B

Size

0

5

10

15

20

25

30

35

40

ECE

(b) Yi-1.5-Chat Family

7B

32B

72B

Size

0

5

10

15

20

25

30

35

ECE

(c) Qwen-2.5-Instruct Family

8B

70B

Size

0

5

10

15

20

ECE

(d) Llama-3-Instruct Family

Vanilla

CAPE

Ours

Is our method effective with different post-training strategies? To demonstrate that our proposed method is agnostic to the post-training strategy, we conduct experiments on a diverse set of Llama3.1-8B models post-trained with different techniques and report the results in Table 3. We use the models released by Ai2 on Hugging Face 2 . The results show that our method consistently improves calibration performance across all tested post-training strategies. For example, DACA reduces the calibration error of the model post-trained with SFT and DPO from 25.193 % to 5.418 % . Additional results on post-trained models with different post-training techniques are provided in Appendix D.5.

5 Discussion

Can DACA be applied to open-ended QA tasks? Previous works estimate confidence scores in open-ended question answering (QA) tasks by reformulating the free-form QA problem into a multiple-choice format [Shen et al., 2024, Kapoor et al., 2024]. Specifically, they pose a binary "Yes" or "No" question to a language model, asking whether its own generated answer is correct or incorrect. This approach, commonly referred to as P(True) in the hallucination detection literature, serves as a well-known baseline. Following prior works, we also adopt the P(True) approach to obtain confidence scores for our experiments. Formally, the confidence score of model f on sample x is defined as p ( Yes | x, f ) . We then define the prediction disagreement between models f and g in open-ended QA tasks as arg max i p i ( x , f ) = arg max i p i ( x , g ) , where i ∈ { 1 , 2 } .

̸

Figure 4 illustrates the calibration performance of our method on the TruthfulQA datasets [Lin et al., 2021], evaluated across models of varying sizes from the Qwen2.5 and LLaMA-3 families. Specifically, we use Qwen2.5-32B and LLaMA-3-70B as the pre-trained models to calibrate the corresponding post-trained models within each family. The results demonstrate that our method consistently reduces calibration error across different models. For example, DACA reduces the vanilla ECE from 30.955 % to 5.244 % on Qwen2.5-32B-Instruct, highlighting its applicability to open-ended QA tasks. Detailed results, including additional metrics, are provided in Appendix D.6.

DACA can benefit selective classification. Selective classification [Geifman and El-Yaniv, 2017] leverages model confidence to decide whether to make a prediction or abstain, thereby improving

7B

14B

32B

72B

Size

0

10

20

30

40

50

ECE

Qwen-2.5-Instruct Family

8B

70B

Size

0

10

20

30

40

ECE

Llama-3-Instruct Family

Vanilla

Ours

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Confidence

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Meta-Llama-3-70B-Instruct

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Confidence

Qwen2.5-72B-Instruct

Vanilla

Ours

reliability by trading off coverage for higher accuracy on accepted examples. This is particularly important when using LLMs for decision-making, where unreliable predictions can lead to significant downstream consequences. Although temperature scaling is accuracy-preserving by design, calibrated confidence scores can nonetheless enhance selective classification by enabling more reliable abstention decisions, thereby improving accuracy on the retained subset.

In Figure 5, we present the accuracy comparison of baselines and our method under varying confidence thresholds ranging from 0.5 to 0.95, where predictions with confidence below the threshold are rejected. A salient observation is that confidence scores calibrated by our method significantly exceed the original accuracy at every confidence threshold, demonstrating improved reliability in selective classification. Notably, the performance gains become increasingly pronounced as the confidence threshold rises. This is attributable to our method's ability to mitigate over-confidence issues, thereby improving the model's accuracy on high-confidence predictions.

6 Related work

Post-training in LLMs. Post-training in Large Language Models (LLMs) is a critical phase that refines models after their initial pre-training [Tie et al., 2025, Kumar et al., 2025], where they learn general language patterns through next-token prediction on vast datasets. In the post-training phase, LLMs undergo a structured enhancement process that typically follows a sequential order. Initially, fine-tuning is employed to adapt the pre-trained model to specific tasks or domains. This step involves updating the model's parameters using curated datasets, which significantly improves its performance on targeted tasks [Yue et al., 2023, Luo et al., 2023]. To optimize resource efficiency, parameterefficient fine-tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA) and Adapters [Hu et al., 2022, Gao et al., 2023, Luong et al., 2024], are often utilized. These methods adjust only a small subset of the model's parameters or introduce a limited number of trainable parameters, achieving comparable performance to full fine-tuning while significantly reducing computational and memory requirements. Following this, reinforcement learning (RL) techniques are applied to

further refine the model's behavior. Methods such as Reinforcement Learning from Human Feedback (RLHF) [Ouyang et al., 2022] and Direct Preference Optimization (DPO) [Rafailov et al., 2023] incorporate dynamic feedback to optimize decision-making and align the model's outputs with user preferences. Together, these strategies transform LLMs into versatile, user-aligned tools for diverse applications. In this work, we address the confidence calibration problem in Post-trained Language Models (PoLMs) by leveraging well-calibrated Pre-trained Language Models (PLMs). Our method aligns the confidence scores of PoLMs with PLMs on samples where both models produce the same prediction.

Confidence Calibration. Confidence calibration has been widely studied to ensure that the confidence levels output by models accurately reflect their true performance. To achieve this, the state-of-the-art calibration methods can be categorized into two paradigms: post-hoc methods [Platt et al., 1999, Guo et al., 2017, Mozafari et al., 2018, Kull et al., 2019, Xiong et al., 2023b, Wang et al., 2024] and regularization methods [Müller et al., 2019, Mukhoti et al., 2020, Hebbalaguppe et al., 2022]. For post-hoc calibration, a representative method is temperature scaling [Guo et al., 2017], which learns a single scalar for rescaling the logit. Recently, several studies have investigated calibration in LLMs [Jiang et al., 2023, Xiao et al., 2022, Chen et al., 2022, Liu et al., 2024b], highlighting that posttraining often leads to overconfidence. One line of work explores fine-tuning methods to encourage well-calibrated numerical and linguistic verbalized confidence [Lin et al., 2022, Kapoor et al., 2024, Tao et al., 2025], while another focuses on training auxiliary models to predict model confidence [Kadavath et al., 2022, Liu et al., 2024b, Ulmer et al., 2024] or estimate temperature parameters for unseen tasks [Shen et al., 2024]. However, these approaches typically require labeled data and, in some cases, are computationally expensive. Other works [Xie et al., 2024, Tian et al., 2023] examine post-trained LLMs and show that carefully designed prompts can elicit better-calibrated uncertainty estimates. Distinct from prior approaches, our work is the first to leverage unlabeled data for post-hoc confidence calibration, offering both efficiency and flexibility.

7 Conclusion

In this paper, we introduce Distance-Aware Confidence Alignment ( DACA ), an unsupervised posthoc method designed to calibrate overconfident PoLMs. To the best of our knowledge, this is the first approach that uses unlabeled data for the post-hoc calibration of LLMs. The core idea behind DACA is to decouple the influence of prediction disagreement when aligning confidence between PoLMs and well-calibrated PLMs. Specifically, DACA optimizes the temperature parameter using only agreement examples-those in which the PLM and PoLM make identical predictions-ensuring that confidence alignment occurs only when the PLM's scores serve as a trustworthy proxy for correctness. Extensive experiments demonstrate the effectiveness of DACA in calibrating PoLMs across a wide range of models and common datasets. This method can be easily adopted in practical settings, as it can be applied to both open-sourced and API-based LLMs and is computationally efficient.

Limitations. Our method involves an additional inference step using pre-trained models, leading to a modest increase in computational cost. Additionally, filtering out disagreement examples may reduce the pool of unlabeled examples available for calibration. However, this trade-off is generally acceptable, given the wide availability of unlabeled data. Future work could explore how to leverage these disagreement examples to further improve calibration.

Acknowledgment

Beier Luo and Hongxin Wei are supported by the Shenzhen Fundamental Research Program (Grant No. JCYJ20230807091809020). We gratefully acknowledge the support of the Center for Computational Science and Engineering at the Southern University of Science and Technology for our research.

References

Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. Findings of the Association for Computational Linguistics: EMNLP , 2023.

Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321 , 2025.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023.

DeepSeek-AI and et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.

Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine , 29(8): 1930-1940, 2023.

Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis , 16(1):64-93, 2024.

Chiwei Zhu, Benfeng Xu, Quan Wang, Yongdong Zhang, and Zhendong Mao. On the calibration of large language models and alignment. Findings of the Association for Computational Linguistics: EMNLP , 2023.

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems , 30, 2017.

Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems , 33:3008-3021, 2020.

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning , pages 1321-1330. PMLR, 2017.

Maohao Shen, Subhro Das, Kristjan Greenewald, Prasanna Sattigeri, Gregory W Wornell, and Soumya Ghosh. Thermometer: Towards universal calibration for large language models. In International Conference on Machine Learning , 2024.

Johnathan Xie, Annie S Chen, Yoonho Lee, Eric Mitchell, and Chelsea Finn. Calibrating language models with adaptive temperature scaling. arXiv preprint arXiv:2409.19817 , 2024.

Azadeh Sadat Mozafari, Hugo Siqueira Gomes, Wilson Leão, Steeven Janny, and Christian Gagné. Attended temperature scaling: a practical approach for calibrating deep neural networks. arXiv preprint arXiv:1810.11586 , 2018.

Gongbo Liang, Yu Zhang, Xiaoqin Wang, and Nathan Jacobs. Improved trainable calibration method for neural networks on medical imaging classification. arXiv preprint arXiv:2009.04057 , 2020.

Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786 , 2025.

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021.

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024.

Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa : A large-scale multi-subject multi-choice dataset for medical domain question answering, 2022.

Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence , volume 29, 2015.

John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers , 10(3):61-74, 1999.

Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in Neural Information Processing Systems , 32, 2019.

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems , 34:15682-15694, 2021.

Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 , 2019.

Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations , 2022.

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 , 2023.

Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305.16355 , 2023.

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 , 2022.

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234 , 2022.

Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633 , 2021.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems , 36:11809-11822, 2023.

Peter J Huber. Robust estimation of a location parameter. In Breakthroughs in statistics: Methodology and distribution , pages 492-518. Springer, 1992.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 , 2024.

Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652 , 2024.

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 , 2024a.

Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319 , 2019.

Mingjian Jiang, Yangjun Ruan, Sicong Huang, Saifei Liao, Silviu Pitis, Roger Baker Grosse, and Jimmy Ba. Calibrating language models via augmented prompt ensembles. ICML 2023 Workshop on Deployable Generative AI , 2023.

Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305.14975 , 2023.

Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306.13063 , 2023a.

Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR workshops , 2019.

Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review , 78(1):1-3, 1950.

Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, and Andrew Wilson. Calibration-tuning: Teaching large language models to know what they don't know. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024) , pages 1-14, 2024.

Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 , 2021.

Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. Advances in Neural Information Processing Systems , 30, 2017.

Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, et al. A survey on post-training of large language models. arXiv preprint arXiv:2503.06072 , 2025.

Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, et al. Disc-lawllm: Fine-tuning large language models for intelligent legal services. arXiv preprint arXiv:2309.11325 , 2023.

Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747 , 2023.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR , 1(2):3, 2022.

Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, et al. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010 , 2023.

Trung Quoc Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. arXiv preprint arXiv:2401.08967 , 3, 2024.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35: 27730-27744, 2022.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems , 36:53728-53741, 2023.

Miao Xiong, Ailin Deng, Pang Wei W Koh, Jiaying Wu, Shen Li, Jianqing Xu, and Bryan Hooi. Proximity-informed calibration for deep neural networks. Advances in Neural Information Processing Systems , 36:68511-68538, 2023b.

Shuoyuan Wang, Jindong Wang, Guoqing Wang, Bob Zhang, Kaiyang Zhou, and Hongxin Wei. Open-vocabulary calibration for fine-tuned clip. arXiv preprint arXiv:2402.04655 , 2024.

Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in Neural Information Processing Systems , 32, 2019.

Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss. Advances in Neural Information Processing Systems , 33:15288-15299, 2020.

Ramya Hebbalaguppe, Jatin Prakash, Neelabh Madan, and Chetan Arora. A stitch in time saves nine: A train-time regularizing loss for improved neural network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16081-16090, 2022.

Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and LouisPhilippe Morency. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. arXiv preprint arXiv:2210.04714 , 2022.

Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, and Heng Ji. A close look into the calibration of pre-trained language models. arXiv preprint arXiv:2211.00151 , 2022.

Xin Liu, Muhammad Khalifa, and Lu Wang. Litcab: Lightweight language model calibration over short- and long-form responses. In The Twelfth International Conference on Learning Representations , 2024b.

Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334 , 2022.

Linwei Tao, Yi-Fan Yeh, Minjing Dong, Tao Huang, Philip Torr, and Chang Xu. Revisiting uncertainty estimation and calibration of large language models. arXiv preprint arXiv:2505.23854 , 2025.

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221 , 2022.

Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. Calibrating large language models using their generations only. arXiv preprint arXiv:2403.05973 , 2024.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our experimental results consistently support the claims presented in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation of this work in the conclusion section 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the proof of our theoretical result in the Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide the detailed implementation details to reproduce our results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We will release the code of our work.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide the data split information in Appendix C and other information in implementation details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Our results are provided in mean and standard error format.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The compute resources are provided in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes, we affirm that our research adheres to the NeurIPS Code of Ethics in all aspects, including considerations related to data usage, transparency, and potential societal impact.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss about the societal impacts of confidence calibration in the introduction. What's more, we also discuss why we need to leverage the unlabeled data to performe confidence calibration.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work does not involve the release of model and dataset. Our models and datasets are all from Hugging Face.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The dataset and model we use are both open-sourced on Hugging Face. Our code is based on PyTorch and vLLM.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We will upload our code in the supplement material.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix

A Over-confidence issue with more post-trained models

In this section, we present evidence that post-training can lead to overconfidence issues with additional PoLMs, as illustrated in Figures 6 and 7. We summarize the PoLMs, along with their post-training technologies and source websites, in Table 4.

0.0

0.2

0.4

0.6

0.8

1.0

Meta-Llama-3.1-8B

Outputs

Gap

ECE: 4.59%

Meta-Llama-3.1-8B

ECE: 4.59%

Meta-Llama-3.1-8B

ECE: 4.59%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Llama-3.1-Tulu-3-8B-SFT

ECE: 11.29%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Llama-3.1-Tulu-3-8B-DPO

ECE: 11.29%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Llama-3.1-Tulu-3-8B

ECE: 20.27%

Pre-trained

Post-trained

0.0

0.2

0.4

0.6

0.8

1.0

Meta-Llama-3-8B

Outputs

Gap

ECE: 3.52%

Meta-Llama-3-8B

ECE: 3.52%

Meta-Llama-3-8B

ECE: 3.52%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

Llama-3-8B-Iter-DPO-179k

ECE: 19.45%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Llama-3-Base-8B-SFT-IPO

ECE: 30.25%

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Llama-3-8B-Self-Instruct-100K

ECE: 13.96%

Pre-trained

Post-trained

B Theoretical Proof

B.1 Proof of Theorem 3.2

Proof. First, we review that the ECE is defined as

Then given a dataset D = { x i , ˜ y i } N i =1 , the ECE of f is given by

where p f ( x ) is the confidence score of f on sample x . In the same way, the ECE of g is given by

where p g ( x ) is the confidence score of g on sample x . Since the confidence level of g is aligned with f , we have that

B.2 Proof of Proposition 3.3

Proof. The KL divergence between the true distribution p ( x ) and the model distribution σ ( g ( x ) /τ ) is given by:

Our goal is to show that D KL ( p ( x ) || σ ( g ( x ) /τ )) is minimized as τ →∞ .

where

First, note that the KL divergence can be expressed as:

where

is the entropy of p ( x ) , a constant, and

is the cross-entropy. Therefore, minimizing D KL ( p ( x ) || σ ( g ( x ) /τ )) with respect to τ is equivalent to minimizing the cross-entropy H ( p ( x ) , σ ( g ( x ) /τ )) .

Next, we analyze the behavior of σ ( g ( x ) /τ ) as τ varies:

̸

As τ → 0 : Since c = arg max g ( x ) , σ ( g ( x ) /τ ) c → 1 and σ ( g ( x ) /τ ) i → 0 for i = c . If p i ( x ) > 0 for some i = c , then -log σ ( g ( x ) /τ ) i →∞ , implying H ( p ( x ) , σ ( g ( x ) /τ )) → ∞ .

As τ →∞ : σ ( g ( x ) /τ ) i → 1 k for all i , since g i ( x ) /τ → 0 . Thus,

̸

Now, for finite τ > 0 , since g c ( x ) > g i ( x ) for i = c (assuming a strict maximum for simplicity), we have σ ( g ( x ) /τ ) c > 1 k , with equality only as τ →∞ . Given that p c ( x ) < 1 k , the model distribution σ ( g ( x ) /τ ) assigns more probability to class c than the uniform distribution for finite τ , while the true distribution p ( x ) assigns less than uniform to class c .

̸

To see why the minimum occurs at τ = ∞ , consider that as τ increases, σ ( g ( x ) /τ ) approaches the uniform distribution, which reduces the cross-entropy by making σ ( g ( x ) /τ ) i closer to 1 k . Since p c ( x ) < 1 k , and typically p i ( x ) for i = c are such that the uniform distribution provides a better approximation than a distribution concentrated on c , the cross-entropy decreases as τ increases.

More formally, one can consider the derivative of H ( p ( x ) , σ ( g ( x ) /τ )) with respect to τ , but the limit behaviors suffice to establish that H ( p ( x ) , σ ( g ( x ) /τ )) is minimized as τ →∞ . Specifically, since H ( p ( x ) , σ ( g ( x ) /τ )) →∞ as τ → 0 and H ( p ( x ) , σ ( g ( x ) /τ )) → log k as τ →∞ , and assuming H ( p ( x ) , σ ( g ( x ) /τ )) is continuous and decreasing in τ , the infimum is achieved as τ →∞ .

Therefore, the temperature parameter that minimizes the KL divergence is:

C Implementation details

Experiment details. We run our experiments on NVIDIA GeForce RTX 4090 and NVIDIA L40 GPU, and implement all methods by PyTorch and vLLM .

Optimizer details. For both TS and DACA, we use the Adam optimizer with a batch size of 256, a learning rate of 0.05, and train for 400 epochs.

Datasets details. For the main experiments, we apply confidence calibration to each of the 57 subjects from MMLU and report the average of the calibration metrics. Specifically, we use the validation split of each subject as the validation set. For the MMLU datasets, we conduct five experiments with five different prompts to calculate the mean and standard deviation of the results, as the validation and test splits are predetermined. We provide the choices of the prompt in Table 5. For other datasets, we use the first prompt and report the mean and standard deviation over five random splits of the validation and test sets, with a test-to-validation ratio of 7:3.

̸

Meta-Llama-3.1-8B-lnstruct

Mistral-7B-nstruct-v0.3

DeepSeek-V2-Lite-Chat

1.0

Outputs

Gap

ECE:11.08%

ECE:18.90%

0.8

ECE:14.10%

0.6

0.4

0.2

0.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Confidence

Confidence

D Detailed results

D.1 More under-confidence results of naive confidence alignment

We present the reliability diagram of more models scaled with naive confidence alignment on the MMLUdataset in Figure 8.

D.2 Extended results across diverse models and datasets

The performance of our method on more datasets and models. We present the average calibration performance with more models across 57 subjects of MMLU in Table 6. In addition, we compare the calibration results of our method and baseline approaches on MathQA and MedMCQA in Table 7 and Table 8, respectively. The results show that our method significantly reduces the miscalibration of PoLMs and achieves performance comparable to TS, which has access to labels. For instance, our method reduces the ECE of DeepSeek-V2-Lite-Chat on MedMCQA from 26.553 % to 1.715 % , while TS reduces to 1.800 % .

D.3 Extension to vector scaling and matrix scaling

We present the results of applying the DACA extension with vector scaling (VS) and matrix scaling (MS) on MedMCQA in Table 9. Across all models, our method consistently reduces the calibration error, regardless of whether VS or MS is used. For example, on Qwen2.5-72B-Instruct, DACA+VS reduces the ECE from 21.720 % to 4.133 % , which is comparable to the oracle VS result of 4.558 % . Similarly, DACA+MS lowers the ECE to 4.407 % , closely matching the oracle MS result of 4.201 % .

D.4 Results of additional large-scale PoLMs

We present the results of our method on DeepSeek-V3 with various PLMs in Table 10. Across all PLMs, our method consistently reduces calibration error. A similar trend is observed, where the lower ECE of the pre-trained model leads to a lower ECE in the scaled post-trained model.

D.5 Results of additional post-training techniques

To evaluate the effectiveness of our method, we perform experiments with more post-trained models, each trained using different post-training techniques. The specific post-training methods applied to each model are listed in Table 4. We present the calibration performance results in Table 11.

D.6 Detailed results for open-ended tasks

For open-ended tasks, we conduct experiments with Qwen2.5 family and Llama-3 family on the TruthfulQA datasets. For the Qwen2.5 family, we choose Qwen2.5-32B as pre-trained models to calibrate all size post-trained models. And for the Llama-3 family, we choose Llama-3-70B as pre-trained models to calibrate all size post-trained models. We present the detailed results in Table 12 to verify the effectiveness of our method.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPost-trained large language models (LLMs) are known to suffer from miscalibration after post-training stages such as supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). This paper proposes leveraging a well-calibrated pre-trained LLM to improve the confidence calibration of post-trained LLMs. Rather than optimizing the negative log-likelihood (NLL) loss on a labeled dataset, a standard approach in existing calibration methods, this work proposes minimizing the KL divergence between the predictive distributions of the post-trained and pre-trained LLMs. The authors further argue that this optimization should be restricted to samples on which both models agree in their predictions. Extensive experiments are conducted, demonstrating the effectiveness of the proposed approach across various settings.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper explores a novel and under-explored setting: unsupervised confidence calibration for LLMs.\n- The proposed method is simple yet effective, with well-justified motivation.\n- Empirical results are convincing and show consistent improvements across various LLMs.\n- The paper is clearly written and easy to follow.\n\n## Weaknesses\n- While the problem setting is crucial, the proposed solution may be impractical in real-world scenarios. It requires deploying both the pre-trained and post-trained LLMs and performing inference with both, which could be computationally expensive.\n- The method is not universally applicable: the temperature parameter is learned specifically for each downstream task. However, the behavior of post-trained LLMs may vary significantly across tasks, potentially requiring different temperature values.\n\nQUESTIONS:\n- Instead of performing post-hoc confidence calibration, is it possible to incorporate the KL loss as a constraint during post-training? This could potentially lead to naturally calibrated predictions without relying on post-hoc temperature scaling.\n\n\n- How well does the proposed method generalize across different tasks? The current approach involves optimizing a temperature parameter for a specific downstream task or domain. Given that different tasks may require different temperature settings, can a **single learned temperature** value effectively calibrate predictions across a diverse set of tasks?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Disagreement-Aware Confidence Alignment (DACA), an unsupervised post-hoc calibration method for post-trained language models (PoLMs). The core insight is that while pre-trained language models (PLMs) are often well-calibrated, post-training (e.g., SFT, RLHF) introduces overconfidence. DACA proposes to align the confidence of a PoLM to that of a PLM using only agreement examples—inputs where both models make the same prediction. This strategy avoids calibration degradation due to disagreement examples, which tend to skew temperature scaling toward under-confidence. The method is simple, effective, and does not require labeled data. It significantly improves Expected Calibration Error (ECE) across multiple datasets and LLM families (including closed-source models like GPT-4o), and can generalize to other post-hoc techniques such as vector and matrix scaling.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The performance is impressive.\n2. The table 2 experiment is interesting, it is a surprise to see such an increase by using other PLMs.\n3. Empirical results are comprehensive across models, datasets (MMLU, MedMCQA, MathQA), and post-training methods (SFT, DPO, RLVR).\n4. Clear experimental baselines and fair comparisons with both supervised and unsupervised methods.\n\n\nWeaknesses:\n1. How you define the confidence distribution of the LLM is important. Should be well defined at the beginning of the analysis. Or people will be lost, how llm expression confidence as traditional models do.\n2. It looks like there are some repeat in Eq6 and Eq7.\n3. To me, the research setting is strange, it is very limited to the white box and even require access to the pre and post training version. Which is hard to obtain in real practice, let alone the multiple inference of the pre trained model. This issue limited the applicability.\n4. The method is limited to multiple option QA.\n5. Table 2 is interesting, I wander if you can run some larger open sources PLMs to achieve even better performance.\n6. Linguistic Confidence is a trending way to express LLM uncertainty, need to be included and discuss [1,2,3]\n7. The method is very limited to the white box and even require access to the pre and post training version. Which is hard to obtain in real practice, let alone the multiple inference of the pre trained model\n\n[1] Revisiting Uncertainty Estimation and Calibration of Large Language Models\n\n[2] Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?\n\nQUESTIONS:\n1. It can be seen that your results can outperform the TS with access to the ground truth label as shown in table 1, could you give some explanations.\n\nLIMITATIONS:\nYes, the limitations are well included in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an unsupervised approach, Disagreement-Aware Confidence Alignment (DACA), to alleviate the overconfidence problem of post-trained language models. Motivated by the intrinsic weakness of temperature scaling, DACA ignores the disagreement samples when optimizing the temperature parameter, demonstrating competitive performance in extensive experiments and shedding light on efficient post-hoc calibration of large language models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. Unsupervised Calibration. As high-quality labeled data is usually expensive to obtain and scale up, DACA explores an unsupervised method for confidence calibration, by innovatively leveraging the well-calibrated confidence of PLMs.\n2. Sufficient Theoretical Analysis. This paper reveals that the failure of naive unsupervised calibration stems from disagreement examples between the two models, thereby providing strong theoretical support for the DACA method.\n3. Extensive Experiments on Classification. DACA’s effectiveness is grounded by sufficient experiments, and it also performs well on (reformatted) QA and selective classification tasks. \n\nWeaknesses:\n\n1. High dependency on the calibration of PLMs. The ceiling performance of DACA is limited to that of the selected PLM. The concept of \"well-calibrated\" PLMs is hard to define.\n2. Like many other post-hoc schemes, DACA learns a single scalar to adjust the model's output distribution, which does not touch on the underlying confidence mechanism of models, and is likely to cause overfitting. On one task, the learned parameter $\\tau$ might be inappropriate to apply to another.\n3. As mentioned by the authors, the disagreement samples are directly eliminated in DACA, which may be critical for better calibration.\n\nQUESTIONS:\n1. In the main experiment, 57 different parameters $\\tau$ are learned for each subject in MMLU. What about learning a single $\\tau$ for all subjects? Is DACA robust under domain shift?\n2. Could you specify the values of $\\pi$ that were observed in your experiments? Furthermore, have you investigated whether the performance of DACA degrades significantly when this ratio is very high (e.g., on extremely difficult tasks), where the agreement examples may become insufficient or distributionally biased?\n3. The definition of `agreement` is ambiguous in real-world LLMs use scenarios. How to improve DACA to help calibrate the confidence of chatbots?\n4. Grammar and typos: L.309, You may add an \"is\" before \"computationally efficient\".\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose Disagreement-Aware Confidence Alignment (DACA), a novel unsupervised method to optimize temperature for post-hoc confidence calibration. They theoretically demonstrate that the confidence of pre-trained language models underestimates the prediction accuracy of post-trained language models on disagreement examples and therefore propose DACA to address this issue. The empirical results show that DACA enhances the calibration of both open-sourced and API-based PoLMs across various dataset.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written and well-structured. The related work section clearly outlines the relationship between existing approaches and the proposed method. The authors first theoretically demonstrate that the confidence of pre-trained language models underestimates the prediction accuracy of post-trained language models on disagreement examples. Then they propose a simple yet practical post-hoc calibration method that can be applied to most existing calibration techniques to enhance performance. Therefore, the proposed method is well-motivated. The authors showcase the superior performance of their proposed methods on multiple popular language models and use appropriate and sufficient evaluation metrics to validate the results. I appreciate how the authors highlight their key findings, such as \"DACA significantly improves the calibration performance of PoLMs\", \"DACA is effective across models of different sizes\", \"DACA is agnostic to the choice of PLMs\", and \"DACA can benefit selective classification.\" These concise summaries effectively emphasize the advantages of their proposed approach. Overall, the proposed approach is both theoretically sound and empirically significant. One potential weakness as noted by the authors is that the disagreement examples are filtered out, which reduces the pool of unlabeled examples available for calibration. It would be helpful if the authors could discuss more about how solve the issue.\n\nQUESTIONS:\nPlease refer to Strengths And Weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}