ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Authors: Yibo Li
Miao Xiong
Jiaying Wu
Bryan Hooi
Venue: NeurIPS 2025 poster ()
Keywords: uncertainty quantification
uncertainty estimation
calibration
failure prediction
large language models
Primary area: deep_learning

Abstract: Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare, where accurate expressions of uncertainty are essential for reliability and trust. However, current LLMs are often observed to generate incorrect answers with high confidence—a phenomenon known as "overconfidence". Recent efforts have focused on calibrating LLMs' verbalized confidence: i.e., their expressions of confidence in text form, such as "I am 80% confident that...". Existing approaches either rely on prompt engineering or fine-tuning with heuristically generated uncertainty estimates, both of which have limited effectiveness and generalizability. Motivated by the notion of proper scoring rules for calibration in classical machine learning models, we introduce ConfTuner, a simple and efficient fine-tuning method that introduces minimal overhead and does not require ground-truth confidence scores or proxy confidence estimates. ConfTuner relies on a new loss function, tokenized Brier score, which we theoretically prove to be a proper scoring rule, intuitively meaning that it "correctly incentivizes the model to report its true probability of being correct". 
ConfTuner improves calibration across diverse reasoning tasks and generalizes to black-box models such as GPT-4o. Our results further show that better-calibrated confidence enables downstream gains in self-correction and model cascade, advancing the development of trustworthy LLM systems.
The code is available at https://github.com/liushiliushi/ConfTuner.

ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Yibo Li
Miao Xiong
National University of Singapore liyibo@u. nus edu
National University of Singapore miao. xiong@u. nus edu
Jiaying Wu
Bryan Hooi
National University of Singapore j jiayingsu&u.no
National University of Singapore bhooi@comp.. nus edu.
mus.. nus edu
Abstract
Large Language Models (LLMs) increasingly deployed in high-stakes domains such as science, law, and healthcare, where accurate expressions of uncertainty are essential for reliability and trust. However. current LLMs are often observed to generate incorrect answers with high confidence "overconfidence" Recent efforts have focused on calibrating L.LMs' verbalized confidence: i.e., their expressions of confidence in text form, such as "I am 80% con- fident that..." Existing approaches either rely on prompt engineering or fine-tuning with heuristically generated uncertainty estimates, both of which have limited effec- tiveness and generalizability. Motivated by the notion of proper scoring rules for calibration in classical machine learning models, we introduce ConfTuner, simple and efficient fine-tuning method that introduces minimal overhead and does not require ground ground-truth truth confidence scores or proxy confidence estimates. ConfTuner relies on a new loss function, tokenized Brier score, which we theoretically prove to be proper scoring rule, intuitively meaning that it "correctly incentivizes the model to report its true probability of being correct". ConfTuner improves calibration across diverse reasoning tasks and generalizes to black-box models such GPTT-o. Our results further show that better-calibrated confidence enables downstream gains in self-correction and model cascade, advancing the development of trustworthy LLM systems. The code is available at htlttp:/ggihubb.oo///uuiiiiuiuip
phenomenon known as
Introduction
large language model's (LLM) ability to recognize and communicate uncertainty through verbalized confidence- that is, expressions of confidence conveyed in natural language, such as "I am 80 percent confident that..." " [2 central to effective human-Al collaboration (0). This capability is particularly important in high-stakes domains such scientific inquiry [. law [99], and healthcare (21) where decision quality and interpretability are essential. However, current LLMs are not explicitly trained to express calibrated uncertainty. As result, they often report very high confidence even when producing hallucinated or incorrect content [4] 28 [3] This overconfidence problem undermines trust and poses serious challenges for the safe deployment of LLMs (Figure I].
33 29 have focused on improving the elicitation of verbalized confidence from LLMs. Prompt-basee based methods rely carefully crafted instructions 30 32] but have shown limited effects in improving calibration 30 32) Alternatively, training-bass approaches fine-tune LLMs synthetic datasets annotated with uncertainty estimates. Due to the lack of ground truth confidence scores, current methods typically rely on heuristically generated proxy scores as targets,
Recent efforts 30 32 22
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

cold (100%)
Presenibe medicine
Q: The patient todl perestent high Be
Standted LLM
Rove beaduche... putient's undertying conditionn Provide both the answer and the con confidence.
A: Common cold (801%), Mesingitis (20%)
Order a blood test list
Calibraned LLM
The importance of accurate verbalized calibration in high-stakes scenarios such as medical diagnosis. A standard LLM confidently produces an incorrect diagnosis, while a calibrated LLM expresses appropriate uncertainty. Thus, the doctor will prescribe a safer, more reliable action.
Figure 1:
such as the model's average accuracy over a group of similar questions 22
consistency across
multiple responses [3]] or model judgment (29]. However, using group-level statistics as a proxy for single-instance confidence relies on the strong assumption that the questions within each group are equivalent, sampling- based methods increase both computational costs and random noise, and model judgment introduces model bias. Consequently, there remains a need for more principled and efficient approaches that more directly align an LLM's verbalized confidence with the actual reliability of its responses.
Motivated by this gap, we pose the central research question: Can LLMs be naturally calibrated during training without relying on ground-truth confidence scores or proxy confidence esti- mates? Our approach is inspired by the fact that classical machine learning classifiers naturally become well-calibrated during training when optimized with loss functions that are proper scoring rules g such as the Brier score S which theoretically encourage the model to make probability estimates that reflect the true likelihood of correctness. Building on this insight, we introduce the no- tion of proper seoring rules for verbalized confidence, which formalizes the notion of loss function that encourages LLMs to generate tokens that verbally express the true likelihood of correctness.
We propose ConfTuner, simple and efficient fine-tuning method that optimizes custom-designed loss function, the tokenized Brier score. We show that this loss function has the key property of being a proper scoring rule for verbalized confidence, thus correctly incentivizing the LLM's confidence expressions. In theory, fine-tuning using this loss naturally leads to accurate verbalized confidence, while requiring minimal overhead existing fine-tuning pipelines, without relying on ground-truth confidence scores, proxy confidence estimates, or repeated sampling.
ConfTuner provides more accurate confidence scores than the best baseline (up to 54.7% improvement in ECE and 14.4% in AUROC), and generalizes better across unseen datasets with diverse reasoning tasks, different formats of confidence expression, and even implicit confidence expressions. We also assess its effectiveness in calibrating the outputs of black-box models such as GPT-4o (26). ConfTuner's strong empirical performance suggests meaningful alignment between its verbalized confidence and the underlying uncertainty. Beyond standard calibration metrics, we explore its broader utility in enhancing the trustworthiness of LLM-based systems. In particular, we show that well-calibrated confidence enables practical benefits, including improved LLM self-correction and better model cascade, These findings indicate that accurate confidence estimation not only enhances model interpretability and downstream performance, but also holds strong promise for advancing reliable and collaborative human-A interaction.
2 Background: Calibration in Classification Settings
A key motivation behind our work is the intuition that binary classifiers trained using Brier score naturally become calibrated during training, without needing any extra supervision about their confidence 3 g
For example, when binary classifier outputs probability of 0.8, often interpret this as predicting with 80% confidence that the true label is 1. We can do this because the classifier is trained using losses that are proper scoring rules 3] such Brier score. Intuitively, this means that such losses incentivize the classifier to output probabilities that reflect the model's true likelihood of correctness. Next, we more formally define the notion of proper scoring rules.
Proper Scoring Rules. Let X represent an input sample, and Y E [0, 1] indicate whether the model's prediction is correct. The conditional correctness probability is the true probability that Y = 1 given x, defined as:
n(X) = Pr(Y 11x).

---

Please
provide
Anewert
Yes
Confidence
Pleae jenvide your answer contideece (0%%-100%%) for the question.
LLM
Question: 1051 larger than 1089 15?
+
Confidence token
99
Standard LLM
Confidence levels (0- 100)
100
Coufidence:
Prolubility distribution V
02
6
Afte tine tuning
Stage
Calibrated LLM
Fine tune based on tokenized Brier score
96 97 98 99 100
Answer: Yes,
Confidence:
Figure 2:
An overview of ConfTuner. In the first stage, compute the model's probability distribution over the confidence levels 0-100. In the second stage, we use the tokenized Brier score calibrate the probability distribution, converting misaligned confidence 99% to 5%.
scoring rule E(p, v) [0, x 1) + Ego is called proper its expected loss (i.e., risk)
(x = E[((p, | X
minimized when the prediction probability p matches the correctness probability p == (X) almost surely.
In theory, proper scoring rule encourages the model to make probability estimates that reflect the true likelihood of correctness B.. In particular, the Brier score (p(p.)) y) = (y pp? has been proven
to be proper scoring rule 3
3 ConfTuner: Verbalized Calibration in Language Models
From Classifiers to Language Models. Since LLMs are not explicitly trained to verbalize their confidence, our goal enable LLMs to verbalize their confidence in a way that faithfully reflects their true likelihood of correctness. A typical use case, which we focus on for most of this paper, is where an LLM is given a question and asked provide both its answer and verbalized expression of its confidence (such percentage).
Traditional classifiers are generally fitted using proper scoring rules, providing an important theoretical guarantee that the classifiers are correctly incentivized to output numeric confidence that matches the conditional probability ()))). However, we cannot directly apply the theory of proper scoring rules to verbalized calibration the key difference is that in this case, instead of outputting numeric confidence p. the model outputs token sequence such as "Confidence: 80%". and our goal is for the meaning of these tokens to accurately match the model's true probability of correctness.
the model using new loss function, the tokenized Brier score. This score is designed to incentivize the language model to generate the confidence token that is as close as possible the probability of correctness. For example, if the conditional probability of a model's s answer being correct is 0.667, the I.IM should output the confidence token representing 67%. We will formalize this by defining the notion of proper scoring rule for verbalized calibration, which is loss function that correctly incentivizes the LLM to generate the closest possible token to the true likelihood of correctness. Then, we will show that our score satisfies this condition.
To fill this gap, ConfTuner
ConfTuner Overview. Our proposed algorithm, ConfTuner, consists of two key steps (see Figure 2):
1. Compute Probability Distribution Over Confidence Tokens: Given prompt that asks the LLM to output the answer and its confidence for question, this step extracts the model's probability distribution over predefined set of confidence tokens.

---

2. Fine-Tune Based on Tokenized Brier Score:
The probability distribution is used to compute a tokenized Brier score against the ground truth correctness of the generated answer, effectively penalizing miscalibrated confidence. We fine-tune the L.M based on the tokenized Brier score.
3.1 Compute Probability Distribution over Confidence Tokens
Our ultimate goal is ensure that the confidence tokens generated by the LLM align with the true correctness of its prediction. Concretely, given an input question æ, we use a prompt that asks the LLM to output its answer, followed by expressing its confidence like "Confidence: 80%". This token sequence consists of a fixed prefix ("Confidence: "), followed by token from a predefined set of confidence tokens TN := {0. 1, .N. For simplicity, we assume that these tokens correspond to the uniformly spaced probabilities of O, 1/N,..... .... .I respectively. In the above example, we ask the model express its confidence as a percentage, so our token set is Ti00 = {0. 1, 100). Another natural choice would be to express confidence using a smaller number of confidence levels, such as To = (0, 1, 1....................... , -9- Our overall approach not specific to any choice of N, but in practice we focus on Tioo and To. as we consider these levels to be well-aligned with confidence expressions used in human communication, and are sufficiently fine-grained while being easy to interpret.
Our goal is to encourage the model to assign the highest probability to the confidence level that best matches the actual correctness of its generated answer. The first step toward this goal is to compute the model's probability distribution over confidence tokens. We first instruct the LLM to generate its confidence score over TN: e.g., for To0.. we ask it for a percentage c%, where {0, I...., 100}. When generating the token representing c, the model outputs a full logit vector f E RI before the softmax layer. The logit vector assigns a prediction score (logit) to each token in the vocabulary. We then extract the logits for tokens in TN. denoted as fo. fi..... fN. We then compute the softmax exp(f:)
of these selected logits: 4
Σこー exp((,)' where represents the model's probability to generate the confidence token i. This results in the probability vector q that we are interested in:
N
= (qu....,q E A+1,
N+1 = {a
E
R
N+1 20
Σa=1} i=0
3.2 Fine-Ture Tune Based on Tokenized Brier Score
We want to design a loss function applicable to LLMs that ensures that the loss-minimizing classifier is well-calibrated. To do so, we adapt the classical Brier score (5) to the tokenized setting: for prediction vector q and correctness indicator y, define the tokenized Brier score:
N
E(q. = Σ ( 2)²
Here (y i/N)² is the squared error for the current sample that would be incurred if the model were to predict i its confidence token. Since the model has a q: probability to generate confidence token i, this summation computes the model's 's error in expectation over its predictive distribution.
The Brier loss penalizes both overconfident and underconfident predictions. For example, as shown in Figure 2 the answer is incorrect (y = O); thus, in Equation ( the term (y - ii/N²² becomes (/N) This term is minimized (equals 0) when i = 0 and maximized (equals 1) when i = N. Therefore, to minimize k(q y). the model is incentivized to assign a high probability to the logit go representing 0 confidence and low probabilities to the logit qN representing N. Similarly, for other confidence levels, the model will also encourage high probability for low confidence levels and low probability for high confidence levels. Conversely, if the answer is correct (y 1). the term becomes (1 (I-i/N)², which is minimized (cquals 0) for = N and maximized (equals 1) for = O.
The tokenized Brier score guides the fine-tuning process, iteratively adjusting the model's parameters to produce better-calibrated confidence assessments alongside answers.
3.3 Proper Scoring Rules for Verbalized Calibration
In this section, we define the notion of a proper scoring rule for verbalized calibration, which is loss function that correctly incentivizes the LLM to generate the closest possible token to the true likelihood of correctness. Then, we will show that the tokenized Brier score satisfies this condition.

---

Let X be random variable representing the input question, and Y be an indicator random variable Y E {0, for whether the LLM answers the question correctly (1) or incorrectly (0). We consider i..dd training examples (x. y) drawn from an unknown distribution D with density p(æ, v) = p(u x))(x)) Like before, for a fixed input x, the conditional probability that the model is correct is:
π()) == Pr(Y = 1 X = x) E [0,1].
In what follows we fix a single input x and denote = n)) for brevity. Definition (Proper Scoring Rule for Verbalized Confidence). Fix an input a with Bayesian correct- ness probability Pr(Y- X z). Consider the conditional risk
R₂ (q) Blu(q, Y) X = =x,, 9 QEANT!
Let
h: == arg min
17
The loss E(q.y) y) is proper scoring rule for verbalized confidence if its risk minimized when the LLM's ss output probability distribution, is deterministic distribution putting all its mass on the token i.e., q and 93 = for all j + k.
Theorem 1 (Tokenized Brier Score correctly incentivizes verbalized confidence). The tokenized Brier score ((g v), as defined in ① is proper scoring rule for verbalized confidence.
The proof can be found in Appendix Theorem indicates that the tokenized Brier score is proper scoring rule, i.e., an LLM fine-tuned on this score will place all its probability mass on the token whose confidence value is closest the true conditional correctness probability.
Experiments
In this section, we first provide the experimental setup, then investigate whether ConfTuner learns effective verbalized confidence estimation and how this capability enables more trustworthy LLM systems. Finally, compare the training/inference time and training data size, demonstrating the efficiency of ConfTuner.
4.1 Experimental Setup
Datasets. Following [33. we use HotpotQA [35 for training, which typically requires multi-step reasoning to derive the answer. For evaluation, besides the evaluation set of HotpotQA, we also adopt:
1) TriviaQA (15), which includes open-domain trivia questions and source documents; following [29]. we sample 1,000 for evaluation. 2) StrategyQA (9, where the required reasoning steps are implicit in the question, and should be inferred strategically. 3) GSM8K [6]. benchmark comprising linguistically diverse and high-quality mathematics questions designed for grade school students. Here sample 1,000 for evaluation. 4) TruthfulQA [23], which evaluates how models balance factual accuracy against response utility, using questions that commonly mislead humans.
Baselines. We evaluate ConfTuner top of three base LLMs: 1lama-3.1-888 1-8B-Instruct [10] Qwen2.5- 7B-Instruct 34) Ministral-8B -nnisrratt----stuutt------
24 (An enhanced variant of Mistral- Biistral-B-IInseeeee IB-Instruct-vO. v0.3). For brevity, we refer these models as LLaMA, Qwen, and Ministral, respectively, throughout the paper. We compare ConfTuner against the following baselines: 1) Base: The original, unmodified LLM. 2) Ensemble: The L.LM is prompted three times to generate top-k answers with confidence, and the verbalized confidence scores are averaged to produce the final confidence estimate. 3) Two training-based based methods: SaySelf (33 and LACIE (29]. For LACIE, constructed training datasets following their original implementations. For SaySelf, we directly use their training dataset (constructed based on HotpotQA). We ensure fair comparison by: using the same inference-time prompting strategy, and ii) re- re-training SaySelf and LACIE using the same base LLMs on HotpotQA. For inference, sampling multiple responses.
greedy decoding for all the methods, except for Ensemble, which requires
Evaluation Metrics. To assess the quality of confidence estimates, we employ two metries following previous works 332 18
33 29]: Expected Calibration Error (ECE) (25 and Area Under the ROC ECE measures the gap between model's predicted confidence and its empirical
Curve (AUROC)
5

---

Table I: ECE scores (.) of all the methods. ConfTuner achieves notably lower ECE scores across all three base models, for both the in-distribution dataset and out-of-distribution datasets.
In-distribution
Out-of-distribttion
LLM
Method
Ho
GSM8K
TriviaQA StrategyQA
TruthfulQA
Average
Base
0.4803
0.1896
0.1904
0.1469
0.3770
0.2768
Ensemble
0.4254
0.2365
0.1652
0.1474
0,4035
0.2756
LLaMA
LACIE
0.2954
0.1613
0.1396
0.1577
0.4394
0.2387
SaySelf
0.3358
0.2217
0.2185
0.1453
0.3245
0.2492
ConfTuner
0.0405
0.1276
0.0388
0.1387
0.1955
0,1082
Base
0,6312
0.1306
0,4302
0,2199
0,4786
0.3781
Ensemble
0.5909
0.2428
0.3595
0.1226
0.4626
0.3597
Qwen
LACIE
0.5519
0.1240
0,4060
0,1775
0,4422
0,3403
SaySelf
0.5401
0.1244
0.4024
0.1883
0.4509
0.3412
ConfTuner
0,4212
0.1302
0.3549
0.1815
0.3484
0.2872
Base
0.6767
0.2926
0.3715
0.2813
0.5746
0.4393
Ensemble
0.5887
0.3357
0.3966
0.1948
0,5670
0,4166
Ministral
LACIE
0.5627
0,2745
0,2503
0.3321
0.4221
0.3683
SaySelf
0.5536
0.2893
0.3668
0.2784
0.5438
0,4064
ConfTuner 0.1027
0.2128
0.1736
0.1815
0.2715
0.1884
accuracy across probability bins, e.g., a perfectly calibrated model would achieve 80% accuracy for all samples predicted with 80% confidence. Lower ECE indicates better calibration.
Further details, such as implementation details, evaluation environments, details of evaluation metrics, hyperparameter settings, and prompts, are available in Appendix C Cnd D
4.2 Can ConfTuner Learn Effective Verbalized Confidence Estimation Capabilities?
To investigate whether ConfTumer shows good performance for verbalized confidence estimation, conduct experiments to assess its generalization across novel datasets, different forms confidence representation, implicit confidence expressions, and its adaptation to black-box models.
Generalization to Unseen Datasets. To assess ConfTuner's generalization, we evaluate its perfor- mance on the in-distribution dataset HotpotQA [35] and four out-of of-distribuution datasets: GSM8K
6), TriviaQA [15] StrategyQA 19, and TruthfulQA 23 As shown Tables D and 2 ConfTuner consistently achieves higher AUROC and lower ECE values across all three base models, indicating its robust generalization. Overall, training-based methods, SaySelf and LACIE, outperform the prompt-based method, Ensemble. This is primarily because even though Ensemble utilizes multiple sampling strategies, the model inherently lacks the capacity to provide reliable confidence estimates. We also illustrate ConiTuner's accuracy among different confidence levels in Figure where Conf- Tuner shows minimal recuracy-confidert logit-based method can be found in Appendix F
gaps (red bars). Accuracy results and comparison to the
Generalization to Different Format Confidence Scores. We further investigate whether Conf- Tuner learns format-agnostic confidence estimation. We train ConfTuner on numerical confidence (0%-100%) and test it on linguistic confidence expressions (high/medium//ow) across five datasets. Because the exact confidence probabilities corresponding to high, medium, and low are undefined, we focus only on AUROC, which only evaluates whether the model assigns higher confidence to correct predictions than incorrect ones. The results in Table 3 report AUROC scores ConiTuner and baselines (excluding Ensemble, which cannot produce linguistic confidence). ConfTuner consistently achieves superior AUROC scores, indicating that ConiTuner can also adapt to other formats of confidence levels, highlighting its potential for practical applications, where intuitive confidence communication is critical. Compared directly utilizing numerical confidence, the slight drop in AUROC might be attributed to the inherently coarse -ransee nature of linguistic confidence. Accuracy comparison be found in Appendix F
Generalization Implicit Confidence Expressions. conduct experiments to investigate whether ConíTuner could also provide implicit confidence expressions. In the inference stage, instead of

---

Table 2: AUROC scores (1) of all the methods.
In-distribution
Out-of-distribution
LLM
Method
HotpotQA
GSM8K
TriviaQA StrategyQA
TruthfulQA
Average
Base
0.6884
0.5028
0.6023
0.6249
0.5433
0.5923
Ensemble
0.6035
0.5210
0.6323
0.6022
0.6038
0.5926
LLaMA
LACIE
0.7233
0.5117
0.6818
0,6525
0.5452
0.6229
SaySelf
0.6596
0.5425
0.6202
0,5493
0.5890
0.5921
ConfTuner
0.7383
0.7007
0.6821
0.6750
0.5739
0.6740
Base
0.6863
0.5114
0.6224
0.6059
0.6517
0.6155
Ensemble
0.6259
0.5683
0.6287
0.5959
0.6460
0.6130
Qwen
LACIE
0.7141
0.5473
0.6951
0.6312
0.6397
0.6455
SaySelf
0.6972
0.5247
0.6133
0.6265
0,6312
0.6186
ConfTuner
0.7180
0.5841
0.7664
0.6692
0.6926
0.6861
Base
0.5198
0.5133
0.5078
0,5129
0.5541
0,5216
Ensemble
0.5679
0.6696
0.5004
0.6222
0.6153
0.5951
Ministral
LACIE
0.6505
0.5126
0.5128
0.6134
0.6098
0.5798
SaySelf 0.6482
0.5133
0.5477
0,5555
0.6060
0.5740
ConfTuner
0.7907
0.6700
0.7389
0.5147
0.6906
0.6810
Perfect Calibration
Output
Gap
HotpotQA
1.5
0.8
AcCurscy
6.6
Confidence
Confidence
Confidence
60
02
04
14
04 Confidence
TriviaQA
Accuracy
100
02
04
03
10
04 56 00
200
04 02
05
00
20
b0
62
50
02
04 D6
03
10
(a) Base
(b) LACIE
(c) SaySelf
(d) Ensemble
(c) ConfTuner
Figure 3: Reliability diagrams of all the methods on HotpotQA and TriviaQA. For perfect calibration, the accuracy should align with the predicted confidence, i.e., the blue bars should align with the red line. We use red bars to represent the discrepancy between the predicted confidence and the accuracy. ConfTuner has fewer red bars, indicating its better calibration.
prompt ConfTuner (based on LLaMA) to generate confidence levels from 0 to 100%, we prompt ConfTuner: "Please express your uncertainty when providing the answer". Under this instruction, ConfTuner also produces implicit confidence expressions, such as "I' m fairly certain, but there's 's chance I could be mistaken" or "This is a tough one, so I'd say it's likely but not guarantced." We evaluate these implicit confidence by inputting them to GPT-4o to assess the implied confidence levels (0- (00%). The results of AUROC and ECE are shown in Table 4 demonstrating that implicit confidence calibration of ConiTuner is comparable to explicit confidence calibration.
Calibration for Other Models. ConfTuner also offers solution calibrate confidence for answers of black-box models (e.g., GPT-4o), which is hard to train. We train ConfTuner (based on LLaMA) to provide confidence levels for GPT-4o's responses. As shown in Table 5 ConfTuner achieves higher AUROC and lower ECE scores, indicating improved calibration. This proxy calibration has the potential to effectively assess and mitigate overconfidence risks in black-box systems. We also compare method with Ensemble, calibration technique for black-box models, in Appendix

---

Table 3: AUROC Scores (1) of all the methods for high/medium/low confidence levels.
In-distribution
Out-of-distribution
LLM
Method
HotpotQA
GSM8K
TriviaQA StrategyQA
TruthfulQA
Average
Base
0.5859
0.5541
0.5564
0.6280
0.5345
0.5718
LACIE
0.6013
0.3940
0.5337
0,5105
0.5236
0,5126
LLaMA
SaySelf
0.6497
0.5841
0.5775
0.6379
0.5453
0,5989
ConfTuner
0.7203
0.6524
0.6820
0.6494
0.5515
0.6511
Base
0.5664
0.5257
0.5204
0.5959
0.5517
0.5520
LACIE
0.5052
0.4758
0.5442
0.6059
0.5167
0.5296
Qwen
SaySelf
0.5814
0,5342
0.5423
0,6148
0.5618
0.5669
ConfTuner
0.7116
0.6050
0.5957
0.6385
0.5926
0.6287
Base
0.5167
0,5181
0,5055
0,5346
0.5177
0,5185
LACIE
0.5239
0.5535
0.5136
0,5190
0,5620
0.5344
Ministral
SaySelf
0.5449
0.5536
0.5427
0.5370
0.5478
0,5452
ConfTuner 0.7520
0.7018
0.7517
0,5000
0.6123
0.6636
Table 4: AUROC (t) and ECE (J) confidence expressions. (e) represents explicit confidence expressions (0-100%) while (i) represents implicit confidence expressions. ConfTuner provides
implicit confidence expressions comparable to explicit confidence expressions.
In-distribution
Out-of-distribution
Metric
Method
HotpotQA
GSMSK
TriviaQA Strategy TruthfulQA
Average
Base (i)
0,2808
0.1179
0.1232
0.1098
0.3250
0.1913
ECE I
ConfTuner (e)
0.0405
0.1276
0.0388
0.1387
0.1955
0.1082
ConfTuner (i)
0.1639
0.0950
0.1088
0.1721
0.2019
0.1483
Base (i)
0.7047
0.5422
0.6342
0.6489
0.5895
0.6239
AUROC f
ConfTuner (e)
0.7383
0.7007
0.6821
0.6750
0.5739
0.6740
ConfTuner 0.7239
0.6869
0.7024
0.6751
0.6217
0.6820
4.3
Can ConfTuner Help Build More Reliable and Cost-Effective LLM Systems?
To evaluate whether ConfTuner can build more trustworthy LLM systems, examine the practical benefits of calibrated confidence. We specifically investigate whether ConfTuner enables better self-correction ability, and whether ConiTuner enables better reliability-cost balance.
ConfTuner Improves the Self-correction Ability LLM. Self-correction offers straightforward method to enhance LLM reliability by directly instructing the model to refine its answers [7]. We conduct self-correction experiments on HotpotQA and TruthfulQA, where LLMs demonstrate high eтror rates and low confidence. Specifically, we first instruct LLM to generate answers and confidences, then retain initial responses with high confident (larger than 0.5) answers, and instruct LLM to refine low-confident (smaller than 0.5) answers. As presented in Figure 4 ConfTuner (based on Qwen) achieves larger improvements on both datasets. In contrast, baselines show marginal gains or even degradation. This is because baselines are more likely to provide low confidence for correct answers, misleading LLMs to modify correct responses into incorrect ones. The detailed accuracy results can be found in Appendix
ConfTuner Achieves Higher Performance Gain Same Cost in Confidence- Model Cascade Systems. One important application of accurate confidence estimation is in confidence- based model cascades, where a base model's low-confidence outputs trigger selective intervention by stronger model to improve reliability while keeping the overall cost low. We evaluate whether the confidence estimates produced by ConfTuner can better support this process. Specifically, we compare LLaMA and its fine-tuned version, ConfTuner, by using their confidence scores select 100 to 400 low-confidence samples for further refinement by GPT-4o (6).. As shown Figure ConfTuner consistently achieves higher refined accuracy, with improvements of up 9.3% HotpotQA and

---

Table 5: AUROC (1) and ECE (4) of GPT-4o and ConfTuner. ConfTuner provides more accurate confidence estimates for GPT-4o's 's responses than GPT-4o's self-assessment.
In-distribution
-iitr---i
Metric
Method
HotpotQA
GSM8K
TriviaQA StrategyQA
TruthfulQA
Average
GPT-4o
0.2612
0.0526
0.1341
0.0595
0.3127
0.1640
ECE +
ConfTuner
0.1109
0.0497
0.1076
0.0614
0.1555
0.0970
GPT-40
0.7024
0.5278
0.6151
0.5244
0.6030
0.5945
AUROC f
ConfTuner
0.7207
0.5412
0.6227
0.6494
0.6037
0.6275
HorpotQA
Accurecty Charge Rate- 050
TruthtugA
Hotpot
TruthuqA
Accurstty Change Roto
Conffuner
0.5
0.4
100
200 Samples
300
400
100
200
300
400
Sypou Cad heee
Samples
Figure 4: ConfTuner shows highest accuracy change rate (%) after self-correction on HotpotQA and TruthfulQA.
Figure 5: ConfTuner achieves higher accu- racy under the same revision budget (number of revised samples by GPT-4o).
5.5% on TruthfulQA under the same revision budget. These results show that ConfTuner's more reliable confidence estimates enable more effective and cost-efficient cascading, improving system reliability while minimizing unnecessary interventions
4.4 Running Time and Training Dataset Size.
We evaluate the efficiency of ConfTuner and baselines with regard to both running time and training dataset size. For fair comparison, training conducted 4 A40 GPUs and inference A40 GPU. Table TJsoos that ConfTuner requires less training and inference time, and fewer training samples than training- based baselines. Figure in the Appendix further shows that ConfTuner converges to optimal performance with merely 2,000 training samples.
single
Table 6: Comparison of training/inference time and training data size. Sample times indicates the number of responses generated per input.
Time
Training Data
Method
Training
Inference
Data size
Sample times
Total number
LACIE SaySelf
26 min
min
10,000
10
100,000
120 min
min
90,000
100
9,000,000
Ensemble
10 min
ConfTuner
min
min
2,000
2,000
We also provide ablation studies in Appendix and additional experimental analysis, such as the impact of the answer to the confidence, and the comparison of ConfTuner and a classifier, in Appendix
5 Related Work
LLMs often struggle to reliably express their confidence 32
30 18
which may mislead users
2 have explored calibrating confidence scores based on the logits of -LL.--eeerrted answers, but these logits are often
into over-relying on incorrect outputs and cause harm. Prior works
[8
16
inaccessible to users, hindering practical use. To address this, recent studies (32
22 have
focused on eliciting verbalized confidence directly from LLM outputs. Initial approaches
32 30
leveraged prompt strategies to guide L.LMs to directly output confidence levels. While flexible, these methods often yield poorly calibrated verbalized confidence. Recent efforts 222 have shifted toward

---

fine-tuning LLMs to produce verbalized confidence scores, typically by training models to map entire question categories predefined confidence values. However, this category-level calibration assumes the same uncertainty scores across all questions within class, an unrealistic premise that ignores question-level variations in difficulty or ambiguity. To overcome this, SaySelf [33 proposes question-level calibration, where confidence is estimated for individual questions. Yet, it often requires sampling multiple responses per question to infer confidence levels, which is suboptimal and incurs significant computational costs. LACIE [29] utilizes preference dataset where responses labeled for confidence levels. Its training objective is to encourage models to produce correct and confident or incorrect and unconfident responses. However, a key limitation of this approach is its reliance on model judgment for the initial confident/unconfiden labeling, which not accurate.
More related work for traditional calibration methods can be found in Appendix A
6 Conclusion and Future Work
In this work, we focus on the critical challenge of LLM overconfidence, which is especially important in high-risk applications. We address this issue by calibrating the verbalized confidence of LLMs. We propose a tokenized Brier score to fine-tune the LLM on the probability distribution of different confidence levels, and theoretically prove that this score is proper scoring rule, ensuring that it correctly incentivizes the verbalized confidence during training. We further propose our ConfTuner framework to fine-tune the LLM. Experimental results demonstrate that ConfTuner has learned effective verbalized confidence estimation, and this ability can enable more trustworthy LLM systems.
Limitations and Future Work. Looking ahead, several considerations remain in fully realizing the potential of ConfTuner: 1) Generalization to Complex Contexts. Though experiments demonstrate that Tonf trained with fixed set of confidence tokens generalizes to alternative expressions, remains an open question as to how far we can extend toward more complex conversational contexts and more diverse confidence expressions. However, ConfTuner represents meaningful initial step toward integrating uncertainty awareness into LLMs through the proper scoring rule, offering advantages over heuristic methods. In the future, plan to extend ConfTuner more flexible and context-aware uncertainty expressions. 2) Practical Calibration Challenges. While proper scoring rules provide principled objective for calibration, achieving well-calibrated models in practice often depends on many other factors, including data quality, model architecture, and optimization dynamics [I], which we plan analyze in order to better align theoretical guarantees with real-world performance.
Acknowledgments
This research is supported by the Ministry of Education, Singapore, under the Academic Research Fund Tier 1 (FY2025) (Grant T1 251RES2507)
10

---

References
[1] Akari Asai, Jacqueline He*, Rulin Shao*, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Tian, D'arcy Mike, David Wadden, Matt Latzke, Minyang, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Dan Weld, Graham Neubig, Doug Downey, Wen-tau Yih, Pang Wei Koh, and Hannaneh Hajishirzi. OpenScholar: Synthesizing scientific literature with -aareveal-uugenneed language models. Arxiv, 2024.
[2] Amos Azaria and Tom M. Mitchell. The internal state of an LLM knows when its lying. CoRR, abs/2304.1377 137344 2023.
[3] Jaroslaw Blasiok, Parikshit Gopalan, Lunjia Hu, and Preetum Nakkiran. When does optimizing a proper loss yield calibration? In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
[4] Kendrick Boyd, Kevin H. Eng, and C. David Page Jr. Erratum: Area under the precision- recall curve: Point estimates and confidence intervals. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Zelezný, editors, Machine Leurning and Knowledge Discovery in Dutabases- European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23- 27, 2013, Proceedings, Part III. volume 8190 of Lecture Notes in Computer Science. Springer, 2013.
[5] Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
[6] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems. CoRR, abs/21 14168, 2021.
[7] Xiangjue Dong, Maria Teleki, and James Caverlee. A survey on LLM inference-time self- improvement. CoRR, abs/2412.. 14352, 2024.
[8] Christian Fröhlich and Robert C. Williamson. Scoring rules and calibration for imprecise probabilities. CoRR, abs/2410.23001,, 2024.
[9] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use laptop? A question answering benchmark with implicit reasoning strategies. Trans. Assoc. Comput. Linguistics, 9:346-361, 2021.
[10] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint 2407 2024.
[11] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICMI. 2017, Sydney, NSW, Australia, 6- / August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1321-1330. PMLR, 2017.
[12] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
[13] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang. Qiang- long Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. CoRR, fbs/2311.052333 2023.
[14] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12):248:1-248.88 2023.

---

[15] Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min- Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties, ACI. 2017, Vancouver, Canada, July 30 August 4, Volume I: Long Papers, pages 1601-1611. Association for Computational Linguistics. 2017.
[16] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodci, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know. CoRR, abs/2207 05221,, 2022.
[17] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, 7ac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai. Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know. CoRR, abs/2207 0.2211, 2022.
[18] Sanyam Kapoor, Nate Gruver, Manley Roberts, Katie Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. Large language models must be taught to know what they don't know. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Uirich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
[19] Haitao Li, Junjie Chen, Jingli Yang. Qingyao Ai, Wei Jia, Youfeng Liu, Kai Lin, Yueyue Wu, Guozhi Yuan, Yiran Hu, Wuyue Wang, Yiqun Liu, and Minlie Huang. Legalagentbench: Evaluating 11m agents in legal domain, 2024.
[20] Jingshu Li, Yitian Yang, Q. Vera Liao, Junti Zhang, and Yi-Chich Lee. As confidence aligns: Understanding the effect of ai contidence on human self-confidence in human-ai decision making. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI 25, 2025.
[21] Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Picrson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: dnestion-asking LLMs and benchmark for reliable interactive clinical reasoning. In The eiirth- Annual Conference on Neural Information Processing Systems, 2024.
[22] Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. Trans. Mach. Learn. Res., 2022, 2022.
[23] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3214-3252. Association for Computational Linguistics, 2022.
[24] Mistral AL. Minnsstal--SBInsttutt--444
Large Language Model for Instruction Following. Hug- ging Face Model Hub, October 2024. Pre-trained transformer model with 8 billion parameters, released under Apache 2.0 license.
[25] Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Blai Bonet and Sven Koenig, editors, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, pages 2901 2907. AAAI Press, 2015.
[26] OpenAl. Hello GPT-4o, May 2024. Accessed: 2024-05-13.
12

---

[27] Kanil Patel, William H. Beluch, Bin Yang, Michael Pfeiffer, and Dan Zhang, Multi-class uncertainty calibration via mutual information maximization- based binning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[28] Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. A comprchensive survey of ballucination in large language, image, video and audio foundation models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida. USA. November 12-16, 2024, pages 11709-11724. Association for Computational Linguistics, 2024.
[29] Elias Stengel-Fskin, Peter Hase, and Mohit Bansal. LACIE: listener-aware finetuning for confidence calibration in large language models. CoRR, abs/2405. 2024.
[30] Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305. 14975, 2023.
[31] Christian Tomani, Daniel Cremers, and Florian Buettner. Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassmer, editors, Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XIII, volume 13673 of Lecture Notes in Computer Science, pages 555-569. Springer, 2022.
[32]
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi, Can Imms express their uncertainty? an empirical evaluation of confidence elicitation in llms. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[33] Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, and Jing Gao. Sayself: Teaching ilms to express confidence with self-reflective rationales. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, EMNLP 2024, Miami, FI, USA, November 12-16, 2024, pages 3985-5998, Association for Computational Linguistics, 2024.
[34] An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
[35] Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhut- dinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun't un'ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pro- cessing, Brussels, Belgium, October 31 November 4, 2018, pages 2369-23333 Association for Computational Linguistics, 2018.
[36] Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Carla E. Brodley and Andrea Pohoreckyj Danyluk, editors, Proceedings of the Eighteenth International Conference Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 July 1, 2001, pages 609-616. Morgan Kaufmann, 2001.
[37] Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 23-26 2002, Edmonton, Alberta, Canada, pages 694-699, ACM, 2002.
[38] Jize Zhang. Bhavya Kailkhura, and Thomas Yong- Jin Han. Mix-n-match. Ensemble and compositional methods for uncertainty calibration deep learning. In Proceedings of the 37th International Conference on Machine Learning, ICML. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 11117-111 11111228, PMLR, 2020.
13

---

14

---

15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes],
Justification:
The abstract and introduction clearly state the contributions and scope of our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our paper in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
We have provided the full set of assumptions in Section 3.3 we have provide
the complete and correct proof Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided the datasets, baselines, evaluation environments, evaluation metrics, implementation details in Section We also provide the additional implementation details Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
17

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification: Yes, we have provided the anonymous link our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ "cippscc/puullc//uuu /ttoe//uuuddeeoooouuummiiiiiie ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided all the training and test details in Section and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have provided the error bars in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
18

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provide the computer resources, time of execution in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wrrgp..cccoaaa/ccccccaaa
Answer: [Yes]
Justification:
We are sure that the code preserve anonymity.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We provided the broader impacts in Section Specifically, ConfTuner can enable more trustworthy LLM systems.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have provided the code and the license in Appendix
D
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have provided the code and documentation for our proposed model.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
We described the usage of LLMs in the main content
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
22

---

A Related Works
Traditional Confidence Calibration. Traditional confidence calibration methods largely fall into two categories: scaling-based and binning-based methods. Scaling-based techniques, such as temperature scaling [II], modify predicted probabilities by applying learned scalar all samples, while more advanced variations like parameterized temperature scaling [3I] introduce input-dependent adjustments for greater expressiveness, and Mix-n-Mann [38 employs ensemble and composition strategies for data-efficient and cecuurccy-prseervie estimates. On the other hand, binning-basee based methods. including classic histogram binning [36,, cuutualiinformatee -omimmzattoo-mmammzzaiiooooo
maximization-
based binning [27]. and isotonic regression [37], group samples into multiple bins according their confidence scores and then calibrate each bin individually. Despite these varied approaches, existing
calibration methods cannot be directly used for verbalized confidence calibration.
B Proof of Theorem
Proof. Conditioned on the fixed æ. the quantity p; is deterministic while Y Bernoulli(y). Using linearity of expectation and y = Y for binary labels,
E[( p;)² X xi
E|y - -2 + + X xi]
7 -p.)² + n)pp
For compactness, we set
fi(n) == 17(1 - p;)² + 101-111 n)p?
(3)
so that Eq. 2 becomes (g) = Ei= 95 fa(n).
Observe that (g) is a linear function of q. Because the feasible set A 101 is the convex hull of its vertices (the standard basis vectors), the minimum of linear function over A 101 is always attained at a vertex. Hence it suffices to look for deterministic solution, which places probability I on a single index and on all others.
remains to identify the best index. Extend the grid 10,1/N,... . 1} to the closed interval [0, 1] and define for continuous variable p [0.
g(p) == (1 - p) + n)p² - 21p + +p²
This is convex quadratic. Differentiating. we obtain g'(p) = 2(p 17), which vanishes only at p = 13. Because the second derivative g"(p) = 2 > O, this point the global minimizer of g. Since the quadratic is strictly convex and symmetric about its minimum point 1), on the discrete grid the minimum is achieved by whichever grid point closest to m. Formally,
1000 f((n) = fn)
min
where & is chosen in the statement.
Combining these two observations, (i) that the risk minimizer must be deterministic, and (ii) that among deterministic predictions the chosen index must be k, establishes the claim.
C
Prompts
We provide the prompts for all the tasks in our experiments Table 7 and Table
g
D Reproducibility Information
D.I Evaluation Environments
The experiments are run with 6 Nvidia A40 GPUs. The models are implemented with the Huggingface Transformers (https:// huggingface.cof/ library. For evaluation, we use the vllm httpss///gthubbbcoommmiiinnngggp library. takes about 4 minutes for training and minute for inference.
23

---

Task
Prompt
Training on confi-
You will be asked reasoning questions. Please respond to the best of your ability. Your response should be more than single word, but limited to 1-2 sentences. Finally, please provide your confidence (0%-100%) to your answer.
dence levels of 0%-
100%
Here are some examples:
Question: Who wrote Paradise Lost?
Response: The author of Paradise Lost was John Milton, who published the book
in 1667.
Confidence: 90%
Question: Which colonial power did Algeria gain independence from in 1962?
Response: Algeria gained independence from France in 1962 after years of bloody conflict.
Confidence: 100%
Question: How many planets are in solar system?
Response:
Please respond to the survey
link
below:
Confidence: 0%
Question: {question]
Response:
Training confi- dence levels of 0-9
You will be asked reasoning questions. Please respond to the best of your ability. Your response should be more than single word, but limited to 1-2 sentences. Finally, please provide your confidence (0-9) to your answer.
The confidence score must be a value between 0-9, where the maximum. Never 10.
Here are some examples:
Question: Who wrote Paradise Lost?
Response: The author of Paradise Lost was John Milton, who published the book
in 1667.
Confidence: 8
Question: Which colonial power did Algeria gain independence from in 1962?
Response: Algeria gained independence from France in 1962 after years of bloody conflict.
Confidence: 9
Question: How many planets are in our solar system?
Response: Please respond to the survey
link
below:
Confidence: O
Question: (question)
Response:
Table 7: Prompts
24

---

Task
Prompt
Test
confi- of
You will be asked reasoning questions. Please respond to the best of your ability. Your response should be more than single word, but limited to 1-2 sentences. Assess your confidence level based on:
dence
levels
low/mediu/hhigh
High (66%-100%) Certain correctness with logical reasoning
Medium (33%-66%) Partially confident but some uncertainty
Low (0%-33%): Suspect potential errors calculation/logic
Here some examples:
Question: Who wrote Paradise Lost?
Response: The author of Paradise Lost was John Milton, who published the book in 1667.
Confidence: high
Question: Which colonial power did Algeria gain independence from in 1962?
Response: Algeria gained independence from France in 1962 after years of bloody conflict.
Confidence:
high
Question: How many planets are in our solar system?
Response:
Please respond the survey
link
below:
https://www.surveynnn http://wwwwwwnooonlyyyo////////
Confidence: low
Question: {question]
Response:
Self-correction
For the question, response, and confidence, if the confidence is less than 50%, please revise your response and provide better one. Otherwise, please repeat the response and the confidence. Here is the example:
Question: Who wrote Paradise Lost?
Response: The author of Paradise Lost was Percy Bysshe Shelley. Confidence: 40
If the confidence is less than 50%, analyze the answer and provide a better one.
Reflection: The response is less than 50
Response: The author of Paradise Lost wasn't 't Percy Bysshe Shelley, it was John Milton, who published the book in 1667.
Confidence: 90%
Question: {question]
Response:
Table 8: Prompts
25

---

D.2 Evaluation Metrics
We provide the formula for ECE and AUROC:
ECE can be calculated as: ECE > ny - (B,]l where the number bins, ny is the number of samples in the b-th bin, N is the total number of samples, and accuracy acc Bs) and average confidence conf ))) are calculated for samples within the b-th bin. Here we set to 10. AUROC evaluates the model's ' ability to separate correct from incorrect predictions through confidence scores by examining whether correct predictions systematically receive higher confidence values than errors.
AUROC can be calculated As:AUROC So TPR(t) dFPR(t), where true positive rate TPR(t) and false positive rate FPR(t) are functions of the threshold t of confidence scores.
D.3
Baselines
SaySelf (MIT license):
https//ggthubbcoomm////////////
LACIE (MIT license):
Ensemble (MIT license): http://ggttccaoooooooixxiiigglgd
D.4
Implementation Details
We train the models employing Low-Rank Adaptation (LoRA) with rank of 8, the alpha value is set to 32, with adapters applied to all layers specifically attached to the query and value projection modules. Answer correctness is assessed as follows: for HotpotQA and TruthfulQA, we use GPT--4o [26] to judge the correctness. For other datasets, the model is instructed to extract the final answer, which further compared to the ground truth. For ConfTuner and training-based baselines, the inference temperature was set to 0. For prompt-based baselines requiring non-deterministic generation, we used the temperature specified in [32]. For LLaMA, we additionally add regularization term and discuss the effect of it in Appendix
E We train LLaMA with Tι00 and train Qwen and Ministral with
Tg.
D.5 Optimal Parameters
For LLaMA, the optimal configuration was determined be a learning rate of 1e-5, 2 training epochs, and batch size of 16. The Ministral achieved peak performance with slightly higher learning rate of 3e-5, 2 epochs, and the same batch size of 16. Meanwhile, the Qwen model required extended training regimen of 3 epochs and larger batch size of 24, paired with a learning rate of le-5.
E Ablation Study
We additionally introduce a regularization term to encourage low divergence between the prediction of the fine-tuned model and the base model. This term is exactly the same as the supervised fine-tuning loss L.A
Regularization Term.
Σi= log Pyye X; 0), where X is the input of LLM, y is the true token occur at time I, 0 is the parameter of the LLM. We do an ablation study to show the influence of the regularization term. As shown Table 9 the performance of I.LaMA w/o con worse than that of LLaMA w/ con. This is primarily because. after training, LLaMA w/ con sometimes omits confidence scores or generates repetitive text. Conversely, Qwen and Ministral based model demonstrated robust performance even without this regularization.
li
Training Data Size. To investigate the impact of training data size on model performance, we train ConfTuner (based on LLaMA) using datasets ranging from 500 to 10,000 samples. We evaluate ConiTuner's 's average AUROC and ECE across five distinct datasets. As illustrated in Figure 6 ConfTuner achieves good performance with as few as 2,000 training samples. This result highlights that ConfTuner develops robust calibration capabilities even from limited data.
Impact of Confidence Forms During Training. during training, we compare two approaches for LLaMA: using continuous 0%-100% confidence
assess the impact of confidence representation
26

---

Table 9: ECE and AUROC metrics for dif- ferent base models with (w/ reg) and without regularization (w/o reg).
AUROC
0.030
LLM
Context
ECE +
AUROC 1
0.20
0.625
0.15
0.800
w/o reg
0,0722
0.7043
0.10
0.39
LLaMA
w/ reg 0.0405
0.7383
Dataset Size
1000
10000
2520
Datase:
5000
7500
10000
w/o reg w/ reg
0.4212
0.718
Qwen
Figure 6: Impact of training data size average AUROC and ECE on five datasets across three base models. ConfTuner achieves good performance with 2,000 samples.
0,4359
0.7242
w/o reg w/ reg 0.1797
0.1027
0.7907
Ministral
0.7338
Table 10: Comparison of ConfTuner trained on different confidence levels.
Context
ECE +
AUROC 1
0-9 0%- 100%
0.0605 0.0405
0.7248
0.7383
scale versus confidence levels from 0 to 9. The results, presented in Table T0, demonstrate that the 0%-100%% scale lead to marginal improvement in performance.
E.1 Ablation Training Distribution Shifts
We further train LLaMA on GSMSK (math problems) instead of HotpotQA (general knowledge from Wikipedia), As shown in Table [T ConfTuner trained on GSM8K performs better on GSMSK and St but worse on HotpotQA, TriviaQA, and TruthfulQA.
Table 11: ECE and AUROC metrics for ConfTuner trained on GSM8K and HotpotQA.
Metric
Method
HotpotQA
GSM8K
TriviaQA
Strategy QA
TruthfulQA
Average
ConTuner (GSM8K)
0.2308
0.0753
0.1000
0.1075
0.2257
0.1479
ECE 4
ConTuner (HotpotQA)
0.0405
0.1276
0.0388
0.1387
0.1955
0.1082
ConTuner (GSM8K)
0.6552
0.7035
0.5978
0.6826 0.6750
0.5822
0,6408
AUROC f
ConTuner (HotpotQA)
0.7383
0.7007
0.6821
0.5739
0.6740
F
Additional Experimental Results
F1 The Impact of the Hidden States of the Answer
We prompt ConfTuner (based on LLaMA) to generate confidence score prior to providing the answer. The results of AUROC and ECE are presented in Table 12 Our findings indicate that outputting confidence before the answer yields poorer performance compared to outputting afterward, suggesting that the hidden states of the answer tokens are informative about the certainty of the response. And ConfTuner still outperforms Base model when outputting confidence first.
F.2 Comparison between ConfTuner and classifier
a precise comparison between (A) ConfTuner, and (B) an LLM with an external linear classifier with the same architecture as the model's original output projection layer. Specifically, this confidence classifier a linear transformation layer, whose input dimension matches the dimension of the model's hidden states, and its output dimension equals the size of the model's vocabulary. The
27

---

Table 12: AUROC (1) and ECE (.) of outputting generating confidence first (c+a) or generating answer first (a+c). Generating the answer first yields better performance, indicating the hidden states of the answer are informative of the confidence scores.
In-distribution
Out-of-distribution
Metric
Method
HotpotQA
GSM8K
TriviaQA
StrategyQA
TruthfulQA
Average
Base (c+a)
0.6909
0.5447
0,5819
0.7094
0.4471
0,5948
AUROC t
ConfTuner (c+a)
0.7263
0.6241
0.6565
0.6787
0.5267
0.6425
ConfTuner (a+c)
0.7383
0.7007
0.6821
0.6750
0.5739
0.6740
Base (cta)
0.4796
0.2082
0.1062
0.5285
0.3761
0.3397
ECE 4
ConfTuner (c+a)
0.0685
0.0953
0.1487
0.2839
0.2889
0.1771
ConfTuner (a+c)
0.0405
0.1276
0.0388
0.1387
0.1955
0.1082
input this classifier is the final hidden state from the L.M''s last layer, corresponding to the last token position in the generated sequence.
(A) and (B) have the exact same architecture, and the only differences between them are (1) End- to-end training: in (A), we train the LLM end to end, but in (B) we train only the final linear laye. (2) Initialization/ parameter sharing: in (A), the output projection layer parameters tied with the
LLM's original embedding matrix, while (B), the classifier's parameters are not tied and randomly initialized.
To further disentangle these effects, we also evaluated third variant: (C) a classifier identical (B), but initialized with the LLM's original embedding matrix.
As shown in Table 13 we have the following observations: (1) the classifier initialized with LLM's original embedding matrix (C) performs better than the classifier with random initialization (B). This indicates that the random initialization might lead noise
noisy gradients), resulting in sub-optimal results. (2) ConfTuner still performs better than the classifier initialized with LLM's original embedding matrix (C). This because the classifier infers only based on the hidden state of the LLM. If the final hidden state does not capture sufficient information about the model's 's confidence, the classifier will be less effective at confidence estimation. In contrast, ConfTuner trains the LLM itself's 's parameters, so the L.M can be trained to preserve the necessary confidence information in the final hidden state.
Table 13: Comparison between ConfTuner, a classifier with random initialization, and classifier initialized with LLM's original embedding matrix.
Metric
Method
HotpotQA
GSM8K
TriviaQA
Strategy QA
TruthfulQA
Average
ConTuner (A)
0.7383
0.7007
0.6821
0,6750
0.5739
0.6740
AUROCT t
classifier+random init (B)
0.6817
0.6025
0.6442
0.5961
0.5428
0.6335
classifier+lIm init (C)
0.7356
0.6518
0.6873
0.6420
0.5626
0.6559
ConTuner (A)
0.0405
0.1276
0.0388
0.1387
0.1955
0.1082
ECE |
classifier+random init (B)
0.0865
0,2983
0.1582
0.2057
0.2493
0.1996
classifier+IIm init (C)
0.0581
0.1685
0.0621
0.1459
0.2206
0.1310
F.3 Accuracy Comparison
Table presents the experimental accuracies for the 0%-100% confidence assessments, while
Table 15
details the accuracies for classifications of high, low, medium confidence. These results
indicate that the base model consistently achieves the highest accuracy. However, ConfTuner also demonstrates comparable performance.
28

---

Table 14: Accuracy comparison of all the methods for 0%-100% confidence.
In-distribution
Out-of-distribution
LLM
Method
HotpotQA
GSM8K
TriviaQA StrategyQA
TruthfulQA
Base
0.3620
0.7970
0.7440
0.7113
0.3732
LACIE
0.1850
0.6850
0.5360
0.6563
0.3354
LLaMA
SaySelf
0.3650
0.7690
0.7380
0.7066
0.3450
Ensemble
0.3150
0.7109
0.7242
0.6807
0.2655
ConfTuner
0.3320
0.7850
0.7200
0.6677
0.3696
Base
0.2900
0.8680
0.5560
0.7083
0.4149
Ensemble
0.2619
0.3719
0,5429
0.7031
0.2864
Qwen
LACIE
0.2880
0.8620
0.5520
0.7021
0.4039
SaySelf
0.2850
0.8640
0,5570
0.7109
0,4002
ConfTuner
0.2860
0.8620
0.5520
0.6764
0.4284
Base
0.3160
0.6980
0.6270
0.6769
0.3782
Ensemble
0.2583
0.4187
0.5940
0.6947
0.2600
Ministral
LACIE
0.2490
0.7110
0.5230
0.6083
0.3341
SaySelf
0.3110
0.6980
0.6250
0.6720
0.3390
ConfTuner
0.3040
0.7080
0.6030
0.6197
0.4321
Table 15: Accuracy comparison of all the methods for 0-9 confidence.
LLM
Method
HotpotQA
GSM8K
TriviaQA
Strategy
TruthfulQA
Base
0.7890
0.7940
0.7390
0.7004
0.3550
LACIE
0.2270
0.3450
0.4770
0.4279
0.3329
LLaMA
SaySelf
0.3470
0.7810
0.7380
0,6930
0.3660
ConfTuner
0.3260
0.7900
0.7200
0.6742
0.3586
Base
0.2920
0.8810
0.5580
0.7148
0,3990
LACIE
0.2810
0,8010
0,4520
0.6306
0,3953
Qwen
SaySelf
0.2980
0.8820
0.5570
0.7122
0,4149
ConfTuner
0.3000
0.8650
0.5580
0.6878
0.4345
Base
0.3070
0.7220
0.6340
0.6790
0.3672
LACIE
0.2800
0.6930
0.5410
0.6067
0.3367
Ministral
SaySelf
0.3180
0.7210
0.6270
0.6681
0.3476
Con/Tuner
0.3030
0.7300
0.6010
0.6231
0,4468
F.4 Comparison to Logit-based Method
We have conducted experiments to compare ConfTuner with a logit-based method, P(True) on the LLaMA base model. The results of ECE and AUROC in Table below show that ConfTuner outperforms P(Truc).
Table 16: Comparison to P(True).
Metric
Method
HotpotQA
GSM8K
TriviaQA
StrategyQA
TruthfulQA
Average
P(True)
0.7132
0.7026
0.7748
0.6352
0.5192
0.6690
AUROC +
ConTuner
0.7383
0.7007
0.6821
0.6750
0.5739
0.6740
P(True)
0.5118
0.1645
0.2309
0.2538
0.5527
0.3427
RCE J
Con'uner
0.0405
0.1276
0.0388
0.1387
0.1955
0.1082
29

---

E5 Comparison with Black-box Calibration Method
We further add black-box calibration baseline, Ensemble 32 which prompts LLMs to generate the top K guesses and their corresponding confidence, then inputs the same prrmpt multiple times, and finally computes the average confidence. The results are shown Table T7 We can see that ConfTuner has significantly better ECE (by 5.3%) and slightly lower AUROC (by 1.4%). Please note that ConfTuner only uses smaller model and prompts once, while Ensemble uses GPT-4o and prompts times, which is more expensive.
Table 17: Performance Comparison Different Methods
Metric
Method
HotpotQA
GSM8K
TriviaQA
StrategyQA
TruthfulQA
Average
GPT-44o
0.2612
0.0526
0.1341
0.0595
0.3127
0.1640
ECE +
Ensemble
0.2016
0.0742
0.1143
0.0438
0.3161
0.1500
ConTuner
0.1109
0.0497
0.1076
0.0614
0.1555
0.0970
GPT-4o 0.7024
0.5278
0.6151
0.5244
0.6030
0.5945
AUROC t
Ensemble
0.7280
0.6280
0,6113
0.6077
0.6301
0.6410
Con'unnrr
0.7207
0.5412
0.6227
0.6494
0.6037
0.6275
F.6
Full Results for Ensemble
Due to space limitations, we provide the results with the standard deviation for Ensemble in Table
18
Table 18: Full results with standard deviation of Ensemble.
Table
Base model
HotpotQA
GSM8K
TriviaQA
StrategyQA
TruthfulQA
LLaMA
0,425410.0417 0.5909=1.0000
0.2365:1.0415
0.1652+:0.0233
0.147440.0004
0.4035:10.200
Table
Qwen
0.2428+00.030
0.3595+:.0252
0.1226+000360
0.4626+0.0112
Ministral
0.5887=000023
0.3357+0.0706
0.3966+1.0050
0.1948:0.0013
0.5670=000651
LLaMA
0.6035:000361
0.5210=0.0359
0.6323:0.0193
0.6022+0.0177
0.6038#0.0176
Table
Qwen
0.6259+0.0176 0.5683+0.0267
0.6287+0.0218
0.5959+t0.332
0.6460=0.0149
Ministral
0.5079=0000200
0.669=#0.0611
0.500410.0563
0.6222+0.0531
0.6153=0.0564
LLaMA
0.3150=0.0508
0.710910.0509
0.7242+000085
0.6807#0.0398
0.2655+0.0391
Table 14
Qwen
0.2619+0.0330
0.3719+0.0450
0.5429+000049
0.7031 +0.4422
0.2864+0.0432
Ministral
0.2583++00051
0.4187+0.0487
0.5940+00.000
0.6947+0.0490
0.2600+0.0083
E.7 Accuracy of Self-correction
We provide the accuracies before and after self-correction in Table 19
Table 19: Accuracy of ConfTuner and baselines on self-correction task. After self-correction,
ConfTuner achieves the highest accuracy.
Method
HotpotQA Before
TruthfulQA Before
After
After
Base
0.283
0.280
0.410
0.405
LACIE
0.280
0.282
0.403
0.406
SaySelf
0.285
0.284
0.400
0,410
ConfTuner
0.283
0.293
0.409
0.425
30


Reviews:
[{"review_id": "crcLHcxAga", "reviewer": "Reviewer_gPzL", "summary": "The paper presents a loss function for training LLMs to be better calibrated in their confidence assessments. The paper presents a theoretical argument for the proposed loss function being a Proper Scoring Rule and empirical testing on a combination of benchmark datasets with different tasks, different models, and different verbalizations of confidence. The paper finds that the ConfTuner loss consistently provides better confidence estimates, including out of domain, relative to other techniques or just using a base model’s estimates.", "questions": "I do have a few questions about the paper\n\n1.\tFor the LoRA training, what was the alpha, and what layers of the neural networks did you attach the adapters to?\n2.\tCould the proposed loss be combined with other losses to improve both the quality of the answers and the calibration in the same training?\n3.\tDoes the confidence calibration carry over to tasks/prompts where there aren’t explicit instructions to evaluate its confidence? In other words, is there any implicit confidence calibration that is learned, such that a prompt that includes things like “…If you don’t know something or are unsure about something, do …” would work better? I think it's just a little clunky to always have the LLM output its confidence on everything, and it would be nice if it had better implicit confidence calibration.", "limitations": "All limitations are successfully addressed in the paper.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper has a strong empirical validation and is well written and easy to understand. I especially appreciate that the paper did out-of-domain testing (i.e., they trained confidence assessments for one dataset and task and then showed how that training carried over to new datasets and tasks). I believe that such a result shows promise for practical usage. The proposed method is also simple to implement, with good theoretical guarantees.\n\nThe weaknesses of the paper are few. I suppose one could argue that the problem of getting accurate confidence estimates from LLMs is of secondary importance to improving the general, and especially reasoning, performance of LLMs. This paper is not likely to be ground-breaking for the field, but I still think it shows a lot of promise as a technique for actual usage.", "was_revised": true, "final_justification": "I will stand by my rating of 'Accept' after participating in the discussion period and reviewing the other comments. I appreciate that the authors' addressed some of the clarity issues with their testing setup and believe the paper does make a contribution.", "rebuttal": "Thank you for your encouraging feedback and insightful comments! We address your main concerns as follows: \n\n> ### W1: Importance of confidence estimation vs core LLM advancement​\n\nImproving the general capabilities of LLMs is indeed important. However, **even as LLMs become more capable, there will inevitably be questions that exceed their knowledge or reasoning abilities**. In such cases, the model’s tendency to be overconfident or fail to accurately express their uncertainty is a fundamental limitation, undermining user trust and making it hard for downstream systems or users to know when caution is warranted. \n\nMoreover, **the ability to express calibrated confidence can also support better reasoning and decision-making**. For instance, in retrieval-augmented generation (RAG) or tool-use settings, a model that can identify and verbalize its uncertainty within its chain-of-thought could trigger additional information-seeking steps, such as searching for more evidence or invoking a tool, to validate or revise its initial answer, allowing for iterative self-correction and better performance.\n\n \n\n> ### Q1: LoRA setting.\n\nThe alpha value is set to 32, with adapters applied to all layers - specifically attached to the query and value projection modules. We will incorporate this implementation detail in our manuscript revision.\n\n> ### Q2: Could the proposed loss be combined with other losses to improve both the quality of the answers and the calibration in the same training?\n\nYes. As a plug-and-play loss, the Brier score is highly flexible and compatible with standard LLM fine-tuning pipelines. **Brier loss can be seamlessly combined with other loss functions, such as those optimizing answer quality**, to enhance both calibration and response accuracy during training simultaneously. In this work, we focused exclusively on improving verbalized confidence calibration, so we only use the Brier loss. Future work could explore integrating it with answer-focused losses to achieve dual improvements.\n\n> ### Q3: Does the confidence calibration carry over to tasks/prompts where there aren’t explicit instructions to evaluate its confidence?\n\n**ConfTuner could generalize to scenarios without explicit instructions.** We conduct new experiments on ConfTuner (based on LLaMA). Specifically, we change the original prompt from `{Question + output confidence (0-100)}` to `{Question + Please express your uncertainty when providing the answer}`. Under this instruction, the LLM also produces implicit confidence expressions, such as \"I'm fairly certain, but there’s a chance I could be mistaken\" or \"This is a tough one, so I’d say it’s likely but not guaranteed.\" We evaluate these implicit confidence by inputting them to GPT-4o to assess the implied confidence levels (0-100). The results of AUROC and ECE are shown in Table 1 and Table 2, demonstrating that **implicit confidence calibration of ConfTuner is comparable to explicit confidence calibration.**\n\nTable 1: AUROC\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.7047 | 0.5422 | 0.6342 | 0.6489 | 0.5895 | 0.6239 |\n| ConfTuner (explicit) | **0.7383** | **0.7007** | 0.6821 | 0.6750 | 0.5739 | 0.6740 |\n| ConfTuner (implicit) | 0.7239 | 0.6869 | **0.7024** | **0.6751** | **0.6217** | **0.6820** |\n\nTable 2: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.2808 | 0.1179 | 0.1232 | **0.1098** | 0.3250 | 0.1913 |\n| ConfTuner (explicit) | **0.0405** | 0.1276 | **0.0388** | 0.1387 | **0.1955** | **0.1082** |\n| ConfTuner (implicit) | 0.1639 | **0.0950** | 0.1088 | 0.1721 | 0.2019 | 0.1483 |\n\nThank you again for your supportive review. Your constructive comments are greatly appreciated for revising our paper. We welcome any additional questions you might have."}, {"review_id": "JHXaMZRJyT", "reviewer": "Reviewer_Fgyi", "summary": "The paper proposes to \"verbalize\" (un)certainty in LLM answers by prepending \"Confidence: <>\" to the answer, where in place of \"<>\", the scores for the token predictions (e.g. 0,1,2,...99,100) are evaluated as the uncertainty estimate. The confidence token predictions are trained using a tokenized Brier score inspired by proper scoring results. The method is evaluated on several standard benchmarks and report strong results, outperforming evaluated baselines.", "questions": "1. How load bearing is the \"verbalization\" part here? Effectively, we are using the output embedding matrix as a 101-class classifier between 0...100. Would training a basic classifier on top of the last hidden state after appending \"Confidence: \" to the generated answer work just as well? Or is actually using the pretrained \"verbalized\" representation of the embedding tokens 0...100 crucial? --- More generally, what are the actual differences between your proposed method and a \"simple\" classification into confidence bins? A proper clarification on this point will alleviate my concerns regarding originality.\n\n2. Would outputting the confidence prediction *before* the answer change how well the method works? In other words, are the hidden states of the actual answer tokens informative about the certainty of the answer or is this mostly extracted from hidden states in the question? This might be a nice additional experiment but not related a major concern regarding the paper.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\nThe method is well-motivated and the introduction of the tokenized Brier score is novel. The authors also include valuable experiments on self-correction and model cascades as evaluations how uncertainty estimation methods can be employed in practice to improve the performance of LLM (systems).\n\n### Weaknesses\nFundamentally, I do not see the difference between the proposed method and a classification problem into confidence bins (which itself can be seen as a discretization of a regression task predicting the confidence) --- See also my question. Therefore, while the method itself is well-motivated and reports string results, I would like to see more discussion on this point or arguments to the contrary.\n\nThere is also an abundant amount of research into uncertainty quantification nowadays (which I will acknowledge is impractically to fully cite, as this would constitute its own survey paper). In particular, e.g. works on regression for a confidence score for a given answer (https://arxiv.org/abs/2404.15993) or \"verbalizing\" confidence as the probability mass put on a special token (https://arxiv.org/abs/2412.06676) seem quite related (particular considering my question about differences to regression/classification).", "was_revised": true, "final_justification": "I am raising the score of my review. The authors have meaningfully engaged with the Reviewers and conducted significant, insightful additional experiments. Additionally, the discussion has been fruitful; the authors have candidly discussed limitations of their method and ablated various components to give a better assessment of the mechanisms behind their method. If the authors add the additional experiments and points of discussion to the paper, it turns an already good work into a strong one. While not groundbreaking for AI as a whole, the paper in that state would be a technically well-executed and meaningful addition to the current research area of uncertainty quantification in LLMs.\n\nTherefore I raise the score to 5 — `Accept`.", "rebuttal": "Thank you for your kind and constructive feedback on our manuscript! We address your main concerns as follows:\n\n> ### W1 & Q1: Difference between ConfTuner and classification method.\n\nConfTuner is superior to the classification method for two reasons: first, using LLM to directly verbalize its confidence is **more seamless and intuitive** than having an external classifier generate confidence scores. Second, by fine-tuning the LLM with a tokenized Brier score, **ConfTuner enhances the model's intrinsic calibration ability.** This is evidenced by:\n\n1) **ConfTuner has good generalization ability to various scenarios and different forms of expressions.** ConfTuner achieves significantly lower ECE scores on out-of-distribution datasets, and also generalizes to different forms of confidence levels (Section 4.2). **We further compare the performance of ConfTuner and the classification method** on the in-distribution and out-of-distribution datasets. We conduct an experiment where we feed the token logits after \"Confidence: \" from the last hidden state into a two-layer MLP classifier to predict confidence scores from 0 to 100, trained with Brier loss. The results, presented in Tables 1 and 2 below, compare this approach to ConfTuner (based on LLaMA). Our findings indicate that the classifier-based method yields poorer performance compared to ConfTuner, especially on OOD datasets, indicating that **the classifier has relatively higher usage of spurious factors that do not generalize well to diverse scenarios**.\n\n2) This improved calibration not only boosts trustworthiness but also supports broader LLM capabilities, such as self-correction (Figure 4). \n\n3) **ConfTuner also generalizes to implicit confidence expressions**, which cannot be done by a classifier. We conduct further tests with a more **open-ended setup**. Specifically, we modified the original prompt from `{Question + Output confidence (0-100)}` to `{Question + Please express your uncertainty when providing the answer}`. This new setting significantly expands the LLM's output space, allowing it to generate diverse natural language expressions of uncertainty, such as \"I'm fairly certain, but there’s a chance I could be mistaken\" or \"This is a tough one, so I’d say it’s likely but not guaranteed.\" We evaluate these implicit confidences by inputting them to GPT-4o to assess the implied confidence levels. The results of AUROC and ECE are presented in Tables 3 and 4 below, demonstrating that **ConfTuner can handle not only classification-like confidence expressions but also more complex and implicit confidence expressions.\n\nTable 1: Classifier vs ConfTuner comparison (AUROC)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Classifier | 0.7340 | 0.5843 | 0.6425 | 0.6219 | 0.5648 |  0.6295 |\n| ConfTuner | **0.7383** | **0.7007** | **0.6821** | **0.6750** | **0.5739** | **0.6740** |\n\nTable 2: Classifier vs ConfTuner comparison (ECE)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Classifier | 0.1253 | 0.2261 | 0.1469 | 0.1526 | 0.3894 |  0.2081 |\n| ConfTuner | **0.0405** | **0.1276** | **0.0388** | **0.1387** | **0.1955** | **0.1082** |\n\nTable 3: Implicit confidence expression performance (AUROC)\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.7047 | 0.5422 | 0.6342 | 0.6489 | 0.5895 | 0.6239 |\n| ConfTuner (explicit) | **0.7383** | **0.7007** | 0.6821 | 0.6750 | 0.5739 | 0.6740 |\n| ConfTuner (implicit) | 0.7239 | 0.6869 | **0.7024** | **0.6751** | **0.6217** | **0.6820** |\n\nTable 4: Implicit confidence expression performance (ECE)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.2808 | 0.1179 | 0.1232 | **0.1098** | 0.3250 | 0.1913 |\n| ConfTuner (explicit) | **0.0405** | 0.1276 | **0.0388** | 0.1387 | **0.1955** | **0.1082** |\n| ConfTuner (implicit) | 0.1639 | **0.0950** | 0.1088 | 0.1721 | 0.2019 | 0.1483 |\n\n> ### W2: Discussion of two related works.\n\nThe regression-based methods [1] rely on an external classifier to calibrate confidence, and can not generalize to different forms or implicit confidence expressions. Similarly, using a single \"I don’t know\" token limits expressiveness to a binary uncertainty indicator [2], lacking the granularity of our approach. By contrast, ConfTuner fine-tunes the LLM itself, enabling **more accurate and diverse verbalized confidence expressions**. We will include this discussion in our future revision.\n\n\n\n> ### Q2: Would outputting the confidence prediction before the answer change how well the method works?\n\nIt’s an interesting experiment and could further elucidate the internal mechanics of ConfTuner. We prompt ConfTuner to generate the confidence score prior to providing the answer. The results of AUROC and ECE are presented in Tables 5 and 6. Our findings indicate that outputting confidence before the answer yields poorer performance compared to outputting it afterward, suggesting that **the hidden states of the answer tokens are indeed informative about the certainty of the response**. And ConfTuner still outperforms Base model when outputting confidence first. We will include this experiment in our revised manuscript.\n\nTable 5: Performance of outputting confidence first (AUROC)\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (confidence+answer) | 0.6909 | 0.5447 | 0.5819 | 0.7094 | 0.4471 | 0.5948 |\n| ConfTuner (confidence+answer) | 0.7263 | 0.6241 | 0.6565 | **0.6787** | 0.5267 | 0.6425 |\n| ConfTuner (answer+confidence) | **0.7383** | **0.7007** | **0.6821** | 0.6750 | **0.5739** | **0.6740** |\n\nTable 6: Performance of outputting confidence first (ECE)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (confidence+answer) | 0.4796 | 0.2082 | 0.1062 | 0.5285 | 0.3761 | 0.3397 |\n| ConfTuner (confidence+answer) | 0.0685 | **0.0953** | 0.1487 | 0.2839 | 0.2889 | 0.1771 |\n| ConfTuner (answer+confidence) | **0.0405** | 0.1276 | **0.0388** | **0.1387** | **0.1955** | **0.1082** |\n\nThanks again for your valuable suggestions, which are extremely useful for our future work on revising the manuscript.​​ Feel free to reach out if anything comes up. Please let us know if anything needs further clarification!\n\n[1] Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach\n\n[2] I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token"}, {"review_id": "ZibdMiP1QV", "reviewer": "Reviewer_wrZQ", "summary": "The authors argue for a loss based on a proper scoring rule for improving the calibrations of LLMs. They tackle the challenge of verbalizing the calibrated confidence of LLMs and towards this they propose a method called ConfTuner which finetunes an LLM on correctness scores to elicit the confidence scores. While a vanilla application of a proper scoring rule based loss function, such as Brier score, would produce a numerical score, the challenge is how to verbalize this numerical score? Towards this end, the author propose a loss function, inspired by a Brier scoring rule, that assigns high probability to low-confidence (verbalized) tokens (and low probability to high-confidence tokens) when the LLM is wrong and the opposite when the LLM is right. When an LLM is finetuned with such a loss function based on correctness, it produces calibrated confidence scores. Experimental results suggest that the generated confidence scores are well-calibrated on the dataset for which finetuning set was used but calibration is significantly worse for out-of-distribution datasets (for which finetuning was not done).", "questions": "See the weaknesses section above.", "limitations": "See the weaknesses section above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n- The paper is well-written and easy to follow. The proposed method is well-motivated.\n- The Brier score inspired loss function is neat and seems to generate well-calibrated scores for in-distribution examples. \n\n** Weaknesses **:\n- The main challenge with finetuning-based calibration approaches is the difficulty in demonstrating that the model is not merely exploiting spurious correlations in (question, answer, confidence) tuples to to achieve low loss on a given dataset, and that it is truly learning to generate a calibrated confidence scores. This clearly manifests in the lackluster out-of-distribution results on many fine-tuning-based calibration methods. I am afraid this work has similarly quite high out-of-distribution calibration errors. Could the authors comment on that and how will they rebut the concern that their method is not merely exploiting the distributional similarity in (question, answer, confidence) tuples for the in-distribution examples and not really learning to emit a calibrated confidence score. \n\n- I think an important baseline of self-evaluation task (proposed in Kadavath et al. (https://arxiv.org/abs/2207.05221) is missing. Please add that.", "was_revised": true, "final_justification": "My post-rebuttal response\n-----------------------------\n\nThank you for providing clarification and adding the P(True) baseline. The AUROC is almost the same as ConfTuner. I keep my original score.", "rebuttal": "Thanks for your positive feedback and valuable suggestions! We address your main concerns as follows:\n\n> ### W1: Concerns of exploiting spurious correlations in (question, answer, confidence) tuples and high out-of-distribution calibration errors.\n\nThe absolute levels of metrics on a given dataset depend significantly on the dataset’s difficulty and the model’s accuracy on that task, which vary from one dataset to another. However, in many cases, **the improvements contributed by our approach over the Base model on out-of-distribution cases are often better than on in-distribution ones**. The results are shown in Table 1 below. Under LLaMA and Qwen models, the AUROC improvements on OOD datasets generally surpass those on the in-distribution dataset (HotpotQA). This suggests that ConfTuner is not merely exploiting distributional similarities in the training data but is learning robust confidence calibration capabilities. \n\nConfTuner not only generalizes to OOD datasets, but also **generalizes to different forms of confidence expressions** (e.g., training on confidence expressed as percentages, and then expressing confidence linguistically as high/medium/low at test time), as demonstrated in Section 4.2. This reinforces that ConfTuner has internalized calibration ability rather than relying on spurious correlations. \n\nFurthermore, we conduct a new experiment to show that **ConfTuner also generalizes to implicit confidence expressions**. Specifically, we modified the original prompt from `{Question + output confidence (0-100)}` to `{Question + Please express your uncertainty when providing the answer}`. This new setting significantly expands the LLM's output space, allowing it to generate diverse natural language expressions of uncertainty, such as \"I'm fairly certain, but there’s a chance I could be mistaken\" or \"This is a tough one, so I’d say it’s likely but not guaranteed.\" We evaluated these implicit confidences by inputting them to GPT-4o to assess the implied confidence levels. The results of AUROC and ECE are presented in Tables 2 and 3 below, demonstrating that **ConfTuner enhances the model's intrinsic calibration ability instead of simply learning spurious correlations**.\n\n\nTable 1: Improvement of AUROC compared to the Base model\n\n| LLM | HotpotQA (In-dist) | GSM8K | TriviaQA | StrategyQA | TruthfulQA | \n|------|-------------------|-------|----------|------------|------------|\n| LLaMA | +7.25% | +39.35% | +13.25% | +8.02% | +5.63% | \n| Qwen | +4.62% | +14.22% | +23.13% | +10.45% | +6.28% | \n| Ministral | +52.12% | +30.53% | +45.51% | +0.35% | +24.64% | \n\n\nTable 2: Implicit confidence expression performance (AUROC)\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.7047 | 0.5422 | 0.6342 | 0.6489 | 0.5895 | 0.6239 |\n| ConfTuner (explicit) | **0.7383** | **0.7007** | 0.6821 | 0.6750 | 0.5739 | 0.6740 |\n| ConfTuner (implicit) | 0.7239 | 0.6869 | **0.7024** | **0.6751** | **0.6217** | **0.6820** |\n\nTable 3: Implicit confidence expression performance (ECE)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|--------|-----------|------------|------------|---------|\n| Base (implicit) | 0.2808 | 0.1179 | 0.1232 | **0.1098** | 0.3250 | 0.1913 |\n| ConfTuner (explicit) | **0.0405** | 0.1276 | **0.0388** | 0.1387 | **0.1955** | **0.1082** |\n| ConfTuner (implicit) | 0.1639 | **0.0950** | 0.1088 | 0.1721 | 0.2019 | 0.1483 |\n\n> ### W2: Comparison to a related work.\n\nOur method focuses on verbalized confidence calibration and differs from the work [1], which uses logits to infer the confidence. Thus, we initially did not include it as a baseline. However, per your request, we have conducted experiments incorporating this baseline (P(True)) on the LLaMA base model. The results of ECE and AUROC in Tables 4 and 5 below show that **ConfTuner outperforms P(True)** [1]. We will add this baseline to our revised manuscript.\n\nTable 4: Comparison to P(True) (AUROC)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|-------|----------|------------|------------|---------|\n| P(True) | 0.7132 | **0.7026** | **0.7748** | 0.6352 | 0.5192 | 0.6690 | \n| ConfTuner | **0.7383** | 0.7007 | 0.6821 | **0.6750** | **0.5739** | **0.6740** |\n\n\nTable 5: Comparison to P(True) (ECE)\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|--------|----------|-------|----------|------------|------------|---------|\n| P(True) | 0.5118 | 0.1645 | 0.2309 | 0.2538 | 0.5527 | 0.3427 | \n| ConfTuner | **0.0405** | **0.1276** | **0.0388** | **0.1387** | **0.1955** | **0.1082** |\n\n\nWe really appreciate your valuable suggestions; they’ll be extremely useful for our future work on revising the manuscript.​​ Please let us know if you have any further questions.\n\n[1] Language Models (Mostly) Know What They Know"}, {"review_id": "AfAbnTq08R", "reviewer": "Reviewer_9LBm", "summary": "The paper that I read is about a significant issue with Large Language Models (LLMs) that frequently overestimate their confidence in their responses. To tackle this problem, the researchers came up with ConfTuner, a brand-new technique for fine-tuning LLMs that enables them to communicate their confidence levels in a well-calibrated manner. They created a \"tokenized Brier score\" loss function that encourages the model to express confidence levels that accurately reflect its likelihood of being correct. The best part is that ConfTuner doesn't require any ground-truth confidence scores and only adds a little computational overhead. The researchers tested ConfTuner on various reasoning tasks and discovered that it outperforms existing calibration techniques and can even be used with black-box models like GPT-4o.", "questions": "1. If we want to elicit confidence $\\in \\mathbb{R}$, why not add a regression head after the llm generated answer and train that with Brier score? I would suggest to ablate against a regression head baseline (e.g., attach a scalar head to the LLM’s final hidden state, trained with Brier loss) to establish the superiority of tokenised confidence.\n\n2. It would be valuable to evaluate how the proposed method performs relative to black-box models where confidence estimation is derived directly from access to output logits.\n\n3. In Experiment section please evaluate against Pair-rank to aggregate topK sampled verbalized confidence scores - https://openreview.net/forum?id=gjeQKFxFpZ\n\n4. The sensitivity of the confidence estimates to varying prompting strategies remains unclear. It would be useful to quantify the incremental gains from fine-tuning, compared to improved prompt engineering and top-k response aggregation using existing models.\n\n5. Limited Ablation on Training Distribution Shifts: The paper could benefit from more extensive analysis on how shifts in the training data distribution affect calibration performance.", "limitations": "The authors acknowledge the limitation of tokenization rigidity, where the use of predefined tokens may constrain flexibility. However, they do not highlight data dependency as a critical factor—calibration quality is highly sensitive to the training data distribution. Performance may degrade when there are shifts in topic or style, such as transitioning from medical to legal QA.\n\nAdditionally, the authors did not address the fairness implications of their approach. There is no analysis on whether calibration quality varies across demographic groups or in cases of ambiguous queries (e.g., non-Western names in TriviaQA), which may introduce biases.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. Novel Formulation of Verbalized Calibration: Adapts classical proper scoring rules (e.g., Brier score) to tokenized confidence expressions, bridging a theoretical gap between traditional classifiers and LLMs.\n2. Practical Efficiency: Minimal overhead: Trains with 2,000 samples (vs. 100k for baselines) and converges in 4 minutes (vs. 26–120 min). Requires no ground-truth confidence or repeated sampling, making it scalable.\n\nWeakness:\n1. Limited Exploration of Complex Contexts: While the paper shows generalization to different confidence formats (numerical percentages, high/medium/low), it doesn't fully explore more complex conversational contexts or diverse natural language expressions of uncertainty.\n2. Black-Box Calibration Limitations: While ConfTuner calibrates GPT-4o outputs, it relies on a proxy model (LLaMA) and cannot access internal states. Experiments lack comparison to other black-box calibration methods.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns, and I have accordingly revised my score. However, the novelty of the work remains unclear to me. The use of Brier score-based loss functions has been well established in the uncertainty estimation literature for deep neural networks. As a result, this work comes across more as a straightforward extension of those ideas to the LLM setting using a simplex-based approach.", "rebuttal": "Thanks for your valuable suggestions! We address your main concerns as follows:\n\n> ### W1: Limited exploration of complex contexts\n\n**ConfTuner can generalize to more complex contexts**. We conduct tests with a more **open-ended setup**. Specifically, we change the original prompt from `{Question + output confidence (0-100)}` to `{Question + Please express your uncertainty when providing the answer}`. This new setting significantly expands the LLM's output space to diverse natural language expressions of uncertainty, such as \"I'm fairly certain, but ...\" or \"I'm not sure\" We evaluated these implicit confidences by inputting them to GPT-4o to assess the implied confidence levels. The results of AUROC and ECE are presented in Tables 1 and 2 below,  showcasing ConfTuner's robustness in diverse conversational contexts.\n\nTable 1: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Base (implicit) | 0.7047 | 0.5422 | 0.6342 | 0.6489 | 0.5895 | 0.6239 |\n| ConfTuner (explicit) | **0.7383** | **0.7007** | 0.6821 | 0.6750 | 0.5739 | 0.6740 |\n| ConfTuner (implicit) | 0.7239 | 0.6869 | **0.7024** | **0.6751** | **0.6217** | **0.6820** |\n\nTable 2: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Base (implicit) | 0.2808 | 0.1179 | 0.1232 | **0.1098** | 0.3250 | 0.1913 |\n| ConfTuner (explicit) | **0.0405** | 0.1276 | **0.0388** | 0.1387 | **0.1955** | **0.1082** |\n| ConfTuner (implicit) | 0.1639 | **0.0950** | 0.1088 | 0.1721 | 0.2019 | 0.1483 |\n\n> ### W2: Comparison of black-box calibration methods for GPT-4o.\n\nBased on your suggestion, **we further add a black-box calibration baseline, Ensemble [1]**, which prompts LLMs to generate the top K guesses and their corresponding confidence, then inputs the same prompt multiple times, and finally computes the average confidence. The results are shown in Tables 3 and 4. We can see that **ConfTuner has significantly better ECE (by 5.3%) and slightly lower AUROC (by 1.4%)**. Please note that ConfTuner only uses a smaller model and prompts once, while Ensemble uses GPT-4o and prompts 3 times, which is more expensive.\n\nTable 3: ECE\n\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| GPT4o | 0.2612 | 0.0526 | 0.1341 | 0.0595 | 0.3127 | 0.1640 |\n| Ensemble | 0.2016 | 0.0742 | 0.1143 | **0.0438** | 0.3161 | 0.1500 |\n| ConfTuner | **0.1109** | **0.0497** | **0.1076** | 0.0614 | **0.1555** | **0.0970** |\n\nTable 4: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| GPT-4o | 0.7024 | 0.5278 | 0.6151 | 0.5244 | 0.6030 | 0.5945 |\n| Ensemble | **0.7280** | **0.6280** | 0.6113 | 0.6077 | **0.6301** | **0.6410** |\n| ConfTuner | 0.7207 | 0.5412 | **0.6227** | **0.6494** | 0.6037 | 0.6275 | \n\n> ### Q1: Ablate against a regression head baseline\n\nConfTuner has two strengths over a regression head:  1) **ConfTuner enhances the model's intrinsic calibration ability, thus has better generalization ability**. ConfTuner could generalize to out-of-distribution datasets, different forms of confidence levels (Section 4.2), and to **diverse and more complex confidence expressions (W1), which cannot be done by a simple regression head**. 2) It offers a **more seamless and natural** way to express confidence.\n\nBrier loss is used for various confidence level tokens instead of a single numeric value, so we further compare the performance of ConfTuner and a classifier on the in-distribution and out-of-distribution datasets. We conducted an experiment where we fed the token logits after \"Confidence: \" from the last hidden state into a two-layer MLP classifier to predict confidence scores from 0 to 100, trained with Brier loss. The results, presented in Tables 5 and 6 below, compare this approach to ConfTuner. Our findings indicate that **the classifier-based method yields poorer performance compared to ConfTuner, especially on OOD datasets**.\n\nTable 5: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Classifier | 0.7340 | 0.5843 | 0.6425 | 0.6219 | 0.5648 |  0.6295 |\n| ConfTuner | **0.7383** | **0.7007** | **0.6821** | **0.6750** | **0.5739** | **0.6740** |\n\nTable 6: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Classifier | 0.1253 | 0.2261 | 0.1469 | 0.1526 | 0.3894 |  0.2081 |\n| ConfTuner | **0.0405** | **0.1276** | **0.0388** | **0.1387** | **0.1955** | **0.1082** |\n\n> ### Q2: Comparison to models where confidence estimation is derived from logits\n\nWe further use a method [3], P(True), where confidence estimation is derived directly from logits of the token “True”. The results of ECE and AUROC are shown in Tables 7 and 8, demonstrating that **ConfTuner performs better than P(True)**.\n\nTable 7:  AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| P(True) | 0.7132 | **0.7026** | **0.7748** | 0.6352 | 0.5192 | 0.6690 | \n| ConfTuner | **0.7383** | 0.7007 | 0.6821 | **0.6750** | **0.5739** | **0.6740** |\n\nTable 8: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| P(True) | 0.5118 | 0.1645 | 0.2309 | 0.2538 | 0.5527 | 0.3427 | \n| ConfTuner | **0.0405** | **0.1276** | **0.0388** | **0.1387** | **0.1955** | **0.1082** |\n\n\n> ### Q3: Comparison to Ensemble (pair-rank).\n\nWe provide the results of Pair-rank in Tables 9 and 10. The results demonstrate that **ConfTuner outperforms Ensemble (Pair-rank)**.\n\nTable 9: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Ensemble(Pair-rank) | 0.4745 | 0.2161 | 0.1535 | 0.1862 | 0.4165 | 0.2894 |\n| ConfTuner | **0.0405** | **0.1276** | **0.0388** | **0.1387** | **0.1955** | **0.1082** | \n\nTable 10: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| Ensemble(Pair-rank) | 0.6237 | 0.5768 | 0.6411 | 0.6388 | **0.6096** | 0.6180 |\n| ConfTuner | **0.7383** | **0.7007** | **0.6821** | **0.6750** | 0.5739 | **0.6740** | \n\n> ### Q4: Sensitivity of ConfTuner to varying prompting strategies & Comparison to improved prompt engineering and top-k response aggregation\n\nEven trained only on numeric confidence (0-100%) prompting, **ConfTuner could generalize to different confidence elicitation prompting strategies**, such as \"high/medium/low\" (Table 3 in our paper), and implicit confidence elicitation (W1).\n\nWe have compared ConfTuner with Ensemble (Tables 1 and 2 in our paper), where the prompting and aggregation strategies are recommended by [1]. We also compare ConfTuner with Ensemble (pair-rank) in Q3. The results show that ConfTuner still achieves better performance. \n\nWe further conduct an experiment on ConfTuner + Ensemble, i.e., use the prompting and aggregation strategies of Ensemble for ConfTuner. The results in Tables 11 and 12 show that **improved prompting and aggregation strategies could yield better performance for ConfTuner**.\n\nTable 11: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| ConfTuner + Ensemble | 0.0518 |**0.0824** | 0.0567 | **0.0736** | **0.1629**  | **0.0855**  |  \n| ConfTuner | **0.0405** | 0.1276 |**0.0388**| 0.1387 | 0.1955 | 0.1082 | \n\nTable 12: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| ConfTuner + Ensemble| 0.7264 | **0.7151**| **0.7079** | **0.6825**  | **0.5973** |   **0.6858**| \n| ConfTuner | **0.7383** | 0.7007 | 0.6821 | 0.6750 | 0.5739 | 0.6740 | \n\n\n> ### Q5 and Limitation 1: Ablation on Training Distribution Shifts\n\nWe further train LLaMA on GSM8K (math problems) instead of HotpotQA (general knowledge from Wikipedia). As shown in Tables 13 and 14. **ConfTuner trained on GSM8K performs better on GSM8K and StrategyQA, but worse on HotpotQA, TriviaQA, and TruthfulQA**. \n\nTable 13: ECE\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| ConfTuner (GSM8K) | 0.2308 | **0.0753** | 0.1000 | **0.1075** | 0.2257 | 0.1479 |\n| ConfTuner (HotpotQA) | **0.0405** | 0.1276 | **0.0388** | 0.1387 | **0.1955** | **0.1082** | \n\nTable 14: AUROC\n\n| Method | HotpotQA | GSM8K | TriviaQA | StrategyQA | TruthfulQA | Average |\n|-|-|-|-|-|-|-|\n| ConfTuner (GSM8K) | 0.6552 | **0.7035** | 0.5978 | **0.6826** | **0.5822** | 0.6408 |\n| ConfTuner (HotpotQA) | **0.7383** | 0.7007 | **0.6821** | 0.6750 | 0.5739 | **0.6740** | \n\n> ### Limitation 2: fairness implications of ConfTuner\n\nThe primary focus of this paper is to establish effective verbalized confidence calibration. Analyzing calibration quality across demographic groups or ambiguous queries is a critical topic and a valuable direction for future work. Indeed, [2] finds that many existing calibrators tend to focus on calibrating overall performance, which can lead to miscalibration in underrepresented groups. We believe that methods that more deeply integrate with the LLM’s own capabilities like ours have potential to alleviate this problem, but a deeper analysis is left for future work. We appreciate your suggestion and will include a discussion on fairness implications and potential biases in our revised manuscript. \n\n​​Thanks again for your constructive and valuable suggestions! We'll incorporate them in future revisions.​​ Please let us know if you have any further questions.\n\n[1] Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms\n\n[2] Proximity-Informed Calibration for Deep Neural Networks.\n\n[3] Language Models (Mostly) Know What They Know"}]

Author rebuttal:

Meta-review: This paper aims to address the overconfidence problem of LLMs and focuses on calibrating LLM's verbalized confidence. It proposes that existing approaches, which rely on prompt engineering or fine-tuning with heuristical uncertainty estimations, have limited effectiveness and generalizability. To mitigate this issue, it proposes a new loss function tokenized Brier score and a method ConfTuner. The experimental results demonstrate that the proposed method improves calibration across multiple tasks and foundation models. The reviewers comment that this paper is well-motivated, has certain theoretical and empirical contributions, and is easy to follow. The authors have adequately address the proposed concerns, such as highlighting the difference between existing methods, adding more discussions and explanations, and including necessary baseline methods. After the rebuttal, all of the reviewers give positive comments and ratings. Therefore, I recommend accepting this paper.
Decision: Accept (poster)