CaTS: Calibrated Test-Time Scaling for Efficient LLM Reasoning
Authors: Chengsong Huang
Langlin Huang
Jixuan Leng
Jiacheng Liu
Jiaxin Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Test-time Scaling
Model Calibration
Efficient inference
Language Modeling
Scaling
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose Self-Calibration, a new unsupervised framework to help model calibrate the confidence and using the confidence to efficiently test time scaling.

Abstract: Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sampling and Self-Consistency with majority voting are simple and effective, they require a fixed number of sampling responses for each query, regardless of its complexity. This could result in wasted computation for simpler questions and insufficient exploration for more challenging ones. In this work, we argue that model confidence of responses can be used for improving the efficiency of test-time scaling. Unfortunately, LLMs are known to be overconfident and provide unreliable confidence estimation. To address this limitation, we introduce Self-Calibration by distilling Self-Consistency-derived confidence into the model itself. This enables reliable confidence estimation at test time with one forward pass. We then design Calibrated Test-Time Scaling (CaTS), adapting common repeated sampling methods, such as self-consistency and Best-of-N to handle queries of various difficulty. We also show that CaTS-SC is provably better than vanilla self-consistency. Experiments on three LLMs across nine datasets demonstrate the effectiveness of our approach. Specifically, applying confidence-based Early Stopping (CaTS-ES) to Best-of-N improves MathQA accuracy from 73.7 to 83.6 with a sample budget of 16 responses, demonstrating the effectiveness of the confidence-based sampling strategy at inference time.

Published as a conference paper at ICLR 2026
CATS: CALIBRATED TEST-TIME SCALING FOR EFFICIENT LLM REASONING
Chengsong Huang', Langlin Huang' Jixuan Leng?
Jiacheng Liu", Jiaxin Huang'
Washington Univeristy in St. Louis
Camegie Mellon University University of Washington
{chengsong, h. www.langlin lin, jiaanniiiiiiiinh/eeee.a...o ji sixuanl @cs. cmu. edu, liujcêcs washington.
ABSTRACT
Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sam- pling and Self-Consistency with majority voting are simple and effective, they require fixed number of sampling responses for each query, regardless of its complexity. This could result in wasted computation for simpler questions and insufficient exploration for more challenging ones. In this work. argue that model confidence of responses can be used for improving the efficiency of test- time scaling. Unfortunately, LLMs are known to be overconfident and provide unreliable confidence estimation. To address this limitation, we introduce Self- Calibration by distilling Self-Consistency-deee -derivedd confidence into the model itself. This enables reliable confidence estimation at test time with one forward pass. We then design Calibrated Test-Time Scaling (CaTS), adapting common repeated sampling methods, such as self-consistency and Best-of-N to handle queries of various difficulty. We also show that CaTS-SC is provably better than vanilla self-consistency. Experiments on three LIMS across nine datasets demonstrate the efffcctiveness of our approach. Specifically, applying confidence- based Early Stopping (CaTS-ES) to Best-of-N improves MathQA accuracy from 73.7 to 83.6 with a sample budget of 16 responses, demonstrating the effectiveness of the confidence-based sampling strategy inference time. Our codes are available at https:://// //tp://gittttt com Chengsong Huang/ Self f- weae-Caalbaaiiiiaoo
INTRODUCTION
Leveraging additional computation dur- ing inference can enhance the quality of responses generated by large language models (LLMs) (Snell al., 2024a; Yao et al., 2023; Wu et al., 2024; Chen et al., 2025a). Among these methods, repeated sampling (Brown et al., 2024) such as Best-of-N (Cobbe al., 2021) and Self- Consistency (Wang et al., 2022b) gener- ate multiple candidate responses and select the final answer by a scoring model or a majority voting rule. While these meth- ods have proven effective, they require a fixed amount of sampled responses for each query regardless of its difficulty and com- plexity. Although increasing the sample size generally improves performance, it also increases computational costs and in- ference time (Amini et al., 2024). This is
87.5
Save 94.2% samples
85.0
82.5
80.0
Save 50.4% samples
Accuracy
77.5
75.0
Save samples
72.5
Self-Consistency
CaTS-SC
70.0
2a
2
2
27
2
Number of Responses (N)
Figure 1: Accuracy over response numbers of vanilla Self-Consistency (SC) CaTSS-C on MathQA using our trained Llama-3. --B-----Ittn-tuu model. The horizon- tal lines mark the response usage difference required for CaTS-SC to reach the same accuracy with vanilla SC.

---

Published as a conference paper at ICLR 2026
particularly inefficient for simple questions like + ?", where few samples are sufficient to find the correct solution (Chen et al., 2024), and extensive sampling is unnecessary.
Previous adaptive sampling strategies (Aggarwal et al., 2023; Li et al., 2024; Wan al., 2024) typically design lightweight stopping criteria to determine whether additional responses should be sampled. However, they often incorporate manually designed features or heuristic rules, such as early stopping when the model generates the same response three times consecutively, which can limit their generalizability across different tasks and models. Therefore, is critical to design lask-independent, model-agnostic approach without heavy reliance on human-designed heuristics.
We propose efficient test-time sampling method by using model confidence for dynamic sam- pling adjustment, since confidence can be seen as an intrinsic measure that directly reflects model uncertainty on different tasks. However, extracting accurate confidence can be challenging since LLMs are known to be overconfident in their own responses (Lin et al., 2022: Xiong et al., 2023; Leng et al., 2024), and their confidences often exceed the actual accuracy, especially in small models. Self-Consistency (Wang al., 2024a) can provide a relatively accurate confidence estimation by aggregating answer counts from multiple sampled solutions (Tian et al., 2023a), but it again requires sampling large number of responses for each query beforehand.
To address this, we introduce Ceff-Calibration to train L.MSs for accurate confidence estimation in only one forward pass, without requiring any numan-labeled data. Specifically, we improve model calibration by distilling
CoCCoositinc-ddevvvve confidence into the model itself. This is done by constructing pseudo training tuples of query, answer, and confidence test time, we propose Calibrated Test-Time Scaling (CaTS), including early stopping for Best-of- N when sampled responses reach a target confidence (CaTS-ES), and Self-Consistency weighted by confidence (CaTS-SC). We also provide a systematic proof of the conditions under which this confidence- weeidtence-eeighted approach is theoretically guaranteed to outperform vanilla Self- Coff-Consistency.
diverse training set. At
Empirical experiments on three LLM architectures across nine datasets demonstrate that our confidence-basee test-time scaling approaches consistently outperform their baseline counterparts under the same sampling budget. Specifically, both CaTS-ES for Best-of-N and CaTS-SC improve MathQA accuracy over their baselines from 73.7 to 83.6 with an average sampling budget of 16 responses. More importantly, our approaches can achieve comparable performance with substantially fewer computational resources. As shown Fig. 1, CaTS-SC can save 94.2% samples to achieve an accuracy of 85.0, compared to standard Self- Self-Consistency, demonstrating that reliable confidence estimation can significantly enhance the computational efficiency of test-time scaling.
2 REPEATED SAMPLING
Repeated sampling (Brown et al., 2024) is a framework that generates multiple responses with Chain-of-Thouge prompting (Wei al., 2022), then verifier to get the final results. We will introduce three fundamental repeated sampling strategies, which aim to enhance response quality.
2.1 BEST-OF-N
For each input query multiple candidate responses {y:} are sampled, where 1 1 < N. function such as an additional reward model or a confidence generator- assigns each response score Score(y:). The simplest selection strategy, known as Best-of-N (Cobbe et al., 2021), chooses the response with the highest score as the final answer.
N, A scoring
2.2 SELF-CONSISSEENY
CofESsnsistencyy (Wang et al., 2022b) (SC) selects the most frequent response among different candidates. Given candidate responses {y/, .2......2 voting. This approach enhances robustness by aggregating diverse model outputs rather than relying on single highest-scoring response.
.....N ). the final answer is determined by majority
2

---

Published as a conference paper at ICLR 2026
Sample query from seed datasets
Group responses by answer
Calculate the Soft
Self Self-Confidence (S5C)
Amy is 10. Jake is Alex's age right How old Alex?
years
(8+10)2 = 9. The answer 9.
0.9
Sample N responses
LLM
years old.
0.33/0.+0..++.33-0.15
Alex is years
0.85
[Amy Jake B.
Alex 10-8 2.....
0.15
Dr
(2) Labelling the Confidence
Alex is 9 years old
(les/No)
Generate confidences for each resporde
SmoothL1( #(Truel
LLM
D₁
optimize
D.
$fS8C
CrossEntropy
D. LLLLM ))
(1) Sampling Responses
(3) Training the LLM
Figure 2: Illustration of the Celf-Calibration framework. Given a query from the seed dataset, we sample N responses from the LLM. We use confidence querying prompt to let LLM assign confidence score to each response. Responses are then grouped by their answers, and the Soft Self-Cone Self-Consistency (SSC) score is computed for each group. During training, all data tuples contribute to improving the model's calibration, while higher-confidenn data used to enhance the LLM's generation ability.
2.3
ADAPTIVE SELF-CONSISTENCY
Adaptive Seff-Consistency (ASC) (Aggarwal et al., 2023) enhances the standard Self-Consistency approach by dynamically adjusting the number of samples based agreement among generated responses. This method iteratively samples responses and calculates the cumulative frequency un (=) and relative frequency fx (2) of each unique answer after samples:
(x(2) k
(=)
Σ 1 z), fx(2) E=1
The sampling process continues until the maximum relative frequency A (x(2) exceeds a predefined threshold r. Formally:
k +
if fax
y = arg mammxxxx2) otherwise.
This adaptive strategy reduces computational costs by limiting the number of required samples while maintaining high accuracy in the final answer selection.
SELF-CALIBRATIS
In this section, we provide an overview of our proposed Self-Calibration framework without re- quiring any ground-truth answer, illustrated in Fig. 2. First, we synthesize a set of cuputt-utput- confidence tuples (z,, V/. G) from seed dataset for training (Sec. 3.2). Using this synthetic dataset, we can train a language model with a combined loss to output calibrated confidence scores (Sec. 3.3).
3.1 CONFIDENCE SCORE ESTIMATION
naive way to obtain confidence score from L.LM P(True) (Kadavath al., 2022). Given the input-output pair (I;, 1s), we construct prompt as ① @ I, where / is confidence querying prompt, "Is the answer correct? (Yes/No)". The confidence score defined as the probability of token "Yes" in the next position, calculated from the raw logits before any temperature scaling is applied. Due to the KV-cache mechanism (Pope et al., 2022), the additional computational cost is roughly equivalent to generating 10 tokens, which is negligible compared to the typically longer input and output sequences, which is detailed Appendix L. Empirical results suggest that ((True) often lacks calibration, leading overconfidence in incorrect answers (Tian al., 2023b). So aim
3

---

Published as a conference paper at ICLR 2026
supervised training to improve the calibration of P(True), helping LLMs produce reliable confidence scores.
3.2 TRAINING DATA GENERATION
Our goal is to create a labeled dataset D, = (ar, y, c); without human annotations, where (æ; y) is query-response pair and an accurate confidence. To achieve this, we first generate multiple candidate answers for each query and ensure diversity via Dynamic Temperature sampling. Next, we calibrate the confidence each candidate through Soft Self-Consistency, which integrates the model's s intrinsic probability estimate with the overall agreement among different responses.
Soft Self- Confisonsst Score. Previous work has shown that self-consistency scores provide strong zero-shot cal- ibration (Wang et al., 2024a), outperforming P(True) or raw logits as confidence measures (Guo et al., 2017a). To further enhance the reliability of the confidence score in the training set, we introduce a soft self-consistency score, which integrates P (True) with self- celf-connist and offers a more accurate and robust confidence estimation.
Table 1: Comparison of ECE scores be- tween different confidence estimation methods of L 1-8B-Insttuut GSM8K and SVAMP.
Method
GSMSK
SVAMP
P(True)
12.03
28.94
For each query z, ent responses, each with an associated confidence score. N, we
the LLM to generate N differ-
SC
4.48
4.94
SSC
3.42
3.75
Given the set of triplets (I, Vn, Cp) where compute the soft self-consistency (SSC) score as:
Eiy.ey
SSC(y)
ΣN G
Using this score, we construct the final training set as (x, Uo. SSC(y:)), where SSC(y; provides calibrated confidence estimation for each response.
As shown in Table 1, SSC achieves lower Expected Calibration Error (Guo et al., 2017b) (ECE, definition of which is shown in Appendix H) scores compared to both P(True) and standard self- consistency on GSM8K and SVAMP datasets, demonstrating its effectiveness in providing more reliable confidence estimates.
Dynamic Temperature. To generate more diverse and high-quality responses, adopt the Entropy-based Dynamic Temperature (EDT) Sampling method (Zhang et al., 2024b) when generating each response y- By adaptively inereasing the temperature when the entropy H of the output distribution low, EDT promotes greater response diversity while preserving output quality. Formally, the temperature T (H) is defined as:
To x M To x M 2 TO: (,
T(H)
otherwise,
where Tp is the base temperature, M is scaling factor, 7 affects the scale of temperature variations, and is threshold that sets the temperature to zero if To MY/H is below TO
3.3 TRAINING OBJECTIVE
We optimize the model's confidence estimation by minimizing the difference between the predicted confidence and the target confidence using the Smoothl.1 loss. To ensure that training on confidence estimation does not degrade the model's reasoning ability, we incorporate the standard generation loss of Chain-of- of-Though answers into the objective (Huang et al., 2022). Specifically, only responses with confidence scores above threshold r) are selected for training to guarantee the quality of the reasoning path. A weighting coefficient w is introduced to balance these two loss terms. The overall loss function is formulated as:
Lisal (0)
Σ (By.B))E
SmoothLi (po
Fz. I), .5) + Σ (- log po (vi z:)).
4

---

Published as a conference paper at ICLR 2026
4 CALIBRATED TEST-TIME SCALING
In this section, we introduce CaTS, which uses reliable confidence scores from the Self- Self-Calibration trained model. The confidence scores serve as a quality indicator for each response, allowing the model to prioritize more reliable answers. We present confidence-guided variants of three popular test-time scaling approaches: Best-of-N, Self-Consistency, and Adaptive Coff--ssssttncy.. Our seliddenee-giided variants achieve comparable performance with significantly reduced computational costs. Finally, we present the conditions under which this method is effective, along with the theoretical proof.
4.1 EARLY STOPPING WITH CALIBRATED CONFIDENCE (CATS-ES)
CaTS-ES improves the efficiency of Best-of-N by terminating the sampling process once response with sufficient confidence is found. Given sequential sampling process where each response yi is assigned a confidence score C₁, we follow this rule:
+ 1,
if <T
Vi.
otherwise.
This means that continue sampling responses one by until response meets the confidence threshold т, and such a response is selected as the final answer, avoiding unnecessary additional sampling and reducing computational overhead.
4.2 SELF-CONSISTENCY WITH CALIBRATED CONFIDENCE (CATS-SC)
CaTS--C extends the traditional Ceff-Consistency approach by incorporating confidence scores into the voting process. Instead of treating all sampled responses equally, we assign each response y/ confidence score c, leading to a weighted aggregation:
y = arg MIE
Σ
a aa
This modification ensures that responses with higher confidence contribute more significantly to the final selection, enhancing robustness by prioritizing more reliable predictions.
4.3 ADAPTIVE SELF-CONSISTENCE WITH CALIBRATED CONFIDENCE (CATS-ASC)
Similar to Self-Consistency with Confidence, we use confidence as the weight when calculating the relative frequency Adaptive Self Self-Consistency. CaTS-ASC will follow the rest part ASC.
k cel (ve z), fx(2)
(x(2) Σ
U1 (z)
4.4 THEORETICAL ANALYSIS
Under the theoretical framework proposed in this work (see Appendix for detailed proof), CaTS- SC is guaranteed to exponentially outperform vanilla self-consistency if the confidence signal satisfies the following inequality:
20g Ug
2UMV PMV
where the terms are defined as: q is the conditional correctness probability, i.c., the true probability that a model's output is correct given its confidence score. The expectation [||] is taken over the distribution of q across all samples.
μMv is the mean margin for vanilla self-consistency (majority voting), representing the average
lead of the true answer over single incorrect competitor, defined as μMV = 4-1

---

Published as a conference paper at ICLR 2026
UMV is the margin variance for vanilla self-consistency (majority voting), representing the uncer-
tainty in the voting process, defined as UMV [1 o)(k 2+
(k
Hy is the mean margin for CaTS-SC, defined as M4g IE 2(kg
q)(k g) c+ 1)2
by is the margin variance for CaTS-SC, defined as Uq
Therefore, our method outperforms the original approach when the provided confidence signal is sufficiently accurate.
EXPERIMENTS
5.1 EXPERIMENT SETUP
(Dubey et al., 2024), Qwen2.5-7B-Instrrue (Team, 2024) and DeepSeek-R DeepSeek-RI-Distill- L---issill--Qen--l.. (DeepSeek-A) AI, 2025). These models represent diverse architectures and training strategies, allowing to test the adaptability of our methods.
Models. We conduct experiments three open-source LLMs:
Seed Datasets. We construct training dataset with diverse reasoning datasets, including:
ARC_easy (Clark et al., 2018), commonsense QA (Talmor et al., 2019), LogiQA (Liu et al., 2020), GSM8K (Cobbe al., 2021), OpenBookQA (Mihaylov et al., 2018), ReClor (Yu et al., 2020), SeiQ (Welbl et al., 2017), SVAMP (Patel et al., 2021) and WinoGrande (Sakaguchi et al., 2019). For each dataset, we randomly sample 2,000 questions (without answers) from the training set to construct our training data. Additional details are shown in Appendix G.
Evaluation Datasets and Prompts. We evaluate our methods on three benchmark datasets: ARC- Challenge (Clark et al., 2018), Object-Counting (Suzgun al., 2022) and MathQA (Amini et al., 2019), covering mathematical and commonsense reasoning tasks in both multiple-choice and open- ended formats. These three datasets considered out-of-domain they differ from the datasets used in training, which we refer as in-domain datasets. To evaluate performance in an in-domain setting, also the test of GSM8K, SVMMP, and ARC easy. The details of datasets, system prompts and the task prompts of each dataset are shown in Appendix C.
Baseline Methods. We include adaptive test-time scaling methods such Early-Stopping Self- Consistency (ESC) (Li et al., 2024) and Reasoning Aware Seff-Conisstenyy (RASC) (Wan et al., 2024) for comparison. ESC divides the sampling process into sequential windows and halts further sampling when high-confidence consensus is reached within window. RASC enhances sampling efficiency by dynamically evaluating both the generated answers and their reasoning paths. To ensure fair comparison, we reproduce these two methods on our Lalf-Calibration trained models. We further introduce an untrained confidence-informe celf-consistency(ISSs (Taubenfeld et al., 2025) as baseline with probability confidence and Self-Certainty (Kang et al., 2025) which uses confidence for ranking.
Experiment Setting. To ensure fair comparison across different test-time scaling methods, use the same sample budgets for each of them. Sample budget refers to the average number of responses each method samples per query. For dynamic methods such as CaTS-ES and CaTS-ASC, we calibrate their threshold for each dataset so that the actual number of samples collected in practice closely matches, but slightly under, the target budget.
5.2 RESULTS
Table 2 shows the accuracy comparison of different methods with sample budget of 16. We observe that CaTS-SC, CaTS-ES, and CaTS-ASC consistently outperform their base counterparts. On Llama-3.1-880 --8-IIntruet CaTS-SC surpasses SC on MathQA (73.7 to 83.6), while DeepSeek- R1-Distill- 1.5B, CaTS-E3 outperforms Best-of-N on ARC RRC.chalee (54.1 to 66.5). These results highlight that integrating calibrated confidence enhances test-time scaling with the same sampling budget. We also provide evaluation results on reasoning datasets such as MMLU pro (Wang et al., 2024c), Hellaswag (Zellers al., 2019), and GPQA (Rein al., 2024) in Table 10.

---

Published as a conference paper at ICLR 2026
Table Accuracy comparison of different test-time scaling methods across three language models when the sample budget equals 16. The evaluation is conducted on three datasets: Obj C. (Ob- jecttCounting), MathQA, and ARC_C. AARC,Challeme number of responses sampled per query. The improvements of CaTS over their baselines are shown in parentheses. Results for sample budget set are shown in Appendix D.
"Sample budget" refers to the average
Llma--3.1 1a-ma-3..-8--nssttiie Insttrutttt
Qwen2.5-7B --B-.--------
DeepSeek-R D----iill---.S
Methods
Obj-C.
MathQA
ARC.C.
JObj.C
MathQA
ARC.C.
JObj.C.
MathQA
ARC.C.
SC
69.1
73.7
85.2
76.8
83.3
90.1
64.9
89.4
60.8
CISC
72.8
81.2
86.4
80.2
81.1
89.2
66.4
90.8
61.7
Self-Certainty
73.2
81.5
86.8
80.6
82.4
89.5
66.9
90.7
62.3
CaTS-SC
76,8 (+7.7)
83.6
(+9.9)
87.7 (+2.5)
81,2 (+4.4)
87.8 (+4.5)
90.8 (+0.7)
70.8 (+5.9)
91.8 (+2.4)
66.5 (+5.7)
Best-of-N
62.3
73.7
84.5
72.4
83.8
48.1
87.8
54.1
CaTS-ES
76.8 (+14.5)
83.6 (+9.9)
87.7 (+3.2)
81.2 (+8.8)
87.8 (+4.0)
90.8 (-0.3)
70.8 (+22.7)
91.6 (+3.8)
66.5 (+12.4)
ASC
67.9
72.7
84.6
77.2
83.2
90.9
64.5
89.4
59.5
ASC
75.2 (+7.3)
81.9 (+9.2)
86.6 (+2.0)
81.6 (+4.4)
87.2 (+4.0)
91.2 (+0.3)
70.4 (+5.8)
91.8 (+2.4) 65.1 (+5.6)
ESC
76.0
81.0
87.1
81.2
86.3
91.0
70.8
91.3
65.6
RASC
76.0
81.4
87.3
81.2
86.4
90.3
70.8
91.4
65.8
ANALYSIS
6.1 PERFORMANCE COMPARISON UNDER DIFFERENT SAMPLE BUDGETS
Increasing the sample budget allows for selecting higher- quality outputs but comes at the cost of greater compu- tational expense. To evaluate this trade-off, we compare different methods across multiple sample budgets and visu- alize their performance trends. All methods use the same responses generated by Self- Self-Calibration trained models. As shown in Figure 3, all methods achieve hetter accuracy as the number of responses increases. Our confidence- guided approaches consistently outperform their original counterparts in most settings. When the sample budget is small, Best-of-N performs better than CaTS-ES because early stopping might stop too soon with low threshold, missing a better response. The results of other models and datasets shown in Appendix F.
Accuracy/%)
60-
66
Sect
A3C
2 Number of Responses (N)
Figure Accuracy over varying sample budgets of different inference strategies on MathQA using self- calbbbaaedteed Qwen- ..5-7B-Instruction
6.2 CALIBRATED CONFIDENCE SCORE COMPARED TO REWARD SCORE FROM REWARD MODELS
We compare self-calibrated con- fidence scores with established open- source reward model approaches. A reward model is an additional scoring model used evaluate the quality of generated responses (Christiano et al.,
Table 3: Accuracy Best-of-16 on two models (Llama- 3. 3-B 3.sttBtInsttuet Instruct and 2w5n-22.-7B- 2.5t7--s--)t three datasets between self-calibrated confidence scores and reward scores from additional reward models.
Model
Dataset
Reward
Confidence
2017). Deployment of a reward model can introduce several limitations: (1)
MathQA
82.1
84.0
Llama
Object Counting
72.6
72.0
Reward scores are often unbounded or require dataset-specific normaliza- tion, thus difficult to apply universal threshold for filtering or reweighting responses; (2) Running an extra re-
ARC. RRC.llalee
86.2
86.6
MathQA
87.5
86.8
Qwen
Object Counting
76.6
76.4
ARC. RRC.Challenge
89.6
89.8
ward model increases inference time;
and (3) A dedicated reward model requires additional GPU memory, and less efficient for large-scale deployment.
7

---

Published as a conference paper at ICLR 2026
Table 4: Self-Calibration results across both in-domain and out-of domain datasets on three different models. We use Self-Cal. to denote Self-Calibration in the table.
Llama-3.1-8h1 ---ma-3.--BBIInttuuee
Qwen2.5-7B- Instruct
DS-RI-Distill-L.5
Dataset
Metric
Vanilla
Self-Cal.
Vanilla
Self-Cal.
Vanilla
Self-Cal.
In-Domain Datasets
ECE
13.70
3.79
87.39
16.88
40.03
GSMSK
AUC +
72.43
75.36
68.61
82.21
64.31
55.57
ACC
77.44
80.43
89.41
88.74
73.38
75.36
ECE
28.03
10.17
89.60
24.49
30.40
12.00
SVAMP
AUC
74.17
75.79
75.10
87.46
49.33
71.27
ACC
72.60
75.29
90.48
92.00
52.27
57.48
ECE
5.45
5.00
57.58
5.62
20.19
11.36
ARC.casy
AUC
81.16
76.89
66.10
76.75
62.89
66.86
ACC
87.73
89.21
92.11
92.01
54.00
56.74
Out-of-Domain Datasets
ECE
7.01
6.03
55.19
10.11
11.42
5,46
ARC.challenge
AUC
80.67
80.41
64.21
78.33
64.07
65.27
ACC
80.87
82.38
89.37
89.05
43.39
45.77
ECE
27.85
9,69
72.41
5.82
47.26
4.60
Object Counting
AUC
53.84
59.47
68.07
81.02
50.39
67.61
ACC
60.68
65.88
72.41
74.22
55.33
58.13
BCE
12.55
8.64
62.35
18.92
13.16
4.34
MathQA
AUC
85.23
87.21
72.48
69.80
78.89
66.09
ACC
44.18
52.34
49.85
64.18
37.69
43.21
For our analysis, we use the following reward models for comparison: for Llama-3. ----Instuuct,, use the reward model from RLHFlow (Dong et al., 2024): for Qwen-2.5, we utilize its official Process Reward Model (PRM)* 2 (Zhang et al., 2025). For PRM, we use the lowest reward score across all steps. We ensure the size of each reward model matches with their corresponding base models.
Table shows that our self-calibrated confidence scores achieve similar performance to reward model scores across all datasets when using Best-of-N. This means that method, by generating approximately 10 additional tokens, achieves a performance comparable to that of an extra reward model of the same size. We further conduct time analysis in Appendix to demonstrate the efficiency of our approach.
6.3 EVALUATION ON CALIRAAAINNAIION
To prove the effectiveness of our Self-Calibration model training to output accurate confidence estimations, we adopt three standard metrics for evaluating model calibration: ECE, Area Under the Receiver Operating Characteristic Curve (AUC) (Hendrycks & Gimpel, 2017), and accuracy (ACC). The explanation and details of these metrics can found Appendix
Table 5: Ablation study results on MathQA and Object Counting in Llama-3. L1-smooth" means using MSE loss instead of L1- smooth.
Results. In Table 4, we compare our models trained on Seff-Calibration objective with their vanilla base models on multiple in-domain and out-of-domain datasets. Self-Calibration trained models consistently lower the ECE score while generally improving accuracy. On GSM8K,
"w/o
MathQA
Object Counting
Caff-Caaibration reduces FCE from 13.70 to 3.79 while improving accuracy from 77.44%
Method
ECE L ACC
ECE | ACC
ours (full)
8.64
52.34
9.69
65.88
to 80.43%. Even in cases where ECE slightly increases, such as ARC_easy for Llama-3, 1--8B- Instruct, accuracy still improves from 87.73% to 89.21%. Moreover, the strong results on out-of- domain tasks demonstrate the generalizability of our method, seen in MathQA, where accuracy improves from 49.85% to 64. 18% for Qwen2.5-7B-Instmun
w/o EDT
9.14
51.44
10.40
62.88
w/o SSC
10.85
52.18
16.02
61.12
w/o L1-smooth
6.43
50.86
10.48
56.48
'https://huggingffaa coo/RHHF1ooom ow a. 1-88--88-------- straa Data hhttps://huggingtf co/ Qwwn//weetteeeee
8

---

Published as a conference paper at ICLR 2026
Ablation Study. We conduct an ablation study to investigate the impact of key components in Self- Calibration, including Dynamic Temperature (EDT), Soft celf-Consistency (SSC), and LJ Ll-smooth loss. Table 5 presents our ablation results on the MathQA and Obj Cou datasets.
Removing the dynamic temperature or the soft self-consistency score leads to noticeable increases in ECE and/or drops in accuracy. Meanwhile, replacing the Ll-smooth objective with MSE achieves slightly lower ECE on MathQA but reduces accuracy on both tasks, suggesting that our chosen loss formulation is more robust overall. These results demonstrate that each module contributes model calibration and reasoning performance.
6.4 CAN OTHER CONFIDENCE QUERYING PROMPTS WORK WELL?
Since our confidence-based approach was trained using a specific confidence querying prompt, we explore whether alternative prompts can achieve similar performance during inference. This analysis crucial for understanding the robustness of confidence querying prompts different from the training prompt.
Table Accuracy comparison between the original prompt and 6 alternative querying prompts on self- calibrated Llama-3.1-88 3.1-8B-Instruct Results are reported
We evaluate 6 alternative prompts (listed in Appendix E. at inference time. Table 6 shows that despite training with a specific prompt, other prompts yield comparable performance, with only minor variations. This suggests that our confidence querying approach is robust to prompt changes and our training framework improves model calibration rather than overfitting to spe- cific prompt.
mean=std-
Dataset
Method
Original
New
CaTS-ES
81.7
81.52.10.30
MathQA
CaTS-ASC
81.9
81.8 81.8040.21
CaTS-SC
82.1
01.63
CaTS-ES
67.2
70.80.11.9
Obj C.
CaTS-ASC
74.8
74.07 +1.07411.00
CaTS-SC
74.4
73.40
RELATED WORK
CaTS-ES
86.2
86.62.10.20
ARC C.
CaTS-ASC
86.6
86.63.4.05
CaTS-SC
86.4
86.35
7.1 TEST-TIME SCALING
Snell et al. (2024b); Chen et al. (2025b) studied optimal test-time compute allocation to significantly enhance efficiency. Self-Enhanced tree search frameworks (Bi et al., 2024; Lample et al., 2022; Koh al., 2024; Zheng al., 2025) aggregate multiple reasoning paths and employ sparse activation strategies. Beyond that, step-wise verifiers are leveraged to dynamically prune the search tree (Wang al., 2022a; Li et al., 2022: Lightman al., 2023a). Combining different versions of the same query can also improve the performance (Huang et al., 2024). Scaling (Chen et al., 2025a; Welleck al.. 2022: Wang et al., 2024b; Chen al., 2023; Madaan et al., 2023; Aggarwal al., 2024) that iteratively refines model outputs to improve performance in complex tasks.
7.2 MODEL CALIBRATION
Model calibration aims to align model's confidence with its accuracy. Prior research has explored scaling- based methods (Deng al., 2023; Guo et al., 2017b; Zhang al., 2020) and nonparametric techniques like binning (Zadrozny & Elkan, 2001). More recent work has introduced verbalized confidence, prompting models to directly output confidence scores (Lin al., 2022). Most studies focus on pre-trale pre-trained and instruction-tundl LLMs (Lin et al., 2022; Han et al., 2024), others investigate RLHF-trained LLMs and propose calibration through prompting strategies (Xiong et al., 2023; Tian et al., 2023b).
7.3 LLM VERIFIER
Our approach is closely related to LLM-based verifiers (Zhao et al., 2025), as both aim to evaluate whether generated response meets correctness criteria. Lightman et al. (2023b) trained verifiers that assess the correctness of generated solutions, enhancing the selection of accurate responses. Zhang et al. (2024a) trained verifiers using next-token prediction enhance reasoning performance in large language models. GenRM (Mahan et al., 2024) is an iterative algorithm that trains LLMs on self- -efereeneeteed reasoning traces align synthetic preference labels.

---

Published as a conference paper at ICLR 2026
8 CONCLUSION
We improve the efficiency of test-time scaling methods LLMs with reliable confidence estimation. Our Self-Calibration enhances LLM confidence estimation in one forward pass, without requiring any labeled data. We then propose CaTS by dynamically adjusting sampling strategies based on calibrated confidence scores. Experiments show that our approaches consistently outperform baselines under the same sample budget. Our findings suggest that reliable confidence estimation and dynamic sampling can substantially enhance the effectiveness and efficiency of repeated sampling.
REPRODUCIBILITY STATEMENT
To help reproduce our work, we present the following:
Datasets and Models: All datasets and models used in experiments publicly available. The specific prompts used to in our experiments are detailed in Appendix C.
Prompts:
Hyperparameters: A full list of hyperparameters for all experiments is provided in Appendix G.
ACKNOWLEDGMENTS
We thank the anonymous reviewers and the area chair for their time, effort, and constructive sug- gestions, which have greatly improved the quality of this paper. We thank Haolin Liu (University of Virginia) for helpful discussions regarding the theoretical part of this paper. This research was supported in part by the NVIDIA Academic Grant Program and WashU Ignite Interdisciplinary Grants.
REFERENCES
Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam. Let's sample step by step: Adaptive- consistency for efficient reasoning and coding with llms. In Conference on Empirical Methods in Natural Language Processing, 2023.
Pranjal Aggarwal, Bryan Parno, and Sean Welleck. Alphaverus: Bootstrapping formally verified code generation through self-improving translation and treefinement, 2024.
Afra Amini, Tim Vieira, and Ryan Cotterell. Variational best-of-n alignment. ArXiv preprint, abs/2407.060577 06057, 2024.
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Konzel-Kedziorsi,, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proc. of NAACL-HLT, pp. 2357-2367, Minneapolis, Minnesota, 2019, Association for Computational Linguisties. doi:
1245.
Bin Bi al. Forest-of-thought: Scaling test-time compute for enhancing Ilm reasoning. ArXiv preprint, abs/2412.09078,, 2024.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher R'e, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. ArXiv preprint, Abs/2407.21787 2024.
Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, and Sercan Ö Anı.. Sets: Leveraging velf-veeriiattion and self-correction for improved test-time scaling. ArXiv preprint, abs/2501.19306. 2025a.
Weizhe Chen, Sven Koenig, and Bistra Dilkina. Iterative deepening sampling as efficient test-time scaling, arXiv preprint arXiv:2502.05:: 05549, 2025b.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+3=? on the overthinking of ol oi-like llss. ArXiv preprint, abs/2412.21 187, 2024.
10

---

Published as a conference paper at ICLR 2026
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. 2023.
Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg. and Dario Amodei. Deep reinforcement learning from human preferences. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances Neural Information Processing Systems 30: Anmual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach. CA, USA. pp. 0299-4309 2017.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv preprint, abs/1803.054577 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. ArXiv preprini, abs/21 14168, 2021.
DeepSeek-AL. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
Ailin Deng, Miao Xiong, and Bryan Hooi. Great models think alike: Improving model reliability via inter-model latent agreement. ArXiv preprint, abs/2305.00188 2023.
Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf. ArXiv preprint, abs/2405.07863. 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, and etc. The llama 3 herd of models. ArXiv preprint, abs/2407.21783, 2024.
Yichao Fu, Xuewei Wang. Yuandong Tian, and Jiawei Zhao. Deep think with confidence, arXiv preprint arXiv:2508. 100008..5200 15260, 2025.
Amirhosein Ghasemabadi, Keith G Mills, Baochun Li, and Di Niu. Guided by gut: Efficient test-time scaling with reinforced intrinsic confidence. arXiv preprint arXiv:2505.20325, 2025.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.), Proc. of ICMI, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 2017a.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.), Proc. ICML, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 2017b.
Haixia Han, Tingyun Li, Shisong Chen, Jie Shi, Chengyu Du, Yanghua Xiao, Jiaqing Liang, and Xin Lin. Enhancing confidence expression in large language models through learning from past experience. ArXiv preprint, 0tth22444.03155 2024.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out- out-of--istribution examples in neural networks. In Proc. of ICLR. OpenReview.net, 2017.
Chengsong Huang, Langlin Huang, and Jiaxin Huang. Divide, reweight, and conquer: A logit arithmetic approach for in-context learning. ArXiv preprint, abs/2410. 10074, 2024.
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. ArXiv preprint, abs/2210. 16100, 2022.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zachary Dodds, Nova Dassarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know, ArXiv preprint, abs/2207.05231 2022.
Zhewei Kang. Xuandong Zhao, and Dawn Song. Scalable best-of-n selection for large language models via self-certainty. arXiv preprint ar0iv:2502. 18581, 2025.

---

Published as a conference paper at ICLR 2026
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents, 2024.
Guillaume Lample, Marie-Anne Lachaux, Thibaut Lavril, Xavier Martinet, Amaury Hayat, Gabriel Ebner, Aurélien Rodriguez, and Timothée Lacroix. Hypertree proof search for neural theorem proving, 2022.
Adrien Le Coz, Stéphane Herbin, and Faouzi Adjed. Confidence calibration of classifiers with many classes. Advances in Neural Information Processing Systems, 37:77686-77725, 2024.
Jixuan Leng, Chengsong Huang. Banghua Zhu, and Jiaxin Huang. Taming overconfidence in llms:
Reward calibration in rlhf. ArXiv preprint, 10000240,,,9,44 2024.
Yifei Li, Zeqi Lin, Shizhuo Zhang. Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making large language models better reasoners with step-aware verifier, 2022.
Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang. and Kan Li. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. ArXiv preprint, abs/2401. 104800 2024.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step, 2023a.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. ArXiv preprint, abs/2305.20050, 2023b.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. ArXiv preprint, abs/2205. 14334, 2022.
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: challenge dataset for machine reading comprehension with logical reasoning. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 20200//01
2020, pp. 3622-3628 ijeai.org, 2020. doi: 10.24963/jccii. 2020/501.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinmmnn with self-feedback, 2023.
Dakota Mahan, Duy Phung, Rafael Rafailov, Chase Blagden, nathan lile, Louis Castricato, Jan- Philipp Franken, Chelsea Finn, and Alon Albalak. Generative reward models. ArXiv preprint, abs/2410. 12832, 2024.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. Proc. of EMNLP, pp. 2381-2391, Brussels, Belgium, 2018. Association for Computational Linguistics. doi: 10. 186538/1//DIS 1260.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080-2094, Online, 2021. Association for Computational Linguistics. doi: 10.18653/w1/202 .nw.cl woaecmmmai..1e. 168.
Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scaling transformer inference. ArXiv preprint, abs/2211.05 2022.
David Rein, Betty Li Hou, Asa Cooper Stickland. Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: graduate-level google-prec proof q&a benchmark. In First Conference Language Modeling, 2024.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande. Communi- cations of the ACM, 64:99-106, 2019.
12

---

Published as a conference paper at ICLR 2026
Charlie Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. ArXiv preprint, abs/2408.033 033314 2024a.
Charlie Snell et al. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. ArXiv preprint, abs/2408.033 03314, 2024b.
Mirac Suzgun, Nathan Scales, Nathanael Scharli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. In Annual Meeting of the Association for Computational Linguistics, 2022.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proc. of NAACL-HILT, pp. 4149-4158, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. doi: 10.18653/v1/ N19-1421.
Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self coniistnsyy in llms. arXiv preprint arXiv:2502.06223, 2025.
Qwen Team. Qwen2.5: A party of foundation models, 2024.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. ArXiv preprint, (iss2305..14977 2023a.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. ArXiv preprint, abs/2305. 14975, 2023b.
Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leveraging reasoning paths for efficient Ilm sampling. 2024.
Ante Wang, Linfeng Song, Ye Tian, Baolin Peng, Lifeng Jin, Haitao Mi, Jinsong Su, and Dong Yu. Self-consistency boosts calibration for math reasoning. In Conference on Empirical Methods in Natural Language Processing, 2024a.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery. and Denny Zhou Self-consistency improves chain of thought reasoning in language models, 2022a.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, and Denny Zhou, Sclf-consistnncy improves chain of thought reasoning in language models. ArXiv preprint, abs/2203.11171. 2022b.
Yifei Wang, Yuyang Wu, Zeming Wei, Stefanic Jegelka, and Yisen Wang. A theoretical understanding of self-correction through in-context alignment, 2024b.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37: 95266-95290, 2024c.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoe Le, and Denny Zhou, Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.11903, 2022.
Johannes Welbl, Nelson F. Liu, and Matt Gardner. Crowdsoureing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pp. 94-106, Copenhagen, Denmark, 2017. Association for Computational Linguistics. doi: 10. 1...86//////11-44433
Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Generating sequences by learning self-correct, 2022.
13

---

Published as a conference paper at ICLR 2026
Menghua Wu, Cai Zhou, Stephen Bates, and Tommi Jaakkola. Thought calibration: Efficient and confident test-time scaling. arXiv preprint (at::2505.18044, 2025.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optinal inference for problem-solving with language models. 2024.
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Ilms express their uncertainty? empirical evaluation of confidence elicitation in 1lms. ArXiv preprint, abs/2306. 112330661106... 13063, 2023.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. ArXiv preprint, abs/2305. 10601, 2023.
Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. Reclor: reading comprehension dataset requiring logical reasoning. In Proc. of ICLR. OpenReview.net, 2020.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Carla E. Brodley and Andrea Pohoreckyj Danyluk (eds.), Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 July 1, 2001, pp. 609-616. Morgan Kaufmann, 2001.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:/905.074 2019.
Jize Zhang, Bhavya Kailkhura, and Thomas Yong-Jin Han. Mix-n-match: Ensemble and compo- sitional methods for uncertainty calibration in deep learning. In Proc. of ICML, volume 119 of Proceedings of Machine Learning Research, pp. 11117-11128 PMLR, 2020.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. ArXiv preprint, abs/2408.152400 2024a.
Shimao Zhang, Yu Bao, and Shujian Huang. Edt: Improving large language models' generation by entropy-based dynamic temperature sampling. ArXiv preprint, abs/2403. 2024b.
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. ArXiv preprint, abs/2501.07301, 2025.
Yulai Zhao, Haolin Liu, Dian Yu, SY Kung, Haitao Mi, and Dong Yu. One token to fool Ilm-as-a- judge. arXiv preprint arXiv:2507.0879. 2025.
Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Xinyu Yang. Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, et al. Parallel-rl: Towards parallel thinking via reinforcement learning. arXiv preprint arXiv:2509.0700 2025.
A THE USE OF LARGE LANGUAGE MODELS (LLMs)
We acknowledge the use of large language models (LLMs) as assistive tools in this research, with usage limited to refining grammar and improving language clarity in the manuscript and writing utility scripts for data preprocessing and postprocessing; all outputs from these models meticulously reviewed, revised, and verified by the authors, who retain full responsibility for all content presented in this paper.
14

---

Published as a conference paper at ICLR 2026
B THEORETICAL FRAMEWORK AND PROOFS
B.1 SETTING AND ASSUMPTIONS
Let A [I....,,K K 2 2, be the of possible answers and let the (unique) ground-trute -ruth answer be '' E A. From an LLM we draw n. samples
(X,, C.) A x [0,
(i=l....,ni)
where X; is the answer of sample i and C. is scalar confidence score of sample i.
Assumption (Conditional i..d.). Conditional on T, the pairs {(X,, Ci) are independent and identically distributed.
Assumption 2 (Monotone Confidence). There exists (uuaa q [0, + (x. 1), strictly increasing, such that for any E AA and any с E [0,
P(X; C. c, T =)
q(c).
We refer to q()) as the conditional correctness probability at confidence
Assumption (Distribution of wrong labels). When X; # T, the allocation of mass among wrong labels does not depend on the particular true class. A canonical and widely used choice is the symmetric model:
1c9(c) K-1 for each b + t.
=
Definition (Weighted voting rules). Given nonnegative weight function h [0, 1] - R4,, define the per-class score and decision
!!h)
T = arg max ((h) a
i=1
Two special cases:
Majority Voting (MV): h =
q-weighted voting (CWMV-q): h(g) 9-
When comparing a true class to a competitor + define the pairwise margin summand
y!!) -XX; b}) In((C))),
that S sp
B.2 MOMENTS OF THE PAIRWISE MARGIN
The next lemma gives exact conditional moments used throughout.
Lemma (Mean and variance of the pairwise summand), Fix a true class a and competitor & a. Under Assumptions 1-3, for any weight h with essential supremum H = |1hlloo 3o and for = (CC),
4is T = E (A)
Kq: l((g));
K -1
(1 - qi) (K2q: + K K-2) (K-1)
Var (((h))
Gs T = a)
h(g))2
and (x < H a.s.
Consequently, with ((C)).
H4 = E (t) = a E K - h(q)
Kg 1
(1
q) (K22 + -2)
2A == E. Var (((1))
q. T = a
T=
=
h(()2
(K 1)2
15

---

Published as a conference paper at ICLR 2026
Proof. Condition a: Under T = we have by Assumptions 2-3:
1 (c)
P(X; a | c, a) q(c), P(X; = b c. c
K
P(X; E | (K (())
K-
Thus y-(h) takes values + and 0 with those probabilities, and
1- 1-g(c)
Kg(c) h((()).
( (K)
c, T= a) = =h(a(e)) a(c) K K-1
K-1
(x, (x) c, a) hh((c))) (((c) + 1-g(c)
For the variance, E yields
Subtracting the squared mean
K-1
Var(Y,")) |,, a) (la(ee)²((()) (9()) k-1
(Kate-1))) (K-1y
(1-ttonK otg h(((c)) (K-1P
where the last equality an algebraic identity. Boundedness immediate since |r/h)| S h(4))) 1 H. Taking outer expectations in gives the stated μ4 and Un-
Positivity of the mean. Since q > almost surely by Assumption and h 2 is not almost surely zero, we have (4 0.
B.3 CONSISTENCY AND MON-ASYMPTOTHE EXPONENTIAL BOUNDS
Theorem (Strong consistency and exponential tails). Assume 1-3 and let H = |1A||o0 < co. For any true class a and any competitor b + O,
(1) P P(S
((h)
T
P
(h)
0 T
n nh
exp
exp
(1)
2 Un + H ph Consequently,
2ch 3 H2
where μu and Uh are given in Lemma
PP P(fn
fn * a T T= < 1K- exp
Ti u
(2)
2 + I ph
and in particular (fN,, + T) + exponentially fast as n + oa (strong consistency).
c(h)) Σ y(() where (y.())) are independent given T Proof. By Definition S S
Each summand is bounded by | < H and has positive mean = E[ T a] > 0
and average conditional variance un from Lemma Apply Bernstein's 's inequality for bounded independent variables to the centered sum E,(,, -MA):
P
(h)
nPu S -niph
I exp
20p + H
which is equation 1. The second inequality follows from μln < H. A union bound over b E AA) (a)
gives equation 2. Exponential decay in n is immediate.
B.4 WHEN q- WEIGHTING IS EXPONENTIALLY BETTER THAN MV
We now compare the error exponents (the constants in the exponential upper bounds) of MV and g-weighting under the same assumptions. Definition 4 (Bernstein error exponent). For weight h with H ||A]locs define the per-pair Bernstein exponent
E(ar+)())
2 + pn
with ph and Up as in Lemma The overall top-1 error satisfies P(Ta + T a) (K 1) expl- gla>-b)()) n) for each fixed + (Theorem 3).
16

---

Published as a conference paper at ICLR 2026
Theorem 5 (Explicit criterion for exponential advantage). Under Assumptions 1-3, with MV (h 1) and q-weighting (h(q) = q) and H 1, define
(1 q)( K -2]
μMV = E
UMV = E
K- 1
(K 1)2
q(Kg 1)
4 ()(K2g + K 2)
Pq IR
04 E
K
(K 1)2
If the inequality
PLA 20MV + 2
(3)
20g + 3/4g
JUMV
holds (for the distribution of (()) induced by C), then for each competitor a,
gga>"((g) >> gl = P(T, # a =T < (K - 1g)) & (K - (KK-1)----nbin (MV))
In words: q-weighting has strictly larger envor exponent and thus exponentially dominates majority voting.
Proof. This is an immediate comparison of the Bernstein exponents from Theorem Under the stated assumption equation 3, have g(a>b) (q) > g(a>b) for every pair (a, yielding the claimed dominance pairwise and hence for the top-1 error by the union bound.
Readable sufficient condition. The technical inequality equation is verifiable from held-ow out data. One can also give transparent sufficient conditions ensuring it holds. A useful example is two-tier confidence mixture:
Lemma 6 (Two-tier sufficient condition). Suppose q(C)) takes values qhi E ( 1) with probability
(0, 1) and go & qmi) with probability
Then the inequality equation 3 holds whenever
(m qhi - 1) (1 x) n) ( (g 9o 1))²
To (K Thi + K - 2) (1 7)
(K² noo + K - 2) KK- - ()((a (Kqui - -1) (1 T)qio (Kqio - 1))
(Kqhi - 1) + 11==((Kmm))) - -1))
T(1 - (K* Ih + - (1- 2o) 222 + +K
(I= 11)
(K qio -1))
In particular, if a and (ghi qio) are high-confidence votes), the inequality is satisfied; thus CWMV-q exponentially dominates MV.
small (ie., there is nontrivial mass of genuinely
Proof. Substitute the two-point distribution into the definitions of (g. Ug. PMV. UMV and check equation 3. The final statement follows by continuity: as т ((uui 9to) grows, the left-hand side increases while the right-hand side decreases, because g-weighting shrinks the contribution of low- confidence votes (reducing the denominator via the 9² factor) while preserving the positive drift on high-confidence votes (numerator).
Binary simplification (K 2). For K 2 the formulas simplify to
PMV E 2q 1). UMV E[ 4q 4) ). and equation becomes
Mg E[ 9(2 1)], 1/2 = =|4 9)1,
(Elg(2g - 1)])²
(k(2q 11)
2 (1 4) + (2 2 1)]
[Ag 4)) 3 1]
C PROMPTS
C. DETAILS ABOUT TEST DATASET
ARC-Challenge includes difficult science questions requiring external knowledge and reasoning. Object-Counting focuses on numerical and spatial reasoning by counting objects. MathQA tests mathematical problem-solving across arithmetic, algebra, and calculus.
17

---

Published as a conference paper at ICLR 2026
C.2 SYSTEM PROMPT
Here we show the system prompt to let the model generate responses for Chain-of Thoughts and format for extracting the final results.
For the following question, provide step-by-ster
explanation of your thought process. Use the format demonstrated below for your response.
"Example Format:
Explanation: <Your detailed explanation here, outlining how you arrived at your answer <Insert your concise answer here, which should include a (answer.type) (e.g., {demo))>
Answer:
Ensure that your response strictly adheres to this format. Explicitly include the words 'Explanation'', Answwe:'.
The answer type includes "option letter" and "number".
C.3 DATASET PROMPTS
We show the prompts for each dataset in Table 7. All datasets and models are open-sourced.
Table 7: Query templates for each dataset.
Dataset
Query Template
gsm8k
Question: (question](a
scig
Question: (question http://hppppooooooooopoooooossss (ofoptions An
commonsense qa
Question: ( htt.:/iooooooooootoososseeeeee \a
winogrande
Question:
(sentencee))opptioos::/)) [option1 1BB.
{opticn2){n
openbookqa
Question:
(guestion///optioonssll (opionsstetttit
reclor
Passage (assage:/nppasee |u[psssse////n/ouuttiioo
(question]laja
Options: (p (opptiosssttett//
math.cm qa
Problem: hogbllaenttetttmmoottooooo//ooe
arc_challenge
Question: (question\\n optiions:\//oppiiiosssttt/
arc_easy
Question: [masttain)yyorrroooossooootossss
logiqa
Articler Articler\n(contee hatte///occcttttt/////eeessssee
(question]lala
Options: /ofoptioosstext//l.
svamp
Question: (Body + Question](n
apqa
(Question)) hncgetions://ooptioosssstto
aqua_rat
Question: (peesttoonkhopprriooooo/opppossr
D FULL MAIN RESULTS
Here show the main results when sample budget 4 in Table 8.
When the sample budget is small, the model has limited opportunities to explore different reasoning paths. In this scenario, output variability is often high, and having an additional confidence signal (as
18

---

Published as a conference paper at ICLR 2026
Table Accuracy comparison of different test-time scaling methods across three language models. The evaluation is conducted on three datasets: Obj C. (Object Counting), MathQA, and ARC C. (ARC_ (RRC.Challegge) "Sample budget" refers the average number of responses sampled per query. The improvements of confidence-augmentee methods over their baselines are shown in parentheses. All methods use the same responses generated by Self-Calibration trained models.
1llma-3.1-BB-mma
Swee2.5--B-Instree
DeepSeck Baacck-RR-iiiiiie R1-Distill-1.5B
Methods
Obj.C.
MathQA
ARC.C.
Obj.C. C.
MatQQ
ARC.C.
Obj.C. -.j.C. MathQA
ARC.C.
Sample budget
SC
65.1
71.5
83.8
74.4
82.7
89.9
57.7
79.2
48.2
CaTS-SC
72.8 (+7.7)
82.1 (+10.6)
86.4 (+2.6)
78.4 (+4.0)
86.9
(+4.2)
90.3 (+0.4)
64,0 (+6.3)
91.2 (+12.0)
63.2 (+15.0)
Best-of-N
60.7
73.5
84.5
72.0
83,4
89.7
50.1
77.8
44.2
CaTS-ES
67.2 (+6.5)
81.7 (+8.2)
86.2 (+1.7) (7) 78.4 (+6.4)
87.1 (+3.7)
90.1 (+0.4)
56.0 (+5.9)
90.6 (+12.8)
59.0 (+14.8)
ASC
67.5
72.7
84.6
75.2
83.2
88.9
55.3
79.1
48.5
CaTS-ASC
(+7.3)
81.9 (19.2)
(+2.0)
80.0
(+4.8)
87.2 (+4.0) (+1.7)
62.8 (+7.5)
91.6 (+12.5)
64.6 (+16.1)
ESC
72.0
78.6
85.8
80.0
86.9
89.6
58.0
91.2
63.0
RASC
72.4
79.0
85.8
80.0
86.4
89.8
62.6
91.2
63.1
ASC w/ Conf.) is essential for filtering out noisy or incorrect responses. This confidence-augmented method helps select the most promising candidate under tight sampling constraints.
However, when the sample budget increases, the model can generate more candidate solutions, which typically raises the chance of hitting the correct answer. In this setting, Early Stopping approach Saraacc--specillffll when coupled with a high confidence threshold can terminate as soon as encounters correct reasoning path.
E
FULL RESULTS OF DIFFERENT CONFIDENCE QUERYING PROMPTS
E. CONFIDENCE QUERYING PROMPTS
We show the confidence querying prompts we used in Sec. 6.4.
1 Is this the correct answer?
1z Does this answer seem right?
J:: Is this the right answer?
L Is the given answer accurate?
Is: Would you say this answer is correct?
To: Is this response correct?
E.2 RESULTS OF DIFFERENT QUERYING PROMPTS
In Table 9, we show the results of different confidence querying prompts for tuned -Lama-3.1-8B Instruct.
Table 9: The results for different confidence querying prompt.
Dataset
Method
2
3
4
5
6
Original
CaTS-ES
81.7
81.4
81.7
81.3
81.1
81.9
81.7
MathQA
CaTS-ASC
81.9
81.9
81.8
81.8
81.4
82.0
81.9
CaTS-SC
81.5
81.4
81.5
81.7
81.9
81.8
82.1
CaTS-ES
70.0
71.6
69.6
68.0
73.6
72.0
67.2
Object_Countingg
CaTS-ASC
73.6
73.6
74.4
73.6
76.0
73.2
74.8
CaTS-SC
72.8
74.0
73.2
72.4
74.4
73.6
72.8
CaTS-ES
86.8
86.4
86.8
86.5
86.8
86.4
86.2
AhC.challe
CaTS-ASC
86.7
86.6
86.6
86.6
86.7
86.6
86.6
CC
86.3
86.1
86.1
86.7
86.3
86.6
86.4
19

---

Published as a conference paper at ICLR 2026
F
RESULTS FOR DIFFERENT SAMPLE BUDGETS
Here, we show the performance under different sample budgets of other datasets and models.
64
ASC
52.5
07
74
77.5
25.0
55
70.0
2)
23
Number Respanses (1)
Number of Responses (N)
Namber of Resporses ON)
(a) ARC_Challenge with Llama-3.1-
(b) Object Counting with Llama-3.1- 1-
(c) MathQA with Llama-3.1-88
8B-Instruct
8B- 8B-Instruct
Instruct
$7.5
91.5-
Exise5
65.0-
91.0-
20.5
BG
60.0-
27.5
69.5-
76
725
58.5
70..
59.0
20
2
Number Responces
Number Responses
Number Responces
(d) ARC.Challenge with Qwen-2.5- (c) Object Counting with Qwen-2.5-
(f) MathQA with Qwcn-2.5-7B- Instruction
7B-Instruction
7B-Instruction
Figure 4: Performance comparison of different inference strategies across datasets and models using CaTS.
G HYPERPARAMETERS
This section details the hyperparameters used in our experiments. We categorize them into training data generation, training process, and response generation
G.1 TRAINING DATA GENERATION
When creating the datasets, we set the number of responses for each query N parameter in dynamic temperature, we follow the default hyperparameter settings from the original paper: To = 0.8, 0.8, 7 = 1.0, and TO = 0.001.
For the
G.2 TRAINING PROCESS
In the training objective, we set the threshold = 0.75 to filter the response used in generation ability training and the weight w 0.1 to balance two losses.
In the training process, we use the AdamW optimizer with learning rate of 5 x 10-5, The total number of training samples set to 100,000, while 1,000 samples are used for evaluation. We employ a batch size of with gradient accumulation steps of 64 to simulate larger effective batch size. The model trained for epoch.
For parameter-efficient fine-tuning, apply LoRA with rank 32, scaling factor 16, and dropout rate 0.05. the whole training examples, the ratio of causal language modeling data 0.7. We train the model multiple datasets with varying proportions of training and evaluation data. Specifically, GSM8K and SVAMP each contribute 15% of the training and evaluation samples. SciQ. CommonsenseeAA Winogrande, OpenBookQA, ReClor, ARC-Easy, and LogiQA each contribute 5 5% of the training and evaluation samples.
During the sample training data selection process, ensure that the data is evenly distributed across different confidence intervals. This balancing strategy prevents overrepresentation of any specific confidence range, allowing the model learn from diverse of samples. By maintaining
20

---

Published as a conference paper at ICLR 2026
1aama-.1--88-nnstteee
Qwen2.S-7BlInssruct
Methods
GPQA
Hellaswag
MMLU pro
GPQA
Hellaswag
MMLU pro
GPQA
Hellaswwg
MMLU
SC
33,28
66.62
49.19
34,72
71.16
52.83
38.32
36.65
34,56
Selt-Certuinty
34.12
68.73
50.44
35.91
72.58
54.21
38.71
37.52
35.12
(+2.3)
(16.2)
53.43 (14.2)
40.91 (16.2)
(15.9)
57.66 (14.8)
(+1.2)
(+3.8)
(+2.9)
Best-of-N
32.26
67.28
47,31
37,24
71.02
$1.87
37.16
36.97
32.63
CaTS-ES
36.55 (14.3)
73.72 (16.4)
53.83
40.40(+3..) (+3.2)
76,72 (15.7)
58,18 (16.3)
39.53(12.4)
40.60 (+3.6)
37.68 (+5.1)
33.79
66.52
49,37
34,72
71.22
$2.53
39.49
36.47
34,42
CaTS-ASC
36.55 (42.8)
72,70 (16.2)
53.63 (44.3)
38,89 (14.2)
76.56 (+5.3)
57.20 (44.7)
40.70 (41.2)
40.04 (43.6)
37.31 (42.9)
52.84
76.96
$6.98
36.83
RASC
34,86
72:36
52.00
37,76
76.76
$7.00
39.32
40.12
36.65
Table 10: The results for additional datasets.
equal number of training examples in each confidence bin, we improve the robustness confidence calibration and reduce potential biases in the learning process.
G.3 RESPONSE GENERATION
When generating the response, we set the temperature equals to 1.0.
H
EXPLANATION OF CALIBRATION ESTIMATION
ECE measures the discrepancy between model's predicted confidence and its actual accuracy, defined
ECE Σ Bml (ace(Bm) conf (Bm)),
where M is the number of bins, Bm represents the set of samples in the m-th bin, and N is the total number of samples. lower ECE value indicates better calibration, meaning the model's confidence aligns more closely with its actual correctness. In our experiments we set m 10.
I RESULTS ON ADDITIONAL DATASETS AND MODELS
We provide the results in three more dataset in Table and Qwen3-44(nonn) (oonn---4nnnnttnnnig in Table 11.
ObjCC
MaribOA
Hellaswag
MMLLUPro
$7.60
56.03
91.29
36.87
20.28
57.00
CaTS-SC
89.15 (+1.55)
87.85 (+1.8)
(+1.25)
38.52 (+1.65)
72.40 (+1.52)
59.65 (+1.75)
Rest-ffN
84.52
36.27
(+1.62)
(+1.78)
92.10 (+1.24)
38.05
(+1.75)
$7.40
ASC
$8.00
86.07
91.37
37.03
70.74
$7.87
CETS-ASC
89.80 (+1.50)
87.95
(+1.888
92.72 (+1.355
38,90 (+1187)
72.60 (+1.86)
59.95 (42.08)
ESC
$7.53
CATS-ESC
(+1.75)
87.80 (+1.72)
(+1.28)
39.20 (+1.,7)
(+1.71)
60.10 (42.11)
Table 11: Performance comparison of CaTS variants and baseline methods across multiple bench- marks. Numbers in parentheses indicate improvements over the corresponding baseline.
J REWARD MODEL TIME COMPARE
When the reward model and generation model are similar in size, inference with both models doubles computational cost and latency, as each input requires two forward passes-one for generation and one for evaluation. In contrast, Self-Calibration integrates confidence estimation into the generation model, enabling single-pass inference and significantly improving efficiency. Empirically, we find that reward model evaluation takes 1.71 seconds per sample (without batching), while Self-Calibration requires only 1.08 seconds, showing the efficiency advantage of method.
K SUPERVISED CORRECTNESS vs. SELF-INDUCED CONFIDENCE
21

---

Published as a conference paper at ICLR 2026
To further examine the effectiveness of our self-induced confidence signal, conduct ablation study in which the confidence supervision is replaced with binary correctness labels (0/1). The results shown in Table 12. Across all benchmarks, supervised correctness provides only slight improve- ments over the baseline but consistently underperforms compared to the self-induced confidence signal. This suggests that the soft, self-consistency deriive supervision offers more informative guidance than discrete correctness labels. The limitations of supervised correctness are particularly evident in the early-stopping (ES) setting. When trained with binary labels, the model's confidence predictions collapse toward the extremes (0 or I), which substantially degrades CaTS-EB IS-ES performance. In contrast, the unsupervised confidence signal remains stable and continues support reliable early stopping.
Overall, these findings demonstrate that directly supervising confidence with correctness labels does not yield performance gains. The self-induced confidence signal is both more informative and more robust, especially in scenarios involving early stopping and confidence-weighted voting.
Method
Obj.C
MathQA
ARC_C
CaTS-SC
76.8
83.6
87.7
CaTS-SC (supervised)
76.2
81.2
87.3
CaTS-ES
76.8
83.6
87.7
CaTS-ES (supervised)
69.8
81.2
86.2
CaTS-ASC
75.2
81.9
86.6
CaTS-ASC (supervised)
74.8
80.2
86.6
Table 12: Ablation results comparing self-induced confidence with supervised correctness across different CaTS variants.
L TOKEN NUMBER ANALYSIS
We report the average token usage in MathQA with Llama-3. -...-nstruut--slf-clllle model, showing that the overall token-level count remains proportional to the sample count. Therefore, the reported efficiency gains remain valid and reflective of actual resource usage in practice.
Response # Self-Consistency # Token
CaTS-SC Token
345.28
355.28
2
690.99
710.87
3
1022.44
1051.99
5
1633.28
1681.15
10
2804.93
2890.71
20
3375.38
3482.24
Table 13: Token usage comparison between Self-Consistency and our method.
M ADDITIONAL RELATED WORK
Recent test-time scaling approaches also leverage intrinsic confidence signals, but differ funda- mentally from our self-calibration framework. Deep Think with Confidence (Fu et al., 2025) extracts token- step-level confidence filter reasoning traces, whereas our method derives soft, self-consisteney-bueee confidence signal that reflects reasoning-lever correctness. Thought Calibration (Wu et al., 2025) relies on trained probes over hidden states to detect reasoning errors, while our approach does not need access to hidden states and labelled data, and generates confidence directly from the model's own reasoning path. Guided by Gut (Ghasemabadi et al., 2025) enhances intrinsic confidence through RL and tree-search mechanisms and relies required ground truth answers, while our lightweight, RL-free calibration yields a stable learnable confidence predictor for both voting and early stopping. Although surrogate confidence calibration has been explored for traditional classifiers (Le Coz et al., 2024), their binary calibration does not extend to multi-step reasoning: in comparison, method performs soft self-calibration tailored to
IsCorrect() function that
22

---

Published as a conference paper at ICLR 2026
LLM reasoning, providing richer and more stable signal without collapsing to hard correctness labels than just labels.
23


Reviews:
[{"review_id": "4w49mnpKFW", "reviewer": "Reviewer_jmso", "summary": "This paper proposes a test-time scaling method for LLMs, CaTS, integrating self-calibrated confidence scores into common repeated sampling methods.", "questions": "* How sensitive is the performance to hyperparameter choices: weighting coefficient w in training objective, threshold in CATS-ES, sampling budget, etc. How easy is it for practitioners to use CaTS? Do default hyperparameters work well across LLMs and benchmarks?\n* Only applicable for white box models. Are there variants of this method for black box reasoning models?\n* How are the hyperparameters chosen: weighting coefficient w in the training objective, threshold in CATS-ES?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Proposes an efficient test-time sampling method that uses self-calibrated confidence scores for dynamic repeated sampling to improve LLM performance.\n* Strong performance improvements across benchmarks and LLM families.\n* Intuitive design of self-calibration approach not requiring ground-truth supervision during training and a single forward pass during inference.\n* The authors provide a theoretical proof of how and when CaTS is better than vanilla self-consistency\n* The methodology is clearly presented and well written with descriptive figures.\n* The authors show calibrated confidence scores compare well with reward model approaches.", "weaknesses": "* Show results on challenging reasoning benchmarks used for reasoning LLM evaluation: More challenging and conventional reasoning benchmarks should be reported on, including MATH-500, AIME, GPQA-diamond, MMLU, etc, since the title focuses on efficient LLM “reasoning”. The benchmarks used might not be challenging for reasoning LLMs, and the transfer of test-time scaling results to harder questions should be investigated.\n* Show results with reasoning LLMs: Apart from DeepSeek-R1-Distill-1.5B, the other models Llama-8B-3.1-Instruct and Qwen2.5-7B-Instruct are not reasoning LLMs. Since the title focuses on efficient LLM “reasoning”, results with reasoning LLMs, including Qwen3, and GPT-OSS should be included.\n* Variance in performance across multiple runs: Reasoning LLM performance has variance across runs (e.g., on AIME). Deep Think with Confidence, Fu et. al. report performance averaged over 64 independent runs. Is the performance in CaTS reported over multiple runs? How low/high is the variance?\n* The OOD results shown are not out-of-domain as such. For example, ARC Easy is present in training, and  ARC Challenge in test. While these results show some generalization, results on test benchmarks from different domains (e.g., GPQA in test) would be better evidence.\n* Consider comparing to more recent work: Test-time scaling (TTS) is an active area of research with recent work also leveraging an LLM’s internal confidence signals without requiring external supervision (see Deep Think with Confidence, Fu et. al., and Thought calibration: Efficient and confident test-time scaling, Wu et. al., and Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence, Ghasemabadi et. al.). Further, creating a surrogate training dataset for confidence calibration has been explored in this prior work (Confidence Calibration of Classifiers with Many Classes, Coz et. al.), although not applied to LLMs. The authors should compare their novelty against these methods, and potentially include some of the recent work as baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4goiioxo8A", "reviewer": "Reviewer_fEfu", "summary": "CaTS proposes a calibrated test-time scaling scheme for LLM reasoning: first Self-Calibration distills a soft self-consistency confidence (SSC) into the model so it can predict correctness in one forward pass. With this calibrated confidence, they introduce CaTS-ES (early-stop Best-of-N) and CaTS-SC (confidence-weighted self-consistency) as drop-in inference policies.  On 3 LLMs/9 datasets it lifts MathQA 73.7 to 83.6 at N=16, matches reward-model selection without a second pass, and can reach the same SC accuracy with up to 94.2% fewer samples.", "questions": "See Weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The problems the paper addresses are important and timely , both how to incorporate confidence signals into LLM training and how to improve the efficiency of test-time scaling. \n2. The proposed soft self-consistency confidence is simple yet effective, and the empirical improvements are both significant and consistent across models and datasets. \n3. The paper is also clearly presented, with a coherent motivation and narrative that make it easy to follow.", "weaknesses": "1.\t**Objective overlaps with reward modeling.**\nThe training target effectively functions like a reward-model score that measures response quality. Given that the method aims to predict P(\\text{correct}) for early stopping and confidence-weighted voting, a natural question arises: what happens if the model is trained directly on binary correctness (i.e., replacing c_j in the training objective with the true correctness label)? Since the training data already include ground-truth answers, it would be valuable to add an ablation that supervises the model with 0/1 correctness and compares it against the self-induced confidence signal. This would reveal whether the proposed self-generated confidence provides any genuine advantage, or suffers drawbacks relative to using correctness as the target.\n2. **Off-policy pseudo-label drift.**\nSSC targets are computed once from the base model’s samples and kept fixed during fine-tuning. As the generator shifts, these pseudo-labels can mismatch the updated distribution and potentially entrench confidently wrong majorities. It would be helpful to analyze sensitivity to this self-distillation bias—e.g., iterative re-labeling with the calibrated model (one extra SSC pass), checkpoint-wise calibration drift (ECE over training).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WJETvzYSDf", "reviewer": "Reviewer_C3JX", "summary": "This paper aims to achieve more efficient test time scaling by incorporating better calibrated confidence estimation per sample. They do this by training the model via self-calibration to estimate its own confidence more reliably. Then, using this self-calibrated model they explore two strategies for test-time scaling: (1) early stopping in BON when the confidence estimate crosses some threshold, and (2) self-consistency and adaptive self-consistency weighted by confidence scores. A variety of experiments and ablations show that this method works, improves accuracy and is generally more efficient compared to baselines.", "questions": "-", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is a good paper. It was fun to read, the proposed method is very interesting / innovative and the breadth of experiments and ablations support the findings nicely.  \nThe self-calibration training framework seems to be new and a very clean approach in terms of computing soft self-consistency scores as the targets.", "weaknesses": "The tasks and domains are restricted to reasoning, so it’s a bit hard to reason about how generally useful this approach can be, especially in terms of the reliability of confidence estimation. But reasoning is a pretty good application to start with to demonstrate the viability of this approach.  \nNot a fan of ECE since it is possible to choose hyperparameters (e.g. number of bins, style of binning) to hide miscalibration issues, but the paper does report AUC as well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5oJxtS7zR6", "reviewer": "Reviewer_ZXsN", "summary": "This paper introduces Calibrated Test-Time Scaling (CaTS), a framework that leverages confidence estimation for adaptively scaling the amount of computation devoted to each test-time query in large language models. The central idea is to distill Self-Consistency-derived confidence into an LLM via a self-calibration procedure, enabling reliable confidence estimation in a single forward pass. CaTS proposes variants of common repeated sampling methods (Best-of-N, Self-Consistency, Adaptive Self-Consistency) that use these calibrated confidence scores to guide early stopping or assign voting weights. The paper presents a mathematical analysis demonstrating conditions under which the confidence-weighted approach provably outperforms vanilla self-consistency. Experiments on three open LLMs across nine datasets support the claims, showing improvements in accuracy per compute over baselines, plus ablations and robustness checks.", "questions": "1. What is CISC in Table 2?\n2. In Table 2, CaTS-SC and CaTS-ES results are almost the same. Is there an explanation?\n3. Figure 4(d) does not appear to behave as expected. Is there an explanation?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Principled Confidence Signal: The proposed self-calibration framework uses a pseudo-labeled dataset, and the design of the soft self-consistency (SSC) score is both intuitive and empirically strong.\nTheoretical Justification: The mathematical analysis provides a precise and nontrivial error bound.\nComprehensive Experiments: Numerous results tables/figures provide strong empirical support.\nComputational Efficiency: Results on token usage and comparisons with reward model-based alternatives show that CaTS substantially reduces inference cost without sacrificing performance.", "weaknesses": "1. Clarity and Terminology Issues\n- \"the model’s calibration\" and \"P(True)\" are not properly introduced. Also, \"P(True)\" and \"P(Yes)\" are used interchangeably.\n- Table 1 caption misses SVAMP.\n- Table 10 is not linked\n2. \"For dynamic methods such as CaTS-ES and CaTS-ASC, we calibrate their threshold for each dataset so that the actual number of samples collected in practice closely matches, but slightly under, the target budget.\" This seems heuristic; there might be a better way to measure the performance of dynamic methods\n3. The paper does not propose new early stopping or dynamic sampling techniques, nor does it introduce a novel threshold-selection strategy. Thus, some claims in the intro are unjustified.\n4. arxiv.org/abs/2502.18581 seems to be similar work, should be included as a baseline for empirical comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)