A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
Authors: Zhi Zhou
Yuhao Tan
Zenan Li
Yuan Yao
Lan-Zhe Guo
Yu-Feng Li
Xiaoxing Ma
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Model
Self Consistency
LLM Reasoning
Test-Time Scaling
Best-of-N
Primary area: deep_learning
TL;DR: We introduce the first theoretical framework for analyzing LLM reasoning errors, and bridge two typical sampling-based test-time scaling methods to achieve both low error and fast convergence.

Abstract: Test-time scaling seeks to improve the reasoning performance of large language models (LLMs) by adding computational resources. A prevalent approach within the field is *sampling-based test-time scaling methods*, which enhance reasoning by generating multiple reasoning paths for a given input during inference. However, despite its practical success, the theoretical foundations remain underexplored. In this paper, we provide the first theoretical framework for analyzing sampling-based test-time scaling methods, grounded in the perspective of confidence estimation. Based on the framework, we analyze two dominant paradigms: self-consistency and perplexity, and reveal key limitations: self-consistency suffers from high estimation error while perplexity exhibits substantial modeling error and possible degradation of the estimation error convergence. To address these limitations, we introduce RPC, a hybrid method that leverages our theoretical insights through two key components: *Perplexity Consistency* and *Reasoning Pruning*. *Perplexity Consistency* combines the strengths of self-consistency and perplexity, boosting the convergence rate of estimation error from linear to exponential while preserving model error. *Reasoning Pruning* prevents degradation by eliminating low-probability reasoning paths.
Both theoretical analysis and empirical results across seven benchmark datasets demonstrate that RPC has a strong potential for reducing reasoning error. Notably, RPC achieves reasoning performance comparable to self-consistency while not only enhancing confidence reliability but also reducing sampling costs by 50%. The code and resources are available at https://wnjxyk.github.io/RPC.

A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
Zhi Zhou
Yuhao Tan Zenan Li 2 Yuan Yao
Lan-Zhe Guo 1,3 Yu-Feng Li 1.4x Xiaoxing Ma 1*
1 State Key Laboratory of Novel Software Technology, Nanjing University, China Department of Computer Science, ETH Zurich, Switzerland School of Intelligence Science and Technology. Nanjing University, China
Schoool of Artifical Intelligence, Nanjing University, China zhouz@lamde nju. edu. cn, liyf@nju. edu. cn, xxm@nju. edu.
Abstract
Test-time scaling seeks to improve the reasoning performance of large language models (LLMs) by adding computational resources. A prevalent approach within test-time scaling methods, which enhance reasoning by
the field is sampling-base generating multiple reasoning paths for given input during inference. However, despite its practical success, the theoretical foundations remain underexplored. this paper, we provide the first theoretical framework for analyzing sampling-basee based test-time scaling methods, grounded in the perspective of confidence estimation. Based on the framework, we analyze two dominant paradigms: cfff-sossitency and perplexity, and reveal key limitations: colf-consistency suffers from high estimation error while perplexity exhibits substantial modeling error and possible degradation of the estimation error convergence. To address these limitations, we introduce RPC, hybrid method that leverages our theoretical insights through two key components: Perplexity Consistency and Reasoning Pruning. Perplexity Consistency combines the strengths of self-consistency and perplexity, boosting the convergence rate of estimation error from linear to exponential while preserving model error. Reason- ing Pruning prevents degradation by eliminating low-probability reasoning paths. Both theoretical analysis and empirical results across seven benchmark datasets demonstrate that RPC has strong potential for reducing reasoning error. Notably, RPC achieves reasoning performance comparable to self-consistency while not only enhancing confidence reliability but also reducing sampling costs by 50%. The code and resources are available at https: http::////jjik gi thub.io/RPG
1 Introduction
Recent advances in large language models (LLMs) have demonstrated their remarkable reasoning capabilities across diverse applications, including problem-solving 33 35), planning (53 13), and decision-making [45 (8). Test-time scaling methods [58] (57 20 can further enhance reasoning performance with additional computation. Among these strategies, the sampling-based test-time scaling method has emerged as simple yet effective technique. Through generating multiple reasoning paths and selecting the most plausible one through confidence estimation mechanism, such as self- colf-consistency [57] and perplexity [00], this approach has achieved significant improvements.
The success of sampling-base based test-time scaling hinges accurately estimating the confidence of sampled reasoning paths, facilitating their comparison and enabling the selection of the most
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

plausible answer in the Best-of-N manner 20 29I. Current confidence estimation approaches can be categorized into two main types: (i) consistency-basee methods, which use pre-defined consistency functions [1 to determine answer confidence, with self-consistency as the primary representative; and (ii) probability-bee -hased methods, which utilize scores from either the internal L.L.M probability [44] or external model (29) to evaluate the confidence of reasoning paths, with perplexity as the leading example. While both have proven effective in practice, a rigorous theoretical understanding remains lacking regarding their underlying mechanisms, inherent limitations, and potential improvements.
This paper addresses this gap by introducing the first theoretical framework for sampling-base based test-time scaling methods in LLM reasoning. Within this framework, reasoning error is decomposed into two components: Estimation Error and Model Error. We apply our framework to analyze two representative methods from each category. self-consistency and perplexity. and identify their limita- tions. Self-consistency. which relies on Monte Carlo estimation, achieves only a linear convergence rate in estimation error, resulting in unsatisfactory performance when the sampling budget is limited. Perplexity utilizes the internal probabilities of L.LMs to achieve exponential convergence rates, but they suffer from high model error, which compromises their effectiveness. Moreover, its convergence rate depends on the magnitude of probabilities and deteriorates significantly when probabilities are low. These insights provide guidance for developing improved methods: An optimal approach should simultaneossly achieve rapid estimation error convergence while maintaining low model error.
To address the above limitations, we introduce novel Reasoning Draasonig-pruniing Perplexity Consistency (RPC) method, which consists of two components: Perplexity Consistency and Reasoning Pruning. Perplexity Consistency integrates the internal probability of the L.M into the self-consistency framework, enabling rapid estimation error reduction that transitions from linear to exponential convergence while maintaining low model error. Reasoning Pruning mitigates the degradation issue of estimation error reduction rate when the internal LLM probability magnitude is low by automatically modeling the probability distribution and removing low-probability reasoning paths. Our theoretical analysis shows that RPC achieves both fast estimation error convergence and low model error, offering significant potential to reduce reasoning error. Our empirical results demonstrate the effectiveness and efficiency of RpC on seven benchmark datasets. Specifically, on four mathematical reasoning datasets, RPC reduces the required sampling budget by at least 50% while maintaining the same level of reasoning performance as self-consistency. When using the same sampling budget, RPC achieves an average improvement of 1. 1.29% over existing methods. Additionally, RPC provides confidence estimates that are more closely aligned with the ground truth compared to existing methods.
To summarize, the main contributions of the paper are:
(I) We propose a theoretical framework that formulates sampling-based test-time scaling in LLM reasoning and decomposes reasoning error into estimation error and model error. This framework enables an analysis of existing techniques, offering guidance for developing improved approaches.
(2) Based on this framework, we introduce a novel test-time scaling method, RPC. which leverages the internal probability of LLMs within the cofficonsistency paradigm and removes low- cow-probaiility reasoning paths, thereby accelerating error reduction and improving reasoning performance.
(3) Our theoretical analysis demonstrates that RPC achieves rapid estimation error convergence, providing strong potential for reducing reasoning error. Empirical results on seven benchmark datasets show that RPC substantially reduces the required sampling budget, improves reasoning accuracy, and enhances the reliability of confidence estimation.
2 Problem and Analysis
In this section, we first formulate the LLM reasoning problem using sampling-bassee based test-time scaling methods. Then, we theoretically decompose the LLM reasoning error into Estimation Error and Model Error, and analyze two representative methods of consistency- and probability-ban methods. Our analysis reveals insights for building an advanced LLM reasoning method.
2.1 Problem Formulation
Given a reasoning problem (æ, where &r represents the input query, and y represents the ground- truth answer. The LLM generates reasoning path t = (f1...., fmm by sequentially sampling tokens
2

---

according to the conditional probability distribution p(t; Ix, tee), where m denotes the length of the reasoning path. The probability of generating the reasoning path is defined as p(t x), a.k.a the confidence of the reasoning path f. An extraction function g(-) maps the reasoning path to the final answer ÿ = g()). We can further extend the probability to the answer ÿ, i.e., the answer confidence, denoted as p(ú .:) Take the mathematical reasoning problem as an example, the reasoning path f could be "1 + = 2. The answer is 2." and the function g(-) extracts the answer from f, resulting in 9 = g() = 2. The reasoning correctness is evaluated by the indicator function I = y]-
The confidence represents the probability that the reasoning path é or answer ý correct, allowing LLMs to select the most reliable solution from multiple candidates in the Best-of-N manner [20 29] However, in practice, accessing the confidence for all possible reasoning paths or answers of LLMs is computationally infeasible. Therefore, we typically estimate the confidence by sampling finite n reasoning paths ,,,.... , la from the LLM sampling distribution y(( xx), which yields the estimated confidence p({ æ)) or p(ù 2) for any reasoning path é or answer ý.
To measure the reasoning performance of LLMs, we use the squared error 22 to penalize reasoning error of confidence for any single reasoning path or answer y):
{t = E ($( x) - T(g(O) = =.) Ep(0) = E (0(0 x) 10 = vi)² -ī)=)))
(I)
where the expectation is taken over all possible combinations of n sampled reasoning paths i,, -...... En- which are used to estimate the confidence p. For notational clarity and simplicity, we omit the explicit form of expectation here and throughout the remainder of the paper.
In the following analysis, we categorize sampling-based test-time scaling methods into two types based confidence estimation aspects: consistency-basee methods and probability- based methods. Consistency- based methods 663 72 estimate confidence by evaluating the agreement among different reasoning paths, with coffisonsitteney [57) serving representative approach. Probability-basee methods (20] [0] estimate confidence using either the internal probability provided by the LLM or the scores from external models, with perplexity (57) being a representative example. Our further analysis is conducted on each representative method.
2.2 Theoretical Analysis
In this section, we theoretically decompose the reasoning error into estimation error and model error. Next, we examine the specific forms of reasoning error for two representative methods, self-consistency (SC) and perplexity (PPI.), to provide insights for improved algorithm design.
2.2.1 Reasoning Error Decomposition
Take the reasoning error Eg) of p(ū (r) in Equation 1 as an example, we can decompose the reasoning eτror into the estimation error and model error as follows.
Proposition 1 (Error Decomposition). For any impul with ground-truth answer y and any possible answer J, let p(ù x) denote the unbiased estimated confidence of ÿ and p(ğ (x) denote the ground truth confidence. Then, the reasoning error Eg(g) can be divided into two components:
Ep(i) = E [ x) - p(0 (=)) + (p(0) x)) 1 - vi)
(2)
Estimation Error
Model
where the expectation is taken over sampled reasoning paths E2...... In for estimating confidence. Remark 1. The detailed proof is provided in Appendix I Proposition I separates the effect of confidence estimation on reasoning eтror from the effect of the LLM's s reasoning capability. The Estimation Error depends solely on the sampling size and the confidence estimation strategy, while the Model Error is invariant and determined by the LLM's reasoning capability. This proposition demonstrates that, apart from the fixed Model Error, which determined by the LLM's inherent rea- soning capability, the reasoning error is bounded by the Estimation Error. Moreover, this proposition provides insights into two directions for improving LLM reasoning performance: (1) reducing the Estimation Error through larger sampling sizes or more accurate confidence estimation methods, and (2) reducing the Model Error by enhancing the LLM'' reasoning capabilities or developing more sophisticated and effective confidence metrics.
3

---

Next, we analyze two representative methods in sampling-basee test-time scaling: self-consisteney (Sc) from consistency-batd based methods and perplexity (PPL) from probability-bad -based methods, using our theoretical framework. Below, we adopt common assumption that L.M sampling follows Bernoulli distribution
[55 allowing us to compute the estimation error for specifie methods.
2.2.2 Analysis of Self- Ceff-Consistecyy
Self-consistency 660 is the representative method for consistency-basee methods, which samples n reasoning paths ",..., In and estimates the confidence of any á using Monte-Carlo estimation by
1 Σ 72
sc) 00
x)
lb = Ø1. Y = g(t))
(3)
Then, the reasoning error of Sc for a given problem (ar y) and any possible ý can be computed by
Eptaco (0) = EE
T 5) T = wl)
(4)
Finally, we conduct decomposition on Sc to illustrate the key factors affecting the reasoning error. Proposition 2 (Sc Reasoning Error Decomposition). For any input with ground-truth answer let p ((S)) (y 2) denote the estimated probability for any possible answer 9 by Sc. Then, the reasoning error giees (ÿ) can be divided into components:
Egeses (i) >>>1-000 r(u x) (1 - p(ū (p())))
x) 1 = ul)²
Estimation Error
Model Error
This proposition reveals that the estimation error of SC consists solely of variance since the sampling is unbiased. It decreases only linearly with increasing sample size, which results in substantial reasoning error when sampling is limited. This analysis suggests that a promising direction for improving Sc is to develop methods that achieve faster estimation error convergence rates.
Remark 2.
The detailed proof is provided in Appendix A.1
2.2.3 Analysis of Perplexity
Perplexity is a representative method for probability- methods that directly utilizes the internal LLM probability p(i) æ:) for any reasoning path î. However, since the number of possible reasoning paths nearly infinite, the ground-truth probability can only be accessed for those paths that are actually sampled. Therefore, the estimated probability of any possible reasoning path f is estimated using the unique set of sampled reasoning paths R Set (f1....,f.)) f.).
plta [x), if 8= ii
B(PLL) (11 x) -
otherwise.
(6)
tER
Similarly, the reasoning error of PPL. is denoted as follows, with the following proposition decompos- ing the reasoning error of PPL.
Egamas (1) = E
i p(( - = vi)²
EER
Proposition (PPL Reasoning Error Decomposition). For any given imput x with ground-truth answer y, let ((PEL) (i) x) denote the estimated probability of by PPL method for any possible reasoning path f. Then, the reasoning error Egtn, (f) can be divided into two components:
Egtins (0) = =(1 -(1-p( ())))(((()))( = v] - p((l2)) x)) (p(6) x)) I[( - vi)
(8)
Extimation Error
Model Error

---

L.M Sampling
Reasoning Prusing
Perplexity Consistency
Reasening
Sampling
CCasistescy
Figure I: Illustration of the RPC approach. The Reasoning Pruning filters out low-probability answers, while the Perplexity Consistency incorporates LLM probabilities into the self-cons framework, resulting in faster convergence of estimation error.
Remark 3. The detailed proof is provided in Appendix A.I Compared with Sc, the estimation error of PPL decreases exponentially, with the rate depending on the magnitude of ground-trume truth confidence p(i Ir). For reasoning paths that are likely to yield the correct answer (i.e., those with non-negligible confidence), PPI. achieves substantially faster convergence rate. However, for reasoning paths with extremely low confidence, the convergence advantage of PPL degrades. Furthermore, the Model Error of PPL is typically larger than that of Sc in practice, and we formally demonstrate this in the ideal case in Appendix 4 This analysis suggests that the degradation issue of Estimation Error and the large Model Error issue are fundamental challenges for improving PPL method.
3 Methodology
Based on our theoretical analysis. we identify three fundamental challenges in existing methods: (I) Sc suffers from slow Estimation Error convergence, leading to efficiency concerns; (2) PPL exhibits large Model Error, compromising its effectiveness; (3) PPL's 's Estimation Error convergence advantage deteriorates significantly for certain cases, resulting in a degradation issue.
To address these challenges, we propose Peasoning-Prruiing Perplexity Consistency (RPC). a novel method with two key components. First, we integrate internal L.M probabilities into the self- consistency framework to create Perplexity Consistency (PC), confidence estimation function that reduces estimation error efficiently while maintaining low model error, thus addressing the first two challenges with theoretical guarantees. Second, we introduce Reasoning Pruning (RP) module that resolves the third challenge by systematically filtering out reasoning paths with low probabilities.
3.1 Perplexity Consistency
To address the efficiency and effectiveness challenges simultaneously, we propose Pc, which directly leverages the LLM's prediction probability like PPL., obtaining the benefit of an exponential conver- gence rate, and also applies the consistency function of Sc to minimize the model error. Formally, for the unique set of sampled reasoning paths R = Sct (f..... En), the estimated probability of any possible answer ÿ is
(PCC) (0
x) = Σ Ilg(f) = (lp(tl=).
(9)
tER
which calculates the cumulative probability of all unique reasoning paths (+ i, E R. and g(t,) = 0}} whose answer ý. Therefore, the squared error of Pc for any possible answer ý
Egtres (i) EE (Z) (g) x)) - 1lū = =
(10)
Now, we present the following theorem, which explores the reasoning error decomposition of Pc. Theorem 4 (Pc Reasoning Error Decomposition). Assume that l = |{E g(t) = ú}l and define
= 1 -p( 1)). Then, the reasoning error E(p(PC) of PC can be divided into two components:
Egtres (0) = a" app( x) (2 = vl - (1 a" )p( xz) + (p() x) 10 = vl)²
Estimation Error
Model Error
5

---

Remark 5. The proof is presented in Appendix A.3] The theorem states that PC successfully fuses the strengths of PPL and Sc: achieves the same level of model error as Sc while ensuring the similar convergence rate as PPI. in the estimation error. Particularly, the convergence rate can be computed as "o""p" as x) (1 E p(ū [x), which is exponential.
Remark 6. The convergence rate is primarily influenced by the magnitude of p(ツ r). most scenarios, it remains exponential, facilitating rapid estimation enror reduction. However, when p( 5) + 0 and np(ý 2) < 1, we only have ax I+nplg z) 32]. resulting in the convergence
rate unexpectedly degenerating to linear result.
3.2 Reasoning Pruning
To address the third degradation challenge, which is also verified in Remark 6 we propose directly pruning these sampled answers, called Reasoning Pruning (RP). The main idea of Rp is that an answer with negligible p(ğ z) is not likely to be the correct answer, so we may prune it directly without significant increase in reasoning error. Specifically..Rr sets p(O æ) 0 when the cumulative probability of all its corresponding reasoning paths is lower than a threshold making the estimation error vanish. However, there are two questions raised for Rp: (1) Although pruning can boost the estimation error reduction, does it potentially increase the model error and thereby increase the overall reasoning error? (2) How should we determine the threshold T for pruning?
For the first question, we theoretically prove that RP can achieve the optimal error reduction for each given problem (Ir, 3) with the optimal threshold
high probability.
Theorem 7 (Effectiveness of Reasoning Path Pruning). Assume that the optimal threshold T = ply r), and let k = |4: g(l,) ÿ, 1,...,77 nn), which refers to the size of samples whose answer y. Hence, RP achieves the optimal error reduction with at least the probability
2kx² -²)))
c
Remark The proof is included in Appendix A..4 The theorem provides guarantee that Rp can achieve the optimal error reduction for each given problem (I, y) at high probability. Note that the optimal error reduction not only boosts the estimation error reduction efficiency but also effectively reduces the model error, thus improving the final reasoning capability of LLMs.
For the second question, we develop an automated strategy to determine the 7 based on the distribution of all sampled reasoning paths. Inspired by open-set recognition [5]. we model the probability distribution of Ω and Q2 as a mixture of two Weibull distributions, representing high and low probability regions. Elaborately, define the PDF of the mixture distribution as:
(x) = w1 fw ki, A1) + W2 A2),
where the Weibull PDF 59 is defined as fw(r;,, A) = { exp (-(#)²). We use the maximum likelihood estimation methods to estimate the parameters, i.e. (k1 A1), (k2, A2). wi. and wa the probability distribution of all sampled reasoning paths for each reasoning problem. We assume that Weibull(k), A1) is the high probability distribution and Weibull (k2, A2) is the other. Then, the probability of the reasoning path being in the high probability distribution is derived by
Wi. fw [w(2;; kn, A1)
Phügh (x) wifw(e (z: K1, A1) + (2; A2)
where x is the value of LLM internal probability. Then, we remove sampled reasoning paths f satisfying Prigh xx)) 0.5, which are more likely to be in the low probability distribution. Moreover, to ensure the algorithm's 's stability when n is limited, we employ the Truncated Mean method 43). retaining outputs with probability greater than the overall mean. This prevents the removal of too many reasoning paths due to the potential inaccurate estimation of the mixture
distribution.
3.3
RPC Method
Overall, we apply the Reasoning Pruning to all sampled reasoning paths I,...., for removing low probability reasoning paths and then compute the confidence based on Perplexity Consistency, forming our proposed Reasoning- prannng Perplexity Consistency (RPC) confidence estimation method. The pseudo-code is shown in Algorithm I in Appendix
B Figure illustrates its complete framework.

---

Table Efficiency comparison of Perplexity Consistency module (Pc) and RPC. The table shows the minimum number of samples needed to exceed the best performance of Sc, with reduction rates in bold when sampling reduced.
MATH
MathOdysse
OlympiadBench
ATME
Method
Accuracy
#Samplings
Accuracy
#Samplings
Accuracy
#Samplings
Accuracy
#Samplings
Best of Sc
50.57
64
28.32
112
11.07
128
9.40
128
Pc
50.63
32
28.51
112
11.07
128
9.00
64
A
+0.06
-50.0%
+0.19
-0.0%
0.00
-0.0%
0.00
-50.0%
RPC
51.16
32
29.31
32
11.07
64
9.50
48
A
+0.59
-50.0%
+0.99
-71.4%
0.00
-500%
+0.10
-62.5%
Experiments
In this section, we conduct experiments to answer the following research questions:
RQI: Efficiency. How does RPC reduce the number of samples required to achieve comparable performance through faster convergence?
RQ2:
Efficacy. How does RPC improve reasoning performance compared to existing methods?
RQ3: Reliability, How does RPC enhance the reliability of confidence estimation compared to existing methods?
Morcover, further discussions are devoted to further demonstrating the effectiveness of RPC.
4.1 Experimental Setting
In this section. we briefly introduce the comparison methods, datasets, and implementation details. Due to space limitations, detailed experimental settings are included in Appendix C
Comparison Methods. We compare three types L.LM confidences: perplexity confidence [57
(PPL., self-consistency confidence [10] (Sc), and verbalized confidence (52) (VERB). The verbalized
confidence is computed based on the probability that the LLM outputs "True" versus "False" when asked an "Is-True" question. For code generation tasks, extracted verbalized confidence scores from the model's numerical likelihood expressions by prompting the LLM.
Datasets. We introduce four popular benchmarks for math reasoning: MATH (25), MathOdyssey [16],
OlympiadBench [23], and AIME [67 (contains problems from 1983 to 2024). As to code gener- ation tasks, evaluate each method three benchmarks, i.e., HumanEval [8], MBPP B3, and introductory-Level problems APPS [24].
Implementation Details. For math reasoning tasks, we evaluate the with 1.8B and parameters (65, as well as the DeepSeekMath-KU 7B model 49] The consistency function Ig is the answer comparison. For code generation tasks, we evaluate the Deepseek-Coder 33B model. The consistency function lc is constructed based on semantic equivalence [42] by clustering code based on given test cases. We set the sample size to 128 for the MathOdyssey, OlympiadBench, and AIME datasets and 64 for the MATH dataset by default. Each experiment repeated 10 times with different random seeds, and the average performance is reported. All experiments were conducted on Linux servers with A800 and H800 GPUs
models
4.2 Empirical Results
RQ1: Efficiency. How does RPC reduce the number of samples required to achieve comparable performance through faster convergence?
evaluate our proposed RPC against the standard self-consistency method using four mathematical benchmark datasets with the Interni M-2-MATTH-Plus 7B model. For the MATH dataset, we set the reasoning path size 64, while we set the number of reasoning paths to 128 for the other datasets with Sc. We then record the best performance and minimum sampling requirements for Sc. For both

---

Accuracy (%)
Accuracy (%%
Accuracy
5C
#Sampling
#Sampling
#Sampling
#Sampling
(a) MATH
(b) MathOdyssey
(c) OlympiadBench
(d) AIME
Figure 2: The accuracy of the with different sample sizes n. Our proposed RPC achieves the best performance across all datasets, validating our theoretical analysis. We also show the performance of single perplexity consistency module (denoted as Pc), which further supports our theoretical findings.
7B model four math reasoning datasets
Table Performance Comparison using BeeenL.M---MATII
7B model measured by accuracy and expected calibration error metrics. The best performance is highlighted in bold. The results show that our RPC outperforms existing methods in majority of cases.
Method
MATH
MathOdyssey
OlympiadRench
AIME
Average
Accurscy())
FCR()
Accuracy())
FCR())
Accuncy(")
ECF())
Accumcy(t)
BCF())
Aoc:(1)
ECR())
0.61
0.88
Sc:
50.37 0.11
6.71 0.18
28.25
0.60
12.23 0.54
11.07
0.15
20.20
0.16
9.40+0..1
14.35 + 0.23
24.82
13.37
Ric
$1.95 0.15
641
0.18
0.18
3142 = 0.75
9.7 = 0.73
11.14
0.15
1886 0.18
9.74 0.23
14.32 0.21
26.11
12.37
RPC and our Perplexity Consistency module (denoted PC), we report the minimum number of samples needed to match or exceed the performance of the Sc in Table
The results of Pc indicate improved convergence rates compared Sc in several cases, while maintaining similar rates in others. These findings support the rapid convergence and degeneration issues of PC in Theorem samples to achieve comparable performance relative to Sc. Moreover, the degeneration issues observed in PC are effectively addressed in RPC, validating both the effectiveness of the Reasoning Pruning module and our Theorem 7
RPC shows significant efficiency improvements by requiring fewer
RQ2: Efficacy. How does RPC improve reasoning performance compared existing methods?
We evaluate the performance of Pc and RPC in Figure across various sample budgets. The results demonstrate that RPC achieves better performance than both PPL (which relies on internal LLM probabilities) and Sc (which employs Monte Carlo sampling). The detailed accuracy results, including mean and standard deviation in Table 2 support these findings.
We also analyze the performance of PC separately. The results indicate that Pc has a faster con- vergence rate than Sc, which aligns with Theorem[4] The significant performance gains from Pc RPC, as shown in Figure 2a and Figure 2b] validate the effectiveness of the Reasoning Pruning module. This suggests that Reasoning Pruning helps reduce model errors when the LLM exhibits good alignment by eliminating incorrect reasoning paths that carry low L.MM probability scores.
RQ3: Reliability, How does RPC enhance the reliability of confidence estimation compared to existing methods?
To evaluate the reliability of confidence estimation, we analyze the ECE results RPC and compari- methods in Table ECE measures the difference between predicted probabilities and empirical accuracy, as directly computing estimation error using ground-truth probabilities is virtually im- practical. The results demonstrate that RpC approach achieves lower ECE values and higher accuracy compared to PPL and Sc, indicating more reliable confidence estimation. We visualize this improvement through reliability diagrams comparing Sc and RPC in Figure 3 on MathOdyssey, which clearly shows the reduced calibration error of RPC.
4.3 Further Discussion
Performance on Code Generation Tasks. To investigate whether our proposed approaches can generalize to other reasoning tasks, such code generation tasks, evaluate RPC and comparison

---

Accuracy (No)
Verb
RPC
10
StammnBooll
(a) Sc
(b) RPC
Figure 4: Performance on three code generation tasks using Deepseek- Deepseek-Coder Coder 33B model. The ex- perimental results show that our RPC achieves the best performance.
Figure 3: The reliability diagrams of Sc and RPC MathOdyssey dataset using IntemL.---MATTH-PIur 7B model.
methods three code generation benchmarks, as illustrated in Figure 4 The results show that RPC achieves the highest accuracy across all datasets, demonstrating its effectiveness beyond mathematics.
Performance Additional Reasoning Tasks. To further validate the effectiveness of RPC, conducted experiments on the GPQA [47] and LogiQA [39 benchmarks using the DeepSeek-RI- Distill-Qwent model [2 with 16 samples. The results in Appendix DD.5 are consistent with those from both math reasoning and code generation tasks, where RPC outperforms existing methods.
Performance across Model Scales and Architectures. To evaluate the generalization ability of our approaches across different model scales and architectures, we conducted additional experiments using MnternLM2MMM M2-MMath-llus 1.8B and DeepSeek-Math 7B models. The results in Table 3 consistent with results Table 2 We additionally report the detailed performance of InternL.M2- Plus 1.8B using diverse sampling budgets in Appendix D.2 The experimental results demonstrate the effectiveness of RPC across different model scales and architectures.
Performance using Advanced Methods and Models. To further verify the effectiveness of RPC across diverse scenarios, we conduct additional experiments using the DeepSeek- -1 RI-Distill-Qwen-11 model [21] which is an advanced model with thinking capability. The results in Appendix D.4] demonstrate that the performance gains of RPC persist when combined with advanced models that have strong reasoning performance, highlighting its potential for enhancing LLM reasoning capability, Moreover, we also combine RPC with two advanced methods: ESC [36] and BoN using the reward model (69, which are advanced versions of Sc and PPL, respectively. The results in Appendix [..] show that RPC applied to both methods consistently outperforms the Sc and their original versions, demonstrating its strong compatibility for integration with advanced methods.
Performance using High Sampling Temperatures. As discussed above, sampling more diverse reasoning paths is important for improving reasoning performance. Therefore, we additionally conduct experiments under high sampling temperatures. As shown in Appendix [..]] RPC can further improve reasoning performance under high sampling temperatures by benefiting from the diverse reasoning paths, while Sc may deteriorate due to increased estimation errors at high temperatures.
Discussion about Computational Overhead. We discuss the computational overhead of RPC in Appendix D.7 Theoretically, RPC introduces only minimal computational overhead compared to Sc. In practice, the additional computational overhead of RPC is negligible compared to the primary computational bottleneck (i.e.. LLM inference time) in reasoning tasks. RPC actually provides an excellent computational trade-off, where the minimal computational overhead is exchanged for significant time savings achieved by reducing the number of required L.M inferences.
Discussion about Slyper-parameters We provide a detailed discussion of the hyperparameters in Appendix D.6 Overall, RPC introduces the Rp module automatically prune reasoning paths with low probability without requiring manual threshold setting. This makes RPC hyperparameter--fre method that robust across diverse reasoning tasks.
Related Work
This paper is related the topics, i.e., LLM Reasoning Boosting and LLM Confidence Estimation.
LLM Reasoning Boosting. Recent research (2 has developed various methods to enhance L.M reasoning. CoT [31] proposes the "Let's think step by step" prompt to guide LLMs in generating

---

Table Performance Comparison of different models and different parameter scales. The accuracy is reported as the mean and stdev. The best performance is highlighted in bold. The results show that our RPC outperforms existing methods major cases.
Method
InternI. Ma--Lll2---Ma--PPP Plas 18B
DeepSeckMath-Ke RL. 7B
MATH
OlympiadBench MathOdysse
AIME
MATH
MathOdyssey
OlympiadBench
AIME
PPL.
33.24 1: 0.24
16.56 0.88
3.08 1: 0.20
1,66 1: 0.15
42.51 1: 0.23
22.34
1,00
5,90 0.31
3.37 1: 0.46
VERB
7.21
2.81 0.26
0.06
0.05
2.55
0.24
2.36
0.12
Sc
36.48 1 0.15
14,52 1: 0.46
5.99 1: 0.17
2.66 1: 0.20
53.33
1: 0.09
36.68 0.65
11.29 4 0.17
9.42 1: 0.23
37,88
16.35
6.52 0.24
0.24
53.37
37.25 0.69
11:30
0.31
structured solutions. For complex problems, Least-to-Most 722 introduces decomposition strategy that breaks down challenges into manageable sub-problems. Few-shot methods [58] 70 leverage carefully selected examples to improve reasoning performance. To enable more comprehensive rea- soning, search- based methods [19] integrate Monte Carlo Tree Search (MCTS). Recent advancements have further enhanced MCTS by incorporating reward models [68,
46
To directly optimize rea- using specialized
soning abilities, researchers have explored fine-tuning approaches (66
34
38
65]
datasets and reinforcement learning techniques [4 41. While these methods focus on generating
accurate reasoning paths, our RPC can build upon them by utilizing multiple sampling strategies, enabling better reasoning performance. Recent studies recognize its computational issues and propose early stopping 37 and dynamic sampling [56] 54] 2 to improve efficiency. While previous studies primarily focus developing specific methods that demonstrate empirical effectiveness, our paper introduces a theoretical framework for analyzing them and provides theoretical insights.
LLM Confidence Estimation. The confidence estimation for LLM can be categorized into three types: (1) perplexity confidence, (2) verbalized confidence, and (3) self-consistency confidence.
Perplexity confidence [28 15 utilizes the geometric mean of LLM prediction probabilities (i.e.,
perplexity [10 6 to evaluate model adherence [44] and prompt quality [64]. Verbalized confi-
dence 30 60 52 directly asks LLMs to express their confidence through various approaches, such as multi-agent deliberation [6]. multi-step evaluation [60], top-k ranking [52], few-shot prompting 40), and reflection 14 [7I]. Self-consistency confidence [57
26 50 measures the agreement among multiple generated answers to improve reasoning performance, with recent work [60] 1 further developing this approach confidence metrie. Recent studies have identified that LLM con- fidence is crucial for enhancing LLM reasoning, such as DeepConf [18]. CISC [5T] and TTSC [27], which align with core idea. Our work provides theoretical explanations support their findings.
6 Conclusion
In this paper, we present a theoretical framework for LLM reasoning problem in sampling-base test-time scaling, providing insights and principled guidance for designing better LLM reasoning methods. This framework decomposes I.LM reasoning error into estimation error and model error, revealing the efficiency issues of the self-consistency method and effectiveness and degeneration issues of the perplexity method. To this end, we introduce (RPC), which integrates internal LLM probabilities into the self-consistency framework for faster estimation error convergence while maintaining low model error and prunes low-probability reasoning paths to address the degeneration issue. Both theoretical analysis and empirical results demonstrate that RPC achieves superior eтror convergence, reasoning performance, and confidence reliability.
paasooninn-pruminn Perplexity Consistency
L Limitations and Future Work. Despite our theoretical analysis providing insights and guidance for designing better L.M reasoning methods, this paper presents only a preliminary step toward building advanced reasoning approaches. We believe theoretical framework can guide future research in this promising direction. Detailed limitations and future works are discussed in Appendix E
Broader Impacts
This paper presents a new theoretical framework for analyzing sampling-basse based test-time scaling methods LLM reasoning, enabling researchers to identify the limitations of existing methods and to provide insights for developing future approaches. Therefore, we believe that this work has the potential to benefit the broader L.M reasoning community by offering a theoretical foundation. As this paper is theoretical study, we do not identify negative societal impacts that need to be discussed.
10

---

Acknowledgments and Disclosure of Funding
We appreciate the anonymous reviewers for their valuable insights and helpful comments. This work is supported by the National Natural Science Foundation of China (Grant No. 624B2068 and 62576162), the srading-edge Technology Program of Jiangsu Science Foundation (BK20232003), the Key Program of Jiangsu Science Foundation (BK20243012), and the Fundamental Research Funds for the Central Universities (0221 14380023).
References
[1] Yasin Abbasi- Ahbasi-Yadkori, Iija Kuzborskij, András György, and Csaba Szepesvári. To helieve or not to helieve your LLM. CoRR, 2024.
[2] Aman Madaan Pranjal Aggarwal, Yiming Yang, and Mausam. Let's sample step by step: Adaptive- consistency for efficient reasoning and coding with llms. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12375 12396, 2023.
[3] Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry. Quoe V. Le, and Charles Sutton. Program synthesis with large language models. CoRR, abs/2108.07733 2021.
[4] Evan Becker and Stefano Soatto. Cycles of thought: Mcasuring LLM confidence through stable explana- tions. CoRR, abs/2406.034m 03441, 2024.
[5] Abhijit Bendale and Terrance F. Boult, Towards open set deep networks. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pages 1563-1572, 2016.
[6] David M. Blei, Andrew Y. Ng. and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3(1):993 (122,299-1022,, 2003.
[7] Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests. In Proceedings of the 11th International Conference on Learning Reprexentations, 2023.
[8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray. Raul Puri, Gretchen Krueger, Michael Petrow, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizaheth Barnes, Ariel Herhert- Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak. Tang, Igor Babuschkin, Suchir Balaji. Shantanu Jain, William Saunders. Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodci, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, abs/2107 03374, 2021.
[9] Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models, 2025.
[10] Stanley F Chen, Douglas Beeferman, and Roni Rosenfeld. Evaluation metrics for language models. 1998.
[11] Xinyun Chen, Renat. Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint 2311123111173111 2023.
[12] Furui Cheng, Vilém Zouhar, Simran Arora, Mrinmaya Sachan, Hendrik Strobelt, and Mennatallah El- Assady. Relic: Investigating large language model responses using self self-consistency. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1-18. 2024.
[13] Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua. Plug-and-play policy planner for large language model powered dialogue agents. Proceedings of the 12th International Conference on Learning Representations, 2024.
[14] Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu. Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. Chain of verificatto reduces hallucination in large language models. In Findings of the Association for Computational Linguistics, pages 3563-3578 2024.

---

[15] Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalay, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention relevance: Towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 5050-5063, 2024.
[16] Meng Fang, Xiangneng Wan, Fei Lu, Fei Xing, and Kai Zou. Mathodyssey: Benchmarking mathematical wroblem-solving skills in large language models using odyssey math data. CoRR, abs/2406.18322 2024.
[17] Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity-base based prompting for multi-step reasoning. In Proceedings of the 11th International Conference on Learning Representations, 2023.
[18] Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawci Zhao. Deep think with confidence. CoRR, abs/2508.1522 15260, 2025.
[19] Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. istar-math, Small Iims can master math reasoning with self- deep thinking. 2025.
[20] Lin Gui, Cristina Garbacea, and Victor Veitch. Bonbon alignment for large language models and the sweetness of best-of-n sampling. In Advances in Neural Information Processing Systems, 2024.
[21] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang. Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, 7. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu
Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong
Dai, Fuli Luo. Guangbo Hao. Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Honghui Ding, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo. Jiashi Li. Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Juniong Li, L. Cai, Jiaqi Ni, Jian Liang. Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao. Kang Guan, Kexin Huang. Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng. Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen, Xiaotao Nie, Xin Cheng. Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang. Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyue Jin, Xiaojin Shen. Xiaosha Chen, Xiaowen Sun. Xiaoxiang Wang. Xinnan Song. Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang. Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yonggiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan. Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie. Ziyang Song. Zizheng Pan, Zhen Huang. Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseck-rl incentivizes reasoning llms through reinforcement learning. Nature, (45(8081):633 638, 2025.
[22] Trevor Hastie, Jerome H. Friedman, and Robert Tibshirani. The Elements Statistical Learning: Data Mining, Inference, and Prediction. Springer Series Statistics. Springer. 2001.
[23] Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi. Zhiyuan Liu, and Maosong Sun. Olympiadhench: challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific prohlems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 3828-3850. 2024.
[24] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazcika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with APPS. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmorks, 2021.
[25] Dan Hendrycks, Collin Burs,, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang. Dawn Song, and Jacob Steinhardt. Mcasuring mathematical problem solving with the math datasct. In Advances in Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
[26] Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via celf-calibration. CoRR, abs/2503.0003/ 2025.
12

---

[27] Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration. CoRR, abs/2503.00031, 2025.
[28] Yuheng Huang, Jiayang Song, Zhijie Wang. Shengming Zhao, Huamingg Chen, Felix Juefei-Xu, and Lei Ma. I Look before you leap: An exploratory study of uncertainty mcasurement for large language models. arXiv preprint arXiv:2307. 10236, 2023.
[29] Yuu Jinnai, Tetsuro Morimura, Kaito Ariu, and Kenshi Abe. Regularized best-of-n sampling to mitigate reward hacking for language model alignment. CoRR, 0bs/2404.010522 2024.
[30] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer EI Showk, Andy Jones, Nelson Elhage. Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hermandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know. CoRR, abs/2207.0522/ 2022.
[31] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Advances in Neural Information Procexsing Systems, pages 22199- 22213, 2022.
[32]
László Kozma. Useful incqualities, 2021.
[33] Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski. Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag. Theo Gutman-Solo Solo. Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems, pages 3843-3851 2022.
[34] Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, and Chang Zhou. MuggleMath: Assessing the impact of query and response augmentation math reasoning. In Proceedings the 62nd Annual Meeting of the Association for Computational Linguistics. pages 10230-10258 2024.
[35] Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, and Brian Ichter. Chain of code: Reasoning with language model-augmented code emulator. In Proceedings of the 41 International Conference on Machine Learning, 2024.
[36] Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. Escape sky-high cost: Early-stopping celf-coonisteney for multi-step reasoning. Proceedings of the 12th International Conference on Learning Representations, 2024.
[37] Yiwei Li, Peiwen Yuan, Shaoxiong Feng. Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. Proceddings of the 12th International Conference on Learning Representations, 2024.
[38] Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang. Xian Zhang, and Xiaoxing Ma. Neura- symbolic data generation for math reasoning. Advances in Neural Information Processing Systems, pages 23488-2333 23515, 2024.
[39] Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, pages 3622-3628, 2020.
[40] Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Calibrating 11m based evaluator. In Proceedings of the 2024 Joint International Conference on Computational Linguisties, Language Resources and Evaluation, pages 2638-2656, 2024.
[41] Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang, Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. In Proceddings of the 13th International Conference on Learning Representations, 2025.
[42] Viktor Malik and Tomáš Vojnar Automatically checking semantic equivalence hetween versions of large-scale projects. In Proceedings of 14th IEFE Conference on Software Testing, Verification and Validation, pages 329-339, 2021.
[43] A Marazzi and C Ruffieux. The truncated mean of an asymmetric distribution. Computational Statistics & Data Analysis, 32(1):79-100 1999.
13

---

[44] Bhuvanashree Murugadoss. Christian Poelitz. Ian Drosos, Vu Le. Nick McKenna, Carina Suzana Negreanu, Chris Parnin, and Advait Sarkar. Evaluating the evaluator: Measuring LLMs' adherence task evaluation instructions. Proceedings of the 39th AAAI Conference on Artificial Intelligence, 2025.
[45] Siqi Ouyang and Lei Li. Autoplan: Automatic planning of interactive decision decision-making tasks with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3114-3128, 2023.
[46] Sungjin Park, Xiao Liu, Yeyun Gong, and Edward Choi. Ensembling large language models with process reward-guided tree search for better complex reasoning. CoRR, abs/2412.155977 2024.
[47] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In Proceedings of the 1st Conference on Language Modeling, 2024.
[48] Elena Shlendorio, Vincenzo Dentamaro, Alessio Lo Cascio, Francesco Germini, Michela Piredda, and Giancarlo Cicolini. Integrating human expertise & automated methods for a dynamic and multi paramer evaluation of large language models' feasibility in clinical decision-makee International Journal of Medical Informatics. 188:105501, 2024.
[49] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, Y. Wu, and Daya Guo. Deepscekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024.
[50] Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in llms. CoRR, abs/2502.06233, 2025.
[51] Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves consisensy in llms. In Findings of the Association for Computational Linguistics, pages 20090-201 11, 2025.
[52] Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 133--442. Association for Computational Linguistics, 2023.
[53] Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subharao Kambhampati. On the planning abilitics of large language models- A critical investigation. Advances in Neural Information Processing Systems, pages 75993-76005, 2023.
[54] Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Dynamic self-consistency: Leveraging reasoning paths for efficient LLM sampling. CoRR, abs/2408. 17017, 2024.
[55] Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, and Ben Athiwaratkun. Reasoning in token economies: Budget-awart evaluation of LLM reasoning strategies. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19916-19999 19939, 2024.
[56] Xinglin Wang, Shaoxiong Feng, Yiwei Li, Peiwen Yuan, Yueqi Zhang, Boyuan Pan, Heda Wang, Yao Hu, and Kan Li. Make every penny count: Difficulty-adaptivv self-consistency for cost-efficient rcasoning. CoRR, abs/2408. 1 13457, 2024.
[57] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In Proceedings of the I 11th International Conference on Learning Representations, 2022.
[58] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Fd Chi, Quoe Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, pages 24824-24833 2022.
[59] Waloddi Weibull. A statistical distribution function of wide applicability. Journal of applied mechanics, 1951.
[60] Miao Xiong. Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Ilms express their uncertainty? an empirical evaluation of confidence elicitation in lmss. In Proceedings of the 12th International Conference on Learning Representations, 2024.
14

---

[61] Ruixin Yang, Dheeraj Rajagopal, Shirley Anugrah Hayati, Bin Hu, and Dongyeop Kang. Confidence calibration and rationalization for LLMs via multi-agent deliberation. In International Conference on Learning Reprexentations Workshop on Reliable and Responsible Foundation Models, 2024.
[62] Xiao-Wer Yang, Jie-Jing Shao, Lan-Zhe Guo, Ben Zhang, Zhi Zhou, Lin-Han Han Jia, Wang -hou Dai, and Yu-Feng Li. Neuro-symbolic artificial intelligence: Towards improving the reasoning abilities of large language models. In Proceedings of the 34th International Joint Conference on Artificial Intelligence. pages 10770-,0778, 2025.
[63] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, pages 11809 11822, 2023.
[64] Yuxuan YAO, Han Wu, Zhijiang Guo, Zhou Biyan, Jiahui Gao, Sichun Luo, Hanxu Hou, Xiaojin Fu, and Lingi Song. Learning from correctness without prompting makes LLM efficient reasoner. In Proceedings of the Ist Conference on Language Modeling, 2024.
[65] Huaiyuan Ying. Shuo Zhang. Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, Yudong Wang, Zijian Wu, Shuaibin Li, Fengzhe Zhou, Hongwei Liu, Songyang Zhang, Wenwei Zhang, Hang Yan, Xipeng Qiu, Jiayu Wang, Kai Chen, and Dahua Lin. Internim- Open math large language models toward verifiable reasoning, 2024.
math:
[66] Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In Proceddings of the 12th International Conference on I Learning Representations, 2024.
[67] Parvez Zamil and Gollam Rabby, Aime problems 1983 to 2024, 2024.
[68] Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. REST-MCTS*: L.L self-training via process reward guided tree search. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems, 2024.
[69] Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. In Findings of the Association for Computational Linguistics, pages 10495-105166 16, 2025.
[70] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In Proceedings of the 11th International Conference Learning Representations, 2023.
[71] Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, and Jianshu Chen. Fact-and-reflection (far) improves confidence calibration of large language models. In Findings of the Association for Computational Linguistics, pages 8702-8718, 2024.
[72] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, and Ed H. Chi. Least-to-most prompting enables complex reasoning in large language models. In Proceedings of the I 1thh International Conference on Learning Representations, 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have claimed that our contributions explicitly the end of introduction
section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have discussed the limitations at the end of conclusion section as well as in the Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: We have provided the full set of assumptions in our theoretical results and a complete proof in the Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All the necessary information for reproducibility is provided in the subsec tion 4. and Appendix C Moreover, we provide all necessary information in our project homepage https:///mjjx: giths..ittuuubu.. io/RPC
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
17

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification:
Yes, the code are provided our supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://mips.co
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7 //nips. hwcp//blccggudeeeeeeeeeeeeeeeeee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All the necessary information provided in Section 4.1 and Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We reported the standard deviation of the experiments in the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided the details of the compute resources Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The authors have reviewed the NeurIPS Code of Ethics and the paper conforms to the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the potential positive and negative societal impacts of our work the Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We have not released any models or datasets that have a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly credited the creators of the assets in our reference.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our algorithm is confidence estimation method based on the sampled LLM reasoning paths.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standarr components.
Please refer to our LLM policy (https://nee) /ttps//////upp.ppeeee. ///Cooffreences222222 for what should or should be described.
22

---

A Theoretical Results
A.1 Proof of Proposition T Proposition
2 and Proposition 3
Proof. In this proof, all the expectations are taken over the combinations of rt-sampled reasoning paths , fn and omit the details of expectation for simplicity. We first prove the general error decomposition in Proposition[] Then, we give the proofs of Proposition 2 and Proposition 3 We adopt common and practical assumption that the LLM sampling be modeled sampling from a Bernoulli distribution [55], which allows us to further compute the estimation error for specific methods.
Proof of General Error Decomposition For any input with ground-truth answer y and any possible answer i, let p(ú a)) denote the unbiased estimated confidence of j and p(ū <x) denote the ground truth confidence. We have
Et)
(p(0 xx) 11 =y1)² =/)²|
(((0) z) -P( 2) p(ù 1x) Ii 3/)
(p(ū .:) - p(ù x)) (p(0 x)) Tly w))
(11)
Estimation Error
Model Error
2 (6() x) p( x)) (p(0lz) 1p m/)
Cross Term
Then, we prove the Cross Term is zero for the unbiased confidence function as follows:
Cross Term (p / z) I[ý y)) E [5 2) p(O =)]
(p( x) -[ý = vl)
(12)
Therefore, we have the general error decomposition:
Eg(D) = (5(0 x) - pl x)) + (p(01x) - 1I0 = w)) 2
(13)
Estimation Error
Model Error
Proof of Error Decomposition for Sc For any input & with ground- truth answer y: let p(Sc) (ú) a) denote the estimated probability of ý by Sc method for any possible answer ÿ. The confidence function of Sc is ((5c)(j) x) = :, E M[ = Øil, where ÿ1 = g(f1),... Un g(fx) and
7,,.... Fr are sampled reasoning paths from the L.M distribution. Apply the error decomposition,
we have
Egescs (i) = E (015
(012) p(0 x) + (r(0 x) - Il = v1)²
(14)
1 n
p(ū (1 x()l=)) (pl .x) Tū =
Proof of Error Decomposition for PPI. Another way is to use the confidence function to build the sampling probability distribution using the unique set of n sampled reasoning paths R. Set (Z,,...,t%) i.i.,,,,)..... i.c.,
p(l [x), if there is ī, -i= In(=Dpp((1)) otherwise
((r (0|x)) = 0,
(15)
FRR
To simplify the analysis, we assume that each e....., E, is differrent from each other, which is reasonable assumption in practical LLM reasoning and slightly affects the value of if this assumption is not satisfied. Then, we have
(PP)) p((] - p(f p(f
(16)
(p)) (11=)) [({(==)]]] = =(1 - -11pp( p((1)))) (((1=)))<((11>)"
23

---

Hence,
Egcons (f) B[(b
(f] (f|=) -
= B((D!PE)) (f - + =
--(1 p((l=)))'((( +2(1-p(t
-11-p(1 =((
+ (p())2) ([g(() x))
(1 - =11-p(f) (2) v) 00--(( p(0 =)) + (p(6)=) -))(() =
(17)
Hence, we complete the proof.
A.2 Model Error Comparison in Ideal Case
Proposition (Comparison of Model Errors). Consider setting with infinite sampling of reasoning paths (n + oo) where each incorrect reasoning path leads to unique answer, that is, g(t:) & g(t;)
for any + j where g(ti) # y and g(tz) + y. The model ervor of self-consistency ( and perplexity (Maude) satisfy:
(SC)) S Model
(18)
The inequality is strict when the consistency function identifies equivalent correct reasoning paths more than once.
Remark 9. Although the assumptions in Proposition are idealized, this special case demonstrates that the consistency function in self-consisenncy achieves better model error than the perplexity method. In practice. the perplexity method always gives larger model error compared to the self-consistency method, as it does not leverage the consistency function to analyze the structural properties of specific reasoning problems. The proof is presented follows.
Proof. We first recall the definitions of the model error for self-consiseency and perplexity:
e(Sc)
( (g(Sc) (0 (x) - II0 - y))
Model
=
(19)
(PPL)
EiEr (D(PM) (f|zz) ()) [[g = v))
Model
=
where R = ) is the set of reasoning paths sampled from the LLM. We can compute the difference between the model error of Sc and PPL follows:
c(Sc) Model
(PrL) Model
Σ
(pise) (g x)) - Ily = vi) -
Σ
(f x)) - I[g(f) = v1) 2
DE (s(t.) [==1...))
ERR
Σ
(2(sc)()lx) 1l = v1) l[g(t) = 51 (BIPPL) (8) xx) - 110 = vi)
(E(a()) (i
IER
(A)
(20)
Assuming infinite samplings, the unbiasedness of SC gives us:
(80)) 1 Σ [[g(E.) = - Σ nt
I[g(() 0] (f x))
(21)
ERR
For any J. consider two cases:
24

---

(i) If 9 = y, then is the correct answer. We have
(A) = (piso) (0 x)) - -1) Σ
(a((t) Ø1
iRR
= (p(50) (5/ (1|xx) + 25(50) (i/ [21) Σ [la(t) = =0] (p(t0)((1(2)) + -20 (é|z))
iER
-0(0) (0lx)2 +1 [()( = 5] p(mm) (f|2r)2 - Σ ((((() 5]
EER
p(00) (0| 1z)
I (( (6) xx) +1 Σ DiER l(g(t) = 01
Σ I[a(t) ú)
iER
(22)
Let (())) [0|z)² B² and Eier I[g(t) = ]] = C, then
B²
(A) - B² +
C C
BBC + C- -B² c21
= C
(23)
1
1 [CC (1)(1-0)
s 0.
Equality holds if ΣieR Tlo(O) ú) = 1. This indicates that (A) < if the consistency function is effective least once, making ΣiER I[o(O) = ÿ] -C>1. 1.
(ii) If ÿ A y, then ÿ is incorrect. Assuming distinct answers for wrong reasoning paths, we have Σ iE I[g(f)= ū] 1, thus
(2180)(0) (01z) - -IlØ = u1)² - - Ello(t) = 5l (E(PA)(E)=) -11 = u1)²
(A) =
ieR
(24)
(p(sc) (0) (+) - -I[- - wi) (pisc) (0 [z) - I1l = wl) 22 = 0,
since only one E R satisfying that g(() equals the incorrect answer ÿ.
Therefore, c.PPL)) than equal the model error of perplexity under assumptions. Moreover, if the consistency function is effective at least once, the model error of self-connistency is strictly less than the model error of perplexity.
c(Sc)
Model I O, indicating that the model error of self-consistency is always less
A.3 Proof of Theorem
Proof. In this proof, all the expectations are taken over the combinations of n-sampled reasoning paths i, L for estimating the confidence (i(Cc and we omit the details of expectation for simplicity. To simplify the analysis, we assume that each In is different from each other, which is reasonable assumption in practical LLM reasoning and slightly affects the value of n this assumption is not satisfied. For any given answer g. the estimated probability of Pc is p x)) Σn 1[g(1:) o)p(t, 22), allowing the reasoning error of Pc can be computed as follows.
Egroo (0) = E PC (0 xx) I/ =
(25)
(D (0 xx) P(O | 2) + p(O x) Ip y))
We define l :=: [(E) gf) ÿ}}, which means that how many reasoning paths whose answers are can be covered given limited sampling budget. Note that we further have [[([g(() ppp(E x)]
25

---

p((0 thus
Elp(Pe)
x)] = E > [[g(ft) = s1p(ts x)]
im
1
(26)
(1 و | (z)) p(0)
a = ")(ū x.),
where a a= = tt( r.). Again, we have
Bpito (0 x²) x()1=))] = - -11 - aa'pppp'' "(ū x²)
(27)
(( (0 (0l=)) p(ù [x))) (1 (1- (1 a)) )) (1 - [1=)))(()))))
p()12))
Hence,
Egeney (0) (p(( (9) - 11i = y1)²]
EE (p (9 x) - p(0 x)) + p .) - - u1 = x/)a
- (1-aa)"piii x()
(1 (1 ())))(( x)) + (0(ū) x) [í -v))²
(28)
which completes the proof.
A.4 Proof of Theroem 7
Proof. Let us set the pruning threshold by == plylp(yll 1z) Then, we have
Egenies (0) = ap(ă 1 (21lg y) (1+ (1+ =
Eximation Error
(29)
(p() x) 1i: = v)) (((p(t) x)) < T]
Model Error
However, we only have an estimation of p(ğ [x), i.e., ((PPC) x²) kE [[(((t ()p(( s Et= p(fx Ir), where i,....,i, & are h sampled reasoning paths whose answer is ÿ. Therefore, the reasoning error of our approximate version can be computed by
EgtRre (0) (p)) vl (1 + a x))
Σ
p(t xx)
-TT m
Estimation Errur
(30)
+ (p(ū) xx) [| w)> Σ p(. _7]
Model Erroe
Hence, we only need to consider the probability + 10tpp(t x) ity, we can obtain that
LT. Using Hoeffding's 's inequal-
1 p(11)) 2)) 7) s exp (-
2k p(ù xx)
P
p(t, (x)
sexp
(31)
We set = tt1l 1))
then
P
1
Σ
p(t 12) 2 5 exp
(-2kh2(1 (-
)²)
(32)
a
Hence, we complete the proof.
26

---

B Pseudo Code of RPC Method
In this section, we provide the pseudo code of RPC. The output of Algorithm | paths with the highest confidence. The extraction function g(-) can be used to parse the reasoning paths to answers.
is set of reasoning
Algorithm 1 Reasoning-pruning Perplexity Consistency
Input:
1: Sampled Reasoning paths ,
2: LLM Internal Probabilities P1, :.....:PP Pn
3: Consisitency function Ic(::-)
Output: Most-confident reasoning paths with probabilities
4: [Phase] 1: Reasoning Pruning)
5: (ki, A1, 122 A2, W1, u12) Model probability distribution parameters from " Pn (Using
Theorem 3.2]
6: Pincmn <- 22:m1 Ps
7: Iretain &- (i (p:) > 0.5 2 Pmean J [PHigh is defined in Theorem 3.2
8: [hhaee Perplexity Consistency]
9: U+ Set(I
10: Aunique + {i ti E and Letain
11: for each reasoning path / E U do
C(f) EieTman [c flp,
13: end for
14: Cmax f max C(I)
15: return {(E, C()) f E U, C(i) = Cmax
C
Detailed Experimental Settings
C.1 Datasets
For mathematical reasoning tasks, we evaluate our proposed methods and comparison methods on four mathematical datasets that include MATH, MathOdyssey, OlympiadBench, and AIME datasets.
MATH dataset (25 is a dataset comprised of challenging competition math problems and we use its 5,000 testing data for evaluation.
MathOdyssey dataset [16] contains 387 problems, covering advanced high-school level, university-level, and Olympiad-level mathematics.
OlympiadBench dataset 23 contains 8.476 Olympiad level mathematics and physics prob- lems. We select the English problems without images, resulting testing dataset of 1,284 problems.
AIME dataset 67 contains 993 test problems collected from the American Invitational Mathematics Examination, spanning from 1983 to 2024.
For code generation tasks, we conduct experiments on three common benchmark datasets. Hu- manEval [ contains 164 hand-written Python programming problems. MBPP (sanitize version) consists of 427 entry-level programming problems. We also include the introductory-leve problems
of APPS 24], which contains 1000 problems.
C.2 Detailes of Mathematical Reasoning Task
For all experiments in the main paper, we use a sampling temperature of 1.0 and set the top-p parameter 0.95. The prompt template follows the recommendation of each LLM.
Prompt for Math Reasoning Tasks. The InternL.M2-MATHH-Plus 1.8B and 7B models are chat models that facilitate conversations between two roles: "user" and "assistant". The prompt for the "user" role provided in Prompt[T contrast, the MeepSeek-Math 7B model operates in non-chat mode, and its corresponding prompt is shown in Prompt
27

---

Prompt 1: Prompt for Internl Mi-ee-lM-2-11
Problem:
[instruction]
Let's think step by step Solution:
Prompt 2: Prompt for DeepSeck-Min Math
(instruction)
Please reason step by step, and put your final answer within \boxed }.
Prompt for Mathematical Verbalized Method. We observed that the tuned math models are challenging prompt for generating confidence. Therefore, we adopted the methods from [52) to calculate the probability based on the likelihood of the first generated "True" token and the first generated "False" token. The corresponding prompt is provided Prompt
Prompt 3: Prompt for Mathematical Verbalized Method
Question: (question)
Proposed Answer: [answer]
Is the proposed answer:
M((A) True
MtBB False? The proposed answer is:
C.3 Detailes of Code Generation Task
Code Generation. On the code generation task, we LLM generate a code snippet to solve given programming problem, and then evaluate its functional correctness based on the ground-truth test cases provided by the dataset. detail, we set the top p to 0.95, and the max generation length to 1024. For code snippet prot-processing, by sopll-bbccttccss(=>) and then we follow [8] to truncate the generated code snippet before the following stop sequences: "Inclass", "Andef". "'n#" "vnif", "Anprint". At the same time, we also obtain the log-probability of each token from the LLM response. For "verbalization" setting, the verbalized confidence is also extracted from the text generated by L.MM along with the code snippet.
first extract the code text from the code block surrounded
Self-consistency Code. We follow 7 to sample 100 test cases for each programming problem from the same model. Then, we achieved self-consistency in code at the semantic equivalence level, which is based on the execution behavior of any two codes this set of test cases. More formally, we implemented the consistency function Tc(:.) ) as an indicator function that indicates whether two codes are semantically equivalent, i.e., TT y) if and only if code æ and y execute the same result on this set of test cases.
Prompt for Generating Code. The prompt for generating code consists header, a functional signature, and docstring and LLM needs to implement the body of this function. An Illustration of has_close__ ces_ccoos_eeeemente elements problem is shown Prompt
Prompt for Generating Test Cases. For generating test cases, we implement the function body with "pass" statement on the basis of the prompt generate the code, and added a comment to require the L.LM to generate test cases for the programming problem. An Illustration of has_close, cwwoseeeeeeentteeeeeeeeeeeeeeeee elements problem is shown in 5
28

---

Prompt 4: Prompt for Generating Code
def has close_ mas_close_eeementee elements (numbers: List [float],
threshold: float) -> bool:
Check if in given list of numbers, are any two numbers closer to each other than given threshold.
Prompt 5: Prompt for Generating Test Cases.
def
has has ees_clleeeeeemente elements (numbers: List [float], threshold: float) -> bool:
Check if in given list of numbers, are any two numbers closer to each other than given threshold.
pass
check the correctness of has welloseeeeemmeneeeeeeeee assert
Prompt for Code Verbalized Method. For generating code with verbalized confidence, we add instructions for generating verbalized confidence, as well format requirements to facilitate the extraction of code and confidence score. We also give simple example to help L.IM understand the format requirements at the end of the prompt. An Illustration of has coss_cloe_____ sll_ccloseeeeeeeeeeee sate__eementt problem is
shown in 6
Prompt 6: Prompt for Code Verbalized Method
Come up with a solution that solves the following programming question and provide your confidence score in this solution like 0%, 10%, ... 100%.
def has cissecloe__e elements (numbers: List [float], threshold: float) -> bool:
Check if in given list of numbers, are any two numbers closer to each other than given threshold. m*n
Format requirement: output in the form of the following example. Do not provide any additional explanations. Here is output example:
Solution:
'pyhhn your code <c.
Confidence:
D Detailed Experimental Results
D.1 Results under High Sampling Temperature.
Using a high sampling temperature enables L.LMs to produce more diverse outputs, potentially enhancing reasoning performance. However, it also leads to increase in estimation error.
29

---

To investigate the effectiveness of our approaches in addressing the estimation error issue, we conducted experiments with higher sampling temperatures (T 1.1 and T 1.3) using the InternLM- 2-MATH-Plus 7B model. The results in Table indicaaee that our RPC approach consistently surpasses baseline methods. Notably, significant performance gap persists between RPC and Sc, indicating that RPC effectively tackles the estimation error issue even under sampling high-temperature.
We also report the performance of RPC with different sampling numbers under the high temperature in Figure and Figure 6 The results demonstrate that the RPC method effectively improves the reasoning performance at higher temperatures,
leverages the increased diversity in sampling to enhance self consissssst.... In contrast, the Sc method's performance deteriorates due to increased estimation errors at higher temperatures.
Table 4: Performance Comparison of different models and different parameter scales. The accuracy reported as the mean and stdev. The best performance highlighted bold. The results show that RPC approach consistently outperforms existing methods.
Temperature = 1.1
Method
MATH
MathOdyssey
OlympiadBench
AIME
PPL
47.35 + 0.16
28.59
1.30
7.27 + 0.23
0.23
6.02 + 0.34
VERB
25.51 + 0.23
9.41
0.44
3.66 + 0.16
3.07 + 0.15
Sc
50.66 0.22
27.89 0.43
10.74 0.15
8.73 + 0.24
RPC
52.58 = 0.14
32.98 0.69
11.00 = 0.24
9.30 + 0.29
Temperature 1.3
Method
MATH
MathOdyssey
OlympiadBench
AIME
PPL
47.58 0.31
26.38 1.41
7.76 + 0.46
6.50 + 0.41
VERB
24.62 - 0.33
8.60 - 0.26
3.11 0.17
2.29 + 0.12
Sc
50.65 0.14
27.61 0.67
10.49 0.18
8.02 + 0.20
RPC
53.12 + 0.14
33.19
0.56
10.91 + 0.18
8.83 + 0.23
Accuney (%)
NO
Accumic (%)
Accuncy (%)
Accunst (%)
12
22
#Sampling B
#Sampling a
#Sampling
#Sampling
(a) MATH
(b) MathOdyssey
(c) OlympiadBench
(d) AIME
Figure 5: The accuracy of
7B model four mathematical reasoning
datasets with different sample sizes n. The sampling temperature is set to 1.1.
FR
Accurncy (%))
NO RuC
Accurccy (15)
Accuracy (%)
19124
45
10T
10:22
#Sampling
#Sampling m
#Samplingn
#Sampling m
(c) OlympiadBench Figure 6: The accuracy of M 7B model four mathematical reasoning datasets with different sample sizes n. The sampling temperature is set to 1.3.
MATH
(b) MathOdyssey
(d) AIME
30

---

D.2 Performance with Different Models Scales
In the Figure 2] we plot the accuracy of the IntemLM-2-MATH-PII 7B model on four mathematical reasoning datasets with different sample sizes n. Here, we further investigate the performance of relateively small model, Mieeml.----MATII can be drawn from these results. For the MathOdysse dataset, the PPI. method shows superior performance compared to other methods, which can be attributed to the relatively low model error of PPL on this dataset, allowing the perplexity- RPC method consistently outperforms the Sc method, which demonstrates its ability to enhance the convergence properties of the Sc method.
1.8B, is presented in Figure 7 Similar conclusions
approach to function effectively. Furthermore, the
Acoumey (%)
O4 Counony
(90) Cemmni
KIN
PPL
#Sampling
#Sampling
#Sampling
ASampling
(a) MATH
(b) MathOdyssey
(c) OlympiadBench
(d) AIME
Figure 7: The accuracy of
1.8B model on four mathematical reasoning
datasets with different sample sizes n.
D.3 Performance with R1 LLMs
We further investigate the performance of each method using RI LLMs with thinking capability [9] Here, we conduct experiments on the DeepSeek- R1 R RI--Disiill-wen- model [1] using 16 samples, where the internal probability of LLM is computed only using the response after the "</think>" token.
The results presented in Table 5
We observe that our RPC method consistently outperforms PPL and Sc across all datasets when using RI LLMs. Moreover, the performance improvements persist even when the base LLM becomes stronger, demonstrating the potential of RPC improve even stronger LLMs and its generalizability across diverse LLMs.
Table 5: Performance comparison
model. The best performance is highlighted in bold. The results show that our RPC is compatible with thinking LLMs and consistently outperforms PPL and Sc across all datasets.
Method
MathOdyssey
AIME
MATH
OlympiadBench
PPL
60.04
72.92
81.81
21.65
Sc
57.22
70.40
82.03
21.93
RPC
61.11
76.47
82.78
22.81
D.4 Performance Comparison with Advanced Methods
In our main paper, we compare RPC with Sc and PPL. to demonstrate its effectiveness in bridging internal probability and self-consistenge which confirms our theoretical analysis. Here, we further combine RPC with two advanced methods: ESC [36] (an advanced version of Sc) and Best-of-N (BoN) using the 1ren2..5MMtthPR... replaces internal probability with rewards from an external reward model) to verify the compatibility of RPC with state-of-the-ant art methods. Similar to Appendix D.3 conduct experiments the DeepSeek-R1-Disse Qwen-7B model using 16 samples.
reward model (RM) [69] (an advanced version of PPL that
As shown in Table 6 RPC using ESC consistently outperforms both Sc and ESC across all datasets, demonstrating its effectiveness when adapted to advanced methods. Similarly, the results in Table 7 show that RPC using RM consistently performs better than or comparable to Sc and BoN using RM across all datasets. Overall, these experimental results demonstrate that RPC is compatible with state-of-tt ofethe-art methods and consistently outperforms them across all datasets.
31

---

Table 6: Performance comparison with ESC method. The best performance is highlighted in bold.
Method
MathOdyssey
AIME
MATH
OlympiadBench
Sc
57.22
70.40
82.03
21.93
ESC
57.17
70.18
81.97
22.03
RPC using ESC
61.03
76.26
82.74
22.81
Table 7: Performance comparison with BoN method using reward model. The best performance is highlighted in bold.
Method
MathOdyssey
AIME
MATH
OlympiadBench
Sc BoN using RM
57.22
70.40
82.03
21.93
58.35
69.35
81.86
21.57
RPC using RM
58.35
71.04
82.16
22.22
D.5 Performance on Additional Reasoning Tasks
In our main paper, we demonstrate the effectiveness RPC on both mathematical reasoning and code generation tasks. Here, we further conduct experiments on two additional reasoning tasks: GPQA [47] and LogiQA [39]. GPQA is a multiple-choice benchmark covering biology, physics, and chemistry, while LogiQA is a benchmark testing logical reasoning abilities. Similar to Appendix D.3
we conduct experiments on the DeepSeck-RI I--Distill-Qwen. model using 16 samples.
As shown in Table 8 we observe that our RPC method consistently outperforms PPL and Sc on both datasets. These experimental results demonstrate that the RPC method generalizable additional reasoning tasks, showing its strong potential for application to diverse real-world reasoning scenarios.
Table 8: Performance comparison on GPQA and LogiQA benchmarks. The best performance is highlighted in bold.
Method
GPQA
LogiQA
PPL
41.46
54.36
Sc
43.00
56.71
RPC
44.09
58,42
D.6 Explanation Hyper-parameetrs
We would like to emphasize that the RPC method introduces Reasoning Pruning module that can automatically determine the pruning of reasoning paths without manually setting thresholds. Therefore, it avoids sensitivity issues associated with manual threshold setting and makes our RPC hyperparameter-free method, which is robust across diverse tasks, including mathematical reasoning, code generation, logical reasoning, etc.
Although our RPC method does not explicitly require hyperparameter tuning. the optimization process of the Reasoning Pruning module still involves some parameter configurations, such as initialization and the range of distribution parameters. Therefore, evaluate the robustness of RPc, we conducted experiments analyzing the impact of different initialization methods and parameter bounds. Specifically, we tested the method with 10 random seeds under various configurations, including different initialization approaches (Fixed Init vs. Zero Init) and different bounds for distribution parameters W1 and W2. The Fixed Init method and parameter bounds of [0.2, 0.8) represent our default optimization settings for RPC. Results presented in Table 9 demonstrate that RPC maintains robust performance across different configurations. Note that these parameter configurations in our method do not require tuning in practical applications.
32

---

Table 9: Performance of RPC with different initialization methods and parameter bounds when automatically modeling internal LLM probability distributions.
Fixed Init
Zero Init
w E 0.2, 0.8]
31.620 + 0.754
31.183 0.821
w 0.15, 0.85]
31.748 + 0.622
32.108 + 0.685
w E (0.1, 0.9]
31.722 + 0.577
32.416 + 0.532
D.7 Explanation on Computational Overhead
In this section, analyze the computational overhead of RPC compared to Sc from both theoretical and practical perspectives.
From theoretical standpoint, Sc requires Onn²) time complexity for answer equivalence computa- tion, where n the number of samplings. Our RPC method consists of two components: Reasoning Pruning with O(k(m² + +n)) time complexity and Perplexity Consistency with O n log np + Rp) time complexity, where m represents the number of mixture distribution parameters, & denotes the optimization iteration count, and up is the number of reasoning paths after pruning (np n)). The overall complexity simplifies to (22k + nk + n²) I 0 + nk + n²), introducing only minimal computational overhead compared to Sc.
From a practical perspective. experiments on MathOdyssey with 128 samplings show that Sc takes 0.006s per question while RPC takes 0.036s. Both processing times are negligible compared to the I.LM inference time required for sampling multiple reasoning paths. Moreover, RPC's slight overhead is well-justified by ability to reduce the required number of samplings while maintaining performance, as demonstrated in Table I which addresses the primary computational bottleneck in reasoning tasks.
Overall, RPC provides an excellent computational trade-off, where the computational overhead of Rp exchanged for significant time savings achieved by reducing the number of LLM inferences.
E Limitations and Future Work
In this paper, we introduce a theoretical framework for analysis the sampling -batid test-time scaling methods of LLM reasoning and new method based on our analyses. While our theoretical framework and approach offer several advantages, there remain some limitations as listed as follows:
1. Our theoretical framework is general to analysis sampling-basedd test-time scaling methods, however, in this paper, we only focus the analysis
typical methods well RPC.
2. Our proposed method is post-hoc approach that relies solely on the sampled reasoning paths and selects answers from these samples. This design makes the method straightforward to use, as it does not require any modifications to the LLM architecture its training process. However, the performance improvements are moderate compared to approaches that involve model training. Although integrating method with trained LLMs may provide additional performance gains, it remains an open question how to utilize our insights to improve the training process of LLMs.
3. Current design of our method is relatively simple, which leaves considerable room for improvement. For example, the perplexity consistency component could be extended to include alternative probability measures, such as rewards from reward model. Furthermore, the reasoning pruning module could be improved by incorporating more advanced techniques to increase its effectiveness.
We also provide the following directions for future work:
Our paper demonstrates that the theoretical frame- work is versatile enough to analyze two typical sampling-bare based test-time scaling methods. It can also readily extended evaluate other advanced methods derived from these two types, with strong potential to adapt to various test-time scaling strategies.
Analyzing Test-Time Scaling Methods:
33

---

2. Exploring Advanced Sampling Strategies: Our theoretical results indicate that better convergence requires an effective sampling strategy sample sufficiently diverse reasoning paths, which is not deeply investigated in this paper. Improved sampling strategies could be designed by drawing inspiration from our theoretical framework.
3. Applying to Diverse Reasoning Tasks: In paper, show that the Rpe method is effective on math, code, and logical reasoning tasks without exploiting task-specific properties. Task-specific methods could be developed based on our theoretical framework to achieve better performance (c.g., by considering the properties of g(-) in different reasoning tasks).
34


Reviews:
[{"review_id": "CNoqZLYLHF", "reviewer": "Reviewer_m4pU", "summary": "They present a theoretical analysis of several methods for aggregating multiple LLM samples: perplexity based and consistency based. They break the model's error into both an estimation error component and a model error component. Using their framework, they then propose two improvements which improve both the estimation/model error theoretically: perplexity consistency and reasoning pruning. They show that combining these techniques can also empirically yield improvements on best of N on several math benchmarks with open LLMs.", "questions": "See weaknesses.", "limitations": "I think they could evaluate on more benchmarks and using more LLMs to strengthen their empirical claims. The gains observed are not that huge.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* They present an interesting theoretically framework for understanding the efficacy of different best-of-N test time compute aggregation methods.\n* Their theoretical framework allows them to come up with an improvement over existing techniques which yields empirical gains.\n* The paper is largely well written\n* They carry out reasonably thorough empirical experiments\n* I like the use of both accuracy and calibration as metrics for evaluation\n\nWeaknesses:\n* The gains from their new method are quite small on most of the tasks they experiment with, with the exception of MathOdessey, which shows much more substantial gains. Why is that? Is there something special about that particular text which lends itself to this method?\n* The ideas derived from their theory feel sort of obvious. I guess it's nice to have all the theory, but if you asked me to list off 20 ideas for ways one could potentially improve over majority voting, I would probably come up with these, and then I would be able to pick them with empirical validation from there. This isn't to necessarily take away from the work, but at points the theory feels a little excessive for what it is helping to deliver in the paper.\n* They do all their experiments using two LLaMA models, but could demonstrate the generality of their approach better by showing improvements using more open LLMs.", "was_revised": true, "final_justification": "I believe the additional experiments they provide help give further empirical evidence for their method, so I am willing to raise my score to a 4.", "rebuttal": "**Dear Reviewer m4pU:**\n\nThank you for your valuable feedback on our paper. We address your concerns as follows.\n\n### **Weakness 1: Explanation for the MathOdyssey Dataset**\n\nTechnically, RPC effectively integrates the strengths of both SC and PPL methods. When these two approaches produce good and complementary results, RPC can leverage this synergy to achieve substantial performance improvements. In the case of the MathOdyssey dataset, Figure 2(b) illustrates this complementarity: as the number of samplings increases, PPL demonstrates a large initial advantage, while SC performance improves later. This complementarity enables RPC to exploit the strengths of both methods, resulting in the significant performance gains observed.\n\n### **Weakness 1 & Limitations: Magnitude of Performance Gains**\n\nWe would like to clarify that RPC's performance gains should be considered in the proper context. RPC is a general framework with two key features: it is a training-free method that needs no extra data collection or model fine-tuning; and it is a post-hoc method, meaning any PPL or SC functions can be directly integrated. Given these points, even modest but consistent accuracy improvements across diverse, challenging reasoning benchmarks are quite significant. Additionally, our approach offers several advantages beyond accuracy:\n(1) **Efficiency**: As demonstrated by our theoretical results and confirmed in Table 1, RPC achieves faster estimation error convergence, potentially reducing the computational costs;\n(2) **Calibration**: As demonstrated in Table 2, RPC also produces more reliable confidence scores, enhancing the reliability of LLM prediction.\n\nIn summary, when evaluated comprehensively across accuracy, efficiency, and calibration, our empirical and theoretical results strongly support the advantages of the RPC approach.\n\n### **Weakness 2: Importance of Theoretical Results**\n\nWe would like to emphasize that the empirical exploration-guided method and the theoretical analysis-guided method represent two distinct research methodologies with different strengths.\nWhile we acknowledge that effective algorithms can emerge through empirical exploration, theoretical analysis provides unique advantages in systematically identifying the fundamental limitations of existing algorithms and developing principled solutions.\n\nAs presented in our paper, our theoretical results reveal the fundamental limitations of SC and PPL methods. We then derive potential solutions by combining these methods and provide theoretical results that reveal both advantages and challenges of this approach. RPC serves as a direct implementation of our theoretical insights, which empirically validates our theoretical framework and demonstrates the effectiveness of our proposed solution. Therefore, we believe that beyond the specific algorithm we proposed, our theoretical results provide valuable insights and make significant contributions to the field.\n\n### **Weakness 3 & Limitations: Experiments with Additional LLMs**\n\nThank you for your suggestion. To address this concern, we conducted additional experiments using DeepSeek-R1-Distill-Qwen-7B [1], a recent LLM with long CoT reasoning capability [2]. Due to computational resource constraints, we focused on two relatively small datasets: MathOdyssey and AIME 1983-2024, using 16 samplings per question for each method. We evaluated both accuracy and calibration metrics across these datasets. \nThe results consistently show that, when a better base model is adopted, RPC still outperforms baseline methods while maintaining good calibration, which further validates the generalizability of our approach across different models.\n\n|  | MathOdyssey Acc. | MathOdyssey ECE | AIME Acc. | AIME ECE |\n| --- | --- | --- | --- | --- |\n| PPL | 60.04 | 20.2 | 72.92 | 4.3 |\n| SC | 57.22 | **11.4** | 70.40 | 5.3 |\n| RPC | **61.11** | **11.4** | **76.47** | **2.5** |\n\n### **Limitations: Experiments with Additional Benchmarks**\n\nThank you for your suggestion. We further validate our proposed method across diverse reasoning domains. Specifically, we evaluate RPC on GPQA [3] (a multiple-choice benchmark covering biology, physics, and chemistry) and LogiQA [4] (a benchmark testing logical reasoning abilities) using the DeepSeek-R1-Distill-Qwen-7B model [1].\n\nWe report the performance of PPL, SC, and RPC using 16 samplings per question. Additionally, we evaluate RPC with only 10 samplings (denoted as RPC*), which reduces LLM inference costs by 37.5% compared to other methods. The results demonstrate that RPC consistently outperforms baseline methods across different reasoning tasks. \nNotably, even with fewer samplings, RPC* still achieves better performance than SC and PPL methods (which use 16 samplings), demonstrating both the effectiveness and efficiency of our approach and its broad applicability.\n\n|  | GPQA | LogiQA |\n| --- | --- | --- |\n| PPL | 41.46 | 54.36 |\n| SC | 43.00 | 56.71 |\n| RPC* (w/ 10 samplings) | 43.58 | 57.59 |\n| RPC | **44.09** | **58.42** |\n\n### **Reference**\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Arxiv 2025.\n\n[2] Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models. Arxiv 2025.\n\n[3] GPQA: A Graduate-Level Google-Proof Q&A Benchmark. COLM 2024.\n\n[4] LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning. IJCAI 2020."}, {"review_id": "jVGR5sFi2h", "reviewer": "Reviewer_bSKW", "summary": "This paper lays out a new way to think about test-time sampling in large language model reasoning, splitting the error into two parts: estimation error from sampling and model error from the LLM itself. It introduces a method called Reasoning-pruning Perplexity Consistency (RPC), which blends the strengths of self-consistency and perplexity while trimming less likely reasoning paths to cut sampling needs by half. Through experiments on seven datasets, including math and coding tasks, RPC shows it can match or beat existing methods in accuracy while making confidence estimates more reliable.", "questions": "1. Have you considered testing RPC on tasks beyond math and coding, like commonsense reasoning or text-based decision-making? Showing it works across diverse domains would make the method’s impact more convincing and could sway my view on its significance.\n2. What’s the computational trade-off for RPC, especially with the Weibull distribution fitting and pruning steps? Providing runtime or resource comparisons with baselines would clarify its feasibility and could strengthen my confidence in its practicality.\n3. How sensitive is RPC’s performance to the pruning threshold τ? An ablation study showing how different τ values affect results would help me trust the method’s robustness and potentially improve my quality assessment.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The theoretical framework is a standout, offering a clear lens to analyze sampling strategies with solid mathematical backing, including detailed proofs tucked in the appendix. \n- The writing flows logically, guiding readers from problem setup to RPC’s design and results, with figures and tables that make the method’s benefits easy to grasp.\n- By slashing sampling requirements by 50% without sacrificing performance, RPC tackles a real bottleneck in deploying LLMs efficiently, which could have broad practical impact.\n- RPC’s blend of perplexity, self-consistency, and a pruning mechanism feels fresh, cleverly building on prior work to address specific shortcomings.\n\nWeaknesses:\n- The experiments stick to math and coding datasets, so it’s unclear how well RPC would handle other reasoning tasks like commonsense or creative problems, limiting its proven scope.\n- The explanation of the Weibull distribution for pruning is a bit thin, which might leave readers puzzled about how it’s implemented or why it’s chosen.", "was_revised": true, "final_justification": "Thank you to the authors for their thoughtful and insightful responses. I maintain my positive score, and I hope this paper will be accepted.", "rebuttal": "**Dear Reviewer bSKW:**\n\nThank you for your valuable feedback on our paper. We address your concerns as follows.\n\n### **Weakness 1 & Question 1: Additional Evaluation Benchmarks**\n\nThank you for your suggestion. To address your concern about the generalizability of our method beyond mathematical and coding tasks, we conducted additional experiments on diverse reasoning domains. Specifically, we evaluated RPC on GPQA [2] (a multiple-choice benchmark covering biology, physics, and chemistry) and LogiQA [3] (a benchmark testing logical reasoning abilities) using the DeepSeek-R1-Distill-Qwen-7B model [1].\n\n|  | GPQA | LogiQA |\n| --- | --- | --- |\n| PPL | 41.46 | 54.36 |\n| SC | 43.00 | 56.71 |\n| RPC | **44.09** | **58.42** |\n\nThese additional experimental results across diverse reasoning benchmarks provide strong empirical support for our theoretical framework and demonstrate the broad applicability of RPC beyond mathematical and coding tasks, addressing your concern about the generalizability of our method.\n\n### **Weakness 2: Explanation of Reasoning Pruning**\n\nThank you for raising this question. Our core idea is to model the probability distribution of LLM samplings as a mixture of two distributions: one for high-confidence reasoning paths and another for low-confidence paths. This allows us to automatically identify which samplings belong to the low-probability distribution for pruning, thereby eliminating the need for manually determined thresholds and their associated sensitivity issues.\n\nRegarding implementation details, we selected the Weibull distribution as our base distribution specifically because it effectively captures low confidence values, which is essential for our pruning. The distribution parameters ($w_1$, $w_2$, $k_1$, $k_2$, $\\lambda_1$, $\\lambda_2$) are estimated using maximum likelihood estimation via SciPy's optimization tools, as detailed in Algorithm 1 (Lines 5-7) in Appendix B. \nAs mentioned in our Limitations in Appendix E, our current implementation of Reasoning Pruning is intentionally straightforward to validate our theoretical framework, with potential for further enhancement through alternative distributions or more advanced techniques.\n\nWe will revise our paper to provide clearer explanations of both the implementation details and motivation.\n\n### **Question 2: Computational Overhead of RPC**\n\nWe appreciate your concern about the computational trade-offs. We would like to clarify that the primary computational bottleneck in reasoning tasks lies in the time consumed by multiple LLM inferences. As demonstrated in Table 1, RPC can achieve the same performance as SC with fewer samplings, showing strong potential to reduce the LLM inference overhead. Therefore, RPC actually provides an excellent computational trade-off, where the minimal computational overhead of RPC is exchanged for significant time savings by reducing the number of required LLM inferences.\n\nWe provide a detailed analysis of computational overhead in Appendix D.5.\nMoreover, taking the LogiQA benchmark mentioned above as an example, the average time for one LLM inference is approximately 0.7s per question. \nThe computational overhead of SC is 0.0001s per question, while the computational overhead of RPC is 0.008s per question, when dealing with 16 samplings. Therefore, the computational overhead of either method is negligible compared to the LLM inference time. Even saving a single LLM inference completely offsets the additional computational overhead of any method.\n\nWe will revise our paper to clarify the computational issue, and we hope our clarification will address your concern.\n\n### **Question 3: Questions about Pruning Threshold $\\tau$**\n\nWe thank you for highlighting this important point.\nTo clarify, our Reasoning Pruning is designed as a parameter-free method that automatically determines the pruning of reasoning paths without requiring manual setting of a threshold $\\tau$. This determination is based on the probability distribution of reasoning paths, as explained in our response to Weakness 2.\nThis automatic determination eliminates any sensitivity issues that would typically arise from manual threshold setting.\nThe consistently superior performance of RPC across diverse tasks and different models demonstrates the effectiveness and robustness of our automatic approach in the Reasoning Pruning component.\n\nIn the revised paper, we will clarify and emphasize the advantage of the Reasoning Pruning module for automatic pruning.\n\n### **Reference**\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Arxiv 2025.\n\n[2] GPQA: A Graduate-Level Google-Proof Q&A Benchmark. COLM 2024.\n\n[3] LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning. IJCAI 2020."}, {"review_id": "Ie3cjtSDLq", "reviewer": "Reviewer_CQT8", "summary": "Test-time sampling is a hot topic to improve the reasoning performance of large language models (LLMs). In this paper, it gives the theoretical analysis for test-time sampling and two dominant paradigms self-consistency and perplexity are analyzed. Meanwhile, the limitations are also revealed. Then RPC is introduced to leverage theoretical insights through two key components: Perplexity Consistency and Reasoning Pruning. The experiments show that the sampling costs are signifcicantly reduced and the confidence reliability is enhanced.", "questions": "1.The challenges to analyze the self-consistency and perplexity for the test-time sampling should be further explained since it is the motivation of the paper and can improve the quality of the paper well.\n2.Since test-time sampling is a hot topic, the works that are published in recent two years should be compared and analyzed.\n3. In this paper, a framework is designed. However, I wonder whether the frameworks can be integated into the previous works and the framework should be further verifed.", "limitations": "The authors state that there are few works to give the theoretical analysis. However, the challenges to give the theoretical analysis should be deeply expained since it is the foundation of the paper.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1.The paper is well organized and it is easy to follow.\n2.The theoretical analysis can expain the reasoning of LLM well. The experiments are well designed.\n\nWeakeness:\n1.The challenges to analyze the self-consistency and perplexity for the test-time sampling should be further explained.\n2.More SOTA methods with test-time sampling should be comapred. \n3.The framework should be further verifed.", "was_revised": true, "final_justification": "Most of my concerns have been addressed and the experiments seem to be promising.", "rebuttal": "**Dear Reviewer CQT8:**\n\nThank you for your valuable feedback on our paper. We address your concerns as follows.\n\n### **Weakness 1 & Question 1 & Limitations: Challenges of Theoretical Analysis**\n\nThank you for your insightful question regarding the challenges of theoretical analysis and our motivation. We would like to clarify the challenges of theoretical analysis as follows.\n\nFirst, a significant gap exists in the theoretical understanding of current methods like Self-Consistency (SC) and Perplexity (PPL). There is no conclusive explanation for their respective strengths or the specific conditions under which each excels. This gap motivated us to develop a robust theoretical framework. Our goal is to not only explain their empirical performance but also to identify their inherent disadvantages and, crucially, devise a method (RPC) that synergistically combines their strengths.\n\nSecond, the fundamental difficulty in combining SC and PPL lies in their distinct methodological underpinnings. Self-Consistency (SC) operates as a sampling-based estimation method, approximating the true probability distribution by aggregating multiple reasoning paths. In contrast, Perplexity (PPL) is a direct estimation method that calculates probabilities straight from the model. The disparate principles governing these two approaches make the theoretical analysis of their combination particularly challenging.\n\n### **Weakness 2 & Question 2: Comparison with Recent Methods**\n\nThank you for your suggestion. We evaluate each method using a recent LLM with the long CoT reasoning capability [2], i.e., DeepSeek-R1-Distill-Qwen-7B [1], to determine whether RPC can enhance performance when combined with SOTA test-time sampling technique. We also compare the best-of-N method (BoN) using a reward model, which selects the samplings according to average process reward scores generated by Qwen2.5-Math-PRM 7B [4]. Additionally, we compare an advanced self-consistency method, i.e., ESC [3]. Due to computational resource constraints, we only evaluate two relative small datasets, i.e., MathOdessey and AIME 1983-2024, with a maximum of 16 samplings per problem as follows.\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| PPL | 60.04 | 72.92 |\n| SC | 57.22 | 70.40 |\n| BoN using RM [4] | 58.35 | 69.35 |\n| ESC [3] | 57.17 | 70.18 |\n| RPC | **61.11** | **76.47** |\n\nThe results demonstrate that our RPC consistently outperforms all comparison methods, demonstrating its effectiveness.\n\n### **Weakness 3 & Question 3: Further Verification on Theoretical Framework**\n\nThank you for your suggestion. Our framework is generally compatible with extended versions of both PPL and SC methods, meaning the PPL or SC functions can be readily substituted with more advanced alternatives.  For instance, consider integrating ESC [3], an advanced self-consistency method. Our theoretical framework readily extends to this by analyzing the error between the ground-truth probability and the estimated probability derived from ESC's specific computation functions. In this scenario, our theoretical analysis (Proposition 2 and Theorem 4) remains valid, and the new SC method does not alter our proposed framework.\n\nTo empirically validate this,  our experimental results below demonstrate that RPC remains effective when the SC function is replaced by ESC or when the PPL function is replaced by reward functions from an external Reward Model (RM). Both RPC using ESC and RPC using RM methods outperform the baseline SC method while maintaining comparable or superior performance relative to their respective counterparts. These findings emphasize the effectiveness of RPC and empirically verify our theoretical framework.\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| SC | 57.22 | 70.40 |\n| ESC [3] | 57.17 | 70.18 |\n| RPC using ESC [3] | **61.03** | **76.26** |\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| SC | 57.22 | 70.40 |\n| BoN using RM [4] | **58.35** | 69.35 |\n| RPC using RM [4] | **58.35** | **71.04** |\n\nWe believe our theoretical framework provides a foundation that can help future researchers identify the limitations of various test-time sampling methods and inspire meaningful improvements to these approaches.\n\n### **Reference**\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Arxiv 2025.\n\n[2] Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models. Arxiv 2025.\n\n[3] Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning. ICLR 2024.\n\n[4] The Lessons of Developing Process Reward Models in Mathematical Reasoning. Arxiv 2025."}, {"review_id": "xVFgT3fpTr", "reviewer": "Reviewer_CyKh", "summary": "The paper titled \"A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning\" presents a novel theoretical framework for analyzing test-time sampling strategies in large language models (LLMs) used for reasoning tasks. The authors introduce a method called Reasoning-Pruning Perplexity Consistency (RPC), which combines the strengths of self-consistency and perplexity-based methods to improve reasoning performance. The paper provides a detailed analysis of existing methods, identifies their limitations, and demonstrates through both theoretical and empirical results that RPC can significantly reduce reasoning error while maintaining high confidence reliability.", "questions": "1. Could the authors provide more details on the implementation of the mixture distribution modeling in the Reasoning Pruning component? How sensitive is the performance to the choice of distribution parameters?\n2. Are there any other state-of-the-art test-time sampling methods that the authors plan to compare RPC with in future work? How does RPC compare to methods that involve fine-tuning or reinforcement learning for reasoning tasks?", "limitations": "The computational complexity of RPC might limit its scalability to very large datasets or real-time applications. The authors should further investigate the trade-offs between performance and computational efficiency.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper introduces a comprehensive theoretical framework that decomposes reasoning error into estimation error and model error. This provides a principled way to analyze and understand the limitations of existing test-time sampling methods.\n2. The proposed RPC method effectively combines the strengths of self-consistency and perplexity, achieving faster convergence rates and lower model error. This hybrid approach is supported by both theoretical analysis and empirical results.\n3. The authors conduct extensive experiments on seven benchmark datasets, demonstrating that RPC outperforms existing methods in terms of reasoning accuracy, efficiency, and confidence reliability. The results are statistically significant and reproducible.\n4. The paper provides detailed proofs and analysis of the theoretical results, which enhances the credibility and depth of the research.\n\nWeaknesses:\n1. Although the authors argue that the computational overhead of RPC is acceptable, the additional processing steps (e.g., mixture distribution modeling) could be a concern for real-time applications where inference speed is critical.\n2. The effectiveness of RPC relies on the quality of the underlying LLM. If the LLM itself has significant reasoning errors or biases, the improvements brought by RPC might be limited.", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer CyKh:**\n\nThank you for your valuable feedback on our paper. We address your concerns as follows.\n\n### **Weakness 1 & Limitations: Computational Overhead of RPC**\n\nWe appreciate your concern about computational efficiency. We would like to emphasize that the primary computational bottleneck in reasoning tasks lies in the time consumed by multiple LLM inferences. \nAs demonstrated in Table 1, RPC can achieve the same or better performance as SC with fewer samplings, showing strong potential to reduce the computational overhead of LLM inference. \nTherefore, RPC actually provides an excellent computational trade-off, where the minimal computational overhead of RPC is exchanged for significant time savings by reducing the number of required LLM inferences.\n\nAs detailed in Appendix D.5, our thorough efficiency analysis shows that RPC maintains a time complexity comparable to SC, with only a negligible overhead of 0.03s per problem when processing 128 samplings on the MathOdyssey dataset. Therefore, RPC demonstrates practicality for real-time applications where inference speed is critical, as it achieves better performance with fewer LLM samplings while introducing only negligible computational overhead compared to existing methods.\n\n### **Weakness 2: Performance of RPC regarding to underlying LLM**\n\nWe appreciate your insightful concern about the dependency on the underlying LLM quality. However, we would like to highlight that this dependency affects all comparison methods (SC and PPL) equally, and we compare the performance using the same LLM to ensure the fairness of experiments. Furthermore, our experiments in Table 3 and Figure 4 demonstrate that RPC consistently delivers performance improvements across different model architectures and sizes, indicating the robustness of our approach.\n\nFrom a theoretical perspective, our framework explicitly addresses this concern by decomposing reasoning error into two components: model error (inherent to the specific LLM) and estimation error (which can be optimized by the algorithms). This decomposition allows us to isolate the impact of the underlying model quality (model error) and focus on improving the estimation process (estimation error). RPC is specifically designed to minimize this estimation error component, making it effective regardless of the base model, as long as the model provides meaningful reasoning signals.\n\n### **Question 1: Details of Reasoning Pruning**\n\nThank you for the question about our implementation details of Reasoning Pruning.\nReasoning Pruning is a parameter-free method that automatically determines the pruning of reasoning paths without requiring manual setting of a threshold. \nIts core idea is modeling LLM probabilities across multiple samplings as a mixture of two distributions: a normal confidence distribution and an extremely low confidence distribution. This approach allows us to automatically identify and prune low-quality reasoning paths without requiring manual threshold setting.\nFor implementation details, we model the LLM probability distribution using the mixture model specified in the equation below Line 211. We estimate the distribution parameters ($w_1$, $w_2$, $k_1$, $k_2$, $\\lambda_1$, $\\lambda_2$) using maximum likelihood estimation implemented with SciPy's optimization tool. This process is described in Algorithm 1 (Lines 5-7) in Appendix B.\n\nA key advantage of our approach is that all distribution parameters are automatically estimated from the probability of LLM samplings rather than being manually configured, eliminating sensitivity concerns. Our extensive experiments across various models and diverse datasets consistently demonstrate the robustness of this automatic approach.\n\n### **Question 2: Future Work of RPC**\n\nThank you for your suggestions. We plan to compare RPC with more methods using more LLMs in future work to evaluate its effectiveness. The following preliminary experiments verify the feasibility of this plan.\n\nWe evaluate RPC using LLM with the long CoT reasoning capability [2], i.e., DeepSeek-R1-Distill-Qwen-7B [1], to determine whether RPC can enhance performance when combined with the SOTA test-time sampling technique. We compare the best-of-N method (BoN), which selects the samplings according to the average process reward scores generated by Qwen2.5-Math-PRM 7B [4]. Additionally, we compare an advanced self-consistency method, i.e., ESC [3]. Due to computational resource constraints, we only evaluate and report two relatively small datasets, i.e., MathOdessey and AIME 1983-2024, with a maximum of 16 samplings per problem as follows. The results demonstrate that our RPC consistently outperforms all comparison methods, demonstrating its effectiveness.\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| PPL | 60.04 | 72.92 |\n| SC | 57.22 | 70.40 |\n| BoN using RM [4] | 58.35 | 69.35 |\n| ESC [3] | 57.17 | 70.18 |\n| RPC | **61.11** | **76.47** |\n\nRegarding fine-tuning or reinforcement learning models, we would like to clarify that RPC is a post-hoc method that can be easily integrated with a trained model or sampled reasoning paths. For example, we have demonstrated that RPC can be directly applied to the DeepSeek-R1-Distill-Qwen-7B model to further improve the performance in the previous table. \n\nMoreover, RPC can further enhance advanced methods. For instance, ESC can be upgraded through RPC (RPC using ESC), and RPC can also leverage reward scores generated by a reward model (RPC using RM). The results, presented in the following two tables, show that both RPC using ESC and RPC using RM methods outperform the SC method while maintaining comparable or superior performance to their original counterparts, demonstrating the effectiveness of RPC.\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| SC | 57.22 | 70.40 |\n| ESC [3] | 57.17 | 70.18 |\n| RPC using ESC [3] | **61.03** | **76.26** |\n\n|  | MathOdessey | AIME 1983-2024 |\n| --- | --- | --- |\n| SC | 57.22 | 70.40 |\n| BoN using RM [4] | **58.35** | 69.35 |\n| RPC using RM [4] | **58.35** | **71.04** |\n\n### **Reference**\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Arxiv 2025.\n\n[2] Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models. Arxiv 2025.\n\n[3] Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning. ICLR 2024.\n\n[4] The Lessons of Developing Process Reward Models in Mathematical Reasoning. Arxiv 2025."}]

Author rebuttal:

Meta-review: This paper presents an interesting theoretical analysis on various methods of ensembling LLM output, which leads to design of Perplexity Consistency and Reasoning Pruning components. Through the discussion in Rebuttal phase, the reviewers reach consensus on accepting this paper.
Decision: Accept (poster)