Confidence Improves Self-Consistency in LLM s (2025)

Abstract: Self-consistency decoding enhances LLMs’ performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally expensive, as sampling many of these (lengthy) paths is required to increase the chances that the correct answer emerges as the most frequent one. To address this, we introduce Confidence-Informed Self-Consistency (CISC). CISC performs a weighted majority vote based on confidence scores obtained directly from the model. By prioritizing high-confidence paths, it can identify the correct answer with a significantly smaller sample size. When tested on nine models and four datasets, CISC outperforms self-consistency in nearly all configurations, reducing the required number of reasoning paths by over 40% on average. In addition, we introduce the notion of within-question confidence evaluation, after showing that standard evaluation methods are poor predictors of success in distinguishing correct and incorrect answers to the same question. In fact, the most calibrated confidence method proved to be the least effective for CISC. Lastly, beyond these practical implications, our results and analyses show that LLMs can effectively judge the correctness of their own outputs, contributing to the ongoing debate on this topic.

## introduction modern large language models (llms) demonstrate strong reasoning capabilities @xcite , driven in part by their capacity to generate a sequence of intermediate reasoning steps that lead them toward a final answer @xcite . selfconsistency @xcite is a popular decoding strategy that further improves llms' reasoning performance by sampling a diverse set of reasoning paths and selecting the most frequent answer as the final output. despite its effectiveness, this approach is also computationally expensive, as it requires generating a large number of (long) reasoning paths to increase the chances that the correct answer emerges as the most frequent one. motivated by recent evidence that llms possess the ability to judge the correctness of their own outputs @xcite , we hypothesize that self-consistency could be made significantly more efficient if the model could review each generated reasoning path before selecting a final answer. we therefore introduce confidence-informed self-consistency (cisc), a lightweight extension of self-consistency. as illustrated in figure 2 , cisc uses the model to generate a self-assessment score for each path and employs these scores in a weighted majority vote. we conducted a comprehensive comparison of cisc and self-consistency, spanning nine llms of various sizes, four datasets covering a wide range of mathematical and commonsense reasoning tasks, and three popular methods for deriving self-assessment confidence scores from the model. our results demonstrate that cisc outperforms self-consistency in virtually all the examined configurations. using the best-performing confidence estimation method, cisc achieves comparable performance to self-consistency while reducing the required number of reasoning paths by over 40% on average (see figure 1 for an example). surprisingly, the most calibrated confidence method is actually the least useful for cisc. we offer a potential explanation: existing confidence evaluation metrics measure the usefulness of confidence scores for comparing answers across different questions, while cisc requires distinguishing correct and incorrect answers for the same question. to address this, we propose the within-question discrimination (wqd) metric that specifically measures this ability, and demonstrate that it can predict the relative performance of cisc with different confidence methods. finally, we conduct a qualitative-analysis and find a significant agreement between model confidence scores and human assessments of the reasoning-paths' quality. specifically, responses identified by the model as low-confidence were also significantly more likely to be flagged by human evaluators as exhibiting signs of low-quality reasoning patterns. to summarize, we contribute practical methods and foundational insights: we consider an auto-regressive language model m with parameters θ. we use p θ (•|x) to denote m 's distribution over the next token given the provided context x. given a question q (e.g., "jane had 4 apples and ate half of her apples. how many apples she has now?"), we denote the model's response as (r, a), where a is the answer (e.g., "2") and r is a reasoning path (or chain-of-thought), a sequence of logical steps supposedly leading up to this answer (e.g., "if jane ate half her apples, this means she ate 2 apples. 4 minus 2 is 2."). as an intuitive example, consider a hypothetical setting where there exist only two possible answers, one correct and one incorrect. for a model that responds with the correct answer 60% of the time, standard majority voting will require 40 samples to reach 90% accuracy 1 . however, a weighted majority vote that weights correct answers twice as much as incorrect ones, will achieve 90% accuracy with less than 10 samples. with this motivation in mind, we build on recent findings suggesting that llms are capable of judging the correctness of their own outputs @xcite @xcite , and incorporate the model's self-assessment of its reasoning paths into the final answer selection: given a question q and responses {(r 1 , a 1 ), . . . , (r m , a m )}, cisc involves: the temperature parameter t controls the relative importance of the answer frequency versus the confidence scores. namely, as @xmath0, the distribution of normalized confidence scores approaches the uniform distribution, and cisc collapses to vanilla self-consistency. conversely, as @xmath1, the softmax normalization approaches the hard maximum function, prioritizing the single response with the highest confidence and disregarding the overall frequency of answers. this may lead cisc to select a different answer than self-consistency (see figure 2 ). ## experimental setup we compare cisc and self-consistency across a range of confidence extraction methods ( §4.1), reasoning tasks ( §4.2) and models ( §4.3). ## main results this section demonstrates cisc's ( §3.1) substantial performance advantage over self-consistency. we compare cisc, using fixed compute budgets of 5 and 10 responses per question, based on the metrics defined in §4.4. cisc outperforms self-consistency across virtually all models and datasets. method. the results are macro-averaged across all models and datasets. cisc outperforms selfconsistency with every confidence method. the p(true) method yields the best results, achieving an average cost reduction of 41% and 46% with budgets of 5 and 10 responses, respectively. figure 3 presents a detailed breakdown of cisc's performance using p(true) across all models and datasets. notably, cisc is effective across nearly all configurations, with some exceeding 67% cost reduction. we provide additional results in appendix c. in particular, confidence normalization improves cisc's performance. we drill down into the importance of the within-question confidence normalization step in cisc. in ## within-question confidence evaluation recent work demonstrated that verbal confidence methods significantly outperform p(true) in terms of calibration @xcite , which is the defacto approach to evaluate the quality of confidence measures. yet, perhaps surprisingly, cisc is more effective with p(true) than with verbal confidence methods ( confidence method cost reduction @ 10 p(true) (w/o normalization) 32% (14.8) p(true) (w/ normalization) 46% (18.6) sp (w/o normalization) 24% (13.1) sp (w/ normalization) 31% (14.6) verbal (w/o normalization) 20% (12.5) verbal (w/ normalization) 30% (14.4) to gain an intuition for the difference between within-question and between-question confidence evaluation, consider the following simple example. imagine a model m and a dataset with two types of questions: questions that m finds "easy" (e.g., answers correctly 95% of the time) and questions that m finds "hard" (e.g., answers correctly 5% of the time). consider a confidence measure that assigns every answer to an "easy" question a confidence of 0.95 and every answer to a hard question a confidence of 0.05. this confidence signal is useless for cisc, as it does not make any distinctions between answers to the same question. on the other hand, it scores well under existing metrics (e.g., it is perfectly calibrated). the above thought experiment shows that the fact that well-calibrated confidence scores can be derived from a model does not necessarily imply the model possesses a capacity to self-assess its own responses. to isolate this specific ability, we design a metric that measures whether the confidence scores can distinguish correct and incorrect responses to the same question: definition 6.1 (within-question discrimination). given a dataset of questions, for each question q, denote the sampled responses by r @xmath2, a i )} m @xmath3, and let r + q , r - q ⊆ r q be the subsets of correct and incorrect responses respectively. we evaluate the within-question discrimination (wqd) of a confidence method c : (r, a) → r as: that is, we compute the fraction of cases where the higher confidence response is indeed the correct response, out of pairs of responses to the same question (exactly one of which is correct). in our work, we use @xmath4.6). to emphasize the importance of within-question evaluation, we test if wqd is more predictive of cisc's success than standard between-question confidence metrics. we compare each confidence method from §4.1 in terms of: (i) standard metrics, such as ece @xcite and brier score @xcite , (ii) wqd, (iii) cisc performance at a budget of 10 samples. we follow previous work @xcite and report the standard metrics after applying temperature scaling @xcite , a technique that fits a single temperature parameter t to the model's confidences to minimize the negative log-likelihood on the data. we use ece-t and brier-t to denote the scaled scores. the results of this comparison, averaged across all datasets ( §4.2) and models ( §4.3), are summarized in the wqd metric prioritizes interpretability, focusing on the discrimination ability of the confidence scores irrespective of the relative magnitude of the confidence values c(r, a) and c(r , a ). however, examining the relationship between wqd and the confidence gap |c(r, a)c(r , a )| offers additional insights. figure 4 illustrates a near monotonic relationship: the within-question discrimination ability (indicated by color) smoothly increases with the confidence gap (x-axis). these findings suggest a fine-grained self-assessment mechanism, where even small differences in confidence scores reflect significant variations in the probability of a response being correct taken together, our findings provide a compelling evidence that llms indeed posses an intrinsic ability to reassess their own responses. ## qualitative analysis in §5 we showed that cisc has clear performance advantages over standard self-consistency, and argued that this suggests llms are capable of selfassessing their confidence in responses to the same question. to facilitate a better understanding of this phenomenon, we asked human evaluators to identify indicators of low-quality model responses (i.e., logical patterns that reduced the evaluators' confidence in the correctness of the llm response). our analysis revealed a strong correlation between the prevalence of these indicators and lower confidence scores assigned by the llm. ## related work confidence signals for llms. there is a long line of work on deriving confidence measures from llms. popular approaches use the agreement across multiple samples @xcite @xcite @xcite , the model's internal representations @xcite @xcite or directly prompting the model to verbalize its confidence @xcite . all papers in this line of work focused on fact-seeking tasks, so confidence is typically derived based on the final answer alone. to the best of our knowledge, our work is the first to apply these approaches to scoring the entire reasoning path. reasoning verification. while learned verifiers have been demonstrated to significantly improve performance on math word problems @xcite @xcite , the ability of llms to perform self -verification and self -correction is still heavily contested, with some works providing positive evidence for such capabilities @xcite @xcite @xcite and others arguing that the gains can mostly be attributed to clever prompt design, unfair baselines, data contamination and using overly simple tasks @xcite @xcite @xcite . this work contributes to this ongoing discussion by presenting multiple lines of evidence supporting llm self-verification. in particular, we demonstrate clear benefits from a simple confidence-based self-verification approach. ## discussion in this work we introduced cisc, a lightweight extension of self-consistency. across diverse models, datasets, and confidence extraction methods, cisc consistently outperformed self-consistency, reducing computation costs by over 40% on average. the performance gains achieved by using modelderived confidence scores provide a practical evidence that llms can effectively judge the quality of their own outputs, contributing to the ongoing debate on this topic @xcite . this is further strengthened by our qualitative evaluation, revealing significant agreement between model confidence and human assessments of response quality. complementing our investigation of llm selfassessment, we address the crucial aspect of evaluating confidence methods. traditional calibration metrics, which assess confidence across different questions, fail to capture a model's ability to distinguish between high and low quality responses to the same question. to overcome this, we introduce the within-question discrimination (wqd) metric and demonstrate its effectiveness. we encourage future research to explore the integration of model self-confidence into more sophisticated reasoning frameworks like tree of thoughts @xcite or graph of thoughts @xcite , believing that harnessing this inherent capability can further boost performance. another promising avenue is training models to produce more accurate intrinsic or verbal confidence @xcite , which would directly improve cisc and related methods. for instance, recent evidence suggests that a better signal can be derived from the model's internal states, even outperforming p(true) @xcite . conversely, cisc and wqd can be used to assess the impact of advancements in confidence generation. access to the model's probabilities. the preferred cisc approach calculates p(true) (as described in §4.1) by examining the model's assigned probability to the verbal confidence token. this method is available in both popular openweights frameworks (e.g., hugging-face (2024a)) and closed-weights frameworks (e.g., openai (2025)). however, this feature may not be universally available across all frameworks. ## ethics statement this work improves llm reasoning efficiency by introducing a new decoding strategy (cisc). while cisc itself introduces no new ethical issues, llms can perpetuate biases and have societal impacts. responsible llm development and deployment, including bias mitigation, are crucial.