Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling (2025)

Abstract: Self-consistency mitigates hallucinations in Large Language Models (LLMs) by sampling multiple reasoning paths, but it lacks a systematic approach to determine the optimal number of samples or select the most faithful rationale. To address this limitation, we introduce Reasoning-Aware Self-Consistency (RASC), a novel framework that enhances sampling efficiency and reasoning faithfulness by dynamically evaluating both outputs and rationales. RASC assesses the quality of reasoning and the consistency of answers for each generated sample, using these assessments to guide early stopping decisions and rationale selection. The framework employs criteria-based stopping and weighted majority voting, enabling more informed choices on when to halt sampling and which rationale to select. Our comprehensive experiments across diverse question-answering datasets demonstrate that RASC outperforms existing methods, reducing sample usage by approximately 70% while maintaining accuracy. Moreover, RASC facilitates the selection of high-fidelity rationales, thereby improving the faithfulness of LLM outputs. Our approach effectively addresses the efficiency-accuracy trade-off in LLM reasoning tasks, offering a new perspective for more nuanced, faithful, and effective utilization of LLMs in resource-constrained environments.

## introduction large language models (llms) have demonstrated remarkable complex reasoning capabilities in various domains @xcite , particularly when employing chainof-thought (cot) prompting @xcite which allows llms to generate reasoning paths (rps) before answers. building upon this, @xcite introduced self-consistency (sc), a decoding strategy that significantly enhances reasoning performance by sampling multiple rps and marginalizing over these samples. this approach has substantially improved llm's performance across various reasoning domains @xcite . however, sc's effectiveness is tied to the number of reasoning paths sampled, creating a critical trade-off between performance and computational cost. while the original paper show that sampling more paths (e.g., 40) generally leads to better performance, there's no systematic method to determine the optimal number of samples for any given task. this ambiguity often results in over-sampling, leading to excessive computational demands @xcite . thus, we need more efficient methods that can maintain sc's reasoning benefits while reducing its computational burden. recent research has introduced early-stopping mechanisms to address the computational costs of self-consistency. adaptive consistency (ac) @xcite employs incremental sampling, terminating when a clear majority emerges based on a predefined probability distribution, while early stopping self-consistency (esc) @xcite segments the preset sample size into windows and stops sampling when answers within a window achieve uniformity. although these methods effectively reduce computational demands, they fail to address two critical limitations in the original self-consistency approach: (1) they treat all samples equally for sampling decisions, basing the stopping criteria solely on the consistency of final answers, with this uniform treatment extending to the use of majority voting; (2) these approaches lack of a formal mechanism to distinguish and prioritize the most faithful and high-quality rationale among the selected samples, which can be critical in domains that require precise explanations such as healthcare and science @xcite : comparison between self consistency and reasoning-aware self consistency (rasc) methods for a reasoning problem. the top half illustrates the standard self consistency approach, generating multiple reasoning paths and using majority voting for the final answer. the bottom half demonstrates rasc's improvements: it assigns scores based on the qualities of both answers and reasoning paths, effectively handling incorrect or irrelevant responses. rasc's resampling and stop condition mechanisms optimize sampling, enhancing efficiency and maintaining accuracy. moreover, rasc enables the selection of the most faithful rationale among generated samples, improving the quality and reliability of the reasoning process in complex tasks @xcite . to address these limitations while preserving the benefits of self-consistency, we propose reasoning-aware self-consistency (rasc), a framework that enhances the original sc approach. as illustrated in fig. 1 , our method assesses both the rationales across samples and the consistency of final answers, which enables more nuanced and efficient sampling. rasc is motivated by the principle that higher-quality rationale often leads to better llm reasoning @xcite . by leveraging this insight, rasc prioritizes robust reasoning in decision-making and enables early stopping when a consistent pattern of high-quality rp and consistency of answers is observed. this approach not only improves efficiency but also allows rasc to identify the most faithful and high-quality chains of thought among selected samples. our approach significantly outperforms existing methods across 10 datasets spanning commonsense, mathematical, and symbolic reasoning, reducing sample usage by 60%-80% while maintaining or improving accuracy. furthermore, experiments demonstrate rasc's ability to select high-fidelity chain-of-thought reasoning, as well as its robust-ness across various datasets, prompting strategies, and multiple base llm models, highlighting its adaptability to different efficiency-accuracy tradeoffs and application needs. this combination of efficient sampling, high-fidelity reasoning selection, and broad applicability positions rasc as a versatile and powerful tool for enhancing the reasoning capabilities of large language models. in summary, our key contributions are as follows: ## reasoning-aware self-consistency self-consistency (sc) and its variants enhance question-answering accuracy in large language models (llms) by generating multiple answers, but often overlook reasoning processes, leading to inefficient sampling. we propose reasoning-aware self-consistency (rasc), a framework that optimizes sampling efficiency while maintaining accuracy by evaluating both reasoning paths and answers. rasc introduces a sufficiency scoring function that combines reasoning quality and answer consistency features, mapped to a continuous score via an optimized classifier. the framework employs a dynamic sampling process with a buffer of high-quality samples, stopping when a predefined capacity is reached. finally, rasc uses weighted majority voting based on the scores to select the final answer and most faithful reasoning path. this approach integrates reasoning evaluation into the self-consistency process, offering a more faithful and computationally efficient method for complex question-answering tasks in llms. ## conclusion through our investigation of sampling-based llm reasoning, we discovered that the quality of intermediate reasoning paths serves as a crucial early indicator for efficient sampling decisions. building on this insight, we developed rasc, a framework that strategically incorporates reasoning path evaluation into the sampling process. our approach not only achieved a 60-80% reduction in required samples while maintaining accuracy. by introducing principled methods for assessing reasoning quality and selecting the most faithful rationale from generated samples, rasc provides a systematic solution to the challenge of balancing computational efficiency with reasoning reliability in llms. the framework's success across diverse reasoning tasks demonstrates that focusing on the quality of intermediate reasoning, rather than just final answers, offers a promising direction for developing more efficient and trustworthy llm reasoning systems.