Uncertainty-Aware Answer Selection for Improved Reasoning in Multi- LLM Systems (2025)

Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities, yet selecting the most reliable response from multiple LLMs remains a challenge, particularly in resource-constrained settings. Existing approaches often depend on costly external verifiers, human evaluators, or self-consistency techniques that require multiple samples from a single model. While multi-LLM systems produce more diverse responses than single models and thus have greater potential, they often underperform compared to single LLM self-consistency. In this work, we propose a calibrated log-likelihood-based selection framework to improve multi-LLM performance. Our approach leverages uncertainty estimation to identify the most confident response while minimizing inference costs. We show that our method outperforms majority voting and exceeds self-consistency performance when using a large number of model calls. Through extensive experiments, we demonstrate improvements of approx. 4%, 3%, and 5% on GSM8K, MMLU, and ARC, respectively, when applying uncertainty-aware selection to multi-LLM systems.

## introduction large language models (llms) have achieved remarkable advancements, with state-of-the-art (sota) models now approaching or even surpassing human performance. with rapid advancements, an array of pre-trained llms and llm apis have become readily accessible, each exhibiting distinct strengths across specialized tasks. for instance, models like gpt-deep research excel in analytical and research-oriented tasks, whereas gpt-o3 is tailored towards programming and coding. similarly, there are domain-specific expert models optimized for tasks in physics, algebra, and other fields. this proliferation raises an important question: how can we optimally leverage the diverse capabilities of multiple specialized llms to produce 1 code: https://github.com/aakriti05/multi-llm-uncertainty the most accurate and reliable answers without incurring additional training costs (given the substantial resource demands associated with training) ? this challenge becomes especially critical when high-quality external judges or evaluators-capable of assessing answers across multiple specialized domains-are unavailable, expensive, or even infeasible due to the superhuman capabilities of modern llms @xcite . existing approaches have tried exploring this but (challenge 1) suffer from reliance on external information such as: (1) external verification models @xcite , (2) human or llm-based judges @xcite @xcite , or (3) reward models trained explicitly for response ranking. these methods involve significant resource overhead and are infeasible when existing models reach superhuman performance. additional major challenge is that existing methods from single-llm literature like majority voting with self-consistency @xcite , selfreflection @xcite , or metricbased selection @xcite (e.g., perplexity, self-certainty) (challenge 2) demand extensive sampling or (challenge 3) are infeasible for multi-llm contexts due to inherent differences in outputs from a multi-llm system. (challenge 4) a simplistic multi-llm adaptation of self-consistency, which selects the most frequently generated answer, does not effectively exploit intermodel reasoning and hence misses potential performance gains @xcite . to address the above challenges, in this paper, we propose a novel and computationally efficient method method to aggregate responses from diverse llms and systematically select the best answer without relying on external verifiers (challenge 1), without requiring extensive sampling (challenge 2), and effectively use diverse multi-llms (challenge 3, 4). specifically, we introduce uncertainty estimation-based answer selection from multi-llm systems, which employs a calibrated log-likelihood-based selection metric that implicitly leverages the inherent knowledge and confidence of the given models, further improving response accuracy and reducing computational overhead. our method is built on the hypothesis that a model (or expert) trained on a specific example will exhibit high confidence (i.e., low uncertainty), while models unfamiliar with the example will show higher uncertainty. assuming the training data is correct, the most confident expert is likely to provide the correct answer and our aim to find that expert. thus, the primary contributions are: it is light weight and simple as it does not require external verifiers, reward models, human or llm judges. 3. our method demonstrates strong empirical performance across both debate (multiround multi-llm discussions) and nondebate (best-of-n with multiple llms) settings on gsm8k, mmlu (6 subsets), and arc datasets showing improvement of 4%, 3% and 5% respectively. it achieves greater improvements in multi-llm settings compared to single llm, further reinforcing the potential of a diverse-answer-generating multi-llm system to boost overall performance. ## uncertainty-aware answer selection we now describe our approach for aggregating and selecting optimal answers in a multi-llm system. while methods for optimal answer selection-commonly used in best-of-n strategy-are well-studied in single-llm contexts @xcite , directly extending them to multi-llm environments poses significant challenges. specifically, diverse llms (1) produce outputs varying significantly in format and length, (2) differ widely in their model weights and architectures, and (3) yield uncertainty scores that are typically incomparable across models due to a lack of calibration. to address these issues, we propose a calibrated log-likelihood metric, which integrates smoothly with the interactive multi-llm debate as well noninteractive multi-llm best-of-n setup. we first detail our proposed metric and then provide theoretical insights and analysis to illustrate its enhanced performance. we also provide discussion on its computational efficiency. figure 1 illustrates how our calibrated selection approach surpasses traditional majority-voting strategies for a multi-llm setup. (note: a non-debate best-of-n setting is equivalent to first round of debate setting.). ## experimental setup and results baselines. we evaluate our proposed metric for multi-llm setups by comparing it against commonly used majority voting with random tiebreaking @xcite , considering both interactive multi-llm debate and non-interactive best-of-n settings. we utilize three models: qwen2.5-7b-instruct, ministral-8b-instruct-2410, and llama-3.1-8b-instruct. the evaluation datasets are gsm8k @xcite , mmlu @xcite (comprising 8 subsets), and arc @xcite . further experimental details are provided in appendix c. additionally, we compare against single-llm best-of-n sampling using the best-performing model (qwen2.5-7b-instruct). this comparison allows us to assess the relative advantage provided by multi-llm systems when using our proposed approach. to ensure fairness, we maintain an equal number of total llm calls across all evaluated scenarios. ## conclusion we introduced a calibrated log-likelihood-based selection framework to enhance multi-llm systems. by leveraging uncertainty estimation, our method selects the most confident response, reducing reliance on costly external verifiers and extensive sampling. our approach outperforms random selection and majority voting with same model calls, making it a cost-effective solution. additionally, we highlight the benefits of diverse model reasoning in multi-llm debate. future work can explore adaptive sampling and extend our method to broader reasoning tasks.