UB ench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions (2025)

Abstract: Despite recent progress in systematic evaluation frameworks, benchmarking the uncertainty of large language models (LLMs) remains a highly challenging task. Existing methods for benchmarking the uncertainty of LLMs face three key challenges: the need for internal model access, additional training, or high computational costs. This is particularly unfavorable for closed-source models. To this end, we introduce UBench, a new benchmark for evaluating the uncertainty of LLMs. Unlike other benchmarks, UBench is based on confidence intervals. It encompasses 11,978 multiple-choice questions spanning knowledge, language, understanding, and reasoning capabilities. Based on this, we conduct extensive experiments. This includes comparisons with other advanced uncertainty estimation methods, the assessment of the uncertainty of 20 LLMs, and an exploration of the effects of Chain-of-Thought (CoT) prompts, role-playing (RP) prompts, and temperature on model uncertainty. Our analysis reveals several crucial insights: 1) Our confidence interval-based methods are highly effective for uncertainty quantification; 2) Regarding uncertainty, outstanding open-source models show competitive performance versus closed-source models; 3) CoT and RP prompts present potential ways to improve model reliability, while the influence of temperature changes follows no universal rule. Our implementation is available at https://github.com/Cyno2232/UBENCH.

## introduction in recent years, significant progress has been made in the development of large language models (llms), including chatgpt @xcite , llama @xcite , qwen @xcite , etc. these advancements have what is the best drug of choice for acute heart attacks? e%3cmxcell%20id%3d%222%22%20value%3d%22%22%20style%3d%22line%3bstrokewidth%3d2%3bhtml%3d1%3bshadow%3d0%3bsketch%3d1%3bfillcolor%3d%234d4d4d%3bgradientcolor%3dnone%3b%22%20vertex%3d%221%22%20paren candidate answer: aspirin (true) what do you think is the likelihood that the answer is correct? 80% 50% llm1 llm2 👍 what is the best drug of choice for acute heart attacks? e%3cmxcell%20id%3d%222%22%20value%3d%22%22%20style%3d%22line%3bstrokewidth%3d2%3bhtml%3d1%3bshadow%3d0%3bsketch%3d1%3bfillcolor%3d%234d4d4d%3bgradientcolor%3dnone%3b%22%20vertex%3d%221%22%20paren candidate answer: antibiotics (false) what do you think is the likelihood that the answer is correct? 10% 60% llm1 llm2 👍 not only piqued the intense interest of both the industrial and academic sectors @xcite , but have also spurred a wave of exploration and research. in this fast-paced development, comprehensively evaluating llms' capabilities is essential. this evaluation helps us understand llms better, improve their performance, and apply them effectively in real-world scenarios, unlocking their potential for innovation across different fields. much effort has been devoted to establishing accurate, authentic, and equitable evaluation systems for llms, such as c-eval @xcite and mt-bench @xcite . however, these benchmarks often focus solely on the performance of models across various tasks, neglecting the credibility of their answers. in some situations, models may not be certain about the confidence level associated with their provided answers, potentially leading to misunderstandings or even harm. specifically, llms are prone to generating misinformation without warning, which may mani- method close? single? close&single? xiong et al. (2023) ✓ ✗ ✗ ye et al. (2024) ✓ ✓ ✗ ubench (ours) ✓ ✓ ✓ to address these challenges, we introduce ubench, a new benchmark. distinct from previous efforts, it is founded on confidence intervals, 1 please refer to §5 for details. based on this, we compare the uncertainty quantification method based on confidence intervals with other powerful methods. we also evaluate 20 state-of-the-art open-source and closed-source models that cover several series and sizes. in addition, we explore the factors that potentially influence model uncertainty, including chain-of-thought (cot) @xcite prompts, roleplaying (rp) @xcite prompts, and the impact of temperature on model uncertainty. our main contributions and findings are summarised below: ## the bench the overall construction and evaluation process of ubench is shown in figure 2 . overall, ubench includes 4 categories, including tasks such as reading comprehension, qa, mathematical reasoning, and more. we provide detailed information on categories, data construction, and the design of prompts. ## experiments 4.1 experimental settings evaluated models. based on ubench, we test 20 popular llms, covering a wide range of open and closed source llms, as shown in ## case study since our methods are heuristic in nature, we attempt to illustrate the advantages of our approach relative to the method by @xcite (hereafter referred to as the hybrid approach) by example. tables 7 and 8 show the responses of mistral-7b to a question from the cosmos qa dataset based on these two methods. due to the length of the text, we only present three answers generated by the hybrid approach. firstly, our method is characterized by shorter questions and answers, reflecting its conciseness and low resource consumption. content-wise, with our prompt strategy (refer to ## conclusion focusing on the assessment of reliability in llms, we present ubench, a new benchmark for uncertainty estimation in llms based on multiple choice questions. the benchmark consists of 12k ten-choice questions in four categories: knowledge, language, understanding, and reasoning. comparative experimental results show that our confidence interval-based method outperforms other sota uncertainty estimation methods. additionally, we assess the reliability of 20 mainstream llms, which include both open and closed sources, on this benchmark. we reveal that even the most advanced llms still exhibit low reliability in their predictions, especially in extreme cases, which pose potential risks. therefore, it is necessary to incorporate uncertainty estimation into the evaluation of llms. further exploratory analysis reveals that the incorporation of cot and role-playing prompt methods generally benefits llms in demonstrating stronger reliability, whereas changes in temperature have varying effects on different llms. additionally, we conduct a case study to demonstrate the effectiveness of our proposed method. we hope that this study will play an important role in the further development and application of llms.