Self-Ensemble: Mitigating Confidence Distortion for Large Language Models (2025)

Abstract: Although Large Language Models (LLMs) perform well in general fields, they exhibit a **confidence distortion problem** on multi-choice question-answering (MCQA), particularly as the number of answer choices increases. Specifically, on MCQA with many choices, LLMs suffer from under-confidence in correct predictions and over-confidence in incorrect ones, leading to a substantially degraded performance. To solve this problem, we propose Self-Ensemble in this work. Our method splits the choices into several groups and ensembles LLM predictions across these groups to reach a final decision. The advantage of Self-Ensemble is its plug-and-play nature, where it can be integrated into existing LLM architecture based on a designed attention mask and positional encoding, without requiring labeled datasets for parameter tuning. Experimental results on three LLMs and datasets demonstrate that Self-Ensemble comprehensively addresses the confidence distortion problem of LLMs, outperforming standard inference as well as baseline methods.

## introduction large language models (llms) have exhibited remarkable performance in processing natural language information, such as the llama @xcite , mistral @xcite , and deepseek @xcite . among various nlp tasks, multi-choice question-answering (mcqa) stands out as a standard and challenging benchmark to evaluate the reliability and reasoning ability of llms @xcite . it can significantly reduce the hallucinations by limiting the answer to a predefined set of choice aligned with human knowledge @xcite . while advanced llms perform well on standard mcqa benchmarks, such as mmlu @xcite , mathqa @xcite , and gsm-8k @xcite , they still face challenges when it comes to numerous choices that are closely related @xcite . in particular, we identify a critical problem: llms suffer from confidence distortion on mcqa tasks based on a comprehensive benchmark analysis. specifically, llm's confidence in the correct choice tends to degrade, while its confidence in incorrect choices increases as the number of choices grows. this problem grows even more pronounced in the presence of additional noisy or partially relevant choices, thereby increasing the unreliable generations and erroneous predictions. to calibrate llm predictions on mcqa, existing work focuses on post-processing llm predictions to better align confidence with correctness. one common method is the vector scaling @xcite . it adjusts the model outputs by applying a learned scaling vector and bias vector to the logits before applying the softmax function, which can smooth or sharpen the probabilities to better align with correctness. another approach is dirichlet calibration @xcite . it generalizes llm's prediction probability on each choice using a dirichlet distribution to maximize the likelihood function on a validation set. however, a major limitation of these methods is their reliance on a labeled validation dataset to optimize calibration parameters. in practical scenarios, collecting highquality validation data can be both labor-intensive and costly, which limits these techniques. to overcome this limitation, we introduce self-ensemble for llm calibration in this work. unlike existing work, self-ensemble calibrates llm predictions without relying on a labeled validation dataset for parameter tuning. specifically, self-ensemble mitigates the confidence distortion problem through a divide-and-conquer process: it splits the choices into several groups of choice subsets; then ensembles llm predictions on each group to achieve the final decision. to plug into existing llm architecture, we design attention masking and positional re-encoding mechanisms that enable the divide-and-conquer process to be executed internally during inference. figure 1 shows the comprehensive performance of self-ensemble on three datasets and llms, where our method shows significant improvement over standard inference and baseline methods by addressing the confidence distortion problem. to summarize, the contributions of this work are as follows: ## confidence distortion and self-ensemble we demonstrate the confidence distortion problem for llms and introduce our self-ensemble method to overcome this problem. ## plugging self-ensemble into llms we propose integrating self-ensemble into existing llm architectures to enable intrinsic selfensemble processing during inference. as illustrated in equations ( 1 ) and (2), self-ensemble requires multiple forward passes through the llm, which are for efficient self-ensemble processing, we design specialized attention masks and positional encoding, such that these multiple calls of llm can be executed in a single forward pass over the con- ## experiment in this experiment, we conduct experiments to evaluate self-ensemble, aiming to answer the following research questions: rq1: can self-ensemble improve on llm's accuracy on many-choice qa? rq2: can self-ensemble mitigate the confidence distortion problems for llms? rq3: how does self-ensemble reshape the scaling of model accuracy with respect to parameter size? ## conclusion in this work, we demonstrate the confidence distortion problem of llms generally on mcqa, particularly in the many-choice setting. to solve this problem, we propose self-ensemble by decomposing a many-choice problem into several fewchoice problems, and aggregating the intermediate results into the final solution. we further integrate self-ensemble with existing llm architecture, enabling an intrinsic self-ensemble process for llm inference. experimental results across various llms and datasets show that self-ensemble can effectively overcome this problem, improving confidence in correct answers and reducing confidence in incorrect answers. this enables self-ensemble to significantly improve the accuracy of llms in mcqa by 8% on average.