The Role of Model Confidence on Bias Effects in Measured Uncertainties for Vision-Language Models (2025)

Abstract: With the growing adoption of Large Language Models (LLMs) for open-ended tasks, accurately assessing epistemic uncertainty, which reflects a model’s lack of knowledge, has become crucial to ensuring reliable outcomes. However, quantifying epistemic uncertainty in such tasks is challenging due to the presence of aleatoric uncertainty, which arises from multiple valid answers. While bias can introduce noise into epistemic uncertainty estimation, it may also reduce noise from aleatoric uncertainty. To investigate this trade-off, we conduct experiments on Visual Question Answering (VQA) tasks and find that mitigating prompt-introduced bias improves uncertainty quantification in GPT-4o. Building on prior work showing that LLMs tend to copy input information when model confidence is low, we further analyze how these prompt biases affect measured epistemic and aleatoric uncertainty across varying bias-free confidence levels with GPT-4o and Qwen2-VL. We find that all considered biases have greater effects in both uncertainties when bias-free model confidence is lower. Moreover, lower bias-free model confidence is associated with greater bias-induced underestimation of epistemic uncertainty, resulting in overconfident estimates, whereas it has no significant effect on the direction of bias effect in aleatoric uncertainty estimation. These distinct effects deepen our understanding of bias mitigation for uncertainty quantification and potentially inform the development of more advanced techniques.

## introduction robust quantification of large language models' (llms) confidence in their answers is vital for trust and safety in critical applications @xcite . without effective confidence ranking, accurate predictions may be 1 https://github.com/xinyiliu0227/uncertainty_ quantification_bias question: the eiffel tower is located in ___? ## related work uncertainty quantification with a single valid answer. traditional machine learning models treat total uncertainty as a measure of confidence when each question has a single valid answer @xcite @xcite . in single-choice classification problems like mmlu @xcite , studies @xcite show that llms are generally well-calibrated. reinforcement learning with human feedback (rlhf) has complicated uncertainty estimation @xcite . studies @xcite show that rlhf-trained llms often overestimate their confidence, raising concerns about the reliability of self-reported uncertainty. moreover, @xcite and @xcite found that self-reflection alone is insufficient for accurately assessing uncertainty. @xcite found that rephrasing and reordering prompts improve uncertainty quantification in single-answer settings. while their approach partially overlaps with ours in textual perturbation, we extend the analysis to multi-answer scenarios that involve aleatoric uncertainty and additional prompt-introduced biases, including image-based biases. crucially, we further examine how these biases affect the two uncertainties differently across varying confidence levels, offering a deeper understanding of the bias mitigation method. uncertainty quantification with a single semantic valid answer. prior work on llm uncertainty with aleatoric components mainly focuses on variability in generating semantically equivalent outputs, using benchmarks such as coqa @xcite , triviaqa @xcite , and ambigqa @xcite . proposed techniques include training auxiliary classifiers @xcite and leveraging internal model states @xcite @xcite , requiring additional training or model access. semantic equivalence has proven to be effective in reducing aleatoric uncertainty from phrasing variability without access to internal model states @xcite . research by @xcite observed that sample-based methods outperform single-inference approaches. building on these findings, we shift focus from phrasing variation to the challenge of multiple semantically valid answers, aiming to capture the distinct characteristics of epistemic and aleatoric uncertainty. uncertainty quantification with multiple semantic valid answers. uncertainty estimation becomes more complex with multiple semantically valid answers. @xcite tackled this by assuming larger models capture aleatoric uncertainty, while a smaller model head is trained to predict it. they also observed that llms are more likely to copy input information when epistemically uncertain compared to aleatorically uncertain. @xcite built on similar findings by using mutual information to estimate epistemic uncertainty, measuring answer distribution dependency on provided hints through iterative prompting. this growing body of work underscores the need to distinguish epistemic from aleatoric uncertainty with multiple semantically valid answers. we extend this by analyzing how biases introduced by relying on a single prompt affect these two measured uncertainties across different model confidences. in addition, @xcite preselected multilabel queries with high entropy (> 0.7) from the wordnet dataset @xcite , where llms achieve near-perfect performance. this approach results in instances with high total uncertainty but correct outputs, which may not reflect real-world data distributions. we use unfiltered datasets to better capture practical challenges. ahdritz et al. (2024) and yadkori et al. (2024) both found that llms are more likely to copy input information under high epistemic uncertainty but not high aleatoric uncertainty. inspired by these findings, we further analyze how these promptintroduced biases impact each type of uncertainty estimation separately, aiming to provide deeper insight. ## experiments prompt template you are given an image and a set of descriptions. your task is to evaluate each description and determine whether it is true based on the image. below are the descriptions: {label_0}: {option_0} {label_1}: {option_1} {label_2}: {option_2} {label_3}: {option_3} provide one index of the descriptions that are true, regardless of the number of descriptions that you believe are true. return your response as a single index without any additional explanations or text. here is an example format for your response: 0 use the provided format and structure for your response. dataset. we use the vl_checklist @xcite and crepe datasets @xcite , which contain numerous images with human-verified positive and negative descriptions. in contrast, some datasets @xcite contain image descriptions but lack multiple correct and incorrect ones per image, while others @xcite include only a limited number. we randomly select two correct and two incorrect descriptions and present them in a random order to ensure unbiased llm evaluation. these datasets evaluate more advanced model capabilities, compositional reasoning @xcite , compared to early multi-label datasets such as wordnet where current llms achieve nearperfect performance. to balance data coverage and budget, we create 1,000 questions from 1,000 images per dataset. evaluation metrics. we adopt the auroc metric for uncertainty quantification, following prior studies @xcite @xcite . auroc measures how well confidence scores rank instances with correct versus incorrect predictions and is robust to class imbalance @xcite . for further analysis, we use linear regression coefficients and p-values to examine how bias-free model confidence influences bias-induced changes in measured epistemic and aleatoric uncertainty. regression coefficients indicate the direction and magnitude of this relationship: a positive coefficient suggests greater bias effects at higher confidence levels, while a negative coefficient implies that higher confidence reduces bias impact. pvalues assess statistical significance, with low values (@xmath0.05) indicating a meaningful effect rather than one due to chance. ## conclusion removing three text-based biases and three imagebased biases improves uncertainty quantification in the presence of aleatoric uncertainty, as measured by auroc on gpt-4o. however, the improvement from image-based bias removal is smaller, likely due to existing image perturbation during training. while uncertainty decomposes into epistemic and aleatoric components, our findings show that lower model confidence amplifies bias effects on measured uncertainties, with a greater amplification observed on epistemic than on aleatoric uncertainty. moreover, while model confidence does not significantly affect the direction of bias-induced changes in measured aleatoric uncertainty, lower model confidence is associated with greater biasinduced underestimation of epistemic uncertainty (i.e. overconfidence). future work may leverage the distinct bias effects on these two types of uncertainty across varying confidence levels to develop more advanced techniques for disentangling them. in addition, our analysis could be extended to pure-text datasets once more challenging or less exposed multi-label benchmarks in llms pretraining become available.