Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models (2025)

Abstract: To leverage the full potential of Large Language Models (LLMs) it is crucial to have some information on their answers’ uncertainty. This means that the model has to be able to quantify how certain it is in the correctness of a given response. Bad uncertainty estimates can lead to overconfident wrong answers undermining trust in these models. Quite a lot of research has been done on language models that work with text inputs and provide text outputs. Still, since the visual capabilities have been added to these models recently, there has not been much progress on the uncertainty of Visual Language Models (VLMs). We tested three state-of-the-art VLMs on corrupted image data. We found that the severity of the corruption negatively impacted the models’ ability to estimate their uncertainty and the models also showed overconfidence in most of the experiments.

## introduction llm-based ai assistants can help us with a wide variety of tasks. the responses generated by these models sound convincing and correct most of the time but it has been shown that they can confidently generate incorrect or even nonsensical answers. in the field of llms, this is known as hallucinations @xcite . currently, the biggest problem with chatgpt-like ai assistants is that they will generate real and hallucinated answers with the same degree of confidence, as seen in figure 1 . as there have already been examples of algorithmic biases with serious consequences in real-world applications of machine learning models @xcite , with the rapid evolution of llms, it is likely that they will have increasingly more responsibilities in practical applications. there are multiple risks involved with deploying these models in highstakes decisions in the real world @xcite . we have to ensure that these models are well-calibrated, meaning that the model's confidence in a response accurately predicts the likelihood of the answer being correct. ## related work in this paper we estimate the model uncertainty by prompting. there is no consensus on the best method to elicit reliable confidence scores from llms. this is a problem as different methods yield different confidence scores so it is hard to compare the calibration of different models. @xcite examined various methods to extract confidence scores from the examined models and found that for models trained with reinforcement learning with human feedback (rlhf) @xcite , the verbalized confidence is better calibrated than other methods that for instance, estimate internal token probabilities by sampling. this finding makes verbalized uncertainty a viable option to estimate uncertainty in vlms. even though estimating the model's confidence by prompting has some drawbacks, verbalized uncertainty is getting more attention and has also been examined by @xcite . their work builds on @xcite as they investigate different prompting methods like chain-of-thought reasoning or top-k. different prompting strategies yielded similar results: llms exhibit overconfidence and the majority of the models' confidence scores fall within the 80-100 range. this paper strengthens their findings and tests their "vanilla" prompting strategy on increasingly corrupted images. since uncertainty estimation is not often incorporated in computer vision applications (valdenegro-toro, 2021), there has not been much research published on the topic. the only paper that examined uncertainty estimation in vlms is @xcite . they also used verbalized confidence estimation on visual questionanswering tasks and found that the models were poorly calibrated, showing severe overconfidence. we build on their research by introducing increasingly corrupted images in the dataset. most research @xcite @xcite has been focused on models applied in classification problems or when it comes to question answering, multiple choice or true/false questions. the main issue with this is that their methods for eliciting confidence scores are not applicable to state-of-theart vlms. while users would like to enjoy the benefits of well-calibrated models, they should not have to deal with the inner workings of the system and instead receive well-calibrated confidence scores in a verbalized form. we tested the models on more complex, open-ended questions which mimics the usage of these models in the real world. we combined the ideas from @xcite to test the models on increasingly corrupted images and @xcite to extend the research into vlms where internal token probabilities are not available. with this paper, we aim to bridge the gap between uncertainty quantification on standard neural networks and vlms. this is important due to the rapid advancement of vlms, and the lack of research on their uncertainty calibration. ## methods we tested the vlms on three different datasets using three different corruptions. the specific details of the experiments, datasets, the used corruption techniques and the evaluation procedure are explained below. ## results here, we report the results of the three experiments. for visual question answering, we were mainly interested in how the ece is affected by the increased corruption severity. since the results seemed to increase linearly, we attempted to fit linear regression lines to the data points and calculated the coefficient of determination r 2 value to test the explanatory power of the linear models. we were also interested in whether there is any connection between a model's refusal rates and their performance. especially at higher severity levels, refusing to answer can improve the model's performance. refusing to answer is not as useful as a correct answer, but it is better than an incorrect guess. in the counting experiment, we only examined the change in accuracy as the corruption severity increased. ## discussion overall we found that increased corruption severity had a negative impact on the three examined models' accuracy and calibration. when the corruption level gradually became higher and the models' accuracy started to decrease, it was not accompanied by decreasing confidence scores. we also found that models that refuse to answer at a higher rate can achieve better accuracy and ece scores. our other main finding is that models are generally overconfident in their responses and output high confidence scores in most of their responses. this overconfidence was present in all three experiments but it was the most severe in the counting problems. interpretation of results. our findings about overconfident models and their high confidence scores are in line with @xcite and @xcite . both of them found that the majority of the confidence scores of llms and vlms fall within the [80, 100] range. while @xcite looked at the performance of llms in different reasoning tasks (commonsense, arithmetic, symbolic), groot and valdenegro-toro (2024) also examined the performance of vlms in visual question answering tasks. our results show that this characteristic of vlms persists when they are tested on corrupted images. higher corruption levels worsening calibration was also found by @xcite who tested different neural network architectures designed for image classification. we show that their findings can be extended to the realm of vlms. as mentioned before, one possible explanation for this overconfidence is the rlhf fine-tuning of these models. it rewards answers that sound more confident so the model learns to express its responses using confident language which influences the confidence scores of verbalized uncertainty. examining the the number of cases where the models refused to answer, we found that higher refusal rates can help the model's calibration as it is nearly impossible to give a correct answer to some highly distorted images. especially with gaussian noise, there were times when the models were fooled by the noise and output completely unrelated answers to the images. this happened less with jpeg compression as it was a less severe corruption than the other two. in case of defocus blur, the models were more likely to recognise the heavy blurring effect on the image and refuse to respond to the question. images corrupted with gaussian noise were less likely to be recognised as corrupted and in some cases they were even confused with pointillistic paintings which is a painting technique from the late 19 th century using small colourful dots that form an image when viewed from a distance. for some examples of model responses, we refer to appendix f. ## conclusions and future work the key conclusions that we obtained from this work are the following: from these results, we can see that there are many things that can be improved when it comes to the calibration of vlms. in the current state of things, users are often presented with confident wrong answers which undermine the trust in these models. this paper contributes to the research in uncertainty estimation of vlms and points out the shortcomings of these models with respect to their calibration. better-calibrated models would be beneficial to millions of users as these models are already widely used by the general public. as the models output high confidence scores, the lower confidence bins were underrepresented in the calibration plots. this issue could be solved with more images in the datasets, but there were some limitations on the number of images that the models could be tested on. for the easy visual question answering experiment, there were tens of thousands more images available from the dataset by @xcite and @xcite . however, all of the images were used from the jus dataset, which put a limit on the number of images in the hard vqa experiment and the counting experiment. since a correct answer to a question could be phrased in multiple ways, the answers had to be manually checked, which made the data-gathering process time-consuming. the used apis also had a limit on the number of requests per minute, which prevented large-scale testing. one way to automate the check for the correctness of the answers could be to use an llm to check the semantic equivalence of the correct reference answer and the response provided by the model. this was not a suitable approach in this paper, as these methods still have limited correlation (spearman rank correlation ρ < 0.7) with manual assessment @xcite . another way could be to use better prompts that restrict the model to one or two-word answers that are easier to check automatically but that would put a limit on the complexity of the tested questions. lastly, we found that some models are more likely to refuse to answer for images with severe corruptions. these samples can therefore not be included in the answer, which can decrease error rate and decrease the recorded miscalibration. this may be considered acceptable, as not giving an answer may be preferable over a random guess, but it is not as good as giving a correct answer. there are methods that attempt to minimize the refusal rate and get more correct answers @xcite , but those were not considered in this study.