Selectively Answering Visual Questions (2024)

Abstract: Recently, large multi-modal models (LMMs) have emerged with the capacity to perform vision tasks such as captioning and visual question answering (VQA) with unprecedented accuracy. Applications such as helping the blind or visually impaired have a critical need for precise answers. It is specially important for models to be well calibrated and be able to quantify their uncertainty in order to selectively decide when to answer and when to abstain or ask for clarifications. We perform the first indepth analysis of calibration methods and metrics for VQA with in-context learning LMMs. Studying VQA on two answerability benchmarks, we show that the likelihood score of visually grounded models is better calibrated than in their text-only counterparts for in-context learning, where sampling based methods are generally superior, but no clear winner arises. We propose AVG BLEU, a calibration score combining the benefits of both sampling and likelihood methods across modalities.

heading: 1 Introduction
text: Reliable Visual Question Answering (VQA) systems should provide a confidence estimate of their own predictions. This introspective skill allows users to decide when to trust or double check its outputs, or lets the system itself selectively decide when to gather more information in order to provide accurate responses.
While this problem has been studied extensively for classification models (Guo et al., 2017) and more recently also for text generation (Cole et al., 2023a), VQA systems have been under-explored. The combination of multiple input modalities can introduce different sources of uncertainty due to incorrectly framed or focused images. This is specially true when images are taken by people with no or limited vision, as is the case for the VizWizVQA dataset (Gurari et al., 2018). Furthermore the questions in VizWiz-VQA are spoken and there-
fore more conversational and contextual, which can introduce additional denotational uncertainty due to ambiguity when the question is under-specified because of shared common ground (Cole et al., 2023a). The multiple crowd-worker annotations in the dataset allow us also to identify confusing or unanswerable questions. It also allows us to see cases when even the unique answer to an unambiguous question can have multiple equivalent surface forms, or be expressed with different levels of specificity, for example in the month and year, or the full date for an expiration date.
Our goal is to study which methods from the literature can help calibrate state-of-the-art VQA systems, and build a selective VQA setup that can trust its confidence estimates when answering. We
4219
see this as a necessary step towards systems that can be used in real life scenarios. Figure 1 introduces three of the methods used in Cole et al. (2023a) as well as our proposed AVG BLEU (inspired in Wan et al. (2023)) on an example from the VizWiz dataset (Gurari et al., 2018).
Our contributions can be summarized as follows: (a) Through the use of several metrics, we study how the calibration methods from text QA perform when applied in two VQA answerability benchmarks and find that likelihood based scoring is better for LMMs than LLMs, without a clear winner. (b) We propose a new sampling based method that tackles limitations and improves all metrics significantly. Concretely, it improves coverage at 80% accuracy by 5 points for the best LMM and coverage at 70% accuracy by 8 points for the best LLM.

heading: 2 Related Work
text: Our paper studies VQA systems with unanswerable question through the VizWiz-VQA (Gurari et al., 2018) and UNK-VQA (Guo et al., 2023) datasets. We show some examples in Figure 2. Through the introduction of unanswerable responses from the annotators, Gurari et al. study a possible form of calibration as binary detection of such images. This setting is however limited in that a model (or person) can assess a question as answerable and be uncertain about whether the answer is a good one, as observed by Chen et al. (2023a).
On the topic of estimating uncertainty, several approaches appear in the literature. Collier et al. (2021) incorporate a latent trainable variable for the covariance matrix among classes. The use of additional binary classifiers— so-called selectors— was investigated for text QA (Kamath et al., 2020; Desai and Durrett, 2020; Jiang et al., 2021). This approach was extended to VQA by Whitehead et al. (2022) and Dancette et al. (2023). They focus on fine-tuned models and the use of selectors. We argue that, with the increasing availability and adop-
tion of LMMs, it is important to study calibration in zero and few-shot scenarios, which pose unique challenges for evaluation (Maynez et al., 2023).
Cole et al. (2023a) study different samplingbased methods to calibrate LLMs in a zero-shot fashion. It focuses on entity based QA, where there is little differences in the surface form of equivalent answers. Due to the open-ended problem, the answers in VizWiz-VQA and UNK-VQA can vary, as illustrated in Figure 1. This variation needs to be considered when designing schemes to measure uncertainty when evaluating multiple samples.

heading: 3 Methods
text: A QA system is said to be well-calibrated when each prediction has a confidence score which can help assess how often it is correct. Formally, the paradigm of selective QA uses a scoring function s that attaches to each QA pair (q, a) a numeric score s(q, a). The score can be compared with a threshold τ so that the system answers—aka triggers—when s(q, a) > τ or abstains otherwise. As shown in Figure 1, various scoring methods can be used. We present first the methods from Cole et al. (2023a) for text-only QA, and then our proposal.
Likelihood The likelihood-based calibration uses the output language model score p(a|q) computed using the chain rule: p(a|q) =∏n
i=1 p(ti|t1, · · · , ti−1, q) where the ti are the tokens that form the answer a.
Sampling Repetition Based on Wilcox (1973)’s Variation Ratio we compute the frequency of the most sampled output divided by the total number of samples, which coincides with the probability of the mode of the empirical distribution. When more samples agree with each other, the answer can be considered to be more trustworthy.
Sampling Diversity Computed as 1− #unique#samples , it is inversely proportional to the number of distinct samples and is zero if all samples are different.
AVG BLEU We propose an approach to grading the similarity among model answers that relies on averaging a measure of semantic similarity. Since there are multiple answers, we consider the average among all the possible pairs as an estimate of the diameter or dispersion of the full set. This is inspired by the measure proposed by Wan et al. (2023) for LLMs. Instead of ROUGE (Lin, 2004), which is not sensitive to
character n-grams, we use BLEU (Papineni et al., 2002) and compute the pairwise weighted average 1 k ∑ i,j p(ai|q)BLEU(ai, aj) over the set of k distinct predicted answers. Other similarity metrics are studied in Section 4.2. We fix the distance between a proper answer and unanswerable to 0.

heading: 4 Experiments
text: In our experiments, we use the validation split of VizWiz-VQA with 4k instances. Each question has up to 10 crowd-worker answers. Examples can be seen in Figure 2. We consider a question answerable if at least one crowd-worker annotated it as such. This corresponds to 75% of the questions.
We also include the validation set of UNKVQA (Guo et al., 2023), consisting of 1K examples, synthetically constructed by modifying the images or text from VQA v2 (Goyal et al., 2017).
On the modeling side, we chose stateof-the-art LMMs LLaVA (Liu et al., 2023), Flamingo (Alayrac et al., 2022), and BLIP-2 (Li et al., 2023). To compare the calibration of LMMs with LLMs, we leverage the human written captions for VizWiz images provided by Gurari et al. (2020). We chose to use gold captions to control the additional errors and uncertainty which could arise from model written ones. Nevertheless we run a study in Section 4.1 using captions from PaLI-X (Chen et al., 2023b) and found that similar results hold. We evaluate state-of-the-art LLMs PaLM-2 (Anil et al., 2023) and Falcon (Almazrouei et al., 2023). We sample ten responses with a tem-
perature of 0.7. We consider the model to trigger if the most likely answer (greedy) does not contain the sequence “unanswerable”. The full prompts and BLIP-2 results can be seen in Appendix A. To simplify the analysis while staying consistent with the official metric defined in Antol et al. (2015), we consider a model answer correct if it matches at least one (instead of three) of the gold answers.
We evaluate the different scoring methods defined in the previous section with intrinsic and extrinsic metrics that we introduce below, over the set of examples where each model triggers a response.
Expected Calibration Error (ECE) Predictions are bucketed into ten same-sized bins, ranked by the confidence. We compute the mean absolute value of the distance between the average confidence score and the accuracy of predictions in each bin, averaged across all non empty bins. This intrinsic evaluation interprets a confidence score to represent a probability, so it computes the difference in the predicted probability of being correct from the observed probability of being correct. ECE is therefore noisy and sensitive to re-scaling.
ROC-AUC Area under the receiver operating characteristic curve measures the ability as a binary classifier for correct and incorrect predictions by integrating over the curve of the rates of true and false positives. It can be interpreted as the probability that the score of a correct output chosen at random is higher than that of an incorrect output.
Coverage@Acc While ECE and ROC-AUC assess absolute and relative calibration respectively, we want a metric closely aligned with selective QA. Therefore, we compute the triggering rate the model can achieve if forced to maintain a certain accuracy among the responses where it triggers. Formally, C@Acc is the maximum coverage such that the accuracy on the C% of most-confident predictions is at least Acc%. For example, if
C@70 = 30, then the system is 70% or more accurate on the top 30% most-confident predictions. We take 100 coverage to mean not all of the examples, but where the model produced an answer.
We show the overall results for VizWiz in Table 1, including the baseline triggering rate and accuracy before incorporating calibration signals. The strength of the models is validated by comparing to fine-tuned models on the task, where a FlanT5 (Base) (Chung et al., 2022) model trained on all the available captions is 74.2% accurate when it triggers, and a PaLI3-5 (Chen et al., 2023c) is 66.3% accurate when it triggers. Details about the fine-tuning experiments are explained in Appendix B. For LLMs, the findings of Cole et al. (2023a) hold, and likelihood is the worst calibration metric, repetition and diversity are superior to it. But, surprisingly, the picture changes for LMMs where we can see that repetition and diversity methods are not reliable. In contrast, AVG BLEU works well across the board, matching or improving the best metric in most cases. The same holds for UNKVQA in Table 2. We omit C@Acc for accuracy below the model overall accuracy since it is 100.
We also evaluated model-based measures of semantic similarity in Section 4.2 beyond BLEU, but interestingly, we found that BLEU has strong performance while requiring negligible compute.

heading: 4.1 Human vs. model written captions
text: In order to restrict the noise and sources of uncertainty when evaluating LLMs for the VQA task, we opted to use gold captions. However, it can be argued that a more realistic setup would require the use of automated captions. With that goal, we evaluated the two LLMs on captions generated by a fine-tuned PaLI-X model (Chen et al., 2023b), which reported state-of-the-art results for the task when using an additional OCR module as input. Interestingly we observe improved results when using the model written captions (higher accuracy and triggering). Upon inspection of examples we see that the generated captions tend to be shorter and more concise, focusing on salient elements of the photo, which leads on average to better downstream answers for VizWiz where many questions are asking about salient objects (ex: “What is this?”).
As is the case for the manual captions, sampling based methods generally outperform likelihood by a large margin. The exception is on the ECE metric, which is expected given that it relies on the align-
ment between the scores and probabilities, can be fixed post-hoc, and is arguably the least practical one. We also observe that, AVG BLEU is able to be better or comparable to best result in every case.

heading: 4.2 Replacing BLEU for dense similarity
text: While BLEU has the advantage of fast execution, it can fail to capture more nuanced forms of similarities among answers. We benchmarked two additional answer similarity metrics, BEM (Bulian et al., 2022) and BLEURT (Sellam et al., 2020). Perhaps surprisingly, the results over the various tasks show very little effect, as seen in Table 4. We speculate that the variability among 10 crowdworker answers diminishes the possible improvements, but it is possible that other VQA tasks could benefit from this approach making the additional compute cost worth the improved calibration.

heading: 5 Analysis and Discussion
text: How can we enhance the reliability of a VQA system? Based on the results presented in this work and previous literature in the effect of calibration scores in user facing applications (Zhang et al., 2020) we can make the following recommendations when implementing a VQA system:
(a) Add a confidence score to the responses based on AVG BLEU that lets the users of VQA systems decide whether they can trust the system response or whether the question at hand is critical and needs a higher confidence in which case they can ask another person.
(b) Automated captioning and LLMs on those captions can be useful to answer visual questions
but likelihood is not a good calibration score in that case (Cole et al., 2023b).
Why are LMMs better calibrated? We speculate that this has to do with the inability for diversity and repetition metrics to capture similarity beyond text exact match and the grounding effect of the multi-modality acting as a regularizer and distributing the likelihood of possible answers in a more meaningful way. We leave further experiments in this direction to future work.
The language modality—e.g. gold captions—is a human generated signal with high information density and communicative intent (Rambow and Walker, 1994). Other non-communicative natural signal modalities have heavy redundancy, noise, and low information density, as observed in other multi-modal tasks (Wei et al., 2023). We can speculate that this produces a regularization effect— meaning the model training objective forces it to spread its bets among many answers. Grounding (Harnad, 1990) the question to sections of the image becomes particularly hard for images taken by visually impaired people in VizWizVQA due to occlusions or lack of framing, introducing an additional source of uncertainty (Chen et al., 2022).
Can we measure accuracy better? In the previous experiments, we perform exact-match (EM) criteria to classify whether the generated answer was correct or not, following the standard VQA accuracy evaluation (Antol et al., 2015).
This approach led to instances where the model, when presented with a question such as “What is my computer screen showing?” produced the response “A system restore”. Despite the relevance of the generated answer, it was erroneously classified as incorrect due to the stringent acceptance criteria that required matching exactly the annotated answers, such as “system restore”, “system restore message”, “system restore pop up”.
To address this limitation, we replicated the experiments incorporating three similarity techniques
for classifying correct answers, chosen based on their widespread adoption and use in comparable experiments (Risch et al., 2021). A similarity threshold was hand picked upon inspecting 20 instances. The results show variations in accuracy. Despite these differences, confidence metrics remained stable across the board, with variations of 8% at most, as shown in Table 6. Further details and experiments can be found in Appendix C.
What errors do models make? We manually inspected 272 errors made by the best two models on VizWiz-VQA presented in the previous section, one LLM and one LMM, with the goal of getting insights on their weaknesses. We find the following three most frequent type of errors. First, the LLM hallucinates more than the LMM and hallucinated answers in LLMs often trigger a response. Second, the gold caption of the LLM may not include the answer to the question. Third, the answer is true but not useful for visually impaired people. Appendix D illustrates each of them.

heading: 6 Conclusions
text: We studied scoring methods to assess the confidence in the predictions of a VQA system in the VizWiz and UNK-VQA datasets. We observed that, as shown previously, sampling based methods might present an advantage over likelihood based estimates in text models, but the picture changes for text-image models. Open ended question with non-entity answers present additional challenges for sampling methods, so the equivalence among possible sampled answers needs to be incorporated as well. Our AVG BLEU score is able to capture the spread of possible samples in a soft way and accomplishes the best results on the calibration evaluations we measured, combining the advantages of sampling and likelihood-based methods.
Limitations
This study was conducted using several models over two datasets, which might limit the generalization of its conclusions. No other VQA datasets to our knowledge account for unanswerable questions in the same rich manner. For that reason, more analysis and datasets should be created to further research in this space and we hope our results serve as an initial step in that direction.
The use of manually curated captions may introduce a source of bias. Our analysis of error patterns reveals a substantial advantage for the LLMs due to its direct access to these gold captions. Addressing this limitation, future work could explore the feasibility and implications of working with automatically generated captions, allowing for a more realistic assessment in real-world scenarios.
Ethical Considerations
In this paper we explore calibration methods for VQA models. This is an under-researched area which is particularly relevant for the visually impaired community for at least two reasons. First, quantifying the uncertainty in the day-to-day questions from this community can help them decide when to trust an automated VQA and when to ask another person. Second, our findings show that metrics in previous work have unstable performances for different kinds of models when evaluated over questions with high uncertainty such as those from visually impaired people.
The use of rating systems and visual question answering in the context of people with low or no vision carries risks since users of such systems cannot easily verify their results, and erroneous answers can lead to serious harm. In addition to accidental failures, models such as those studied can be attacked with the use of adversarial examples (Alzantot et al., 2018) by malicious actors with the aim of taking advantage of a vulnerable population.
For these reasons, before offering VQA systems for this population, it is essential to conduct detailed studies of the use cases and their possible failures, with emphasis on risk mitigation and respect of vulnerable populations (Le Ferrand et al., 2022). These studies should be designed collaboratively with members of the target group to minimize biases about how said system can be used.

heading: Acknowledgments
text: We would like to thank Francesco Piccinno, Srini Narayanan, Luciana Ferrer, and the anonymous reviewers for their time, constructive feedback, useful comments and suggestions about this work. This work used computational resources from both Google DeepMind and CCAD – Universidad Nacional de Córdoba1, which are part of SNCAD – MinCyT, República Argentina.

heading: A Few-shot Experiments
text: We show in Figure 3 and Figure 4 the 4-shot prompts used for LLMs and LMMs respectively, with representative examples chosen from the training set. The experiments were conducted in Tesla T4 GPUs with 16GB of VRAM and it took less than 4 hours for each model we executed. The results from BLIP-2 (Li et al., 2023) are shown in Table 5. We used the official models checkpoints released in https://hf.co. For PaLM2, we used the publicly available API2 and the Flamingo predictions on the dataset were shared by the authors.

heading: B Fine-tuning experiments
text: The Flan-T5 base model was trained using 3 NVIDIA GTX 1080Ti GPUs (GP102, 11 GiB
2https://cloud.google.com/vertex-ai
GDDR5) connected via PCIe 3.0 16x, with 32-bit floating-point precision. The fine-tuning process extended over 4k global steps until convergence was achieved, employing a batch size of 16. The learning rate schedule uses a linear warmup of 800 steps to 1e-5, followed by cosine decay to 0. Model optimization uses Adafactor (Shazeer and Stern, 2018). A sequence of 256 tokens was used for input encoding, while the output decoding utilized an 8-token sequence. The predictions for a fine-tuned PaLI-3 were shared by the authors and we refer to the paper for the training configuration. We show the results of the experiments in Table 7.

heading: C Classifying correct answers
text: As discussed in Section 5, the experiments showcased in this paper used exact-match criteria to classify an answer as correct: only if the output generated by the model was exactly one of the accepted answers it counted as correct. In this section we describe the replication of these experiments, employing identical methodologies, introducing four distinct techniques for classifying correct answers. The selection of these techniques was based on their widespread adoption and their utilization in a comparable experiment outlined in (Risch et al., 2021). We perform this experiment for the LLaVA 13B and PaLM 2 Bison with similar results.
Each technique is briefly described as follows: Exact Match (EM) returns True only if the model’s answer is exactly one of the accepted answers from the dataset. For Cosine Similarity we used allMiniLM-L6-v2 (Wang et al., 2020) as an embedding model due to its widespread use. BEM, as proposed in (Bulian et al., 2022), is a fine-tuned BERT model to classify answer equivalence using the SQuAD Dataset (Rajpurkar et al., 2016).
In Table 8 we compare the performance of the ECE method throughout the four tested techniques to classify correct answers. We can observe how both the accuracy and the value of ECE change, however the best performant method remains constant. These findings suggest that the results pre-
sented in this paper are robust across various answer classification techniques.
D Insights from error analysis
In examining 272 errors from the top-performing language models LLaVA 13B which is an LMM and PaLM 2 Bison which is a LLM, we aimed to identify weaknesses specific to the needs of the vulnerable population. Three recurrent errors were observed which accounted for most errors. Firstly, the LLM tends to hallucinate more than the LMM, with these hallucinated responses influencing the final output. For example Figure 5 illustrates how PaLM 2 Bison hallucinates multiple serial numbers that appear neither in the image nor in the captions. This is less frequent for LLaVA 13B which refrains from answering in most of these cases.
Secondly, Figure 6 shows an example of a question that is not possible to answer with the information in the gold captions. This is to be expected a caption cannot include all the information present
in an image. This is a limitation of the methodology of using captions instead of images for VQA.
Lastly, the third kind of frequent error observed are answers that are factually correct, but are not useful for individuals with visual impairments. A example of this kind of error is shown in Figure 7.
These findings shed light on critical areas for improvement in models designed for this particular user group, showing that well calibrated systems are important in this domain.

heading: Future Work
text: Addressing this limitation, future work could explore the feasibility and implications of working with automatically generated captions, allowing for a more realistic assessment in real-world scenarios. Ethical Considerations
In this paper we explore calibration methods for VQA models. This is an under-researched area which is particularly relevant for the visually impaired community for at least two reasons. First, quantifying the uncertainty in the day-to-day questions from this community can help them decide when to trust an automated VQA and when to ask another person. Second, our findings show that metrics in previous work have unstable performances for different kinds of models when evaluated over questions with high uncertainty such as those from visually impaired people. The use of rating systems and visual question answering in the context of people with low or no vision carries risks since users of such systems cannot easily verify their results, and erroneous answers can lead to serious harm. In addition to accidental failures, models such as those studied can be attacked with the use of adversarial examples (Alzantot et al. , 2018) by malicious actors with the aim of taking advantage of a vulnerable population. For these reasons, before offering VQA systems for this population, it is essential to conduct detailed studies of the use cases and their possible failures, with emphasis on risk mitigation and respect of vulnerable populations (Le Ferrand et al. , 2022). These studies should be designed collaboratively with members of the target group to minimize biases about how said system can be used