Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization (2023)

Abstract: Vision-and-language (V&L) models pretrained on large-scale multimodal data have demonstrated strong performance on various tasks such as image captioning and visual question answering (VQA). The quality of such models is commonly assessed by measuring their performance on unseen data that typically comes from the same distribution as the training data. However, when evaluated under out-of-distribution (out-of-dataset) settings for VQA, we observe that these models exhibit poor generalization. We comprehensively evaluate two pretrained V&L models under different settings (i.e. classification and open-ended text generation) by conducting cross-dataset evaluations. We find that these models tend to learn to solve the benchmark, rather than learning the high-level skills required by the VQA task. We also find that in most cases generative models are less susceptible to shifts in data distribution compared to discriminative ones, and that multimodal pretraining is generally helpful for OOD generalization. Finally, we revisit assumptions underlying the use of automatic VQA evaluation metrics, and empirically show that their stringent nature repeatedly penalizes models for correct responses.

## introduction visual question answering (vqa) is the task of automatically answering natural language openended questions about images. tackling vqa involves multiple skills, such as language and visual understanding, integrating information between the two (vision and language) modalities, and commonsense and knowledge based reasoning. one of the goals of the vqa research has been fostering the development of systems that are able to answer any open-ended question about any image. this motivation has inspired a fruitful line of research in designing vqa benchmarks (e.g., @xcite @xcite @xcite and models (e.g., @xcite @xcite . in this work, we investigate if recent pretrained vqa models can indeed answer any open-ended question about images or if they are mostly suitable for answering questions from the vqa benchmarks they are optimized for. in other words, are models learning to solve the task or learning to solve the datasets? we believe the former is more aligned with the goal of building real-world vqa systems. to measure whether models learn to solve the task of vqa, we believe we need to examine their out-of-distribution (ood) generalization capabilities: how they perform on examples drawn from a distribution other than that of the training set. in this work, we extensively evaluate ood generalization of current pretrained v&l models by conducting cross-dataset evaluations (without any adaptation to the test domain). through our extensive experiments, we provide in-depth discussion on the following questions: domain adaptation in vqa. some studies @xcite @xcite have explored domain adaptation of vqa models from one vqa benchmark to another. our focus, instead, is on evaluating zero-shot crossbenchmark generalization without any adaptation. this allows us to assess the robustness of current models towards unforeseen distribution shifts. our work is similar to that of @xcite and @xcite , who study ood generalization in vision and text. zero-shot vqa with pretrained models. in an emerging line of research @xcite @xcite , large-scale pretrained unimodal models @xcite are repurposed to tackle vqa in zero-shot or few-shot fashion. while such zero-shot vqa evaluations are a better test of generalization than iid evaluations, our focus, differently, is on investigating whether models can generalize to unseen datasets upon being taught the task by showing examples from one dataset. moreover, this line of work does not focus on a thorough analysis of models in ood settings (which is hard to define for these models due to the massive amount of data they are pretrained on). ## experimental setup in this section, we present our framework to examine ood generalization in vqa. we examine two pretrained transformers across five benchmarks. ## out-of-distribution generalization we examine to what extent our models learn to solve a specific vqa benchmark by latching on dataset-specific correlations, as opposed to learning more general skills required in vqa. we fine-tune a pretrained model on the train split of one benchmark (e.g., gqa) and evaluate it on the validation split of a different one (e.g., vg). overall, we evaluate models by fine-tuning them on each benchmark and testing them against all benchmarks. if pretrained models are indeed learning the vqa skill, we expect to see a small drop in performance between the iid and ood settings. the results are presented in fig. 1 , with different evaluation benchmarks grouped on the x-axis. first, across all models and for each benchmark, we see a notable drop in the vqa accuracy from the iid to the ood setting. while such a drop might be anticipated, we found the extent of the drop surprising given the impressive performance of current pretrained vl models. for all models shown, the largest drops are observed when evaluating models on the vizwiz benchmark. moreover, even the smallest performance drop, which happens when fine-tuning models on vqav2 and evaluating them on vg, remains relatively large (i.e., 5.3 points for albef gen ). these results show that pretrained models are largely learning the fine-tuning benchmark without learning to solve the vqa task. second, we observe that fine-tuning on vqav2 results in the lowest drop in iid to ood performance across all conditions-the vqav2 bar (shown in blue in fig. 1 ) is the closet to the iid one for gqa, vg, and vizwiz. we conclude that fine-tuning on vqav2 yields a model that best generalizes to ood settings in our benchmarks. this result is not simply due to the size of the finetuning benchmark as vg is larger than vqav2. similarly, all the models achieve highest ood performance on vqav2. we conjecture that vqav2 is the most diverse benchmark of our selection. ## the effect of multimodal pretraining previous work has shown that pretraining on multimodal (i.e., image-text) data improves iid performance (e.g., @xcite ; here, we ask if multimodal pretraining can help in ood settings as well. we repeat the experiments in sec. 4 without pretraining our models on multimodal data; instead we train the models on the train split of one benchmark and test it on the validation split of another. fig. 3 shows the difference between the vqa accuracy of models with and without multimodal pretraining: each bar shows the gap between a bar in fig. 1 and the equivalent experiment without multimodal pretraining. we observe that multimodal pretraining is helpful in almost all conditions, since the majority of values displayed in fig. 3 are positive. pretraining is improving ood performance likely because it can reduce the gap between the train and ood test data by potentially exposing the model to a more diverse set of data points during pretraining. in our model mm pt vqav2 vqa-cp drop cf-vqa -53.6 63.5 9.9 vilbertdisc no 66.7 42.5 24.2 vilbertdisc yes 67.0 42.9 24.1 albefdisc no 64.0 40.1 23.9 albefdisc yes (4m) 70.0 44.4 25.6 albefdisc yes (14m) 70.3 45.2 25.1 albefgen no 61.4 36.6 24.8 albefgen yes (4m) 71.0 49.2 21.8 albefgen yes (14m) 72.1 49.6 22.5 we observe that multimodal pretraining is more effective for the generative albef compared to the discriminative albef (cf. the shaded and solid bar with the same color in fig. 3 bottom). for the vilbert model, we generally do not observe such a pattern-discriminative and generative models mostly show comparable improvements due to multimodal pretraining. we observe only small improvements when increasing the size of the multimodal pretraining dataset for the albef model (see fig. 8 in app. b for more details). ## evaluation on vqa-cp in this section, we evaluate the models 4 on the vqa under changing priors dataset (vqa-cp; @xcite ). this dataset is designed such that, for every question type, train and test splits have different prior distributions of answers. thus, models that overfit to answer priors in training data and lack sufficient visual grounding show poor generalization on the vqa-cp test set. for comparison, we also evaluate the performance of counterfactual vqa (cf-vqa; @xcite , a state-of-art method on vqa-cp, which does not use either the transformer architecture nor multi-4 albef and vilbertdisc (using the official codebase). modal pretraining. however, it explicitly tackles the language (i.e., question and answer) biases in vqa-cp. tab. 3 shows that for all the transformer-based models, there is a large drop in the performance (at least 22%) from vqav2 to vqa-cp. thus, in spite of advances in the transformer architecture and pretraining on diverse datasets, models are still overfitting to answer priors in the training data and lack sufficient visual grounding @xcite . however, the drop is much less for cf-vqa (10%), suggesting that incorporating inductive biases specific to the generalization problem (i.e., modeling language bias) helps more than the transformer architecture or scaling up the amount of pretraining data. we also observe that the drop from vqav2 to vqa-cp is often lower for the generative albef than the discriminative al-bef (except for albef without any multimodal pretraining). thus, generative models are more robust than the discriminative ones, especially when they are pretrained (similarly to the observations made in sec. 4.2). as for the effect of pretraining, for generative albef, pretraining helps reduce the drop from vqav2 to vqa-cp. however, for discriminative models, pretraining does not seem to help generalization (in fact, it worsen albef). ## qualitative analysis to dig deeper into the potential causes of the poor ood generalization of our pretrained models, we perform a qualitative study. to this end, we randomly sample and manually examine failure cases in top-30 answer classes with the highest performance drop when moving from iid to ood evaluation. we only focus on answer classes that are present in both the train and test splits, ensuring that performance drop is not due to the absence of answer classes in the training set. we report the top-5 classes that contribute the most to the drop in performance for each ood setting in tab. 11 (app. c). we notice that the following answer classes appear frequently across different ood settings: yes/no answers, directions (left/right), colors, and numbers. in the following, we discuss a few major potential causes for the poor ood generalization, and mention vilbert disc responses as examples in the discussion, although similar observations hold for other models. poor reasoning skills. models evaluated on gqa, but fine-tuned on another dataset, show the highest performance drop on classes such as "yes", "no", "right", "left", "top", and "bottom". for instance, vilbert disc fine-tuned on vqav2, and evaluated on gqa underperforms vilbert disc that has been both fine-tuned and evaluated on gqa by 24% for the answer class "no." upon qualitative examination, we find that for many of such failure cases, the gqa questions are more compositional and hence require more complex reasoning (e.g., "are there both bison and zebras in the image?", "is the cheese to the right or to the left of the empty plate?") than the questions for the same answer classes in other datasets (e.g., from vqav2 train set: "is the tv turned on?", "which hand is the man holding up?"). this study re-affirms previous findings that vqa models lack sufficient logical, spatial, and compositional reasoning skills @xcite but for the more recent, pretrained transformer models. overfitting to the answer priors. previous studies have shown that vqa models tend to be biased towards the prior distribution of answers in the training set (per question type) @xcite . we find that this limitation exists in the more recent pretrained models as well, and it is especially hurtful in the ood settings because the priors need not be the same across train and test sets, unlike in the iid settings. for instance, vilbert disc fine-tuned on vqav2 predicts "2" for a lot questions with target answer "1" in the vg test set. similarly, sometimes vilbert disc fine-tuned on vg incorrectly predicts "helmet" for vqav2 test questions such as "what is the skateboarder wearing to protect his head?", "what protective gear is he wearing?" when the skateboarder is not wearing anything. this indicates that the model is relying on answer priors rather than visual grounding. our experimental results on vqa-cp (sec. 6) directly quantify the extent of such limitations in current models. overfitting to the question format. we observe instances of models failing to correctly answer questions when the format of the questions changes between the fine-tuning and test sets. for instance, questions about "chair" in the vqav2 fine-tuning set are mostly of the form "what is . . . sitting on?" whereas in the gqa test set, they are mostly of the form "what kind of furniture is . . . ?". thus, the "chair" class accuracy of vilbert disc fine-tuned on vqav2 drops from 48% when tested on vqav2 to 38% on the gqa test set. similarly, vilbert disc fine-tuned on gqa fails terribly for "dog" and "cat" classes on the vg test set (accuracy drops of 47% and 43% respectively between gqa-gqa (fine-tuned on gqa, tested on gqa) and gqa-vg). gqa questions are mostly of the form "what animal . . . ?" or "what kind of animal . . . ?" whereas vg questions often do not mention the word "animal" and are of the form "who is . . . ?" or "what is . . . ?" (e.g., "who is holding the frisbee?", "what is on the leash?"). to the best of our knowledge, no previous work has reported such behavior of vqa models (i.e., they tend to overfit to the question format). finally, we observe cases where correct model responses are evaluated as incorrect by the vqa evaluation metric, as such responses differ from the ground-truth answers. in the next section, we provide examples of such cases and examine the impact of stringent evaluation metric on poor ood generalization by engaging human raters to evaluate responses. ## human evaluation in our qualitative study, we observed that the stringent nature of the standard vqa evaluation metrics (i.e., performing string matching of model responses with a small set of ground-truth answers) repeatedly penalizes models for correct responses because those responses do not exist in the set of ground-truth answers (fig. 5 ). for example, the evaluation metric fails to take into account differences (between model response and ground-truth) due to specificity of the answers (e.g., "on table" vs. "table", "pizza slices" vs. "pizza"), synonyms, and different interpretations of the question (e.g., fig. 5 right). in this section, we aim to quantify how robust standard vqa metrics are by performing human human evaluation yields notably higher accuracies than the automatic evaluation. this is shown in fig. 6 , where the increase can be up to 33.5% when moving from automatic to human evaluation. 5 this implies the current automatic metrics miss out on a lot of correct responses due to their stringent nature. interestingly, this increase in model accuracy from automatic to human evaluation is higher for vilbert gen than vil-bert disc for all the benchmarks. this is expected because the generative model is more likely to produce longer, more varied answers, which might not be awarded using automatic metric but are still correct responses. moreover, human evaluation helps ood settings more than the iid settings for most of the benchmarks (e.g., gqa, vqav2). this is also expected, because in the ood settings, a model might not learn the format of the test answer ("on table" vs. "table ", "clear vs. sunny") from the train set (unlike in the iid settings) and hence it is more likely to be penalized by the automatic accuracy metric. thus, we conclude that the currently used accuracy metrics for vqa are not robust, especially for generative models and ood evaluation settings. hence, to more accurately evaluate the goodness of our models, we need to develop better evaluation metrics for vqa. even after human evaluation, models still exhibit poor ood generalization. although human evaluation improves the models' accuracies and more so for the ood than the iid settings, we observe that the models' performance in ood settings is still worse compared to that of iid set-tings, albeit with reduced margin (see app. d for quantitative results). we also note that while vil-bert disc usually outperformed vilbert gen with the automatic evaluation, vilbert gen outperforms vilbert disc for all the test sets under human evaluation. this reinforces the observations in sec. 4.2 regarding stronger ood generalization of generative models over discriminative ones. ## conclusion in this study, we show that, despite their impressive performance when evaluated on test data drawn from the same distribution as the training data, recent v&l models perform poorly in outof-distribution (ood) settings. we conclude that these models learn to solve specific benchmarks as opposed to the skill of visual question answering (vqa). interestingly, in most cases, we observe that the generative models are more robust to ood generalization compared to the discriminative ones. moreover, pretraining the models on large imagetext data often helps in ood generalization. our results also highlight the importance of human evaluation for a more accurate assessment of model performance: we find that the current vqa automatic metrics miss out on a notable number of correct model responses. human evaluation is especially important as the community is shifting towards generative vqa models which, unlike discriminative ones, can produce answers that go beyond those seen in a training/fine-tuning dataset. finally, to make progress towards more capable models, we need more rigorous evaluation protocols that shed light on models' strengths and short-comings. we believe testing models in ood settings is a step towards this direction as it helps evaluate models for general skills required to solve the task as opposed to benchmark-specific correlations.