Are VQA Systems RAD ? M easuring Robustness to Augmented Data with Focused Interventions (2021)

Abstract: Deep learning algorithms have shown promising results in visual question answering (VQA) tasks, but a more careful look reveals that they often do not understand the rich signal they are being fed with. To understand and better measure the generalization capabilities of VQA systems, we look at their robustness to counterfactually augmented data. Our proposed augmentations are designed to make a focused intervention on a specific property of the question such that the answer changes. Using these augmentations, we propose a new robustness measure, Robustness to Augmented Data (RAD), which measures the consistency of model predictions between original and augmented examples. Through extensive experimentation, we show that RAD, unlike classical accuracy measures, can quantify when state-of-the-art systems are not robust to counterfactuals. We find substantial failure cases which reveal that current VQA systems are still brittle. Finally, we connect between robustness and generalization, demonstrating the predictive power of RAD for performance on unseen augmentations.

## introduction in the task of visual question answering (vqa), given an image and a natural language question about the image, a system is required to answer the question accurately @xcite . while the accuracy of these systems appears to be constantly improving @xcite @xcite , they are sensitive to small perturbations in their input and seem overfitted to their training data @xcite . to address the problem of overfitting, the vqa-cp dataset was proposed @xcite . it is a reshuffling of the original vqa dataset, such that the distribution of answers per question type (e.g., "what color", "how many") differs between the train and test sets. using vqa-cp, @xcite demonstrated the poor out-of-distribution generalization of many vqa systems. while many models were subsequently designed to deal with the vqa-cp dataset @xcite @xcite , aiming to solve the out-of-distribution generalization problem in vqa, they were later demonstrated to overfit the unique properties of this dataset @xcite . moreover, no measures for robustness to distribution shifts have been proposed. in this work we propose a consistency-based measure that can indicate on the robustness of vqa models to distribution shifts. our robustness measure is based on counterfactual data augmentations (cads), which were shown useful for both training @xcite and evaluation @xcite . cads are aimed at manipulating a specific property while preserving all other information, allowing us to evaluate the robustness of the model to changes to this property. for example, consider transforming a "what color" question to a "yes/no" question, as depicted in figure 1 . the counterfactual reasoning for such a transformation is: "what would be the question if it had a yes/no answer?". while vqa models have seen many of both question types, their combination (yes/no questions about color) has been scarcely seen. if a model errs on such a combination, this suggests that to answer the original question correctly, the model uses a spurious signal such as the correlation between the appearance of the word "color" in the question and a particular color in the answer (e.g. here, color ⇒ white). further, this example shows that some models cannot even identify that they are being asked a "yes/no" question, distracted by the word "color" in the augmented question and answering "green". our robustness measure is named rad: robustness to (counterfactually) augmented data (section 2.1). rad receives (image, question, answer) triplets, each augmented with a triplet where the question and answer were manipulated. it measures the consistency of model predictions when changing a triplet to its augmentation, i.e., the robustness of the model to (counterfactual) augmentations. we show that using rad with focused interventions may uncover substantial weaknesses to specific phenomenon (section 3.2), namely, users are encouraged to precisely define their interventions such that they create counterfactual augmentations. as a result, pairing rad values with accuracy gives a better description of model behavior. in general, to effectively choose a model in complex tasks, complementary measures are required @xcite . thus, it is important to have interpretable measures that are widely applicable. note that in this work we manipulate only textual inputs -questions and answers, but rad can be applied to any dataset for which augmentations are available. in particular, exploring visual augmentations would be beneficial for the analysis of vqa systems. further, representation-level counterfactual augmentations are also valid, which is useful when generating meaningful counterfactual text is difficult @xcite . our augmentations (cads) are generated semiautomatically (section 2.2), allowing us to directly intervene on a property of choice through simple templates. as in the above example, our augmentations are based on compositions of two frequent properties in the data (e.g., "what color" and "yes/no" questions), while their combination is scarce. intuitively, we would expect a model with good generalization capacities to properly handle such augmentations. while this approach can promise coverage of only a subset of the examples in the vqa and vqa-cp datasets, it allows us to control the sources of the model's prediction errors. we conduct extensive experiments and report three key findings. first, for three datasets, vqa, vqa-cp, and visdial @xcite , models with seemingly similar accuracy are very different in terms of robustness, when considering rad with our cads (section 3). second, we show that rad with alternative augmentation methods, which prioritize coverage over focused intervention, cannot reveal the robustness differences. finally, we show that measuring robustness using rad with our cads predicts the accuracy of vqa models on unseen augmentations, establishing the connection between robustness to our controlled augmentations and generalization (section 4). ## robustness to counterfactuals in this section, we first present rad (section 2.1), which measures model consistency on questionanswer pairs and their augmented modifications. then, we describe our template-based cad generation approach (section 2.2), designed to provide control over the augmentation process. ## robustness with rad and cads in the following, we perform experiments to test the robustness of vqa models to augmentations. we describe the experimental setup, and evaluate vqav2, vqa-cpv2, visdial models, each on our augmentations and on other alternatives. foot_1 ## measuring generalization with rad to establish the connection between rad and generalization, we design experiments to demonstrate rad's added value in predicting model accuracy on unseen modified examples. concretely, we generate 45 butd (vqa) and lmh (vqa-cp) instances, differing by the distribution of question types observed during training (for each model instance we drop between 10% and 99% of each of the question types "what color", "how many" and "what kind" from its training data; see appendix e for exact implementation details). for each of the above models we calculate rad values and accuracies in the following manner. we split the validation set into two parts: d (features) and t (target). consider a pool of four original question sets that are taken from their corresponding modifications: y/n c, y/n hm, y/n wk, reph. then we have four possible configurations in which d is three sets from the pool and t is the remaining set. for each model and for each configuration, we compute model accuracy on d (accuracy(d)) and on the modifications of questions in t (the predicted variable y(t ) = accuracy(t )) which are modified with the target augmentation of the experiment. we also compute the rad values of the model on the modified questions in d which are generated using the other three augmentations (rad(d, d ), and rad(d , d)). then, we train a linear regression model using accuracy(d), rad(d, d ), and rad(d , d), trying to predict y(t ). we perform this experiment four times, each using a different configuration (different augmentation type as our target), and average across the configurations. ## conclusion we proposed rad, a new measure that penalizes models for inconsistent predictions over data augmentations. we used it to show that state-of-theart vqa models fail on cads that we would expect them to properly address. moreover, we have demonstrated the value of our cads by showing that alternative augmentation methods cannot identify robustness differences as effectively. finally, we have shown that rad is predictive of generalization to unseen augmentation types. we believe that the rad measure brings substantial value to model evaluation and consequently to model selection. it encourages the good practice of testing on augmented data, which was shown to uncover considerable model weaknesses in nlp @xcite . further, given visual augmentations, which we plan to explore in future work, or linguistic augmentations, rad is applicable to any classification task, providing researchers with meaningful indications of robustness.