Human-Adversarial Visual Question Answering

1 Introduction . Visual question answering ( VQA ) is widely recognized as an important evaluation task for vision and language research . Besides direct applications such as helping the visually impaired or multimodal content understanding on the web , it offers a mechanism for probing machine understanding of images via natural language queries . Making progress on VQA requires bringing together different subfields in AI – combining advances from natural language processing ( NLP ) and computer vision together with those in multimodal fusion – making it an exciting task in AI research . Over the years , the performance of VQA models has started to plateau on the popular VQA v2 dataset [ 20 ] – approaching inter-human agreement – as evidenced by Fig . 1 . This raises important questions for the field : To what extent have we solved the problem ? If we haven ’ t , what are we still missing ? How good are we really ? An intriguing method for investigating these questions is dynamic data collection [ 29 ] , where human annotators and state-of-the-art models are put “ in the loop ” together to collect data adversarially . Annotators are tasked with and rewarded for finding model-fooling examples , which are then verified by other humans . The easier it is to find such examples , the worse the model ’ s performance can be said to be . The collected data can be used to “ stress test ” current VQA models and serve as the next iteration of the VQA benchmark helping drive further progress . The commonly used VQA dataset [ 20 ] was collected by instructing annotators to “ ask a question about this scene that [ a ] smart robot probably can not answer ” [ 4 ] . One way of thinking about our proposed human-adversarial data collection is that it explicitly ensures that the questions can not be answered by today ’ s “ smartest ” models . ∗Equal contribution . Correspondence to advqa @ fb.com . †Work done as an intern at Facebook AI Research . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . This work is , to the best of our knowledge , the first to apply this human-adversarial approach to an image and language multimodal problem . We introduce Adversarial VQA ( AdVQA ) , a large evaluation dataset of 46,807 examples in total , all of which fooled the VQA 2020 challenge winner , the MoViE+MCAN [ 45 ] model . We evaluate a wide range of existing VQA models on AdVQA and find that their performance is significantly lower than on the commonly used VQA v2 dataset [ 20 ] ( see Table 1 ) . Furthermore , we conduct an extensive analysis of AdVQA characteristics , and contrast with the VQA v2 dataset . We hope that this new benchmark can help advance the state of the art by shedding important light on the current model shortcomings . Our findings suggest that there is still considerable room for continued improvement , with much more work remaining to be done . 2 Related Work . Stress testing VQA . Several attempts exist for stress testing VQA models . Some examine to what extent VQA models ’ predictions are grounded in the image content , as opposed to them relying primarily on language biases learned from the training dataset . The widely used VQA v2 dataset [ 20 ] was one attempt at this . The dataset contains pairs of similar images that have different answers to the same question , rendering a language-based prior inadequate . VQA under changing priors ( VQA-CP ) [ 2 ] is a more stringent test where the linguistic prior not only is weak in the test set , but is adversarial relative to the training dataset ( i.e. , answers that are popular for a question type during training are rarer in the test set and vice versa ) . Other complementary datasets test model robustness against certain conditions , such as different logicial compositions [ 18 ] , question rephrasings [ 52 ] , suprious correlations [ 1 ] , and consistency in answering sub-questions [ 51 ] . Datasets which benchmark specific capabilities in VQA models also exist , such as reading and reasoning about text in images [ 57 ] , leveraging external knowledge [ 44 ] , and spatial [ 26 ] or compositional reasoning [ 28 ] . Other vision and language datasets , such as Hateful Memes [ 31 ] , have also tried to make sure the task involves true multimodal reasoning , as opposed to any of the individual modalities sufficing for arriving at the correct label . Compared to the previous work , with AdVQA , we aim to have a more holistic and general dataset that pushes the boundaries of the state-of-the-art on VQA . Saturating prior work . The VQA v2 [ 20 ] challenge has been running yearly since 2016 and has seen tremendous progress , as can be seen in Figure 1 . From simple LSTM [ 23 ] and VGGNet [ 54 ] fused models to more advanced fusion techniques ( MCB [ 16 ] , Pythia [ 27 ] ) to better object detectors ( MCAN [ 61 ] , MoViE+MCAN [ 45 ] , BUTD [ 3 ] ) to transformers [ 59 , 56 ] and self-supervised pretraining ( MCAN [ 61 ] , UNIMO [ 38 ] ) , the community has brought the performance of models on the VQA v2 dataset close to human accuracy , thus starting to saturate progress on dataset . As we show through AdVQA , however , saturation is far from achieved on the overall task , and hence there is a need for new datasets to continue benchmarking progress in vision and language reasoning . natural language inference [ 46 ] , question answering [ 5 , 50 ] , sentiment analysis [ 49 ] , hate speech detection [ 60 ] , next video-and-language event prediction [ 35 ] and dialogue safety [ 15 ] . To the best of our knowledge , ours is the first work to explore an image and language multimodal human-adversarial benchmark.1 3 AdVQA . The aim of this work is to investigate state of the art VQA model performance via human-adversarial data collection . In this human-and-model-in-the-loop paradigm , human annotators are tasked with finding examples that fool the model . In this case , annotators are shown an image and are tasked with asking difficult but valid questions that the model answers incorrectly . We collected our dataset using Dynabench [ 29 ] , a platform for dynamic adversarial data collection and benchmarking . For details on our labeling user interface , please refer to the supplementary material . The VQA v2 dataset is based on COCO [ 39 ] images . We collected adversarial questions on both val2017 COCO images and testdev2015 COCO images . We then random sampled the collected set down to 2 questions per val2017 COCO images ( 10,000 ) and 1 question per testdev2015 COCO images ( 36,807 ) . The random sampling was done to balance the annotator ’ s contributions . The data collection involves three phrases ( Figure 2 ) : First , in the question collection phase , Amazon Mechanical Turk workers interact with a state-of-the-art VQA model and self-report image-question pairs that the model failed to answer correctly . Second , in the question validation phase , a new set 1Concurrent work [ 36 ] jointly available with AdVQA at https : //adversarialvqa.org also focuses on the same problem but uses images from a different source . of workers validates whether the answers provided by the VQA model for the image-question pairs collected in the first phrase are indeed incorrect . Finally , in the answer collection phase , we collect 10 ground truth answers for the image-question pairs validated in the second phase . In what follows , we provide further details for each of these steps . 3.1 Question Collection . In this phase , annotators on Amazon MTurk are shown an image and are tasked with asking questions about this image . The interface has a state-of-the-art VQA model in the loop . For each question the annotator asks , the model produces an answer . Annotators are asked to come up with questions that fool the model . Annotators are done ( i.e. , can submit the HIT ) once they successfully fooled the model or after they tried a minimum of 10 questions ( whichever occurs first ) . To account for the fact that it may be hard to think of many questions for some images , annotators are given the option to skip an image after providing non-fooling three questions . We use the VQA Challenge 2020 winner – MoViE+MCAN [ 45 ] – trained on the COCO 2017 train set as the model in the loop . This is to ensure that the adversarial benchmark we collect is challenging for the current state-of-the-art in VQA . The interface provides a digital magnifier to zoom into an area of the image when the mouse hovers over it , allowing workers to examine the image closely if necessary . See Figure 3b . Keyboard shortcuts were provided for convenience . 3.2 Question Validation . Note that in the question collection phase , annotators self-report when they have identified a question that is incorrectly answered by the model . Whether or not this was actually the case is verified in the question validation phase . Two different annotators are shown the image , question , and answer predicted by the model , and asked whether the model ’ s answer is correct . As an additional quality control , the annotator is also asked whether the question is “ valid ” . A valid question is one where the image is necessary and sufficient to answer the question . Examples of invalid questions are : “ What is the capital of USA ? ” ( does not need an image ) or “ What is the person doing ? ” when the image does not contain any people or where there are multiple people doing multiple things . If the two validators disagree , a third annotator is used to break the tie . Examples are added to the dataset only if at least two annotators agree that the question is valid . 3.3 Answer Collection . In the final stage of data collection , following [ 20 , 57 ] , we collect 10 answers per question providing instructions similar to those used in [ 20 ] making sure that no annotator sees the same question twice . In addition , as an extra step of caution , to filter bad questions that might have passed through last two stages and to account for ambiguity that can be present in questions , we allow annotators to select “ unanswerable ” as an answer . Further , to ensure superior data quality , we occasionally provide annotators with hand-crafted questions for which we know the non-ambiguous single true answer , as a means to identify and filter out annotators providing poor quality responses . 3.4 Human-Adversarial Annotation Statistics . The statistics for the first two stages ( question collection and validation ) are shown in Table 2 . We find that annotators took about 5 tries and on average around 4 minutes to find a model-fooling example . For computing the model error rate , we can look at the instances where annotators claimed they had fooled the model , and where annotators were verified by other annotators to have fooled the model . The latter has been argued to be a particularly good metric for measuring model performance [ 29 ] . We also further confirm that the model was indeed fooled by running the model-in-the-loop on a subset of examples in which human agreement was 100 % and found the accuracy to be 0.15 % on val , and 0.18 % on test splits of AdVQA respectively . Interestingly , the “ claimed ” model error rate , based on self-reported model-fooling questions , is similar to that of text-based adversarial question answering , which was at 44.0 % for RoBERTa and 47.1 % for BERT [ 5 ] . The validated error rate for our task is much higher than e.g . for ANLI [ 46 ] , which was 9.52 % overall , suggesting that fooling models is a lot easier for VQA than for NLI . It appears to be not too difficult for annotators to find examples that the model fails to predict correctly .

Summary: This paper proposes AdVQA, a new (adversarial) test set for the Visual Question Answering.  Annotators and a SOTA model are put in the loop to make sure the model can be fooled. Experiments show that other VQA models tend to perform poorly on AdVQA, and a comprehensive analysis discusses the questions and answers that are in this dataset (compared with VQA and TextVQA).