Human-Adversarial Visual Question Answering

1 Introduction . Visual question answering ( VQA ) is widely recognized as an important evaluation task for vision and language research . Besides direct applications such as helping the visually impaired or multimodal content understanding on the web , it offers a mechanism for probing machine understanding of images via natural language queries . Making progress on VQA requires bringing together different subfields in AI – combining advances from natural language processing ( NLP ) and computer vision together with those in multimodal fusion – making it an exciting task in AI research . Over the years , the performance of VQA models has started to plateau on the popular VQA v2 dataset [ 20 ] – approaching inter-human agreement – as evidenced by Fig . 1 . This raises important questions for the field : To what extent have we solved the problem ? If we haven ’ t , what are we still missing ? How good are we really ? An intriguing method for investigating these questions is dynamic data collection [ 29 ] , where human annotators and state-of-the-art models are put “ in the loop ” together to collect data adversarially . Annotators are tasked with and rewarded for finding model-fooling examples , which are then verified by other humans . The easier it is to find such examples , the worse the model ’ s performance can be said to be . The collected data can be used to “ stress test ” current VQA models and serve as the next iteration of the VQA benchmark helping drive further progress . The commonly used VQA dataset [ 20 ] was collected by instructing annotators to “ ask a question about this scene that [ a ] smart robot probably can not answer ” [ 4 ] . One way of thinking about our proposed human-adversarial data collection is that it explicitly ensures that the questions can not be answered by today ’ s “ smartest ” models . ∗Equal contribution . Correspondence to advqa @ fb.com . †Work done as an intern at Facebook AI Research . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . This work is , to the best of our knowledge , the first to apply this human-adversarial approach to an image and language multimodal problem . We introduce Adversarial VQA ( AdVQA ) , a large evaluation dataset of 46,807 examples in total , all of which fooled the VQA 2020 challenge winner , the MoViE+MCAN [ 45 ] model . We evaluate a wide range of existing VQA models on AdVQA and find that their performance is significantly lower than on the commonly used VQA v2 dataset [ 20 ] ( see Table 1 ) . Furthermore , we conduct an extensive analysis of AdVQA characteristics , and contrast with the VQA v2 dataset . We hope that this new benchmark can help advance the state of the art by shedding important light on the current model shortcomings . Our findings suggest that there is still considerable room for continued improvement , with much more work remaining to be done . 2 Related Work . Stress testing VQA . Several attempts exist for stress testing VQA models . Some examine to what extent VQA models ’ predictions are grounded in the image content , as opposed to them relying primarily on language biases learned from the training dataset . The widely used VQA v2 dataset [ 20 ] was one attempt at this . The dataset contains pairs of similar images that have different answers to the same question , rendering a language-based prior inadequate . VQA under changing priors ( VQA-CP ) [ 2 ] is a more stringent test where the linguistic prior not only is weak in the test set , but is adversarial relative to the training dataset ( i.e. , answers that are popular for a question type during training are rarer in the test set and vice versa ) . Other complementary datasets test model robustness against certain conditions , such as different logicial compositions [ 18 ] , question rephrasings [ 52 ] , suprious correlations [ 1 ] , and consistency in answering sub-questions [ 51 ] . Datasets which benchmark specific capabilities in VQA models also exist , such as reading and reasoning about text in images [ 57 ] , leveraging external knowledge [ 44 ] , and spatial [ 26 ] or compositional reasoning [ 28 ] . Other vision and language datasets , such as Hateful Memes [ 31 ] , have also tried to make sure the task involves true multimodal reasoning , as opposed to any of the individual modalities sufficing for arriving at the correct label . Compared to the previous work , with AdVQA , we aim to have a more holistic and general dataset that pushes the boundaries of the state-of-the-art on VQA . Saturating prior work . The VQA v2 [ 20 ] challenge has been running yearly since 2016 and has seen tremendous progress , as can be seen in Figure 1 . From simple LSTM [ 23 ] and VGGNet [ 54 ] fused models to more advanced fusion techniques ( MCB [ 16 ] , Pythia [ 27 ] ) to better object detectors ( MCAN [ 61 ] , MoViE+MCAN [ 45 ] , BUTD [ 3 ] ) to transformers [ 59 , 56 ] and self-supervised pretraining ( MCAN [ 61 ] , UNIMO [ 38 ] ) , the community has brought the performance of models on the VQA v2 dataset close to human accuracy , thus starting to saturate progress on dataset . As we show through AdVQA , however , saturation is far from achieved on the overall task , and hence there is a need for new datasets to continue benchmarking progress in vision and language reasoning . natural language inference [ 46 ] , question answering [ 5 , 50 ] , sentiment analysis [ 49 ] , hate speech detection [ 60 ] , next video-and-language event prediction [ 35 ] and dialogue safety [ 15 ] . To the best of our knowledge , ours is the first work to explore an image and language multimodal human-adversarial benchmark.1 3 AdVQA . The aim of this work is to investigate state of the art VQA model performance via human-adversarial data collection . In this human-and-model-in-the-loop paradigm , human annotators are tasked with finding examples that fool the model . In this case , annotators are shown an image and are tasked with asking difficult but valid questions that the model answers incorrectly . We collected our dataset using Dynabench [ 29 ] , a platform for dynamic adversarial data collection and benchmarking . For details on our labeling user interface , please refer to the supplementary material . The VQA v2 dataset is based on COCO [ 39 ] images . We collected adversarial questions on both val2017 COCO images and testdev2015 COCO images . We then random sampled the collected set down to 2 questions per val2017 COCO images ( 10,000 ) and 1 question per testdev2015 COCO images ( 36,807 ) . The random sampling was done to balance the annotator ’ s contributions . The data collection involves three phrases ( Figure 2 ) : First , in the question collection phase , Amazon Mechanical Turk workers interact with a state-of-the-art VQA model and self-report image-question pairs that the model failed to answer correctly . Second , in the question validation phase , a new set 1Concurrent work [ 36 ] jointly available with AdVQA at https : //adversarialvqa.org also focuses on the same problem but uses images from a different source . of workers validates whether the answers provided by the VQA model for the image-question pairs collected in the first phrase are indeed incorrect . Finally , in the answer collection phase , we collect 10 ground truth answers for the image-question pairs validated in the second phase . In what follows , we provide further details for each of these steps . 3.1 Question Collection . In this phase , annotators on Amazon MTurk are shown an image and are tasked with asking questions about this image . The interface has a state-of-the-art VQA model in the loop . For each question the annotator asks , the model produces an answer . Annotators are asked to come up with questions that fool the model . Annotators are done ( i.e. , can submit the HIT ) once they successfully fooled the model or after they tried a minimum of 10 questions ( whichever occurs first ) . To account for the fact that it may be hard to think of many questions for some images , annotators are given the option to skip an image after providing non-fooling three questions . We use the VQA Challenge 2020 winner – MoViE+MCAN [ 45 ] – trained on the COCO 2017 train set as the model in the loop . This is to ensure that the adversarial benchmark we collect is challenging for the current state-of-the-art in VQA . The interface provides a digital magnifier to zoom into an area of the image when the mouse hovers over it , allowing workers to examine the image closely if necessary . See Figure 3b . Keyboard shortcuts were provided for convenience . 3.2 Question Validation . Note that in the question collection phase , annotators self-report when they have identified a question that is incorrectly answered by the model . Whether or not this was actually the case is verified in the question validation phase . Two different annotators are shown the image , question , and answer predicted by the model , and asked whether the model ’ s answer is correct . As an additional quality control , the annotator is also asked whether the question is “ valid ” . A valid question is one where the image is necessary and sufficient to answer the question . Examples of invalid questions are : “ What is the capital of USA ? ” ( does not need an image ) or “ What is the person doing ? ” when the image does not contain any people or where there are multiple people doing multiple things . If the two validators disagree , a third annotator is used to break the tie . Examples are added to the dataset only if at least two annotators agree that the question is valid . 3.3 Answer Collection . In the final stage of data collection , following [ 20 , 57 ] , we collect 10 answers per question providing instructions similar to those used in [ 20 ] making sure that no annotator sees the same question twice . In addition , as an extra step of caution , to filter bad questions that might have passed through last two stages and to account for ambiguity that can be present in questions , we allow annotators to select “ unanswerable ” as an answer . Further , to ensure superior data quality , we occasionally provide annotators with hand-crafted questions for which we know the non-ambiguous single true answer , as a means to identify and filter out annotators providing poor quality responses . 3.4 Human-Adversarial Annotation Statistics . The statistics for the first two stages ( question collection and validation ) are shown in Table 2 . We find that annotators took about 5 tries and on average around 4 minutes to find a model-fooling example . For computing the model error rate , we can look at the instances where annotators claimed they had fooled the model , and where annotators were verified by other annotators to have fooled the model . The latter has been argued to be a particularly good metric for measuring model performance [ 29 ] . We also further confirm that the model was indeed fooled by running the model-in-the-loop on a subset of examples in which human agreement was 100 % and found the accuracy to be 0.15 % on val , and 0.18 % on test splits of AdVQA respectively . Interestingly , the “ claimed ” model error rate , based on self-reported model-fooling questions , is similar to that of text-based adversarial question answering , which was at 44.0 % for RoBERTa and 47.1 % for BERT [ 5 ] . The validated error rate for our task is much higher than e.g . for ANLI [ 46 ] , which was 9.52 % overall , suggesting that fooling models is a lot easier for VQA than for NLI . It appears to be not too difficult for annotators to find examples that the model fails to predict correctly .

Summary: This paper works on the topic of visual question answering and aims to test how well state-of-the-art models perform on adversarial examples. Specifically, the authors collect a new adversarial benchmark on VQA, named "Adversarial VQA" (AdVQA), including a large evaluation dataset of 28,522 examples in total. The data samples in AdVQA are collected by putting both human annotators and state-of-the-art models in the collection loop: Annotators are requested to create questions that can fool the state-of-the-art models, and these questions are further verified by other annotators. The authors benchmark state-of-the-art models on these collected adversarial questions and their performance drops significantly.  The main contribution of this paper is the collected AdVQA dataset which includes adversarial examples created by human. The experimental results suggest future research directions on VQA.