Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper explores measuring VQA models under adversarial settings, by having two competing models, when reasoning over a CLEVR scene as usual, and the other seeks to make the scene more challenging to answer. These settings reveal the weaknesses of visual reasoning models, and demonstrate that their apparent almost perfect performance on the standard CLEVR dataset may give imprecise impression about their “true” reasoning capabilities, as the adversarial settings may better reveal. 

Update: Following the detailed and thorough response by the authors that addressed most of my concerns, I'm happy to increase my score.
main_review: 
**Prior work**
* **Prior works about adversarial evaluation**: This type of adversarial analysis is not new in ML for QA models in particular (e.g. has been explored for reading comprehension over SQuAD and also over real-images) but it still interesting and unexplored in the context of CLEVR.
* **Prior work about consistency**: The particular sort of adversarial analysts that the paper explores is actually about measuring consistency - asking the same question over different scenes that hold the same relevant relations to make the question's answer unchanged, and checking wether the model indeed keeps its answer consistent over the different scenes. Such sort of evaluation has been proposed e.g. as part of the GQA dataset a couple of years ago, by pre-computing variety of semantically-equivalent questions and then measuring the consistency over them. 
* **Related work**: The related work section gives a good coverage though in describing prior papers in related fields.

**The approach**
* **Problem statement**: The authors propose a clearly-defined approach and present it with a good degree of detail and preciseness. 
* **adversarial scene manipulation**: A strength of the approach is that compared to other adversarial experiments that tend to change the inputs to be valid but not natural (e.g. a random or unreadable sequence of words that fails a textual QA model, or changes to pixel values that impact a visual models abut can't be noticed by a person), the sort of adversarial modifications performed by the proposed approach are completely natural and explainable (manipulating objects within the scene). 
* **Perceptual vs. reasoning failures**: At the same time, I do worry about whether maybe some of the adversarial modifications are potentially just too visually-difficult for the model rather than really exposing real reasoning failures. There are multiple causes the could lead for that: (1) moving an object to be too occluded by another object so to be hard to identify, (2) or moving an object to be near another object of the same color in a particular position that will make the model harder to distinguish between them, (3) or making objects too similar in x or in y position such that it's hard to determine if one it behind/in front / to the left/right of another, or (4) putting a metallic object at a very particular position in the scene and/or partially occluded such that it will be hard to notice it's metallic reflection. Some of the qualitative examples indeed look like that is potentially the source for the change in the model answer.  I think there are 2 ways to mitigate this potential issue: First, constrain the adversarial model so to prevent it from making valid but "visually unfair" modifications (e.g. making object too close to each other etc), and second quantitatively measuring how the modification impact the performance of a pre-trained CLEVR object detector. On the standard CLEVR data a pre-trained detector can reach almost a perfect accuracy, and in the cases its performance will go down too due to the adversarial modification, we will know that the source of the failure was in perception rather than in reasoning. Overall I recommend the authors to explore these options further in order to give isolate the reasoning failures and strengthen the claims in the paper.
* **Other datasets**: Another important way to increase the confidence that the failures happen due to reasoning and not perception, which is always a good practice in general - is to test the approach also on another dataset. e.g. the NLVR or other 2d scenes could be good candidates, since their perceptual elements is easier than in clevr and so they are less prone to adversarial perception failures.
* **Adversarial vs. pre-computing questions**: I actually think that the pre-computed version as in GQA potentially allows simpler means for carrying out the consistency evaluation (by simply testing a trained model over a set of test questions). On the other hand, the adversarial conditions do enable looking for questions that will be especially hard for the particular evaluated model -- on a sense these settings therefore are more challenging, since they focus on looking for particular blind spots of the model, rather than measuring its consistency over randomly sampled equivalent scenes -- i.e. it could be that out of the space of semantically equivalent scenes, the model stays consistent in 95% of them, and the adversarial model finds those few rare cases it doesn't, not giving quantitative indication to how rare these cases are. Random sampling of multiple equivalent scenes would in contrast give a better estimation of the degree of consistency. Overall, I think while each approach has positives and negatives, it could be nice to include comparison in model's performance in the adversarial vs. randomly sampled equivalent scenes settings, in order to give a bigger more detailed picture about models' degree of consistency.
* **Black-box evaluation**: The black-box aspect of the approach -- the fact that it measures models performance and reasoning capabilities just by looking at their textual answers can be both a strength and a weakness: on the one hand, it make the evaluation more easily performed on a wider variety of models, but on the other, it gives a narrow view into why models fail when they do. Especially for CLEVR, there are multiple models that are e.g. based on  computing attention maps, executable programs, structured representations and other indications for their ongoing reasoning rather than just predicting an answer. I think it will be good to pick couple of such models and explore in the paper how they perform internally when failing vs. when being consistent -- are the any interesting patterns in the computed programs or attentions when models fail? Are there particular settings, scenes or questions that make models to be particularly vulnerable? having such sort of analysis would help a lot getting further insight not just into whether models are good or bad reasoners, but also for why are they not good enough and what could be done to improve them.

**Writing quality**
* The writing clarity of the paper is good, the idea and content are easy to follow, the motivation is clearly explained, and there are good visualizations that help understanding. 
* **Spaces**: One small note regarding presentation of the manuscript: I guess due to space constraints there are no paragraphs at all -- the whole paper after the first couple pages is a one long block of text -- which makes it harder to read visually, so strongly recommend getting it back to the standard format in terms of spaces etc.

summary_of_the_review: Overall, I think the paper proposes an interesting evaluation but is still missing empirically in multiple aspects that could strengthen the paper:
* Tracking potential reasons for the failure source (perceptual or reasoning) and making sure to reduce ways for the adversarial model to place objects in ways that make them too hard from the perceptual side to answer correctly.
* Comparing adversarial to randomly-sampled scenes to measure the degree of consistency given a question rather than penalize the model for the existence of a single way per question to fool it. 
* Explore the model at least one more additional environment (e.g. 2d)
* Explore additional indications about models behavior when they do and do not fail, to give further insight about potential reasons for why they are in/consistent in different cases.

I therefore give the paper a marginal score, and encourage the authors to update the paper to make the evaluation more extensive!
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a black-box adversarial attack on models trained on the CLEVR dataset. CLEVR dataset is widely considered solved, especially with program synthesis-based models which seem to achieve near-perfect accuracy on CLEVR with a fairly small amount of training examples. Moreover, due to their synthetic and limited nature, it can be reasonably assumed that they would be able to answer *any* variation of CLEVR scenes within the bounds of the CLEVR-universe (Barring intentionally OOD test sets such as CoGENT). However, this paper shows that it is possible to fool all classes of the CLEVR model using an adversarial player that simply re-configures the scene (which should have the same answer as before). While the drops inaccuracy is not drastic, it is still an interesting finding, especially for program synthesis-driven methods. The paper also ends with an experiment about whether a purely data-driven approach can ever learn to be fully "robust". 
main_review: To this reviewer, this is an interesting paper, that passes the bar of acceptance in ICLR. The experiments are thorough and well-explained. The writing is clear and detailed and the results are interesting. However, there are a few concerns that prevent me from giving a better score. Details Below

Strengths:

S1. Thorough Experiments: The choice of both data-driven (RN, FiLM), ground-truth program based (IEP, TBD) as well as a host of experiments clearly show the efficacy of the proposed model. There is very little to complain about ablations, details, statistical tests, and thoroughness in experiments.

S2. Clear Writing: The writing is clear and easy to understand. It also has enough details to easily allow reproduction by an expert.
 
S3. Interesting Results: The results are not always surprising. As expected, the program-based models are more robust to adversarial attacks compared to "from-scratch" methods. Regardless, some of the results for several strong non-program-based models still come as a surprise to me and raise interesting questions about the tested algorithms as well as learning as a whole. To me, this is the biggest strength of the paper.

Weaknesses:

W1. Experimental Validation has some weird issues: Maximal scores are, as I understand it, measure the loss of accuracy in the "worst" minigame compared to overall accuracy. I think that is an incorrect way to report it. In my opinion, it should be compared to the accuracy in *THAT* particular mini-game as opposed to comparing it with average accuracy for the model. Please clarify this choice. 

W2. Some details are missing: How are the manipulations done and are steps taken to avoid unintended issues, e.g., occlusions, that would make a question unanswerable/ambiguous? 



summary_of_the_review: Overall, this is a solid paper with really interesting results. There are a few issues, noted in weaknesses, that prevent me from giving it a higher score. I will happily assign a higher score after rebuttal if those points are clarified.


UPDATE: I have read the author's responses and all other reviews carefully. Like others, I had some concerns about scene occlusion, etc. issues, but the authors have satisfactorily answered that. I also appreciate the authors' commitment to update the paper according to the reviews. That being said, I do not think the revisions have made for any *new* information for me to change my scores. I still think this is an interesting paper, albeit a bit limited in scope. I tend to keep my score as is ( ~ a weak accept)
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to measure the reasoning capabilities of VQA models by training an "adversary player" that can manipulate the image and fool the evaluated models. Given a sample, the "adversary player" moves objects such that the groundtruth answer remains valid and the scene is in-distribution. If the evaluated VQA model produces a different output, the "adversary player" receives a positive reward and is then optimized with a reinforcement learning algorithm.
main_review: Strengths:

The paper proposes a general framework to evaluate the reasoning capabilities of current VQA models. This evaluation could be useful as previous VQA models are likely to overfit the dataset bias instead of understanding and reasoning the question.


Weaknesses:

The experimental results don't reveal much new information. The early models(SAN, FiLM, and RN) are simple and do not consider the reasoning abilities. TbD and Mdetr perform object-level reasoning. Thus it is predictable that they can maintain the average performance given the manipulated samples. Other State-of-the-art models on CLEVR, such as MAC[1] and NS-CL[2], are also likely to have consistent answers. The experimental results do not expose any new weakness of the evaluated VQA methods.

[1] Drew A. Hudson, Christopher D. Manning, Compositional Attention Networks for Machine Reasoning, ICLR 2018

[2] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu, The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision, ICLR 2019
summary_of_the_review: Overall, this paper proposes a feasible way to evaluate the reasoning abilities of any black-box VQA model. Although the experimental results haven't revealed new problems, the proposed method can inspect current largely pretrained vision-language models to see if they perform reasoning on various tasks. Thus I endorse acceptance.
review_title: null

Meta-review: Introducing an adversarial agent that re-configures the rendered scenes of CLEVR to demonstrate that models that appear achieve super-human performance are actually easily fooled due to their lack of ability to reason, provides a nice insight into limitations with existing approaches and correspondingly how we evaluate on some benchmarks.  There is a persistent concern that the results are only on CLEVR, and that the adversarial examples are not really disproving reasoning but rather issues with vision.  However, overall reviewers were generally positive about the aims the work.
Decision: Accept (Poster)