Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Given recent finding shows that ViTs are more robust than CNN, this paper investigates an intriguing question:
“Under what kinds of perturbations do ViTs become weaker learners compared to CNNs". They propose a Patch-Fool to fool the attention mechanism. Their investigation leads to some interesting findings and might inspire more interesting future work. 
main_review: The strengths of this work include (a) extensive experiment to benchmark the robustness of different ViT varinst; (b) proposing a new attacj framework; (c) insightful findings.

Weaknesses: (a) Given those existing findings regarding the robustness comparison between ViT and CNN, this work might look a little incremental. This work clearly recognizes those works and I think it is not necessarily a negative point. (b) The authors mention that  their work might inspire innovative defense techniques. I do not see the rationale behind this claim. I suggest the authors remove this claim or illustrate it more clearly.

Minor suggestions and discussions: 
A recent work [1] has also investigated the MLP-Mixer beyond ViT. It is suggested to discuss it. Is is possible to apply the proposed attack method also to MLP-Mixer. [1] also discusses universal attacks and I am curious whether the above attack method can be extended beyond Image-dependent attack to universal ones. Do the authors think the reasons that ViT are weaker against Patch-Fool might be explained from the shift invariance perspective [1,2]?


[1] Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs
[2] Shift Invariance Can Reduce Adversarial Robustness
summary_of_the_review: I believe this work is solid and provides insightful finding, and thus I vote for acceptance of this work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the robustness of vision transformers (ViT) from the perspective of adversarial attacks on patches, where the attack algorithm is particularly designed to fool the attention mechanism. While some prior works show that ViT has better adversarial robustness compared to CNNs, this paper shows that ViT has worse robustness against patch attacks, when only a few patches are manipulated and the perturbations are dense within the patches.
main_review: Strengths
* This paper proposes a particular algorithm to attack vision transformers by considering the attention mechanism.
* This paper successfully identifies the vulnerability of ViT against dense patch attacks, while ViT was more robust under traditional Lp perturbations or natural perturbations in previous works.
* The paper has comprehensive experiments on benchmarking the robustness of ViT and CNN models, in terms of different patch selection strategy, number of perturbed patches, the sparsity of perturbation within each patch. 

Weaknesses
* This paper doesn't consider the robust training for ViT to improve the robustness and evaluate the robustness of more robust models.
* I think "ARE VISION TRANSFORMERS ALWAYS ROBUST AGAINST ADVERSARIAL PERTURBATIONS?" in the title is not very meaningful, because it's known that it's impossible for a model to be robust against all perturbations so far, and even on the perturbations used in existing works, ViT is not robust. ViT was just relatively more robust than CNN. 
summary_of_the_review: This paper makes an important contribution on identifying a particular vulnerability of ViT against patch attacks compared to CNNs and has comprehensive empirical results, though this paper does not consider robust training for the models.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper propose a new attack on Vision Transformers (ViTs) called Patch-Fool. The attack proceeds by first picking a patch which contributes the most (in the self attention calculation) to other patches and then perturbed it adversarially wrt cross entropy + attention based loss. The results show that this kind of attack degrades performance significantly wrt prior work. The authors then perform various ablation studies to justify their architecture/loss choices.
main_review: To my knowledge, the approach is novel and performance gains (degradation in this case) non-trivial. Overall, the paper is fairly easy to understand and the evaluation is fair.
 
Here are some of my thoughts/criticisms:
1. My main concern is wrt to the lack of example of images of adversarial examples. From whaat I have seen, there is only one image in Fig 1 which shows the output of the technique proposed. The authors later go on to show that increasing the number of patches seems to make the attack more effective. However, at the point, is the image even adversarial in the traditional definition of the word? (i.e.  change in image is imperceptible to human eyes). Patch-Fool does not have an \epsilon factor like traditional robust training seems to have, so, essentially, we can perturb the image by a large amount to make the attack more potent and potentially make large changes to the image. Maybe the authors could also quantify how far these adversarial examples are via L2 distance. 
2. Related to the previous point, if we perturb all patches in the image, accuracy drops drastically for both CNNs and ViTs. So, can this be considered a procedure to generate adversarial examples for both architectures simultaneously? The authors partially answer this in the sparse case in Table 7 but I was curious about the full perturbation case. Again, without an example image to look at, it is hard to say whether all patches being changed is even an adversarial example in aforementioned sense. 
3. What are the numbers in table 3? I realized its accuracy on a later reading but the authors should mention this in the caption
4. A minor point about terminology - I have seen “Robust accuracy” used in the literature to mean accuracy of a robust model (i.e. model trained via robust training) whereas here, I assume, the authors used a model trained on clean data and fed it adversarial images. This seems to be implied in the paper but would be useful to mention to avoid confusion.

Formatting/Typos:
1. Section 4.3 - “according a predefined value l” should be  “according to a predefined value l”
2. The title for section 5.2 is the last line of the page. I understand the authors are trying to adhere to page limits but it breaks the “flow” readability-wise. I would encourage the authors to correct this.
summary_of_the_review: Ultimately, I think there is a good idea here but the lack of qualitative examples makes a fair evaluation of this technique not possible. I would not recommend publication of this manuscript in its current state.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a Patch-Fool attack which fools the self-attention mechanism of ViTs to evaluate the robustness of ViTs and CNNs-based models. Prior works investigated the robustness of ViTs and CNNs under the adversarial attacks designed mainly for CNNs and conjectured that ViTs are more robust than CNNs. However, in this paper, the authors attack a specific single patch and develop an attention-aware attack framework against which they find that ViTs are weaker learners than CNNs. Moreover, by developing Spare Patch-Fool, the authors find that ViTs are more vulnerable than CNNs under high perturbation density. 

main_review: Strengths: 
1. This paper provides a new perspective for evaluating the robustness of ViTs which is novel and insightful.
2. From my point of view, the conclusion that the perturbation density is the key factor that influences the robustness ranking between ViTs and CNNs is significant to this field and can provide a better understanding of ViTs’ robustness.  
3. This paper is well-written and easy to follow. The experiments are extensive and solid enough. 

Weaknesses: 
1. I notice that the robustness benchmarks of this paper are under the non-robust models. Some recent works have also proposed adversarial training for ViTs such as [1]. Can the authors provide the results on adversarial trained ViTs and CNNs for better benchmarking the robustness of these models?

[1] Shao, Rulin, et al. "On the adversarial robustness of visual transformers." arXiv preprint arXiv:2103.15670 (2021).

summary_of_the_review: This paper proposes an effective Patch-Fool Attack for evaluating the robustness of ViTs, which is novel and the key finding of this paper is significant. I suggest acceptance.
review_title: null

Meta-review: This paper provides an interesting study on the adversarial robustness comparisons between ViTs and CNNs, and successfully challenges the previous belief that ViTs are always more robust than CNNs on defending against adversarial attacks. Specifically, as revealed in this paper, when the attacker considers the attention mechanisms, the resulting patch attack can hurt ViTs more. 

Overall, all the reviewers enjoy reading this paper and appreciate the comprehensive robustness comparisons between ViTs and CNNs. The reviewers were concerned about the missing experiments about adversarial training, vague statements about the inspiration for future defenses, visualization of adversarial examples, etc. All these concerns are well addressed during the discussion period, and all reviewers reach a consensus on accepting this paper.

The final version should include the experiments, visualizations, and clarifications provided in the rebuttal. In addition, please release the code as promised.
Decision: Accept (Poster)