Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper benchmark tests several implementation details on adversarial training with Vision Transformers. Moreover, this paper proposes two modules, i.e., ARD and PRM to improve the adversarial robustness.
strengths_and_weaknesses: There are a number of benchmark tests, which is really appreciated.
The proposed ARD and PRM modules look interesting.

questions: In table 1, why ViT-B has the exact same awful performance for natural images and adversarial examples before the gradient clip? Does that mean that ViT-B does not work on natural image classification at all? And about the accuracy on Cifar-10 dataset, the 10.00% seems pretty magical. Do you explore whether this number is because the adversarial examples from nine out of ten classes can fool the model 100 percent, while the remaining one can not fool the model at all, or the number 10.00% just magically happens?
limitations: While there are lots of benchmark tests that provide empirical hints for practical applications, I personally expect discussions about the reason for such observations. For example, why ViT-B suffers the gradient explosion while Swim-B does not? How about other Transformer models? Why “SGD+piecewise” performs better than “AdamW+cyclic”, and is that true for all Transformer models?
With such discussions, I think it would be more useful in guiding practical applications.

Moreover. some conclusions are very difficult to be applied in practice. For example, the authors claim that “it is a good choice to use advanced data augmentations (e.g., Cutmix and Mixup), …”, but “RandAugment is a more advanced technique, which is too difficult for AT of ViTs”, then how “advanced” the data augmentation should be used? 

The proposed ARD and PRM modules look interesting, but their effectiveness is not well justified by the experiments, especially combining them together shows lower performance in many cases. Is there any reason for this phenomenon?
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper analyzes and improves the robust training performance with vision transformers. If first rigorously analyze the impact of various factors on robust training and then proposes novel methods, namely attention random dropping (ARD) and perturbation random masking (PRM), to further boost the performance. 
strengths_and_weaknesses: This paper is among the first works to analyze robust training performance on vision transformers. It uniquely takes a deep dive into the fields by rigorously analyzing the impact of various factors in robust training on the final performance.

----

The major limitation of this work is the highly sub-optimal performance of vision transformers in adversarial training. For example, after all proposed innovations (including pre-training, warmup strategies, careful augmentations, etc.) the best robust accuracy (AA) achieved on cifar-10 is 50.01 with ViT-B. In contrast, a ResNet-18 achieves 51.06 robust accuracy (AA) without any pre-training (so no extra data or other bags of tricks) [Source: RobustBench]. Note that a ResNet-18 has at least 10x (if not more) lower number of parameters than ViT-B. With extra data, the ResNet-18 network can further achieve 57.67 robust accuracy (AA), much better than the best achieved with very large ViTs in the paper. 

The aforementioned critique also signals towards another critical comparison that is missing in the paper: A comparison of ViTs and CNNs performance. ViTs continued to be adopted in the community over CNN since they outperformance CNNs. To make a case for the use of ViTs in a robust training regime, one needs to show a similar benefit of ViTs over CNNs. However, I couldn’t find any such comparison in the paper (as I argued above the performance of ViTs is far suboptimal than CNNs in the current paper).  

Another troubling trend on the performance front (cifar-10) is that Swin, which is much better architecture than baseline ViT-B, has poorer performance (table 1, 2). Not just for Swin, but this phenomenon is also true for other modern architectures that are better than baseline ViT. This makes me question whether the proposed innovations are any effective at lower resolutions since they are unable to utilize better architectures. Only at higher resolution (ImageNette), they perform better with better architectures. This should be clearly clarified in the paper. 

questions: I am also puzzled why a very small number of epochs are chosen (40 by default). ViTs are generally trained for longer schedules (often 300 epochs), especially when trained from scratch. Was computational cost the only reason behind this design choice? 

Similarly, another puzzling trend is in fig 2. In fig. 2a, pre-training on ImageNet-1k achieves better performance than pertaining on much bigger ImageNet-21K. This is strange as ImageNet-21K tends to always perform better. Similarly in fig. 2b, the training accuracy of the model itself doesn’t converge to very high values, suggesting that the bottleneck in pre-training from scratch is the optimization process, not generalizing. Given the large number of parameters in ViTs, shouldn’t they be able to fit the data on the training set?
limitations: The limited performance of ViTs, in comparison to CNNs, is not highlighted in the paper. I encourage authors to discuss this limitation with concrete comparisons.
review_title: null

final_score: rating: 9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: 
Summary:

I think this article makes a huge contribution to the adversarial robustness of ViT. It is not just a simple application of adversarial training suitable for CNN to ViT, but it has made substantial contributions, including detailed and reasonable analysis and proposing new AT techniques suitable for ViT. I like this paper very much and look forward to its early publication. It would undoubtedly be a great contribution to the community if the authors could publish the trained robust models in time. I'm looking forward to it being included in the robustness library.

strengths_and_weaknesses: 
(Positive) This paper provides a comprehensive evaluation of various training techniques across several datasets, thus bringing the first implementation benchmark for adversarial training of ViTs.


(Positive) It is not just a simple application of adversarial training suitable for CNN to ViT, but it has made substantial contributions, including detailed and reasonable analysis and proposing new AT techniques suitable for ViT.

(Positive) CNN and ViT are indeed very different, so the AT of CNN should be different from that of ViT. For example, compared to CNN, ViT requires Gradient Clipping and pre-training, prefers advanced DA, and is more prone to robust overfitting.


(Positive) Although we know the difference between CNN and ViT under vanilla training, relying on this known knowledge is not enough. For example, we know that ViT favors AdamW in Vanilla training, but the authors found that adversarial training of ViT favors SGD.

(Positive) The authors put forward two distinctive AT methods, PMR and ARD, specifically for the characteristics of ViT.

(Positive) Indeed, as the authors say, although ViT and CNN are very different, ViT trained only by vanilla training is not significantly robust (only a small effect under extremely small interference).

(Positive) According to the protocol on CIFAR-10, it is reasonable for the authors to change the input of ViT to 32x32.

(Positive) Similarly, in my experience, it is reasonable for the authors to use AA to test the robustness of ViT.

(Positive) It is reasonable for the authors to use PGD-10 to perform AT on ViT.


(Positive) The authors' results on the AT for ViT are reasonable and hopefully will be published sooner for people to use.


(Negative) Figure 2 shows that the effect of pre-training with ImageNet-1k is slightly higher than that of ImageNet-21k. Can the authors give some reasonable explanations?



(Negative) The content of formula (4) to formula (7) is too simple, and the existence of these formulas is unnecessary. These formulas can be simplified or omitted unless the authors want to use these formulas to give theoretical results to their method.


(Question) It can be seen from Table 3 that the effect of PRM is better than that of ARD in general. I hope the authors give some reasonable explanations.

(Positive) The authors demonstrate that their method can be combined with and improve the performance of existing AT methods. This is very inspiring.

(Positive) The authors are commendable for their hyperparameter analysis of the number of epochs for warmup.


questions: 

Please answer the questions in "Strengths And Weaknesses."


limitations: 
The authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents two empirical strategies to improve the effectiveness of adversarial training in vision transformer models. The authors first analyzed the effects of gradient clipping, pre-training, data augmentations, optimizer, and training schedules. They further proposed two tricks, attention random dropping (ARD) and perturbation random masking (PRM) to improve the adversarial robustness. Evaluations show that both methods could contribute to the robustness enhancements. 
strengths_and_weaknesses: Strengths:
1. The paper is well written, well-organized, and easy to follow.
2. The evaluation is thorough and sufficient.

Weakness:
1. There is no in-depth analysis or ablations on how these different vectors affect the adversarial training.
2. The technical novelty is limited, and the authors fail to provide intuition on why these two tricks would work. 
3. Although the authors provide the chain rule of gradient propagation for these two tricks, the overall idea is still similar to dropout-style regularization. Therefore, the authors need to provide more baselines on different regularization methods.
questions: 1. The authors do not deliver a further in-depth analysis or ablation study on the reason behind these two tricks. 
2. In Table 3, the combination of ARD and PRM fails to deliver better results. I assume the reason could be the regularization will be too strong, but I expect to see the answers from the authors.
limitations: There is no potential negative societal impact of this work.
review_title: null

Meta-review: metareview: The paper studies how to properly conduct adversarial training on ViTs to obtain adversarially robust models. The paper mainly tried to conduct a comprehensive study on ViT adversarial training and identified several important training heuristics that can improve the robustness of ViTs. Among four reviewers, three consider this as a borderline paper and one strongly supports this work. After several discussions, we think this paper will build a solid foundation for future adversarial robustness studies on ViT models so decide to accept the paper. However, we do hope the authors carefully revise their paper based on the reviewers' suggestions. More specifically, the main concern from reviewers is the correctness issue pointed out by Reviewer 63WY. We hope the authors can carefully check and explain those issues in their final version. 
confidence: Certain
Decision: Accept