When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
Authors: Yichuan Mo
Dongxian Wu
Yifei Wang
Yiwen Guo
Yisen Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Vision Transformer
Adversarial Training
Robustness

Abstract: Vision Transformers (ViTs) have recently achieved competitive performance in broad vision tasks. Unfortunately, on popular threat models, naturally trained ViTs are shown to provide no more adversarial robustness than convolutional neural networks (CNNs). Adversarial training is still required for ViTs to defend against such adversarial attacks. In this paper, we provide the first and comprehensive study on the adversarial training recipe of ViTs via extensive evaluation of various training techniques across benchmark datasets. We find that pre-training and SGD optimizer are necessary for ViTs' adversarial training. Further considering ViT as a new type of model architecture, we investigate its adversarial robustness from the perspective of its unique architectural components. We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs. Our code is available at https://github.com/mo666666/When-Adversarial-Training-Meets-Vision-Transformers.

When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
Yichuan Mo!
Dongxian Wu²
Yifei Wang Yiwen Guo
Yisen Wang
Key Lab. of Machine Perception (MoE), School of Intelligence Science and Technology. Peking University
2 'The University of Tokyo
3 School of Mathematical Sciences, Peking University Independent Researcher Institute for Artificial Intelligence, Peking University
Abstract
Vision Transformers (ViTs) have recently achieved competitive performance in broad vision tasks. Unfortunately, on popular threat models, naturally trained ViTs shown to provide more adversarial robustness than convolutional neural networks (CNNs). Adversarial training is still required for ViTs to de- fend against such adversarial attacks. In this paper, we provide the first and comprehensive study on the adversarial training recipe of ViT's via extensive evaluation of various training techniques across benchmark datasets. We find that pre-training and SGD optimizer are necessary for ViTs' adversarial training. Further considering VIT as a new type of model architecture, we investigate its adversarial robustness from the perspective of its unique architectural compo- nents. We find, when randomly masking gradients from some attention blocks or masking perturbations
some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up line of work to explore the architectural information inside the newly designed models like ViT'. Our code available httpp:///ithuu.. com momm/o666666/ When- Alee--ddveeaaral--1e -raniiiaaa-aainn-n--eeeee -iieets-iision Wwwiiiio--rrnss' Tranoooreerssssssrrrrroooo ormers
Introduction
Recent years have witnessed the overwhelming advances of Vision Transformers (ViTs) 2 Different from widely deployed Convolutional Neural Networks (CNNs) of local convolutional operations on input images, ViTs adopt self-attention mechanisms [5] sequence of image patches. Based on large-scale pre-training, ViT's have achieved competitive and even better performance compared to CNNs in several fields such as semantic segmentation [6]. object detection [7]. and image generation (8). Despite their great success on a growing number of vision tasks, ViTs fail in providing more adversarial robustness than CNNs on popular threat models 10]. To defend against such adversarial examples crafted by adding human-imperceetible perturbations to images [i 12), ViTs still demand adversarial training (adversarial training) 16), an approach that incorporates adversarial examples into training and obtains notable empirical robustness.
that adopt series
Notably, the implementation details of adversarial training make a big difference on CNNs backbones 17]. which motivates us to comprehensively evaluate various training techniques for adversarial training of ViTs and thus provide the first implementation benchmark for the training recipes of ViT's under adversarial training. Since adversarial training is quite different from natural training, different insights discovered, for example, pre-training is useful for adversarial robustness and
"Correponding author: Yisen Wang (yisen.wang @ pku. edu.ed..cn).
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

SGD behaves better than AdamW for ViTs' adversarial training. Based on our empirical findings, we provide a practical recipe for adversarial training of ViTs to help researchers in their future work.
Although ViT' can achieve reasonable adversarial robustness via the above training recipes of adversarial training, it is unexplored whether or not we are able to further improve its robustness through the specific architectural information from ViT', after all ViT's are kind of model totally different from CNNs. Interestingly, when we randomly mask gradients from some attention blocks or mask perturbations on some patches during adversarial training, the robustness ViTs can be further improved. We term these two simple methods as Attention Random Dropping (ARD) and Perturbation Random Masking (PRM) respectively, and conduct extensive experiments verify their effectiveness.
Our main contributions are summarized as follows:
We comprehensively evaluate training techniques for ViT's under the setting of adversarial training on popular threat models, and discover many interesting insights that are different from the natural training scenario.
Based on the unique architectural information from ViTs, we propose Attention Random Dropping (ARD) and Perturbation Random Masking (PRM) warming-up up strategy to improve the adversarial robustness of ViTs. In particular, ARD randomly masks gradients from some attention blocks while PRM randomly masks perturbations on some patches.
Our work not only provides the first implementation benchmark (bags of tricks) for adversar- ially trained ViTs, but also reminds researchers of the potential of architectural information inside the new brand of models like ViTs.
2 Related Work
2.1 Training Strategies for ViTs
Due to the lack of inductive bias [.. ViTs require more training techniques to reach their performance
potential. Dosovitskiy et al ( first pretrained ViTs on the large-scale datasets (ImageNet-22 k or JFT-300M [18] and achieved competitive performance on downstream tasks. Touvron et al [19
applied strong data augmentations such as Randaugment (00 and Mixup [21 to train ViTs from
scratch. They also find AdamW (22) performs better than SGD [23] for training ViTs. Other studies [1/44] (5) adopted gradient clipping to stabilize and accelerate the convergence ViTs. Steiner et al
(26 conducted a comprehensive study on data augmentation and model regularization of ViTs to improve their natural accuracy. For the robustness of ViTs, the training strategies, especially under adversarial training, are not fully investigated yet. That is exactly what we are doing in this paper.
2.2 Adversarial Robustness of ViTs
Given a clean example & with its class label and model fa(-) with its parameters 0, the goal of an adversary is to find adversarial perturbation å that fools the network into making incorrect prediction for the perturbed image (i.e., (x + 5) + y), while the perturbation norm does not exceed (i.e., ||ō S e),
= arg max [(fe(z + 6).8).
Se
where -(,,-) is usually the cross entropy loss. Previous studies 27 generate adversarial examples, and Project Gradient Descent (PGD) 13
proposed several methods
becomes the most popular
one:
8 + II, (ō + a sign ())
+ (x)
To defend against adversarial attacks, adversarial training (adversarial training) and its variants 13 28, 14 29 become the most promising and effective method 32.32 which directly incorporates adversarial examples into training:
N
= arg min
Σ
max L(folx: + 62), 9:)-
N
Despite extensive understandings of adversarial robustness on CNNs, only a few studies focus on the robustness of ViTs, which can be classified into the following categories: 1) naturally trained

---

-on-aaversarially trained) ViTs: In [3 9 34 10), they revealed that naturally trained ViTs are more robust than CNNs against extremely small perturbations (e.g., under threat model of 0.001
on ImageNet, which is much smaller than the popular threat model of 4/255 R 0.0157): 2) adversarially trained ViTs: In 35 2] they demonstrated that adversarially trained ViTs fail in providing more adversarial robustness on popular threat models compared to CNNS.
However, these previous works all lack a comprehensive study on the adversarial training process of ViTs, which may result in an incomplete and biased impression on the robustness of adversarially trained ViTs. Therefore, under the setting of adversarial training of ViTs, we for the first time comprehensively evaluate bags of training tricks across benchmark datasets, and provide a whole adversarial training recipe for ViTs to obtain remarkable robustness.
3 Strategies for Adversarial Training of ViTs
At the first glance, we may think there are already many works studying the training strategies of ViTs. The BIG difference from them lies on the training paradigm: previous research focuses more on the natural training of ViTs while here we focus on ViTs' adversarial training. Since adversarial training is quite different from natural training, different insights might be discovered.
In this section, comprehensively evaluate several training techniques in terms of data (pre-training and data augmentation) and training (optimizer, learning rate schedule, and gradient clipping) across various architectures (vanilla ViT and Swin) and datasets (CIFAR-10 10 and Imagenette). so as to provide the first training recipe for adversarial training of ViTs.
Datasets. Since adversarial training is cime-consuming, small datasets remain popular for adversarial
robustness 32), Here, we datasets of CIFAR- 10 [36] and Imagenette [37] (a subset of 10 classes from ImageNet-IK] Note that the latest version of Imagenette (imagenette-v2") reshuffles the sampled subset of ImageNet- IK and then splits the training and validation set. This leads to an overlapping issue between the pre- pre-training set (ImageNet- and the reshuffled validation set. To avoid the leakage of validation data during pre- pre-training, we select the previous version of Imagenette (imagenette-v
Architectures. We use the vanilla VIT (ViT-B) I and Swin B 2 as baseline models. ViT first suc- cessfully introduced Transformers from natural language processing to computer vision and achieved competitive recognition performance compared to CNNs. With the help of multi-stage hierarchical architecture, Swin became a general-purpose backbone and achieved SOTA performance on several downstream tasks. By default, all models are pretrained on ImageNet- smaller image size of CIFAR-10, 10, we downsample the patch embedding kernel from 16 x 16 to 4 x 4.
To accommodate the
Threat Models. We apply the commonly-used loo attack (oo0 S E) and set 8/255 for CIFAR- 10 and Imagenette. When evaluating the adversarial robustness, we apply 20-step PGD (PGD-20) [13 with the step size 2/255 and Auto-Attack (AA) (31] that is the strongest attack to verify the empirical robustness via an ensemble of diverse attacks (i.e., two variants of PGD attack, FAB attack
38 and Square Attack (3)..
Training Settings. All models (unless otherwise specified) are pre-trained on ImageNet- IK and adversarially trained for 40 epochs using SGD with weight decay le-4, and an initial learning rate 0. that is divided by 10 at the 36-th and 38-th epoch. Simple data augmentations such as random crop with padding and random horizontal flip are applied. During adversarial training, we use PGD-10 with step size 2/ 2/255 to craft adversarial examples.
3.1 The Usefulness of (Natural) Pre-training
The success of ViTs heavily relies on natural pre-training on large datasets M. Meanwhile, natural pre-training is commonly believed to be useless for improving the adversarial robustness of naturally trained ViTs [9] A question is naturally raised here: under adversarial training of ViTs, whether natural pre-training is useful or not.
https://s3. http://ssshsaaaaaaaoocccaaaaaaaa
https://s3. awazamaws.acooomm
tgz
https://github. httpa///www....gaaaaaaatrraoaaaa
3

---

luat
Irageta-S24
Iragehe--22
From scnacc
Modex
(a) Adversarial training curve of ViT-B CIFAR-10
(b) Final robustness of different pre-training strategies
(Left: with pre-training, Right: training from scratch)
(Left: CIFAR-10, Right: Imagenette)
Figure 1: Effects of different pre-training strategies on adversarial training of ViTs.
We first plot the learning curve of ViT-B with and without pre-training. Specifically, one model is updated from an initialization from natural pre-training on ImageNet-1K, K, while the other is from random initialization. In Figure aa). find that ViT without pre-training achieves low robustness because under- under-fitting on the training data hinders the performance (the training robustness only ~ 30%). Mcanwhile, VIT with pre-traine pre-training fits training data better and improves robustness by notable margin (+20%). This suggests that the current random initialization for weight parameters of ViTs not suitable for adversarial training, even small datasets like CIFAR-10, while natural pre-training on large datasets is useful for successful adversarial training of ViTs. This conclusion seems different from the previous research [9] This is because, although both study adversarial robustness, we focus on the effects of natural pre-training for adversarially trained ViTs, while they focus more on the robustness of naturally trained models.
Further, explore if a larger dataset can bring more robustness. Before adversarial training CIFAR-10, 10, we adopt 3 pre-training strategies respectively: 1) natural pre-training on ImageNet- 21K (an extremely large dataset with 14M images from 21K classes), 2) natural pre-training on ImageNet- 1K (a large dataset with more than 1.2M images from 1K classes), and 3) no pre-training (i.e., adversarial training from scratch). The final robustness under AA is shown in Figure [ Compared adversarial training from scratch, pre-training both ImageNet-IK and ImageNet-211 can improve the robustness of ViT' by notable margin (~~ 16% on average), while ImageNet-21K fails in providing higher robustness (it slightly hurts the performance) compared to ImageNet-IK. Similar trends can also be observed on Imagenette.
In conclusion, natural pre-training is necessary for ViTs to fit the training data during adversarial training, while larger dataset is unable to provide better robustness.
3.2 The Necessity of Gradient Clipping
Table 1:
The performance (%) of adversarially trained ViT,
Swin, and DeiT with or without (w/o) gradient clipping.
Swin B VIT-B DeiT-S
CIFAR-10 10
Imagenette
cradient Norm
Model
Natural
AA
Natural
AA
ViT-B ViT-B w/o GC
85.53
48.33
91.40
64.20
10.00
10.00
10.00
10.00
Swin-B
83.75
45,28
95.60
74.00
Swin-B w/o GC
82.42
44.90
95.00
73.00
11a
100
200
300
450
300
Iteration
DeiT-S
84.13
47.26
90.40
60.20
Figure 2: The fn-norm of gradients during adversarial training of ViTs.
DeîT-S w/o GC
10.00
10.00
10.00
10.00
In natural training, some Transformers implementations (e.g., ViTs and Swin) apply gradient clipping, while the others (e.g.. DeiT) do not. Here, we explore whether it is necessary to use gradient clipping during adversarial training. First, find that, without gradient clipping, it difficult adversarially train ViT or DeiT on small datasets. For example, the final robustness of ViT-B and DeiT-S is only 10% both CIFAR- 10 and Imagenette in Table [ To study the difficulty, visualize the norm of gradient during adversarial training in Figure 2 and find the gradient explodes at the very beginning of training. Then adopt gradient clipping [40] again to adversarial training of ViTs. Specifically,

---

ensure the maximum of gradient vector is under the 1o-norm. In Table 1 with the help of gradient clipping, ViT-B (DeiT-S) achieves the robustness of 48.33% (47.26%) and 64.20% (60.20%) on CIFAR-10 10 and Imagenette. Although Swin-B can be adversarially trained directly, gradient clipping still improves the performance by notable margin. In comparison, gradient clipping has almost no effect on the robustness of CNNs, as shown in Appendix [ In summary, for ViT's, although gradient clipping is optional in natural training, it seems necessary for their adversarial training.
3.3 The Effect of More Training Epochs
The natural training of ViTs usually requires more epochs for better natural accuracy (e.g., 300 epochs for Swin [2] ImageNet-IRe IK) compared to CNNs (e.g., 90 epochs for ResNet [4] on ImageNet- IK). Adversarial training of ViTs is then supposed require longer training epochs for better adversarial robustness (9]. Here, we further investigate it by adversarially training ViT's for 80 epochs (the default setting is 40 epochs) with the learning rate decay the 75-th and 78-th epoch. Figure 3]) illustrates that neither ViT-B nor Swin-B benefits from the longer training, which potentially hurts the final robustness instead.
(a) Adversarial training curve for more epochs (Left: CIFAR-10, Right: Imagenette)
Robustness with different data augmentations (Left: ViT-B CIFAR-10, Right: Swin-B on Ima- genette)
Figure Learning curves of adversarial training with longer training epochs or advanced data augmentations. The robustness is evaluated under PGD-20.
3.4
The Effect of Advanced Data Augmentations
In adversarial training of CNNs, majority of advanced data augmentations like Mixup 21 and Randaugment (20 are proven to be unsuccessful (without considering model weight averaging)
41 42
which is contrary to natural training. Here, explore if these advanced data augmentations
can improve robustness for adversarial training of ViTs. Specifically, we incorporate CutMix (3),
Mixup 21, and Randaugment 20 to adversarial training of ViTs respectively. In Figure 3()). on CIFAR-10, ViT-B obtains higher robustness with CutMix or Mixup or both, while behaving worse with Randaugment. We also find similar phenomena of Swin-B on Imagenette. This is because Randaugment too difficult for adversarial training of ViTs. In conclusion, a suitable combination of data augmentations can directly improve the adversarial robustness of ViTs, which is different from CNNs (needing model weight averaging). the following experiments, we always adopt CutMix and Mixup to adversarial training of ViT's by default.
Table 2: The performance (%) of adversarially trained ViT-B and Swin-B by different optimizers and learning rate schedulers.
ViT-B
Swin-B
Dataset
Optimizer
Natural
AA
Natural
AA
AdamW+cyclic
78.67
46.16
78.33
45.20
Adam Wiieeeeeise
80.76
46.76
10.00
10.00
CIFAR-10
SGD+cyclic
83.06
48.91
81.83
45.46
SGD+piecewise
83.16
49.06
83.36
46.89
AdamW+eycll
92.00
66.80
93.40
72.40
Adam spiceweee
87.40
59.40
10.00
10.00
Imagenette
SGD+cyclic
93.20
66.60
95.40
74.40
SGD+piecewise
93.40
67.00
96.40
74.60
5

---

3.5 The Effect of Optimizer and Learning Rate Scheduler
Recalling that SGD achieves the best adversarial robustness in CNNS 13 41 By contrast, the default optimizer of ViTs (natural training) is AdamW [22]. Here, we investigate what is the appropriate optimizer for adversarial training of ViTs in order to obtain better robustness. We adversarially train models based on every combination of the (p (AdamW and SGD) and If fixing
the learning rate schedule (cyclic and piecewise). The results are summarized in Table 2 the learning rate scheduler, whatever cyclic or piecewise, SGD almost all outperforms Adam W in terms of adversarial robustness. When fixing the optimizer, we find that cyclic learning rate scheduler is suitable for Adam W while the piecewise learning rate scheduler is a better choice for SGD. Furthermore, SGD+pieeeeise achieves better robustness compared to Adam W +cyclii in all datasets and models, i.e., on -IFAR-10, SGD piecewise helps ViT-B improve the natural accuracy by 4.49% and the adversarial robustness by 2.90% over AdamW + cyclic. In summary, the combination of SGD and piecewise learning rate scheduler may be a good choice for adversarial training of ViTs.
3.6 Insights on the Adversarial Training of VITs
With the above comprehensive evaluation of different training techniques, we can provide a practical adversarial training recipe for ViTs. First, pre-training and gradient clipping are almost necessary. Second, it would be better if incorporating advanced data augmentations (e.g., Cutmix and Mixup) and adopting SGD optimizer with piecewise learning rate scheduler. Third, there seems need to train longer epochs for the robustness improvement.
Table 3: The comparison of adversarially trained ViTs and CNNs.
(a) CIFAR-10 and Imagenette
(b) ImageNet-IK
Model
ResNet50
Swin- Ti
WRN-50-2
Swin-S
Model
WRN-50-2
Swin-B
#Parameters
22.5M
27.5M
66.9M
48.8M 48.8M
Natural AA
68.46
74.36
CIFAR-10 (AA)
49.02
43.98
51.33
44.88
38.14
38.61
Imagenette (AA)
61.00
71.80
62.00
73.80
To show the potential of ViT's adversarial training, we adversarially train VITs following the above recipe and CNNs following the default setting, ensuring both of them own similar numbers of parameters. In Table Ia), CIFAR-10, similar to previous findings 9 ViTs indeed do not advance the adversarial robustness compared to CNNs, while we think that we should not be overly pessimistic about this, after all CIFAR-10 10 is low-resolution dataset, while ViTs are usually trained on high- resolution datasets. We might need a better adaption method for this input size difference. On the positive side, on high-resolution datasets like ImageNette (Table B(a)) and ImageNet- 1K (Table 3(b)), the robustness results totally different. On Imagenette, Transformers of different sizes (Swin-Ti for small size, and Swin-S for medium size) outperform their competitors (ResNet50 or WideResNett-5022 respectively) with a similar even smaller number of parameters by a notable margin (> 10%) On ImageNet-1K K (a more difficult dataset for adversarial tasks), we achieve robustness of 38.61 % [l which is higher than the current state-of- the-art result of 38. 14% on RobustBench. This
indicates the great potential for ViTs in terms of adversarial training.
4 An Architectural Perspective for Adversarially Robust VITs
In contrast CNNs, VITs are completely different architecture. For example, it divides images into sequences of patches before feeding into the model, and it directly integrates global information across the entire images even in the lowest layers using self-attention M1 This fundamental difference naturally raises a question: whether or not we are able to further improve its robustness based on such specific architectural information. In this section, we explore the impact of these two most obvious architectural changes (multi-head attention and patch-based based image splitting) inside ViTs on the ad adversarial robustness.
5 AAaamW with initial learning rate 5e-4 and weight decay 0.3 and SGD with the default setting in Section 'For the experimental details, please refer to Appendix D

---

1.
ML.P
MI.P
Badonwed Propaggtion
Norm
Norm
Parwart
Mulb- Head Attention
Norm
Embedded
Gradient
(a) ARD removes the gradients flowing through the multi-head attention modules with probability during the back propagation while keeping forward propa- gation unchanged.
(b) PRM randomly masks a proportion of perturba- tion with probability during the forward propagation while keeping backward propagation unchanged.
Figure 4: The diagram of our proposed method to improve the adversarial robustness of ViTs.
4.1 Attention Random Droppping (ARD)
In ViTs, embedded patches are repeatedly processed by encoder blocks, any of which consist of multi-head attention and a MLP layer, shown in aa) For the i-th block, its input 2;-1 is first processed by multi-head attention (-),),, and intermediate result z (=--1) z is further processed by the remaining parts (.) (including MLP layer and skip connection). For
ViT with L blocks, its output of the last (i.e., L-th) block is
Z1 ((L) (2 -f( (A (EL-1) ZL-1).
According to the chain rule in calculus, the gradient of a loss function L with respect the input of the last block 11.1 can then be decomposed as
Oc
OL dz,. OL (5()) OALL)
Dzz-1 + 1).
(5)
Hzl-1
d2g dZL-1
02g dz-
Here, we propose the Attention Random Dropping (ARD), which randomly drops the gradient flowing the multi-head attention (1)).
Oc
OL 0f(h)
0A(L) (uz θzL-1
+ 1),
(6)
dzl-1
dzy dz
where we set ur to with the probability of p and to otherwise. We continue to use the chain rule to obtain the gradient of ARD with respect to the input image
Oc
Oc
Of())
0A40)
Ozo
II
(u:
+ 1)
dr
dzL
dz
θz,-1
Ox
where zo the input embedded patches and all random variables uN are i.i.d. Figure 4a) also illustrates the backward propagation using ARD.
When ARD meets adversarial training, apply ARD as warming up strategy. After the warming- up periods, the training becomes vanilla adversarial training. In particular, the parameter p in ARD decreases linearly from 1 to 0 during the first ne epochs, i.e., p 1 n/Tn. where is the current epoch (n S n.w)) Here, p dynamic parameter along with the training process. At the beginning, relatively weak adversarial samples (large p) are used to warm up while as the training continues, the probability p should be gradually smaller to 0 to ensure the strength of the generated adversarial samples during the whole adversarial training process.
4.2 Perturbation Random Masking (PRM)
ViTs always split input images into non-overlapping patches, and then process them further. Con- sidering this patch-based image splitting, we introduce the Perturbation Random Masking (PRM).
7

---

Specifically, the perturbation on a patch will be masked with probability & during adversarial example generation:
8' c- M M
0C(f(x +0), + )),))
(8)
8 x- n. 88' sign(
08
where 8 is the adversarial perturbation, and M the mask to remove the perturbations on some patches. The second equation is exactly the update rule of PGD attack.
Similar ARD, incorporate PRM split into J patches, the perturbations on [J J kJ patches will be masked for each iteration during adversarial example generation within adversarial training, and we linearly decrease k from 1 to 0 at the first nw epochs, i.e., also a dynamic parameter along with the training process.
warming-up strategy. Considering image that is
n//n Nhe where the current epoch (n S ntw). Here, same as p.
4.3 Overall Algorithm
Obviously, ARD and PRM are two individual methods that have impact on two different aspects. ie.. ARD influences the backward propagation while PRM influences the update of adversarial perturbations. Thus, we can easily combine them as mixed warming-up strategy:
8' M
OC
Of()
0A
Ozo
(9)
- 11. 8' + a sign(
(u:
+ 1))
Ozl.
oz
0z-1
08
i=1
The details of ARD and PRM based adversarial training for ViTs are summarized Appendix C
4.4 Evaluation on Benchmark Datasets
In this section, we evaluate the effectiveness of ARD and PRM for the adversarial robustness of ViTs on benchmark datasets.
Experimental Settings. For models, we apply 4 kinds of ViTs (vanilla ViT M. DeiT 19), ViT 44 and Swin (2) and 3 kinds of scales (base, small, and tiny) for each respectively. For robustness evaluation, we also adopt 20 steps CW (27 (E.z version of CW loss optimized by PGD-20) and 100 steps PGD [3] in addition to PGD-20 and AA. We evaluate the performance on: 1) the vanilla setting (the basic setting determined in Section 33). 2) the setting with ARD, 3) the setting with PRM, and 4) the setting with ARD and PRM. To reveal the potential of our proposed methods, we select the
hyperparameter nh via grid search over (5, 10, 15, 20) for each combination of methods (+ARD, + +PRM, or + +both) and architectures (ViT-S, VIT-B, DeiT-Ti, et al.).
Experimental Results. In Table 4 after comparing the same kind of ViTs with different scales, we observe that increasing the size of ViTs can improve both the accuracy on clean examples and the robustness against adversarial attacks. Interestingly, the better architecture in natural training on the high-resolution dataset (e.g., Swin on ImageNet- - fails to lead better robustness in adversarial training on the low-resolution dataset. We conjecture the inductive bias (multi-scale objects and the high resolution of pixels on high-resolution images) inside Swin is longer valid low-resolution images, which results in unsatisfactory performance. Moreover, diving into each VIT model with different architectural approaches, robustness and accuracy, and the combination of both methods can improve the performance further. For example, compared to the vanilla adversarial training which achieves the robustness of 44.88% with Swin-S on CIFAR-10, we increase the robustness 46.00 46.04% (+ 1.16%) using ARD or 46.01 % (+ 1.13%) using PRM, while we further improve the performance to 46.17% (+ 1.29% using their combination. Similar results are also observed large datasets (i.e. ImageNet-Ikg in Appendix
find that our proposed ARD PRM has already boosted the
4.5 Combination with Other Defense Methods
Although the above discussions are mainly concentrated on standard adversarial training, we further demonstrate that our methods: ARD and PRM, can be casily combined with other stronger defense methods. We adopt TRADES [14] and MART [15] here because of their outstanding performance evaluated in (32. The settings are the same as in Section 4.4 except for two points: 1) we do not apply

---

Table 4: Performance (%) of ARD and PRM with different ViT variants on benchmark datasets. Note that 'B' denotes base, 'S' denotes 'small,, and 'Ti' denotes 'tiny". . Here we do not report the results of ViT-Ti and DeiT-B because ViT-Ti does exist and the architecture of DeiT-B is the as ViT-B. The best results are in bold.
CIFAR-10
Imagenette
Model
Method
Natural
CW-20
PGD-20
PGD-100
Natural
CW-20
PGD-20
PGD-100
vanilla
79.59
48.22
50.86
50.73
46.37
90.40
64.00
63.80
63.00
VIT-S
+ARD
81.70
49.07
51.72
51.42
47.12
91.40
64.20
64.60
64,20
62.80
+PRM
81.77
49.03
51.67
51.46
47.22
92.00
64.60
64.60
64.20
+both
81.86
49.09
51.73
51.46
47.33
91.40
64,20
65.20
64.60
63.00
vanilla
83.16
51.11
52.98
52.71
49.06
93.40
68.00
68.80
68.00
67.00
+ARD
84.21
51.61
53.41
53,10
49.65
94.80
69,40
68.60
68.20 28.20
68.20
VITBB
+PRM
84.31
52.16
53,79
53.47
50.01
94,40
70.60
69.80
69.40
69.20
+both
84,90
52.27
53.80
53.51
50.03
95.00
70.60
70.00
69.60
69.60
vanilla
75.46
45,40
48.
47,96
43.62
82,40
55.60
56,00
55,80
53.80
DeñT-Ti
+PRM
79.09
47
49,76
49.53
45.68
89 20
63,40
62.80
62.60
61.80
90.20
vanilla
81.43
49.53
51.88
51.75
47.40
92.20
64.20
64.60
63.40
63.40
DeiT-S
+ARD
81.76
49.49
$1.65
$1.43
47.55
90.80
67.00
66.00
66.00
65.80
+PRM
83.02
50.47
52.50
$2.26
48.27
...00
66.20
65.80
65.40
65.00
+both
83.04
50.52
52.52
52.36
48.34
91.00
67.00
66.60
66.20
65.80
vanilla
53.09
30.87
33.63
33.61
29.65
63.60
37.40
ConVIT-Ti
+ARD
79.87
47.54
50.14
49,90
45.63
90.20
64.00
63.60
63.40
63.20
+PRM
45.55
48.14
90.40
+both
80.28
47.47
49.86
49.55
45.42
90.40
65.40
65.00
64.80
64.40
vanilla
54.03
31.73
34.61
ConVTT-S
+ARD
84.06
50.83
52,72
52.44
48.71
94,00
68.40
67.80
67.60
67.20
+PRM
84.05
50.79
52.96
52.56
48.72
67.40
67.20
67.00
+both
84,32
50.94
53,10
52.81
48.85
94,40
68.80
68,20
67.80
67.60
vanilla
61.54
35.63
38.77
38.71
34.21
92.20
69.20
68 20
68 20
68.00
ConViT-B
+ARD
85.36
51.51
53.16
52.96
49.16
93.80
70.60
71.00
70.20
69.60
+PRM
85.48
51.48
52.83
$2.48
49.28
94.20
70.00
69. .40
69.40
69.20
+both
85.80
51.47
53.36
$3.05
49.33
95.20
72.60
73.00
72.20
70.60
vanilla
79.34
45.74
47.95
47.75
43.98
94.80
72.80
72.80
72.40
71.80
Swin Ti
+PRM
81.94
46.93
48.56
48.35
44.91
96.20
74.40
74.40
73.80
73.40
+both
48.87
96.20
vanilla
79.34
46.56
48.53
48.32
44.88
95.40
74.60
74.00
74.00
73.80
Swin-S
+ARD
82.07
47.84
49.56
49.31
46.04
96.00
75.60
75.00
74.80
74.60
+PRM
84,24
48.17
49.63
49.38
46.01
96.00
75.60
75.00
74.60
74.40
+both
84.46
48.52
50.02
46.17
96,00
76.00
75.00
75.00
74.80
vanilla
83.36
48.22
50.19
49.88
46.89
96.40
76.80
75.80
75.20
74.60
Swin-B
+ARD
81.24
47.64
49.19
48.83
44.38
97.00
78.00
77.20
76.40
75.80
+PRM 84.07
49.68
50.95
50.66
47.25
96.80
77.40
76.20
76.00
75.80
+both 84.16
49.78
51.47
$1.19
47.50
97.20
78.00
77.40
77.20
76.20
CutMix and Mixup for data augmentations to keep in line with their original papers, and 2) we always set The as 10 for simplicity. The experiments are conducted on DeiT-Ti, Con ViT-TT,i,, and Swin-Ti. As shown Table 5 the robustness and accuracy of TRADES and MART are improved on both datasets. For example, for DeiT-Ti on CIFAR-10, 10, AA robustness increases by 0.77% on TRADES and 1.63% on MART. It demonstrates that ARD and PRM can further improve the performance of existing defense methods.
4.6 Hyperparameter Analysis
For simplicity of our proposed warming-up up strategy, number of warming-up epochs for both ARD and PRM. Taking CIFAR-10 as an example, we firstly analyze this shared hyperparameter n E {0, 5, 10, 15, 20, 25 on four ViT variants (ViT, DeiT, Con ViT, and Swin), and the settings are the same as Section 4.4 AA used to evaluate the robustness. In Figure 5ja). the robustness is significantly improved as long adopting our proposed warming-up up strategy into adversarial training, i.e., n,, > O. We obtain the highest robustness at the best new (e.g., hyperparameter, which allows us to tune nuw easily.
the same hyperparameter nh as the
15 for Swin-S). Besides, the improvements are not sensitive to the
To further exploit the proposed method, we perform experiments on Swin-S using two individual hyperparameters n and ni, for ARD and PRM respectively. When fixing one (e.g., n(,) as 15 (the

---

Table 5: Performance (%) of our proposed ARD and PRM when combined with other defense methods. The best results are in bold.
Model
Method
CIFAR-10
Imagenette
Natural
CW-20
PGD-20
PGD-100
AA
Natural
CW-20
PGD-20
PGD- (0D
AA
IRADES
78.70
46.78
49.63
49.58
46.25
88.00
63.00
62.60
62.40
61.20
Deff-Ti
+Ours
80.24
47.60
51,02
50.97
47.02
89.00
63,20
64,40
64,00
61.80
MART
71.7
45.95
49.52
49.37
44.34
80.40
55,40
56.20
56.00
52.60
+Ours
74.89
47.60
51.18
51.16
45.97
86.40
61.80
63.40
63.20
62.20
TRADES
77.70
45.09
48.71
48.63
44.65
83.80
58.80
60.40
60.20
$7.80
CTnViT
80.02
47.33
50.10
50.08
46.75
89.20
66.20
65.60
65.00
64.60
+MART
63.68
38.97
42.80
42.77
37.62
61.80
36.40
41.80
41.60
35.40
+Ours
74.89
47.60
$1.18
45.97
88.00
65.00
64,40
64.40
63.40
TRADES
79.41
46.45
49.3
49.23
45,74
93.60
73.80
73.40
73.00
72,00
Swin-
80.71
46.36
75.60
74.20
74.20
+MART
75.19
46.10
49.82
49.71
44.54
92,40
71.60
70.20
69.60
68.60
+Ours
77.37
50.44
50.28
45.28
96.20
80.00
70.80
70.60
70.00
+1.
44.
ROTIMARRIS C-15)
(a) Sensitivity n.
nw on overall approach. (Left: AA Robustness, Right: Accuracy)
(b) Sensitivity of n. on each method individually. (Left: AA Robustness, Right: Accuracy)
Figure 5: Performance of ViTs under different hyperparameter configurations...
best one in the last experiment), we change another hyperparameter (e.g., m²) as {0, 5, 10, 15, 20, 25). In 5 (b), for robustness, these situations show different trends, i.e., longer warm-up with ARD (the orange line) increases the final robustness and a longer warm-up with PRM (the blue line) decrease the final robustness. We can find good balance when n = nn 15. The former has improved the performance enough, and the latter has not decreased too much. As a result, we set n n New by default.
5 Conclusion
In this paper, we conducted comprehensive experiments to investigate various training techniques to bring the first implementation benchmark for adversarial training of ViTs. We find that, under adversarial training. pre-training and gradient clipping are necessary while SGD is preferred over AdamW. W. Besides, taking the architectural information into consideration, we incorporated two simple but effective methods, ARD and PRM, into adversarial training to improve the adversarial robustness further. Extensive experiments demonstrated the effectiveness of our proposed methods. We hope our work not only provides the first implementation benchmark for adversarial training of ViTs, but also reminds researchers of the potential of architectural information contained within newly designed models like ViTs.
Acknowledgment
Yisen Wang partially supported by the NSF China (No. 62006153), Project 2020BD006 supported by PKU-Baidu Fund, and Open Research Projects of Zhejiang Lab (No. 2022RC0AB05).
References
[1] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
10

---

[2] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
[3] Alex Krizhevsky, Ily Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, 2012.
[4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
[6] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, al. Rethinking semantic segmentation from scquence-to-sequeece perspective with transformers. In CVPR, 2021.
[7] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020.
[8] Yifan Jiang, Shiyu Chang, and Zhangyang Wang. Transgan: Two pure transformers can make one strong gan, and that can scale up. In NewrlPS, 2021.
[9] Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yo Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers. In arXiv, 2021.
[10] Ahmed Aldahdooh, Wassim Hamidouche, and Olivier Deforges. Reveal of vision transformers robustness against adversarial attacks. In arXiv, 2021.
[11] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
[12] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In ICLR, 2015.
[13] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
[14] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
[15] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
[16] Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NeurlPS, 2020.
[17] Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. In ICLR, 2021.
[18] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In ICCV, 2017.
[19] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In ICML, 2021.
[20] Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with a reduced search space. In NeurlPS, 2020.
[21] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. ICLR, 2018
[22] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2018
[23] Herbert Robbins and Sutton Monro. A stochastic approximation method. The Annals of Mathematical Statistics, 1951.

---

[24] Liangqiong Qu. Yuyin Zhou, Paul Pu Liang, Yingda Xia, Feifei Wang, Fei-Fei, Ehsan Adeli, and Daniel Rubin. Rethinking architecture design for tackling data heterogeneity in federated learning. In CVPR, 2022.
[25] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV, 2021.
[26] Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. In Transactions on Machine Learning Research, 2022.
[27] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, In S&P, 2017.
[28] Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
[29] Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In ICLR, 2022.
[30] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
[31] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-frre attacks. In ICML, 2020.
[32] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. In NeurlPS Datasets and Benchmarks Track, 2021.
[33] Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness vision trans- formers to adversarial examples. In ICCV, 2021.
[34] Philipp Benz, Soomin Ham, Chaoning Zhang, Adil Karjauv, and In So Kweon. Adversarial robustness comparison of vision transformer and mip-mixer to cnns. In arXiv, 2021.
[35] Yutong Bai, Jieru Mei, Alan Yuille, and Cihang Xie, Are transformers more robust than cnns? In NeurIPS, 2021.
[36] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Tech Report, 2009.
[37] Jeremy Howard. Imagenette. http:///gtthhbboooo//sssaaaaaaet
[38] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In ICML, 2020.
[39] Maksym Andriushchenko, Francesco Croce. Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black- box adversarial attack via random search. In ECCV, 2020.
[40] Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. Understanding the exploding gradient problem. In arXiv, 2012.
[41] Leslic Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
[42] Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. In arXiv, 2020.
[43] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. In ICCV, 2019.
[44] Stéphane d' Ascoli, Hugo Touvron, Matthew L Leavitt, Ari Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In ICML, 2021.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix A
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix A
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material as URL)? [No] We will release upon acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 3
(c) Did you report error bars (e.g., with respect the random seed after running experi-
ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, cloud provider)? [Yes] See Section B
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [Yes] See Section 3
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you's 're using/curating? [No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "wxZ0_qhGtSS", "reviewer": "Reviewer_UDNi", "summary": "This paper benchmark tests several implementation details on adversarial training with Vision Transformers. Moreover, this paper proposes two modules, i.e., ARD and PRM to improve the adversarial robustness.", "questions": "In table 1, why ViT-B has the exact same awful performance for natural images and adversarial examples before the gradient clip? Does that mean that ViT-B does not work on natural image classification at all? And about the accuracy on Cifar-10 dataset, the 10.00% seems pretty magical. Do you explore whether this number is because the adversarial examples from nine out of ten classes can fool the model 100 percent, while the remaining one can not fool the model at all, or the number 10.00% just magically happens?", "limitations": "While there are lots of benchmark tests that provide empirical hints for practical applications, I personally expect discussions about the reason for such observations. For example, why ViT-B suffers the gradient explosion while Swim-B does not? How about other Transformer models? Why “SGD+piecewise” performs better than “AdamW+cyclic”, and is that true for all Transformer models?\nWith such discussions, I think it would be more useful in guiding practical applications.\n\nMoreover. some conclusions are very difficult to be applied in practice. For example, the authors claim that “it is a good choice to use advanced data augmentations (e.g., Cutmix and Mixup), …”, but “RandAugment is a more advanced technique, which is too difficult for AT of ViTs”, then how “advanced” the data augmentation should be used? \n\nThe proposed ARD and PRM modules look interesting, but their effectiveness is not well justified by the experiments, especially combining them together shows lower performance in many cases. Is there any reason for this phenomenon?\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "There are a number of benchmark tests, which is really appreciated.\nThe proposed ARD and PRM modules look interesting.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qAa-TZmKDAQ", "reviewer": "Reviewer_nQpy", "summary": "\nSummary:\n\nI think this article makes a huge contribution to the adversarial robustness of ViT. It is not just a simple application of adversarial training suitable for CNN to ViT, but it has made substantial contributions, including detailed and reasonable analysis and proposing new AT techniques suitable for ViT. I like this paper very much and look forward to its early publication. It would undoubtedly be a great contribution to the community if the authors could publish the trained robust models in time. I'm looking forward to it being included in the robustness library.\n", "questions": "\n\nPlease answer the questions in \"Strengths And Weaknesses.\"\n\n", "limitations": "\nThe authors adequately addressed the limitations and potential negative societal impact of their work.\n\n", "rating": 9, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\n(Positive) This paper provides a comprehensive evaluation of various training techniques across several datasets, thus bringing the first implementation benchmark for adversarial training of ViTs.\n\n\n(Positive) It is not just a simple application of adversarial training suitable for CNN to ViT, but it has made substantial contributions, including detailed and reasonable analysis and proposing new AT techniques suitable for ViT.\n\n(Positive) CNN and ViT are indeed very different, so the AT of CNN should be different from that of ViT. For example, compared to CNN, ViT requires Gradient Clipping and pre-training, prefers advanced DA, and is more prone to robust overfitting.\n\n\n(Positive) Although we know the difference between CNN and ViT under vanilla training, relying on this known knowledge is not enough. For example, we know that ViT favors AdamW in Vanilla training, but the authors found that adversarial training of ViT favors SGD.\n\n(Positive) The authors put forward two distinctive AT methods, PMR and ARD, specifically for the characteristics of ViT.\n\n(Positive) Indeed, as the authors say, although ViT and CNN are very different, ViT trained only by vanilla training is not significantly robust (only a small effect under extremely small interference).\n\n(Positive) According to the protocol on CIFAR-10, it is reasonable for the authors to change the input of ViT to 32x32.\n\n(Positive) Similarly, in my experience, it is reasonable for the authors to use AA to test the robustness of ViT.\n\n(Positive) It is reasonable for the authors to use PGD-10 to perform AT on ViT.\n\n\n(Positive) The authors' results on the AT for ViT are reasonable and hopefully will be published sooner for people to use.\n\n\n(Negative) Figure 2 shows that the effect of pre-training with ImageNet-1k is slightly higher than that of ImageNet-21k. Can the authors give some reasonable explanations?\n\n\n\n(Negative) The content of formula (4) to formula (7) is too simple, and the existence of these formulas is unnecessary. These formulas can be simplified or omitted unless the authors want to use these formulas to give theoretical results to their method.\n\n\n(Question) It can be seen from Table 3 that the effect of PRM is better than that of ARD in general. I hope the authors give some reasonable explanations.\n\n(Positive) The authors demonstrate that their method can be combined with and improve the performance of existing AT methods. This is very inspiring.\n\n(Positive) The authors are commendable for their hyperparameter analysis of the number of epochs for warmup.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "WF8uwTbD0WL", "reviewer": "Reviewer_63WY", "summary": "This paper analyzes and improves the robust training performance with vision transformers. If first rigorously analyze the impact of various factors on robust training and then proposes novel methods, namely attention random dropping (ARD) and perturbation random masking (PRM), to further boost the performance. ", "questions": "I am also puzzled why a very small number of epochs are chosen (40 by default). ViTs are generally trained for longer schedules (often 300 epochs), especially when trained from scratch. Was computational cost the only reason behind this design choice? \n\nSimilarly, another puzzling trend is in fig 2. In fig. 2a, pre-training on ImageNet-1k achieves better performance than pertaining on much bigger ImageNet-21K. This is strange as ImageNet-21K tends to always perform better. Similarly in fig. 2b, the training accuracy of the model itself doesn’t converge to very high values, suggesting that the bottleneck in pre-training from scratch is the optimization process, not generalizing. Given the large number of parameters in ViTs, shouldn’t they be able to fit the data on the training set?", "limitations": "The limited performance of ViTs, in comparison to CNNs, is not highlighted in the paper. I encourage authors to discuss this limitation with concrete comparisons. ", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is among the first works to analyze robust training performance on vision transformers. It uniquely takes a deep dive into the fields by rigorously analyzing the impact of various factors in robust training on the final performance.\n\n----\n\nThe major limitation of this work is the highly sub-optimal performance of vision transformers in adversarial training. For example, after all proposed innovations (including pre-training, warmup strategies, careful augmentations, etc.) the best robust accuracy (AA) achieved on cifar-10 is 50.01 with ViT-B. In contrast, a ResNet-18 achieves 51.06 robust accuracy (AA) without any pre-training (so no extra data or other bags of tricks) [Source: RobustBench]. Note that a ResNet-18 has at least 10x (if not more) lower number of parameters than ViT-B. With extra data, the ResNet-18 network can further achieve 57.67 robust accuracy (AA), much better than the best achieved with very large ViTs in the paper. \n\nThe aforementioned critique also signals towards another critical comparison that is missing in the paper: A comparison of ViTs and CNNs performance. ViTs continued to be adopted in the community over CNN since they outperformance CNNs. To make a case for the use of ViTs in a robust training regime, one needs to show a similar benefit of ViTs over CNNs. However, I couldn’t find any such comparison in the paper (as I argued above the performance of ViTs is far suboptimal than CNNs in the current paper).  \n\nAnother troubling trend on the performance front (cifar-10) is that Swin, which is much better architecture than baseline ViT-B, has poorer performance (table 1, 2). Not just for Swin, but this phenomenon is also true for other modern architectures that are better than baseline ViT. This makes me question whether the proposed innovations are any effective at lower resolutions since they are unable to utilize better architectures. Only at higher resolution (ImageNette), they perform better with better architectures. This should be clearly clarified in the paper. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "415yUK1kCD", "reviewer": "Reviewer_JwsQ", "summary": "This paper presents two empirical strategies to improve the effectiveness of adversarial training in vision transformer models. The authors first analyzed the effects of gradient clipping, pre-training, data augmentations, optimizer, and training schedules. They further proposed two tricks, attention random dropping (ARD) and perturbation random masking (PRM) to improve the adversarial robustness. Evaluations show that both methods could contribute to the robustness enhancements. ", "questions": "1. The authors do not deliver a further in-depth analysis or ablation study on the reason behind these two tricks. \n2. In Table 3, the combination of ARD and PRM fails to deliver better results. I assume the reason could be the regularization will be too strong, but I expect to see the answers from the authors.", "limitations": "There is no potential negative societal impact of this work.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written, well-organized, and easy to follow.\n2. The evaluation is thorough and sufficient.\n\nWeakness:\n1. There is no in-depth analysis or ablations on how these different vectors affect the adversarial training.\n2. The technical novelty is limited, and the authors fail to provide intuition on why these two tricks would work. \n3. Although the authors provide the chain rule of gradient propagation for these two tricks, the overall idea is still similar to dropout-style regularization. Therefore, the authors need to provide more baselines on different regularization methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept