On the Adversarial Robustness of Vision Transformers
Authors: Rulin Shao
Zhouxing Shi
Jinfeng Yi
Pin-Yu Chen
Cho-Jui Hsieh
Venue: ICLR 2022 Submitted (2022)
Keywords: vision transformer (ViT)
adversarial robustness

Abstract: Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides the first and comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs). This observation also holds for certified robustness. We summarize the following main observations contributing to the improved robustness of ViTs:
   1) Features learned by ViTs contain less low-level information and are more generalizable, which contributes to superior robustness against adversarial perturbations. 
   2) Introducing convolutional or tokens-to-token blocks for learning low-level features in ViTs can improve classification accuracy but at the cost of adversarial robustness.
   3) Increasing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. But for a pure transformer model, simply increasing the size or adding layers cannot guarantee a similar effect.
   4) Pre-training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs. 
   5) Adversarial training is also applicable to ViT for training robust models.
Furthermore, feature visualization and frequency analysis are conducted for explanation. The results show that ViTs are less sensitive to high-frequency perturbations than CNNs and there is a high correlation between how well the model learns low-level features and its robustness against different frequency-based perturbations.

Under review as a conference paper at ICLR 2022
ON THE ADVERSARIAL ROBUSTNESS OF VISION TRANSFORMERS
Anonymous authors
Paper under double-blind review
ABSTRACT
Following the success in advancing natural language processing and understand- ing, transformers are expected to bring revolutionary changes to computer vision. This work provides comprehensive study on the robustness of vision transform- ers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs). This observation also holds for certified robustness. We summarize the following main observa- tions contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less low-level information and more generalizable, which contributes to superior robustness against adversarial perturbations. 2) Introducing convolu- tional or tokens-to-token blocks for learning low-Ievel features in ViTs can im- prove classification accuracy but at the cost of adversarial robustness. 3) Increas- ing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. But for pure transformer model, simply increasing the size or adding layers cannot guarantee a similar effect. 4) Pre-training without adversarial training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs. 5) Adversarial training is also applicable to ViT for training robust mod- els. Furthermore, feature visualization and frequency analysis are conducted for explanation. The results show that ViTs are less sensitive to high-frequency per- turbations than CNNs and there a high correlation between how well the model learns low-level features and its robustness against different frequency-based per- turbations.
INTRODUCTION
Transformers are originally applied in natural language processing (NLP) tasks as a type of deep neural network (DNN) mainly based on the self-attention mechanism (Vaswani al.) (2017): De- vlin et al. (2018); :): Brown et al. (2020)) and transformers with large-scale pre-training have achieved state-of the-atte-rt results on many NLP tasks (Devlin al. (2018); et al. (2019); Yang et (2019); Sun et al. (2019)). Recently, Dosovitskiy et al. (020)) applied a pure transformer directly to se- quences of image patches (i.e., a vision transtormer, ViT) and showed that the Transformer itself can be competitive with convolutional neural networks (CNN) on image classification tasks. Since then transformers have been extended to various vision tasks and show competitive or even better performance compared to CNNs and recurrent neural networks (RNNs) (Carion et al. (2020); Chen al. (2020); Zhu et al. (2020). While ViT and variants hold promise toward a unified machine learning paradigm and architecture applicable different data modalities, it remains unclear on the robustness of ViT against adversarial perturbations, which is critical for safe and reliable deployment of many real-word wealwwrldd applications.
In this work, we examine the adversarial robustness of ViTs on image classification tasks and make comparisons with CNN baselines. As highlighted in Figure [[a). our experimental results illustrate the superior robustness of ViTs than CNNs in both white-bon and black-box attack settings, based on which we make the following important findings:

---

Under review as a conference paper at ICLR 2022
VIT/DeTT
Transformer
WISST
T2T-VT
15M
T2T
Transformer
VITA/16
50
T2TTatt-3 34
45
Swin- Tran-Trasforrer
Cas betracze
Swe Sol
Swin Transfermer Swin Trunsfermer
35
30
Reshattoknn
Reshetlo
Hybrid
Res-ViT
CNN
Transfommer
SEResNet
CNN
CNN
CNN
CNNS
70
72
74
76
78
8D
82
CNN
CNN
Accuracy
(a) Robust Accuracy Clean Accuracy
(b) Model Architecture
Figure 1: (a) Robust accuracy v.s. clean accuracy. The robust accuracy is evaluated by AutoAttack (Croce & Hein (020 The "Hybrid" class includes CNN-ViT, T2T-ViT and Swin-T as introduced Section Models with attention mechanisms have their names printed the center of the cir- cles. ViTs have the best robustness against adversarial perturbations. Introducing other modules ViT can improve clean accuracy but hurt adversarial robustness. CNNs are more vulnerable to adversarial attacks. (b) Vision transformers and CNNs investigated in this paper.
Features learned by ViTs contain less low-level information and benefit adversarial robustness. ViTs achieve lower attack success rate (ASR) of 51.9% compared with minimum of 83.3% by CNNs in Figure [(a) They are also less sensitive to high-frequency adversarial perturbations.
Using denoised randomized smoothing (Salman et al. 2020), ViTs attain significantly better cer- tified robustness than CNNs.
It takes the cost of adversarial robustness to improve the classification accuracy of ViTs by intro- ducing blocks to help learn low-level features as shown Figure [(a)
Increasing the proportion of transformer blocks in the model leads better robustness when the model consists of both transformer and CNN blocks. For example, the attack success rate (ASR) decreases from 87.1% to 79.2% when 10 additional transformer blocks are added TiT-ViT-14. However, increasing the size of pure transformer model cannot guarantee similar effect, e.g.. the robustness of ViT-S/16 better than that of ViT-B/16 in Figure [(a)
Pre-tres Pre-training without adversarial training on larger datasets does not improve adversarial robustness though it is critical for training ViT.
The principle of adversarial training through min-max optimization (Madry et al. 2017 y Zhang al. 2019)) ) can be applied train robust ViTs.
2 RELATED WORK
Transformer Vaswani et al. (017)) has achieved remarkable performance many NLP tasks, and its robustness has been studied on those NLP tasks. Hsieh al.) (2019); Jin al. (220 Shi & Huang (2020): Li et al. (2020): Garg & Ramakrishnan (2020): Yin et al. 2020) conducted adversar- ial attacks on transformers including pre-trained models, and their experiments transformers usu- ally show better robustness compared other models based Long short-term memory (LSTM) or CNN, with theoretical explanation provided in Hsich et al. (2019). However, due to the discrete nature of NLP models, these studies are focusing discrete perturbations (e.g., word character substitutions) which are very different from small and continuous perturbations in computer vision tasks. Besides, Wang et al. (2020a) improved the robustness of pre-trained transformers from
2

---

Under review as a conference paper at ICLR 2022
information information--thorrtiic perspective, and (Shi al. (2020) Ye et al. 2020); Xu et al. (2020) studied the robustness certification of transformer based models. To the best of our knowledge, this work one of the first studies that investigates the adversarial robustness (against small perturbations in the input pixel space) of transformers on computer vision tasks. There are some concurrent works studying the adversarial robustness of ViTs. We supplement their contribution in Appendix G
3 MODEL ARCHITECTURES
We first review the architecture of models investigated in experiments including several vision transformers (ViTs) and CNN models. A detailed table of comparison is given in Table
3.1
VISION TRANSFORMERS
We consider the original VIT Dosovitskiy et al. (2020) and its four variants shown in Figure ((b)
Vision transformer (ViT) and data-efficient image transformer (DeiT): ViT (Dosovitskiy al. 2020 mostly follows the original design of Transformer (Vaswani et al. (2017) Devlin et al. 20188) on language tasks. For 2D image E "Xxx with resolution H W and C' channels, is divided into sequence of N = pd
H-W flattened 2D patches size P Px E R- (1 i < N). The patches are first encoded into patch embeddings with simple convolutional layer, where the kernel size and stride of the convolution is exactly P x P. In addition, there are also position embeddings to preserve positional information. Similar to BERT (Devlin al. 2018)) a large-scale pre-trained model for NLP, a special [CLS] token is added to output features for classification. DeiT (Touvroo al. (202T) further improves the ViT's performance using data augmentation or distillation from CNN teachers with an additional distillation token. We investigate ViT-(S.B,LL/IE DeiT-S/16 and Dist-DeiT-B/6 defined the corresponding papers in the main text and discuss other structures in Appendix
F
Hybrid CNN and VIT (CNN- Dosoviiskiy al. (2020) also proposed hybrid architecture for ViTs by replacing raw image patches with patches extracted from CNN feature map. This is equivalent to adding learned CNN blocks to the head of ViT as shown in Figure [(b]] Following Dosovitskiy et al. (2020), we investigate ViT-B/16-Re in our experiments, where the input sequence
is obtained by flattening the spatial dimensions of the feature maps from ResNet50.
Hybrid of T2T and VIT (T2T-ViT): Yuan et al. (2021) proposed to overcome the limitations of the simple tokenization ViTs, by progressively structurizing an image to tokens with a token-to-token (T2T) module, which recursively aggregates neighboring tokens into one token such that low-level structures can be better learned. T2T-ViT was shown to perform better than VIT when trained from scratch on midsize dataset. We investigate T2T-ViT-14 14 and T2T- 2TT-ViT-24 24 in our experiments.
Hybrid of shifted windows and ViT (Swin-T): Liu et al. (021) computes the representations with shifted windows scheme, which brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. We investigate Swin-S/4 in the main text and discuss other structures in Appendix
3.2 CONVOLUTIONAL NEURAL NEWORKS
We study several CNN models for comparison, including ResNet18 (He et al. (2016)), ResNet50- 32x4d (He et al. (2016)). ShuffleNet (Zhang et al. (2018)). MobileNet (Howard et al. (2017)) and VGG16 (Simonyan Zisserman| (2014). We also consider the SEResNet50 model, which uses the Squeeze-and- Exxeitttinni (SE) block (Hu et al. (2018)) that applies attention channel dimensions to fuse both spatial and channel-win wise information within local receptive fields at each layer.
The aforementioned CNNs are all trained on ImageNet from scratch. For better comparison with pre-trained transformers, we also consider two CNN models pre-trained on larger datasets: ResNeXt-32x4d-sst (Yalniz al. (2019)) pre-trained YFCC100M (Thomee et al. (2015)), and ResNet50-swsll pre-trained on IG- IB -B-T-Bggtte (Mahajan et al.] (2018) using semi-weakly super- vised methods (Yalniz et al. 2019).. They are both fine-tuned ImageNet.
3

---

Under review as a conference paper at ICLR 2022
4 ADVERSARIAL ROBUSTNESS EVALUATION METHODS
We consider the commonly used & -norm bounded adversarial attacks to evaluate the robustness of target models. An Lo attack is usually formulated as solving a constrained optimization problem:
max L (xxdv, .v) S.t. xadv
xx - -xol SC
(1)
where Xy is clean example with label y, and we aim to find adversarial example xade within an to ball with radius centered at Xe, such that the loss of the classifier & (xrdu, y) is maximized. We consider untargeted attack in this paper, so attack is successful if the perturbation successfully changes the model's prediction. The attacks as well as randomized smoothing method used in this paper are listed below.
White-box attack Four white-box attacks are involved in our experiments. The Projected Gradi- ent Decent (PGD) attack (Madry et al. (2017) solves Eq. byy iteratively taking gradient ascent:
adv X
Clipxo.e (x, (x²do + a sgn (VxJ (x, adv :y)))))
where xqdu stands for the solution after iteeaatons, and Clipxo.e() (.) denotes clipping the values to make each x1+1, within [xo.; 6, Xo.i + e), according to the Eoo threat model. As special case, Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2014)) uses single iteration with 1. AutoAttack (Croce& & Hein (020)) is currently the strongest white-box attack which evaluates ad- versarial robustness with a parameter- free ensemble of diverse attacks. We also design a frequency- based attack for analysis, which conducts attack under additional frequency constraint:
edv free
IDCT(DCT(X)))))) (x pgd
xo) O Mj) + Xo,
(3)
where DCT and ICCT stand for discrete cosine transform and inverse discrete cosine transform respectively, xpdd stands for the adversarial example generated by PGD, and My stands for the
mask metric defined the frequency filter which is illustrated in Appendix B We found this design similar to Wang et al. al. (2020b).
Black-box attack We consider the transfer attack which studics whether an adversarial pertur- bation generated by attacking the source model can successfully fool the target model. This test not only evaluates the robustness models under the black-box setting, but also becomes a sanity check for detecting the obfuscated gradient phenomenon (Athalye al. (2018)). Previous works have demonstrated that single-step attacks like FGSM enjoys better transferability than multi-step attacks (Kurakin al. 20177). We thus FGSM for transfer attack in our experiments.
Denoised Randomized Smoothing We also evaluate the certified robustness of the models using randomized smoothing, where the robustness is evaluated as the certified radius, and the model is certified to be robust with high probability for perturbations within the radius. We follow Salman et al. (2020) to train DnCNN (Zhang et al. (2017)) denoiser Dg for each pre-trained model f with the "stability" objective with Lcs denoting cross entropy and / denoting Gaussian distribution:
Lstab Ber.s.een,.fox(((fodt + fxx))) where NN(0, o²)).
Randomized smoothing is applied on the denoised classifier f o Da for robustness certification:
g (x) = arg max (x + 8)) = c] where 8 ~ (0,
(5)
<cy
The certified radius is then calculated for the smoothed classifier as (Cohen et al. 2019):
R
2 ( (ps) a (p())))),
(6)
where t the inverse of the standard Gaussian CDF, PA = -(fx=(=( CA) is the confidence of the 1 predicted class Ca. and P3 max (x + 5) = c) is the confidence for the second top class. Accordingly, given a perturbation radius, the certified accuracy under this perturbation radius can be evaluated by comparing the given radius to the certified radius R.
4

---

Under review as a conference paper at ICLR 2022
5 EXPERIMENTAL RESULTS
In the experiments, we show the superior robustness of ViTs over CNNs against both white-box attacks and transfer attacks. We also study their certified robustness with the denoised randomized smoothing technique and conduct preliminary adversarial training experiments on VIT. Based on the experiments, we analyze what grants ViTs to adversarially robust through different lens.
For all experiments, we load pre-trained ViT models and CNNs from the PyTorrc image models library (timm, Wightman (2019) and torchvision (Paszke et al. (20T9h) respectively. We evaluate the clean accuracy of each model on the whole test set of ImageNet- Ik (Deng et al. (2009)), and we sample 1,000 test examples to evaluate robust accuracy and attack success rate (ASR). Note that lower ASR means better robustness. Experimental results CIFAR-10 10 are reported in Appendix For adversarial training we use CIFAR-10 (Krizhevsky et al. 2009)) as detailed in Section 3.4
5.1 ROBUSTNESS UNDER WHITE-Box ATTACKS
Settings We use PGD and AutoAttack to study the robustness under white-box attacks. We con- sider attack radius e from {0.001, 0.003, 0.005, 0.01). For PGD attack, fix the attack steps to Witer = 40 with other parameters following the default setting of the implementation in Foolbox RRuber et al. 2020)). AutoAttack does not require any hyper-parameter tuning.
Results We present the results using PGD attack in Table I and the results using AutoAttack in T The ASR is approximately 100% on all the models when e is large, e.g. when = 0.01. But for smaller attack radii, ViT models have lower ASR than CNNs under both PGD attack and AutoAttack. For example, when = 0.001, the ASR for ViT-S/16 is only 44.6% while the ASRs for CNNs are least 70.0%. And under the same attack radius, the ASR of AutoAttack for ViT- S/16 is only 51.9% compared to 93.9% of ShuffleNet. Besides, the results show that AutoAttack is much stronger than PGD attack under the same c. These results demonstrate that ViT is more robust than CNNs under these white-box attacks. We also visualize the clean/robust accuracy tradeoff and model size of these models in Figure [(a)
Table I: Attack success rate (%) of target mod- els against 40-step PGD attack with different radii and the clean accuracy ("clean acc").
Table 2:
Attack success rate (%) of target models against Autokt with different attack radii, in a similar format as Table !
Clean Acc
PGD Attack radius
Clean Acc
AutoAttack attack radius
Model
0.001
0.003
0.01
Model
0.005
VIT-S/16
77.6
44.6
75.4
89.8
99.0
VIT-S/16
77.6
51.9
94.0
99.5
100.0
VIT-B/16
75.7
51.1
85.4
94.0
99.1
VIT-B/16
75.7
60.2
94.6
99.4
100.0
VIT-L/16
79.2
44.9
76.6
90.1
98.2
VIT-L/16
79.2
53.4
91.5
99.0
100.0
VIT-B/16-Res
84.0
54.5
91.6
97.7
99.9
VIT-B/16-Res
84.0
72.3
99.1
100.0
100.0
T2T-ViT-14
80.1
62.9
93.0
98.2
100.0
T2T-VIT-14
80.1
87.1
99.9
100.0
100.0
T2T-VIT-24
82.2
52.3
87.7
96.6
99.8
T2T-VIT-24
82.2
79.2
99.7
100.0
100.0
Deit-S/16
77.7
51.1
82.4
92.9
98.9
Dist-Deit-S/16
79.3
56.9
96.3
99.8
100.0
Dist-Deit-B/16
81.8
44.4
82.3
95.5
99.6
Dist-Deit-B11
81.8
57.3
96.6
99.8
100.0
Swin-S/4
81.8
60.0
87.6
96.8
99.8
Swin-S/4
81.8
92.1 I
99.9
100.0
100.0
SEIRssNe150
75.7
64.6
95.1
99.2
99.9
78.4
100.0
ResNeXt-32xAd-si
80.3
77.0
97.1
98.8
99.5
ResNeXt-32x4d-ssi
80.3
93.5
100.0
100.0
100.0
ResNet50
81.2
75.3
97.1
99.6
ResNet50
91.9
100.0
ResNer18
70.0
75.1
98.0
99.4
99.9
ResNer18
70.0
85.7
99.6
100.0
100.0
CesNet50.32x44
77.6
71.8
96.8
99.6
ResNet50.
77.6
86.8
100.0
ShuffleNet
69.4
85.0
99.4
99.8
100.0
ShuffleNet
69.4
93.9
100.0
100.0
100.0
MohileNet
71.9
83.3
99.6
100.0
MobileNet
71.9
92.2
100.0
100.0
100.0
VGG16
71.6
73.7
96.8
98.7
99.4
VGG16
71.6
83.3
99.5
100.0
100.0
5.2 ROBUSTNESS UNDER TRANSFER ATTACK
We also conduct transfer attack to test the adversarial robustness the black-box setting as described in Section We consider attacks with -nom perturbation no larger than 0.1 and present the results in Figure When the ViTs serve as the target models and CNNs serve as the source models, as shown in the lower left of each subplot, the ASR of the transfer attack is quite low. On the other hand, when the ViTs are the source models, the adversarial examples they generate have higher ASR when transferred to other target models. As result, the first three rows and the last seven columns
5

---

Under review as a conference paper at ICLR 2022
attack radius 0.1
attack radius 0.01
12T41124
-64
5404
LINO
care
0.10
Ko30
8:30
0391
Raa00 and
6-M
4:me 4.2%
0.714
7m 44
6:44
-84
6274
-01
V5G16
-62
V0222
-92
-50
-02
Figure 2: ASR of transfer attack using FGSM with different attack radii. The rows stand for the surrogate models used to generated adversarial examples. The columns stand for the target models. Darker rows correlate to the source models that generate more transferable adversarial examples. Darker columns mean that the target models are vulnerable to transfer attack." attack."Re550e "Res50-ssI" and "Res50-see swsl" are in short of CeesNcXt-32x4dax and "ResNet50-swsl" respectively. Results for more radii can be found in Appendix
Table 3: Results of adversarial training for dif- ferent models using PGD-7 (7-step PGD attack) and TRADES respectively on CIFAR-10. VIT- B/4 is a variant of ViT-B/16 where we down- sample the patch embedding kernel from 16x 16 to 4 x to accommodate the smaller image size CIFAR-10. We report the clean accuracy (%) and robust accuracy (%) evaluated with PGD- 10 and AutoAttack respectively. Each model is trained using only 20 epochs to reduce the cost.
VIT-5/16
ResNet18
Certified Accuracy
ResNet50
02-
05
6.
Model
Method
Clean
PGD-10
AutoAttack
Radius
PreAcdResNer18
PGD-7
77.3
48.9
44.4
Figure 3: Certification of ViT-S/16, ResNet50 and ResNet18 ImageNet classifiers using de- noised randomized smoothing (a = 0.25).
TRADES
27.6
49.4
44.9
WidbResNar-4-100
PGD-7
80.3
52.2
48.4
TRADES
81.6
VIT-B/4
PGD-7
85.9
51.7
47.6
TRADES
darker than the others. Besides, for the diagonal lines in the figure where FGSM actually attacks the models in a white-box setting, we can observe that ViT' are less sensitive to attack with smaller CNNs, and T2T modules make ViTs more robust such one-step attack. In addition, ad adversarial examples transfer well between models with similar structures. As ViT-S/16, VIT-B/16 and ViT-L/16 have similar structures, the adversarial examples generated by them can transfer well to each other, and it is similar for T2T-ViTS and CNNs respectively.
radii compared
5.3 CERTIFIED ROBUSTNESS
Settings We train the denoisers using the stability objective for 25 epochs with a noise level of a = 0.25, learning rate of 10-5 and a batch size of 64. We iterate through the ImageNet dataset to calculate the corresponding radii according to Eq. 6 and report the certified accuracy versus
different radii as defined in Salman et al. (2020) in Figure
Results As shown in Figure VIT-S/16 has higher certified accuracy than ResNet18, showing better certified robustness of vision transformers over CNNs. We also found that, in the same set- tings, training a Gaussian denoiser for the ResNet18 is harder than for the ViT-S/16. Accuracy of VIT-S/16 with denoiser at noise of 0.25 is 64.84% (4.996% without any denoiser), while accu- racy of ResNet50 and ResNet18 with denoiser at the same noise is 47. 47.782% (5.966% without any denoiser).

---

Under review as a conference paper at ICLR 2022
5.4 ADVERSARIAL TRAINING
Settings We also conduct a preliminary experiment on adversarial training for ViT. For this ex- periment we use CIFAR-10 (Krizhevsky et al. (2009) with = 8/255 and the ViT-B/16 model. Since originally this ViT was pre-trained on ImageNet with image size 224 x 224 and patch size 16 x 16 while image size on CIFAR-IO is 32 x 32, we downsample the weights for patch embed- dings and resize patches to 4 x 4, so that there are still 8 x 8 patches and we name the new model as ViT-B/4. Though ViT originally enlarged input images on CIFAR- for natural fine-tuning and evaluation, we keep the input size as 32 x 32 to make the attack radius comparable. For training, we use PGD-7 (PGD with 7 iterations) (Madry et al. (2017) and TRADES (Zhang et al. (2019) methods respectively, with no additional data during adversarial training. We compare ViT with two CNNs. ResNet18 (He et al. (2016)) and WideRes.Net 34- 10 Zaaoruykoo & Komodakis (2016)) To save training cost, train each model for 20 epochs only. although some prior works used around hundreds of epochs (Madry et al. 2017);[ Pang et al. (2020)) and are very costly for large models. We use batch size of 128, an initial learning rate of 0.1, an SGD optimizer with momentum 0.9, and the learning rate decays after 15 epochs and 18 epochs respectively with a rate of 0.1. While we use a weight decay of 5 10- for CNNs suggested by Pang al. (2020) that 5 x 10-4 is better than 2 x 10-4, we still use 2 x 10- for VIT as we find 5 x 10 causes an underfitting for ViT. We evaluate the models with PGD-10 (PGD with 10 iterations) and AutoAttack respectively.
Results We show the results in Table The ViT model achieves higher robust accuracy compared to ResNet18, and comparable robust accuracy compared to WideRes.Net-34-1 10, while ViT achieves much better clean accuracy compared to the other two models. Here ViT does not advance the robust accuracy after adversarial training compared to large CNNs such as WideResNett34-10. We conjecture that VIT may need larger training data or longer training epochs to further improve its robust training performance, inspired by the fact that on natural training ViT is not able to perform well either without large-scale pre-training. And although ViT-ViT improved the performance of natural training when trained from scratch, our previous results in Table I and Table 2 show that the T2T-VIT structure may be inherently less robust. We have also tried Wong et al. (2020) which was proposed to mitigate the overfitting of FGSM conduct fast adversarial training with FGSM, but we find that it can still cause catastrophic overfitting for ViT such that the test accuracy on PGD attacks remains almost O. We conjecture that this fast training method may be not suitable for pre- trained models or require further adjustments. Our experiments in this section demonstrate that the adversarial training framework with PGD TRADES is applicable for transformers on vision tasks, and we provide baseline results and insights for future exploration and improvement.
6 REASONING OF ADVERSARIAL ROBUSTNESS
In this section, we present extended analysis to dissect the source of improved adversarial robustness in ViTs. We also verify that ViT's improvement is not caused by insufficient attack optimization, and an explanation from Hopfield network perspective is provided (see Appendix D for details).
Learning LoweLevel Structures Makes Models Less Robust One interesting and perhaps sur- prising finding is that ViTs have worse robustness when modules that help to learn local structures are added ahead of the transformer blocks. For example, T2T-ViT adds several T2T modules to the head of ViT which iteratively aggregates the neighboring tokens into one token in each local percep- takes the features generated by ResNet as inputs, which has the same effect
tive field. ViT-B/16- as incorporating a trained CNN layer in front of the transformer blocks. Both modules help to learn local structures like edges and lines (Yuan et al. 2202
When the features learned by ResNet are introduced, the ASR of ViT-B/16 rises from 51.1% to 54.5% of ViT-B/16-Res under PGD attack, and from 60.2% to 72.3% under AutoAttack, with attack radius e = 0.001. A similar phenomenon can be observed by comparing the ASR of ViTs and T2T-VITs. The ASR of T2T- 22T-ViT-14 14 is 18.3% higher under PGD attack and 35.2% higher under AutoAttack compared with the ASR of ViT-S/16, under attack radius c = 0.001.
One possible explanation is that the introduced modules improve the classification accuracy by membering the low-level structures that repeatedly appear in the training dataset. These structures such as edges and lines are high-frequent and sensitive to perturbations. Learning such features
7

---

Under review as a conference paper at ICLR 2022
makes the model more vulnerable to adversarial attacks. Examination of this hypothesis is con- ducted in Section 6.1
Increasing the Proportion of Transformer Blocks Can Improve Robustness Hendryykk et al. (2019) mentioned that larger model does not necessarily imply better robustness. It can be con- firmed by our experiments where VIT-S/16 shows better robustness than larger ViT-B/16 under both PGD attack and AutoAutack. In this case, simply adding transformer blocks to the classifier cannot guarantee better robustness. However, we recognize that for mixed architecture that has both T2T and transformer blocks. it is useful to improve adversarial robustness by increasing the proportion of the transformer blocks in the model. As shown in Tables D and 2 T2T- has lower ASR than VTT- under both attacks. Besides the transformer block, we find that other attention mechanism modules such as SE block also improves adversarial robustness as SEResNe150 has the least proportion of attention, the ASR of SEResNet50 is higher than ViT and T2T-ViT models but lower than other pure CNNs. These two findings are coherent since the attention mechanism is fundamental in transformer blocks.
Pre-training without Adversarial Training Does Not Improve Robustness Pre-training is criti- cal for ViT's to achieve competitive standard accuracy with CNNs trained from scratch (Dosovitskiy iet al. (2020). However, pre-training may not be the main reason of better robustness. To illus- trate this point, we include CNNs pre-trained
large datasets and fine-tuned on ImageNet-Ik to check the effect of pre- te-irrainng on adversarial robustness, CNNs pre-trained on large datasets IG- 1B-Targeted (Mahajan et al.] (2018) and YFCC100M (Thomee et al. (2015)) that are even larger than ImageNet-2II4 used by VIT, ResNet50-swsl and ResNeXt-32240
still have similar or even higher ASR than ResNet18 and ResNet50-32x44 that are not pre-trained. This supports our ob- servation that pre-training in its current form may not be able to improve adversarial robustness. Hendryeks et al. (2019) also reported that pre-training without adversarial training techniques can- not improve adversarial robustness. The resilience of ViT against perturbations corresponds more to the transformer structure rather than pre-training.
6.1 FREQUENCY STUDY AND FEATURE VISUALIZATION
Table 4: Frequency study and the ASR (%) of the target models against PGD attack. In the "Low- pass" column, only low-frequent adversarial perturbations are preserved and added to the input images. the "High-pass" column, only high-freap fighffreueet perturbations can pass through the filter. The "Full-pass" mode is the same as the traditional PGD attack. We set the attack step fixed to 40 and vary the attack radius to different values as shown in the second row.
Low-pass
High-pass
Full-pass
Model
0.001
0.003
0.005
0.01
0.1
0.001
0.083
0.005
0.01
0.1
0.001
0.003
0.005
0.01
0.1
VIT:5766
16.0
31.9
35.3
4032
41.8
19.2
39.3
49.4
59.6
76.6
44.5
73.4
89.3
99.0
100.10
BIT
28.1
35.7
39.7
44.2
50.4
33.7
46.9
56.0
66.6
78.1
51.1
85.4
94.0
99.1
100.0
VIT-L/16
25.1
35.9
41.7
49.8
58.0
27.1
37.7
43.4
52.5
71.1
44.9
76.6
90.1
98.2
100.0
ViT-B/16-Res
18.6
24.9
37.1
84.0
92.7
96.7
97.7
T2T-VIT-14
22.0
22.8
24.0
24.2
25.7
50.4
79.5
90.9
96.9
98.6
62.9
93.0
98.2
100.0
100.0
T2T-VIT-24
20.8
25.6
41.7
82.3
52.3
87.7
96.6
100.0
ResNet50-swsl
21.8
25.1
26.3
28.4
27.5
54.7
87.6
95.0
97.8
96.5
75.3
97.1
98.6
99.6
100.0
RosNec50-32k4d
25.0
33.7
37.3
41.0
38.5
52.3
82.9
92.6
96.7
96.5
71.8
96.8
98.8
99.6
99.9
Here conduct a frequency study and feature visualization to support claim that ViTs more robust because they learn to focus less on high-frequency features than other models. Such property can be diminished when introducing other modules to the ViT's structure, causing the inferior adversarial robustness of hybrid ViTs.
We design frequency study to verify our hypothesis that ViTs are adversarially more robust compared with CNNs because ViTs learn less high-frequency features. As defined in equation 3 for adversarial perturbations generated by PGD attack, we first project them to the frequency domain by DCT. We design three frequency filters as shown in Figure 5 in Appendix B the full-pass filter, the low-pass filter, and the high-pass filter. We take 32 x 32 pixels in the low-frequency area out of 224 224 pixels as the low-pass filter, and 192 192 pixels in the high-frrequency area as the high-pass filter. Each filter allows only the corresponding frequencies to pass through- when the adversarial perturbations go through the low-pass filter, the high-frequency
Frequency Study
8

---

Under review as a conference paper at ICLR 2022
SiokeNNe51
Vil-acto-Re Res
Figure 4: Feature visualization: The learned low-level structure features are highlighted in blue (obviously perceptible) and green (minorly perceptible). The CNNs in the first row learn more low-level features compared with the ViTs in the second row. The ViTs pay more attention to the low-level structures and their feature maps become noisier when ResNet features introduced (ViT-B/16-Res) or neighboring tokens are aggregated into one token recursively (T2T-ViT-24).
components are filtered out and vice versa, and the full-pass filter makes no change. We then apply these filters to the frequencies of the perturbations, and project them back to the spacial domain with the IDCT. We test the ASR under different frequency areas, and show the results in Table A
The ASR of VIT is relatively low in the "High-pass" column when only the high-frequencies of the perturbations are preserved. contrast, CNNs show significantly higher ASR in the "High-pass" column than in the "Low-pass" column. It reflects that CNNs tend to be more sensitive to high- frequency adversarial perturbations compared to ViTs. We also observe that adding modules that learn low-level structures makes the models more sensitive to high-frequency perturbations. T2T- ViT-14, T2T-VTT-24 and ViT-B/16-Red have higher ASR in the "High-pass" column and lower ASR in the "Low-pass" column compared with vanilla ViTs, which verifies our hypothesis that low-level features are less adversarially robust, Besides, when adding more transformer blocks to the TiT-ViT model, the model becomes less sensitive the high frequencies of the adversarial perturbations, e.g., the T2T-ViT-24 VIT-24 has an 8.7% lower ASR than that of the T2T-ViT-14 14 in the "High-pass column.
Feature Visualization We follow the work of Yuan et al. (222)) to visualize the Iearned features from the first blocks of the target models in Figure We resize the input images resolution of 224 x 224 for CNNs and a resolution maps from the first block are in the same shape of 112 112. Low-level features like lines and edges are highlighted in blue (obviously perceptible) and green (minorly perceptible). As shown in Figure 4 CNNs like ResNee5O-swsll and ResNet50-32x4d learn features with obvious edges and lines. Minorly perceptible low-level features are learned by T2T-ViT-24 and VřT-B/16-Res. While is hard to observe such information the features maps learned by ViT-B/16.
1792 x 1 1792 for ViTs and T2T-ViTs such that the feature
The feature visualization combined with the frequency study shows that the model's vulnerability under adversarial perturbations is highly relative the model's tendency to learn low-level high- frequency features. Techniques that help the model learn such features may improve the perfor- mance on clean data but the risk of sacrificing adversarial robustness.
CONCLUSION
This paper presents the first study on the robustness of ViTs against adversarial perturbations. Our results indicate that VIT's are more robust than CNNs on the considered adversarial attack and certi- fied robustness settings. Moreover, we show that the features learned by ViT's contain less low-level information, contributing improved robustness against adversarial perturbations that often con- tain high-freeuency components; introducing convolutional blocks in ViTs can facilitate learning low-level features but has negative effect on adversarial robustness and makes the models more sensitive to high-frequency perturbations. We also demonstrate adversarial training for VIT. Our work provides deep understanding of the intrinsic robustness of ViTs and can be used to inform the design of robust vision models based on the transformer structure.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Rima Alaifari, Giovanni S Alberti, and Tandri Gauksson. Adef: an iterative algorithm to construct adversarial deformations. arXiv preprint arXiv:1804.07729. 2018.
Faisal Alamri, Sinan Kalkan, and Nicolas Pugeault. Transformer-encoder detector module: Us- ing context to improve robustness to adversarial attacks object detection. arXiv preprint arXiv:2011.0697., 2020.
Ahmed Aldahdooh, Wassim Hamidouche, and Olivier Deforges. Reveal of vision transformers robustness against adversarial attacks. arXiv preprint arXiv:2106.03734. 2021.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of secu- rity: Circumventing defenses to adversarial examples. International Coference on International Conference on Machine Learning, 2018.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference Computer Vision, pp. 213-229. Springer, 2020.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and llya Sutskever. Generative pretraining from pixels. In International Conference on Machine Learning, pp. 1691- 1703. PMLR, 2020.
Xiangning Chen, Cho-Jui Hsieh, and Boqing Gong. When vision transformers outperform resnets without pretraining strong data augmentations. arXiv preprint arXiv:2106.01548, 2021.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In International Conference on Machine Learning, pp. 2206- 2216. PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 / 1929, 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur, Sharpness-aware minimiza- tion for efficiently improving generalization. arXiv preprint arX1v:2010.01412, 2020.
Siddhant Garg and Goutham Ramakrishnan. Bae: Bert-based adversarial examples for text classifi- cation. arXiv preprint 2xXx::2004.0197700 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre- mre-trainng can improve model robustness and uncertainty. In International Conference on Machine Learning, pp. 2712-2721. PMLR, 2019.
10

---

Under review as a conference paper at ICLR 2022
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 2017.
Yu-Lun Hsich, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, and Cho-Jui Hsieh. On the robustness of self-attentive models. In ACL, pp. 1520-1529, 2019.
Jie Hu, Li Shen, and Gang Sun. Squeere-and-excitttion networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132-7141, 2018.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8025, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Dmitry Krotov and John Hopfield. Dense associative memory is robust to adversarial inputs. Neural computation, 30(12):3151-316 2018.
Dmitry Krotov and John J Hopfield. Dense associative memory for pattern recognition. Advances in Neural Information Processing Systems, 2016.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL httptts:///ppeeeeiiiee net net/forum? Beet//orummiddBBmmA..... 3ddd3mddd433mm33
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial
attack against bert using bert. arXiv preprint wxXi:2204.00998. 2020.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 19X.:11977.11922 2019.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103. 14030, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1rXivv/706.066 2017.
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 181- 196, 2018.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. arXiv preprint arXiv:2 104.06600, 2021.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. arXiv preprint arXiv:2105.10497 2021a.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Fahad Shahbaz Khan, and Fatih Porikli. On improving adversarial transferability of vision transformers. arXiv preprint arXiv:2106.041000 0469, 2021b.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. arXiv preprint arXiv:2010.00467, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. arXiv preprint arXiv: 1912.0173,, 2019.
11

---

Under review as a conference paper at ICLR 2022
Sayak Paul and Pun-Yu Chen. arXiv:2105.07581, 2021.
Vision transformers are robust learners.
arXiv preprint
Yao Qin, Chiyuan Zhang, Ting Chen, Balaji Lakshminarayanan, Alex Beutel, and Xuezhi Wang. Understanding and improving robustness of vision transformers through patch-based negative augmentation. arXiv preprint arXiv:2110.07858, 2021.
Hubert Ramsauer, Bernhard Schäft, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gru- ber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, Victor Greiff, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.022 02217, 2020.
Jonas Rauber, Roland Zimmermann, Matthias Bethge, and Wieland Brendel. Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensor- flow, and jax. Journal of Open Source Software, 5(53):2607, 2020. doi: 10.21 00.21105/joss.02007
URL httphttt////o..rgge/e 10. 21102/ josss////sss.. 02607
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. arXiv preprint arXiv:2003.01900. 2020.
Hadi Salman, Saachi Jain, Eric Wong, and Aleksander Madry. Certified patch robustness via smoothed vision transformers. arXiv preprint arXiv:2110.07719 2021.
Zhouxing Shi and Minlie Huang. Robustness to modification with shared words in paraphrase identification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Prucessing: Findings, pp. 164-171, 2020.
Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, and Cho-Jui Hsieh. Robustness verifi- cation for transformers. arXiv preprint arXiv:2002.06622 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 14XX:::14099156, 2014.
Yu Sun, Shuohuan Wang. Yukun Li, Shikun Feng. Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. Emiie: Enhanced representation through knowledge integration. arXiv preprint arXiv: 1904. 09223,, 2019.
Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xian- glong Liu, Dawn Song, Alan Yuille, et al. Robustart: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211, 2021.
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. The new data and new challenges in multimedia research. arXiv preprint 1rXiv:1503.0000
1(8), 2015.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint 17X7::776.03762,, 2017.
Boxin Wang, Shuohang Wang, Y. Cheng, Zhe Gan, Jia, Bo Li, and Jing jing Liu. Infobert: Improving robustness of language models from an information theoretic perspective. ArXiv, abs/2010.02329, 2020a.
Hans Shih-Han Wang, Cory Cornelius, Brandon Edwards, and Jason Martin. Toward few-step adversarial training from frequency perspective. In Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence, SPAI '20, pp. 11-19, New York, NY, USA, 2020b. Association for Computing Machinery. ISBN 97814503761 12. doi: 10.114573850033410922
URL http:: //o:: http://o.o.....ooo//o// 1ttp://...111/333800 0008885000334119222
Ross Wightman. Pytorch image models.
https:// httt://///thbb.../ com/ /tt.i/ii..ommmiiiamamaaa///
pytorch- mage-mods 2019.
12

---

Under review as a conference paper at ICLR 2022
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03%0 2020.
Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33, 2020.
I Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-seas scale semi- supervised learning for image classification. arXiv preprint arXiv:/905.00546,, 2019.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V
Le. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint 190600/900. 2019.
Mao Ye, Chengyue Gong, and Qiang Liu. Safer: A structure-free approach for certified robustness to adversarial word substitutions. arXiv preprint 14Xv: 2020.
Fan Yin, Quanyu Long, Tao Meng, and Kai-Wei Chang- On the robustness of language encoders against grammatical errors. arXiv preprint arXiv:2005.05663 2020.
Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. tokens-to-token vit: Training vision transformers from scratch on imagenet. arXiv preprint arXiv:2101.11986, 2021.
Sergey Zagoruyko and Nikos Komodakis. 16X1v:1605.07146, 2016.
Wide residual networks.
arXiv preprint
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference Machine Learning, pp. 7472-7482. PMLR, 2019.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26 17:33122-1155, 2017.
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6848-6856, 2018
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arX1v:2010.04159, 2020.
13

---

Under review as a conference paper at ICLR 2022
SUPPLEMENTAL MATERIAL
In this supplemental material, provide more analysis and results in experiments.
A TARGET MODELS
Summary of the target models investigated in the main are shown in Table The weights of these models are all publicly available at Paszke et al. (2019); Wightman (2019) such that our experiments can be easily reproduced.
Table 5: Comparison of the target models investigated in the main text.
VIT backbone
Pretraining
Model
Layers
Hidden size
Attention
Params
Pretraining dataset
Scale
VIT-S/16
786
Self-atiention
49M
TmageNet-21I
14M
VIT-B/16
12
786
Self-attention
87M
ImageNet-21K
14M
VIT-L/16
24
1024
Self-attention
304M
ImageNet-21K
14M
ViT-B/16-Res
786
Self-attention
87M
ImageNet-21K
14M
T2T-ViT- 14
384
Self-attention
22M
T2T-ViT-24
512
Self-aticntion
64M
DeiT-S/16
12
384
Self-attention
22M
Dist-DeiT-B/16
12
768
Self-attention
87M
Swin-S/4
(2,2,18,2)
96
Self-attention
SUM
SEResNer50
Squiazz-and-Excitation
Squeeze-and-Be Excitation
28M
ResNeXt-32x44a
25M
YFCC100M IG-IB-TBRE 1B-Targeted
100M
ResNet5U-swsll
26M
940M
ResNet18
12M
ResNet50-32xdd
25M
ShuffleNer
2M
MobileNet
4M
VGGI6
138M
B FREQUENCY FILTERS
We show the design of frequency-filters in Figure
filter out
pass through
Full Pass
Low Pass
High Pass
192
Figure 5: Filters for the frequency-based attack. The frequencies corresponding to the red part filtered out, and the frequencies corresponding the green part pass through. "Full Pass" means all of the frequencies are preserved. "Low Pass" means only low-frequent components are preserved. "High Pass" preserves the high-frequent part.
C TRANSFER ATTACK RESULTS
Transfer attack results using more attack radii are provided in Figure
14

---

Under review as a conference paper at ICLR 2022
atteck radius = 0.1
attack redius 0.08
attack radius 0.05
attack radius 0.02
attack radius 0.01
attack radius 0.005
Figure 6: ASR transfer attack using FGSM with different attack radii. The rows stand for surrogate models used to generated adversarial examples in the white-box attack approach. The columns stand for target models. Darker rows correlate to source models that generate more transferable adversarial examples. While darker columns mean that the target models more vulnerable to the transfer attack. "Res50-ssI" and "Res50-swsl" are short of "ResNectt-324ddsss and "Res-ss50-ssssl" respectively.
D THE SOURCE OF ADVERSARIAL ROBUSTNESS
In this section we examine the source of the adversarial robustness revealed in our experiments.
The improved robustness of VIT is caused by insufficient attack optimization. We first demonstrate that the better robustness of ViTs in white-box attacks is not caused by the difficult optimization in ViT by plotting the loss landscape with sufficient attack steps.
ResNet18, = 0.03
ResNet18, = 0.01
120
Cross-Entropy Loss
VIT-S/16, E 0.03
VIT-S/16, E = 0.01
60-
20
100
200
300
400
500
Number of Iterations
Figure 7: Cross entropy loss versus varying PGD attack steps for VIT-S/16 and RestNet18, The dashed lines corresponds to larger attach radius of 0.03 and the full lines to smaller attack radius of 0.01.
15

---

Under review as a conference paper at ICLR 2022
Figure shows the cross entropy loss versus varing PGD attack steps for ViT-S/16 and R As shown in the figure, ViT's loss curves converge at a much lower value than RestNet18, suggesting that the improved robustness of VIT is not caused by insufficient attack optimization.
VIT-S146
VIT-B-16
VIT-1/16
8:33
ELOP4
0.314
01753
8.154
ILSST
8.233
BLTT4
8,997
8:44
6.225
1L277
0,200
0.22
8.080
0L542
8.57
I7744
0355)
1625
4.55
01943
8.23
00119
6.31
1UR
10099
8.67
0.287
0.00
0.946
01003
6477
0A43
8.63
0.50
0.822
8.03
8.65
CS1
100
121-N11-14
VIE-921--44
R.229
6206
E754
0277
14
1L75
0.75
8.75
5J533
10
0,705
0L525
Saga
6.224
A7715
8435
8.05
O
GALE
0O03
BA12
0.01
8.82
8.500
8822
0003
SERIANA150
BATS
AEES
TEACE
8.347
10750
6.72
TEAST
0.555
ILOCE
EL16A
5.077
8.42
0.044
0.556
0.31
0.903
8.456
0.02
12912
5.51
6.34
0.044
0.61
2.80
8315
6505
IUNO
8862
0,00
0.03
0255
0364
Figure 8: Adversarial accuracy of the target models against PGD attack with different attack radii ("eps") and attack steps ("steps"). When the attack radius and attack steps are increased, the adver- sarial accuracy of the target model decreases to zero. Darker blocks stand for more robust models against PGD attack.
Figure 8 shows the robust accuracy of more target models against PGD attack with different attack radii ("eps") and attack steps ("steps"). Vision transformers have darker blocks than CNNs', which stands for their superior adversarial robustness against PGD attack.
A Hopfield Network Perspective The equivalence between the attention mechanism in transform- ers to the modern Hopfield network (Krotov & Hopfield (2016)) was recently shown in Ramsauer eet al. (2020). Furthermore, simple Hopfield network (one layer of attention- like network) and dataset (MNIST), improved adversarial robustness was shown in Krotov & Hopfield (2018). There- fore, the connection of attention in transformers to the Hopfield network can be used to cxplain the improved adversarial robustness for ViTs.
E EXPERIMENTS ON CIFAR-10
We choose the ImageNet the benchmark because ViTs can hardly converge when training directly on small datasets like Cifar. Therefore, we finetune the ViTs instead. As shown in Table ViT-B/4 performs higher robust accuracy than WideResNet, which is consistent with the trend ImageNet.
Table 6: Robust accuracy of ViT-B/4 and WideResNet against PGD-10 attack with different attack radii.
Model
0.001
0.003
0.01
0.03
ViT-B/4
0.9202
0.6242
0.0994
0.0103
WideResNet
0.7744
0.5923
0.0854
0.0000
F EXPERIMENTS ON SOTA VIT STRUCTURES
In this section, we supplement the experimental results of recently proposed SOTA ViTs.
16

---

Under review as a conference paper at ICLR 2022
computes the representations with shifted windows scheme which brings greater efficiency by limiting self-attention computation to non-overlapping local win- dows while also allowing for cross-wings cross-window connection.
Swin-Trasnformat (Liu et al. 2021
DeÎT (Touvron et al. 2021) further improves the ViTs' performance using data augmentation or distillation from CNN teachers with an additional distillation token.
SiM-VIT (Chen et al. 2021 uses sharpness-awaar aware minimization (Foret et al. 2020 to train ViTs from scratch on ImageNet without large-scale pretraining strong data augmentations.
Table 7 summarizes the information of models investigated our experiments. The window size of the swin transformers in Table 7 is 7. The pre-trained weights of these models available in t timm package.
Table 7: SOTA ViT models investigated in our experiments.
Model
Layers
Hidden size
Heads
Params
D
Touvron et al.
2021
12
192
6M
Deit-S/16
Touvron et al.
2021
12
384
22M
Deit-B/16 Touvron el al.
2021
12
768
12
87M
Dist-Deit-1716
Touvron et al.
2021
12
192
6M
Dist-Deit-S/16
Touvron el al.
2021
384
22M
Dist-Deit-B/16
louvron et al. 202
12
768
12
87M
ViT-SAM-B/16
Chen et al.
2021
12
768
12
87M
ViT-SAM-B/32 CChnn et al..
2021
12
768
12
88M
Swin-T74 (Liu et al.
2021
(2,2,6,2)
96
(3,6,12,24)
28M
Swin-S/4
Liu ct al.
2021
(2,2,18,2)
96
(3,6,12,24)
50M
Swin-B/4
Liu al.
2021
(2,2,18,2)
128
(4,8,16.32)
88M
Swin-I/4
Liu et al.
202T
(2,2,18,2)
192
(6,12,24,48)
197M
Table 8: Robust accuracy (%) of ViTs described in agbinss 40-step PGD attack with different attack radii, and also the clean accuracy ("Clean"), A model is considered to be robust if the robust accuracy is higher.
Model
Clean
0.001
0.003
0.005
0.01
Deit-T716
72.3
36.8
8.3
0.3
Deit-S/16
77.7
48.9
17.6
7.1
1.1
Deit-B/16
81.3
46.6
14.3
6.0
0.9
Mat-Deit-T/1e
74.4
40.6
5.7
0.7
0.2
79.3
52.4
15.1
4.3
0.3
Dist-Deit-B/16
81.8
55.6
17.7
4.5
0.4
ViT-SAM-B/16
76.7
63.4
37.0
20.1
3.8
ViT-SAM-B/32
63.8
53.2
32.3
19.7
3.1
Swin-T/4
78.8
33.5
6.0
1.2
0.1
Swin-S/4
81.8
40.0
12.4
3.2
0.2
Swin-B/4
82.3
38.8
11.1
4.1
0.3
Swin-L/4
84.2
38.7
11.1
2.9
0.4
Table shows the clean and robust accuracy of ViTs Table against 40-step PGD attack with different radii. And results for AutoAttack shown in Table g twin-transformers introduce shifted windows scheme that limit self self-attention computation to non-overlapping local windows, which harms the robustness Tokens-to- Token scheme according to the above results.
17

---

Under review as a conference paper at ICLR 2022
Table Robust accuracy (%) of ViTs described in Table 7 againss AutoAttack with different attack radii, and also the clean accuracy ("Clean"). A model is considered to be more robust if the robust accuracy higher.
Model
Clean
0.001
0.003
0.005
0.01
Deit-T/16
72.3
23,4
0.5
0.0
0.0
Deit-S/16
77.7
30.2
1.2
0.0
0.0
Deit-B/16
81.3
20.4
0.3
0.1
0.0
Dist-Deit-T/B
74.4
31.1
0.8
0.1
0.0
Dist-Deit-S/Ie
79.3
43.1
3.7
0.2
0.0
Dist-Deit-B/16
81.8
42.7
3.4
0.2
0.0
ViT-SAM-B/16
76.7
59.8
26.0
8.4
0.1
ViT-SAM-B/32
63.8
48.9
23.6
9.7
0.8
Swin-T/4
78.8
6.8
0.1
0.0
0.0
Swin-S/4
81.8
7.9
0.1
0.0
0.0
Swin-B/4
82.3
2.4
0.1
0.0
Swin-L/4
84.2
4.3
0.1
0.0
0.0
G DISCUSSION OF CONCURRENT AND RELATED WORKS
In the context of computer vision, one earliest relevant work Alamri al. (2020) which applies transformer encoder in the object detection task and reports better adversarial robustness. But the model they considered is a mix of CNN and transformer instead of the VIT model considered in this paper. Besides, the attacks they applied were relatively weak, and there lacks study and explanation on the benefit of adversarial robustness brought by the transformers.
Recently, there are many concurrent or follow-up works investigating the adversarial robustness of vision transformers. Many of them have cited our work and shown valuable discussion extended experiments. We acknowledge their contributions below. Mahmood et al. (202T) test the adversarial robustness the transformer in both white-box and black-box settings, analyze the security
simple ensemble defense of CNNs and transformers, and find such ensemble defense is effective for the black-box setting. Qin et al. (202T) and Salman et al.] (202T) investigate the adversarial robustness of VITs through the lens of their special patch-based architectural structure. [Naserr al.
(2021b) investigate whether the weak transferability of adversarial patterns from nigh-peeforming
weak attack. [Aldahdooh 202T): Naseer et al.] (2021a): Paul & Chen (202T;; Tang et al. (202T) study the adversarial robustness from different views, e.g., preprocessing defense methods (Aldahdooh al. [202I), shape
VIT models, reported in work, is result of weak features
et al.
recognition capability (Naseer et al. 202 and natural adversarial examples (Paul & Chen 2021
Tang et al. 2021).
Our work differs from these concurrent and related works by focusing more on the origin of the adversarial robustness of vision transformers. We discuss the superior adversarial robustness of VITs through the lens of frequency, and find that the ViT's are especially robust to high-frequency adversarial perturbations. We also apply denoised randomized smoothing and show VIT also has superior certified robustness than CNN models. Our study provides insight on understanding the source of ViT's adversarial robustness and designing more robust architectures.
H
ROBUSTNESS AGAINST ADVERSARIAL DEFORMATION
Besides additively perturbing the correctly classified image, ADef (Alaifari et al. 2018 iteratively applies small deformations to the clean data. We show the robust accuracy against such perturbations Table I0, which is in accordance to the results of PGD and AutoAttack.
18

---

Under review as a conference paper at ICLR 2022
Table 10: Robust accuracy (%) against AFef under the default setting described in Alaifari et al.
2018).
Model
ViT-S/16
VGG16
DenseNet
MobileNet
ResNet18
Robust Accuracy
12.4
10.8
11.1
11.7
11.8
19


Reviews:
[{"review_id": "uI0DvmaICvC", "reviewer": "Reviewer_WHoT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe study on the adversarial robustness of ViT is comprehensive.\nThe analyst behind the observation is provided.\nThe paper is clearly written and easy to follow.\n\nWeakness:\n1. The authors state that ViT is more robust than CNN. It is only true when a weak attack is applied. When a standard attack (with perturbation range epsilon=0.03/0.01), both ViT and CNN can be fooled with 100% fooling rate. The observation indicates that both are equally vulnerable under standard attack. Hence, the statements or claims made in this paper should be carefully formulated.\n\n2. The observations on the difference in model robustness are presented in this work. However, the difference is always attributed to the model architecture difference in this work. Compared to ResNet, the ViT and DeiT are trained/finetuned in a different setting, e.g. different data augmentation, different training schedules. It is not clear how the different training/finetuning settings contribute to the different model robustness. It might be too early to attribute the observation to model architectures.\n\n3. Similarly, the author lists the following point as one of their findings:\nPre-training on larger datasets does not improve adversarial robustness though it is critical for training ViT.\nHowever, previous work shows pre-training can improve model robustness [1]. More discussion should be provided hier. The claims should be careful. \n4. In Figure 3, the author shows ViT-small shows higher certified robustness than ResNet18. In the work DeiT, they show that the CNN counterpart of DesT-small is ResNet50 instead of ResNet18. If it is too hard to compare them in a 100% fair fashion. Please show the curves of both ResNet50 and ResNet18.\n\n5. The adversarial training on ViT is also studied in this work. However, the experiments are conducted on CIFAR10 datasets. The ViT is expected to behave well only on a large dataset. What is the performance of ViT in a standard-setting on CIFAR10 dataset?\n\n[1] Hendrycks, Dan, Kimin Lee, and Mantas Mazeika. \"Using pre-training can improve model robustness and uncertainty.\" International Conference on Machine Learning. PMLR, 2019.", "summary_of_the_paper": "This work studies the adversarial robustness of vision transformers comprehensively. The author first presents their finding via empirical experiments. Concretely, they show the robustness of vision transformers from the perspective of adversarial attack, transferability, and certified robustness, adversarial training defense. Then, they analyze the reason behind the observation. ", "summary_of_the_review": "ViT demonstrates the potential to work as an alternative to CNNs. The adversarial robustness of ViT is indeed an important topic. As claimed by the author, this work makes the first comprehensive study on this topic. However, some concerns listed in weaknesses above remain to be addressed. Therefore, I rate this paper blew the acceptance threshold. I happy to raise my rating if the concerns are well addressed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VnE9RbpAr5Q", "reviewer": "Reviewer_7ATA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n(Negative) Throughout the paper, the authors claimed that this is the first paper to introduce an adversarial attack on ViT to examine the robustness of ViTs. But actually, until today, there have been a number of papers working on this topic, e.g., [r1-r4]. Especially, [r4] has been published in ICCV 2021. I understand that the authors might argue that this paper was submitted to Arxiv on 29 Mar 2021, which is earlier than some of the other papers. But, the authors should understand the following facts. First, please note that, [r4] was submitted to Arxiv on 26 Mar 2021, which is even earlier than the authors' paper. Moreover, [r4] has been published by ICCV 2021, and the authors' paper is unpublished yet. Second, because this paper is still not yet published and currently there has been a number of paper workings the same ideas of this paper, it would not be appropriate for the authors to overstate this is the first work in ViT robustness, considering many of the concurrent works provide more deep insights in this topic. Third, [r5] was the earliest paper studying the robustness of non-local attentive models among all of these works. Fourth, the insight of this paper is relatively shallow. There are not many valuable discoveries in this article, so the author keeps discussing the same thing from different angles. Fifth, I understand that the author wrote this paper many months ago, and they don't want to improve their paper because they think many \"advance papers\" are later than theirs. But I encourage the authors to improve their paper with more advanced insights to catch up with the recent developments. For example, the author could propose some practical solutions to ViT robustness. I also have some experience that I was the first to propose some techniques, but my paper was rejected. Then, the later paper working on a similar idea was published. I was frustrated. But I continued to improve my paper, and finally, my paper was accepted by another venue as an oral presentation. \n\n\n[r1] Intriguing Properties of Vision Transformers\n\n[r2] Vision Transformers are Robust Learners\n\n[r3] RobustART: Benchmarking Robustness on Architecture Design and Training Techniques\n\n[r4] Understanding Robustness of Transformers for Image Classification (ICCV 2021) 26 Mar 2021\n\n[r5] Feature Denoising for Improving Adversarial Robustness.\n\n\n(Negative) It would be good not to say that this work provides the first study on the robustness of vision transformers (ViTs) against adversarial perturbations. There have been other works discussing the ViT adversarial robustness. It would be good for the author to recognize the contribution of the community. It would be good for the authors to be humble.\n\n(Negative) The following observations are natural and not new: \"we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs).\"\n\n(Negative) The following observations are natural and not new: \"Features learned by ViTs contain less low-level information and are more generalizable, which contributes to superior robustness against adversarial perturbations.\" Existing papers found that ViTs do better than CNNs in adversarial robustness but do worse than CNNs in natural noise robustness.\n\n(Negative) The following observations are complementary to the above observations. They thus are less informative: \"Introducing convolutional or tokens-to-token blocks for learning low-level features in ViTs can improve classification accuracy but at the cost of adversarial robustness.\"\n\n(Negative) The following observations are complementary to the above observations. So these observations are less informative. Increasing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. This makes readers feel that there are not many valuable discoveries in this article, so the author keeps discussing things from different angles.\n\n(Negative) I'm afraid I have to disagree with the authors' following claim: \"Pre-training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs.\" I think the authors performed the wrong pre-training. Pre-training on the larger dataset should be robust pre-training, i.e., it should be adversarial training, but NOT vanilla pre-training. In my experiments, doing robust pre-training on a larger dataset significantly improves adversarial robustness.\n\n\n(Negative) The following observations are too natural and contain little information: \"ViTs are less sensitive to high-frequency perturbations than CNNs and\nthere is a high correlation between how well the model learns low-level features and its robustness against different frequency-based perturbations.\"\n\n(Negative) The following descriptions need to be corrected: \"it remains unclear on the robustness of ViT against adversarial perturbations, which is critical for safe and reliable deployment of many real-world applications,\" given that the literature has fully studied the robustness of ViT.\n\n\n(Negative) The following descriptions need to be corrected: \"In this work, we conduct the first study on examining the adversarial robustness of ViTs on image classification tasks and make comparisons with CNN baselines.\"\n\n\n(Neutral) The following observation is natural and ok: \" Using denoised randomized smoothing (Salman et al., 2020), ViTs attain significantly better-certified robustness than CNNs.\"\n\n(Positive) The following description is correct and insightful: \"short-term memory (LSTM) or CNN, with a theoretical explanation provided in Hsieh et al. (2019). However, due to the discrete nature of NLP models, these studies are focusing on discrete perturbations (e.g., word or character substitutions) which are very different from small and continuous perturbations in computer vision tasks.\"\n\n(Negative) The following description needs to be modified: \" To the best of our knowledge, this work is the first study that investigates the adversarial robustness (against small perturbations in the input pixel space) of transformers on computer vision tasks.\"\n\n(Negative) The following description needs to be modified: \"In the context of computer vision, the most relevant work is Alamri et al. (2020), which applies transformer encoder in the object detection task and reports better adversarial robustness.\"\n\n(Negative) In the related work section, all the above-mentioned works should be discussed and compared to acknowledge the community's contribution.\n\n(Positive) I am happy to see that the authors also evaluate the certified robustness of the models using randomized smoothing, where the robustness is evaluated as the certified radius,\n\n(Negative) The following observations are too natural: \" Adversarial training can be applied to train robust ViTs.\"\n\n\n(Positive) The following finding is interesting and valuable: \"We conjecture that ViT may need larger training data or longer training epochs to improve further its robust training performance, inspired by the fact that on natural training ViT is not able to perform well either without large-scale pre-training.\"\n\n\n(Positive) The results in Table 4 are fascinating, valuable, and beneficial. I really like the results in this table.\n\n\n(Negative) Please use text instead of figures to give a sub-caption to each subfigure in Figure 4. The current version is lossy.\n", "summary_of_the_paper": "\nThis paper studies the adversarial robustness of ViTs. There are several significant strengths and weaknesses in this paper. Especially, the novelty of this paper against one published paper is limited. It would be good to see my detailed comments below.\n\n", "summary_of_the_review": "\nBalancing the strengths and weaknesses of the proposed method, I would like to recommend a rating of weak rejection for this paper.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "569wz5mdTPK", "reviewer": "Reviewer_x14o", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Some findings lack explanations and reasons. For example, it is important to give a reason why ViTs have better adversarial robustness than convolutional neural networks. Does this advantage come from the splitted several tokens in ViTs? If yes, could you please conduct more experiments on different sizes of tokens? It would be better to provide more explanations for this.\n\n2. More general Transformer architectures should be explored in the paper. The authors mainly focus on ViTs, and the observations are mainly for ViTs. Recently, Swin Transformer gains more attention in computer vision tasks. The authors only discuss one type of Swin Transformer, i.e., Swin-S/4. It would be better to explore more types of Transformer and draw conclusions for the more general cases of Transformer.\n\n3. The experiments should be improved.  The authors only consider PGD attack and AutoAttack. These two types of attack methods are linear, i.e., add small perturbation on clean data. It would be better to conduct more experiments on other nonlinear attack methods, e.g., ADef [1], which applies small deformations to the clean data. \n\n    [1] ADef : an Iterative Algorithm to Construct Adversarial Deformations. ICLR 2019\n\n4. In Table 1, the attack success rate (ASR) of  Deit-S/16 is the best result, but the ASR of ViT-L/16 is 98.2. Is there a mistake in these two results?\n", "summary_of_the_paper": "This paper provides a comprehensive study on the robustness of ViTs against adversarial perturbations. The authors found that 1) ViTs has better adversarial robustness than convolutional neural networks; 2) Introducing convolutional or tokens-to-token blocks can improve the classification accuracy but at the cost of the adversarial robustness; 3) More proportion of transformers has better robustness; 4) Pre-training on larger datasets does not improve adversarial robustness; 5) Adversarial training is applicable to ViTs. In addition, many experiments verify the findings on white-box, transfer attack settings and adversarial training.  ", "summary_of_the_review": "It would be better to provide more explanations for some findings, and further improve the experiments for more general Transformer cases and different types of attack methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0VNoCI9NDB", "reviewer": "Reviewer_FTwY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \\\\\n(1) As claimed, this work provides the first and comprehensive study on the robustness of ViT. \\\\\n(2) This study provides some interesting (but not surprising) insight.\n(3) The paper has a clear structure and is easy to follow.\n\nWeaknesses: \\\\\n(1) The insights of this work cannot be exploited for understanding or improving the adversarial robustness. This is the main reason that I believe this work might be below the ICLR bar. \\\\\n(2) Why is ViT more robust than CNN? This work claims that ViTs have less low-level information and more generalizable. How do you define high-level and low-level? Why does high generalization contribute to superior robustness? Any proof or reference? It would be interesting if the authors can provide more concrete insight into the mechanism behind the reported phenomenon. \\\\\n\n\nSome works [1-10] might be worth a check: \\\\\nThey do not affect my rating of this submission.\n\n[1] Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation\n[2] Certified Patch Robustness via Smoothed Vision Transformers\n[3] Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs\n[4] Towards Transferable Adversarial Attacks on Vision Transformers\n[5] On Improving Adversarial Transferability of Vision Transformers\n[6] Reveal of Vision Transformers Robustness against Adversarial Attacks\n[7] Intriguing Properties of Vision Transformers\n[8] Vision Transformers are Robust Learners\n[9] On the Robustness of Vision Transformers to Adversarial Examples\n[10] Towards Robust Vision Transformer", "summary_of_the_paper": "This paper performs a study on the adversarial robustness of vision transformers. It provides some insight on vision transformer from the robustness perspective.", "summary_of_the_review": "Overall, this work has some merits but I expect more than that to get in for ICLR. Some investigation is rudimentary and is suggested to provide more deep insight. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies the adversarial robustness of vision transformers. The authors conclude that vision transformers are generally more adversarially robust than the convolutional neural networks. Several interesting empirical conclusions are made for the robustness property of vision transformers. Sufficient empirical experiments are conducted. Overall, the paper is well-written, well-organized, and interesting. However, there are some concerns about the current version. (1) There are some concurrent works having similar empirical findings, which have been formally published and would weaken the interest of readers in the paper. (2) The reviews suggest that the authors use the insights from the paper to design more robust and effective vision transformers. The four reviewers have unanimous recommendations below the acceptance threshold. We therefore cannot recommend acceptance. However, we believe that by taking the comments, the next version would be a very strong paper.
Decision: Reject