On the Adversarial Robustness of Vision Transformers
Authors: Rulin Shao
Zhouxing Shi
Jinfeng Yi
Pin-Yu Chen
Cho-Jui Hsieh
Venue: Accepted by TMLR ()

Abstract: Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides a comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with MLP-Mixer and convolutional neural networks (CNNs) including ConvNeXt, and this observation also holds for certified robustness. Through frequency analysis and feature visualization, we summarize the following main observations contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less high-frequency patterns that have spurious correlation,  which helps explain why ViTs are less sensitive to high-frequency perturbations than CNNs and MLP-Mixer, and there is a high correlation between how much the model learns high-frequency features and its robustness against different frequency-based perturbations. 2) Introducing convolutional or tokens-to-token blocks for learning high-frequency features in ViTs can improve classification accuracy but at the cost of adversarial robustness. 3) Modern CNN designs that borrow techniques from ViTs including activation function, layer norm, larger kernel size to imitate the global attention, and patchify the images as inputs, etc., could help bridge the performance gap between ViTs and CNNs not only in terms of performance, but also certified and empirical adversarial robustness. Moreover, we show adversarial training is also applicable to ViT for training robust models, and sharpness-aware minimization can also help improve robustness, while pre-training with clean images on larger datasets does not significantly improve adversarial robustness. 

Zhouxing Shi Published in Transactions on Machine Learning Research (10/2022)
On the Adversarial Robustness of Vision Transformers
Rulin Shao
rulins cx.cmu.edu
Carnegie Mellon University
zshi Cel.csuucll.ee.
University of California, Los Angeles
Jinfeng Yi JD Al Research
yijinfeng jd.com
Pin-Yu Chen IBM Research
pin-yu.chen @ ibm.com
Cho-Jui Hsieh
choksieh @ @cs. cc.ucla.ede
University of California, Los Angeles
Reviewed on ReenReview: https://ppenneriiwwweet///oo
Abstract
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides comprehen- sive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial ro- bustness when compared with MLP-Mixer and convolutional neural networks (CNNs) including ConvNeXt, and this observation also holds for certified robustness. Through frequency analysis and feature visualization, we summarize the following main observations contributing to the im- proved robustness of ViTs: 1) Features learned by ViTs contain less high-frequency patterns that have spurious correlation, which helps explain why ViTs are less sensitive to high-frequency pertur- bations than CNNs and MLP-Mixer, and there is a high correlation between how much the model learns figh-freqeency features and its robustness against different frequency-based perturbations. 2) Introducing convolutional tokens-to-token blocks for learning high-frequency features in ViTs improve classification accuracy but the cost of adversarial robustness. 3) Modern CNN designs that borrow techniques from ViTs including activation function, layer norm, larger ker- nel size to imitate the global attention, and patchify the images inputs, etc., could help bridge the performance gap between ViT' and CNNs not only in terms performance, but also cer- tified and empirical adversarial robustness. Morcover, we show adversarial training also ap- plicable to ViT for training robust models, and sharpness-aware minimization can also help im- prove robustness, while pre-training with clean images on larger datasets does not significantly improve adversarial robustness. Codes available at https: thtb. Ru inShao/ on on-the- anteeeavverriiaa robustness of visual transformer
1 Introduction
Transformers are originally applied in natural language processing (NLP) tasks as type of deep neural network (DNN) mainly based on the self-attention mechanism (Vaswaní et al.) 2017 Devlin et al. 2018 Brown et al. 22020). and transformers with large-scale pre- pre-training have achieved statc-of-the-an results on many NLP tasks (Devlin et al. 2018 Liu al.] 2019: Yang et al.] 2019 Sun et al., (219). Recently, Dosovitskiy al. (2020) applied pure trans- former directly to sequences of image patches (i.e., vision transformer, ViT) and showed that the Transformer itself can be competitive with convolutional neural networks (CNN) image classification tasks. Since then transformers

---

Published in Transactions on Machine Learning Research (10/2022)
65
VIT-S/16
1.
60
Vir-8/16
VIT-L/16
55
VIT-SAM-8/16
Dist DefT ants
VIT-8/16-Res
160
T2T-VT-144
VIT-S/16
50
T2TV/T-24
DeT--S16
DeiT-S/16
Parameters
VIT-L/16
45
Dist B/16
Robust Accuracy (%)
Swin-5/4
40
SEResNe150
VIT-8/16
Swin-S14
ResNext-S2xtd-ssi
35
ResNet50-swsi
MLP 8
ResNet18
30
RessMet50332xcr
ShuffleNet
Hybrid
MT-&16-Ree
MobileNet
25
20
ConvNeXt-S
MLP-Mixee P-Mixer-B/16
SEReSNeISO
T2T-VIT-24
15
CNN
12T-VI--1
10
5
0
70
72
74
78 Clean Accuracy (%)
76
80
82
84
Figure 1: Robust accuracy V.S. clean accuracy. The robust accuracy is evaluated by AutoAttack (Croce & Hein 2020a). The "Hybrid" class includes CNN-Vi ViT, T2T-ViT and Swin-T introduced Section Models with attention mechanisms have their names printed at the center of the circles. ViTs have the best robustness against adversarial perturbations. Introducing other modules ViT can improve clean accuracy but hurt adversarial robustness. CNNs are more vulnerable adversarial attacks.
have been extended to various vision tasks and show competitive or even better performance compared to CNNs and recurrent neural networks (RNNs) (Carion al. 2020 Chen al. 2020 Zhu al. 2020). While ViT and its variants hold promise toward unified machine learning paradigm and architecture applicable different data modalities, it is critical to study the robustness of ViT against adversarial perturbations for safe and reliable deployment of many real-world applications.
In this work, we examine the adversarial robustness ViTs on image classification tasks and make comparisons with CNN and MLP baselines. As highlighted in Figure D our experimental results illustrate the superior robustness of VYTs than CNNs and MLP-Mixer in various settings, based on which we make the following important findings:
Features learned by ViTs contain less high- fighffrequency information and benefit adversarial robustness. ViTs achieve higher robust accuracy (RA) of 59.8% compared with maximum improvement of 16.7% by CNNs in Figure They are also less sensitive high-frequency adversarial perturbations.
Using denoised randomized smoothing (Salman et al. 2020). ViTs attain significantly better certified robustness than CNNs.
takes the cost of adversarial robustness to improve the classification accuracy ViTs by introducing blocks to help learn low-level features as shown in Figure
Increasing the proportion transformer blocks in the model leads to better robustness when the model consists of both transformer and CNN blocks. For example, the robust accuracy (RA) increases from 12.9% to 20.8% when additional transformer blocks are added T2T- 12T-ViT-14, 14. However, increasing the size of pure transformer model cannot guarantee similar effect, e.g., the robustness of ViT-S/16 is better than that of ViT-B/16 in Figure]
2

---

Published in Transactions on Machine Learning Research (10/2022)
The principle of adversarial training through min-max optimization (Madry et al. 2017 Zhang al. 2019) can be applied to train robust ViTs. Pre-training on larger datasets without adversarial training does not improve adversarial robustness. But sharpness-aware optimization (Foret et al. 2020 Chen et al. 2021 benefits both the adversarial robustness and clean accuracy of ViTs.
2 Related Work
Transformer (Vaswani et al. 2017) has achieved remarkable performance on many NLP tasks, and its robustness
has been studied on those NLP tasks. Hsieh et al. (2019); Jin et al. (2020); Shi & Huang 2020)); Li et al. (2020);
Garg & Ramakrishnan 2020); ; Yin et al. (2020) conducted adversarial attacks on transformers including pre-trained
models, and in their experiments transformers usually show better robustness compared to other models based on Long short-term memory (LSTM) or CNN, with theoretical explanation provided in Hsiih et al. (2019). However, due the discrete nature of NLP models, these studies are focusing on discrete perturbations (e-g., word or character substitutions) which are very different from small and continuous perturbations in computer vision tasks. Besides, Wang et al. (2200a improved the robustness of pre- trained transformers from an information-theoretis perspective, and[Sh et al. 2020 et (202); Xu et al. (2020) studied the robustness certification of transformer-based models. Recently, there are some concurrent works studying the adversarial robustness of ViT'. In the context of computer vision, one earliest relevant work is Alammi et al. (220)) which applies transformer encoder in the object detection task and reports better adversarial robustmess. But their considered model is mix of CNN and transformer instead of the ViT model considered in this paper. Besides, the attacks they applied were relatively weak, and there lacks studies and explanations the benefit of adversarial robustness brought by the transformers.
Recently, there are many concurrent or follow-up works investigating the adversarial robustness of vision transformers. Many of them have cited a preprint version of our work and shown valuable discussion or extended experiments. We acknowledge their contributions below. Mahmood et al. (202T) test the adversarial robustness of the transformer in both white-box and black-box settings, analyze the security of simple ensemble defense CNNs and transformers, and find such ensemble defense is effective for the black-box setting. QQin al.) (202T) and Salman et al. (2021) investigate the adversarial robustness of ViT's through the lens of patch-based architectural structure. Herrmann et al.) (2022) further designs pyramid adversarial training with augmentation techniques to improve the VIT's both sanity and robust performance. Naseer et al.] (2021b) investigate whether the weak transferability of adversarial patterns from high-performing ViT models, as reported in our work, is result of weak features or a weak attack. AAldhdoooh et al. [2021): [aseer et al.] (2021a): Paul & Chen (202T): Tang et al.] (2021); Mao et al. (2022) study the adversarial robustness from different views, e.g., preprocessing defense methods (Aldahdooh et al. 202T), shape recognition capability (Naseer et al., 2021a), natural adversarial examples (Paul & Chen 2021 Tang et al. 202T), and detailed robust components in ViTs (Mao et al.] 2022).. (Jeeveswaran. et al., 2022) studies object detection and semantic segmentation tasks, and shows ViTs are more robust to distribution shifts, natural corruptions, and adversarial attacks in both tasks.
Our work differs from these concurrent and related works by focusing more on the origin of the adversarial robustness of vision transformers. We discuss the superior adversarial robustness of ViTs through the lens of frequency, and find that the ViT' are especially robust to high-frequency adversarial perturbations. We also apply denoised randomized smoothing and show ViT also has superior certified robustness than CNN models. Our study provides insight on understanding the source of ViT's adversarial robustness and designing more robust architectures. And we cover various basclines in the experiments for comprehensive comparison, including the recently proposed ConvNeXt (Liu et al. 2022), MLP-Mixer ("oolstikhin et al. 2021). and Triin-Transfrmmrrs (Liu et al. 2021). etc.
To the best of our knowledge, our work is the first study that investigates the certified adversarial robustness of trans- formers on computer vision tasks, and explains the source of ViTs' superior adversarial robustness from a frequency perspective. We also show that ViTs have superior adversarial robustness (against small perturbations in the input pixel space) in both box and blac--box settings. And we investigate various SOTA models including the re- cently proposed Cenv (Liu et al. 2022) and MI.P-Mixer (Tolstikhin et al. 2021) in the experiments.
3

---

Published in Transactions on Machine Learning Research (10/2022)
3 Model Architectures
We first review the architectures of models investigated in our experiments. A summary of the target models investi- gated in the main text is shown in T More experimental results of ViT variants can be found in Appendix B The weights of all the investigated models are all publicly available checkpoints (Paszke et al. 2019 Wightman 2019 and they are well-tuned separately by their designers. Different from Bai al. (2021) which uses the same training setting for all models, we follow Paul & Chen (02T) to take different neural network models trained and tuned to the optimum individually rather than using common but potentially suboptimal setting, since one setting could be biased to some models as well.
Table 1: Comparison of the target models investigated in the main text. More variants (Deit, ViT-SAM, Swin-ViT, etc.) can be found in Appendix B
ViT backbone
Pretraining
Model
Layers
Hidden size
Attention
Params
Pretraining dataset
Scale
ViT-S/16
786
Self-attention
49M
ImageNet-21K
14M
ViT-B/16
12
786
Self-atteniion
87M
ImageNet-21K
14M
ViT-L/I6
24
1024
Self-attention
304M
ImageNet-21K
14M
ViT-B/I6-Res
12
786
Self-attention
87M
ImageNet-2IK
14M
T2T-ViT-14
384
Self-attention
22M
T2T-ViT-24
24
512
Self-attention
64M
DeiT-S/16
12
384
Self-attention
22M
Dist-DeiT-B/16
768
Self-attention
87M
Swin-S/4
(2,2,1822)
96
Sclf-attention
50M
SEResNet50
Squeeze-and-Exciiatton
28M
ResNeXt-32x4d-sh
25M
YFCC100M IG-IB-Targeted
100M
ResNet50-swsl
26M
940M
ResNet18
12M
ResNet50-32x4d
25M
ShuffleNet
2M
MobileNet
4M
VGG16
138M
3.1
Vision Transformers
For vision transformer, consider the original VIT (Dosovitskiy al. 2020) and its variants:
Vanilla VIT with different training schemes (VIT, DeiT, ViT-SAM): The original ViT (Dosovitskiy et al. 2020 mostly follows the original design of Transformer (Vaswani et al., 2017 Devlin et al, 2018 on language tasks. For 2D image I, E ##xWCC XC (1 < S N) with resolution H x W and C channels, it is divided into a sequence H-W flattened 2D patches of size P x P, E RRP-C (1 S i i N). The patches are encoded into patch
of N = pr- embeddings with simple convolutional layer, where the kernel size and stride of the convolution is exactly P x P. DeiT (Touvron et al.] 2021) further improves the ViT's performance using data augmentation or distillation from CNN teachers with an additional distillation token. We investigate ViT-(S,B,L//16,, DeiT-S/16 and Dist-DeiT-B/16 as defined in the corresponding papers in the main text and discuss other structures in Appendix B ViT-SAM (Chen) al. 2021 uses sharpness-awar minimization (Foret al. 2020) train ViTs from scratch on ImageNet without large-scale pretraining strong data augmentations. We include ViT-SAM-BI BB16 in the main text.
(2000) also proposed hybrid architecture for ViTs by replacing raw image patches with patches extracted from CNN feature map. This is equivalent to adding learned CNN blocks to the head of ViT. We investigate ViT-B/16-Res in our experiments, where the input sequence is obtained by flattening the spatial dimensions the feature maps from ResNet50.
Hybrid of CNN and VIT (CNN-ViT): Dosovitskiy et al.
Hybrid of T2T and VIT (T2T-ViT): Yuan al. (02T)) proposed overcome the limitations of the simple tokeniza- tion in ViTs, by progressively structurizing an image to tokens with token-to-token (T2T) module, which recursively
4

---

Published in Transactions on Machine Learning Research (10/2022)
aggregates neighboring tokens into one token such that low-level structures can be better learned. T2T-VIT was shown to perform better than VIT when trained from scratch midsize dataset. We investigate T2T-V and T2T-ViT-24 in our experiments.
Hybrid of shifted windows and VIT (Swin-T); Liu et al. 2021 computes the representations with shifted windows, which brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-wints woodsw connection. We investigate Swin-S/4 in the main text and discuss other structures in Appendix B
3.2 Baselines
We study several CNN models for comparison, including R (He et al., 2016), ResNec50-32x4d (He et al. 2016), ShuffleNet (Zhang al. 2018), MobileNet (Howard et al. 2017). and VGG16 (Simonyan & Zisserman 2014). We also consider the SEResNe(50 model, which uses the Squceze-ande Excitattin (SE) block (Hu al. 2018) that ap- plies attention to channel dimensions. The recently proposed MLP-Mixx (Tolstikhin et al. 2021 and ConvNeXt (Liu et al. 2022) are also included for comparison.
The aforementioned MLP and CNNs are all trained on ImageNet from scratch. For a better comparison with pre- trained transformers, we also consider two CNN models pre-trained on larger datasets: ResNeXt-32xxd-sst (Yalniz et al. 2019) pre-trained on YFCC100M (Thomee et al. 22155, and ResNet50-swsl pre-trained on IG-1B- B-TTagette (Mahajan et al. 2018) using semi-weakly supervised methods (Yalniz al. 2019), and then fine-tuned on ImageNet.
4 Adversarial Robustness Evaluation Methods
We consider the commonly used lo -nrm bounded adversarial attacks to evaluate the robustness of target models. An loc attack is usually formulated as solving a constrained optimization problem:
max (xcare xadv
s.t. ||xadr xolloo S,
(1)
where Xo is a clean example with label y, and we aim to find an adversarial example xade within loo ball with radius centered at Xu. such that the loss of the classifier L (xadv, y) is maximized. We consider untargeted attack in this paper, so an attack is successful if the perturbation successfully changes the model's prediction. The attacks as well as randomized smoothing method used in this paper are listed below.
Projected Gradient Descent Attack (Madry et al. 2017) Projected Gradient Decent (PGD) attack (Madry et al. 2017) solves Eq. by iteratively taking gradient ascent:
X ade
Clipxs, (xcdo + a sgm (vxJ (xade, ,)))),
where xede stands for the solution after t iterations, and Clipxo.e() (-) denotes clipping the values to make each xade within [xp.s - c, Xg,; + c) according to the lo threat model. As special case, Fast Gradient Sign Method (FGSM)
[Goodfellow al. 2014) uses single iteration with 1.
AutoAttack (Croce & Hein 2020a) parameter-free ensemble of diverse attacks, including two variants of PGD attacks (APGD-CE, APGD-DLR), optimization based attack (FAB & Hein 2020b)) and a query based black-box attack (Square Attack et al. 2019)).
AutoAttack (Croce & Hein, 2020a)
Transfer Attack We consider the transfer attack which studies whether adversarial perturbation generated by at- tacking the source model can successfully fool the target model. This test not only evaluates the robustness of models under the black-box setting, but also becomes a sanity check for detecting the obfuscated gradient phenomenon (Atha-) Iye et al. 2018). Previous works have demonstrated that single-step attacks like FGSM enjoys better transferability than multi-step attacks (Kurakin al. 2017). We thus use FGSM for transfer attack in our experiments.
Frequency- Filleere Attack for Frequency Analysis tional frequency filter to force the adversarial perturbations being within specific frequency domain (a tieeuuruy
our frequency study, we conduct PGD attack with an addi-
5

---

Published in Transactions on Machine Learning Research (10/2022)
filter
pass through
Full Pass
Low Pass
High Pass
32
Y
224
192
Figure 2: Filters for the frequency-ben bassed attack. The frequencies corresponding to the red part are filtered out, and the frequencies corresponding to the green part can pass through. "Full Pass" means all of the frequencies are preserved. "Low Pass" means only frequency components are preserved. "High Pass" preserves the high feeh-freuuecy part.
domain low-frequency domain):
edv freg
(DCT(DCT(xX))))) (x xx) + M,) Xo.
(3)
where DCT and IDCT stand for discrete cosine transform and inverse discrete cosine transform respectively, x stands for the adversarial example generated by PGD, and My stands for the mask metric defined the frequency filter as illustrated in Figure 2
Denoised Randomized Smoothing(Salman et al., 2020) for Certified Robustness Analysis We also provide prov- able certified robustness analysis using denoised randomized smoothing (Salman et al., 2020). The model is certified to be robust with high probability for perturbations within the radius, so the robustness is evaluated as the certified radius. We follow Salman et al. (2020) to train a DnCNN (Zhang et al. 2017) denoiser D9 for each pre-trained model f with stability objective Staa::
Lstab ae
f(x:))
where 5 ~ N(0,0²1 follows Gaussian distribution. Then randomized smoothing is applied the denoised classifier f Do for robustness certification:
g(z) arg max P[ 6)) c] where 5 ~ N(00 eEY
(5)
The certified radius (Cohen et al. 2019) is then calculated for the smoothed classifier as:
R
2 (Φ (PA) - + (PB)),
(6)
where 4-1 is the inverse of the standard Gaussian CDF, PA P(f( + d) CA) is the confidence of the top-1 predicted class ca. and Pn maXcşeaa P(f(x++) c) is the confidence for the second top class. Accordingly, given perturbation radius, the certified accuracy under this perturbation radius can be evaluated by comparing the given radius to the certified radius R.
5 Experiments
In this section, we compare the adversarial robustness of ViTs and CNNs from different perspectives: First, we show ViT' are more robust to high-frequency perturbations than CNNs using crafted frequency-filtree attacks and feature visualization. Second, we conduct a comprehensive empirical study on the comparison between VITs and CNNs regarding the adversarial robustness against diverse white-box attacks and black-box attacks, and the transferability of the adversarial examples from ViTs CNNs and vice versa. Finally, we provide a provable certified robustness comparison using denoised randomized smoothing.

---

Published in Transactions on Machine Learning Research (10/2022)
ResNat50.wool
ResNest50-32544
SERiNNat50
VIT-1816
ViT-IWU6-Res
12EAIT-24
Figure 3: Feature visualization: The learned low-level structure features are highlighted in blue (obviously perceptible) and green (minorly perceptible). The CNNs in the first row learn more low-level features compared with the ViTs in the second row. The ViTs pay more attention the low-level structures and their feature maps become noisier when ResNet features are introduced (ViT-B/16-Red 6-Res) neighboring tokens are aggregated into one token recursively (T2T- ViT-24).
Unless otherwise specified, Clean Accuracy (CA) stands for the accuracy evaluated on the entire ImageNet-IK lk (Deng et (all))000))) test set, Robust Accuracy (RA) stands for the accuracy on the adversarial examples generated with 1,000 test samples. Higher RA stands for better adversarial robustness. Adversarial training and evaluation results on CIFAR-10 are reported Appendix and Appendix C
5.1 Frequency Study Using Filtered-PGD and Feature Visualization
Wang et al. 2020b) has shown that CNN models could take up high-frequency patterns that are almost imperceptible to humans but have distribution correlation with labels, leading to a trade-off between robustness and accuracy. How- ever, we show the original ViT models possess superior adversarial robustness against high-frequency perturbations compared with CNNs. We also show that some variants that introduce non-transformer modules (e.g., ResNet blocks and T2T blocks) ViTs to improve clean accuracy. can diminish VITs' original adversarial robustness against high- frequency perturbations, leading to inferior adversarial robustness hybrid ViTs (e.g.. ResViTs and T2T-ViTs). We craft frequency -filtered PGD attack to study models' resistance different frequency components. Besides, we feature visualization to facilitate the illustration of our hypothesis, showing that ViTs are paying less attention to the high-frequency patterns in the images.
Table 2: RA (%) of the target models against frequency- filtered PGD attack in our frequency study. In the "Low-pass" column, only low-frequency adversarial perturbations are preserved and added the input images. In the "High-pass"" column, only frghffreueency perturbations are preserved. The "Full-par mode preserves all frequency and is the same the traditional PGD attack. We set the attack step fixed to 40 and vary the attack radius to different values.
Low-pass
High-pe
Full-pass
Attack Radius
0.001
0.003
0.005
0.001
0.003
0.005
0.01
0.001
0.003
0.005
0.01
0.1
VIT-S/16
74.0
68.1
64.7
59.8
56.2
70.8
60.7
50.6
40.4
23.4
55.4
24.6
10.2
1.0
0.0
VTT-B/16
71.9
64.3
60.3
55.8
49.6
66.3
53.1
44,0
33.4
21.9
48.9
14,6
6.0
0.9
0.0
VIT-L/16
74.9
64.1
58.3
50.2
42.0
72.9
62.3
56.6
47.5
28.9
55.1
23.4
9.9
1.8
0.0
VYT-B/16-Res
83.1
81.4
80.4
79.0
75.1
62.9
29.2
16.0
7.3
3.3
45.5
8.4
2.3
0.1
0.0
121-VIT-14
78.0
77.2
76.0
75.8
49.6
20.5
9.1
1.4
37.1
7.0
0.0
0.0
T2T-VIT-24
80.2
79.2
78.4
77.7
74.4
58.3
31.1
17.7
8.2
3.1
47.7
12.3
0.2
0.0
MLP-Mixer-B/
69.4
64.7
62.4
60.0
59.8
56.6
32.5
19.5
6.7
1.3
34.5
3.8
0.0
0.0
0.0
ConvNeXi-S
80.3
79.0
77.9
76.4
74.3
53.1
24.4
14.4
6.8
6.2
15.9
0.0
0.0
0.0
0.0
ResNet50-swal
78.2
74.9
73.7
71.6
72.5
45.3
12.4
5.0
2.2
3.5
24.7
2.9
1.4
0.4
0.0
RcsNet50-32x4d
75.0
66.3
62.7
59.0
61.5
47.7
17.1
3.3
3.5
28.2
3.2
0.4
0.1
7

---

Published in Transactions on Machine Learning Research (10/2022)
5.1.1 Frequency Study
VIT's are more resistant to high-frequency perturbations and have less bias towards high-frequency features that have spurious correlation with labels. We design frequency study to verify our hypothesis that ViTs are adversarially more robust compared with CNNs and MLP-Mixer because ViTs learn less high-frequency features. As defined in equation ]] for adversarial perturbations generated by PGD attack, we first project them to the frequency domain by DCT. We design three frequency filters illustrated in Figure 2 the full-pass filter, the low-pass filter, and the high-pass pass filter. We take 32 x 32 pixels in the low-frequency area out of 224 x 224 pixels as the low-pass filter, and 192 x 192 pixels in the high-frequency area as the high-pass filter. Fach filter allows only the corresponding frequencies to pass through when the adversarial perturbations go through the low-pass filter, the high-frequency components are filtered out and vice versa, and the full-pass filter makes no change. We then apply these filters to the frequencies of the perturbations, and project them back to the spacial domain with the IDCT. We test the RA under different frequency areas, and show the results in Table 2
The RA of ViTs are much higher in the "High-pass" column when only the high-frequencies of the perturbations are preserved. In contrast, CNNs show significantly lower RA in the "High-pass" column than in the "Low-pass" column. It reflects that CNNs tend to be more sensitive to high-frequency adversarial perturbations compared to ViTs, As shown in the Table 2 CNNs have moderate robust accuracy drop w.r.t. low-frequency perturbations but severe decrease against higu-fryy perturbations when increasing the perturbation rates. For example, when increasing the attack radius from 0.001 to 0.01, the robust accuracy of ResNet50-32x4d decreases from 75.0% to 59.0% (by 16.0%) for low-pas: adversarial attack, but 47.0% to 3.3% (by 43.7%) for high-past adversarial attack, indicating the -eehffreueecyy perturbations are the main cause to the low robust accuracy for CNNs. However, when increasing the attack radius from 0.001 to 0.01, ViT-B/16 decreases from 71.9% to 55.8% (by 16.1% for low-pass adversarial attack, but only 66.3% to 33.4% (by 32.9%) for high-ffrequeeccy-eruuubattonssss showing much better resistance to high-fie freuufrey perturbations compared with CNNs.
Introducing CNN T2T blocks makes ViTs less robust to high-frequency perturbations. One interesting and perhaps surprising finding is that ViTs have worse robustness when modules that claimed to help learning local struc- tures are added ahead of the transformer blocks. For example, TIT-VIT adds several T2T modules to the head of ViT which iteratively aggregates the neighboring tokens into one token in each local perceptive field. ViT-B/16-Res takes the features generated by ResNet as inputs, which has the same effect as incorporating a trained CNN layer in front of the transformer blocks. Both modules help to learn local structures like edges and lines (Yuan et al.. 202T). We observe that ResNet and T2T modules that could help improve the CA of the hybrid VITs makes the models more sen- sitive to high-frequency perturbations. T2T-ViT-14 14, T2T-ViT-24 and ViT-B/16-Res have lower RA in the "High-pass" column and higher RA in the "Low-pass" column compared with vanilla ViTs, which correlates with the previous observations that high-frequenee features are less adversarially robust. When adding more transformer blocks to the T2T-ViT model, the model becomes less sensitive to the high frequencies of the adversarial perturbations, e.g., the T2T-ViT-24 has an 8.7% higher RA than that of the T2T-ViT-14 in the "High-pass" column.
One possible explanation is that the introduced modules improve the classification accuracy by remembering the high-frequency patterns that repeatedly appear in the training dataset. These structures such as edges and lines are high-frequency and sensitive to perturbations (Wang et al., 2020b}. Learning such features makes the model more vulnerable to adversarial attacks. Examination of this hypothesis is conducted though feature visualization.
Increasing the Proportion of Transformer Blocks Can Improve Robustness Hendrycks et al. (209 mentioned that larger model does not necessarily imply better robustness. can be confirmed by our experiments where ViT- S/16 shows better robustness than larger ViT-B/16 under both PGD attack and AutoAttack. In this case, simply adding transformer blocks to the classifier cannot guarantee better robustness. However, recognize that for mixed architecture that has both T2T and transformer blocks, it is useful to improve adversarial robustness by increasing the proportion of the transformer blocks in the model. As shown in Table 2 (we will also show similar observations in Tables
3 and 4. T2T-ViT-22 has higher RA than T2T-ViT 14 under both attacks. Besides the transformer block, we find that other attention mechanism modules such as SE block also improves adversarial robustness as SeResNet50 has the least proportion of attention, the RA of SEResNet50 lower than ViT and T2T- ViT models but higher than other pure CNNs. These two findings are coherent since the attention mechanism is fundamental in transformer blocks.
8

---

Published in Transactions on Machine Learning Research (10/2022)
5.1.2 Feature Visualization
We hypothesis that ViTs are more resistant to high-frequency perturbations because they are learning less high- frequency features that have high correlation with labels but less semantic meanings. To facilitate the illustration, we follow the work of Yuan et al. (202T) to visualize the learned features from the first blocks of the target models in Figure 3 We resize the input images to resolution of 224 x 224 for CNNs and resolution of 1792 X 1792 for ViTs and ViT- such that the feature maps from the first block are in the same shape of 112 x 112. Features with high-frequency patterns like lines and edges are highlighted in blue (obviously perceptible) and green (minorly perceptible). As shown in Figure CNNs like ResNet50-swsl and ResNet50-32x4l learn features with obvious edges and lines. While it is hard to observe such information in the feature maps learned by ViT-B/16.
The frequency study combined with the feature visualization shows that model's vulnerability against adversarial perturbations is relative to the model's tendency learn high-frequency low-level features that could have correlation with labels but little semantic information. Techniques that help the model learn such features may improve the accuracy on clean data but at the risk of sacrificing adversarial robustness. We are still facing the trade-off between CA and RAA
5.2 Empirical Study: Adversarial Robustness under Various Adversarial Attacks
In this section, test adversarial robustness against PGD,, AutoAttack and transfer attack, and provide observations we drew from this empirical study.
5.2.1 Robustness under PGD and AutoAttack
Settings We take attack radius e E {0.001, 0.003, 0.005, 0.01} for PGD and AutoAttack evaluation. For PGD attack, we fix the attack steps to Nster = 40 with other parameters following the default setting of the implementation in Foolbox (Rauber et al. 2020). No hyper-parameter tuning is needed in AutoAttack.
Results We present the results of PGD and AutoAttack in Table and Tablerrespectively The RA is approximately 0.0% on all the models when e = 0.01 is large,indicating models trained without any adversarial augmentations are vulnerable to large perturbations. However, such vulnerability doesn't 't hold equally for all models within mild perturbations: For smaller attack radii, ViT models have higher RA than CNNs under both PGD attack and AutoAttack. For example, when 0.001, the RA for V is 55.4% while the RA for CNNs is at most 30.0%. Moreover, under the same attack radius, the RA of AutoAttack for ViT-S/16 is 48.1% compared to 6.1% of ShuffleNet, which is a large gap. We visualize the clean/robust accuracy tradeoff and model size of these models in Figure | and VIT models are noticeably on the upper right over CNN models.
Introducing ResNet or T2T blocks decreases the RA under both PGD and AutoAttack. When the features learned by ResNet are introduced, the RA of VIT-B/16 decreases from 48.9% to 45.5% of ViT-B/16-Res under PGD attack, and from 39.8% 27. 77.7% under AutoAttack, with attack radius 0.001. A similar phenomenon can be observed by comparing the RA of ViT's and ViT's The RA of T2T- is 18.3% lower under PGD attack and 35.2% lower under AutoAttack compared with the RA of ViT-S/16, under attack radius 0.001.
We also verify that pre-training with clean images does not improve robustness. Dosoviiskiy et al., 2020) points out that pre-training critical for ViTs to achieve competitive standard accuracy with CNNs trained from scratch. However, we note that pre-training doesn't bring better robustness ViTs. To illustrate this point, we include CNNs pre-trained large datasets and fine-tuned on ImageNet-Ik 1k to check the effect of pre- gre-training adversarial robust- ness. CNNs pre-trained on large datasets IG-IB- Targeted (Mahajan et al.) 2018) and YFCC100M (Thomee et al. 2015) that are even larger than ImageNet-21k used by ViT, ResNet50- and ResNNeXt-32x4d-s still have similar or even lower RA than ResNer18 and Res.Net50-32x4d that are not pre-trained. This supports our observation that pre- mre-traniig in its current form may not be able to improve adversarial robustness, which is also in accordance with Hendrycks et al. (2019). Besides, the results show that SAM can further improve model's adversarial robustness. This is because the SAM objective is formulated as a min-max optimization similar to adversarial training: but instead of adding adversarial perturbations to the input space, SAM adds adversarial perturbations to the weights.
9

---

Published in Transactions on Machine Learning Research (10/2022)
Table 3:
Clean Accuracy (%) and Robust Accuracy (%) of target models against 40-step PGD attack with different
radii, More results of the SOTA VITs can be found in Appendix B
CA
RA against PGD
Attack Radius
0.001
0.003
0.005
0.01
ViT-S/16
77.6
55,4
24.6
10.2
1.0
VIT-B/16
75.7
14.6
0.9
VIT-L/16
79.2
55.1
23.4
9.9
1.8
ViT-SAM-B/16
76.7
63.4
37.0
20.1
3.8
ViT-B/16-Res
84.0
45.5
8.4
2.3
0.1
T2T-ViT-14
80.1
37.1
7.0
1.8
0.0
T2T-ViT-24
82.2
47.7
12.3
3.4
0.2
Deit-S/16
77.7
48.9
17.6
1.1
Dist-Deit-B/16
81.8
55.6
17.7
4.5
0.4
Swin-S/4
81.8
40.0
12.4
0.2
MLP-MixeT-B/16
73.8
41.9
10.7
4.3
0.4
ConvNeXt-S
82.7
42.4
8.1
2.6
0.0
SBResNet50
75.7
35.4
4.9
0.8
0.1
ResNeXt-32x4d-ssl
80.3
23.0
2.9
1.2
0.5
ResNet50-swsl
81.2
24.7
2.9
1.4
0.4
ResNet18
70.0
24.9
2.0
0.6
ResNet50-32x4d
77.6
28.2
3.2
1.2
0.4
ShuffleNet
69.4
15.0
0.6
0,2
0.0
MobileNet
71.9
16.7
0.4
0.0
0.0
VGG16
71.6
26.3
3.2
1.3
0.0
Table 4: Clean Accuracy (%) and Robust Accuracy (%) of target models against AutoAttack with different attack radii More results of the SOTA ViTs can be found in Appendix B
CA
RA against AutoAttack
Attack Radius
0.001
0.003
0.005
0.01
ViT-S/16
77.6
48.1
6.0
0.5
0.0
ViT-B/16
75.7
39.8
5.4
0.6
0.0
VIT-L/16
79.2
46.6
8.5
1.0
0.0
VTT-SAM-B/16
76.7
59.8
26.0
8.4
0.1
VYT-B/16-Res
84.0
27.7
0.9
0.0
0.0
T2T-ViT-14
80.1
12.9
0.1
0.0
0.0
T2T-ViT-24
82.2
20.8
0.3
0.0
0.0
Dist-Deit-S/16
79.3
43.1
3.7
0.2
0.0
Dist-Deit-B/IC
81.8
3.4
0.0
Swin-S/4
81.8
7.9
0.1
0.0
0.0
MLP-Mixer-BIII
73.8
0.0
ConvNeXt-S
82.7
15.9
0.0
0.0
0.0
SEResNe150
75.7
21.6
0.0
ResNeXI-32x4d-ssl
80.3
6.5
0.0
0.0
0.0
ResNet50-swsl
81.2
0.0
0.0
ResNet18
70.0
14.3
0.4
0.0
0.0
ResNet50-32x4d
77.6
13.2
0.0
ShuffleNet
69.4
6.1
0.0
0.0
0.0
MobileNet
71.9
0.0
0.0
VGG16
71.6
16.7
0.5
0.0
0.0
5.2.2 Transferability of Adversarial Examples from ViTs to CNNs and Vice Versa
We also conduct transfer attack to test the adversarial robustness in the black-box setting as described Section We consider attacks with form perturbation no larger than 0.1 and present the results in Figure When the
10

---

Published in Transactions on Machine Learning Research (10/2022)
attack radius = 0.1
attack radius = 0.01
1.0
10
VT 806
71710
VI 826
1.14%
-0.3
008
127 VTT 14
case
0.70N
6.724
0.78
T2TW/714
0
0.797
0.288
0.58 0.001
T27 24
6,50
6:645
0.533
6.500
6753
6:44
T2TVIT24
0322
2:24
8.440
0.350
6.465
0.02
06
And00
Red50
Ro000 smll
5487
0/883
C:10T
5:13
DJRKE
CATD
5.09
5388
6:34
0:331
0.381
0311
0333
3.38
Resto 32440
Res50
E2445
0:111
6:141
0.10
CAST
DATA
DAMA
Mobiitket
0788
8:316
0:223
0.347
-0.
00
5393 2872 CATS C3ss
6121 CLW
CAM
0229
6:27
8.277
6.31
2.14
Shuffellet
ShutfleNet
-0.0
-000
VEGLE
Rodiett
12P47--34 720-5532
Target model
Target
model
Figure 4: Target model error rate on adversarial examples (i.e., 1.0 RA) against transfer attack using FGSM with different attack radii. The rows stand for the surrogate models used to generated adversarial examples. The columns stand for the target models. Darker rows correlate to the source models that generate more transferable adversarial examples. Darker columns mean that the target models are more vulnerable to transfer attack. "Res50-ssI" and "Res50- swsl" are in short of "ResNeXt-32xx44a. and "ResNet50-swsl"" respectively. Results for more radii can be found in Appendix B
VITs serve as the target models and CNNs serve as the source models, as shown in the lower left of each subplot, the RER of the transfer attack is quite low. On the other hand, when the ViTs are the source models, the adversarial examples they generate have higher RER when transferred other target models. As result, the first three rows and the last seven columns are darker than the others. Besides, for the diagonal lines in the figure where FGSM actually attacks the models in a white-box setting, we can observe that ViTs are less sensitive to attack with smaller radii compared to CNNs, and T2T modules make ViTs more robust to such one-step attack. In addition, adversarial examples transfer well between models with similar structures. As ViT-S/16, ViT-B/16 and ViT-L/16 have similar structures, the adversarial examples generated by them can transfer well to each other, and it is similar for T2T-ViTs and CNNs respectively.
The finding that ViTs rely less on geghffreuencc patterns also helps us understand why ViT adversarial examples are better transferred to CNNs as shown in Figure 4 As studied in (Ilyas al. 2019), they found and called those features less prone to be attacked by adversarial attacks as "robust features". which is consistent with our understanding of the "high-level" features that are preferable for neural networks to learn. And those noise-like "non-robust" features are similar to our high-frequency features that contain low-level information. Since the "non-robust" high-frequency patterns are generated with specific models, adversarial perturbations with regard to such patterns should be assumed to be harder to transfer. Since ViTs are less sensitive to such high-frequency patterns, we assume the adversarial perturbations will be forced to rely less on model-specific feature patterns and thus be casier to transfer. is interesting to notice that ViTs' adversarial examples could be even stronger than the CNN counterparts with large attack radii, and we think it's because the transferable adversarial perturbations need larger attack radii which could be further explored.
5.3 Provably Certified Robustness Comparison Using Denoised Randomized Smoothing
Settings We train the denoisers using the stability objective for 25 epochs with a noise level of a = 0.25, learning rate of 10 and a batch size of 64. We iterate through the ImageNet dataset to calculate the corresponding radii according to Eq. 6 and report the certified accuracy versus different radii as defined in (Salman et al., 2020) in Table
Results As shown in Table 5 VIT-S/16 has higher certified accuracy than ResNet18 and ResNet50, showing better certified robustness of vision transformers over CNNs. We also found that, in the same settings, training a Gaussian denoiser for the ResNet is harder than for the VIT-S/16. Accuracy of ViT-S/16 with denoiser noise of 0.25 is
11

---

Published in Transactions on Machine Learning Research (10/2022)
Table 5: Certified robust accuracy w.r.t. different radii using denoised randomized smoothing (Salman et al.
2020
(a 0.25). Pure ViTs are highlighted with gray shadows.
Radius
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
VIT-S/16
0.5944
0.5452
0.4936
0.4424
0.3972
0.3428
0.2820
0.2044
0.0
DeiT-S/16
0.6352
0.5880
0.5380
0.4948
0.4476
0.4004
0.3408
0.2604
0.0
Dist-DeiT-S/16
0.6072
0.5600
0.5176
0.4716
0.4172
0.3620
0.3108
0.2360
0.0
SAM-ViT
0.6320
0.6040
0.5520
0.5360
0,5040
0.4600
0,4160
0.3560
0.0
T2T-ViT-14
0.4044
0.3816
0,3580
0.3348
0.3044
0.2660
0.2276
0.1816
0.0
VGG16
0.3772
0.3220
0.2796
0.2372
0.1964
0.1580
0.1176
0.0768
0.0
ResNet50
0.4584
0,4096
0.3604
0.3140
0.2676
0.2208
0.1820
0.1268
0.0
SEResNet50
0.4880
0.4440
0.3880
0.3360
0.2920
0.2680
0.2160
0.1760
0.0
ConvNext-S
0.5160
0.4760
0.4320
0.3920
0.3480
0.2880
0.2440
0.1880
0.0
64.84% (4.996% without any denoiser), while accuracy of ResNet50 and ResNet18 with denoiser at the same noise is 47.782% (5.966% without any denoiser).
Pure ViTs possess better certified robust accuracy than CNNs. As shown in Table 5 pure ViTs (VIT-S/16, DeiT- S/16, Dist-DeiT-S/16 and SAM-ViT) have higher certified robust accuracy than CNNs. Introducing T2T blocks ViTs cause the model to have inferior certified robust accuracy radii smaller than 0.5. While sharpness-aware minimization helps further improve ViTs* certified robust accuracy.
than CNNs especially for small radii, e.g..
Modern CNN design helps bridge the performance gap between CNNs and ViTs. The design of modern non- transformer models, e.g. ConyNext, has borrowed many techniques from transformers. For example, using larger kernel size to imitate the global attention mechanism of transformers, following the transformers to change stem to "Patchify", using invertible bottleneck as transformers do, substituting BN with LN, replacing ReLU with GeLU, etc, All these changes are targeted to imitate the transformer's operation without introducing the attention blocks. MLP- Mixer also does similar modifications. Our experiments (Table 3 Table 4 and Table show that such modification helps bridge the performance gap between CNNs and transformers not only in terms of clean accuracy, but also empirical and certified robust accuracy. We also show that CNNs with attention mechanism, i.c. SEResNe150 in our experiments, also has better certified robustness than CNNs.
5.4 Extended Analysis
we also conduct sanity check to verify that ViT's improvement is not caused by insufficient attack optimization, and an explanation from Hopfield network perspective provided. Besides, we verify that adversarial training could be directly applied to ViTs in Appendix
In Appedix A
6 Conclusion
This paper presents comprehensive study on the robustness of ViT's against adversarial perturbations. Our results in- dicate that ViTs are more robust than CNNs on the considered adversarial attacks and certified robustness settings. We show that the features learned by ViTs contain less low-level information, contributing improved robustness against adversarial perturbations that often contain fighffreyuency components. Also, introducing convolutional blocks in ViTs can facilitate learning low-level features but has negative effect on adversarial robustness and makes the mod- els more sensitive to high-frequency perturbations. Moreover, both the sanity performance and the (certified and empirical) adversarial robustness are improved in the modern CNN designs that leverage techniques from ViTs to imitate the global attention behavior. We also demonstrate adversarial training for VIT. Our work provides a deep understanding of the intrinsic robustness of ViTs and can be used inform the design of robust vision models based the transformer structure.
12

---

Published in Transactions on Machine Learning Research (10/2022)
References
Rima Alaifari, Giovanni S Alberti, and Tandri Gauksson. Adef: an iterative algorithm construct adversarial defor- mations. arXiv preprini
2018.
Faisal Alamri, Sinan Kalkan, and Nicolas Pugeault. Transformer-encoder detector module: Using context to improve robustness adversarial attacks on object detection. arXiv preprint arXiv:2011.06973 2020.
Ahmed Aldahdooh, Wassim Hamidouche, and Olivier Deforges. Reveal of vision transformers robustness against adversarial attacks. arXiv preprint arXiv:2106.03734 2021.
Anish Athalye. Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumvent- ing defenses to adversarial examples. International Coference International Conference Machine Learning. 2018.
Yutong Bai, Jieru Mei, Alan L Yuille, and Cihang Xie. Are transformers more robust than cnns? Advances in Neural Information Processing Systems, 34:26831-26843, 2021.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, In European Conference Computer Vision, pp. 213 229. Springer, 2020.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International Conference on Machine Learning, pp. 1691-1703. PMLR, 2020.
Xiangning Chen, Cho-Jui Hsieh, and Boqing Gong. When vision transformers outperform resnets without pretraining or strong data augmentations. arXiv preprint arXiv:2106.01544 2021.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fre free attacks. In International Conference on Machine Learning, pp. 2206-2216. PMLR, 2020a.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference Machine Learning, pp. 2196-2205 PMLR, 2020b.
Francesco Croce, Maksym Andriushchenko, and Matthias Hein. Provable robustness of relu networks via maximiza- tion of linear regions. In the 22nd International Conference on Artificial Intelligence and Statistics, pp. 2067-2066. PMLR, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and patlern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. arXiv preprint arXiv:1810.04800 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11999 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-awar minimization for efficiently improving generalization. arXiv preprini arXiv:2010.01412, 2020.
Siddhant Garg and Goutham Ramakrishnan. Bae: Bert-based adversarial examples for text classification. arXiv preprint arXiv:2004.01970, 2020.
13

---

Published in Transactions on Machine Learning Research (10/2022)
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1412.657, 2014.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Identity mappings in deep residual networks. In European conference on computer vision, pp. 330-645. Springer, 2016.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference Machine Learning, pp. 2712-2721. PMLR, 2019.
Charles Herrmann, Kyle Sargent, Lu Jiang, Ramin Zabih, Huiwen Chang, Ce Liu, Dilip Krishnan, and Deqing Sun. Pyramid adversarial training improves vit performance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13419-13429, 2022.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.0486) 2017.
Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, and Cho-Jui Hsieh. On the robustness of self-attentive models. In ACL, pp. 1520-1529, 2019.
Jie Hu, Li Shen, and Gang Sun. Squeere-and-exciiation networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7132-7141, 2018.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adver- sarial examples are not bugs, they are features. Advances neural information processing systems, 32, 2019.
Kishaan Jeeveswaran., Senthilkumar Kathiresan., Arna Varma., Omar Magdy., Bahram Zonooz., and Elahe Arani. A comprehensive study of vision transformers on dense prediction tasks. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications Volume 4: VISAPP, pp. INSTICC, SciTePress, 2022. ISBN 978-989-758--8 758-555-55
doi: 10,5220000119911800001111
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? strong baseline for natural lan- guage attack text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8025, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Dmitry Krotov and John Hopfield. Dense associative memory robust adversarial inputs. Neural computation, 30 (12):3151-1167, 2018
Dmitry Krotov and John J Hopfield. Dense associative memory for pattern recognition. Advances in Neural Informa- tion Processing Systems, 2016.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Pro- ceedings. OpenReview.net, 2017. URL BttLLttts///ppeeeiiiiee worp:r/ewwweeeeeeeeoomuummmmma
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. arXiv preprint arXiv:2004. 2020.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 197. 2019.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103. 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. arXiv preprint arXiv:2201.03555 2022.
14

---

Published in Transactions on Machine Learning Research (10/2022)
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: (tte::700.000885 2017.
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming Hc, Manohar Paluni, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings the the European
Conference on Computer Vision (ECCV), pp. 181-196, 2018.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. arXiv preprint arXiv:2104.02610, 2021.
Xiaofeng Mao, Gege Qi, Yuefeng Chen, Xiaodan Li, Ranjie Duan, Shaokai Ye, Yuan He, and Hui Xue. Towards robust vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12042-12051, 2022.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsoee Hsuan Yang, Intriguing properties of vision transformers. arXiv preprint arXiv:2105. 2021a.
Muzammal Naseer, Kanehana Ranasinghe, Salman Khan, Fahad Shahbaz Khan, and Fatih Porikli. On improving adversarial transferability of vision transformers. arXiv preprint N.KV::2006.04169, 2021b.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. arXiv preprint arXiv:2010.00467 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv: 191200073, 2019.
Sayak Paul and Pin-Yi Yu Chen. Vision transformers are robust learners. arXiv preprint arXiv:2105.0758/ 2021.
Yao Qin, Chiyuan Zhang, Ting Chen, Balaji Lakshminarayanan, Alex Beutel, and Xuezhi Wang. Understand- ing and improving robustness of vision transformers through patch-based negative augmentation. arXiv preprint arXiv:2110.07858, 2021.
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Hol- zleitner, Milena Pavlović, Geir Kjetil Sandve, Victor Greiff, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217, 2020.
Jonas Rauber, Roland Zimmermann, Matthias Bethge, and Wieland Brendel. Foolbox native: Fast adversarial attacks benchmark the robustness of machine learning models in pytorch, tensorflow, and jax. Jourmal of Open Source Software, 5(53):2607, 2020. doi:
URL bttps::/abi....
http://00.111111ss11 000.0........77.0 02607
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and J Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. arXiv preprint arXiv:2003.01900 2020.
Hadi Salman, Saachi Jain, Eric Wong, and Aleksander Mądry. Certified patch robustness via smoothed vision trans- formers. arXiv preprint arXiv:21 2021.
Zhouxing Shi and Minlie Huang. Robustness to modification with shared words in paraphrase identification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 164- 171, 2020.
Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, and Cho-Jui Hsich. Robustness verification for trans- formers. arXiv preprint arXiv:2002.06622 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXiv:1409.1556. 2014.
Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. Ernie: Enhanced representation through knowledge integration. arXiv preprint 1rXiv:/904.099 2019.
15

---

Published in Transactions on Machine Learning Research (10/2022)
Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, et al. Robustart: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211 2021.
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and 1(8), 2015.
Li-Jia Li. The new data and new challenges in multimedia research. arXiv preprint
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, al. Mip-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601, 2021.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Train- ing data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint arXiv: 2017.
Boxin Wang, Shuohang Wang, Y. Cheng, Zhe Gan, R. Jia, Bo Li, and Jing jing Liu. Infobert: Improving robustness of language models from an information thcoretic perspective. ArXiv, abs/2010.02329, 2020a.
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8684-8694, 2020b.
Ross Wightman. Pytorch image models. httess//gttuu...oo////attttttttr http:////ooooo---aammmemeeeeeeee 2019.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.33994, 2020.
Kaidi Xu, Zhouxing Shi. Huan Zhang, Yihan Wang, Kai- Kei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33, 2020.
Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546. 2019.
Zhilin Yang, Zihang Dai, Yiming Yang. Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint /0X1::/906.00277, 2019.
Mao Ye, Chengyue Gong, and Qiang Liu. Safer: A structure-free approach for certified robustness to adversarial word substitutions. arXiv preprint arXiv:2005.1442. 2020.
Fan Yin, Quanyu Long, Tao Meng, and Kai-Wei Chang. On the robustness of language encoders against grammatical errors. arXiv preprint arXiv:2005.05683, 2020.
Li Yuan, Yumpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens- to-token vit: Training vision transformers from scratch on imagenet. arXiv preprint arXiv:2101. 11986, 2021.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv: 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically princi- pled trade-off between robustness and accuracy. International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep for image denoising. IEEE transactions image processing, 26(7):3142-3155, 2017.
16

---

Published in Transactions on Machine Learning Research (10/2022)
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6848-6856, 2018.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint XXiv:2010.041599 2020.
17

---

Published in Transactions on Machine Learning Research (10/2022)
Supplemental Material
In this supplemental material, we provide more analysis and results our experiments.
A The Source of Adversarial Robustness
In this section examine the source of the adversarial robustness revealed in our experiments.
The improved robustness of VIT is not caused by insufficient attack optimization. We first demonstrate that the better robustness of ViTs in white-box attacks is not caused by the difficult optimization in ViT by plotting the loss landscape with sufficient attack steps.
ResNet18, 0.03
ResNet18, 0.01
Cross- EnEropp Loss
VIT-S/16, 0.03
200
VIT-S/16, & 0.01
80
Number of Iterations
Figure 5: Cross entropy loss versus varying PGD attack steps for VIT-S/16 and RestNet18. The dashed lines corre- sponds to larger attach radius of 0.03 and the full lines to smaller attack radius of 0.01.
E= 0,004
E=00008
VIT-S/16
60
VIT-S/16
ResNet18
ResNet18
40
40
20
10
20
30
40
20
40
E=0.01
E=0.03
60
VIT-S/16
VIT-S/16
ResNet18
30
ResNet18
40
10
10
20
30
40
0
10
20
30
40
Figure 6: Robust accuracy versus varying PGD attack steps.
The attack radii used for evaluation shown in subtitles.
18

---

Published in Transactions on Machine Learning Research (10/2022)
Figure shows the cross entropy loss versus varing PGD attack steps for VIT-S/16 and ResNet18. Figure shows the robust accuracy versus varing PGD attack steps. As shown in the figures, ViT's loss curves converge at much lower value than RestNet18, suggesting that the improved robustness of VIT is not caused by insufficient attack optimization.
VIT-S/16
VIT-R/16
VTT-1/16
0.709
0.314
0.153
0.534
6.715
0.691
0.667
0.216
12
0.715
0.297
100
0.425
IL735
9.625
0.52
0.413
0.355
0/004
0,768
0.456
0/186
0.789
0.567
IL442
I1289
0.228
06
EL/66
0,416
0,142
IL187
0.12K
06
0.714
0.552
0.425
0.291
6.22
66
0,683
0,316
0.158
0,013
0.022
0.533
9.219
0/91
0,025
0.019
0,003
6.31
0.144
0.51
0.021
0.545
0.214
0.05
1L017
0,005
0.2
IL415
0.137
0.554
0.013
0.189
022
0:55
0.211
0.098
00034
0,017
6.2
0.565
0.207
0.06
0.005
0.001
0.475
0.167
8.041
0
0.084
0.542
0.291
0.062
0.03
0.015
0.983
8.055
BI0SS
0.L
0.1
8.001
0.903
0,805
0.008
0.01
0.1
8.001
8.003
0.85
0.608
0.01
0.8
Epoliuus
Equlluna
Epelloxx
VIT-RR1G-Re4
V2T-VIT-14
VIT-T2T-24
1.8
1/1
1.
12229
0,200
1L293
6776
11:75
0.75
0:89
0.002
IL3135
1.7152
12.722
0.887
9,609
0,547
0.363
0.262
0
0L744
9.589
8,476
0.335
0.262
0.001
0,779
0.657
0,534
0:42
0,371
0.011
Slegs
0.731
B-406
IL135
B.061
66
IL665
0.409
8,251
IL124
0.184
IL715
0.511
0.161
8.338
1.155
6.6
0.51
0.14
ALRAS
1L002
0J002
IL444
0.184
8:166
1LO05
0
0.56
0.198
1.06
8.02
0L011
0.43
ILEU7
1L002
0,001
0:37
8J163
M112
0.2
1.478
0.31
0,035
D/004
0
0L006
0.046
8.07
0.903
Epelions
Egolions
Epstions
SeResNe150
ResNet50-swal
ResNer50-32xid
0.726
0.706
ILO7S
9,616
0,123
IL793
0,149
6.717
IL663
0.621
0.147
0,756
0.707
0.672
8.617
0.565
0.106
0.783
0.552
0,395
0.242
0.1G8
0
0.724
9.496
0.32
0.165
0.186
0.007
IUS
0.685
0.494
0.33
0.168
0.125
0.001
Steps
0.61
BL335
0L173
0.066
0/836
0.007
0.262
8.115
0,042
0.028
0.004
RUS
0,593
0.275
0.136
0.034
0.083
0L002
0.4
30
BJ036
6.117
0.011
0.012
044
9033
8/177
0.011
0.009
0/022
0.02
8/166
0.903
0,865
0.006
00
60
Epstions
Epstions
Figure 7: Adversarial accuracy of the target models against PGD attack with different attack radii ("eps") and attack steps ("steps"). When the attack radius and attack steps are increased, the adversarial accuracy of the target model decreases to zero. Darker blocks stand for more robust models against PGD attack.
shows the robust accuracy of more target models against PGD attack with different attack radii ("eps") and attack steps ("steps"). Vision transformers have darker blocks than CNNs', which stands for their superior adversarial robustness against PGD attack.
Figure 7
A Hopfield Network Perspective The equivalence between the attention mechanism in transformers to the modern Hopfield network (Krotov & Hopfield (016)) was recently shown in (Ramsauer et al. 2020). Furthermore, on simple Hopfield network (one layer of attention- network) and dataset (MNIST), improved adversarial robustness was shown in (Krotov & Hopfield 2018). Therefore, the connection of attention in transformers the Hopfield network can be used to explain the improved adversarial robustness for ViTs.
B Experiments on SOTA ViT Structures
In this section, we supplement the experimental results of recently proposed SOTA VITs.
Swin-Trie Trasnformer (Liu et al., 2021 computes the representations with shifted windows scheme which brings greater efficiency by limiting self-attention computation to non-overlgpping local windows while also allowing for cross-window connection.
DeiT (Touvron et al., 2021) further improves the ViTs' performance using data augmentation or distillation from CNN teachers with an additional distillation token.
SAM-VIT (Chen et al. 2021) uses sharpness-aware minimization (Foret et al. 2020) to train ViTs from scratch on ImageNet without large-scale pretraining or strong data augmentations.
Table summarizes the information of models investigated in our experiments. The window size of the swin trans- formers in Table 6 is 7. The pre-trained weights of these models are available in t package.
19

---

Published in Transactions on Machine Learning Research (10/2022)
Table 6: SOTA ViT models investigated in our experiments.
Model
Layers
Hidden size
Heads
Params
Deit-T/16 (Touvron al.,
2021
12
192
3
6M
Deit-S/16
Touvron et al.
2021
12
384
6
22M
Deit-B/Ie B/16
Touvrro et al.. 2021
12
768
12
87M
Dist-Deit-T/16
Touvron et al.
2021
12
192
6M
Dist-Deit-S/16
Touvron et al..
2021
12
384
6
22M
Dist-Deit-B/16 Touvron et al.
2021
12
768
12
87M
ViT-SAM-B/16
(Chen et al..
2021
12
768
12
87M
ViT-SAM-B/32 (Chen al.
2021
12
768
12
88M
Swin-T/4 (Liu et al.
2021
(2,2,6,2)
96
(3,6,12,24)
28M
Swin-S/4
Liu et al.
2021
(2,2,182)
96
(3,6,12,24)
50M
Swin-B/4
Liu al.
2021
(,,,,882)
128
(4,8,16,32)
88M
Swin-L/4
(Lu et al.
2021
(,,2,182)
192
(6,12,24,48)
197M
Table 7: Robust accuracy (%) of ViTs described in Table against 40-step PGD attack with different attack radii, and also the clean accuracy ("Clean"), A model is considered to be more robust if the robust accuracy is higher.
Model
Clean
0.001
0.003
0.005
0.01
Deit-T/16
72.3
36.8
8.3
2.6
0.3
Deit-S/16
77.7
48.9
17.6
7.1
1.1
Deit-B/16
81.3
46.6
14.3
6.0
0.9
Dist-Deit-T/ie
74.4
40.6
5.7
0.7
0.2
Dist-Deit-S/Un
79.3
52.4
15.1
4.3
0.3
Dist-Deit-Bie
81.8
55.6
17.7
4.5
0.4
ViT-SAM-B/16
76.7
63.4
37.0
20.1
3.8
ViT-SAM-B/32
63.8
53.2
32.3
19.7
3.1
Swin-T74
78.8
33.5
6.0
1.2
0.1
Swin-S/4
81.8
40.0
12.4
3.2
0.2
Swin-B/4
82.3
38.8
11.1
4.1
0.3
Swin-L/4
84.2
38.7
11.1
2.9
0.4
Table 7 shows the clean and robust accuracy of ViTs in Table 0 against 40-step PGD attack with different radii. And results for AutoAttack are shown in Table g Swin-transformers introduce shifted windows scheme that limit self- attention computation to non-overlapping local windows, which harms the robustness as Tokens-to- tokkeen scheme according to the above results.
C Experiments on Cifar-10
We choose the ImageNet as the benchmark because ViTs can hardly converge when training directly on small datasets like Cifar. Therefore, we finetune the ViTs instead. As shown Table ViT-B/4 performs higher robust accuracy than WiteResNet, which is consistent with the trend ImageNet.
20

---

Published in Transactions on Machine Learning Research (10/2022)
Table 8: Robust accuracy (%) of ViTs described Table against AutoAttack with different attack radii, and also the clean accuracy ("Clean"). A model is considered to be more robust if the robust accuracy higher.
Model
Clean
0.001
0.003
0.005
0.01
Deit-T/16
72.3
23.4
0.5
0.0
0.0
Deit-S/16
77.7
30.2
1.2
0.0
0.0
Deit-B/16
81.3
20.4
0.3
0.1
0.0
Dist-Deit-T/Ii
74.4
31.1
0.8
0.1
0.0
Dist-Deit-S/16
79.3
43.1
3.7
0.2
0.0
Dist-Deit-B/16
81.8
42.7
3.4
0.2
0.0
ViT-SAM-B/16
76.7
59.8
26.0
8.4
0.1
ViT-SAM-B/32
63.8
48.9
23.6
9.7
0.8
Swin-T7/ T/4
78.8
6.8
0.1
0.0
0.0
Swin-S/4
81.8
7.9
0.1
0.0
Swin-B/4
82.3
2.4
0.1
0.0
0.0
Swin-L/4
84.2
4.3
0.1
0.0
0.0
Table 9: Robust accuracy of ViT-B/4 and WideResNet against PGD-10 attack with different attack radii.
Model
0.001
0.003
0.01
0.03
ViT-B/4
0.9202
0.6242
0.0994
0.0103
WideResNet
0.7744
0.5923
0.0854
0.0000
Robustness Against Adversarial Deformation
Besides additively perturbing the correctly classified image, ADef (Alaifari et al.] 2018) iteratively applics small
deformations the clean data. We show the robust accuracy against such perturbations in Table TO] which is in accordance the results of PGD and AutoAttack.
Table 10: Robust accuracy (%) against AFef under the default setting described in Alaifari et al. 2018
Model
VIT-S/16
VGG16
DenseNet
MobileNet
ResNet18
Robust Accuracy
12.4
10.8
11.1
11.7
11.8
E Transfer Attack Results
Transfer attack results using more attack radii provided in Figure
F
Adversarial Training
Settings We also conduct a preliminary experiment on adversarial training for VIT. For this experiment we use CIFAR-10 (Krizhevsky al. 2009) with 6 8/255 and the ViT-B/16 model. Since originally this ViT was pre- trained on ImageNet with image size 224 x 224 and patch size 16 16 while image size on CIFAR-10 32 32, downsample the weights for patch embeddings and resize patches to 4 x 4. so that there are still 8 patches and name the model as ViT-B/4. Though ViT originally enlarged input images fine-tuning and evaluation, we keep the input size as 32 x 32 to make the attack radius comparable. For training, we use PGD-7 (PGD with 7 iterations) (Madry et al. 2017) and TRADES (Zhang et al. 2019) methods respectively, with no additional data during adversarial training. We compare VIT with two CNNs, ResNet18 (He et al., 2016) and WideResNet-344 34 (Zagoruyko & Komodakis 2016). save training cost, we train each model for epochs only, although some prior works used around hundreds epochs (Madry et al.) 20177] Pang et al. (2020) and are very costly for large models. We batch size of 128, initial learning of 0.1, SGD optimizer with momentum 0.9,
CIFAR-10 10 for natural
21

---

Published in Transactions on Machine Learning Research (10/2022)
attack radius 0.1
attack radius 0.08
attack radius 0,05
-22
-3
VEEW
attack radius 0.02
attack radius 0.01
attack radius 0,005
Ro 54
Fentars 2211
Figure 8: ASR of transfer attack using FGSM with different attack radii. The rows stand for the surrogate models used to generated adversarial examples in the white-box attack approach. The columns stand for the target models. Darker rows correlate to the source models that generate more transferable adversarial examples. While darker columns mean that the target models are more vulnerable to the transfer attack. "Res50-ss1" and "Res50-swsl" are in short of "ResNeXt-32x4d-ss"" and "ResNet50-swsl" respectively.
Table 11: Results of adversarial training for different models using PGD-7 (7-step PGD attack) and TRADES respec- tively on CIFAR-10. ViT-B/4 is a variant of ViT-B/16 where we downsample the patch embedding kernel from 16 16 to 4 x to accommodate the smaller image size on CIFAR-10. We report the clean accuracy (%) and robust accuracy (%) evaluated with PGD-10 and AutoAttack respectively. Each model is trained using only 20 epochs to reduce the cost.
Model
Method
Clean
PGD-10
AutoAttack
PGD-7
77.3
48.9
44.4
PreActResNet18
TRADES
77.6
49.4
44.9
PGD-7
80.3
52.2
48.4
WideResNet-34-10
TRADES
81.6
53.4
49.3
PGD-7
85.9
51.7
47.6
VIT-B/4
TRADES
85.0
53.9
49.2
and the learning rate decays after 15 epochs and 18 epochs respectively with a rate of 0.1. While we use weight decay of 5 x 10-4 for CNNs as suggested by Pang et al. (2020) that 5 x 10-4 is better than 2 x 10-4, we still use 2 x 10-4 for VIT as we find 5 x 10-4 causes an underfitting for VIT. We evaluate the models with PGD-10 (PGD with 10 iterations) and AutoAttack respectively.
Results We show the results in Table II The ViT model achieves higher robust accuracy compared to ResNet18, and comparable robust accuracy compared to WideResNet-344 10, while VIT achieves much better clean accuracy compared the other two models. Here ViT does not advance the robust accuracy after adversarial training compared to large CNNs such as WideResNNet34-10.. We conjecture that ViT may need larger training data longer training
22

---

Published in Transactions on Machine Learning Research (10/2022)
epochs to further improve its robust training performance, inspired by the fact that natural training VIT is not able to perform well either without large-scale pre-training- And although T2T-ViT improved the performance of natural training when trained from scratch, our previous results in Table 3 and Table 4 show that the T2T-ViT structure may be inherently less robust. We have also tried Wong et al. (2020) which was proposed to mitigate the overfitting FGSM to conduct fast adversarial training with FGSM, but we find that can still cause catastrophic overfitting for VIT such that the test accuracy on PGD attacks remains almost We conjecture that this fast training method may be not suitable for pre-trained models or require further adjustments. Our experiments in this section demonstrate that the adversarial training framework with PGD or TRADES is applicable for transformers vision tasks, and provide baseline results and insights for future exploration and improvement.
23


Reviews:
[{"review_id": "aRBMk7pmZN", "reviewer": "Reviewer_BF1i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ This paper companies a broad range of vision models, including CNNs, transformers and their combinations, on the same settings.\n+ It's interesting to see that the adversarial examples produced from ViT can transfer well to CNNs.\n\nWeaknesses:\n- Most adversarial attacks are PGD's and something similar. Other types of attacks are missing, such as optimization based attack or query based attack. It is unclear whether the current findings would also hold on those types of adversarial attacks.\n- The attack radius used for the experiments are quite small, and most results are reported with $\\epsilon$ less than or equal to 0.01. According Figure 8 in Appendix B, growing $\\epsilon$ from 0.01 to 0.1 will lead to a huge attack success rate increase. It would be interesting to provide a denser result curve wrt $\\epsilon$, and analyze whether the trend aligns with the low-frequency and high-frequency story. Also in Figure 7 of Appendix B, it would be better to show the accuracy or ASR wrt number of iterations -- the cross entropy loss metric does not align with other experiment results reported in the paper.\n- In Figure 4, can we add more analysis about why ViT adversarial examples are better transferred to CNNs? When $\\epsilon$ is 0.1, their adversarial examples are even stronger than the CNN counterparts. But when $\\epsilon$ goes lower to 0.01, the transferability goes bad. Does that relate to the conjecture authors provide regarding why transformers are more robust?\n- In Table 5, when adding more transformer blocks to T2T-ViT, the low-pass attack rate also goes down. Any explanation to that?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses adversarial robustness for vision transformers, especially on the classification tasks. Authors test a few white-box attacks and transfer-based black-box attacks on CNNs, transformers and their combinations. Through experiments, authors claim that vision transformers are usually more robust to adversarial attack, and adding more CNN components into transformers will degrade the robustness. Authors claim that transformers focus less on low level pattern and verifies the claim with experiments and visualizations.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Some nitpickings:\n* Page 4 line 5: should $x_i\\in\\mathbb{R}^{N \\times (P^2 \\cdot C\\)}$ or $x_i\\in\\mathbb{R}^{P^2\\cdot C}$? It seems that $x_i$ is only one patch (e.g. 16 by 16 one).\n* Page 5 line 4: what is the subscript $i$ in $x^{adv}_{t+1,t}$? I don't see $i$ in Eq 1 or Eq 2.\n* Page 10 last paragraph: \"Figure 3 in Appendix ??\". It should be in the main paper?\n* Figure 2 seems to be too big and doesn't contain useful information. Section 3.1 already covers everything.", "broader_impact_concerns": "No broader impact concerns regarding the submission.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RYOIXczXNz", "reviewer": "Reviewer_Z4jg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n— A comprehensive study is conducted across different architectures and comparison is performed. Such a study can benefit the community in understanding the properties of these architectures.\n\n— Different attacks are considered to understand the robustness of transformer architectures.\n\n— The Frequency study helps in understanding the type of features learned by transformer architectures.\n\n**Weaknesses:**\n\n— Although a thorough evaluation is conducted across architectures, it raises a question whether these are fair evaluations. [1] show that vision transformers might appear more robust than CNNs, but they need to account for the different training strategies and the architectural details such as the activation functions used (ReLU vs GeLU). With comparable training strategies, CNNs are shown to be as robust as ViTs on different attacks. Hence, the comparisons shown in this paper might not be a fair reflection on the whether robustness stems from the details described in [1,2] or due to other factors. \n\n— Authors conduct a frequency study and show that CNNs are more sensitive to high-frequency perturbations than transformers, thereby contributing to difference in robustness. However, previous works [3,4] have already studied this for CNNs. Hence, novelty and contributions of the proposed work remains limited. \n\n[1] - Are Transformers More Robust Than CNNs?,arXiv:2111.05464\n\n[2] - Can CNNs Be More Robust Than Transformers? , arXiv:2206.03452\n\n[3] -  High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks, arXiv:1905.13545\n\n[4] - Dissecting the High-Frequency Bias in Convolutional Neural Networks, CVPR 2021 workshop", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The proposed work conducts an extensive study on the difference between adversarial robustness of Vision transformer (ViT) and CNN architectures. Authors consider different networks and observe that transformers are more robust than CNNs for different attack settings such as white-box, transfer and certified robustness. Authors also show that adversarial training also benefit transformers achieving better robustness than CNNs. A frequency analysis is also performed where it is shown that ViTs contain less low-level information, thereby contributing to the robustness. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "— A comparable setting similar to [1,2] above can help understand what aspects of vision transformer contribute to the robustness. A discussion on the difference in inferences drawn compared to [3,4] can also be included in the frequency study section.\n\n— Robustness transfer for different computer vision tasks such as object detection can also be considered for evaluation. \n\n— Authors show that Sharpness Aware Minimization (SAM) can improve robustness significantly. It would be interesting to see if this can be applied to other architectures as well. \n\nMinor:\n\n—  Page 10, reference missing to appendix", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AeMKA0UFep", "reviewer": "Reviewer_UVpe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The investigation of adversarial robustness is an important topic, and it is interesting to evaluate novel architectures, such as ViT, MLP-Mixer, ConvNeXt-S. \n2. This work provides an extensive collection of robustness experiments on ViTs.\n3. As far as I can tell, this work indeed presents the first study that investigates the certified adversarial robustness of ViTs, which is interesting.\n\n## Weaknesses\n1. The focus of this work is scattered. For nearly every part of the work, a separate work could be done. Specific examples:   \n(a) While this might be the first work that evaluates the certified robustness of ViTs, the evaluation is limited to only three models. It would have been great, if more models would have been evaluated, such as ViT-B/16, ViT-L/16, T2T-ViT-14, ConvNeXt-S and SEResNet50.   \n(b) As the authors state, the adversarial training part is only a preliminary experiment and its results section raises more questions than it provides insights.   \nIn fact, I noticed that for most of the discussed points, dedicated works have been published, which go into much more detail and provide much more insight.\n\n2. A large body of similar works exists, and it is hard to differentiate what makes this work unique from other works. This discussion should not take place in Appendix A, but should be presented in the main manuscript, since it is crucial for the reader to understand, why reading this work is valuable. Especially, the following similarities stood out to me, which should be discussed. It should be clearly stated, how this work distinguishes itself in terms of technical novelty, but also in terms of insight, which cannot be observed in the other works.  \n(a) The phenomenon that vision transformers are more robust against adversarial perturbations (Figure 1, Table 2) has been widely observed in most other works.  \n(b) Transfer Attacks for ViTs have been discussed in [Naseer et al. (2021b), A, Mahmood et al. (2021), D].  \n(c) Adversarial Training of ViTs has been discussed in [B] and many works exist on the general robustness of ViTs [C, Salman et al. (2021)].  \n(d) A similar frequency study with similar observations has been performed in [Paul & Chen, D].\n\n3. Some results appear to be contradicting:  \n(a) The abstract states: “ViTs possess better adversarial robustness when compared with MLP-Mixer and convolutional neural networks (CNNs) including ConvNeXt.” However in Table 2: ConvNeXt (a modern convolutional neural network) is more robust than T2T-ViT-14? This contradicts the above claim and I am missing an in-depth discussion regarding this observation (exceeding the one provided in Sec. 6.1).   \n(b) Similar to (a) and more prominent, in Table 3 the CNNs ConvNeXt-S, SEResNet50, ResNet18, ResNet50-32x4d, and VGG16 appear all to be more robust than the transformer architectures T2T-ViT-14 and Swin-S/4.  \n(c) While “ViT-S/16 shows better robustness than larger ViT-B/16”, the even larger model Vit-L/16 also shows better robustness than ViT-B/16, and is on par or even more robust than ViT-S/16, which contradicts the explanations in the paper. \n\n4. The authors mention a possible explanation for the contradicting results in Table2 and Table3, where at least ConvNeXt is more robust than T2T-ViT-14 as the “remembering the low-level structures that repeatedly appear in the training dataset”, which is accompanied by visualization in Section 6.1. Unfortunately, this remains only a possible explanation without a definite answer, a proof, or more convincing experiments to support this explanation. Also, experiments to rule out other possibilities are missing.\n\n5. The authors claim that low-level features are less adversarially robust, which is linked to the frequency study. However, the link between frequency, robustness, and low-level features is not clear to me. I hope the authors could provide a proof, an extended discussion with supporting citations, or an extensive supportive experimental evaluation. \n\n[Naseer et al. (2021b)] On Improving Adversarial Transferability of Vision Transformers  \n[A] Towards Transferable Adversarial Attacks on Vision Transformers; AAAI 2022  \n[Salman et al. (2021)] Certified patch robustness via smoothed vision transformers  \n[B] Pyramid Adversarial Training Improves ViT Performance; ArXiv 2021  \n[C] Towards Robust Vision Transformer; CVPR 2022  \n[Salman et al. (2021)] Certified patch robustness via smoothed vision transformers  \n[Paul & Chen (2021)] Vision Transformers are Robust Learners  \n[Mahmood et al. (2021)] On the Robustness of Vision Transformers to Adversarial Examples  \n[D] Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs; BMVC 2021  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work studies the adversarial robustness of vision transformers (ViTs) from various angles. The adversarial robustness is studied through various experiments: white-box attacks, transfer attacks, black-box attacks, as well as certified robustness. The main finding is that “ViTs possess better adversarial robustness when compared with MLP-Mixer and convolutional neural networks”. This finding is further analyzed through a frequency study and a feature visualization. Additionally, the authors showed that adversarial training is applicable to ViTs.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "This work is a collection of robustness experiments on ViTs. However, the experiments fail to tell a coherent story, with many contradicting results. Additionally, due to the diversity of the experiments, they were not performed and evaluated in-depth, leaving many open questions. I hope the authors can address my points in the weakness section and convince me of the unique contributions of their work and how they relate to existing literature. ", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KU3unk3T3H", "reviewer": "Reviewer_N2u2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The topic of this paper is very relevant to the community.\n2. The paper is easy to follow.\n3. The study from the perspective of frequency is interesting.\n\nWeaknesses:\n1. The main claim of this paper is ViTs are more robust than CNNs, Mixer-MLP. The claim is only true when a very small perturbation range is allowed. When a large perturbation is applied, they are all equally vulnerable.\n\n2. The third observation that Adversarial training is also applicable to ViT for training robust models is trivial. The claim that pre-training without adversarial training on larger datasets does not significantly improve adversarial robustness is simply an observation, which cannot explain the core claim that ViTs are more robust.\n\n3. In Table 2, the author report Attack Success Rate (%) on different models, which is inappropriate. The author evaluates the clean accuracy of each model on the whole test set of ImageNet-1k, which includes the false classified images even under no perturbation. Hence, The reported score is not true ASR, which can be misleading.\n\n4. This work studies the robustness of vision transformers. However, some points are supported by weak experiments. For instance, the adversarial training experiments are conducted on CIFAR10. The contribution is not solid given this is an empirical study. \n\nGiven the limited novelty and the weakness above. This is not good enough for acceptance.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary: This work studies the adversarial robustness of vision transformers. The authors first show that ViTs possess better adversarial robustness than MLP-Mixer, CNNs, and ConvNeXt. Furthermore, the authors provided three empirical observations to explain why ViTs are more robust.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Remove the trivial Adversarial Training experiments or Make the experiments more comprehensive.\nReorganize the paper to better present the main findings.", "broader_impact_concerns": "I have no particular concerns regarding the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper received reviews from four expert reviewers in the field. Both Reviewer Z4jg and Reviewer N2u2 are satisfied with the author's paper revision and recommend to accept. Reviewer BF1i did not enter the final recommendation, but the AE reads the comments and the authors' revision and believes that there are no outstanding critical concerns. Reviewer UVpe suggested that most of their technical concerns have been sufficiently address. There was a remaining concern regarding the novelty with respect to several concurrent work (posted on arXiv). As 1) novelty is not one of TMLR's acceptance criteria and 2) arXiv publication timelines are irrelevant and should be ignored, the AE does not find this concern a valid ground for diminishing the contributions of this work.

Considering all the factors, the AE agrees with the reviewers that extensive analysis of adversarial robustness for vision transformers provides valuable insights to the community. The authors' revision also significantly improves the clarity of the exposition. The AE thus recommends to accept as is. 
Decision: Accept (poster)