Year: 2021

Abstract:
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides a comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs). This observation also holds for certified robustness. We summarize the following main observations contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less low-level information and are more generalizable, which contributes to superior robustness against adversarial perturbations. 2) Introducing convolutional or tokens-to-token blocks for learning low-level features in ViTs can improve classification accuracy but at the cost of adversarial robustness. 3) Increasing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. But for a pure transformer model, simply increasing the size or adding layers cannot guarantee a similar effect. 4) Pre-training without adversarial training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs. 5) Adversarial training is also applicable to ViT for training robust models. Furthermore, feature visualization and frequency analysis are conducted for explanation. The results show that ViTs are less sensitive to high-frequency perturbations than CNNs and there is a high correlation between how well the model learns low-level features and its robustness against different frequency-based perturbations.

text: Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides a comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs). This observation also holds for certified robustness. We summarize the following main observations contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less low-level information and are more generalizable, which contributes to superior robustness against adversarial perturbations. 2) Introducing convolutional or tokens-to-token blocks for learning low-level features in ViTs can improve classification accuracy but at the cost of adversarial robustness. 3) Increasing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. But for a pure transformer model, simply increasing the size or adding layers cannot guarantee a similar effect. 4) Pre-training without adversarial training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs. 5) Adversarial training is also applicable to ViT for training robust models. Furthermore, feature visualization and frequency analysis are conducted for explanation. The results show that ViTs are less sensitive to high-frequency perturbations than CNNs and there is a high correlation between how well the model learns low-level features and its robustness against different frequency-based perturbations.

heading: 1 INTRODUCTION
text: Transformers are originally applied in natural language processing (NLP) tasks as a type of deep neural network (DNN) mainly based on the self-attention mechanism (Vaswani et al. (2017); Devlin et al. (2018); Brown et al. (2020)), and transformers with large-scale pre-training have achieved state-of-the-art results on many NLP tasks (Devlin et al. (2018); Liu et al. (2019); Yang et al. (2019); Sun et al. (2019)). Recently, Dosovitskiy et al. (2020) applied a pure transformer directly to sequences of image patches (i.e., a vision transformer, ViT) and showed that the Transformer itself can be competitive with convolutional neural networks (CNN) on image classification tasks. Since then transformers have been extended to various vision tasks and show competitive or even better performance compared to CNNs and recurrent neural networks (RNNs) (Carion et al. (2020); Chen et al. (2020); Zhu et al. (2020)). While ViT and its variants hold promise toward a unified machine learning paradigm and architecture applicable to different data modalities, it remains unclear on the robustness of ViT against adversarial perturbations, which is critical for safe and reliable deployment of many real-world applications.
In this work, we examine the adversarial robustness of ViTs on image classification tasks and make comparisons with CNN baselines. As highlighted in Figure 1(a), our experimental results illustrate the superior robustness of ViTs than CNNs in both white-box and black-box attack settings, based on which we make the following important findings:
• Features learned by ViTs contain less low-level information and benefit adversarial robustness. ViTs achieve a lower attack success rate (ASR) of 51.9% compared with a minimum of 83.3% by CNNs in Figure 1(a). They are also less sensitive to high-frequency adversarial perturbations.
• Using denoised randomized smoothing (Salman et al., 2020), ViTs attain significantly better certified robustness than CNNs.
• It takes the cost of adversarial robustness to improve the classification accuracy of ViTs by introducing blocks to help learn low-level features as shown in Figure 1(a).
• Increasing the proportion of transformer blocks in the model leads to better robustness when the model consists of both transformer and CNN blocks. For example, the attack success rate (ASR) decreases from 87.1% to 79.2% when 10 additional transformer blocks are added to T2T-ViT-14. However, increasing the size of a pure transformer model cannot guarantee a similar effect, e.g., the robustness of ViT-S/16 is better than that of ViT-B/16 in Figure 1(a).
• Pre-training without adversarial training on larger datasets does not improve adversarial robustness though it is critical for training ViT.
• The principle of adversarial training through min-max optimization (Madry et al. (2017); Zhang et al. (2019)) can be applied to train robust ViTs.

heading: 2 RELATED WORK
text: Transformer (Vaswani et al. (2017)) has achieved remarkable performance on many NLP tasks, and its robustness has been studied on those NLP tasks. Hsieh et al. (2019); Jin et al. (2020); Shi & Huang (2020); Li et al. (2020); Garg & Ramakrishnan (2020); Yin et al. (2020) conducted adversarial attacks on transformers including pre-trained models, and in their experiments transformers usually show better robustness compared to other models based on Long short-term memory (LSTM) or CNN, with a theoretical explanation provided in Hsieh et al. (2019). However, due to the discrete nature of NLP models, these studies are focusing on discrete perturbations (e.g., word or character substitutions) which are very different from small and continuous perturbations in computer vision tasks. Besides, Wang et al. (2020a) improved the robustness of pre-trained transformers from an
information-theoretic perspective, and Shi et al. (2020); Ye et al. (2020); Xu et al. (2020) studied the robustness certification of transformer-based models. To the best of our knowledge, this work is one of the first studies that investigates the adversarial robustness (against small perturbations in the input pixel space) of transformers on computer vision tasks. There are some concurrent works studying the adversarial robustness of ViTs. We supplement their contribution in Appendix G.

heading: 3 MODEL ARCHITECTURES
text: We first review the architecture of models investigated in our experiments including several vision transformers (ViTs) and CNN models. A detailed table of comparison is given in Table 5.

heading: 3.1 VISION TRANSFORMERS
text: We consider the original ViT (Dosovitskiy et al. (2020)) and its four variants shown in Figure 1(b).
Vision transformer (ViT) and data-efficient image transformer (DeiT): ViT ( Dosovitskiy et al. (2020)) mostly follows the original design of Transformer (Vaswani et al. (2017); Devlin et al. (2018)) on language tasks. For a 2D image x ∈ RH×W×C with resolution H ×W and C channels, it is divided into a sequence ofN = H·WP 2 flattened 2D patches of size P ×P , xi ∈ R
N×(P 2·C) (1 ≤ i ≤ N). The patches are first encoded into patch embeddings with a simple convolutional layer, where the kernel size and stride of the convolution is exactly P × P . In addition, there are also position embeddings to preserve positional information. Similar to BERT (Devlin et al. (2018)), a large-scale pre-trained model for NLP, a special [CLS] token is added to output features for classification. DeiT ( Touvron et al. (2021)) further improves the ViT’s performance using data augmentation or distillation from CNN teachers with an additional distillation token. We investigate ViT-{S,B,L}/16, DeiT-S/16 and Dist-DeiT-B/16 as defined in the corresponding papers in the main text and discuss other structures in Appendix F.
Hybrid of CNN and ViT (CNN-ViT): Dosovitskiy et al. (2020) also proposed a hybrid architecture for ViTs by replacing raw image patches with patches extracted from a CNN feature map. This is equivalent to adding learned CNN blocks to the head of ViT as shown in Figure 1(b). Following Dosovitskiy et al. (2020), we investigate ViT-B/16-Res in our experiments, where the input sequence is obtained by flattening the spatial dimensions of the feature maps from ResNet50.
Hybrid of T2T and ViT (T2T-ViT): Yuan et al. (2021) proposed to overcome the limitations of the simple tokenization in ViTs, by progressively structurizing an image to tokens with a token-to-token (T2T) module, which recursively aggregates neighboring tokens into one token such that low-level structures can be better learned. T2T-ViT was shown to perform better than ViT when trained from scratch on a midsize dataset. We investigate T2T-ViT-14 and T2T-ViT-24 in our experiments.
Hybrid of shifted windows and ViT (Swin-T): Liu et al. (2021) computes the representations with shifted windows scheme, which brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. We investigate Swin-S/4 in the main text and discuss other structures in Appendix F.

heading: 3.2 CONVOLUTIONAL NEURAL NEWORKS
text: We study several CNN models for comparison, including ResNet18 (He et al. (2016)), ResNet5032x4d (He et al. (2016)), ShuffleNet (Zhang et al. (2018)), MobileNet (Howard et al. (2017)) and VGG16 (Simonyan & Zisserman (2014)). We also consider the SEResNet50 model, which uses the Squeeze-and-Excitation (SE) block (Hu et al. (2018)) that applies attention to channel dimensions to fuse both spatial and channel-wise information within local receptive fields at each layer.
The aforementioned CNNs are all trained on ImageNet from scratch. For a better comparison with pre-trained transformers, we also consider two CNN models pre-trained on larger datasets: ResNeXt-32x4d-ssl (Yalniz et al. (2019)) pre-trained on YFCC100M (Thomee et al. (2015)), and ResNet50-swsl pre-trained on IG-1B-Targeted (Mahajan et al. (2018)) using semi-weakly supervised methods (Yalniz et al. (2019)). They are both fine-tuned on ImageNet.

heading: 4 ADVERSARIAL ROBUSTNESS EVALUATION METHODS
text: We consider the commonly used `∞-norm bounded adversarial attacks to evaluate the robustness of target models. An `∞ attack is usually formulated as solving a constrained optimization problem:
max xadv
L ( xadv, y ) s.t. ∥∥xadv − x0∥∥∞ ≤ , (1) where x0 is a clean example with label y, and we aim to find an adversarial example xadv within an `∞ ball with radius centered at x0, such that the loss of the classifier L ( xadv, y ) is maximized. We consider untargeted attack in this paper, so an attack is successful if the perturbation successfully changes the model’s prediction. The attacks as well as a randomized smoothing method used in this paper are listed below.
White-box attack Four white-box attacks are involved in our experiments. The Projected Gradient Decent (PGD) attack (Madry et al. (2017)) solves Eq. 1 by iteratively taking gradient ascent:
xadvt+1 = Clipx0, (x adv t + α · sgn ( ∇xJ ( xadvt , y )) ), (2)
where xadvt stands for the solution after t iterations, and Clipx0, (·) denotes clipping the values to make each xadvt+1,i within [x0,i − ,x0,i + ], according to the `∞ threat model. As a special case, Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2014)) uses a single iteration with t = 1. AutoAttack (Croce & Hein (2020)) is currently the strongest white-box attack which evaluates adversarial robustness with a parameter-free ensemble of diverse attacks. We also design a frequencybased attack for analysis, which conducts attack under an additional frequency constraint:
xadvfreq = IDCT(DCT(x adv pgd − x0) Mf ) + x0, (3)
where DCT and IDCT stand for discrete cosine transform and inverse discrete cosine transform respectively, xadvpgd stands for the adversarial example generated by PGD, and Mf stands for the mask metric defined the frequency filter which is illustrated in Appendix B. We found this design similar to Wang et al. (2020b).
Black-box attack We consider the transfer attack which studies whether an adversarial perturbation generated by attacking the source model can successfully fool the target model. This test not only evaluates the robustness of models under the black-box setting, but also becomes a sanity check for detecting the obfuscated gradient phenomenon (Athalye et al. (2018)). Previous works have demonstrated that single-step attacks like FGSM enjoys better transferability than multi-step attacks (Kurakin et al. (2017)). We thus use FGSM for transfer attack in our experiments.
Denoised Randomized Smoothing We also evaluate the certified robustness of the models using randomized smoothing, where the robustness is evaluated as the certified radius, and the model is certified to be robust with high probability for perturbations within the radius. We follow Salman et al. (2020) to train a DnCNN (Zhang et al. (2017)) denoiser Dθ for each pre-trained model f with the “stability” objective with LCE denoting cross entropy and N denoting Gaussian distribution:
LStab = E(xi,yi)∈D,δLCE(f(Dθ(xi + δ)), f(xi)) where δ ∼ N (0, σ 2I). (4)
Randomized smoothing is applied on the denoised classifier f ◦ Dθ for robustness certification:
g(x) = arg max c∈Y
P[f(Dθ(x+ δ)) = c] where δ ∼ N (0, σ2I). (5)
The certified radius is then calculated for the smoothed classifier as (Cohen et al., 2019):
R = σ
2 (Φ−1(pA)− Φ−1(pB)), (6)
where Φ−1 is the inverse of the standard Gaussian CDF, pA = P(f(x+δ) = cA) is the confidence of the top-1 predicted class cA, and pB = maxc6=cA P(f(x+ δ) = c) is the confidence for the second top class. Accordingly, given a perturbation radius, the certified accuracy under this perturbation radius can be evaluated by comparing the given radius to the certified radius R.

heading: 5 EXPERIMENTAL RESULTS
text: In the experiments, we show the superior robustness of ViTs over CNNs against both white-box attacks and transfer attacks. We also study their certified robustness with the denoised randomized smoothing technique and conduct preliminary adversarial training experiments on ViT. Based on the experiments, we analyze what grants ViTs to be adversarially robust through different lens.
For all experiments, we load pre-trained ViT models and CNNs from the PyTorch image models library (timm, Wightman (2019)) and torchvision (Paszke et al. (2019)) respectively. We evaluate the clean accuracy of each model on the whole test set of ImageNet-1k (Deng et al. (2009)), and we sample 1,000 test examples to evaluate robust accuracy and attack success rate (ASR). Note that lower ASR means better robustness. Experimental results on CIFAR-10 are reported in Appendix E. For adversarial training we use CIFAR-10 (Krizhevsky et al. (2009)), as detailed in Section 5.4.

heading: 5.1 ROBUSTNESS UNDER WHITE-BOX ATTACKS
text: Settings We use PGD and AutoAttack to study the robustness under white-box attacks. We consider attack radius from {0.001, 0.003, 0.005, 0.01}. For PGD attack, we fix the attack steps to niter = 40 with other parameters following the default setting of the implementation in Foolbox (Rauber et al. (2020)). AutoAttack does not require any hyper-parameter tuning.
Results We present the results using PGD attack in Table 1 and the results using AutoAttack in Table 2. The ASR is approximately 100% on all the models when is large, e.g. when = 0.01. But for smaller attack radii, ViT models have lower ASR than CNNs under both PGD attack and AutoAttack. For example, when = 0.001, the ASR for ViT-S/16 is only 44.6% while the ASRs for CNNs are at least 70.0%. And under the same attack radius, the ASR of AutoAttack for ViTS/16 is only 51.9% compared to 93.9% of ShuffleNet. Besides, the results show that AutoAttack is much stronger than PGD attack under the same . These results demonstrate that ViT is more robust than CNNs under these white-box attacks. We also visualize the clean/robust accuracy tradeoff and model size of these models in Figure 1(a).

heading: 5.2 ROBUSTNESS UNDER TRANSFER ATTACK
text: We also conduct transfer attack to test the adversarial robustness in the black-box setting as described in Section 4. We consider attacks with `∞-norm perturbation no larger than 0.1 and present the results in Figure 2. When the ViTs serve as the target models and CNNs serve as the source models, as shown in the lower left of each subplot, the ASR of the transfer attack is quite low. On the other hand, when the ViTs are the source models, the adversarial examples they generate have higher ASR when transferred to other target models. As a result, the first three rows and the last seven columns
are darker than the others. Besides, for the diagonal lines in the figure where FGSM actually attacks the models in a white-box setting, we can observe that ViTs are less sensitive to attack with smaller radii compared to CNNs, and T2T modules make ViTs more robust to such one-step attack. In addition, adversarial examples transfer well between models with similar structures. As ViT-S/16, ViT-B/16 and ViT-L/16 have similar structures, the adversarial examples generated by them can transfer well to each other, and it is similar for T2T-ViTs and CNNs respectively.

heading: 5.3 CERTIFIED ROBUSTNESS
text: Settings We train the denoisers using the stability objective for 25 epochs with a noise level of σ = 0.25, learning rate of 10−5 and a batch size of 64. We iterate through the ImageNet dataset to calculate the corresponding radii according to Eq. 6, and report the certified accuracy versus different radii as defined in Salman et al. (2020) in Figure 3.
Results As shown in Figure 3, ViT-S/16 has higher certified accuracy than ResNet18, showing better certified robustness of vision transformers over CNNs. We also found that, in the same settings, training a Gaussian denoiser for the ResNet18 is harder than for the ViT-S/16. Accuracy of ViT-S/16 with denoiser at noise of σ = 0.25 is 64.84% (4.996% without any denoiser), while accuracy of ResNet50 and ResNet18 with denoiser at the same noise is 47.782% (5.966% without any denoiser).

heading: 5.4 ADVERSARIAL TRAINING
text: Settings We also conduct a preliminary experiment on adversarial training for ViT. For this experiment we use CIFAR-10 (Krizhevsky et al. (2009)) with = 8/255 and the ViT-B/16 model. Since originally this ViT was pre-trained on ImageNet with image size 224 × 224 and patch size 16 × 16 while image size on CIFAR-10 is 32 × 32, we downsample the weights for patch embeddings and resize patches to 4 × 4, so that there are still 8 × 8 patches and we name the new model as ViT-B/4. Though ViT originally enlarged input images on CIFAR-10 for natural fine-tuning and evaluation, we keep the input size as 32 × 32 to make the attack radius comparable. For training, we use PGD-7 (PGD with 7 iterations) (Madry et al. (2017)) and TRADES (Zhang et al. (2019)) methods respectively, with no additional data during adversarial training. We compare ViT with two CNNs, ResNet18 (He et al. (2016)) and WideResNet-34-10 (Zagoruyko & Komodakis (2016)). To save training cost, we train each model for 20 epochs only, although some prior works used around hundreds of epochs (Madry et al. (2017); Pang et al. (2020)) and are very costly for large models. We use a batch size of 128, an initial learning rate of 0.1, an SGD optimizer with momentum 0.9, and the learning rate decays after 15 epochs and 18 epochs respectively with a rate of 0.1. While we use a weight decay of 5× 10−4 for CNNs as suggested by Pang et al. (2020) that 5× 10−4 is better than 2× 10−4, we still use 2× 10−4 for ViT as we find 5× 10−4 causes an underfitting for ViT. We evaluate the models with PGD-10 (PGD with 10 iterations) and AutoAttack respectively.
Results We show the results in Table 3. The ViT model achieves higher robust accuracy compared to ResNet18, and comparable robust accuracy compared to WideResNet-34-10, while ViT achieves much better clean accuracy compared to the other two models. Here ViT does not advance the robust accuracy after adversarial training compared to large CNNs such as WideResNet-34-10. We conjecture that ViT may need larger training data or longer training epochs to further improve its robust training performance, inspired by the fact that on natural training ViT is not able to perform well either without large-scale pre-training. And although T2T-ViT improved the performance of natural training when trained from scratch, our previous results in Table 1 and Table 2 show that the T2T-ViT structure may be inherently less robust. We have also tried Wong et al. (2020) which was proposed to mitigate the overfitting of FGSM to conduct fast adversarial training with FGSM, but we find that it can still cause catastrophic overfitting for ViT such that the test accuracy on PGD attacks remains almost 0. We conjecture that this fast training method may be not suitable for pretrained models or require further adjustments. Our experiments in this section demonstrate that the adversarial training framework with PGD or TRADES is applicable for transformers on vision tasks, and we provide baseline results and insights for future exploration and improvement.

heading: 6 REASONING OF ADVERSARIAL ROBUSTNESS
text: In this section, we present extended analysis to dissect the source of improved adversarial robustness in ViTs. We also verify that ViT’s improvement is not caused by insufficient attack optimization, and an explanation from Hopfield network perspective is provided (see Appendix D for details).
Learning Low-Level Structures Makes Models Less Robust One interesting and perhaps surprising finding is that ViTs have worse robustness when modules that help to learn local structures are added ahead of the transformer blocks. For example, T2T-ViT adds several T2T modules to the head of ViT which iteratively aggregates the neighboring tokens into one token in each local perceptive field. ViT-B/16-Res takes the features generated by ResNet as inputs, which has the same effect as incorporating a trained CNN layer in front of the transformer blocks. Both modules help to learn local structures like edges and lines (Yuan et al. (2021)).
When the features learned by ResNet are introduced, the ASR of ViT-B/16 rises from 51.1% to 54.5% of ViT-B/16-Res under PGD attack, and from 60.2% to 72.3% under AutoAttack, with attack radius = 0.001. A similar phenomenon can be observed by comparing the ASR of ViTs and T2T-ViTs. The ASR of T2T-ViT-14 is 18.3% higher under PGD attack and 35.2% higher under AutoAttack compared with the ASR of ViT-S/16, under attack radius = 0.001.
One possible explanation is that the introduced modules improve the classification accuracy by remembering the low-level structures that repeatedly appear in the training dataset. These structures such as edges and lines are high-frequent and sensitive to perturbations. Learning such features
makes the model more vulnerable to adversarial attacks. Examination of this hypothesis is conducted in Section 6.1.
Increasing the Proportion of Transformer Blocks Can Improve Robustness Hendrycks et al. (2019) mentioned that larger model does not necessarily imply better robustness. It can be confirmed by our experiments where ViT-S/16 shows better robustness than larger ViT-B/16 under both PGD attack and AutoAttack. In this case, simply adding transformer blocks to the classifier cannot guarantee better robustness. However, we recognize that for mixed architecture that has both T2T and transformer blocks, it is useful to improve adversarial robustness by increasing the proportion of the transformer blocks in the model. As shown in Tables 1 and 2, T2T-ViT-24 has lower ASR than T2T-ViT-14 under both attacks. Besides the transformer block, we find that other attention mechanism modules such as SE block also improves adversarial robustness – as SEResNet50 has the least proportion of attention, the ASR of SEResNet50 is higher than ViT and T2T-ViT models but lower than other pure CNNs. These two findings are coherent since the attention mechanism is fundamental in transformer blocks.
Pre-training without Adversarial Training Does Not Improve Robustness Pre-training is critical for ViTs to achieve competitive standard accuracy with CNNs trained from scratch (Dosovitskiy et al. (2020)). However, pre-training may not be the main reason of better robustness. To illustrate this point, we include CNNs pre-trained on large datasets and fine-tuned on ImageNet-1k to check the effect of pre-training on adversarial robustness. CNNs pre-trained on large datasets IG1B-Targeted (Mahajan et al. (2018)) and YFCC100M (Thomee et al. (2015)) that are even larger than ImageNet-21k used by ViT, ResNet50-swsl and ResNeXt-32x4d-ssl, still have similar or even higher ASR than ResNet18 and ResNet50-32x4d that are not pre-trained. This supports our observation that pre-training in its current form may not be able to improve adversarial robustness. Hendrycks et al. (2019) also reported that pre-training without adversarial training techniques cannot improve adversarial robustness. The resilience of ViT against perturbations corresponds more to the transformer structure rather than pre-training.

heading: 6.1 FREQUENCY STUDY AND FEATURE VISUALIZATION
text: Here we conduct a frequency study and feature visualization to support our claim that ViTs are more robust because they learn to focus less on high-frequency features than other models. Such property can be diminished when introducing other modules to the ViT’s structure, causing the inferior adversarial robustness of hybrid ViTs.
Frequency Study We design a frequency study to verify our hypothesis that ViTs are adversarially more robust compared with CNNs because ViTs learn less high-frequency features. As defined in equation 3, for adversarial perturbations generated by PGD attack, we first project them to the frequency domain by DCT. We design three frequency filters as shown in Figure 5 in Appendix B: the full-pass filter, the low-pass filter, and the high-pass filter. We take 32 × 32 pixels in the low-frequency area out of 224 × 224 pixels as the low-pass filter, and 192 × 192 pixels in the high-frequency area as the high-pass filter. Each filter allows only the corresponding frequencies to pass through – when the adversarial perturbations go through the low-pass filter, the high-frequency
components are filtered out and vice versa, and the full-pass filter makes no change. We then apply these filters to the frequencies of the perturbations, and project them back to the spacial domain with the IDCT. We test the ASR under different frequency areas, and show the results in Table 4.
The ASR of ViT is relatively low in the “High-pass” column when only the high-frequencies of the perturbations are preserved. In contrast, CNNs show significantly higher ASR in the “High-pass” column than in the “Low-pass” column. It reflects that CNNs tend to be more sensitive to highfrequency adversarial perturbations compared to ViTs. We also observe that adding modules that learn low-level structures makes the models more sensitive to high-frequency perturbations. T2TViT-14, T2T-ViT-24 and ViT-B/16-Res have higher ASR in the “High-pass” column and lower ASR in the “Low-pass” column compared with vanilla ViTs, which verifies our hypothesis that low-level features are less adversarially robust. Besides, when adding more transformer blocks to the T2T-ViT model, the model becomes less sensitive to the high frequencies of the adversarial perturbations, e.g., the T2T-ViT-24 has an 8.7% lower ASR than that of the T2T-ViT-14 in the “High-pass” column.
Feature Visualization We follow the work of Yuan et al. (2021) to visualize the learned features from the first blocks of the target models in Figure 4. We resize the input images to a resolution of 224 × 224 for CNNs and a resolution of 1792 × 1792 for ViTs and T2T-ViTs such that the feature maps from the first block are in the same shape of 112 × 112. Low-level features like lines and edges are highlighted in blue (obviously perceptible) and green (minorly perceptible). As shown in Figure 4, CNNs like ResNet50-swsl and ResNet50-32x4d learn features with obvious edges and lines. Minorly perceptible low-level features are learned by T2T-ViT-24 and ViT-B/16-Res. While it is hard to observe such information in the features maps learned by ViT-B/16.
The feature visualization combined with the frequency study shows that the model’s vulnerability under adversarial perturbations is highly relative to the model’s tendency to learn low-level highfrequency features. Techniques that help the model learn such features may improve the performance on clean data but at the risk of sacrificing adversarial robustness.

heading: 7 CONCLUSION
text: This paper presents the first study on the robustness of ViTs against adversarial perturbations. Our results indicate that ViTs are more robust than CNNs on the considered adversarial attack and certified robustness settings. Moreover, we show that the features learned by ViTs contain less low-level information, contributing to improved robustness against adversarial perturbations that often contain high-frequency components; introducing convolutional blocks in ViTs can facilitate learning low-level features but has a negative effect on adversarial robustness and makes the models more sensitive to high-frequency perturbations. We also demonstrate adversarial training for ViT. Our work provides a deep understanding of the intrinsic robustness of ViTs and can be used to inform the design of robust vision models based on the transformer structure.

heading: A TARGET MODELS
text: Summary of the target models investigated in the main text are shown in Table 5. The weights of these models are all publicly available at Paszke et al. (2019); Wightman (2019) such that our experiments can be easily reproduced.

heading: B FREQUENCY FILTERS
text: We show the design of frequency-filters in Figure 5.

heading: C TRANSFER ATTACK RESULTS
text: Transfer attack results using more attack radii are provided in Figure 6

heading: D THE SOURCE OF ADVERSARIAL ROBUSTNESS
text: In this section we examine the source of the adversarial robustness revealed in our experiments.
The improved robustness of ViT is not caused by insufficient attack optimization. We first demonstrate that the better robustness of ViTs in white-box attacks is not caused by the difficult optimization in ViT by plotting the loss landscape with sufficient attack steps.
Figure 7 shows the cross entropy loss versus varing PGD attack steps for ViT-S/16 and ResNet18. As shown in the figure, ViT’s loss curves converge at a much lower value than RestNet18, suggesting that the improved robustness of ViT is not caused by insufficient attack optimization.
Figure 8 shows the robust accuracy of more target models against PGD attack with different attack radii (“eps”) and attack steps (“steps”). Vision transformers have darker blocks than CNNs’, which stands for their superior adversarial robustness against PGD attack.
A Hopfield Network Perspective The equivalence between the attention mechanism in transformers to the modern Hopfield network (Krotov & Hopfield (2016)) was recently shown in Ramsauer et al. (2020). Furthermore, on simple Hopfield network (one layer of attention-like network) and dataset (MNIST), improved adversarial robustness was shown in Krotov & Hopfield (2018). Therefore, the connection of attention in transformers to the Hopfield network can be used to explain the improved adversarial robustness for ViTs.

heading: E EXPERIMENTS ON CIFAR-10
text: We choose the ImageNet as the benchmark because ViTs can hardly converge when training directly on small datasets like Cifar. Therefore, we finetune the ViTs instead. As shown in Table 6, ViT-B/4 performs higher robust accuracy than WideResNet, which is consistent with the trend on ImageNet.

heading: F EXPERIMENTS ON SOTA VIT STRUCTURES
text: In this section, we supplement the experimental results of recently proposed SOTA ViTs.
Swin-Trasnformer (Liu et al., 2021) computes the representations with shifted windows scheme which brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection.
DeiT (Touvron et al., 2021) further improves the ViTs’ performance using data augmentation or distillation from CNN teachers with an additional distillation token.
SAM-ViT (Chen et al., 2021) uses sharpness-aware minimization (Foret et al., 2020) to train ViTs from scratch on ImageNet without large-scale pretraining or strong data augmentations.
Table 7 summarizes the information of models investigated in our experiments. The window size of the swin transformers in Table 7 is 7. The pre-trained weights of these models are available in timm package.
Table 8 shows the clean and robust accuracy of ViTs in Table 7 against 40-step PGD attack with different radii. And results for AutoAttack are shown in Table 9. Swin-transformers introduce shifted windows scheme that limit self-attention computation to non-overlapping local windows, which harms the robustness as Tokens-to-Token scheme according to the above results.

heading: G DISCUSSION OF CONCURRENT AND RELATED WORKS
text: In the context of computer vision, one earliest relevant work is Alamri et al. (2020) which applies transformer encoder in the object detection task and reports better adversarial robustness. But the model they considered is a mix of CNN and transformer instead of the ViT model considered in this paper. Besides, the attacks they applied were relatively weak, and there lacks study and explanation on the benefit of adversarial robustness brought by the transformers.
Recently, there are many concurrent or follow-up works investigating the adversarial robustness of vision transformers. Many of them have cited our work and shown valuable discussion or extended experiments. We acknowledge their contributions below. Mahmood et al. (2021) test the adversarial robustness of the transformer in both white-box and black-box settings, analyze the security of a simple ensemble defense of CNNs and transformers, and find such ensemble defense is effective for the black-box setting. Qin et al. (2021) and Salman et al. (2021) investigate the adversarial robustness of ViTs through the lens of their special patch-based architectural structure. Naseer et al. (2021b) investigate whether the weak transferability of adversarial patterns from high-performing ViT models, as reported in our work, is a result of weak features or a weak attack. Aldahdooh et al. (2021); Naseer et al. (2021a); Paul & Chen (2021); Tang et al. (2021) study the adversarial robustness from different views, e.g., preprocessing defense methods (Aldahdooh et al., 2021), shape recognition capability (Naseer et al., 2021a), and natural adversarial examples (Paul & Chen, 2021; Tang et al., 2021).
Our work differs from these concurrent and related works by focusing more on the origin of the adversarial robustness of vision transformers. We discuss the superior adversarial robustness of ViTs through the lens of frequency, and find that the ViTs are especially robust to high-frequency adversarial perturbations. We also apply denoised randomized smoothing and show ViT also has superior certified robustness than CNN models. Our study provides insight on understanding the source of ViT’s adversarial robustness and designing more robust architectures.

heading: H ROBUSTNESS AGAINST ADVERSARIAL DEFORMATION
text: Besides additively perturbing the correctly classified image, ADef (Alaifari et al., 2018) iteratively applies small deformations to the clean data. We show the robust accuracy against such perturbations in Table 10, which is in accordance to the results of PGD and AutoAttack.

Review summaries:
 This paper studies the adversarial robustness of ViTs. There are several significant strengths and weaknesses in this paper. Especially, the novelty of this paper against one published paper is limited. It would be good to see my detailed comments below.  

This paper provides a comprehensive study on the robustness of ViTs against adversarial perturbations. The authors found that 1) ViTs has better adversarial robustness than convolutional neural networks; 2) Introducing convolutional or tokens-to-token blocks can improve the classification accuracy but at the cost of the adversarial robustness; 3) More proportion of transformers has better robustness; 4) Pre-training on larger datasets does not improve adversarial robustness; 5) Adversarial training is applicable to ViTs. In addition, many experiments verify the findings on white-box, transfer attack settings and adversarial training.  

This work studies the adversarial robustness of vision transformers comprehensively. The author first presents their finding via empirical experiments. Concretely, they show the robustness of vision transformers from the perspective of adversarial attack, transferability, and certified robustness, adversarial training defense. Then, they analyze the reason behind the observation.