Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?

1 INTRODUCTION . The recent performance breakthroughs achieved by vision transformers ( ViTs ) Dosovitskiy et al . ( 2020 ) have fueled an increasing enthusiasm towards designing new ViT architectures for different vision tasks , including object detection ( Carion et al. , 2020 ; Beal et al. , 2020 ) , semantic segmentation ( Strudel et al. , 2021 ; Zheng et al. , 2021 ; Wang et al. , 2021 ) , and video recognition ( Arnab et al. , 2021 ; Liu et al. , 2021b ; Li et al. , 2021b ; Fan et al. , 2021 ) . To fulfill the goal of deploying ViTs into real-world vision applications , the security concern of ViTs is of great importance and challenge , especially in the context of adversarial attacks ( Goodfellow et al. , 2014 ) , under which an imperceptible perturbation onto the inputs can mislead the models to malfunction . In response , the robustness of ViTs against adversarial attacks has attracted increasing attention . For example , recent works ( Bhojanapalli et al. , 2021 ; Aldahdooh et al. , 2021 ; Shao et al. , 2021 ) find that in addition to ViTs ’ decent task performance , they are more robust to adversarial attacks compared with convolutional neural networks ( CNNs ) under comparable model complexities . In particular , ( Shao et al. , 2021 ) claims that ViTs focus more on capturing the global interaction among input/feature patches via its self-attention mechanism and the learned features contain less low-level information , leading to superior robustness to the local perturbations introduced by adversarial attacks . A natural response to this seemingly good news would be determining whether ViTs are truly robust against all kinds of adversarial perturbations or if their current winning in robustness is an inevitable result of biased evaluations using existing attack methods that are mostly dedicated to CNNs . To unveil the potential vulnerability of ViTs , this work takes the first step in asking an intriguing question : “ Under what kinds of perturbations do ViTs become weaker learners compared to CNNs ? `` , and makes the following contributions : • We conduct extensive experiments to benchmark the robustness of different ViT variants , which integrate recent advances and new arts in ViTs ’ architecture design , over that of CNNs under existing adversarial attacks ; • We propose a new attack framework , dubbed Patch-Fool , aiming to fool the self-attention mechanism by attacking the basic component ( i.e. , a single patch ) participating in selfattention calculations . Our Patch-Fool attack features a novel objective formulation , which is then solved by Patch-Fool ’ s integrated attention-aware patch selection technique and attention-aware loss design ; • We evaluate the robustness of both ViTs and CNNs against our Patch-Fool attack with extensive experiments and find that ViTs are consistently less robust than CNNs across various attack settings , indicating that ViTs are not always robust learners and their seeming robustness against existing attacks can be overturned under dedicated adversarial attacks ; • We further benchmark the robustness of both ViTs and CNNs under a sparse variant of Patch-Fool , dubbed Sparse Patch-Fool , and discover that the perturbation density , defined as the number of perturbed pixels per patch , highly influences the robustness ranking between ViTs and CNNs , where our Patch-Fool is an extreme case of high perturbation density . We believe our work has opened up a new perspective for exploring ViTs ’ vulnerability and can provide insights to both future architecture designs and training schemes for robustifying ViTs towards their real-world deployment . 2 RELATED WORKS . Vision transformers . Motivated by the great success of Transformers in the natural language processing ( NLP ) field ( Vaswani et al. , 2017 ) , ViTs have been developed by splitting an input image into a series of image patches and adopting self-attention modules for encoding the image ( Dosovitskiy et al. , 2020 ) , and been shown to achieve competitive or superior performance over CNNs via dedicated data augmentation ( Touvron et al. , 2021 ) or self-attention structure ( Yang et al. , 2021 ; Graham et al. , 2021 ; Liu et al. , 2021a ) . As such , there has been tremendously increased attention on applying ViTs to various computer vision applications , such as self-supervised learning ( Caron et al. , 2021 ; Chen et al. , 2021b ; Xie et al. , 2021 ; Li et al. , 2021a ) , object detection ( Carion et al. , 2020 ; Beal et al. , 2020 ) , and semantic segmentation ( Strudel et al. , 2021 ; Zheng et al. , 2021 ; Wang et al. , 2021 ) . The achievable performance of ViTs are continuously refreshed by emerging ViT variants , which provide new arts for designing ViT architectures . For example , convolutional modules have been incorporated into ViTs for capturing low-level features ( Xiao et al. , 2021 ; Wu et al. , 2021 ; Graham et al. , 2021 ; Peng et al. , 2021 ) , and replacing the global self-attention mechanism with local self-attention modules ( Liu et al. , 2021a ; Dong et al. , 2021 ; Liang et al. , 2021 ; Liu et al. , 2021b ; Chu et al. , 2021 ) . Motivated by ViTs ’ impressive performance and the growing interest in deploying ViTs into real-world applications , this work aims to better understand the robustness of ViTs and to develop adversarial attacks dedicated to ViTs . Adversarial attack and defense . Deep neural networks ( DNNs ) are known to be vulnerable to adversarial attacks ( Goodfellow et al. , 2014 ) , i.e. , imperceptible perturbations onto the inputs can mislead DNNs to make wrong predictions . As adversaries , stronger attacks are continuously developed , including both white-box ( Madry et al. , 2017 ; Croce & Hein , 2020 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016 ; Moosavi-Dezfooli et al. , 2016 ) and black-box ones ( Chen et al. , 2017 ; Ilyas et al. , 2018b ; Andriushchenko et al. , 2020 ; Guo et al. , 2019 ; Ilyas et al. , 2018a ) , which aggressively degrade the performances of the target DNN models . In response , various defense methods have been proposed to improve DNNs ’ robustness against those adversarial attacks . For example , purification methods ( Guo et al. , 2017 ; Xie et al. , 2017 ; Xu et al. , 2017 ; Song et al. , 2017 ; Shi et al. , 2021 ) applied onto the inputs purify the adversarial samples back to the distribution of clean ones ; random smoothing ( Cohen et al. , 2019 ; Li et al. , 2018 ) onto the inputs certifiably guarantees DNNs ’ robustness against adversarial perturbations ; detection-based methods ( Metzen et al. , 2017 ; Feinman et al. , 2017 ; Lee et al. , 2018 ) equipped with additional detection modules distinguish adversarial examples from clean ones before feeding them into the target DNN models ; and adversarial training ( Shafahi et al. , 2019 ; Madry et al. , 2017 ; Wong et al. , 2019 ; Tramèr et al. , 2017 ) , which augments the training set with adversarial samples generated on the fly during training , is currently one of the most effective defense methods . There has been a continuous war between adversaries and defenders and the readers are referred to ( Akhtar & Mian , 2018 ; Chakraborty et al. , 2018 ) for more attack and defense methods . Robustness of vision transformers . Driven by the impressive performance recently achieved by ViTs in various vision tasks , their robustness has gained increasing attention . A consistent observation drawn by pioneering works that study ViTs ’ robustness is that ViTs are more robust to adversarial attacks than CNNs since ViTs are more capable of capturing the global interactions among patches , while CNNs focus on local features and thus are more vulnerable to local adversarial perturbations . In particular , ( Bhojanapalli et al. , 2021 ) shows that ViT models pretrained with a sufficient amount of data are at least as robust as their ResNet counterparts on a broad range of perturbations , including natural corruptions , distribution shifts , and adversarial perturbations ; ( Aldahdooh et al. , 2021 ) finds that vanilla ViTs or hybrid-ViTs are more robust than CNNs under 𝐿𝑝-based attacks ; and ( Shao et al. , 2021 ) further explains that ViTs ’ learned features contain less low-level information and are more generalizable , leading to their superior robustness , and introducing convolutional blocks that extract more low-level features will reduce the ViTs ’ adversarial robustness . In addition , ViTs ’ adversarial transferability has also been studied : ( Mahmood et al. , 2021 ) shows that adversarial examples do not readily transfer between CNNs and transformers and ( Naseer et al. , 2021 ; Wei et al. , 2021 ) propose techniques to boost the adversarial transferability between ViTs and from ViTs to CNNs . In parallel , ( Mao et al. , 2021 ) refines ViTs ’ architecture design to improve robustness . In our work , we challenge the common belief that ViTs are more robust than CNNs , which is concluded based on evaluations using existing attack methods , and propose to customize adaptive attacks utilizing ViTs ’ captured patch-wise global interactions to make ViTs weaker learners . 3 EVALUATING ROBUSTNESS OF VITS AND CNNS UNDER EXISTING ATTACKS . Although various comparisons on robustness of ViTs and CNNs have been explored in pioneering works ( Bhojanapalli et al. , 2021 ; Aldahdooh et al. , 2021 ; Shao et al. , 2021 ) , their evaluation suffers from one of the following limitations : ( 1 ) only adopt weak attack methods , ( 2 ) only adopt early ViT designs without considering recently advanced ViT architectures , and ( 3 ) do not adopt the official and latest pretrained models and suffer from inferior clean accuracies . To this end , we extensively evaluate the robustness against common white-box attacks of several representative ViT variants , which cover the popular trends in designing ViT architectures , including ( 1 ) using local self-attention ( Swin ( Liu et al. , 2021a ) ) , which adopts the attention mechanism within a local region instead of the global ones in vanilla ViTs to capture low-level features and reduce the computational cost , and ( 2 ) introducing the inductive bias of CNNs to build hybrid models ( LeViT ( Graham et al. , 2021 ) ) . 3.1 EVALUATION SETUP . Models and datasets . We evaluate the robustness of three ViT families ( i.e. , DeiT ( Touvron et al. , 2021 ) , Swin ( Liu et al. , 2021a ) , and LeViT ( Graham et al. , 2021 ) ) and two CNN families Query Point Adv Img from Patch-Fool Layer 2 Original Img Adv Img from PGD-20 Layer 6 Layer 12 Figure 1 : Comparisons among the attention maps in the intermediate layers of DeiT-S generated by the clean inputs , the adversarial inputs under PGD-20 attacks ( 𝜖 = 0.003 ) , and the proposed Patch-Fool attack , respectively . In particular , we average the attention scores across all the attention heads in each layer and visualize the attention score of each token for a given query token ( the center patch in the red box in our show case ) , following ( Kim et al. , 2021 ) . We can observe that the difference in attention maps between clean and adversarial inputs generated by PGD-20 keeps small across different layers ; In contrast , the proposed Patch-Fool notably enlarges the gap between clean and adversarial attention maps , demonstrating a successful attack for ViTs . ( ResNet ( He et al. , 2016 ) and VGG ( Simonyan & Zisserman , 2014 ) ) on ImageNet using their official implementation and pretrained models . Note that we adopt DeiT models without distillation , which only improves the training schedule over vanilla ViTs , for a fair comparison . Attack settings . We adopt four adversarial attacks ( i.e. , PGD ( Madry et al. , 2017 ) , AutoAttack ( Croce & Hein , 2020 ) , CW-𝐿𝐼𝑛 𝑓 ( Carlini & Wagner , 2017 ) , and CW-𝐿2 ) with different perturbation strengths . In particular , for the CW-𝐿𝐼𝑛 𝑓 and CW-𝐿2 attacks , we adopt the implementation in AdverTorch ( Ding et al. , 2019 ) and the same settings as ( Chen et al. , 2021a ; Rony et al. , 2019 ) ; For AutoAttack , we adopt the official implementation and default settings in ( Croce & Hein , 2020 ) .

Summary: This paper propose a new attack on Vision Transformers (ViTs) called Patch-Fool. The attack proceeds by first picking a patch which contributes the most (in the self attention calculation) to other patches and then perturbed it adversarially wrt cross entropy + attention based loss. The results show that this kind of attack degrades performance significantly wrt prior work. The authors then perform various ablation studies to justify their architecture/loss choices.