On Improving Adversarial Transferability of Vision Transformers 

1 INTRODUCTION . Transformers compose a family of neural network architectures based on the self-attention mechanism , originally applied in natural language processing tasks achieving state-of-the-art performance ( Vaswani et al. , 2017 ; Devlin et al. , 2018 ; Brown et al. , 2020 ) . The transformer design has been subsequently adopted for vision tasks ( Dosovitskiy et al. , 2020 ) , giving rise to a number of successful vision transformer ( ViT ) models ( Touvron et al. , 2020 ; Yuan et al. , 2021 ; Khan et al. , 2021 ) . Due to the lack of explicit inductive biases in their design , ViTs are inherently different from convolutional neural networks ( CNNs ) that encode biases e.g. , spatial connectivity and translation equivariance . ViTs process an image as a sequence of patches which are refined through a series of self-attention mechanisms ( transformer blocks ) , allowing the network to learn relationships between any individual parts of the input image . Such processing allows wide receptive fields which can model global context as opposed to the limited receptive fields of CNNs . These significant differences between ViTs and CNNs give rise to a range of intriguing characteristics unique to ViTs ( Caron et al. , 2021 ; Tuli et al. , 2021 ; Mao et al. , 2021 ; Paul & Chen , 2021 ; Naseer et al. , 2021b ) . Adversarial attacks pose a major hindrance to the successful deployment of deep neural networks in real-world applications . Recent success of ViTs means that adversarial properties of ViT models also become an important research topic . A few recent works explore adversarial robustness of ViTs ( Shao et al. , 2021 ; Mahmood et al. , 2021 ; Bhojanapalli et al. , 2021 ) in different attack settings . Surprisingly , these works show that large ViT models exhibit lower transferability in black-box attack setting , despite their higher parameter capacity , stronger performance on clean images , and better generalization ( Shao et al. , 2021 ; Mahmood et al. , 2021 ) . This finding seems to indicate that as ViT performance improves , its adversarial feature space gets weaker . In this work , we investigate whether the weak transferability of adversarial patterns from high-performing ViT models , as reported in recent works ( Shao et al. , 2021 ; Mahmood et al. , 2021 ; Bhojanapalli et al. , 2021 ) , is a result of weak features or a weak attack . To this end , we introduce a highly transferable attack approach that augments the current adversarial attacks and increase their transferability from ViTs to the unknown models . Our proposed transferable attack leverages two key concepts , multiple discriminative pathways and token refinement , which exploit unique characteristics of ViT models . Our approach is motivated by the modular nature of ViTs ( Touvron et al. , 2020 ; Yuan et al. , 2021 ; Mao et al. , 2021 ) : they process a sequence of input image patches repeatedly using multiple multi-headed self-attention layers ( transformer blocks ) ( Vaswani et al. , 2017 ) . We refer to the representation of patches at each transformer block as patch tokens . An additional randomly initialized vector ( class token1 ) is also appended to the set of patch tokens along the network depth to distill discriminative information across patches . The collective set of tokens is passed through the multiple transformer blocks followed by passing of the class token through a linear classifier ( head ) which is used to make the final prediction . The class token interacts with the patch tokens within each block and is trained gradually across the blocks until it is finally utilized by the linear classifier head to obtain class-specific logit values . The class token can be viewed as extracting information useful for the final prediction from the set of patch tokens at each block . Given the role of the class token in ViT models , we observe that class tokens can be extracted from the output of each block and each such token can be used to obtain a class-specific logit output using the final classifier of a pretrained model . This leads us to the proposed self-ensemble of models within a single transformer ( Fig . 1 ) . We show that attacking such a self-ensemble ( Sec . 3 ) containing multiple discriminative pathways significantly improves adversarial transferability from ViT models , and in particular from the large ViTs . Going one step further , we study if the class information extracted from different intermediate ViT blocks ( of the self-ensemble ) can be enhanced to improve adversarial transferability . To this end , we introduce a novel token refinement module directed at enhancing these multiple discriminative pathways . The token refinement module strives to refine the information contained in the output of each transformer block ( within a single ViT model ) and aligns the class tokens produced by the intermediate blocks with the final classifier in order to maximize the discriminative power of intermediate blocks . Our token refinement exploits the structural information stored in the patch tokens and fuses it with the class token to maximize the discriminative performance of each block . Both the refined tokens and self-ensemble ideas are combined to design an adversarial attack that is shown to significantly boost the transferability of adversarial examples , thereby bringing out the true generalization of ViTs ’ adversarial space . Through our extensive experimentation , we empirically demonstrate favorable transfer rates across different model families ( convolutional and transformer ) as well as different vision tasks ( classification , detection and segmentation ) . 1Average of patch tokens can serve as a class token in our approach for ViT designs that do not use an explicit class token such as Swin transformer ( Liu et al. , 2021 ) or MLP-Mixer ( Tolstikhin et al. , 2021 ) 2 BACKGROUND AND RELATED WORK . Adversarial Attack Modeling : Adversarial attack methods can be broadly categorized into two categories , white-box attacks and black-box attacks . While the white-box attack setting provides the attacker full access to the parameters of the target model , the black-box setting prevents the attacker from accessing the target model and is therefore a harder setting to study adversarial transferability . White-box Attack : Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2014 ) and Projected Gradient Descent ( PGD ) ( Madry et al. , 2018 ) are two initially proposed white-box attack methods . FGSM corrupts the clean image sample by taking a single step within a small distance ( perturbation budget ) along the objective function ’ s gradient direction . PGD corrupts the clean sample for multiple steps with a smaller step size , projecting the generated adversarial example onto the -sphere around the clean sample after each step . Other state-of-the-art white-box attack methods include Jacobian-based saliency map attack ( Papernot et al. , 2016 ) , Sparse attack ( Modas et al. , 2019 ) , Onepixel attack ( Su et al. , 2019 ) , Carlini and Wagner optimization ( Carlini & Wagner , 2017 ) , Elastic-net ( Chen et al. , 2018 ) , Diversified sampling ( Tashiro et al. , 2020 ) , and more recently Auto-attack ( Croce & Hein , 2020b ) . We apply white-box attacks on surrogate models to find perturbations that are then transferred to black-box target models . Black-box Attack and Transferability : Black-box attacks generally involve attacking a source model to craft adversarial signals which are then applied on the target models . While gradient estimation methods that estimate the gradients of the target model using black-box optimization methods such as Finite Differences ( FD ) ( Chen et al. , 2017 ; Bhagoji et al. , 2018 ) or Natural Evolution Strategies ( NES ) ( Ilyas et al. , 2018 ; Jiang et al. , 2019 ) exist , these methods are dependent on multiple queries to the target model which is not practical in most real-world scenarios . In the case of adversarial signal generation using source models , it is possible to directly adopt white-box methods . In our work , we adopt FGSM and PGD in such a manner . Methods like ( Dong et al. , 2018 ) incorporate a momentum term into the gradient to boost the transferability of existing white-box attacks , building attacks named MIM . In similar spirit , different directions are explored in literature to boost transferability of adversarial examples ; a ) Enhanced Momentum : Lin et al . ( Lin et al. , 2019 ) and Wang et al . ( Wang & He , 2021 ) improve momentum by using Nesterov momentum and variance tuning respectively during attack iterations , b ) Augmentations : Xie et al . ( Xie et al. , 2019 ) showed that applying differentiable stochastic transformations can bring diversity to the gradients and improve transferability of the existing attacks , c ) Exploiting Features : Multiple suggestions are proposed in the literature to leverage the feature space for adversarial attack as well . For example , Zhou et al . ( Zhou et al. , 2018 ) incorporate the feature distortion loss during optimization . Similarly , ( Inkawhich et al. , 2020b ; a ; Huang et al. , 2019 ) also exploit intermediate layers to enhance transferability . However , combining the intermediate feature response with final classification loss is non-trivial as it might require optimization to find the best performing layers ( Inkawhich et al. , 2020b ; a ) , and d ) Generative Approach : Orthogonal to iterative attacks , generative methods ( Poursaeed et al. , 2018 ; Naseer et al. , 2019 ; 2021a ) train an autoencoder against the white-box model . In particular , Naseer et al . show that transferability of an adversarial generator can be increased with relativistic cross-entropy ( Naseer et al. , 2019 ) and augmentations ( Naseer et al. , 2021a ) . Ours is the first work to address limited transferability of ViT models . The Role of Network Architecture : Recent works exploit architectural characteristics of networks to improve the transferability of attacks . While Wu et al . ( 2020 ) exploit skip connections of models like ResNets and DenseNets to improve black-box attacks , Guo et al . ( 2020 ) build on similar ideas focused on the linearity of models . Our work similarly focuses on unique architectural characteristics of ViT models to generate more transferable adversarial perturbations with the existing white-box attacks . Robustness of ViTs : Adversarial attacks on ViT models are relatively unexplored . Shao et al . ( 2021 ) and Bhojanapalli et al . ( 2021 ) investigate adversarial attacks and robustness of ViT models studying various white-box and black-box attack techniques . The transferability of perturbations from ViT models is thoroughly explored in ( Mahmood et al. , 2021 ) and they conclude that ViT models do not transfer well to CNNs , whereas we propose a methodology to solve this shortcoming . Moreover , Mahmood et al . ( 2021 ) explores the idea of an ensemble of CNN and ViT models to improve the transferability of attacks . Our proposed ensemble approach explores a different direction by converting a single ViT model into a collection of models ( self-ensemble ) to improve attack Figure 2 : Adversarial examples for ViTs have only moderate transferability . In fact transferabililty ( % ) of MIM ( Dong et al. , 2018 ) perturbations to target models goes down as the source model size increases such as from DeiT-T ( Touvron et al. , 2020 ) ( 5M parameters ) to DeiT-B ( Touvron et al. , 2020 ) ( 86M parameters ) . However , the performance of the attack improves significantly when applied on our proposed ensemble of classifiers found within a ViT ( MIME & MIMRE ) . transferability . In essence , our proposed method can be integrated with existing attack approaches to take full advantage of the ViTs ’ learned features and generate transferable adversaries .

Summary: In this paper,  authors improve adversarial transferability of ViT by Self-Ensemble and Token Refinement method. 'Self-Ensemble' implies it treats each transformer block of ViT as the 'last block' and apply a shared classifier to it. So that the trivial discriminative path can be attacked. 'Token Refinement' tries to improve the classification ablity of shallow blocks by using extra non-shared module on each block's output tokens. This paper successfully improved the adversarial ability of ViT attacks. Sufficient experimental results and detailed description ensure its reproducibility.