On Improving Adversarial Transferability of Vision Transformers 
Authors: Muzammal Naseer
Kanchana Ranasinghe
Salman Khan
Fahad Khan
Fatih Porikli
Venue: ICLR 2022 Spotlight (2022)
Keywords: Vision Transformers
Adversarial Perturbations

Abstract: Vision transformers (ViTs) process input images as sequences of patches via self-attention; a radically different architecture than convolutional neural networks (CNNs).  This makes it interesting to study the adversarial feature space of ViT models and their transferability. In particular, we observe that adversarial patterns found via conventional adversarial attacks show very \emph{low} black-box transferability even for large ViT models. We show that this phenomenon is only due to the sub-optimal attack procedures that do not leverage the true representation potential of ViTs. A deep ViT is composed of multiple blocks, with a consistent architecture comprising of self-attention and feed-forward layers, where each block is capable of independently producing a class token. Formulating an attack using only the last class token (conventional approach) does not directly leverage the discriminative information stored in the earlier tokens, leading to poor adversarial transferability of ViTs.Using the compositional nature of ViT models, we enhance transferability of existing attacks by introducing two novel strategies specific to the architecture of ViT models.  \emph{(i) Self-Ensemble:} We propose a method to find multiple discriminative pathways by dissecting a single ViT model into an ensemble of networks. This allows explicitly utilizing class-specific information at each ViT block. \emph{(ii) Token Refinement:} We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT.Our token refinement systematically combines the class tokens with structural information preserved within the patch tokens. An adversarial attack when applied to such refined tokens within the ensemble of classifiers found in a single vision transformer has significantly higher transferability and thereby brings out the true generalization potential of the ViT's adversarial space. Code: https://t.ly/hBbW.

Published as a conference paper at ICLR 2022
ON IMPROVING ADVERSARIAL TRANSFERABILITY OF VISION TRANSFORMERS
Muzammal Naseeråt
Kanchana Ranasinghe Fatih Porikli
Salman Khan
Fahad Shahbaz Khan S3
i Fuustralian National University,
.Stony Brook University, "Qualcomm USA
*Linköping University
Mohamed bin Zayed University of AI,
muzammal wanaaeeeaanu edu.au au
ABSTRACT
Vision transformers (ViTs) process input images as sequences of patches via self- attention; radically different architecture than convolutional neural networks (CNNs). This makes interesting to study the adversarial feature space of ViT models and their transferability. In particular, we observe that adversarial patterns found via conventional adversarial attacks show very low black-box -bx transferability even for large ViT models. We show that this phenomenon is only due the sub-optimal attack procedures that do not leverage the true representation potential of ViTs. deep ViT is composed of multiple blocks, with a consistent architecture comprising of self-attention and feed-forward layers, where each block is capable of independently producing class token. Formulating an attack using only the last class token (conventional approach) does not directly leverage the discriminative information stored in the earlier tokens, leading to poor adversarial transferability of ViT's. Using the compositional nature of ViT models, we enhance transferability of existing attacks by introducing two novel strategies specific to the architecture of VIT models. (i) Self-Ensemble: We propose method to find multiple discrimi- native pathways by dissecting a single ViT model into an ensemble of networks. This allows explicitly utilizing class-speciff information at each ViT block. (ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT. Our token refinement systematically combines the class tokens with structural information preserved within the patch tokens. An adversarial attack when applied to such refined tokens within the en- semble of classifiers found in single vision transformer has significantly higher transferability and thereby brings out the true generalization potential of the ViT's adversarial space. Code: https: / http//tlll//////tt /hBbW.
INTRODUCTION
Transformers compose a family of neural network architectures based on the self-attention mecha- nism, originally applied in natural language processing tasks achieving state-of-the-at performance (Vaswani et al., 2017; Devlin et al., 2018; Brown et al., 2020). The transformer design has been subsequently adopted for vision tasks (Dosovitskiy al., 2020), giving rise a number of successful vision transformer (ViT) models (Touvron et al., 2020; Yuan et al., 2021; Khan et al., 2021). Due to the lack of explicit inductive biases in their design, ViTs are inherently different from convolutional neural networks (CNNs) that encode biases e.g., spatial connectivity and translation equivariance. ViTs process an image as a sequence patches which are refined through a series of self-attention mechanisms (transformer blocks), allowing the network to learn relationships between any individual parts of the input image. Such processing allows wide receptive fields which can model global context as opposed to the limited receptive fields of CNNs. These significant differences between VITs and CNNs give rise to a range of intriguing characteristics unique to ViTs (Caron et al., 2021; Tuli et al., 2021; Mao et al., 2021; Paul & Chen, 2021; Naseer al., 2021b).
Adversarial attacks pose a major hindrance to the successful deployment of deep neural networks in real-world applications. Recent success of VIT's means that adversarial properties of VIT models

---

Published as a conference paper at ICLR 2022
A VIT as Single Classifier
VIT as Self-Ensemble
Shared Classifier
Shared Local Norm
Classifior
Patch Tokens
Block
n Block
VIT
Class
Token
Local Norm
Figure I: Left: Conventional adversarial attacks view ViT as a single classifier and maximize the prediction loss (c.g., cross entropy) to fool the model based on the last classification token only. This lcads to sub-optimal results as class tokens in previous VIT blocks only indirectly influence adversarial perturbations. In contrast, our approach (right) effectively utilizes the underlying ViT architecture create self-ensemble using class tokens produced by all blocks within VIT to design the adversarial attack. Our self-ensemble enables to use hierarchical discriminative information learned by all class tokens. Consequently, an attack based on our self-ensemble generates transferable adversaries that generalize well across different model types and vision tasks.
also become an important research topic. A few recent works explore adversarial robustness of ViTs (Shao et al., 2021: Mahmood et al., 2021; Bhojanapalli et al., 2021) in different attack settings. Surprisingly, these works show that large ViT models exhibit lower transferability in black-box attack setting, despite their higher parameter capacity, stronger performance on clean images, and better generalization (Shao et al., 2021; Mahmood et al., 2021). This finding seems to indicate that as VIT performance improves, its adversarial feature space gets weaker. In this work, we investigate whether the weak transferability of adversarial patterns from high- segh-performing ViT models, as reported in recent works (Shao et al., 2021; Mahmood et al., 2021; Bhojanapalli et al., 2021), is result of weak features or weak attack. To this end, introduce a highly transferable attack approach that augments the current adversarial attacks and increase their transferability from ViTs to the unknown models. Our proposed transferable attack leverages two key concepts, multiple discriminative pathways and token refinement, which exploit unique characteristics of ViT models.
Our approach is motivated by the modular nature of ViTs (Touvron et al., 2020; Yuan al., 2021; Mao et al., 2021): they process a sequence of input image patches repeatedly using multiple multi-headed self-attention layers (transformer blocks) (Vaswani et al., 2017). We refer to the representation of patches at each transformer block as patch tokens. An additional randomly initialized vector (class token') is also appended to the set of patch tokens along the network depth to distill discriminative information across patches. The collective set of tokens is passed through the multiple transformer blocks followed by passing of the class token through a linear classifier (head) which is used to make the final prediction. The class token interacts with the patch tokens within each block and is trained gradually across the blocks until it is finally utilized by the linear classifier head to obtain class-specifes logit values. The class token final prediction from the set of patch tokens at each block. Given the role of the class token in VIT models, we observe that class tokens can be extracted from the output of each block and each such token can be used to obtain a class-specific logit output using the final classifier of a pretrained model. This leads us to the proposed self-ensemble of models within a single transformer (Fig. 1). We show that attacking such selfennsemlle (Sec. 3) containing multiple discriminative pathways significantly improves adversarial transferability from ViT models, and particular from the large ViT's.
be viewed extracting information useful for the
Going one step further, we study if the class information extracted from different intermediate VIT blocks (of the self-ensemble) can be enhanced to improve adversarial transferability. To this end, we introduce a novel token refinement module directed at enhancing these multiple discriminative pathways. The token refinement module strives to refine the information contained in the output of each transformer block (within single ViT model) and aligns the class tokens produced by the intermediate blocks with the final classifier in order to maximize the discriminative power of intermediate blocks. Our token refinement exploits the structural information stored in the patch tokens and fuses it with the class token to maximize the discriminative performance of each block. Both the refined tokens and self-ensemble ideas combined to design adversarial attack that is shown to significantly boost the transfcrability of adversarial examples, thereby bringing out the true
Averagee of patch tokens can serve as class token in our approach for VIT designs that do not use an explicit class token such as Swin transformer (Liu et al., 2021) Mi.er (Tolstikhin al., 2021)
2

---

Published as a conference paper at ICLR 2022
generalization of ViTs' adversarial space. Through our extensive experimentation, we empirically demonstrate favorable transfer rates across different model families (convolutional and transformer) as well as different vision tasks (classification, detection and segmentation).
2 BACKGROUND AND RELATED WORK
Adversarial Attack Modeling: Adversarial attack methods can be broadly categorized into two categories, white-box attacks and black-box attacks. While the white-box attack setting provides the attacker full access to the parameters of the target model, the black-box setting prevents the attacker from accessing the target model and is therefore a harder setting to study adversarial transferabilityy.
White-box Attack: Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014) and Projected Gradient Descent (PGD) (Madry et al., 2018) are two initially proposed white-box attack methods. FGSM corrupts the clean image sample by taking single step within small distance (perturbation budget& e) along the objective function's 's gradient direction. PGD corrupts the clean sample for multiple steps with a smaller step size, projecting the generated adversarial example onto the e-sphere around the clean sample after each step. Other state-of-the-an white-box attack methods include Jacobian mucobian-based saliency map attack (Papemot et al., 2016). Sparse attack (Modas et al., 2019), One- pixel attack (Su et al., 2019), Carlini and Wagner optimization (Carlini & Wagner, 2017), Elastic-net (Chen et al., 2018), Diversified sampling (Tashiro et al., 2020), and more recently Auto-attack (Croce & Hein, 2020b). We apply white-box attacks surrogate models to find perturbations that are then transferred to black-box target models.
Black-box Attack and Transferability: Black-box attacks generally involve attacking a source model to craft adversarial signals which are then applied on the target models. While gradient estimation methods that estimate the gradients of the target model using black-box optimization methods such as Finite Differences (FD) (Chen et al., 2017; Bhagoji et al., 2018) or Natural Evolution Strategies (NES) (Ilyas et al., 2018; Jiang et al., 2019) exist, these methods are dependent on multiple queries to the target model which is not practical in most real-world scenarios. In the case of adversarial signal generation using source models, it is possible directly adopt white-box methods. In our work, we adopt FGSM and PGD in such manner. Methods like (Dong et al., 2018) incorporate a momentum term into the gradient to boost the transferability of existing white-box attacks, building attacks named MIM. In similar spirit, different directions are explored in literature to boost transferability of adversarial examples; a) Enhanced Momentum: Lin et al. (Lin al., 2019) and Wang et al. (Wang& & He, 2021) improve momentum by using Nesterov momentum and variance tuning respectively during attack iterations, b) Augmentations: Xie el al. (Xie al., 2019) showed that applying diffcrentiable stochastic transformations can bring diversity to the gradients and improve transfcrability of the existing attacks, c) Exploiting Features: Multiple suggestions are proposed in the literature to leverage the feature space for adversarial attack as well. For example, Zhou et al. (Zhou et al., 2018) incorporate the feature distortion loss during optimization. Similarly, (Inkawhich et al., 2020b;a; Huang al., 2019) also exploit intermediate layers enhance transferability. However, combining the intermediate feature response with final classification loss is non-trivial as it might require optimization to find the best performing layers (Inkawhich et al., 2020b;a), and d) Generative Approach: Orthogonal to iterative attacks, generative methods (Poursaced et al., 2018; Naseer et al., 2019; 2021a) la) train an autoencoder against the white-box model. In particular, Naseer et al. show that transferability of an adversarial generator can be increased with relativistic cross-entropy (Naseer et al., 2019) and augmentations (Naseer et al., 2021a). Ours the first work to address limited transferability of ViT models.
The Role of Network Architecture: Recent works exploit architectural characteristics of networks to improve the transferability of attacks. While Wu et al. (2020) exploit skip connections of models like ResNets and DenseNets to improve black-box attacks, Guo et al. (2020) build on similar ideas focused on the linearity of models.
Our work similarly focuses on unique architectural characteristics of ViT models to generate more transferable adversarial perturbations with the existing white-box attacks.
Robustness of ViTs: Adversarial attacks on ViT models are relatively unexplored. Shao et al. (2021) and Bhojanapalli et al. (2021) investigate adversarial attacks and robustness of ViT models studying various white-box and black-box attack techniques. The transferability of perturbations from ViT models is thoroughly explored in (Mahmood et al., 2021) and they conclude that ViT
3

---

Published as a conference paper at ICLR 2022
Figure 2: Adversarial examples for ViTs have only moderate transferability. In fact transferabililty (%) MIM (Dong et al., 2018) perturbations to target mod- cls goes down as the souree model size increases such from DeiT-T (Touvron et al., 2020) (5M parameters) DeïT-B (Touvron et al., 2020) (86M parameters). However, the performance the attack improves sig- nificantly when applied on our proposed ensemble of classifiers found within VIT (MIM _ & MIM#*). )
DelT Reshwel 52
DeiT - T2T-24
70
MIM
70
MIM
Fooling Rate 199
05
65
MIMA
Mija
660
60
55-
Fooling
55
50
50
DaiT-T
DeiT-S
D
DeiT-T
DWiT-S
DeiT-B
models do not transfer well CNNs, whereas propose methodology to solve this shortcoming. Moreover, Mahmood et al. (2021) explores the idea of an ensemble of CNN and ViT models to improve the transferability of attacks. Our proposed ensemble approach explores a different direction by converting single ViT model into collection of models (self-ensemble) to improve attack transferability. In essence, our proposed method can be integrated with existing attack approaches to take full advantage of the ViTs' learned features and generate transferable adversaries.
3 ENHANCING ADVERSARIAL TRANSFERABILITY OF VITS
Given a clean input image sample with a label y. source VIT model F and target model M which is attek, the goal of an adversarial attack is generating an adversarial signal, x', using the information encoded within F, which can potentially change the target network's prediction (M(x')) ((ra)argmaa> + y). A set of boundary conditions are also imposed on the adversarial signal control the level of distortion relation to the original sample, i.e., | ='Ilp e, for a small perturbation budget e and p-norm, often set to infinity norm (loc).
Preliminaries:
Motivation: The recent findings (Shao al., 2021; Mahmood al., 2021) demonstrate low black-box transferability of ViTs despite their higher parametric complexity and better feature generalization. Motivated by this behaviour, we set-up simple experiment of our own to study the adversarial transferability of ViTs (see Fig. 2). We note that transferability of adversarial examples found via momentum iterative fast gradient sign method (Dong et al., 2018) (MIM) at fo S 16 on DeiT (Touvron et al., 2020) does not increase with model capacity. In fact, adversarial transferability from DeiT base model (DeiT-B) on ResNet152 and large vision transformer (ViT-L (Dosovitskiy et al., 2020)) is lower than DeiT tiny model (DeiT-T). This is besides the fact that DeiT-B has richer representations and around 17x more parameters than DeÎT-T. We investigate if this behavior is inherent to ViTs or merely due to sub-optimal attack mechanism. To this end, we exploit unique architectural characteristics of ViTs to first find an ensemble of networks within a single pretrained ViT model (self-ensemble, right Fig. 1). The class token produced by each self-attention block is processed by the final local norm and classification MLP-head to refine class-specific information (Fig. 2). In other words. our MIM and MIM R.F. variants attack class information stored in the class tokens produced by all the self-attention blocks within the model and optimize for the adversarial example (Sec. 3.1 and 3.2). Exploring the adversarial space of such multiple discriminative pathways in self-ensemble generates highly transferable adversarial examples, as we show next.
3.1 SELF-ENSEMBLE: DISCRIMINATIVE PATHWAYS OF VISION TRANSFORMER
A VIT model (Dosovitskiy et al., 2020; Touvron et al., 2020). F. with n transformer blocks can be defined F = (Ji о /2 fa o... Jn) o g, where fa represents a single ViT block comprising of multi-head self-attention and feed-forward layers and is the final classification head. To avoid notation clutter, we assume that g consists of the final local norm and MLP-head (Touvron al., 2020; Dosovitskiy et al., 2020). Self-attention layer within the vision transformer model takes a sequence of m image patches as input and outputs the processed patches. We will refer to the representations associated with the sequence of image patches as patch tokens, P. E Rm (where d is the dimensionality of each patch representation). Attention in ViT layers is driven by minimizing the empirical risk during training. In the case of classification, patch tokens are further appended with the class token (Q& E IR. xd). These patch and class tokens are refined across multiple blocks (fr) and attention in these layers is guided such that the most discriminative information from patch tokens is preserved within the class token. The final class token is then projected to the number of classes by the classifier, g. Due to the availability of class token each transformer block, create
4

---

Published as a conference paper at ICLR 2022
Figure 3: Distribution of discriminative infor- mation across blocks of DeiT models. Note how multiple intermediate blocks contain fea- tures with considerable discriminative infor- mation as measured by top-1 accuracy on the ImageNet val. set. These are standard mod- cls pretrained ImageNet with no further training. Each block (x-axis) corresponds a classifier Fx as defined in Equation
DeiT-B
Accuracy Top-1 (%6
DeiT-S
60
DeiT-T
40
10
11
Blocks
ensemble of classifiers by learning shared classification head at each block along the ViT hierarchy. This provides us an ensemble n classifiers from a single VIT, termed as the self-ensemble:
A II
Fk =
f: og,
where & = 1,2....,7..
(1)
We note that the multiple classifiers thus formed hold significant discriminative information. This is validated by studying the classification performance of each classifier (Eq. 1) in terms of top-1 (%) accuracy ImageNet validation set, demonstrated in Fig. 3. Note that multiple intermediate layers perform well on the task, especially towards the end of the ViT processing hierarchy.
For an input image with label y, an adversarial attack can now be optimized for the ViT's self- ensemble by maximizing the loss at each VIT block. However, we observe that initial blocks (1-6) for all considered DeiT models do not contain useful discriminative information as their classification accuracy is almost zero (Fig. 3). During the training of VIT models (Touvron et al., 2020; Yuan et al., 2021; Mao et al., 2021), parameters are updated based on the last class token only, which means that the intermediate tokens are not directly aligned with the final classification head, g in our self-ensemble approach (Fig. 3) leading to a moderate classification performance. To resolve this, we introduce token refinement strategy to align the class tokens with the final classifier, g. and boost their discriminative ability, which in turn helps improve attack transfcrability.
3.2 TOKEN REFINEMENT
As mentioned above, the multiple discriminative pathways within ViT give rise to an ensemble of classifiers (Eq. 1). However, the class token produced by each attention layer is being processed by the final classifier, g. This puts an upper bound on classification accuracy for each token which is lower than or equal to the accuracy of the final class token. Our objective is to push the accuracy of the class tokens intermediate blocks towards the upper bound as defined by the last token. For this purpose, we introduced a token refinement module to fine-tune the class tokens.
Our proposed token refinement module illustrated Fig. inserted between the outputs of each block (after the shared norm layer) and the shared classifier head. Revisiting our bascline ensemble method (Fig. 1), we note that the shared classifier head contains weights directly trained only on the outputs of the last transformer block. While the class tokens of previous layers may be indirectly optimized to align with the final classifier, there exists a potential for misalignment of these features with the classifier: the pretrained classifier (containing weights compatible with the last layer class token) may not extract all the useful information from the previous layers. Our proposed module aims to solve this misalignment by refining the class tokens in a way such that the shared (pretrained) classifier head is able to extract all discriminative information
acts as an intermediate layer
Figure 4: Recent VITs process 196 image patches, leading to 196 patch tokens. We rearranged these to create 14x14 feature grid which is processed by convolutional block extract structural information, fol- lowed by average pooling to create a single patch token. Class token is refined via a MLP layer before feeding to the classifier. Both tokens subsequently merged.
Munet
Shaned Lonel
Y" Elecc
Fist
Tecel
Suttermect Motlle
Chatter
W7 or Sef-erobbe with feyfard Tolonz
Token Reyfoement Medsil
5

---

Published as a conference paper at ICLR 2022
80
Original Class laken
UMS Original Class Token
Refined Class
Accuracy Top-1
00-
Refined Class Taken
Accurscy Top
-15
40
40
20
43
10
2
7
11
12
9
10
12
Block Number (DelT-B)
Block Numbbe (D=T-TT)
Original Class Taken
Figure 5: Self-Ensemble for DeiT (Touvron et al., 2020): We measure the top-1 accuracy on ImageNet using the class-token of each block and compare to our refined tokens. These results show that fine-tuning helps align tokens from intermediate blocks with the fi- nal classifier enhancing their classification performance. Thus token refinement leads to strengthened discrimina- tive pathways allowing more transferable adversarics.
1000
Refined Class Token
Accurary Tap-1
60.
20-
10.
9
10 11 12
Block Number (DeiT-S)
contained within the class tokens of each block. Moreover, intermediate patch tokens may contain additional information that is not at all utilized by the class tokens of those blocks, which would also extract both patch tokens and the class token
be addressed by our proposed block. Therefore, from each block and process them for refinement, as explained next.
Patch Token Refinement: One of the inputs the token refinement module is the set of patch tokens output from each block. We first rearrange these patch tokens to regain their spatial relationships. The aim of this component within the refinement module is extract information relevant to spatial structure contained within the intermediate patch tokens. We believe that significant discriminative information is contained within these patches. The obtained rearranged patch tokens are passed through a convolution block (standard ResNet block containing a skip connection) to obtain spatially aware feature map. which is then average pooled to obtain single feature vector (of same dimension as the class token). This feature vector is expected to extract all spatial information from patch tokens.
-Class Token Refinement: By refining the class tokens of each block, we aim to remove any misalignment between the existing class tokens and the shared (pretrained) classifier head. Also, given how the class token does not contain a spatial structure, we simply use a linear layer to refine it. We hypothesize that refined class token at each block would be much more aligned with the shared classifier head allowing it to extract all discriminative information contained within those tokens.
We obtain the refined class token and the patch feature vector (refined output of patch tokens) and sum them together to obtain a merged token. While we tested multiple approaches for merging, simply summing them proved sufficient.
Merging Patch and Class Token:
Training: Given a VIT model containing & transformer blocks, we plugin instances of our token refinement module to the output of each block illustrated in Figure 4. We obtain the pretrained model, freeze all existing weights, and train only the k token refinement modules for only a single epoch on ImageNet training set. We used SGD optimizer with learning rate set to 0.001. Training finishes in less than one day on a single GPU-V100 even for large ViT model such as DeïT-B.
As expected, the trained token refinement module leads to increased discriminability of the class tokens, which we illustrate in Figure 5. Note how this leads to significant boosting of discriminative power especially in the earlier blocks, solving the misalignment problem. We build on this enhanced discriminability of the ensemble members towards better transferability, as explained next.
3.3 ADVERSARIAL TRANSFER
Our modifications to ViT models with respect to multiple discriminative pathways and token refine- ment are exploited in relation to adversarial transfer. We consider black-box attack perturbations that are generated using a source (surrogate) ViT model. The source model is only pretrained on ImageNet, modified according to our proposed approach and is subsequently fine-tuned to update only the token refinement module for a single epoch. We experiment with multiple white-box attacks, generating the adversarial examples using a joint loss over the outputs of each block. The transferability of adversarial examples is tested on a range of CNN and ViT models. Given input sample x and its label y, the adversarial object for our self-ensemble (Eq. 1) for the untargeted attack is defined as,

---

Published as a conference paper at ICLR 2022
Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014)
Convolational
Transforers
Source (L)
Attack
B1T50
Res152
WRN
DN201
VTT-L
T2T-24
ThT
VIT-
T2T-7
LGG19
PGSM
23.34
28.56
33.92
33.22
113.18
10.78
12.96
25.08
29.90
MNAS
FGSM
233.16
39.82
40.10
44.34
16.60
22.56
25.82
34.10
48.96
FGSM
29.74
37.10
38.86
42,40
44.38
35.42
50.58
73.32
57.62
Deit-T
FGSM 6
30.34
39.60
41.42
45.58
48.34
35.08
51,00
80.74
62.82
FGSM
30.18
39.82
41.26
46.06 (37)
46.76
32.68.27)
48.00,
80.10 (65)
63,90
FGSM
25.44
31.04
33,58
36.28
36.40
33,41
41,00
58,78
43,48
Deit-S
FGSM 5
30.82
38.38
41.06
46.00
47.20
39.00
51.44
78.90
56.70
FGSM RB
94484
12.8)
46,26
51.88
47.92
39.86 (6.5)
(5.7
82.00
66.20, (2.7)
FGSM
22,54
31.58
33.86
34.96
30.50
27.84
33.08
50.24
40,50
Deit-B
FGSM 5
31.12
41.46
43.02
47.12
42.28
35.40
46.22
73.04
$7.32
FGSM R.
45 48, 11.)) 42
11.68..112
36.60 wa.
49.60).1..5
74.40 65.92
Projected Gradient Decent (PGD) (Madry et al., 2018)
VGG19
PPG
19.80
28.56
33.92
33.22
S5.94
10.78
12.96
13.08
29.90
MNAS
PPD
19.44
36.28
36.22
40.20
88.04
18.04
21.16
19.60
41.70
PGD
14.22
23.98
24.16
26.76
335.70
21.54
44.24
86.86
53,74
Deit-T
PGD
14.42
24.58
25,46
28.38
39.84
21.86
45,08
88.44
53,80
PG
22.46,1824)
(4.64(110))
37.62
40,56
58.60
26.58
55.52
96.34, (0.5)
66.688
PGD
18.78
24.96
26,38
30,38
37,84
33,46
60,62
84,38
47,14
D
P
18.98
27.72
29.54
32,90
44.30
35.40
64,76
89.82
52,76
PGDRE RB
28.96, (02)
38.92
(440)
(2.84
46.82
60.86
40,30
76.10 (153)
97,32,
(1.54
PGD
18.68
25.56
27.90
30.24
34.08
31.98
52,76
69.82
39.80
Deit-B
8
23.64
32.84
35.40
38.66
43.56
37.82
64.20
82,32
51.68
PGD A.
19.92..190)
49.
10, 53.38
56.96.
56.90..m..)
45.70..13.)
79.56
94. 10
74.78-15.
Table 1: Fool rate (%) on 5k ImageNet val. adversarial samples at S 16. Perturbations generated from our proposed self-ensemble with refined tokens from vision transformer have significantly higher success rate.
[F ((lla))aaaax # yl. s.t. || x' c, k E (1,2.....
where [.] is indicator function. In the case of target attack, the attacker optimizes the above objective towards a specific target class instead of an arbitrary misclassification.
EXPERIMENTS
We conduct thorough experimentation on a range of standard attack methods to establish the per- formance boosts obtained through our proposed transferability approach. We create for adversarial attacks with < 16 and observe their transferability by using the following protocols:
Source (white-box) models: We mainly study three vision transformers from DeiT (Touvron et al., 2020) family due to their data efficiency. Specifically, the source models are Deit-T, DeiT-S, and DeiT-B (with 5, 22, and 86 million parameters, respectively). They trained without CNN distillation. Adversarial examples are created on these models using an existing white-box attack (c.g.. FGSM (Goodfellow al., 2014), PGD (Madry et al., 2018) and MIM (Dong et al., 2018)) and then transferred to the black-box target models.
Target (black-box) models: We test the black-box transferability across several vision tasks in- cluding classification, detection and segmentation. We consider convolutional networks including BiT-ResNNet50 (BiT50) (Beyer al., 2021), ResNet152 (Res152) (He al., 2016), Rede-ResNet-50-2 (WRN) (Zagoruyko & Komodakis, 2016), Nense (DN201) (Huang et al., 2017) and other ViT models including Token-to- TokeTT transformer (T2T) (Yuan al., 2021), Transformer in Transformer (TnT) (Mao et al., 2021), DINO (Caron et al., 2021), and Detection Transformer (DETR) (Carion et al., 2020) as the black-box target models.
We use ImageNet training set to fine tune our proposed token refinement modules. For evaluating robustness, we selected 5k samples from ImageNet validation set such that random samples from each class that are correctly classified by ResNet50 and ViT small (ViT-S) (Dosovitskiy et al., 2020) are present. In addition, we conduct experiments on COCO (Lin et al., 2014) (Sk images) and MASCAL-VOCII (Everingham al., 2012) (around 1.2k images) validation set.
Datasets:
7

---

Published as a conference paper at ICLR 2022
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong et al., 2018)
Convolutional
Transformers
Source (1)
Attack
BrT50
Res152
WRN
DN201
VIT-L
T2T-24
TnT
VIT-S
T2I-7
vGG19...
MIM
36.18
46.98
54.04
57.32
112.80
21.84
25.72
28.44
47.74
MNAS
MIM
34.78
54.34
55.40
64.06
118.8
34.54
38.70
40.58
60.02
MIM
36.22
45.56
47.86
53.26
63.84
48.44
72.52
96.44
77.66
Deit-T
MIM E
34.92
45.58
47.98
54.50
67.16
46.38
71.02
97.74
78.02
MIM
(2.04(58)
(4.02
58.48.
63.00
79.12, (533)
49.86 1.4
77.80
99.14(12)
85,50,
MIM
38.32
45,06
47,90
52.66
63.38
58.86
79.56
94.22
68.00
Deit-S
MIM E
40.66
49.52
52,98
58.40
71.78
61.06
84.42
98.12
74.58
MIM RE
13.70
61.72.
65.10
(1.74
84.30, (209) 66.32, 1.5)
(1252
(9.42.122)
39.08.121))
MIM
36.98
44,66
47.98
$2.14
57.48
54,40
70.84
84.74
$9.34
Deit-B
MIM E
45,30
54,30
58.34
63.32
70.42
61.84
82.80
94.46
73.66
MIM RM
70. 15
74.08
81 31..28
69.6
92. 221.
94, v0.
89.72..30.n
MIM with Input Diversity (DIM) (Xie et al., 2019)
VGG19
DIM
46.90
62.08
68.30
73.48
16..86
30.16
34.70
35.42
58.62
MNAS
DIM
43.74
62.08
68.30
73.48
|25.06
42.92
47.24
52.74
71.98
DIM
57.56
68.30
70,06
77.18
62.00
70.16
82.68
89.16
86.18
Deit-T
DIM
60.14
70.06
69.84
78.00
66.38
72.30
85.98
93.72
90.78
DIM
62.10 (45)
70.78
70.78
18.40
67.58
68.56,11
84.18
93.36
91.52,
DIM
59.00
62.12
63,42
67,30
62.62
73.84
79.50
82.32
74,20
S
DIM
68.82
74.44
75.34
80.14
76.22
84,10
91.92
94.92
88.42
RI
14.12))
81.30, (9.15)
(2.64
86.98; (968)
78.88, (6.3)
85,26, 11.4)
(3.22
(6.56
93.60.1944
DIM
56.24
59.14
60.64
64.44
61.38
69.54
73.96
76.32
64.44
Deit-B
DIM
73.04
78.36
80.28
83.70
79.06
85.10
91.84
94,38
86.96
DIM
80. 13.55)
84.92
36.36
(9.24
1779
-4.
92.28,
95.26,.15.) 93.42
Table 2: Fool rate (%) on 5k ImageNet val. adversarial samples at S 16. Perturbations generated from our proposed self-ensemble with refined tokens from vision transformer have significantly higher success rate.
Evaluation Metries:
We report fooling rate (percentage of samples for which the predicted label is flipped after adding adversarial perturbations) to evaluate classification. In the case of object detection, we report the decrease in mean average precision (mAP) and for automatic segmentation, we use the popular Jaccard Index. Given the pixel masks for the prediction and the ground-truth, it calculates the ratio between the pixels belonging to intersection and the union of both masks.
Baseline Attacks: We show consistent improvements for single step fast gradient sign method (FGSM) (Goodfellow et al., 2014) as well as iterative attacks including PGD (Madry et al., 2018), MIM (Dong al., 2018) and input diversity (transformation to the inputs) (DIM) (Xie et al., 2019) attacks. Iterative attacks ran for 10 iterations and we set transformation probability for DIM to default 0.7 (Xie al., 2019). Our approach is not limited to specific attack settings, but existing attacks can simply be adopted to our self-ensemble ViTs with refined tokens. Refer to appendices A-J for extensive analysis with more ViT designs, attacks, datasets (CIFAR10 & Flowers), computational cost comparison, and latent space visualization of our refined token.
4.1 CLASSIFICATION
In this section, discuss the experimental results adversarial transferability across black-box classification models. For a given attack method 'Attack', , we refer 'Attack and 'Attack as self-ensemble and self-ensemble with refined tokens, respectively, which are the two variants of our approach. We observe that adversarial transferability from ViT models to CNNs is only moderate for conventional attacks (Tables 1 & 2). For example, perturbations found via iterative attacks from DeiT-B to Res152 has even lower transfer than VGG19 However, the same attacks when applied using our proposed ensemble strategy (Eq. 1) with refined tokens consistently showed improved transferability to other convolutional as well as transformer based models. We observe that models without inductive biases that share architecture similarities show higher transfer rate of adversarial perturbations among them (e.g., from DeiT to ViT (Dosovitskiy et al., 2020)). We further observe that models trained with the same mechanism but lower parameters are more vulnerable to black-box attacks. For example, ViT-S and T2T-T more vulnerable than their larger counterparts, ViT-L and T2T-24. Also models trained with better strategies that lead to higher generalizability are less vulnerable to black-box attacks e.g., BiT50 is more robust than 155 (Tables and 2).
8

---

Published as a conference paper at ICLR 2022
Figure Ablative Study: Fooling rate of intermediate lay- ers under MIM (white-box) at- tack using our self-ensemble ap- proach. We obtain favorable im- provements for our method.
Fool Race (N)
Fare
HIM
MBH HIH
MI
HIV
8
HNY
Blocks (Dell:T)
Blocks
Blocks (DelTa)
Source (-+)
DeiT-T
DeiT-S
DefT-B
Table Cross-Task Transfcrability (classift- cation-vdetection) Object Detector DETR (Car- ion et al., 2020) is fooled. mAP at [0.5:0.95] IOU on COCO val. set. Our sese-ln approach with refined token (RF) significantly improves cross-task transferability. (lower the better)
No Actack
MTM
MIRE
M.M
M MTM
MTM
MTMRE
24.0
19.7
23.7
19.0
22.9
16.9
38.5
DTM
Digee
B.M
D Mit
DTM
DTM
20.5
13.7
20.3
12.0
19.9
11.1
Source (-+)
DeiT-T
DeiT-S
DeiT-B
Table 4: Cross- Task Transferability (classi- peaftion->segmettatiinn DINO (Caron et al., 2021) is fooled. Jaccard index metric is used evaluate segmentation performance. Best ad- versarial transfer results are achieved using our method. (lower the better)
No Attack
MIM
MINRE
NMM
MI
MIM
MIN
32,5
31.6
32.5
31.0
32.6
30.6
42.7
DIK 31,9
RIN
DI
DIM
DINRE
31.4
31.7
31,3
32,0
31.0
Clean Image
Adv Image
Clean Image
Adv Image
Clean Image
Adv Image
Figure 7: Visualization of DETR failure cases for our proposed DIM attack gener- ated from DeiT-S source model. (best viewed zoom)
The strength of our method is also evident by blockwise fooling rate in white-box setting (Fig. 6). It is noteworthy how MIM fails to fool the initial blocks of ViT, while our approach allows the attack to be effective in the intermediate blocks for the last class token. This ultimately allows us fully exploit ViT's adversarial space leading to high transfer rates for adversarial perturbations.
4.2 CROSS-TASK TRANSFERABILIT
Self-attention is the core component of transformer architecture regardless of the task; classification (Dosovitskiy et al., 2020; Touvron et al., 2020; Yuan et al., 2021; Mao et al., 2021), object detection (Carion et al., 2020), or unsupervised segmentation (Caron et al., 2021). We explore the effectiveness of our proposed method two additional tasks: object detection (DETR) (Carion et al., 2020) and segmentation (DINO) (Caron et al., 2021). We select these methods considering the use of transformer modules employing the self-attention mechanism within their architectures. While the task of object detection contains multiple labels per image and involves bounding box regression, the unsupervised model DINO trained in a self-supervised manner with no traditional image-level labels. Moreover, DINO uses attention maps of ViT model to generate pixel-level segmentations, which means adversaries must disrupt the entire attention mechanism to degrade its performance.
We generate adversarial signals on source models with classification objective using their initial predictions as the label. In evaluating attacks on detection and segmentation tasks their optimal setting, the source ViT need to process images of different sizes (e.g.. over 896 x96x88 886 pix for DETR). To cater for this, we process images in parts (refer appendix G) which allows generation of stronger adversaries. The performance degradation of DETR and DINO on generated adversaries are sum- marised Tables & 4. For DETR, we obtain clear improvements. the more robust DINO model, our transferability increases well with the source model capacity compared to the baseline.
CONCLUSION
We identify key shortcoming in the current state-of-the-art approaches for adversarial transferability of vision transformers (ViTs) and show the potential for much strong attack mechanisms that would exploit the architectural characteristics of ViTs. Our proposed novel approach involving multiple discriminative pathways and token refinement is able to fill in these gaps, achieving significant performance boosts when applied over range of state-of-the-art attack methods.

---

Published as a conference paper at ICLR 2022
Reproducibility Statement: Our method simply augments the existing attack approaches and used open source implementations. We highlight the steps to reproduce all of the results presented in paper, a) Attacks: We used open source implementation of patchwise attack (Gao et al., 2020) and Auto-Atrack (Croce & Hein, 2020b) (refer Appendix B) with default setting. Wherever necessary, we clearly mention attack parameters, e.g., iterations for PGD (Madry et al., 2018), MIM (Dong et al., 2018) and DIM (Xie et al., 2019) in section 4 (Experiments: Baseline Attack). Similarly, transformation probability for DIM is set the default value provided by the corresponding authors that is 0.7, b) Refined Tokens: We fine tuned class tokens for the pretrained source models (used to create perturbations) using open source code base (https: //githubs com/pytoooooooooooooo //pp/yyorccccc details for fine tuning in section 3.2. Further, we will publicly release all the models with refined tokens, c) Cross-Task Attack implementation: We provided details in section 4.2 and pscudo code in appendix G for cross-task transferability (from classification to segmentation and detection), and d) Dataset: We describe the procedure of selecting subset (5k) samples from ImageNet val. set in section 4. We will also release indices of these samples to reproduce the results.
wtw.aampplssteeeemmeeeee /tt///aaeer//////e//// /masterrmmageeettt We provided training
Since our work focuses on improving adversarial attacks models, in the short run our work can assist various parties with malicious intents of disrupting real-world deployed deep learning systems dependent ViTs. However, irrespective of our work, the possibility of such threats emerging exists. We believe that in the long run, works such as ours will support further research on building more robust deep learning models that can withstand the kind of attacks we propose, thus negating the short term risks. Furthermore, a majority of the models used are pre-trained on ImageNett (ILSVRC' 12). We also conduct our evaluations using this dataset. The version of ImageNet used contains multiple biases that portray unreasonable social stereotypes. The data contained is mostly limited to the Western world, and encodes multiple gender / ethnicity stereotypes Yang et al. (2020) while also posing privacy risks due to unblurred human faces. In future, we hope to use the more recent version of ImageNet Yang et al. (2021) which could address some of these issues.
Ethics Statement:
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query- efficient black box adversarial attack via random search. In European Conference on Computer Vision, pp. 484 501. Springer, 2020. 14, 15
Lucas Beyer, Xiaohua Zhai, Amélic Royer, Larisa Markeeva, Rohan Anil, and Alexander Kolesnikov. Knowledge distillation: A good teacher is patient and consistent. arXiv preprint 1XXx:22066002237, 2021. 7
Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. Practical black-hox attacks on deep neural networks using efficient query mechanisms. In ECCV, 2018.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification. ArXiv, abs/2103. 14b/2600..4486, 2021. 2,3
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 1416 2020. 1
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. European Conference on Computer Vision, pp. 213-229. Springer, 2020. 7, 9
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P, 2017.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint WXXv:2204.14294 2021. 1, 7,
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsich. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In AlSec, 2017. 3
Pin- Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsich. Ead: clastic-nct attacks to deep neural networks via adversarial examples. In AAAI, 2018.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference on Machine Learning, pp. 2196-2205. PMLR, 2020a. 14, 15
10

---

Published as a conference paper at ICLR 2022
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter- attacks. ICML, 2020b. 3, 10, 14, 15, 16
Jacob Devlin, Ming-Wes Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:/810.04805, 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018. 3, 4, 7, 8, 10, 14, 17, 18, 19, 23
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov. Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 110000011922 11929, 2020. 1,4, 7, 8.
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results, 2012. 7
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Hengtao Shen. Patch-wise attack for fooling deep neural network. In European Conference on Computer Vision, 2020. 10, 14,
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In ICLR, 2014. 3, 8, 23
Yiwen Guo, Qizhang Li, and Hao Chen. Backpropagating linearly improves transferability of adversarial examples. In Advances in Neural Information Processing Systems, 2020. 3
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016. 7, 20, 24
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinherger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017. 7
Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim, Enhancing adversarial example transferability with an intermediate level attack. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4733-4743 2019. 3
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In ICML, 2018. 3
Nathan Inkawhich, Kevin Liang, Lawrence Carin, and Yiran Chen. Transferable perturbations of deep feature distributions. In International Conference Learning Representations, 2020a. URL https: //openreview.ccr net http://eeee//t/eeeeeeeeeeee wttpo/oomm.o..a00//oa
Nathan Inkawhich, Kevin J Liang, Binghui Wang, Matthew Inkawhich, Lawrence Carin, and Yiran Chen. Perturbing across the feature hicrarchy to improve standard and strict blackbox attack transfcrability. arXiv preprint arXiv:2004. 14861, 2020b. 3
Linxi Jiang, Xingjun Ma. Shaoxiang Chen, James Bailey, and Yu-Gang Jiang. Black-hox adversarial attacks on video recognition models. In ACM MM, 2019. 3
Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. arXiv preprini 1rXiv:2101.0116, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 14
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv:1908.068 06281, 2019.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft coco: Common objects context. European conference on computer vision, pp. 740-755. Springer, 2014.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arxiv: 14030, 2021. 2, 14, 16
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In ICLR, 2018. 3, 7, 8, 10, 14, 16, 17, 23
11

---

Published as a conference paper at ICLR 2022
Kaleel Mahmood, Rigel Mahmood, and Marten van Dijk. On the robustness of vision transformers adversarial examples. ArXiv, abs/2104,02610, 2021. 2, 3,4
Yuxin Mao. Jing Zhang, Zhexiong Wan, Yuchao Dai, Aixuan Li, Yunqiu Lv, Xinyu Tian, Deng-Ping Fan, and Nick Barnes. Transformer transforms salient object detection and camouflaged object detection, arXiv preprint arXiv:2104.00127, 2021. 7,2.5.799
Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Sparsefool: a few pixels make big
difference. In CVPR, 2019. 3
Muzammal Nascer, Salman Khan, Harris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 2019. 3
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. On generating transferable targeted perturbations. arXiv preprint arXiv:2 144,44 2021a. 3, 14
Muzammal Nascer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Intriguing properties of vision transformers, 2021b. I
Maria- Breia-Elenaaaa Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 722-729. IEEE, 2008. 14
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In & 2016.
Sayak Paul and Pin- Yu Chen. Vision transformers are robust learners, 2021.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Seikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2888 2011. 21
Omid Poursaced, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4422-4431, 2018.
Rulin Shao, Zhouxing Shi, Jinfeng Yi. Pin- Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers, ArXiv, abs/2103. 15670, 2021. 2, 3,
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks In IEEE Transactions on Evolutionary Computation. IEEE, 2019.
Yusuke Tashiro, Yang Song. and Stefano Ermon. Output diversified initialization for adversarial attacks. arXiv preprint arXiv:2003.0m 06878, 2020.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakoh Uszkoreit, Mario Lucie, et al. Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv: 10X.v:2005.0160) 2021. 2, 14, 20, 22
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12877, 2020. 1, 2, 4, 5, 6, 7, 9, 14, 15
Shikhar Tuli, Ishita Dasgupta, Erin Grant, and Thomas L., Griffiths. Are convolutional neural networks or transformers more like human vision?, 2021.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008. 21
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, arXiv preprint 1rXiv:1706.03760 2017. 1,2
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. arXiv preprint 1xXi:2103.155777 2021.
Dongxian Wu. Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR, 2020.
Haiping Wa, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions vision transformers. arXiv preprint arXiv:2 18808, 2021. 14, 20, 22
12

---

Published as a conference paper at ICLR 2022
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity, In Proceedings of the IEEE/CVF Conference on Computer Vision and Patten Recognition, pp. 2730-277 2739, 2019. 3, 8, 10, 14, 17, 18, 19, 23
Kaiyu Yang, Klint Qinami, Fei-Fei, Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In faaect, pp. 547-558, 2020.
Kaiyu Yang, Jacqueline Yau, Fei-Fei, Deng. and Olga Russakovsky. A study face obfuscation in imagenet. arXiv preprint arXiv:2103.0619) 2021. 10
Yuan, Yunpeng Chen, Tao Wang. Weihao Yu, Yujun Shi, Francis Tay, Jiashi Feng, Shuicheng Yan. Tokens-to-token vit: Training vision transformers from scratch on imagenet. arXiv preprint arXiv:2101. 2021. 1, 2, 7, 15
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:6005.07146, 2016. 7
Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, and Yong Yang. Transferable adversarial perturbations. In Proceedings of the European Conference on Computer Vision (ECCV), pp- 452 467, 2018.
13

---

Published as a conference paper at ICLR 2022
Appendix
Adversarial perturbations found with ensemble of models are shown to be more transferable (Dong et al., 2018; Naseer et al., 2021a). In appendix A. we demonstrate the effectiivnness of our self- ensemble to boost adversarial transferability within ensemble of different models. Our approach augments and enhances the transferability of the existing attack. We further demonstrate this with recent attacks, Patchwise (Gao et al., 2020) and Auto-Attack (Croce & Hein, 2020b) in appendix B. Auto-Attack a strong white-box attack which is a combination of PGD with novel losses and other attacks such as boundary- based, FAB (Croce & Hein, 2020a) and, query based Square attack (Andriushchenko et al., 2020). This highlights that our method can be used as plug-and suug-and-lay with existing attack methods. We then evaluate adversarial transferability of Auto-Attack and PGD (100 iterations) at 4, 8, and 16 in appendix C. Our self-ensemble with refined tokens approach consistently performs better with these attack settings as well. In appendix D, we extended our approach to other dataset including CIFARIO (Krizhevsky et al., 2009) and Flowers (Nilsback & Zisserman, 2008) datascts. We highlight vulnerability of Swin transformer (Liu et al., 2021) against our approach in appendix E. We showed the results on full ImageNet validation set (50k samples) in appendix F. This demonstrates the effectiveness of our method regardless of dataset task. We discuss the generation of adversarial samples for images of sizes that are different to the source ViT models' input size (e-g., greater than 224) in appendix G. We provide computational cost analysis in Appendix H and visualize the latent space of refined tokens in Appendix H. Finally, we extended our approach to diverse ViT designs (Wu et al., 2021: Tolstikhin et al., 2021) and CNN in Appendices I and J.
A SELF-ENSEMBLE WITHIN ENSEMBLE
We created an ensemble of pre-trained Deit models (Touvron et al., 2020) including Deit-T, Deit- S and DeiT-B. These models have 12 blocks (layers) and differ in patch embedding size. These models are trained in similar fashion without distillation from CNN. As expected, adversarial transferability improves with an ensemble of models (Table 5). We also note that unlike such multiple-model ensemble approaches, our self-ensemble attack achieves performance improvements with minimal increase in computational complexity of the attack that from a single ViT. Our self-ensemble extended three classifiers ensemble an ensemble of 36 classifiers. Such multi-mode ensemble combined with our proposed self-ensemble with refined tokens approach leads to clear attack improvements in terms of transferability (refer Table 5).
Projected Gradient Decent (PGD) (Madry al., 2018)
Convolutional
Transformers
Source (4)
Attack
Res152
WRN
DN201
T2T-24
T2T-7
TnT
ViT-S
PGD Deit-[T+S+B] PGDE E
46.1 1
48.5
51.0
62.4
64.0
81.5
85.7
49.4
50.7
56.7
62.8
68.9
83.4
87.4
RG
64. (
69.0(220.5)
75. (0240
(3.7
87.0
93.5
95.3
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong al., 2018)
MIM
62.6
66.6
70.2
77.2
76.2
86.4
87.4
Deit-[T+S+B] MIM
67.3
68.0
73.5
76.2
78.7
87.9
87.9
MIM RE
76.7(414.)
80.6 84.0
84.0
89.3
93.1
93.6,46.)
MIM with Input Diversity (Xie et al., 2019)
DIM
77.2
77.9
79.7
80.7
81.4
85.0
85.7
Deit-[T+S+B] DIM E
77.8
78.0
80.7
84.6
81.5
86.1
86.4
DIM RE
83.1
84.8.669
86.0
84.9
87.2 87.2
87.3
89.0 43.3)
Table 5: Self-ensemble within Ensemble: Fool rate (%) 5k ImageNet val. adversarial samples at < 16.
Our proposed celfeensemble with refined tokens have significantly higher success rate of adversarial perturbations found within an ensemble models (Deit-T, Deit-S and Deit-B).
14

---

Published as a conference paper at ICLR 2022
B SELF-ENSEMBLE FOR MORE ATTACKS: PATCHWISE AND AUTO-ATTACK
conducted additional experiments with recent methods such as Patch-wist wise attack Gao al. (2020) and Auto-Attack Croce & Hein (2020b) to show case that approach does not need special setup and can be used as plug-and-play with the existing attacks, Patch-wise Gao et al. (2020) is a recent black-box attack while Auto-Attack is combination of PGD with novel losses and other attacks such as boundary- based, FAB (Croce & Hein, 2020a) and, query based Square attack (Andriushchenko al., 2020). In each case, attack transferability increased significantly using approach of self-ensemble with refined tokens (refer Table 6).
Patchwise (PI) attack (Gao et al., 2020)
Convolutional
Transformers
Source (4)
Attack
Res152
WRN
DN201
T2T-24
T2T-7
TnT
ViT-S
Pl
54.1
58.5
62.6
41.1
65.3
55.6
87.0
Deit-T
P
56.0
61.3
65.3
42.8
68.8
58.0
89.9
PI
60.4()6.3
67.3(48.8)
(1.4
44. (43)
(4.8(49.5)
64.0
95.6(+8.6
PI
53.1
57.6
63.9
49.6
59.7
64.1
84.2
Deit-S
F
57.0
62.6
66.8
51.8
65.0
71.6
89.5
RI
63.6
70.4(+128)
(-.9)
53. (43.5)
(+.36)
78.4(+14.3)
96.2
PT
53.3
56.6
60.2
47.8
54.0
59.8
74.7
Deit-B
60.7
64.4
70.0
53.0
63.3
71.4
86.6
RI
70.0
73.8
57.709.9) (7.6
77.3
(
(4.0(
Auto-Attack (AA) (Croce Hein, 2020b)
AA
15.9
19.6
19.0
11.0
34.3
19.9
52.0
Deit-T
A
14.0
15.8
17.1
10.3
27.9
18.0
49.3
RARE
20.0
24.2
(7.7()..7)
13.4G2.4)
40.0
28. (+8.2)
58.8(668)
AA
23.5
22.9
24.8
21.0
40.4
36.2
61.3
Deit-S
AA
21.9
23.3
25.8
22.2
38.6
37.8
60.3
RA
29.3
32.9(+10)
(41.9)
30.2
(1.9(+11.5)
(9.7(-13.5)
68.5
AA
25.6
28.5
28.4
23.4
39.0
39.9
55.7
Deit-B
AA
26.6
29.2
31.5
26.1
40.7
40.2
55.8
AA RE
37.4
40.4
42.6
36.3
56.7
55.6
69.5
Table 6: Fool rate (%) on 5k ImageNet val. adversarial samples at e S 16.
Auto-attack and patchwise attacks when applied to our proposed self-ensemble with refined tokens have significantly higher transfer rate to unknown convolutional and other vision transformers.
C SELF-ENSEMBLE FOR DIFFERENT E:
4, 8, 16
We evaluate adversarial transferability boost provided by our method also hold for large number of attack iterations such PGD with 100 iterations and Auto-Attack which is based on multiple random restarts, thousands of queries, etc. We further evaluate adversarial transfer at various perturbation budgets, such as 4, 8, and 16. Our self-ensemble with refined tokens approach consistently improves black-box adversarial transfer under such attack setting, thus making them not only powerful in white-box but also in black-box setting. Results are presented in Table 7.
D SELF-ENSEMBLE FOR MORE DATASETS: CIFAR 10 AND FLOWERS
We extended our approach to CIFARIO and Flowers datasets. Since VIT training from seratch on small datasets is unstable Touvron al. (2020); Yuan et al. (2021), fine-tuned ViTs trained ImageNet on CIFARIO and Flowers. Transferability of adversarial attacks increases with notable gains by using approach based self-ensemble and refined tokens (refer Tables 8, 10 and 11).
15

---

Published as a conference paper at ICLR 2022
Auto-Attuck (Croce & Ieiin, 2020b)
Source (U)
Attack
Convolational
Transformers
Res1:52
WRN
DN201
T2T-24
T2T-
TaT
VIT-S
AA
4.2/9.0/15.9
4.3/8.9/11.6
5.4/9.3/19.0
3.3/4.8/11.0
10.6/18.8/34.3 3.1/
17.6/33.7/2.00
Deit-T
AAE
4.486615588
5.0996611.11
3.7/4.8/11.3
10.8/17.6/27
5.7/10.0/18.0
18.5/34.4/49.3
AA AA 88
6.3/12.1/20.0
7./12.9/22.22
7.313.9/27.7
4.7/9.8/13.4
15.9/28.3/00.0
9.2/15.3/28.11
28.7/47.6/5888
AA
8.4/12.9/23.5 7.9/13.2/22.
8.0/14.7/24.8
17.3/11.321
15.9/25.0/444
12.4/20.6/36.2
20.7/42.0061.3
Deit-S
AA
8.8/12.5/21.9 9.0/15. 7.6/13.6/23.3
9.1/14.3/22.2
16.0/25.2/38.6
18.3/27 2/37.8
28.9/44, 9/60.3
28.9/44.9/60.3
AA
9.5/18,5/29.3
10.5/19.7/32.9
12.0/22.9/36.77
11.6/18.130.22
24.6/37.2/51.9
23.3/35.2/49.7
47685
AA
8.2/14.4/25.6
8.4/13.6/28.5
9.8/16,3/28.4
1.7/12.4/23.4
15.4/22.4/39.0
63.9/21.6/99.9
16.6/32.8/55.
Deit-B
AA
8.8/15.3/26.6
9.1/17.4/29.2
11.1/19.3/31.5
9.5/15.7/26.1
17.3/26.3/0077
16.2/27.0/40.2
31.3/37,0/55.
AARE
12.1/21.5/37.44 13.3/24.5/00.4
14.2/26.7/42.6 12.6/11.5/6.3
25.0/39.5/56.7 26.9/40.6/55.6 29.65
Prujected Grudient Decent (PGD) (Madry et 2018)
PGD
6.3/15.2/27.5
8.1/17.9/30.1
8.7/20.3/33.8
7..0/.6/24.2
18.6/30.8/56.1 13.7/33.6/45.3 33.6/63.
Deit-T
PGD
8.5/18.630.5 9.0/18.2/30.7
10.7/19.2/34.5
7.9/10.0/26.3
23.1/39.8/58.9 14.9/35.847.1
52.3/75.0/90.3
PGD
12.3/28.9/4066
18.8/23.5/44.0
16.5/29.1/48.5
15.0/26.833.11
30.9/53.//3311
29.2/52.3/69.7
08.2/80.1/988.
PGD
9.3/16.8/30.1 14.9/20.5/32.7
9.8/23.5/35.3 3
16.3//0.3/31.5
26.9/36.3/51.
26.8/40.6/56.1
60.9/62.5/844
Deit-S
PGD
12.3/17.0/55. 1//36
11.2/26.3/33,
17.0/22.2344.8
32.5/41 2/53.0 O
51,2//7.7/90,
PGD RE
17.8/30.1/44.7 18.7/33.6/50.6
20.7/35.6/54.9
23.1/34.1/45.8
48.5/67.8/8006
40.1/65.9/85.0
67.8/88.8/98.4
PGD
13.0/20/4/3.22 11.5//8.6/33.8
14.7/28.9/46.
14,77199//30.0
15.3/24.8/37.9 20.2/35.2/50.7
20.6/44 8/65.2
Deit-B
F
13.5/21.5/40.1
17.1/28.4/44.3 17.8/30.7/47.
16.7/35.7/45.0
26.7/42.2/57.6 36.8/60. 1/733/1/733.
44.8/67.
Rt MD
19.1/38.9/00.8 23.3/40.8/6..7 30.2/56.870.88 19.6/40./56.11 49.6/66.6/80.2 45.9/69.9/88.7 59.6/82.6/98.6
Table 7: Fool rate (%) on 5k ImageNet val. at various perturbation budgets. Auto-attack and PGD (100
iterations)
evaluated S 4/8/10 Our method consistently performs hetter.
Projected Gradient Descent (PGD) (Madry et al., 2018)
Transformers
Convolutional
Source (L)
Attack
T2T-24
T2T-7
Res50
PGD
26.8 / 41.9
61.0
25.4 / 39.2
Deit-T
PGDE
30.6 42.3
49.6 60.9
35.2 / 42.2
RGDRE
37.7 57.3
65.4/ 80.4
49.5 57.2
PGD
35.6 47.6
54.9
27.2/ 36.1
Deit-S
PGD
35.4 47.8
46.9 59.6
31.2 40.6
PGDRE
46.6 / 65.5
51.2/ 61.9
PGD
29.1 / 40.4
33.5 / 43.7
26.2 / 34.6
Deit-B
PGD
30.1 43.0
40.7 54.8
33.4 / 41.0
PGDRE
40.3 / 44.2
61.4/78.0
52.3 / 65.7
Table 8: Self-ensemble with refined tokens for CIFAR10: PGD fool rate (%) on CIFAR10 test set (10k
samples). In each table cell, performances are shown for two perturbation budgets i.e., S 8/16.
E VULNERABILITY OF SWIN TRANSFORMER
We show how the black-box vulnerability of VIT architecture without explicit class token (Swin transformer (Liu et al., 2021)) increases against our approach (refer Table 12).
F FULL IMAGENET VALIDATION SET
In line with most existing attack methods, we use subset ImageNet (5k samples, 5 samples from each class) for all our experiments. Our subset from ImageNet is balanced sample from each class. We will release the image-indices of ours subset publicly to reproduce the results.
Additionally, re-run best performing attacks (MIM and DIM) the whole ImageNet val. set (50k samples) to validate the merits of our approach ( refer Table 13). Our approach shows considerable improvements in attack transferability regardless the dataset size.
16

---

Published as a conference paper at ICLR 2022
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong et al., 2018)
Transformers
Convolutional
Source (4)
Attack
T2T-24
T2T-7
Res50
MIM
45.8 70.2
63.9 / 81.2
50.6 72.2
Deit-T
MIM
48.8 / 70.1
65.7 81.0
57.3 74.00
MIM RE
57.8/ / 83.0
78.8 / 99.7
57.8/77.2
MIM
62.3
77.0
/ 69.6
Deit-S
MIM
62.1/78.0
65.2 82.3
52.3 / 77.44
MIM
70.3 88.8
82.0 /9.44
60.2 79.3
MIM
/71.1
51.6 72.7
50.6 68.7
Deit-B
MIM
5
60.8 79.6
55.6 72.4
MIM RE
62.3 74.8
75.8 / 89.7
63.6 81.3
MIM with Input Diversity (DIM) (Xie et al., 2019)
DIM
56.4 / 90.1
67.1 / 89.1
67.4 85.4
Deit-T
DIM E
62.6 / 90.2
77.8 90.3
68.4 85.6
DIM RE
70.2 92.2
80.4 93.6
72.5 89.0
DIM
66.9 90.5
70.8 / 86.5
67.7 / 83.8
Deit-$
DIM E
73.7 / 90.8
71.2 89.2
69.7 /7.2
DIM RE
75.9 /94.5
82.6 / /93.7
75.3 92.6
DIM
71.7 86.9
63.6 82.4
60.3 / 83.3
Deit-B
DIM E
73.0 933.6
72.4 90.3
68.7 89.8
DIM RE
75.7 / 95.0
77.5 / 93.0
72.4 / 92.8
Table Self-ensemble with refined tokens CIFARIO: MIM
rate (%) on CIFARIO lest set (10k samples). each table cell, performances are shown for two perturbation budgets i.c., é S 8/16.
Projected Gradient Descent (PGD) (Madry et al., 2018)
Transformers
Convolutional
Source (L)
Attack
T2T-24
T2T-7
Res50
PGD
22.4 / 28.8
31.3 /1.3
16.3 / 26.8
Deit-T
PGDE
25.6 / 35.5
35.6/48.7
25.4 / 33.8
PGDRE
26.0 / 37.6
39.5 55.9
38.4/ 47.2
PGD
20.7 / 26.9
30.8 41.3
17.4 26.2
Deit-S
PGD
23.2/ 31.8
34.4 47.3
25.0 32.6
RGD
25.3 /5.44
40.1 / 56.1
43.2 52.0
PGD
20.6 26.9
32.0 411.3
18.4 27.2
Deit-B
PGDE
23.5 / 32.0
34.3 48.0
29.3 / 36.3
RGD
27.0 / 39.0
39.4 56.9
49.8 / 57.0
Table 10: Self-ensemble with refined tokens for Flowers: PGD fool rate (%) on Flowers test set. In each table
cell, performances are shown for two perturbation budgets i.e., I 8/16.
G CROSS-TASE TRANSFERABILITY WITH LARGER IMAGES (>224)
generate adversarial signals on source models with a classification objective using their initial predictions as the label (Algorithm !). In the case of VIT models trained on 224x224 images, rescale all larger images to a multiple of 224x224, split them into smaller 224x224 image portions, obtain the prediction of the source (surrogate) model for each smaller image portion, and apply the attack separately for each image portion using the prediction as the label for the adversarial objective function (Algorithm 2). For example, the case of DETR, we image size of 896 896, split
17

---

Published as a conference paper at ICLR 2022
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong et al., 2018)
Transformers
Convolutional
Source (4)
Attack
T2T-24
T2T-7
Res50
MIM
26.3 / 45.2
38.1 / 64.6
35.8/ 54.6
Deit-T
MIM
27.5 / 47.9
39.4 / 65.9
37.8 56.9
MIM RE
28.0 48.6
42.2 69.5
47.2 64.9
MIM
24.1 24.1/43.5
37.0/ 63.3
35.3 53.2
Deit-S
MIM
25,4/45.6
38.3 65.5
38,6/ 53.8
MIM
26.7 47.1
43.0 69.6
51.8 69.5
MIM
23.7 433.4
37.8 / 63.4
39.2 56.3
Deit-B
MIM E
25.2
45.2
39.0 65.0
40.3 60.0
MIM
28.7 / 49.2
43.7 69.3
56.3 71.2
MIM with Input Diversity (DIM) (Xie et al., 2019)
DIM
24.2 / 44.4 36.5 / 62.6
45.8 62.5
Deit-T
DIM
26.0 / 44.7
37.9/ / 62.9
47.2 62.8
DIMRE
27.3 / 46.9
40.1 / 67.1
54.7/70.4
DIM
23.1 / 42.0
/
47.2 64.1
Deit-S
DIM
24.0 / 42.8
/
48.1 /
DIM RE
27.1 47.0
68.7
5
DIM
23.4 42.9
36.3 61.5
46.3 64.3
Deit-B
DIM
24.4 / 44.5
37.1 63.4
46.4 66.0
DIM
30.0 / 50.2
44.3 / 69.4
58.4 / 78.8
Table 11: Self-ensemble with refined tokens for Flowers: MIM and DIM fool rate (%) on Flowers test set. In each table cell, performances are shown for two perturbation budgets i.c., S 8/16.
Source (()
PGD
PGD F
PGDRE RE
MIM
MIM
MIM
DIM
DIM
DIM RE
Deit-T
17.1
18.3
23.8
36.0
37.9
40.1
52.0
54.4
56.7
Deit-S
26.7
28.2
36.2
47.1
49.9
56.0
61.5
69.6
71.8
Deit-B
29.3
35.7
46.8
49.4
57.7
66.7
60.3
74.9
77.5
Table 12: Swin Transformer (patch-4, window-7): Fool (%) 5k ImageNet val. perturbation budget, < method increases the black-box strength of adversarial attacks against Swin Transformer.
it into 16 portions of size 224x224, and use these to individually generate adversarial signals which we later combine to build joint 896x896 sized adversary.
Algorithm 1 Cross-Task Attack
We run a generic attack on cross- tasks where a single image level label is not available. The source model is used to generate label (its original prediction) which used the target for the white- box attack. The generated adver- sarial signal is then applied onto the target task.
for samples, in (dataloader))
orig_shape = somples.shappe
# run attack
4
with torch ()__gra(():
clean_out
model (samples)
label clean_oug out argmax (dim=-1) (dim=-))ddetcchl
adv generate adversary model, samples, label)
done running attack
10
samples adv 11
continue evaluation
18

---

Published as a conference paper at ICLR 2022
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong et al., 2018)
Source (L)
Attack
Convolutional
Transformers
Res152
WRN
DN201
T2T-24
T2T-7
TnT
VIT-S
MIM
48.2
52.2
55.7
53.3
75.3
73.0
93.1
Deit-T
MIM
49.9
53.6
58.5
52.8
76.6
73.5
96.5
MIM RE
57.8(+9.8)
(2.0
(5.2(49.5)
55.4(+2.1)
84.7(+)
(8.8
(9.5(+6.4)
MIM
48.3
46.3
56.4
60.5
70.3
80.2
92.5
Deit-S
MIM E
53.1
50.7
60.2
63.3
78.0
85.6
97.1
MIM RE
(+.85)
(420))
(5.0()
69.1()8.6)
(2)3
96. (+15.)
99.5
MIM
41.6
45.0
55.9
56.5
60.0
68.4
70.2
Deit-B
MIM F
58.3
60.3
65.8
64.9
76.3
85.6
83.0
MIM
(5.9))
80.2
84.8 84.8+28.9 +28.9)
(25.8
90.1.30..)
97. (428.7)
(-26))
MIM with Input Diversity (DIM) (Xie et al., 2019)
DIM
66.4
67.0
72.0
70.1 70.2
82.8
80.5
90.2
Deit-T
DIM E
70.2
70.9
77.0
87.2
83.5
95.3
DIMRE
(2.8(+6.4)
74.8
79.7(47.7)
73.2
89.0
85.4
97.8(+7.6)
DIM
60.3
60.2
65.0
74.2
67.0
81.5
80.1
Deit-S
DIM
77.6
76.8
81.3
88.3
89.1
92.6
95.6
DIM
(2.0
83.1 (422.9)
87.2
(9.97)
89.1
96.4
(+.3
DIM
58.4
59.2
65.3
70.5
65.3
75.0
77.2
Deit-B
DIM
79.6
82.7
84.8
87.3
88.3
90.4
95.7
DIM RE
87.0
85.6
91.2
87.0
91.7(+26.4)
93.0
(6.1(-18.9))
Table 13: Fool rate (%) on the whole ImageNet val. (50k) adversarial samples at S 16. Our method remains effective and significantly increase adversarial transforability on large datasct as well.
Algorithm Attack for different input sizes
We run modified version of the attack for cross-task exper- iments requiring specific input sizes not compatible with the ViT models' input size. We split larger images into smaller im- age portions, generate adversar- ial signals separately for each portion, and combine these to obtain a joint adversarial signal relevant the entire image. For each image portion, we use the source model to generate la- bel (its prediction for that image portion), and use it as the target when generating the adversarial signal.
import product
for samples, targets in (dataloader):
sriig_shape samples.shape
run attack product list proouuct ([O, 2, 31, [0, 1, 31) temp torch.cat(1
2,244+ samples(:, 1,224xx:228 224x (1+x), 22422: 224 (1+y)]
x, y in product listl. dim=D)
10
11
with torch. ()
12
clean
model (temp)
13
label clean_out. arrmma (rgmax(dim=-1)) (dim-=1).detach)
14
15
model, temp, label) temp torch. zeros like (sampless
adv generate adversary
17
for idx, (x, y) in enumerate (product. list):
18
templ :: 22,xxx2224+ (1+x), :24xy::24+ (1+y)
adv[orig_share [0] +idx: sidx:orig soriorrrgiiiii shapesssppplle [0] (1dx+1))
20
adv temp
21
assert orig_share
adv. shape
22
done running attack
23
samples adv
24
continue model evaluation
H COMPUTATION COST AND INFERENCE SPEED ANALYSIS
In this section, we analyze the parameter complexity and computational cost of our proposed approach. The basic version of our self-ensemble (Attack only off the shelf pretrained ViT without adding any additional parameters. However, we introduce additional learnable parameters with token refinement module to solve misalignment problem between the tokens produced by an intermediate
19

---

used se- lected samples (Sec. 4) from ageNet validation set for this ex- periment and all iterative attacks Published as a conference paper at ICLR 2022
Class Token
Patch Tokens
Figure Our proposed refinement module (Sec. 3.2) processes patch tokens using a convolutional block (He et al., 2016) while the class token is processed by linear layer. Class and patch tokens are the outputs of intermediate ViT black. The convolutional layers have a filter size of 3x3xd. We rearrange the number of patch tokens into 14x14 grid before feeding them to convolutional block. The embedding dimension (d) of the class token and each patch token dictates the number of parameters and inference compute cost within convolutional and MLP layers the refinement modulc.
1xx14xd
3x3, d
p 1xxxl
Relu
MLP
Relu
Average Pool
Kernel size=14
Table 14: We compare inference speed (in minutes) of attacks on the conventional model against the attack on our proposed self- ensemble (with and without refine- ment module).
Attack Inference Speed Analysis
Attacks
Model
Self-ensemble
Refined Tokens
FGSM
PGD
MIM
DIM
0.19
1.48
1.49
1.51
Deit-T
0.19
1.59
1.69
1.6
0.23
2.12
2.13
2.15
X
0.34
3,22
3.21
3.23
Deit-S
0.36
3.34
3.32
3.35
(PGD, MIM, DIM) ran for 10 it- erations. Inference speed is com-
0.54
5.19
5.22
5.24
Deit-B
X
0.97
9.45
9.32
9,27
puted using Nvidia Quadro RTX
X
1.0
9.66
9.43
9.44
6000 with Pytorch library.
1.64
16.15
15.88
15.9
block and the final classifier of a vision transformer (Sec. 3.2). Our refinement module processes class and patch tokens using MLP (linear layer) and Convolutional block (Fig. 8) and its parameter complexity is dependent on the embedding dimension of these pretrained tokens. For example, DeiT-S produces tokens with embedding dimension of size R-384 and our refinement module adds 1.47 million parameters to its sub-model within the self-ensemble trained with refined tokens.
Fine tuning with additional parameters provides notable gains in recognition accuracy. Our approach increases top-1 (%) accuracy (averaged across self-ensemble) by 12.43, 15.21, and 16.70 for Deit-T, Deit-S, and DeiT-B, respectively. Similar trend holds for the convolutional vision transformer (Wu al., 2021) and MLP-Mixer (Tolstikhin al., 2021) well (Fig. 10).
Further, we analyze inference speed for different attacks Table 14. The computational cost difference between the attack conventional model and the attack our self-ensemble (without refinement module) is only marginal. As expected, attacking self-ensemble with refinement module slightly expensive, however, the notable difference is only with very large models such as DeiT-B.. In fact, the increase in computational cost is a function of the original complexity of the pre-trained ViT model e.g.. DeiT-B with an original parametric complexity of 86 million generates high-dimensional tokens 784, which leads to higher compute cost in our refinement module.
H.1 LATENT SPACE OF REFINED TOKENS
We visualize the latent space of our refined tokens in Fig. We randomly selected 10 classes from ImageNet validation set distributed across the entire dataset. We extracted class tokens with and without refinement from the intermediate blocks (5,6,7.8) 8) of Deit-T, Deit-S, and Deit-B. Our refined tokens have lower intra-class variations i.e., feature representations of samples from the same class are clustered together. Furthermore, the refined tokens have better inter-class separation than the original tokens. This indicates that refinement minimizes the misalignment between the final classifier and intermediate class tokens, which leads to more disentangled representations. Attacking such disentangled representations across the self-ensemble allows us find better adversarial direction that leads to more powerful attacks.
20

---

Published as a conference paper at ICLR 2022
Closs Taken St Withoot Refiemmen
Ches Token Er Without Refinement
Onas Token 7%, Without Reffnement
Ches Toben B: Withost Refinement
Class Tokens from intermediate blocks of D are projected from 1992 to R
Taken Rotheemen
Class Token with Rafinament
Class Tohen E: With Rafinament
Clocs Totan with Reftnamant
Class Tokan a: With Reftrement
Class Tokens from intermediate blocks of DeiT-S are projected from 3334 to R.2
Cass Token 5t Without Beffnementt
Clace Toban E: Without Refinesentt
Class Token Withost Rafinement
Clacs Toben B: Withost Refmement
Wieh
Class Tokens from intermediate blocks of DeiT-B are projected from 781 to IR2.
Figure 9: Class Tokens t-SNE visualization: We extracted class tokens from intermediate blocks (used for creating individual models within self-ensemble) and visualize these in 2D space via i-SNE (Van der Maaten & Hinton, 2008). Our refined tokens have lower intraclass variations i.c., feature representations of the same class samples are clustered together. Further refined tokens have better interclass separation than original tokens. We used skleam (Pedregosa et al., 201 and perplexity is set to 30 for all the experiments. (best viewed in zoom)
21

---

Published as a conference paper at ICLR 2022
Accuracy Top-1 (%)
80
Original Class Token
10.0
+0.0
JO0
+000
Refined Class Token
+2.3
100
5.0
0.
100
60
6.
5.6
0-
6..
00
0-6
+28.0
0.0.6
10.
100
+27.9
9.
20
+18.8
+2.7
+4,2
1
2
7 Block Number (CVT-T)
4
9
10
Accuracy Top-1 (%)
80
190
Original Class Token
+0
+0.0
+0.0
13.8
Refined Class Token
27.8
5
19
60
80
19
+47.4
:6
56
+0.0
40
4338
5
+78.0
c
5.
6
5
20
115.3
6
19
+2.8
+6,9
+8
5
50
1
2
3
4
5
7
8
9
10
11
12
13
14
15
16
Block Number (CVT-S)
Accuracy Top-1 (%)
80
Original Class Token
+0.0
+0.0
+0.0
+0,2
Refined Class Token
+0,7
90
9
60
6.
**
6
+4.0
00
6.
+6
5.6
16.
0-0
16
+8,8
5.0
5
10
10
20
+11.2
+13.4
+9,9
26
50
00
5.
0-
0
0
0.
1
2
3
6
7
10
11
12
Block Number (Mixer-B)
Figure 10: Self-Ensemble for CvT (Wu et al., 2021): We measure the top-1 (%) accuracy on ImageNet validation set using the class-token and average of patch tokens of each block for CvT and MI.P-Mixer and compare to our refined tokens. These results show that fine-tuning helps align tokens from intermediate blocks with the final classifier enhancing their classification performance. An interesting observation is that pretrained MLP-Mixer lower final accuracy than CvT models, however, its early blocks show more discriminability than CvT. This allows a powerful self-ensemble which in turn boost adversarial transferability (Table 15).
SELF-ENSEMBLE FOR HYBRID VIT AND MLP-MIXER
In this section, apply our proposed approach diverse set of ViT models including hybrid vision transformer (convolutional vision transformer (CvT) (Wu et al., 2021)) as well as a MLP-Mixer (Tolstikhin al., 2021). CvT design incorporates convolutional properties into the ViT architecture. We apply our approach on two CvT models. CvT-Tiny (CvT-T) and CvT-Small (CvT-S). We create 10 and 16 models within self-ensemble of CvT-T and CvT-S, respectively. On the other hand, MLP-Mixer thrives simplistic architecture design. It does not have self-attention layers or class token. We use an average of patch tokens as class token in this case and create 12 models within self-ensemble of MLP-Mixer-BB using the same setup as described in Sec. 3.2. The refined class tokens show clear improvements in top-1 1 (%) accuracy ImageNet validation set (Fig. 10). Similarly attacking self-ensemble with refined tokens has non-trivial boost in attack transferability (Table 15). The successful application of approach such diverse ViT designs highlights the generality of our method.
(Mixer-B). We train refinement module for each of these models
22

---

Published as a conference paper at ICLR 2022
Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014)
Convolutional
Transformers
Source (1)
Attack
BiT50
Res152
WRN
DN201
VIT-L
T2T-24
ToT
VIT-S
T2T-7
FGSM
23.82
28,34
29.66
30.84
16.74
26.06
28.96
30.28
41.74
CVT-T
FGSM e
27.66
34.42
35.58
38,66
18.26
30,98
35.38
38.42
51.46
FGSM RM
30.24
(7.74
39.44.00)
42.52.
18.30
28.52.2.5)
35.00....)
39.30_..0
.9.50
FGSM
19.50
24.04
26.56
27.42
14.6
21.48
22.58
25.80
36.82
CvT-S
FGSM R
30.0
37.30
39.08
42.04
19.48
32.38
34.56
39.24
52.92
FGSM RR
33,10
20
41.84.-15.))
45.66-183)
19.44
32. 12:12:
35.98.-13.4)
41.266-15.5:
59.38
FGSM
25.00
31.02
33.00
34.54
29.28
29.24
35.08
52.44
40.86
Mixer-B
FGSM 8
33.08
39.06
41.62
46.30
36.76
32.64
45.66
73.48
58.14
FGSM R2
(222 45.26 45. 45.12..12.1
48.65..1411
38.54.03.
36.70.22)
47.22
80.44 zum (3.58
Projected Gradlent Decent (PGD) (Madry et al., 2018)
PGD
20.400
23.82
25.98
25.76
8.16
27.34
31.30
20.74
46.16 16
CVT-T
PGD R
21.36
26.06
28.86
28.18
8.78
28.66
33.38
23.40
50.14
RGDRE
23.20
28.44
29.56..3.)
31.28
8.80
26.92,.0
(1.32
22. -2.14-114)
.7.70
PGD
19.80
22.98
25.14
24.28
7.92
24.92
25.24
18.40
40.72
CVT-S
P
25.58
30,28
32.76
34.32
9.18
32.34
34.52
24.08
55.90
RGDRR
27.12
30,56
32.28..3.1)
33.78.
8.66
30.22
33.76
23.00
58.68.1158)
PGID
12,92
16.70
17.96
19.20
14.68
16.98
23,90
42.32
29.46
Mixer-B
PGD
24.74
32,36
35.48
37.78
28.22
31.42
49.50
76.54
58.00
PGD RM
26.50
39.50 39.50 -4
38.60
32.56 17.9
33.30
-4.68..008)
82,90 21.0
62, .3.53
Momemtum Iterative Fast Gradient Sign Method (MIM) (Dong al., 2018)
MIM
39.30
42.68
45.94
48.88
20.38
48.74
53.50
45,46
65.94
CVT-T
MIM
42.24
45.66
49.88
54.02
21.46
50.28
56,26
49.46
71.60
RI
48.92....)
52.00
58.18..04
60.22..11.)
20.24:au
50.54...3
58.74....)
50.94.ss)
81.36,1154)
MIM
36.60
39,94
42.64
44.8
18.48
45.06
44,92
39.12
58.64
CVT-S
MIM
47.04
50.72
55.02
59.56
23.06
56.06
57.84
51.20
76.54
MIM RM
53,26,0267)
(1.04
13.6
(1.22
22.24
55.58
-2.80
52.60 13.5) 79.26, (206)
MIM
26.96
32.66
35.28
38.42
33.96
34.68
45.08
68.16
46.88
Mixer-B
MIM
42.62
49.82
52.98
59.32
51.72
49,90
68.86
92.58
75.48
MIM RE
46.50...os 22.30 26.20.22.5) 62.56
.3.44..0.s:
.2.00
22.2
94.66 -6.5) .0.12
MIM with Input Diversity (DIM) (Xie et al., 2019)
DIM
61.94
61.60
64.22
67.72
36.9
69.80
76,20
65.70
73.68
CVT-T
DIM
72.22
72.94
75.88
80.80
41.54
79.24
86.12
76.02
85.62
DIM RB
(8.)
(5.
80.90,
10.7) 85.74 (100)
40.06 1.12) (8.62
89.34, 13.1)
77.88,
122) 92.42
DIM
51.64
55.28
54.16
57.22
30.98
60.1
62.36
53.30
62.46
CVT-S
D
74.96
75.88
79.56
83.5
44.06
80.94
86.40
76.86
88.06
DIM
80.45,a08
78.94..237)
81.64..27 5) 25.74..2
42.78.-11.)
22.36.22.3)
88.50.-a6.1)
77.66-213)
22.10))2)))
DIM
48.64
49.08
52.14
57.44
39.28
$7.24
64.00
71.88
63.02
Mixer-B
DIM R
69.32
72.46
74.98
80.38
55.96
74.66
84.66
91.86
88.78
DIM RB
(2.2)
76.72
80
84.66 (272)
60.43 2122
82.17
88.62
95.22 233) 92.92 (29.))
Table 15: Fool rate (%) on 5k ImageNet val. adversarial samples at S 16. Perturbations generated from our
proposed self-ensemble with refined tokens from vision transformer have significantly higher success rate.
J CAN SELF-ENSEMBLE IMPROVE TRANSFERABILITY FROM CNN?
Our proposed idea of the self-ensemble can be applied to a CNN model (e.g., ResNet50) with an average pooling operation over the intermediate layer outputs. However, due to the varying channel dimension of feature maps across a pretrained CNN, building an ensemble with shared classifier is not as forwight-forward as in ViT (where equidimensional class token is available at each level). For example, ResNet50 has four intermediate blocks that produce feature maps of sizes 12 2256x56x38 5112x28 28
R 1024x14x12 and 2048x777 respectively. Then, the final classifier processes the average pooled features from the last block. Since there is mismatch between feature dimensions of the intermediate layers and the final classifier, therefore applying the basic version of self-ensemble approach (Attack F) is not possible for the pretrained ResNet.
An intermediate layer is essential to project intermediate feature vectors to the same dimension as the final feature vector in the case of CNNs. Therefore, for the refined embedding variant of
23

---

Published as a conference paper at ICLR 2022
FGSM: DeiT-S vs ResNet-50
Fool Rate Improvement e
DeiT-S (E)
DeiT-S (RE)
ResNet-50 (RE)
25
20
10
15
10
BIT Res152 WRN DN201 ViT-L T2T-24 TnT VîT-S T2T-7
Target Models
PGD: DeiT-S ResNet-50
Fool Rate Improvement (96)
30
7/A
DeiT-S (E)
BB
DeiT-S (RE)
ResNet-50 (RE)
25
20:
15
5
BIT50 Res152 WRN DN201 ViT-L T2T-24 TnT
ViT-S
T2T-7
Target Models
MIM: DeiT-S vs ResNet-50
Fool Rate Improvement 196)
DeiT-S (E)
DeiT-S (RE)
ResNet-50 (RE)
25
20
15
10
BIT50 Res152 WRN DN201 VIT-L T2T-24 24 TnT VIT-S T2T-7
Target Models
DIM: DeiT-S ResNet-50
Fool Rate Improvement (96)
30
K/A
DeiT-S (E)
DeiT-S (RE)
ResNet-50 (RE)
25
20
15
BIT50 Res152 WRN DN201 VIT-L T2T-24 TnT VIT-S T2T-7
Target Models
Figure 11: Self-Ensemble for ResNet50 (He et al., 2016): We report relative improvement after adding refinement module for Resnet50 and Deit-S mdoels. Note that the basic version of self-ensemble can applicd to ResNet50 due to varying channel dimension across different layers. Feature refinement improves adversarial transfer from ResNet50, however rolative gains are significant for vision transformer, Deit-S.
our approach, we finetune ResNet50 features to create self-ensemble with the procedure described in Sec. 3.2. We report the relative improvements of self-ensemble with refined features w.r.t the original (single) model and compare ResNet50 with Deit-S (Fig. 11). Both these off-the-shelf models, ResNet50 (25 million parameters) and Deit-S (22 million parameters), comparable in terms of their computational complexity. We observe that feature refinement also helps to boost adversarial transferability from the ResNet50 model. However, adversarial transferability improvement of our self-ensemble with token refinement on ViTs is considerably better than for the case of ResNet50.
These results suggest that the proposed approach is well-suited for improving adversarial transferabil- ity of VIT models.
24


Reviews:
[{"review_id": "fI3tb3Zg3BK", "reviewer": "Reviewer_aX8L", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n\n###Summary###\n\nThis paper enhances transferability of vision transformers (ViT) by introducing two strategies specific to the architecture of ViT models, i.e. Self-Ensemble and Token refinement. Specifically,  Self-Ensemble finds multiple discriminative pathways by dissecting a single ViT model into an ensemble of networks, leading to an explicit utilization of class-specific information at each ViT block. In addition, the proposed Token Refinement can potentially enhance the discriminative capacity at each block of ViT. \n\nThe high-level motivation of this paper is that the adversarial patterns found via conventional adversarial attacks show very low black-box transferability for ViT models. The authors claim that this phenomenon is due to the sub-optimal attack procedures that do not leverage the true representation potential of the ViTs. Thus, this paper introduces a highly transferable attack approach that augments the current adversarial attacks and increases their transferability from ViTs to the unknown models. \n\nThe paper conduct experiments on a range of standard attack methods to establish the performance boost obtained through the proposed transferablility approach by using the Source (white-box) models, Target (black-box) models. The experiments conducted on ImageNet/COCO dataset, and PASCAL demonstrate that the proposed method can outperform the baselines on some of the experimental settings. The paper also provides a detailed analysis of the model and experimental results. \n\n### Novelty ###\n\nThe paper proposes two novel strategies about how to enhance the transferability of ViT models, i.e. refined tokens and self-ensemble. When those ideas are combined to an adversarial attack, it significantly boosts the transferability of adversarial examples, thereby bringing out the true generalization of ViTs’ adversarial space. These two ideas are interesting and should be heuristic for the research community, especially for those who have a strong interest in designing ViT models. \n\n\n###Clarity###\n\nThe paper is overall logically clear and easy to follow. The tables and figures are well illustrated by the captions. The writing and presentation of this paper are good enough to convey the ideas.\n\n###Pros###\n\n1) The paper proposes two interesting ideas to enhance the generalization ability of the ViT models, i.e. refined tokens and self-ensemble. These two methods are well-motivated. For self-ensemble, it creates an ensemble of classifiers by learning a shared classification head at each block along the ViT hierarchy, leading to an explicit utilization of class-specific information at each ViT block. For refined tokens, the paper proposes both patch token refinement and class token refinement to tackle the misalignment between the features with the classifier. \n\n2) The paper provides extensive experiments on multiple datasets, settings, and vision tasks. The results show that both proposed techniques can boost the performance, compared with many SOTA baselines.  Through the experiments, this paper empirically demonstrates favorable transfer rates across different model families (convolutional and transformer) as well as different vision tasks (classification, detection, and segmentation).\n\n3) The paper provides a detailed empirical analysis to demonstrate the effectiveness of the proposed methods. \n\n###Cons###\n\n1) While self-ensembling is useful to enhance the transferability of the ViT models, it also makes the models more complex. Thus, some computational costs and inference speed analysis are desired. \n\n2) One of the main assumptions of proposing the patch token refinement and class token refinement is there exists misalignment between the feature and classifier in ViT models. It will be helpful to provide some empirical evidence to validate this assumption (even some material in the appendix). \n\nBased on the summary, cons, and pros, the current rating I am giving now is \"weak accept\". I would like to discuss the final rating with other reviewers, ACs.\n", "summary_of_the_paper": "\nThis paper enhances transferability of vision transformers (ViT) by introducing two strategies specific to the architecture of ViT models, i.e. Self-Ensemble and Token refinement. Specifically,  Self-Ensemble finds multiple discriminative pathways by dissecting a single ViT model into an ensemble of networks, leading to an explicit utilization of class-specific information at each ViT block. In addition, the proposed Token Refinement can potentially enhance the discriminative capacity at each block of ViT. \n\nThe high-level motivation of this paper is that the adversarial patterns found via conventional adversarial attacks show very low black-box transferability for ViT models. The authors claim that this phenomenon is due to the sub-optimal attack procedures that do not leverage the true representation potential of the ViTs. Thus, this paper introduces a highly transferable attack approach that augments the current adversarial attacks and increases their transferability from ViTs to the unknown models. \n\nThe paper conduct experiments on a range of standard attack methods to establish the performance boost obtained through the proposed transferablility approach by using the Source (white-box) models, Target (black-box) models. The experiments conducted on ImageNet/COCO dataset, and PASCAL demonstrate that the proposed method can outperform the baselines on some of the experimental settings. The paper also provides a detailed analysis of the model and experimental results. \n", "summary_of_the_review": "Thia paper proposes two strategies specific to the architecture of ViT models, i.e. Self-Ensemble and Token refinement to enhance the transferability of the ViT models.  The experiments conducted on ImageNet/COCO dataset, and PASCAL demonstrate that the proposed method can outperform the baselines on some of the experimental settings. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "do-2UT-eUX", "reviewer": "Reviewer_U2B6", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, I find the paper presents an interesting and important work w.r.t the adversarial security of ViTs. The main strengths of the paper are:\n\n- The paper is well organized and well-written. \n- The proposed approaches are well-motivated by empirical findings.\n- The paper has several experiments that the effectiveness of these approaches.\n\nI also have some concerns/questions about the discussions in the paper:\n\n- In my view, the addition of the ensemble module and refined tokens is similar to an approach that combines both CNN and ViT. In Figure 6, it is also evidence that the addition of the refined tokens is important. This begs the question of whether adversarial samples of hybrid CNN/ViT models have similar transferability performance as of the proposed methods in the paper? Currently, it seems like the paper only studies one particular type of ViTs, and the generality of the proposed methods, as well as the generality of the motivation, can be limited.\n- Does the same trend follow w.r.t adversarial transferability to other large CNN models such as BiT?\n\n", "summary_of_the_paper": "The paper looks at Vision Transforms (ViTs) models and transferability of adversarial examples, which is previously known to be challenging between ViTs to CNNs and vice versa. The paper leverages the discriminative information stored in the lower layers' tokens and proposes two methods that modify and fine-tune the ViTs in order to extract the adversarial samples. The paper conducts extensive experiments to show the effectiveness in adversarial transferability across different models, including CNN's and ViT's variants, and tasks.", "summary_of_the_review": "I find that the paper presents an interesting study on the transferability of adversarial samples of ViTs. In general, I think it is a good paper, but also have some concerns about the generality of the work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "QfUhRestkN", "reviewer": "Reviewer_HKsh", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n1. This is a novel study on ViT adversarial attack modeling, and the self-ensemble method seems to make sense.\n2. This work conducts sufficient experiments, and shows good transferablity in a variety of situations.\n3. The method of this paper is a general framework. It is not limited to specific calculation method (such as saliency maps and gradients), but is augmented calculation object. So I hope this paper will have a broader impact.\n\nWeekness\n1. The author claims that the design of this attack method comes from the different inductive biases between Transformer and CNN style models. However, this paper dosen‘t mention why this inductive bias would cause the previous attack method to fail, nor dose it mention why that difference would make the method more effective.\n2. In addition, this method seems to be able to improve the transferablity of models with other paradigms. If this method can improve CNNs' adversarial tranferablity better, it is difficult to say that this is a method of \"improving adversarial transferability of ViT model\". This paper lacks relevant ablation studys.", "summary_of_the_paper": "In this paper,  authors improve adversarial transferability of ViT by Self-Ensemble and Token Refinement method.\n'Self-Ensemble' implies it treats each transformer block of ViT as the 'last block' and apply a shared classifier to it. So that the trivial discriminative path can be attacked.\n'Token Refinement' tries to improve the classification ablity of shallow blocks by using extra non-shared module on each block's output tokens.\nThis paper successfully improved the adversarial ability of ViT attacks. Sufficient experimental results and detailed description ensure its reproducibility.", "summary_of_the_review": "In summary, this paper proposes novel methods and shows good reproducibility. Proposed self-ensemble and token reinforcement gets good results in many situations. But I still have doubts about whether this method is for Transformer’s inductive bias.\nI am inclined to accept this paper, but I would like to see a more detailed discussion.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0spBr3XAF57", "reviewer": "Reviewer_NMLw", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem approached by the paper is interesting and the proposed approach is novel to the best of my knowledge. The work is well-structured and written in a clear and concise manner with sufficient experimental verification. \n\nStrengths:\n- well written\n- sufficient experimental validation including different vision tasks and datasets\n- a good overview of previous black-box and white box attacks\n\nWeaknesses:\n- will the method work on images with lower dimensions\n-  it would have been interesting to include a Future Works section\n- a more thorough discussion as why the approach works\n- what effect do surrogate model and black-box target model topological differences have on adversarial transferability?\n- enhance the quality of images\n- ", "summary_of_the_paper": "By incorporating self-ensemble and token refinement methods, the authors have devised a method that takes use of the modularity of ViT models to improve their adversarial transferability. Their experiments have demonstrated adversarial transferability across different convolution and transformer model families for different vision tasks.", "summary_of_the_review": "Overall, I vote for marginal accept. I like the idea of mining the relation between blocks and handle it by the proposed self-ensemble and token refinement method. My major concern is about the clarity of the paper and discussion as when and why the method works. Hopefully the authors can address my concern in the rebuttal period. \n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors enhance the adversarial transferability of vision transformers by introducing two novel strategies specific to the architecture of ViT models: Self-Ensemble and Token Refinement method. Comprehensive experiments on various models (including CNN's and ViT's variants) and tasks (classification, detection, and segmentation) successfully verify the effectiveness of the proposed method.

In general, the problem studied is relevant and important. The paper is well-written and well-motivated with empirical findings. The proposed two strategies are novel, simple to implement, and effective in practice. Following the author's response and discussion, the average score increases from 6 to 7.5, with most concerns well addressed. AC believes that the paper should be highlighted at the ICLR conference.
Decision: Accept (Spotlight)