Title: Improving Adversarial Transferability via Intermediate-level Perturbation Decay
Venue: NeurIPS 2023
Accepted: yes
Keywords: adversarial examples

black-box attack

adversarial transferability
Abstract: Intermediate-level attacks that attempt to perturb feature representations following an adversarial direction drastically have shown favorable performance in crafting transferable adversarial examples. Existing methods in this category are normally formulated with two separate stages, where a directional guide is required to be determined at first and the scalar projection of the intermediate-level perturbation onto the directional guide is enlarged thereafter. The obtained perturbation deviates from the guide inevitably in the feature space, and it is revealed in this paper that such a deviation may lead to sub-optimal attack. To address this issue, we develop a novel intermediate-level method that crafts adversarial examples within a single stage of optimization. In particular, the proposed method, named intermediate-level perturbation decay (ILPD), encourages the intermediate-level perturbation to be in an effective adversarial direction and to possess a great magnitude simultaneously. In-depth discussion verifies the effectiveness of our method. Experimental results show that it outperforms state-of-the-arts by large margins in attacking various victim models on ImageNet (+10.07% on average) and CIFAR-10 (+3.88% on average). Our code is at https://github.com/qizhangli/ILPD-attack.

Improving Adversarial Transferability via Intermediate-level Perturbation Decay

Abstract

Intermediate-level attacks that attempt to perturb feature representations following an adversarial direction drastically have shown favorable performance in crafting transferable adversarial examples. Existing methods in this category are normally formulated with two separate stages, where a directional guide is required to be determined at first and the scalar projection of the intermediate-level perturbation onto the directional guide is enlarged thereafter. The obtained perturbation deviates from the guide inevitably in the feature space, and it is revealed in this paper that such a deviation may lead to sub-optimal attack. To address this issue, we develop a novel intermediate-level method that crafts adversarial examples within a single stage of optimization. In particular, the proposed method, named intermediate-level perturbation decay (ILPD) , encourages the intermediate-level perturbation to be in an effective adversarial direction and to possess a great magnitude simultaneously. In-depth discussion verifies the effectiveness of our method. Experimental results show that it outperforms state-of-the-arts by large margins in attacking various victim models on ImageNet ( +10.07% on average) and CIFAR-10 ( +3.88% on average). Our code is at https://github.com/qizhangli/ILPD-attack.

1 Introduction

Adversarial examples can be crafted without getting access to essential information ( e.g. , architecture and parameters) of the victim models [34]. Due to their applicability of assessing AI safety in many real-world deployment environments, recent years have witnessed a steady increase in the study of black-box adversarial examples [29, 4, 11, 46, 8, 16]. A considerable proportion of black-box attacks rely on transferability of adversarial examples, i.e. , a phenomenon where adversarial examples crafted targeting a white-box substitute model can also fool unknown victims with a moderate success rate.

Despite being crucial to many attacks, the adversarial transferability has not been fathomed out. Some work attempts to regulate the intermediate-level perturbation (discrepancy between the intermediatelevel representation of adversarial examples and that of benign examples) to show a large magnitude and align with an directional guided that can effectively maximize the prediction loss of a model [20, 26, 43, 52]. To achieve this, they choose to maximize the scalar projection of the intermediate-level perturbation onto a directional guide. The determination of the directional guide of such methods are various, and each was well-motivated. Though effective, such an objective inevitably deviates the intermediate-level perturbation from the well-motivated directional guide (as will be discussed in Section 3.1), and it may deteriorate the attack performance on the substitute model and lead to sub-optimal transferability despite showing a large perturbation magnitude in the middle layer.

In this paper, we propose a method that encourages the intermediate-level perturbation to possess a greater magnitude than a directional guide by its nature and to be in the same adversarial direction as that of the guide. This is achieved by introducing intermediate-level perturbation decay (ILPD)

in a single stage of optimization. Extensive experiments were conducted to show that our ILPD outperforms existing state-of-the-arts by large margins in attacking various victim models on CIFAR10 and ImageNet. Moreover, such a method can further be readily integrated into a variety of existing transfer-based attacks.

2 Background and Related Work

Adversarial examples can be generated in white-box and black-box settings.

White-box attacks. In white-box settings, the architecture and parameters of the victim model are known to the adversary, and the gradient of the victim models can therefore be directly utilized to generate adversarial examples. Given a victim model f : R n → R c which has been trained to classify any input, a typical way of generating adversarial examples is to maximize the prediction loss L ( f ( x + ∆x ) , y ) of f while constraining the ℓ p norm of the perturbation ∆x to a benign input x , i.e. , max ∥ ∆x ∥ p ≤ ϵ L ( f ( x + ∆x ) , y ) , where ϵ is the perturbation budget and y is the ground-truth. FGSM [14] derives a single gradient step to obtain the perturbation ∆x = ϵ · sign( ∇ x L ( f ( x ) , y )) in an ℓ ∞ setting. If not otherwise mentioned, we denote L ( f ( · ) , · ) as L f ( · , · ) for simplicity of notation. For performing stronger attacks, iterative methods have also been proposed, e.g. , I-FGSM [25] and PGD [32] which update the perturbation at the j -th iteration as

where η is the update step size and the Clip ϵ ∆x operation ensures that the norm of perturbation does not exceed ϵ .

Black-box attacks. Issuing attacks in black-box settings is much more challenging than in white-box settings due to the lack of information of the victim model. In general, neither the architecture nor the parameter of the victim model is accessible in the black-box settings, and the attacker can only obtain the prediction confidence or even only the predicted label, providing any input. Two lines of methods have been developed for crafting adversarial examples under such circumstances, i.e. , query-based methods and transfer-based methods. Query-based methods advocate searching along promising directions [4, 5, 49, 48] or estimating the gradient of victim models via zeroth-order optimization [6, 22, 23, 41], while transfer-based methods train substitute models to craft adversarial examples and can be query-free.

Adversarial transferability. As the core of transfer-based attacks, the transferability of adversarial examples from substitute models to unknown victim models has been studied widely over recent years, and a variety of methods have been proposed to enhance it. Related to our work, some previous work also achieves improved transferability by modifying the gradient computation on substitute models. For instance, Guo et al. [15] advocated removing non-linear activations during back-propagation, especially in later layers of DNNs. Wu et al. [45] suggested to take gradients more through skip connections, if exist, on the substitute models. In particular, intermediate-level representations were taken special care of in some prior arts [20, 26, 16, 43, 52].

3 Perturbation Decay in the Feature Space

We revisit some existing attacks that operate on intermediate-level representations and identify their possible limitations in Section 3.1 and propose a novel method to improve them in Section 3.2.

3.1 Revisiting Intermediate-level Attacks

As transfer-based attacks, intermediate-level attacks are performed with at least a substitute model. Given a model input x , the output of an α -layer substitute model can be simplified as f ( x ) = W T α ϕ α -1 ( W T α -1 . . . ϕ 1 ( W T 1 x )) , where, for i ∈ { 1 , . . . , α } , ϕ i is an activation function, and W i ∈ R n i -1 × n i parameterizes a layer in the model. Assume that the input is of n dimensions and there are c classes for prediction, then we have n 0 = n , and n α = c . Existing intermediate-level attacks decompose the model into a β -layer feature extractor h and an ( α -β ) -layer classifier g , i.e. , we can formulate the model as f ( x ) = g ( h ( x )) for any x . A motivation of intermediate-level attacks is that different models might share the same low-level representations in h and it suffices to operate on the output layer of h [20, 26].

A typical work of intermediate-level attack is ILA [20]. It attempts to regulate the intermediate-level perturbation ( i.e. , discrepancy between the intermediate-level representation of adversarial examples and that of benign examples) to show a large norm and to be close to a directional guide. The directional guide in ILA is chosen as the discrepancy from benign representations that could be reached in the output space of h and leads to maximum prediction loss of f , given the perturbation budget. Since white-box attacks all devote to achieving maximal prediction loss, ILA can be performed with simply a u -step baseline attack ( e.g. , I-FGSM) and the intermediate-level perturbation achieved by the baseline attack as the directional guide.

Let x + ∆x j be the adversarial input achieved at the j -th step of the baseline attack, we denote z ( j ) h := h ( x + ∆x j ) as the intermediate-level representation of x + ∆x j . Further, denote by v the directional guide, then we can formulate it as v = ( z ( u ) h -z (0) h ) .

ILA maximizes the scalar projection, which is

where 1 / ∥ v ∥ 2 is omitted as it does not affect the optimal solution. With such a learning objective, the norm of the vector projection is maximized if v and h ( x + ∆x ) -h ( x ) form an acute angle. Since the direction of intermediate-level perturbation is not restricted to be in exactly the same direction as that of the directional guide, the obtained intermediate-level perturbation often deviates from the directional guide in practice. Yet, the adversarial perturbations are generally not robust enough [1], and such directional deviations normally leads to decrease in the prediction loss of the substitute model, even with large perturbation magnitude in the middle layer.

A quick experiment was conducted to examine the impact of such directional deviations. The experiment was conducted on CIFAR-10 [24] using a VGG-19 [36] as the substitute model and another VGG-19 with the same h as the victim model, just to simplify the analyses. The directional guide v of ILA was obtained by 10steps I-FGSM following prior work [26, 16, 27]. We evaluated the cross-entropy prediction loss of the models with various intermediate-level perturbations to demonstrate how the attack performance can be affected by the scalar projection (onto v ) and the directional deviation (from v ). Somewhat unsurprisingly, as depicted in Figure 1, when fixing the direction of intermediatelevel perturbations, the perturbations with larger magnitude leads to more significant attack per-

Figure 1: How the prediction loss of (a) the substitute model and (b) the victim model vary with the scalar projection onto v and the direction deviation from v ( i.e. , the cosine similarity between v and h ( x + ∆x ) -h ( x ) ). The purple box represents the results achieved by ILA.

formance on both the substitute model and the victim model. More enticingly, with intermediate-level perturbations showing similar scalar projection, those deviating more from the directional guide leads to lower prediction loss on the victim model and hence worse black-box attack performance. As illustrated in the figure, the ILA perturbation only slightly deviates from the directional guide v (with a cosine similarity of roughly 0.90), yet its attack performance is already inferior to that of the intermediate-level perturbations along the direction of v with even smaller magnitude . Follow-up work of ILA, e.g. , [26, 16], mostly focuses on developing more powerful directional guides, yet the deviation remains and the same problem persists.

3.2 Our Solution

Given Figure 1, one may consider that a straightforward improvement of the intermediate-level attacks is to instead maximize the magnitude of (intermediate-level) perturbation only along the direction of v . Nevertheless, most existing intermediate-level attacks consists of two separate stages which sequentially seeks v and maximizes perturbation magnitude, and they fail to take into account whether there exists a larger intermediate-level perturbation along the direction of v , given the perturbation budget in the input space of f .

In this context, we aim to develop a novel method that seeks intermediate-level perturbations to be in an adversarial direction and to possess larger norms (or say larger magnitudes) compared to a directional guide in the same direction simultaneously. As with previous methods, we consider a direction to be adversarial if a guide along this direction achieves large prediction loss by itself when being fed to the substitute classifier g . The intermediate-level perturbation with a relatively larger magnitude can simply be obtained by amplifying the directional guide by γ × in the middle layer. However, such an intermediate-level perturbation may not be reachable with a limited perturbation budget, i.e. , ∥ ∆x ∥ p ≤ ϵ , in the input space.

In light of all these considerations, we turn to the 'dual' optimization problem and opt to maximize the prediction loss of g with a γ × 'decayed' intermediate-level perturbation ( i.e. , ( h ( x + ∆x ) -h ( x )) /γ which is regarded as the directional guide v ). That is,

Since the method seems decays the intermediate-level perturbation from the benign features, we call it intermediate-level perturbation decay (ILPD) . Apparently, setting γ = 1 means attack without the perturbation modification and it makes the method equivalent to the baseline multi-step attack. While, enforcing γ > 1 leads to decayed perturbation as desired. The operation of h ( x + ∆x ) /γ + ( γ -1) h ( x ) /γ ) resembles the mixup augmentation [51], except there is no randomness for γ and γ is suggested to be relatively large to encourage larger perturbation magnitude and larger loss.

Figure 2 demonstrates the performance of our ILPD, together with that of ILA. Solid dots ● , ● , . . . , ● represent achievable performance of our ILPD with 1 /γ ∈ { 1 . 0 , 0 . 9 , . . . , 0 . 1 } , while the blue cross mark ✖ represents the performance of ILA in the figure. From the vertical axis, it can be seen that the ILPD adversarial examples are capable of tricking the victim model into showing significantly larger prediction loss ( 4 . 08 -> 4 . 54 ), despite possessing lower perturbation magnitude in the feature space, comparing with ILA examples. Given γ in the concerned range, ILPD-obtained perturbation directions are very effective in the sense of being adversarial. See the shaded area which illustrates how the attack performance varies with the perturbation magnitude along with these directions in the middle layer.

6.5

of thevictimmodel

ILA

6.0

ILPD, 1/v = 1.0

5.5

ILPD, 1/v = 0.9

Morepowerfulattack

5.0

...

ILPD, 1/v = 0.1

4.5

Predictionloss

4.0

3.5

3.0

2.5

2.0

7

8

9

10

11

Intermediate-levelperturbationmagnitude

It is worth noting that we normally suggest a relatively large γ with γ ≥ 2 ( i.e. , 1 /γ ≤ 0 . 5 ). The formulation of ILPD leads to a γ × larger perturbation magnitude anyway, than that of ( h ( x + ∆x ) -h ( x )) /γ which is regarded as the directional guide v . However, given g being non-linear in practice, a γ × amplified intermediate-level perturbation does not truly lead to γ × larger prediction loss of g . In particular, the obtained loss is more difficult to be guaranteed especially when γ is extremely large, e.g. , γ ≫ 10 or, equivalently, 1 /γ ≪ 0 . 1 . Therefore, we simply apply with 0 . 1 ≤ 1 /γ ≤ 0 . 5 in our main experiments in Section 5. Comprehensive ablation studies are deferred to Section 5.4.

4 Analysis from the Gradient Alignment Perspective

Recall that the diversity in input gradients across models is the main obstacle to adversarial transferability, thus, in this section, we analyze ILPD from the perspective of gradient alignment between the substitute and victim models, in order to shed more light on its effectiveness.

4.1 Impacting Factors of Gradient Alignment

Aiming at compromising an unknown victim model f t , a baseline transfer-based attack ( e.g. , I-FGSM) can generate adversarial example on a white-box substitute model f . In fact, if ∇ x L f ( x + ∆x j , y ) ≈

(a) Similarity of input gradients

(b) Success rate

70

0.32

65

s

60

0.30

50

60

nd

0.28

rat

40

55

ine

=

0.26

Success

Success

Cosi

betwe

a

D

0.24

I-FGSM

ReplaceVz,L(zg，y)

30

20

I-FGSM

Replace Vz,L(zg,y)

50

0.22

Replace{D}

Replace{D;}

45

Replaceboth

10

Replaceboth

40

0

20

40

60

80

100

0

20

40

60

80

100

60

80

100

Stepj

Stepj

Stepj

∇ x L f t ( x + ∆x j , y ) , then the obtained adversarial examples will be as powerful on the victim model. Yet, the input gradients are often not well-aligned, and, as depicted in Figure 3(a), after some initial iterations, the input gradient of L f ( x + ∆x j , y ) and that of L f t ( x + ∆x j , y ) may be increasingly different as j increases, which is probably a major obstacle to the adversarial transferability of multi-step attacks.

Let us delve deep into this. For simplicity of notations, the activation functions ϕ i is restricted to ReLU. Then, the input gradient ∇ x L f ( x + ∆x j , y ) can be written as the product of three terms:

following the chain rule, where D i ( z ( j ) h ) provides a binary matrix whose diagonal entry is set to 1 if it corresponds to a nonzero activation within the i -th layer, and it is set to 0 if otherwise. If there exist max pooling layers, they can be similarly formulated, and the gradient through these layers can also be given as a binary matrix in { D i } .

The input gradient of a victim model f t ( i.e. , ∇ x L f t ( x + ∆x j , y ) ) can be similarly formulated as in Eq. (4). The diversity between ∇ x L f ( x + ∆x j , y ) and ∇ x L f t ( x + ∆x j , y ) probably comes from some difference in the activation masks in { D i } and/or diversity in the gradients of L w.r.t. the logits ( i.e. , ∇ z g L ( z g , y ) where j is dropped for simplicity of notation). Note that the gradients of h are not concerned as it has been assumed in prior work that different models might share a similar h .

By replacing 1) activation masks in { D i } and/or 2) the gradient of L w.r.t. the logits ( i.e. , ∇ z g L ( z g , y ) ) with those obtained on the victim model, we are capable of analyzing how much impact they actually make. An experiment is thus conducted, using ten pairs of substitute and victim models with the VGG-19 architecture. Figure 3 shows that the transferability can be remarkably improved by replacing any of them. Replacing ∇ z g L ( z g , y ) seems more effective at initial iterations while replacing the activation masks leads to slightly more improvement at the end. Given the cross-entropy loss which evaluates the discrepancy between the prediction probabilities and the one-hot ground-truth vector y , we have ∇ z g L ( z g , y ) = p -y where p = s ( z g ) is a vector containing all prediction probabilities and s is the softmax function. That said, at initial iterations, the difference in p between the models is a more influential factor, and, at later iterations, the difference in { D i } becomes dominant.

As the adversarial example is optimized on the substitute model, the prediction of a I-FGSM example is very likely to be less incorrect on the victim model even at the first iteration. Difference in p across models leads to difference in the input gradient, and it is possible that the difference become more and more severe, forming a vicious circle in some sense.

4.2 Analysis of ILPD and Some Other Attacks

Recall that the input gradient obtained by ILPD at the j -th iteration is

(a) Similarity of input gradients

(b) Success rate

70

65

0.30

I-FGSM

gradients

ILPD

60

60

0.28

similarity

50

ndu!

0.26

40

E

55

Cosine

Success

Success

a

0.24

30

50

betwee

0.22

20

I-FGSM

45

10

ILPD

40

0

20

40

60

80

100

0

20

40

60

80

100

60

80

100

Stepj

Stepj

Stepj

with the gradient of L w.r.t. to some less adversarial intermediate-level perturbation ˜ z ( j ) h . Less adversarial intermediate-level perturbations leads to less incorrect predictions, which probably aligns better with that of the victim model.

Figure 4 shows that ILPD leads to more aligned input gradient across models, especially after the first 20 iterations. The attack success rates are compared in Figure 4(b), and it can be seen that, thanks to the improved gradient alignment, ILPD is capable of generating more transferable adversarial examples in comparison to I-FGSM, achieving +4 . 00% absolute gains at the 100 -th iteration. More detailed analyses show that the ILPD examples lead to activation masks and prediction probabilities better aligned with those of the victim model.

In addition to our ILPD, some other intermediate-level methods can also be seen as beneficial to gradient alignment. In particular, for ILA [20], we have

where the Taylor series is adopted to obtain the approximation. Therefore, optimizing Eq. (2) at the j -th step of an iterative optimizer can be considered as adopting the gradient w.r.t. z (0) h as a replacement of the gradient w.r.t. z ( j ) h . Without injecting noise or augmenting data at each iteration, ILA++ [26, 16] and NAA [52] can also be approximately regarded as replacing the gradient w.r.t. z ( j ) h with the gradient w.r.t. z (0) h .

In addition to these efforts, Guo et al. proposed LinBP [15] which back-propagates linearly in later layers. It can be regarded as replacing each D i with an identity matrix for computing input gradients. Though effective, it mainly takes care of { D i } and somehow ignores the difference of ∇ z g L ( z g , y ) across models.

5 Experiments

Experimental settings will be introduced in Section 5.1. We will compare our solution with a variety of state-of-the-arts to evaluate its effectiveness in Section 5.2. We combine our solution with some existing methods to further improve the adversarial transferability in Section 5.3, and we perform some ablation studies in Section 5.4.

5.1 Experimental Settings

Our experiments were conducted on CIFAR-10 [24] and ImageNet [35] under ℓ ∞ constraint in the black-box setting. We focus on untargeted attacks and use I-FGSM attack as the back-end method, just like in previous work [46, 11, 28, 42]. We set the perturbation budget to ϵ = 4 / 255 and 8 / 255 for attacks on CIFAR-10 and ImageNet, respectively. Further, on CIFAR-10, we used a VGG-19 with batch normalization [36] as the substitute model, and a different VGG-19 with batch normalization,

a ResNet-18 [18], a PyramidNet [17], GDAS [10], a WRN-28-10 [50], a ResNeXt-29 [47], and a DenseNet-BC [19] 2 as the victim models, following some prior work [15, 26, 16]. All victim models (except for the VGG-19 which was trained by ourselves) were pre-trained by others and directly collected for experiments. On ImageNet, we used a ResNet-50 [18] model as the substitute model, and collect 13 victim models, including a different ResNet-50 [18], a VGG-19 [36], a ResNet-152 [18], a Inception v3 [37], a DenseNet-121 [19], a WRN-101 [50], a RepVGG-B1 [9], a MLP Mixer-B [38], a ConvNeXt-B [31], a ViT-B [13], a DeiT-B [39], a Swin-B [30], and a BEiT-B [3] 3 .

The image size of adversarial examples was 224 × 224 on ImageNet and 32 × 32 on CIFAR-10, without crop, following prior work [28, 42]. We performed adversarial attacks on all test data in CIFAR-10 and 5000 randomly sampled examples from the ImageNet validation data. The adversarial examples obtained at each iteration of the attack should be clipped to [0, 1], in order to ensure that only valid images are fed to DNNs. We run 100 iterations with a step size of 1 / 255 for all attack methods on both CIFAR-10 and ImageNet in this section, and we set a smaller step size of ϵ/ 100 for Section 4 as we care more about the trend of overfitting before exhausting the perturbation budget. ILPD was performed at the output of the fourth VGG block for VGG-19 on CIFAR-10 and the output of the last building block of the second ResNet meta layer for ResNet-50 on ImageNet, with γ tuned in the range satisfying 0 . 1 ≤ 1 /γ ≤ 0 . 5 . The advanced version of ILA++ [16] and NAA [52] are adopted as competitors. Since both methods inject some random transformations to the benign examples, when computing gradients, we also add Gaussian noise with a standard deviation of 0.05 to the benign examples at each iteration, for fair comparison. For competitors, we directly adopt their official implementations and hyper-parameters, and we attack on the same CIFAR-10 and ImageNet images for all methods. All experiments are performed on an NVIDIA V100 GPU.

5.2 Comparison with State-of-the-arts

The comparison results are summarized in Table 1 and 2. For ImageNet, 13 victim models, including four vision transformers, one multi-layer perceptron model, and a model that has the same architecture of the substitute model, are adopted. For CIFAR-10, 7 victim models are adopted. We compare our method with a gradient stabilization method ( i.e. , MI-FGSM [11]), some input augmentation methods ( i.e. , DI 2 -FGSM [46], TI-FGSM [12], SI-FGSM [28], and the recent Admix [42]), compelling intermediate-level attacks ( i.e. , ILA++ [16] and NAA [52]), some back-propagation modification methods ( i.e. , SGM [45] and LinBP [15]), and a very recent method called TAIG [21].

Experiment results in the tables show that our method outperforms all these competitors by large margins. Specifically, it can be seen that with the assistance of our ILPD, it is possible to achieve an average success rate of 57.06% on ImageNet, which outperforms the second best ( i.e. , ILA++) by +10.07% . On CIFAR-10, we achieve 62.82% , which outperforms the second best by +3.88% .

Attacking some newly designed models is challenging when using ResNet as the substitute model. The I-FGSM baseline can only achieve a success rate of less than 10% against these models (see the right half of Table 1) and even < 5% against some vision transformers. However, our method

achieves impressive improvements in attacking these models. For instance, compared with the IFGSM baseline, when attacking the MLP-Mixer and ConvNeXt models, our ILPD obtains +20 . 82% and +34 . 86% absolute gains, respectively, and when attacking the vision transformers, our ILPD obtains an absolute gain of +19 . 14% .

Attack robust models. We also tested our method on the task of attacking robust models. We collected a robust Inception v3 and a robust EfficientNet-B0 from the timm [44] repository, and obtain a robust ResNet-50 and a robust DeiT-S [40] from Bai et al. [2]'s open-source repository 4 as the victim models. Adversarial examples generated on the ResNet-50 substitute model were applied for this experiment. Our ILPD is compared with

DI 2 -FGSM, NAA, and ILA++, which are the most competitive methods in Table 1, and we summarize the results in Table 3. It shows that our method outperforms competitors in attacking these robust models and getting +4 . 13% improvement, compared with NAA, which is the second best method. In addition to the robust models, we also test our method on two advanced defense methods, i.e. , random smoothing [7] and NRP [33]. Specifically, our ILPD achieves a success rate of 12 . 90% in attacking a smoothed ResNet-50 on ImageNet, while the second best method (NAA) and the baseline show 11 . 84% and 7 . 20% , respectively. For NRP, the adversarial examples generated by our method can achieve an average success rate of 20 . 03% after purification. Meanwhile, the second best method (DI 2 -FGSM) achieves 14 . 54% , and the baseline achieves 9 . 39% .

5.3 Combination with Existing Methods

Our method can be readily combined with other attacks. In this section, we evaluate the combination performance. We tried combining our method with different sorts of attacks, including a gradient stabilization method, i.e. , MI-FGSM [11] and two input augmentation methods, i.e. , DI 2 -FGSM [46] and Admix [52]. The results are reported in Table 4. It shows that the adversarial transferability can indeed be further improved by combining our method with these prior arts. Specifically, when combining with DI 2 -FGSM, the best average success rate, i.e. , 63.23% , is obtained.

(a) ResNet-50

(b) VGG-19

55

60

50

55

SuccessRate(%)

(%)

45

SuccessRate(

50

40

45

35

40

30

35

LinBP

ILA++

LinBP

ILA++

25

NAA

ILPD

30

NAA

ILPD

01-1 1-2 1-3 2-1 2-2 2-3 2-4 3-1 3-2 3-3 3-4 3-5 3-6 4-1 4-2 4-3

1

2345

6

7

8

9

10111213141516

Position

Position

5.4 Ablation Study

The effect of position. As an intermediate-level attack, our method needs to choose a position to split f in h and g . Such a position selection affects the performance of all intermediate-level attacks. We vary the position and compare our method with ILA++ [16], NAA [52], and LinBP [15] in Figure 5. The performance on ImageNet and CIFAR-10 are both tested, using ResNet-50 and VGG-19 as the substitute models, respectively. Apparently, our ILPD demonstrates consistently higher adversarial transferability on all choices of the position. In particular, our method significantly outperforms ILA++, NAA, and LinBP when choosing a lower layer to split f into h and g . When using ResNet-50 as the substitute model, choosing any of the first eleven layers ( i.e. , from '0' to '3-3') can achieve an average success rate of more than 53% on ImageNet, even simply choosing the first layer will lead to an average success rate of 53 . 12% , which is already superior to all competitors in Table 1.

The effect of γ . We evaluate ILPD with varying γ on ImageNet and CIFAR-10 in Figure 6. The victim models are the same as the ones in Table 1 and Table 2. It can be seen that using 1 /γ < 1 always leads to improved performance compared with the baseline (which is equivalent to adopting 1 /γ = 1 ). On ImageNet, the optimal performance is obtained with 1 /γ = 0 . 1 ( i.e. , γ = 10 ) and ResNet-50, while, for CIFAR10, 1 /γ = 0 . 3 ( i.e. , γ = 10 / 3 ) is more effective. That is, on ImageNet, we aim to maximize the prediction loss of a perturbation which is cut by

(a) ImageNet

(b) CIFAR-10

(%)

60

Success Rate(%)

64

Rate

50

60

Success

40

56

30

52

20

48

0.0

0.20.40.60.81.0

0.0

0.20.40.60.81.0

1/y

1/y

Figure 6: How the average success rate changes with γ using (a) ImageNet trained ResNet-50 and (b) CIFAR-10 trained VGG-19 as the substitute models.

10 × in the middle layer. It is possible that employing γ > 10 leads to even better performance on ImageNet, yet we do not aim to carefully tune γ and would like to leave it to future work.

6 Conclusion

In this paper, we have revisited some intermediate-level attacks and shown that, limited by their twostage training scheme, the obtained intermediate-level perturbations will deviate from the directional guides and such a deviation largely impairs the attack performance of the crafted adversarial examples. In order to overcome this limitation, we have developed a method that performs attacks through a single stage of optimization and guarantees the obtained intermediate-level perturbation possesses a larger magnitude when compared with a directional guide and exactly follows its direction that can effectively increase the prediction loss of the model. The developed method is formed as adopting intermediate-level perturbation decay when computing gradients. Extensive experimental results on ImageNet and CIFAR-10 have shown that our method outperforms state-of-the-arts by large margins in attacking various victim models, including convolutional models, multi-layer perceptron model, vision transformers, and robust models. Moreover, we have shown that our method can be easily combined with prior methods to craft more transferable adversarial examples.

7 Acknowledgment

This material is based upon work supported by the National Science Foundation under Grant No. 1801751 and 1956364.

References

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning , pages 284-293. PMLR, 2018.

Yutong Bai, Jieru Mei, Alan Yuille, and Cihang Xie. Are transformers more robust than cnns? In Thirty-Fifth Conference on Neural Information Processing Systems , 2021.

Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations , 2021.

Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR , 2018.

Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decisionbased attack. arXiv preprint arXiv:1904.02144 , 2019.

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security , pages 15-26. ACM, 2017.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning , pages 1310-1320. PMLR, 2019.

Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning , pages 2196-2205. PMLR, 2020.

Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13733-13742, 2021.

Xuanyi Dong and Yi Yang. Searching for a robust neural architecture in four gpu hours. In CVPR , 2019.

Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR , 2018.

Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4312-4321, 2019.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015.

Yiwen Guo, Qizhang Li, and Hao Chen. Backpropagating linearly improves transferability of adversarial examples. In NeurIPS , 2020.

Yiwen Guo, Qizhang Li, Wangmeng Zuo, and Hao Chen. An intermediate-level attack framework on the basis of linear regression. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Dongyoon Han, Jiwhan Kim, and Junmo Kim. Deep pyramidal residual networks. In CVPR , pages 5927-5935, 2017.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , 2016.

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR , 2017.

Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transferability with an intermediate level attack. In ICCV , 2019.

Yi Huang and Adams Wai-Kin Kong. Transferable adversarial attack based on integrated gradients. arXiv preprint arXiv:2205.13152 , 2022.

Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In ICML , 2018.

Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. In ICLR , 2019.

Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In ICLR , 2017.

Qizhang Li, Yiwen Guo, and Hao Chen. Yet another intermediate-leve attack. In ECCV , 2020.

Qizhang Li, Yiwen Guo, Wangmeng Zuo, and Hao Chen. Making substitute models more bayesian can enhance transferability of adversarial examples. In The Eleventh International Conference on Learning Representations , 2023.

Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv:1908.06281 , 2019.

Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In ICLR , 2017.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10012-10022, 2021.

Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11976-11986, 2022.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR , 2018.

Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A selfsupervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 262-271, 2020.

Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 , 2016.

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. IJCV , 2015.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR , 2015.

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR , 2016.

Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing Systems , 34:24261-24272, 2021.

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning , volume 139, pages 10347-10357, July 2021.

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning , pages 10347-10357. PMLR, 2021.

Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, and ShinMing Cheng. Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks. In AAAI , 2019.

Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 16158-16167, 2021.

Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren. Feature importanceaware transferable adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7639-7648, 2021.

Ross Wightman. Pytorch image models. https://github.com/rwightman/ pytorch-image-models , 2019.

Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Rethinking the security of skip connections in resnet-like neural networks. In ICLR , 2020.

Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In CVPR , 2019.

Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR , 2017.

Ziang Yan, Yiwen Guo, Jian Liang, and Changshui Zhang. Policy-driven attack: learning to query for hard-label black-box adversarial examples. In ICLR , 2021.

Ziang Yan, Yiwen Guo, and Changshui Zhang. Subspace attack: Exploiting promising subspaces for query-efficient black-box attacks. In NeurIPS , 2019.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC , 2016.

Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations , 2018.

Jianping Zhang, Weibin Wu, Jen-tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su, and Michael R Lyu. Improving adversarial transferability via neuron attribution-based attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14993-15002, 2022.

A Experiment Results of Using Other Substitute Models

We compare our method with other intermediate-level attacks using various substitute model on ImageNet. The comparison results are summarized in Table 5. Three vision transformers and one multi-layer perceptron model are considered as the substitute models, including a ViT-B [13], a DeiT-B [39], a Swin-B [30], and a MLP Mixer-B [38]. The victim models are the same as those in Table 1. For ILPD, h is always split to be the first two blocks of these models and we fix 1 /γ = 0 . 1 . For NAA and ILA++, we tested the results of all possible intermediate layer selections and report the best results among them. The average success rates are compared as follows, and the victim models are the same as those in Table 1 of our paper. Experiment results in the table show that our method outperforms all these competitors by large margins.

Substitute Method

ResNet

-50

ResNet

-152

DenseNet

-121

WRN

-101

Inception

v3

RepVGG

-B1

VGG

-19

Mixer

-B

ConvNeXt

-B

ViT

-B

DeiT

-B

B

BEiT

-B

Average

ViT-B

ILA++

22.12%

17.44%

28.20%

21.28%

28.52%

29.30%

32.22%

47.06%

19.00%

97.22%

40.12%

29.50%

43.34%

29.84%

NAA

31.90%

27.64%

45.74%

35.36%

46.56%

44.98%

46.36%

58.32%

26.76%

94.58%

60.62%

40.52%

57.52%

43.52%

ILPD

34.96%

32.64%

43.78%

37.38%

44.02%

44.60%

44.36%

58.82%

36.38%

96.42% 61.92% 54.16% 66.54%

46.63%

DeiT-B

ILA++

20.92%

14.40%

24.76%

18.60%

25.24%

29.46%

33.00%

33.94%

18.24%

27.22%

87.54%

18.38%

19.72%

23.66%

NAA

29.74%

25.04%

36.52%

30.26%

38.26%

39.18%

39.42%

45.78%

31.38%

39.42%

75.32%

31.54%

37.62%

35.35%

ILPD

40.14%

36.20%

42.02%

36.80%

40.82%

44.80%

43.76%

53.34%

45.68%

51.82%

81.74%

46.36% 49.64%

44.28%

Mixer-B

ILA++

22.10%

14.46%

24.52%

19.08%

27.96%

25.48%

27.64%

96.64%

15.08%

22.76%

23.76%

12.68%

14.24%

20.81%

NAA

33.28%

23.24%

42.92%

28.86%

41.56%

41.48%

43.88%

92.62%

26.54%

42.76%

52.88%

24.64%

29.06%

35.93%

ILPD

39.84%

30.84%

42.56%

34.40%

41.88%

44.44%

42.56%

91.56%

40.80%

49.88% 58.26% 36.38% 38.74%

41.72%

Swin-B

ILA++

12.68%

8.36%

13.44%

9.22%

14.30%

18.92%

22.78%

12.64%

19.84%

9.14%

7.64%

83.96%

9.80%

13.23%

NAA

16.36%

13.10%

22.72%

16.54%

22.90%

26.78%

28.84%

22.04%

28.80%

17.70%

19.94%

71.90%

21.16%

21.41%

ILPD

31.04%

27.44%

32.66%

25.88%

28.06%

43.08%

42.52%

32.06%

55.48%

40.14% 35.42% 89.56% 47.42%

36.77%

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the limitations of intermediate-level attacks, highlighting how their two-stage training scheme leads to perturbations that deviate from directional guides, significantly impairing attack performance. To overcome this limitation, the authors propose a single-stage optimization method that ensures intermediate-level perturbations have larger magnitudes and precisely follow the directional guides, effectively increasing model prediction loss. Extensive experiments on ImageNet and CIFAR-10 demonstrate that the proposed method outperforms existing techniques when attacking various victim models. Furthermore, the paper shows that the proposed method can be easily combined with prior approaches to craft more transferable adversarial examples.\n\nSTRENGTHS:\nThis paper recognizes the limitations of current intermediate-level attacks and proposes a novel method named ILPD. This method crafts adversarial examples using a single stage of optimization, encouraging the intermediate-level perturbation to be both in an effective adversarial direction and possess a significant magnitude simultaneously. A large number of experiments demonstrate that ILPD outperforms SOTA methods by large margins.\n\nWEAKNESSES:\nThe method seems sensitive to $\\gamma$ and the intermediate layer to split h and g.\n\nQUESTIONS:\n1 Line 200-201. It is reasonable that the gradients of $h$ are not concerned, but why $W_{\\beta+1}$ (after the intermediate layer) not considered? \n\n2 If change the surrogate model, do you need to adjust the intermediate position and $\\gamma$?\n\nLIMITATIONS:\nThe method appears to be sensitive to the parameters $\\gamma$ and the choice of intermediate layer for splitting h and g. However, in real-world scenarios, it is often impractical to tune these parameters effectively based on the victim models.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose an intermediate-level attack method with a single stage of optimization to improve attack transferability. When computing gradients, it adopts intermediate-level perturbation decay to process a larger magnitude. The experiment validations on the ImageNet and CIFAR-10 datasets are given to demonstrate its effectiveness to improve the attack transferability.\n\nSTRENGTHS:\n1. The proposed method presents a single stage with a larger magnitude rather than two stage training.\n2. The proposed method is a pluggable module that can be deployed in other attack methods.\n3. The experiments are thorough with various victim models, including CNNs, MLP, vision transformers, and robust models.\n\nWEAKNESSES:\n1. From Figure 1, there is no obvious difference between the victim model and the substitute model. The reviewer is confused that these two figures are less related to the statement in Section 3.1. \n2. The construction of this paper is disordered. The authors introduce the existing intermediate-level attack method with the proposed method, which covers the contribution of the proposed method. The authors should highlight its own contribution.\n3. For the experiments, the authors only use the ResNet-50 model as the substitute model to test the attack transferability on other models. However, a cross-model evaluation could be better, i.e., using MLP or transformers to test on other types of models. \n4. The writing of this paper needs to be improved. \n- For the introduction, the authors do not provide a brief introduction of the proposed method. \n- For the related work, the white-box attack is less related to this paper. And existing intermediate-level attack methods are missed.\n- There are also some typos.\na) In line 31, be discuss -> be discussed.\nb) In line 327, there exists two duplicate ‘to all competitors’.\n\nQUESTIONS:\n1. Figure 1 does not give an intuitive visualization about the problem of existing intermediate-level attacks. The motivation of this paper should be stated clearly.\n2. Lack of related works with existing intermediate-level attack methods. The whole construction of this paper should be improved.\n3. How about the transferability with other types of models as the substitute model?\n4. Fix the typos and polish this paper again.\n\nLIMITATIONS:\nLimitations not clearly elaborated.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an approach to enhancing the transferability of adversarial examples in a black-box scenario. Previous studies have typically followed a two-stage process involving the derivation of effective guiding directions and subsequently maximizing the perturbation magnitudes accordingly. Unfortunately, this approach often deviates from the intended guides, resulting in suboptimal performance. In contrast, the authors introduce a one-stage method that utilizes a decayed perturbation function and an analysis tool to investigate the sources of deviation. Through extensive experiments, the authors demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS:\n1. The paper is well-written, and the motivation is sound.\n2. The analysis provides insights and could serve as a tool for future work.\n3. Extensive experiments verify the effectiveness of the proposed method.\n\nWEAKNESSES:\n1. Despite the empirical effectiveness, the proposed method is not well-explained. Or, at least, it is unclear to me. The authors may like to elaborate more on how the \"dual\" equation (Eq. 3) is derived and provide insights into the equation. There seem to be some alternatives. For example, one can maximize $\\mathcal{L}(h(x+\\Delta x))$. What would be the role of $h(x)$ in this formulation?\n\n2. Following the previous point, the proposed method looks incremental or empirical finding to me if it lacks proper insights.\n\n3. In L152, what is the link to the mixup? The authors mentioned it, but it needed further discussion in depth.\n\n4. In Section 4, the analysis lacks the baseline ILA. The audiences will be interested in whether the proposed method improves the prior work as motivated by the authors in the introduction.\n\nQUESTIONS:\n1. The experiment settings seem to be different from prior work. What does the *different* mean in Sec 5-1? Why didn't the authors report the same architecture attack performance?\n2. Meanwhile, the performance of ILA++ reported in the paper is different from the original paper. What would be the root reason for it?\n\nLIMITATIONS:\n1. The authors do not report the variance of the experimental results.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors first introduce the observation that traditional Intermediate-Level Attack (ILA) and some subsequent works deviate from the directional guides, resulting in sub-optimal transferability. With such observation, the authors propose Intermediate-Level Perutbration Decay (ILPD), a one-staged optimization that decays the intermediate perturbation strength in return for amplified directional guide, and thus higher adversarial transferability.\nExperiments on ImageNet and CIFAR-10 show that the proposed ILPD method outperforms SOTA transfer-based attacks in different types of models as well as robust models.\n\nSTRENGTHS:\n1. Claims and speculations in the paper are well grounded by theoretical analysis as well as empirical studies. From the hypothesis of directional deviation in Figure 1, then the introduction of the ILPD and verification in Figure 2, studies and discussion are organized layer by layer, which is an enlightening experience to read through.\n2. Apart from showing the analysis of ILPD, the authors extend the discussion to previous works such as ILA++, NAA and LinBP. I particularly like the authors’ attempt to replace $D_i$ and $\\nabla_{z_g}L(\\mathbf{z}_g, y)$ to study the transferability in the gradient aspect.\n3. Although the modification the authors propose is a simple math trick by scaling up the influence from the directional guide, the optimization falls back to one-staged and no longer resembles lLA, which is sufficiently novel.\n4. The experiments are conducted in a large variety of models, both normal and robust, both CNNs and ViTs. Ablation study and hyper-parameters study are performed to understand the method in an in-depth manner.\n\nWEAKNESSES:\nI could not spot any major weaknesses in this paper. The following points are mostly minor and understandable:\n\n1. Although the authors perform evaluations on robust models, the choice of models does not align with what used to be evaluated in the previous works (i.e. the NeurIPS 2017 Competition, some models with ensemble adversarial training, etc.). \n\n2. The number of attack iterations is fixed to be 100, which is quite large considering most baselines perform only 10-20 iterations in their experiments. Plus the observation in Figure 4(b), does it imply the proposed method has a slow convergence and requires more iterations to yield transferable attacks?\n\n3. The performance of ILPD seems to be quite sensitive to the choice of $\\gamma$. For example, Figure 2 suggests setting $1/\\gamma \\approx 0.5$ might probably obtain good transferability. However, in Figure 6(a), setting $1/\\gamma = 0.5$ can have a success rate >15% lower than the optimal choice. For a new dataset, it is hard to determine a good starting $\\gamma$ due to the difference in Figure 6(a) and 6(b).\n\n#### Grammar mistakes:\n- Line 148: the method **seems decays**\n- Line 212: **a** I-FGSM example\n- Line 214:  the difference **become**\n- Line 254 **a** Inception v3\n\nHere are only a few. The authors are recommended to check the paper thoroughly for the remainings.\n\n---\n\nIn summary, this paper is very well-written and provides a lot of evidence to support its claims. The only weaknesses that I could identify are largely outweighed by its strengths. Therefore I recommend accepting this paper.\n\nQUESTIONS:\n1. Regarding the ILA++ baseline, I would like more clarification on the base attack in ILA++ for directional guidance. Since two attacks are applied in series, how are they balanced to sum up to 100 iterations?\n\nLIMITATIONS:\nWeakness #3 is briefly discussed in the paper. To me, the search for hyper-parameters will be the major limitation of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper notes that ILA (intermediate level attack), which maximizes projection onto a guide attack, suffers when the direction of the attack ends up differing from the direction of the guide, even when the projection is large. Thus, they propose that instead of allowing the direction to vary from a guide, we should simply look for attacks that have large intermediate magnitudes.\n\nThis motivates their approach (called Intermediate Level Perturbation Decayed). They then demonstrate that this approach results in significantly improved performance across a wide range of models.\n\nSTRENGTHS:\nOverall, I think the issue the authors identify with ILA is well motivated, and the improvements are fairly significant. In particular, the experiments investigating gradient alignment are fairly convincing about what ILPD is doing, as well as the analysis of existing attacks.\n\nWEAKNESSES:\nMy primary issue with this paper was that I found the presentation confusing. Although the motivation for why we might prefer an approach that doesn't differ in direction from an original \"guide\" attack is solid, the actual motivation of the approach used was much less clear to me, and not particularly intuitively presented. From my understanding, the attack is essentially \"simulating\" reducing the strength of the perturbation at some intermediate stage, but this interpretation isn't presented in the paper.\n\nIn particular, upon first reading, this line was particularly confusing to me.\n\n> seeks intermediate-level perturbations to be in an adversarial direction and to possess larger norms (or say larger magnitudes) compared to a directional guide in the same direction simultaneously\n\nSince if there was a perturbation of the input that could lead to a strictly larger perturbation along the same direction as a guide, I would expect that to be already found by standard optimization procedures. I believe the part I missed from my first reading is that there is no \"directional guide\" - in comparison with ILA (a 2 step procedure), this is a 1 step attack.\n\nAlthough I think I now have some intuitive understanding of the paper's attack, it doesn't quite match the author's explanation, and I think the paper would benefit from improving the clarity of section 3.2\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}