Rethinking the Backward Propagation for Adversarial Transferability
Authors: Xiaosen Wang
Kangheng Tong
Kun He
Venue: NeurIPS 2023 poster (2023)
Keywords: Adversarial examples
Convolutional neural networks
Adversarial transferability
Backward propagation
TL;DR: We propose a new method to enhance the transferability of adversarial examples by modifying the gradient calculation of non-linear layers, ReLU and maxpool.

Abstract: Transfer-based attacks generate adversarial examples on the surrogate model, which can mislead other black-box models without access, making it promising to attack real-world applications. Recently, several works have been proposed to boost adversarial transferability, in which the surrogate model is usually overlooked. In this work, we identify that non-linear layers (e.g., ReLU, max-pooling, etc.) truncate the gradient during backward propagation, making the gradient w.r.t. input image imprecise to the loss function. We hypothesize and empirically validate that such truncation undermines the transferability of adversarial examples. Based on these findings, we propose a novel method called Backward Propagation Attack (BPA) to increase the relevance between the gradient w.r.t. input image and loss function so as to generate adversarial examples with higher transferability. Specifically, BPA adopts a non-monotonic function as the derivative of ReLU and incorporates softmax with temperature to smooth the derivative of max-pooling, thereby mitigating the information loss during the backward propagation of gradients. Empirical results on the ImageNet dataset demonstrate that not only does our method substantially boost the adversarial transferability, but it is also general to existing transfer-based attacks. Code is available at https://github.com/Trustworthy-AI-Group/RPA.

Rethinking the Backward Propagation for Adversarial Transferability
Xiaosen Wang",, Kangheng Tong²*, Kun Heat
'Huawei Singular Security Lab
School of Computer Science and Technology, Huazhong University of Science and Technology {xiaosen, tongkangheng, brooklet60)@hust. eerausst.dduc.. en
Abstract
Transfer-based attacks generate adversarial examples on the surrogate model, which mislead other black-box models without access, making promising attack real-world applications. Recently, several works have been proposed to boost ad- versarial transferability, in which the surrogate model is usually overlooked. In this work, we identify that non-linear layers (e.g. ReLUU, max-pooling, etc.) truncate the gradient during backward propagation, making the gradient w.r.f. input image impre- cise to the loss function. We hypothesize and empirically validate that such trunca- tion undermines the transferability of adversarial examples. Based on these findings, we propose a novel method called Backward Propagation Attack (BPA) to increase the relevance between the gradient w.r.t. input image and loss function so as generate adversarial examples with higher transferability. Specifically, BPA adopts non-monotonic function as the derivative of ReLU and incorporates softmax with temperature to smooth the derivative of max- max-pooling, thereby mitigating the infor- mation loss during the backward propagation of gradients. Empirical results on the ImageNet dataset demonstrate that not only does our method substantially boost the adversarial transferability, but also general to existing transfer-ban based attacks. Code is available at https: btttps://githbb com wwwoo//rusttooth-oooo -Iw--AAI-------- RPA
Introduction
Deep Neural Networks (DNNs) have been widely applied in various domains, such as image recog- nition 37 B 16), object detection (34] face verification [52 41,, etc. However, their suscepti- bility to adversarial examples (39 10. which are carefully crafted by adding imperceptible per- turbations to natural examples, has raised significant concerns regarding their security. In recent years, the generation of adversarial examples, aka adversarial attacks, has garnered increasing at-
tention 31 21 6, 55 24 the research community. Notably, there has been a significant advancement in the efficiency and applicability of adversarial attacks [19 making them increasingly viable real-world scenarios.
57
59
51
By exploiting the transferability of adversarial examples across different models [29]. transfer- based attacks generate adversarial examples on the surrogate model fool the target models 57
43
58). Unlike other types of attacks 2 19 47) transfer-basse attacks do not require direct access to the victim models, making them particularly applicable for attacking online interfaces. Consequently, transfer-based attacks have emerged prominent branch of adversarial attacks. However, it is worth noting that the early white-box attacks [10 31 21 often exhibit poor transferability despite demonstrating superior performance within the white-box setting.
"The first authors contribute equally.
Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

To this end, different techniques have been proposed to enhance adversarial transferability. such
as cimentummbased attacks 66
27
43
46
59), imput transformations [57
45
30
8
48
42].
advanced objective functions 17
54
49
and model-related attacks [23
53
12
50
Among
these techniques, model-related attacks are particularly valuable due to their ability to exploit the characteristics of surrogate models. Model-related attacks offer unique perspective on adversarial attacks by leveraging the knowledge gained from surrogate models, which can also shed new light on the design of more robust models. In despite of their potential significance, model-related attacks have been somewhat overlooked compared to other types of transfer-based attacks.
Since transfer-based attacks mainly design various gradient ascend methods to generate ad adversarial examples on the surrogate model, in this work, we first revisit the backward propagation procedure. We find that non-linear layers (e.g., activation function, max- max-pooling, etc.) often truncate the gradient of loss w.r.f. the feature map, which diminishes the relevance of the gradient between the loss and input image. We assume and empirically validate that such gradient truncation undermines the adversarial transferability. Based on this finding, we propose Backward Propagation Attack (BPA), which modifies the calculation for the derivative of ReLU activation function and max-pooling layers during the backward propagation process. With these modifications, BPA mitigates the negative impact of gradient truncation and improves the transferability of adversarial attacks.
Our main contribution can be summarized as follows:
To our knowledge, this the first work that proposes and empirically validates the detrimental effect of gradient truncation on adversarial transferability. This finding sheds new light on improving adversarial transferability and might provide new directions to boost the model robustness.
We propose a model-related attack called BPA, that adopts a non-monotonic function as the derivative of the ReLU activation function and incorporates softmax with temperature to calculate the derivative of max-pooling. With these modifications, BPA mitigates the negative impact of gradient truncation and enhances the relevance of gradient between the loss function and the input. Extensive experiments on ImageNet dataset demonstrate that BPA could significantly boost various untargeted and targeted transfer-base based attacks and outperform the baselines with substantial margin, emphasizing the effectiveness and superiority of our proposed approach.
2
Related Work
In this section, we provide a brief overview of the existing adversarial attacks and defenses.
2.1 Adversarial Attacks
Existing adversarial attacks can be categorized into two groups based on the access to target model, namely white-box attacks and black-box attacks. the white-box setting [10 33 31 2], attackers have complete access to the structure and parameters of the target model. In the black-box setting, the attacker has limited or no information about the target model, making it applicable in the physical world. Black-box attacks can be further grouped into three classes, i.e., score-based attacks [1]. query-based attacks 3 22 47,, and transfer- based attacks 66 57 43 Among the three types of black-box attacks, transfer- based attacks generate adversarial examples on the surrogate model without accessing the target model, drawing increasing interest recently.
Since MI-FGSM 66] integrates momentum into I-FGSM (21 to stabilize the update direction and achieve improved transferability, various momentum- based attacks have been proposed to generate transferable adversarial examples. For instance, NI-FGSM 27 leverages Nesterov Accelerated Gradient for better transferability. VMI-FGSM [43] refines the current gradient using the gradient variance from the previous iteration, resulting in more stable updates. EMI-FGSM [46) enhances the momentum by averaging the gradient of several data points sampled in the previous gradient direction.
On the other hand, input transformations that modify the input image prior to gradient calculation have proven highly effective in enhancing adversarial transfcrability, such as DIM (56). TIM [ SIM 277, Admix (45) SSA (30 and so on. Among these attacks, Admix introduces a small segment of an image from different categories, while SSA applies frequency domain transformations to the input image, both of which have demonstrated superior performance generating transferable adversarial examples.

---

Several studies have explored the utilization of more sophisticated objective functions to enhance transferability in adversarial attacks. ILA [17] employs fine-tuning techniques to increase the similarity of feature differences between the original or current adversarial example and a benign sample. ATA (54) maximizes the disparity of attention maps between a benign sample and an adversarial example. FIA [49 minimizes a weighted feature map in an intermediate layer to disrupt significant object-aware features.
A few works have emphasized the significance of the surrogate model in generating highly transferable adversarial examples. Ghost network (3 attacks a set of ghost networks generated by densely applying dropout at the intermediate features. On the other hand, another line of work focuses on the gradient during backward propagation. SGM [54] adjusts the decay factor to incorporate more gradients from the skip connections of ResNet to generate more transferable adversarial examples. LinBP 12 performs backward propagation in a more linear fashion by setting the gradient of ReI.U as constant of 1 and scaling the gradient of residual blocks. In this work, we find that the gradient truncation introduced by non-linear layers undermines the transferability and modify the backward propagation so to generate more transferable adversarial examples.
2.2 Adversarial Defenses
The existence of adversarial examples poses significant security threat to deep neural networks (DNNs). To mitigate this impact, researchers have proposed various methods, among which adver- sarial training has emerged as widely used and effective approach [TO] 21 31]. By augmenting
the training data with adversarial examples, this method enhances the robustness of trained models against adversarial attacks. For instance, Tramèr al. [40] introduce ensemble adversarial training, technique that generates adversarial examples using multiple models simultaneously, which shows superior performance against transfer-basee attacks.
Although adversarial training is effective, it comes with high training costs, particularly for large-scate scale datasets and complex networks. Consequently, researchers have proposed innovative defense methods as alternatives. Guo et al. [T1] utilize various input transformations such as JPEG compression and total variance minimization to eliminate adversarial perturbations from input images. Xie et al. [56] mitigate adversarial effects through random resizing and padding of input images. Liao et al. (25
propose training high-level representation denoiser (HGD) specifically designed to purify input images. Nasser (32) introduce a neural representation purifier (NRP) by a self-supervised adversarial training mechanism to purify the input sample. Various certified defenses aim to provide a verified guarantee in a specific radius, such as randomized smoothing (RS) [5].
3 Methodology
In this section, we analyze the backward propagation procedure and identify that the gradient truncation introduced by non-linear layers undermines the adversarial transferability. Based on this finding, we propose Backward Propagation Attack (BPA) to mitigate such negative effect and gain more transferable adversarial examples.
3.1 Backward Propagation for Adversarial Transferability
Given an input image x with ground-truth label y. a classifier f with successive layers (e.g.. 2+41 (f)(())) 20 2)) predicts the label f(x) f((2)) y with high probability. Here o(-) is a non-linear activation function (e.g., ReLU) or identity function there is no activation function after é-th layer f.. The attacker aims to find an adversarial example add adhering the constraint of al - =|Ip S e, but resulting in (x * f(x)
for untargeted attack and f(2ade = y for targeted attack. Here is the maximum perturbation magnitude, y/s is the target label, and lp denotes the p-norm distance. For brevity, the following description will focus on non-targeted attacks with
p xc. Let Hx, y; 9) denote the loss function of classifier (e.g., the cross-entropy loss). Existing white-box attacks often solve the following constrained maximization problem using the gradient V 0):
adv
argmax J( y; 0).
(1)
3

---

Based on the chain rule, we can calculate the gradient as follows:
V (z, 0J (z,
dø (o((Fi(z)))
azs+1 OzR
II
(2)
0f((z))
Oz:
Zzk
dx
i=++1
where < < 1 is the index of an arbitrary layer. Without loss of generality, we explore the backward propagation when passing the k--th layer follows:
A fully connected or convolutional layer followed by non-linear activation function. Taking ReLU activation (i.e., Φu) for example, the j-th element in the gradient w.r.f. the k-th feature, [ lj. will be one if žk. > 0 and otherwise, daz will be zero. These zero gradients in 05k+1 can lead to the truncation of gradient of the loss function aVa.y: w.r.f the input image. As Ozs
dzz
a result, the gradient is effectively limited or weakened to some extent.
Max-pooling layer. As shown in Fig. [ max-pooling calculates the maximum value (orange block)
within a specific patch. Hence, the derivative 05k+1 will be a binary matrix, containing only ones dak
at locations corresponding to the orange blocks. In this case, approximately 3/4 of the elements in the given sample will be zeros. This means that max max- pooling tends to discard significant portion of the gradient information contained in az. Dxr
resulting in a truncated gradient.
The truncation of gradient caused by non-linear layers (e.g., ac- tivation function, max-pooling) can limit or dampen the flow of gradients during backward propagation, which decays the rele- vance among the gradient between the loss and input. Considering that many existing attacks rely on maximizing the loss by leverag- ing the gradient information, we make the following assumption:
0.1
0.2
1.9
1.4
0.0
-05
2.3 0.7
-0.4 0.9
1.0
-20
0.7 0.6
0.5
1.7
The truncation of gradient [ y; 0) intro- duced by non-line non-linear layers in the backward propagation process decays the adversarial transferability.
Assumption
Figure 1: A max-pooling layer with 2 x 2 kernel size and stride s 2 on 4 feature map in the forward propagation.
To validate Assumption T. we conduct several experiments us- ing FGSM, I-FGSM and MI-FGSM. The detailed experimental settings are summarized in Sec. 4.1
Randomly mask the gradient. To investigate the impact of gradient truncation on adversarial transferability, we introduce a random masking operation increase the probability of gradient truncation between stage 3 and stage 2 of ResNet-550. Fig. 2a ralillustrates the attack performance with various mask probabilities. As the mask probability increases, more zeros appear in the derivative, indicating a higher degree of gradient truncation. Consequently, the larger truncation probability renders the gradient less relevant to the loss function, decreasing the attack performance of the three evaluated methods. These findings validate our hypothesis that the truncation of gradient negatively impacts adversarial transferability and highlight the importance of preserving gradient information to maintain the effectiveness of adversarial attacks across various models.
Recover the gradient of ReLU or max-pooling layers. In contrast, it expected that mitigating the truncation of gradient replaced the zeros in the derivative of ReLU or max poollpn operations with ones, using various replacement probabilities. a) In Fig. 2b as the probability of replacement increases, fewer gradients are truncated across ReLU, resulting in improved adversarial transferability on all the three attacks. Notably, these attacks achieve their best performance when the derivative consists entirely of ones, which aligns with LinBP 12]. b) As illustrated in Fig. when the ratio of ones in the derivative of max-pooling increases (i.e., the replacement probability mcreases), the attack performance initially improves, reaching peak around 0.3. Subsequently, the attack performance gradually decreases but remains superior to vanilla backward propagation. This hightlights that we need more suitable approx ipproximation imation for the derivative calculation of max-pooling, which is detailed in Sec. 3e2. These results suggest that decreasing the probability of gradient truncation in max-pooling is beneficial for enhancing adversarial transfcrability.
improve the adversarial transferability. To explore this, randomly
Overall, these findings validate Assumption D that the truncation of gradients negatively impacts adversarial transferability. By preserving gradient information and carefully adjusting the replacement probabilities, it is possible to improve the effectiveness of adversarial attacks across different models.

---

FGSM
I-FGSM
MI-FGSM
(S)
30
.20
25
25
20
20
0.2
U.4
0.6
0.8
0.2
0.4
U.B
0.8
0.2
U.4
0.6
U.S
Prob.
Prob.
Prob.
(a) Randomly mask the gradient
(b) Recover the gradient of ReLU
(e) Recover the gradient of max-pooling
Figure 2: Average untargeted attack success rates (%) of FGSM, I-FGSM and MI-FGSM when randomly mask the gradient, recover the gradient of ReLU or max- mao-looling layers, respectively. The adversarial examples are generated on ResNet-55 and tested on all the nine victim models illustrated in Sec.
Raw data is provided in Appendix A.1
3.2 Mitigating the Negative Impact of Gradient Truncation
In Sec. 3.1 we demonstrate that reducing the probability gradient truncation in non-linear layers can enhance adversarial transferability. However, setting all elements in the corresponding derivative to one is not optimal for generating transferable adversarial examples. Here we investigate how to modify the backward propagation process non-linear layers to further enhance the transferability.
0.5
Curs
LiBBP
Within the standard backward propagation procedure. the elements comprising the derivative depend on the magnitudes of the associated feature map. This observation provides an impetus for considering the intrinsic characteristies of the underlying features when diminish- ing the probability of gradient truncation. To this end, we modify the gradient calculation for the ReLU activation function and max-pooling in the backward propagation procedure as follows:
vallina
Figure 3: Various candidate derivatives of ReLU function.
Gradient calculation for ReLU. To ensure precise gradient calculation, it is important exclude extreme values from consideration when calculating the gradient, while still maintaining the relationship between the elements in the derivative and the magnitude of the feature map. Among the family of ReLU activation functions, SiLU [14] provides a smooth and continuous gradient across the entire input range and is less susceptible to gradient saturation issues. Hence, we propose using the derivative of SiLUU to calculate the gradient of ReLU during the backward propagation process, i.e., 0a.+1 = oz4) (1 + 2; (1 - (z;))) where o(.) is the Sigmoid function. This '2;
formulation allows our gradient calculation to reflect the input magnitude mainly within the input range around [-5, 5]. while closely resembling the behavior of ReLU when the input is outside this range. As shown in Fig. 3 our proposed gradient calculation method demonstrates improved alignment with the input's magniuude compared to both the original derivative of ReLU and the derivative used in LinBP. By leveraging the smoothness and non- montoonotyy of SiLU, we can obtain more accurate and reliable gradient information for ReLU.
Gradient calculation for max-pooling. Similar to the gradient calculation for ReLU, it essential exclude extreme values and ensure that the gradient remains connected the magnitude of the feature map. Furthermore, in the case of max-pooling, the summation of gradients within each window should remain at one to minimize modifications to the gradient. To address these considerations, we propose using the softmax function calculate the gradient within each window w of the max max-pooling operation:
0zk+1 Zzk
Enew etuu
$.j.w
where is the temperature coefficient to adjust the smoothness of the gradient. If the feature Zk.i.j is related to multiple windows (i.e., the stride is smaller than the size of max-pooling), we sum its gradient calculated by Eq. Eq.3in in each window the final gradient.
5

---

Attacker
Method
v3
N/A
16.34
13.38
36.86
36.12
13.46
17.14
10.24
9.46
5.52
7.90
FGD
LinBP
27.22
23.04
59.34
59.74
22.68
33.72
16.24
13.58
7.88
BPA
35.36
30.12
70.70
68.90
32.52
42.02
22.72
19.28
12.40
SGM
33.78
28.84
63.06
65.84
31. 11
41.54
19.56
17.48
10.98
MI-FGSM
LinBP
29.82
68.66
30.24
16.58
9.94
Ghost
29.76
23.68
57.28
56.
25.00
34.76
17.10
14.76
9.50
BPA
47.58
41.22
80.54
79.40
44.70
54.28
32,06
25.98
17.46
N/A
42.68
36.86
68. 82
66.68
40.78
46.34
27.36
24.20
17.18
SGM
50.04
44.28 28
77.56
79.34
48.58
56.86
32.22
27.72
19.66
FML-FGSM
LinBP
47.70
40.40
77.44
78.76
41 .48
52.10
28.58
24.06
16.60
Ghost
47.82
41.42
75.98
73,40
44.84
52.78
30.84
27.18
19.08
BPA
55,00
48.72
85.44
83.64
52.02
60.88
38.76
33,70
23.78
NAA
29.10
26.08
58.02
59.10
27.60
39.16
15.12
12.30
7.86
SGM
35.64
32.34
65.20
71.22
34.20
46.72
17.10
13.86
9.08
ILA
LinBP
37.36
34,24
71.98
72.84
35.12
48
19.38
14,10
9.28
Ghost
28.
7.90
BPA
47.62
43.50
81.74
80.58
47.88
60.64
27.94
20.64
14.76
N/A
35.78
29.58
64,70
25.
34.18
17.30
11.44
SGM
45.22
38.98
70.22
78.44
35.30
46.06
26.28
21.64
14.50
Ghost
36.44
28.62
61.12
66.80
24.90
33.98
20.58
16.84
10.82
51.36
44.70
50.00
26.44
Table 1: Untargeted attack success rates (%) of various adversarial attacks nine models when generating the adversarial examples on ResNet-50 w/wo various model-related methods.
In practice, adopt the above strategies calculate the gradient of ReLU and max-pooling during the backward propagation process. This approach allows us to circumvent the issue of gradient truncation introduced by these non-linear layers. We refer to this modified backward propagation technique as Backward Propagation Attack (BPA), which can be applied to existing CNNs to adapt various transfer-based attack methods.
Experiments
In this section, we conduct extensive experiments on standard ImageNet dataset the effectiveness of the proposed BPA. We first specify our experimental setup, then we conduct series of experiments to compare BPA with existing state-of-the-ar attacks under different settings. Additionally, we provide ablation studies to further investigate the performance and behavior of BPA.
35 to validate
4.1 Experimental Setup
Dataset. Following LinBP [2]. we randomly sample 5,000 images pertaining to the 1,000 categories from ILSVRC 2012 validation [35] which could be classified correctly by all the victim models.
Models. We select ResNet-50 13 and VGG-19 37 as our surrogate model for generating adversar- ial examples. As for the victim models, consider six standardly trained networks, i.e., Inception-v3 (Inc-v3) 13,, Inception Inception-Resnet-y Resnet-en (IncRes-v2) [38], DenseNet [16], MobileNet-v2 (36), PNAS-
Net (28). and SENet [15]. Additionally,
adopt three ensemble adversarially trained models, (Inc-v3ans3). inse-inception-v3 (Inc-v3ens4), and ens-adv-Inception. 40 To address the issue of different input shapes required by these models, we adhere to the official pre-processing pipeline, including resizing and cropping techniques.
namely we--3--dv--nneptione
ResNet-v2 (IneRes-v2ens)
Baselines. We adopt three model-related methods as baselines, i.e., SGM [53), LinBP [12 and Ghost (23,, and evaluate their performance to boost adversarial transferability of iterative attacks (PGD (3I), momentum-based attacks (MI-FGSM 66, VMI-FGSM 43]), advanced objective functions (ILA
and input transformation based attacks (SSA (0).
8/255 to align with existing works. We run the attacks T 10 iterations with step size a 1.6 for untargeted attacks and T 300 iterations with step size 1 1/255 for targeted attacks. We set the momentum decay factor 1 1.0 and sample 20 examples for VMI-FGSM. The number of spectrum transfor- mations and tuning factor is set to N 20 and p 0.5, respectively. The decay factor for SGM is
Hyper-parameters. We adopt the maximum magnitude of perturbation

---

Attacker
Method
Inc-v3
IncRes-v2
DenseNet
MobileNet
PNASNet
Inc-V3ema
Inc-V3
IncRes-ve V2ass
SGM
23.68
19.82
51.66
55.44
22.12
30.34
13.78
12.38
7.90
SGM-BPA
43.44
38.14
77.66
81.50
41,42
53.56
27.20
22.58
14.70
LinBP
27.22
23.04
59.34
59.74
22,68
33.72
16.24
13.58
7,88
PGD
LiP
39.08
34,80
77,80
76.86
40.50
50,26
25.66
22.46
15,10
Gbost
17.74
13.68
42.36
41.06
13.92
11.60
10.34
6.04
Chosi+BPA
34.62
29.28
69.48
69.20
29.98
41.60
22.68
18.88
11.48
SGM
33.78
63.06
84
41.54
19.56
17.48
10.98
SGM+BPA
56.04
49.10
83.32
88.08
$2,96
63.30
36.10
29.78
20.98
LinBP
35.92
29.82
68.66
69.72
30,24
41,68
19.98
16.58
9.94
MI-FGSM
LiaRP4BPA
48.74
43.96
83,30
83.52
50.00
59,22
32.60
28.42
20.32
Ghost
29.76
23 23.68
$7,28
56.10
25.00
34.76
17.10
14.76
9.30
Ghost+BPA
50.42
42.84
83.02
81.24
44,70
$6.50
32.46
26.82
18.34
Table 2: Untargeted attack success rates (%) of various baselines combined with our method using PGD and MI-FGSM. The adversarial examples are generated on ResNet-50.
Y = 0.5 and the random range of Ghost network is À 0.22. We follow the setting of LinBP to modify the backward propagation of ReLU in the last eight residual blocks of ResNet-50. We set the temperature coefficient 10 for ResNet-50 and for VGG-19.
4.2 Evaluation on Untargeted Attacks
To validate the effectiveness of our proposed method, we compare BPA with several other model- related methods (i..., SGM, LinBP, Ghost) on ResNet-50 to boost various adversarial attacks, namely PGD, MI-FGSM, VMI-FGSM, ILA and SSA. Here we adopt ResNet-50 as the surrogate model since SGM is specific to ResNets. However, it is worth noting that BPA is general various surrogate models with non-linear layers and we also report the results on VGG-19 in Appendix [A.2] To further validate the effectiveness of BPA, also consider more input transformation based attacks, different perturbation budgets and conduct evaluations on CIFAR-10 dataset (20 in Appendix A.3 A..5 We
measure the attack success rates by evaluating the misclassification rates of the nine different target models on the generated adversarial examples.
We can observe from Table that the model-related strategies can consistently boost performance of the five typical attacks on nine models. Among the baseline methods, LinBP generally achieves the best performance, except for VMI-FGSM where SGM surpasses LinBP. By addressing the issue of gradient truncation, BPA consistently improves the performance of all the five attack methods and achieves the best overall performance. On average, BPA outperforms the runner-up attack by a significant margin of 7 7.84%, 11.19%, 5.08%. 9.17%, 2.25%, respectively. These results highlight the effectiveness and generality of BPA in generating transferable adversarial examples compared with existing model-related strategies. The performance improvement achieved by BPA on SGM and LinBP, which also modify the backward propagation, validates our hypothesis that reducing the gradient truncation introduced by non-linear layers is beneficial for enhancing the adversarial transferability. This emphasizes the importance of carefully considering the backward propagation procedure when generating transferable adversarial examples.
Evaluations the single baseline.
Evaluations by combining BPA with the baselines. The primary objective of BPA is to mitigate the negative im- pact of gradient truncation on adver- sarial transferability, which is not con- sidered by the baselines. Hence, is expected that BPA can also boost the performance of these basclines. For validation, we integrate BPA with the baseline methods to enhance the per- formance of PGD and MI-FGSM at- tacks. The results of these combina- tions are presented
Attacker
Method
HGD
R&P
NIPS
JPEG
RS
NRP
N/A
9.34
5.00
6.00
11.04
8.50
11.96
SGM
16.80
7.50
9.44
13.96
10.50
12.76
PGD
LinBP
16.80
7.68
10.08
15.76
10.50
13.14
Ghost
6.42
11.92
9.50
12.06
BPA
23,96
12.02
15.60
22.52
14.00
14.08
N/A
16.64
8.04
9.92
16.68
13.00
13.32
SGM
24.80
11.02
13.16
20.26
14.00
14.38
MI-FGSM
LinBP
21.98
10.32
13.26
20.56
12.50
13.22
Ghost
17.98
8.88
10.64
18.52
13.50
13.84
34.30
17.84
30.86
17.50
15.96
Table 3: Untargeted attack success rates (%) of several at- tacks on six defenses when generating the adversarial exam- ples on ResNet-50 w/wo various model-related methods.
Table 2 We can observe that BPA can effectively boost the adversarial transferability of various baselines. On average, BPA can boost the best baseline (i.e., LinBP) with a remarkable margin of 13.23% and 20.94% for PGD and MI-FGSM, highlighting
7

---

Attacker
Method
Inc-v3
IncRes-v2
DenseNet
MobileNet
PNASNet
SENet
Inc-v3mns
Inc-v3enet
IncRes-V2ena
NIA
0.54
0.80
4.48
2,04
1.62
2.26
0.18
0,08
0.02
SGM
2.56
3.12
15.08
8.68
5.78
9.84
0.62
0.18
0.04
PGD
LinBP
5,30
4.84
16.08
8.48
7.26
7.94
1.50
0.54
0,28
Gbost
1.34
2.14
10.24
4.74
3,90
6.64
0.36
0.16
0.10
BPA
8,76
9.74
23.76
13.42
14,66
13.76
2.52
1,02
0.72
N/A
0.16
0.26
2.06
0.90
0.42
1.22
0.00
0.02
0.02
SGM
0.74
0.76
5.84
3.24
1.66
3.70
0.00
0.02
0.00
MI-FGSM
LinBP
3.30
3,00
13.44
6.26
5.50
7.18
0.30
0.10
0.02
Gbost
0.66
0.76
5.48
2.14
1.58
3.38
0.08
0.02
0.00
BPA
5.68
7.30
23.34
12.16
12.50
14.56
0.60
0.12
0.06
Table 4: Targeted attack success rates (%) of various attackers on nine models when generating adversarial examples on ResNet-50 w/wo model-related methods using PGD and MI-FGSM.
the high effectiveness and superiority of BPA. Such high performance also validates its excellent generality to various architectures and supports our hypothesis about gradient truncation.
Evaluations defense methods. To further evaluate the effectiveness of BPA, we also assess its performance on six defense methods using PGD and MI-FGSM, namely HGD 251. R&P [56), - JPEG 11,, RS [5] and NRP [32] The results presented in Table We can observe that our BPA method successfully enhances both the PGD and MI-FGSM attacks, leading to higher attack performance against the defense methods. The results suggest that BPA effectively enhance adversarial attacks against range of defense techniques, reinforcing its potential as powerful tool for generating transferable adversarial examples.
In summary, BPA exhibits superior transferability compared to various baseline methods when evaluated using range of transfer-basee based attacks. It also exhibits good generality to further boost existing model-related approaches and achieves remarkable performance on several defense models, highlighting its effectiveness and versatility in generating highly transferable adversarial examples.
4.3 Evaluation on Targeted Attacks
To further evaluate the effectiveness of BPA, we also investigate its performance in boosting targeted attacks. Zhao et al. [60] identified that logit loss can yield better results than most resource-intensive attacks regarding targeted attacks. Here we adopt PGD and MI-FGSM to optimize the logit loss on ResNet-50 w/wo various model-related methods. The results are summarized in Table Without the model-related methods, both PGD and MI-FGSM exhibit poor attack performance. However, when these methods applied, the attack performance improves significantly. Notably, our BPA method achieves the best attack performance among all the baselines. This highlights the high effectiveness and excellent versatility of our proposed method in boosting targeted attacks and exhibits its potential to improve adversarial attacks in a wide range of scenarios. We also provide the results on VGG-19 in Appendix A.6
4.4 Ablation Study
To gain further insights into the effectiveness of BPA, we perform parameter studies on two crucial aspects: the position of the first ReLU layer to be modified and the temperature coefficient t for max-pooling. Additionally, we conduct ablation studies to investigate the impact of diminishing the gradient truncation of ReLU and max-pooling separately. We also provide more discussions about BPA Appendix A.7
On the position of the first ReLU layer to be modified. ReLU activation functions are densely applied in existing neural networks. For instance, there are totally 17 ReLU activation functions in ResNet-50. Intuitively, the truncation in the latter layers has greater impact on gradient relevance compared to the earlier layers. As BPA aims recover the truncated gradients by injecting imprecise gradients into the backward propagation, it is essential to focus on the more critical layers. To identify these important layers and evaluate their impact on transferability, we conduct the BPA attack using MI-FGSM by modifying the ReLU layers starting from the i-th layer, where 1 < < 17. As shown in Fig. 4al modifying the last ReLU layer alone significantly improves the transferability of the attack, showing its high effectiveness. As we modify more ReLU layers, the transferability further
8

---

Inev3
IncRes-v2
Net
MobileNe:
PNASNel
SENet
Inev3enea
Ine-dened
IncRes-2enna
Atack succoss DiEs (5))
40
ARack
02 The position of the first ReL. layer to modiffied
0
3
15
Temperature coeffleient
(a) Attack success rate (%% of BPA using MI-RGSM by modifying the ReLU layers starting from the layer. Here 3-0 indicates first ReLU layer the third stage
(b) Artack success rate (%) of RPA using MI-PGSM with various temperature coefficients S & 15) in Eq. B for the max-pooling layer
Figure 4: Syper-parameee studies on the position of the first ReLu layer to be modified and the temperature coefficient for the max-poote max-pooling layer.
Altacker
ReLU
Max-pooling
Inc-v3
IncRes-v2
DenseNet
MubileNet
PNASNet
SENet
Inc-433ne5
Ine-v3mnet
IncRes-v
X
16:34
13.38
36.86
36.12
13:46
17.40
10.24
9.46
5.52
34.00)
62.80
34.96
14.38
x
20.36
16.
44.66
42.82
17.12
21.52
13.20
11.88
7.74
35.36
30.12
70.70
68.90
32,52
42.02
22.72
19.28
12.40)
26.20
21. 10.50
51.50
49.68
22.92
30.12
16:22
14.58
9.00
MI-PGSM
74.42
10.94
X
34.16
02
61
59.42
32.24
37.32
21.74
19.96
14.70
47.5%
41.22
80.54
79.40
44.70
54.28
32,06
25.98
17.46
Table 5: Untargeted attack success rates (%) of PGD and MI-FGSM when generating adversarial examples on ResNet-50 w/wo modifying the backward propagation of ReLU or max-pooling.
improves and remains consistently high for most models. However, for a few models (e.g.. PNASNet), modifying more ReLU layers leads to a slight decay on performance. To maintain high level of performance across all nine models, we modify the ReLU layers starting from 3-0 ReLU layer, which also adopted LinBP [2].
On the temperature coefficient / for max-pooling. The temperature coefficient / plays a crucial role in determining the distribution of relative gradient magnitudes within each window. For example, when = O, the gradient distribution becomes a normalized uniform distribution. To find an appropriate temperature coefficient, we conduct the BPA attack using MI-FGSM with various temperatures. As shown in Fig. 4b when f = 0, the attack exhibits the poorest performance but still outperforms the vanilla MI-FGSM. As we increase the value of the attack's 's performance consistently improves and reaches a high level of performance after = 10. By selecting a suitable temperature coefficient, we ensure that the gradient distribution within each window is well-balanced and contributes effectively to the adversarial perturbation. Thus, we adopt = 10 in our experiments.
A Ablaioon studies ReLU and max-pooling. As stated in Sec. 3.1 hypothesize that the gradient truncation caused by non-linear layers, such as ReLU and max-pooling in ResNet-50, has detrimental effect adversarial transferability. To further validate this hypothesis, conduet ablation studies by comparing the performance of PGD and MI-FGSM attacks using the vallina backward propagation, the backward propagation modified by either ReLU or max-pooling, and both modifications combined. As shown in Table adopting the modified backward propagation with either ReLU or max-pooling results in a significant improvement in adversarial transferability for both PGD and MI-FGSM attacks. Considering the presence of only one max- max-pooling layer in ResNet-50, the average performance improvement of 4.07% and 7.58% for PGD and MI-FGSM highlights the high effectiveness of BPA and underscores the efficacy of BPA in addressing the issue of gradient truncation. Furthermore, when both ReLU and max-pooling layers are modified in backward propagation, PGD and MI-FGSM exhibit the best performance. This finding supports the rational design of BPA and highlights the importance of mitigating gradient truncation in both ReLUU and max-pooling layers to achieve optimal adversarial transferability.

---

5 Conclusion
In this work, we analyzed the backward propagation procedure and identified that non-linear layers (e.g., ReLU and max-pooling) introduce gradient truncation, which undermined the adversarial transferability. Based on this finding, we proposed a novel attack called Backward Propagation Attack (BPA) to mitigate the gradient truncation for more transferable adversarial examples. In particular, BPA addressed gradient truncation by introducing a non-monotonic function as the derivative of the ReI.U activation function and incorporating softmax with temperature to calculate the derivative of max-pooling. These modifications helped to preserve the gradient information and prevented significant truncation during the backward propagation process. Empirical evaluations on ImageNet dataset demonstrated that BPA can significantly enhance existing untargeted and targeted attacks and outperformed the baselines by remarkable margin. Our findings identified the vulnerability of model architectures and raised a new challenge in designing secure deep neural network architectures.
6 Limitation
Our proposed BPA modifies backpropagation process for gradient calculation, making it only suitable for gradient-basee based attacks. Besides, BPA modifies the derivatives of non-linear layers, such as ReLU and max-pooling. Consequently, it may not be directly applicable to models lacking these specific components, such as transformers. In the future, we will investigate how to generalize our BPA to such transformers by refining the derivatives of some components, e-g., softmax. This endeavor to enhance the generality and versatility of BPA will be an essential aspect of ongoing research, paving the way for the broader applicability of the proposed method and facilitating its adoption in various deep learning models beyond those with ReLU and max-pooling layers.
7 Acknowledgement
This work is supported by National Natural Science Foundation (U22B2017, 62076105) and Interna- tional Cooperation Foundation of Hubei Province, China (202IEHBO1)).
References
[1] Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-Basse Adversarial Attacks: Reliable
Attacks Against Black- Box Machine Learning Models. In Proceedings of the International Conference on Learning Representations, 2018.
[2] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE symposium on security and privacy (sp). pages 39-57, 2017.
[3] Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattack: A Query-efficient Decision-
based Attack. In 2020 ieee sympoxium on security and privacy (sp). pages 1277-1294, 2020.
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models. Proceedings of
the 10th ACM workshop on artificial intelligence and security, pages 15-26, 2017.
[5] Jeremy M. Cohen, Elan Rosenfeld, and J Zico Kolter. Certificd Adversarial Robustness via Randomized Smoothing. International Conference on Machine Learning, pages 1310-1320, 2019.
[6] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting Adversarial Attacks with Momentum. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9185-9193, 2018
[7] Yinpeng Dong. Tianyu Pang. Hang Su. and Jun Zhu. Evading Defenses to Transferable Adversarial Examples by Translation invvaraan Attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4312 4312-4321, 2019.
[8] Zhijin Ge, Fanhua Shang, Hongying Liu, Yuanyuan Liu, Liang Wan, Wei Feng, and Xiaosen Wang. Improving the Transfcrability of Adversarial Examples with Arbitrary Style Transfer. In Proceedings of the ACM International Conference Multimedia, 2023.
[9] Zhijin Ge, Fanhua Shang, Hongying Liu, Yuanyuan Liu, and Xiaosen Wang. Boosting Adversarial Transferability by Achieving Flat Local Maxima. In Proceedings of the Advances in Neural Information Processing Systems, 2023.
[10] lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In Proceedings of the International Conference on Learning Representations. 2015.
10

---

[11] Chuan Guo, Mayank Rana. Moustapha Cisse, and Laurens van der Maaten. Countering Adversarial Images using Input Transformations. In Proceedings of the International Conference Learning Representations, 2018.
[12] Yiwen Guo. Qizhang Li. and Hao Chen. Backpropagating Linearly Improves Transferability of Adversarial Examples. In Proceedings of the Advances in Neural Information Processing Systems, pages 85-95, 2020.
[13] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-778,
2016.
[14] Dan Hendrycks and Kevin Gimpel. arXiv:1066.0 08415, 2016.
Gaussian Error Linear Units (GELUs).
arXiv preprint
[15] Jie Hu, Li Shen, and Gang Sun. Squeeze-and--xcitation Networks, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7132-7141, 2018.
[16] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely Connected Convolutional Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4700-4708, 2017.
(17] Qian Huang, Isay Katsman, Horace Hc, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing Adversarial Example Transferability with an Intermediate Level Attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4733-4742, 2019.
[18] Tianjin Huang, Vlado Menkovski, Yulong Pei, YuHao Wang, and Mykola Pechenizkiy. Direction- aggregated attack for transferable adversarial examples. ACM Journal on Emerging Technologies in Computing Systems (JETC), 18(3):1-22, 2022.
[19] Andrew Tlyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-hox Adversarial Attacks with Limited Queries and Information. Proceedings of the International Conference on Machine Learning, pages 2137-2146, 2018.
[20] Alex Krizhevsky, Geoffrey Hinton, et al. Learning Multiple Layers of Features from Tiny Images. 2009.
[21] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial Machine Learning at Scale. In Proceedings of the International Conference on Learning Representations, 2017.
[22] Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li. QEBA: Query-Efficient Boundary- based Blackbox Attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pateen Recognition, pages 1221 1230,, 2020.
[23] Yingwei Li, Song Bai, Yuyin Zhou, Cihang Xie, Zhishuai Zhang, and Alan Yuille. Learning Transferable Adversarial Examples via Ghost Networks. In Proceedings of the AAAI Conference on Artificial Intelligence,
pages 1 11458- 1465,, 2020.
[24] Siyuan Liang. Xingxing Wei. Siyuan Yao, and Xiaochun Cao. Efficient adversarial attacks for visual object tracking. In Eumpean Conference Computer Vision, pages 34-50, 2020.
[25] Fangzhou Liao, Ming Liang. Yinpeng Dong. Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against
Adversarial Attacks using High-level Representation Guided Denoiser. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1778-1787, 2018.
[26] Jiadong Lin, Chuanbiao Song. Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv: 06281, 2019.
[27] Jiadong Lin, Chuanbiao Song. Kun He, Liwei Wang, and John E. Hoperoft. Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks. In Proceedings of the International Conference on Learning Representations, 2020.
[28] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy, Progressive Neural Architecture Search. In Proceedings of the European Conference on Computer Vision, pages 19-34, 2018.
[29] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into Transferable Adversarial Examples and Black-box Attacks. In Proceedings of the International Conference on Learning Representations, 2017.
[30] Yuyang Long, Qilong Zhang. Boheng Zeng, Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency Domain Model Augmentation for Adversarial Attack. In Proceedingy of the European Conference on Computer Vision, pages 549-566, 2022.
[31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks, In Proceedings of the International Conference on Learning Representations, 2018
[32] Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A Self- supervised Approach for Adversarial Robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 262-271, 2020.
[33] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The Limitations of Deep Learning in Adversarial Settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pages 372-387, 2016.

---

[34]
Shaoqing Ren, Kaiming He, Ross Girshick. and Jian Sun. Faster R-CNN: Towards Time-Time&& Object
Octection with Region Proposal Networks. In Proceedings of the Advances in Neural Information Processing Systems, 2015.
[35]
Olga Russakovsky. Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet Large Scale Visual Recognition Challenge. International Journal of Computer Vision. 115:211-252. 2015.
[36] Mark Sandler. Andrew Howard. Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4510-4520, 2018
[37] Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the International Conference on Learning Representations, 2015.
[38] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, Inception-resnet and the Impact of Residual Connections on Learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 2017.
[39] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing Properties of Neural Networks. In Proceedings of the International Conference on Learning Representations, 2014.
[40] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. In Proceedings of the International Conference on Learning Representations, 2018.
[41] Hao Wang. Yitong Wang. Zheng Zhou, Xing Ji, Dihong Gong. Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large Margin Cosine Loss for Deep Face Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[42] Kunyu Wang. Xuanran He, Wenxuan Wang, and Xiaosen Wang. Boosting Adversarial Transferability by Block Shuffle and Rotation. arXiv preprini arXiv:2308. 0123308..220 10299, 2023.
[43] Xiaosen Wang and Kun He. Enhancing the Transferability of Adversarial Attacks through Variance Tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages
1924-1933, 1933,193,222 2021.
[44] Xiaosen Wang. Kun He, Chuanbiao Song, Liwei Wang, and John F. Hoperoft. AT-GAN: An Adversarial Generator Model for Non-Constrained Adversarial Examples. arXiv preprint arXiv: / 1904. 007.07793, 2019.
[45] Xiaosen Wang. Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the Transferability of Adversarial Attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16158-16167, 2021.
[46] Xiaosen Wang, Jiadong Lin, Han Hu, Jingdong Wang, and Kun He. Boosting Adversarial Transferability through Enhanced Momentum. In Proceedings of the British Machine Vision Conference, page 272, 2021.
[47] Xiaosen Wang. Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, and Wei Liu. Triangle Attack: A Query-efficione
periision-baed Adversarial Attack. In Proceedings of the European Conference on Computer Vision, pages 156-174, 2022.
[48] Xiaosen Wang, Zeliang Zhang, and Jianping Zhang. Structure Invariant Transformation for better Adver- sarial Transfcrability. In Proceedings of the IEEF/CVF International Conference on Computer Vision, 2023.
[49] Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhang Qin, and Kui Ren. Feature Importance- aware Transferable Adversarial Attacks. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pages 7639-7648, 2021.
[50] Zhiyuan Wang, Zeliang Zhang. Siyuan Liang, and Xiaosen Wang. Diversifying the High-level Features for better Adversarial Transferability. In Proceedings of the British Machine Vision Conference, 2023.
[51] Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint arXiv: 1811. 12641, 2018.
[52] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A Discriminative Feature Learning Approach for
Deep Face Recogmition. In Proceedings of the European Conference on Computer Vision, 2016.
[53] Dongxian Wu, Yisen Wang. Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets. In Proceedings of the International Conference on Learning Representations, 2020.
[54] Weibin Wu. Yuxin Su. Xixian Chen, Shenglin Zhao, Irwin King, Michael R Lyu. and Yu- Wing Tai. Boosting the Transfcrability of Adversarial Samples via Attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1161-1170, 2020.
[55] Chaowei Xiao, Bo Li, Jun-Yar Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating Adversarial Examples with Adversarial Networks. arXiv preprint arXiv: 1801_026 0026100 2018.
[56] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating Adversarial Effects
Through Randomization. In Proceedings of the International Conference on Learning Representations.
2018.
12

---

[57] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai. Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving Transferability of Adversarial Examples With Input Diversity. In Proceedings the IEEE/CVF Conference on Computer Vixion and Puttern Recognition, pages 2730-2739. 2019.
[58] Yifeng Xiong, Jiadong Lin, Min Zhang, John E. Hopcroft, and Kun He. Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2022, pages 14963-14972, 2022.
[59] Jianping Zhang, Jen-tse Huang, Wenxuan Wang. Yichen Li, Weibin Wu, Xiaosen Wang, Yuxin Su, and Michael R Lyu. Improving the Transferabilityy of Adversarial Samples by Path-Augmented Method. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5173-8182 2023.
[60] Zhengyu Zhao, Zhuoran Liu, and Martha Larson. On Success and Simplicity: A Sccond Look Transferable Targeted Attacks, Advances Neural Information Processing Systems, 34:6115-611 6128, 2021.
13

---

A Appendix
In the appendix, we provide extra results about proposed BPA. We first provide the raw data for Fig. 2 Considering that our main evaluations are conducted on ResNet-50 since the baseline SGM primarily focuses on the residual connection, we also conduct the evaluations on VGG-19 to further validate the effectiveness and generality of our proposed BPA. Then we provide the evaluations using more input transformation based attacks, and different perturbation budgets and also evaluate various methods on CIFAR- 10 datasets. To further gain insights into the superiority of BPA, we also offer further discussions and explorations about the motivation and design.
A.1 Raw Data for Fig.
Due to the limited space, we only report the average attack success rates in Fig. 2 generality various deep models, we also report the raw data in Tab. Al We can see that the attacker exhibits the same trends various deep models, which is consistent with the results in Fig.
To validate its
Prob.
Inv-v3
InvRes-12
DenseNet
MobileNet
PNASNet
SENet
Inc-v3
Inc-v3mmet
IncRes-v2rns
0.00
26.38
21.38
51.26
49.62
22.64
30.22
15.94
14.44
8.68
0.10
26.14
21.62
51.30
49.78
22.12
29.32
15.92
14,24
8.88
0.20
25.72
21.56
50.72
49.02
22.18
28.90
15.82
13.82
8.38
21.24
48.74
29.12
15,52
13,82
8.54
0.40
24.96
20.78
50.02
48.32
21.48
28.64
15.04
13.92
8.46
0,50
24.94
20.16
49.52
47.66
20.64
27.92
14.42
13.22
8.28
0.60
24.94
19.70
48.28
47.20
20.34
27.92
14,28
12.84
8.14
0.70
23.18
18.78
46.96
46.08
18.92
26.42
13.96
12.72
7.74
0.80
17.18
44.52
44.38
11.72
7.00
0.90
20.22
15.78
39.82
41.16
14.80
20.84
11.22
10.54
6.34
1,00
0.02
1.10
0.00
0.02
0.02
0.02
3.76
4.30
1.28
Table Al: Original data for Fig. 2a
Prub.
Inv-x3
InvRes-v2
DenseNet
MobileNet
PNASNet
SENet
Inc-v3
Inc-v3mmut
IncRes-v2
0.00
26.38
21.38
51,26
49.62
22.64
30.22
15.90
14.42
8.68
0.10
28.
23.22
$4,40
$1.62
24.82
32.38
16.98
15.34
9,26
0.20
29.34
24.80
$7.28
55.02
26.60
34.14
17.70
16.12
10.04
25.70
$6.52
36.00
17.92
16.26
0.40
31.34
26.36
60.50
$7.78
28.66
36.68
18.74
16.24
10.18
0.50
31.86
26.76
60.74
58.64
29.46
37.68
19.12
16.74
10.42
0.60
31.68
27.02
61.00
58.66
29.80
37.58
18.94
16.66
10.76
0.70
31.90
26.36
60.62
59.40
29.52
37.90
18.88
16.72
10.40
0.80
26.82
59.16
37.90
18.80
16.40
0.90
31.80
26.56
60.42
$8.90
29.06
37.50
18.98
16.78
10.52
1.00
32.00
26.52
60.44
59.44
29.28
37.46
18.78
16.52
10.30
Table A2: Raw data for Fig. 2b
Prub.
Inv-x3
InvRes-v!
DenseNet
MobileNet
PNASNet
SENet
Inc-v3ensa
Inc-y3ense
IncRes-v2uus
0.00
26.08
21.40
$1.30
49.64
22.50
30.26
15.74
14.44
8.68
0.10
28.60
23.78
56.14
53.10
26.28
32.78
18,20
16.64
11.30
0.20
29.66
24.14
56.98
54.72
27.18
33.54
19.62
18.04
12.86
0.30
28.92
24.06
57.16
$4.36
31.58
20.50
19.28
13.40
0.40
28.22
23.46
56.12
$3.16
26.72
30.24
20.62
19.44
14.04
0.50
27.50
21.68
54.40
52.06
25.54
28.48
20.34
19.02
14.14
0.60
26.70
20.72
53.02
50.10
24.24
27.42
19.92
19,30
14.04
0.70
25.82
20.14
51.38
48.36
22.94
26.12
20.24
19.70
0.80
25.00
19.04
50.02
47.36
24.12
20.22
19.68
14.56
0.90
24.70
18.34
48.98
46.36
21.36
23.34
20.38
19.68
14.18
1.00
24.04
18.10
48.34
45.08
21.02
22.40
20.38
20.10
14.18
Table A3: Raw data for Fig.
2c

---

Attacker
Method
PNASNet
v3
IncRes- V2nn
NIA
12.52
9.70
25.82
32.20
13.18
13.82
7.64
7.00
4.14
PGD
Ghost
13.18
9.72
25.78
12.50
12.80
13.68
8.12
7.90
4.48
14.06
8.78
NIA
19.74
15.32
37.02
43.42
21.16
23.02
11.46
10.08
5.96
MI-FGSM
Ghost
19.88
15.34
36.44
43,20
21.84
24.06
11.54
10.30
6.00
BPA
36.88
29,98
68.58
45.98
43.06
17.68
11.94
NIA
37.20
29.58
58.20
62.20
40.88
38.86
21.14
17.62
11.10
VMI-FGSM
Ghost
36.94
29.75
58.32
62.16
41.32
38. 38.96
21.18
17.58
11.20
RPA
51.60
43.00
74.08
78.74
59.54
54.74
32.88
30.04
20.18
NIA
16.08
13.8
31.28
42.62
19.72
25.
8.76
7.70
4.62
LinBP
17.08
14.54
32.74
44.40
20.16
27.08
8.44
4.54
ILA
Ghoot
16.56
14.08
31.80
41.90
20.12
25. 25.98
8.84
7.84
4.76
BPA
29.70
25.06
50.84
61.52
38.84
41.20
15.30
12.36
8.30
N/A
33.52
26.38
50.86
60.26
30.94
30.78
17.06
14.52
8.78
SSA
LinBP
35.70
28.08
53.76
63.52
32.32
34.18
18.64
16.10
9.36
Ghost
33.52
25.92
51.31
60.50
30.96
30.02
17.16
14.74
8.74
BPA
50.36
40.68
70,90
78.86
51.64
47.86
29.52
26.50
18.30
Table A4: Untargeted attack success rates (%) of various adversarial attacks on nine models when generating the adversarial examples on VGG-19 w/wo various model-related methods.
Tne-v3
IncRes-v2
MobileNet
Inc-V3
Inc-V3ears
IncRes-v2een
LinBP
13.52
10.28
27.60
34.36
14.16
15.12
8.32
7.88
4.20
LinBP+BPA
21.10
16.48
40,92
50.54
25.28
24.86
12.34
7.16
PGD
Ghost
13.18
9.72
25.78
32.50
12,80
13.68
8.12
7.90
4.48
Chost-BPA
26.34
20.22
49.14
58.02
34.96
31.22
15.60
13.60
8.56
LinBP
20.28
15.24
36.84
44.44
20.66
23.28
10.92
9.52
5.48
LiA
32,08
24.58
63.16
10.22
ML-FGSM
Gbost
19.88
15.34
43.20
24.06
11.54
10.30
Ghost-BPA
37.12
30.50
60.60
69.00
45.80
43.10
21.28
17.38
11.92
Table A5: Untargeted attack success rates (% various baselines combined with our method using PGD and MI-FGSM. The adversarial examples are generated on VGG-19.
A.2 Additional Evaluation Untargeted Attacks
To validate the generality of BPA various architectures, further validate the effectiveness of our proposed BPA on VGG-19. Specifically, we first conduct untargeted attacks on VGG-19 following the setting in Sec. 4.2 Here take LinBP and Ghost as baselines.
Evaluations on the single baseline. As shown Table A4 model-related methods consistently achieve better attack performance than the attacks the original models, showing the effectiveness of these methods. Compared with LinBP and Ghost, our proposed BPA exhibits superior performance across all five attacks. On average, BPA outperforms the runner-up method with remarkable margin of 14.21% 44 16.44%, 14. 15%, 11.80%, 13.64% for PGD, MI-FGSM, VMI-FGSM, ILA, and SSA, respectively. These results consistent with the findings reported in Sec. 4.2 for ResNet-50. The superior performance of BPA not only validates its effectiveness but also highlights its generality to different architectures.
Evaluations by combining BPA with the baselines. Similar Sec. 4.2 we also integrate BPA into LinBP and Ghost to further boost the performance. The results in Table A5 indicate that BPA can significantly improve the attack performance PGD and MI-FGSM. For instance, considering MI- FGSM attack, integrating BPA results in clear performance improvement of 11.42% and 16.46% for LinBP and Ghost, respectively. These findings are consistent with the results obtained on ResNet-50, as discussed in Sec. 4.2 These results further highlight the effectiveness and superiority of BPA in boosting the adversarial transferability of existing attacks, which are not limited to the surrogate models.
Evaluations on defense methods. Finally, we evaluate these model-related approaches on defense methods and report the results in Table A6 Notably, our BPA method consistently enhances the performance of PGD and MI-FGSM attacks, yielding superior results against the defense methods compared to other model-related methods. On average, BPA outperforms the runner-up method with
15

---

Method
MohileNet
PNASNet
Inc-
IncRes- V2
N/A
45,00
38.56
71.64
70.08
41.60
48.56
28.52
24.82
16.48
DIM
LinIPP
45.74
38.38
76.34
77.58
39.28
50.20
27.40
22,60)
15.50
Ghest
BPA
59.20
50.86
87.70
86.92
55.40
62.68
40.32
34.84
24.42
SGM
41.18
35.22
71.12
72.20
41.24
47.88
25.50
23.66
16.28
Ghest
37.08
29.36
66.48
63.26
33.26
39.74
23.58
21.14
14.54
BPA
50.86
87.70
86.92
55,40
62.68
34.84
N/A
42.68
33.76
70.12
65.70
34,90
39.
26.36
23.48
15.22
52.70
SIN
LinBP
50.46
41.06
78.22
77.14
38.82
47.64
30.38
25.08
17.14
Ghost
47.46
37.92
77.14
72.82
38.66
46.02
29.14
24.58
16.28
BPA
52.40
43.44
80.12
76.38
45.42
50.28
39.16
36.32
26.06
SGM+BPA
62.26
54.12
88.04
88.28
56.80
63,46
46.08
39,80
30.68
N/A
45.00
38.56
71.64
70.08
41.60
48.56
28.52
24.82
16.48
SGM
52.72
45.22
79.42
82.34
49.50
58.66
32,42
28.20
19.26
DA
LinBP
45.74
38.38
76.34
77.58
39.28
50.20
27.40
15.50
Ghost
45.20
37.86
72.70
72.34
40.32
48.50
27.44
23.98
15.78
34.84
Table A7: Untargeted attack success (%) of various imput-transformation-bee attacks on nine models when generating the adversarial examples on ResNet-50 w/wo various model-related methods.
margin of 5.32% and 7.78% for PGD and MI-FGSM, respectively. These findings further underscore the high effectiveness of BPA in improving the performance of various attacks and highlight its versatility in enhancing adversarial attacks across different architectural models.
In conclusion, the results obtained
for untargeted attacks on VGG-19 align with the findings presented for ResNet-50 Sec. 4.2 The sig- nificant and consistent improvement in performance across various ar- chitectures validates our motivation that addressing the gradient trunca- tion issue caused by non-linear layers can enhance adversarial transferabil- ity. These findings also strongly sup- port the high effectiveness and utility of our BPA to boost adversarial trans- ferability.
Attacker
Method
HGD
R&P
NIPS-r3
JPEG
RS
NRP
N/A
5.44
3.16
3.54
8.36
8.45
11.26
LinBP
5.28
3.26
3.88
9.14
9.00
11.76
PGD
Ghost
5.68
3.16
3.70
9.10
8.50
10.98
5.76
12.18
8.00
MI-FGSM
LinBP
8.06
4.75
5.34
11.56
8.50
12.32
Ghost
9.14
4,92
5.78
12.32
8.50
12.08
BPA
24.36
11.50
14.30
22.38
14.00
13.12
Table A6: Untargeted attack success rates (%) of several attacks on six defenses when generating the adversarial ex- amples on VGG-19 w/wo various model-related methods.
A.3 Additional Evaluation on Various Input Transformation based Attacks
In Table ! we compare our BPA with various model-related attacks when combined with other attacks, including gradient-based (PGD), momentum-bant (MI-FGSM, VMI-FGSM), objective- related (ILA) and input transformation based (SSA) attack. Due the page limit, we only evaluate BPA with one up-to-date transformation based attack (SSA). Here we further evaluate its generality
to other input transformation based attacks, namely DIM 57
TIM 77 SIN (6) and DA [8] in
Table A7 As we can see, our BPA can significantly boost these input transformation based attacks. Compared with existing model-related attacks, BPA consistently exhibits better transferability for TIM, DIM, and DA. For SIN, BPA exhibits comparable attack performance with SGM on standardly trained models but much better transferability on adversarially trained models. To integrate the advantage of SGM and our BPA, we combine BPA with SGM for SIN, which outperforms the baselines with a clear margin. These results further validate its superiority in boosting adversarial transferability.
A.4 Additional Evaluation on Another Widely Adopted Perturbation Budget
Both 8/255 and 16/255 are widely adopted perturbation budgets adversarial learning. Sec. 4.2 mainly adopt = 8/255 for evaluation. To further validate the effectiveness of our BPA, also
16

---

evaluate BPA using PGD with 16/255. As shown in Table A8 BPA consistently outperforms the baselines with clear margin. showing its remarkable effectiveness in boosting adversarial transferabilityy with various perturbation budgets.
Attacker
Method
DenseNet
MobileNet
PNASNet
SENNet
v3
Inc Inc-y3onnd
IncRes V2
N/A
36.42
29.90
65.08
62.46
30.40
38.60
21.26
18.70
12.60
SGM
49.62
41 (1.60
77.78
80.14
46.04
56.24
29.28
24.18
17.20
PGD
LinBP
60.48
53.12
87:24
86.60
53.84
67.74
36.60
28.86
20.24
Ghoot
39.90
31.42
67.98
67.84
32.26
40.74
22.62
19.68
13.76
BPA
72.06
66.56
93.42
92.28
70.26
78.74
49.76
40.80
31,08
Table A8: Untargeted attack success rates (%) of PGD on nine models when generating adversarial
examples on ResNet-50 w/wo model-related methods using e = 16/255.
A.5 Additional Evaluation on CIFAR-10 Dataset
Existing transfer-based attacks mainly validate their effectiveness Ima- geNet dataset. For a fair compari- son, also evaluate BPA on Im- ageNet dataset. Here we also conduct experiments CIFAR-10 using PGD) with e 205 in Table A9, BPA exhibits better trans- ferability than the baselines, showing its high effectiveness and generality to various datasets and models.
Attacker
Method
WRN
ResNeXt
DenseNet
pyramidaet
gdas
N/A
65.94
65.66 .6.66
63.56
16.96
50.00
LinBP
67.88
67.46
65.02
18.18
51.78
PGD
Ghost
66.46
65.52
62.92
17.32
49.18
BPA
74.38
73,80
69.66
20.74
57.16
8 on VGG-19. As shown
Table A9: Untargeted attack success rates (%) of several
attacks on five models for CIFAR-10 dataset when generating the adversarial examples on VGG-19 19 w/wo various model- related methods.
A.6 Additional Evaluation on Targeted Attacks
Targeted attacks are more challenging than untargeted attacks. To further validate the effectiveness and generality of BPA, we also perform the targeted attack on VGG-19, following the experimental settings in Sec. The results are summarized Table A10 is interesting that LinBP decays the targeted attack performance on VGG-19. Since there is no skip connection in VGG-19, LinBP only modifies the derivative of ReLU, which might introduce an imprecise gradient. This highlights the significance that BPA excludes extreme values from consideration when calculating the gradient for better transferability. It is evident that our BPA achieves the best attack performance among various methods. Overall, BPA outperforms LinBP and Ghost by 18% and 7.90% for PGD, and 2.43% and 2.46% for MI-FGSM. These results further validate the effectiveness of BPA in targeted attacks, demonstrating its superiority over the baselines. The improved performance of BPA showcases its potential and generality in enhancing targeted attacks on various models.
Attacker
Method
Inc-v3
IncRes-v2
DenseNet MobileNet
PNASNet
SENet
Inc-v3
Inc-V3euut
IncRes-V2....
N/A
1.26
1.26
3.80
2.72
5.10
4.32
0.10
0.04
U.02
LinBP
1.26
1.22
3.44
2.24
4.26
3.52
0.26
0.12
0.02
PGD
Gbost
1.34
1.26
3.88
2.52
5.26
4.40
0.12
0.06
0.02
BPA
6.70
7,30
19.44
12.56
23.34
17.32
1,80
0,76
0,74
N/A
0.18
0.10
1.00
0.92
1.02
1.14
0.00
0.00
0.02
MI-FGSM
LinBP
0,24
0.20
1.18
0.86
0.94
1.02
0.02
0.00
0.02
Ghost
0.22
0.14
0.94
0.74
1.04
1.12
0.00
0.02
0.02
BPA
1.24
1.24
5.60
4.22
7,06
6,80
0.12
0.02
0.04
Table A10: Targeted attack success rates (%) of various attackers on nine models when generating adversarial examples on VGG-19 w/wo model-related methods using PGD and MI-FGSM.
A.7 Additional Evaluation on Modifications of ReLU Layers with BPA and LinBP
In this work, we find that the truncation of non-linear layers (e.g.. ReLU, max-pooling) decays the relevance between the gradient w.r.. the input and loss. Making the model more linear can enhance such relevance, thus improving the transferability. However, making the model more linear is not optimal. Taking ReLU for example, we compare the transferability of PGD on LinBP and BPA by
17

---

solely changing the derivatives of ReLU. Here LinBP makes the model more linear than our BPA. As shown in Table ATT BPA exhibits better transferability than LinBP, which supports our argument.
Attacker
Method
Ino-v3
IncRes-v2
DenseNet
MobileNet
PNASNet
SENet
Ino-V3emat
InC-V3
IncRes-V2ena
N/A
16.34
13.38
36.86
36.12
13.46
17.14
10.24
9.46
5.52
PGD
LinBP
27.22
23.04
59.34
59.74
22.68
33.72
16.24
13.58
7.88
BPA
29.38
24.00
62.80
61.82
24.98
34,96
17.52
14.38
8.90
Table A11: Untargeted attack success rates (%) of PGD on nine models when generating ad adversarial examples on ResNet-50 with modifications on ReI.U layers.
A.8 Comparison between Random Replacement and BPA on Max-pooling Layers
For max-pooling, the initial replacement of zeros in the gradient with ones results in an increase in relevance and subsequent improvement in attack performance. However, the number of replaced zeros increases, the vallina BP faces challenges in accurately discerning which elements are critical that related to the magnitude of input values. This indiscriminate replacement introduces substantial error, leading to a decay in attack performance. Hence, we also compare randomly replacing the zeros with ones using the probability of 0.3 (the best reult in Fig. 2cp and BPA for max- maxx-oooing on ResNet-50. As shown in Table|A12 BPA exhibits better transferability, which further validates our motivation and indicates that the effectiveness of our BPA not solely attributed to achieving linearity.
Attacker
Method
Inc-v3
IncRes-v2
DenseNet
MobileNet
PNASNet
SENet
Inc-v3
Inc Inc-v
IncRet- x2
N/A
16.34
13.38
36.86
36.12
13:46
17.14
10.24
9.46
5.52
PGD
(eplace(0.3)
14:52
11.94
37.52
36.02
13.84
17.28
10.34
10.56
6.48
BPA
20.26
16.16
44.66
42.82
17.12
21.52
13.20
11,88
7.74
Table A12: Untargeted attack success rates (%) of PGD on nine models when generating adversarial examples on ResNet-50 with modifications on max-pooling layers.
A.9 Relevance between Gradient w.r.f. Input and Loss Function
The relevance between gradient w.t. input and loss function indicates the sensitivity of the loss function to changes the input when taking a small step in the direction of gradient. This relevance can be formally defined as follows:
Definition (Relevance between gradient w.r.t. input and loss function) Given an input x, loss function J (x) and a step size the relevance between gradient w.r.t. input and objective function can be defined Jiz+e ((z))
ReLU helps address the vanishing gradient issue during the training of deep models by eliminating some gradients. However, BPA focuses on boosting transferability of adversarial examples generated on such ReL.Uacctivate, networks. To effectively generate adversarial examples, it is crucial that the gradient w.r.t. the input provides a reliable direction to maximize the loss. Unfortunately, the truncation of ReLU makes the calculated gradient unable to provide such exactly precise direction, egg., making the gradient weakened to some extent. Similarly, max-pooling also truncates the gradient during the backpropagation, causing the gradient unable to indicate exactly precise direction.
As summarized above, the truncation of ReLU and max-pooling drops gradient (introduces zeros) in the backpropagation process, which decays the relevance. We also calculate the Relevance using vallina backpropagation and our backpropagation on ResNet-50 using 1, 000 images. BPA achieved the relevance of 240.51, while the gradient calculated by the vallina backpropgation achieves lower relevance(14)) (14..33) which supports our hypothesis that non-linear layers can decay the relevance.
18


Reviews:
[{"review_id": "Fw2RP6jzir", "reviewer": "Reviewer_5iKz", "summary": "This paper introduces the Backward Propagation Attack (BPA) to improve the transferability of adversarial examples generated based on the iterative gradient ascent procedure. The paper first empirically demonstrates that the non-linear layers such as ReLU and max-pooling truncate the loss gradient, and such a truncation procedure negatively affects the transferability of adversarial perturbations. Motivated by this observation, the paper proposes to modify the gradient computation for the ReLU and max-pooling layers. Particularly, consider a trained source model from which the adversarial perturbations are generated. BPA proposes to replace the derivative of the ReLU functions in the source model with the derivative of the SiLU function. Similarly, the derivative of the max-pooling function in the source model is replaced with a softmax function with a tunable temperature coefficient. Finally, results on randomly sampled 5000 Imagenet images demonstrate the improved transferability of adversarial perturbations generated based on PGD and MI-FGSM.", "questions": "Questions/clarifications:\nIn the abstract, why does the truncation make the gradient imprecise? The ReLU and the max-pooling layers are part of the network definition, so the derivative is exactly what it is supposed to be and I do not see the lack of precision. What does it mean by the relevance between gradient wrt input and the loss function? Since the term \"relevance\" appears later in the paper as well (eg. ln45, 58, 148, 315), having a more rigorous definition is warranted.\n\nFigure 2 is a crucial element in motivating the proposed method. Is the success rate averaged over nine victim models? What is the variance among the results?\n\nLn 124 of Sec. 3.1: the paper first establishes that $z_i$ is the input to the function $f_i$ which returns $z_{i+1}$. When predicting the label, why is $z_l$ used as an input to the function $f_{l+1}$, shouldn't it be $z_{l+1}$?\n\nLn 138: Shouldnt the j-th element of $\\frac{\\partial z_{k+1}}{\\partial z_k}$ be 1 if $f(z_{k})_j >0$? \n\nLn 139 \"As a result, the gradient is effectively limited or weakened to some extent\". ReLU was designed to address the vanishing gradient problem in training deep models. By having a step function as its derivative, it allows some gradient information to pass freely while simultaneously not updating other weights. Therefore, this is exactly how the backward propagation was designed to be with ReLU-activated networks.\n\nLn 414: Similarly, this is exactly how bp is supposed to behave with max-pooling.\n\nLn 149: What does it mean by \"decaying the relevance\"? \n\nLn 185-186: Is this a prior result? or a claim made in this paper supported by evidence?\n\nSuggestions: Could we replace the y-axis (\"Attack success rate\") with the adversarial loss value on the source model? That is to plot $\\ell(x+\\Delta x)$, where $\\Delta x$ is generated using fgsm/i-fgsm/mi-fsgm on the source model modified using the various method. If we observe a higher loss value after the modification, it indeed shows that the derivative of those nonlinear layers prevents the perturbations from reaching maxima. On the other hand, if we observe a lower loss value after the modification, this means the truncation effect of the derivative of the non-linear functions leads to perturbations overfit to the source model, despite having higher losses.\n\nOther minor suggestions:\nFigure 1 should not come after Figure 2.", "limitations": "There is no discussion of the limitation. For instance, the proposed paper is only suitable for attack algorithms that are based on gradient computations. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: Modifying the backward propagation procedure of the non-linear layers when computing the loss gradient during the process of generating adversarial examples is an interesting approach to improve the transferability of the perturbations. \n\nQuality: The paper is well-written. The proposed BPA method is motivated using empirical observations. The evaluations include the transferability of perturbations under both targeted and untargeted settings.\n\nClarity: The motivation behind the proposed method as well as the structure of the paper is clear.\n\nSignificance: The adversarial robustness of deep learning models against transfer-based perturbations poses a practical security concern. This paper proposes a method that can be used in combination with existing gradient-based attack algorithms to improve the transferability of perturbations. \n", "weaknesses": "The clarity of the paper can be improved. There are some terms that are key to understanding the message which the author wants to convey, but the lack of rigorous definitions makes it difficult to understand. (See Questions for a list of clarifications )\n\nThe evaluations in the paper are performed based on perturbations bounded by the L_inf norm with an $\\epsilon$ of $\\frac{8}{255}$. In Ln 242, it states that the choice of $\\epsilon$ aligns with existing work (which one?), whereas most existing work considers $\\frac{16}{255}$ (LBP, MI, VI/VMI, etc). Especially since one of the main contributions of the proposed method is the improvement of transferability over other sota methods (ie. \"remarkable margin\" ln284, ln296, ln 358), it is important to make sure the evaluations and the comparison with other methods are performed fairly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We address your concerns below.\n\n**Q.1**. Perturbation budget.\n>**A.1**. Both $\\frac{8}{256}$ and $\\frac{16}{256}$ are widely adopted perturbation budgets in adversarial learning. In this paper, we follow the setting of baselines [1,2] to set $\\epsilon=\\frac{8}{256}$ for fair comparison. As you said, there are many transfer-based attacks that have adopted $\\frac{16}{256}$. Here we also evaluate BPA using PGD with $\\epsilon=\\frac{16}{256}$. As shown in Tab. 1 (APDF), BPA consistently outperforms the baselines with a clear margin, showing its remarkable effectiveness in boosting adversarial transferability. We will add the complete results in the revision.\n>\n>[1] Guo et al. Backpropagating Linearly Improves Transferability of Adversarial Examples. NeurIPS 2020.\n>\n>[2] Li et al. Learning Transferable Adversarial Examples via Ghost Networks. AAAI 2020.\n\n**Q.2**. Why does the truncation make the gradient imprecise?\n>**A.2**. Please refer to **G.1** for details. The relevance between gradient w.r.t. input and loss function indicates the sensitivity of loss function to changes in the input when taking a small step in the direction of gradient. This relevance can be formally defined as follows:\n>\n>**Definition 1 (Relevance between gradient w.r.t. input and loss function)** Given an input $x$, loss function $J(x)$ and a step size $\\epsilon$, the relevance between gradient w.r.t. input and objective function can be defined as $\\frac{J(x+\\epsilon \\cdot \\nabla_xJ(x)) - J(x)}{\\epsilon}$.\n>\n>As shown in **A.6**, BPA can increase the such relevance compared to vallina backpropagation, which further validates our motivation. We will clarify it in the revision. Thanks.\n\n**Q.3**. Is the success rate averaged over nine victim models in Fig. 2? What is the variance among the results?\n>**A.3**. Fig. 2 reports the average success rates on nine victim models due to page limit. As you asked, we calculate the variance of each data point in Tab. 2-4 (APDF). Since the attack success rates varies across different models, especially on standardly trained model (e,g., DenseNet) and adversarially trained model (e.g., Inc-v3$_{ens3}$), the large variance is expected and cannot provide any meaningful information. We also provide the original data for MI-FGSM of Fig. 2 (a) in Tab. 5 (APDF), which exhibit the consistent trend for each model. We will report these data in the appendix for reference.\n\n**Q.4**. Ln 124 & Ln 138.\n>**A.4**. They are typos. We will correct it in the revision.\n\n**Q.5**. Ln 139 \"As a result, the gradient is effectively limited or weakened to some extent\". ReLU was designed to address the vanishing gradient problem in training deep models. By having a step function as its derivative, it allows some gradient information to pass freely while simultaneously not updating other weights. Therefore, this is exactly how the backward propagation was designed to be with ReLU-activated networks. Ln 414: Similarly, this is exactly how bp is supposed to behave with max-pooling.\n>**A.5**. ReLU helps address the vanishing gradient issue during the training of deep models by eliminating some gradients (referred to as the truncation of ReLU in this paper). However, this paper focus on boosting transferability of adversarial examples generated on such ReLU-activated networks. To effectively generate adversarial examples, it is crucial that the gradient w.r.t. the input provides a reliable direction to maximize the loss. Unfortunately, as discussed in A.2, the truncation of ReLU makes the calculated gradient unnable to provide such exactly precise direction, i.e., making the gradient weakened to some extent. Similarly, max-pooling also truncates the gradient during the backpropagation, cuasing the gradient unnable to indicate an exactly precise direction. \n\n>In Fig. 2, we empirically validate that recovering such dropped gradient (though not precise) can effectively boost adversarial transferability. Motivated by this finding, we modify the backpropoagation for ReLu and max-pooling layers to recover the dropped gradient for better transferability.\n\n**Q.6**. Ln 149: What does it mean by \"decaying the relevance\"?\n>**A.6**. As responsed in **A.2** and **A.5**, the truncation of ReLU and max-pooling drops gradient in the backpropagation process, which decays the relevance. Based on Definition 1, we calculate the Relevance using vallina backpropagation and BPA on ResNet-50 with 1000 images. The gradient calculated by vallina backpropgation achieves the relevance of $149.23$, lower than BPA of $240.51$. This supports our hypothesis that non-linear layers can *decay the relevance*. We will add them in the Appendix and make it clearer.\n\n**Q.7**. Ln 185-186: Is this a prior result? or a claim made in this paper supported by evidence?\n>**A.7**. It is a claim made in this paper. As shown in Tab. 1 and Tab. 5, by only modifying ReLU, BPA achieves better performance than LinBP, supporting this claim. **G.2** further validates this claim.\n\n**Q.8**. Could we replace the y-axis (\"Attack success rate\") with the adversarial loss value on the source model?\n>**A.8**. In Fig. 2, we validate our hypothesis that we can increase such relevance to boost adversarial transferability. Thus, we adopt the Attack success rates as our y-axis. Since adversarial attacks aim to maximize the loss value to generate adversarial examples, there exists a positive relationship between the attack success rate and the average loss value. Following your suggestion, we have calculated the adversarial loss value, which shows a similar trend as attack success rate in Fig. 2. This analysis reinforces the findings drawn from the attack success rates, further supporting the hypothesis that increasing relevance leads to improved adversarial transferability.\n\n**Q.9**. Fig. 1 should not come after Fig. 2. & Limitation\n>**A.9**. We will swap their order. Please refer to **G.3** for limitation. Thanks."}, {"review_id": "xd7aLl2HKA", "reviewer": "Reviewer_NESJ", "summary": "This paper introduces a new way to improve adversarial transferability by removing gradient truncation in the surrogate model. Specifically, the method replaces gradients of relu and maxpool with softened version to avoid gradient truncation. Experiments are conducted over many models and show gains upon existing methods.", "questions": "See above", "limitations": "Not addressed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The method seems well motivated, novel and straightforward.\n- The empirical gain is substantial and general across different settings\n- The paper is well written overall and easy to follow.", "weaknesses": "It is not clear to me whether the improvement is caused by the extra design to align with the function better, or just simply by making the model more linear as suggested in LinBP. For the relu case, it seems that replacing gradient with 1 is the best as shown in prelimiary study. It is not clear whether this is still the case in the general experiments due to lack of ablation. For maxpool, Fig 2 also shows that the attack is best at some middle point. Does this relate to linearity too? And why does the success rate first increases then decreases?\n ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We address your concerns below.\n\n**Q1**. It is not clear to me whether the improvement is caused by the extra design to align with the function better, or just simply by making the model more linear as suggested in LinBP. For the ReLU case, it seems that replacing gradient with 1 is the best as shown in the preliminary study. It is not clear whether this is still the case in the general experiments due to the lack of ablation. For max-pooling, Fig 2 also shows that the attack is best at some middle point. Does this relate to linearity too? And why does the success rate first increase and then decrease?\n\n> **A.1**. 1) In this work, we find that the truncation of non-linear layers (e.g., ReLU, max-pooling) decays the relevance between the gradient w.r.t. the input and loss (see **G1** for more details). Making the model more linear can enhance such relevance, thus improving the transferability. However, making the model more linear is not optimal. Taking ReLU for example, we compare the transferability of PGD on LinBP and BPA by solely changing the derivatives of ReLU. Here LinBP makes the model more linear than our BPA. As shown in the following table, BPA exhibits better transferability than LinBP, which supports our argument. \n> | | Inc-v3 | IncRes-v2 | DenseNet | MobileNet | PNASNet | SENet | Inc-v3$_{ens3}$ | Inc-v3$_{ens4}$ | IncRes-v2$_{ens}$ |\n> |-|-|-|-|-|-|-|-|-|-|\n> |LinBP| 27.22 | 23.04 | 59.34 | 59.74 | 22.68 | 33.72 | 16.24 | 13.58 | 7.88|\n> |BPA| **29.38** | **24.00** | **62.80** | **61.82** | **24.98** | **34.96** | **17.52** | **14.38** | **8.90** |\n> \n> 2\\) In the preliminary, we validate our hypothesis that we can increase such relevance to boost adversarial transferability. To increase the relevance, we randomly replace the zeros in the gradient of ReLU with ones. The increasing attack performance can validate our hypothesis but it does not yield an optimal solution.\n>\n> 3\\) For max-pooling, the initial replacement of zeros in the gradient with ones results in an increase in relevance and subsequent improvement in attack performance. However, as the number of replaced zeros increases, the vallina BP faces challenges in accurately discerning the critical elements that are related to the magnitude of input values. This indiscriminate replacement introduces a substantial error, leading to a decay in attack performance. Hence, we adopt Equation (3) to address such issue. We also compare randomly replacing the zeros with ones using the probability of $0.3$ (the best results in Figure 2 (c)) and our BPA for max-pooling on ResNet-50. As shown in the following table, BPA exhibits better transferability, which further validates our motivation and indicates that the effectiveness of our BPA is not solely attributed to achieving linearity.\n> \n> | | Inc-v3 | IncRes-v2 | DenseNet | MobileNet | PNASNet | SENet | Inc-v3$_{ens3}$ | Inc-v3$_{ens4}$ | IncRes-v2$_{ens}$ |\n> |-|-|-|-|-|-|-|-|-|-|\n> |LinBP| 14.52 | 11.94 | 37.52 | 36.02 | 13.84 | 17.28| 10.34 | 10.56 | 6.48 |\n> |BPA| **20.26** | **16.16** | **44.66** | **42.82** | **17.12** | **21.52** | **13.20** | **11.88** | **7.74**|\n> \n> We will add the above results and discussion in the revision to clarify it. Thanks."}, {"review_id": "CdzMVHu3IR", "reviewer": "Reviewer_gcCC", "summary": "In this work, the authors find that non-linear layers make the gradient imprecise, leading to limited adversarial transferability. Based on this finding, they propose Backward Propagation Attack (BPA) to increase such relevance. In particular, BPA adopts a non-monotonic function as the derivative of ReLU and incorporates softmax function to smooth the derivative of max-pooling. Extensive experiments have shown the high effectiveness of the proposed BPA.", "questions": "1.What is the temperature coefficient for max-pooling?", "limitations": "N/A", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.The paper is well-written and easy to follow.\n2.The motivation is clear and reasonable. Intuitively, the non-linear layers can indeed truncate the gradient during the backward propagation.\n3.The proposed method is novel and interesting. Simply adjusting the gradient calculation of non-linear layers can significantly boost the adversarial transferability, which also validates the motivation.\n4.Different from some works which is limited to some specific architectures, the proposed method is general to existing CNNs.\n5.The authors have conducted experiments on both untargeted and targeted attacks, which can validate its high effectiveness and show good generality when combined with existing model-related methods.", "weaknesses": "1.For ReLU activation, the authors adopt the derivative of SiLU function. As shown in Figure 3, the derivative of SiLU function is mainly different from that of ReLU function in the range of [-5, 5]. I am curious how many elements the derivative matrix are in such range so that BPA can boost the transferability.\n2.Recently, transforming the input image for gradient calculation has shown great effectiveness in boosting adversarial transferability, such as DIM, Admix, SSA and so on. It is expected to see the results on such attacks to further validate the effectiveness of BPA on various transfer-based attacks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We address your concerns below.\n\n**Q.1**. For ReLU activation, the authors adopt the derivative of SiLU function. As shown in Figure 3, the derivative of SiLU function is mainly different from that of ReLU function in the range of [-5, 5]. I am curious how many elements the derivative matrix are in such range so that BPA can boost the transferability. \n\n> **A.1**. As you suggested, we have examined the range of elements in the derivative matrix and observed that the majority of the elements fall within the interval $[-2,2]$. Additionally, more than $99.98\\%$ elemnts are in this range $[-5,5]$. These findings strongly suggest that our gradient candidate possesses the ability to effectively handle most elements, making our BPA effective in boosting adversarial transferability.\n\n**Q.2**. It is expected to see the results on such attacks to further validate the effectiveness of BPA on various transfer-based attacks.\n\n> **A.2**. Our BPA modifies the backpropagation process, which is general to various transfer-based attacks. To validate its effectiveness, we adopt gradient-based (PGD), momentum-based (MI-FGSM, VMI-FGSM), objective-related (ILA) and input transformation based (SSA) attacks for evaluation. Due to the page limit, we only evaluate BPA with one up-to-date transformation based attack (SSA) in our paper. As you suggested, we further evaluate its generality to other input transformation based attacks in the following tables, namely DIM and TIM. As we can see, our BPA can significantly boost these input transformation based attacks and exhibits better transferability than other model-related works, which further validates its superiority in boosting adversarial transferability.\n>\n> **DIM**:\n> | Attacker | Inv-v3 | InvRes-v2 | DenseNet | MobileNet | PNASNet | SENet | Inc-v3$_{ens3}$ | Inc-v3$_{ens4}$ | IncRes-v2$_{ens}$ |\n> | -------- | ------ | --------- | -------- | --------- | ------- | ----- | --------------- | --------------- | ----------------- |\n> | N/A | 45.00 | 38.56 | 71.64 | 70.08 | 41.60 | 48.56 | 28.52 | 24.82 | 16.48             |\n> | SGM | 52.72 | 45.22 | 79.42 | 82.34 | 49.50 | 58.66 | 32.42 | 28.20 | 19.26 |\n> | LinBP | 45.74 | 38.38 | 76.34 | 77.58 | 39.28 | 50.20 | 27.40 | 22.60 | 15.50 |\n> | Ghost | 45.20 | 37.86 | 72.70 | 72.34 | 40.32 | 48.50 | 27.44 | 23.98 | 15.78             |\n> | BPA | **59.20** | **50.86** | **87.70** | **86.92** | **55.40**  | **62.68** | **40.32** | **34.84** | **24.42** |\n> \n> **TIM**:\n> | Attacker | Inv-v3 | InvRes-v2 | DenseNet | MobileNet | PNASNet | SENet | Inc-v3$_{ens3}$ | Inc-v3$_{ens4}$ | IncRes-v2$_{ens}$ |\n> | -------- | ------ | --------- | -------- | --------- | ------- | ----- | --------------- | --------------- | ----------------- |\n> | N/A      | 32.58  | 26.66     | 58.44    | 55.44     | 29.26   | 34.84 | 21.38           | 18.76           | 13.54             |\n> | SGM      | 41.18  | 35.22     | 71.12    | 72.20     | 41.24   | 47.88 | 25.50           | 23.66           | 16.28             |\n> | LinBP    | 43.20  | 36.30     | 75.08    | 74.08     | 39.14   | 46.90 | 27.62           | 23.62           | 16.74             |\n> | Ghost    | 37.08  | 29.36     | 66.48    | 63.26     | 33.26   | 39.74 | 23.58           | 21.14           | 14.54             |\n> | BPA      | **59.20**  | **50.86**     | **87.70**    | **86.92**     | **55.40**   | **62.68** | **40.32**           | **34.84**           | **24.42**             |\n\n\n**Q.3**. What is the temperature coefficient for max-pooling?\n\n>**A.3**. We set the temperature coefficient $t=10$ for ResNet-50 and conducted a parameter study in Section 4.4. We will add it into hyper-parameters in Section 4.1. Thanks."}, {"review_id": "Mfl8gmnUUN", "reviewer": "Reviewer_g444", "summary": "This paper  have developed a new method, Backward Propagation Attack (BPA), to enhance the transferability of adversarial examples. BPA uses a non-monotonic function as the derivative of ReLU and incorporates softmax with temperature to smooth the derivative of max-pooling, reducing information loss during gradient backward propagation. The method has demonstrated significant improvements in adversarial transferability on the ImageNet dataset.", "questions": "- Could your technique potentially work in conjunction with input-transformation-based attacks such as TIM/SIN/DIM  for further improving the transferability?", "limitations": "the authors adequately addressed the limitations", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**Originality**: To the best of my knowledge, the idea that modifying the derivative of max-pooling, Relu activation for improving adversarial transferability is new. \n\n **Clarity**: This paper is well-structured and easy to follow. \n\n**Significance**: To understand and improve the transferability of adversarial examples is important in the domain of adversarial examples ", "weaknesses": "- The chosen baselines are exclusively drawn from model-related attacks. It might be beneficial to include a few from input-transformation-based attacks. Furthermore, it would provide a more comprehensive evaluation to examine if your technique could potentially work in conjunction with input-transformation-based attacks such as TIM[1]/SIN[2]/DIM[3] for further enhancing the transferability.\n\n- There's a relevant piece of work that hasn't been included.[4]\n\n[1]  Dong, Yinpeng, et al. \"Evading defenses to transferable adversarial examples by translation-invariant attacks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.\n\n[2] Lin, Jiadong, et al. \"Nesterov accelerated gradient and scale invariance for adversarial attacks.\" arXiv preprint arXiv:1908.06281 (2019).\n\n[3] Xie, Cihang, et al. \"Improving transferability of adversarial examples with input diversity.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n[4] Huang, Tianjin, et al. \"Direction-aggregated attack for transferable adversarial examples.\" ACM Journal on Emerging Technologies in Computing Systems (JETC) 18.3 (2022): 1-22.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We address your concerns below.\n\n**Q.1**. The chosen baselines are exclusively drawn from model-related attacks. It might be beneficial to include a few from input-transformation-based attacks. Furthermore, it would provide a more comprehensive evaluation to examine if your technique could potentially work in conjunction with input-transformation-based attacks such as TIM[1]/SIN[2]/DIM[3] for further enhancing the transferability.\n\n> **A.1**. In Table 1, we compare our BPA with various model-related attacks when combined with other attacks, including gradient-based (PGD), momentum-based (MI-FGSM, VMI-FGSM), objective-related (ILA) and input transformation based (SSA) attacks. Due to the page limit, we only evaluate BPA with one up-to-date transformation based attack (SSA). As you suggested, we further evaluate its generality to other input transformation based attacks, namely DIM, TIM and SIN in the following tables. As we can see, our BPA can significantly boost these input transformation based attacks. Compared with existing model-related attacks, BPA consistently exhibits better transferability for TIM and DIM. For SIN, BPA exhibits comparable attack performance with SGM on standardly trained models but much better transferability on adversarially trained models. To integrate the advantage of SGM and our BPA, we combine BPA with SGM for SIN, which outperforms the baselines with a clear margin. These results further validate its superiority in boosting adversarial transferability. We will add them in the revision. Thanks.\n> \n> **DIM**:\n>|Attacker|Inv-v3|InvRes-v2|DenseNet|MobileNet|PNASNet|SENet|Inc-v3$_{ens3}$|Inc-v3$_{ens4}$|IncRes-v2$_{ens}$ |\n>|-|-|-|-|-|-|-|-|-|- |\n>|N/A|45.00|38.56|71.64|70.08|41.60|48.56|28.52|24.82|16.48|\n>|SGM|52.72|45.22|79.42|82.34|49.50|58.66|32.42|28.20|19.26|\n>|LinBP|45.74|38.38|76.34|77.58|39.28|50.20|27.40|22.60|15.50|\n>|Ghost|45.20|37.86|72.70|72.34|40.32|48.50|27.44|23.98|15.78|\n>|BPA|**59.20**|**50.86**|**87.70**|**86.92**|**55.40**|**62.68**|**40.32**|**34.84**|**24.42**|\n> \n> **TIM**\n>|Attacker|Inv-v3|InvRes-v2|DenseNet|MobileNet|PNASNet|SENet|Inc-v3$_{ens3}$|Inc-v3$_{ens4}$|IncRes-v2$_{ens}$ |\n>|-|-|-|-|-|-|-|-|-|-|\n>|N/A|32.58|26.66|58.44|55.44|29.26|34.84|21.38|18.76|13.54|\n>|SGM|41.18|35.22|71.12|72.20|41.24|47.88|25.50|23.66|16.28|\n>|LinBP|43.20|36.30|75.08|74.08|39.14|46.90|27.62|23.62|16.74|\n>|Ghost|37.08|29.36|66.48|63.26|33.26|39.74|23.58|21.14|14.54|\n>|BPA|**59.20**|**50.86**|**87.70**|**86.92**|**55.40**|**62.68**|**40.32**|**34.84**|**24.42**|\n>\n> **SIN**\n>|Attacker|Inv-v3|InvRes-v2|DenseNet|MobileNet|PNASNet|SENet|Inc-v3$_{ens3}$|Inc-v3$_{ens4}$|IncRes-v2$_{ens}$ |\n>|-|-|-|-|-|-|-|-|-|-|\n>|N/A|42.68|33.76|70.12|65.70|34.90|39.90|26.36|23.48|15.22|\n>|SGM|52.78|43.04|79.92|80.22|45.10|52.70|31.94|27.44|18.74|\n>|LinBP|50.46|41.06|78.22|77.14|38.82|47.64|30.38|25.08|17.14|\n>|Ghost|47.46|37.92|77.14|72.82|38.66|46.02|29.14|24.58|16.28|\n>|BPA|52.40|43.44|80.12|76.38|45.42|50.28|39.16|36.32|26.06|\n>|SGM+BPA|**62.26**|**54.12**|**88.04**|**88.28**|**56.80**|**63.46**|**46.08**|**39.80**|**30.68**|\n> \n> [1] Dong, Yinpeng, et al. \"Evading defenses to transferable adversarial examples by translation-invariant attacks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.\n>\n> [2] Lin, Jiadong, et al. \"Nesterov accelerated gradient and scale invariance for adversarial attacks.\" arXiv preprint arXiv:1908.06281 (2019).\n> \n> [3] Xie, Cihang, et al. \"Improving transferability of adversarial examples with input diversity.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n**Q.2**. There's a relevant piece of work that hasn't been included.[4]\n> **A.2** Direction-Aggregated adversarial attack (DA) utilizes the aggregated direction during the attack process to prevent the generated adversarial examples from overfitting to the white-box model. This approach aligns well with our BPA, making them compatible for combined evaluation. To further validate the effectiveness of our BPA, we integrate it with DA-MI-FGSM and present the results in the following table. The results demonstrate that BPA can significantly boost the adversarial transferability of DA-MI-FGSM, surpassing the performance of other model-related attacks. This further validates the superiority of BPA in boosting adversarial transferability with good generality to various attacks. We will add it in the revision. Thank you for your valuable feedback and contributions to strengthening our research.\n>|Attacker|Inv-v3|InvRes-v2|DenseNet|MobileNet|PNASNet|SENet|Inc-v3$_{ens3}$|Inc-v3$_{ens4}$|IncRes-v2$_{ens}$|\n>|-|-|-|-|-|-|-|-|-|-|\n>|N/A|45.00|38.56|71.64|70.08|41.60|48.56|28.52|24.82|16.48|\n>|SGM|52.72|45.22|79.42|82.34|49.50|58.66|32.42|28.20|19.26|\n>|LinBP|45.74|38.38|76.34|77.58|39.28|50.20|27.40|22.60|15.50|\n>|Ghost|45.20|37.86|72.70|72.34|40.32|48.50|27.44|23.98|15.78|\n>|BPA|**59.20**|**50.86**|**87.70**|**86.92**|**55.40**|**62.68**|**40.32**|**34.84**|**24.42**|\n>\n> [4] Huang, Tianjin, et al. \"Direction-aggregated attack for transferable adversarial examples.\" ACM Journal on Emerging Technologies in Computing Systems (JETC) 18.3 (2022): 1-22."}, {"review_id": "onOSGjK2I7", "reviewer": "Reviewer_YaZN", "summary": "This paper presents the Backward Propagation Attack (BPA) as a solution to the issue of gradient truncation in transfer-based attacks within adversarial machine learning. Through empirical validation, the authors highlight the negative impact of gradient truncation on adversarial transferability and propose modifications to the backward propagation process. BPA effectively enhances the relevance of the gradient between the loss function and the input, resulting in significant improvements in various untargeted and targeted transfer-based attacks compared to baseline approaches. This work not only provides valuable insights for enhancing adversarial transferability but also suggests promising directions for bolstering model robustness.", "questions": "1.\tThe paper proposes modifications to the backward propagation process to mitigate gradient truncation. Could the authors provide more theoretical justification for the choice of these specific modifications?\n2.\tThe experimental evaluation is conducted on the ImageNet dataset. Have you considered testing the proposed BPA method on other datasets to assess its generalizability? It would be valuable to understand how the approach performs across different domains and datasets.\n3.\tIt would be beneficial to discuss the limitations of the proposed BPA method. Are there any specific scenarios or conditions where the effectiveness of BPA may be compromised? Besides, could the authors explain the main improvements of BPA compared with LinBP?\n4.\tThe paper primarily focuses on the enhancement of transfer-based attacks. Could you discuss potential applications or implications of the proposed BPA method in other areas of adversarial training, such as these gradient-based white-box attacks?\n", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper demonstrates originality by identifying and addressing the issue of gradient truncation in transfer-based attacks, which has not been extensively explored before. The proposed Backward Propagation Attack (BPA) introduces modifications to the backward propagation process, mitigating the detrimental effect of gradient truncation. The empirical evaluation on the widely used ImageNet dataset showcases the high quality of the research, with rigorous experimentation and comparisons against baselines. The clarity of the paper is commendable, effectively communicating the problem, the proposed solution, and the experimental methodology. The findings are significant as they contribute to advancing transfer-based attacks, provide new insights into adversarial transferability, and have potential implications for improving model robustness.", "weaknesses": "While the paper presents valuable contributions, there are a few areas that could be improved. Firstly, the theoretical underpinnings of the proposed modifications to the gradient truncation during the backward propagation process could be further elaborated to enhance the understanding of the approach. Additionally, the limitations of the proposed techniques could be further discussed to provide a comprehensive view of its effectiveness, such as the potential performance decrease caused by replacing the ReLU function. Furthermore, the paper focuses mainly on the ImageNet dataset, and it would be beneficial to explore the generalizability of the approach on other datasets or domains. Overall, addressing these weaknesses would strengthen the paper and provide further insights into the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We address your concerns below.\n\n**Q.1**. Theoretical justification.\n>**A.1**. The theoretical explanation of transfer-based attacks is elusive, rendering the selection of specific modifications for boosting transferability a complex challenge. In this work, we find that the truncation of non-linear layers engenders a decline in the relevance between gradient w.r.t. input and loss function, which hampers adversarial transferability. Our approach endeavors to enhance the relevance by designing new derivatives for ReLU and max-pooling layers, which are non-zero and related to the magnitude of input. As shown in Fig. 4 (b), we have conducted parameter studies for max-pooling. To substantiate the rationale behind the modification for ReLU, we conduct empirical justification as follows:\n>\n>+ **Derivative Range Analysis**. We examine 1000 images and find that the majority of the elements in the derivative matrixs of ReLU fall within the interval $[-2,2]$ and more than $99.98\\%$ elements are in the range $[-5,5]$. As shown in Fig. 3, the derivative of SiLU function is mainly different from that of ReLU function in the range of $[-5, 5]$. Hence, SiLU can work for almost all the elements. \n>+ **Scale the magnitude**. We add a scale factor $s$ for the derivative of SiLU to explore its amplitude influence. As shown in Tab. 6 (APDF), when increasing $s$, the attack performance firstly increases and achieves the peak at $s=0.8$. When we continue to increase the scale factor, it acts like LinBP which sets all the elements to ones and the performance starts to decay. Notably, the standard SiLU configuration ($s=1.0$) falls short of delivering optimal transferability. This underscores its untapped potential for further fortification of adversarial transferability.\n>\n>The empirical validations substantiates that while the chosen modifications exhibit promise in boosting transferability, the optimal choice may be yet undetermined. We will explore the potential modifications and try to derive theoretical justification in future work. Thanks.\n\n**Q.2**. Results on other datasets.\n>**A.2**. Existing transfer-based attacks mainly validate their effectiveness on ImageNet dataset. For a fair comparison, we also evaluate our BPA on ImageNet dataset. Here we conduct experiments on CIFAR-10 using PGD with $\\epsilon=\\frac{8}{255}$ on VGG-19. As shown in the table below, BPA exhibits better transferability than the baselines, showing its high effectiveness and generality to various datasets and models. We will add the complete results in the revision.\n>|Attacker|WRN|ResNeXt|DenseNet|pyramidnet|gdas|\n>|-|-|-|-|-|-|\n>|N/A|65.94|65.66|63.56|16.96|50.00|\n>|LinBP|67.88|67.46|65.02|18.18|51.78|\n>|Ghost|66.46|65.52|62.92|17.32|49.18|\n>|BPA|**74.38**|**73.80**|**69.66**|**20.74**|**57.16**|\n\n**Q.3**. Limitation and the main improvements of BPA.\n>**A.3**. Please refer to **G.3** for limitation. We summarize the main improvements of BPA compared with LinBP as follows:\n>+ **Motivation.** Different from LinBP which makes the deep model more linear, we find that the truncation of non-linear layers decays the relevance between the gradient w.r.t. input and loss function. It makes the gradient provide an imprecise direction to maximize the loss in some cases. This also explains why making the model more linear by LinBP results in better transferability. \n>+ **BPA considers non-linear layers.** Different from LinBP which only focuses on ReLU, we focus on recovering the gradient of non-linear layers, including ReLU and max-pooling. As shown in Table 5, with max-pooling, our BPA can achieve much better transferability than solely modifying ReLU, validating the effectiveness of considering a broader set of non-linear layers.\n>+ **BPA considers the magnitude of input.** Different from LinBP which uniformly sets all elements of ReLU's gradient to one, our BPA sets the gradient based on the magnitude of input matrix. It helps highlights the crucial elements based on the gradient, resulting in better transferability. To validate its advantage,  we compare the transferability of PGD on LinBP and BPA by solely changing the derivatives of ReLU. As shown in the following table, BPA exhibits better transferability than LinBP, supporting our argument.\n>\n>||Inc-v3|IncRes-v2|DenseNet|MobileNet|PNASNet|SENet|Inc-v3$_{ens3}$|Inc-v3$_{ens4}$|IncRes-v2$_{ens}$|\n>|-|-|-|-|-|-|-|-|-|-|\n>|LinBP|27.22|23.04|59.34|59.74|22.68|33.72|16.24|13.58|7.88|\n>|BPA|**29.38**|**24.00**|**62.80**|**61.82**|**24.98**|**34.96**|**17.52**|**14.38**|**8.90**|\n>\n>Though both LinBP and BPA modify the back-propagation process, they have distinct motivations and strategies to achieve better transferability. Considering a wider range of non-linear layers and modifying the gradient based on input magnitude, BPA achieves better adversarial transferability.\n\n**Q.4**. Potential applications of BPA.\n>**A.4**. As you suggested, we adopt FGSM to attack ResNet-50 with $\\epsilon=\\frac{8}{255}$ on ImageNet dataset. Integrating BPA with FGSM achieves a white-box attack success rate of $93.56\\%$, which outperforms that of the vanilla model ($86.76\\%$). This underscores the effectiveness of BPA for white-box attacks.\n>\n>Due to limited time and resources, we do not evaluate BPA for adversarial training. Smooth activation functions [1] can improve adversarial training, which shows the potential applicability of our BPA for adversarial training and warrants further exploration in future work.\n>\n>Note that this work mainly focuses on transfer-based attacks and does not discuss these potential applications in the paper. As you said, the proposed motivation and method might not be limited to transfer-based attacks. We will explore its potential application in adversarial learning in the future. Thanks for your insightful comments.\n>\n>[1] Xie et al. Smooth Adversarial Training. arXiv Preprint arXiv:2006.14536, 2020."}]

Author rebuttal:
We provide some experimental results in the attached PDF, denoted as APDF. We address the common concerns as follows:

> **G.1** If the gradient of a function is non-zero at a point $x$, the direction of gradient signifies the path in which the function experiences the steepest increase from $x$. Considering a function $f(x) = ReLU(wx+b)$ where $w >0, b, x \in R$, we have that $f(x)=0$ if $x\le-\frac{b}{w}$. Our goal is to maximize $f(x)$ starting from the data point $x_0$, which is analogous to adversarial attacks aimed to maximize the loss function. If $x_0 > -\frac{b}{w}$, $\frac{\partial f(x)}{\partial x}=w$ tells us that increasing $x$ will elevate $f(x)$. However, due to the truncation of ReLU, $\frac{\partial f(x)}{\partial x} =0$ when $x_0 \le -\frac{b}{w}$, fails to provide a clear direction for maximizing $f(x)$, rendering the optimization process ineffective. This truncation issue also arises during the backpropagation process in deep models, which we refer to as *"imprecise gradient"* in this paper. Thus, the *imprecise gradient* means that the gradient w.r.t. input image fails to indicate the direction to maximize/minimize the loss instead of the lack of precision for gradient calculation. 

> **G.2** To validate that setting all elements in the corresponding derivative to ones is not optimal, we compare the transferability of PGD on LinBP and BPA by solely changing the derivatives of ReLU. As shown in the following table, BPA exhibits better transferability than LinBP, which validates our claim.
>| | Inc-v3 | IncRes-v2 | DenseNet | MobileNet | PNASNet | SENet | Inc-v3$_{ens3}$ | Inc-v3$_{ens4}$ | IncRes-v2$_{ens}$ |
>|-|-|-|-|-|-|-|-|-|-|
>|LinBP| 27.22 | 23.04 | 59.34 | 59.74 | 22.68 | 33.72 | 16.24 | 13.58 | 7.88|
>|BPA| **29.38** | **24.00** | **62.80** | **61.82** | **24.98** | **34.96** | **17.52** | **14.38** | **8.90** |

> **G.3** Limitation: Our proposed BPA modifies backpropagation process for gradient calcaulation, making it only suitable for gradient-based attacks. Besides, BPA modifies the derivatives of non-linear layers, such as ReLU and max-pooling. Consequently, it may not be directly applicable to models lacking these specific components, such as transformers. In the future, we will investigate how to generalize our BPA to such transformers by refining the derivatives of some components, e.g., softmax. This endeavor to enhance the generality and versatility of BPA will be an essential aspect of ongoing research, paving the way for the broader applicability of the proposed method and facilitating its adoption in various deep learning models beyond those with ReLU and max-pooling layers.
>
> We will add the limitation discussion in the revision.

Meta-review: The paper presents a simple approach that improves the transferability of adversarial attacks. Based on the observation that the truncation of the gradients is highly correlated with the transferability of attack, the authors proposed to replace the gradient computations in ReLU and max-pooling to more smooth functions, and demonstrated the consistent improvement in attack success rates.

Overall, all reviewers were satisfied with the clarity and empirical validation of the approach. The paper initially received two Accepts, two Borderline Accepts, and one Borderline Reject. Primary concerns raised by the reviewers were about comparisons with various transfer attacks (e.g., the ones based on input transformation), comparisons on more datasets, clarifications on some claims (e.g., is linearity enough to enhance the relevance of the input to the loss?) and terminology (e.g., term “relevance”). The authors adequately addressed most concerns in the rebuttal, hence all reviewers recommend acceptance. The AC agrees with the reviewers’ decision. The authors should update the camera-ready paper to include the changes suggested by the reviewers during the review process.
Decision: Accept (poster)