Boosting Adversarial Transferability by Achieving Flat Local Maxima
Authors: Zhijin Ge
Hongying Liu
Xiaosen Wang
Fanhua Shang
Yuanyuan Liu
Venue: NeurIPS 2023 poster (2023)
Keywords: Adversarial attack
Adversarial transferability
Black-box Attack

Abstract: Transfer-based attack adopts the adversarial examples generated on the surrogate model to attack various models, making it applicable in the physical world and attracting increasing interest. Recently, various adversarial attacks have emerged to boost adversarial transferability from different perspectives. In this work, inspired by the observation that flat local minima are correlated with good generalization, we assume and empirically validate that adversarial examples at a flat local region tend to have good transferability by introducing a penalized gradient norm to the original loss function. Since directly optimizing the gradient regularization norm is computationally expensive and intractable for generating adversarial examples, we propose an approximation optimization method to simplify the gradient update of the objective function. Specifically, we randomly sample an example and adopt a first-order procedure to approximate the curvature of the second-order Hessian matrix, which makes computing more efficient by interpolating two Jacobian matrices. Meanwhile, in order to obtain a more stable gradient direction, we randomly sample multiple examples and average the gradients of these examples to reduce the variance due to random sampling during the iterative process. Extensive experimental results on the ImageNet-compatible dataset show that the proposed method can generate adversarial examples at flat local regions, and significantly improve the adversarial transferability on either normally trained models or adversarially trained models than the state-of-the-art attacks. Our codes are available at: https://github.com/Trustworthy-AI-Group/PGN.

Boosting Adversarial Transferability by Achieving Flat Local Maxima
Zhijin Gel; Hongying Liu?r, Xiaosen Wang"; Fanhua Shang", Yuanyuan Liu!t
'School of Artificial Intelligence, Xidian University
Medical College, Tianjin University, China
$ Huawei Singular Security Lab *College of Intelligence and Computing, Tianjin University zhij jaijiigestt.. xidian. edu. en, hyliu2009@tju. edu. cn xiaosenChust. edu. en, fhshang@tju. edu. en, yyliuxxidian, edu. cn
Abstract
Transfer-based attack adopts the adversarial examples generated on the surrogate model attack various models, making it applicable the physical world and attracting increasing interest. Recently, various adversarial attacks have emerged to boost adversarial transferability from different perspectives. In this work, inspired by the observation that flat local minima are correlated with good generalization, we assume and empirically validate that adversarial examples flat local region tend to have good transferability by introducing penalized gradient norm to the original loss function. Since directly optimizing the gradient regularization norm is computationally expensive and intractable for generating adversarial exam- ples, we propose an approximation optimization method simplify the gradient update of the objective function. Specifically, randomly sample exam- ple and adopt a first-order procedure to approximate the Hessian/vector product, which makes computing more efficient by interpolating two neighboring gradi- ents. Meanwhile, in order to obtain more stable gradient direction, we randomly sample multiple examples and average the gradients of these examples to reduce the variance due to random sampling during the itcrative process. Extensive ex- perimental results the ImageNet-compatibe dataset show that the proposed method can generate adversarial examples at flat local regions, and significantly improve the adversarial transferability on either normally trained models adver- sarially trained models than the state-of-the-an attacks. Our codes are available at: https://github.com Trustworthy--l--ooooe
Introduction
great number of works have shown that Deep Neural Networks (DNNs) are vulnerable to adversarial
examples [T0 16.
38 26 50), which generated by applying human-imperceptible perturbations
on clean input to result in misclassification. Furthermore, adversarial examples have an intriguing
property of transferability [10 [II 61 59 57,, i.e., the adversarial example generated from the surrogate model can also misclass other models. The existence of transferability makes adversarial attacks practical to real-world applications because hackers do not need to know any information about the target model, which introduces a series of serious security problems in security-sensitiig applications such as self-driving [65] [3) and nice-recognition 44 30
"Equal Contribution. Corresponding authors
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Although several attack methods [5] 16 26 28 have exhibited great attack effectiveness in the white-box setting, they have low transferability when attacking black-box models, especially for attribute that the reason for adversarial examples shows weak transferability due to dropping into poor local maxima or overfitting the surrogate model, which is not likely to transfer across models. To address this issue, many methods have been proposed from different perspectives. Gradient optimization attacks 100 31 49 644 attempt to boost black-box performance by advanced gradient calculation. Input transformation attacks [ B |I 55 15 55 aim to generate adversarial examples with higher transferability by applying various transformations to the inputs. Especially, the above methods are mainly proposed from the perspective of optimization and generalization, which regard the process of generating adversarial examples on the white-box model as a standard neural network training process, and treat adversarial transferability as equivalent to model generalization [1]] Although these methods can improve the transferability of adversarial examples, there are still some gaps between white-box attacks and transfer-basee black-box attacks.
some advanced defense models 36 47
Previous works [10 31 61]
Inspired by the observation that flat minima often result in better model generalization [24 66 [4]
we assume and empirically validate that adversarial examples at flat local region tend to have good transferability. Intuitively, we can achieve the flat local maxima for adversarial examples using a gradient regularization norm but it is computationally extensive to solve such a problem. To address this issue, we theoretically analyze the optimization process and propose novel attack called Penalizing Gradient Norm (PGN). In particular, PGN approximates the Hessian/vedd interpolating the first-order gradients of two samples. This approximation allows to efficiently generate adversarial examples at flat local regions. To eliminate the error introduced by approximation, PGN incorporates the average gradient of several randomly sampled data points to update the adversarial perturbation. Our main contributions can be summarized as follows:
product by
To the best of our knowledge, it is the first work that empirically validates that adversarial examples located in flat regions have good transferability.
We propose a novel attack called Penalizing Gradient Norm (PGN), which can effectively generate adversarial examples at flat local regions with better transferability.
Empirical evaluations show that PGN can significantly improve the attack transferability on both normally trained models and adversarially trained models, which can also be seamlessly combined with various previous attack methods for higher transferability.
2 Related Work
In this section, we provide a brief overview of the adversarial attack methods and introduce several studies on the flat local minima for model generalization.
2.1 Adversarial Attacks
In general, adversarial attacks can be divided into two categories, i.e., white-box attacks and black- box attacks. In the white-box setting, the target model is completely exposed to the attacker. For instance, Goodfellow et al. [16] proposed the Fast Gradient Sign Method (FGSM) to generate adversarial examples with one step of gradient update. Kurakin et al. (26) further extends FGSM to an iterative version with a smaller step size a, denoted as I-FGSM. Madry et al. [36] extends I-FGSM with a random start to generate diverse adversarial examples. Existing white-box attacks have achieved superior performance with the knowledge of the target model. On the other hand, black-box attacks are more practical since they only access limited or no information about the target model. There are two types of black-box adversarial attacks [21]: query-baser and transfer-basee attacks. Query- based attacks [3] 6 53 often take hundreds or even thousands of quires to generate adversarial examples, making them inefficient in the physical world. In contrast, munsfer-basee 56 48) generate adversarial examples the surrogate model, which can also attack other models without accessing the target model, leading to great practical applicability and attracting increasing attention.
attacks [6] 49 52 54
Unfortunately, adversarial examples crafted by white-box attacks generally exhibit limited transfer- ability. To boost adversarial transferability, various methods are proposed from the perspective of optimization and generalization. MI-FGSM [10] integrates momentum into I-FGSM to stabilize the update direction and escape from poor local maxima at each iteration. NI-FGSM B1 adopts
2

---

Nesterov's 's accelerated gradient [40] to further enhance the transferability. Wang et al. [49] tunned the gradient using the gradient variance of previous iteration to find more stable gradient direction. Wang et al. [52] enhanced the momentum by accumulating the gradient of several data points in the direction of the previous gradient for better transferability.
Data augmentation, which has shown high effectiveness in improving model generalization [37 63 2 g has been widely studied to boost adversarial transferability. Xie et al. [6]] adopted diverse input patterns by randomly resizing and padding to generate transferable adversarial examples. Dong er al. [II] utilized several translated images to optimize the adversarial perturbations, and further calculated the gradients by convolving the gradient at untranslated images with a kernel matrix for high efficiency. SIM [31] optimizes the adversarial perturbations over several scaled copies of the input images. Admix [31] mixes up set of images randomly sampled from other categories while
maintaining the original label of the input. Spectrum simulation attack (SSA) [35] transforms the input image in the frequency domain to craft more transferable adversarial examples.
Besides, some methods improve adversarial transferability from different perspectives. For instance, Liu et al. [32] proposed an ensemble attack, which simultancously attacks multiple surrogate models. Wu et al. 55 employed an adversarial transformation network that can capture the most harmful deformations to adversarial noises. Qin et al. 42] injected the reverse adversarial perturbation at each step of the optimization procedure for better transferability.
2.2
Flat Minima
After Hochreiter er al. (20) pointed out that well-generalized models may have flat minima. the connection between the flatness of minima and model generalization has been studied from both empirical and theoretical perspectives [24] 41 14 [6). Li et al. (27) observed that skip connections promote flat minima, which helps explain why skip connections are necessary for training extremely deep networks. Similarly, Santurkar et al. 43 found that BatchNorm makes the optimization landscape significantly smooth in the training process. Sharpness-Aware Minimization (SAM) [14) improves model generalization by simultaneously minimizing the loss value and sharpness, which seeks the parameters in the neighborhoods with uniformly low loss values. Jiang et al. (23] studied 40 complexity measures and showed that a sharpness-based measure has the highest correlation with generalization. Zhao et al. [66] demonstrated that adding a gradient norm of the loss function can help the optimizer find flat local minima.
3 Methodology
3.1 Preliminaries
Given an imput image r with its corresponding ground- label y. the deep model f with parameter 0 expected to output the prediction fz; 0) y with high probability. Let B,(B)) (a' || x=Ip S E} be an e-ball of an input image where e >> 0 is a pre-defined perturbation magnitude, and - lp denotes the 4, Ly-norm (e.g, the -n-omm) The attacker aims to find an example æade E (x) that misleads the classifier (2ndv, () * y. Let [zz y; ()) be the loss function (e.g., cross-entropy loss) of the classifier f. Existing white-box attacks such as [66] 26 I0 usually generate adversarial examples by solving the following maximization problem:
max ax 0)
gade (B,()) (a))
These attack methods mainly generate adversarial examples through gradient iterations. For instance, I-FGSM 26 iteratively updates the adversarial perturbation as follows:
ade = 1I (xr) 2 + a sign( (V Ladn (2qde y: 0))] x0 = x,
(2)
where Ig. (-) projects an input into (x(x), a e/T, and is the total number of iterations. For black-box attacks, the gradient is not accessible so that we cannot directly solve Problem ①I like I-FGSM. To address such issue, transfer-based attacks generate adversarial examples on an accessible surrogate model, which can be transferred to fool the target models.
3

---

3.2 Flat Local Maxima Tend to Improve Adversarial Transferability
In general, the adversarial transferability equivalent to model generalization if we analogize the optimization of perturbation with the model training process [9] Existing works [24 11] have demonstrated that flat local minima tend to generalize better than their sharp counterparts. This inspires us to assume that adversarial examples at a flat local region w.r.t. the loss function tend to have better transferability across various models. rigorous description is given as follows:
Assumption 1. Given any small radius Ç > 0 for the local region and two adversarial ex- amples u1 and 222 xadv for the same inpul image x, if max a yr 0)ll < E J y; (2, with high probability, aq tends to be more transferable than xnde across various models. 222
SAM [4 has indicated that a flat local minimum is an entire neighborhood having both low loss and low curvature. Zhao et al. [66] also demonstrated that if the loss function has a smaller gradient value, this would indicate that the loss function landscape flatter. Here we adopt the maximum gradient in the neighborhood to evaluate the flatness of the local region. Following Assumption [ we introduce regularizer, which minimizes the maximum gradient in the (-ball of the input, into Eq. [ as follows:
max
Maadd y; 0) x
X
max (anddo)
| [1r, y3 0)1
audd (
where x 2 0 is the penalty coefficient. During the optimization process, we penalize the maximum gradient to perceive the sharper regions that can not be identified by using the averaged gradients. By optimizing Problem (3), we can find that the adversarial example with small gradient of data points in its neighborhood should be at a flat local region. However, it is impractical to calculate the maximum gradient in the Ç-ball. Hence, we approximately optimize Eq. (3) by randomly sampling an example x' E (x(d"d)) at each iteration using I-FGSM [26] and MI-FGSM 10). respectively.
As shown in Fig. [ the regularizer of gradients can significantly boost the adversarial transfer- ability of either I-FGSM or MI-FGSM. In gen- eral, the average attack success rate is improved by a clear margin of 5.3% and 7.2% for I-FGSM and MI-FGSM, respectively. Such remarkable performance improvement strongly validates As- sumption RAP [42 also advocates that the
I-FGSM wio Reg.
Attack success rates
60
L.FGSM wt Reg.
MI-PGISM wif Reg.
MI FGSM Reg.
40
20
adversarial example should be located at flat local region. However, to the best of our knowl-
Inc-v4
IocRes +2
Res 101
Res 152
Ince' en
Incor3 anod
beRes beees--dens velns
edge, it is the first work that empirically vali- dates that flat local maxima can result in good adversarial transferability. Based on this finding, we aim to improve the adversarial transferability by locating the adversarial examples in a flat local region, and a detailed approach for finding flat local maxima will be provided in Sec. 3.3
Targeted Models
Figure 1: The average attack success rates (%) of I-FGSM and MI-FGSM w/wo the gradient regular- ization on seven black-box models. The adversar- ial examples are generated on Inc-v3.
3.3 Finding Flat Local Maxima
Although we have verified that adversarial examples at a flat local region tend to have better transfer- ability, the optimization process involved in Eq. (3) is computationally expensive and intractable for generating transferable adversarial examples. The detailed ablation study is summarized in Sec. 4.6 To address this challenge, we propose an approximate optimization approach that efficiently optimizes Eq. (3) to generate more transferable adversarial examples.
Since x' in Eq. (3) is close to ado with small Ç. we assume that the effects of maximizing J (2adv yy: 8) and maximizing J(z',y 0) are expected to be equivalent. Then we can approximately simplify Eq. (3) as follows:
max gadd (z)
"d y: 0) R [ 0) - X V y; 0)1),,,
s.t. x' E Be(ando),
(4)
where is a loss function. Here a penalized gradient norm introduced into the original loss function J for achieving flat local maxima.

---

Algorithm 1 Penalizing Gradient Norm (PGN) attack method
Input: A clean image with ground-truth label y, and the loss function J with parameters 6. Parameters: The magnitude of perturbation c; the maximum number of iterations, T; the decay factor μi; the balanced coefficient d; the upper bound (i.e., ( of random sampling in Ç-ball; the number of randomly sampled examples, N.
1: 90 = 0, x0th I. a = =<Ti;
2: for = 0, 1........ T do
3: Set g 00;
4: for 0, 1, 1............................... . 1 do
5:
Randomly sample an example x' E BB (agdu))
6:
Calculate the gradient at the sample x', 9 V z' (x' yi 0);
7:
Compute the predicted point by x² = x'
g
11
8:
Calculate the gradient of the predicted point, 9
J J[x" y: 0);
9:
Accumulate the updated gradient by g 9 N [[1 8) g + 6 g"]:
10: end for
11: 9c-1 f 9t + T
12: Update 2944 via 2241 = (T). [agde a a sign(g(+1)]
13:
end for
14:
return aadv= =xgdo
Output: An adversarial example adv,
In practice, it is computationally expensive to directly optimize Eq. 4 since we need to calculate its Hessian matrix. Note that existing adversarial attacks typically rely on the sign of the gradient, rather than requiring an exact gradient value. Thus, we approximate the second-order Hessian matrix by using the finite difference method to accelerate the attack process.
Theorem 1 (Finite Difference Method (T). Given a finite difference step size and one normalized (amyy)))))
gradient direction vector IV, (z.,000:0)) the Hessiun/vector product can be upproximated by the first-order gradient as follows:
V (Z, (2,
With the finite difference method, we can solve Eq. (4) approximately by using the Hessian/vector product for high efficiency. particular, we can calculate the gradient at each iteration follows.
Corollary 1. The gradient of the objective function
(4) al the I-th iteration can be approximated as:
V Zzgao Lix 0) Re (1 8) Vaz Jarh y5 0) 5 Zz; Ma's yi 0) Mets +ave>
(6)
where I'e point randomly sampled in B& (agde)) is the iteration step size, and 8 balanced coefficient.
A
The detailed proof of Corollary i provided in the Appendix. From Corollary D we approximate the gradient of Eq. (4) by interpolating two neighboring gradients. This approximation technique significantly enhances the efficiency of the attack process.
In Eq. (4) we approximate the loss at the input image aade and the maximum gradient By (xade) by randomly sampling an example x' E B& (aadv). However, this approach introduces variance due to the random sampling process. To address this issue, we randomly sample multiple examples and average the gradients of these examples to obtain a more stable gradient. Such averaging helps reduce the error introduced by the approximation and randomness, resulting in more accurate gradient estimation. This, in turn, allows us to effectively explore flat local maxima and achieve better transferability.
In short, we introduce a novel attack method, called Penalizing Gradient Norm (PGN). The PGN attack aims to guide adversarial examples towards flatter regions by constraining the norm or magnitude of the gradient. The details of the PGN attack are outlined Algorithm Since PGN derived by optimizing Eq. (3) to achieve a flat local maximum, it can be seamlessly integrated with existing gradient-basse based attack methods and input transformation-bent based attack methods, leveraging their strengths to further improve adversarial transfcrability.
5

---

(a) MI-FGSM
(b) NI-FGSM
(c) VMI-FGSM
(d) EMI-FGSM
(e) RAP
( PGN
Figure 2: Visualization of loss surfaces along two random directions for two randomly sampled adversarial examples on the surrogate model (i.e., Inc-v3). The center of each 2D graph corresponds to the adversarial example generated by different attack methods (see more examples in the Appendix).
Experiments
In this section, we conduct extensive experiments on the ImageNet-compatinn dataset. We first provide the experimental setup. Then we compare the results of the proposed methods with existing methods on both normally trained models and adversarially trained models. Finally, we conduct ablation studies to study the effectiveness of key parameters in our PGN. The experimental results were performed multiple times and averaged to ensure the experimental results were reliable.
4.1 Experimental Settings
Dataset. We conduct our experiments on the ImageNet-compatible dataset, which is widely used in previous works 4 4,33]
It contains 1,000 images with the size of 299 x 299 3, ground-truth labels, and target labels for targeted attacks.
Models. To validate the effectiveness our methods, we test attack performance five popular pre-trained models, including Inception-v3 (Inc-v3) 45). Inception-và (Inc-v4), InseptionResnet-
v2 (IncRes-v2) [46], ResNet-110 (Res- 101), and ResNet-152 (Res-152) [18] We also consider
adversarially trained models including, Inc-v3ens3. Inc-v4 and IncRes- v2ens
47).
Baselines. We take five popular gradient-based iterative adversarial attacks as our baselines, including. MI-FGSM [10), NI-FGSM [3I], VMI-FGSM [49], EMI-FGSM [52]. RAP [2). We also integrate the proposed method with various input transformations to validate the generality of our PGN, such DIM [II), TIM 61, SIM [II, Admix [51). and SSA [35].
Hyper-parameters. We set the maximum perturbation of the parameter e = 16.0/255, the number of iterations T = 10, and the step size = e/T. For MI-FGSM and NI-FGSM, we set the decay factor / = 1.0. For VMI-FGSM, we set number of sampled examples N 20 and the upper bound of neighborhood size 3 1.5
For EMI-FGSM, we set the number of examples N = 11, the sampling interval bound r = 7, and adopt the linear sampling. For the attack method, RAP, we set the step size 2.0/255, the number of iterations K 400, the inner iteration number T 10, the late-start KLS = 100, the size neighborhoods e,, 16.0/255. For our proposed PGN, we set the number of examples N 20, the balanced coefficient 0.5, and the upper bound of 3.0
4.2 Visualization of Loss Surfaces for Adversarial Example
To validate that our proposed PGN method can help the adversarial examples find a flat maxima region, we compare the loss surface maps of the adversarial examples generated by different attack methods on the surrogate model (i.e., Inc-v3). Each 2D graph corresponds to an adversarial example, with the adversarial example shown at the center. and compare the loss surfaces in Fig. 2 each row represents the visualization of one image. From the comparison, we observe that our PGN method can help adversarial examples achieve flatter maxima regions compared to the basclines. The adversarial examples generated by method are located
randomly select two images from dataset

---

Table The untargeted attack success rates (%ist,, over 10 random runs) of various gradient- based attacks in the single model setting. The adversarial examples are crafted on Inc-v3, Inc-v4, IncRes-v2, and Res- 101 by MI-FGSM (MI), NI-FGSM (NT), VMI-FGSM (VMI), EMI-FGSM (EMI), RAP, and our PGN attack methods, respectively. Here indicates the white-box model.
Model
Attack
Inc-v3
Inc-v4
IncRes-v2
Res-101
Inc-V3ensd Ine-V3ens4
IncRes-v2ena
100.0+00.0
51.0++0.77
45.8+0.60
49.0+0.24
22.6-0.52
02.0+
10.9:0.24
NI
100.0++0.0*
61.4+0.42
59.6+0.54
57.2+0.18
22.5=0.37
22.7+0.35
11.5++0.26
VMI
100.0
74.810.58
69.9
63.510.67
41.6=0.54
41.610.54
25.011.34
Inc-v3
EMI
100.0.1..0
80.710.58
77.1=0.37
72.410833
33.0=0.6
31.910.49
17.0:10.48
99.910.10#
84.5110.9
79.3
76.5100.5
56.9=0844
51.310.62
31.9:0.55
PGN
100.0++0.0*
90.6+0.67
89.3=0.75
81.2
64.6=0.75
65.6=0.94
45.3+
MI
57.2110.66 100.010.00
46.110.14
51.510.33
19.1-0:66
18.4.0.23
10.2.10.36
NI
62.8.10.43 100.010.00
52.7.0.34
56.710.19
19.2-0.25
18.3.10.7
11.710.29
VMI
77.6+0.65
99.8=0.10"
69.8=0.41
66.7+:033
41.1=0.87
41.2+0.54
27.0=t024
Inc-v4
EMI
84.2:+0.22
99.7+0.10"
75.0=0.70
74.4:0664
31.5=0044
28.0::.65
16.2
RAP
85.3+0074 09,5
79.5=0.62
77.2+0.42
45.2=0.69
46.8=0.48
29.3+0.51
PGN
91.2+10.58
99.6+
8.76
83.5+0.53
67.0-0.68
6.62
49.1
58.2::0.2
52.410.41
99.310.21
50.710:26
22.0=0.37
22.0=0.31
13,820.43
NI
60.3+:0.5
57.1
99.5=0.17
55.3+0.35
18.3=0.18
19.3:0.29
12.1
VMI
78.2400.6
77.0=:0.57
99.1=0.36
66.010.48
47.6=0.69
43.3=0.36
37.7+0.37
IncRes-v2
EMI
85.2+0.78
83.3+0.29
99.7+0.18
74.0+
38.4-0488
33.8+00.33
24.1+00.8
RAP
87.1+0.75
84.2+00.5
99.4+0028
79.4+0.64
50.3-0.47
49.8+0.89
40.2+:0.54
PGN
92.0+-0.69
92.3++063
99.8+
83.5+0.41
7..6-0.75
0.65
66.62+
MI
51,5+0026
42.2+:0.55
36.3+0.24
100.0+0.00
18.7=0.32
16.6+0.14
9.0-0.22
NI
55.6++0.35
46.9+0.41
40.8+0.28
100.0+0.0*
17.5=0.57
17.6+0.42
9.2-0.24
VMI
75.0+00.40
69.2+0.59
63.0=0.84
100.0+0.00
35.9=0.41
35.7+0.87
24.1
Res-101
EMI
74.3100.55
71.7.10.77
62.6.1.299
100.011.00
25.7=00.4
24.6.10.98
13.3110.8
RAP 80.4110.75
75.51
68.0-1084
99.910.10*
40.3
39.910.73
30.4
PGN
86.2.10.84
83.3110.6
77.8-0.69
100.010.0*
63.1 1.32
50.8
in larger and smoother flat local maxima. This confirms that our method can generate adversarial examples located in the flat maximum. With the adversarial examples located in more flat local maxima, PGN exhibits much better transferability than existing attacks shown in Sec. 4 which further supports our motivation.
4.3 Attack a Single Model
We conduct series of gradient-based attacks under single-model setting and report the attack success rates, which indicate the misclassification rates of the target models when using adversarial examples inputs. The adversarial examples generated on four different models: Inc-v3, Inc-v4, IncRes-v2, and Res- 101, respectively. The results are summarized in Table [
From the results, we observe that our PGN method not only maintains high attack success rate on white-box models but also significantly improves the attack success rate on black-box models. For example, when generating adversarial examples on Inc-v3, VMI-FGSM, EMI-FGSM, RAP achieve the attack success rates 74.8%, 50.7% and 84.5%, respectively on Inc-v4. In comparison, our PGN method achieves impressive attack success rate of 90.6%, surpassing RAP (the best baseline) by a margin of 6.1%. Moreover, when targeting adversarially trained models, our PGN attack method consistently outperforms other gradient-bart based attacks, improving the attack success rate by at least 11.8% compared to state-of-the-ee methods on average. This convincingly validates the high effectiveness of our proposed method against both normally trained and adversarially trained models. Such outstanding results highlight the effectiveness of locating adversarial examples in flat regions for improved transferability, which is consistent with our motivation.
4.4 Attack an Ensemble of Models
In addition to attacking a single model, we also evaluate the performance of our PGN method in an ensemble-model setting to further validate its effectiveness. In this subsection, adopt the ensemble attack method 100, which creates an ensemble by averaging the logit outputs of different models. Specifically, the adversaries are generated by integrating three normally trained models, including
7

---

Table 2: The untargeted attack success rates (%) of various gradient based attacks on eight models in the multi-model setting. The adversarial examples are generated on the ensemble models, i.e. Inc-v3, Inc-v4, and IncRes-v2. v2. Here * indicates the white-box model.
Attack
Inc-v3 Inc-v4 IncRes-v2 Res-10) 101
Res- I52
InC- InC-V3enst IncRes-v
Avg.
MI
99.8*
99.5*
97.8*
66.8
68.4
36.7
36.9
22.6
66.06
NI
100.0*
99.9*
99.6*
74.6
74.0
37.7
37.1
22.2
68.14
VMI
99.9*
99.5*
98.0*
82.2
81.6
66.8
64.1
50.8
80.36
EMI
100.0*
99.9*
99.7*
92.3
93.0
62.8
59.5
40.0
80.90
RAP
100.0*
99.4*
98.2*
93.5
93.4
70.1
69.8
58.9
85.41
PGN
100.0*
99.9*
99.6*
94.2
94.6
88.2
86.6
79.2
92.78
Table 3: The untargeted attack success rates (%) of our PGN method, when is integrated with DIM,
TIM, SIM, Admix, and SSA, respectively. The adversarial examples generated Inc-v3. Here *
indicates the white-box model.
Attack
Inc-v3 Inc-v4 IncRes-v2 Res-101
Inc-v Inc-v3ena3
Inc-v3ened
IncRes-v2.
Avg.
DIM
99.7*
72.2
67.3
62.8
32.8
30.7
16.4
54.56
PGN-DIM
100.0
93.6
91.9
87.3
78.3
77.5
59.8
84.06
TIM
99.9*
51.6
47.2
47.8
29.6
30.7
20.5
46.76
PGN-TIM
100.0
87.6
84.1
75.0
78.1
77.6
65.5
81.13
SIM
100.0
70.5
68.2
63.8
37.5
37.8
22.0
57.11
PGN-SIM
100.0
92.5
91.2
84.0
76.1
75.7
59.0
82.64
Admix
100.0
78.6
77.3
69.5
41.6
40.3
24.1
61.63
PGN-Admix
100.0
93.1
92.2
85.5
76.9
77.2
60.2
83.57
SSA
99.7*
88.3
86.8
77.7
56.7
55.3
35.2
71.39
PGN-SSA
99.8
89.9
89.7
82.9
69.2
67.8
47.1
78.03
Inc-v3, Inc-v4, and IncRes-v2. All the ensemble models are assigned equal weights and we test the performance of transferability both normally trained models and adversarially trained models.
The results, presented in Table demonstrate that our PGN method consistently achieves the highest attack success rates in the black-box setting. Compared previous gradient-based attack methods, PGN achieves an average success rate 92. 78%, outperforming VMI-FGSM. EMI-FGSM, and RAP by 12. 42., 11.88%, and 7.37%, respectively. Notably, method exhibits even greater improvements against adversarially trained models, surpassing the best attack method RAP by over 18.4% on average. These results validate that incorporating penalized gradient norms into the loss function effectively enhances the transfcrability of adversarial attacks, which also confirms the superiority of our proposed method in adversarial attacks.
4.5 Combined with Input Transformation Attacks
Existing input transformation-ber based attacks have shown great compatibility with each other. Similarly. due the simple and efficient gradient update process, our proposed PGN method can also be combined with these input waanformation-basee methods to improve the transferability of adversarial examples. further demonstrate the efficacy into these input transformations i.e., DIM, TIM, SIM, Admix, and SSA. We generate adversarial examples on the Inc-v3 model and test the transferability of adversarial examples on six black-box models.
proposed PGN method, we integrate our method
The experimental results are shown in Table When combined with our gradient update strategy, can significantly improve the adversarial transferability of these input transformation- based attack methods in the black-box setting. At the same time, our method also has great improvement after combining these methods. For example, DIM only achieves an average success rate of 54.56% on the seven models, while when combined with our PGN method can achieve an average rate of 84.06% which is 29.5% higher than before. Especially, after combining these input transformation-bant methods, our PGN tends achieve much better results on the ensemble adversarially trained models, compared with the results in Table I It is significant improvement and shows that method

---

Table 4: Comparison of the approximation effect between directly optimizing the Hessian matrix
(Hm) ) and using the Finite Difference Method (FDM) to approximate. "Time" represents the total running time on 1,000 images, and "Memory" represents the computing memory size.
Attack Hm
FDM Inc-v3 Inc-v4 IncRes-v2 Res-101 101 Res- 152
Time (s) Memory (MiB)
X
X
100.0*
27.8
19.1
38.1
35.2
52.31
1631
I-FGSM
/
x
100.0*
39.2
30.2
47.0
45.5
469.54
7887
/
100.0*
37.9
28.6
45.7
44.6
96.42
1631
has good scalability and can be combined with existing methods to further improve the success rate of transfer-basee based black-box attacks. In addition, our PGN method can also be combined with various gradient-based attack methods to enhance the transferability of previous works. The more experimental results are shown in the Appendix.
4.6 Ablation Study on Finite Difference Method
In this subsection, we will analyze and experimentally verify the effectiveness of the finite difference method in accelerating the approximation of the second-order Hessian matrix. We first thcoretically analyze the accelcration effect of the finite difference method and substantiate our theoretical analysis with comparative experiments.
Theoretical analysis. For the baseline attack method, I-FGSM 26),, the gradient is computed only once per iteration. Thus, its computational complexity is (n). where represents the image size. However, when we introduce the penalty gradient term, the need arises to compute the second-order Hessian matrix, leading to a theoretical computational complexity of On²)) To address this, we propose the finite difference method as an approximation the Hessian matrix, which requires the computation of the gradient twice in each iteration, effectively yielding computational complexity of (22).. This theoretically promises significant improvements in computational efficiency.
Experimental comparison. To validate the effectiveness of the finite difference method in acceler- ating computation, we compare the runtime and computational memory before and after using the -ifte-differenc method. These experiments were conducted using codes executed on an RTX 2080 Ti with a CUDA euvironment. We employed I-FGSM and evaluated the total running time on 1,000 images (excluding data loading time) and the attack success rate on black-box models. The outcomes are presented in Table Directly optimizing Eq. 4 results in better attack performance with high computational resources. With the finite difference method (FDM), we can better approximate the performance of direct optimization of the second-order Hessian matrix, which significantly reduces the running time and the computational memory. Furthermore, owing to the relatively modest image size and the comparatively small number of parameters compared to the model, the accelerated com- puting capabilities of CUDA enable the actual running time to be less than the theoretical estimates.
4.7 Ablation Study on Hyper-parameters
In this subsection, we conduct a series of ablation experiments to study the impact of different parameters, including the balanced coefficient ö and the upper bound of Ç-ball, and the study of the number of sampled examples N will be illustrated in the Appendix. To simplify our analysis, we only consider the transferability of adversarial examples crafted on the Inc-v3 model.
The balanced coefficient 8. In Sec. 3.3 we introduce a balanced coefficient to represent the penalty coefficient À (see the Appendix for details). Compared to studying the parameter of X, studying d will be much easier because it divides the scope of the parameter learning. As shown in Fig. 3a| we study the influence of the ò in the black-box setting where is fixed to 3.0 x As we increase 8, the transferability increases and achieves the peak for these black- box models when = 0.3. This indicates that when averaging these two gradients the performance is best. Therefore, set 5 0.5 in our experiments.
The upper bound of Ç-ball. In Fig. 3b we study the influence of the upper bound neighborhood size for random sampling, determined by parameter Ç, on the success rates the black-box setting. In our experiments, use uniform sampling, which reduces the bias due to uneven sample distribution. As

---

Atacc MUDDOSTTTES
60
Inced
IncRes-v2
Inc-vd
IncRes v2 Res
20
Res-101
152
20
Res- 101
TncRos-V2ens
Inc-vRenad
Inc Reves
0.1 0.2 0.3
0.5c 2e 2.5c 3e be 4.5c &e upper neighburiood
The buhincd coefficient
(a) The hyper-parameter
(b) The hyper-parameter,
Figure 3: The untargeted attack success rates (%) six black-box models with various hyper- parameters d or Ç. The adversarial examples are generated by PGN on Inc-v3.
we increase Ç. the transferability increases and achieves the peak normally trained models when <= 3.0 but it is still increasing against the adversarially trained models. When 4.5 the performance of adversarial transferability will decrease on seven black-bor models. To achieve the trade-off for the transferability normally trained models and adversarially trained models, we set < = 3.0 e in our experiments.
5
Conclusion
Inspired by the observation that flat local minima often result in better generalization, we hypothesize and empirically validate that adversarial examples at flat local region tend to have better adver- sarial transferability. Intuitively, we can optimize the perturbation with a gradient regularize in the neighborhood of the imput sample to generate an adversarial example in flat local region but is non-trivial to solve such an objective function. To address such an issue, we propose novel attack method called Penalizing Gradient Norm (PGN). Specifically, PGN approximates the Hessian/vector product by interpolating the first-order gradients of two samples. To better explore its neighborhood, PGN adopts the average gradient of several randomly sampled data points to update the adversarial perturbation. Extensive experiments on the ImageNet-compatibe dataset demonstrate that PGN can generate adversarial examples at more flat local regions and achieve much better transferability than existing transferbbase attacks. Our PGN can be seamlessly integrated with other gradient- based and input transformation-Dase attacks to further improve adversarial transferability, demonstrating its versatility and ability to improve adversarial transferability across different attack scenarios.
Limitation
Although have experimentally verified that flat local minima can improve the transferability of adversarial attacks, there is still a lack of theoretical analysis regarding the relationship between flatness and transferability. An existing perspective is that transferability may be related to the generalization ability of flatness. We will also keep studying the theoretical connection between transferability and flat local minima in future work. We hope our work sheds light the potential of flat local maxima in generating transferable adversarial examples and provides valuable insights for further exploration in the field of adversarial attacks.
Acknowledgments
We want thank the anonymous reviewers their valuable suggestions and comments. This work was supported by the National Natural Science Foundation of China (Nos. 62276182, 12, 62072334, and 61976164), and the National Science Basic Research Plan Shaanxi Province of China (No. 2022GY-061).
10

---

References
[1] Neculai Andrei. Accelerated conjugate gradient algorithm with finite difference hessian/vector product approximation for unconstrained optimization. Journal of Computational and Applied Mathematics, 230(2):500-582, 2009.
[2] Markus Bayer. Marc-André Kaufhold, and Christian Reuter. A survey on data augmentation for text classification. ACM Computing Surveys, 55(7):1-39, 2022.
[3] Wieland Brendel, Jonas Rauber, and Matthias Bethge. Reliable attacks against black-box machine learning models. arXiv preprint arXiv: 2017.
Decision-basce adversarial attacks:
[4] Junyoung Byun, Seungju Cho, Myung-Joon Kwon, Hee-Seon Kim, and Changick Kim. Improv- ing the transferability of targeted adversarial examples through object-based diverse input. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15244-15253, 2022.
[5] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, pages 39-57, 2017.
[6] Pin- Yu Chen, Huan Zhang. Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 2017.
[7] Zhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu, Longhui Wei, and Qi Tian. Visformer: The friion-friendly transformer. In Proceedings of the IEEF/CVF International Conference on Computer Vision. pages 589-598, 2021.
[8] Phillip Chlap, Hang Min, Nym Vandenberg, Jason Dowling, Lois Holloway, and Annette Ha- worth. A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 65(5):545-563, 2021.
[9] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of International Conference on Machine Learning, pages 1310-1320. PMLR, 2019.
[10] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9185 9193, 2018
[11] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by wranslatonniivariian attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4312-4321, 2019.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of International Conference on Learning Representations, 2021.
[13] Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. Robust physical attacks on machine learning models. arXiv preprint arXiv: :tfi::777.08945 2(3):4, 2017.
[14] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware mini- mization for efficiently improving generalization. In Proceedings of International Conference on Learning Representations, 2021.
[15] Zhijin Ge, Fanhua Shang, Hongying Liu, Yuanyuan Liu, Liang Wan, Wei Feng, and Xiaosen Wang. Improving the Transferability of Adversarial Examples with Arbitrary Style Transfer. In Proceedings of the ACM International Conference Multimedia, 2023.

---

[16] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In Proceedings of International Conference on Learning Representations. 2015.
[17] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. In Proceedings of International Conference on Learning Representations, 2018.
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[19] Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1 11936-1199 11945, 2021.
[20] Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural Computation, 9(1):1-42, 1997.
[21] Zhichao Huang and Tong Zhang. Black-box adversarial attack with transferable model-bases embedding. arXiv preprint arXiv: /9/ I. 140, 2019.
[22] Xiaojun Jia, Xingxing Wei, Xiaochun Cao, and Hassan Foroosh. Comdefend: An efficient image compression model to defend adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2019.
[23] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. arXiv preprini 1rXiv:111..02178, 2019.
[24] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large- batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv: wtw.l:609904448 2016.
[25] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical Report, 2009.
[26] Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial Intelligence Safety and Security, pages 91- 2018
[27] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Proceedings of Advances in Neural Information Processing Systems, 31, 2018.
[28] Siyuan Liang, Xingxing Wei, Siyuan Yao, and Xiaochun Cao. Efficient adversarial attacks for visual object tracking. In European Conference on Computer Vision, pages 34-50, 2020.
[29] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2018.
[30] Shang-Sheng Lin, Chia-Y Yi Hsu, Pin-Yu Chen, and Chia-Mu Yu. Real-world adversarial examples via makeup. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2854-2858, 2022.
[31] Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In Proceedings of International Conference on Learning Representations, 2019.
[32] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In Proceedings of International Conference on Learning Representations, 2017.
[33] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012-10022, 2021.
12

---

[34] Zihao Liu, Qi Liu, Tao Liu, Nuo Xu, Xue Lin, Yanzhi Wang, and Wujie Wen. Feature distillation: Dnn-oriented jpeg compression against adversarial examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 860-868, 2019.
[35] Yuyang Long, Qilong Zhang, Boheng Zeng, Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency domain model augmentation for adversarial attack. In Proceedings of the European Conference on Computer Vision, pages 549-566. Springer, 2022.
[36] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 777770666082 2017.
[37] Agnieszka Mikotajezyk and Michal Grochowski. Data augmentation for improving deep learning in image classification problem. In 2018 International Interdisciplinary PhD Workshop (IIPhDW), pages 117-122. IEEE, 2018.
[38] Seyed-Mohsen
Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2574-2582, 2016.
[39] Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 262-271, 221, 2020.
[40] Yurii Nesterov. A method for unconstrained convex minimization problem with the rate of convergence o (1/k" 2). In Doklady An Ussr, volume 269, pages 543-547, 1983.
[41] Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In Proceedings of Advances in Neural Information Processing Systems, 30, 2017.
[42] Zeyu Qin, Yanbo Fan, Yi Liu, Li Shen, Yong Zhang, Jue Wang, and Baoyuan Wu. Boosting the transferability of adversarial attacks with reverse adversarial perturbation. arXiv preprint arXiv:2210.055: 059666 2022.
[43] Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does batch normalization help optimization? In Proceedings of Advances in Neural Information Processing Systems, 31, 2018
[44] Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 ACM on Asia Conference on Computer and Communications Security, pages 1528-1540, 2016
[45] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2812 2818-2211 18-2826, 2016.
[46] Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI Conference Artificial Intelligence, 2017.
[47] Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick Medaniel. Ensemble adversarial training: Attacks and defenses. In Proceedings of International Conference on Learning Representations, 2018.
[48] Kunyu Wang, Xuanran He, Wenxuan Wang, and Xiaosen Wang. Boosting Adversarial Transfer- ability by Block Shuffle and Rotation. arXiv preprint arXiv:2308. 10299, 2023.
[49] Xiaosen Wang and Kun He. Enhancing the transfcrability of adversarial attacks through variance tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021.
[50] Xiaosen Wang, Kun He, Chuanbiao Song, Liwei Wang, and John E. Hopcroft. AT-GAN: A Generative Attack Model for Adversarial Transferring on Generative Adversarial Nets. arXiv preprint arXiv:1904.07799 2019.
13

---

[51] Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16158-16167, 2021.
[52] Xiaosen Wang, Jiadong Lin, Han Hu, Jingdong Wang, and Kun He. Boosting adversarial transferability through enhanced momentum. In Proceedings of the British Machine Vision Conference, 2021.
[53] Xiaosen Wang. Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, and Wei Liu. Triangle attack: A query-efficient decision-based adversarial attack. In Proceedings of The European Conference on Computer Vision (ECCV), pages 156-174, 17.. Springer, 2022.
[54] Xiaosen Wang, Kangheng Tong, and Kun He. Rethinking the Backward Propagation for Adversarial Transferability. In Proceedings of the Advances in Neural Information Processing Systems, 2023.
[55] Xiaosen Wang, Zeliang Zhang, and Jianping Zhang. Structure Invariant Transformation for better Adversarial Transferability. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4607-4619, 2023.
[56] Zhiyuan Wang. Zeliang Zhang, Siyuan Liang, and Xiaosen Wang. Diversifying the High-level Features for better Adversarial Transferability. In Proceedings of the British Machine Vision Conference, 2023.
[57] Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint arXiv:1811.12641, 2018.
[58] Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In Proceedings of International Conference on Learning Representations, 2020.
[59] Weibin Wu, Yuxin Su, Michael Lyu, and Irwin King. Improving the transferability of adver- sarial samples with adversarial transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021.
[60] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversar- ial effects through randomization. In Proceedings of International Conference on Learning Representations, 2018
[61] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2730-27222 2019.
[62] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Proceedings of Network and Distributed System Security Symposium, 2018.
[63] Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Yan Liu. How does data augmentation affect privacy in machine learning? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10746-1075 2021.
[64] Jianping Zhang, Jen tse Huang, Wenxuan Wang, Yichen Li, Weibin Wu, Xiaosen Wang, Yuxin Su, and Michael R. Lyu. Improving the Transferability of Adversarial Samples by Path- Augmented Method. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8173-8182, 2023.
[65] Yang Zhang, Hassan Foroosh, Philip David, and Boqing Gong. Camou: Learning physical vehicle camouflages to adversarially attack detectors in the wild. In Proceedings of International Conference on Learning Representations, 2018
[66] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In Proceedings of International Conference on Machine Learning, pages 26982-26992. PMLR, 2022.
14

---

A Proof of Corollary
In this work, we introduce penalized gradient norm the original loss function, which helps the adversarial examples to achieve flat maximum. Then we randomly sample an example x' in the neighborhood of the adversarial example zndv and simplify the objective function as follows:
max Llaado ys 0) [J(z 0: x. y: 0)112) gadvEB, (x))
s.t. x' E (a(add)).
(7)
Gradient-based attacks require calculating the gradient the objective function during practical optimization, thus the gradient of the current loss function (7) can be expressed as follows:
V addv Lizado
0) V J(r' 0; À
V V Ja' 0)112)
Jar y:
(8)
V x' J(' 0; À V J( 8: 0)
J 0)11
In practice, it computationally expensive to directly optimize Eq. 8 since we need to calculate the Hessian matrix. In this work, we approximate the second-order Hessian matrix using the finite difference method to accelerate the attack process. Specifically, local Taylor expansion would be employed approximate the operation results between the Hessian matrix and the gradient vector.
A.1 Proof of Theorem
Proof. According to the Taylor expansion, we have
V 2 (x+ + Ar. 0) VzJ(z, + (Z, 0)Ar + O(1An1))),
(9)
where Ar = a U, a is small step size, and v a normalized gradient direction vector. Here, we denote = V, (z.
Therefore, the Hessian/vector product can be approximated by the first-order gradient as follows: (xx
(z VIJ(I
+ a U. y: V J y:
(10)
A.2 Proof of Corollary
Proof. From Eqs. 8 and
10 the gradient of the loss function L(.) can be expressed as:
V J(r yz 0)
V nadds adx .0:0) = s J(x v:0) - X J.J( :::
J yı 0)ll
J(r' 0) J + +a- v.y. a. 0) - V J y:
(11)
= (1 v J 0) - Je +a va.v,
We introduce balanced coefficient 5 and denote as 8 Hence, the gradient of the objective function 7 at the f-th iteration can be approximated as:
uad L(aadd y; 0) ~ (1 5) V [o,, v: + + Vg J( + +.,,,;0)
(12)
B Visualization of Loss Surfaces
Implementation details. Given that the adversarial example zade typically has a large number of dimensions, visualizing the loss function against all dimensions becomes infeasible. To this end, we randomly select two directions, denoted as r'1 and 72, from a Gaussian distribution with the same dimension adv Next, we calculate the loss change by varying the magnitudes of
15

---

1y and k2, representing the scaling factors applied to and r2, respectively, which enables us to visualize the loss function using two-dimensional plot. This approach provides slice of the loss function, allowing us to analyze its behavior and understand the impact of perturbations along different directions.
Visualization of loss surfaces for more adversarial examples. We visualize five randomly selected images in the ImageNet-compattbn dataset. The adversarial examples are generated by various gradient-based attack methods on Inc-v3. These selected images include the three images that can be successfully transferred using our PGN method but cannot be transferred using other baseline methods. As shown in Fig. 4 we can observe that our method generates visually similar adversaries as other attacks. Hence, our method demonstrates the capability guide adversarial examples towards larger and smoother flat regions. This observation substantiates the effectiveness of our PGN method in generating adversarial examples that reside within flat regions, thereby shedding light on the potential role of flat local maxima in generating transferable adversarial examples.
C
Combined with Other Categories of Attacks
C.1
Gradient-based Attacks
Our PGN attack method can also be combined with various gradient-based attacks. The core of our method involves updating gradients by interpolating the first-order gradients from two samples to approximately minimize the gradient norm. contrast, conventional gradient-basee methods typically utilize single example for gradient updates. To evaluate the efficacy of our strategy, we incorporate this interpolation approach into previous gradient-based methods, such as I-FGSM (BIM), MI-FGSM, NI-FGSM, VMI-FGSM, EMI-FGSM, and RAP. To simplify the experimental setup, we omitted random sampling and directly substituted the gradient update process of these methods with proposed strategy.
The experimental results are presented in Table Notably, when our gradient update strategy is integrated, there is remarkable improvement in the adversarial transferability of the gradient- based attack methods in the black-box setting. For example, RAP alone achieves an average success rate of 68. 08.80% across the seven models. However, when combined with our PGN method, the average success rate rises 73.91%, exhibiting a significant improvement of 5.11%. This outcome underscores the robust scalability of our approach, as it seamlessly integrates with existing methodologies further amplify the success rate of transfer-based attacks.
Table 5: Untargeted attack success rates (%) of our PGN method, when it is integrated with I-FGSM (BIM), MI-FGSM, NI-FGSM, VMI-FGSM, EMI-FGSM, and RAP, respectively. The adversarial examples are generated on Inc-v3. indicates the white-box model.
Attack
Inc-v3
Inc-v4 IncRes-v2
Res- 101-101
Inc-v3enea
Inc-v3enet
IncRes-v2..
Avg.
BIM
100.0*
28.1
20.6
26.7
11.9
11.9
5.0
29.17
PGN-BIM
100.0
33.1
25.3
30.4
13.7
13.8
6.1
31.77
MI
100.0*
50.8
46.3
48.9
23.3
22.2
11.7
43.31
PGN-MI
1
56.4
53.3
55.0
24.7
24.7
11.8
46.54
NI
100.0
62.3
59.8
57.8
22.4
22.3
11.8
48.06
PGN-NI
68.6
65.8
61.9
25.7
25.8
12.8
51.51
VMI
1
75.2
70.2
66.0
41.7
40.9
24.8
59.83
PGN-VMI
79.8
75.9
69.8
45.1
46.1
27.8
63.50
EMI
1
81.6
77.0
72,0
32.8
32,2
17.5
59.01
PGN-EMI
100.0
83.6
81.7
75.9
35.2
34.4
17.5
61.19
RAP
99.9
84.6
79.3
76.6
57.2
51.3
32.7
68.80
PGN-RAP
100.0
90.3
86.4
85.7
59.5
56.3
39.3
73.91
C.2 Skip Gradient Method
Moreover, our method can also combined with the Skip Gradient Method (SGM) [58]. By using MI-FGSM as the backbone method, first compare PGN method with SGM the
16

---

MI-FGSM
NI-FGSM
VMI-FGSM
EMI-FGSM
RAP
PON
Raw Image
Figure 4: Visualization of adversarial examples with their corresponding loss surfaces along two random directions. Here, we randomly sampled five images and generated the adversarial examples on Inc-v3. These selected images include the three images (i.e.. the last three ones) that can be successfully transferred using our PGN method but canmot transferred using other baseline methods. The loss surfaces are also calculated Inc-v3.
17

---

ResNet-18 18 model. Since SGM modifies the backpropagation and is compatible with our PGN, we also integrate PGN into SGM evaluate its generality other attacks. As shown Table our PGN consistently exhibits better attack performance than SGM, which further shows its superiority in boosting adversarial transferability. Besides, PGN-SGM outperforms both PGN and SGM, showing its remarkable compatibility with various attacks.
Table 6: Untargeted attack success rates (%) of our PGN method with SGM method the single model setting. The adversarial examples crafted Res-18.
Source:Res- 18 Res-101
Res-152
Inc-v3
Inc-v4
IncRes-v2
Inc-v3ene3
InC-v3enal
Avg.
MI
82.8
73.3
54.5
48.7
33.9
17.4
18.1
46.96
SGM
89.1
82.5
66.0
58.8
45.4
20.8
20.8
54.77
PGN
95.9
92.9
77.9
75.9
58.7
33.7
36.3
67.33
PGN-SGM
96.7
93.8
80.1
76.3
62.8
33.8
37.0
68.64
D Ablation Studies for More Hyper-parameters
D.1
The Number of Sampled Examples, N
Aftck 5000054 ratck (R)
60
40
Ino-v4
IncRes-v2
Re-101
Rey-152
20
IncRes-VZena
25
The namber sampled examples.
Figure 5: Untargeted attack success rates (%) on six black-box models with the different number of sampled samples N. The adversarial examples are generated by PGN on Inc-v3.
In this study, we employ random sampling of multiple examples and calculate the average gradients of these examples to mitigate the variance resulting from random sampling during the iterative process. To investigate the influence of the number of sampled examples, denoted as N, we conduct ablation experiments analyze this parameter. As illustrated in Figure 5 when N method demonstrates the lowest level of transferability. However, as we increase the value of N, the transferability exhibits rapid improvement until N 12, after which it gradually converges for normally trained models. Notably, when > 12, a slight performance improvement can still be achieved by increasing the number of sampled examples in our PGN method. To strike balance between transferability and computational overhead, we set N = 20 in our work. This observation further substantiates that sampling random examples from the vicinity of the adversarial example effectively facilitates neighborhood exploration. Consequently, it stabilizes the gradient update process and encourages the discovery of flatter regions by the adversarial example.
our
D.2 Finite Difference Step-size
In this work, we simply set the finite difference step-size to 1.6/255 6.27e-3 to avoid redundant hyperparameters. In order to study the effect of the smaller step-size the finite
18

---

difference method, we performed more experiments. We investigate the transferability of adversarial examples when the alpha is set to le-4 and lc-5, and the experimental results are reported in Table The results show that smaller step-sizes cannot improve adversarial transferability. This is because the smaller the step-size, the closer the two neighboring gradients are, leading our method degrade to MI-FGSM.
With the current choice of a, our method also approximates the actual Hessian/vector product. We compared the cosine similarity of perturbations generated by the Hessian/vector product and the finite difference method. The average cosine similarity of 1,000 images about 0.9035. Note that existing adversarial attacks typically rely on the sign of the gradient, rather than requiring an exact gradient value. Thus, we can approximate the second-order Hessian matrix by using the finite difference method to accelerate the attack process.
Table 7: Untargeted attack success rate (%) when using smaller step sizes in the finite-difference method. indicates the white-box model.
Size
Inc-v3
Inc-v4
IncRes-v2
Res-101
6.27e
100.0*
91.6
90.6
89.1
le
100.0*
86.4
84.6
82.6
le
100.0*
85.3
85.0
82.4
E Differences between Our method and Related Works
There are two previous works similar to our proposed attack method (i.e., PGN), where one Reverse Adversarial Perturbation (RAP) 42), which is an adversarial attack method. And the other is Gradient Norm Penalty (GNP) (66), which was proposed to improve generalization. To demonstrate the novelty and significance of the proposed method, we give detailed discussion about the differences and similarities compared to these two works.
E.1 Differences with RAP
Reverse adversarial perturbation (RAP) is closely related work that encourages adversarial examples to be located at region with low loss values. However, there are still many differences between our method and RAP.
Empirical verification. Both RAP and our PGN aim to achieve flat local optima for more transferable adversarial examples. Moreover,this is the first work to provide empirical verification that adversarial examples in flat local optima are more transferable.
Methodology. RAP injects worst-case perturbations into the optimization process to maximize the loss and achieve flat local optima. In contrast, PGN introduces a gradient norm into the loss to achieve a flat local optimum. Hence, the proposed method differs significantly from RAP.
Efficiency. Our method is more computationally efficient than RAP. In RAP, the outer loop is 400, and the inner loop is 8. contrast, our PGN method employs finite differences to approximate the second-order Hessian matrix and samples multiple points to obtain more stable gradient, which requires only 10 outer loops and 20 inner loops for the computation. Here we also compared the time consumed for each batch of images between RTX 2080 with a CUDA environment. We chose Inc-v3 as the source model and set the batch size 10. Experimental results show that RAP will consume 207.1 1 19s in each batch size, while method only consumes 29.85s, which validates that our PGN is more computationally efficient.
method and RAP. The experiment
conducted on
E.2 Compared with Gradient Norm Penalty
Zhao et al. [66] proposed a gradient norm penalty method in deep learning to improve generalization, which is similar to our method. However, there are also some differences between these two works, which are summarized follows:
Different goals. Zhao et al. primarily focus on computing flat local minima improve generalization during model training. In contrast, our main objective is investigate the potential impact of flat
19

---

local minima on the transferability of adversarial attacks. Previous research in the field of adversarial attacks has shown limited attention the relationship between flat local optima and adversarial transferability.. Consequently, our work significantly contributes new insight into the domain of adversarial attacks.
Objective function. A distinguishing feature of our proposed method lies in its emphasis on the gradient information surrounding the adversarial example. Specifically, we approximate the second- order Hessian matrix using the finite difference method, which plays a crucial role in our approach. The work presented by Zhao et al. also supports and provides theoretical foundations for the feasibility of our method within the realm of udversarial attacks.
In summary. main motivation in this work is explore whether flat local optimum improve adversarial transferabilitty. To the best of our knowledge, it is also the first work that penalized the gradient norm and the finite difference method support motivation in the field of adversarial attacks.
Additional Experimental Results
F.1
Evaluated on Transformer-based Models
To further validate the effectiveness of our PGN method, we evaluated the performance on the Transformer-based models, i.e. ViT [2], PiT [19) Visformer (7], and Swin (33]. The adversarial examples are generated on Inc-v3 and the attack success rates are summarized in Table [ It can be
seen that PGN can consistently outperform the baselines on these Transformers, showing its high effectiveness and generality to various architectures.
Table 8: Untargeted attack success rate (%) of adversarial examples in Transformer-based models.
The adversarial examples are generated on Inc-v3.
Attack
ViT
PiT
Visformer
Swin
Avg.
MI
18.9
18.1
23.7
22.2
20.73
NI
20.0
19.9
25.5
25.5
22.73
VMI
28.4
34.0
41.1
40.8
36.08
EMI
26.0
27.3
36.9
36.7
31.73
RAP
35.2
41.8
50.6
50.2
44.45
PGN
44.7
53.2
65.5
64.5
56.98
F.2 Attack Defense Models
In this subsection, besides normally trained models and adversarially trained models, we further validate the effectiveness of our methods other defenses, including Rit-Red (62), ComDefend 22],
JPEG [17. HGD [29], R&P [60], NIPs-r3 (39 FD 34 NPR 33), and RS [9]. The adversarial examples generated model 1/3.
ensemble of Inc-v3, Inc-v4, and IncRes-v2, v2, and the weight for each
The experimental results are reported in Table In the context of ensemble models, it is evident that our algorithm can considerably enhance existing attack methods. For instance, VMI, EMI, and RAP achieve average success rates of 54.91%. 91, 61.59%, and 69.94%, respectively. against the six defense models. In contrast, our proposed PGN method achieves an average success rate of 77. 77.08%, surpassing them by 22.17%. 15.49%. and 7. 7.14%, respectively. This notable improvement demonstrates the remarkable effectiveness of our proposed method against adversarially trained models well other defense models. Consequently, it poses more substantial threat advanced defense models. These findings further validate that the discovery of adversarial examples within flat regions can significantly enhance the transferability of adversarial attacks.
E33 Attack Success Rates on CIFAR-10
To further illustrate the effectiveness of our PGN method on different datasets, we conduct experiments on CIFAR-10 (25). we set the hyperparameters as follows: maximum perturbation 8/255, number
20

---

Table 9: Untargeted attack success rates (%) six defense models. The adversarial examples
crafted on the ensemble models, i.e. Inc-v3, Inc-v4 and IncRes-v2.
Attack
ComDefend
JPEG
NIPs-r3
FD
R&P
HGD
Bit-Red
NPR
RS
AVG.
MI
54.9
49.5
29.9
51.7
22.2
24.8
23.8
36.5
30.3
39.96
NI
56.9
50.8
29.3
53.6
23.1
22.3
23.9
38.1
80.7
36.52
VMI
72.4
71.5
58.1
67.8
50.6
54.3
39.0
44.9
35.6
54.91
EMI
78.2
74.1
69.1
74.8
60.1
64.8
47.6
46.8
38.8
61.59
RAP
89.5
88.1
81.0
79.6
73.1
72.3
54.6
49.6
41.7
69.94
PGN
93.7
91.3
88.3
85.7
83.6
82.5
72.1
51.3
45.2
77.08
of iterations T' 10, and step size a 1/255. We compare our PGN method with various gradient- based attacks, including MI-FGSM. NI-FGSM, VMI-FGSM, EMI-FGSM, and RAP. The adversarial examples are generated on the VGG-16 ResNet-50,, and DenseNet-122 121 models, respectively. The results in Table [O)cTTary show that our PGN method can enhance the attack transferabilityy on the CIFAR-I0 dataset. This verifies our motivation
adversarial examples located in flat local regions tend to exhibit better transferability across diverse models. Moreover, our attack method shows superior performance when applied to other datasets, reinforeing its versatility and effectiveness.
Table 10: Untargeted attack success rates (%) on the CIFAR-I0 dataset for the attack methods in the single model setting. The adversarial examples crafted VGG-16, ResNet-50 (Res-50), and DenseNet- 121, respectively.
Attack MobileNet VGG-19 GoogLeNet
Inc-v3 DenseNet-121 DenseNet- 169
Res-34
Res-50
MI 52.18
57.56
47.29
52.74
40,96
42.40
41.72
41.93
NI
56.13
61.36
49.19
54.87
37.61
39.74
38.74
38.90
VMI
66.14
68.05
60.89
65.63
55.62
57.21
55.35
56.46
EMI
70.69
74.36
66.78
70.56
59.98
63.04
60.47
61.83
RAP
77.98
78.43
73.41
77.86
68.30
69.74
65.48
66.27
PGN
85.97
86.73
82.82
85.59
72.48
74.66
71.62
72.95
(a) Untargeted attack success rates (%)
the adversarial examples crafted VGG-16.
Attack
MobileNet VGG16
VGG19
GoogleNet Inc-v3
DenseNet-121 DenseNet- 169
Res-34
MI
70.42
67.37
65.8
63.06
69.02
72.39
73.34
67.78
NI
71.97
65.57
63.76
63.28
69.13
71.03
72.78
65.02
VMI
77.60
74.27
73.26
71.11
75.99
76.80
77.68
73.54
EMI
80.11
78.66
77.43
76.34
78.12
79.68
80.12
77.24
RAP
86.92
84.24
83.46
80.68
81.75
83.54
84.98
81.36
PGN
90.88
88.68
88.07
85.79
89.53
89.93
90.91
87.19
(b) Untargeted attack success (%) for the adversarial examples crafted Res-50.
Attack
MobileNet
VGG16
VGG19
GoogLeNet
Inc-v3
DenseNet-169
Res-34
Res-50
MI
63.47
60.64
60.08
57.39
63.35
71.09
61.99
67.37
NI
66.86
61.80
60.85
60.30
66.54
75.92
61.57
69.25
VMI
71.28
68.49
68.01
65.40
70.62
75.51
68.42
72.50
EMI
74.36
75.66
73.54
70.41
75.23
78.94
73.64
77.45
RAP
79.98
80.22
78.68
76.39
80.55
84.25
78.65
80.03
PGN
86.73
85.12
84.66
81.82
86.26
88.35
83.30
86.21
(c) Untargeted attack
rates (%) for the adversarial examples crafted DenseNet-121.
21


Reviews:
[{"review_id": "3mLW0MlczS", "reviewer": "Reviewer_PZXp", "summary": "In this work, the authors first assume and empirically validate that adversarial examples at flat local minima tend to have better adversarial transferability. Based on this finding, they introduce a regularizer on the gradients in the neighborhood of the input sample to achieve flat local minima. To make the attack more computationally efficient, they propose Penalizing Gradient Norm (PGN) attack, which approximates the second-order Hessian matrix by interpolating two Jacobian matrixes.", "questions": "In Line 9 of Algorithm 1, g’ and g* are only related to i-th sampled example x’. Is it a typo? I think it should accumulate all the gradients of N sampled examples.\nWhat is the difference between the two symbols L and J? I think they are both loss functions. Definitions of these two loss functions should be clarified.", "limitations": "No.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It is the first work that empirically validates that adversarial examples at flat local minima have better adversarial transferability.\n\nThe approximation on the second-order Hessian matrix is reasonable with theoretical support.\n\nThe proposed method is simple yet effective. Extensive experiments have shown that PGN can significantly boost adversarial transferability compared with existing methods.\n\nThe visualization in Figure 2 validates that PGN can achieve better flat local minima than existing attacks, which further supports their motivation.", "weaknesses": "Results on vision transformers, such as ViT, Swin, etc. would better be included.\n\nEvaluations on more defense methods, such as randomized smoothing and denoising should be conducted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer PZXp,\nThanks for your valuable and insightful comments. We address your concerns as follows:\n\n**Q.1.** Results on vision transformers, such as ViT, Swin, etc. would better be included.\n\n> **A.1.** As you suggested, we further adopt four mainstream vision transformers to evaluate the effectiveness of our method, i.e., ViT [R1], PiT [R2], Visformer [R3], and Swin [R4]. The adversarial examples are generated on Inc-v3 and the attack success rates are summarized in the following table. It can be seen that our PGN can consistently outperform the baselines on these transformers, showing its high effectiveness and generality to various architectures. We will report the complete results in our final paper.\n> \n> |Method\t|ViT\t|PiT\t|Visformer\t|Swin|\n> | :----: | :----: | :----: | :----: | :----: | \n> |MI\t|18.9\t|18.1\t|23.7\t|22.2 |\n> |NI\t|20.0\t|19.9\t|25.5\t|25.5 |\n> |VMI\t|28.4\t|34.0\t|41.1\t|40.8 |\n> |EMI\t|26.0\t|27.3\t|36.9\t|36.7 |\n> |RAP\t|35.2\t|41.8\t|50.6\t|50.2 |\n> |PGN (ours)\t|**44.7**\t|**53.2**\t|**65.5**\t|**64.5** |\n> \n> [R1] Alexey et al. \"An image is worth 16x16 words: Transformers for image recognition at scale.\" ICLR. 2021.  \n> \n> [R2] Heo et al. \"Rethinking spatial dimensions of vision transformers.\" ICCV. 2021.  \n> \n> [R3] Chen et al. \"Visformer: The vision-friendly transformer.\" ICCV. 2021.  \n> \n> [R4] Liu et al. \"Swin transformer: Hierarchical vision transformer using shifted windows.\" ICCV. 2021.  \n\n\n**Q.2.** Evaluations on more defense methods, such as randomized smoothing and denoising should be conducted.\n\n> **A.2.** Following your suggestion, we further evaluate our PGN and other gradient-based attacks on three advanced defense models, i.e., Feature Distillation (FD) [R1], Randomized Smoothing (RS) [R2], and Neural Representation Purifier (NRP) [R3]. The adversarial examples are generated on the ensemble models, i.e. Inc-v3, Inc-v4 and IncRes-v2, and the attack success rates are summarized in the following table. As we can see, our PGN can consistently exhibits better attack performance than the baselines on these advanced defenses. These results futher validate the superiority of our proposed PGN. We will add the complete results in our final paper.\n>\n> | Method | FD | RS | NPR|\n> | :----: | :----: | :----: | :----: | \n> | MI | 51.7 | 30.3 | 36.5 |\n> | NI | 53.6 | 30.7 | 38.1 |\n> | VMI| 67.8 | 35.6 | 44.9 |\n> | EMI| 74.8 | 38.8 | 46.8 |\n> | RAP| 79.6 | 41.7 | 49.6 |\n> | PGN (ours)| **85.7** | **45.2** | **51.3** |\n>\n> [R1] Liu et al. \"Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples.\" CVPR. 2019.  \n> \n> [R2] Cohen et al. \"Certified adversarial robustness via randomized smoothing.\" ICML. 2019.  \n> \n> [R3] Naseer et al. \"A self-supervised approach for adversarial robustness.\" CVPR. 2020.  \n\n**Q.3.** In Line 9 of Algorithm 1, g’ and g* are only related to i-th sampled example x’. Is it a typo? I think it should accumulate all the gradients of N sampled examples. What is the difference between the two symbols L and J? I think they are both loss functions. Definitions of these two loss functions should be clarified.\n> **A.3.** Thanks for pointing out this typo. \n> 1) In fact, the symbol $\\bar{g}$ is intended to accumulate the gradients of multiple sampled examples, and the correct representation should be $\\bar{g} = \\bar{g}+(1-\\delta)\\cdot g'+ \\delta \\cdot g^{\\ast}$. Specifically, in each iteration, $\\bar{g}$ is initialized to $0$. Then $N$ different examples $x'$ will be randomly sampled in the neighborhood of $x_t^{adv}$. Subsequently, the gradients $g'$ and $g^{\\ast}$ related to example $x'$ will be computed. Finally, these gradients will be accumulated into $\\bar{g}$ as the final gradient. We will correct this typo in Algorithm 1 in the revised version.\n>\n> 2) The symbols $L$ and $J$ represent two different loss functions. Where $J$ is the original loss function of the classifier $f$ (e.g., the cross-entropy loss), and $L$ is our proposed loss function that introduces a penalized gradient norm to the original loss function to achieve flat local maxima. To avoid confusion, we will add the above clarifications to the final version of the paper. \n\nWe appreciate your efforts in improving the clarity and accuracy of our paper, and we will make the necessary revisions to address these issues."}, {"review_id": "LpaygG3Epn", "reviewer": "Reviewer_xHRb", "summary": "This paper aims to boost adversarial transferability by using the Penelizing Gradient Norm, which can restrict adversarial examples located in flat regions. The writing is good, and it is easy to read. The experiments demonstrate that the proposed method achieves good results. ", "questions": "See above weakness. ", "limitations": "The proposed method is similar to the previous work [a].\n\n[a] Penalizing gradient norm for efficiently improving generalization in deep learning. ICML 2022.", "rating": 5, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The motivation is clear, and the writing is well.\n\n-  The analysis in Sec.3.2 is interesting, which can verify the assumption. \n\n- The proposed method is simple but effective. \n\n", "weaknesses": "1. In Theorem 1, the authors briefly introduce the finite difference method, which is fundamental for efficiently approximating a second-order Hessian matrix. Although this is an interesting solution, it is better to verify this approximation in experiments if Eq. 6 is a good approximate solution to the objective function in Eq. 4. On one hand, I think the results of solving Eq. 4 directly should be reported, which can show that the approximated solution will not affect the performance. On the other hand, the running time and complexity analysis are also considered, which can show that this key design of PGN actually works well. \n\n2. In Table 1, we first observe that PGN is a good solution for boosting transferability. But, we can also observe that the source model IncRes-v2 can achieve the best average score. Does this mean that the loss surface of this model is in a more smooth region? \n\n3. I understand this paper focuses on boosting adversarial transferability. However, for black-box attacks, query-based adversarial attacks are also widely studied. Therefore, I am interested in if the Penelizing Gradient Norm is a generally method for black-box attacks, not limited to transfer attacks.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer xHRb,\nThanks for your valuable and insightful comments. We address your concerns as follows:\n\n**Q.1.** In Theorem 1, the authors briefly ... key design of PGN actually works well. \n>**A.1.** As you suggested, we first theoretically analyze the acceleration effect of the finite difference (FD) method. For the baseline attack method I-FGSM, the gradient is computed only once per iteration. Thus, its computational complexity is $O(n)$, where $n$ represents the image size. However, upon introducing the penalty gradient term, the need arises to compute the second-order Hessian matrix, leading to a theoretically computational complexity of $O(n^2)$.  To address this, we use the finite difference method as an approximation to the Hessian matrix, which requires the computation of the gradient twice in each iteration, effectively yielding a computational complexity of $O(2n)$. This theoretically promises significant improvements in computational efficiency.\n>\n> Additionally, we substantiate our theoretical analysis with comparative experiments. These experiments were conducted on an RTX 2080 Ti with a CUDA environment. We employed I-FGSM and evaluated the total running time on 1000 images (excluding data loading time) and the attack success rate on black-box models. (**The results please refer to the response to Q.3. of Reviewer 2kUx**).  Directly optimizing Eq. 4 results in better attack performance with high computational resources. With the finite difference method, we can better approximate the performance of direct optimization of the second-order Hessian matrices, which significantly reduces the running time and the computational memory. Furthermore, owing to the relatively modest image size ($299\\times 299\\times 3$) and the comparatively small number of parameters compared to the model, the accelerated computing capabilities of CUDA enable the actual running time to surpass the theoretical estimates. We will add this discussion in the revision.\n\n**Q.2.** In Table 1, we first ... is in a more smooth region?\n> **A.2.** We tested the smoothness over the loss function of the adversarial examples generated by Inc-v3 and IncRes-v2 models, respectively. To characterize the flatness of the loss surface, we define the slope $k=\\frac{J(x_0)-J(x_i)}{\\||x_0 - x_i \\||_2}$, where $x_0$ represents the center point of the 2D map in Fig. 2, and $x_i$ represents the points sampled near the center point $x_0$. Notably smaller values of $k$ mean flatter. To derive comprehensive insights, we randomly selected a subset of images ($S_i$, which denotes the $i$-th image) and uniformly sampled 10 points near the center to calculate their average slope $k$. \n>\n> |Averaged slope $k$|$S_0$|$S_1$|$S_2$|$S_3$|$S_4$|\n> |:-:|:-:|:-:|:-:|:-:|:-:|\n> |Inc-v3|0.013|0.254|0.016|0.024|0.009|\n> |IncRes-v2|0.004|0.122|0.018|0.020|0.007|\n>\n> From the table, it can be concluded that most of the adversarial examples generated by the IncRes-v2 model have smaller slopes. We also counted the slopes of 1000 images on the Inc-v3 and IncRes-v2 models, respectively. The results show that 76.4% of the images exhibit smaller slopes on the IncRes-v2 model. The observations from these experiments confirm that the IncRes-v2 model facilitates the generation of adversarial examples that exist in smoother regions of the loss landscape. In our future work, we will investigate the possible reason and design effective method to train the model with more smooth region as the surrogate model for better transferability.  Thanks for your insightful comments.\n\n**Q.3.** I understand this paper ... not limited to transfer attacks. \n> **A.3.** Thanks for this valuable comment. Unfortunately, our PGN might not generalize to query-based attack methods. Here, we provide the following analysis. \n> 1) The goal of query-based attack methods is different from ours. The goal of our method is more concerned with improving the transferability of the adversarial examples on different black-box models, while the query-based method mainly focuses on the attack success rate on the source model.\n> 2) Query-based attack methods mainly exploit limited output information such as labels and logits, while our PGN needs to utilize the gradient to the objective loss function. Hence our method is not inherently applicable on query-based attack methods.\n\n**Q.4.** The proposed method is similar to the previous work [R1].\n> **A.4.** Indeed, both our PGN and [R1] penalize the gradient norm. However, there are significant differences between these two works, which are summarized as follows:\n> 1) **Different goals**. While [R1] primarily focuses on enhancing flat local minima to improve generalization during model training, our main objective is to investigate the potential impact of flat local minima on the transferability of adversarial attacks. Previous research in the field of adversarial attacks has shown limited attention to the relationship between flat local optima and adversarial transferability. Consequently, our work significantly contributes new insights to the domain of adversarial attacks.\n> 2) **Objective function**. A distinguishing feature of our proposed method lies in its emphasis on the gradient information surrounding the adversarial example. Specifically, we approximate the second-order Hessian matrix using the finite difference method, which is crucial for our approach. The work presented in [R1] also supports and provides theoretical foundations for the feasibility of our method within the realm of adversarial attacks.\n>\n> In summary, our main motivation in this work is to explore whether flat local optimum can improve the adversarial transferability.  To the best of our knowledge, it is also the first time to use penalized the gradient norm and finite difference method to support our motivation in the field of adversarial attacks.\n>\n> [R1] Zhao et al. \"Penalizing gradient norm for efficiently improving generalization in deep learning.\" ICML 2022."}, {"review_id": "c0PBkS6k4V", "reviewer": "Reviewer_8Bpx", "summary": "The paper proposes a method called Penalizing Gradient Norm (PGN) to improve the transferability of adversarial perturbations. The method is motivated by the observation that encouraging the flatness of the local landscape for adversarial examples can lead to better transferability, and thus PGN regulates the process of gradient-based adversarial attack algorithms by penalizing the magnitude of the loss gradient with respect to the input. Since such a regularization process requires the input Hessian which is a computationally expensive process, PGN utilizes the finite difference method to approximate the Hessian matrix. Experiments on the ImageNet-compatible dataset demonstrate that the proposed method can improve the transferability of untargeted attacks in comparison to other baseline methods.", "questions": "The issue around Sec. 3.2 stems from the lack of rigor in Assumption 1.\nHow is a local region defined? What is the definition of flatness, and how to measure it?\nFollowing the assumption, why is the maximum l2 norm used in (3)? not a value averaged over l2 norm of data points sampled around x?\nConsider inputs that fail to transfer, but now are transferable because of the modified objective. Are they indeed situated in a flat region?\nMore importantly, why is (3) necessary to validate the assumption? Since the assumption is agnostic to the attack algorithm, it should be true to any adversarial attack methods: I-FGSM, MI-FSGM, VIM-FGSM, etc. As such, given any attack algorithm, we should compare the flatness between adversarial examples that are transferable and those which fail to transfer.\n\nLn 163 requires clarification, why do we have such expectations? \n\nThe FD method circumvents the expensive computation of the input Hessian, and the approximation becomes accurate with decreasing value of \\alpha. Why is the \\alpha used in FD the same as the \\alpha used in the iterative process of generating adversarial examples? To achieve an accurate approximation of the Hessian, shouldn't the stepsize used in FD be very small? \\alpha = \\eps/T seems to be a large value to me. \n\nWith the current choice of \\alpha, how close is the approximation (5) to the actual Hessian?\n\nEvaluation:\nIs PGN based on standard ifgsm, or one of the momentum variants? Since RAP is the closes method to PGN, is RAP used in the evaluation based on the standard ifgsm as well?\n\nLn303: The statement of \"flat local minima result in better generalization\" being a fact is quite strong. \n\nAlso, I would suggest the author proofread the paper. There are several very noticeable typos. For instance, even the name of the method is spelled incorrectly as \"Penalizing\" (Ln48, Ln55)", "limitations": "I suggest the author include a brief discussion of the limitation of the proposed work.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Originality: The proposed method is original and intuitive. \n\nClarity: The general structure of the paper is very clear: moving from validating an assumption to proposing an algorithm, and finally evaluating the proposed method with empirical results. \n\nSignificance: The proposed method addresses a practical security concern of deep learning models. The proposed method improves the transferability of the adversarial perturbations compared to existing gradient-based methods. Extensive empirical evaluations were performed to demonstrate the efficacy of the proposed method.", "weaknesses": "Reverse adversarial perturbation (RAP) is a closely related work that encourages adversarial examples to be located at a region with low loss values. To demonstrate the novelty and significance of the proposed method, the paper needs a detailed discussion of the differences and similarities compared to RAP.\n\nOne of the major contributions claimed by the paper is the empirical validation that \"adversarial examples located in flat regions have good transferability\", and it is mainly covered in Sec. 3.2.\nPutting aside the significance of the contribution, the authors should be very careful about the claims and statements in Sec 3.2. The assumption and the followed empirical validation both suffer from the lack of rigor and thus weaken the significance of the contributions. Please see the Questions section for additional discussions.\n\nSome technical details in Sec 3.3 require clarifications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nDear Reviewer 8Bpx,\nThanks for your valuable and insightful comments. We address your concerns as follows:\n\n**Q.1.** PGN v.s. RAP\n>**A.1.** As you suggested, we discuss the similarities and differences between PGN and RAP methods as follows.\n>1) **Empirical verification**. Both RAP and PGN aim to achieve flat local optima for more transferable adversarial examples. Besides, we are the first work that provides an empirical verification that adversarial examples in flat local optima are more transferable.\n>2) **Methodology**. RAP injects worst-case perturbations into the optimization process to maximize the loss and achieve falt local optima. In contrast, PGN introduces a gradient norm into the loss to achieve a flat local optimum. Hence, the method we employed differs significantly from RAP. \n>3) **Efficiency**. Our method is more computationally efficient. In RAP the outer loop is 400 and the inner loop is 8. In contrast, our PGN method employs finite differences to approximate the Hessian matrix and samples multiple points to obtain a more stable gradient, which requires only 10 outer loops and 20 inner loops for the computation. Thus, our method not only improves computational efficiency but also achieves a higher attack success rate. \n\n**Q.2.** The issue around Sec. 3.2...which fail to transfer.\n>**A.2.** \n>1) In mathematics, we define the example $x'$ in the $\\epsilon$-neighborhood of the input image $x$ as a local region. Flatness indicates the maximum gradient value in the neighborhood of the sample. The smaller the gradient value is, the flatter it is. To characterize the flatness of the loss surface, we define the slope $k=\\frac{J(x_0)-J(x_i)}{\\||x_0 - x_i \\||_2}$, where $x_0$ represents the center point of the 2D map in Fig. 2, and $x_i$ represents the points sampled near the center point $x_0$. \n>2) During the optimization process, there might be saddle points. If we use averaged gradient, saddle points will reduce the size of the averaged gradient value, making us unable to perceive the sharper regions. Hence we use the maximum gradient, which can better perceive the worst-case gradient.\n>3) To address your concerns, we selected images (about 286 images) that can be transferred using our PGN method but cannot be transferred using MI-FGSM. We calculate the average slope $k$ of adversarial examples on the Inc-v3 model for these 286 images. **See the first table in PDF**. The adversarial example generated by our PGN have smaller slopes, confirming that our method does craft adversarial examples in flatter local regions.\n>4) The goal of adding the penalized gradient norm is to make the gradient value smaller, and a smaller gradient value also means that the local region is flatter. Thus, if we verify that Eq.3 can improve the adversarial transferability, it naturally verifies that our assumption is valid, i.e., flat local optima have better transferability. \n>5) Our assumption is agnostic to the attack algorithm. Hence, PGN is suitable for any gradient-based attack methods. To address your concerns, we combine PGN with these gradient-based attack methods to generate adversarial examples on Inc-v3 and compute their flatness (averaged on 1000 images). **See the second table in PDF**. Our PGN method can effectively improve the adversarial transferability of existing attacks and make the adversarial examples located in flatter regions. \n\n**Q.3.** Ln 163 requires...expectations?\n>**A.3.** Intutitively, it is expected that the loss of these data points in a small neighboorhood are similar. To address your concerns, we evaluated the value of $J(x_t^{adv})$ and $J(x')$ during the iteration process. In each iteration, we first calculated the value of the loss function for the adversarial example $x_t^{adv}$, then we randomly sampled a sample $x'$ in the neighborhood of the $x_t^{adv}$ and calculated the loss function $J(x')$. **See the third table in PDF**. It can be observed that both loss function values are relatively approximate. We can also observe that the expectation of $J(x')$ is smaller than that of $J(x^{adv})$, and maximizing a smaller value during the optimization process is more beneficial for our optimization.\n\n**Q.4.** $\\alpha=\\epsilon/T$ seems to be a large value \n>**A.4.** We conduct PGN using smaller step sizes **see the fourth table in PDF**. We can see smaller step sizes cannot improve adversarial transferability. This is because the smaller the step size, the closer two neighboring gradients are, leading our method to degrade to MIFGSM. To avoid redudant hyper-parameters, we simply adopt the step size of $\\alpha$ in this paper.\n\n**Q.5.** With the current...the actual Hessian? \n>**A.5.** To address your concerns, we compared the cosine similarity of perturbations generated by the Hessian matrix and finite differences method. The average cosine similarity of 1000 images is about 0.9035. It can be observed that the current finite difference can effectively approximate the adversarial perturbation generated by the Hessian matrix.\n\n**Q.6.** Evaluation: Is PGN...well?\n>**A.6.** For fair comparison, our PGN and RAP adopt MI-FGSM for evaluation.\n\n**Q.7.** Ln303: The...being a fact is quite strong.\n>**A.7.** Existing works [R1, R2, R3] have shown that more flat local minima often result in better model generalization from empirical and theoretical perspectives. We will rephrase that sentence as \"Inspired by the observation that flat local minima often result in better generalization\". \n>\n>[R1] Keskar et al. On large-batch training for deep learning: Generalization gap and sharp minima. ICLR. 2017.\n>\n>[R2] Neyshabur et al. Exploring generalization in deep learning. NeurIPS. 2017.\n>\n>[R3] Foret et al. Sharpness-aware minimization for efficiently improving generalization. ICLR. 2021.\n\n**Q8-Q10**\n> We have corrected the typos and a discussion of the limitations will be added in the revised paper. Technical details in Sec 3.3 have be provided in Appendix. "}, {"review_id": "U3EtVWU0Ay", "reviewer": "Reviewer_2kUx", "summary": "The authors empirically observed that the adversarial data lies on the flat local maxima yields enhanced attack transferability. Inspired by this observation, this paper proposes a regularization to help the gradient-based attacks to find the adversarial data at the flat local maxima. This regularization penalizes the gradient norm around the adversarial data and can be efficiently computed via the finite difference method. The empirical results validate the effectiveness of the proposed method in improving attack transferability.", "questions": "1. Could the authors provide the error bar?\n\n2. Could the authors provide a comparison between the proposed method and the previous work [1]? \n\n3. Could the authors discuss how the effectiveness of the finite difference method in speeding up optimization empirically/theoretically? What are the computational consumption and the attack success rate with/without the finite difference method? \n\n4. What is the performance of the proposed method compared the baseline methods evaluated on the transformer-based models (i.e., vision transformers)?\n\n[1] Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets, ICLR 2020.\n", "limitations": "The motivation of the proposed method lacks theoretical support.\n\n", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The motivation of the proposed method is clear. The observation in Figure 1 is interesting and inspiring.\n\n2. The authors utilized the finite difference method to make the computation more efficient.\n\n3. It seems that the proposed method can significantly improve attack transferability. \n\n", "weaknesses": "1. The proposed method is not theoretically motivated, which degrades its soundness. The authors claimed the optimization of perturbation equals the model training process using an analogy. However, it lacks supportive theoretical results to support this claim. Since the aforementioned claim is not solidly proven, the reason for the adversarial data at flat local maxima yielding better transferability seems unclear.\n\n2. The paper does not provide the standard variance of the reported results to validate their significance. \n\n3. The paper lacks some empirical and theoretical analyses of the effectiveness of the finite difference method in speeding up optimization. I think the optimization of the gradient norm is very important for the proposed method. Therefore, the effect of the acceleration method is worth studying.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 2kUx,\nThanks for your valuable and insightful comments. We address your concerns as follows:\n\n**Q.1.** Could the authors provide the error bar?\n> **A.1.** Following your suggestions, we will add all error bars to the final version of the paper. Here we provide part of the error bars as follows. From the results, it can be seen that the standard deviation of our PGN method is not very large, which indicates that our method is stable.\n>\n> |Source:Inc-v3|Inc-v4|IncRes-v2|Res-101|\n> |:-:|:-:|:-:|:-:|\n> |VMI|74.81$\\pm$0.57|70.22$\\pm$0.69|75.84$\\pm$0.47|\n> |EMI|81.12$\\pm$0.47|76.90$\\pm$0.72|80.43$\\pm$0.63|\n> |RAP|82.44$\\pm$0.51|79.27$\\pm$0.64|81.43$\\pm$0.72|\n> |PGN |91.61$\\pm$0.73|90.62$\\pm$0.94|89.14$\\pm$0.68|\n\n**Q.2.** Could the authors provide a comparison ... and the previous work [R1]?\n> **A.2.** As you suggested, we compare our PGN with SGM [R1] using Res-18 model with MI-FGSM as the backbone method. Since SGM modifies the backpropagation and is compatible with our PGN, we also integrate PGN into SGM to evaluate its generality to other attacks. As shown in the following table, our PGN consistently exhibits better attack performance than SGM, which further shows its superiority in boosting adversarial transferability. Besides, PGN+SGM outperforms PGN as well as SGM, showing its remarkable compatibility with various attacks. We will add it in the revision.\n> \n> |Source:Res-18|Res-101|Res-152|Inc-v3|\n> |:-:|:-:|:-:|:-:|\n> |MI|82.8|73.3|54.5|\n> |SGM|89.1|82.5|66.0|\n> |PGN|95.9|92.9|77.9|\n> |PGN+SGM|96.7|93.8|80.1|\n> \n>[R1] Wu et al. \"Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets.\" ICLR 2020.\n\n**Q.3.** Could the authors discuss ... with/without the finite difference method?\n>**A.3.** As you suggested, we first theoretically analyze the acceleration effect of the finite difference (FD) method. For the baseline attack method I-FGSM, the gradient is computed only once per iteration. Thus, its computational complexity is $O(n)$, where $n$ represents the image size. However, upon introducing the penalty gradient term, the need arises to compute the second-order Hessian matrix, leading to a theoretically computational complexity of $O(n^2)$.  To address this, we use the finite difference method as an approximation to the Hessian matrix, which requires the computation of the gradient twice in each iteration, effectively yielding a computational complexity of $O(2n)$. This theoretically promises significant improvements in computational efficiency.\n>\n> Additionally, we substantiate our theoretical analysis with comparative experiments. These experiments were conducted on an RTX 2080 Ti with a CUDA environment. We employed I-FGSM and evaluated the total running time on 1000 images (excluding data loading time) and the attack success rate on black-box models. The results are presented in the following table. Directly optimizing Eq. 4 results in better attack performance with high computational resources. With the finite difference method, we can better approximate the performance of direct optimization of the second-order Hessian matrices, which significantly reduces the running time and the computational memory. Furthermore, owing to the relatively modest image size ($299\\times 299\\times 3$) and the comparatively small number of parameters compared to the model, the accelerated computing capabilities of CUDA enable the actual running time to surpass the theoretical estimates. We will add this discussion in the revision.\n> |w/o or w/ FD|Inc-v4|IncRes-v2|Res-101| Times (Total)| Memory Size|\n> |:-:|:-:|:-:|:-:|:-:|:-:|\n> |I-FGSM (Backbone)|27.8|19.1|38.1|52 s|1631 MiB|\n> |Hessian matrix+w/o (FD)|39.2|30.2|47.0|469 s|7887 MiB|\n> |Hessian matrix+w/ (FD)|37.9|28.6|45.7|96 s|1631 MiB|\n\n**Q.4.** What is the performance ... the transformer-based models?\n>**A.4.** Please refer to the response to Q.1 of Reviewer PZXp.\n\n**Q.5.** The proposed method is not ... transferability seems unclear.\n>**A.5.** In [R1], Lin et al. analogize the adversarial example generation process to the standard neural model training process, where the input $x$ can be viewed as parameters to be trained and the target model can be treated as the training set. From this prespective, the transferability of adversarial examples is equivalent to the generalization of the normally trained models. Intuitively, training model and adversarial perturbation generation are both optimization problems, which derives such analogy. Besides, this analogy has been widely accepted by nemerous works, such as exploring better optimization methods (NI [R1], VMI [R2]) or data augmentation methods (ODI [R3], DITL [R4]), which are effective in improving the transferability of adversarial examples.\n>\n> In fact, it is difficult to provide a theory to support the analogy between adversarial transferability and model generalization in the field of adversarial attacks. In this work, we are inspired by this analogy and attempted to enhance the transferability of adversarial examples from a new perspective. Hence, we try to explore flat local minima to enhance the transferability of adversarial examples. We assumed and experimentally verified that the flat local optima are related to the transferability of the adversarial examples, and the loss surface maps of the generated adversarial examples in Fig. 2 also validates our motivation. Also, we will keep studying the theoretical connection between transferability and flat local minima in our future work.\n>\n> [R1] Lin et al. \"Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks.\" ICLR. 2020.\n>\n> [R2] Wang et al. \"Enhancing the Transferability of Adversarial Attacks through Variance Tuning.\" CVPR. 2021.\n>\n> [R3] Byun et al. \"Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Input.\" CVPR. 2022.\n>\n> [R4] Yuan et al. \"Adaptive Image Transformations for Transfer-based Adversarial Attack.\" ECCV. 2022."}]

Author rebuttal:
Dear Reviewers and Area Chairs:

We sincerely appreciate all of your precious time and constructive comments.  The results of the partial response data (Tables) are in the pdf file.

Meta-review: Overall, this is a borderline paper.  The reviewer 8Bpx gave 3 score and provided many valuable comments. AC read the discussion between authors and Reviewer 8Bpx. The main concern of this paper is the Sec 3.2 is not rigorous. AC agrees with the reviewer's comment by reading the paper. However, when reading the rebuttal, AC feels that the authors provide good explanation. Although this paper lacks of  supportive theoretical results, overall, this paper provides new insights and understanding in term of transferability of adversarial examples. AC tends to accept this paper. However, AC indeed viewed the reviewers' comments are very valuable and the authors should add them in the revised version.
Decision: Accept (poster)