Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness
Authors: Mingyuan Fan
Xiaodan Li
Cen Chen
Wenmeng Zhou
Yaliang Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Adversarial examples; transferability; transfer-based attack; theoretical analysis
Primary area: safety_in_machine_learning

Abstract: A prevailing belief in attack and defense community is that the higher flatness of adversarial examples enables their better cross-model transferability, leading to a growing interest in employing sharpness-aware minimization and its variants. However, the theoretical relationship between the transferability of adversarial examples and their flatness has not been well established, making the belief questionable. To bridge this gap, we embark on a theoretical investigation and, for the first time, derive a theoretical bound for the transferability of adversarial examples with few practical assumptions. Our analysis challenges this belief by demonstrating that the increased flatness of adversarial examples does not necessarily guarantee improved transferability. Moreover, building upon the theoretical analysis, we propose TPA, a Theoretically Provable Attack that optimizes a surrogate of the derived bound to craft adversarial examples. Extensive experiments across widely used benchmark datasets and various real-world applications show that TPA can craft more transferable adversarial examples compared to state-of-the-art baselines. We hope that these results can recalibrate preconceived impressions within the community and facilitate the development of stronger adversarial attack and defense mechanisms.

Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness
Mingyuan Fan', Xiaodan Li Cen Chen Wenmeng Zhou', Yaliang Li* 'School of Data Science & Engineering, East China Normal University, China
2 The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China Alibaba Group, Hangzhou, China Alibaba Group, Bellevue, WA, USA
fmy26609660gaiil com, cenchenddase ecnu, edu. cn, (fiona. 1xd, wenmeng zwm, yaliang (aleeaallbble Sallibbba-iine inc. com
Abstract
A prevailing belief in attack and defense community is that the higher flatness adversarial examples enables their better cross-model transferability, leading to growing interest employing sharpness-aware minimization and its variants. How- ever, the theoretical relationship between the transferability of adversarial examples and their flatness has not been well established, making the belief questionable. theoretical investigation and, for the first time,
To bridge this gap, embark derive theoretical bound for the transferability of adversarial examples with few practical assumptions. Our analysis challenges this belief by demonstrating that the increased flatness of adversarial examples does not necessarily guarantee improved transferability. Moreover, building upon the theoretical analysis, we propose TPA, a Theoretically Provable Attack that optimizes a surrogate of the derived bound to craft adversarial examples. Extensive experiments across widely used benchmark datasets and various real-world applications show that TPA can craft more trans- ferable adversarial examples compared tftteoof-hhe-rt baselines. We hope that these results can recalibrate preconceived impressions within the community and facilitate the development stronger adversarial attack and defense mechanisms. The source codes are available in https: : //g thub. com foon/mmy666/ TPA
Introduction
The transferability of adversarial examples 5 suggests a phenomenon where adversarial examples designed fool one local proxy model can also be generalized to mislead other unknown target models, even with different model parameters and architectures. This intriguing property has significant implications for various applications, ranging from the robustness assessment of large- However, it is observed that the generated adversarial examples often tend to be overly specialized the proxy model, showing limited transferability [27 30 36). Driven by this, researchers have dedicated substantial efforts to techniques (24 35).
scale models 110 27 to data privacy protection [17 23 32]
developing wunsferability-enhhnee
Most cunnferrbblitty-ehancii techniques have drawn inspiration from an analogy between the transferability of adversarial examples and the generalization capability of models [25]
36 44] For
example, borrowing the idea of Mixup (46), Admix 377 attempts to blend adversarial examples with other samples to generate more transferable adversarial examples. Recently, Foret et
[4 observed
a strong correlation between the flatness of the model's 's loss landscape and its generalization capability,
"Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

thus advocating for the optimization of the worst-case loss within a small perturbation radius to regulate the loss landscape. The empirical performance of such sharpness- aware aware minimization is quite impressive, sparking a surge of interest among researchers and catalyzing the development of its variants. Given the above considerable progress, the exploitation of sharpness-aware minimization and its variants to enhance the transferability of adversarial examples has become one of the hottest
research topics [22 15 42 45).
techniques based on flatness achieve state-of-the oftt-offthh-arr art performance [5 30 42,, the theoretical relationship between adversarial transferabil- ity and flatness has not been well established. As such, it remains questionable whether adversarial examples converging to flat extreme points, referred as flat adversarial examples. necessarily possess better transferability. In fact, the relationship between flatness and generalization capability still under debate and not well-solved. Recent works [ provoking finding, i.e., flat extreme points do not always lead to better generalization ability, making the relationship between adversarial transferability and flatness even more mysterious. Given the aforementioned gap in our current understanding, we first theoretically investigate the following critical question:
Although empirical results show that transferabiliy--enhaeeee
39 have brought to light a thought-
Q1: Are flat adversarial examples necessarily associated with improved transferability?
Our theoretical examination yields negative answer to Q1. Specifically, we derive bound of adversarial transferability and demonstrate that, despite being a contributing term, flatness alone is insufficient to determine the value of the bound. Then, our attention naturally shifts to explore Q2:
Q2: Can our theoretical analysis guide to develop a more principled transfer-based attack?
One straightforward idea for addressing the Q2 is to directly optimize the derived bound. However, the optimization of this bound requires higher-order gradient information, which prohibitively expensive in gigh-dimensionl spaces. To circumment this issue, we introduce Theoretically Provable Attack (TPA), which employs a theoretically- erooruically-grrondeed surrogate of the original bound as an alterna- tive. The optimization of the surrogate only requires first-order gradient information, rendering TPA more computationally efficient and practical.
It should be stressed that the surrogate in TPA is theoretically supported, ensuring that its attack effectiveness enjoys theoretical guarantees. Morcover, the empirical evaluation of TPA in the benchmarks and three kinds of real-world applications demonstrates the superior performance of TPA over the state-of-the-arr attack methods. Our contributions are three-fold:
To our best knowledge, we are the first to derive a theoretical bound on the transferability of adversarial examples, with few practical assumptions. Contrary to the prevalent belief among the community, we demonstrate that the flatness of adversarial examples does not necessarily guarantee enhanced transferability. Our theory also advances the understanding of the intrinsic characteristics governing transferability and lays solid theoretical foundation for future research in this area.
Building upon our theoretical analysis, introduce TPA which optimizes surrogate of the original bound to avoid direct computation of high-order gradients. We demonstrate the effectiveness of the surrogate the surrogate of the derived bound.
We conduct extensive experiments in the benchmark dataset ImageNet, showing the impres- sive performance of TPA. To our best knowledge, TPA is the first transfer-bused attuck to achieve an average 90% allack success rale against transformer architectures. We also test the effectiveness of TPA in three kinds of real-world applications that are under-explored in the existing works, including Google Vision Systems, advanced search engines, as well as multimodal large model applications (c.g., GPT-4 and Claude3).
2 Related Work
2.1
blatdess-based Optimization Methods
Intuitively, the loss landscape of model on training data mirrors that on test data, suggesting that slight translation of the loss landscape over training data can approximate the loss landscape over

---

test data. Given this, flat extreme point is believed to enjoy smaller performance gap between training and testing data than sharp extreme points. i... better generalization ability. Capitalizing on this intuition, SAM 6 7 [4] incorporates the following flatness term or its variants into its loss function to improve the model's generalization capability:
max [(Fo+p(x),)) y) ((Fo(z), le
(I)
where (z. (2, C, F6 and are training data pairs, loss function, model parameterized by 8, and perturbation budger, respectively. Equation 1 evaluates the loss landscape's flatness at point 0 by quantifying the increase in loss when moving from / to neighboring worst-hehaved parameter value. Subsequent studies explored various flatness metrics, such as Fisher determinant [20] and gradient
norms [47 48). for regularizing loss landscapes. Nevertheless, recent works [ 39 have cast a critical light. Dinh et al. 3 demonstrated the existence of networks that have a sharp loss landscape yet manage to work well on test data, Andriushchenko et al. [1] empirically revealed that there not strong correlation between flatness and generalization ability, particularly in large neural networks.
2.2 Transferabillit--nnannnng Methods
Transfrability-chhannnng methods can be roughly divided into imput-regularization -ased methods, optimization- based methods, and model-based methods. In each iteration, wouurrrguuaizatnon-baaeee
methods g 36 44 ensemble multiple transformed inputs to craft adversarial examples, and the distinction between these methods is reflected in the adopted transformation techniques. DI [44 suggests to resize and pad inputs, while TI [5] translates the inputs and SI [25] scales the inputs. By observing that existing transformations are all applied on single image, Admix [37] attempts to admix inputs with an image from other categories. Unlike the spatial domain transformations mentioned above, SSA 27 perturbs inputs in frequency domains to produce more diverse transformed inputs. BSR [35] segments inputs into non-overlapping blocks, shuffling and rotating the blocks randomly.
25 and model-based methods 50 For instance, MI 4 adopts Momentum optimizer. Besides, SGM [40] refines the back- propappopagttion procedure to amplify the gradients of early layers, due to that the features learned by early layers are more shared over different models. StyLess (24] employs stylized networks to prevent adversarial examples from using non-robust style features. More recent advancements delved into the exploration of flatness- based optimization methods to improve the transferability of adversarial examples. Qin et [30] introduced RAP, which tailors Equation to craft flatness-enhanced adversarial examples. Some studies [15] 42 chose to punish the gradients of the generated adversarial examples.
Optimization-basse methods leverage more advanced optimizers enhance the transferability via the lens of model per se [22
3 Theoretical Analysis
In this section, we embark on a theoretical exploration of adversarial transferability, commencing with the canonical setup of transfer-basee based adversarial attacks. The primary objective of transfer- based attacks is to make adversarial examples crafted using the local proxy model F' as effective as possible for the target model F". Formally, for a given natural samples z with ground-truth labels y, the vanilla transfer-basee attack solves the following optimization task to craft adversarial noises 5* for as:
8 = arg arg min L(F(x + 8), v), s.t., ||81|cc S.
where L is cross-entropy loss and E is perturbation budget. Despite the high attack effectiveness of x + 8* on the proxy model, their effectiveness significantly diminishes when transferred to the target model. To better study this problem, we decouple transferability into two factors: the local effectiveness term, which measures the loss of generated adversarial examples on the proxy model, along with the transfer-relate related loss term, which quantifies the change in the loss of adversarial examples when transferring from the proxy model to the target model. The local effectiveness term and the transfer-related loss term can be evaluated by L(F(z + 6),y) and D(x + d,y) = L(F'(z + 6)..) v) - L(F(s + v), respectively. Intuitively, transferable adversarial examples are characterized by high attack performance on the proxy model together with minimal increase in the transfer-related loss term upon moving to the target model. Before presenting our theoretical results, we first make several practical assumptions.
3

---

Assumption 3.1. The underlying distribution of x is continuous and bounded: there exists a constant B₁ such that Vx. it holds that p(z) S B
Assumption 3.2. The gradient norm of p(x) is bounded: there exists a constant Bg such that Vz, holds that V p(22)1 S Bz.
Assumption 3.3. The probability of adversarial examples occurring in nature is less than that of natural samples: p(r + 6) S p(r).
Assumption 3.4. Proxy-model- -eenertteeeeteed adversarial examples have greater loss on the proxy model than on the target model: (x + 8).v) S C(F(x + 6), y)-
Assumption 3.5. The proxy model is based on ResNet-like architecture, where each layer applies linear transformation followed by ReLU activation.
Theorem 3.1. (See Appendix for Proof.) Let Assumption 3.1 that:
3.5
stand. For small |81| it holds
px(x) (1) +, 8,1) s E EptollID(E w)1B + + 1 v x)1
inherent model difference component
+ (1+ + C)Bp())) 11181 V log Fax + 5)113) +2 Eptr) {11811 Σ 1 Foz 6)i,
(3)
The forst-order gradient component
The second-order gradlent component
where C== Bi+ 81 and [i, i] denotes the element at the intersection of the i-th row and the j-th
column in the given matrix. Let the sum of the three terms on the right side of Equation 3 be denoted as K. Then, we have:
[C(F(x 2)11 101 +6) 6), K)1.
We use p(x) to denote the underlying distribution of and posit Assumption 3.1 3.5 Assumption 3.1 arises naturally since natural samples do not occur with infinite probability. Building upon Assumption 3.1. straightforward deduction is that the gradients of p(x)) are bounded; otherwise, there exists at least one sample that appears with infinite probability. Assumptions a333 Assumption 3.4 are naturally valid. Regarding Assumption 3.3 adversarial samples are specifically crafted rather than commonly occurring in nature. Moreover, discussions about transfer-based attacks only make sense under the conditions set by Assumption 3.4 Assumption 3.5 is satisfiable because the attacker can fireely choose which proxy model to use. Theorem 3.1 establishes bounds for [| + 8, y) and [|C(F"(z + 8). y) I1. We mainly discuss the bound [z + 6, 1). which composed of three key components, namely the inherent model difference, the first-order gradient, and the second-order gradient components. For the sake of simplicity in our discussion, ||ō] considered constant.
The inherent model difference component. This component is intuitively interpreted as a measure of the similarity between the proxy and target models by evaluating their output differences regarding natural samples x. A higher value of the component implies that adversarial examples produced by the proxy model are less likely to transfer to the target model.
In practice, the value of this component tends to be modest due to the following reasons. Firstly, the target model is expected perform well on æ, misclassification of the target model to would render crafting adversarial examples for æ trivial. Secondly, as the attacker is capable of fine-tuning the proxy model on x, the loss of x in the proxy model can also be small. Furthermore, if the diserepancy between the two models is small, their gradient differences also are minor as well Thus, both (xx, (z. 11 and [z, 1) are of small values.
Moreover, this component illuminates the selection criteria for an appropriate proxy model. While previous studies have highlighted that proxy and target models exhibiting structural and training data similarities tend to facilitate the transferability of adversarial examples across these models, our theoretical insights suggest a more nuanced criterion. Specifically, a proxy model needs only yield predictions for r that are closely aligned with those of the target model, thereby relaxing the requirements of architectural and data similarities.
'In fact, there is C' < 1. and using C' of 1 could be useful in certain situations, such as calculating the specific value of upper bound for [1 + 6, 1).12 12. Here, for the sake of the tightness of bound, we have not done

---

The first-order gradient component and the second- order gradient component. Our analysis here chal- lenges the prevailing belief [12 15
35 [Rxoost="]
35 (2oasta") 2xsinntsF))
30
42
45 that
=bbll++ +bal
flat adversarial examples necessarily induce improved transferability. In Theorem 3.1] the factors that are relevant to the direction ò include the first-order and second-order gradient components. The first-order gradient component serves as a direct measurement of the flatness associated with adversarial examples. Nonetheless, the presence of the second-order gradient component unveils that transferability is not exclusively contingent on flatness. other words, an adversarial example that appears flat but has higher second-order gradient component might not be optimal for the target model. To elucidate this point, consider the function f()) = sin(x²) as an example. Figure presents the norms of first-order gradient Yı and the second-order
Bust Sabitii for
Figure 1: order gradient, the second-order gradient of y sin x2. The black stars symbolize the location where the minimum values of y/ and y/ are achieved.
The visualization of the first-
gradient of f(x), as well as their sum ya- Upon examination, it is observed that optimizing the norms 1/1 alone does not yield optimal solutions for optimizing V/3 This highlights the inadequacy of minimizing only the first-order gradient component in ensuring the minimization of the bound.
4 The Proposed Approach TPA
4.1 Overview
In Section 3] the adversarial transferability is decoupled into the local effectiveness term and the waanserability-relatee loss term and we derive a bound for the latter. It is natural to jointly optimize both the local effectiveness term and the bound for generating more transferable adversarial examples. The first term maintains the effectiveness of generated adversarial examples against the proxy model while the bound controls the performance degradation upon transfer to the target model, thus achieving better transferability. However, the bound involves the second-order gradient component that is quite expensive to optimize. To this end, as shown in Section 4.2 TPA instead develops computationally feasible surrogate for the bound that retains optimization integrity, substantiated through theoretical validation. Section 4.3 introduces an approximate solution to efficiently tackle the optimization target.
4.2 Optimization Formulation
The optimization target of TPA is formalized as follows:
8"
arg min L(F(x 8),8) AEA-CI-LO +5 A),v)ll),,
(5)
s.t., 1601 Sc. 0,
where U(-b, b) is uniform distribution between -b and b, À is penalty magnitude, and 5 + A denotes a sample extracted uniformly from the region around - CCF((++0)) 0)) y) as the local effectiveness term, while [/ (z 5 A), serves surrogate for our bound. Notice that our proposed surrogate is designed to regulate both the first- and second- order gradient component, excluding the inherent model difference component. This is because the inherent model difference component remains a constant term given specific perturbation norm constraint and the proxy-target model pair.
8. In Equation [ we employ
We next demonstrate the effectiveness of the surrogate. The surrogate minimizes the gradients of samples around x + 8, intuitively leading to minimization of the gradient norms of x +-+ themselves as well. Our main investigation is determining if the surrogate can effectively minimize the second- order gradient component. The second-order gradient quantifies the rate at which the first-order gradient changes. Through penalizing the gradient norms of samples around x + 5, the surrogate naturally encourages the gradient norms of samples around + 8 towards zero, thereby implicitly
5

---

moderating the second-ores -oder gradient. Theoretically, is demonstrated as follows:
V Vlog F(x + 8 w)ld v log F + 8][1
Σ
V2 log + =
lim
- H|
=|| lim (VVL(((VV+++++)))) ( p=0
p)) (L +6),
According Equation 6 if the gradient norms of the samples around x + 8 approach zero, the second-order gradient components also tend towards zero. Thus, the surrogate is capable of effectively regulating both the first- and second-order gradients.
We see that the optimization target of 'PPA bears a high visual similarity to those in some studies 15 42 which penalties the gradient norm of x + 6. Despite this superficial similarity, there exists fundamental difference in their effects. As claborated in Section 3 penalizing only the gradient norm of is unable to regulate the second-order gradient component.
4.3
Approximate Solution
The highly non-linear and non-convex nature of F()) render the analytic solution of Equation 5 to be hardly derived 21 As result, standard practice for solving Equation is employing gradient-based optimization methods [2]. However, as shown Equation 7 the gradients of Equation 5 invollve Hessian matrix evaluated at multiple points x + 8 + A. for A, 1,. each of which is troublesome evaluate in high-dimension spaces
29
N
- V[((F( + +))
VI A.)
N
VC(F + + A.).)) VL(F(x + A.).y)la
= -
Ho+6+A
N
To get rid of directly evaluating Hessian matrix in Equation] 7
we propose an approximate estimation for H H VC(F(=+6-A.)..) [ To achieve this, we utilize Taylor expansion on C(F(z + 5
A, + Φ), v). assuming Φ is small enough for making expansion feasible:
[(F(x + 8 A, + Ø). y) = C(F(x + A.), v) + VL(F((z + Ai).
(8)
Furthermore, by differentiating both sides of Equation g we obtain:
VL(F(x 5 A p). v) (L 5 A.). H.TStA.
Notice that, the desired item H4+6+A, | arises if setting along the direction (.)) By implementing the idea, there is:
vC(
[
VL(F(x CC[F(++0++ A:),
H VC(F(( + 2)1l
(10)
VL(F + + k [V ).v) VL(F x + 8 Ai).
+C(F@++++AM+
A
k
Wherein, h VC(F(rt&+A)..))
[VZ[F(2+++A) 1. and & is small constant to cater ø being small. By substituting Equation 10 into Equation 7 obtain the approximately estimated gradients of Equation 5 Com- pared to directly evaluating Hessian matrix that requires quadratic storage and cubic computation time [13 29 our approximate solution only involves first-order gradients (linear computation time) so as to make TPA more efficient computationally. Moreover, linear expansion used in Equation results in approximation error of O(k) 33
5
Simulation Experiment
5.1 Setup
Dataset. We randomly select 10000 images from ImageNet.

---

Table 1: The attack success rates (%) of different attacks on normal models. Best results are in bold.
Novic
Metiol
Acelicio
Dxachet121
1xcesaaV)
MobicNAV2
Sprcccaa
StatieNaN2
Rightt
MIASN41
WaddloNNc00
VGG-3
100
$7:0)
15.41
08.33
06.51
36: 55
26.11
13:54
22.91
81:13
90:11
8.00 0.59
03.50
15.88
10037
03:33
08:35
09634
17.58
12.81
43:30
66.82
13.54
13.51
48.13
(83.0
13.61
55.38
33.39
35.29
08.33
00:34
41:83
5908
06.33
13:00
63:38
36.55
3131
55.
131.00
05:33
81.56
76.33
77-41
10.90
17.00
18340
18.90
03.00
01.55
05.33
56.31
58.64
190.00
07.00
65:16
V5.53
04.53
14.81
01.44
6.:30
01.00
10.13
77.55
Ope
09.55
93.09
99.52
96.70
99.52
99.72
99.82
$4.56
97.37
99.82
99.44
95.55
05.53
94.95
15.44
$5.02
35.30
30.57
10.54
77.55
12.17
16.90
$1.30
90.36
32.33
45.43
55.23
06.76
33
10
70.38
11
78.05
03.88
76.87
1.
10
11
106.1
63
58.57
57.50
62:42
4318
61.35
06.48
13.55
63.00 03
17.01
41.30
55A
066.2
13
$3.06
58.33
05.33
16.20
05.33
53.34
54.56
93.11
37.53
71.46
55.08
00.99
5433
64:22
V9.55
99.67
98.39
96.85
99.01
9964
96.11
V7.96
98.77
08.57
05.57
06.39
96.37
75.35
1564
Ng
14.23
$2.5
26.13
7000
87.68
01:13
05.51
76.30
$6.10
$7.08
85:38
47.00
01.45
DeseNel21
76.33
61.35
92333
06.15
15
5691
70.41
06.33
45.15
56.76
16.14
70.4
76.13
12.30
73.00
38.51
41.31
KAY
07:23
55.48
05.36
9434
68.77
01.31
94.10
15.04
51.11
03.1
09.93
09.33
99.34
98.32
9992
99.57
99.82
94.11
91.37
98.33
99.00
99.63
98.98
91.98
NI
01:23
83.09
75.08
20.31
90.87
10.55
T2.3
86.35
$8.55
90.
6151
Desoladdn
05.1
0335
81.%
62.37
37.80
0011
70.30
06.11
95.44
19:0
56.97
65.33
29.1
11330
83.10
3534
79.15
0.3
5516
BAP
M601
88 00
6185
1550
05.36
1906
91.88
0359
6379
8355
02.6
01.1
93.96
90.77
9603
98.31
9485
98.34
90.55
98.36
55A
Clers
28
10
16
10
16
12
14
Dettarbation Balact
Pertsbriis Badact
Pectaciaaics Budget
Figure 2: The attack effectiveness of TPA with varying perturbation budgets. The adversarial examples are crafted on ResNet50.
Models. We consider 14 models including ResNet50, DenseNet121, MobileNetV2, EfficientNet, VGG19, 333 WideResNet50., MNASNet, RegNet, ShuffleNetV2, SqueezeNet, ConyNet, ViT, and Swin. with the first twelve being convolutional networks and the final two based on transformer architectures. Moreover, also validate the performance of TPA secured models including adversarial training with L2 BT] and Loc (34] constraint as well as robust training with
Styled ImageNet (SIN) and the mixture of Styled and natural ImageNet (SIN-IN) [16]
MI NI 255 TI [5) VT (36), SSA (27), RAP 30) BSR (35 and Self-Universality 38 Self-Universality a targeted attack and thus we only report its performance
Baselines. Nine state-of- of- the attacks are used as competitors for TPA: BIM 21. DI 44
targeted setting.
Evaluation metric. Attack success rate (ASR, 1) is used as the evaluation metric that is defined as the misclassified rate of adversarial examples by target models.
Hyperparameter configurations. For baselines, we set hyperparameters used in their original papers. For TPA, set À b b 16, k 0.05, N 10. Moreover, for all methods, set iteration of 2d of 16, and step size of 1.6.
5.2 Attack Results
Attack results normal models. Here examine the attack effectiveness of our method on both convolutional neural networks and transformer-like neural networks and Table I reports the attack results. The following observation can be made. Simply speaking, as shown in Table
*Since our approximation involves Hessian matrix that is known to accelerate convergence, we increase the number of iterations to ensure the convergence of all attacks. See Appendix for attack results with iteration of 10. Overall, the increased iterations did not significantly affect the attack performance (less than 1%).
7

---

Table 2: The attack success rates (%) of different methods secured models. Three different robust training methods are considered: adversarial training with L2 perturbation (L.2 {0.03 ~ 5)) [1 and Lo perturbation (AdvIneV3 and EasAdvIncResV [34, robust training with Styled ImageNet (SIN) and the mixture of Styled and natural ImageNet (SIN-IN) [16].
The best results are in bold.
Proxy Model
Method
AdvIneV3
EnshdviancRsV2
SIN
SIN-IN
L2:0.03
L2-0.15
L2-0.1
L2.0..
63.67
64.06
61.30
75:46
96.05
82.33
83.47
74.74
73.05
77.98
43.11
75.13
96.79
83.57
83.70
74.52
73.33
78.11
45.03
75.5
92.04
78.27
78:
75.61
69:57
70:04
71.53
25.74
RoNNr100
50.89
77.27
75.51
77:22
75.05
74.21
72.76
67.04
70.92
70.87
75.61
SSA
30.43
70.31
97.63
61.61
69.06
70.38
71.25
11.02
04
81:77
90.23
75.06
73.48
BSR
64.012
98.
85:57
K3.09
78.31
78.31
76.25
80.1
Ous
72.81
99.52
96.20
96.45
96.45
93.65
91.76
$8.19
33.85
42.
63.47
62.76
58.76
45.15
71
K2.55
82.96
90.13
74.57
75.36
44.74
71 79
83 83
62: 70
83.04
80.08
74.50
75.13
75.28
78
54.97
72. 50
50
77:27
76 86
75.58
69.40 46
70:36
71. 1..56
75
ResNet 52
VT
51.22
43
63.1
65. 10
72:
98.62
75.58
76.07
73.64
71.23
66.61
68.11
69.72
69.06
71.02
1.
75 A5
SSA
51.30
41.94
67 68
$6.05 63
79.34
79:31
76.61
69.29
70.20
71, 20
75
RAP
52.38
43.36
76
85.45 47
61:12
81:31
79.54
75.42
74.95
72. 16
75
BSR
45.5
80.05
58 97
84.95
K1.76
77.57
77.60
m1
80.42
Our
71.66
79.46
99.21
99.80
95.64
95.43
95.15
91.76
90.05
65.36
84.72
41.99
34.0%
46
40 36
63.09
63.11
62.55
60.90
27
MI
59.31
43.03
21.25
79.44
81.71
85:27
50.05
74.64
76.02
75.77
78.65
N]
19.03
45.05
72.07
30.31
61.55
83.14
79,80
74.87
75.09
73.79
78.57
DI
55.00
47.19
75:21
77. 1.19
75.99
27.07
75,99
69.31
70.09
71, 18
75 69
DemseNet121
VI
T1
00.26
49.13
51.73
40.71
62. 24.24
64.22
75. 25.22
67.55
70.25
73.62
76. 13
73.57
71.96
74.21
67.60
68.55
69.57
68:85
71.10
71,05
75
75.54
SSA
61.84
$1.85
69.00 08
34.00
79:64
79:35
76,94
69.67
70.51
71,45
75.77
RAP
62.17
$3.002
75.62
$4.90
61:62
81.10
79.65
75:45
74.78
72,00
75.00 15.64
BSR
64.02
56.3
81 14
88.3 28.25
K5.14
84.69)
K2.24
78.22
77.39
76.15
79.78
Our
72.19
79.64
99.36
99.64
95.31
95.69
95.56
92.65
90.52
$6.63
N5.71
BIM
42,07
34.15
48:93
40.99
62.8%
63.62
62.78
60.94
64.11
68.42
74.13
MI
59.92
45.20
71.35
$1.53
K2.9%
83.19
M0.03
75.54
76.20
75.71
75.60
N1
59.90
44.41
72:27
$2.42
K2.55
83:06
80.15
75.28
75.79
73.66
78.21
D)
56.51
48.70
74 69
50.03
27.12
77.45
76.75
70.05
71.49
71.71
75.82
DemseNer2011
VT
T1
61.76
50.08
41.20
$2.86
62. 88
65.22
69. 34
76.51
74.21
76.66
74.29
76.36
74.59
73.60
67.68
68.67
70.26
69.67
71.07
71.02
75.64
75
SSA
62.30
53.76
69.55
85.05
79.64
79.39
76.76
70.20
70:92
71.38
75.99
RAP
63.17
54.07
75.
85:27
80.76
81 06
80.16
74.74
74.
71. 70
75 75.33
BSR
65.02
56.5
81 -1
88 88.09
85. 20
85. 10
83.06
77.45
77.56
76. 17
80
Ours
70.99
78.21
98.98
99.67
95.41
95.94
94.92
91.56
89.44
85.51
34.57
Table 3: The targeted attack success rates of different methods. The proxy model is ResNet50.
Method
DeaseNet121
EfflcientNet
InceptionV3
ConvNet
WideResNet50
VGG19
VIT
Swin
MI
4,20
0.60
0.00
0.20
5.20
0.80
0.00
0.20
DI
1.60
0.20
0.20
0.00
2.00
0.70
0.00
0.10
VT
0.50
1.10
5.40
1.30
0.40
SSA
5.40
2.30
1.50
0.70
6.60
3.60
0.80
1.20
RAP
8.20
6.70
12.20
7.10
5.80
BSR
19.40
14.00
8.60
7.90
14.50
9.30
7.70
6.50
Self-Universality
22.00
18.00
9.60
8.70
15.70
8.50
9.70
6.70
Ours
31.90
25.20
14.30
12.20
22.60
13.20
13.40
9.80
method defeats all the baselines by a significant margin and the winning attack. For instance, when employing ResNet50 as the proxy model, our attack witnesses an average gain of roughly 9% on attack success rate, even when compared to the state-of-the-arr cansfer-based attack SSA.
Attack results on secured models. We evaluate the effectiveness of TPA on models defended by robust training. We stick employing undefended models proxy models and this is more challenging setting, given that proxy and target models are more divergent. The attack results are reported in Table and we draw the following conclusions. Similar the attack results normal models, TPA still can craft more transferable adversarial examples against robust models. In particular, as can be seen in Table 2
TPA enhances ASRs over SSA by large margin.
Targeted attack results. We also investigate the attack effectiveness of TPA for targeted attacks, which is more challenging attack setting compared to untargeted attack setting. Following the 3 reports the targeted attack results of different attack methods. As can be seen in Table 3 the attack effectiveness of TPA on the targeted setting is so striking, even when compared to Seff Universses,,,,y, which is specifically designed for targeted
evaluation setting for targeted attacks [49] Table
attacks. compared with baselines.
Attack results with varying perturbation budgets. Small perturbation budgets e raise the difficulty in conducting based attacks and we further evaluate the attack effectiveness of different
8

---

Table 4: The scoring for the effectiveness of adversarial examples against real-world applications. We randomly extract 100 samples from ImageNet and generate adversarial examples for them using TPA and ResNet50. We enlist a volunteer assess the consistence between the image contents with the predictions made by applications. A lower rating reflects a higher effectiveness of the attack.
Score (1)
Classification
Object Detection
Google Search
Bing Search
Yandex Search
Baidu Search
GPT-4
Claude5
7
21
10
4
15
12
2
9
4
16
21
17
10
30
30
82
32
methods with varying perturbation budgets. Figure iillustrates the ASRs of VT, SSA, and TPA. In short, TPA still dominates all settings in terms of ASRs.
Evaluation in Real World Applications
This section evaluates the performance of TPA against real applications, which is more challenging but also leads to a more reliable evaluation due to the following three reasons:
Application architecture and complexity. world applications are likely to employ sophisticated models that are difficull to replicate in controlled research environment. Besides, such applications probably incorporate practical defenses.
Training setting. Publicly available models mostly share similar training recipes (ImageNet). However, the recipes of real industry environments far more complicated.
Structure of the output. Real-world systems often output multiple hierarchical labels with associated confidences, instead of logits. Along this, the output space of real applications is vastly larger than that of proxy models, i.e., inconsistency in label space.
Google MLaaS platforms. We attack Google Cloud Vision Application including image classi- fication and object detection, which is recognized as one of the most advanced Al services. We craft 100 adversarial examples against the application. We first collect the application' 's responses to these examples and ask a volunteer to score the consistency between these examples with the corresponding responses [ The scoring ranges from (totally wrong) to 5 (precise), with higher scores indicating weaker attack performance. See the appendix Cor further details about the scoring process. shooss the striking effectiveness of TPA against Google Service, while Appendix [ provides visualization results. The ASRs for image classification and object detection 70% and 80%, respectively, considering scores of or 2 as a successful attack. Besides, we find that ineffective adversarial examples often involve entities of persons, label not covered in ImageNet. As result, TPA is not guided to corrupt the features of "Person", leading to the ineffectiveness of these adversarial examples.
around
Reverse image search engines. Given an image of interest, reverse image search engines enable searching for the most similar ones and creates great convenience and benefits. The task is remarkably different from image classification and object detection and here we test the effectiveness of TPA against search engines. We attack Top-4 search engines suggested by the site, including Google Bing Yandex and Baidu We reuse adversarial ones crafted for Google Service and score their effectiveness ranging from 1 to 5, inversely related to the similarity of retrieved images the original image. Table 4 reports attack results and Appendix D shows the images retrieved by four search engines for original and adversarial ones. Four engines present notable vulnerabilities against adversarial examples by TPA, particularly Baidu Picture Search, which returns completely unrelated images.
Multi-modal AI Platforms. Multimodal Al platforms, which process and integrate various types of data, are inherently more difficult to attack due to their complexity and their ability for contextual integration across multiple modalities. We evaluate the effectiveness of our method on the multimodal Al platforms of OpenAl and Amazon, specifically on GPT-4 and Claude3, respectively. Table reports the numerical results where TPA achieves ASRs as high 56% and 62% for GPT-4 and Claude3 scores of 1 or 2 are deemed as a successful attack. Specifically, Figure provides an
'See Appendix E for more information the recruitment of volunteers and the evaluation procedure.

---

GPT-4 Response: The image appears to be digital painting animated, glitch-like effect applied a photograph a green cabbage or lettuce being chopped with knife. The colors are vivid, and the motion suggests that the knife in the act of slicing through the vegetable.
Claude3 Response: This image shows various plastic hags filled with different types of produce and vegetables. The hags contain items like lettuce or leafy greens, carrots, and what appears to be herbs or greens of some kind. The plastic bags allow the contents to visible from the outside.
Figure 3: Example.
example wherein both GPT-4 and Claude3 fail to accurately classify the green fruit and mistakenly perceive a knife and plastic bag instead.
7 Ending Remark
In this paper, we established bound for adversarial transferability and demonstrated that flat adversarial examples do not inherently enjoy improved transferability. Drawing upon our theoretical insights, we developed TPA, which optimizes both the local effectiveness term and a surrogate the transfer-related loss term to generate adversarial examples. We conducted extensive experiments to validate the effectiveness of TPA across standard benchmarks and multiple real-world applications. hope that theoretical analysis will inspire further research that delves deeper into adversarial transferability, driving forward the development of more sophisticated and effective adversarial attacks and defenses.
Acknowledgments and Disclosure of Funding
This work supported by the National Natural Science Foundation of China under grant number 62202170, the Alibaba Group through the Alibaba Innovation Research Program, and the Open Research Fund of The State Key Laboratory of Blockchain and Data Security, Zhejiang University.
References
[1] Maksym Andriushchenko, Francesco Croce, Maximilian Mueller, Matthias Hein, and Nicolas Flammarion. A modern look at the relationship between sharpness and generalization. In In- ternational Conference on Machine Learning, 2023. URL httpss//// http:://appi.smmaiiccccheeee semanticscholas org hts//CrrpuslD ID: 256846369
[2] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. pages 2206-2216 PMLR, 2020.
[3] Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima generalize for deep nets. In International Conference on Machine Learning, 2017. htttsstps://ap
semanticscholarr wwwpollll.rrrooooooeee Cttr//CrrpsID:::: 7636159
[4] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9185-9193, 2018.
[5] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses transferable adversarial examples by cansllatio-invariann attacks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4307 4316, 2019.
[6] Jiawei Du, Hanshu Yan, Jiashi Feng. Joey Tianyi Zhou, Liangli Zhen, Rick Siow Mong Goh, and Vincent Y. F. Tan. Efficient sharpness- aware minimization for improved training of neural networks. ArXiv, abs/2110.031 2021. URL httpt://appppp.. http://ppp..mmaaiccccchhecoceeee
org/ Org/CorpusIDD 23841943416
[7] Jiawei Du, Daquan Zhou, Jiashi Feng, Vincent Y. F. Tan, and Joey Tianyi Zhou. Sharpness-aware training for free, ArXiv, abs/2205. 14083, 2022. URL F.Rl.htts.: //apii http://p...eaaattccccooorrrrrrr.
CorpusID:249152070
10

---

[8] Mingyuan Fan, Cen Chen, Chengyu Wang, and Jun Huang. Exploiting pre- pre-trained models and low-frequency preference for cost-effective transfer-basee based attack. ACM Transactions on Knowledge Discovery from Data.
[9] Mingyuan Fan, Cen Chen, Chengyu Wang, and Jun Huang. On the trustworthiness landscape of state-of-the-art generative models: A comprehensive survey. arXiv preprint arXiv:2307. 16680, 2023.
[10] Mingyuan Fan, Cen Chen, Chengyu Wang, Wenmeng Zhou, and Jun Huang. On the robustness of split learning against adversarial attacks. In ECA/ 2023, pages 668-675. IOS Press, 2023.
[11] Mingyuan Fan, Cen Chen, Chengyu Wang, Wenmeng Zhou, and Jun Huang. On the robustness of split learning against adversarial attacks. In ECAI, 2023.
[12] Yan Fang. Zhongyuan Wang, Jikang Cheng, Ruoxi Wang, and Chao Liang. Promoting adversar- ial transferability with enhanced loss flatness. 2023 IEEE International Conference on Multi- media and Expo (ICME), pages 1217-1222, 2023. htt.hhtps://appp semanticscholaar
org/CorpusID : 261126990
[13] Roger Fletcher. Practical methods of optimization. 1988.
[14] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. ArXiv, abs/2010.01412, 2020. URL
wrwp.ooooroooosoa://20 222134499,
[15] Zhijin Ge, Fanhua Shang, Hongying Liu, Yuanyuan Liu, and Xiaosen Wang. Boosting ad- versarial transferability by achieving flat local maxima. ArXiv, abs/2306.05225,, 2023. URL
259108222
[16] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet- trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, Boaet/foouummdddyygjooeeeee
2019. UuRLiattss//operrvviie
[17] Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Un- learnable examples: Making personal data unexploitable. ArXiv, abs/2101.0453 004898, 2021. URL sww.ppi...emmaiiiscchhccoooooooo /ttt::://pp..sematticceeee https://appi
wwtp/oorpc///DDDDD 231592390
[18] Jinyuan Jia, Xiaoyu Cao, Binghui Wang, and Neil Zhenqiang Gong. Certified robustness for top-k predictions against adversarial perturbations via randomized smoothing. 2020. URL https://api.. https://api.semaatiichhbbb
org/ wwwooogg//rroeee : 59842968
[19] Xiaojun Jia, Xingxing Wei, Xiaochun Cao, and Hassan Foroosh. Comdefend: An efficient image compression model to defend adversarial examples. 2019 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6077-6085, 2018. URL https
Corg/Corpusllin ID: 54434655
[20] Zhiwei Jia and Hao Su. Information- local minima characterization and regularization. In International Conference on Machine Learning, pages 4773-4783 PMLR, 2020.
[21] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial examples in the physical world. 2016.
[22] Yingwei Li, Song Bai, Yuyin Zhou, Cihang Xie, Zhishuai Zhang, and Alan Loddon Yuille. Learning transferable adversarial examples via ghost networks. ArXiv, abs/1812.03413. 2020.
[23] Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. In International Conference on Machine Learning, pages 20763-20786. PMLR, 2023.
[24] Kaisheng Liang and Bin Xiao. Styless: Boosting the transferability of adversarial examples. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8163 8172, 2023. TRRLhttps::0 //api sewanticcchhlaarr
:ttp:/oorpsII:::::000 258299992
[25] Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv: Learning, 2020.
[26] Zihao Liu, Qi Liu, Tao Liu, Yanzhi Wang, and Wujie Wen. Feature distillation: Dnn- oriented jpeg compression against adversarial examples. 2019 IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 860-868, 2018. URL httpps
//api. semanticcccic org/ arg/oorpusID:386355

---

[27] Yuyang Long, Qi li Zhang, Boheng Zeng, Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency domain model augmentation for adversarial attack. 2022.
[28]
Muzammal Naseer, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, and
Fatih Murat Porikli. A self-supervised approach for adversarial robustness. 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 259-268, 2020. URL
https://api... serareiismataccchooommo .oweaarrrrrgggooee :nrg/CorpusID::2000 2195590220
[29] Yaguan Qian, Yu qun Wang, Bin Wang, Zhaoquan Gu, Yu- Yu-Shuang Guo, and Wassim Swaileh. Hessian-free second-orde -oder adversarial examples for adversarial learning. ArXiv, abs/2207.01396, 2022.
[30] Zeyu Qin, Yanbo Fan, Yi Liu, Li Shen, Yong Zhang, Jue Wang, and Baoyuan Wu. Boosting the transferabiliity of adversarial attacks with reverse adversarial perturbation. In Advances in Neural Information Processing Systems, 2022.
[31] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? NIPS'20, Red Hook, NY, USA, 2020.
Curran Associates Inc. ISBN 9781713829546.
[32] Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing. In Proceedings of the 40th International Conference on Machine Learning, pages 29894-29918, 2023.
[33] Elias M. Stein and Rami Shakarchi. Real analysis: Measure theory, integration, and hilbert spaces. 2005.
[34] Tramèr, D Boneh, Kurakin, Goodfellow, N Papernot, and P McDaniel. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations, ICLR 2018- C018-Conference Track Proceedings, 2018.
[35] Kunyu Wang. Xuanran He, Wenxuan Wang, and Xiaosen Wang. Boosting Adversarial Trans- ferability by Block Shuffle and Rotation. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2024.
[36] Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. 202/ IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1924-1933. 1933, 2021.
[37] Xiaosen Wang, Xu He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. 202/ IEEE/CVF International Conference on Computer Vision (ICCV),
pages 16138-16147, 2021.
[38] Zhipeng Wei, Jingjing Chen, Zuxuan Wu, and Yueping Jiang. Enhancing the unif-universaliity for transferable targeted attacks. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1 12281-12290, 2023. URL https://api: serrrtieemmiiiiicmm cistaticcrccarrrr
org/Corpus ID: : 257663620
[39] Kaiyue Wen, Tengyu Ma, and Zhiyuan Li. Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization. ArXiv, abs/2307.11007 2023. URL.
http:/ppp..enntiicccchchoolla org/CorpusIDD 2599991128
[40] Dongxian Wu, Yisen Wang. Shutao Xia, James Bailey, and Xingjun Ma. Skip connections matter:
On the transferability of adversarial examples generated with resnets. ArXiv, abs/2002.05990, 2020.
[41] Lei Wu and Weijie J. Su. The implicit regularization of dynamical stability in stochastic gradient descent. ArXiv, abs/2305.17490, 2023. URL htttp:://api..mmaniicccht
htg//ooppuID:22899550
[42] Tao Wu, Tie Luo, and Donald C. Wunsch. Gup attack: Transferable adversarial examples via gradient norm penalty. 2023 IEEE International Conference on Image Processing (ICIP), pages 3110-3114, 2023. URL http:::///// http:///aapp..aaiiiieeeeeeeeeee semanticscholar. org/ heg//CorpussD ID: :59501222299080
[43] Cihang Xie, Jianyu Wang. Zhishuai Zhang, Zhou Ren, and Alan Loddon Yuille. Mitigating adversarial effects through randomization. ArXiv, abs/1711.01991, 2017. URL https://api semanticscholar. wwtp:o//ooo.::332555550
[44] Cihang Xie, Zhishuai Zhang, Jianyu Wang, Yuyin Zhou, Zhou Ren, and Alan Loddon Yuille. Im- proving transferability of adversarial examples with input diversity. 2019 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2725-2734, 2019.
12

---

[45] Yinhu Xu, Qi Chu, Haojie Yuan, Zixiang Luo, Bin Liu, and Nenghai Yu. Enhancing adversarial transferability from the perspective imput loss landscape. In International Conference on Image and Graphics, 2023. URL https://app semanticscholar. arrg/orrpsIID
265068861
[46]
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
URL /ttpittps///pprrreriee.... meet/Forummiid-11001111
[47]
Xingxuan Zhang, Renzhe Xu, Han Yu, Hao Zou, and Peng Cui. Gradient norm aware min- imization seeks first-order flatness and improves generalization. 2023 /EEF/CVF Confer- Computer Vision and Pattern Recognition (CVPR), pages 20247-20257, 2023. URL https://api.
semaaniccccctt https://ap..smmaiiccchee
org/ Corpus 257365008
[48] Yang Zhao, Haoxian Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently
improving generalization in deep learning. ArXiv, abs/2202.03599, 2022. URL https
//api.semanticschol.... org. Cwrg/orrpssDD-... 246652190
[49] Zhengyu Zhao, Zhuoran Liu, and Martha Larson. On success and simplicity: A second look transferable targeted attacks. In NeurlPS, 2021.
[50] Yao Zhu, Yuefeng Chen, Xiaodan Li, Kejiang Chen, Yuan He, Xiang Tian, Bo Zheng, Yaowu Chen, and Qingming Huang. Toward understanding and boosting adversarial transferability from a distribution perspective. IEEE Transactions on Image Processing, 31:6487-6501, 2022.
13

---

A Proof of Theorem 3.1
We first demonstrate a bound on our transfer loss term -
By employing Taylor expansion on D(x + 6, 5, = () C(F derive the following expression:
D v) D( + 6. v) (L
(11)
Let us denote the underlying distribution of x as p(x). We employ cross-entropy loss as the loss function. We use [y) and (z)) denote the prediction probabilities of classifying input as y. We consider minimizing
|| (z + 8) y (L + 8). y) I12 over p(z). There is:
/ pla)l((Vlog(((+ 6)
log F(x 5) T00]T8811 81
/ Vloll01@1 8)]] v log F(r + d0ell
(by ||ab|l - ||all [[]l])
(12)
/ [[)] F' 30 log F(x de
2 ( Vog Fr + [y] V log F(x + (ydydr.
The following proof is divided into two main steps. The first step involves deriving a bound on the second integral. The second step focuses deriving bound on J p(x)] V log F' + d)llll2dr. Finally, we combine the results from both steps to obtain the bound for the equation above.
Step I: we handle the second integral of the above expression here: J
F'(z
[y] V log Fxx + [y]dr. We have:
1 p(=))l01l8 F' 5)
V log F(x [yldr
p(x)
Fl + 1812 0) [l] V log F(x + 5yddd
2 / (())]161 VFF [)
log F(x + 5)
(use F' + 5)(y) s 1)
M v log F(x + Hullidd
(13)
[0(e)1011F((=+ log F(x + 0) 0
Ef f (t001111FF(+++))00)))))0 F(x 1 iddr
Σ/
F(x +) i)de (use p(oo) 0)
-- E/
log F(x + d)
Since adversarial examples generated on the proxy model are often also somewhat effective against the target model, this implies that V FF [)
V log F(r + [bl > O. Furthermore, by using F' + [b] 1, we can derive the first inequality in the above equation. [i] and [ refer i-th element in gradient and element in the i-th row and i-th column of Hessian matrix, respectively. From the fourth line the fifth line, we use integration by parts. Then, we suggest p( p(+00) = p(-00) which is natural in the distribution of interest. In the last line, we use F' + 8) S F(x 8y < O. Since is crafted for the proxy model, naturally follows that F' + w) < F(x 0)(y), y). i.e., Assumption 3.4 Moreover, the second derivative of log(.) is negative, and the second derivative of F(x+ [()))))(v) R.
and 2 log F(r + ()
O, which leads to 2 log 6x++))(v)) [v] [i, i)
The second derivative being zero can be derived from Assumption 3.5 which is easily verifiable.
14

---

Step 2: now we derive the bound of p(z) Fx 60bellddd. Notice that here we have:
/ p(x + (IV log F (x + 6) b V log F(a: + (2
(14)
/ Vl
F (x)) | log F(x)
Based on Equation
we obtain:
/ p(æ + +6]| log 0)
/ p(z + 8)1 Vlog + 5) dz
(15)
/ Fo)) log Flx))
For F'
((g 11 by using ||||| ||b|| S |||||
||0||
I|a - -/1,, we have:
/
V log ((x)(v]1B)d
(16)
p(s))lV((ogF ('())v)) log F(x))v)))ddr.
Thus, there is:
/ p(s + +l]le log F'(x d)d
$ / plx + 6)]l 6)ll + d)ll
(17)
/ p(z)))V((ogFF F' log F())lll)dd.
Due to p(x + 8) S p(r), there is:
/ p(x+ + [ F( + d)
$ / p(z))| Vog)]lloog//
log F(x + 12
(18)
+ / [(a))lV((og F" (x) - log F (x))
Furthermore, we let:
C = max p(x)
p(r + 6)
p(x)
(notice Vp(x) 2
max
p(r) + V
B1
(19)
B1 Vp(x:) T5
B
max
B1 By |811
B
B₁ + By 18112
Notice that:
p(x 0) p(a + 8) p(x)
min p(x + +))
B, + Bg B2|10112
2 p(x)
p(x).
(20)
p(x)
p(x)
B1
15

---

For Equation 17
we have:
/ p(a)]lV Fxx ddd
sc / ((t) F(x + +6) 1yll
(21)
c (M))) (=)lui log (())ll))dd
Combining results. Combining Equation 12 and
we have:
/ (l +* 1y6 v log F(x + 5) To
< ple))1l13(1V00ggFF(++
V log F(x dvll))
+ + p(=)11115(( log F(x + b)lll ild
(22)
5 (1+1) [oa)l 6y1ll2d
+ +2
43 |0 d]
+C /
log Fxx (x)
In the second line of Equation 13 we use F(x +5) S 1. Then, according to - -(x
(L + 5). (L + 5), 3) we have
Ep(s)]]D(x 6, 1 < Bp(=)) (xx 1) Ep(e VF +0),)) VL 8), (6,
(23)
By substituting the result from Equation 22 into the above expression, we can obtain the bound presented in Theorem [.1 The bound of L(F (x 5), y) can similarly be obtained using basic norm inequalities, which will not elaborate on here.
The applicability for big 5. In our proof, we apply Taylor expansion which demonstrates effec- tiveness for small changes in the input, i.e., small 5. Our discussion extends the applicability of our theory when faced with sizable values. In fact, the generation adversarial examples is an iterative process. Taylor expansion remains accurate with small update steps. Moreover, our proof mainly relies on p(x) 2 p(z + 6). This implies that as long as we acknowledge that a larger 5 leads decrease in p(x + 0), our theory continues hold. It naturally follows that larger typically correlates with diminished classification accuracy.
B
Supplementary Experiments
B.1 Attack Results on Other Defenses
We here evaluate the attack performance of TPA against other defenses including R&P 43 NIPS-R3
FD [26). ComDefend [19], RS (18), NRP (28). These defense mechanisms deviate significantly from adversarial training as they rely on data processing techniques to purify adversarial examples
into normal ones. The detailed settings follow Long al. (27). repoo the attack performance of VT, SSA, RAP, and our method against these defenses. TPA still consistently surpasses baselines by a large margin.
B.2 Attention Visualization for Targeted Attacks
Figure shows more visualizations of attention maps. As can seen, the adversarial examples produced by VT and SSA only can slightly decrease the attention of the target model to entities of
16

---

Table 5: The attack success rates (%) of attacks against various defenses. We use ResNet50 as the proxy model.
Attack
NIPS-R3
ComDefend
NRP
VT
64.53
70.69
73.4
87.33
52.14
43,97
SSA
72.10
75.32
76.06
91.27
61.86
52.91
RAP
93.15
92,44
92.58
95.59
76.09
76,28
Ours
95.81
97.44
94.02
97.64
85.54
83.47
images, while TPA can significantly distract the attention of the target model from the entities to trivial regions.
(a) Original
(b) VT
(c) SSA
(d) Ours
Figure 4: We conduct targeted attacks and visualize attention maps of the target model to the resultant adversarial images.
B.3 Impact of Hyperparameters
TPA involves four hyperparameters that can significantly impact the attack performance:
À is balance factor between the loss of resultant adversarial examples and the flatness of the region around the adversarial ones. The bigger À attaches more attention to promoting adversarial examples toward flat regions.
suggests how wide the region around adversarial examples desired to be flat.
When comes the implementation TPA, Hessian matrix is approximately evaluated and inappropriate & probably induces non-trivial enrors.
17

---

We extract N samples around adversarial examples to estimate the flatness of the region around the adversarial ones. Generally, the estimation accuracy raises with increasing N.
The impact of A. Figure a illustrates the attack success rates of crafted adversarial examples by TPA with varying À against target models. We see that as increases, the attack effectiveness of TPA presents the tendency to climb initially and decline thereafter. If À is small like À 0.1, most attention of Equation S is put into optimizing L while ignoring the flatness of the region around crafted adversarial examples, i.e., degrading into vanilla transfer-based attacks and causing that the adversarial ones are more likely to be trapped by model-specified sharp regions. As a remedy, properly increasing À can grab part of the attention of Equation to focus the flatness of the surrounding region, so that the resultant adversarial examples have more chance to evade the model-specified sharp regions and more transferable. However, it should be stressed that too bigger induces Equation 5 to only focus on the flatness and de-emphasize whether the region poses a threat to the proxy model, leading to reduction in the attack success rate. Therefore, carefully adjusting À is necessary.
92
90
DenseNet121 EmMleenNet
IneepiionY3
DenseNec121
8S
.66
EmfcleetNet
InceprtioN
80
80
75
0.1
0.5
8
12
14
16
(a) Impact of X
(b) Impact of b
90.0
900
381
1e
85.0
Effcclenteet Inceptiony3s
Inseption
82
80.0
80
0.01
0.03
0,85
0.07
0.00
20
k
(c) Impact of &
(d) Impact of N
Figure 5: The attack effectiveness of TPA with varying X E (0.1,0.5, 1,5, 10). (1, 4,8, 12, 16). h E (0.01, 0.03, 0.05, 0.07, 0.09}, N e (5, 10, 15, 20). The proxy model is ResNe150. We =
The impact of b. We examine the attack effectiveness of TPA by changing the hyperparamter b and 5ig shows the attack results over different proxy-target pairs. Overall, the attack performance of TPA steadily increases with increasing 6. The reason for it is that, a small 6 makes the extracted samples too close to the resultant adversarial examples and causes that the flatness of the region around the resultant adversarial examples cannot be effectively evaluated. Furthermore, increasing b alleviates the issue so as to boost the transferability of produced adversarial examples. Moreover, interestingly, we find that, when employing ResNet50 and EfficientNet, increasing b from 14 to 16 slightly hurts the transferability of produced adversarial examples, and this is probably attributed to the extracted samples being little far from the crafted adversarial examples.
The impact of k. Figure 5(c) shows the influence of the hyperparameter k: to the performance of TPA and we observe that increasing l weakens the attack effectiveness of TPA. In fact, shown in
18

---

Section 4.3
Hessian matrix is approximately replaced by gradient difference, i.e., Equation 10 and the feasibility of the approximation solution depends on a small & to omit the error of approximation induced by Taylor expansion. Hence, it is intuitive why bigger k incurs degradation on the attack success rates of TPA.
The impact of N. Figure d(d) shows the attack performance of TPA with different N. Intuitively, by sampling more instances from the region around the resultant adversarial examples, i.e., increasing N, we can make a more accurate estimation of the flatness of the region, which in turn can decrease estimation errors and then strengthen the transferability of generated adversarial examples. As expected, Figure d) validates this point that increasing N promotes the transferability of produced adversarial examples.
B.4 Attacks with Varying Iteration
We here investigate the impact of iterations attack performance. 6 the performance of different attacks with varying numbers of iterations. can observed that there is negligable change in attack performance when the number of iterations is increased from 10 to 20. The results suggest that the attack methods achieve convergence with 10 iterations.
Table 6: The attack effectiveness of different attacks with varying iterations. We use ResNet50 the proxy model.
Attack
ResNet50
DenseNet121
EfficientNet
VT (10 iter)
100.00
88.76
81.51
VT (20 iter)
100.00
89.62
81.63
SSA (10 iter)
100.00
95.29
90.72
SSA (20 iter)
100.00
95.51
91.56
Ours (10 iter)
99.80
99.08
99.25
Ours (20 iter)
99.85
99.69
99.52
C Score Implication
We begin by gathering predictions from Google MLaaS platforms for both label detection and object detection over crafted adversarial examples. We assign scores to these predictions along five discrete levels, with scores of through 5 indicating the degree accuracy: totally wrong, slightly incorrect, strange but not incorrect, relatively reasonable, and precise. A score of suggests that the images do not contain the objects predicted. A score of indicates minor errors in the predictions, such as identifying flowers instead of trees in tree image. A score of 3 suggests that the main object in the image is not correctly recognized, for example, predicting stones, roads, or tires for a car image. A score of 4 denotes accurately identifying the general type of the main object in the images, while score of 5 indicates that the system can fully and correctly identify the main object in the image. Next, volunteer (See Section P for more information the recruitment of volunteers and the evaluation procedure) asked to rate the consistency between the predictions and the images based on these scores For search engines, similar evaluation procedure is followed, where assess the similarity between the retrieved images and their corresponding original images.
D A visualization of different attacks against Google Service and Search Engines
Figure Souureoshann shows visualization of different attacks against Google Service, and use the format (attack method)-(tasee to denote the attack used to craft adversarial examples and the corresponding test task.
For image classification, consistent with our intuition, the original image is predicted into plant-related categories. However, the returned predictions for the adversarial examples produced via VT and SSA are close to the ground-truth label of the original image. Therefore, the adversarial examples cannot be deemed to be threats against Google Cloud Vision. In contrast, the adversarial examples
19

---

generated by our method indeed mislead Google Cloud Vision, where the predictions with the highest confidence are bird, water, and beak, and these predictions are fairly irrelevant to the original images.
For object detection, we can obtain similar conclusions. The original image is correctly detected. The adversarial examples produced by VT seem fool Google Cloud Vision to some extent, while SSA can fully mislead Google Cloud Vision. Also, our adversarial examples trick Google Cloud Vision with higher confidence than VT.
Figure 7 visualizes an example of TPA against four state-of-the-art search engines. We observe that search engines fetch high-quality and similar images for normal samples. However, when input the generated adversarial examples, the quality of retrieved images noticeably deteriorates, particularly in the case of Baidu.
Rpals
Foweeigg Plari
Dlnating
Pira Pelholooy
Fuai
(a) Original- Classification
(b)
VT-Classification
Temostial Plrnt
Vitte
Adaptat
Deak
Frue
Fowarin Paant
Omes
(c) SSA-Classitication
(d) Ours- Curs-Classffication
(c) Siiinal--OOjett-Dteeeee
(f) VT-Ohject-Detectie
(g) SSA- ASSA-Objet-Detectiie Detection
(h) Ours- Object-Detection
Detection
Figure 6: The attack results on Google Cloud Vision including image classification and object detection tasks. We do not know any knowledge of it and the proxy model is ResNet50.
E
Ethics Statement
This paper designs a novel approach to enhance the transferability of adversarial examples. While this approach is casy-to-implement and seems harmful, it is believed that the benefits of publishing TPA outweigh the potential harms. It is better to expose the blind spots of DNNs as soon as feasible because doing so can alert deployers to be aware of potential threats and greatly encourage AI community to design corresponding defense strategies.
For the human assessment process (Section G we generated adversarial examples and collected the predictions of different applications on these adversarial examples. Throughout the entire process, all communication with the volunteer, including recruitment, was conducted anonymously. Similarly,
20

---

(a) Original-Google
(b) Ours-Google
Original-Binn Bing
(d) Ours-Bing
(c) Original- Yandex
(f) Ours- Yandex
(g) Original- -Bidu
(h) Ours-Bait Baidu
Figure 7: An example for attacking four state-of-the-art search engines.
do not knew the personal information regarding to the volunteer. The volunteer unaware of our specific objectives, ensuring that there was no interest between the volunteer and us. The volunteer came from certain university and possessed normal discernment abilities. During the rating process, the volunteer were unaware of whether a given sample was an adversarial example or the attack method used to generate it. Therefore, there was no bias towards any particular type of attack method from the volunteer. Overall, the evaluation process was relatively fair.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims reflect the contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Our appendix contains ethics statement.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

Justification:
Our appendix contains complete proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide source codes.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification:
We provide source codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuiu wwwiiicgguie//eeeeeeeeeis CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We make detailed experiment settings in our paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We only run one trial.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: Common GPUs capable of running experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp//ppp....pppoo.o.ccccccaaaa
Answer:
[Yes]
Justification:
We follow code of ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We explain this in Introduction.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
25

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not releases any models datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We do this.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whw.ree.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
26

---

Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Our appendix contains this.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There risks in experiment for study participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "Ejii7AlYKm", "reviewer": "Reviewer_x5ig", "summary": "The paper proposes a theoretical investigation into the relationship between the flatness of adversarial examples and their transferability. The authors challenge the prevailing belief that flatter adversarial examples necessarily have better transferability. They introduce a new method called Theoretically Provable Attack (TPA), which optimizes a surrogate of the derived transferability bound, enabling the generation of more transferable adversarial examples. The paper includes extensive experiments demonstrating the effectiveness of TPA on various benchmarks and real-world applications.", "questions": "1. In transfer attacks, the target model ($\\hat{F}$ here) is black-box and unknown, while the statement in line 160 of the paper that \"a proxy model needs only yield predictions for x that are closely aligned with those of the target model\" is not feasible in actual attack scenarios.\n\n2. Furthermore, in Equation (3), the derived upper bound is only related to the target model $\\hat{F}$ in the first term, which is confusing. It is noted that the derivation of the second-order gradient component comes from Equation (12). My question is why the gradient of the target model $ \\hat{F}(x + \\delta)$ disappears from the fourth to the fifth line in Equation (12). Is this due to the application of the integration by parts formula? \n\n3. Please provide a performance comparison of the TPA method with existing methods that only constrain the gradient norm [11, 41]. Is the core difference that TPA uses uniformly distributed noise? \n\n If the authors can satisfactorily address the above questions, I can consider raising my score.", "limitations": "see the questions.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "This paper addresses a highly worthy research topic. The theoretical understanding of adversarial transferability is still under-explored. The experimental results on benchmarks are also very impressive. The authors claim that merely constraining the gradient norm at the adversarial examples is not sufficient to enhance model transferability; it is also necessary to consider second-order gradient information.", "weaknesses": "The theoretical analysis in the paper establishes an upper bound that appears to be quite loose (due to Taylor approximations and inequality relaxations). As a result, it lacks sufficient insights into how this upper bound inspired the design of the TPA method presented in Equation (4) of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1:** The tightness of our bound.\n\n**Response:**\nWe would like to provide some clarifications regarding our bound.\nFirst, as pointed out by Reviewer QVii, the first and second terms in Eq.3 should be squared.\nThe revised bound in Theorem 1 is: $$\\mathbb{E} \\{ ||D(x+\\delta,y)||_2^2 \\leq \\mathbb{E}  ||D(x,y)||_2^2  +  C \\mathbb{E} ||\\delta||_2^2 ||\\nabla D(x,y)||_2^2 \\}  + (1+C) \\mathbb{E} \\{ ||\\delta||_2^2 || \\nabla \\log F(x+\\delta) ||_2^2 \\} + 2  \\{ \\sum \\mathbb{E} ||\\delta||_2^2 |\\nabla^2 \\log F(x+\\delta)[i,i]| \\}.$$\nWe denote the terms on the right-hand side of the inequality as $Q$.\nMoreover, in our response to Reviewer QVii, we detail the lower bound for the squared loss of the generated adversarial examples on the target model: $$\\mathbb{E} ||L(F'(x+\\delta),y)||_2^2 \\geq \\mathbb{E} || L(F(x+\\delta),y) ||_2^2 - \\mathbb{E} || D(x+\\delta,y) ||_2^2 \\geq  \\mathbb{E} || L(F(x+\\delta),y) ||_2^2-C.$$\n\nWe here conduct an empirical evaluation to examine the effectiveness of our bound.\nWe craft 1000 adversarial examples using ResNet50 against DenseNet121.\nOur estimates show that the sum of squared losses for the examples on the proxy and target models is approximately 280.90 and 76.37, with $\\mathbb{E}  ||D(x+\\delta,y)||_2^2$ of 69.27.\nWe calculate the value of $Q$ to be 170.75, setting $C$ to 1 due to $C \\leq 1$.\nThe difference between 170.75 and 69.27 is somewhat non-trivial.\nHowever, when we translate this difference into probabilities, it becomes quite minor.\nSpecifically, the loss for the target model on the generated adversarial examples should approximate $10.50$ ($\\mathbb{E} || L(F(x+\\delta),y) ||_2^2-C=280.90-170.75=10.50^2$).\nThis implies that the probability of the target model correctly classifying the examples is at most $e^{-10.50} \\approx 2.75 \\times 10^{-5}$, which is a rather small number.\nIn summary, our bound is indeed tight and practical, due to the exponential mechanism and the typically high loss of the generated adversarial examples in the target model.\nWe also evaluate several other models, as detailed in the table below.\n\n| Target Model | $\\mathbb{E} \\Vert L(F(x+\\delta),y) \\Vert_2^2-C$ | Probability Bound |\n|:------------:|:---------:|:--:|\n| EfficientNet |   182.03   |  4.81 $\\times 10^{-5}$ |\n| InceptionV3  |   189.53   |  7.05 $\\times 10^{-5}$ |\n| MobileNetV3  |  161.91   |  1.83 $\\times 10^{-5}$ |\n| ViT          |   215.56  |  0.0003 |\n\n\n---\n\n**Question 2:** The target model is black-box and unknown.\n\n**Response:**\nWhat we commonly refer to as a black-box scenario actually permits some queries to the target model but restricts access to its architecture and parameters.\nThis is common across various AI applications, e.g., Google's AI services, which allow users to input data and receive predictions.\nMoreover, a truly inaccessible target model would negate the possibility of feeding adversarial examples into it, making black-box attacks trivial.\nTherefore, limited access is practical and reflects actual attack scenarios.\n\n---\n\n**Question 3:** Why does the gradient of the target model $F'(x+\\delta)$ disappear from the fourth to the fifth line in Equation (12).\n\n**Response:**\nThe probability of the target model correctly predicting $x+\\delta$ should be higher than that of the proxy model, that is, $F'(x+\\delta) \\geq F(x+\\delta)$.\nConsequently, we have $-F'(x+\\delta) \\leq -F(x+\\delta).$\nAdditionally, considering the second derivative of $log(x)$, which is $-\\frac{1}{x^2} \\leq 0$ we have:\n$$- \\sum \\int p(x)||\\delta||_2^2 F'(x+\\delta) \\nabla^2 \\log F(x+\\delta)[i,i] dx \\geq - \\sum \\int p(x)||\\delta||_2^2 F(x+\\delta) \\nabla^2 \\log F(x+\\delta)[i,i] dx.$$\nWe have added a clear explanation of this step to improve the manuscript's readability.\n\n---\n\n\n**Question 4:** Comparison of the TPA method with [11, 41].\n\n**Response:**\nWe employ ResNet50 as the proxy model to craft adversarial examples for 1000 natural samples, using our attack and the attacks proposed in [11,41].\nThe results, summarized in the table below, demonstrate that our attack significantly outperforms those described in [11,41].\nNotice that the attacks presented in [11] and [41] are almost identical in terms of their objective functions and optimization methods, resulting in nearly identical attack success rates.\n\n| Target Model | [11] | [41] | Ours |\n|:------------:|:----:|:----:|:----:|\n| EfficientNet | 81.6 | 81.2 | 99.7 |\n|     VGG19    | 86.3 | 86.5 | 98.5 |\n|    ConvNet   | 65.8 | 65.5 | 94.6 |\n|      ViT     | 51.1 | 50.7 | 93.8 |\n\nFormally, the key difference between our attack and those in [11, 41] lies in the use of uniformly distributed noise.\nDespite its simplicity, this additional random noise plays a unique and fundamentally important role in enhancing performance, as our theoretical analysis illustrates.\nThis is also the primary reason why our method achieves notably higher success rates compared to [11, 41].\n\n---\n\n[11] Boosting adversarial transferability by achieving flat local maxima\n\n[41] Gnp attack: Transferable adversarial examples via gradient norm penalty", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yIfv6fq9MT", "reviewer": "Reviewer_QVii", "summary": "This paper focuses on the transferability of adversarial examples. The authors first derive an upper bound for the transferability loss used in the paper. Then, they propose a new loss function based on the derived bound to increase the adversarial transferability. The proposed TPA method is tested in both classic models and real applications.", "questions": "See the weakness part.", "limitations": "The authors did not discuss the limitations.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. This paper is well-organized.\n2. The proposed method is tested in real-world applications.", "weaknesses": "- The theoretical claims lack a strong and important assumption. Theorem 3.1 is based on a strong assumption that the source model and the target model have the same loss on all inputs ($L(F'(x),y)-L(F(x),y)\\approx 0$), which is used in Line 482 in Appendix A. However, this strong and important assumption is not explicitly stated in Theorem 3.1. Moreover, this assumption is not verified and may be not practical in different models.\n- Theorem 3.1 cannot well reflect the bound of the adversarial transferability of inputs. First, there seems to be a typo w.r.t. the definition of $D(x+\\delta,y)$ in Line 125. It is supposed to be $D(x+\\delta,y)=\\Vert L(F'(x+\\delta),y) - L(F(x+\\delta),y) \\Vert$. Second, according to the proof in Appendix A, the term on the left side of the inequality should be $\\Vert D(x+\\delta,y)\\Vert_2^2$. Please check the claim and the proof. Third, Theorem 3.1 only provides a bound for the \"transfer-related loss term\", but the adversarial transferability of adversarial examples also depends on the local effectiveness term. Thus, it is unclear whether the loss function based on this bound conflicts with the local adversarial loss.\n- The attack success rates in experiments are not reported with the error bar or the standard deviation.\n- It is unknown in Section 6 how the used 100 adversarial examples are crafted, e.g., which proxy model is used, which dataset is used, and how 100 examples are selected.\n- The evaluation in Section 6 is conducted by only one volunteer, which is unreliable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1\\&2:** The theoretical claims lack a strong and important assumption. \\& Theorem 3.1 cannot well reflect the bound of the adversarial transferability of inputs.\n\n**Response:**\nFor first issue, we do not assume $L(F'(x),y)-L(F(x),y) \\approx 0$ (See the derivation below).\n\nFor second issue, it is a typo and it should be $D(x+\\delta,y)= L(F'(x+\\delta),y) - L(F(x+\\delta),y)$.\nAs you mentioned, the left side of our bound (Theorem 1) should be in squared terms.\n\nFinally, based on revised Theorem 3.1, we can derive the bound of adversarial transferability.\nLet us briefly restate the revised proof to address your questions.\n\nBased on $\\mathcal{L}(F'(x),y)-\\mathcal{L}(F(x),y) = L(F'(x+\\delta),y) - \\nabla L(F'(x+\\delta),y)^\\top \\delta- L(F(x+\\delta),y)+\\nabla L(F(x+\\delta),y)^\\top \\delta$, we have $ D(x+\\delta,y) =  D(x ,y) + \\nabla L(F'(x+\\delta),y)^\\top \\delta - \\nabla L(F(x+\\delta),y)^\\top \\delta$.\n\nTaking the $L_2$-norm on both sides and then taking expectiation, we obtain $\\int p(x) ||D(x+\\delta,y)||_2^2dx\\leq \\int p(x) ||D(x,y)||_2^2dx+\\int p(x)|| \\nabla L(F'(x+\\delta),y)^\\top \\delta-\\nabla L(F(x+\\delta),y)^\\top \\delta||_2^2dx$.\n\nThe main result in Appendix A is Equation 21, which does not use $L(F'(x),y)-L(F(x),y) \\approx 0$. Therefore, we have\n$$\\int p(x) || \\nabla L(F'(x+\\delta),y)^\\top \\delta-\\nabla L(F(x+\\delta),y)^\\top \\delta||_2^2 dx \\leq (1+C) \\int p(x) ||\\delta||_2^2 || \\nabla \\log F(x+\\delta) ||_2^2  dx + 2 \\sum \\int p(x) ||\\delta||_2^2 |\\nabla^2 \\log F(x+\\delta)[i,i]| dx + C \\int  p(x) ||\\delta||_2^2 ||\\nabla (\\log F'(x) - \\log F(x))||_2^2 dx.$$\n\nCombining the above equations, we get our revised bound (revised Theorem 1):\n$$\\mathbb{E} ||D(x+\\delta,y)||_2^2 \\leq \\mathbb{E}\\{ ||D(x,y)||_2^2 + C ||\\delta||_2^2 ||\\nabla D(x,y)||_2^2 \\}  + (1+C) \\mathbb{E} \\{ ||\\delta||_2^2 || \\nabla \\log F(x+\\delta) ||_2^2 \\}+2\\mathbb{E} \\{ \\sum \\mathbb{E} ||\\delta||_2^2 |\\nabla^2 \\log F(x+\\delta)[i,i]| \\}.$$\n\nNow, let us consider the bound for adversarial transferability.\nSpecifically, based on $L(F'(x+\\delta),y) = D(x+\\delta,y)+L(F(x+\\delta),y)$ and taking the $L_2$-norm on both sides and applying basic norm properties, we get: $$ ||L(F'(x+\\delta),y)||_2^2 \\geq | || L(F(x+\\delta),y) ||_2^2 - || D(x+\\delta,y) ||_2^2 |. $$\n\nSince $L \\geq 0$ and higher loss of $x+\\delta$ on the proxy model than the target model, there is $\\mathbb{E} ||L(F'(x+\\delta),y)||_2^2 \\geq \\mathbb{E} || L(F(x+\\delta),y) ||_2^2 - \\mathbb{E} || D(x+\\delta,y) ||_2^2.$\nThe bound of $\\mathbb{E} || D(x+\\delta,y) ||_2^2$ is already provided. Thus we can obtain the bound of $\\mathbb{E} ||L(F'(x+\\delta),y)||_2^2$ (See Response for Reviewer #x5ig's Q1).\nThe loss of adversarial examples on the proxy model is our \"local effectiveness term\". \nThis inspires design of our attack, as shown in Equation 4 in the original manuscript, where the first term maximizes the local effectiveness term and the second term minimizes the bound about transfer-related loss term.\n\n---\n\n**Question 3:** The attack success rates in experiments are not reported with the error bar or the standard deviation.\n\n**Response:**\nWe have conducted experiments to calculate the error bars and standard deviations and included them in revised manuscript.\nBelow are some results, where we use ResNet50 as the proxy model and run attacks in five trials to report (ASR $\\pm$ standard deviation).\nOur method not only achieves ASRs but also enjoys smaller deviations.\n\n| Attack |   DenseNet121  |  EfficientNet  |   InceptionV3  |     ConvNet    |       ViT      |\n|:------:|:--------------:|:--------------:|:--------------:|:--------------:|:--------------:|\n|   RAP  | 95.05$\\pm$0.41 | 95.15$\\pm$0.38 | 93.76$\\pm$0.56 | 90.62$\\pm$0.37 | 62.54$\\pm$0.37 |\n|   BSR  | 96.98$\\pm$0.36 | 95.07$\\pm$0.41 | 93.39$\\pm$0.33 | 88.27$\\pm$0.21 | 82.14$\\pm$0.36 |\n|  Ours  | 99.68$\\pm$0.08 | 99.55$\\pm$0.04 | 98.74$\\pm$0.11 | 94.37$\\pm$0.07 | 93.54$\\pm$0.08 |\n\n---\n\n**Question 4:** It is unknown in Section 6 how the used 100 adversarial examples are crafted, e.g., which proxy model is used, which dataset is used, and how 100 examples are selected.\n\n**Response:**\nWe randomly select 100 samples from the benchmark evaluation dataset ImageNet [1].\nUsing these samples, we generate 100 adversarial examples with our attack method, employing the default hyperparameters and ResNet50 as the proxy model.\nWe have added these details in the revised manuscript.\n\n---\n\n**Question 5:** The evaluation in Section 6 is conducted by only one volunteer, which is unreliable.\n\n**Response:**\nWe have recruited two additional volunteers to conduct evaluation.\nThe table below reports the average scores and variance from all three volunteers (the original evaluator + two additional volunteers).\nThe results show a high degree of consistency provided by the three volunteers, reinforcing the superior performance of our method.\n\n\n| Score | Classification | Object Detection |  Google Search |   Bing Search  |  Yandex Search |  Baidu Search  |      GPT-4     |     Claude3    |\n|:-----:|:--------------:|:----------------:|:--------------:|:--------------:|:--------------:|:--------------:|:--------------:|:--------------:|\n|   5   |     2$\\pm$1    |   2.33$\\pm$0.58  |     0$\\pm$0    |     0$\\pm$0    |     0$\\pm$0    |     0$\\pm$0    |  1.67$\\pm$0.58 |  0.33$\\pm$0.58 |\n|   4   |     7$\\pm$2    |  21.67$\\pm$0.58  | 10.67$\\pm$1.15 |  6.33$\\pm$0.58 |  6.33$\\pm$1.53 |  4.33$\\pm$0.58 | 13.33$\\pm$1.53 | 11.33$\\pm$0.58 |\n|   3   | 13.33$\\pm$0.58 |   7.67$\\pm$0.58  |    17$\\pm$1    |    11$\\pm$1    | 12.33$\\pm$1.15 |     5$\\pm$1    | 28.33$\\pm$1.15 |    26$\\pm$1    |\n|   2   |     9$\\pm$1    |   5.33$\\pm$2.31  |    17$\\pm$1    | 20.67$\\pm$0.58 | 16.67$\\pm$1.53 | 10.33$\\pm$0.58 | 29.67$\\pm$0.58 |   31$\\pm$2.65  |\n|   1   | 68.67$\\pm$1.15 |    63$\\pm$2.65   | 55.33$\\pm$0.58 |    62$\\pm$1    | 64.67$\\pm$1.53 | 80.33$\\pm$1.53 |    27$\\pm$1    | 31.33$\\pm$1.15 |\n\n---\n\n[1] On success and simplicity: A second look at transferable targeted attacks", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pe66Gc6lAm", "reviewer": "Reviewer_dRSh", "summary": "The paper investigates the relationship between the transferability of adversarial examples and the flatness of adversarial examples. The paper shows that flatness alone is not sufficient to guarantee transferability. Based on this theoretical result, it derives an optimization method for adversarial examples that improves transferability. This proposed method is evaluated empirically and compared to a wide range of baselines.", "questions": "- Eq. 17: why is $p(x+\\delta)\\leq p(x)$?\n- The usage of D in the statement of Thm. 3.1 makes a mapping of terms in the proof to the result unnecessarily cumbersome.\n- How does the proposed method compare to PGN [1] which improves transferability through flatness?\n- The computational problem of measuring flatness, i.e., the second order gradient components, can be alleviated by considering relative flatness [2], which has also been applied to adversarial examples [3]. Could this be used as an alternative or means to improve the proposed method?\n\n[1] Ge, Zhijin, et al. \"Boosting adversarial transferability by achieving flat local maxima.\" Advances in Neural Information Processing Systems 36 (2023): 70141-70161.\n[2] Petzka, Henning, et al. \"Relative flatness and generalization.\" Advances in neural information processing systems 34 (2021): 18420-18432.\n[3] Walter, Nils Philipp, et al. \"The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective.\" arXiv preprint arXiv:2405.16918 (2024).", "limitations": "The paper clearly states assumptions and limitations throughout the manuscript. It would, however, be beneficial to justify the theoretical assumptions and discuss the resulting limitations (e.g., the assumption on smoothness of the target distribution, or that probabilities go to zero for $x\\rightarrow\\pm\\infty$).", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- sound theoretical motivation and result\n- the assumptions are reasonable and clearly stated\n- comprehensive empirical evaluation\n- evaluation on real-world applications\n- interesting insights on the link between flatness and transferability of adversarial examples", "weaknesses": "- the paper does not discuss how its insights can be used to improve defense mechanisms\n- The presentation of the the proof of Thm. 3.1 could be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:** The presentation of the the proof of Thm. 3.1 could be improved & The usage of D in the statement of Thm. 3.1 makes a mapping of terms in the proof to the result unnecessarily cumbersome.\n\n**Response:**\nIn response to your suggestion, we have revised the proof and replaced $D$ to enhance clarity and readability.\n\n---\n\n**Q2:** why is $p(x+\\delta) \\leq p(x)$?\n\n**Response:**\nNatural samples are drawn from the real-world distribution that the model was designed to handle.\nIn contrast, adversarial examples are not a direct product of this distribution; they are artificially crafted through specific techniques that exploit the model’s vulnerabilities.\nTherefore, their occurrence in real-world scenarios, where data is not typically manipulated in such a adversarial manner, is far less common, i.e., $p(x+\\delta) \\leq p(x)$.\n\n---\n\n**Q3:** How does the proposed method compare to PGN [1] which improves transferability through flatness?\n\n**Response:**\nWe employ ResNet50 as the proxy model to craft adversarial examples for 1000 natural samples, using our attack and PGN.\nThe results, summarized in the table below, demonstrate the superior performance of our attack over PGN.\nFor instance, our attack achieves an impressive ASR of 99.7% on EfficientNet, significantly surpassing the 81.6% achieved by PGN.\n\n| Target Model | PGN [1] | Ours |\n|:------------:|:----:|:----:|\n| EfficientNet | 81.6 | 99.7 |\n|     VGG19    | 86.3  | 98.5 |\n|    ConvNet   | 65.8 | 94.6 |\n|      ViT     | 51.1 | 93.8 |\n\n---\n\n\nQ4: The computational problem of measuring flatness, i.e., the second order gradient components, can be alleviated by considering relative flatness [2], which has also been applied to adversarial examples [3]. Could this be used as an alternative or means to improve the proposed method?\n\n**Response:**\nWe have carefully read [2,3] and found them to be quite enlightening.\nIndeed, relative flatness shares a significant similarity to the second-order gradient component in our bound.\nNonetheless, calculating relative flatness presents a considerable challenge, as shown in [3], which only addresses the relative flatness concerning the penultimate layer.\nSimilarly, directly penalizing the relative flatness of adversarial examples poses substantial computational difficulty.\nWe value the implications of these inspiring works [2,3] and will include them in the revised manuscript.\nWe also consider exploring relative flatness as a promising avenue for future research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to express our sincere appreciation for the efforts and feedback from all reviewers. We have taken into account reviewers' comments and suggestions, which have greatly enriched the quality of this manuscript.

As noted by some reviewers, there are minor errors and ambiguities in our proof. We have fixed these issues to enhance the manuscript's readability.  **Importantly, these revisions do not affect the core insights and contributions of this manuscript.**

This analysis is grounded in practical assumptions and unveils a nuanced relationship between the transferability of adversarial examples and their flatness. We believe that this insight will provide the community with a deeper understanding of transferability, paving the way for future research.

Moreover, we have incorporated reviewers' constructive suggestions regarding experiments and other relevant interesting literature.

Overall, the reviewers' expertise and constructive feedback have significantly enhanced the clarity and depth of this manuscript. We believe that the revised manuscript now presents a more convincing and compelling study, and we once again extend our heartfelt gratitude to the reviewers for their efforts and feedback.

Meta-review: After the rebuttal and discussions, reviewers all agree that the paper has made good contribution to the theoretical and practical understanding of adversarial transferability. AC would recommend acceptance for this paper, conditioned on the case that the following comments will be addressed by the authors. Note that some of comments are technical and need to be faced seriously for the correctness/soundness of the paper.

1. The assumptions of $p(x+\delta)\le p(x)$, $F'(x+\delta)>F(x+\delta)$ and $1\ge F'(x+\delta)$ and the justification behind them should be clearly stated.

2. The proof of Theorem 3.1 needs a revision.

3. Some typos should be fixed.

Please take these comments into consideration in the revised version of the paper.
Decision: Accept (poster)