Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability
Authors: Junqi Gao
Biqing Qi
Yao Li
Zhichang Guo
Dong Li
Yuming Xing
Dazhi Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: Adversarial Attacks; Generative Attack; Transferable Targeted Attack

Abstract: The transferability of adversarial perturbations provides an effective shortcut for black-box attacks. Targeted perturbations have greater practicality but are more difficult to transfer between models. In this paper, we experimentally and theoretically demonstrated that neural networks trained on the same dataset have more consistent performance in High-Sample-Density-Regions (HSDR) of each class instead of low sample density regions. Therefore, in the target setting, adding perturbations towards HSDR of the target class is more effective in improving transferability. However, density estimation is challenging in high-dimensional scenarios. Further theoretical and experimental verification demonstrates that easy samples with low loss are more likely to be located in HSDR. Perturbations towards such easy samples in the target class can avoid density estimation for HSDR location. Based on the above facts, we verified that adding perturbations to easy samples in the target class improves targeted adversarial transferability of existing attack methods. A generative targeted attack strategy named Easy Sample Matching Attack (ESMA) is proposed, which has a higher success rate for targeted attacks and outperforms the SOTA generative method. Moreover, ESMA requires only $5\%$ of the storage space and much less computation time comparing to the current SOTA, as ESMA attacks all classes with only one model instead of seperate models for each class. Our code is available at https://github.com/gjq100/ESMA

Perturbation Towards Easy Samples Improves
Targeted Adversarial Transferability
Junqi Gao".. Biqing 23 Yao Lit.1, Zhichang Guo!, Dong Li Yuming Xing', Dazhi Zhang'
Sccooo of Mathematics, Harbin Institute of Technology Department of Control Science and Engineering, Harbin Institute of Technology 1 Department of Electronic Engineering, Tsinghua University "Frontis.Al
Equal contribution. t Corresponding author. [gjunqi97, qibiging7, mathgzc, arvinlee826) @ @ggmail.co yyaoli5508 xyuming, zhangdazhi @hit.edu.cn
Abstract
The transferability of adversarial perturbations provides an effective shortcut for black-box attacks. Targeted perturbations have greater practicality but are more dif- ficult to transfer between models. In this paper, we experimentally and theoretically demonstrated that neural networks trained on the same dataset have more consistent performance in low sample density regions. Therefore, the target setting, adding perturbations towards HSDR of the target class is more effective in improving transferability. However, density estimation is challenging in sigh--imenssioall scenarios. Further theoretical and experimental verification demonstrates that easy samples with low loss are more likely to be located HSDR. Perturbations towards such easy sam- ples in the target class avoid density estimation for HSDR location. Based the above facts, we verified that adding perturbations to easy samples in the target class improves targeted adversarial transferability of existing attack methods. A generative targeted attack strategy named Easy Sample Matching Attack (ESMA) is proposed, which has higher success rate for targeted attacks and outperforms the SOTA generative method. Moreover, ESMA requires only 5% of the storage space and much less computation time comparing the current SOTA, as ESMA attacks all classes with only one model instead of seperate models for each class. Our code is available at hhttps: //github. com gjq100/SM110
(HSDR) of each class instead of
Introduction
Deep learning models exhibits substantial computational capacity in many downstream tasks, but are vulnerable adversarial attacks 2 Such attacks tend be transferable B as well as real-world achievable [5], which makes it implementable in black-box scenarios. Targeted attacks known to be more difficult to transfer [4 6 compared with non-targeted attacks.
Intuitively, directions and transferability of adversarial attacks are closely related, but their relationship rarely discussed. 77 found that samples in low density regions of ground truth distribution are more susceptible to adversarial attacks. They also verified that adversarial perturbations that aligned with the low density direction of ground truth distribution can lead better transferability. For targeted attack scenarios, the direction that can bring more transferability has not been fully researched yet is not clear.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Findings and arguments. In non-targeted scenarios, directing towards low-density regions of the ground-truth truth distribution can improve adversarial transferability 77. However, this ap- proach is not direct enough for targeted attacks. As shown in Figure I perturbations point- ing at Sangs--appe-e--ni-----ee
(HSDR) of the target domain are more effective than that pointing at low-density regions of the ground-truth distribution, it perturbates samples more di- rectly to the target discrimination region. Moreover, we demonstrate and theoretically prove that neural networks tend to have more consistent outputs in the HSDR of each class, which means that perturbations pointing to the HSDR of the target class are also transferable between different models. However, density estimation for samples in high dimension is challenging. Fortunately, by the definitions of hard samples and easy samples in [8] we found fact that helps, in each cate- gory, easy samples with smaller losses are more likely to locate in HSDR. We provide theoretical and experimental assurance for this fact. Based on this fact, we can directly perturbate towards such easy samples of the target domain to improve transferability without density estimation.
Related works. Adversarial attacks can be divided into white-box attacks and black-box attacks. In the white-box setting, attackers have access to the model's 's structure and parameters, while in the black-box setting, they have no such information but only access the input and output of the model. This is typically the scenario encountered in real-world situations. Black-box attacks include query- based attacks (9] 10. IT] and transfer-based attacks [12] 13 14. Conducting too many queries impractical in real-world applications. In contrast, transfer-based attacks are more feasible as they do not require queries. Transfer- based attacks often require the use of white-box surrogate model to produce adversarial perturbations. In terms of the way adversarial perturbations are generated, there are two approaches: iterative instance-specific methods and generative methods. Iterative instance- specific methods utilize model gradients to iteratively add perturbations to specified samples (e.g.. FGSM [3] C&W [5], PGD [5]). To enhance the transferability of adversarial samples, subsequent work combines these methods with various techniques, such introducing momentum [12 [3] and considering input transformations [3 [4 16
Towarde low denutry seglon ground tuth disenbutinn
Figure 1: A schematic example of our motivation, plotting the probability den- sity (darker the color represents larger the density) and samples for three pop- ulations (orange, cyan, and green). The black line indicates the Bayesian dis- criminant boundary.
17 during iterations, training auxiliary classifiers 18 19,, or substituting different loss functions 20 21 22). However, instance-specin -ppecfffc methods are primarily designed for non-targeted scenarios, often lacking effectiveness in targeted settings. Although relatively good results have been achieved for targeted attacks [20, 21 (33, instance-specific methods still require iteratively creating perturbations for each specified sample, while generators trained for generating adversarial perturbations can be generalized on more samples after training
24 25 26 27
The leading perturbation generative method currently is TTP 224 which employs target-specific GAN to align clean and augmented data from the source domain with the target domain data. However, TTP necessitate training a dedicated generator for each class. This significantly increases storage requirements and training time. In our work, we design a generator for simultancous attacks on all target classes, which uses class embedding information for each target class. To construct these embeddings, we adopt techniques similar to SNE (28) to align the surrogate model's output logits for target classes with the generator's 's embeddings. This enables latent features learned by the surrogate model to guide the construction of well-structured class embeddings. Building on our findings, train multi-class perturbation generator simultaneously perturb the source domain samples towards easy samples with low loss of each target class. Experiments on the ImageNet dataset shown that our method can obtain a better target transfer success rate than TTP, while requiring much less storage, Our method also has certain advantages in training time.
Therefore, our contribution can be summarized as follows:
We found that deep learning models have more consistent outputs in HSDR of each class, as demonstrated theoretically and through experiments (Section 2.1 This implies that adding

---

adversarial perturbations pointing to the HSDR of the target class results in better targeted adversarial transferability.
We experimentally and theoretically verified that easy samples with low loss in early- stopping models are likely to be located in HSDR, which allows us to directly add per- turbations pointing to such samples of the target class to improve targeted adversarial transferability (Section 2.2]. This avoids density estimation of high-dimensional samples, which is challenging and computationally expensive.
We introduced the Easy Sample Matching Attack (ESMA) (Section 3). ESMA achieves a higher targeted transfer success rate compared to SOTA generative attacks TTP. Furthermore, it only needs one model to perform attacks for all target classes (Section D which requires much less storage space than TTP (only about 1/20 of the storage space in 10 targets case), and requires less training time.
Our findings and conclusions can not only provide guidance for target transfer attacks, but more importantly, they reveal the consistency of deep learning models in the HSDR. and the correlation between sample difficulty and local sample density. Not only that, the design of our multi-target perturbation generative model can provide new reference for subsequent related research.
2
Main conclusions
In this section, we combine illustrative experiments and theoretical proof's to illustrate our two conclusions in turn, and we use the following notations and definitions. Z X' x y Rd x I is the sample space. The i.i.d. dataset S {2i, 3e } consists of n sample pairs (c), S I n. Given y. the conditional distribution of is Dajy- Feature mapping f : X + RK where K is the number of class, the class of feature mapping f E F. Specif- ically, the parametrized class Fw := {fw X - EK E W]. where W is the parameter space. Define F = { So sw sw Fw, So fm(( = ((fm(())) S denotes Softmax- transformation. The Softmax-Cross- Cross-Entropy loss is denoted as (soa(;))) : T x - RR Let Sj ((x, E : j}. I, = {i: (I., E S.).C, (ar E X: (I, E =
Definition (Gj zo. r)..r)-Local sample density) Given a class j E [K], (Fo, yo) E Sy, the (j. r) Local sample density:
EiET, 1(x; E B))),T))
P(i.za.))
volB(sa, r)
where 3(1o,r)= ERd: || xooll Sr), volB(1o, r) denotes the volume of B(xo.r). r).
Now use the notation I I(jj)a))) {i: (24, 31) Sy, E B(au, r), (Io. yo) S,)}, C(j.m..)
{x x E Cy vi E B( r), (ro. yo) E Sj)}. For simplicity, we denote Exce (w (x), as ((w,
and define the local empirical risk R(j (w) [aaao.nn EiELg l(w,I;)
2.1 The Output Consistency of Different Deep Learning Models in HSDR
We conduct an experiment to explain why samples in low-density regions of ground-truth distribution are vulnerable to attacks, and verify the consistency of the output of the deep learning model in HSDR. A dataset consist of 200 samples is constructed by sampling from two 2-d Gaussian distributions with equal probability, which is then used to train a neural network for classification. Subsequently, we plot the discriminant region of the Bayes' criterion (which has the minimum error rate) for the known ground truth prior distribution and compared it with the discriminant region of the trained classifier. In addition, train two other neural networks with different structures and parameter quantities, and plot the output differences of the three neural networks (Figure 2 The discriminant region in Figure (ag can reach the Bayesian error rate. The intersection of the two population distributions in the middle of the probability density curve (as shown in Figure 2 belongs to the len-deysity region of the ground-truth distribution and also to the misclassified region. In such a region, even with minimal expected error, the Bayesian discriminant criterion will classify samples with relatively small probability density of single population into another class. These samples can be thought of as "outliers". The trained NN classifier discriminates samples into their original categories, causing
3

---

CTp
4000
(a)
(b)
(C)
(d)
Figure 2: (a): Bayesian discriminant region, darker the color indicate higher the confidence probability. (b): Classifier discriminant region, the probability density curves the two population distributions plotted, the white part represents the low-density region of ground truth joint distribution, and we boxed out the small pits in the Bayesian misclassified region. (c): Classifier discriminant region with samples. We boxed an outlier. (d): Output differences between three different classifiers, darker purple indicates greater difference output between different classifiers
difference in decision boundaries from the Bayesian prior classifier and creating small pits as shown in Figure 2(a) Most of the samples around the pit (Figure (Figure[(c)) belong to another class due to its lower population density compared to the other class. Hence, perturbing the samples from this pit (i.e. outliers) towards the discriminant region of the other class becomes easier. If other trained neural networks can accurately classify such outliers, they will also generate similar pits. By adding perturbations pointing these pits samples from another class, they will be perturbed into such pits, enabling the transfer of adversarial samples between different classifiers, which explained the findings of [7].
However, as we stated in Figure 1_ perturbations towards the low-density regions of the ground-truft distribution are not direct enough for targeted attacks, while perturbations towards the HSDR of the target class are more direct. Combining this with Figure [(d) we observe that different classifiers have more consistent outputs in the HSDR. With Theorem we theoretically verified this, which also
implies that perturbing samples to the HSDR the target class leads to better targeted adversarial transferability.
Theorem (Local output consistency) For target class j E [K]. and two different parametrized class Fwr = {fwi wi E Wi}. Fwz == {fwz w2 E W2), assume that
(( EiEIu (f5 - SSa (z:)) S n, then for any sample (ro, yo) E Sy. in the neigh-
borhood B(xo, r), with probability at least 1 - 6, the following holds:
ELEDD. v [1 (x) - f (x)
Cij,zo.r)ll.
(N
Kdd/2
dd/2 log (2K/6)
< 00
(2 2d log² (1aa VP(j.ooo)) + +1
+ 7.
P(j.wa.)) P(j. 1d+
Remark. Theorem suggests that different models have a more consistent output near the samples in HSDR, specifically, the difference between the outputs has bound of order Ô (v Kd4//2
P().=o.r) 24,z
When the number of classes and dimensions are fixed, larger sample density results in more consistent performance, weaker consistency within smaller neighborhoods. Note that this local consistency is weakened as the dimension increases, but the relative output consistency between HSDR and LSDR does not change.
However, a very practical problem that sample points in high-dimensional space tend to be very discrete due to the curse of dimentionality 229), which makes it difficult find a suitable local neighborhood size to calculate the local density of samples, moreover, in the case of large datasets, calculating local density can be computationally expensive. But the conclusion of the next section can help us find the sample located in the HSDR using carly-stopping models without directly calculating the local density of the sample.
2.2 Correlation between Local Sample Density and Sample Difficulty
In this section we illustrate the correlation between sample density and sample difficulty. Hard samples have attracted much attention in various tasks because of their significance for training

---

g proposed that for the convergent model, the difficulty of the sample can be measured by the loss gradient norm of the sample, easy samples has relatively small loss gradient norm, while the loss gradient norm of difficult samples is relatively large, especially, those with too large gradient norm may be outliers.
convergence and model generalization 8 33 31 32
We use the following theoretical analysis to illustrate that for a trained classifier, the samples in the HSDR tend to have a smaller local empirical risk. At the same time, smaller loss gradient norms ensure that losses are more consistent in small neighborhoods, which implies those samples with both smaller loss and loss gradient norms guarantee less local empirical risk in the neighborhood where the sample is located. To illustrate the following conclusions, we make several mild assumptions:
Assumption (Smoothness assumption) E(w, x)) satisfies the following conditions of Lipschitz continuous gradien:
V (w - Vul(wy, x| S 41 [w - w22l Vwn W2 E W, [V Val(w, x2)1l S L2 ||l x2ll V21, X.
Assumption / x)|| S for all w W.
Assumption (Polyak-Lojasievicc Condition) R(j. (w) satisfies the PL-condition:
1
V Ru (w))) 2 a (Rizo.))) Ris,nor))
2
where R(joo)))))
1 [swaore
ΣiEt a(w,x:) is the local empirical risk, R (j.ro.r)
inf [. (w). WEW
Assumption 4 For vi E Iijaory the following holds:
V w R Riaan)) (w), V (wl 23)) 2 B V R ()
Assumption and were made in 33
34 and 35 where Assumption 2 can actually be derived
from Assumption I when the input space is bounded, which is usually satisfied. Even a non-convex function can still satisfy Assumption 3 [4], the inequality in Assumption B iiplics that all stationary
points are global minimum [36], which is proved in recently works [37, 38 for over-parameterized
DNNs. Assumption is reasonable when local neighborhood radius r is small.
Algorithm Mini-batch SGD
Require: Initialized weights w total steps T,, sample set S, batch size M and step size Ne- 1: for t = 1 + T' do
2: Randomly sample M different samples B from S, where batch size [ = M, corresponding indicator set denote Igt.
3:
w²+1 = =w²
1
ΣiEI,, V Vwl(wt (lwl(ww))))
M5
4: end for
Return: "''1
Consider the local empirical risk under Algorithm I we use Theorem 2 to illustrate the convergence rate relies on local density.
Theorem (Optimization relies on local density) Given a learnable parametrized class F [fw X' + R. w E WW. let w' updated by Algorithm under Assumption 23 and Me But and T L1 then with probability at least 1 5. holds
4 set
23μ'
(00 Rig I (1-27 (P()soor)) (Rij.orr) R Ris.oo))
(12)
12
where (Pis.0or)) max {
P///2 T($+1)M
}. which a non-descending funciiio
Im(T/8) 2M
of P(j.ao.))
5

---

According to theorem 2 the local empirical risk R(j-zo.r)) (w) will reach a more faster convergence rate in HSDR and, relativlly, a relatively slower convergence rate in LSDR, thus for early-stopping models, they has a lower local empirical risk in HSDR. Combined with the following Proposition I we show that smaller gradient norm guarantees a smaller local empirical risk. For overfitting models, the local empirical risk of each neighborhood where samples are located may be very small, but for carly-stopping models, this relativity of local empirical risk with respect to local sample density can be maintained.
Proposition 1 Under Assumption given any (14) Ve) E for any X that satisfies || x0| S the following holds:
(ew, x;) l (w,x))
3
l (w, v.) $ (V, x,,,
3L2r
+ 2
r
r
Remark. Proposition Iasiiiionnsseee Suggests that minimizing the loss leads to a smaller norm of the loss gradient. However, this constraint becomes less tight as the neighborhood radius decreases. Compared with the loss itself, the loss gradient norm further guarantees the proximity of local loss, especially when r is small. Therefore, samples with smaller loss and smaller loss gradient norm are more likely to be in a neighborhood with smaller local empirical risk.
Using the above conclusion, for an early-stopping model, samples with smaller losses and loss gradient norms tend to be located in HSDR. Still in the example the previous section, we train three models with early-stopping, and the learning rate adjusted with the number of steps, and then plotted Figure 3 The model has more consistent outputs in HSDR. When the loss and gradient norms of sample are small, the region where the sample is located has smaller local empirical risk. Samples in HSDR have smaller local empirical risk, which is consistent with our theoretical analysis. Additionally. the rightmost graph of Figure B] shows that samples with smaller loss and gradient norms often locate in HSDR, validating our conclusions.
Therefore, we can determine whether sample is more likely to located in HSDR or LSDR by evaluating whether it has smaller loss and gradient norms simultaneously. This eliminates the need to calculate local sample densities to find samples in HSDR.
03
0.1
01
06-1
Dunely
Lin-Gaddon:
Lock DoniD
Figure 3: The first figure depicts the difference in output of three models under different local sample densities P(y....)) divided into different bins. The second figure shows the local empirical risk R.y.,,.)) of samples under different sum of loss and gradient norms (Loss+Gradnorm) For Loss+Gradnorm, first normalize both variables separately and then add them up to eliminate magnitude differences. The third figure represents the local empirical risk of local sample densities in different values. The fourth figure displays the local density under different Gradnorsss The neighborhood radius r is taken as 0.4.
We conduct transfer attack experiments on three baselines on the CIFARIO dataset, three different models are chosen as victims. Combining our perspectives above, we use algorithm 2 to select the anchor of each target class for guiding the addition of adversarial perturbations. We implement our strategy by simply using squared loss to match anchor points, i.e. minimizing |] (a:do) Gtarett || (choosing 9 = 10 and с 16 pixels). For comparison, we also use cross-entropy (CE) loss to calculate adversarial examples the vanilla way, as well as using squared error without screening mechanism (i.e., randomly selecting the target anchor in the target class) to exclude the influence of different losses. The results are shown Table T our strategy indeed helps to enhance the transferability of target attacks, which further confirms our viewpoints.

---

Algorithm 2 Target Anchor Screening
Require: Early-stopping classifier fw. screening parameter and sample set S.
1: for = 1 <- do
2: loss; &ace (f Yi), gradnorm; (f y-lll-
3: end for
4: For each class [K], select the g-th smallest loss and the gradient norm among the samples in that class as thresholds thr and thrs sradnomm
5: for k 1 f- K do
6:
Ak { i E I& loss; < thr loss gradnorm, thrg
aradnorm
OK TA Eser, (w)())):
del
7: end for
Table 1: Targeted transfer success rates. "left/middllerightt" represent attacks using regular CE loss, square loss with randomly selected target anchors, and square loss with target anchor screening, respectively. In parentheses, the clean accuracy is indicated.
Sec:Res3495.44%
Src:VGG10942279)
Atack
VGG16
DDnsee121
RRe34
DDensel21 121
-Res24
VCG16
MIM 14.308/12444/144444 19 10000/000000000000000000000 14 905001//1.
13.419//22
1991 17 139//444
177 11 21t.2/111222///// 17%/1101.77111115 2.5%%
13.59%/12550//133
23%6/14 50%/15..1 12.49%
(12.88% 11.145/
10.1%%
(1140%
15.74%/14 8/15 11 949.99711199999222222 09%
15.4605/14.39/5559999
10% 17.69918898 29.509/13322%/1512222
12.979/22.1 1991347% 18.113916.889//888844 18.18 12.
3 Training Strategy of ESMA
We present our training strategy in this section, shown in Figure 4 Our training strategy is carried out two steps, the first step we pre-train the generator's embedding representations, and the second step is find easy samples of each target class based previous conclusions, and then use these samples to guide the generator to generate perturbations from the source domain to the target domain.
Pre-trained Embeddings Guided by Latent Features To obtain better embeddings that more effectively rep- resents inter-class information, since the latent space of deep learning mod- els often extracts enough class infor- mation [39]. we treat the output fea- tures of the local model as a of a priori embedding. In order to make the generator embedding have sim- ilar structure to such priori embed- ding, we refer the idea of manifold learning, using strategy similar to the SNE algorithm 28). Let μú =
Target Domnin
Samples
Sargles
Css
Genarator
Sample
fitsase 15
W x ((2,)s
Generator
Pertarlate projection and Cip
Seane redie
Leu
Source Domain
E,, Eiez, 4, where 4 f(;). and then we design the following manifold matching loss. Generator embeddings of various class were pulled the manifold that output features in to obtain embeddings with better structure.
Figure 4: Training strategy of ESMA.
Next, we denote the generator embedding of class j as Ej. The four matrices S M Eaue M Scon E satisfy M.SSue || Mill, M.E.ue ||e: eill M respectively. Let
40P4 M. M.Euss
er
TAilll,
Te, 11le,
Se Suc exp (M M
Free
exp( C
ΣR exp(1s
S Seze
M
i.j
R exp (M Pesc
exp( M
M
Froe
exp( (M
ΣK exp (M
Ek_ expl Bo0
then manifold matching loss is as follows:

---

LM Eis Maj
log
M. + Eis Mis
Peu
L log
Mi
+\1 Eis
Soon Mij
log
Mi M
Eis M. log M...
Becn Mid
A2
The last regular term is prevent losses from collapsing. Àι and A2 are hyperparameters. The pre- trained embedding with our strategy has a larger Euclidean distance and a smaller cosine similarity, which greatly alleviates the previous clustering phenomenon. More detailed analysis and discussion are provided in Appendix B
Training of Multi-target Adversarial Perturbation Generators After pre-training embedding, we freeze the parameters of the embedding layer, combined with our previous conclusions, we select several easy samples in each class form target anchor sets A,, and match them with the output of our generator in feature space, the generator we use is a Unet with Resblocks. We propose the following objective easy sample feature matching loss to train the multi-class adversarial generator.
LEM = Σ J=1
Σ MMl ( Ij)
Σ IZZ
als (az) (clip, (W Go (sa)))),
where (z) clip (min (x 1 E, max (x, e))), W is a differentiable Gaussian kernel with size 3 3, such smoothing operations have been demonstrated to further improve transferability [24). The measure of the distance between the two features d(.,.) is Smooth L.1 loss, which has a unique optimal solution that is not sensitive to exceptional values [40] Then, our final training strategy can be represented by algorithm
Algorithm 3 Training Strategy of ESMA
Require: Generator Gg with praite embeddings, anchors ag, ls E [K] and Total epochs N.
1: for t 1 <- N do
2: for
n do 3: target, Uniform((1,, K})-
4: 5/ * target, then
5: Random choice an anchor a, from Aj.
6: Gradient descent step on LEM.
7: end if
8: end for
9:
end for
4 Experiments
In this section, we verify the effectiveness of our method through experiments ILSVRC2012 dataset 41). To evaluate the effectiveness of different components in our strategy, we conduct ablation experiments in Section 4.3 Other ablation experiments can be found in Appendix
4.1 Experiment Setup
Dataset The dataset used comes from the ILSVRC2012 dataset [41] in which we selected classes as the training set, which refer to the ten classes used for TTP training in (24), they are 24, 99, 198, 245, 344, 471, 661, 701, 802, 919, We train generators using images of these classes in the training set (1300 images per class), and use the images in the validation set (50 images per class) as the validation dataset the targeted attack.
Models We use the four networks used in 24 as source models ResNet-50 42
(Res50), VGG-
(Res152). Except the above four models, we also select three models from the Inception series: Inception-v3 (5) (Inc-v3), Inception-v4 [46] (Inc-v4), Inception-ResNet-v2 (46) (IncRes-v2) and transformer vision model, VIT 47 As the analysis and fundamental assumptions of this paper are based the condition of training data from the same distribution, in order to explore the transferability between models trained on training data with different distribution, conduct additional transfer attack on two
19-bn 43 (VGG19bn), DenseNet- 121
44 Densel21), ResNet-152 42

---

udversarially trained models, Inc-v3-adv 48 and IncRes-v2-ens [49. Results are shown in E.2 In addition, we also test the adversarial transferability of ESMA on the scenario where the source model is an ensemble of different models, corresponded results are reported in E.
Baselines We select many iterative instance-specific attack benchmarks, MIM 12). SI-NIM B
TIM [4), DIM [16]. and advanced iterative instance-specifie attacks that are competitive in target
setting, Po-TI-Trip (0)), Logit [2I], Rap-LS [23). FGS (50), DMTI-Logit-SU (51]. 1-1-SI-TI-
DIM [52]. Generative adversarial attacks HGN [53] and TTP [24] also included in our comparision,
where TTP is the current SOTA generative method. As generative attack, TTP requires training
class-dependent generator specifically for each class.
Parameter Setting For the parameter settings of different attack methods, we refer to the default settings in [20]. total iteration number T = 20, step size a = </T. where the fo perturbation for the stochastic imput diversity in DIM,
restriction is set 16. Momentum factors μ is set set the probability of applying input diversity as 0.7. For TIM, the kernel-ennthh is set to 7, which is more suitable for targeted attacks. For Po- Po-TI-Trip. the weight of triplet loss À is set to 0.01, while the margin y is set to 0.007. Referring to [21], the number of iteration steps of Logit is set to 300, and for RAP-LS, we choose 400 iteration steps, Kr.s is set 100, and en is for TTP, since we used a relatively small training set, we added 10 epochs to the original paper [24] settings ensure the performance of the model, and the learning rate of Adam optimizer is
12/255 [23]. Then
(3, = 5,, 32 = .999). Finally, for our model, we used the Adam W optimizer to train 300 epochs with learning rate of 1e- 4 (350 for cases where the source model is VGG19bn or Densel21), the value of used for sample screening is set to 2.
Evaluation Setting We train the generator the training set of the selected ten classes, verify the targeted transferality on the validation set, for each target class, we use the 450 images of the remaining classes the source data and perturb them to the target class. For the iterative instance- specific attacks, we directly attack these instances and test their targeted transfer success rate. All methods (including training) were implemented single NVIDIA RTX A5000 GPU.
4.2 Results
The results of our experiments, reported in Table 3 ESMA outperforms current SOTA gen- erative attack TTP in targeted transfer success rates. Our approach demonstrates significant advantages in terms of efficiency and effectiveness, with TTP having a parameter count of 7.84M per model compared to ESMA's 4.40M. Additionally, ESMA achieves average train- ing time that is 78, 78.4% of TTP. The experimental results also strongly supports our viewpoint.
Buttbe
4.3 Ablation Studies
Doee
Tomer Sirae
To validate the effectiveness of the two components in our strategy design, we compared the cases with (w/) and without (w/o) pre- trained embeddings and target anchor screening. The results are shown in Figure In the without pre-trained embeddings, the performance is significantly weaker than the case with pre-trained embeddings across different training durations. Moreover, the case with target anchor screening shows a noticeable improvement com- pared randomly selecting target anchors.
Figure 5: Comparison of targeted attack transfer suc- cess rates with (w) pre- trained embeddings and with- out (w/o) pre-trained embed- dings at different training epochs. Sre:Res50.
To verify that ESMA can indeed increase the sample density of the the samples used for
target class for the original samples, adversarial testing in table First, we calculate the sample density of the target class for the clean samples P(target,s...)) (here is set to 600). Then, we apply ESMA to generate adversarial samples a:dd and calculate the sample density of the target class for these adversarial samples p(arret,, .ry. In both cases, the density calculation results were averaged across all target classes for each sample. We normalize the results by dividing them by the maximum value among all results in both cases. Additionally, we further bin and count the number of test samples in different intervals under different perturbation constraints. The results are shown in Table 2 The

---

target class sample density of the samples perturbated using ESMA has increased in bins with larger values, which confirms that ESMA can indeed enhance the target class sample density.
Table 2: Comparison of sample density (binned).
|Counts[0-0.] 0.1-0.2 0.2-0.3 0.3-0.4
0.5-0.6 0.6-0.7
0.7-0.8
0.8-0.9 0.9-1.0
Clean 32
35
31
45
39
51
57
51
61
48
Case for 8/265:
|ESMA]
Clean 32
35
31
45
39
51
57
51
61
48
Case for
16/256
JESMAI 25
Table Targeted transfer success rates. "Src" indicates the source model. For TTP and ESMA, the numbers in parentheses indicate training duration in seconds.
VGGL9ta
-Inocv3
-tlnc-vi
LlccRes-222
MIM
L3331
3,4618
3,6716
0.51%
0.31%
0.221
0.16%
1.38%
DIM
13.3015
26,9515
23.8016
3.11%
3.80%
1.2015
0.47%
10 10.09%
SENIM
1,07%
1,91%
Res50
32.2016
5.09%
3.38%
14.52%
SI-PGS-M
20.75%
32,685
2.925
25.4915
36.3096
36.8915
148816
12675
10,0246
1.1%
10 721
DTMI-Logit SU
33 33.655
78.9155
79 000%
18.56%
3.9
8.718
3.33%
36 16/
RAP-ES
60.045
83.4016
50.3896
22.106
17 17.445
10.829
3.00%
TTP195339 7)
111
73,675
72 BAR
33 09
35.2%%
275
13.05%
07%
ESMAR13060.1
$1.29%
83.8996
S1.7896
37,5316
39.29%
17.291
21.25%
$1.76%
Attack
VGG1199
Kec50
Mes152
-IInc-v4
IncRes-v2
TIM
8.98%
2.8255
8.87%
3.485
5.51%
1.915
0.11%
73%
DIM
10.209
13.901
9.18%
3.085
1.2758
0.60%
19%
Po-TI- Tri
11.29%
16.805
11.78%
3.49%
4.54%
2.965
0.96%%
7.7%%
SI-POS
16.67%
S'L-IITI--IM
12.04%
11,9855
13.89'5
31.71%
12.0115
1.1675
3.27%
19%
DIMI- Logit SIU
33.38%
$5.93
34.1616
13. 5555
12 365
1.75%
3.62%
21 185
30.67%
15.18%
9.655
3882
HGN
TTP (09611.6)
21.00%
58 022
6915
23.00%
13.24%
17.11%
7023
29.28'
Sre
Attack
Res50
Denseel 121
-RRs152
Inp-v3
-Inc-04
-TacResv22
VVT
AVG
MIM TIM
0.20%
102
3,8055
4.58%
188
1.47%
109%
035615
0.107
1.93%
DIM
2.98%
0.338
1.1
100%
116%
0.37%
0.18%
1711%
VGG19bn
Po TL Top
4,5615
6.27%
2.42%
1.69%
1.47%
0.877
0.27%
2.51%
SI-PGS MM
4.12%
5.71%
1.97%
1.56%
1.25%
0.685
0.25%
22/2%
S'I-SI-TI-DIM
Logue
22.1815
30.4796
11.51%
3.51%
6.7156
1.911
0.98%
11 12%
DIML-LLgii Liggi-- SUU
23.47%
31.47%
12.6015
0.13%
7.51%
2,0771
1.11%
02%
HGN
34 185
31 11.015
0.366
11.16%
8.57%
T.715
1.76%
15 51%
TIPL131319.21
36.7015
34,4450
22.111
11.826
10.40%
2.2018
2.5%%
ESMAIT10209221
11.73%
Src
Attack
VGG19bn
Rox50
Ditse 121
-Tn--44
IncRes-v2
VTT
AVG
MTM TIM
1.11%
5.695
1.70%
0.69%
0.31%
0.18%
0.18%
644%
766
76.50 64
21.566
1.60%
4.23%
1.04
107%
7875
SE-IIM
0.825
1655
166
0.365
0.37%
0.344
0.11%
695
Res152
PS Top
14.33%
33.3016
29,81%
8.53%
6.25%
4.711%
1.36%
14.44%
SI-PCIS
13.8%
39,229
24.0%
7.98%
5.11
4.315%
0.95%
13.99 91.91
S S'I-SI-TI-DIL
19.295
36.649
30.78%
14.36%
11.42%
10.845
2975
30/3
Loget
15 13.84%
15.22%
61.31%
15.33%
10 10.60%
8.115
3. 766
20 541
27.708
17,47%
RAP-ES
18,9296
14.33%
3
TINN
61.405
73.314
67.80%
33.474
26 26.77%
11.31%
[1 1196%
4
TTP 13323
63.315
70.736
36.73%
13.6455
5.62%
79.939
Conclusion
In this work, we provided novel perspective the transferability of targeted adversarial attacks. The study theoretically and experimentally demonstrated that adding perturbations towards HSDR of the target class can further enhance the transferability of targeted attacks. We also proposed method for identifying samples within HSDR, which avoided the impracticality of density estimation in nigh-dimensional data. Building upon these insights, introduced improved generative multi-target attack strategy ESMA, which surpassed previous generative targeted attacks in both effectiveness and efficiency. We believe that our insights not only provide guidance for targeted attacks but also offer insights for dataset selection. Further discussion on these points would be included in our future work.
10

---

References
[1] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
[2] Biqing Qi, Bowen Zhou, Weinan Zhang, Jianxing Liu, and Ligang Wu. Improving robustness of intent detection under adversarial attacks: A geometric constraint perspective. IEEE transactions on neural networks and learning systems, PP, 2023.
[3] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015.
[4] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In ICLR. OpenReview.net, 2017.
[5] Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEEE S&P, 2017.
[6] Nathan Inkawhich, Wei Wen, Hai (Helen) Li, and Yiran Chen. Feature space perturbations yield more transferable adversarial examples. In CVPR, 2019.
[7] Yao Zhu, Jiacheng Sun, and Zhenguo Li. Rethinking adversarial transferability from data distribution perspective. In ICLR, 2022.
[8] Buyu Li, Liu, and Xiaogang Wang. Gradient harmonized single- stage detector. AAAI, 2019.
[9] Nicolas Papemot, Patrick McDaniel, Ian Goodfellow, Somesh Jha. Berkay Celik, and Ananthram Swami. Practical black-bor box attacks against machine learning. In ACM, 2017.
[10] Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black- box adversarial attacks with limited queries and information. In ICML, 2018.
[11] Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput., 23(5):828 841, 2019.
[12] Yinpeng Dong. Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018
[13] Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2020.
[14] Yinpeng Dong, Tianyu Pang. Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by
attacks. In CVPR, 2019.
[15] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep lcarning models resistant to adversarial attacks. In ICLR, 2018.
[16] Junhua Zou, Zhisong Pan, Junyang Qiu, Xin Liu, Ting Rui, and Wei Li. Improving the transferability of adversarial examples with diverdd--eeess-uuuut eiverse and region fitting, In ECCV. 2020.
[17] Junyoung Byun, Seungju Cho, Myung-Joon Kwon, Heeseon Kim, and Changick Kim. Improving the transferability of targeted adversarial examples through object-based diverse input. In CVPR, 2022.
[18] Nathan Inkawhich, Kevin J. Liang, Lawrence Carin, and Yiran Chen. Transferable perturbations of deep feature distributions. In ICLR, 2020.
[19] Nathan Inkawhich, Kevin J. Liang, Binghui Wang, Matthew Inkawhich. Lawrence Carin, and Yiran Chen. Perturbing across the feature hierarchy to improve standard and striet blackbox attack transferability. In NewrIPS, 2020.
[20] Maosen Li, Cheng Deng, Tengjiao Li, Junchi Yan, Xinbo Gao, and Heng Huang. Towards transferable targeted attack. In CVPR, 2020.
[21] Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. On success and simplicity: second look at transferable targeted attacks. In NewrIPS, 2021.
[22] Chaoning Zhang, Philipp Benz, Adil Karjauv, Lae-Won Cho. Kang Zhang, and In So Kweon. Investigating top-k white-box and transferable black-box attack. In CVPR, 2022.
11

---

[23] Zeyu Qin, Yanbo Fan, Yi Liu, Li Shen. Yong Zhang, Jue Wang, and Baoyuan Wu, Boosting the transfer- ability of adversarial attacks with reverse adversarial perturbation. In NewrlPS, 2022.
[24] Muzammal Naseer, Salman H. Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. On generating transferable targeted perturbations. In ICCV, 2021.
[25] Omid Poursaced, Isay Katsman, Bicheng Gao, and Serge J. Belongic. Generative adversarial perturbations. In CVPR, 2018.
[26] Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R. Venkatesh Babu. NAG: network for adversary generation. In CVPR, 2018.
[27] Muzammal Naseer, Salman H. Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. In NeurlPS, 2019.
[28] Geoffrey E. Hinton and Sam T. Roweis. Stochastic neighbor embedding, In Suzanna Becker, Sebastian Thrun, and Klaus Obermayer, editors, NewrlPS, 2002.
[29] Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. 2009.
[30] Abhinav Shrivastava, Abhinav Gupta, and Ross B. Girshick. Training region- based object detectors with online hard example mining. In CVPR, 2016.
[31] Xiaolong Wang, Abhinav Shrivastava, and Abhinav Gupta. A-fast-renn: Hard positive generation via adversary for object detection. In CVPR, 2017.
[32] Tsung- Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár, Focal loss for dense object detection. In JCCV. 2017.
[33] Aman Sinha, Hongseok Namkoong, and John C. Duchi. Certifying some distributional robustness with principled adversarial training. In ICLR, 2018.
[34] Maher Nouiched, Maziar Sanjabi, Tianjian Huang, Jason D. Lee, and Meisam Razaviyayn. Solving class of non-convex min-max games using itcrative first order methods. In NeurlPS, 2019.
[35] Yisen Wang. Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICMI,, 2019.
[36] Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the polyak-tojasiewicz condition. ECMI/PKDD, 2016.
[37] Simon S. Du, Jason D. Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. In ICML, 2019.
[38] Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over- parameterization In ICML, 2019.
[39] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.
[40] Ross B. Girshick. Fast R-CNN. In ICCV, 2015.
[41] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, pages 248 255, 2009.
[42] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016
[43] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. In ICI.R, 2015.
[44] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In CVPR, 2017.
[45] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.
[46] Christian Szegedy. Sergey loffe. Vincent Vanhoucke, and Alexander A. Alemi. Inception-vé v4, inception- resnet and the impact of residual connections on learning. In AAAI, 2017.
12

---

[47] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. ICLR. 2021.
[48] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian J. Goodfellow, Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
[49] Alexey Kurakin, lan Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang. Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang. Zhishuai Zhang. Zhou Ren, Alan L. Yuille, Sangxia Huang, Yao Zhao. Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, and Motoki Abe. Adversarial attacks and defences competition. CoRR, abs/1804.00097, 2018.
[50] Lianti Gao, Qilong Zhang, Xiaosu Zhu, Jingkuan Song, and Heng Tao Shen. Staircasc sign method for boosting adversarial attacks. CoRR, abs/2104.09722, 2021.
[51] Zhipeng Wei, Jingjing Chen, Zuxuan Wu, and Yu-Gang Jiang. Enhancing the self-universality for transferable targeted attacks. In CVPR, 2023.
[52] Yuyang Long. Qilong Zhang. Boheng Zeng. Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency domain model augmentation adversarial attack. In ECCV, 2022.
[53] Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Boosting transferability of targeted adversarial examples via hicrarchical generative networks. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassmer, editors, ECCV, 2022.
[54] Dylan J. Foster and Alexander Rakhlin. Lo vector contraction for rademacher complexity. CoRR. abs/1911 1.06468,,, 2019.
[55] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. MIT Press, 2012.
[56] Shai Shwlar-S and Shai Ben-David, Understanding Machine Learning From Theory to Algorithms. Cambridge University Press, 2014.
[57] Yunwen Lei, Ürün Dogan, Ding Xuan Zhou, and Marius Kloft. Data-dependent generalization bounds for multi-class classification. IEEE Trans. Inf. Theory, 65(5).2995 3021, 2019.
[58] Yong Liu. Refined learning bounds for kernel and approximate k-means. Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6- 14, 2021, virtual, 2021.
[59] Wassily Hoeffding. Probability incqualities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-33 1963.
13

---

Supplementary Material:
Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability
A Proofs
A.1 Proof of Theorem
First, we need the definition below:
Definition 2 ((i, k. To, r--Lolal Output Rademacher Complexity) Let f = (Sw...., 10 FS. == {f wo E W). given B( = {x || xoll I r). (zo. vo) E Sj. then the empirical (j, k zo, -local outpu Rademacher Complexity is defined by:
...., 5
R. (FF)) = Bd n
os Sx (a:i)
sup fur E
where o,e I( ii.d random variables satisfies IP (a, = =o; Correspondingly, the expected (i, To, rl-lccal output Rademacher complexity is de- fined as R(i
P(o;=-
Es
Ra
((i (FF))
) a, C(Arar)) where S(iza.))
((x,)) Sj : B( r), (zo, vo) E S).
We first prove the lemma and use to prove theorem and we need lemma and before proving the lemma 1.
Lemma Given B(xs, r). (Zo, Mo) E Sj. S(i and f E FF., then with probability at least
1 - 8, the following holds for any f E J and E [K]:
EznDalz [f x E C(iiro.n) Es(a.oni))
[fSt]]
K
log(2/0)
S 40
log?
[
1ow
2 [((
where K is the number of classes, O and 6" are constants.
Lemma (loc Contraction Inequality (54) Let S ( RRK) and let Φ RK lR be L-Lipschitz with respect to the 1s norm, iee. (16(0) - [I') I L. o'll Vu, o' RRK For
any > 0, there exists a constant C >> 0 such that if (o(f(x)) [I I 5, then
F C LVK max Rrn (Fi) +o
Çn max; Rr (F:)
where R2 ( Eo /su Ei 0r (f (x,))],, R2 (Fi) = supsex" R2 (F,)
Lemma Given B(Io,r), (2o, Ho) E S₁, S(g.zo.r) and FS, with probability at least 1 - ó, holds:
[i log(1/5)
(( < R( (FS)
Proof of Lemma Let St ((. be a dataset that has at most one element different with a( its
elements are x E I(), then

---

E
sup
Σ
Oism (ii)
Sr
- Es
sup
of St (xlp)
1 EF
iE (o
1 EF *
<E
sup
Σ (ET(s.zgir)
Ji Sk (x:))
Σ (ET(simp
JiShi S& (xi)
8 EF
<E>
sup ( = Sk
oi (f
Se (I.) for (x1))
S 1,
iET
the last inequality is because (. differs from S(j.zo.r) with at most one element, and f E [0, 1]. Then use McDiarmid's inequality 155 Theorem D.3), we have
Rlixz (FS!) S ( + () log(1/6)
holds with probability at least -. 6. In the following we use the notation: E Es [ = Issaori EiEri for (x;), then define:
((S()aa)))) = sup f Et
ExwDaly [S(((r) | x E Cin.amm) Es
[
Lemma Given B( r), (Ios yo) following holds for any f F
E Sg and f E FSS, with probability at least 5, the
-Bs EANDAY [fSt (x) 1 C(j
[
2
R R(i (FFS))
)+
log(1/0)
2 [
Proof of Lemma 4 Similarly, let (....)) be a dataset that has at most one element different with S(j then we have
(SS - - = sup
E EgwDaly [fSt x E C(j,ro.r)] - Est (.soiin [
Sk eF 5k
E EynDalz [f (x) x E C(j.ro:)) - Escioon [fSs]]
sup
fu e
sup
E syy ts
[ Es [AS]]
Sk S
(2 - S (zi))
sup Sk 5k
(EIO
[(j
then use McDiarmid's inequality, with probability least 8, the following holds:
log(1/8)
(( I Es Esg..n. ()((()))))) | Iz E C(izoon)] + V [I.oo..))
(1)

---

then we could use the standard symmetrization technique ([56), Theorem 3.1) to derive the following:
((.0o))))))) 24 C(i.zo.r)]
=Bsts
sup 1 EF
BanDalo [f (x) x E C(saeai)) Essiionr [ az E ((.2or))
-Bsis.agrr)
sup 10 EF
Esy (
[ [ E.sis [set | E C(sssr)) Ls E Cy.som))
[ss Bsstroom
[1 Estringrr [ | ((s.or))
sup Ax Exil
Jensen
Σ (EIUiaiie
( (x.) 1 (1:)) |ac, ((.2sir)
())
sup far EF
Σ iELeropir
as (1w)(r)) - (x:))
sup Jut E
(Since and S" have same distribution)
E B S
Chagichio
sup SC EF
Σ
os (fet (2:) f (2:)
10,, E C(iaoir)
Jensen
2.s
sup EF
Σ
aifw (xs)
| C(iaroi))
[
VEtisiom
R R..ooo (FS)),
[i
combine with we can get:
2
log(1/0)
R. [fSt C(j,roir)]
[
R(.aoo)) Se
[(
2 ((
holds with probability at least 1 6- 6. Proof of Lemmal ! Let du IRK R satisfies (b((u) Uk. where (U1, (v1,..., Uk) E RKK Then
R. R.f Fi),
since (dx(v) 2x (v') = I0 - U S - Va, o' E R i.. de is 1-Lipschitz. For f e
FS we have | S 1, therefore, joe (f (r)) V I (a)ll S 1, then use Lemma 2 choose
L - 1,5 1,a z. there exists constant C 0 such that
[1 (rS)
R1 (0r o F&) S Ct VK max R(awo)) (F&t log?
(2)
max: feli.ro.))
where [liwair) (FS:) = sup
R ((
(FSS).
and we denote
((x, v) Z C(sieoi)) } as Z(..
3

---

z Fuivon For (j.o))))) E (1.tair)
Vs E F we have
Σ
auf S (xi)
= Bo
Σ
out (zi)
iEZ(i.zg.r)
R
as St (Ii))
Jensen
Σ
S (2i)
S nn,
the penultimate inequality uses the Khintchine-Kahinn Kahane Inequality 57 Therefore,
R(.,o.)) (FSi) Eg
sup fw EF
ff (a+)
I Vn.
(EI().mo:))
which means Ra
() (FS)) S /n.
Choose (ø, 0) 2 (i fer E j satisfies
Eg
Σ
JsJ
iSk (x)
=
sup
Eg
Σ
off Su (x)
2E r
E
iE iETejaape))
then, similar Lemma 3 in we have
sup Fisanml (irg:r)
(FS))
sup
Eg
sup 1 EF (EI(s.
Σ
of (x4)
S1s.00.)) E2
sup
Eg
sup 1 EF iEl
oi (x))
sup
E.
of (())(()) (z)
Jensen =EZ)),00
EF
(EI13.20.r
E.
1Sk
AET(irair)
V2
FSp (i)
2
iET
2 [2.
7Sx (i)
similarly, the penultimate inequality also uses the Khinnehine-Kahnee Inequality [57]
Below we de-
Zes. 8
note (S (i) as fun Choose 8d max bonk, which satisfies max; R ickS K
and @ = max C combine with we have 1<k5K
2 V2
R.. (dx o FF) sww / K [To log?
(3)
fw

---

holds for all & [K]. Then use Lemma
3 and Lemma with probability at least 1 6, holds
EAwDAIN [1 (a) | x E C(
Es (s....)))
[
2
R4i00.17 (Fil)) 3// [I())oo)))
(2/6)
[(iworr)
then combine with 3 we have
ExwDalo [f (z) x C(ixrrrr Escingir [
(22
K
log(2/8)
S 40
log²
3/
V [1s.20.r)
b
[IG..oo.))
holds with probability at least -.66 6.
Proof of Thervem For the inequality Lemma guarantee with probability at least 1 - 8, the inequality
by taking = K for each [K], we can
EENDaIo [f (x) Ciirain)
[
K
log(2K/0)
S 40
log²
3 V 2
V [(izor))
1
simultaneously holds for each E [K]. Then for two different parametrized class FSt
{fer ww E W1} and F&. == {f w E W], just take C = max (C c and b = min {bwi, _.,", the following simultaneously holds for any i E (1,2} and E [K]:
E BanDa [SS((r) x C(A.a..)) - Bst
[
/2 [iwworr)
K
log(2K/8))
I 40
log?
+
Iig.aorr
2 (Ijaao.)
According to the assumption
EvEresoorr
S 7. which means (1S (xx) - 1 (ar))
S& mi
Es
[ I y, then we have
E DaND [AS (I) x E C(iioor))] BuNDaix [f (z) w C(
BanDaix [ (x) x E Cisoo.)) Bsss.oo [
+ Bis... [
Es [SS
EuwDat [f$t(2) x E C(ssorr)] - Bstsinorrs [fSt]]
+
V / V2
K log²
1og((KK/)
8C
6 v I(j.ro.r)
+7,
thus,
Eanyaaly Daly
[ (x) x Cy.nno)) - EENDa [f((x) a E Cij.oor))ll
EawDaay [r (xx) C.J.oo.r)] ENNDEI [. Ciz.a.ril
max kE(K] [
122 [(saroor>
K
(2K/2)())
< 8C
log2
+ 6
+
2 I (i.ar))))
By the definition, we have [(r.
volB P(i
r). and note that
5

---

2r
x2 vol r) r(5+1) pd S 6r
va
therefore we can derive that with probability at least 1 - 8, the following holds:
||BNnDEI [smn (a)
| x E Cli.oo:n] - BEwDaiN [f
x E C(j:
Kd4/2
2n (3p)(Fu,
d4/2 1o/21og(2K/0
S S8GV
r) log?
+ V.
P(iwoor)) r) 1 75
b
which completes the proof.
A.2
Proof of Theorem
First, note that
[((,,,)))) -
(E(w), ((w(((w))))))) x;))
=
VEI(S.m0ir)
Σ
[w(,, - 8 (w0,
I(j:zo:r)
$
L1
[w]
W2ll
the second inequality holds because of Assumption of ) (w), we have
Then due to the Lipschitz continuous gradient
L1
R ( (w))) S (V a R (w)"), w - w0)
1| - w0'
+
V Vulli (w0),
1
Σ
V Vwl(w' xIi)
=
M
IEI
L Lini
1
+
V
2
M
iEZgi
Lnng
S
-1p
V Rij
Σ
(voelw'',
M
VEIn2
the inequality holds because of Assumption with Assumption we have
2 prescribe notation & [g (I combined
1
V w (w.o)..o.))(
Vul(w',
Tle
M
VE ge
/ V ((iiion (w')
1)t
Σ
V Vwe(w", xzs)
M
iET ge (Ttesap)))
Σ
Vwe(wo' x2;)
M
VEI gi (ecingor)
(iitoioo + (M (M- -x) G2
||
M
M
(Ri,.oor)) Ry R.)r))))) + G2
M
choose m. Bpt due to T S 28 G2 < Linida
In

---

Note that & ~ Hypergeometric (n, [(t-mom)) M) by Hoeffding's 's inequality 59], we have
IP (Et [(s. - -Me) Se 22
take 7 =c-2M6² we have
&
[(jaorr)
In (T/5)
M
M
2M
holds simultaneously for all & (1, T) with probability at least 6, which means
23
(Ri,soo))((0") R(sooor))
M
2 p2p max {
(Fizzanl M
In (T/0)
0 } (Reseor)(w")) Risseam)
2M
Therefore,
(w)
R ())
I R (w') R(j,
.0 .00) (Ris Riraom)) + L1G²
2
In (T/5)
1
max
t
M
2M
2,82 1222
2
{
((i
In (T/5)
.00) (B(jaa.))(uo)) - R R(s.oo.))
+ L2
1
max
t
(
r
+1+
M
2M
2,3 k2²
holds simultaneously for all / (1,2. T} with probability at least - 5.
A.3
Proof of Proposition I
Under Assumption
due to the Lipschitz continuous gradient, for V(z,, Mi) E S, we have
Lg
(w, 21) 2 l (w. .) + V 1D x) , x²)
||2, - x 1
(4)
2
L2
(w, < <((w))) + (1 x) 24 -x) +
||3, - x 1|
(5)
2
La
(ww, x) < E (w. 2;)) + w. 2.) - x,) +
|| - x |
(6)
2
holds for Vo E X. By (4), choose 5j |V [
( which satisfies || x/| r, then
have
V (w, x|
L2r²
8 (w, 2 W. I;
+ +r /V x)1
|
L
2 r V (w, x)1l
2
L27
V (we .))
V. (w, + V (w, x;)1|
2
Lzr²
2 r (IVV (w)
|2 xw) V [x((,,))) x)),)))))
2
L21"
2 r ([V (w,+:))
La 121 - x11)
2
3Lzr2
r V (w. 2+)1
2

---

which means
8 (ww, x:) + 322r
|V (7,6(wxzj)) x;)1 1
2
Use (5), we have
L2
E (w, E (mx, < (V (w, , - x) +
|1 1:12
2
Sr r V (w,,)] L2r?
< r (l (w, x;)11 + [z((w,x,)) - -Vz((wxx))) + L2r²
2
3L2r²
< r V (w, xw,,,)
2
then use (6), we have
L
(ww x) - ((w, 1 (V (w 2;), x -xx)
+
|| - x1
2
Sr rr (w, x.)l + L2r²
Thus
l (w, xz) - Ll (w,x))
3
S V (w),
(8)
2
Combining (7) and (8), we get
Ie (w, xx) l (w, a)
3
l ((w, v.)
3L2r
[ (w, xw)x;))1
r +
2
B Details of Pre-training Embeddings
actual experiments, was found that if the embedding not pre-trained first, the model convergence speed will be quite slow. We tried to freeze the parameters of the embedding layer after pre-training multiple epochs, but found that even with the same number of epochs each pre-training, the internal structures between the embedding vectors are very different, and there may be clustering between different class embedding (as shown in Figurd6). the distance between multiple classes is very close, which leads to redundant information from many other classes in embedding.
To address this issue and enable the generator's 'ss embeddings to better reflect class information, we employ the following loss, utilizing the feature outputs of the source model as guidance for embedding pretraining:
Besc ij
31 log LJ
LM = Eis
M
i.j
log
+ Eij M
3I T
M Seun 1.3
Eis M onn
i.j
log M.
Ei, Man
Ecer Mad log M.,
However, directly optimizing the L. leads to the issue of loss collapse. We analyse each component of the loss. First, note that for Vi, [K],
M < -1 S M < 1. Then
M Feso exp( (xppMM Beos
I exp M. expl (M
1+ expl M. Bron
I- -1) exp(2)
M Seme ΣK exp exp(M
1+ I+Eny exp (M. Ma
Under the settings in this paper, K = 10, so both terms analyse above are positive with lower bounds. Sex In the optimization process, their ratio will not approach zero. As for M it is deterministic positive constant. Therefore, when it serves the numerator, it will not lead to ratio close to zero
inside the logarithm. However, for ME if it becomes close to zero when used as the numerator, will cause loss collapse. Note that
5.j
MM.
Fro M
exp
1
1.)
ΣK exp
M
1 Enss exp (M M.
K max exp (M 15 KSK
8

---

18
76
Colse
Codtre
Eiciddion
Figure 6: The top row displays the result of directly trained for 9 epochs, while the bottom row shows the result of directly trained for 49 epochs. The numbers 9 represent different class labels in sequential order. The two heatmaps on the left depict the Euclidean distance and cosine similarity between the embeddings of each class. The images the right display the distances after PCA projection onto 2-dimensional plane (normalized and unnormalized, representing cosine similarity and Euclidean distance, respectively).
To prevent M, i.
Fox from becoming too small, we need to ensure:
> 0,
K max exp K
(ME M
which means we need ensure that
Bes (M # t00, 11 vis K,
max exp s K
since M,eee E.E are non-negative. Also, note that MEE.: = le- e/ S ||el| + le Therefore, by imposing the following constraint in the optimization:
min L s.t. |ee
S SUiSiiSK
We can ensure that 5 < 2U, which in turn guarantees
max exp M Fas + +0,, V1 Sig K,
15 K
Ees thus avoiding the issue of M, becoming too small and causing loss collapse.
By explicitly incorporating the above constraint as a regularizer into the optimization objective, we obtain the final manifold matching loss:
LM =M= Ma
Sex log M M.eeue
+ Eis M. log
Frie
M Seue
Peue
+\1 En, Stan i.j
log
E,s M.can log M..,
M..,
+ Ae K
|e-1|
In our experiments, use the AdamW optimizer with learning rate of 1.5c for 15000 epochs to optimize the embeddings, while setting Y and X2 to 0.01. The results, as shown in Figure] 7 demonstrate significant improvement the clustering of embeddings between different classes.
The trained embedding denoted as ej E T2 j E [K], where ej E Rd4 E [K]. For the utilized Resblock, let the input and output channels be Onn and Cout respectively. The fully connected layer

---

Figure The results obtained by optimizing LM-
MLP(.) E Rd + RC Firstly, we employ the initial convolution kernel Conv, to map the raw data from the input h of the current layer, transitioning it from [Cin, H, W] to [Cout. H. W]. Then perform addition operation between this mapped result and the embedding transformation obtained through the MLP. Formally expressed as:
h' = Convi + MLP((e)
where y is the label of the imput. Subsequently, we apply a second convolution to this output, followed by LayerNorm operation, and finally, pass through attention operation before adding to the residual. Formally stated as:
output (ttennton(LL((onnv(())) + h)
C Additional Ablation Studies
To investigate the impact of different settings during the training phase, we conduct several experi- ments. First, we investigate the output consistency of the model for different values of q used for sample screening, as shown in Figure 8 A smaller q implies a stricter selection, which leads to an increase in output consistency (in terms of mean), but also results in a larger standard deviation. In this case, q = is a relatively suitable choice.
Then we validate the impact of different choices of d(-:-) on targeted transferability. The results are shown in Figure 9
where d(-,.) was chosen as the Smooth L1 loss, which significantly improves the targeted transferability compared to L1 and L2 losses.
D Implementation Details
In the exploratory experiments conducted in Section the three different network architectures we employed are depicted in Table
Table 4: Architectures of NN Classifiers
Model
Hidden Layer
Hidden Layer
Hidden Layer 3
Model
Linenrt2,
Lincar
100
ReIJU
Rel.U
Model
100)
150)
ReLU
ReLU
ReLU
Lincant 150, 2)
Model 3
Lincur12. ReLU
Lineur120,
Lineant 20,
ReUU
ReLU
Lineun 20,
For the experiments CIFARIO mentioned in Section we used the PyTorch framework. The optimizer is SGD, with an initial learning rate of 0.1, weight decay of 5e - 4, and momentum of 0.9. We apply cosine annealing learning rate decay, with maximum number of epochs (I (Tmaa) 200. Tolerance of Early-stopping is set to 30. Additionally, we used a batch size of 128 during training and applied the following data augmentations:
Randomly crops the input image to size of 32 x 32 pixels with padding of 4 pixels.
10

---

12
4=10
4=20
9-50
Density
Ma 0.25 M 0.2011000110 16:0.10
q
MOn O.3IC
6
4
DETE SD
0.0
0.2
0.4
0.8
Cosine Similarity
Figure 8: Density plots of output similarity between models under different g selections. For the screened data, we calculate the cosine similarity between the output of the source model and the output of all target models, and take the average represent the output similarity of the data across different models. "q=all" indicates no screening, and the dashed line represents the average for different values of
7/7
0.9
0.5
2.0.0
Smooth LI
Success Rate
0.1
0,2
Demsel21
Res152
In:Resv22
Target Model
Figure 9: Targeted transferability (Src:Res50) with different chosen of d(,;) :).
Randomly flips the input image horizontally with a probability of 0.5.
After data augmentation, the input is then transformed into a tensor and normalized with mean of (0.4914, 0.4822, 0.4465) and standard deviation of (0.2023, 0. 0.1994, 0.2010).
For the experiments in Section 4 we used the structure of following models: ResNet-50, VGG19bn, DenseNet-121, ResNet-152 from the torchvision library, and others from the timm library. The torchattacks library is utilized for MIM, SI-NIM, TIM, and DIM.
E
Additional Experimental Results
E.1
Ensemble Model Source
In Table 5,
we report the results of the case of an ensemble model as the source model. The selected ensemble model are consist of three models: Res-50, Inc-v3, and Inc-v4, while the other models
http://ggtubbbooerrrrrr/////aeae wtt.:/aaaaaaaaatttat-aa-aaa-a-ee pytorch

---

not involved in the ensemble are treated as target models. ESMA still demonstrates outstanding adversarial transferability.
Table 5: Targeted transfer success rates. The source model an ensemble of Res-50, Inc-v3, and Inc-v4.
Sre:Enemmlee -Res152
Attack
VGG19bn
+ Densel21
IncRes-v2 -VT
AVG
MIM
1.76%
3.40%
3 29%
107%
0.22%
1.95%
TIM
9.96%
16.13%
76%
8.27%
1.11%
10 10.05%
DIM
10.271
16.31%
15 22%
8.20%
0.87%
10.17%
SI-NIM
1.16%
1.67%
1.49%
0.93%
0.13%
1,08%
Po-TI-Trpp
14,22%
21.76%
18.69%
12.00%
1.6%%
13.68%
SI-FGS
15.72%
12.20%
1.53%
15.02%
S'--SI-I-IMM
17.82%
21.07%
21 36%
16.38%
2.38%
15.00%
Logat
16.195
27.56%
17.51%
36.96/S
2.20%
20.06%
DTMI Logit SU
18.11%
30.16%
73%
38.64%
2.645
71.
RAP-CS
48 98%
72.91%
70.67%
26.53%
4.22%
44 44.6%
HGN
55 10%
67.37%
63.54%
49.3%
26.78%
52.53%
TIP 12225,2)
60.02%
71.35%
67.44%
55.42%
30.04%
56.85%
SSMA(12303..
62.51
74.11
72.61%
62.33%
32.70%
60.85%
E.2 Models Trained on Different Distributions as Victim
Given that our discussion is based on the condition that the source and target models trained on the same distribution data, to explore the transferability between models trained on training data with different distribution, models, Inc-v3-ady and IncRes-v2-ens. The results are reported in Table 6 ESMA's performance is slightly weakened but still competitive. Considering that this validation is not directly related the viewpoints, conclusions, and statements of our paper, we will further discuss the transferability on target models trained on data with distribution shift our future work.
conducted additional transfer attacks two adversarially trained
Table 6: Targeted transfer success rates. "Sre" indicates the source model.
Src
Target
MIM
TIM
DIM NIM Po To-TI-Trip Trip SI-FGS M 1 DIM Logit DTMI Logit SU RAP-L
HGN
TTP
ESMA/Oun)
Res50
IncRos-<2en
10.11% 0,3695
(0.11% 0.22%
0.18% 0.11%
0.22%
0.24%
0.76%
0.89%
012095
0.225
0.27%
0L38%
2.12%
5.53%
3.25%
Sic
Targel
MIM
TIM
DIM
SI-NIM Po-TI-Trip SI-FGS S TI-D-IDDDM DIM Logil DTMI-Logii-SU RAP-L
[IGN
TTP ESMA(Oun))
Dense121
-ac--33yt
Inckkes-2an
10.13%
0.09% 0.18% 0.13% 0.11% 0.31%
0.20%
0.84%
0.22%
0.20%
0.225
0.18%
2.39%
1888
3.18%
3.50%
1.83%
Sec
Target
MIM
TIM
DIM
SI-NIM Po-TI-Trip SUPGSS s SIS--TEDD Logit DEMI-LogiE-SU KAP-LS
HGN
TTP ESMA/Ous)
VGG19bn
Inc-v
10.13% 0.11%
1666
0.118
0.11%
0.22%
0.16%
0.13%
0.13%
0.18%
0.20%
0.49%
0.31%
Inza
0.00% 0,07% 0.09% 0.01%
0.11%
0,138
0.22%
0.00%
0,078
0.09%
0.53%
0.60%
0.22%
Sie
Target
SI-NIM Po-TI-Trip SLFGS SPI-SI-T--DMN Logit L'MMi-Logi
ESMA/Ous)
Res152
IncRess2gges
30.18% O. 0.20
(0005% 0.3095
0.27%
0.33%
0.93%
1.025
0.27%
0.266
0.315
0.345
0.38%5
4.55%
4.26%
5.09%
5.00%
Sie
Target
MIM
TIM
DIM SI-NIM Po-TI-Trip SI-FGISM 8"1-SI-TI-DIM Lngit DTMI-I RAP-LS
HGN
TTP
FSMA(Oun))
Fasemble
Iat-V3gdy [0.20% 0.60PS
0.40% 0.13%
0.69%
0.609S
2.51%
0.53%
0.534
0L595
22.67%
24.44%
17,00%
Incetes>22ex (0.115 0.9695 0.42%
0.09%
0.87%
0.669%
251%
(120X6
0.24%
0.58%
21.25%
23.33%
20182%6
F Fooling Google Lens
We show that adversarial examples perturbated using ESMA can deceive Google Lens in image search, shown in Figure 10 and
12

---

French Bulldog
(b) Google Lens: French Bulldog
ESMA (Target:Goose)
(d) Google Lens: Goose
ESMA (Target:Hippo)
(1) Google Lens: Hippo
Figure 10: The original image is of a French Bulldog, which can be intentionally perturbed to yield different search results, such goose in the second and hippopotamus in the third row.
13

---

7963
Model T
(h) Google Lens: Model
Your Combat vtra raalicic prakelled
7963
ESMA (Target:Cannon)
(d) Google Lens: Cannon
7963
(e) ESMA Grarget:Grey Owl)
(1) Google Lens: Grey Owl
Figure 11: The original image is of a Ford Model T, which can be intentionally perturbed to yield different search results, such cannon in the second row and gray owl in the third row.
14


Reviews:
[{"review_id": "zJF3g4RrOw", "reviewer": "Reviewer_YLC1", "summary": "This paper finds that optimizing perturbations towards easy examples located in HSDR can improve targeted adversarial transferability. The authors provide toy experiments and theoretical proof to illustrate these conclusions. They further utilize easy examples to guide the targeted attack in the GAN-like framework. This framework makes it feasible to perform the multi-target attack.", "questions": "Please conduct more experiments as discussed above.", "limitations": "None.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Exploring targeted adversarial transferability in terms of High-Sample-Density-Regions is interesting.\n2. The proposed framework support multi-target adversarial perturbation generation.\n3. The motivation of this paper is clear.\n", "weaknesses": "The experiments are insufficient. They miss previous works, hierarchical generative networks for multi-target attack [A], and iterative attacks [B,C,D, E]. Besides, the authors only conduct experiments on naturally trained models, and neglect to consider adversarially trained models and other defense methods. And the authors should attack more diverse network architectures, such as ViTs.\n\n\n[A] Yang X, Dong Y, Pang T, et al. Boosting transferability of targeted adversarial examples via hierarchical generative networks[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 725-742.    \n[B] Zhao Z, Liu Z, Larson M. On success and simplicity: A second look at transferable targeted attacks[J]. Advances in Neural Information Processing Systems, 2021, 34: 6115-6128.   \n[C] Qin Z, Fan Y, Liu Y, et al. Boosting the transferability of adversarial attacks with reverse adversarial perturbation[J]. Advances in Neural Information Processing Systems, 2022, 35: 29845-29858.    \n[D] Wei Z, Chen J, Wu Z, et al. Enhancing the Self-Universality for Transferable Targeted Attacks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 12281-12290.    \n[E] Byun J, Cho S, Kwon M J, et al. Improving the transferability of targeted adversarial examples through object-based diverse input[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15244-15253.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thorough review of our paper and the valuable feedback you have provided. Allow us to address your inquiries in the following response:  \n\n**Concern1**： They miss previous works, hierarchical generative networks for multi-target attack [A], and iterative attacks [B, C , D, E].  \n\n**Response to Concern1**: Sincerely appreciate your insightful feedback. We have incorporated the generative attack algorithms [A], [B], and [C] into our original comparisons, denoted as HGN, Logits, and RAP-LS respectively. As the work presented in reference [D] focuses on transferability in the context of UAP scenarios, and reference [E] employs 3-D transformations to enhance transferability, which significantly alter the original images themselves, they do not directly align with our research scenario. However, for the sake of completeness in discussing related work, these mentioned papers are duly referenced and acknowledged in our article.   \n\n**Concern2**：Besides, the authors only conduct experiments on naturally trained models, and neglect to consider adversarially trained models and other defense methods.   \n\n**Response to Concern2**: We greatly appreciate your valuable suggestions. As the theoretical analysis and fundamental assumptions of this paper are based on the condition of training data from the same distribution, the previous versions did not encompass adversarial training models or models with defense mechanisms. Inspired by your insightful reminder, we have conducted an evaluation of model transferability under adversarial training conditions, and have included this result in the appendix of the article. We have observed certain performance degradation with the proposed methods, which aligns with our initial assumptions. In our subsequent work, we will specifically address the challenges of transferability in scenarios involving distributional non-consistency. Your input has been instrumental in guiding our research direction.  \n\n**Concern3**：And the authors should attack more diverse network architectures, such as ViTs.  \n\n**Response to Concern3**: We greatly appreciate your suggestion regarding conducting experiments with additional model architectures. We have taken this suggestion into account and have included the Vision Transformer (Vit) as one of the evaluated models. This addition allowed us to validate the effectiveness of our method on the Vit model, further emphasizing its superiority.  \n\nOnce again, thanks for all the insights and suggestions. Regarding the included comparative experiments, you can find them highlighted in blue in the PDF attached within the Author Rebuttal.\n"}, {"review_id": "ZICN8T1UK6", "reviewer": "Reviewer_tYob", "summary": "The paper presents both theoretical and experimental evidence showing that incorporating perturbations towards the High-Sample-Density-Regions (HSDR) of the target class can significantly enhance the transferability of targeted attacks. To address the challenges of density estimation in high-dimensional data, a novel method for identifying samples within HSDR is proposed. Leveraging these insights, the authors introduce ESMA, which outperforms existing generative targeted attacks in terms of effectiveness and efficiency.", "questions": " 1 According to Algorithm 3,  the target class is randomly sampled from a uniform distribution in the training stage. How does the generator generate a target adversarial example given a specific target class in the reference stage? \n\n2 Is the model employed to select target anchors different from the source model in ESMA? This information is not clear.", "limitations": "The writing and the usage of notation need to be enhanced.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper provides theoretical and experimental evidence to support the observation that deep learning models exhibit more consistent outputs in the High-Sample-Density-Regions (HSDR) of each class (Section 2.1). This finding establishes a foundation for the proposed approach. The paper demonstrates through experimental and theoretical analysis that easy samples with low loss in early-stopping models are commonly found in HSDR. Leveraging this insight, the proposed method directly adds perturbations targeting these samples of the target class, resulting in improved targeted adversarial transferability. This approach circumvents the need for computationally expensive density estimation of high-dimensional samples. Finally the paper introduces the Easy Sample Matching Attack (ESMA)  which surpasses state-of-the-art generative attacks such as TTP in terms of targeted transfer success rate. Notably, ESMA only requires one model to perform attacks for all target classes significantly reducing storage space requirements and training time compared to TTP.", "weaknesses": "There are some grammar mistakes (in lines 231-238, the sentences in the whole paragraph are all connected with commas, making it difficult to read) and the notion is a little bit confusing (e.g. in Algorithm 2, $a_k$ is in the feature space, but in the equation in line 251, $a_j$ is in the input space).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for highlighting these issues, and we greatly appreciate your thorough examination of our work. Below are our responses to each of these questions and suggestions:  \n\n**Concern1**：There are some grammar mistakes (in lines 231-238, the sentences in the whole paragraph are all connected with commas, making it difficult to read) and the notion is a little bit confusing (e.g. in Algorithm 2, $a_k$ is in the feature space, but in the equation in line 251, $a_j$ is in the input space).  \n\n**Response to Concern1**: We greatly appreciate your identification of the detailed issues in our paper. Following your suggestions, we have made the following revisions:  \n\n**Original**:\nTo obtain better embeddings that more effectively represents inter-class information, since the latent space of deep learning models often extracts enough class information, we treat the output features of the local model as a set of a priori embedding, in order to make the generator embedding have a similar structure to such a priori embedding, we refer to the idea of manifold learning, using a strategy similar to the SNE algorithm, let $\\mu_j =\\frac{1}{\\left|\\mathcal I_j\\right|}\\sum_{i\\in \\mathcal I_j}l_i$, where $l_i=f(x_i)$, and then we design the following manifold matching loss , generator embeddings of various class were pulled to the manifold that output features in to obtain embeddings with a better structure.  \n\n**Revised**:\nTo obtain better embeddings that more effectively represent inter-class information, since the latent space of deep learning models often extracts sufficient class information, we treat the output features of the local model as a set of priori embeddings. In order to make the generator embedding have a similar structure to such priori embeddings, we draw inspiration from manifold learning and employ a strategy similar to the SNE algorithm. Specifically, we introduce $\\mu_j =\\frac{1}{\\left|\\mathcal I_j\\right|}\\sum_{i\\in \\mathcal I_j}l_i$, where $l_i=f(x_i)$, and then formulate the following manifold matching loss. This loss encourages the generator embeddings of various classes to be attracted to the manifold representing the output features, thus yielding embeddings with a better structure.  \n\nFurthermore, we have corrected the reference from $f(a_j)$ in line 251 to $a_j$.\nAnd we performed a thorough review of the entire paper and carried out a comprehensive refinement to enhance its overall quality.  \n\n**Question1**： According to Algorithm 3, the target class is randomly sampled from a uniform distribution in the training stage. How does the generator generate a target adversarial example given a specific target class in the reference stage?  \n\n**Response to Question1**: Sincerely appreciate your inquiry. In the reference stage, adversarial samples are generated directly by designating the target classes, eliminating the need for random selection. Each individual sample undergoes an attack targeting all other specified classes. It's important to note that these target classes can be customized and explicitly specified according to the experiment's requirements.  \n\n**Question2**：Is the model employed to select target anchors different from the source model in ESMA? This information is not clear.  \n\n**Response to Question2**: We sincerely appreciate your question, as it greatly contributes to the refinement of our article. In reality, the selection of target anchors is conducted based on the source model. Our pre-trained embeddings and target anchor selection solely rely on information from the source model. We consciously opted for this approach to maintain fairness and equity in our study. We will also clarify this in the article.  \n\nThank you once again for pointing out issues and raising questions. Your input has been immensely valuable to us.\n"}, {"review_id": "1pOD85pHC9", "reviewer": "Reviewer_YJBk", "summary": "This paper presents a hypothesis that pushing adversarial samples towards high-density regions can enhance the transferability of target attacks. The author provides theoretical analysis and experimental validation to support the rationale behind this hypothesis. Based on this hypothesis, the author designs a new generative attack method that improves the adversarial transferability of target attacks.", "questions": "*  I suggest the authors conduct ablation studies to verify their claim.\n*  Does making the output of adversarial examples match that of the target samples improve the density of adversarial examples?", "limitations": "The authors adequately addressed the limitations", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The idea of pushing ushing adversarial samples towards high-density regions in target attack is well-motivated\n* The theoretical analysis presented in this paper is insightful and solid.", "weaknesses": "* The lack of persuasive experimental evidence is a major concern.\n    1. Insufficient ablation study: The paper introduces multiple modules in the proposed method, but the author fails to include an ablation study. Without such an analysis, it becomes challenging to ascertain whether the proposed method genuinely enhances adversarial transferability. For example, it would be valuable to investigate if the transferability improvement persists when pretrained embeddings are not utilized.\n    2. Limited impact of easy example-based adversarial attacks: The experiments presented in Table 1 demonstrate that the adversarial attacks based on easy examples have limited impact on transferability. Additionally, the experimental description in this section lacks sufficient detail. It would be helpful to clarify whether the baseline methods in Table 1 are based on cross-entropy. If they are, I recommend the author redefine the baselines, such as using target anchors generated from randomly selected samples of the target class.\n    3. Lack of validation experiments:  Experiments are required to assess whether the adversarial examples generated by the proposed method exhibit higher density/smaller loss and loss gradients compared to those generated by baseline methods. It is essential for the author to include such validation experiments to provide empirical evidence supporting the claims regarding the improved density and reduced loss characteristics of the generated adversarial examples.\n\n* The analysis and proposed solution in the paper exhibit a gap. The conclusion drawn in the paper suggests that samples with lower loss and smaller gradients tend to have stronger transferability. A straightforward approach to address this would be to minimize not only the loss but also the gradient of the loss during the attack. It raises the question of why the author did not employ this direct and simple solution. Instead, the author chose to modify the loss to make the output of adversarial examples align with the target anchor. This approach introduces a gap between the previous analysis and the proposed solution.\n\n* Section 3 lacks clarity in its description. The role of pretrained embedding is not well-defined. I recommend the author to provide a detailed formula or explanation of how pretrained embedding is utilized in the generator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely appreciate your valuable suggestions. Here is our sincere response to each of your suggestions (due to word limitations, a portion of the questions you raised has been indicated with ellipsis) :  \n\n**Concern1**：The paper introduces multiple modules in the proposed method ...... when pretrained embeddings are not utilized.  \n\n**Response to Concern1**: Thank you for your suggestions. In the initial submission, we included some ablation experiments in Appendix C, but indeed did not cover experiments on the performance improvement of pretrained embeddings. Appendix B provides an analysis of the pretrained embeddings. Based on your suggestion, we have incorporated a ablation experiment on pre-trained embeddings in Appendix C. We tested the performance difference of ESMA with and without pre-trained embeddings at different training stages. The results indicate that the model performs noticeably weaker without the pre-trained embedding operation, confirming the effectiveness of this module. We sincerely acknowledge the valuable contribution of your suggestion.  \n\n**Concern2**：The experiments presented in Table 1 demonstrate that the adversarial attacks based on easy examples have limited impact on transferability. Additionally, the experimental description in this section lacks sufficient detail. It would be helpful ...... generated from randomly selected samples of the target class.\n\n**Responce to Concern2**: We extend our sincere gratitude for your valuable input. Your insights have the potential to further enhance the quality of our paper. In Table 1, we aimed to offer a straightforward validation of the broad effectiveness of our approach in enhancing target attack transferability. The baseline incorporated therein does indeed entail attacks using standard cross-entropy. Following your suggestion, we have included an additional baseline experiment involving random anchor sample selection for comparison. The performance of the baseline with randomly selected anchors is inferior to the case with sample screening. The comparative ablation experiment on random sample selection and target anchor screening for ESMA has been included in Appendix C of the initial submission.  \n\n**Concern3**：Experiments are required to assess whether the adversarial examples generated by the proposed method ...... improved density and reduced loss characteristics of the generated adversarial examples.  \n\n**Response to Concern3**: We sincerely appreciate your insightful suggestion, as it holds significant merit. However, directly substantiating the elevation in density presents a formidable challenge. However, directly verifying the increase in density is quite challenging. Our analytical conclusion is that samples with smaller loss and loss gradient norms in the training set often correspond to HSDR, but it does not necessarily imply that samples with smaller loss and loss gradient norms are always within the HSDR. Therefore, conducting this validation experiment is indeed difficult. Your understanding is greatly appreciated.  \n\n**Concern4**：The conclusion drawn in the paper suggests that samples with lower loss and smaller gradients ...... This approach introduces a gap between the previous analysis and the proposed solution.  \n\n**Response to Concern4**: We are immensely grateful for your insightful inquiry. Specifically, our perspective can be likened to proposing a necessary condition: samples that satisfy this condition within the training set are more likely to reside within the HSDR. However, it does not imply that merely possessing low loss and low loss gradients guarantees a sample's location within such a region. Thus, directly optimizing for loss and loss gradients is insufficient. As a result, we adopt a strategy of aligning by selectively screening such samples as anchor points from the early-stopping model's training set. Your thoughtful consideration is highly appreciated.  \n\n**Concern5**：Section 3 lacks clarity in its description. The role of pretrained embedding is not well-defined. I recommend the author to provide a detailed formula or explanation of how pretrained embedding is utilized in the generator.  \n\n**Response to Concern5**: We sincerely appreciate your question, as it contributes to further enhancing the clarity of our article. Following your valuable suggestion, we have provided a comprehensive explanation of the specific embedding utilization approach in the appendix of our paper. Allow us to elucidate the details here as well. The trained embedding is denoted as $e_j\\in\\mathbb R^{d_1}, j\\in [K]$, where $e_j\\in\\mathbb R^{d_1}, j\\in [K]$. For the utilized Resblock, let the input and output channels be $C_{in}$ and $C_{out}$ respectively. The fully connected layer $\\mathrm{MLP}(\\cdot)\\in \\mathbb R^{d_1}\\rightarrow \\mathbb R^{C_{out}}$. Firstly, we employ the initial convolution kernel Conv1 to map the raw data from the input $h$ of the current layer, transitioning it from [$C_{in}$, H, W] to [$C_{out}$, H, W]. Then we perform an addition operation between this mapped result and the embedding transformation obtained through the MLP. Formally expressed as:  \n\n\\begin{equation*}\nh' = \\mathrm{Conv1}(h)+\\mathrm{MLP}(e_{y})\n\\end{equation*}\nwhere $y$ is the label of the input. Subsequently, we apply a second convolution to this output, followed by a LayerNorm operation, and finally, pass it through an attention operation before adding it to the residual. Formally stated as:\n\\begin{equation*}\n\\mathrm{output}=\\mathrm{Attention}(\\mathrm{LN}(\\mathrm{Conv2}(h'))+h)\n\\end{equation*}  \n\nOnce again, we sincerely appreciate all your insights and suggestions. Your input has been immensely valuable to us. Regarding the added images and tables, you can find them in the PDF file attached in the author rebuttal. The additional experiments are denoted with blue markings in the tables.\n"}, {"review_id": "fwaM0fR0G4", "reviewer": "Reviewer_YEMS", "summary": "The paper theoretically and experimentally demonstrated that adding perturbations towards High-Sample-Density-Regions (HSDR) of the target class can further enhance the transferability of targeted attacks. To avoided the impracticality of density estimation in high-dimensional data, authors propose a method for identifying samples within HSDR. The experimental results demonstrate the effectivenss of  the proposed method.", "questions": "Please refer to weakness.", "limitations": "This paper only considers CNNs and it is doubtful whether it is valid on ViTs.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The experiments demonstrate the effectiveness of the proposed ESMA, which can outperform baselines by a large margin.\n- The idea of adding perturbations towards HSDR is intersting and effective.\n- To avoided the impracticality of density estimation in high-dimensional data, this paper propose a method for identifying samples within HSDR, which is practical.", "weaknesses": "- The recent transfer-based attack [1,2,3] is missing. I suggest authors discuss or compare propsed method with these paper. \n- The experiments are not extensive, i.e., only two tables. More ablation studies should be presented. For example, \n   - Will the results be really poor if a difficult sample is selected to form target anchor sets? Or why author don't report the results of ESMA in Table 1.\n   - Does ESMA still outperform the existing algorithm if the source model is an ensemble of models.\n\n\n\n[1] Zhang et al. Staircase sign method for boosting adversarial attacks, 2021\n\n[2] Zhao et al. On Success and Simplicity: A Second Look at Transferable Targeted Attacks, NeurIPS 2021\n\n[3] Long et al. Frequency Domain Model Augmentation for Adversarial Attack, ECCV2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable feedback. Below is our response to your suggestions:  \n\n\n**Concern1**：The recent transfer-based attack [1,2,3] is missing. I suggest authors discuss or compare propsed method with these paper.  \n\n**Response to Concern1**: We sincerely appreciate your suggestions, as they greatly contribute to improving our article. Following your advice, we have incorporated these works into the references. Regarding [1] and [2], we acknowledge their competitive nature as baseline methods and have included them in our comparative experiments (referred to as SI-FGS$^2$M and Logit, respectively, in our comparison). As for [3], we observed that it does not provide any discussion or experiments in the context of target attack scenarios; thus, we have not included it in our comparisons. However, for the comprehensiveness of the related work in the article, all mentioned papers are referenced and explained within the article.  \n\n**Concern2**：Will the results be really poor if a difficult sample is selected to form target anchor sets? Or why author don't report the results of ESMA in Table 1.  \n\n**Response to Concern2**: Sincerely appreciate your invaluable suggestion. In fact, we have already included comparative ablation experiments with and without anchor point screening in Appendix C of the initial draft. It is true that when randomly selecting target anchors, there is a noticeable degradation in performance. Table 1 was specifically designed to validate the broad applicability of our approach in enhancing target attack transferability. It provides a comparison between the baseline and the baseline with the additional screening mechanism, excluding ESMA. Initially, our comparison involved attacks using standard cross-entropy and attacks utilizing the square of the distance to target anchors, highlighting the improvement achieved through screening. To ensure even greater rigor, we have conscientiously included the scenario of randomly selecting anchor points in Table 1's baseline, thereby further enriching our analysis. Once again, I sincerely thank you for your input and for helping us refine our work.  \n\n**Concern3**：Does ESMA still outperform the existing algorithm if the source model is an ensemble of models.  \n\n**Response to Concern3**: Your questions are of immense help to us. In response to your inquiries, we utilized an ensemble model composed of ResNet-50, Inception-v3, and Inception-v4 as the source model. We then conducted tests on models outside of this ensemble, and the results of these tests have been documented in Appendix E. The experimental outcomes demonstrate that our method continues to exhibit superior performance even when the source model is an ensemble.  \n\n**Concern4**：This paper only considers CNNs and it is doubtful whether it is valid on ViTs.  \n\n**Response to Concern4**: Thank you sincerely for your valuable input. Your concern has truly sparked our inspiration, as the inclusion of ViT in transfer attack research is relatively uncommon at present. Taking your concern to heart, we have conscientiously incorporated ViT as one of the target models. The experimental results continue to affirm the effectiveness of our approach. We are deeply grateful for your insightful suggestion, as it has significantly contributed to the enhancement of our work.  \n\nOnce again, we sincerely appreciate your valuable input and suggestions. Regarding the experiments we added, we have included the tables in the supplementary PDF file attached to the author rebuttal. The experiments we added are marked in blue in the table."}]

Author rebuttal:
We sincerely appreciate the valuable insights and suggestions provided by the esteemed reviewers during the review process. We extend our gratitude for the time and effort you dedicated to scrutinize our paper. Your reviews are of paramount importance to our research endeavor, and your insightful recommendations and feedback have played a crucial role in further refining and enhancing the quality of our paper. For each reviewer's comments, we have addressed each point individually. The following outlines the modifications we have made based on the reviewers' feedback:  

1) We have restructured the references. In order to enhance the comprehensiveness of the literature review and to capture readers' interest, we have incorporated the following six relevant works in the field of adversarial attacks into the references section. Additionally, we have expanded the literature review portion in the introduction.  

>[1]Yang X, Dong Y, Pang T, et al. Boosting transferability of targeted adversarial examples via hierarchical generative networks[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 725-742.  
>[2] Zhao Z, Liu Z, Larson M. On success and simplicity: A second look at transferable targeted attacks[J]. Advances in Neural Information Processing Systems, 2021, 34: 6115-6128.  
>[3] Qin Z, Fan Y, Liu Y, et al. Boosting the transferability of adversarial attacks with reverse adversarial perturbation[J]. Advances in Neural Information Processing Systems, 2022, 35: 29845-29858.  
>[4] Zhang et al. Staircase sign method for boosting adversarial attacks. arXiv:2104.09722.   
>[5] Wei Z, Chen J, Wu Z, et al. Enhancing the Self-Universality for Transferable Targeted Attacks[C]//Proceedings of the IEEE/CVF
 Conference on Computer Vision and Pattern Recognition. 2023: 12281-12290.  
>[6] Byun J, Cho S, Kwon M J, et al. Improving the transferability of targeted adversarial examples through object-based diverse input[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15244-15253.  


2) Building upon the existing comparative experiments, we have expanded our evaluation to include a comparison with the methods proposed in the works [1], [2], [3], and [4], denoted as HGN, Logit, RAP-LS, and SI-FGS$^2$M respectively. Additionally, we have introduced the Vision Transformer (ViT) as another tested model. Furthermore, we have included a comparison involving an ensemble model as the source model, which is presented in the appendix. The experimental results consistently affirm the superior transferability of our approach even amidst these additional comparisons.  


3) Due to the theoretical analysis and foundational assumptions of this paper being built upon the premise of training data from the same distribution, we conducted additional transfer attack tests on adversarially trained models to explore the transferability of models trained on different distribution data. The experimental results indicate a certain degree of performance degradation in the proposed method, aligning with our initial assumptions. In subsequent work, we will concentrate on addressing the challenges posed by the perturbation transferability in scenarios with different distributions.  


4) We have introduced additional ablation experiments comparing models with and without pre-trained embeddings. The experimental results validate the effectiveness of the proposed pre-training operation. Furthermore, in Appendix B, we have included a formulaic representation of the methodology for utilizing pre-trained embeddings to provide readers with greater clarity.  


5) We have thoroughly reviewed the grammar, symbols, and formulas within the paper. We have made revisions to address the specific points highlighted by the reviewers. And we have subsequently polished the entire article. This was done to enhance the overall clarity of the content, enabling readers to better comprehend the paper.  


All additional experimental tables and figures have been documented in the attached PDF file. For the newly added results, we have utilized blue font for differentiation. Once again, we express our sincere gratitude for the reviewers' patient reading and highly valuable suggestions.

Meta-review: This paper studies the adversarial robustness of neural networks. Specifically, the authors aim to improve the targeted adversarial transferability, where adversarial examples for one "source" white-box model are used to fool another "target" model in a black-box fashion. The main insight is that optimizing perturbations towards easy examples located in the high sample density regions (HSDRs) improves targeted adversarial transferability. The paper contains both a theoretical analysis of why this is effective, as well as experiments on several source and target networks, including ResNet and Inception. The paper also contains toy examples (e.g., Gaussian mixture models) to demonstrate the approach.

Reviewers found many positive aspects to the paper. They found the general approach of using HSDRs to be compelling, intuitive, and effective. They also appreciated that the method can support multi-target perturbations. The reviewers had many questions about the experiments, but overall, they found that the new proposed algorithm (ESMA) to be quite effective and better than several prior works / baselines. The reviewers also found the method to be fairly practical and usable, given that the authors specifically explain how to implement it for high dimensional datasets (i.e., ImageNet-1k).

On the negative side, the reviewers had many questions about ablation studies and related work. For example, there is a question about whether the proposed method works without using pre-trained embeddings (this seems to be a key assumption). During the discussion, the authors added a variety of new experiments to (at least partially) address these concerns. Some other concerns were around tying claims about the density to the adversarial attack effectiveness, as well as trying out model ensembles. The authors have added new experiments to help illuminate these topics. For example, the authors have added new experiments to answer: "how the enhancement of transferability can be unequivocally attributed to HSDR rather than other contributing factors?"

In summary, the discussion was extremely fruitful, and assuming all the new experiments end up in the final version of the paper, the authors' claims seem to be very well supported. The original results already demonstrated significant improvements. Therefore, I recommend acceptance of the paper. 

I strongly encourage the authors to incorporate all of the additions that they mention during the discussion in the final version of the paper, both in the appendix and with some references in the main paper to improve easy identification of answers for the main reviewer concerns.
Decision: Accept (poster)