AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks
Authors: Jin Li
Ziqiang He
Anwei Luo
Jian-Fang Hu
Z. Jane Wang
Xiangui Kang
Venue: NeurIPS 2024 poster (2024)
Keywords: Adversarial Attacks
Imperceptibility
Diffusion Models
Deep Neural Networks
Primary area: diffusion_based_models

Abstract: Imperceptible adversarial attacks aim to fool DNNs by adding imperceptible perturbation to the input data. Previous methods typically improve the imperceptibility of attacks by integrating common attack paradigms with specifically designed perception-based losses or the capabilities of generative models. In this paper, we propose Adversarial Attacks in Diffusion (AdvAD), a novel modeling framework distinct from existing attack paradigms. AdvAD innovatively conceptualizes attacking as a non-parametric diffusion process by theoretically exploring basic modeling approach rather than using the denoising or generation abilities of regular diffusion models requiring neural networks. At each step, much subtler yet effective adversarial guidance is crafted using only the attacked model without any additional network, which gradually leads the end of diffusion process from the original image to a desired imperceptible adversarial example. Grounded in a solid theoretical foundation of the proposed non-parametric diffusion process, AdvAD achieves high attack efficacy and imperceptibility with intrinsically lower overall perturbation strength. Additionally, an enhanced version AdvAD-X is proposed to evaluate the extreme of our novel framework under an ideal scenario. Extensive experiments demonstrate the effectiveness of the proposed AdvAD and AdvAD-X. Compared with state-of-the-art imperceptible attacks, AdvAD achieves an average of 99.9% (+17.3%) ASR with 1.34 (-0.97) $l_2$ distance, 49.74 (+4.76) PSNR and 0.9971 (+0.0043) SSIM against four prevalent DNNs with three different architectures on the ImageNet-compatible dataset. Code is available at https://github.com/XianguiKang/AdvAD.

AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks
Jin Li', Ziqiang He', Anwei Luo', Jian-Fang Hu', Z. Jane Wang", Xiangui Kang' GGunngdogg Key Lab of Information Security, School of Computer Science and Engincering, Sun Yat-Sen University Electrical and Computer Engineering Dept, University of British Columbia
Abstract
Imperceptible adversarial attacks aim to fool DNNs by adding imperceptible pertur- bation to the input data. Previous methods typically improve the imperceptibility attacks by integrating common attack paradigms with specifically designed perception-baset losses or the capabilities of generative models. In this paper, we propose Adversarial Attacks in Diffusion (AdvAD), a novel modeling frame- work distinct from existing attack paradigms. AdvAD innovatively conceptualizes attacking as a non non--oranen parametric diffusion process by theoretically exploring basic modeling approach rather than using the denoising generation abilities of reg- ular diffusion models requiring neural networks. At each step, much subtler yet effective adversarial guidance is crafted using only the attacked model without any additional network, which gradually leads the end of diffusion process from the original image a desired imperceptible adversarial example. Grounded in a solid theoretical foundation of the proposed non- tan-parametric diffusion process, AdvAD achieves high attack efficacy and imperceptibility with intrinsically lower overall perturbation strength. Additionally, an enhanced version AdvAD-X is proposed to evaluate the extreme of our novel framework under an ideal scenario. Extensive experiments demonstrate the effectiveness of the proposed AdvAD and AdvAD-X. Compared with sate-offtthe-ar imperceptible attacks, AdvAD achieves an average of 99.9% (+17.3%) ASR with 1.34 (-0.97) l2 distance, 49.74 (+4.76) PSNR and 0.9971 (+0.0043) SSIM against four prevalent DNNs with three dif- ferent architectures on the compebil- dataset. Code is available at bttps://githhe wtt./oooo/////eeee hatm//iiiniKaag/Ak///.. AdvAD.
Introduction
Deep Neural Networks (DNNs) are shown to be vulnerable to adversarial attacks [], 2] (i.e., add maliciously crafted perturbations the input data), posing serious security concerns to real-world applications [3]. The research of adversarial attacks also plays an important role in proactively exposing potential threats, as well as promoting model robustness and corresponding defense methods [4-11]. Many attacks [12-15] focus on maximizing the attack success rate and transfcrability under relatively lenient restrictions (i.e.. Loo l2 norm) of adversarial perturbation, but they could have poor stealthiness and imperceptibilityy since the crafted adversarial examples can be easily detected by the Human Visual System (HVS) [16]. Therefore, imperceptible adversarial attacks [17-23], aiming to maintain attacking efficacy while improving imperceptibility, have attracted considerable attention.
Current imperceptible adversarial attacks could be summarized into categories: 1) perturbation- based attacks devised on perceptual characteristics, and 2) unrestricted attacks. The first one is motivated by the fact that adding adversarial perturbations different components of an image has
"Coresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

varying perceptual quality levels to the HVS. By studying components such as image color [19], texture complexity [21], frequency spectrum [23, 24], etc., these methods design corresponding perceptual-base loss functions and incorporate them to the optimization process to craft adversarial examples where the adversarial perturbation constrained and hidden within specific image regions. Instead of injecting noise-like adversarial perturbations, unrestricted attacks heavily but reasonably modify attributes of images like semantic content to perform attacks. Apart from early work that adopts GANs [25]. some recent methods combine the prevalent diffusion models [26-28] into the adversarial optimization process in an image edition-like way of repeatedly adding noise and denoising to eliminate the noise pattern within the final adversarial examples [29,30]
30] or optimize the embedding of latent diffusion models [31, 32]. However, due to the uncertainty of generative models and the unrestricted setting itself, some unrestricted adversarial examples inevitably exhibit obvious unnatural texture or semantic changes and lose the imperceptibilitty especially for images with complex content. Although previous methods have equipped attacks with imperceptibility utilizing various designs mentioned above, it remains an essential challenge of achieving imperceptible adversarial attacks: How attack with inherently minimal perturbation strength from modeling perspective?
To address this fundamental challenge, we propose Adversarial Attacks in Diffusion (AdvAD), a brand new modeling framework distinct from common attack paradigms of gradient ascending [2] or optimization with adversarial losses [17]. The proposed AdvAD explores a novel non parammemtttic diffusion process for attacks, which fully inherits two key merits of diffusion models: i) the modeling philosophy of converting a difficult task into a series of simple sub-tasks, and ii) solid theoretical foundation. Specifically, AdvAD achieves high attack efficacy with intrinsically lower perturbation strength by innovatively modeling the attack process as a decomposed diffusion trajectory from an initialized noise yet more effective (for attack performance) adversarial guidance is calculated and injected with two cooperating, theoretically grounded non-parametric modules called Attacked Model Guidance (AMG) and Pixel-level Constraint (PC), which gradually leads the end of this trajectory from the original image distribution desired adversarially conditioned distribution based on the theory of diffusion models (e.g., deterministic iiffusiin[27]] conditional sampling [33, 34], etc.).
adversarial example. At each step, a much subtler (for imperceptibility)
Here, we would like to clarify that the proposed diffusion process for attacks is considered as non -on-aammetrric since it does not require additional networks as needed in regular diffusion models for noise estimation. AdvAD firstly initializes fixed diffusion noise, which is then ingeniously manipulated at each step via the adversarial guidance crafted by the proposed AMG and PC modules using only the attacked model with theoretically derived equations. In this way, the proposed AdvAD is facilitated with the modeling approach of diffusion models rather than their denoising or generative capabilities, which avoids the negative impact like semantic content changes caused by the uncertainty of generative models and also promises relatively low computational complexity. Based on Adv we further propose an enhanced version AdvAD-X ('X' for 'eXtreme') with two extra strategies to squeeze the extreme performance in an ideal scenario of the proposed new modeling framework with unique properties, which also possesses theoretical significance and provides new insights for revealing the robustness of DNNs. In summary, our main contributions are:
Addressing the essential challenge of imperceptible adversarial attacks from a novel modeling perspective for the first time, we theoretically explore and derive the basic modeling of diffusion models to perform attacks with inherently lower perturbation strength through a non-parametric diffusion process that requires no additional networks.
We propose two attack versions, AdvAD and AdvAD-X. For the basic AdvAD, the AMG and PC modules cooperate to craft much subtler yet effective adversarial guidance which is progressively injected via initialized diffusion noise at each step. and AdvAD-X further reduces the perturbation strength to an extreme level in an ideal scenario with theoretical significance.
Extensive experiments are conducted to evaluate the effectiveness of our methods in terms of attack success rate, imperceptibility, and robustness. Experimental results demonstrate the superiority of the novel modeling approach for imperceptible adversarial attacks.
2 Preliminaries
Adversarial Attacks. Given an original image Fori with ground-truth label ygt and a classifier f(-) satisfying f(oor) Fgt. normal untargeted attacks aim to craft the adversarial example Dody that
2

---

misleads the classifier, formulated as:
f(wado) & ygt:
s.t.IlZadv - Eoriilp S,
(I)
where ||. lpp represents Ip-norm that is usually implemented with Loo -nom to limit the distance between Eade and Tors within an upper bound of budget Ç. In this paper, we focus on the more general setting of untargeted attacks. More information on related works is provided in Appendix A.
Deterministic Diffusion Process. the deterministic situation of DDIM [27] with or = 0. for an image Do and pre-defined diffusion coefficients oo:T E (0, 1 for step E [0: ?), my in the Forward process of adding noise to 2 is given by H = va,zo + vI - E where E ~ V I)) represents Gaussian noise. For the Backward denoising steps, unlike the DDPM [26] based on Markov chains that each state directly depends on the previous one, DDIM employs a non-Markovian approach. In the backward process, each step first involves calculating "prediction" of final step x!. from current Zt, then adding noise to it again to obtain 2t-1, expressed as:
E4
-0,00 (Tte) + /1 -1- 11-001))
04
(2)
where (2)))) is a estimated diffusion noise using a pretrained neural network 0 for current step, and the term in the first parenthesis represents the predicted æ.. derived by a simple variation of Eq. (2).
Conditional sampling. Song et al. [34] propose the conditional sampling technique for the score- based generative models with score function v p(x,) [35], kind of generative model has close relationship to diffusion models. Without loss of generality, for condition (e.g., class label, mask, etc.) and corresponding conditional distribution pl a (y) score- score-based model can sample from p( x by modifying the score function at each step of t to V Vz log(p xt (p(g(())) if p(1) (x)) is known. Subsequently, with the connection between the score function and the noise e, of diffusion models as Va, log p(æ;; = -1 1/vVT Or [34], this joint distribution could be expanded to the deterministic process of DDIM, achieved by updating the noise 6, to e, each step [33]:
e 6t vI
a+ V x, log p(ylar))
3 Proposed Adversarial Attacks in Diffusion
3.1 Overview
From novel modeling perspective, we propose Adversarial Attacks in Diffusion (AdvAD) to attack with inherently smaller perturbation strength through non non-parametri diffusion process for the first time. As shown in Figure 1, different from previous attack paradigms that employ gradient ascending or optimization with varying kinds of adversarial losses, AdvAD innovatively performs attack within decomposed non-parametric diffusion trajectory starting from an initialized noise, in which very subtle yet effective adversarial guidance is crafted and injected to gradually push the end of this trajectory to desired adversarially conditioned distribution from the original image.
Intuitively, given the original image Tori with an initialized Gaussian noise Eo ~ (0, I), fixed diffusion trajectory from Ø7 to &n (ān Zori) can be easily obtained using DDIM Backward for the deterministic diffusion process as:
KT Var + vI - aT
(4)
D.
VT- V1-0
04-160-
(5)
Vai
With this deterministic diffusion trajectory of the original image, performing adversarial attacks within requires solving two main problems: i) directing the final result of this diffusion process to desired adversarial example rather than the original image; ii) ensuring the modified trajectory (denoted as &;, é for step t) close to the original trajectory (Ø,, co for step f) of the clean image to achieve the imperceptibility of attacks. To fulfill the dual purposes, we propose two theoretically grounded modules, called Attacked Model Guidance (AMG) and Pixel-level Constraint (PC) to work together. At each step, AMG utilizes only the attacked model f(.) to produce the adversarial guidance without requiring any additional networks, synergistically collaborated with PC to constrain and streamline the diffusion process injected with the guidances.
3

---

AMG Atterled Mocki Cariknoo
PC
Powd-looe Corstrirt Iraiisione Enfiiatoon Neioos (()
Modd
Goomskorhh Labe
7 Segg
Start
Step:
Step:
Step:
End Expmple
Original Looge
Ire. Naw Eg.
Kn
Ey-11
R
Eig
X
Re43
Xadt 1mu
No
fron Porn--Prrmeeiicc Diffusion without Adiitiinal Networks for Egi
Step:
Actieved by Munipulating Inthioieed fa Fach Stop.
Ec+1l
Ec+l
..... >- x Adversorial Guidance
é
Sample of Diffusion
Predicted Result
with AMG
Next Sample
MarlMa Procsss
AMG
PC
Eq (7)
Eig (10)
R;
R
Figure 1: Overview of the proposed Adversarial Attacks in Diffusion (AdvAD) that models the attack as non-parametric diffusing process. At each step, Attacked Model Guidance (AMG) module adopts the non-Markovian process for approximating Lade using de to craft adversarial guidance and injects it into the initialized diffusion noise, then Pixel-level Constraint (PC) module imposes restriction produce the noise for the next step and serves to control the whole process precisely.
3.2 Attacked Model Guidance Module
By viewing the attack process as a toarbbutioon----sttiuutttt transformation through a non- parametric diffusion process, the proposed AMG module theoretically integrates the conditional sampling technique of diffusion models to craft the adversarial guidance only using the attacked model f(.). For untargeted attacks, the ultimate goal is modifying Tori with f(Eori) Vgt to Badv that f + Vgt. which can be regarded as directing the determined distribution p(Zors) of the original Vgt).
diffusion trajectory to distribution of Tads with the attacked model as (( Thus, we regard fI + Ygr as an adversarial condition, and employ the conditional sampling technique to the original trajectory by manipulating the diffusion noise to achieve this, expressed as:
E = En vI a₁ V log pf (aadv) + Uoa (.,)
(6)
-0 /I a Vv, log(1 p(f(xadv) yot k.)).
However, Eq. (6) is unsolvable since Lade is unknown during the diffusing process. To address this, inspired by the properties of deterministic non-Markovian DDIM that final diffusion result is firstly predicted each step, we calculate ae via the equation of DDIM non-Markovian process with ē from the previous step, and use it to approximate a Lddv expressed as:
/I a aq εl+1
Bady R t
Vot
The accurate error upper bound and convergence of this approximation are given Proposition in conjunction with the proposed PC module, and the validity of this approximation can also be explained intuitively from the premise of our method. That is, we have xς = Tors for all step t in the original diffusion trajectory, and the modified trajectory should be very close to the original one, so that the relationship between a! and Eade should satisfy 2! Badv.
With Eq. (7), the term of p( (WFad) = yoa (2) in Eq. (6) can be written as p( Myg (2,)) p(f( Ugt) since 4 is calculated from &t which is exactly the output logits of f(af) with Softmar(-) function for the class Vgt- Denoting this term as the classification probability of the attacked model as p (ygel (.)), we can obtain the solvable equation of AMG module that injects adversarial guidance the initialized diffusion noise using only f(-) without any additional network:
C AMG f()) Ugt) cu vI aj Va, log(1 p (Ugel (()))
(8)
At this point, in addition the benefits from modeling, this calculation process of AMG also plays role in endowing AdvAD with imperceptibility. As the attack progresses, the probability py, the term

---

Algorithm 1 AdvAD
Input: Attacked model f(), image Wori with label Vgt, budget step T;
Output:
Adversarial example zadv
1: Initialize pre-defined diffusion coefficients cxo:r E (0, 10.1)
2: Initialize Eo (O, I):
D Initialize and fix diffusion noise en- Align with data range of diffusion process. > Forward process of adding noise e, to Tors- > Non-parane parametric diffusion process.
Transform the range of wors to [-1, 11;
4: Calculate σr via Eq. (4);
5: Set &r := ær, 4r+1 == Eo:
6: = do
7: Calculate # via Eq. (7);
Approximation of # R aade- Align with data range of image. b Inject adversarial guidance. Constraint modified diffusion noise. One step backward from to 1. b Endpoint of the process. Return actual 8-bit image Dadv-
Transform the range of # to [0, 255];
9: Calculate & with AMG via Eq. (8);
10: Calculate & with PC via Eq. (10);
11:
Calculate - via Eq. (11);
Transform the range of &o to [0, 255]:
13:
return adde = int8 int8(round (round (f))
of log(1 - py) as well as coefficient v1 as gradually approach O, which means the strength of injected adversarial guidance gradually converge 0 in AdvAD, while common classification losses (e.g, Cross-Entropp Log Loss, etc.) used in other attack paradigms may increase on the contrary. Further analysis and experiments on this property are provided in Proposition and Sec. 4.5.
3.3 Pixel-level Constraint Module
Collaborating with AMG, the PC module is introduced to impose precise control and streamline the modified diffusion trajectory for attacks. A straightforward choice is to design PC for & that constrains each & using T., thus ensuring æe close to o and the final Tade close to Cori- However, such a "hard" constraint directly applied to & will impair the effectiveness of AMG and disrupt coherence of the transforming trajectory. Therefore, we formulate a more suitable PC for é in Theorem 1.
Theorem Given diffusion coefficients u0 E 1|", the Tori, 3t. 6n from the original trajectory, Rr & from the modified trajectory, and a variable E, if and eo satisfies
Var
- S V1 11-07 OT
E,
(9)
for all E [T' 1), then it follows that ||ēt- Telloc S (Vai- V1- as vI-ar
yar
), ||
Zorillas
S
8, and [00- - Zorslloo~ S & hold true.
According to Theorem 1, the PC for é is implemented
é, PC(()) =Pice (co. (E))
(10)
where Piue (c.e) is a projection operation that constrains the output ē of AMG(-) to é; based on a loo -orm ball of cu to satisfy Eq. (9). After PC, the diffusion noise é for next step is obtained, and the e.-1 can be calculated using the deterministic DDIM backward equation as:
f
/ 1
(11)
The elaborate PC for & directly cooperates with AMG to constrain the diffusion noise, which streamlines the whole diffusion process and can serve to simultaneously control the terms of &,, &6, and e.. satisfying the premise that two trajectories are close and ensuring the effectiveness of AdvAD. The complete pseudo code of AdvAD is provided in Algorithm 1.
Subsquently, based Theorem further give propositions about AdvAD as:
Proposition 1 Under the conditions of Theorem 1, by denoting constrained è; Eu -g we have
3 Lodu = Bori
X
(12)
5

---

where d
-I- Vae
VI-ar1 var-i
and ||0:1bo S
for 7n-a7
Proposition 2 Under the conditions of T'heormm 1, the upper bound on the error of the approximation in Eq. (7) can be expressed
Dade 21 S 2. VI Vot V1 0T
V
(13)
Proposition I explicitly states the much subtler and decreasing strength of the adversarial guidance injected at each step of AdvAD's non-parametric diffusion process, and also allows for a quantitative analysis (as in Sec. 5.5). Proposition indicates the validity and convergence of the approximation of Bado a in AMG (Eq. (7)). It is evident that as t goes from T to or increases from to 1, the upper bound on the approximation error rapidly converge from 2 to O. The detailed derivations of the mentioned PC for e,, proofs of Theorem and Proposition 1, 2 are provided in Appendix B.
3.4 AdvAD to AdvAD-X: Extreme Version
Building upon AdvAD, we further propose scheme called AdvAD-X ('X' for 'eXtreme") with two extra strategies called Dynamic Guidance Injection (DGI) and CAM Assistance (CA), aiming to squeeze the extreme performance of our novel modeling framework in an ideal scenario that is usually overlooked but has theoretical significance.
DGI and CA Strageties. As aforementioned, the attack capability of AdvAD comes from the very subtle yet effective adversarial guidance crafted by AMG and PC, and the intensity of guidance will decrease to 0 as the process progresses. Thus, the DGI is naturally emerged as dynamic skipping stragety skip the unnecessary calculation and injection of adversarial guidance, especially for those steps in the later process. With DGI, AdvAD-X dynamically avoids the execution of AMG and PC and adopts original Eo as the diffusion noise for the steps where the 2 x Lady is already able to mislead the attacked model, reducing the accumulated guidance strength as well as the computational complexity. On the other hand, inspired by the Class Activation Mapping (CAM) [36] identifies critical regions of an image about a decision made of a classifier, our CA strategy calculates a mask (if available) m ranging from 0 to 1 of Zori with (-) and Vgt using GradCAM [37] to further suppress the strength of adversarial guidance within the non-critical image regions in those steps that are not skipped. The equation of AMG with CA strategy can be modified as:
? = Co m vI 04 V V., log(1 (1 - Py (y (2))
(14)
Ideal Scenario. Equipped with DGI, AdvAD-X omits large number of adversarial guidance that is injected by default in AdvAD, while the absolute strength of guidance in each of the remaining steps also suppressed by CA, successfully reducing the final adversarial perturbation an extreme level. This extreme case leads to a problem that in the default setting of attacking with 8-Bit RGB images, the adversarial perturbation of pixels where the intensity is less than 0.5 will be erased due to the quantization. However, in practice, the input of DNNs is normalized as floating-poin data type to avoid gradient problems during training [38, 39J, and white-box attack allows access to the entire of DNNs. Therefore, for AdvAD-X we specifically consider an ideal scenario that directly input the raw final adversarial example in floating -ointt data to DNNs without quantization evaluate the extreme performance of AdvAD-X The pseudo code of AdvAD-X X is provided in Appendix C.
4 Experiments
4.1
Experimental Setup
Dataset. In line with prior studies [15, 19, 32, 40], our experiments are conducted on the ImageNet- compatible Dataset containing 1,000 images of ImageNet [41] classes with size of 299 299, and the images are resized to standard input size of 224 x 224 in all experiments. Models. We select the widely used CNNs of ResNet-50 [42] and enhanced ConvNeXt-Bace [43], Swin Transformer-Ie
htttp://githhbb con/ cleverhans 1ab/ cl cleverhans/ /tree wwwee//eaeeeeeeeeereeeeeeeeeeeee wew.ettterr/eeeeaaa.. v3. eexamples/niieeeeee adversarial, ctttrrsarraa_copetttiiiio conpetitionn dataset

---

Table 1: Results of untargeted white-box attack success rate (ASR) and other evaluation metrics for imperceptibility when employing different attacks and attacked models. The reported running times obtained using RTX 3090 GPU same machine. t and blue mean the results of AdvAD-X are obtained with floating-poin data type in the ideal scenario as described in Sec 3.4.
Model
Artack Method
Time (6)
ASR (%6
Lx
La
PSNR
SSIM
FID 1
LPIPS L MUSIQ
PGD [2
25
98.6
0.031
8.17
33.53
0.8830
35.25
0.0517
52.24
NCF [40
2739
89.9
0.783
75.16
14.79
0.6374
58.99
0.3052
12.12
ACA [32
82239
89.8
0.839
52.42
18.00
0.5659
69.57
0.3381
55.47
ResNet-50
DillAllack |31]
34954
96.6
0.743
30.51
22.63
0.6750
55.29 25.29
0.1130
55.67
[42]
DiPPGD 301
6057
92.1
0.246
11.43
30.95
0.8902
22.18
0,0315
53.05
AdyDrop 21
193
96.8
0.062
3.17
41.91
0.9872
5.57
0.0061
$4.96
PerC-AL [9]
4085
98.8
0.131
2.05
46.35
0.9894
8.62
0.0029
$5.84
SSAH
428
99.7
0.033
2.65
43.73
0,9911
4.48
35,49
AdvAD (ours)
2201
99.7
0.010
1.06
$1.84
0,9980
2.42
0,0005
$6.35
AdvAD-X (ours)
806
100.0
0.002
0.34
63.62
0.9997
0.23
0,0001
$6.59
PGD [2]
127
99.9
0.031
7.98
33.74
0.8845
32.03
0.0386
$1.85
NCF 40
5222
59.4
0.750
72.89
15.10
0.6616
50.52
0 0.2844
70
ACA B2
81149
82.2
0.835
$2.16
18.05
0.5676
68.45
O 0.3421
55. 11
Nenx
Dil'Atack
DinPGD 30]
[3]
35417
6.125
97.8
76.9
0.754
0.245
31.70
11.45
30.94
28
0.6610
0.8908
21.05
72.22
0.1277
0.0306
$4.75
$4.80
-Base [43]
DroD 21
K38
96.9
0.057
3.26
41.69
0.9864
6.42
0.0055
$4,80
ParC-LL [9]
18271
10.3
SSAH [24]
3423
84.6
0.026
2.24
45.19
0.9928
3.04
0.0011
55.78
AdvAD (ours)
15240
100.0
0,016
1.49
48.61
0.9964
5.07
0.0009
55.97
AdvAD-X (ours)
5245
99.8
0.004
0.64
58.01
0.9993
0.62
0.0001
56.43
PGD [2]
98.5
0.031
7,85
33.88
0.8861
21.5 34
51.91
NCF 40
4690
63.7
0.733
69.92
15.48
0.6822
47.17
0.2709
49.77
ACA [32
83706
79.6
0.831
50.70
18.31
0.5157
64.83
0 0.3341
55.65
Swin Trans,
DifPGD [0]
DiffArtack [3]]
36736
6499
69.1
89.7
0.244
0.741
11,26
30.45
31.10
22.67
0.8945
0.6727
53.32
16.19
0.0276
0.1 143
$5.25
55.72
-Base [44]
AdvDrop
623
97.2
0.063
3.37
41.43
0.9853
5.22
O 0.005S
$4.73
ParC-AL
15258
95.6
D. 1.144
2.15
45.93
0.9882
3.33
0.0015
06
SSAH [24]
17.77
96.3
0,035
2.41
44.60
0.9927
2.57
0.0010
$5.53
AdvAD (ours)
9729
100.0
0.013
1.19
50.57
0.9978
1,70
0,0004
56.17
AdvAD-XT (ours)
5243
99.7
0.005
0.52
60.29
0.9995
0.25
0.0001
56.47
7.99
$2.37
0.3329
DillAtack
VisionMannhu
DiPGDD
11.75
30.68
0.8845
Small [46]
AdvDrop
97.0
0.076
4.42
39.30
0,9761
8.02
PerC-AL
10400
6.5
SSAH [24]
1204
49.8
0.028
1.95
46.41
0.9946
2.08
0.0018
55.96
Ad+AD (ours)
6154
99.7
0.016
1.62
47.94
0.9960
3.6Z
0.0017
56.17
AdvAD-Xt (ours)
4021
99.4
0.005
0.69
58,90
0,9989
0.51
0.0004
56.50
[44] with Transformer [45] architecture, and VisionMamba--Snin [46] with the recently emerged advanced Mamba [47] architecture. Attacks. We choose classic PGD [12] and seven attacks that claim having imperceptibility comparison methods, including normal imperceptible attacks of AdyDrop [21], PerC-AL [19], SSAH [24], and unrestricted attacks of NCF [40], ACA [32], Diff'ttack [31], Dif [30], and the generative capability of diffusion models are utilized the last three attacks. For our proposed AdvAD and AdvAD-X, we set & = 8/255 and T 1000 for all experiments unless specifically mentioned. All the other comparison methods are evaluated The results of AdvAD-X
using their official open-source code with the default hyper-parameneeeeeeee are obtained in the ideal scenario with float-pointing raw data as described in Sec. 3.4. Evaluation Metries. Attack success rate (ASR) is used to evaluate the attack efficacy, and seven metrics are adopted to comprehensively assess the imperceptibility. including l2 and loo distances for absolute perturbation strength; Peak Signal-to-Noiie three network-based metries, i.e., Learned Perceptual Image Patch Similarity (LPIPS) [49], Fréchet Inception Distance (FID) [50], and a non-reference metric MUSIQ [51] for image quality.
Ratio (PSNR), Structure Similarity (SSIM) [48], and
4.2 Comparison with State-of-the-art Methods
White-Box Attacks. Table 1 reports the untargeted attack performance and imperceptibility of ten methods against four attacked models. It is evident that the proposed AdvAD with novel modeling framework consistently demonstrates superior performance in terms of both ASR and imperceptibility. For the normal imperceptible adversarial attacks, the absolute adversarial perturbation strength of AdvAD in 1x and l2 distance are only 0.014 and 1.34 in average, which about half of the state-of- the-art restricted imperceptible attack SSAH, and AdvAD maintains almost 99.9% ASR, supporting our key idea that it inherently reduces the strength of perturbation required for attacks from a modeling perspective. When attacking more advanced models from ResNet to VisionMamba, AdvAD always demonstrates the best ASR and imperceptibility. yet other methods tend have some performance degradation (c.g., PerC-AL and SSAH for ConvNeXt and VisionMamba). For unrestricted attacks,
7

---

Figure 2: Visualizations of adversarial examples and corresponding perturbations crafted by nine imperceptible attacks. Perturbations are amplified as marked in top-right for the convenience of observation. Please zoom observe the details of the images with original resolution of 224 x 224.
Table Results of ASR against defenses for robustness evaluation, including three post- pess-ppocessing purification methods and four adversarial training white-box robust models.
Post Purifications (Normal Res-50)
Attack Adversarial Training Model
Allack Method
NRP
DS
Diflusion
Inc-V3
Res-50
Swin-B [S3]
ConvNeXI-B
All Avg.
[4]
[52]
[5]
Avg.
[8]
[9]
[S3]
Avg.
AdvDrop [1]
50.2
30.1
37.1
39.1
93.7
72.4
31.2
37.3
58.7
50.3
Pe [19]
30.3
28.8
25.4
28.2
99.9
46.1
8.2
7.0
40.3
35.1
SSAH [24]
25.6
28.0
11.0
21.3
91.2
84.6
16.8
47.4
60.0
43.5
AdvAD (ours)
51.5
29.5
31.2
37.4
98.9
79.3
60.2
62.7
75.3
59.0
AdvAD-X (ours)
13.4
27.6
10.2
17.1
57.2
45.2
18.0
16.2
34.2
26.8
is expected for them to perform poorly in the quantitative metrics, but if the results are poor for all image quality metrics, it usually indicates that the images damaged. Meanwhile, since the optimizer may not find the global optimal solution, the optimization-banl based methods tent to show sub-optimal ASR. For AdvAD-X, surprisingly, the perturbation strength reduced to an extremely low level with still high attack efficacy in the ideal scenario with floating-pont raw data.
Visualization. The visualizations of adversarial examples againt ResNet-50 in Figure 2 clearly show the characteristics of different imperceptible attacks against ResNet-50. For the first image with a relatively simple and clear object, the unrestricted attacks of NCF, DiffAttack and ACA perform attacks by modifying the semanties fairly, while DifiPGD uses denoising to avoid significant semantic modifications, but often has lower ASR as in Table However, for the image with complex content, the unrestricted attacks result in obvious unnatural color, texture, artifacts and semantic changes. For the normal attacks with perceptual- -based restrictions, by amplifying the noises, can be seen that AdvDrop has obvious gridding effect due the blocking operation in DCT operation, and the perturbation strength in PerC-AL and SSAH is also related to the edge or texture components of the image. In contrast, our AdvAD continuously maintains uniform and lower perturbation which is very difficult to be seen even in the adversarial examples with x5 noise. For AdvAD-X, the perturbations are very slight modifications to the decimal places of the floating-point it is still difficult to be seen even after x 100 magnification. More quantitative comparisons and visualizations are provided in Appendix D.I, D.2.
data for each pixel, thus
4.3 Robustness
The robustness of attacks is also evaluated against defense methods, including purification methods of
NRP [4], DS [52], diffusion- purification [5] and adversarial training robust models of Inc- V3 [8], Res-50 [9], Swin-B [53], ConvNeXt-B [53]. Two classic image transformation defenses of JPEG compression [54], Bit-depth reduction [55], and another type of defense, random smoothing [1 I], are also included. Considering the robustness and transferability of attacks are comparable only under close perturbation budget, the unrestricted attacks are not included in this and the next section.
8

---

type
Clean Part-A
AdvÁD AdxDiop
AdAID-X
SCAM
Table 3: Results of imperceptible attacks against random smoothing defense. Adversarial examples are crafted using only the base model, and then 100 rounds of random smoothing are applied to obtain the final ASR, a the variance of smoothing noise.
100.0
100
76-
75-
ASR/%)
ASR
0.25
0.50
1.00
ASRT
12
ASR1
12b
ASR1
12
Bi-depth Reductoon
Compression
clean
17.3
30.3
46.8
AdvDrop
25.2
5.97
33.5
6.21
48.
5.61
Figure 3: Rubostness on JPEG compression and Bit-depth reduction with different factors.
SSAH
21.8
13.84
32.4
14.82
46.9
13.68
AdvAD (ours)
28.2
2.41
36.8
2,51
50.4
2,08
Table 4: Transferability and effect of T of the proposed AdvAD. * means white-box ASR.
Model
Attack Method
Res-50
Mob-V2
Inc-V3
VGG-19
12 4
PSNR 1
SSIM
FID. 4
LPIPS 1
SSAH [24]
99.7"
15.5
20.4
2.65
43.73
0.9911
4.48
0.0021
AdvAD (7=1000)
22.6
15.1
1.06
$1.84
0.9980
2.42
0.0005
Ros-50 [42]
AdvDrop [2]]
96.8
17.3
23.1
15.8
3.17
41.91
0.9872
5.57
0.0061
PerC-AL [19]
98.8"
22.4
23.8
17.4
2.05
46.35
0.9894
8.62
0.0029
AdvAD (T=000)
100.0"
23.5
24.9
19.9
1.97
46.04
0,9912
7.15
0.0426
PGD [12]
98.6"
41.4
36.7
36.0
8.17
33,53
0,8830
35.25
0.0817
AdvAD (T=10)
100.0"
44.3
37.6
42,9
7.21
34.63
0,9015
30.84
0.0547
7.7
97.8"
19.8
45.24
2.95
Mob V2
AdvAD (7=1000)
9.7
99.7
21.3
14.8
0.94
$3.08
0,9982
1.46
0.0004
[50]
AdvDrop [2]]
9.7
97.7"
22.7
15.0
3.16
41.94
0,9873
4,88
0.00644
PerC-AL [19]
12.7
99.8"
23.3
17.8
2.16
45.67
0,9879
8.77
0.0032
AdvAD 100)
12.2
100.0"
23.4
17.9
1.83
46.68
0,9919
4,73
0.0020
PGD [12]
29.9
99.9*
35,3
37.9
8.29
33.41
0,8803
34.57
0,0500
AdvAD (T=10)
30.6
100.0
35,3
7.23
34.60
0.9006
27.25
As shown in Table 2, the proposed AdvAD demonstrates the best robustness in overall average compared with other imperceptible attacks of AdvDrop. PerC-AL and SSAH. Specifically, when at- tacking robust models, AdvAD achieved an much higher average ASR of 75.3%. For post-processing purifications aim at eliminating adversarial perturbations, despite the inherently lower perturbation strength, AdvAD still maintains the best or second-best ASR against different purifications, which is comparable to AdvDrop with much higher perturbation strength. Similarly, for the results of classic image transformation defenses in Figure 3. Adv also exhibits advantages in most of the factors. In addition, since random smoothing is not a truly end-to-end method but method that uses the base model make multiple predictions on aaggeeed images, we adpot a semi-white-bor setup to fully test the attack performance as described in the caption. Table 3 shows the experimental results, and the PerC-AL is not included because it fails to attack in this setting. can be seen that for all Ø, our AdvAD continuously achieves the best ASR with smaller perturbation strength.
We suppose the robustness of AdvAD mainly benefits from two aspects. Firstly, AdvAD performs attacks during unique non-parametric diffusion process with adversar- ial guidance, which may be casier to break through ex- isting adversarial training models using common attack paradigms. On the other hand, the inherently lower per- turbation crafted by AdvAD spread across the images more uniformly rather than gathering in some areas can be seen in the visualization, making more difficult to be eliminated. For AdvAD-X, it is anticipated to exhibit weak robustness since the extremely low perturbation in
(a) Effect step Adu
heeeage Stecktte ASR15)
nee Am 19
& 12 1
(b) The wwwerabilltttyy..oo Imperceptib Iit Relationship Atracks
16
the ideal scenario is easy to defense.
31.3 25.0
373
40.0
42.5
43.0
47.5
20.0
325
4.4 Transferability and Effect of Step T on AdvAD
Figure 4: More results of (a) effect of step T on AdvAD and (b) transferability- imperceptibility relationship of attacks.
Table 4 reports the ASRs of black-box attacks and the corresponding results of imperceptibility. We also test AdvAD with different step of T for comprehensive evalua- tion. Consistent with the diffusion models, larger T denotes a finer decomposition granularity of the entire process, corresponding to the strength of adversarial guidance at each step. Thus, AdvAD with a larger T exhibits better imperceptibilityy while a smaller implies stronger black-box transferability. Notbly, though there is clear negative correlation between imperceptibility and transferability, our

---

1.6-
A
56-4-
16
Table 5: Results of AdvAD and AdvAD-X with
36-4-
smaller & against Res-50. Step T is fixed as 1000.
value
Pa-4
0.5
8
Atiack
ASRT
lab
PSNRT
SSIMT
FID
0o-4-
AdvAD
98.6
0.93
53.27
0.9986
1.78
0.0
4/255
AdvAD:Xt
100.0
0.29
65.07
0.9998
0.18
1000
750
200
255
1000
750
200
250
AdvAD
96.1
0.82
54.85
0.9989
1.33
2/255
AdvAD-xt
99.4
0.27
65.95
0.9998
0.15
Figure 5: Values of De (left) and ||62||oo (right) of Eq. (12) throughout the diffusion process.
AdvAD
87.4
0.66
57.87
0.9593
0.77
1/255
AdvAD.XT
94.8
0.26
66.42
0.9998
0.14
AdvAD exceeds all comparison attacks in both of transferability and imperceptibility at different comparable levels, demonstrating the effectiveness of the proposed novel modeling framework.
To further elaborate the relationship between transferability and imperceptibility of AdvAD, as well as the optimal trade-off in practice, we plot two line graphs in Figure 4 under more values of T' As shown in Figure (a), as the value of T on the horizontal axis changes, the relationship between imperceptibilityy and transferability shows a clear proportional trend as mentioned above, consistent across different surrogate models. For the optimal trade-off, we consider that the intersection point of the two curves represents balance between imperceptibility and transferability. Accordingly, for the ResNet-50 and MobileNetV2 models, the optimal values of T are 50 and 25, respectively. Moreover, Figure 4 (b) illustrates more direct curves of this relationship and the positions of other comparison methods within Note that, all the other comparison methods are located to the lower left of the curve of AdvAD. This indicates that our method consistently achieves the best results in both transferability and imperceptibility compared with other state-of-the-ard restricted imperceptible attacks, demonstrating the effectiveness of our AdvAD as new attack framework with flexibility through the proposed non- con-parmmetric diffusion process.
4.5 Analysis
Eq. (12) in Practice. With the derived analytical formulation of Proposition 1, in Figure 5, we illustrate the actual values of X and ||6:||>> of Eq. (12) using 100 randomly selected images. While Proposition 1 indicates that the upper bound of || is invariant with respect to step 4, the actual strength of the adversarial guidance produced by AMG rapidly decreases as the process progresses, which validates the unique property given at the end of Sec. 3.2. With the similarly decreasing Å the whole term of AeIlö, loc representing oe distance of the guidance at step t also decreases from about 0.0008 to O, supporting that the proposed modeling framework performs imperceptible attacks with inherently small perturbation strength. Performance with Smaller ç. The results of AdvAD and AdvAD-X with smaller & for PC module are shown in Table 5. As decreases from 8 to 2, the imperceptibility is naturally improved because of the upper bound of perturbation becomes lower, yet the ASR of 94.8% only drops slightly. When 5 1/255, AdvAD still holds 87.4% ASR with 57.87 PSNR and 0.9993 SSIM, which means a large number of examples still can fool the DNN with maximum of +1 modification for each pixel, demonstrating the effectiveness of the adversarial guidance injected in the proposed diffusion process for attacks. Moreover, we provide the ablation study of AdvAD-X and additional discussions in Appendix D.3, D.4.
5 Conclusion and Outlook
In this paper, we propose a novel, fundamental modeling framework distinct from existing paradigms to tackle the challenge of imperceptible attacks. By exploring and deriving basic theory of diffusion models, the proposed AdvAD performs attacks through a non -araameeiiiic diffusion process with adversarial guidance, achieving inherently lower overall perturbation strength with high attack efficacy from modeling perspective. Besides, the proposed AdvAD-X evaluates the extreme of this novel modeling framework and further reduces the perturbation strength to an extremely low level in an ideal scenario. Extensive experimental results support the effectiveness and progressiveness of the proposed methods. Beyond imperceptibility, AdvAD holds the potential to become general and extensible attack paradigm thanks to the solid theoretical foundation and the innovative, controllable iuffusion-based process for attacks. In addition, we also hope the new observation that AdvAD-X can successfully attack with extremely small perturbation using floating-pon point raw data can bring inspiration for revealing the robustness and interpretability (e.g., decision boundaries) of DNNs.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by NSFC (Grant No. 62072484), Natural Science Foundation of Guangdong Province (Grant No. 2514050000889) and Guangdong Key Laboratory of Information Security (No. 202381212060026,,
References
[1] C. Szegedy. Zaremba. Sutskever, Bruna, Erhan, Goodfellow, and R. Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
[2] Goodfellow, Shlens, and Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations. 2015. 2015.
X. Yuan, P. He, Q. Zhu, and Li. Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems, 30(9):2805-2224, 2019.
M. Nascer, S. Khan, M. Hayat, F. S. Khan, and Porikli. A self-supervised approach for adversarial robustness. Proceedings of IEEE/CVF Conference on Computer Vision and Puttern Recognition. pages 262-271, 2020. 2020.
[5] Lee and Kim. Robust evaluation diffusion- based adversarial purification. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 134- 144, 2023.
[6] N. Singh, F. Croce, and M. Hein. Revisiting adversarial training imagenet: and generalization across threat models. Advances in Neural Information Processing Systems, 36, 2024.
Architectures, training
A. Luo, C. Kong, Huang, Y. Hu, X. Kang, and A. Kot. Beyond the prior forgery knowlcdge: Mining critical clues for general face forgery detection. IEEE Transactions on Information Forensics and Security, 19:1168-1182, 2023.
[8] F. Tramèr. A. Kurakin, N. Papernot, I. Goodfellow, Boneh, and McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Reprexentations, 2018.
H. Salman, A. Ilyas, L. Engstrom, A. Kapoor, and Madry. Do adversarially robust imagenet models transfer better? Advances Neural Information Processing Systems, 33:3533-3544 2020.
[10] A. Luo, E. Li, Y. Liu, X. Kang, and Z Wang. capsule network based approach for detection of audio spoofing attacks. In /CASSP 2021-202/ IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6359-6363. IEEE, 2021.
[11] Cohen, E. Rosenfeld, and Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pages 1310-1320. PMLR, 2019.
[12] A. Madry, A. Makelov, Schmidt, D. Tsipras, and Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018
[13] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9185-9193, 2018.
[14] Y. Zhang, Y. Tan, T. Chen, Liu, Q. Zhang, and Li. Enhancing the transferability adversarial examples with random patch. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCA1-22, pages 1672-1678, 2022.
[15] Z. Wei, J. Chen, Z. Wu, and Y. Jiang. Enhancing the universiiyyyyliity for transferable targeted attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12281 - 12290, 2023.
[16] M. Sharif, L. Bauer, and M. Reiter. On the suitability of Ip-norms for creating and preventing adversarial examples. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition Workshops, pages 1605-1613, 2018.
[17] N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pages 39-57. IEEE, 2017.
[18] B. Luo, Y. Liu, L. Wei, and Q. Xu. Towards imperceptible and robust adversarial example attacks against neural networks. In Proceedings the AAAI Conference Artificial Intelligence, volume 32, 2018.
11

---

[19] Z. Zhao, Z. Liu, and M. Larson. Towards large yet imperceptible adversarial image perturbations with perceptual color distance. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1039-1048, 2020.
[20] C. Laidlaw, S. Singla, and Feizi. Perceptual adversarial robustness: Defense against unseen threat models. In International Conference on Learning Representations, 2021.
[21] R. Duan, Y. Chen, D. Niu, Y. Yang, A. Qin, and Y. Hc. Advdrop: Adversarial attack to dnns by dropping information. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7506-7515, 2021.
[22] Z. Chen, Wang, Huang, W. Zhao, Liu, and Guan. Imperceptible adversarial attack via invertible neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 414-424, 2023.
[23] S.Jia, C. Ma, T. Yao, B. Yin, S. Ding, and X. Yang. Exploring frequency adversarial attacks for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2022.
[24] C. Luo. Q. Lin, Xie, B. Wo, Xie, and Shen. Frequency- drivee imperceptible adversarial attack on semantic similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pateenn Recognition, pages 15315-15324, 2022.
[25] Y. Song. R. Shu. N. Kushman, and S. Ermon. Constructing unrestricted adversarial examples with generative models. Advances in neural information processing systems, 31, 2018.
[26] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 33:6840 6851, 2020.
[27] Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Reprexentations, 2020.
[28] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Cision and Pattern Recognition, pages 10684-10695, 2022.
[29] X. Chen, X. Gao, Zhao, K. Ye, and C. Xu. Adydiffuser: Natural adversarial example synthesis with diffusion models. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 4562-4572, 2023.
[30] H. Xue, A. Araujo, B. Hu, and Y. Chen. Diffusion-basse adversarial sample generation for improved stealthiness and controllability. In Thirty-seventh Conference Neural Information Processing Systems, 2023.
[31] Chen, H. Chen, Chen, Y. Zhang, Z. Zou and Shi. Diffusion models for imperceptible and transferable adversarial attack. arXiv preprint prxiv:2305.08999 2023.
[32] Z. Chen, B. Li, S. Wu, K. Jiang. S. Ding, and W. Zhang. Content-based unrestricted adversarial attack. Advances in Neural Information Processing Systems, 36, 2024.
[33] P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8777 809488994422 2021.
[34] Y. Song, J. Sohl-Dickstein, Kingma, A. Kumar, Ermon, and B. Poole. Score-based generative mod- eling through stochastic differential equations. International Conference on Learning Representations. 2020.
[35] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[36] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2016.
[37] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE International Conference Computer Vision, pages 618 62666,262222 2017.
12

---

[38] A. Krizhevsky. I. Sutskever, and G. E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[39] S. loffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448 456. pmir, 2015.
[40] S. Yuan, Q. Zhang, L. Gao, Y. Cheng, and J. Song. Natural color fool: Towards boosting black-box unrestricted attacks. Advances in Neural Information Processing Systems, 35:7546-7560, 2022.
[41] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Sathcesh, S. Ma, 7. Huang, A. Karpathy, A. Khosla, M. Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-22 15-22. 2015.
[42] K. He, X. Zhang, S. Ren, and Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer VVision and pattern recognition, pages 770-778, 2016.
[43] Z. Liu, H. Mao, C. Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976-11986, 2022.
[44] 7. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, 7. Zhang. S. Lin, and Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012-10022, 2021.
[45] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[46] L. Zhu, Liao, Q. Zhang, X. Wang, Liu, and Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In International conference on machine learning, 2024.
[47] A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint 12.00223 2023.
[48] Z. Wang, C. Bovik, H. R. Sheikh, and E, Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4))600-612, 2004.
[49] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unrcasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018.
[50] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[5i] J. Ke, Q. Wang, Y. Wang, P. Milanfar, and F. Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148-5157, 2021.
[52] H. Salman, M. Sun, G. Yang, A. Kapoor, and 7. Kolter. Denoised smoothing: A provable defense for pretrained classifiers. Advances Neural Information Processing Systems, 33:21945-21955 2020.
[53] C. Liu, Y. Dong, W. Xiang, X. Yang, H. Su, J. Zhu, Y. Chen, Y. He, H. Xue, and S. Zheng. A comprehensive study on robustness of image classification models: Benchmarking and rethinking. arXiv preprint arXiv:2302. 14301, 2023.
[54] N. Das, M. Shanbhogue, S. Chen, Hohman, S. Li, L. Chen, M. E. Kounavis, and D. Chau. Shield: Fast, practical defense and vaccination for deep learning using jpeg compression. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 196-204, 2018.
[55] C. Guo, M. Rana, M. Cisse, and I.. van der Maaten. Countering adversarial images using input transforma- tions. In International Conference on Learning Representations, 2018
[56] Sandler, A. Howard, M. Zhu, A. Zhmoginov, and Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4510-4520 2018
13

---

Contents
1
Introduction
2
Preliminaries
3
Proposed Adversarial Attacks Diffusion
3.1
Overview
3.2
Attacked Model Guidance Module
3.3
Pixel-level Constraint Module
3.4
AdvAD to AdvAD-X: Extreme Version
4
Experiments
4.1
Experimental Setup
4.2
Comparison with State-of-the-art the-art Methods
4.3
Robustness
8
4.4
Transferability and Effect of Step on AdvAD
4.5
Analysis
10
5
Conclusion and Outlook
10
Related Work
15
B Derivations and Proofs
15
B.1 Straightforward PC for d
15
B.2 Proof of Theorem
16
B.3 Proof of Proposition 1
18
B.4 Proof of Proposition
20
C Algorithm of AdvAD-X
21
Additional Experiments
21
D.1 Additional Quantitative Comparisons
D.2 Additional Visualizations
22
D.3 Ablation Study of AdvAD-X
22
D.4 Additional Discussions
14

---

A Related Work
Beginning with the attack paradigm of Fast Gradient Sign Method (FGSM) [2], there are numerous great works focusing on the imperceptibility of adversarial attacks have been proposed [17-23]. In contrast to another line of attacks aimed at improving the attack success rate and the transferability for black-box models with a more lenient limitation of perturbation strength [13-15], imperceptible adversarial attacks are dedicate to accomplish attacks using as minimal perturbation as possible while deceiving human perception. Among them, PerC-AL [19] improves the imperceptibility by alternating between the classification loss and perceptual color difference when updating perturbations. AdvDrop [21] uses Discrete Cosine Transform (DCT) to discard details in images that are imperceptible for humans. SSAH [24] limits perturbation to high-frequency components using Discrete Wavelet Transform (DWT) to make it undetectable. Similarly, AdvINN [22] also utilizes the DWT and exploits invertible neural networks to specially perform targeted attacks. In addition, with unrestriced setting [25], some recent works have incorporated the capabilities of generative models (e.g., diffusion models [26]) into common attack frameworks to make adversarial examples
natural and enhance
the imperceptibility. DiffAttack [31] and ACA [32] combine the optimization of adversarial losses with the Stable Diffusion [28] generate unrestricted adversarial examples, while Diff-PGD [30] and AdvDiffusee [29] incorporate the classic PGD method [12] into the diffusion steps to make the adversarial examples undergo denoising processing.
Compared to these traditional restricted imperceptible attacks or the recent unrestricted imperceptible attacks (e.g., diffusion- diffusion-basedl), the proposed AdvAD is completely novel approach distinet from existing attack paradigms. It the first pilot framework which innovatively conceptualizes attacking non-pon non-parametric diffusion process by theoretically exploring fundamental modeling approach of diffusion models rather than using their denoising or generative abilities, achieving high attack efficacy and imperceptibility with intrinsically lower perturbation strength. Following the setting of restricted attack, the modeling of AdvAD is theoretically derived from conditional sampling of diffusion models, supporting its attack performance and imperceptibility, and does not require any loss functions, optimizers, or additional neural networks.
B Derivations and Proofs
In this section, we first introduce the specific straightforward Pixel-level Constraint (PC) for é; that simply mentioned at the beginning of Section 3.3 of the main text as an intuitive preliminary, then we provide the detailed proofs of Theorem I and Proposition 1, 2 given in the PC for é,
B.1 Straightforward PC for 6t
For each known He in the fixed diffusion trajectory of the original image Pori, and modified à in the attacking trajectory with the proposed Attacked Model Guidance (AMG) leading to the adversarial example Vade. the objective of PC is to control and constrain these two trajectories to be close, ensuring the final dop (i.e., Fadv) close Bo (i.e., Bori). It is obvious that straightforward way to achieve the goal by directly constrain every â using T. Thus, in PC for &t, we can utilize the restriction of adversarial examples and the relationship between Bede, & and i to derive the constraint for E. Given the budget the desired restriction of adversarial examples is
Dzado - Eorilloo S.
(15)
Next, since the é, is unconstrained in the case of PC for &,, adopt the initialized eo to calculate #c approximating Fade as:
(E0) G. VI- CgED
Fodv
(16)
/tt
where ag = 1. and 01:T E (0, 1]" is a pre-defined decreasing scalar sequence. And the 2, of the original trajectory is calculated as:
c Vattori + vI arEo
(17)
15

---

By substituting Eq. (16) and Eq. (17) into Eq. (15), the constraint for Dr can be easily derived, denoted as:
1
Ft
S
V Vat
3t
(18)
Va
20
[
Tt bo I Var
In this way, the PC for ét can be implemented the start of each step to achieve the basic restrictions by employing projection operation to åt according to Eq. (18). However, this direct modification of & at each step obviously disrupts the entire diffusion trajectory from noise to our desired adversarial distribution, and can not achieve the final imperceptibility. Additionally, it is observed that the estimation of Φ in each step employs a fixed Eg. impairing the adversarial guidance crafted by AMG.
B.2 Proof of Theorem
Therefore, carefully analysis the important noise term in our diffusion- bbaedd modeling approach for adversarial attacks, and present Theorem to support our PC for & as described the main text. The proof of Theorem is provided as follow.
Theorem Given diffusion coefficients ar. E (0, 1J²", the Cori. Tt. En from the original trajectory, Lt & from the modified trajectory, and variable E, if', and Eo satisfies
VaT
l|ε: - colloo S -T- 0T
E,
(19)
for all E [Z': 1}, then it follows that ||ē; - Telloc S (Var - V1 at
V1-ar ) ||
Jas
Korillos 10o
S
ç, and |:0 Tors|loo S E hold true.
Proof We prove Theorem using mathematical induction.
Initial case. For trajectories of the adversarial example Badv and original image Dori that start from ēT+1 = 6n. ②τ = Ty and ④4 2G, we can unfold the formula for computing the 2- with the updated ēT as:
49 TT-1
8--1
09
ET
VaT-1
VT
Var-1
Pr TTnnre Var
+ VI
aT-
II aT ET
OTT
/
TTêT
(20)
VOT
For from the fixed trajectory where Ef always equals to Dori, we have:
TT VT-aTEo aT
Fori
(21)
Varr
With क. = -r., Eq. (20), Eq. (21), and the relationship of (e colloo I ---oo
Jar
E, we have:
VaT
[ē - colloo S
E
v1
aTêT
VI aTED
5
VT
VoT
-1-0y -
67
S
VaT
/ /T
40
89-1lla
(22)
16

---

Meanwhile, for the relationship between 6r-1 and øT at the initial step, we have:
④T /1 aTêr
(TT-1
-1T-1
x
Var-1
+ VI -1- 1êT
vaT
FT
VI OT
VVT-I
V -T-
VaT
VOT ivr aT
aT-1
(êT ED)
VOT
100
V1 -T-1
/T--VV QT
[ê ED 100
(23)
VaT
VaT
aT
V1-aT-1 QT-1
||êr- |coloo
(24)
VaT
/OT
VT-- VI- QT-1
E,
(25)
Oer
where the transition from Eq. (23) Eq. (24) is obtained with the real constant value of at. At this point, it can be seen that Theorem 1 holds in the initial case.
Inductive step.
Assuming the theorem holds for some arbitrary step k. where T 2 k- > we have:
VaT
[ -Eolooo S
5.
(26)
v1 0T
|& Zorilloc = =1(0f --/llx0 S E,
(27)
and
0 Tolloo S (Vox= VI- ak
(28)
VT
Ort
Based on the inductive hypothesis, we next show the validity of the theorem at step k 1. Similar to Eq. (20), we unfold the calculation of 2h-1 with 2y and éμ as:
2x-1
/1 ai- Es
0
0k-1
ak-1
Vak-1
i vT To
Vox
+ V
06-166k
V ak
Ek
Vak-1
/k--
kb V1 CLER
(29)
Vor
And the 2h_1 can be denoted as:
VI -a,Cu
Zors
(30)
Vau
Consequently, by substituting Eq. (29) and Eq. (30) into
| - 56
we have:
Gg
-T-agé
Ft V1 0..00
x6
akk
Vak
1
II ai
(Et
-x. +
(ED εt.
(31)
ak
v ak
|| Tx
+
atk
1122
Cu |0
(32)
/ak
VI as
VOT
VI
akk
<
++
[
Eolloo
(33)
Vas
/1
ar
Vax
17

---

where Eq. (31) To Eq. (32) utilizes the triangle inequality property lf --norm, and Eq. (33) is obtained
with Eq. (28). Then, given the imposed condition of Eq. (19), we can get:
Vor
[ 60
5
αT -I- VoT
vI
vI
1
++
collo0 &
Vag 0T
ak
h--1 1 Tb-1l 5
1 Dori
(34)
And the relationship between -x-1 and 2x- at step k can be expressed as:
-8t-1
vI
axêk
a2-1
+ V 1la---166 ax-1Ex
Vork
T6 v I Cg
Vak-1
V ak-1E0
V 0kk
v 01-1
0k--1 O%k
(0x Tb) + V ak-1
(Ez - E0)
(35)
Vak
vaz
v 0k
[6 Tx 01100 + Vl-ak-1 ak-1
112 culloc
(36)
Vas
Vax
0k-1
(vak v1-
OT
++
V
Vak-iv
ak
/1- V1-ak-1 ak-1
v ax
v1 0T
Vak
v1
C0
(37)
(VO-1--11-0 /1
oOT
(38)
at
where the triangle inequality property is utilized again to obtain Eq. (36)frum Eq. (35), then Eg. (28) and Eq. (34) is substituted obtain Eq. (37). Obviously, for the case of step k it is also consistent with the theorem.
Conclusion. Therefore, by extending the truth of the theorem, from arbitrary step k. to k - 1, and given its established validity at the initial case, the principle of mathematical induction allows us to
.aT _ [ || Eorilloo S & hold true for every conclude that [, - Zz loo I (vor v11 at V
step t E [1 T}. For t = 0 and ogg = 1, we have 20 an and To Dori, thus it is obvious that ||00 Fori los < &. This concludes the proof of the whole Theorem
B.3 Proof of Proposition
Next, we prove Proposition about de and 6e by expanding and rearranging the recursive formulas in our diffusion process for attacks.
Proposition 1 Under the conditions of Theorem 1, by denoting constrained é; Eu 8c. we have
T X
Kads Dori
(39)
t=
where >
VT-or Vas
1-a--1 v
and |001lo 5
E. VI-oor
y Of
18

---

Proof 2 With Eq. (19) in Theorem 1, by denoting é = Eu- - 8,, we have: | 10c = l|ê; - coll20 -
VT S, and the -r-1 1 in the initial step can be written as:
yaT
VOT
VOT (VTT-IV--
hr-1-
v V1-OT-) GT-1
(Cr)
VoT
VaT
yar-1
VV-
vI 0T-1
DT
vor-1
(E0 - (6)-
(40)
VoT
VoT
OT--
Applying the recursion formula twice, we have:
VOT-2
V Er
h--2-
QT-1
(êr)
/T-11
VoT
VaT
22-
0T-1
V 0T
(êr-1) )
V
var-2
V1 - OeT
0T--1
v -2
(E0 δT)
VoT
VOT
VaT-1
1T
V1 OT
v0r-2
(co 81-1)
(41)
Var
VQT-2
Similarly. for -T-3, we have:
VT hT-3=
V 1T
VaT
VOT IV 1- QT
V1 0T
(êT)
VOT-2
-1
vaT
VaT
VQV-2V
QT-1
V
0T
(éT-1) )
VOT-I
/TT 1 0T-2
CT-3
(ê
VOT-2
/OT -3
or-3
OT
OT
(60-
VaT
/or-1
QT-1
QT-2
V
(E0 òT-1)
/TT-1
/TT-2
VaT-3
QT-2
0T-3
(Eo 6T-2)
(42)
/OT-2
Vor-3
It is obvious that the coefficients of each term exhibit a clear regular pattern related to the step t. Following this pattern, we can accordingly get the expression of & as:
Vai Āt
a1-aa
VI- QT-1
4:
Vor
(E0 δr)
VoT
VOT
VOT-I
(/--T-1
0T V1-OT-
Vas
(co - -8T-1)
Var-1
Var-2
11
0442
01+1
Vos
(co 5142)
Va2+2
Vatti
1-at at
-Vai
(Eo 81+1)
(43)
Var+1
Vai
19

---

And the final To can be expressed as:
v 000
-T-
-T
v1 aT-1
Lo
RT
voo
(EO δr)
V
VaT
VaT-I
VI-aa-
aT-
QT
-V00
(E0 - 8T-1)
VT--2
v1 -
v1
Vau
(cu
82)
02
v a1
Vao
Q1
0x0
(ED 81)
(44)
Vai
Vai
Note that in Ciy = 1 Eq. (44), and the coefficients of cu can be mostly eliminated. Thus, by defining X as:
A VI-ai ai
11-0--1
(45)
Va:
1
we can rearrange Eq. (44) into:
TT vI
aTED
T
OrT
t=1
KT VI
aT
T
+
A
VaT
t=
Tori
X
(46)
This concludes the proof of where Eo is the final output of the diffusing process and Fads Sy.
Proposition
B.4 Proof of Proposition
Finally, prove Proposition 2 about the validity and convergence of the approximation Fado N..
Proposition 2 Under the conditions of Theorem and Proposition 1, the upper bound on the error of the approximation in Eq. (22) can be expressed as
VI-ai. aa. Vat
VOT
Fadv
(47)
W1-TT aT
Proof 3 With Eq. (43) and the definitions of Proposition 1, de and a can be written as:
Vat VoT
4.
du (E0 8k),
(48)
k=H+1
and
00 T: VI -ar
1
T
de (cD 0b) - VI- ai
(co - 6(+1).
(49)
Vau
v /or
Vai
k=t+1
For Rodv. we have:
2T
-- aTEO
Radv
t0
d,dr
(50)
VaT
1=1
With Eq. (49) and Eq. (50), we can obtain:
VI at 8
Eade
#0
A
(51)
ya
20

---

Thus, we have:
vI- at
Fade 12
XRdx
81+1
Vai
AgOx
+
Ott
1++1
+ v1-ar
As |
10:++1100
Var
VaT [
-I- at
Var
AR
+
6
v1 oT
Vor
vI VI-aT aT
v VT
-1-
a4
VuT
++
5
v1 uT
Va
v1 OT
VI
a0
vaT
VI- at
vaT
+
val.
Va
VI
var
vI 0T
VI
Or
/TT
2.
(52)
Vag
VI
OT
This concludes the proof of Proposition 2.
C Algorithm of AdvAD-X
Algorithm AdvAD-X
Input: Attacked model f(-), image Cori with label Myt. budget &, step 7';
Output: Adversarial example aade
1: Initialize pre-defined diffusion coefficients ourr (0, 1J
2: Initialize Eu (0, I):
Initialize and fix diffusion noise co- b Align with data range diffusion process. bi Forward process of adding noise Eo to Mori- D Non-parametrie diffusion process.
3: Transform the range Fori to [-1, 1];
4: Calculate के via Eq. (4);
5: Set का ãr, ēr+1 60:
6: Calculate mask of wart using GradCAM:
Mask m for the CA strategy.
7: for To 1 do
8: Calculate & via Eq. (7):
> Approximation of a =s Wadu-
9: Transform the range of a to [0, 255];
b A lign with data range of image.
10:
// DGI strategy for performing AMG and PC dynamically.
11:
if f(x)) == Vge then
12:
Calculate é, using m with AMG via Eq. (14);
AMG module and CA strategy. Same PC module as AdvAD.
13:
Calculate & with PC via Eq. (10);
14:
else
15:
Set é: ED:
Skip the operations of current step. One step backward from t > Endpoint the process. Directly return Ladv in raw floating-pon point data for the ideal scenario.
16:
Calculate á; Eq. (11);
1.
17:
Transform the range of & to [0, 255];
18:
return Ladv Zo:
D Additional Experiments
D.1 Additional Quantitative Comparisons
Table reports the untargeted attack performance and imperceptibilityy of ten methods on Vgg- 19, MobileNet-V2, and WideResNet-500 models. The results indicate that the proposed AdvAD and AdvAD-X, leveraging a novel modeling framework, consistently achieve superior performance. These findings further underscore the effectiveness of the proposed approach.
21

---

Table 6: Additional results of untargeted white-box attack success rate (ASR) and other evaluation metrics for imperceptibility when employing different attacks and attacked models. The reported running times are obtained using a RTX 3090 GPU same machine. t and blue mean the results of AdvAD-X are obtained with floating-point data type in the ideal scenario as described in Sec 3.4.
Model
Attack Method
Time (5) 1
ASR (%)
Lau +
12
PSNR t
SSIM f
FID 1
LPIPS
MUSIQ T
PGD
47
100.0
0.031
8.47
33.23
0,8771
43.15
0.0508
53.51
NCF
3288
92.8
0.794
75.2
14.77
0.6391
57.45
0.3077
49.27
ACA
83123
93.4
0.832
51.22
18.22
0.5767
66.78
0.3277
55.61
DiffArtack
34163
97.0
0.769
31.92
22.23
0,6632
59.08
0.1235
57.22
1gg-19
DitPGD
5770
93.9
0.246
20.72
0.0317
55.23
268
97.5
0.062
1.23
5.90
0.0061
51.93
PerC-AL
8671
100.0
0.142
2.12
45.92
0,9885
10.78
0.0028
56.91
SSAH
948
85.5
0.022
2.35
44.62
0.9920
4.25
0.0017
53.45
AdvAD (ours)
4370
99.5
0.009
1.05
52.13
0,9979
2.62
0.00M0S
56.31
AdvAD-X (ours)
1967
0.001
0.32
64.76
0,9997
0,0001
56.56
PGD
10
19133
0.031
8.29
33.41
0.88003
34.37
0.0500
32.00
NCF
2503
92.5
0.784
76.02
14.69
0.6373
56.23
0.3090
49.37
ACA
83118
92.8
0.835
50.70
18.30
0.5786
64.90
0.3254
56.17
DiTAAtack
34723
0.739
30.51
0.1143
56.01
MobileNet-V2
DiiFGD
5941
11.43
19.22
0,0309
AdyDrop
97.7
0.063
3.16
41.94
4.88
0,0064
54.91
PerC-AL
3187
99.8
0.118
2.16
45.67
0,9879
8.77
0,0052
55.59
SSAH
265
97.8
0.026
2.18
45.24
0,9930
2.94
0.0016
55.78
AdvAD (ours)
992
99.6
0.008
0.94
53.07
0,9982
1.46
0.00044
56.37
AdvAD-X (ours)
388
100.0
0.001
0.24
0,9998
0.11
0,0001
PGD
42
96.0
0.031
8.2
33.3
D.BSIU
35 394
0.0521
52.43
NCF
2971
89.7
0.777
74.05
14.98
0.6473
56.01
0.2965
49.45
ACA
84163
88.0
0.838
53.17
17.89
0.5619
68.27
0.3442
$5.47
DiftArtack
34072
95.1
0.747
30.61
0.6737
0.1137
55.
Wele
DifPGD
S965
0.0317
16
AdyDrop
96.5
0.062
1.28
0.9863
6.21
0.0060
54.917
PerC.Af.
6655
97.8
0.133
1.91
0.9906
56.07
SSAH
738
95.7
0.028
2:2T
33.2T
0.9933
3.95
0.0015
55.88
AdvAD (ours)
3845
99.9
0.010
1.10
51.54
0,9979
2.84
0.00M66
56.33
AdvAD-X (ours)
1477
100.0
0.002
0.38
62.54
0.9996
0.33
0,0001
56.58
D.2 Additional Visualizations
More visualizations of adversarial examples and perturbations under the attacked model of ResNet-50 displayed in Figure 6. The visualizations provide clear insight into how different methods accomplish imperceptible attacks. Our AdvAD and AdvAD-X methods execute imperceptible attacks with lower overall intensity of perturbations. Notably, for images with salient objects (e.g.. the bridge in the second examples), the perturbation intensity of AdvAD also naturally increases in the object regions during the gradient-based adversarial guidance calculation, while AdvAD-X, equipped with the dynamic strategy, still shows uniform and lower overall perturbation intensity.
D.3 Ablation Study of AdvAD-X
From AdvAD AdvAD-X, Table 7 shows the effect of the two strategies of CA and DGI. It can be observed that adding CA in each step of AdvAD slightly improves impercepbility while maintaining the attack success rate of 100%. However, the DGI strategy significantly reduces the iterations of performing AMG and PC from 1000 to 3.97, which indicates that our framework theoretically only requires very little injected adversarial guidance successfully perform attacks, proving the performance of our modeling method as well as the effectiveness of the adversarial guidance. In AdvAD-X, which finally uses both CA and DGI, the guidance strength in each step is further suppressed, resulting in a slight increase in the total number of iterations required adaptively, but the final perturbation strength continues to decrease a more extreme level.
Table 7: Ablation study of the proposed CAM Assistance (CA) and Dynamic Gradient Injection (DGI) strategies in AdvAD-X. As marked with all the results in this experiment are obtained with attacking normal ResNet-50 using the floating- point raw data to align with the setting of AdvAD-X. The term of Iter. indicates the number of iterations that the AMG and PC are performed.
Attack
Iter.
ASRT
[2]
PSNRT
SSIMT
FID
AdvAD
1000
100.0
0.97
52.60
0.9984
2.3894
AdvADeCAT
1000
100.0
0.89
53.27
0.9987
2,2033
AdvAD+DGI
3,97
0.34
63.60
0.9997
0.2317
AdvAD-XT
4.05
100.0
0.34
63.62
0.9997
0.2301
22

---

(ous)
v100
NCF
DEtAtadd
AGA
DITRG
AdDkpp
PoC.LL
SSAH
AdkADiouu
AdAD X (ous)
-100
ADA
Adtttemp
ANAAT Goed
DUTPCD
AdAAD (ous)
1503
1100
NCP
ACA
DMTPOD
Addlxop
PeCALL
SSAH
AdAD Sound
ARADX (oas)
Ad/Tmp
<100
Figure 6: Additional imperceptible adversarial examples and corresponding perturbations generated by various methods. Perturbatons are amplified and shown for the convenience of observation. Please zoom to observe the details of the images.
23

---

D.4 Additional Discussions
Discussion on Proposition In the previous sections, we obtained Proposition 1 ttrouuhh extensive derivations, which reformulates the AdvAD attack process using À and d, While this formulation does not represent the actual attack procedure, enables post-analysis after the completion of attacks. In Proposition 1, although the upper bound of 8 is theoretically independent of the step I, both 6, and the coefficient A gradually decrease with & quantitative results of Figure due to the unique properties of AdvAD. Thus, it emerges hypothesis that whether modifying the coefficient of gradient term of traditional attacks like PGD to decay incrementally could also achieve the imperceptibility. To isolate the impact of this hypothesis, we conduct experiments with modified version of PGD with step size decay as:
31-1 11ç ( + X 17 sign Ve, Cce (xx))
(53)
where X is the same coefficient in Eq. (39) of Proposition for alignment, and is fixed small factor for the initial step size. We have searched a lot of values of v) to determine the optimal range, and the results of attacking three models with different architectures under three typical values of n are presented in Table 8.
Table 8: Results of PGD + step size decay strategy and the proposed AdvAD.
Model
Attack Method
Param.
Time
ASR
1.0
12
PSNR 1
SSIM 1
PGD + Step size decay in Egg (53),
S
2272
99.9
0.016
1.80
46.75
0.9947
PGD Step size docay in Eq, (53),
30-5
1=1000,
2228
99.0
0.008
1.17
50.41
0.9974
Re
PGD size decay Eq. (53).
1e:5
<=//255
2306
7.1
AdvAD (ours)
2201
99.7
0.010
1.06
51,84
0.998
PGD Step size decay Eig (53). 1) 5e:5
8725
98.0
0.008
1.28
49.88
0.9975
PGD Step size decay in Eq. (53). nj
Tw1000,
8728
0.064
0.94
52.47
0.9985
Swin-Base
PGD Step size decay in Fq.. (53),
(=8/255
3.9
AdvAD (ours)
9729
0.013
1.19
50.57
0.9978
PGD Step size docay in Eq. (55), 50-5
6350
0.008
1.63
47.76
0.9959
PGD Step size decay in Eq.
3e-5
T-1000,
6393
78.3
0.004
1.10
51.05
0.9979
VisionMamba-Small
PGD Step size decay in Fq. (53),
le-5
$=8/255
6348
2.5
AdvAD (ours)
61.54
99.7
0.016
1.62
47.94
0.9960
It can be observed that for PGD with this strategy, the ASR is clearly proportional to r,. the impercep- tibility is inversely proportional to 7j. However, regardless of how 1) is adjusted, this strategy can not simultaneously match AdvAD in both ASR and imperceptibiliity. Firstly, for 1j = 5e-5, when attacking VisionMamba, ASR of this strategy is 10.5% lower than AdvAD with close PSNR, and the strategy has 0.2% higher ASR but a 5,09 dB lower PSNR for ResNet50. For 7) = 3c-5, the ASR against VisionMamba and Swin further degrade, being 10.9% and 21.4% lower than AdvAD, respectively. Finally, for 7 = 1e-5, the modified PGD with step size decay fails to attack all the models. Never- theless, although this step size decay strategy performs worse than AdvAD, it indeed enhances the imperceptibility of attacks compared to the original PGD in some cases, which further validates motivation and modeling approach. This is because, while this strategy follows completely different technical route than AdvAD, it similarly uses subtler perturbations to progressively push adversarial examples closer to the model's decision boundary. To this end, we leave further research on the potential of this strategy to future work.
Limitation. As the primary focus of AdvAD is the imperceptibility, although it achieves better transfcrability at lower perturbation strength compared with other restricted imperceptible attacks, its transferability is inevitably weaker than other black-box attack methods that operate in larger perturbation spaces and are specifically designed for transferability (like the unrestricted ones). However, the proposed AdvAD is essentially a general attack paradigm with novel modeling approach and a solid theoretical foundation. By relaxing the constraint of perturbation strength and incorporating enhanced designs for the transfcrability into the proposed framework of non- -arammeettiic diffusion process, AdvAD also has significant potential to be modified into a specific black-box attack, and we also leave this aspect for future research.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We present the motivation, innovation, overview of methods, experimental performance and the main contributions of our paper in the abstract and introduction. These claims are further explained and verified in Sec. 3 and Sec. 4, and detailed proofs are given in Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification; For the proposed AdvAD that mainly focuses on the imperceptibility, we point that there is a trade-off between the imperceptibiliity and transferability for the restricted attacks. The relevant discussions and experimental results are given in See. 3.4, Sec. 4.3, and Appendix D.4. Additionally, for the proposed AdvAD-X, although it shows amazing performance under an ideal scenario using raw floating-por point data for attacking and brings theoretical value, the very small floating-pont point perturbations will be easily erased by the quantization process when the image is actually stored.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
25

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The proposed attack method is built solid theoretical foundation, which is derived from the derivation of the diffusion models. The specific methods and theoretical properties introduced in detail in Sec. 3, and the detailed proofs of the proposed theorem and two propositions are given in Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We present all the formulas, calculation processes, algorithms, and hyperparam- eters of the proposed methods in detail. And we have tested that by fixing the random seed, our method can produce consistent results with the same input on our machine, providing good reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
26

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they (c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We open-source the code GitHub and give the Python environment re- quirements, dataset preparation, running commands, etc. Following our instructions, the experimental results given in the paper can be casily reproduced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
ceabbmmissiooooiccyy.. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wwtpppccc/caeacccccccccccc.c guwdeeg/////////egg Sttpl////uuiieeee/eSuuiiiiii
/CodeSubmic cettuuimsssioonii sesiiisssnooiiiieeeeee seiinnooicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: One of the main contributions of our paper is to propose a novel adversarial attack modeling framework based
por-maraettric diffusion process. Benefiting from the proposed modeling approach, our attack method only needs two simple hyperparameters accurately control the entire attack process, without the need for complex training or optimization processes in other paradigms.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
27

---

7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Only Figure 5 reports actual statistic numerical curve of theoretical upper bound the actual execution process, which is calculated from actual data using a confidence level of 0.85 and shows the confidence interval. other experiments. due to limited computing resources, the error bar under multiple runs is not included. However, mentioned above, the methods proposed in this paper can obtain consistent results in multiple runs by fixing the random seed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emorr bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have indicated that all the experiments are conducted on single NVIDIA RTX 3090 GPU, and the running time required for all methods to attack different models are included in Table 1, Table 5, and Table to compare the computational complexity while providing a reference.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https : htt///nurip... httpppppcccpppbccctccccccccccaaa
28

---

Answer: [Yes]
Justification: We ensure that the research conducted in the paper complies with the NeurIPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., . if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
As research on adversarial attacks of deep neural networks, the significance lies in revealing possible attack algorithms and the vulnerabilities of the models advance. in order to help promote corresponding defense methods or the model robustness, and improve the safety of deep neural networks in real-world applications.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
29

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The dataset we used is under the MIT License, and it has been properly cited in the paper with its URL.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
30

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "sZMF8zTOgC", "reviewer": "Reviewer_K1mG", "summary": "This paper studies a task about generating the imperceptible adversarial noise using the diffusion model. The proposed method theoretically models the attack process as a non-parametric diffusion process. Extensive experiments demonstrate the effectives of the proposed method.", "questions": "see weakness", "limitations": "see weakness", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The writing is good and easy to follow.  \n(2) The technique novelty is good, which studies the imperceptible adversarial perturbation generation as a non-parametric diffusion process. The behind theory is solid.  \n(3) Authors conduct extensive experiments and compare many baselines to show the effectiveness of the proposed method.", "weaknesses": "(1 ) The adopted defense methods are not strong, lacking some strong baselines, like the random smoothing.  \n[1] Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. \"Certified adversarial robustness via randomized smoothing.\" international conference on machine learning. PMLR, 2019.\n\n(2) Could the generated adversarial examples used for black-box (or transfer-based) attack settings? What's the limitation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time in processing our manuscript and the valuable feedback. Our point-by-point responses are as follows.  \n    &nbsp;  \n**Weakness 1: Defense of random smoothing.**  \n**Re:** In Table 3 of the paper, we have evaluated the robustness of proposed AdvAD using numerous advanced defense methods, including three post-processing purification methods and six adversarial training robust models to defense attacks. Random smoothing is indeed another strong defense method, and we have conducted experiments to test the performance of our attack on it following your suggestion.  \nSince random smoothing is not a truly end-to-end model but a method that uses the base model to make multiple predictions on noise-augmented images, we adpot a semi-white-box setup to fully test the attack performance. Specifically, we craft adversarial examples using only the base model without smoothing process, then apply an additional 100 rounds (n=100) of random smoothing voting on the generated adversarial examples and tested the final attack success rate. The results are as follows:  \n|         |  &emsp;&emsp; $\\\\sigma = 0.25$   |    |  &emsp;&emsp; $\\\\sigma = 0.50$    |    | &emsp;&emsp; $\\\\sigma = 1.00$   |    |\n|---------|-----------------------------|----|-----------------------------|----|-----------------------------|----|\n|         | **Top-1 Error (%)**    |  $l _2$  &emsp;&emsp;       | **Top-1 Error (%)**             | $l _2$   &emsp;&emsp;        | **Top-1 Error (%)**             | $l _2$  &emsp;&emsp;         |\n| **clean**   | 17.3                      |  /  | 30.3                      |  /  | 46.8                      |  /  |\n| **AdvDrop** | 25.2 (+7.9)             | 5.97 | 33.5 (+3.2)             | 6.21 | 48.7 (+1.9)             | 5.61 |\n| **SSAH**    | 21.8 (+4.5)             | 13.84 | 32.4 (+2.1)            | 14.82 | 46.9 (+0.1)            | 13.68 |\n| **AdvAD**   | **28.2 (+10.9)**             | **2.41** | **36.8 (+6.5)**           | **2.51** | **50.4 (+3.6)**            | **2.08** |   \n\nIt can be seen that for all random smoothing defenses using base models pre-trained with different variances ($\\\\sigma$), the proposed AdvAD achieves a higher attack success rate with smaller $l_2$-norm perturbations compared to other state-of-the-art imperceptible attacks (attack ), which further demonstrates the effectiveness of the proposed AdvAD. In the revised version, we will include the relevant references and results.\n\n**Weakness 2: Transferability and limitation.**  \n**Re:** Sure. The proposed AdvAD achieves the best imperceptibility with inherently minimal perturbation strength through a novel non-parametric modeling approach, while also surpassing other state-of-the-art restricted imperceptible attacks in transferability at the same time. The transferability of AdvAD is directly related to the step $T$, that is, the imperceptibility is positively correlated with the size of $T$, while transferability is negatively correlated with it. In Table 3 of the original manuscript, we have reported the relevant experimental results, and we have further discussed transferability in more detail in the global author rebuttal section. In Table R1, we provided results corresponding to additional values of $T$, and in Figure R1, we plotted the relationship curve between transferability and imperceptibility to find the optimal trade-off, conducting a more comprehensive evaluation. All experimental results demonstrate the superiority of AdvAD. Please refer to that section for more details.  \n\nRegarding limitations, although AdvAD achieves better transferability at lower perturbation strength, its primary focus remains on the imperceptibility of the attack. Consequently, its transferability is inevitably weaker than other  black-box attack methods that operate in larger perturbation spaces and are specifically optimized for transferability. However, the proposed AdvAD is essentially a general attack paradigm with a novel modeling approach and a solid theoretical foundation. Therefore, by relaxing the constraint of perturbation strength and incorporating enhanced designs for the transferability, AdvAD also has significant potential in the direction of black-box attacks. We leave this aspect for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n9UrbXcoa5", "reviewer": "Reviewer_ygiL", "summary": "In this paper, the authors propose the Adversarial Attack in Diffusion method called AdvAD, which crafts imperceptible perturbations from the model perspective without the need for additional networks.\n\nDuring the non-parametric diffusion process, the proposed AdvAD method introduces Attacked Model Guidance (AMG) and Pixel-level Constraint (PC) modules to ensure both imperceptibility and transferability of the adversarial examples. \n\nMoreover, an enhanced version AdvAD-X is further introduced to achieve better imperceptibility and lower computational complexty.\n\nExperiments conducted on CNN, ViT, and Mamba architectures using the ImageNet benchmark dataset demonstrate that the proposed AdvAD method achieves superior performance compared to baseline methods in terms of attack success rate, imperceptibility, and robustness.", "questions": "1. How are non-critical image regions defined for AdvAD-X? Is this definition universally applicable to all images, or is it adjusted for each image separately?\n\n2. Why were other attack methods (such as NCF, ACA, Diff-Attack, Diff-PGD, etc.) not considered in the transferability experiment shown in Table 3?\n\n3. As only a few values (e.g., 10, 100, 1000) are discussed, how can the optimal value of the step $T$ be chosen to ensure both imperceptibility and transferability in practice?", "limitations": "The proposed AdvAD-X method is only suitable for ideal scenarios where the crafted adversarial example is directly input to DNNs without undergoing quantization.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed AdvAD method reformulates the generation of imperceptible adversarial examples as a non-parametric diffusion process, effectively eliminating the negative effects caused by the uncertainty of generative models.\n\n2. The proposed AMG and PC modules inject adversarial guidance progressively at each step without the need for additional networks, and they have a solid theoretical guarantee that the error caused by approximation is strictly bounded.\n\n3. Experimental results verify the effectiveness of the proposed AdvAD method, which achieves higher attack success rate, lower perturbation strength, and better image quality.", "weaknesses": "1. In section 3.4, the details of AdvAD-X are not explained clearly. The exact step of adopting original diffusion noise in the DGI strategy is not discussed. Additionally, in the CA strategy, the definition of non-critical image regions is not introduced.\n\n2. In section 4.4, the transferability experiment lacks comprehensiveness. As shown in Table 3, the proposed AdvAD method is only compared to a limited number of baseline methods.\n\n3. In Table 3, with a limited budget of updating iterations, the proposed AdvAD (T=10) performs very similarly to the classic PGD method when using Mob-V2 as a surrogate model, improving the black-box ASR by less than 1%.  \n\n4. The experimental settings for Table 4 are not fully introduced, such as the surrogate model used for crafting adversarial examples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time in processing our manuscript and the valuable feedback. Our point-by-point responses are as follows.  \n    &nbsp;  \n**Weakness 1 & Question 1: Details of AdvAD-X: exact step with DGI strategy, definition of non-critical image regions.**  \n**Re:** Due to the page limitation of the main paper, we placed the ablation study results of AdvAD-X in **Appendix C.3**, including the number of iterations of AdvAD-X (column 2, Table 6). For your convenience, we transcribes it here as follows:  \n\n| Attack | Iter. | ASR ($\\\\uparrow$) | $l_2$ ($\\\\downarrow$) | PSNR ($\\\\uparrow$) | SSIM ($\\\\uparrow$) | FID ($\\\\downarrow$) |\n| :-------- | :-------: | :-------: | :------: | :-------: | :-------: | :------: |\n| AdvAD$\\\\textcolor{blue}{\\\\dagger}$            | 1000 | **100.0** | 0.97 | 52.60 | 0.9984 | 2.3894 |\n| AdvAD + CA$\\\\textcolor{blue}{\\\\dagger}$   | 1000 | **100.0** | 0.89 | 53.27 | 0.9987 | 2.2033 |\n| AdvAD + DGI$\\\\textcolor{blue}{\\\\dagger}$ | **3.97**  | **100.0** | **0.34** | 63.60 | **0.9997** | 0.2317 |\n| AdvAD-X$\\\\textcolor{blue}{\\\\dagger}$        | 4.05  | **100.0** | **0.34** | **63.62** | **0.9997** | **0.2301** |   \n\nFrom AdvAD to AdvAD-X,  the effect of the two strategies of CA and DGI are shown in this table. It can be observed that adding CA in each step of AdvAD slightly improves impercepbility while maintaining the attack success rate of 100\\%. However, the DGI strategy significantly reduces the iterations of performing AMG and PC from 1000 to 3.97, which indicates that our framework theoretically only requires very little injected  adversarial guidance to successfully perform attacks, proving the performance of our modeling method as well as the effectiveness of the adversarial guidance. In AdvAD-X, which finally uses both CA and DGI, the guidance strength in each step is further suppressed, resulting in a slight increase in the total number of iterations required adaptively, but the final perturbation strength continues to decrease to a more extreme level.  \n\nFor the CA strategy, we adopt the GradCAM (Gradient-weighted Class Activation Mapping) to obtain mask about non-critical image regions for each image separately (as mentioned in lines 201-204 of the main paper). GradCAM is a widely used technique to indicate important regions of the input image that a classification model focuses on when making decisions about a particular category. Specifically, we  calculate the heatmap via GradCAM as a mask $\\\\boldsymbol{m}$ with the same resolution of image. In this mask , each pixel value ranges from 0 to 1 with lower score representing lower importance, thus the non-critical regions are defined. Consequently, AdvAD-X suppresses the intensity of injected adversarial guidance at each step by performing element-wise multiplication with the mask $\\\\boldsymbol{m}$, which can be expressed as:    \n$$\\\\boldsymbol{\\\\hat{\\\\epsilon}}_t = \\\\boldsymbol{\\\\epsilon} _0 - \\\\boldsymbol{m} \\\\cdot \\\\sqrt{1-\\\\alpha_t} \\\\nabla _{\\\\boldsymbol{\\\\hat{x}}_t}\\\\text{log}(1-p _f(y _{gt}|\\\\boldsymbol{\\\\hat{x}} _{t}^{0}))$$\nAdditionally, we have provided the pseudo code for the AdvAD and AdvAD-X algorithms in the PDF file of the global author rebuttal for better understanding. We also hope the discovery that AdvAD-X can successfully attack with extremely low perturbation strength with floating-point raw data can bring new inspiration to reveal the robustness and interpretability (e.g., decision boundaries) of DNNs.  \n\n**Weakness 2 & Question 2: Other attack methods (NCF, ACA, Diff-Attack, Diff-PGD) are not considered in Table 3.**   \n**Re:** \n1) Rather than only focusing on transferability, the purpose of Table 3 is to show that although one of the main advantages of the proposed method is the imperceptibility achieved by the novel modeling approach with inherently low perturbation strength, our AdvAD also surpasses other state-of-the-art restricted imperceptible attack methods in transferability, even at much smaller perturbation cost. This is despite the evident positive correlation between transferability and perturbation strength as described in line 292-295. Additionally, more than a comparison of transferability, Table 3 also serves as an ablation study to illustrate the effect of step $T$, demonstrating AdvAD could be flexibly adjusted through $T$ as a novel and general attack framework.  \n2) The NCF, ACA, and other methods you mentioned are unrestricted attacks, which dose not impose objective limitation on perturbation strength, resulting in significant alterations to the images (as shown in Table 1, their $l _2$ distances are x10-x75 greater than AdvAD) and inevitably producing unnatural artifacts or unreasonable semantic modifications sometimes (as visualizations in Figure 2 and 5). Given this substantial difference in perturbation cost, comparing the transferability of unrestricted and restricted methods is inappropriate. Therefore, we did not include these methods in Table 3. Instead, as aforementioned, we compared our method with other state-of-the-art restricted imperceptible attacks and demonstrated that our approach surpasses them in both imperceptibility and transferability, simultaneously.  \n\n**Weakness 3 & Question 3: Similar performance to PGD using Mob-V2 as a surrogate model, and the optimal value of $T$.**  \n**Re:**  In the global author rebuttal, we have further discussed transferability of the proposed AdvAD and provide more experimental results to demonstrate its effectiveness, including more values of $T$, the optimal value selection, more comprehensive comparisons with other attacks, etc. Please kindly refer to the corresponding section for more details.  \n\n**Weakness 4: Experimental settings of Table 4.**  \n**Re:**   We apologize for this omission. The attacked model adopted in Table 4 is the classic ResNet50, and the total step $T$ is consistently set to 1000 for all $\\\\xi$. We have added the above description of the experimental setting in the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x7MQNLMwjA", "reviewer": "Reviewer_jCt2", "summary": "This paper proposes a new method, AdvAD, for generating imperceptible adversarial attacks against deep neural networks (DNNs). AdvAD is based on a novel non-parametric diffusion process, which initializes a fixed diffusion noise and then manipulates it at each step using adversarial guidance crafted by two modules: Attacked Model Guidance (AMG) and Pixel-level Constraint (PC). This process gradually leads the image from its original distribution to a desired adversarial distribution. The paper also introduces an enhanced version, AdvAD-X, which aims to achieve extreme performance under an ideal scenario. Extensive experiments demonstrate the effectiveness of both AdvAD and AdvAD-X in terms of attack success rate, imperceptibility, and robustness compared to state-of-the-art methods.", "questions": "What are the specific differences between AdvAD and existing diffusion-based attack methods like DiffAttack and ACA? A detailed comparison would help clarify the novelty of the proposed approach.\n\nWhat are the potential defensive measures against AdvAD, and how can they mitigate the threat posed by this attack?\n\nHow does the computational complexity of AdvAD compare to other state-of-the-art methods, especially for large-scale datasets or models?", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper's main strength lies in its novel approach to generating imperceptible adversarial attacks. The use of a non-parametric diffusion process is innovative and shows promising results in terms of attack effectiveness and imperceptibility. The paper is well-written and easy to follow, with clear explanations of the proposed method and experimental setup.", "weaknesses": "The paper has several weaknesses. The technical novelty is limited, as the core idea of using diffusion models for adversarial attacks has been explored in prior work (e.g., DiffAttack, ACA). The paper lacks a clear comparison with these existing diffusion-based attack methods, making it difficult to assess the incremental contribution of AdvAD. \n\nAnother thing is how much cost is it really to generate adversarial examples this way. I mean, one important use of generating adversarial examples is for adversarial training. So if it becomes really expensive to generate adversarial examples, such a use wouldn't be really meaningful. Then what is the use of these adversarial examples really?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time in processing our manuscript and the valuable feedback. Our point-by-point responses are as follows.  \n    &nbsp;  \n**Weakness 1 & Question 1: Clear and detailed comparison with existing diffusion-based attack methods.**  \n**Re:** Compared to other recent notable works exploring adversarial attacks based on diffusion models, the proposed AdvAD is a completely novel approach distinct from existing attack paradigms. It is the first pilot framework which innovatively conceptualizes attacking as a non-parametric diffusion process by theoretically exploring fundamental modeling approach of diffusion models rather than using their denoising or generative abilities, achieving high attack efficacy and imperceptibility with intrinsically lower perturbation strength. To further highlight and clarify our contributions, we have provided a detailed comparison below between AdvAD and other diffusion-based attack methods in various aspects.  \n1) **Motivation.** Existing diffusion-based attacks achieve imperceptibility by utilizing the denoising or generative abilities of diffusion models to heavily modify texture or semantic attributes of images in an image edition-like way rather than inject typical adversarial noises. In contrast, the proposed AdvAD aims to address the fundamental challenge of imperceptible attacks, that is, performing attack with inherently minimal perturbation strength from a modeling perspective.  \n\n2) **Theory of Attack.** Although diffusion models are introduced in the existing diffusion-based attacks, their theoretical foundations still adhere to prior attack paradigms, either by integrating the optimization of adversarial loss functions (e.g., cross-entropy) or classical attack methods (e.g., PGD) with nested loops of forward and backward diffusion steps. In comparison, we propose a novel non-parametric diffusion process for performing attacks as a new paradigm. The modeling of AdvAD is theoretically derived from conditional sampling of diffusion models, supporting its attack performance and imperceptibility, and does not require any loss functions, optimizers, or additional neural networks.  \n\n3) **Methodology.** Among the existing diffusion-based attacks, DiffAttack and ACA combine the optimization of adversarial losses with the Stable Diffusion to generate unrestricted adversarial examples, while Diff-PGD and AdvDiffuser incorporate the classic PGD attack into the normal diffusion process. Furthermore, as typical diffusion models, these attacks require using pre-trained neural networks to estimate the noise term at each diffusion step. For our AdvAD, the adversarial example is crafted via an uni-directional non-parametric diffusion process from an initialized and fixed diffusion noise to final sample. At each step, a much subtler (for imperceptibility) yet more effective (for attack performance) adversarial guidance is calculated via the proposed AMG and PC modules without additional networks, then injected by manipulating the diffusion noise.  \n\n4) **Effectiveness.** Due to the uncertainty of generative models and the unrestricted attack setting, existing diffusion-based attacks inevitably cause unnatural and unreasonable artifacts in the crafted adversarial examples, especially for images with complex content. Benefiting from our modeling approach, the proposed AdvAD is able to apply precise control over the perturbations and accomplish imperceptible attacks with inherently low perturbation strength. Extensive experimental results have also demonstrated the superiority of AdvAD.  \n\nBased on AdvAD, we further propose an enhanced version AdvAD-X to explore the extreme performance in an ideal scenario for the first time, which also possesses theoretical significance and provides new insights for revealing the robustness of DNNs.  \n\n**Weakness 2 & Question 3: Computational complexity.**    \n**Re:** We respect and concur with your point of view. One of the important roles of adversarial attacks is to promote corresponding defense methods and enhance the robustness of DNNs, and algorithm efficiency is a key factor to consider in practice of adversarial training.  \n\nIn column 3 of Table 1, we have reported the total time cost of each attack for processing 1000 natural images. Thanks to our novel non-parametric and uni-directional diffusion process, which does not require additional networks or nested loops of forward and backward diffusion steps, the computational complexity of AdvAD is mainly concentrated in taking partial derivatives to obtain adversarial guidance (note that this is not the gradient of loss function) at each step. Thus the cost of AdvAD is significantly lower than ACA and DiffAttack, which contain Stable Diffusion within loops of optimization. For Diff-PGD, it is more time-consuming than AdvAD when attacking small models (ResNet-50, 1.5x) but slightly faster for larger models (Swin-Base, 0.8x). Additionally, for non-diffusion optimization-based attacks, the computational complexity of AdvAD is lower than PerC-AL and comparable to the others, while the performance of AdvAD is much better.  \n\n**Question 2: Potential defensive measures against AdvAD.**    \n**Re:** In our research, we have considered the robustness of AdvAD against defense methods and presented the experimental results in Section 4.3. It can be observed that due to the inherently low perturbation strength of AdvAD, it is relatively more susceptible to defense methods based on purification, which is similar to other state-of-the-art restricted imperceptible attack methods. However, the results indicate that AdvAD can relatively easily bypass robust models. We hypothesize that this is because these robust models typically use previous attack algorithms based on adversarial losses or gradient ascent for adversarial training, whereas AdvAD, as a new attack paradigm, is not included. Thus, we also anticipate that AdvAD will further advance the development of related defense fields.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zT1TEhWR1G", "reviewer": "Reviewer_nUXD", "summary": "This work proposes a novel adversarial attack framework called Adversarial Attacks in Diffusion (AdvAD). Unlike prior methods that rely on generative models or specific loss functions, AdvAD formulates attacking as a non-parametric diffusion process. This approach theoretically explores a fundamental modeling strategy instead of leveraging the denoising or generation capabilities of diffusion models with neural networks. AdvAD iteratively refines the attack by crafting subtle adversarial guidance based solely on the targeted DNN, without requiring any additional network. This process progressively transforms the original image into an imperceptible adversarial example.", "questions": "1.\tThe paper claims that AdvAD achieves superior imperceptibility compared to other methods. However, after multiple readings, the justification for this remains unclear. While the text mentions \"much subtler yet effective adversarial guidance at each step\", \"grounded on diffusion model's theoretical foundation\", and various diffusion-related formulations, it lacks a clear explanation (or theoretical verification) of how using non-parametric diffusion leads to attacks with lower perturbations.\n2.\tBased on my understanding, AdvAD shares similarities with traditional attacks like PGD, but with a fixed number of optimization iterations (controlled by the total diffusion model timesteps T) and a gradually decaying learning rate (controlled by the $α_t$ value). I hypothesize that the decayed learning rate is a key factor contributing to imperceptibility. The learning rate starts low (at the beginning of T steps) and gradually decreases to 0 (at the end of 0 steps). This suggests that small perturbations are added first, followed by even smaller ones, potentially leading to adversarial examples closer to the classification boundary and thus exhibiting higher imperceptibility. Could the authors comment on the validity of this hypothesis?\n3.\tFollowing the previous question, to isolate the impact of the decayed learning rate strategy, it would be beneficial to include comparative experiments. These experiments would directly apply a small learning rate initialization and a decayed learning rate schedule to a traditional attack algorithm. If this approach achieves similar performance to AdvAD, it would suggest that the emphasis on diffusion model theory might not be necessary. If not, the authors should highlight the specific benefits of the diffusion model mechanism that contribute to imperceptibility.\n4.\tDoes the selection of the initial Gaussian noise sample $ε_0$ affect the performance of AdvAD?\n5.\tIn Table 1, is the FID metric computed between the perturbed images and the \"imagenet-compatible dataset\"? If the goal is to assess the realism of the adversarial examples compared to natural images, a more comprehensive evaluation would be to compare them with the raw ImageNet images. The \"imagenet-compatible\" dataset, with only 1000 images, might not adequately capture the entire distribution of natural images.\n6.\tLine 248 states that \"the optimizer usually cannot find the global optimal solution, and optimization-based methods tend to show sub-optimal ASR.\" How can AdvAD achieve better ASR compared to these optimization-based methods?\n7.\tIn Table 2, which classifier was used to generate the adversarial samples? The robustness of the adversarially trained model Inc-V3_{adv} and the ensemble model Inc-V3_{ens4} appears very low (almost 100% ASR). Additionally, were Res-50, Swin-B, and ConvNeXt-B clean classifiers or adversarially trained?\n8.\tLine 292 suggests that \"a larger T exhibits better imperceptibility, while a smaller T implies stronger black-box transferability.\" Can the authors elaborate on this trade-off?\n9.\tThere are a few minor writing errors throughout the paper, such as \"a' adversarial\" (Line 140), \"fistly\" (Line 144), and \"an' scheme\" (Line 191).\n10.\tThere are specific questions regarding the proofs of theorems and propositions in the Appendix:\n\n(1)\tIn the second row equality of Eq. 6, shouldn’t $\\epsilon_T$ be $\\epsilon_{T+1}$ in the first term on the numerator of the first term?\n\n(2)\tIn Line 546, it Is clear that $\\epsilon_{T+1}$ satisfies the inequality as $\\epsilon_{T+1}$ equals $\\epsilon_0$, but how does this hold for $\\epsilon_{T}$ as well?\n\n(3)\tEq. 20, third row inequality: How is this inequality derived?", "limitations": "Yes, limitations are identified in the paper.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe concept of leveraging the core principles of diffusion models to craft adversarial examples is intriguing.\n2.\tThe writing style in the Abstract and Introduction sections is engaging.\n3.\tThe effectiveness of AdvAD is validated through experiments.", "weaknesses": "While I have carefully reviewed the work several times, some uncertainties remain. These are addressed in the following \"Questions\" section. Due to these uncertainties, the initial recommendation is a borderline decision, which can be revised upwards or downwards based on the authors' response.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time in processing our manuscript and the valuable feedback. Our point-by-point responses are as follows.  \n\n**Question 1:  Explanation and verification of AdvAD.**   \n**Re:** \n1) **Intuitive explanation.** The superior imperceptibility of AdvAD first comes from its modeling philosophy inherited from diffusion models. For image generation tasks, one of the widely recognized advantages of diffusion models lies in breaking down the complex problem of directly generating images (e.g., GANs) into a series of simpler tasks, gradually pushing a Gaussian distribution toward the image distribution by approximating the noise at each diffusion step, thus to significantly reduce the difficulty of generation with a divide-and-conquer manner. Similarly, compared to previous attack frameworks that directly optimize or inject perturbations into images, the proposed _non-parametric diffusion process_ allows AdvAD to decompose attacking into the relatively simpler task of imposing _adversarial guidance_ by manipulating diffusion noise at each diffusion step to progressively guide the endpoint of diffusion from the original image to a desired adversarial example. Thus, the adversarial guidance injected at each step could be subtler but more effective, ultimately achieving imperceptible attacks with inherently lower perturbation strength.  \n\n2) **Theoretical verification.** As a completely novel attack paradigm without any loss function, optimizer, or additional neural network, the essence of AdvAD exactly lies in the _subtler_ (for imperceptibility) yet _more effective_ (for attack efficacy) adversarial guidance, and both the characteristics are theoretically grounded.  Firstly, the conditional sampling technique enables diffusion models to sample from a given conditional distribution (Eq. 3). Derived from it, the proposed AMG module treats the goal of attack as a condition and gradually moves the original diffusion trajectory of the image towards this adversarially conditional distribution through a non-parametric diffusion process (Eq. 4-8), while the PC module synergistically ensures the process is streamlined and controllable (Theorem 1), providing guarantee of successful attack. Secondly, beyond the aforementioned modeling philosophy, the imperceptibility is also supported in the unique property of adversarial guidance as described in line 156-161. In addition, our Proposition 1, 2 and experimental results (especially for Sec. 4.5) provide further analysis and validation of this point.  \n\n**Question 2, 3: Hypothesis and PGD with decaying learning rate.**  \n**Re:**   \n1) As shown in Eq. 9, the $l _\\\\infty$ upper bound of the guidance strength is irrelative with step $t$. This means that there is no 'decaying learning rate/step size' coefficient for the guidance during the non-parametric diffusion process, and no loss function (e.g., CrossEntropy) is used as in traditional algorithms for gradient ascent. We assume your hypothesis originates from the two decreasing curves about Proposition 1 in Figure 4. Actually, Proposition 1 is an analytical conclusion drawn from extensive formula derivations in Appendix B.3, which demonstrates that the effect of adversarial guidance on the entire process diminishes from strong to weak. This is because, due to the nature of the non-parametric diffusion process, the earlier the guidance is injected, the greater its cumulative impact on the subsequent steps, corresponding to more movement distance toward the target distribution for the noisy sample at that step.  \n\n2) We conduct experiments of the PGD+decaying step size strategy. We modify the PGD attack as: $x ^{t+1} = \\\\Pi(x ^t + \\\\lambda _t \\\\cdot \\\\eta \\\\cdot sign(\\\\nabla _{x _t}L _{CE}(x ^t, y)))$, where $T=1000$, $\\\\lambda _t$ is consistent with our Proposition 1 for alignment,  and $\\\\eta$ is a fixed small factor for the initial learning rate. We explored a lot of $\\\\eta$ values to determine the optimal range, and the results attacking three models with different architectures with three typical $\\\\eta$ are presented in Table R2 of the attached PDF. It can be observed that for PGD with this strategy, the ASR is clearly proportional to $\\\\eta$, while the imperceptibility is inversely proportional to $\\\\eta$. However, regardless of how $\\\\eta$ is adjusted, this strategy _can not_ simultaneously match AdvAD in both ASR and imperceptibility. Firstly, for $\\\\eta$ = $5\\\\times10^{-5}$, when attacking VisionMamba, ASR of this strategy is 10.5% lower than AdvAD with similar PSNR. For ResNet50, the strategy has a 0.2% higher ASR but a 5.09 dB lower PSNR. When $\\\\eta$ = $3\\\\times10^{-5}$, the ASR against VisionMamba and Swin further degrade, being 10.9% and 21.4% lower than AdvAD, respectively. When $\\\\eta=1\\\\times10^{-5}$, this method fails to attack all the models. \n\n**Question 4: Initial Gaussian noise.**  \n**Re:** All Gaussian noises are randomly initialized and have almost no impact because they merely serve as a medium of the injected adversarial guidance. With various seeds, the effect on the final results is akin to slight random fluctuations, at ±0.1% for ASR and ±0.02 for PSNR.  \n  \n**Question 5: Calculation of FID.**  \n**Re:** For adversarial attacks, our goal is to assess the similarity between adversarial examples and their corresponding clean images to measure the imperceptibility of attacks, rather than the gap with the natural image distribution. Therefore, we adopt the FID calculation method consistent with previous work. Moreover, in Table 1, we have included results of the MUSIQ metric, which assesses the realism of adversarial examples without reference images.  \n\n**Due to the character limit, point-by-point responses to the remaining questions are placed in Part 2 of the global author rebuttal. Please kindly refer to that section for details.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wm3ee9d4Hv", "reviewer": "Reviewer_ytMn", "summary": "In this paper, a diffusion-based adversarial attack method is proposed and extensive experiments were conducted to show the effectiveness of the method.", "questions": "same", "limitations": "same", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. the paper is well-written and easy to read.\n2. both adversarial attack and diffusion model are hot topics.\n3. both Linf and L2 results are given in the experiment.\n4. An ablation study on T is provided\n5. The experiment's results are convincing.", "weaknesses": "1. because this paper proposed a new algorithm, at least a pseudo code should be provided to show how the algorithm works.\n2. In the table1, it shows that the performance of the model measured in FID sometimes does not match that in other metrics measuring similarity, e.g., psnr, ssim. For example, in the experiment on visionmemba, the FID of the proposed method is not the smallest, as that in other similar metrics. Can the author give some explanation about this?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time in processing our manuscript and the valuable feedback. Our point-by-point responses are as follows.  \n    &nbsp;  \n**Weakness 1: Pseudo code.**  \n**Re:** We appreciate your attention to this detail. We acknowledge that due to page limitations, we did not include the pseudo code in our initial submission. Instead, we made every effort to optimize our presentation and used clear illustrations with corresponding equation numbers (Figure 1) to introduce the workflow of our algorithm. We apologize for any difficulties this may have caused in understanding our manuscript. We have provided the complete pseudo code of our AdvAD and AdvAD-X in the attached PDF of the global author rebuttal, and it has also been included in the revised manuscript to enhance the clarity. Additionally, we have submitted runnable source code in the Supplementary Material for reproducibility, and we will also release the code after this work is accepted for publication.  \n\n**Weakness 2: The results of FID are not always the best.**  \n**Re:** It can be observed that the FID results of AdvAD are the second lowest only when attacking ConvNeXt and VisionMamba, with SSAH having the lowest results in these cases. Upon careful examination, we consider this to be due to the poor attack performance of SSAH on these two models, for the following reasons: Firstly, please note that the ASRs of SSAH are only **84.6%** and **49.8%** (ours are **100%** and **99.7%**) when attacking these two backbones (as shown in Table 1). As a method for directly optimizing the feature space in imperceptible attacks, SSAH shows such a low attack success rate, indicating that the added adversarial perturbations have minimal impact on the high-dimensional features within the attacked model layers. In other words, when SSAH attacks fail, the high-dimensional features of the crafted adversarial examples and the original images should be closer. Meanwhile, FID measures the Fréchet distance between the high-dimensional features of adversarial examples and original images extracted by a pre-trained Inception-V3. Therefore, the features of the adversarial examples generated by SSAH and the original images in the Inception-V3 model should also be closer, resulting in a lower FID score compared to ours. When all the attacks work normally, due to the inherently lower perturbation strength of AdvAD, the FID score, along with other metrics, achieves the best results among the comparison attacks including SSAH, demonstrating the effectiveness of our novel modeling approach.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their valuable feedback and recognition of our contributions. We also appreciate they find the proposed method is novel (**K1mG**, **ygiL**, **nUXD**, **ytMn**), theory is solid (**K1mG**, **ygiL**), paper is well-written (**K1mG**, **jCt2**, **nUXD**, **ytMn**), and the experiments have demonstrated the effectiveness of our AdvAD and AdvAD-X (**K1mG**, **ygiL**, **jCt2**, **nUXD**, **ytMn**).  
    &nbsp;  
### **Part 1: Responses to the common concerns of step $T$ and transferability of AdvAD.**  
We find that three reviewers (**K1mG**, **ygiL**, **nUXD**) are curious about the effect of step $T$ and the transferability exhibited by the proposed AdvAD. Although the primary focus of our method is to achieve imperceptible attacks with inherently minimal perturbation strength from modeling perspective, AdvAD is actually a complete novel and general attack framework that distinct from previous gradient-ascent or optimization-based ones since there are no loss functions, optimizers, or additional neural networks required. By conceptualizing attacking as a non-parametric diffusion process with adversarial guidance, AdvAD can flexibly adjust the decomposition granularity of the diffusion trajectory through the step $T$. A larger $T$ corresponds to a finer decomposition granularity, resulting in subtler yet effective adversarial guidance, which lead to final adversarial examples with lower perturbation strength. Conversely, a smaller $T$ corresponds to a coarser granularity, resulting in higher perturbation strength, which favors transferability. Despite of the contradiction between perturbation strength and transferability, compared to other state-of-the-art restricted imperceptible attacks, AdvAD still demonstrates both better imperceptibility and transferability simultaneously as shown in Table 3 of the manuscript. To further demonstrate the effect, we present five more values of $T$ in Table R1 of the attached PDF.  

Additionally, to elaborate the relationship between transferability and imperceptibility of AdvAD, as well as the optimal trade-off in practice, we have plotted two line graph according to Table R1. As shown in Figure R1 (a), as the value of $T$ on the horizontal axis changes, the relationship between imperceptibility and transferability shows a clear proportional trend mentioned above, consistent across different surrogate models. For the optimal trade-off, we consider the intersection point of the two curves to represent a balance between imperceptibility and transferability. Accordingly, for the ResNet-50 and MobileNetV2 models, the optimal values of  $T$ are 50 and 25, respectively. Moreover, Figure R1 (b) illustrates more direct curves of this relationship and the positions of other comparison methods within it. Note that, all the other comparison methods are located to the lower left of the curve of AdvAD. This indicates that our method consistently achieves the best results in both transferability and imperceptibility, fully demonstrating the effectiveness of the proposed AdvAD and the novel non-parametric diffusion based modeling approach. All the supplementary results have been included in the revised manuscript.  
    &nbsp;  
### **Part 2: Responses to the remaining questions of Reviewer  **nUXD**.**  
**Question 6: Why AdvAD achieves better ASR compared to the optimization-based methods?**  
**Re:** Optimization-based methods require balancing attack performance and imperceptibility through loss functions and may fall into local optima. In contrast, our method uses inherently small yet effective adversarial guidance to gradually push the non-parametric diffusion towards an adversarially conditioned distribution, without including any penalty terms for imperceptibility, thereby achieving a higher ASR.  

**Question 7: Settings of Table 2.**  
**Re:** As stated in the first row, the left half of Table 2 employs a standard ResNet50 to generate adversarial examples for three post-processing defense methods, while the right side presents the results of white-box attacks on adversarially trained robust models. Among these, Inc-V3 represents an earlier (2018) classic robust model, whereas the others are state-of-the-art adversarial training models recently published (e.g., [48] at NIPS 2023), thus exhibiting enhanced robustness.  

**Question 8, 9: Step $T$ and typos.**  
**Re:** We have elaborated on the trade-off issue in Part 1 of this author rebuttal. For writing errors, we thank you for kindly pointing them out, and we have carefully proofread and corrected all typos in the revised version.  

**Question 10 (1), (2): Initial case of Proof 1.**  
**Re:** As depicted in Figure 1 and Eq. 7 of the main text, the $\\hat{x}_t^0$ is calculated using $\\hat{\\epsilon} _{t+1}$ from the previous step, thus to streamline the whole process. Therefore, the $\\epsilon$ is initialized at step $T+1$ and $x$ at $T$, and the initial case we need to prove is from $T$ to $T-1$, which means deriving the relationship of $\\hat{x} _{T-1}$, $\\bar{x} _{T-1}$, etc., given the relationship of  $\\|\\hat{\\epsilon} _{T} - \\epsilon _0 \\| _\infty$ obtained by imposing the PC module.

**Question 10 (3): Eq. 20.**  
**Re:** Sincerely thank you for your carefulness and pointing the issue of Eq. 20. We are sorry that the sequence of Eq. 20 is mistakenly written upside-down. The correct process should be from bottom to top because $\\|\\hat{\\epsilon} _{k} - \\epsilon _0 \\| _\infty$ is actually not a conclusion but the condition that should be satisfied by applying the PC module as stated in Eq. 9, 10 of the main text. Substituting this condition into the left side of the inequality in the third row now, the inequality and the relationship of $\\|\\hat{x} _{k-1}-x _{ori}\\| _\\infty$ can be easily derived. This typographical error does not affect the correctness of the proof, and we have corrected it in the revised version.

Meta-review: This manuscript studies an imperceptible adversarial perturbation generation as a non-parametric diffusion process along with solid theoretical analyses.

The reviewers' comments have been properly addressed and most reviewers are satisfied with authors' responses.
However, some experiments are somewhat tricky.
For example, the comparison results in Table 2 indicate that the proposed attack AdvAD exhibits inferior success rates than other attacks (e.g., PGD and AdvDrop) under purification-based defenses.
The use of average ASR shows bias results.
Thus, the AC disagrees with the conclusion in Lines 272~273.

On the other hand, the AC agrees with the authors' argument that ``this is the first work to theoretically derive the fundamental modeling of diffusion models for performing imperceptible adversarial attacks with inherently low perturbation strength through a novel non-parametric diffusion process, and extensive experiments have demonstrated its effectiveness.''

So, the AC tends to accept this submission!
Decision: Accept (poster)