DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification
Authors: Mintong Kang
Dawn Song
Bo Li
Venue: NeurIPS 2023 poster (2023)
Keywords: adversarial attack
adversarial purification
adversarial robustness
diffusion model

Abstract: Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples and achieve state-of-the-art robustness. Recent studies show that even advanced attacks cannot break such defenses effectively, since the purification process induces an extremely deep computational graph which poses the potential problem of gradient obfuscation, high memory cost, and unbounded randomness. In this paper, we propose a unified framework DiffAttack to perform effective and efficient attacks against diffusion-based purification defenses, including both DDPM and score-based approaches. In particular, we propose a deviated-reconstruction loss at intermediate diffusion steps to induce inaccurate density gradient estimation to tackle the problem of vanishing/exploding gradients. We also provide a segment-wise forwarding-backwarding algorithm, which leads to memory-efficient gradient backpropagation. We validate the attack effectiveness of DiffAttack compared with existing adaptive attacks on CIFAR-10 and ImageNet. We show that DiffAttack decreases the robust accuracy of models compared with SOTA attacks by over 20\% on CIFAR-10 under $\ell_\infty$ attack $(\epsilon=8/255)$, and over 10\% on ImageNet under $\ell_\infty$ attack $(\epsilon=4/255)$. We conduct a series of ablations studies, and we find 1) DiffAttack with the deviated-reconstruction loss added over uniformly sampled time steps is more effective than that added over only initial/final steps, and 2) diffusion-based purification with a moderate diffusion length is more robust under DiffAttack.

taak: Evasion Attacks Against Diffusion-Based Adversarial Purification
Mintong Kang
Dawn Song UC Berkeley dawnsong@berkeley.ooo edu
Bo Li UICC 1bo@illinois. edu
UIUC mintong2@1llnois..edh
Abstract
Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples and achieve state-of-the-arr robustness. Re- cent studies show that even advanced attacks cannot break such defenses effectively, since the purification process induces an extremely deep computational graph which poses the potential problem of vanishing/exploding gradient, high memory cost, and unbounded randomness. In this paper, we propose an attack technique DiffAttack to perform effective and efficient attacks against diffusion-basee purification defenses, including both DDPM and acore-based approaches. In particular, we propose a reoontedrecconttuuctiin loss at intermediate diffusion steps induce inaccurate density gradient estimation to tackle the problem of vanishing/exploding gradients. We also provide segment-wise forwarding-backe backwarding algorithm, which leads to memory-efficient gradient backpropagation. We validate the attack effectiveness of DiffAttack compared with existing adaptive attacks CIFAR-II and ImageNet. We show that DiffA decreases the robust accuracy of models compared with SOTA attacks by over 20% CIFAR- 10 under foo attack 10% on ImageNet under foc attack (e = 4/255). We conduct a series of ablations studies, and we find 1) DiffAttack with the deviated-recomt recosttcrrronntttucttonn loss added over uniformly sampled time steps is more effective than that added over only initial/final steps, and 2) diffusion-baste baased purification with a moderate diffusion length is more robust under DiffAttack.
8/255), and over
1
Introduction
Since deep neural networks (DNNs) are found vulnerable to adversarial perturbations improving the robustness of neural networks against such crafted perturbations has become important, especially in safety-critical applications [18, 5 (44) In recent years, many defenses have been
52
20
proposed, but they are attacked again by more advanced adaptive attacks 77 30 [2]. One recent line of defense (diffusion-basee purification) leverages diffusion models to purify the input images and achieves the state-of-the-art robustness. Based on the type of diffusion models the defense
utilizes, diffusion-bar -based purification can be categorized into score-based purification 34 which uses
the score- score-based diffusion model 49 and DDPM-basee purification 4 67 51
55
56 which uses
the denoising diffusion probabilistic model (DDPM) [25). Recent studies show that even the most advanced attacks I12 34 cannot break these defenses due the challenges of vanishing/exploding gradient, high memory cost, and large randomness. In this paper, we aim to explore the vulnerabilities of such diffusion-basse purification defenses, and design more effective and efficient adaptive attack against diffusion-bass based purification, which will help to better understand the properties of diffusion process and motivate future defenses.
In particular, the diffusion-based purification defenses utilize diffusion models to first diffuse the adversarial examples with Gaussian noises and then perform sampling remove the noises. In this way, the hope is that the crafted adversarial perturbations can also be removed since the training
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

distribution of diffusion models is clean 49 25 time steps) is usually large, and at each time step, the deep neural network is used to estimate the gradient of the data distribution. This results in an extremely deep computational graph that poses great challenges of attacking it: vanishing/explodingg gradients, unavailable memory cost, and large randomness. To tackle these challenges, we propose deviated-rrcoostrucce algorithm and integrate them as an effective and efficient
The diffusion length (i.e.. the total diffusion
loss and
a segment-wise hewarding-backwaree attack technique DiffArtack.
Essentially, our deviated recontruutonnnn loss pushes the reconstructed samples away from the diffused samples at corresponding time steps. It is added at multiple intermediate time steps to relieve the problem of vanishing/exploding gradients. We also theoretically analyze the connection between it and the score-matching loss [26], and we prove that maximizing the devviated-rcoontruutti induces inaccurate estimation of the density gradient of the data distribution, leading to higher chance of attacks. To overcome the problem of large memory cost, we propose a segment-wise algorithm to backpropagate the gradients through a long path. Concretely, we first do forward pass and store intermediate samples, and then iteratively simulate the forward pass of a segment and backward the gradient following the chain rule. Ignoring the memory cost induced by storing samples (small compared with the computational graph), our approach achieves (1) memory cost.
loss
Finally, we integrate the deviated- reconstruction loss and segment- wise forwarding backwarding algorithm into DiffAttack, and empirically validate its effectiveness on CIFAR- 10 and ImageNet. We find that (1) DiffAttack outperforms existing attack methods B4 60 53
I 2 by large margin
for both the score-based purification and DDPM- purification defenses, especially under large perturbation radii; (2) the memory cost of our efficient segment- wise forwarding-backkwrddie backwarding algorithm does not scale up with the diffusion length and saves more than 10x memory cost compared with the baseline 44) (3) a moderate diffusion length benefits the robustness of the diffusion- -ased purification since longer length will hurt the benign accuracy while shorter length makes it easier to be attacked; (4) attacks with the deviated-reconssruution loss added over uniformly sampled time steps outperform that added over only initial/final time steps. The effectiveness of DiffAttack and interesting findings will motivate us to better understand and rethink the robustness of diffusion-biis based purification defenses.
We summarize the main technical contributions as follows:
We propose DiffAttack, a strong evasion attack against the diffusion -baed adversarial purification defenses, including score-based and DDPM-based purification.
We propose a deviattddreconsstuution loss to tackle the problem of vanishing/exploding gradient, and theoretically analyze its connection with data density estimation. backwarding
We propose a segment-wise borwarding-bcckwardinngg algorithm to tackle the high memory cost challenge, and we are first to adaptively attack the DDPM-based purification defense, which is hard to attack due to the high memory cost.
We empirically demonstrate that DiffAttack outperforms existing attacks by a large margin on CIFAR- 10 and ImageNet. Particularly, DiffAttack decreases the model robust accuracy by over 20% for foc attack (e = 8/255) on CIFAR- 10, and over 10% on ImageNet under Lx attack (e = 4/255).
We conduct a series of ablation studies and show that (1) a moderate diffusion length benefits the model robustness, and (2) attacks with the deviateedrecoonstrutiin loss added over uniformly sampled time steps outperform that added over only initial/final time steps.
2 Preliminary
There are two types of diffusion-based purification defenses, DDPM- purification, and score- based purification, which leverage DDPM [46] 25) and score-based diffusion model [49] to purify the adversarial examples, respectively. Next, we will introduce the basic concepts of DDPM and score-based diffusion models.
Denote the diffusion process indexed by time step L with the diffusion length T by (x+) DDPM constructs a discrete Markov chain {x with discrete time variables i following p(x//x,1) = N(x;;; /1 3, where 33 is a sequence of positive noise scales (e.g.. linear scheduling, cosine scheduling (3)). Considering at == 1 - Bc. āc == 111-1 and

---

ot VB((1 - (1-))/11 āc), the reverse process (i.e., sampling process) can be formulated as:
1- at cn (xx::)) vI- ai
x Vas
x
0.2
(1)
where z is drawn from (0,, I). Eø parameterized with 0 is the model to approximate the perturbation in the diffusion process and is trained via the density gradient loss La:
3f
La Ee,e
20 a.) |le colVaixo vI āge, 1)1
(2)
where e is drawn from (0,, I) and is uniformly sampled from (7] (1, T).
Score-based diffusion model formulates diffusion models with stochastic differential equations (SDE). The diffusion process {x, J is indexed by a continuous time variable t E [0, 1]. The diffusion process can be formulated as:
dx = f(, t)dt + g()ddw
where fxx t) R ~ R the drift coefficient characterizing the shift of the distribution, gt) is the diffusion coefficient controlling the noise scales, and is the standard Wiener process. The reverse process is characterized via the reverse time SDE of Equation (3)
dx = (xx, - 9()))) log (x))ddt + 9())dw
where Vx (x) the time-dependent score function that can be approximated with neural networks so parameterized with 0, which is trained via the score matching loss L, 126 47]
C. = E. [x(t) (xx, t) - Vx, (xx)((
(5)
where A 10 IR. is weighting function and i is uniformly sampled over [0, 1].
3 DiffAttack
3.1
Evasion attacks against diffusion-based purification
60) The adversarial images are transformed into latent representations, and then the purified images are sampled starting from the latent space using the generative models. The process is expected to remove the crafted perturbations since the training distribution of generative models is assumed to be clean.
class of defenses leverages generative models for adversarial purification A3
48
45
With diffusion models showing the power of image generation recently 15 39] diffusion-basse
adversarial purification has achieved SOTA defense performance 34 4
We first formulate the problem of evasion attacks against diffusion based purification defenses. Suppose that the process of diffusion-based purification, including the diffusion and reverse process, denoted by P R Rn where is the dimension of the input Xo, and the classifier is denoted by F 1Rn + [K] where K is the number of classes. Given an input pair (Xo, y), the adversarial example šn satisfies:
arg max (P(()))) # s.t. d(xo, Xo) S 6maz <E(K)
(6)
where Fi(-) is the i-th element of the output, F. x 1n IR is the distance function in the input space, and 8maz is the perturbation budget.
Since directly searching for the adversarial instance Na based on Equation (6) is challenging, often use a surrogate loss L to solve an optimization problem:
max ((F((()))) v) s.t. d(xo, Xo) 5mar
%o
where P(-) is the purification process with DDPM (Equation (1) or score- based diffusion (Equa- tions 3 and 4D), and the surrogate loss L is often selected as the classification-guidee CW loss 7 Cross-Entropy loss and difference of logits ratio (DLR) loss [12]. Existing adaptive
loss, such as
attack methods such as PGD 30 and APGD attack [2] approximately solve the optimization
problem in Equation ( via computing the gradients of loss L with respect to the decision variable
Xo and iteratively updating Xo with the gradients.
3

---

t
T73
t-21/3
Adv. Attacks
DiffAttack
fides
Liko
Lase
"Dog"
Classifier
"Fish"
(Adv. Target)
Diffusion process
Reverse process
La Deviated creenssttonen (..)3.2)
Segment-wise for warddnnn backwarding (Sec.33)
Figure 1: DiffAttack against diffusion- based adversarial purification defenses. DiffAttack features the
loss that addresses vanishing/exploding gradients and the segment-wise forwarding- backwarding algorithm that leads to memory-efficiee gradient backpropagation.
However, we observe that the gradient computation for the diffusion- based purification process is challenging for three reasons: 1) the long sampling process of the diffusion model induces an extremely deep computational graph which poses the problem vanishing/exploding gradient (2). 2) the deep computational graph impedes gradient backpropagation, which requires high memory cost [60 4], and 3) the diffusion and sampling process introduces large randomness which makes the calculated gradients unstable and noisy.
To address these challenges. we propose
prwiated-recoosstuuctoo loss (in Section 3.2) and a segment- wise forwarding backwarding algorithm (in Section 3.3) and design an effective algorithm DiffAttack by integrating them into the attack technique (in Section 3.4.
3.2 reviattedreconnttuuction loss
In general, the surrogate loss C in Equation is selected as the ceassificationguudee loss, such as CW loss, Cross-Entropy loss, DLR loss. However, these losses can only imposed at the classification layer, and induce the problem of vanishing/exploding gradients [2] due to the long diffusion length. Specifically, the diffusion purification process induces an extremely deep graph. For example, DiffPure applies hundreds of iterations of sampling and uses deep UNet with tens of layers as score estimators. Thus, the computational graph consists of thousands of layers, which could cause the problem of gradient vanishing/cxploding. Similar gradient problems generic score- based generative purification (Section 4, 5.1 in (60)). Backward path differentiablee approximation (BPDA) attack (2) is usually adopted overcome such problems, but the surrogate model of the complicated sampling process is hard to find, and a simple identity mapping function is
also mentioned with
demonstrated to be ineffective in the case [34 4 60)
To overcome the problem of ssploding/vanishinn
gradients, we attempt to impose intermediate guidance during the attack. It is possible to build set of classifiers on the intermediate samples in the reverse process and use the weighted average of the classification-giieed loss at multiple layers as the surrogate loss L. However, we observe that the intermediate samples are noisy, and thus using classifier F' that is trained on clean data cannot provide effective gradients. One solution is to train set of classifiers with different noise scales and apply them to intermediate samples impose classification -uuiddd loss, but the training is too expensive considering the large diffusion length and variant noise scales at different time steps. Thus, we propose a rrronttcdre loss to address the challenge via imposing discrepancy for samples between the diffusion and reverse processes adversarially to provide effective loss intermediate time steps.
Concretely, since a sequence of samples generated the diffusion and reverse processes, effective loss imposed on them would relieve the problem of vanishing/exploding gradient and benefit the optimization. More formally, let x,, x be the samples at time step in the diffusion process and the reverse process, respectively. Formally, maximize the
loss Ldes

---

formulated as follows:
max Edev = E, (at) d(xi x()]
(8)
where a-) is Limedeependet weight coefficients and d(x,, x)) is the distance between noisy image X in the diffusion process and corresponding sampled image x in the reverse process. The expectation
over is approximated by taking the average of results uniformly sampled time steps in [o, 7'), and the loss at shallow layers in the computational graph (i.e., large time step t) helps relieve the problem of vanishing/exploding gradient. The conditional expectation over Xx, X given Xo is approximated by purifying Xo multiple times and taking the average of the loss.
Intuitively, the the reverse process away from the sample x; at the corresponding time step in the diffusion process, and finally induces an inaccurate reconstruction of the clean image. Letting qe (x) and (((x) be the distribution of x, and X,, we can theoretically prove that the distribution distance between q (x()) and (x) positively correlates with the score-matching loss of the score-based diffusion or the density gradient loss of the DDPM. other words, maximizing the deviated-reconstructee loss in Equation (8) induces inaccurate data density estimation, which results in the discrepancy between the sampled distribution and the clean training distribution.
loss in Equation (8) pushes the reconstructed sample x in
Theorem 1. Consider adversarial sample 3o= Xg ò. where xa is the clean example and â is the perturbation. Pe (x)))) (..e((x),g))
are the distribution of &, where x, represents the reconstruction of X, in the reverse process. DTV(.,)) measures the total variation distance. Given a VP-SDE parameterized by 3()) and the score-based model so with mild assumptions that V| log Pe (x) - so(x, t)l1 S L DTV(Pe.P') S Cre, and a bounded score function by M (specified in Appendix C.T. I. we have:
1 2 B [1x f) Vx log (x) C1 + v 10--2expie
Drv 26) S
+
(9)
(1 + 8M) S." B(t)dt, C2 = (8(1 (1 - (z)))=
Proof sketch. We first use the triangular inequality to upper bound Drv(g: g)) with Drv pa) Drv (Pt. p4,) + Drv (p', 4)) DTv (qt, pr) can be upper bounded by a function of the Hellinger distance H (qt.))). which can be calculated explicitly. (Pv(Pt,f)) can be upper bounded by the reconstruction error Cre by assumption. To upper bound Drv (pe. 4)), we can leverage Pinker's inequality to alternatively upper bound the KL-divergence between pt and df which can be derived by using the Fokker-Planck equation [4] in the reverse SDE.
loss can indicate a large total variation distance Pry(qe-) (.). which is the lower bound of function with respect the score-matching loss Ft, ||,x||=s so ( (x,
Remark. A large deviated-reconstructie
Vx log gf (x)]l2 (in RHS of Equation (9)) Therefore, we show that maximizing the deviated- reconstruction loss implicitly maximizes the score-matching loss, and thus induces inaccurate data loss and
density estimation to perform effective attack. The connection of
the density gradient loss for DDPM is provided in Thm. 3 in Appendix
3.3 Segment-wise wwwarding-backwarddee algorithm
Adaptive attacks against diffusion- based purification require gradient backpropagation through the forwarding path. For diffusion- purification, the memory cost scales linearly with the diffusion length T and is not feasible in realistic application. Therefore, existing defenses either use
45 or consider adaptive attacks only for small diffusion length 44, but the approximation can induce error and downgrade the attack performance a lot. Recently, DiffPure [34] leverages the adjoint method (28) to backpropagate the gradient of SDE within reasonable memory cost and enables adaptive attacks against score-based purification. However, it cannot be applied to discrete process, and the memory efffccenf gradient backpropagation algorithm unexplored for DDPM. Another line of research g proposes the technique of gradient checkpointing to perform gradient backpropagation with memory efficiency, Fewer activations are stored during forwarding passes, and the local computation graph is constructed via recomputation. However, we are the first to apply the memory-efficient backpropagation technique to attack diffusion purification defenses and resolve the problem of memory cost during attacks, which is realized as a challenging problem by prior attacks against purification defenses [34, 60] Concretely, we propose a segment-wise forwarding-backe backwarding algorithm, which leads memory-efficient gradient computation of the attack loss with respect to the adversarial examples.
surrogate model for gradient approximation [55 56 60
5

---

We first feed the input Xg to the diffusion- purification process and store the intermediate samples
Xi, X2,..., Xy in the diffusion process and Xy. x......., X in the reverse process sequentially, For ease of notation, we have X = fa(x)) and x = fr (x(+1)) for t E [0, T 1]. Then we can
backpropagate the gradient iteratively following:
ac OC 0xL
ac (x(+))
(10)
0x/+1 Ox, 0x1+1
Öx!
0x(+1
At each time step in the reverse process, we only need to store the gradient 0C/0x,, the intermediate sample X/-1 and the model f, to construct the computational graph. When we backpropagate the gradients at the next time step t 1, the computational graph at time step & will longer be reused, and thus, we can release the memory of the graph at time step t. Therefore, we only have one segment of the computational graph used for gradient backpropagation in the memory at each time step. We can similarly backpropagate the gradients in the diffusion process. Ignoring the memory cost of storing intermediate samples (usually small compared to the memory cost of computational graphs), the memory cost of our segment wise forwarding-backwardiin backwarding algorithm is (1) (validated in Figure 3)
We summarize the detailed procedures Algorithm in Appendix B It can be applied to gradient backpropagation through any discrete Markov process with a long path. Basically, we 1) perform the forward pass and store the intermediate samples, 2) allocate the memory of one segment of the computational graph in the memory and simulate the forwarding pass of the segment with intermediate samples, 3) backpropagate the gradients through the segment and release the memory of the segment, and 4) go to step 2 and consider the next segment until termination.
3.4 DiffAttack Technique
Currently, AutoAttack 12 holds the state-of-the-art attack algorithm, but it fails to attack the diffusion-based purification defenses due to the challenge of vanishing/exploding gradient, mem- ory cost and large randomness. To specifically tackle the challenges, we integrate the deviated- reconstruction loss (in Section (32) and the segment-wise bowwarding-bacwwardii algorithm (in Section 3.3) ) as an attack technique DiffAttack against diffusion-based purification, including the score- based and DDPM-based purification defenses. The pictorial illustration of DiffAttack is provided in Figure [
Concretely, we maximize the surrogate loss L as the optimization objective in Equation 7
max L = Lets + ALdev
(1I)
where Less is the CE loss or DLR loss, Ldev is the devviated-reconstrucie loss formulated in Equation (8). and À is the weight coefficient. During the optimization, we use the segment-wise algorithm for memory-efficient gradient backpropagation. Note that Ldev suffers less from the gradient problem compared with Lets, and thus the objective of Ldev can be optimized more precisely and stably, but it does not resolve the gradient problem of Lota. On the other hand, the optimization of Liee benefits the optimization of Leis in the sense that Ldev can induce a deviated reconstruction of the image with larger probability of misclassification. controls the balance of the two objectives. A small À can weaken the reeanterdrrcontttttttion object and make the attack suffer more from the vanishing/exploded gradient problem. while a large À can downplay the guidance of the classification loss and confuse the direction towards the decision boundary of the classifier.
forwarding-bakkwardiin
Attack against randomized diffusion-based purification. DiffAttack tackles the randomness problem from two perspectives: 1) sampling the diffused and reconstructed samples across different time steps multiple times as in Equation (8) (similar to EOT (3)), and 2) optimizing perturbations for all samples including misclassified ones all steps. Perspective provides a more accurate estimation of gradients against sample variance of the diffusion process. Perspective 2) ensures a more effective and stable attack optimization since the correctness of classification is of high variance over different steps in the diffusion purification setting. Formally, the classification result of sample can be viewed as a Bernoulli distribution (i.e.. correct or false). We should reduce the success rate of the Bernoulli distribution of sample classification by optimizing them with a larger attack loss, which would lead to lower robust accuracy. In other words, one observation of failure in classification does not indicate that the sample has a low success rate statistically, and thus, perspective 2) helps

---

Table Attack performance (Rob-Ace (%)) of DiffAttack and AdjAttack purification on CIFAR-10.
34
against score- coore-based
Models
Cl-Acc
l Attack
Method
Rob-Ace
Diff.
AdjAttack
70.64
8/255
-23.76
DiffAttack
46.88
0.1
89.02
600
AdjAttack
82.81
WideResNee-28-100
4 4/255
-10.93
DiffAttack
71.88
AdjAttack
78.58
0.075
91.03
l2
0.5
-14.52
DiffAttack
64.06
AdjAttack
71.29
8/255
-25.98
DiffAttack
45.31
0.1
90.07
80
AdjAttack
81.25
WideResNett-0-116
4/255
-6.25
DiffAttack
75.00
AdjAttack
80.60
0.075
92.68
8
0.5
-10.29
DiffAttack
70.31
continue optimizing the perturbations towards lower success rate (i.e., away from the decision boundary). We provide the pseudo-coder of DiffAttack in Algorithm 2 in Appendix D.I
4 Experimental Results
In this section, we evaluate DiffAttack from various perspectives empirically. As a summary, we find that 1) DiffAttack significantly outperforms other SOTA attack methods against diffusion-based defenses on both the score-based purification and DDPM- based purification models, especially under large perturbation radii (Section 4.2 and Section 4.3): 2) DiffAttack outperforms other strong attack methods such as the black-box attack and adaptive attacks against other adversarial purification defenses (Section 4.4); 3) a moderate diffusson length T benefits the model robustness, since too long/short diffusion length would hurt the robustness (Section 4.5 4) our proposed segment-wisg -ise forwarding backwarding algorithm achieves (m)-mmyrryy cost and outperforms other baselines by large margin (Section 4.6); and 5) attacks with the rwiiatedrecoonstuuutioo loss added over uniformly sampled time steps outperform that added over only initial/final time steps (Section 4.7).
4.1 Experiment Setting
Dataset model. We validate DiffAttack on CIFAR-I0 27 and ImageNet 133 We consider different network architectures for classification. Particularly, WideResNet-28-11 10 and WeteResNet- 70-16 61 are used on CIFAR-10, and ResNet-50 [23]. WideResNNet-00- (WRN-50-2), and ViT (DeiT-S) [16] used on ImageNet. We use pretrained score-based diffusion model 49 and DDPM 25 to purify images following 4.
Evaluation metric. The performance of attacks is evaluated using the robust accuracy (Rob-Ace), which measures the ratio of correctly classified instances over the total number of test data under certain perturbation constraints. Following the literature [12), we consider both foo and l attacks under multiple perturbation constraints e. We also report the clean accuracy (Cl-Ace) for different approaches.
Baselines. To demonstrate the effectiveness DiffAttack, we compare it with 1) SOTA attacks against score-based diffusion adjoint attack (AdjAttack) (34), 2) SOTA attack against DDPM-bases diffusion Diff--BDA attack [4) 3) SOTA black-box attack SPSA (53) and square attack [.] and 4) specific attack against EBM-based purification joint attack 600. We defer more explanations of baselines and experiment details to Appendix D.2 The codes are publicly available at hhttps
7/g1 thub. cow Soon//kangmintoggDDfff...

---

Table 3: Attack performance (Rob-Acc (%)) of DiffAttack and Dift-BPDA purification on CIFAR-10.
against DDPM-based
Architecture
T
Cl-Ace lp Attack
Method
Rob-Acc
Diff.
Diff-BPDA
75.00
8/255
-20.31
DiffAttack
54.69
100
87.50
80
Dift-BPD BPDA
76.56
WideResNNer-8-110
4/255
-13.28
DiffAutack
63.28
Diff-BPDA
76.56
75
90.62
l2
0.5
-8.59
DiffAttack
67.97
Diff-BPDA
74.22
8/255
14.84
DiffAttack
59.38
100
91.21
80
Diff-BPDA
75.78
4/255
-8.59
DiffAttack
67.19
Diff-BPDA
81.25
75
92.19
8
0.5
-9.37
DiffAttack
71.88
4.2
Attack against score-based purification
DiffPure
34 presents the state-of-the-
art adversarial purification performance using the score-hased diffusion models [49] It proposes strong adaptive at- tack (tdjAttack) which the adjoint method (28) to efficiently backpropa- gate the gradients through reverse SDE. Therefore, we select AdjAttack as the strong baseline and compare DiffAttack with it. The results on CIFAR- 10 in Table show that DiffAttack achieves much lower robust accuracy compared with AdjAttack under different types of attacks (fo and & attack) with multiple perturbation constraints e. Concretely, DiffAttack decreases the robust accuracy of models by over 20% under fox attack with e = 8/255 (70.64% + 46.88% on WideResNet-28-I1 and 71. 1.. - 45.31% % on WideResNNet70--1e. well to large-scale datasets ImageNet as shown in Table 2 Note that the robust accuracy of the state-of- the-art non-diffusion-based purification defenses 38 21 achieve about 65% robust accuracy on CIFAR- 10 with WideResNet-258 of score-based purification under Adj AdjAttack in the same setting is 70.64%. However, given the strong DiffAttack, the robust accuracy of score-based purification drops to 46.88% It motivates us to think of more effective techniques further improve the robustness of diffusion-based purification in future work.
Table 2: Attack performance DiffAttack and AdjAttack (34 against score-based adversarial purification with dif- fusion length 0.015 on ImageNet under foo attack
(e 4/255).
Models
Cl-Acc
Method
Rob-Ace
Diff.
AdjAttack
40.93
Re
67.79
-12.80
DiffAttack
28.13
AdjAttack
44.39
WRN-50-2
71.16
-13.14
DiffAttack
31.25
AdjAttack 43.18
DeiT-S
73.63
-10.37
DiffAttack
32.81
The effectiveness of DiffAttack also generalizes
under &x = 8/255 attack (e 8/255), while the performance
4.3 Attack against DDPM-basse purification
Another line of diffusion- based purification defenses 4 55 56] leverages DDPM [46] to purify the images with intentionally crafted perturbations. Since backpropagating the gradients along the diffusion and sampling process with relatively large diffusion length is unrealistic due to the large memory cost, BPDA attack [2] is adopted as the strong attack against the DDPM-based purification. However, with our proposed segment-wise gradients within a small budget of memory cost, and our best knowledge, this is the first work to achieve adaptive gradient-based adversarial attacks against DDPM-based purification. We compare Diff'tttack with Diff-BPDA attack 4 on CIFAR-10, and the results Table demonstrate that DiffAttack outperforms the baseline by large margin under both loo and &g attacks.
sewarrdnn-backwwee
algorithm, we can compute the

---

4.4 Comparison with other adaptive attack methods
Besides the AdjAttack and Diff-BPDA attacks against existing diffusion-basee purification defenses, we also compare DiffAttack with other general types of adaptive attacks: 1) black-box attack SPSA [53] and 2) square attack (1, as well as 3) adaptive attack against score-based generative models joint at- tack (Score / Full) [60]. SPSA at- tack approximates the gradients by ran- domly sampling from pre-defined dis- tribution and using the miff-difffeeenc method. Square attack heuristically searches for adversarial examples in a low-dimensional space with the constraints of perturbation patterns. Joint attack (score) updates the input by the average of the classifier gradient and the output of the score estimation network, while joint attack (full) leverages the classifier gradients and the difference between the input and the purified samples. The results in Table show that Diff'Attack oulperforms SPSA, square attack, and joint atluck by a large margin on score-bused and DDPM-based purification defenses. Note that joint attack (score) cannot be applied to the DDPM-based pipeline due to the lack of a score estimator. AdjAttack fails the DDPM- bessed pipeline since it can only calculate gradients through SDE.
Table 4: Robust accuracy (%) of DiffAttack compared
with other attack methods CIFAR-10 with WideResNet- 28-10 under too attack (e 8/255).
Method
Scsre-based
DDPM-based
SPSA
83.37
81.29
Square Attack
82.81
81.68
Joint Attack (Score)
72.74
Joint Attack (Full)
77.83
76.26
Diff-BPDA
78.13
75.00
AdjAttack
70.64
DiffAttack
46.88
54.69
4.5 Robustness with different diffusion lengths
We observe that the diffusion length plays an extremely important role in the effectiveness of adversarial purifi-
100
eCeon Acc a Acc
100
aChon Acc aRebust Ace
Accuracy (8)
Accuracy (56)
80
cation. Existing DDPM-base Mosed purifica- tion works (56 55] prefer a small dif-
40
fusion length, but we find it vulnerable under our DiffAttack. The influence of the diffusion length T on the perfor- mance (clean/robust accuracy) of the purification defense methods is illus- trated in Figure We observe that I) the clean accuracy of the purification defenses negatively correlates with the diffusion lengths since the longer diffusion process adds more noise to the input and induces inaccurate reconstruction of the input sample; and 2) moderate diffusion length benefits the robust accuracy since diffusion- purification with a small length makes it easier to compute the gradients for attacks, while models with a large diffusion length have poor clean accuracy that deteriorates the robust accuracy. We also validate the conclusion on ImageNet in Appendix D.3
0.00 0.10 220224440222222220000
60 100
Diffusion length T (a) Score-hased purification (b) DDPM- purification
Diffusion length
Figure 2: The clean/robust accuracy (%) of diffusion- based pu- rification with different diffusion length T' under DiffAttack on CIFAR-10 10 with WideResNet-22 10 under L attack (e 8/255).
4.6 Comparison of memory cost
Recent work 4 computes the gradients of the
45,000
Blau et 2022 EL DiffAttack
diffusion and sampling process to perform the gradient-based attack, but it only considers small diffusion length (e.g., 14 on CIFAR- 10).
40,000
35,000
Memon Cost
30,000
25,000
20,000
They construct the computational graph once and for all, which is extremely expensive for
15,000
10,000
memory cost with a large diffusion length. We
5,000
use segment- wise forwarding-backwardin backwarding algorithm in Section to avoid allocating the memory for the whole computational graph. In this part, we validate the memory efficiency of our approach compared to []. The results
T=3
T=5
T-10
T-15
T-20
T=30
T-1000
Diffusion length T
Figure 3: Comparison of memory cost of gradient back- propagation between 4 and DiffAttack with batch size 16 on CIFAR- 10 with WideResNet-225 28-1- 1-00 under too attack.

---

in Figure 3 demonstrate that 1) the gradient backpropagation of [4] has the memory cost linearly correlated to the diffusion length and does not scale up to the diffusion length of 30, while 2) DiffAttack has almost constant memory cost and is able to scale up to extremely large diffusion length (T = 1000). The evaluation is done on an RTX A6000 GPU. Appendix [D33] we provide comparisons of runtime between DiffAttack and and demonstrate that DiffAttack reduces the memory cost with comparable runtime.
4.7 Influence of applying the revilatedrecoosstuuciinn loss at different time steps
We also show that the time steps at which we apply the
Decreased Rob-Acc [%)
+ score-based
DDPM-based
reviated-reconstructiin loss also influence the effectiveness of DiffAttack. Intuitively, the loss added at small time steps does not suffer from vanishing/exploding gradients but lacks supervision at consequent time steps, while the loss added at large time steps gains strong supervision but
25
20
15
10
suffers from the gradient problem. The results in Figure 4 show that adding sampled time steps (Uni(O,T)) achieves the best attack per-
5
loss to uniformly
0
(0, (0,7/3) (2T/3,T) Uni(0,7)
formance and tradeoff compared with that of adding loss to the same number of partial time steps only at the initial stage ((0, T/3)) or the final stage (2T/33 T). For fair com- parisons, uniformly sample T/3 time steps (identical partial stage guidance (0, T/3), (2T /3, T)) to impose Leev.
Figure 4: The impact of applying lae at different time steps on decreased robust accuracy (%). T is the diffusion length and Uni(0, 7) represents uniform sampling.
Related Work
Adversarial purification methods purify the adversarial input before classification with generative
models. Defense-gan [43 trains GAN to restore the clean samples. Pixeldefend 48 utilizes an
autoregressive model purify adversarial examples. Another line of research (50 22
17
24 60)
leverages energy-based model (EBM) and Markov chain Monte Carlo (MCMC) to perform the purification. More recently, diffusion models have seen wide success in image generation
[S
(2]31]99 They are also used to adversarial purification [34 62 57 51 55] 56) and demonstrated to achieve the iuffusion-bbsed purification and show the effectiveness in different settings, which motivates future work to improve the robustness of the pipeline.
robustness. In this work, we propose DiffAttack specifically against
Adversarial attacks search for visually imperceptible signals which can significantly perturb the prediction of models [52 20). Different kinds of defense methods are progressively broken by advanced attack techniques, including white-box attack 6 2 32 and black-box attack ||53 53 35
12 37
59 [9] propose systematic and automatic framework attack existing defense methods.
Despite attacking most defense methods, these approaches are shown to be ineffective against the diffusion-based purification pipeline due the problem of vanishing/exploding gradient, memory cost, and randomness. Therefore, we propose Diff'Attck to specifically tackle the challenges and successfully attack the diffusion-basee purification defenses.
Conclusion
In this paper, we propose DiffAttack, including the deviated-reconstrnctio loss added on intermediate samples and a segment-wise DiffAttack outperforms existing adaptive attacks against diffusion-basel purification by a large margin. We conduct series of ablation studies and show that a moderate diffusion length benefits the model robustness, and attacks with the time steps outperform that added over only initial/final time steps, which will help to better understand the properties of diffusion process and motivate future defenses.
baaudiin-baccwarrding algorithm. We empirically demonstrate that
loss added over uniformly sampled
Acknolwdgement. This work is partially supported by the National Science Foundation under grant No. 1910100, No. 2046726, No. 2229876, DARPA GARD, the National Aeronautics and Space Administration (NASA) under grant no. 8ONSSC20M0229, the Alfred P. Sloan Fellowship, and the Amazon research award.
10

---

References
[1] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: query-efficient black-box box adversarial attack via random search. In European Conference on Computer Vision, pages 484-501 Springer, 2020.
[2] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pages 274-283, PMLR, 2018.
[3] Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adver- sarial examples. In International conference on machine learning, pages 284-293, PMLR, 2018.
[4] Tsachi Blau, Roy Ganz, Bahjat Kawar, Alex Bronstein, and Michael Elad. Threat model- agnostic adversarial defense using diffusion models. arXiv preprint arXiv:2207.0008) 2022.
[5] Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li. Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks. In 2021 IEEE Symposium on Security and Privacy (SP), pages 176-194. IEEE, 2021.
[6] Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 3-14, 2017.
[7] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. leee, 2017.
[8] Chang, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham. Re- versible architectures for arbitrarily deep residual neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
[9] Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.066 064447, 2016.
[10] Zhaoyu Chen, Bo Li, Shuang Wu, Kaixun Jiang, Shouhong Ding, and Wenqiang Zhang. Content-based unrestricted adversarial attack. arXiv preprint arXiv:2305. 10665, 2023.
[11] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670 2020.
[12] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. pages 2206-2216 PMLR, 2020.
[13] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages
Ieee, 2009.
[14] Lue Devroye, Abbas Mehrabian, and Tommy Reddad. The total variation distance between high-dimensional gaussians. arXiv preprint arXiv: arXiv:1810.0869) 2018.
[15] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in Neural Information Processing Systems, 34:8780-8799 2021.
[16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 10X1:20100.1122 2020.
[17) Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32, 2019.

---

[18] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-wond -wolld attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1625-1634, 2018
[19] Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger B Grosse. The reversible resid- ual network: Backpropagation without storing activations. Advances in neural information processing systems, 30, 2017.
[20] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.65722 2014.
[21] Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. Advances in Neural Information Processing Systems, 34:4218 4233, 2021.
[22] Will Grathwohl, Kuan-Chieh Wang, Jörn- Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly energy based model and you should treat it like one. arXiv preprint 19Xiv:9112.0326. 2019.
[23] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[24] Mitch Hill, Jonathan Mitchell, and Song-Chun Zhu. Stochastic security: Adversarial defense using long-run dynamies of energy-based models. arXiv preprint arXiv:2005. 2020.
[25] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[26] Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
[27] Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
[28] Xuechen Li. King-Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradi- ents for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pages 3870 38822 PMLR, 2020.
[29] Xiang Ling, Shouling Ji, Jiaxu Zou, Jiannan Wang. Chunming Wu, Bo Li, and Ting Wang. Deepsec: A uniform platform for security analysis of deep learning model. In 2019 IEEE Symposium Security and Privacy (SP), pages 673-690. IEEE, 2019.
[30] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[31] Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Image synthesis and editing with stochastic differential equations. arXiv preprint 2.X1::2008.0103. 2021.
[32] Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, and Dietrich Klakow. Logit pairing methods can fool gradient-based attacks. arXiv preprint arXiv:1810.1240 2018.
[33] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[34] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. In International Conference on Machine Learning (ICML), 2022.
[35] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference computer and communications security, pages 506-519, 2017.
12

---

[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. al. Pytorch: An imperative style, terf-performanc deep learning library. Advances in neural information processing systems, 32, 2019.
[37] Maura Pintor, Luca Demetrio, Angelo Sotgiu, Ambra Demontis, Nicholas Carlini, Battista Biggio, and Fabio Roli. Indicators of attack failure: Debugging and improving optimization of adversarial examples. arXiv preprint arXiv: 2.Xiv:2106.099443 2021.
[38] Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.0// 022///03.994.. 2021.
[39] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[40] Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-imagge diffusion models. ACM SIGGRAPH 2022 Conference Proceedings, pages 1-10, 2022.
[41] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205. 1 1487, 2022.
[42] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[43] Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint 180. 2018.
[44] Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
[45] Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-supervision. arXiv preprint arXiv:2101.09387 2021.
[46] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamies. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
[47] Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pages 574-584. PMLR, 2020.
[48] Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv: 10766, 2017.
[49] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[50] Vignesh Srinivasan, Csaba Rohrer, Arturo Marban, Klbus-Robert Müller, Wojeiech Samek, and Shinichi Nakajima. Robustifying models against adversarial attacks by langevin dynamics. Neural Networks, 137:1-17, 2021.
[51] Jiachen Sun, Weili Nie, Zhiding Yu, Z Morley Mao, and Chaowei Xiao. Pointdp: Diffusion- driven purification against adversarial attacks on 3d point cloud recognition. arXiv preprint a208 09801, 2022.
13

---

[52] Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfel- low, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv:1312.6199
2013.
[53] Jonathan Uesato, Brendan O'Donoghue, Pushmeet Kohli, and Aäron van den Oord. Adversarial risk and the dangers evaluating against weak attacks. In icm12018, 2018.
[54] Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. arXiv preprint arXiv:2306. / 1698, 2023.
[55] Jinyi Wang, Zhaoyang Lyu, Dahua Lin, Bo Dai, and Hongfei Fu. Guided diffusion model for adversarial purification. arXiv preprint 145V:22005.1469,, 2022.
[56) Quanlin Wu, Hang Ye, and Yuntian Gu. Guided diffusion model for adversarial purification from random noise. arXiv preprint arXiv:2206. 10875, 2022.
[57] Chaowei Xiao, Zhongzhu Chen, Kun Jin, Jiongxiao Wang, Weili Nie, Mingyan Liu, Anima Anandkumar, Bo Li, and Dawn Song. Densepure: Understanding diffusion models towards adversarial robustness. arXiv preprint arXiv:2211.00322 2022.
[58] Haotian Xue, Alexandre Araujo, Bin Hu, and Yongxin Chen. Diffusion-based adversarial sample generation for improved stealthiness and controllability. arXiv preprint arXiv:2305.1644 2023.
[59] Chengyuan Yao, Pavol Bielik, Petar Tsankov, and Martin Vechev. Automated discovery of adaptive attacks on adversarial defenses. Advances in Neural Information Processing Systems, 34:26858 1116688-26670, 2021.
[60] Jongmin Yoon, Sung Ju Hwang, and Juho Lee. Adversarial purification with score-based generative models. In International Conference on Machine Learning, pages 12062-12072. PMLR, 2021.
[61] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint 1605v /46, 2016.
[62] Kui Zhang, Hang Zhou, Jie Zhang, Qidong Huang, Weiming Zhang, and Nenghai Yu. Ada3diff: Defending against 3d adversarial point clouds via adaptive diffusion. arXiv preprint 1724422211.16247, 2022.
14

---

A Broader Impact and Limitations
Broader impact. As an effective and popular way to explore the vulnerabilities of ML models, adversarial attacks have been widely studied. However, recent diffusion- based purification is shown hard to attack based on different trials, which raises an interesting question of whether it can be attacked. Our paper provides the first effective attack against such defenses to identify the vulnerability of diffusion-based purification for the community and inspire more effective defense approaches. In particular, we propose an effective evasion attack against diffusion-based purification defenses which consists of deeiiateddreconsstuction loss at intermediate diffusion steps to induce inaccurate density gradient estimation and segment-wise forwarding- www.gg-bbcaaanrrnn algorithm achieve memory- efficient gradient backpropagation. The effectivvnesss of the deviiatddrcconsstuuciin loss helps us to better understand the properties of diffusion purification. Coneretely, there exist adversarial regions in the intermediate sample space where the score approximation model outputs inaccurate density gradients and finally misleads the prediction. The observation motivates us to design more robust sampling process in the future, and one potential way is to train more robust score- score-based model. Furthermore, the segment- wwise forwarding- barkwwadibkaa algorithm tackles the memory issue of gradient propagation through a long path. It can be applied to the gradient calculation of any discrete Markov process almost within constant memory cost. To conclude, our attack motivates us to rethink the robustness of line of SOTA ceffusion-based purification defenses and inspire more effective defenses.
Limitations. In this paper, we propose an effective attack algorithm Diff'Attack against diffusion- based purification defenses. A possible negative societal impact may be the usage of DiffAttack in safety-critical scenarios such as autonomous driving and medical imaging analysis to mislead the prediction of machine learning models. However, the foundation of Diff DiffAttack and important findings about the diffusion process properties can benefit our understanding of the vulnerabilities of diffusion- based purification defenses and therefore motivate more effective defenses in the future. Concretely, the effectiveness of DiffAttack indicates that there exist adversarial regions in the intermediate sample space where the score approximation model outputs inaccurate density gradients and finally misleads the prediction. The observation motivates us to design more robust sampling process in the future, and one potential way is to train a more robust score-based model. Furthermore, to control a robust sampling process, it is better to provide guidance across uniformly sampled time steps rather than only at the final stage according to our findings.
Algorithm 1 Segment-wist wise
algorithm (Py -orehhh-e pseudo-codes)
1: Input: fr, fa. OC/0x/, x,, x (i E [2')
2: Output: OL/0xo
3: for 1= 1 to To do
4: Creat (x)) + x(-1)
5: <' (--00//xx (x)(
6: OC/Ox/ aauttugraaa CC' xx)
7: Release (frnosseGrraaffffee + x(-1)
8: end for
9: 0C/OxT= + 0
10: for IT
11:
Creat_Craph(fe (fa(x:))
+ x(+1)
12: L'
13: OL/0x, + auto auto_grradd grad(L', (L',)) x.)
14:
Release_Graph(Ae
+ x(+1)
15: end for
B Efficient Gradient Backpropagation
In this section, we provide the PyTorch-like pseudo-codes of the segment-wise forwarding- backwarding algorithm. At each time step / in the reverse process, we only need to store the gradient OC/0x, the intermediate sample x and the model fr to construct the computational graph. When we backpropagate the gradients at the next time step + 1, the computational graph at
15

---

time step will no longer be reused, and thus, we can release the memory of the graph at time step t. Therefore, we only have one segment of the computational graph used for gradient backpropagation in the memory at each time step. We can similarly backpropagate the gradients in the diffusion process.
C Proofs
C.1 Proof of Thm.
Assumption C.I. Consider adversarial sample X6 + ō, where Xq is the clean example and 5 is (x)
the perturbation. reconstruction of at time step in the reverse process. We consider a score-based diffusion model with a well-trained score-based model 8y parameterized by with the clean training distribution. Therefore, we assume that 80 can achieve low score-matching loss given a clean sample and reconstruct it in the reverse process:
are the distribution of &,,,, where x, represents the
V Po - (x(xxt)11 S L.
(12)
DTV (pt, S Ere
(13)
where Drv(.;" (-)-) is the total variation distance. Lu and Ere are two small constants that characterize the score-matching loss and the reconstruction error.
Assumption C.2. We assume the score function of data distribution is bounded by M:
|x)og I M, V log qi (x) l12 S M
(14)
Lemma C.1. Consider adversarial sample Xo == Xo + where X the clean example and the perturbation. pe (x). (x)) are the distribution of X,,,,, where x, represents the reconstruction of Xt in the reverse process. Given a VP-SDE parameterized by B(-) and the score-base -bsed model 8a with Assumption C.2
we have:
Vx logg'((x) - Vx log (x) + 4M² S." B(s)ds (15)
J."
DKI(Ph (1) = 2 (x))
Proof. The reverse process of VP-SDE can be formulated as follows:
= = freo(x, t, P. (eeedddt-rreotteeeeee (t) where free p, -00 log))(x)), grev vB(1)
(16)
Using the Fokker-Plance equation 44 in Equation (16). we have:
(x)(x)
1
Vx
2 "0 (1) (xrk(()))
(17)
Ot.
Vx (0)
logr/ - freu(x,t, px(xx)
(18)
Similarly, applying the Fokker-Plance equation the reverse SDE for (x), we can get:
(x((x)
Øt (Gomeen (1) log (x, d6(x))
Vx
(19)
We use the notation (x)
Ge lxg Frew t. and h₂(x)
Brev (t) logg((( (x, (x,t,,, t, qt). Then according to 34 (Theorem A.1), under the assump- tion that (x) and of (x) are smooth and fast decaying (i.e.,
(x)/Oxi]
0, lim (iim..-ool (x)@ log of (x)/Ox,] 0), we have:
0D d - pl(x)(1, - (x(xxt)) IV log - Vx lx (xx))xx
(20)
01
16

---

Plugging in Equations 18 and 19 we have:
f 2
gore (1)) logpl(x))))) Vx logg((x)
Öt
+ (x) Vx Fxlosn(2))11x<ooop/c/ Vx log (x)])xx
(21)
Finally, derive as follows:
DRI(P 16) = S." J (1,,(x)( gree /
Vx Vx
(22)
+3())l logpo (x) Vx log Vx [xbggo())""(<<ooor(((( Vx log (x))
(23)
1 (5hee
<
(+) (8)
V log / ( (24)
2 [' [(s) V log
1
Vx log lo) (2d Ids 4M2 f( B(s) (25)
Theorem (Thm. 1 in the main text). Consider adversarial sample Xo := XD 5, where X9 is the clean example and ò the perturbation. pe px))),,, (x). (x),)((( are the distribution of X,,,,, where X, represents the reconstruction of x, in the reverse process. DTv(.;)) measures the total variation distance. Given a VP-SDE parameterized by B(-) and the score-based model 88 with mild assumptions that V x log Pr(x) 8x 1)11 < Lu. Drv(P,, S tres and bounded score function by (specified with detaily in Appendix
we have:
1 Bexxx [x 1) Vx log (x)) Ci
Drv(g, vi) -
2
(26)
+V 2
+
where C1 = (In + +MM Se (()ddt, C2 = 8(1 - III (1 - B₂))"
Proof.
Since we consider VP-SDE here, have:
1 -= g(t) BBt)) f(x, t) =
(27)
(x, t) = 3 B(t) (x), (rev(t) = 3(t)
(28)
Using the triangular inequality, the total variation distance between qa and of can be decomposed as:
Drv g() S Drv(qe.P)) Drv (pe. Drv (d.P')
(29)
According to Assumption C.1 we have Drv (pr, (pr,pt)= S ere and thus, we only need to upper bound Dry (qt, Pt) and Drv (rrv('1, P.) respectivel..
Using the notation ae := 1 ((t) and ā := 11, 0g Cig. we have:
x, Ps A Axxx;; v /,x0, (1 āz ()).), X - == v /oxXo (1 I)
(30)
Therefore, upper bound the total variation distance between q, and p. follows:
(a) Drv(ge.Pt)
S 22H (x,, xx)
(31)
(8) 22 1 -expl-
1 858}
(32)
8(1 ax)
2 2expf
(10112)
(33)
8(1 a.)
17

---

where we leverage the inequality between the Hellinger distance H( and total variation distance
in Equation (a) and we plug the closed form of Hellinger distance between two Gaussian
distribution 14 parameterized by f41 E1. 12, E2 in Equation (b2)
(UW(,, E2))² 2 det(E,) 1/4 det (ete(()))) (22)
E1 + E2
-1
exp( (MI u2)²
(wi
[2))
Σ1 + E2
1/2
2
det
2
(34)
Then, we will upper bound Drv (14 9.) We first leverage Pinker's inequality to upper bound the total variation distance with the div-deeegeec::
(P S V D
(35)
Then plug in the results in Lemma C.I to upper bound K Lp/ df and it follows that:
Dry (1 (c)
(36)
$V 2 (KL(P)
(37)
4 J.
1 Vx ox + 2M2 J. B(s)ds
(38)
T
Ex V so(x, 1 Vx lolog (x)
(22))) B(s)ds
(39)
(a) Lu + 2M3)
J." B(s)ds + 1 (lx0xxx () Vx log (x)
(40)
where in Equation (a), we leverage the fact that ((-) is bounded in [0, 1].
Combining Equations 29 and 40), we can finally get:
Drv(a, a') S V
E 1xx N. tx(xxt)
Vx lo + +1 + V2 2-2 2-2exp(-2210013 + ere
(41)
where C = Lu + 2M2) f:t B(s)ds and C2 8(1 IT === (1 B.))
C.2
Connection between the deviated-recconsttuciion loss and the density gradient loss for DDPM
Theorem 3. Consider adversarial sample Na :- Xu + 5, where xa is the clean example and is the perturbation. Pe px( (x)xx(())) reconstruction of x4 in the reverse process. Given a DDPM parameterized by 3(-) and the function approximator Sa with the mild assumptions that || (x, t) <(x,,t)] L.. Drv pb)) S Eye- and a bounded score function by M (i.e., [(((x,)) & M) where E(.,,) represents the mapping function of the true perturbation, have:
the distribution of X, where x², represents the
Drv(ge. 9.) S
2 2expl-C2
Σ A(k. Ollea(82 et8 k31l + + + 2M) Σ x(k, 0) 1m0+1
kmst3
(42)
+ + 1 2 2expl (1 11 (1- 8.))118133 t
1 ITC 4= (1 - -Ba)
where C2 = (11-- / 11 k= 3x)) VII (1 -.), C2
and
8(1 11 11:0011100 (1 BB)))B
- 1-11 II: Ba)
1(k.
v1 - 1 == - 3,))
18

---

Proof. For ease of notation, we use the notation: ag := Br and ā == II. ag From the DDPM
sampling process 25 we know that:
1 -
x/-1 p=:=
X
so (x(x)t) 0,2
(43)
V
v1 Cx
1 -
_ de
X
(x/,
0,2
(44)
V
v1
ai
Ct--
where of
Bt.
Ht,g and Mt.p represent the mean of the distribution of and p/, respectively. Then from Equations (43
and we have:
1 az VaivI-a (so(2,, - so(x(, t))
Pt.
Pt.p
( - 11-1pp)
(45)
Vāa
Applying Equation
45
iteratively, we get:
pTa
PTp
II. IV (14-1,9 11-1p)
Σ
-
(so (xx R)) so (x R:)
ai
k=t
(46)
On the other hand, MT.q - HT, can formulated explicitly considering the Gaussian distribution at time step T' in the diffusion process:
PT, - MT = vār(Xo - xxo) = vārō
(47)
Combining Equations
46 and (7). we can derive that:
I pl,
(48)
(1 ar 111 -141 Va.
(n= Varl1612 +
Σ
100( k) 10)
(49)
v1 an
kmt+1
an M.
< (1 vorll00
Σ
((( +le -x((x( la)
(50)
vI ax
Le Σ A(k Σ A(k 1) (()) -
(51)
s <(Tart Var|16112 (vT= 2M)
Σ x(R00)+ Σ X(kt)]leo(X( -e(2) A)112
(1 az ) 1-_1va
where A(k, t)
0%
We then leverage the closed form formulation of the Hellinger distance between two Gaussian distributions [14] parameterized by μ1, M2, Σz:
expl- 1 (2 Ea), E2)) =1 det(E,) 1/4 det dete)())))
- (m1 122²²
E +
(M - [22)
del Σ1 + Σg
1/2
2
(53)
Applying it to distribution pi and g/o we have:
H2 gi) =1 1 -exp
1- āc
|| Me
8(1 1)8,
(54)
< 1- exeppCCC [1181122 (VIn + +MM Σ x(k,t)+ Σ
{(k,t)]lso 1.)
c( kt)l2
kmt41
(55)
19

---

1 0z
where C = (I Vār and C2 8(1 āt -3
Finally, it follows that:
Drv q SDvv pt) (p.( p/) )+ + Drv ((..
(56)
s <V2 20xp( (1 a.)]lo 3) (2H(()(.P))
(57)
8 2 Ccx
Cill812 + (vLn 2M)
Σ +(K,))+
Σ k=++1
A(k, (82,, - A)lla
(58)
+
2cxp(
8
(1 ac)111112) +
(59)
-exp
Σ
X(k. H)l, Callilaa + (vbu + 2M) Σ A(k,
(60)
+ 1/2 2- 2exp( 5(1 11 8.)31161133 fre
(61)
11 (1 BB,)
where C1 = (I 1 k=1
Bk)) II 8=1 Be). C2
and
8(1 11 -1 (1 -B))B
t(k,t) 8611 II: 3s)
11. 11 32)
D Experiment
D.1 Pseudo-code of DiffAttack
Given an input pair (x. y) and the perturbation budget, we notate L := Cele + XLdee (Equation (8) the surrogate loss, II the projection operator given the perturbation budget and distance metric, 17 the step size, a the momentum coefficient, Niter the number iterations, and W the set of checkpoint iterations. Concretely, we select Lat the cross-entropy loss in the first round and DLR loss in the second round following (34) The gradient of the surrogate loss with respect to the samples is computed by forwarding the samples and backwarding the gradients for multiple times and taking the average to tackle the problem of randomness. We also optimize all the samples, including the misclassified ones, to push them away from the decision boundary. The gradient can be computed algorithm in Section which enables DiffAttackk
with our segment-wise wrrwarding-bacwward be the first fully adaptive attack against the DDPM-based purification defense. The complete procedure is provided in Algorithm 2
D.2 Experiment details
We pretrained -bsed diffusion models 499
CIFAR-10, guided diffusion models [5
ImageNet, and DDPM (25) on CIFAR-10 to purify the images following the literature [34] 4 55 56
Due to the high computational cost, we follow [4 to randomly select fixed subset of 512 images sampled from the test set to evaluate the robust accuracy for fair comparisons. We implement DiffAttack in the framework of Attokt [12. and
the same hyperparameters. Specifically, the number of iterations attacks (Nier) is 100, and the number of iterations to approximate the gradients (EOT) is 20. The momentum coefficient a is 0.75, and the step size 17 initialized with 2e where is the maximum f no-mo of the perturbations. The balance factor À between the classification- guided loss and the deviated-recome reconstruction loss in Equation (8) is fixed 1.0 and a<-) is set the reciprocal of the size of sampled time steps in the evaluation. We consider 8/255 and 4/255 for Lo attack and 0.5 for lz attack following the literature
We use randomly selected 3 seeds and report the averaged results for evaluations. CIFAR-10 is under the MIT license and ImageNet is under the BSD 3-clause license.
More details of baselines. In this part, we illustrate more details of the baselines 1) SOTA attacks against bcore-based diffusion adjoint attack (AdjAttack) (34), 2) SOTA attack against DDPM-based
20

---

Algorithm 2 DiffAltack
1: Input: ::= Lels + ALdevs II, (x, v). 77, Niters W {wo. ..... wn
2: Output: x
3: x(0) t x
4: x(1) t IT (x + + 210; [(x(0), .y)
V
5: Lmux f maxx(((x(0) 1), [(x y)
6: x + ( if Lmxx = [(x (0), 4) else x t x()
7: for
Ninor -1 do
8: z (1++1) t 11 (x(1)) + 17 2 ((x(1), 4))
((++1) - II ((()) + az (x(1)+1 x (k)) + (1 a)( -x(-11)
9:
if ((x(1-1),
.y) Lmaxx then
11:
& ((++1) and LImax ((x(++1),
12:
end if
k W then
14:
f n/2
end if
16: end for
diffusion Diff-BDDA attack 4, 3) SOTA black-box attack SPSA [53 and square attack 11 and 4) specific attack against EBM-based purification joint attack (score/full) [60]. AdjAttack selects the surrogate loss C as the cross-entropy loss and DLR loss and leverages the adjoint method (28) to efficiently backpropagate the gradients through SDE. The basic idea is to obtain the gradients via solving augmented SDE. For the SDE in Equation (4), the augmented SDE that computes the gradients aL/ 0xx of back=propagating through it is given by:
X/T
X
sdeint
f. g. W. 0,T
(62)
ac
ac Ox
where 8C is the gradient of the objective L w.r.t. the x/, and 3x
frev t)
f(xx; z),
(x,) 2)) 1x
Brey (1) 0g
(63)
g(t)
(ww1111 -t)
((t)
w(1 -t)
with 1, and 0,, representing the d-dimensional vectors of all ones and all zeros, respectively and few (x, t) = - V log (x)), (1) := 33(t).
SPSA attack approximates the gradients by randomly sampling from a pre-defined distribution and using the finite-difference method. Square attack heuristically searches for adversarial examples in low-dimensional space with the constraints the perturbation pattern (i.c.. constraining the square shape of the perturbation). Joint attack (score) updates the input by the weighted average of the classifier gradient and the output the score estimation network (i.e., the gradient of liklliieldoood with respect to the input), while joint attack (full) leverages the classifier gradients and the difference between the input and the purified samples. The update of the joint attack (score) formulated as follows:
x レ x + r7 (X ('signnss (x) + (1 - 10--'')aa (x v))
(64)
The update of the joint attack (full) is formulated follows:
x サ x + 7 (N'sign(F(PPS)) - (1 (1-11)))-aee ()ssgn((V 8)
(65)
where 7) is the step size and X' the balance factor fixed 0.5 in the evaluation.
21

---

Table 5: Comparisons of gradient backpropagation time per balch(sscoondd/Mmmrry cost (MB)
between 4 and DiffAttack. We evaluate on CIFAR-10 10 with WideRes.Net 10 with batch size 16.
Method
T = 5
T= 10
= 15
T= 20
T= 30
T 1000
4
0.45/14,491 0.44/2,773
0.83/23,735 0.85/2,731
1.25/32,905 1.26/2,805
1.80/38,777 771
DiffAttack
1.82/2.819
2.67/2,884
85.81/3,941
Table 6: The clean / robust accuracy (%) of diffusion-basee purification with different diffusion lengths T under DiffAttack. The evaluation is done on ImageNet with ResNet-50 under &oo attack (e 4/255).
T 50
T = 100
T = 150
T = 200
71.88 12.46
68.75 24.62
67.79 28.113
65.62 / 26.83
D.3 Additional Experiment Results
Efficiency evaluation. We evaluate the wall clock time per gradient backpropagation of the segment- wise forwarding-bcckwardin backwarding algorithm for different diffusion lengths and compare the time efficiency well the memory costs with the standard gradient backpropagation in previous attacks 4 The results in Table 5 indicate that the segment-wise forwarding-backwardiee
algorithm consumes
comparable wall clock time per gradient backpropagation compared with [4] and achieves much better tradeoff between time efficiency and memory efficiency. The evaluation is done on an RTX A6000 GPU with 49,140 MB memory. In the segment-wise forwarding backwarding algorithm. we require one forward pass and one backpropagation pass in total for the gradient computation, while the standard gradient backpropagation in [4] requires one backpropagation pass. However, since the backpropagation pass is much more expensive than the forward pass [36], our segment- wise forwarding- wrrwardingbbackwardee warding algorithm can achieve comparable time efficiency while significantly reducing memory costs.
serwardig-bbccwwarding
More ablation studies ImageNet. We conduct more evaluations on ImageNet to consolidate the findings in CIFAR-10. We evaluate the clean/robust accuracy (%) of diffusion-based purification with different diffusion lengths T under DiffAttack. The results in Table 6 indicate that 1) the clean accuracy of the purification defenses negatively correlates with the diffusion lengths, and 2) moderate diffusion length benefits the robust accuracy under DiffAttack.
Tansferability of DiffAttack. ACA [0] and Diff-PGD attack (58) explore the transferability of unrestricted adversarial attack, which generates realistic adversarial examples to fool the classifier and maintain the photorealism. They demonstrate that this kind of semantic attack transfers well to other models. To explore the transferability of adversarial examples by by f--norm-bass based DiffAttack, evaluate the adversarial examples generated on score- score-based purification with on defenses with pretrained WRN-50-2 and DeiT-S. The results in Table 7
indicate that DiffAttack also transfers better than Adj Attak and achieves much lower robust accuracy on other models.
Ablation study of balance factor A. As shown in Equation (II) À controls the balance of the two objectives. A small can weaken the deviated-reconstruee from the vanishing/cxploded gradient problem, while large À can downplay the guidance of the classification loss and confuse the direction towards the decision boundary of the classifier. The results in Table & show that selecting À as 1.0 achieves better tradeoffss empirically, for experiments.
object and make the attack suffer more
fix it as 1.0
D.4 Visualization
In this section, we provide the visualization of adversarial examples generated by DiffAttack. Based on the visualization on CIFAR- 10 and ImageNet with different network architectures, we conclude that the perturbation generated by DifTAttack is stealthy and imperceptible to human eyes and hard to be utilized by defenses.
22

---

Table 7: Robust accuracy (%) with Lo attack (e 8/255) against score-based diffusion purification CIFAR-IO. The adversarial examples optimized the diffusion purification defense with pretrained ResNet-50 and evaluated on defenses with other types of models including WRN-50-2 and DeiT-S.
ResNet-50
WRN-50-2
DeïT-S
AdjAttack
40.93
52.37
54.53
DiffAttack
28.13
37.28
39.62
Table 8: The impact of different loss weights À on the robust accuracy (%). We perform Loo (e = 8/255) against score-based diffusion purification CIFAR-10 with WideResNet-28-10 and diffusion length 0.1.
== 0.1
X 1.0
À 10.0
54.69
46.88
53.12
Clean
Ground truth label
airplane
ship
automobile
frog
automobile
truck
frog
automobile
Adversarial
Prediction
deer
truck
ship
bird
truck
ship
dog
truck
Figure 5: Visualization of the clean images and adversarial samples generated by DiffAttack CIFAR-10 with Ex attack (e 8/255) against score-based purification with WideResNeet28--
Clean
Ground truth label
deer
deer
airplane
bird
truck
dog
truck
deer
Adversarial
Prediction
horse
dog
automobile
frog
automobile
cat
automobile
horse
Figure 6: Visualization of the clean images and adversarial samples generated by DiffAttack CIFAR-10 with for attack (c = 8/255) against score-based purification with 700-
23

---

Clean
Ground truth label
urchin
dock
Irish terrier
cabbage butterfly
Adversarial
Prediction
coral reef
ship
Norwich terrier
ringlet
Figure 7: Visualization of the clean images and adversarial samples generated by DiffAttack on ImageNet with Lo attack (e 4/255) against score-based purification with WideResNet-50-2.
Clean
Ground truth label
mushroom
safe
minivan
stove
Adversarial
Prediction
corn
mailbox
moving van
oven
Figure 8: Visualization of the clean images and adversarial samples generated by DiffAttack ImageNet with lo attack (e 4/255) against score-based purification with DeiT-S.
Clean
Ground truth label
screen
home theater
candle
coffee
Adversarial
Prediction
computer
desk
book case
red wine
Figure 9: Visualization of the clean images and adversarial samples generated by DiffAttack on ImageNet with larger perturbation radius: lor attack (e 8/255) against score-based purification with 550.
24


Reviews:
[{"review_id": "bEuLGBG4Gq", "reviewer": "Reviewer_3df8", "summary": "The paper introduces an adversarial attack focused on diffusion-based purification techniques. In diffusion-based purification, a (malicious) input is first noised and then denoised by a diffusion model before being fed through the classifier. The paper aims at solving the issues that arise when using backpropagation-based PGD attacks due to the size of the denoising diffusion graph. The two main contributions are:\n\n - A per-timestep deviated reconstruction loss that maximizes the distance between the noisy version of the original image at a certain timestep and the denoising reconstruction. \n - Segment-wise forwarding-backwarding which is essentially gradient checkpointing for diffusion graphs\n\nIn the evaluation. they test their method against a variety of SOTA baselines on CIFAR10 and ImageNet and are able to beat previously best achieved robust accuracies by a margin.\n\n\n", "questions": " - Especially in classifier guidance, it is common to denoise the current latent x_t to get a coarse estimate of x_0 and then calculate the gradient through this single-step approximation (For example \"UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance\" Li et Al 2022 ). You mention several strategies in 3.2 to prevent gradient obfuscation by introducing additional losses at intermediate steps. Would it be possible to apply such an approach to your problem and do you expect it to work better/worse than your deviated-reconstruction loss?", "limitations": " - I think the paper is very applicable to diffusion-based purifications and the authors do not mention strong limitations either. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": " - The evaluation protocol seems reasonable to me and the results are good across the board compared to the given baselines.\n - Overall the presentation is clear\n - Both gradient checkpointing and the deviated reconstruction loss are quite simple ideas that seem to work well at solving the problem at hand. ", "weaknesses": " - I think the formulation of the segment-wise forwarding-backwarding algorithm makes it seem more novel than it truly is. Gradient checkpointing has been used to compute gradients in deep computational graphs by trading memory for compute in a lot of papers. I agree that diffusion graphs are well suited for this approach as the diffusion latents that are stored are very compact in size compared to the activations inside the score-net, but I still believe that it would be clearer to the reader to just refer to it as gradient-checkpointing. \n - I personally find some arrows in Figure 1 to be a bit confusing. I do not really understand the red arrows DiffAttack in the top part of the graph at t=0 and it's not super clear how they interact with the grey arrows for segment-wise forwarding-backwarding. I understand how the method works from the description in the text and I think it's a good idea to have a figure illustrating gradient flow in the paper but I believe clarity could be improved here. \n - In my opinion, Section 3 should contain a brief description of diffusion-based purification defenses in mathematical notation as the paper is explicitly aimed at attacking them. \n - I think it should be clearly stated at (11) that the goal is to maximize the objective similar to (6)/(7). Maybe it would even improve clarity to add a max close to (8) as minimization is more common in deep learning literature\n - I like Section 4.7 but I think it could be improved by showing results without the added deviated-reconstruction loss at all.\n - I believe the theoretical connection between Theorem 1 and the rest of the paper is rather weak. The Theorem only connects total variation to a mismatch between the ground-truth score function and the network estimate of the score. I think this would be a lot more useful if the paper would actually maximize the total-variation distance or if the Theorem could be stated in terms of the quantity being maximized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: Related work of the gradient checkpointing technique.\n\nThanks for the suggestion. We will add discussions of the related work in our revision. To the best of our knowledge, prior works [a,b,c] propose the technique of gradient checkpointing to perform gradient backpropagation with memory efficiency. They store fewer activations during forwarding passes and construct the local computation graph via recomputation. However, we are the first to apply the memory-efficient backpropagation technique to attack diffusion purification defenses and resolve the problem of memory cost during attacks, which is realized as a challenging problem by prior attacks against purification defenses [d,e].\n\n*[a] Chen, Tianqi, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. \"Training deep nets with sublinear memory cost.\" arXiv preprint arXiv:1604.06174 (2016).*\n\n*[b] Chang, Bo, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham. \"Reversible architectures for arbitrarily deep residual neural networks.\" In Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1. 2018.*\n\n*[c] Gomez, Aidan N., Mengye Ren, Raquel Urtasun, and Roger B. Grosse. \"The reversible residual network: Backpropagation without storing activations.\" Advances in neural information processing systems 30 (2017).*\n\n*[d] Nie, Weili, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat and Anima Anandkumar. “Diffusion Models for Adversarial Purification.” International Conference on Machine Learning (2022).*\n\n*[e] Yoon, Jongmin, Sung Ju Hwang, and Juho Lee. \"Adversarial purification with score-based generative models.\" In International Conference on Machine Learning, pp. 12062-12072. PMLR, 2021.*\n\n> Q2: Improvements of presentations in Figure 1.\n\nWe apologize for the confusion in the figure. The red arrow in the top part denotes that the second line of examples is generated by DiffAttack, while the black arrow refers to the examples by standard adversarial (adv.) attacks. We will use another type of arrow to differentiate them from the arrows used for the classification flow. We compute the gradients with respect to the sample right before the classifier using standard techniques, and then the gradient backpropagation of DiffAttack is performed using a segment-wise forwarding-backwarding manner (the bidirectional grey dash line).   \n\n> Q3: Mathematical formulation of diffusion purification defenses.\n\nThanks for the suggestion. We will include the formulation of DDPM purification process (equation 1) and that of the score-based purification process (equation 3,4) with equation 7 for better understanding from a mathematical perspective. \n\n> Q4: Formulations related to optimization.\n\nThanks for the suggestion. We emphasize that we maximize the loss objective in equation 11. We also add a symbol of $\\max$ before equation 8 to clarify that we actually maximize the objective during DiffAttack as the following: $\\max \\mathcal{L}_{dev} = \\mathbb{E}_t [\\alpha(t) \\mathbb{E} _{\\mathbf{x}_t,\\mathbf{x}'_t|\\mathbf{x}_0} d(\\mathbf{x}_t, \\mathbf{x}'_t)]$.\n\n> Q5: Presentation of the results in Section 4.7.\n\nThanks for the suggestion. We also add the result without deviated-reconstruction loss and provide the results in Table 10 in the one-page rebuttal PDF.\n\n> Q6: Clarification of the connection between Theorem 1 and the rest of the paper.\n\nTheorem 1 indicates that deviated-reconstruction loss in Equation (8) induces inaccurate data density estimation, which results in the discrepancy between the sampled distribution and the clean training distribution. Therefore, it demonstrates that the deviated-reconstruction loss can induce inaccurate reconstruction of the clean distribution and benefit the optimization of adversarial examples. In the analysis, the total variation distance is used to quantify the distribution distance for rigorous derivation. In practice, we cannot compute the total variation distance between complex data distributions and can only optimize via finite samples. Therefore, we design the deviated-reconstruction loss in Equation 8 for the ease of optimization on sampled instances. \n\n> Q7: Discussion of the application of another strategy in the setting.\n\nUPainting [f] adds image-text matching guidance at the intermediate steps of the sampling process to promote the effectiveness of conditional scene generations. The main difference with DiffAttack is that UPainting attempts to promote cross-modal alignment while DiffAttack attempts to relieve the problem of gradient obfuscation via intermediate guidance. The idea of providing classification-guided regularization based on a one-step approximation of the clean image looks promising. However, after some initial trials of evaluations, we found that the accuracy of one-step denoised sample is pretty low for samples at the initial stage. Specifically, the accuracy is below $30\\\\%$ for half of the time steps on CIFAR-10 with WideResNet-28-10. Therefore, it is a concern that the classification guidance would not be effective. It would be interesting for future work to use more sophisticated methods to provide more effective classification guidance. For example, training a classifier with small Gaussian augmentations would possibly benefit a good intermediate classifier since the one-step approximation of the clean image is clearly inaccurate and mixed with Gaussian noises.\n\n*[f] \"UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance\" Li et al 2022.*"}, {"review_id": "iuWhY4vdIf", "reviewer": "Reviewer_EofG", "summary": "This paper proposes a framework, DiffAttack, to attack against diffusion-based adversarial purification defenses. To address the issue of gradient vanishing/exploding, the paper introduces the concept of deviated-reconstruction loss and theoretically analyzes its relationship with density gradient estimation. To overcome the challenge of high memory cost, the paper presents a segment-wise forwarding-backwarding algorithm. Empirical experiments demonstrate the effectiveness of the proposed method and provide observations that contribute to a better understanding of the properties of the diffusion process.", "questions": "1. Line 151 does not provide a specific equation for α(·).\n2. The paragraph from lines 157-165 should be placed before the paragraph from lines 147-156.\n", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The content is rich.\n2. The theoretical analysis is solid.\n3. The experimental results show improvements compared to the competing algorithm.\n", "weaknesses": "1. Section 3.2 discusses that maximizing the deviated-reconstruction loss in Equation (8) can lead to inaccurate data density estimation, which results in a sampling distribution that is inconsistent with the clean training distribution. However, it does not provide a detailed explanation of how this helps alleviate gradient vanishing/exploding.\n2. Additional experiments should be conducted to investigate the transferability of DiffAttack.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: Detailed explanation of the benefits of deviated-reconstruction loss to alleviate gradient vanishing/exploding.\n\nThanks for the suggestion. We will add the following explanations in Section 3.2. The deviate-reconstruction loss is applied at the intermediate time steps, and thus, the regularization is applied at relatively shallow layers of the computational graph compared to the standard classification loss, which is added at the end layer of the graph. The loss at relatively shallow layers has a shortened path of gradient backpropagation, thus relieving the gradient vanishing/exploding problem compared to the case only with the classification loss.\n\n> Q2: Additional experiments of transferability of DiffAttack.\n\nACA [a] and Diff-PGD attack [b] explores the transferability of unrestricted adversarial attack, which generates realistic adversarial examples to fool the classifier and maintain the photorealism. They demonstrate that this kind of semantic attack transfers well to other models. To explore the transferability of adversarial examples by $\\ell_p$-norm-based DiffAttack, we evaluate the adversarial examples generated on score-based purification withResNet-50 on defenses with pretrained WRN-50-2 and DeiT-S. The results in Table 9 in the one-page rebuttal PDF indicate that DiffAttack also transfers better than AdjAttack and achieves much lower robust accuracy on other models.\n\n*[a] Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., & Zhang, W. (2023). Content-based Unrestricted Adversarial Attack. arXiv preprint arXiv:2305.10665.*\n\n*[b] Xue, H., Araujo, A., Hu, B. and Chen, Y., (2023). Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability. arXiv preprint arXiv:2305.16494.*\n\n\n> Q3: Specifications for $\\alpha(\\cdot)$.\n\n$\\alpha(\\cdot)$ is a time-dependent coefficient to regularize different time steps with different importance factors. We perform related ablation studies in Section 4.7 and observe that adding deviated reconstruction loss to uniformly sampled time steps achieves the best attack performance and tradeoff compared with that of adding loss to the same number of partial time steps only at initial/final stages. Specifically, the uniform sampling strategy corresponds to the formulation $\\alpha(t)=1/k$ for $t ~ \\text{mod} ~ (T/k) =0$, where $k \\in Z^+$ controls the interval of uniform sampling. We will formulate the uniform sampling strategy explicitly in our revision.\n\n> Q4: Rearrangement of the explanations in line 157-165.\n\nThanks for the suggestion. We rearrange the explanations in line 157-165 and the explanations of equation 8 in Section 3.2 for better coherency."}, {"review_id": "HNtH2ffyHp", "reviewer": "Reviewer_yE5n", "summary": "This paper proposes a new adaptive attack to unify the robustness evaluation of diffusion-based purification defenses, including both score-based and DDPM-based diffusion models. The new attack mainly includes two novel techniques to overcome the challenges of evaluating diffusion-based purification defenses. The first technique is a regularizer that encourages a large distance between the forward images and their counterparts in the reverse process, which is proposed to tackle the vanishing or exploding gradients problem. The second technique is a forward-backward algorithm that allows for gradient back-propagation segment by segment, which is proposed to reduce the high memory cost. Evaluations on CIFAR-10 and ImageNet show that the proposed adaptive attack can lower the robust accuracy by a large margin while maintaining low memory costs.\n", "questions": "I am willing to raise my score if the following questions are adequately answered or clarified, ordered by importance.\n\n1. Quality 4, 5\n2. Quality 3\n3. Quality 6 (merge to Quality 3 if they are relevant)\n4. Originality 1\n5. Quality 7\n6. Fix Clarity 8", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "### Originality\n\n* **Novel attack techniques.** The overall attack is novel. The proposed regularizer is not surprising but requires some insight to apply properly. The segment-wise forward-backward algorithm is also novel to my knowledge.\n\n### Quality\n\n* **Comprehensive evaluations.** The evaluation includes two models on CIFAR-10 and three models on ImageNet, with two purification defenses and several baseline adaptive attacks.\n* **Good ablation studies.** Other studies of the diffusion lengths, memory cost, and the proposed loss term are relatively sufficient.\n* **The theorem unifies attacks against score-based and DDPM-based purification defenses.** The presented two theorems show that increasing the new loss term can increase the data estimation loss of both DDPM-based and score-based purification defenses, although these theorems are not sufficiently discussed or motivated.\n\n### Clarity\n\nN/A\n\n### Significance\n\n* **Timely topic.** Diffusion-based purification is a fast-developing direction in adversarial example defenses. Given the well-known challenges and the lacked systemized evaluation of these defenses, this paper tackles an important problem in adversarial example defenses.\n* **Strong results.** It is good to see that the proposed attack can lower the robust accuracy by around 10% to 20%. The memory cost is also very low compared with previous approaches.\n", "weaknesses": "### Originality\n\n**1. [Moderate] Missing related work for memory-efficient gradient back-propagation.**\n\nOne major contribution of this paper is the memory-efficient gradient back-propagation algorithm. However, this paper did not discuss this literature and potential previous approaches for improving memory efficiency, either for the general optimization problem or for the diffusion model specifically. It is also claimed that this algorithm *\"can be used for any discrete Markov process.\"* This is a strong claim and seems to contribute to a broader area. Yet, without a solid discussion of related work, it is unclear how to position this algorithm in the literature. For example, is there any existing memory-efficient algorithm that is also able to tackle the memory overheads when evaluating diffusion-based purification defense? If so, a further ablation study may be necessary to show why the algorithm in this paper is still needed. If not, it is recommended to highlight this contribution more.\n\n**2. [Minor] Attack framework vs. attack techniques.**\n\nThe proposed methods are more like novel attack techniques rather than an attack framework. As far as I can tell from Algorithm 2, the attack generally follows the existing PGD or APGD attack framework (gradient update + projection). It is suggested to tune down this claim if the authors could not differentiate the proposed attack's framework from existing ones.\n\n### Quality\n\n**3. [Major] Did not address the motivated challenge of unbounded randomness.**\n\nThe authors called out the *\"unbounded randomness\"* problem in many places throughout the paper (e.g., lines 6, 32, 43, 133, 226, 391). However, the paper did not explain how this problem was resolved by the two proposed techniques. The missing explanation is confirmed by the summarized technical contributions in L69-83, where the two proposed techniques both tackle one of the remaining two challenges. There are two places discussing the randomness problem but likely for some other contexts: the expectation in Equation (8) and the optimization decision in L234-240. However, neither of these discussions could explain the \"unbounded randomness\" challenge in the original evaluation.\n\nPlease clarify if the proposed attack could tackle the randomness problem. If not, it is suggested to discuss why the proposed attack does not need to tackle the randomness problem to obtain strong results. If that is indeed the case, I am curious if the results in this paper would connect to the recent studies showing that randomness might be unhelpful in defenses [A, B]. Otherwise, is it possible to lower the robust accuracy even more if the randomness problem is explicitly tackled? Combined together they could strengthen this paper by showing some insights into the most challenging problem in evaluating purification defenses (gradient, memory, or randomn[[2023-06-28]]ess).\n\n[A] On the Limitations of Stochastic Pre-processing Defenses. NeurIPS 2022.\n[B] Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators. arXiv 2023.\n\n**4. [Major] It is unclear how the gradient issue was resolved.**\n\nThe authors claimed that the proposed deviated-reconstruction loss in Equation (8) could resolve the gradient problems, including both gradient obfuscation (L48, L72, etc.) and gradient vanishing/exploding (L10, L141, etc.). However, it is unclear how these problems are resolved by the proposed loss term.\n\nThis issue has three folds.\n\n1. **[Minor] It is unclear what exact problem is being resolved** when the authors mixed the concepts of \"gradient obfuscation\" and \"vanishing/exploding gradients.\" Specifically, \"gradient obfuscation\" is a general concept that also includes non-differentiable defense components, where the gradient is undefined and hence zero, yet \"vanishing/exploding gradients\" specifically refers to the defined but overly small or large gradients in differentiable defense components. While these are minor conceptual confusion, it is suggested to be consistent in the wording and clarify if it is the \"non-differentiable\" or \"differentiable but vanishing/exploding\" problem that the technique aims to solve, as their solutions typically require different strategies.\n2. **[Moderate] It is unclear if the gradient issue indeed exists in previous evaluations.** While it is true that deep networks could have the problem of vanishing or exploding gradients, they have not been explicitly observed by previous evaluations of purification-based defenses, such as DiffPure [29]. Given this, it is hard to justify if the proposed techniques could obtain stronger results because they have indeed resolved the gradient issues or some other undiscovered reasons.\n3. **[Major] It is unclear why adding a loss term B would resolve the gradient problem of the existing loss term A.** Specifically, if there were a gradient problem in the original robustness evaluation (with only $\\mathcal{L}\\_{cls}$), it means that the loss term $\\mathcal{L}\\_{cls}$ has already exhibited some form of gradient obfuscation (be it non-differentiable or vanishing gradients). In this case, why would adding a new loss term $\\mathcal{L}\\_{dev}$ would be able to resolve the gradient problem in $\\mathcal{L}\\_{cls}$? For example, if the total loss shows non-vanishing gradients, it is very likely that the gradient of $\\mathcal{L}\\_{cls}$ is still vanishing, and the observed gradient solely comes from the added $\\mathcal{L}\\_{dev}$. While it is explained how $\\mathcal{L}\\_{dev}$ can relieve the gradient problem (L153), it only explains how $\\mathcal{L}\\_{dev}$ relieve the gradient problem of itself but not $\\mathcal{L}\\_{cls}$.\n\n**5. [Major] Unclear effectiveness of the proposed regularizer.**\n\nGiven that it is unclear how the gradient issue was resolved, I feel the effectiveness of $\\mathcal{L}\\_{dev}$ should require more exploration from two perspectives.\n1. If it indeed solves the gradient problem of $\\mathcal{L}\\_{cls}$, the authors should provide evidence showing that the problem (1) exists before adding $\\mathcal{L}\\_{dev}$, and (2) disappears after adding $\\mathcal{L}\\_{dev}$.\n2. If it cannot solve the gradient problem of $\\mathcal{L}\\_{cls}$ but mostly optimizes the gradient problem of itself, then it seems to me that making forward-backward samples deviated is more effective than increasing the classification loss, which is an interesting insight. In this case, I would recommend the authors explore the rationale behind this observation, and highlight the efforts of resolving gradient problems in $\\mathcal{L}\\_{dev}$ rather than $\\mathcal{L}\\_{cls}$. Additionally, an ablation study of the loss weight $\\lambda$ should be included.\n\n**6. [Moderate] Unclear discussion about randomized purification.**\n\nThe discussion at L234 is unclear (even with L674-676).\n* First of all, if a single prediction has a high variance, the ultimate prediction should be an ensemble of multiple predictions, e.g., through majority vote. After that, it seems reasonable for attacks not to evaluate misclassified samples. From my understanding, this choice is not to reduce the computational cost, but a fact that you do not need to attack it -- you already find an adversarial example with distance zero.\n* I did not see the logical connection between L234-L237 and L237-L240, mostly because it is unclear if the first part refers to the fixed clean image or the sampled images during the attack.\n* Also, the effectiveness of this choice was not evaluated. Is this choice a must-have for the final results?\n\n**7. [Minor] Visualization of purified noise.**\n\nSince this paper was able to circumvent purification defenses, it is suggested to also visualize the adversarial noise before and after the purification defense. Right now only the pre-purification adversarial examples are provided.\n\n### Clarity\n\n**8. [Major] The presentation is confusing and contrived.**\n\n* Be consistent when describing the gradient problem (detailed in Quality-4.1). The current presentation uses \"gradient obfuscation\" and \"vanishing/exploding gradients\" interchangeably, but the previous term is a general concept that also includes non-differentiable problems. In Section 3.2, jumping between these two terms and the proposed technique is very distracting. It is suggested to use only one of them and be clear about what specific problem you are trying to solve.\n* The intuition behind Equation (8) is not clear until paragraph L166. It is suggested to explain the intuition (deviation, not the gradient problem) first.\n* The paragraph L157 comes out of the blue and is very dense, it might fit more at the end of Section 3.2.\n* Theorem 1 also comes out of the blue. The use of this theorem is unclear until L189. It is suggested to explain the remark (L187) or at least its high-level idea before introducing the theorem.\n* The use of Theorem 3 was presented without any discussion.\n* It is a bit hard to determine what exact defense was evaluated, it would be good to specify the defense name rather than citations.\n\n**9. [Minor] Typos and notations.**\n\n* Undefined terms\n\t* L151, $d()$\n\t* L172, \"data density estimation\"\n\t* L177, VP-SDSE\n\t* L206, $f\\_d$ and $f\\_r$\n * Typos\n\t* L142, Path -> Pass\n\t* L152, sampled -> reconstructed.\n\n### Significance\n\nN/A", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable suggestions, and we are glad that the reviewer finds our work novel, timely, with strong results. \n> Q1: Related work for memory-efficient back-propagation\n\nThanks for the suggestion. Prior works [a,b] propose the technique of gradient checkpointing for backpropagation with memory efficiency. They store fewer activations in forwarding passes and construct local computation graphs via recomputation. However, we are the first to apply a memory-efficient technique to attack purification defenses and resolve the problem of memory cost during attacks, which is a challenging problem as pointed out by [c,d]. We will add these detailed related work discussions in our revision.\n> Q2:  Attack framework vs. attack techniques\n\nThanks for the suggestion. We will update our claim to make it clear as an attack technique.\n> Q3: Clarification of unbounded randomness\n\nThanks for the great question. We clarify that DiffAttack tackles the randomness problem from two perspectives: 1) sampling multiple times in Eq. (8), and 2) optimizing perturbations for all samples including misclassified ones in all steps. Perspective 1) provides a more accurate estimation of gradients. Perspective 2) ensures a more effective attack optimization since the correctness of classification is of high variance over different steps. For instance, the classification result of a sample can be viewed as a Bernoulli distribution. We should reduce the success rate of Bernoulli distribution of samples by optimizing them with a larger attack loss, which would lead to a lower robust accuracy. In other words, one observation of failure in classification does not indicate that the sample has a low success rate statistically, and thus, 2) helps to continue optimizing the perturbations towards a lower success rate (i.e., away from decision boundary). \n> Q4 1): Gradient obfuscation vs. vanishing/exploding gradients\n\nThanks for pointing it out. As gradient obfuscation is a more general concept, we will follow the suggestion and keep using vanishing/exploding gradients in our revision.\n> Q4 2): Gradient issues in prior evaluations\n\nDiffusion purification process induces an extremely deep graph. Specifically, DiffPure applies $100$ iterations of sampling and uses deep UNet with tens of layers as score estimators. Thus, the computational graph consists of thousands of layers, which could cause the problem of gradient vanishing/exploding. Prior work [d] also mentions the gradient problem (Section 4, 5.1 in [d]) and attempts to bypass it by BPDA attack, which is shown to be less effective compared with DiffAttack in Table 4. We will make it more clear.\n> Q4 3): Clarification of how added loss resolves gradient problem\n\nThanks for the question. We propose the deviated-reconstruction loss, which is applied at the intermediate time steps of the computational graph, while the standard classification loss can only be applied at the end of the graph. Therefore, our proposed $L _{dev}$ will suffer less from the gradient problem compared to $L _{cls}$. We will clarify that *adding $L _{dev}$ will not solve the gradient problem of loss $L _{cls}$*. However, our $L _{dev}$ can lead to a deviated reconstruction of the image, which is easier to be misclassified and thus guides the optimization of searching adversarial examples effectively. \n> Q5: More explanations of the added regularizer\n\nThanks for the feedback. $L _{dev}$ suffers less from the gradient problem compared with $L _{cls}$ (illustrated in Q4 (3)), and thus the objective of $L _{dev}$ can be optimized more easily, but it does not solve the gradient problem of $L _{cls}$. On the other hand, the optimization of $L _{dev}$ benefits the optimization of $L _{cls}$ in the way that $L _{dev}$ can induce a deviated reconstruction of the image with a larger probability of misclassification. We follow the suggestion to add ablation studies of the weights $\\lambda$ to show the interplay of $L _{dev}$ and $L _{cls}$. A small $\\lambda$ can weaken the deviated-reconstruction object and makes the attack suffer more from the vanishing/exploded gradient problem, while a large $\\lambda$ can downplay the guidance of the classification loss and confuse the direction towards the decision boundary of the classifier. The results in Table 8 in the rebuttal PDF show that selecting $\\lambda$ as 1.0 achieves better tradeoffs empirically, so we fix it as 1.0 for experiments.\n> Q6: Discussion about randomized purification\n\nThanks for the question. As discussed in Q3, we can view classification results in steps as a Bernoulli distribution and optimize all samples towards a lower attack success rate. We agree that doing multiple evaluations and making optimization decisions can be an alternative to tackle the randomness problem with an accurate estimate of success rate. \nIn L234-237, we point out the randomness problem induced by only optimizing the fixed clean images, and then we illustrate the proposed optimization technique in DiffAttack in L237-240. We will add these explanations for better clarification.\n> Q7: Visualization of samples after purification\n\nFollowing the suggestion, we add additional visualization in Figure 9 in the rebuttal PDF, showing that adversarial examples before and after the purification are stealthy.\n\n> Q8: Presentation and notations\n\nThank you for the feedback. We will consistently use vanishing/exploded gradients. We illustrate the intuition of Eq. 8 (L166-173) before discussions in L148. We add high-level illustrations before Thm. 1 and 3. We also keep using names for all defenses.\n\n*[a] Chang et al. Reversible architectures for arbitrarily deep residual neural networks. AAAI 2018.*\n\n*[b] Gomez et al. The reversible residual network: Backpropagation without storing activations. NIPS 2017.*\n\n*[c] Nie et al. Diffusion Models for Adversarial Purification. ICML 2022.*\n\n*[d] Yoon et al. Adversarial purification with score-based generative models. ICML 2021.*"}, {"review_id": "DOnXLhL8ha", "reviewer": "Reviewer_Vu7G", "summary": "This paper aims to generate adv-samples that can effectively attack diffusion-based purification. Two key strategies are proposed: (a) deviated reconstruction loss which is used as a surrogate loss to relieve gradient obfuscation existing in the long computational graph; (b) segment-wise forward-backward strategy is applied to relieve the high memory cost to calculate accurate $L_{cls}$. The experiments show that the method proposed is effective.", "questions": "refer to `Weaknesses`", "limitations": "NULL", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, the paper is well-written and easy to read, the clarification is clear and \n\n- The problem is important since diffusion-based purification is proven to be effective and model-agnostic.\n- The proposed deviated reconstruction is proven to be effective.\n- The segment-wise forward-backward is effective and well-design\n- The experiments results are extensive.\n\n", "weaknesses": "I mainly have some questions here:\n\n- Will the segment-wise forward-backward affect the speed compared with the original forward-backward?\n- Intuitively, the adv-samples generated by DiffAttack are (1) hard to be purified by one specific DM, (2) hard to denoise at each stage in a given DM. When the budget $\\epsilon$ is larger, e.g. 8, 16 for imagenet, what will the adv-samples be like? \n- The code is not provided in the supp materials, it will be better to show some demo code\n- Do well need to carefully design the $\\lambda$ weights for the two losses?\n- (Some related paper) Can DiffAttack also transfer better to other networks? This topic is quite related to some recent papers, which prove that only attacking the diffusion model can be transferred well. [1, 2]\n\nI am willing to raise my score if most of the questions are solved.\n\n\n\n[1] Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., & Zhang, W. (2023). Content-based Unrestricted Adversarial Attack. arXiv preprint arXiv:2305.10665.\n\n[2] Xue, H., Araujo, A., Hu, B. and Chen, Y., (2023). Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability. arXiv preprint arXiv:2305.16494.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: The speed of segment-wise forward-backward compared with the original forward-backward.\n\nDue to the page limits, in line 361-362, we mention that we provide comparisons of runtime between DiffAttack and [a] (standard forward-backward) in Appendix D.3. The results demonstrate that DiffAttack reduces the memory cost with comparable runtime. The reason is that we require one forward-backward pass for each segment while standard methods only require one backward pass, but since the backpropagation pass is much more expensive than the forward pass. Therefore, our segmentwise forwarding-backwarding manner demonstrates similar runtime as the original forward-backward.\n\n*[a] Blau, Tsachi, et al. \"Threat model-agnostic adversarial defense using diffusion models.\" arXiv preprint arXiv:2207.08089 (2022).*\n\n> Q2: Visualizations of adversarial samples with a larger perturbation on ImageNet.\n\nWe follow your suggestion to add a visualization of the adversarial examples by DiffAttack with a larger perturbation. The results in Figure 10 in the one-page rebuttal PDF indicate that the perturbation generated by DiffAttack is stealthy and imperceptible to human eyes and hard to be utilized by defenses.\n\n> Q3: Codes of DiffAttack.\n\nWe provided the anonymous link of the code repo in line 693 in the supplementary material. We will make it more clear in the main text.\n\n> Q4: Ablation studies of weights for the two losses $\\lambda$.\n\nWe follow the suggestion to add ablation studies of the weights $\\lambda$ to show the interplay of $\\mathcal{L} _{dev}$ and $\\mathcal{L} _{cls}$. According to Equation (11), $\\lambda$ controls the balance of the classification-guided loss $\\mathcal{L} _{cls}$ and the deviated-reconstruction loss $\\mathcal{L} _{dev}$. A small $\\lambda$ can weaken the deviated-reconstruction object and makes the attack suffer more from the gradient obfuscation problem, while a large $\\lambda$ can downplay the guidance of the classification loss and confuse the direction towards the decision boundary of the classifier. We evaluate the effectiveness of different loss weights $\\lambda$ in Table 8 in the one-page rebuttal PDF. The results show that selecting $\\lambda$ as $1.0$ achieves better tradeoffs empirically. For consistency, we fix $\\lambda$ as $1.0$ for all the experiments.\n\n> Q5: Transferability of DiffAttack to other networks.\n\nACA [b] and Diff-PGD attack [c] explores the transferability of unrestricted adversarial attack, which generates realistic adversarial examples to fool the classifier and maintain the photorealism. They demonstrate that this kind of semantic attack transfers well to other models. To explore the transferability of adversarial examples by $\\ell_p$-norm-based DiffAttack, we evaluate the adversarial examples generated on score-based purification withResNet-50 on defenses with pretrained WRN-50-2 and DeiT-S. The results in Table 9 in the one-page rebuttal PDF indicate that DiffAttack also transfers better than AdjAttack and achieves much lower robust accuracy on other models.\n\n*[b] Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., & Zhang, W. (2023). Content-based Unrestricted Adversarial Attack. arXiv preprint arXiv:2305.10665.*\n\n*[c] Xue, H., Araujo, A., Hu, B. and Chen, Y., (2023). Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability. arXiv preprint arXiv:2305.16494.*"}]

Author rebuttal:
We thank all the reviewers for their comments and valuable feedback. We are glad that the reviewers find our work novel, effective, and well-written. We have made the following major updates following the reviews’ suggestions to further improve our work.

1. We add related work of memory-efficient gradient back-propagation to the literature review, following the suggestions from Reviewer yE5n and 3df8.

2. We make more clarifications of the benefits of the deviated-reconstruction loss to the gradient problems, following the suggestions from Reviewer yE5n.

3. We provide more visualizations to demonstrate the stealthiness of DiffAttack, following the suggestions from Reviewer yE5n and Vu7G.

4. We add additional ablation studies of the loss weights, following the suggestions from Reviewer yE5n and Vu7G.

5. We explore the transferability of DiffAttack for different model architectures, following the suggestions from Reviewer Vu7G and EofG.

6. We improve the presentations of the arrows in Figure 1 and the mathematical formulation of the optimization problem, following the suggestions from Reviewer 3df8.

Meta-review: This paper introduces an attack on diffusion based defenses to adversarial examples. The attack is effective and improves attack success rates by 10-20%. The reviewers raise a number of important points of clarification, but almost all of these were addressed to both the reviewers and my satisfaction during the discussion period. Ultimately after discussion all reviewers recommend acceptance and I agree here: the paper makes a valuable contribution. I encourage the authors to make the clarifications to the paper that helped the reviewers see what was being done.
Decision: Accept (poster)