DiffHammer: Rethinking the Robustness of Diffusion-Based Adversarial Purification

Authors: kaibo  wang
xiaowen  fu
yuxuan  han
yang  xiang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 14:26:30

Abstract: Diffusion-based purification has demonstrated impressive robustness as an adversarial defense. However, concerns exist about whether this robustness arises from insufficient evaluation. Our research shows that EOT-based attacks face gradient dilemmas due to global gradient averaging, resulting in ineffective evaluations. Additionally, 1-evaluation underestimates resubmit risks in stochastic defenses. To address these issues, we propose an effective and efficient attack named DiffHammer. This method bypasses the gradient dilemma through selective attacks on vulnerable purifications, incorporating $N$-evaluation into loops and using gradient grafting for comprehensive and efficient evaluations. Our experiments validate that DiffHammer achieves effective results within 10-30 iterations, outperforming other methods. This calls into question the reliability of diffusion-based purification after mitigating the gradient dilemma and scrutinizing its resubmit risk.

Main paper: 
 introduction: 
 The vulnerability of deep neural networks (DNNs) to adversarial samples hinders their application in security-critical domains, where attackers can deceive DNNs by introducing carefully crafted noises [25, 9] . To mitigate this issue, numerous defense strategies have been proposed to enhance their robustness, among which diffusion-based purification has emerged as a promising approach [22, 26, 3] . Diffusion models [11, 24] are designed to construct stochastic processes from noisy data distributions to cleaner ones. As a result, the presence of small adversarial noise can be drowned out by larger noise, which can then be iteratively denoised using the diffusion model [22, 26] or purified through a diffusion-involved optimization [3] . The iterative algorithm and stochasticity of diffusion enhance their purification capabilities empirically, yet they also present a challenge in evaluating their robustness [16, 13] . There are ongoing concerns regarding their effectiveness:Whether their effectiveness originated from inherent robustness or insufficient evaluation?The Expectation of Transformation (EOT) [2] method allows attacks to adapt across stochastic purifications by averaging the gradients of sampled purifications. It maximizes attack success rate under the assumption that most purifications share a common vulnerability, i.e., susceptible to a same adversarial noise. However, high stochasticity in diffusion-based purification challenges this assumption, rendering EOT-based attacks ineffective. Purifications with unshared vulnerability (S 0 , shown in Figure 1 ) will provide inconsistent gradients in the attack, leading to a gradient dilemma. Moreover, obtaining gradients for purification is time-consuming, and this dilemma further increases computational overhead. d, e ) Selective attack targets on S 1 to avoid the gradient dilemma. We identify purifications from S 1 in the E-step and aggregate the gradients of these purifications in the M-step.To address this, we introduce a selective attack method, DiffHammer2 , based on expectation maximization (EM) [5] . As shown in Figure 1 , our algorithm bypasses the gradient dilemma by iteratively identifying vulnerable purifications in the E-step and aggregating their gradients in the M-step. This approach focuses on shared vulnerabilities, akin to using a hammer on weak spots rather than the entire structure. Additionally, we enhance attack efficiency by reducing the diffusion process's backpropagation complexity from O(N ) to O(1) through gradient grafting.Another issue with diffusion-based purification is insufficient evaluation. Attackers aiming for a single success, such as logging in, can achieve a higher chance of success by resubmitting an adversarial sample to the stochastic defense [20] . This resubmit risk is underestimated in 1-evaluation tailored for deterministic defenses, especially given the high stochasticity of the diffusion process. To address this, we upgraded 1-evaluation to N -evaluation and seamlessly integrated it into our DiffHammer framework, offering a more comprehensive risk assessment without extra time costs.We revisit the robustness of diffusion-based purification within the DiffHammer framework. DiffHammer surpasses other state-of-the-art attacks [4, 21, 13, 16, 1, 2] by circumventing the gradient dilemma and achieving near-optimal effectiveness within 10-30 iterations. Under N -evaluation protocol and our selective attack, we find that the risk associated with diffusion-based purification is significantly underestimated. Mainstream purifications [22, 26, 3] fail to withstand even 10 resubmit attacks and may degrade the performance of robust models, indicating that their robustness potential remains underutilized. We hope that DiffHammer's insights into evaluating diffusion-based purification robustness will foster more robust defenses. Our main contributions are summarized as follows:• We identified the limitation of EOT-based attacks and proposed an efficient and effective evaluation, termed DiffHammer, to comprehensively diagnose the adversarial risks of diffusion-based purification. • We introduce a selective attack that avoids the gradient dilemma by targeting the shared vulnerabilities, with the process expedited by gradient grafting. • We validate the effectiveness of DiffHammer through extensive experiments on mainstream purifications, revealing underestimated risks through standardized N -evaluation. 
adversarial attacks: 
 Given an image x ∈ R d and its label y ∈ [K], a classifier f : R d → R K with a preprocessor ϕ : R d → R d will classify it as f [ϕ(x)] = arg max k=1,...,K f k [ϕ(x)], where K is the number of classes. The attacker aims to add an imperceptible adversarial noise r on the image to make it misclassified, i.e., f [ϕ(x + r)] ̸ = y. Adversarial noise is typically obtained by maximizing the loss function L while imposing ℓ p -norm constraint for imperceptibility. For instance, projected gradient descent (PGD) [21] updates ∥r∥ ∞ ≤ ϵ bounded adversarial samples as:EQUATIONwhere α is step size and Π indicates a projection operator.Preprocessors tailored to defend against adversarial noise may lead to vanishing, exploding, or inaccurate gradients. Attackers can resort to approximating the gradient using Backward Pass Differentiable Approximation (BPDA) [1] , where the preprocessor is typically treated as an identity mapping during the backpropagation:EQUATIONEOT [2] can be utilized as an additional approach to estimate gradients of stochastic preprocessors, whose effectiveness is derived from the assumption that different preprocessors have shared vulnerabilities. Given N sampled preprocessors ϕ i , the estimated gradient is:EQUATIONThe diffusion model [11, 24, 14] establishes a link between the clean data distribution p 0 (x) and the noisy distribution p 1 (x) through forward (noising) and backward (denoising) processes. The forward process x t can be expressed as a stochastic differential equation (SDE) for t from 0 to 1 [24] :EQUATIONwherex 0 ∼ p 0 (x), w t ∈ R d is a standard Wiener process, f : R d × R d → R dis the drift coefficient and g : R → R is the diffusion coefficient which is designed so that p 1 (x) follows a standard Gaussian distribution N (0, I d ). DDPM [11] can be viewed as a special case of f (x, t) := -β(t)x/2 and g(t) := β(t), where β(t) is the noise scheduler, usually set as a linear function w.r.t. t.The evolution of reverse-time SDE (t from 1 to 0) then corresponds to the generation of samples:EQUATIONwhere dt is an infinitesimal negative time step and wt is a standard reverse-time Wiener process, and ∇ xp t (x) is known as time-dependent score function and is typically estimated by the neural network.Well-trained diffusion models possess the ability to accurately model the score function and denoise noisy samples, enabling adversarial purification. Considering the small magnitude of adversarial noise, Diffpure [22] therefore floods the adversarial noise while preserving the semantic information by running a forward process from 0 to t * , and then purifies it through a denoising process. GDMP [26] leverages the distance from the initial samples as guidance to further preserve semantic information.Leveraging the fact that diffusion model approximates the score function of x ∼ p 0 (x) more accurately, likelihood maximization (LM) [3] optimizes adversarial samples to minimize estimation error, aligning their distribution with p 0 (x). 
diffhammer: 
 To address ineffectiveness in the attack phase and insufficiency in the evaluation phase, we propose DiffHammer to assess the robustness of diffusion-based purification. First, DiffHammer overcomes the gradient dilemma by designing a selective attack (Section 3.1) using the EM algorithm. Furthermore, we seamlessly integrate N -evaluation (Section 3.2) into the algorithm, enabling more accurate resubmit risk diagnosis and facilitating the attack by providing approximate gradients. Here P is the probability density associated with the stochastic purification ϕ, and p denotes the probability density in logits of f given a specific ϕ. We refer to the gradient of loss L(f [ϕ(x + r)], y) w.r.t. x, ϕ(x + r) as the full gradient (∇ x L ϕ ) and the approximate gradient (∇ ϕ(x+r) L ϕ ), respectively. Without ambiguity, we sometimes abbreviate the loss as L ϕ and gradients as ∇ x L ϕ , ∇ ϕ L ϕ .Assumption. We assume that diffusion-based purifications have unshared vulnerabilities, dividing ϕ into two sets. We denote S 1 as the largest set of ϕ with shared vulnerability, i.e., S 1 = arg max P ({ϕ : f [ϕ(x+r ⋆ )] ̸ = y for a same r ⋆ }) and S 0 = S1 as the set of ϕ compromised by inconsistent adversarial noise r. We assume that optimization towards ϕ ∈ S 0 suffers from the gradient dilemma, leading to ineffective attacks. Our task is to identify S 1 and design r for ϕ ∈ S 1 .Indicating whether a ϕ belongs to S 1 or S 0 by z = 1 or 0, we denote q(z) as the estimated distribution for z for given ϕ. According to the Jensen's inequality, we maximize a lower bound of our goal:EQUATIONwhere Q(q, r) represents the evidence lower bound, and KL(q ∥ p(z | A, r)) is the KL divergence between q and the posterior distribution of z. Given the coupling of variables r and z in the optimization process, we employ the EM algorithm [5] as a solver that alternates between optimizing r (maximizing Q(q, r)) in the M-step and estimating z (minimizing KL divergence) in the E-step.During the M-step, we maximize Q(q, r) w.r.t. r to raise the objective's lower bound while keeping q(z) fixed. By omitting terms unrelated to r, i.e., q(z) ln q(z) and the priors ln p(z = 0), ln p(z = 1), the goal simplifies as maxEQUATIONwhere the second term is further disregarded in the optimization according to our assumption.In adversarial attacks, the objective E ϕ ln p(A | r, z = 1) is typically replaced by maximizing the loss function L ϕ (x + r), achieved through average gradient-based methods. Consequently, the M-step update can be integrated into existing attack algorithms like PGD [21] and AA [4] as a plug-in, with the difference being the reweighting of the gradient from each ϕ by q(z = 1). We further employ stepwise-EM [18] to linearly interpolate the current gradient and the previous gradient with weight t -α for online updating, where t is the number of iterations and α is a hyperparameter. This approach intuitively optimizes the adversarial noise towards the shared vulnerability of S 1 , thus avoiding the gradient dilemma in S 0 .During the E-step, we update q(z = 1) to p(z = 1 | A, r) for a given r, thereby eliminating the KL divergence, which is the gap between the objective and its lower bound. Notice that S 1 is defined as the largest set of ϕ susceptible to the same adversarial noise r ⋆ , where usually r ⋆ ̸ = r. We can estimate p(z = 1 | A, r) by approximating r ⋆ with by-products L ϕ (x + r) and ∇ x L ϕ x+r in the observation of misclassification event A. Denote the difference between r ⋆ and r as ∆r, the loss L ϕ (x + r ⋆ ) w.r.t. r ⋆ can be linearly approximated since ∆r is not excessively large, with both r and r ⋆ bounded by ∥•∥ p ≤ ϵ:EQUATIONHigher loss increases the likelihood of misclassification, so we map L ϕ (x + r ⋆ ) to p(A | r ⋆ ) using a monotonically increasing function σ : R → [0, 1]. By the definition of S 1 , r ⋆ compromises as many ϕ as possible. We find r ⋆ through the following optimization:EQUATIONAfter determining r ⋆ , the probability q(z = 1) that ϕ belongs to S 1 can be estimated as σ(L ϕ (x+r ⋆ )).Remark. We use the empirical average w.r.t. N instances ϕ i , i = 1, . . . , N in each iteration to estimate expectations. In the E-step, we optimize r ⋆ in a low-cost manner, as no model is involved. We reweight the gradient by σ(L ϕi (x + r ⋆ )) in the M-step and then use it in an off-the-shelf attack algorithm to update the adversarial noise r. The primary time cost arises from calculating the gradient ∇ x L ϕi due to the complexity of the purification process. In Section 3.1.2, we will describe how to reduce the O(N ) complexity to O(1) through gradient grafting.Algorithm 1: DiffHammer Input :Data (x, y), classifier f with stochastic purification ϕ, number of resubmit M , off-the-shelf attack algorithm Output :Robustness Rob in M resubmit attacks Initialize r (0) ; for t ← 1 to T do Evaluation for t -1 iteration and input for t iteration;EQUATIONϕ i L ϕ i // for attack; E-step: identify the set with shared vulnerability;∆r (t) = arg max i σ(L (t) ϕ i + ∆r T ∇ (t) ϕ i L ϕ i ) ; q (t) i = σ(L (t) ϕ i + ∆r T ∇ (t) ϕ i L ϕ i ) //M-step: estimate the full gradients' aggregation;g(t) = i q (t) i ∇ (t) ϕ i L ϕ i /N // aggregation in ϕ stage;Select φ according to Equation 11 // representative ϕ; g (t) = Backward ( φ(x + r (t-1) ) T g(t) ) // gradient grafting; g (t) = t -α g (t) + (1 -t -α )g (t-1) // stepwise update; r (t) = AttackAlgorithm (r (t-1) , g (t) ); end Rob (T ) = Eval (r (T ) , M ) with ϕi, i = 1, • • • , N ; return Rob = min(Rob (t) , t = 1, • • • , T ) Classifier PurificationTo address the efficiency challenges in robustness evaluation for diffusion-based purification, we aim to estimate the weighted gradient aggregation i q i (z = 1)∇ x L ϕi with minimal computational cost. Our approach aggregates low-cost approximate gradient ∇ ϕi L ϕi in the early stage to estimate the weighted full gradient, which can be expressed asEQUATIONassuming ∂ϕ i /∂x can be approximated by ∂ φ/∂x. As illustrated in Figure 2 , We first compute the weighted approximate gradient N i=1 q i (z = 1)∇ ϕi L ϕi , which is then grafted onto φ for backpropagation to estimate the gradient expectation, reducing complexity in backpropagation x ← ϕ i (x) to O(1). BPDA [1] uses I to approximate ∂ϕ i /∂x, offering computational simplicity but potentially compromising performance due to oversimplification. Our gradient grafting enhances estimation with one additional backpropagation, achieving a better balance between efficiency and effectiveness. Further related design details are discussed below. E-step in ϕ stage. Although r ⋆ in E-step (Equation 9) involves the full gradient ∇ x L ϕi , our main interest is in σ(L ϕ (x + r ⋆ )). Therefore, we can optimize in the stage of ϕ rather than x to avoid dependence on the full gradient. We express ∆r in the stage of ϕ as ∆r = ϕ(x + r ⋆ ) -ϕ(x + r). The optimization, aimed at attacking as many ϕ as possible, becomes max E ϕ σ(L ϕ (x + r) + ∆r T ∇ ϕ L ϕ ), relying only on the approximate gradient. We end up deriving σ(L ϕ (x + r ⋆ )) from optimized ∆r.Choice of φ. We aim to select a ϕ with representative vulnerability for backpropagation, whose adversarial noise can also affect other ϕ. For a single-step attack on ϕ with adversarial noise g(∇ ϕ L ϕ ) (e.g. sign(∇ ϕ L ϕ ) in ℓ ∞ case), we choose φ among ϕ i using the strategy:EQUATIONThis consistent optimization goal of attacking as many ϕ as possible maintains the shared vulnerability from the ϕ stage to the x stage. The choice of φ involves discrete optimization within a finite set, solvable by traversal when N is not very large.Our selective attack (targeting S 1 ) enhances both the effectiveness and efficiency of the EOT-based attack (targeting S 0 ∪ S 1 ). (1) The EOT-based attack is a specific instance of our algorithm when all ϕ share a common vulnerability. In this case, with S 0 = ∅, q(z = 1) = When r A and r B are significantly different, S 1 tends to be A or B, whereas the other becomes S 0 and provides neutralized gradients. Empirical results indicate that the gradient dilemma is common in diffusion-based purification, highlighting the necessity of selective attacks. Traditional 1-evaluation is insufficient for assessing the robustness of diffusion-based purification, particularly against resubmit attacks. As a stochastic defense, the model produces inconsistent results for even the same queries, enabling attackers to achieve desired outcomes through resubmissions. In cases where attack costs are manageable and even a single success is advantageous, e.g., login, defenders should focus on the model's robustness over M resubmissions. Thus, we propose using Nevaluation as a robustness evaluation protocol for two reasons:(1) As detailed in Theorem 2, 1-evaluation are significantly biased in estimating M -resubmit risk. Due to high stochasticity, it is common for ϕ to have unshared vulnerabilities, as shown in Figure 3 . Merely success or failure record in 1-evaluation fails to capture critical probability information for resubmit risk estimation. For instance, with DiffPure, 46.3% of samples have a attack success rate (ASR) P (S 1 ) ∈ (0, 1), leading to a 17.9% overestimation of 10-resubmit robustness in 1-evaluation ( Rob (2) N -evaluation can be integrated into the attack's loop without extra burden. Samples ϕ i , i = 1, • • • , N serve both as an evaluation for the previous iteration and input for the current, which will not cause information leakage, as each evaluation involves unseen instances.Our DiffHammer framework integrates selective attacks and N -evaluation, as detailed in Algorithm 1. Notably, our N -resubmit robustness extends the traditional 1-submit metric to better assess risks in real-world deployments, orthogonal to our attack algorithm's design. Consequently, DiffHammer offers a more comprehensive risk assessment and enhances attack effectiveness across various metrics.Table 1 : Performance of attacks against diffusion-based purification on CIFAR10. Metrics include Avg./Wor. Rob, (%) and iterations (it.) taken to reach 90% best performance (in parentheses, failure to reach is noted as N/A). Attack algorithms include SOTA white-box attacks with EOT: BPDA [1] , PGD [21] , AA [4] , DA [13] , and DiffHammer (DH, ours); and transfer-based attacks (denoted by †): DMI [29] , TMI [8] , VMI [27] , SVRE [30] .DiffPure [22] GDMP [26] LM [ 4 Experiments 
experimental setup: 
 Baselines. We evaluated the robustness of three diffusion-based purification defenses: DiffPure [22] , GDMP [26] , and LM [3] . For purification, we used a pre-trained score-based diffusion model [24] in DiffPure and GDMP and an EDM model [14] in LM. To ensure a fair comparison, we employed the WideResNet-70-16 [32] as the classifier across all tests.We selected three state-of-the-art attack algorithms equipped with EOT [2] for baseline evaluation:BPDA [1] , PGD [21] , and AA [4] . For DiffPure and GDMP, AA was upgraded to DA [13] by incorporating deviated-reconstruction loss. Our Diffhammer adopts AA as the default attack algorithm. We conducted three restarts totaling 150 iterations to thoroughly evaluate the robustness of the model. Additional configurations for defense and attack are detailed in Appendix C.1. Substitute gradient attacks [22, 31] tailored to diffusion-based processes are found to be inferior to PGD with full gradients [16] , so we leave comparisons with these methods in the Appendix C.4. Additionally, we evaluate verifiable DiffSmooth classifiers [34] , and the results are presented in the Appendix C.3.Evaluation metrics. Consistent with prior work, we use subsets of the CIFAR10 [15] , CI-FAR100 [15] , and ImageNettete [12] (a subset of 10 easily classified classes from Imagenet [6] , more suited for robustness evaluation) with sizes of 512, 512, and 256 as datasets, respectively. The evaluation protocol is N -evaluation with N = 10, with the average robustness (Avg.Rob,EQUATIONi /N S) and worst-case robustness (Wor.Rob,1 -i (max i A (j)i )/S) as metrics. Here A (j) i indicates whether sample j was attacked at the i-th evaluation, and S is the dataset size. We also reported the iterations taken to reach 90% of the best attack effect among all attacks as a metric of efficiency. Experimental results for CIFAR100 can be found in Appendix C.2, and adversarial samples visualization are shown in Appendix C.9. We examine the effectiveness and efficiency of various attack methods under different settings, with results for CIFAR and ImageNettete presented in Table 1 ,2, respectively. Our findings are as follows.(1) DiffHammer shows superior attack effectiveness across datasets and different norms (ℓ ∞ , ℓ 2 ) or attack budgets. This effectiveness arises from selective gradient aggregation, which avoids the gradient dilemma. Its performance improves significantly in high-stochasticity scenarios with pronounced gradient dilemmas, e.g., large-scale datasets like ImageNettete or GDMP with multiple purification rounds. (2) As shown in Figure 4 (other settings can be found in Appendix C.5), DiffHammer typically requires only 10-30 iterations to reach near-optimal results, allowing for rapid model robustness assessment. Other methods might occasionally bypass sampling from S 0 for similar effectiveness, which incurs unnecessary computational costs. (4) Most models show robustness below 50% with 10 resubmits, indicating that a limited number of resubmits can undermine diffusion-based purification, raising concerns about their reliability in practical applications.We verify the gradient dilemma in diffusion-based purification by examining clustering effects and forgetting phenomena. Our EM algorithm clusters the gradients ∇ x L ϕ into S 0 and S 1 . The distribution of silhouette coefficients (SC) using cosine similarity, as displayed in Figure 5 (a), indicates that gradients in these sets differ significantly. A direct result of this gradient dilemma is attack forgetting-where gradients ∇ x L ϕ in consecutive iterations are inconsistent, causing the effects of previous attacks to be forgotten. As the ASR in t -1 iteration shown in Figure 5 (b), DiffHammer maintains attack consistency by identifying S 1 , thus enhancing efficiency. We provide a toy example explaining the gradient dilemma in Appendix C.6. This dilemma may arise from a non-clustered data distribution (e.g., different breeds of dogs in the dataset), which imposes divergent gravitational pulls for purification and inconsistent perturbations needed to corrupt their features.Transfer-based attacks offer a potential solution by treating adversarial samples like models and aiming to improve generalization on a dataset of ϕ. As shown in 1, Data augmentation-based approaches, DMI [29] and TMI [8] , provide improvement in some cases. VMI [27] and SVRE [30] , which aim to reduce gradient variance, perform worse. This unexpected outcome is attributed to the gradient dilemma: generalizing to ϕ from S 1 is beneficial, while generalizing to ϕ in S 0 may be harmful. Thus, DiffHammer acts as a data-selection approach, contributing to data-centric design in transfer attacks. [33] and AWP [28] . The original adversarial robustness (Rob) without purification is listed in parentheses.Classifier AWP [28] (Avg.Rob / Wor.Rob, Rob:60.0) ↓ TRADES [33] (Avg.Rob / Wor.Rob, Rob:53.1) ↓ Purification DiffPure [22] GDMP [26] LM [3] DiffPure [22] GDMP [26] LM [ Diffusion-based purification has been explored to enhance the robustness of adversarially trained (AT) models [22] . We revisited this concept using N -evaluation with TRADES [33] and AWP [28] as AT classifiers, as shown in Table 3 . Our observations are as follows: (1) Robust models mitigate the gradient dilemma, resulting in similar performance for most attacks. This occurs because adversarial training implicitly regularizes the approximate gradient ∇ ϕ L ϕ [23] , reducing the full gradient ∇ x L ϕ of some ϕ towards zero. DiffHammer remains an effective evaluation tool, as diffusion-based purification occasionally reintroduces the gradient dilemma. (2) In most cases, diffusion-based purification weakens the robustness of the AT model. It improves robustness in a few instances but increases the risk of resubmit attacks by 11.7%-24.8%. Therefore, finding a better combination of diffusion-based purification and robust classifiers remains an open question. We conducted ablation experiments on different components in the default settings (grafted gradient ∂ φ/∂xE(q ϕ ∇ ϕ L ϕ ), α = 0.5, N = 10), with results shown in Table 4 . The use of a selective approximate gradient E(q ϕ ∇ ϕ L ϕ ) leads to a significant performance drop, highlighting the importance of the purification gradient ∂ φ/∂x in attacks. The grafted gradient causes only slight performance degradation compared to the full gradient E(q ϕ ∇ x L ϕ ) but avoids a ∼ 10× time burden. We quantify the time cost of different approaches in the Appendix C.7. A smaller hyperparameter α makes the algorithm rely more on the current iteration's gradient, and an appropriate α = 0.5 achieves a better tradeoff in memorizing and learning. We tested effectiveness at 10-evaluations with N samples of ϕ, finding that a proper N = 10 efficiently identifies S 1 for evaluation. When N = 20, attack effectiveness is slightly boosted with more time overhead. The importance of N -evaluation for resubmit risk estimation is verified in the Appendix C.8. 
discussion and insights: 
 Diffusion models remain a promising solution to the adversarial samples due to their fine-grained modeling of data distributions. We advocate for enhancing the robustness of diffusion-based purification through standardized and powerful evaluation methodologies. Here are some insights into purification-based defense and attack:(1) For deploying stochastic defenses, defenders should consider the potential number of resubmissions M by attackers and are advised to assess resubmit risk with N -evaluation where N ≥ M . Additionally, robustness overestimation due to the gradient dilemma can be avoided by using selective attacks. (2) On the attack side, adversarial samples are sensitive to defenses with high stochasticity.Thus, modern data-centric designs may help to enhance adversarial transferability. (3) On the defense side, the goal of stochastic defense is to achieve P (S 0 ) → 1, meaning samplex cannot be attacked for some purifications and cannot be attacked by a same adversarial noise for others. Therefore, purification needs to be coordinated with adversarial training at a more granular level. 
related work: 
 Evaluation for adversarial purification. As test-time adaptive defenses, the iterative process and stochasticity of diffusion-based purification complicate robustness evaluation. Typically, evaluation protocols report only the ASR from a single evaluation. The risk of resubmit attack, referred to the nag factor, is considered in [20] but is not fully explored. Regarding evaluation methods, although there are gradient estimators AdjAttack and attacks score-attack based on the characteristics of the diffusion process, both were found to be inferior to attacks based on the exact gradient [16] , such as PGD [21] , AA [4] , or DiffAttack [13] with reconstruction loss. These methods mitigate stochasticity through the EOT [2] , which inevitably inherits its shortcomings.Transfer-based attacks. Transfer-based attacks aim to generalize attacks from seen defenses to unseen defenses. Data augmentation and improved optimization are the main approaches to enhance transferability. Attackers can craft adversarial noise resistant to defenses through input transformations [29] and gradient smoothing [8] . In the presence of multiple defenses, momentum [7] , variance reducing [27, 30] enhance generalizability. Most transfer-based attacks assume that most defenses share vulnerabilities, enabling generalization in defenses, but this assumption may be invalid in diffusion-based purification. 
conclusion: 
 In this paper, we address the limitations of EOT-based attacks in diffusion-based purification, attributed to the gradient dilemma, by introducing an effective and efficient method called DiffHammer. First, we propose a selective attack strategy that targets vulnerable purifications without encountering the gradient dilemma, enhancing evaluation efficiency through gradient grafting. Second, we incorporate N -evaluation within the loop to quantify the risk of achieving at least one successful attack in practice. We demonstrate DiffHammer's superior performance through comprehensive experiments and anticipate it will offer valuable insights for future designs of robust diffusion-based purification methods. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: All experiments are performed on 1 NVIDIA GeForce RTX 4090 with a memory of 24564 MB.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We make broader impacts in the first paragraph of Appendix A.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: We don't include data or models that have a high risk for misuse. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We cite properly the dataset and code of original owners in Section 4.1 Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: Broader Impact. The vulnerability of deep neural networks to adversarial samples limits their broader application. Diffusion-based purification has shown surprising robustness against EOTbased attacks, making it a promising defense. In this paper, we identify that EOT-based attacks often overestimate model robustness due to the gradient dilemma. Additionally, traditional 1evaluations are insufficient for assessing resubmit risk in stochastic defenses. We propose an effective robustness evaluation framework, DiffHammer, which circumvents the gradient dilemma and incorporates N -evaluation. Our selective attack approach applies to stochastic defenses beyond diffusion-based purification, enhancing the understanding of stochasticity in robustness and inspiring improved adversarial defenses. Our N -evaluation method uncovers resubmit risks in diffusion-based purification, providing comprehensive insights into potential threats from stochastic defenses and informing precautionary measures. Furthermore, we enhance attack efficiency through gradient grafting, enabling rapid iteration of attacks and defenses. We believe that the insights and techniques introduced in our work will guide the development of more robust diffusion-based purification methods, ultimately enhancing the security of machine learning systems.Limitations. In this work, we present a new attack algorithm called DiffHammer that is effective against diffusion-based purification defenses. While this has potential misuse in safety-critical domains like healthcare or finance, the core insights uncovered could also inform the development of more robust diffusion-based defenses going forward. Specifically, we introduce a selective attack strategy to overcome the challenge of the gradient dilemma. However, the underlying reasons behind this gradient dilemma are still not fully understood. Our experiments are limited to image classification, and while our approach can extend to other tasks, specific task-related designs need exploration. Overall, this research uncovers vulnerabilities in current diffusion-based defenses, but the findings could also spur innovations to make these defenses more secure in the future. Proof. The obejective for EOT-based attacks is max r P (A)L A (r) + P (B)L B (r), while the obejective for attacks targeting either A or B is max r P (A)L A (r) or max r P (B)L A (B) and the optimal points are r A and r B , respectively. For any r ∈ R d , there exist θ ∈ R s.t. r = r A + θ(r B -r A ). Therefore, the following inequality holds: Proof. First, we provide detailed proof for the uniformly minimum-variance unbiased estimator (UMVUE) case, where we shall use the Lehmann-Scheffé theorem.EQUATIONTheorem 3 (Lehmann-Scheffé theorem [17] ). Let X 1 , • • • , X N be random variables from an unknown distribution f (x; θ) where θ ∈ Ω is a parameter in the parameter space. Suppose T (X) is a sufficient and complete statistic. If and only if condition for η(T ) to be UMVUE of g(θ) is thatE θ [η(T )] = g(θ).We denote the results of N resubmissions as A 1 , • • • , A N , and the probability of successful attack for each resubmission as p.Then A 1 , • • • , A N follows Bernoulli distribution, i.e. A i i.i.d.∼ Bin(1, p). According to the definition, Rob = (1-p) M . Since the Bernoulli distribution is one of the exponential family of distributions, the sample sum N i=1 A i is known to be the complete and sufficient statistics. Naturally, we haveEQUATIONIf N ≥ M , the conditional expectation can be written asEQUATIONwhich do not depend on p since the sum A 1 + • • • A N is sufficient, and therefore can be used as an estimator according to Rao-Blackwell Theorem:EQUATIONDue to the uniqueness of the UMVUE, the estimator is only available when N ≥ M .Next, we show that when N ≤ M , the maximum likelihood estimator (MLE) tends to overestimate Rob in expectation, i.e., E( Rob(N )M LE ) > Rob. Following the notation above, the MLE estimator for p is n/N . Due to the transformation invariance of the MLE, the MLE estimator for Rob := (1 -p) M is (1 -n/N ) M ; here, n follows binomial distribution Bin(N, p). Therefore, the expectation of the MLE estimator isEQUATIONTherefore, estimating resubmit risk with MLE requires a larger N . When N ≤ M , MLE will underestimate such risk. When N goes larger, the MLE estimator is asymptotically normal. SinceEQUATIONDiffPure [22] . DiffPure adds noise to potentially adversarial samples x 0 through the forward process:EQUATIONwhere α t is a predefined scheduler and ᾱt = t i=1 α i , ϵ is a standard Gaussian noise from N (0, 1). The reverse process gradually removes the Gaussian noise along with the adversarial noise:EQUATIONwhere ϵ θ (x t , t) is a parameterized neural networks, z is standard Gaussian noise in each step and standard deviation σ t = 1-ᾱt-1 1-αt (1 -ᾱt ). In the experiment, t * is set to 0.1 (ℓ ∞ ) and 0.075 (ℓ 2 ) in CIFAR [15] , and in ImageNettete [12] , t * is set to 0.05. We use the surrogate process proposed in [16] , i.e., applying different time intervals in attack and defense to improve the efficiency. The time interval for the attack is set to 0.01 (CIFAR, ℓ ∞ ; ImageNettete), 0.015 (CIFAR, ℓ 2 ). The time interval for defense is set to 0.002 (ImageNettete), 0.005 (CIFAR).GDMP [26] . Compared to DiffPure, GDMP adds gradient guidance for image similarity D(x t , x t ′ ) in forward and reverse processes to preserve semantic information. The update of x t-1 can be written as:EQUATION)where s is a scale of guidance andx t ′ = √ ᾱt ′ x 0 + √ 1 -ᾱt ′ ϵ.In addition, GDMP observes that multiple rounds of purification with smaller t * help to improve robustness.In the experiment, t * is set to 0.036 in CIFAR [15] , and 0.03 in ImageNette [12] . The rounds of purification are 4 and 2, respectively. The time interval for the attack is set to 0.006 (CIFAR) and 0.003 (ImageNette). The time interval for defense is set to 0.003 (CIFAR), 0.001 (ImageNette).LM [3] . LM is the preprocessor in the robust diffusion classifier (RDC) that maximizes the lower bound of the log-likelihood through the following optimization process:EQUATION)wherext = √ ᾱt ′ x + √ 1 -ᾱt ′ ϵ and η is a predefined threshold.The above optimization in the experiment was solved by the projected gradient method with a step size of 0.1 and 5 iterations. t is uniformly sampled from a uniform distribution of 0.4-0.6. It is worth noting that the RDC consists of the LM and a diffusion classifier (DC). We find that the purifier LM is not sufficient to provide reliable robustness. We conjecture that the robustness of RDC mainly stems from the diffusion-based classifier.Each attack is performed with three restarts, consisting of 50 iterations per restart, totaling 150 iterations. Samples with an attack success rate below 50% are restarted to save time. We utilize the APGD [4] as the update algorithm for DiffHammer and BPDA [1] . The step size in PGD [21] is set to 0.007, and the momentum coefficient is set to 1. In VMI [27] and SVRE [30] , we replace the gradient sampled from the neighborhood of x for variance estimation with approximate gradients sampled on ϕ, adapting to stochasticity defense. Other attacks employ default parameter settings. We evaluate the effectiveness of various attacks on diffusion-based purification using CIFAR-100 [15] . Unlike its predecessor, CIFAR10, which had 10 categories, CIFAR100 contains 100 categories, providing a greater fine-grained classification challenge. For consistency, we employ the EDM unconditional diffusion model [14] as the purifier and a pre-trained PyramidNet [10] as the classifier.Given the increased complexity, we set the attack budget to 4/255. As shown in Table 5 , DiffHammer consistently achieves superior attack results and efficiency across all settings, indicating the persistence of the gradient dilemma in finer-grained purification.Notably, finer classifications seem benefit less from diffusion-based purification, with classifiers with diffusion-based purification even demonstrating less than 50% robustness without adversarial attacks. This may be due to current diffusion models' limitations in fine-grained generation, potentially introducing vulnerabilities during reconstruction. Thus, designing diffusion models with enhanced alignment for improved robustness in fine-grained classification remains an open challenge.DiffSmooth [34] enhances certified and empirical robustness through a purified classifier in the inner loop and a sampling-based certification process in the outer loop. Since certified robustness provides a theoretical lower bound on model robustness, we focus on attacking the inner-loop purified classifier. DiffSmooth's purification mechanisms involve single-step purification and majority voting, which can mitigate the risk of resubmission. Unlike DiffPure, DiffSmooth uses a single update:EQUATIONwhere t * satisfies ᾱt * σ 2 = (1 -ᾱt * ) and σ is a predifined noise scale. DiffSmooth applies local smoothing through Gaussian sampling in the neighborhood of purified x0 , and results are determined by majority voting with scale σ ′ : Random smoothing models are primarily used to defend against adversarial noise under the ℓ 2 norm. We compared the impact of different attacks on DiffSmooth with settings of ℓ 2 : 0.5 and ℓ 2 : 1.0.EQUATION)For the ℓ 2 : 0.5 setting, we used σ = 0.5, σ ′ = 0.25, applying ResNet-110 trained with Gaussian smoothing as the classifier. For the 1.0 setting, σ = 0.25, σ ′ = 0.12, with SmoothAdv-trained ResNet-110 as the classifier. A pretrained score-based diffusion model [24] acted as the purifier. As the result shown in Table 6 , majority voting was found to suppress the gradient dilemma, reducing DiffHammer's impact, though it still achieved better attack results with significant efficiency gains.While majority voting defends against occasional resubmissions, it also poses challenges: (1) It can make the stochastic defense model resemble a deterministic one, potentially reducing stochasticity benefits.(2) It imposes a significant time burden, intensified by the diffusion process's duration. DiffSmooth mitigates this with single-step purification, but this results in more fragile purification.Stochastic and iterative algorithms for diffusion-based purification yield challenging gradients computation, so a line of works aimed at approximating gradients in attack algorithms. This includes the Adjoint method [22] and Joint methods (score / full) [31] .Adjoint method obtain the gradient ∇ x L = √ ᾱt * ∇ x t * L through an augmented SDE of Equation 5, which is solved as:EQUATIONwhere sdeint is an SDE solver that sequentially takes six inputs: initial value, drift coefficient, diffusion coefficient, Wiener process, initial time, end time; andf ([x; z], t) = f (x, t) ∂f (x,t) ∂x z g(t) = -g(t)1 d 0 d w(t) = -w(1 -t) -w(1 -t)with 1 d and 0 d representing the d-dimensional vectors of all ones and all zeros, respectively.The gradient in the joint attack (score) is a weighted sum of the approximate gradient and the estimated score function:EQUATIONwhere s θ (x) is the estimated score function ∇ x log p(x), ∇ ϕ L is the approximate gradient, and λ is a balance factor set as 0.5 in the evaluation. The gradient in the joint attack (full) utilizes the difference between the original sample and the purified sample:EQUATIONWhile substitute gradient methods can produce gradients based on diffusion purification with lower time complexity, they have been found to inadequately evaluate robustness [16, 13] . We assess the effectiveness of substitute gradients across different defenses, as shown in the Table 7 . The substitute gradient fails to achieve 90% of DiffHammer effect in all settings, indicating a notable overestimation of model robustness. Specifically, the joint attack (score/full) only enhances the approximate gradient by the difference between samples and score function, lacking information from the diffusion process.Although the adjoint method theoretically provides an accurate gradient, in practice, numerical errors accumulate, leading to inaccurate estimates.To address this, we utilize computational graph reconstruction proposed in [13] to obtain accurate gradients for DiffPure [22] and GDMP [26] . We progressively backpropagate the gradient by saving intermediate variables during iterations. Additionally, we derive the gradient of the optimization process in LM [3] using the Hessian vector product trick. This approach enhances existing gradientbased attacks with accurate gradient computation, establishing a more reliable baseline.We present performance curves of the attack under various settings. Figure 6 compares DiffHammer with a transfer-based attack, showing that DiffHammer achieves superior results and faster convergence. Momentum-based attacks like DMI [29] and TMI [8] demonstrate good efficiency. However, transfer-based attacks yield sub-optimal results due to attempts to generalize across all purifications. Figure 7 and 8 illustrate performance curves under different constraints (ℓ ∞ : 4/255 and ℓ 2 : 0.5). Notably, DiffHammer significantly improves performance and efficiency under the ℓ ∞ : 4/255 setting. With a limited attack budget, DiffHammer leverages the EM algorithm's fast convergence to quickly identify vulnerable set of ϕ. In contrast, attack performances are more similar under the ℓ 2 : 0.5 setting, likely because the ℓ 2 constraint allows greater attack intensity on certain pixels, targeting shared vulnerabilities. Figure 9 and 10 compare attack effectiveness across datasets, revealing that DiffHammer excels in the ImageNette task, where complex purification processes seem more prone to introducing gradient dilemmas.To investigate the origins of the gradient dilemma, we constructed a Gaussian mixture toy example where the two diagonal components belong to the same category as shown in Figure 11 (a). We use ReFlow [19] as our diffusion model to produce near-linear purification trajectories. In this scenario, diffusion-based purification pulls samples slightly towards the origin (dashed line in Figure 11(a) ) before diffusing them back into the data distribution (pink line in Figure 11(a) ). Given that the optimal classifier is a heteroscedastic classifier aligned with the coordinate axes, a sample's gradient direction is either horizontal or vertical, depending on which cluster the noised sample is closer to.We demonstrated the proportion of S 1 for each sampled point x in Figure 11(a) , where values nearing 50% indicate the presence of a gradient dilemma. Simulation results revealed that samples along the diagonal experience severe gradient dilemmas, which is consistent with intuition. In this toy example, the gradient dilemma arises from a non-clustered data distribution, which imposes divergent gravitational pulls for purification. Consequently, we hypothesize that real-world data distributions (c) LM [3] Figure 13 : MLE estimation of the risk of 5 resubmit attacks for different N -evaluation. The reference value is represented by the solid line and its standard deviation by the dashed line.may also exhibit non-clustered features. For instance, in the misclassification of a cat as a dog, considering the existence of various dog breeds, it is more efficient and effective to distort the features of one specific breed rather than all breeds collectively.Our gradient grafting method enables the acquisition of gradients with lower time complexity for efficient attacks. We utilize only byproducts of N -evaluation, with the primary overhead being the computation of approximate gradients, which is less costly than full gradients. Our algorithm's computational cost involves N approximate gradients and one full gradient, whereas a full gradientbased algorithm with 1-evaluation requires N full gradients. When comparing computational times for varying N , we find that increasing N slightly raises costs as the result shown in Figure 12 .Compared to the N full gradient computations in the EOT-based attack, DiffHammer computes the full gradient only once, and the cost of N approximate gradients is acceptable. As N increases, the efficiency gain from gradient grafting becomes more significant.In a threat model where an attacker tries to obtain at least one successful attack by resubmitting, inappropriate evaluation can lead to an underestimated risk. For example, in labs that use facial recognition as authentication, users have up to five attempts. Therefore, the estimation of the 5resubmit risk is critical for the defender. The defender can perform N resubmit evaluations and use MLE as an estimator. We quantify the MLE-estimated risk for DiffHammer with N -evaluation, N = 1, . . . , 10 on a 5-resubmit attack, where the reference value is an empirical average of multiple trials. As the results in Figure 13 show, as the number of resubmissions increases, the estimation gets closer to the reference value with a smaller variance and tends to be stable and accurate when N ≥ M . It is worth noting that when N ≪ M , the estimation is either significantly underestimated (N = 1) or suffers from large variance (N = 2). Therefore, we recommend at least N -evaluation with N ≥ M as a means of evaluation. We visualize adversarial samples generated by DiffHammer against DiffPure [22] under various settings (CIFAR: Figure 14 ,15,16; CIFAR100: Figure 17 ; ImageNette: Figure 18 ). These samples are imperceptible even with a perturbation budget of 8/255. However, subtle adversarial perturbations can lead to significant differences in model decisions, highlighting the need for stronger adversarial defenses. NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.The codes are publicly available at https://github.com/Ka1b0/DiffHammer. 


Decision: The authors propose a sufficient and efficient attack named DiffHammer against diffusion-based purification.
The reviewers agree that the paper makes interesting analysis and conclusions and please make sure to incorporate the improvements in the final version of the paper/

Reviews: review: paper_summary: paper_summary: This paper reveals two limitations of Eot-based attacks in diffusion-based purification: gradient dilemma and underestimation of resubmit attacks' risk. The authors introduce N-time evaluation to evaluate the risk of resubmitted attacks sufficiently. Then, they propose an EM-based attack to solve the gradient dilemma, which achieves satisfactory results compared to both Eot-based and transfer-based attacks. Finally, this paper guides the future improvement of diffusion-based purification.
strength_weakness: strength_weakness: In experiments:

- In line 232, the author would better explain why they chose "BPDA [1], PGD [21], and AA [6]" as baselines. The published years of [1], [6], and [21] are 2018, 2020, and 2017, it seems these methods are relatively old works. The authors would better consider a comparison with newer methods, such as "Diffusion models for adversarial purification. ICML 2022, https://arxiv.org/abs/2205.07460".
- In lines 239 and 240, the author would better provide the measuring unit of time.
- The authors would better add horizontal and vertical labels for the figures in their paper, including figures 2, 4, 5, and 6.
- In line 268, "as shown in DiffHammer retains the memory of...", it seems to be "as shown in Figure 4, DiffHammer retains the memory of...". In addition, the author would better explain why the transfer-based attack DTI only appears in Table 1 and is not compared in Figure 5 and Figure 4. In subsection 4.3, the authors also don't analysis DTI.
- In line 253, the author would better explain how is "14%" and "28%" calculated or obtained from the experimental results.
questions: questions: Please check weaknesses part.
limitations: limitations: I think that the authors have adequately addressed the limitations mentioned in this paper and the potential negative societal impact of their work is controllable.
paper_summary: paper_summary: Diffusion-based purification methods have gained recognition for their robustness against adversarial attacks. However, concerns arise regarding the adequacy of current evaluation methods, particularly in addressing the gradient dilemma inherent in these techniques. This paper introduces DiffHammer, an advanced evaluation framework utilizing an EM-based attack and N-time evaluation to overcome these challenges. DiffHammer identifies vulnerabilities in purification clusters more effectively than traditional methods, significantly enhancing the assessment of diffusion-based purification robustness. Their experiments demonstrate that DiffHammer outperforms existing approaches by identifying more adversarial samples and highlighting previously underestimated security risks.
strength_weakness: strength_weakness: 1. The paper describes that when the purification process contains unshared vulnerabilities, existing EoT-based N-averaging can encounter the gradient dilemma. This gradient dilemma might lead to the inability to generate effective attack samples, resulting in insufficient evaluation of diffusion-based purification methods. However, the paper lacks theoretical proof or experimental validation for this gradient dilemma.

2. The paper mentions that diffusion model-based purification methods like Diffpure, GDMP, and LM have issues with resisting resubmit attacks. However, to my knowledge, DiffSmooth[1], another diffusion model-based purification method, takes into account multiple denoised outputs for the same input sample when making predictions, and it claims to have a certifiably robust pipeline. This defense method appears to be more powerful, and the authors should consider including such methods when evaluating the robustness of diffusion-based adversarial purification techniques. Additionally, it would be valuable to investigate whether resubmit attacks still pose a problem for such robust defense methods.

[1] Zhang J, Chen Z, Zhang H, et al. {DiffSmooth}: Certifiably robust learning via diffusion models and local smoothing[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 4787-4804.
questions: questions: See weaknesses.
limitations: limitations: The authors have clearly described the limitations.
paper_summary: paper_summary: The paper proposes a new adversarial attack framework against diffusion-based purification defenses. The paper first explains the advantage of using N-time evaluation as the metric for randomized defenses. Then, it proposes an E-M based adversarial attack, which empirically shows SOTA ASR.
strength_weakness: strength_weakness: I think the randomness of diffusion purification is worthy of noticing and exploring in an EM framework with distribution-level analysis, which is a viable way from a high level. However, the major concern is that the paper does not convey the attack analysis in the EM framework very clearly. It might be a problem of presentation, but additional explanations are definitely needed for a fair judgment of the method.

1. Some definitions are not clear and rigorous. Line 141: missing rigorous definition of $g$. Line 143 says that $g(\phi)$ depends on a cluster, but the formulation does not show dependence on the cluster. Also, definition of “cluster” is missing.
2. Since sec 3.2 is not quite clearly written to me, I guess that you assume the adversarial perturbations for a given sample $x$ for a given defense $\phi_i$ follows a Gaussian with mean $r$ and covariance $\Sigma$. Is that correct? If so, the Gaussian assumption should be clearly emphasized with motivations of why you use Gaussian. Will assumptions of other distribution work? Is Gaussian important here? Empirical evidence would be beneficial.
3. Here’s another question: if you assume $g(\phi_i)$ denotes Gaussian perturbation regarding $\phi_i$ (according to equation 6, which include multiple $\phi_i$), then there should be $N$ Gaussians for $N$ defenses $\phi_1,..,\phi_N$? But there is just one $r$ and $\Sigma$ here.   
4. Also, line 156 indicates that $q_i$ denote $N$ independent distribution on $z$, then should there be $N$ Bernoulli distribution param $\alpha$ here?
5. $q^{(t)}$ seems not defined in line 159.
6. Line 160: what does optimizing $r$ with higher $z$ mean? Should it be optimizing $r$ towards a high objective in eq. (6)?
7. In line 143, the paper indicates that $g(\phi)$ follows a normal distribution, which means that $g(\phi)$ is a random vector. But in equation (8), $g(\phi_i)$ is treated like a deterministic vector. Should $g()$ be inside the expectation?
8. In E-step, the constant $C$ looks quite important in Equation (9), how to select it? What is the rationale for approximation with a constant?
9. Logging in problem is one practical case for N-times evaluation, but is there any practical scenario for image misclassification?
questions: questions: Please refer to the weakness part for concrete questions.
limitations: limitations: Discussed in Appendix A.
paper_summary: paper_summary: This work proposes a new attack evaluation for diffusion-based purification methods: the 1 + N evaluation, which incorporates expectation maximization-based attacks and N-time evaluation. This method is helpful to evaluate the worst-case robustness of stochasticity-based defense methods.
strength_weakness: strength_weakness: 1.	One of the main conclusions on the advantages of N-evaluation is not that impressive. In fact, I think it is quite intuitive and the innovation is questionable.  
2.	This setting will significantly increase the attack cost. If N is large enough, nearly all stochasticity-based defense methods have the risk of being broken. However, I do not think this is a practical and meaningful setting.
3.	The results cannot be compared with other methods directly. In addition to ASR, the absolute robustness should be reported in Table 1 so that these numbers can be directly compared with the results in other related literature [1, 2].
4.	Insufficient experimental results. Only the simple dataset, CIFAR-10, is considered. More datasets, especially ImageNet, should be incorporated.

[1] Robust Evaluation of Diffusion-Based Adversarial Purification, ICCV, 2023.

[2] Robust Classification via a Single Diffusion Model, ICML, 2024
questions: questions: Please see comments above.
limitations: limitations: N/A

Comments: title: Response for W2 and W4
Thanks for response
Response to the single Gaussian assumption
Thanks for rebuttal
comment: Hi, We kindly look forward to receiving your feedback to give us the opportunity to further improve our paper and address your questions. Thank you!
Thank you for your question and feedback, and our response to your concerns is as follows:

**W2:** In scenarios such as login, authentication, etc., the reward for an attacker to obtain at least one attack is high, while attempts are costly or limited. In such scenarios, the defender needs to consider worst robustness under a practical  $N$-evaluation setting. In such a scenario-dependent setting with practical $N$, stochastic defenses are not so inevitably attacked and should maintain a certain level of robustness. For example, the attacker needs to pay for API calls to attack, or may be blocked for a while when 5 incorrect authentications are submitted. In these cases, threats with $N=5$ need to be considered, while threats with $N=100$ are extreme/impossible. In short, the choice of $N$ is attack scenario-dependent, and $N$-evaluation can be adapted to these scenarios by setting a proper $N$.

When considering robustness in the average sense, the stochastic defense yields different outputs for each submit, so its accuracy should be expected not only for the data, but also for the submit. Thus robustness corresponds to our Avg. Rob metric, which also requires $N$-evaluation to estimate.

The importance of $N$-evaluation in evaluating stochastic defenses is also discussed in \[1\]. Our contributions on $N$-evaluation are (1) we statistically prove that 1-evaluation underestimates the risk of resubmits, and (2) attacks based on $N$-evaluation are more effective and efficient compared to those based on EOT. Overall, $N$-evaluation helps to assess the both average or worst case robustness of stochastic defenses in a wider range of scenarios, and can be used as an aid for better attacks even in strict scenarios where defenders only consider 1-evaluation threats. 

**W4:** We tested the effectiveness of our algorithm on the high resolution dataset restricted-ImageNet, which is ImageNet reorganized into 9 classes. Images are still 256*256 and from ImageNet1k, with the only difference being that fewer classes make the attack more challenging, which is widely used for evaluating model robustness \[2-4\]. Otherwise the 1000 classes in ImageNet1k make the attack easy and the robustness of the model is almost 0. As the results are shown in Table 2, our method DH performs better in different defenses. Our improvement is even more significant in denoising-based DiffPure and GDMP, which suggests that diffusion models on large-scale data are more likely to introduce gradient dilemmas to the defense. Thus, DiffHammer can be used as a better evaluation tool in different datasets.

We appreciate your feedback and will clarify our contribution and add experimental results from other datasets in the revision. We sincerely appreciate it if you could let us know any additional questions or concerns. We are keen to provide a satisfactory response.

\[1\] Lucas K, Jagielski M, Tramèr F, et al. Randomness in ml defenses helps persistent attackers and hinders evaluators[J]. arXiv preprint arXiv:2302.13464, 2023.

\[2\] Chen H, Dong Y, Wang Z, et al. Robust classification via a single diffusion model[J]. arXiv preprint arXiv:2305.15241, 2023.

\[3\] Tsipras D, Santurkar S, Engstrom L, et al. Robustness may be at odds with accuracy[J]. arXiv preprint arXiv:1805.12152, 2018.

\[4\] Yang Y Y, Rashtchian C, Zhang H, et al. A closer look at accuracy vs. robustness[J]. Advances in neural information processing systems, 2020, 33: 8588-8601.
Thanks for the rebuttal. Part of my concerns have been addressed, except for W2 and W4. 

W2: I still do not think that N-evaluation is a practical and meaningful setting considering that nearly all stochasticity-based defense methods have the risk of being broken if N is large enough. In addition, if N=5 or N=10 would be recommended, why don't we use N=100 or even larger to evaluate? Conversely, when we discuss robustness, we often report the accuracy under attacks, which is the expectation of accuracy over the dataset. 1-evaluations naturally meet this perspective. 

W4: High-resolution and large-scale datasets, in particular, the ImageNet-1K, should be evaluated to enhance the results.

Based on these concerns, I think this work to be a borderline case and will keep my rating.
We sincerely appreciate your acknowledgment and your positive feedback on our work! It is extremely valuable for helping us strengthen the paper.  We will make sure to clarify and enhance the rigor of the methods section in our revision.
Thanks for the authors' efforts for further clarification! My concerns are now addressed and I raise the score to 6. I hope the authors  can make the method part more clear and rigorous in revision.
We sincerely appreciate your acknowledgment and your positive feedback on our work!
Thank you for your question. To define the cluster, we follow the definition in \[1\], where clusters are defined by "determine $m$ clusters (subsets) of individuals in $I$, and those individuals assigned to the same cluster are similar yet individuals from different clusters are different (not similar)". 

Our assumption is that, for a class of purifications that can be attacked by the same adversarial noise, the gradients of purifications in this class follow a Gaussian distribution with mean $r$. We estimate $r$ as an approximation to the adversarial noise. We set a null-informative prior on other purifications that are attacked (but not by $r$) or that are not attacked. Thus, we have only a shared set of Gaussian distribution parameters.

In fact, it is not necessary to assume a Gaussian mixture model for all attacked  purifications and assign $m$ sets of parameters ($r_1,.... .r_m;\Sigma_1,... ,\Sigma_m;\alpha_1,... ,\alpha_m$) where $\alpha_1>\alpha_2... ,>\alpha_m$. The reason for this is that the EM algorithm is locally convergent and we are only concerned with the cluster mean $r_1$ that has the largest proportion ($\alpha_1$). Purification from $\mathcal{N}(r_1,\Sigma_1)$ is more likely to occur in the first few batches of observations due to the larger probability $\alpha_1$, which allows the estimated $r$ in EM algorithm to approach $r_1$ initially and optimize towards $r_1$ thereafter. Although the EM algorithm also has a probability of converging to a suboptimal, this also has a sufficient attack effect and avoids the additional overhead of maintaining $m$ sets of parameters. Therefore, we only assume a set of Gaussian parameters $r,\Sigma,\alpha$ to make our algorithm efficient and concise.


\[1\] Duran B S, Odell P L. Cluster analysis: a survey[M]. Springer Science & Business Media, 2013.
Thank the efforts of the authors in the rebuttal!

I still want to make sure I correctly understand the assumptions here.

"call a set with high intraclass similarity and low interclass similarity a cluster" is still not a rigorous and clear definition. Do you assume the gradients for successful attack follow a Guassian? And you just assume a single Gaussian since "$r,\Sigma$" are shared?
Thank you for your responses.  I think it solves most of my concerns, and I will keep my score.