Sample-specific Noise Injection for Diffusion-based Adversarial Purification

Authors: Yuhao Sun, Jiacheng Zhang, Zesheng Ye, Chaowei Xiao, Feng Liu
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: diffusion-based adversarial purification, adversarial purification, adversarial robustness, accuracy-robustness trade-off

Abstract:
Diffusion-based purification (DBP) methods aim to remove adversarial noise from the input sample by first injecting Gaussian noise through a forward diffusion process, and then recovering the clean example through a reverse generative process. In the above process, how much Gaussian noise is injected to the input sample is key to the success of DBP methods, which is controlled by a constant noise level $t^*$ for all samples in existing methods. In this paper, we discover that an optimal $t^*$ for each sample indeed could be different. Intuitively, the cleaner a sample is, the less the noise it should be injected, and vice versa. Motivated by this finding, we propose a new framework, called Sample-specific Score-aware Noise Injection (SSNI). Specifically, SSNI uses a pre-trained score network to estimate how much a data point deviates from the clean data distribution (i.e., score norms). Then, based on the magnitude of score norms, SSNI applies a reweighting function to adaptively adjust $t^*$ for each sample, achieving sample-specific noise injections. Empirically, incorporating our framework with existing DBP methods results in a notable improvement in both accuracy and robustness on CIFAR-10 and ImageNet-1K, highlighting the necessity to allocate distinct noise levels to different samples in DBP methods. Our code is available at: https://anonymous.4open.science/r/SSNI-F746.

Decision: reject
Reviewer scores: 6, 5, 5, 6

Reviews and discussion:
<thread>
creation date:
1729750033853

thread_type:
official_review

<thread_object>
creator: reviewer_ijtf

<content>
summary:
In this paper, the authors proposed to purify the adversarial examples via diffusion models with sample specific noise levels. Hence the adversarial purification can both sufficiently remove the adversarial attacks and preserve the semantic information. In order to obtain the sample-specific noise levels, the authors started from their other key finding that the difference between the clean image and the data point in the purification process can be measured using the score norm. In their experiments, the proposed method SSNI demonstrated both higher standard accuracy and robust accuracy than the baseline diffusion-based purification methods

strengths:
1. *Originality*: It is an original approach to measure the difference between the clean data and the data point sampled from the reverse process in the diffusion model, using the score norm. Given this score norm, it is also novel to compute sample specific noise levels for purification on the adversarial examples.

2. *Clarity*: The core method sample-specific score-aware noise injection is clearly articulated. It is easy to follow from the framework of the method to the design of experiments. The key contributions are also explicitly presented with high clarity.

weaknesses:
1. *Presentation*: The presentation of this work should be improved, especially for the visual presentation. For example, Figure 1 would be clearer to deliver the observations using the ImageNet dataset, because it has higher-resolution images which allow the audience to distinguish the objects.


2. *Quality*: I would highly recommend the authors to have a sufficient check on their lemmas and propositions, especially the proofs. Based on the coherent and flawless proofs, the lemmas and the propositions would be more convincing and solid. For example, the plus or minus sign in the proof of Proposition 1 when they apply the Cauchy-Schwarz inequality. More examples can be seen in **Questions**.

questions:
1. In terms of the significance and the novelty of this paper, could the authors compare the novelty of this work with Diffusion Models Demand Contrastive Guidance for Adversarial Purification to Advance [1] in ICML 2024 in terms of the $t^*$ selection? It would be better to present the difference to highlight the contributions in this submitted paper.

2. For Lemma 3, there is a minor question on in the proof: it seems that given $K>0$, $(1 - \epsilon - K\epsilon) || \mathbf{x} || \geq (1-2\epsilon) ||\mathbf{x}||$ may not hold. Would the authors please elaborate it?

3. For Proposition 1, could the authors explain why the derivative of $\bar{\alpha}_t$ is $\frac{\partial \bar{\alpha}_t}{\partial t} = -\beta_t \bar{\alpha}_t$, given $\bar{\alpha}_t= \prod^t (1-\beta_t)$? 

4. BPDA+EOT as a white-box attack has been shown to achieve lower attack success rate than those utilizing direct gradients of the defense process when evaluating diffusion-based purification methods [2]. It would be better to elaborate the reason why BPDA+EOT is chosen to evaluate the proposed diffusion-based purification method.
    

[1] Bai, M., Huang, W., Li, T., Wang, A., Gao, J., Caiafa, C.F. & Zhao, Q.. (2024). Diffusion Models Demand Contrastive Guidance for Adversarial Purification to Advance, Proceedings of the 41st International Conference on Machine Learning.

[2] Minjong Lee and Dongwoo Kim. Robust evaluation of diffusion-based adversarial purification. In ICCV, 2023.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730635295198

thread_type:
official_review

<thread_object>
creator: reviewer_mled

<content>
summary:
The work provide a method to adapt the diffusion step for each sample in adversarial purification based on diffusion model in order to enhance the trade-off between robust acc and test acc. The paper use the norm of score to reflect how much a data point deviates from the clean data distribution and utilize the metric to design the diffusion step function. The work show a theoretical explanation that the score norm can act as a proxy for distinguishing between noise levels.

strengths:
Good theory and experimental results

weaknesses:
None

questions:
How to adapt bias in the reweighting function in different datasets and network ?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730701041099

thread_type:
official_review

<thread_object>
creator: reviewer_lrdc

<content>
summary:
This paper proposes a new framework, called Sample-specific Score-aware Noise Injection (SSNI). SSNI uses a pre-trained score network to estimate how much a data point deviates from the clean data distribution, then adaptively adjusts the injected noise level for each sample, achieving sample-specific noise injections for diffusion-based purification.

strengths:
A new framework that adaptively adjusts the injected noise level for each sample, thus achieving sample-specific noise injections for diffusion-based purification.

weaknesses:
There are some unclear points that need more clarification; and insufficient theoretical justifications; see below for detailed questions.

questions:
How is the standard accuracy calculated? Are the clean samples also purified before applying the pre-trained classifier network?

What is the value of t within s_\theta in eq (5)?

It is a bit confusing to me how the theoretical results in sec 3 support the claimed relationship between score norm and the optimal noise level. It seems like that Lemma 3 and Prop 1 are about the score norm of $p_t(x)$ where $p_t$ is the marginal distribution of $x_t$, generated by injecting sequential Gaussian noise to clean data $x_0$. In practice, the data may be attacked in different ways and contained by non-Gaussian noise. I think it would be more beneficial to present the theoretical insights for the score norm evaluated on contaminated data under certain attacks (maybe measured by the contamination level $\epsilon$), and how this norm depends on the contamination level $\epsilon$, thus showing that it will be significantly larger than the score norm of clean data.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730715983264

thread_type:
official_review

<thread_object>
creator: reviewer_2mwz

<content>
summary:
Neural networks are highly susceptible to adversarial noise, which compromises their robustness in critical applications. Diffusion-based purification (DBP) methods address this by introducing Gaussian noise in a controlled process to counteract adversarial noise, typically using a fixed noise level $t*$ across all samples. However, this uniform approach can lead to suboptimal results, as the ideal noise level varies by sample. In response, the authors propose Sample-specific Score-aware Noise Injection (SSNI), a framework that adaptively adjusts $t*$ based on a sample’s deviation from the clean data distribution, estimated through score norms. By tailoring noise levels—lower for cleaner samples and higher for noisier ones—SSNI enhances both accuracy and robustness across benchmarks like CIFAR-10 and ImageNet-1K, outperforming traditional DBP methods.

strengths:
1. The need for sample-based noise injection is intuitive and well-motivated. I really like Figure 1.
2. The paper is well-written and easy to follow.

weaknesses:
First, I would like to make a note that my background is more on diffusion rather than adversarial perturbation. Thus, I am not sure about the significance of the empirical performance of the proposed method, compared to the baselines. I will leave the assessment of this part to the other reviewers more qualified than me.

My biggest concern with the paper is that a lot of the arguments and designs feel hand-wavy without proper principles or justification. I will write the detailed point below.

1. I would like to point out that diffusion models are never trained to model the score directly, but scaled versions of its variants, and only represent scores through reparametrizations. Typically, the training objective of diffusion models is parameterized so that the model's prediction has a **constant** norm across noise levels (for the ease of training), for example, the $\epsilon$-prediction in DDPM and the more modern EDM [1,2]. The difference in terms of the score norm comes only from $t$ through reparametrization. Essentially, in terms of diffusion model training, they are not trained to differentiate the norm of the scores across noise levels. This property holds for the two models the authors used in experiments. Thus, I am wondering what exactly the authors observe here in Figure 4 in Appendix B. How is this score norm calculated? Is this calculated through a pre-trained model or through some analytic solution? If the pre-trained models' output norm really correlates with the sample $x$ (and we should keep the input of time/noise level constant), then it is really an undesired outcome of those models. Then what about the more modern, better-trained models, like EDM?
2. I am also confused by the logic that the further "away" the sample is from the "clean" distribution, the more noise we need to inject for purification. In particular, why are we adding a proportional amount of noise estimated in the sample again? For example, if the procedure estimates the model is at time $t$, Algorithm 1 essentially forwards the sample to $t+t'$ time right? I don't understand the logic behind this choice. Also, now when you run the backward sampling process, do you start at $t+t'$ or $t'$? Essentially, when you do the forward and backward, are you treating the test (start) sample $x$ as a clean or a noisy one in terms of diffusion?
3. The two realizations of the reweighting function have no justification at all. Also, how is the bias term $b$ selected?

In all, I feel the main insights of the paper are based on some misunderstanding of the current diffusion models. Also, a lot of the design choices feel questionable, and thus heuristic to me. In the end, I feel unconvinced by the paper's argument and proposal. However, I am happy to be corrected.

[1] Karras et al. Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS 2022.

[2] Karras et al. Analyzing and Improving the Training Dynamics of Diffusion Models. CVPR 2024.

questions:
1. The sub-caption of Figure 1 can be simplified (shared).
2. Judging from Equation 7, are you suggesting to evaluate the pre-trained diffusion model on many $t$s? If so, this requires many network evaluations, right?
3. On Line 344, I think you mean "the key idea is to scale the coefficient" right? Instead of scaling $\|EPS(x)\|$.
4. The time overhead reported in Section 5.5 is not very meaningful. Some relative measurements are more desired.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733305294379

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Revision Update and Summary of Changes (part 1/2)

comment:
We appreciate the thorough feedback from all reviewers and have addressed each concern through additional experiments and clarifications.

We outline our key responses and improvements to the manuscript in this summary.

**Responses to Reviewer 2MWZ**
1. **Comprehensive background introduction**
- Concerns: The reviewer stated that **"I would like to make a note that my background is more on diffusion rather than adversarial perturbation."**
- Response: We **identified potential misunderstandings** due to insufficient background information on Diffusion-Based Purification (DBP) and adversarial perturbations provided in the **Preliminary (Section 2)**. We **clarify the problem settings** in diffusion-based generative tasks versus adversarial classification and **outline the high-level objective, motivation, research gap and contributions** of this study, ensuring that readers from diverse backgrounds can fully understand the context.

2. **Observation in Figure 4**
- Concerns: What exactly the authors observe here in Figure 4 in Appendix B?
- Responses: We explained that the y-axis represents the EPS score norm and clarified that **"with increased attack strength, the score norms also increase."**. We clarify that our study **leverages the observation** that data farther from the original distribution tends to exhibit greater momentum (score norm) towards high-density areas in the vector field, as supported by [1].

[1] Generative Modeling by Estimating Gradients of The Data Distribution, NeurIPS 2019.

3. **Score Norm Computation**
- Concerns: How is this score norm calculated?
- Responses: We **clarified** that the score is estimated using the pre-trained score network based on **score matching principles**, not directly computed from the diffusion model. The norm is then derived mathematically from these estimated scores.

4. **Noise Injection**
- Concerns: Why is a proportional amount of noise added again to the sample?
- Response: We **clarified** by distinguishing adversarial perturbations from the Gaussian noise injected during diffusion. We emphasized that adding Gaussian noise to adversarial examples is **a well-established defense mechanism** in DBP.

5. **Timestep Setting**
- Concerns: Algorithm 1 essentially forwards the sample to $t+t’$ time, right? Also, now when you run the backward sampling process, do you start at $t+t’$ or $t’$?
- Responses: We clarified the **potential misunderstanding** that no additional $t^{\prime}$ is introduced. Instead, we adapt the optimal timestep $t^*$ based on the sample’s score norm. We have **revised** Algorithm 1’s notation for better readability.

6. **Test Sample**
- Concerns: Are you treating the test (start) sample x as a clean or a noisy one in terms of diffusion?
- Responses: We **clarified** and **explained** that test samples are treated uniformly $(t=0)$ regardless of whether they are adversarial or natural.

7. **Bias Term**
- Concerns: How is the bias term b selected?
- Responses: We justified the reweighting functions between **lines 336–363** of the revision and conducted an ablation study on the bias term $b$. We appended these results to **Appendix J**.

8. **Function Evaluation**
- Concerns: Are you suggesting to evaluate the pre-trained diffusion model on many ts? If so, this requires many network evaluations, right?
- Responses: We have **corrected** the unclear definition and clarified that EPS computation **does not involve repeated evaluations** of the diffusion model.

10. **Other Minor Updates**
- We **simplified** the caption of Figure 1 as suggested.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a refinement of the DBP framework by tailoring noise levels based on the samples. The work is well motivated as the original DBP had kept the question of suitable noise level open. The reviewers mostly agree that the work makes a reasonable advance, however there are quite a few concerns regarding the work including details regarding score norm scale, limited clarity in some aspects of the technical exposition, and aspects of the empirical evaluation. There is also literature on attacking DBP, which is not considered/cited in the work and is a major concern, e.g., see King et al., DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification, NeurIPS'23. To their credit, the authors have addressed some of the concerns from the reviewers, e.g., norm scaling, etc., but other concerns persist.