Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness
Authors: Yiquan Li
Zhongzhu Chen
Kun Jin
Jiongxiao Wang
Jiachen Lei
Bo Li
Chaowei Xiao
Venue: NeurIPS 2024 poster (2024)
Keywords: Consistency Model
Diffusion Purification
Certified Robustness
Randomized Smoothing
Primary area: safety_in_machine_learning

Abstract: Diffusion Purification, purifying noised images with diffusion models, has been widely used for enhancing certified robustness via randomized smoothing. However, existing frameworks often grapple with the balance between efficiency and effectiveness. While the Denoising Diffusion Probabilistic Model (DDPM) offers an efficient single-step purification, it falls short in ensuring purified images reside on the data manifold. Conversely, the Stochastic Diffusion Model effectively places purified images on the data manifold but demands solving cumbersome stochastic differential equations, while its derivative, the Probability Flow Ordinary Differential Equation (PF-ODE), though solving simpler ordinary differential equations, still requires multiple computational steps. In this work, we demonstrated that an ideal purification pipeline should generate the purified images on the data manifold that are as much semantically aligned to the original images for effectiveness in one step for efficiency. Therefore, we introduced Consistency Purification, an efficiency-effectiveness Pareto superior purifier compared to the previous work. Consistency Purification employs the consistency model, a one-step generative model distilled from PF-ODE, thus can generate on-manifold purified images with a single network evaluation. However, the consistency model is designed not for purification thus it does not inherently ensure semantic alignment between purified and original images. To resolve this issue, we further refine it through Consistency Fine-tuning with LPIPS loss, which enables more aligned semantic meaning while keeping the purified images on data manifold. Our comprehensive experiments demonstrate that our Consistency Purification framework achieves state-of-the-art certified robustness and efficiency compared to baseline methods.

Consistency Purification: Effective and Efficient
Diffusion Purification towards Certified Robustness
Yiquan Li
Zhongzhu Chen
Kun Jin 2
Jiongxiao Wang
Jiachen Lei²
Bo Li
Chaowei Xiao
'University of Wisconsin-Madison: University of Michigan-Amn Arbor;
CCalifornia Institute of Technology UUniversity Illinois Urbana-Champaign
Abstract
Diffusion Purification, purifying noised images with diffusion models, has been widely used for enhancing certified robustness via randomized smoothing. How- ever, existing frameworks often grapple with the balance between efficiency and effectiveness. While the Denoising Diffusion Probabilistic Model (DDPM) offers efficient single-step purification, falls short in ensuring purified images reside on the data manifold. Conversely, the Stochastic Diffusion Model effectively places purified images on the data manifold but demands solving cumbersome stochastic differential equations, while its derivative, the Probability Flow Ordinary Differen- tial Equation (PF-ODE), though solving simpler ordinary differential equations, still requires multiple computational steps. In this work, we demonstrated that an ideal purification pipeline should generate the purified images on the data manifold that as much semantically aligned to the original images for effectiveness in one step for efficiency. Therefore, we introduced Consistency Purification, an dfficiency-effeettvvene Pareto superior purifier compared the previous work. Consistency Purification employs the consistency model, one-step generative model distilled from PF-ODE thus generate on-manifold purified images with a single network evaluation. However, the consistency model is designed not for purification thus it does not inherently ensure semantic alignment between purified and original images. To resolve this issue, we further refine it through Consistency Fine-tuning with LPIPS loss, which enables more aligned semantic meaning while keeping the purified images on data manifold. Our comprehensive experiments demonstrate that our Consistency Purification framework achieves state-of-the-art certified robustness and efficiency compared to baseline methods.
Introduction
Diffusion models were first proposed for high-quality image generation 2 3
and have been
extended generative tasks across various modalities, including audio 7 8. video 9 [0], and 3D object [2] [3). A diffusion model for image generation typically involves two key processes: (I) forward diffusion process, which transforms the source image into isotropic Gaussian by gradually adding Gaussian noise, and (2) the reverse diffusion process, which uses a Deep Neural Network (DNN) perform iterative denoising starting from random Gaussian noise.
Due to the inherent denoising capability of diffusion models, there have been widely applied to improve the robustness of DNNs. This enhancement is achieved by Diffusion Purification [14] 15 which purifies the network inputs to reduce the effects of various types of unforeseen corruptions adversarial attacks. Among these, particularly suitable and effective scenario of purification to improve certified robustness through randomized smoothing 19 for image
16
17 18
"The first four authors contributed equally. Correspondence to: Jiongxiao Wang <jwaag2999@wiss.eeuu>
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

classification tasks. This method guarantees tight robustness in the & norm with smoothed classifier. However, many previous works [99 20 22
23 24 have shown that it still requires
retraining with Gaussian augmented examples for each noise level to optimize the smoothed classifier. Diffusion models, capable of purifying Gaussian perturbed images before classification, can be seamlessly integrated with any base classifier to produce smoothed classifier for arbitrary noise levels. This integration has been demonstrated to effectively enhance certified robustness, as supported
by numerous studies [8 25 26 27
Gaussian Noise
Consistency Purification
Augmented Imago
Input
Fine-tuned Consistency Model
One-step
Off-the-shelf Classifier
Robust Prediction
Purification
Consistency Model
Additive Gaussian
Noise Sampting
Clean Images
Optimize Parameters
Purified Images
LPIPS
Consistency Fine-tuning
Figure An illustration of Consistency Purification framework.
However, current diffusion purification for certified robustness via randomized smoothing still faces significant trade-offs between efficiency and effectiveness. Although Denoising Diffusion Probabilistic Model (DDPM) [28] only requires one single network evaluation in the purification process [25]. it generates the mean of the posterior data distribution conditioned the noisy sample, which does not necessarily locate on the data manifold and may exhibit ambiguity during classification. To further improve diffusion purification, various methods such DenscPure (26). Local Smoothing 27 and Noised Diffusion Classifiers (29] are applied. However, these methods are considerably less efficient as they require multiple times of the computational costs compared to one-step DDPM. Another promising approach involves using the Probability Flow Ordinary Differential Equation (PF-ODE) 3. It has offered method to accelerate the sampling process 4 and achieved a closer distribution to the original data, well balancing efficiency and effectiveness. However, several computational steps are still needed to solve the ODE numerically.
To find a Pareto superior solution in terms of efficiency and effectiveness, we introduce a new frame- work, Consistency Purification, which integrating consistency models into diffusion purification with Consistency Fine-ture -unint. The consistency model is novel category of diffusion models that learns the trajectory of the PF-ODE that transits the data distribution to the noisy distribution. is trained to map any point along this trajectory back to its starting point. This property is desirable for diffusion purification,
allows images with any scale of Gaussian noise be directly purified to the clean images. Distilled from pre-trained diffusion model by simulating the PF-ODE trajectory, consistency model can generate high-quality in-distribution images single step, thereby ensuring both efficiency and effectiveness. However, since consistency models are primarily trained for image generation, may not suffice to guarantee that the purified image that maintains the same semantic meaning the original image. To address this issue. propose adding Consistency Fine-tuning step into the purification framework, which further fine-tunes the consistency model using Leamned Perceptual Image Patch Similarity (LPIPS) (30 loss, aiming minimize the perceptual differences between the purified and original images, thereby ensuring better semantic alignment, while the same time, ensuring the purified images still lie the data manifold.
We show that Consistency Purification is Pareto superior compared to baselines from two aspects. First of all, compared with effective methods like DensePure (26). Local Smoothing 27 and Noised Diffusion Classifiers [29], Consistency Purification much more efficient since it enables single-step purification. Secondly, compared with efficient method like (onestep-DDPM (25), we provide both

---

theoretical analysis and experiment results to support the effectiveness improvement of Consisteney Purification. In Example [.T]] we show an one-dimensional example demonstrating that consistency model can generate on-manifold purified samples while(onestep-DDDPN does not have this property.
In Theorem 3.3 we show important theoretical result that given purifier, the lower the transport from the original distribution to the purified distribution (a measure of distance between probability distributions, see [31)), the higher the probability that the purified sample is sufficiently close to the original sample, and thus the better purification outcomes. Our experiment results verify that both the integration of consistency model in Consistency Purification and the further Consistency Fine-tuning decreases such transport and achieves better semantic alignment between purified samples and original samples.
Beyond the validation of our theory, we conduct comprehensive experiments to demonstrate the empirical improvements of Consistency Purification. Compared to various baseline settings, our approach has shown significant improvements, achieving an average 5% gain in performance over the previous Dieestep-DDPM under the same cost with single-step purification. These observations underscore our success in finding a Pareto superior diffusion purification framework in both efficiency and effectiveness for certified robustness.
2 Backgrounds
Randomized Smoothing [9). Randomized smoothing is designed to certify the robustness of a
given classifier under &2 norm perturbations. Given a base classifier and an input æ, randomized smoothing first defines the smoothed classifier by g(z) = arg max, PP (e0, (f + E) c), where is the noise level, which controls the trade-off between robustness and accuracy. [19] shows that g(x) induces the certifiable robustness for æ under the l norm with radius R, where
(6-61(0 (pa) t- (p'())) where PA and pB are the probability of the most probable class and "runner-up" class respectively; + is the inverse of the standard Gaussian CDF. The PA and Pa can be estimated with arbitrarily high confidence via the Monte Carlo method.
R= (
Continuous- Time Diffusion Model 3.. The diffusion model has two components: the diffusion process followed by the reverse process. Given an input random variable 2o p, the diffusion process adds isotropic Gaussian noises to the data so that the diffused random variable at time t is (zop 2, / + e,,, s.t., e, ~ (O I), and of (1 ar a and we denote I, ~ Pt. The forward diffusion process can also be defined by the stochastic differential equation
dæ ((x, t) + G(t)dw.
(SDE)
where Tu ~p, D: Rd x + R is the drift coefficient and typically has the form D(x t) D(t)., G: : R + I is the diffusion coefficient, dt is an infinitesimal time step, and w(t) IR" is the standard Wiener process.
The reverse process exists and removes the added noise by solving the reverse-time SDE 32
dæ [tt)== G(I) log (==)ddt + G(I)dw,
(reverse-SDE)
where (r( denotes the marginal distribution at time t, and (t)) is a reverse-time standard Wiener process. defined the probability flow ODE (PF ODE) which has the same marginal distribution as reverse-SDE but can be solved much faster
da [D()== - - V log px)) dt.
(PF-ODE)
As shown in 4, the perturbation kernel of SDE has the general form (oo( (=)))= = N (æ=();;
(perturbation-Kernel)
where s() = exp (1o (())E) and o(t) = v 1o
o())² [E)
dg. Under this formulation, PF-ODE can
written
a(t) s(t) (t)) logn (- (-8): dt a())
dx
where denotes the time derivative and p . (1() denotes the marginal distribution at time (. In our context, we use the EDM parameter 4 where s(t) and (t) = t which gives us probability flow ODE
dx = -V log Pt (x)dt.
(EDM-ODE)
3

---

We use { [eeo.. and { he to denote the diffusion process and the reverse process generated by SDE and reverse-SDEr respectively, which follow the same distribution. We also use (E denote the reverse process generated by PP--DD which has the same marginal distribution as {ot he([.,1) and {&e bejo.11 given t.
DDPM constructs discrete Markov chain (ao, L1. :.. Zi ... mN as the forward process for the training data Tg ~ p. such that P(z, Max vI B, B, where 0 < f1 $2
Discrete-Time Diffusion Model (DDPM 28
.xx 1 are prede-
fined noise scales such that v approximates the Gaussian white noise. Denote 0; I1 N (1 8.). we have P(x,) Nax; vājaz. (1 a,) i... 2z(20. c) = Vā + (1 -,) No
The reverse process of DDPM learns reverse direction variational Markov chain pa (z:-1 (e;)) (( i); (a i)). 28 defines ED as function approximator predict from æ, such that ( i) = - 43 VI-a; (2(())))) Then the reverse time samples are generated by
&
- -ー c; T-a.
8. ca- ()) + VBie,e~ NO0, I), and the optimal parameters O" are
obtained by solving 0 arg ming Ezo.s |||e e + (1 i)) also provided one-step approximate reconstruction of zy from any 2t,
2o R t
(r. v1 (0)6((1)) /Vat-
(onestep-DDPMM)
Consistency Model 33 Given a solution trajectory of PF- the consistency model is defined as D: (z), t) H T.- The model exhibits the property of self conissennss,,n,y ensuring that its outputs are consistent for arbitrary pairs of (Ix, t) from the same PF-ODE trajectory; specifically, D( t) D(zv,t') t') for all t, E [e, T As shown by the definition, consistency models are suitable for one- shot denoising, allowing for the recovery of Φ; from any noisy input æ, in one network evaluation. Two distinct training strategies can be employed for training the consistency models: distillation mode and isolation mode. The primary distinction lies in whether the models distill the knowledge from pre-trained diffusion models or train from initial parameters. According to the experiments reported in (33), consistency models trained in the distillation mode have been shown to outperform those trained in isolation mode for generating high-quality images. Consequently, our paper only considers consistency models trained in the distillation mode.
3 Theoretical Analysis
In this section, we provide theoretical explanations on the advantages of Consistency Purification, with focus on its purification performance improvement terms of certified robustness over (25).
As demonstrated in BL. PF-ODE PF-OODEmmiinainn maintains the marginal distribution of Peverse-SDEr thereby estab- lishing deterministic mapping between the noisy distribution æ; and the data distribution 2%g. In other words, PF-ODE guarantees that the purified sample lies on the data manifold, unlike onestep-DDDMM which lacks this assurance. We present here simple one dimensional example for Illustration.
Example 3.1. Consider one-dimensional space with data set consisting of two samples {y),y2), where y1 and 122
11 The distribution can be represented as a mixture of Dirac delta distri- butions: Pdau( 1 (ō(x y1) + 6(x y2))). By setting s(t) = 1 and a (t) = in perturbation
{1=-))² + -e- -(()))) Then
kernel the distribution at time t becomes: Pr (x)
2t
( }
d log (w) dz
21 (α)
#e
From the derivative formula dlogpe(m) it's evident that x 0 is an equilibrium point, and the dæ right-hand side expression is Lipschitz continuous around x = 0 by L L'Hôpital's 's rule. Thus, according to the Lindelff theorem, any trajectory starting on either side of x will not cross this point. PF--DD drives px))) closer to the Diruc delta distribution Pant (x) as l approaches zero, any initial point on positive/negative side of x: = will eventually approach 1 or -1, iee. the data manifold. Furthermore, in this example, PF FDE generates not only purified sample the data

---

manifold but also closest to the noisy sample. This property is desirable as it establishes a relatively large "robust" neighborhood around each true data point, which implies high certified robustness and a significant certified radius, which will be further discussed later. With the consistency model, we do not need solve the ODE but rather directly map the noisy sample to either 1/ depending on its location relative to a = 0.
For comparison, given any a and i, the onestep-DDPM will ouiput a posterior mean that is
) - )
2
+1
The posterior mean will be near 1 or -1 -1 only when t is sufficiently small compared to |z||. Otherwise, deviates from the data manifold. In the case when t large, the posterior mean will be close to zero, locating in an ambiguous classification region. In adversarial purification [23 26 [4]. we rypically select t based on the variance of the noise added to the data sample rather than using an very small t. This practice helps avoid significant deviations in the posterior mean estimation due to the imperfect estimation of score/noise. With very small i, even slight bias in score/noise estimation can lead to a substantial deviation, resulting in denoised sample even farther from the data manifold represented by (d)a(())
Additionally, PF-ODE is deterministic, eliminating the overhead of majority voting required when using reeerse- SDE as a purifier [26). The consistency model, which reduces ODE solving to a one-step mapping. further ensures purification has the same efficiency as (onestepDDPMI while keeping the in-distribution property.
Though the consistency model enjoys both in-distribution property and one-step efficiency, it does not guarantee that the purified sample has the same semantic meaning as the original sample. This is because the derivation PF--EE only guarantees a mapping between noisy distribution and data distribution, which is sufficient for generation, but not enough for denoising purposes.
To address this concern, we first delineate the desired characteristics of the purifier. As evidenced ideal purifier should yield a purified output situated within proximate vicinity of the original input. It is generally presumed that such purified outputs retain the semantic meaning of the original inputs with a high probability. The disparity in semantic consistency between the noisy input and the purified output generated by PF-ODE arises due to the proximity of the purified output to other samples. In this regard, we propose quantifying this disparity through the notion of transport between the data distribution and the purified distribution, derived by introducing Gaussian perturbations to the data distribution and subsequently applying denoising via PF-ODE Given an original sample x, Gaussian noise and purifier d, the mapping in the transport process defined as T æ + d(x e), which is probabilistie. We aim to demonstrate that a diminished transport between the data distribution and the purified distribution is conducive to higher likelihood of the purified output being situated in proximity to the original sample, thereby preserving its semantic meaning.
in prior works 14] 25 26 34
We will leverage the following definition.
Definition 3.2. Given the data distribution p. Gaussian noise timestep 6, and purifier d, we define Re x + d(x + tc) and the "transport" under Ir between the data distribution and purified distribution as Tu. (p) = | - (z)] p(x)dx.
Intuitively, transport measures the distance between the original and purified samples, which should be small by an effective purifier. Below, we quantify this intuition and present our main theorem. See the detailed proof in Appendix B
Theorem 3.3. Given the transport Tz, (p) between the data distribution p and the corresponding purified distribution under 9G, then for any > O, the probability that the distance between the original sample a and purified sample â = Tt(Φ) is larger than is upper bounded by T.,(p)
Remark 3.4. By Theorem 3.3 the efficacy of the purifier hinges on two crucial factors: the transport Tz, (p) and the radius A theoretically perfect purifier would yield zero transport; however, this is unattainable due to the inherent randomness of gt. Typically, we can optimize the parameter t to minimize the transport, denoted as T* = min; [p((p] In the context of classification tasks, the selection of r also depends on the robustness of the classifier; a more robust classifier allows larger to be chosen, thereby guarantee better purification efficacy.
5

---

FID at different
4.5
Loss
0.25
0.5
1.0
Trarspore
60.3
155.3
350.3
96.8
205.7
383.6
Lg
102.1
214.8
375.4
3.0
onestep-DDPM
LPIPS
20.5
100.9
338.1
Consistency Purification
Consistency f'me--uning
0.3
0.4
0.5
6-6
0.7
0.8
0.9
1.0
Table 1: FID between purified and clean images on CIFAR-I0 test set using different types of fine-tuning loss functions with noise level a {0.25, 0.5, 1.0).
Sigma
Transport between purified im- ages and clean images with noise level E {0.25, 0.5, 1.0).
Figure 2:
For ensuring consistency in semantic meaning between the original and purified samples, it is insufficient merely to minimize their distance; it is also necessary that the purified sample resides on the data manifold, which is the in-distribution property we previously mentioned. To concurrently achieve both objectives, rather than solely focusing on minimizing the Fuclidean distance between the original and purified samples, we opt minimize the Learned Perceptual Image Patch Similarity (LPIPS) loss between them. This strategy aids in mitigating the risk of the purified sample deviating from the data manifold, thereby preserving semantic meaning. In Table [ we show that using LPIPS is better than & and & loss for Consistency Fine-tuning when we want to guarantee the generated images are n--iitribution, where lower FID scores indicate better in in-distribution properties.
Figure 2 validates the effectiveness of Consistency Purification based on our results Theorem 3.3 shows that both the integration of consistency model in Consistency Purification and the further Consistency Fine-tuning can decrease the transport from the original distribution to the purified distribution. Specifically,
can see that Consistency Purification achieves lower average distance from the purified sample to the original sample compared with Mrrestep-DDMM and Consistency Fine-tuning further decreases this average distance, indicating both components result in lower transport and thus a better semantic alignment between purified samples and original samples.
Method
We propose framework, Consistency Purification, with further improved version using Consis- tency Fine-tuning.
4.1 Consistency Purification
We introduce Consistency Purification, directly applying consistency model as a purifier to integrate with base classifier into smoothed classifier for randomized smoothing.
Following Diffusion Denoised Smoothing outlined in [25] it is necessary to establish mapping between Gaussian noise augmented images required by randomized smoothing and the noised image in the ODE trajectory of consistency model. For given consistency model purifier De. any noisy input x, ~ Axxx t2 I) can be recovered to the trajectory's 's start by directly passing it through the model with time x. D9(2,.
When comparing this to the image augmented with additive Gaussian noise Ers ~ N(x,, o² T2)), which is required by randomized smoothing, observe that ærs and 2; share the same formula when t = a. However, since the variances a E (m may not be used during the training of the consistency model, empirically select the nearest time step from the discrete time steps used in training for each a.
For the entire time horizon |e, T] with N - sub-interval boundaries 4 = 12 <... < <N T, the time steps used in training computed by: t; (c1/p4 i-1 NN1(T1/// - e/P)), where 7.

---

Given the variance a of Gaussian noise used in randomized smoothing, we select the corresponding time step 1 for Consistency Purified Smoothing by 1" = [t=lor (m 4
4.2 Consistency Fine-tuning
To optimize the consistency model for aligning semantic meanings during purification, we fune-tune the purifier Dg by minimizing the following loss function: Lo = [E] Do( (x) (1l IIPP.. where the expectation is taken with ~ Pdate- the distance computed by the Learned Perceptual Image Patch Similarity [30]. Pdato represents the distribution of the training data, from which clean images æ are sampled. { m denotes the uniform distribution over m different noise scales o; used for randomized smoothing. Typically, we select the scale set 0; E {0.25, 0.5, 1.0), which is commonly used to compute the certified radius via randomized smoothing.
U [[, [= 3h N(xr, aa. x² I). Here LPIPS denotes
After obtaining the fine-tuned consistency model purifier Dz- can replace the original model used in Consistency Purified Smoothing to purify any noised image Bns with Gaussian variance o;, resulting in the final purified image Tp by xpp Dy- (Brr::1,))
We present the detailed algorithm of our Consistency Purification in Appendix A
Experiments
In this section, we begin by detailing the experimental settings, followed by our main results. Additionally, we conduct ablation studies to further demonstrate the effectiveness of our framework. All experiments are conducted with 1 XXVIIDIA RTX A5000 24GB GPU.
5.1 Experimental Settings.
Dataset. We evaluate the Consistency Purification framework on both CIFAR-10 [35 and ImageNet- 64 (36). CI contains 32 x 32 pixel images across 10 different categories while ImageNet-64 includes 64 x 64 pixel images across 1000 categories. Due to limited computational resources, we select 500 test images for CIFAR- 10 from the 10,000 CIFAR- 10 test set, choosing every 20th example in sequence (c.g., the 1st, 21st, 41st, etc.). Similarly, for the ImageNet-64 dataset, we sample 500 test examples from its 50,000 test examples using fixed interval of 100.
Consistency Purification. For CIFAR- 10, demonstrate the effectiveness of Consistency Purifi- cation, we first perform purification with a public unconditional consistency model [37]. After that, to further improve the performance, we fine-tune the model with noise levels a sampling from (0.25, 0.5, 1.0]. shown the (+ Consistency Fine-tuning). However, currently there is no publicly available unconditional consistency model checkpoint for the ImageNet dataset that can be used directly for purification purposes. The only available model is the conditional consistency model on ImageNet-64. Thus, here we trained an unconditional consistency model on ImageNet-64, initializing it with the existing conditional consistency model checkpoint. Details of the training process are included in Appendix C Additionally, we also conduct Consistency Fine-tuning on ImageNet-64 model with noise levels {0.05, 0.15, 0.25).
Baselines. For comparative analysis of CIFAR-10, we conduct baseline experiments under various settings. The first baseline involves onestep-DDPM, where we employ the 50-M unconditional improved diffusion models from 2 utilizing the one-shot denoising method (25) for purification. Given that our consistency model is distilled from an EDM model 4.. we include EDM as our baselines, applying both one-shot denoising (onestep-EDM) -DMM and ODE solver (PF-ODE EDM) for purification. Additionally, we include the recent advancement in diffusion purification methods, Diffusion Calibration, as a baseline following [38], which fine-tunes the diffusion model with the guidance of classifier Nede 10 to improve the purification accuracy under the specific classifier. While for ImageNet-64, due to the lack of public unconditional EDM model, we only include the comparison baseline with onestep-DDPM.
Randomized Smoothing Settings. We set N = 10000 for both CIFAR-10 and ImageNet as the number of sampling times used in randomized smoothing. We compute the certified radius for each test example at three different noise levels a E (0.25, 0.5, 1.0) for CIFAR-10 and a e {0.05, 0.15, 0.25} for ImageNet-64. Then calculate the proportion of test examples whose radius
7

---

exceeds specific threshold The highest accuracy among these noise levels is reported as the certified accuracy at c.
Classifiers. For the classifier used after purification for CIFAR-10, we employ ViT-B/16 model [39, which is pretrained ImageNet-21k (36) and finetuned CIFAR-10 dataset. In ablation studies, we also use ResNet [40] and WideResNet [41] trained on CIFAR-I0. For ImageNet-64, we make up-sampling on the x64 images and directly apply ViT-B/16 as the classifier.
5.2 Main Results.
We present the certified accuracy of Consistency Purification for both CIFAR-10 and ImageNet- 64 dataset, with the results presented in Table 2 We also include the purification steps which decide whether the purifier needs multiple evaluation times through the networks (Multi Steps) other than a single network evaluation (One Step). As observed from Table Consistency Purification significantly outperforms DnsS for both CIFAR-10 and ImageNet-64 with even higher certified accuracy with Consistency Fine-tuning. Besides, for CIFAR-IO, I0, the results also suggest effectiveness of Consistency Purification with Consistency Fine-tuning when compared with more baseline methods such as onestep-EDD PF-ODE EDM and Diffusion Calibration. For the detailed certified accuracy evaluation of fine-gra ane-grained at different noise levels σ, present the results in Figure 3 compared with the pnette--DDPM setting. All results have demonstrated that Consistency Purification is able certify robustness with both efficiency and effectiveness.
Table 2: Certified Accuracy of Consistency Purification for CIFAR-10 and ImageNet-64.
CIFAR-10
Certified Accuracy (%)
Method
Purification Steps
0.0
0.25
0.5
0.75
1.0
Mmestpp-DDPMP3
One Step
87.6
73.6
55.6
39.2
29.6
onestep-EDM
One Step
87.4
76.2
58.8
40.8
32.4
PF-ODE EDM
Multi Steps
89.6
77.0
60.4
42.6
34.0
Diffusion Calibration 38
One Step
90.2
76.4
57.2
42.6
32.4
Consistency Purification
One Step
90.4
77.2
59.8
42.8
33.2
+ Consistency Fine-tuning
One Step
90.2
79.4
62.4
43.8
35.4
ImageNet-64
Certified Accuracy at (%)
Method
Purification Steps
0.0
0.05
0.15
0.25
0.35
onestep-DDPM [25
One Step
55.2
44.8
33.4
15.2
8.8
Consistency Purification
One Step
62.4
54.2
35.2
19.8
13.0
Consistency Fine-tuning
One Step
68.6
58.0
37.4
23.4
17.4
0.7
ane-stap DDPM 0.25
one-stop DDPM a 0.05
one step DDPM = 1.00
DDPM
0.6
one step DDPM o 0.25
0.15
Consistency Fine-tuning
Consistency Fine-tuning
Cettifted Accuraty
0.6
Consistency Fine-tuning - 0.50
Certified Nccuraty
Consistency Fine-tuning 0.15
Consistency Fine-tuning
0.4
Consistency Fine-tuning
0.4
0.3
0.2
0.1
Radius
Figure 3: Certified Accuracy of Consistency Purification with fine-grained radius c. The left figure shows results on CIFAR-10, 10, the right figure shows results on ImageNet-64. The lines demonstrate the certified accuracy of various radius under different Gaussian noise levels a.

---

To better illustrate the significant improvement in certified robustness brought by Consistency Purification, we present visualizations of images after diffusion purification in Figure for CIFAR- 10 at a noise level of 0.5, compared with the onestep-DDPM approach. As shown, our method produces significantly higher-quality purified images than onestep-DIPMM, Furthermore, these purified images achieve notably higher classification accuracy when evaluated by the same classifier. Additional visualization examples for ImageNet-64 are included Appendix
(a) Purified images by DeSM
(b) Purified images by Consistency Purification
Figure 4: Visualization of purified images after the diffusion purification by applying onestep- -DMM and Consistency Purification on CIFAR-10 with 0.5 noise level. Identical noise patterns are applied to images at corresponding locations. A green border indicates that the purified image is correctly classified, while a red border denotes misclassification by the classifier.
5.3 Ablation Studies.
We conduct various ablation studies to evaluate the effectiveness of our proposed method.
Comparison with Noa-Diffusion-basee Baselines. To compare Consistency Pu- rification with various non-diffusion-bar approaches, we conducted additional exper- iments compute the certified accuracy under three non-diffusion-baset based methods first proposed training a classifier with noisy images to ensure certi- fied robustness. Subsequent works [22 (42 build on [9'ss 's methodology, attempting to enhance the smoothed classifier by adding prediction consistency regularization, or incorporating per-sample bias. The experimental results presented Tabel 3 show that our method surpasses all previous non-diffusion-batt methods in achieving higher certified accuracy, particularly with significantly high clean performance at e= 0.0. Furthermore, in contrast to non non-diffusion--ert based methods, which incur significant costs by requiring additional fine-tuning of robust classifiers for each specific noise level, our method can be applied directly to any off-the-shelf classifiers, significantly broadening its practical applications.
Certified Accuracy e%
Methods
0.0
0.25
0.5
0.75
1.0
Randomized Smoothing
19
74.8
59.2
42.0
31.8
22.0
Consistency Regularization
22
74.4
66.0
56.2
41.4
32.8
Aces [22
74.6
66.4
57.0
43.6
32.8
19
42.
Consistency Purification
90.4
77.2
59.8
42.8
33.2
Consistency Fine-tuning
90.2
79.4
62.4
43.8
35.4
Table 3: Certified Accuracy of Consistency Purification compared with non-diffusion-basee baseline methods.
Fine-tuning Loss Functions. To further demonstrate that LPIPS loss is the best choice considering both on- on-manifold purification and semantic meaning alignment, we assess the certified accuracy of Consistency Purification using different loss functions during Consistency Fine-tuning. Instead of LPIPS distance between the clean and purified images as the loss function, we experiment with E and f distances. Results in Table 4 indicate that Consistency Purification with I.PIPS loss achieves the highest Certified Accuracy. In contrast, fine-tuning with f and lz distances compromises the purification performance for certification. This demonstrates that fine-tuning with LPIPS loss function effectively aligns semantic meanings, whereas ε or l distances may hurt them.
Noise Levels Sampling Schedules during Consistency Fine-tuning. In our experiments of Consistency Fine-tuning, we simply select the same sampling schedules of noise levels a 2450.25, 0.5. 1.0}, uniformly sampling a used in randomized smoothing. To empirically demonstrate its effectiveness, we compare this approach with continuous sampling schedules where o ~ 24(0, 1). Results presented in Table 5 show that our discrete sampling schedule achieves higher certified accuracy. This indicates that fine-tuning with a discrete scale, aligned with the noise levels used in randomized smoothing, enhances certified robustness.

---

Table 4: Certified Accuracy of Consistency Pu- rification with different loss functions during fine-tuning for CIFAR-10. "--" represents the setting without fine-tuning.
Table 5: Certified Accuracy of Consistency Pu- rification with continuous and discrete sampling schedules during fine-tuning for CIFAR-10. represents the setting without fine-tuning.
Certified Accuracy at
Certified Accuracy at e
Distance
0.0
0.25
0.5
0.75
1.0
Schedules
0.0
0.25
0.5
0.75
1.0
90.4
77.2
59.8
42.8
33.2
90.4
77.2
59.8
42,8
33.2
89.4
76.4
59.6
42.4
31.4
[0.1]
89.0
76.2
59.8
43,2
33.8
90.0
77.0
59.8
42.4
33.4
(0.25, 0.5, 1.0)
90.2
79.4
62.4
43,8
35.4
LPIPS
90.2
79.4
62.4
43.8
35.4
Generalizability with Different Classifiers. We compute certified accuracy with various classifiers to test if our framework maintains its effectiveness with arbitrary classifiers. The results, presented in Table compare Consistency Fine-tuning with Diffusion Calibration, alternative method to fine-tune diffusion models for improving the certified robustness. When evaluated across different method outperforms Diffusion Calibration except certified accuracy at = 0.0 on WRN28-10 model. It is worth noting that the Diffusion Calibration, which requires a specific classifier for guidance during fine-tuning, exhibits limitations, only achieving comparable performance with the guidance classifier WRN28-10. This demonstrates the advantages of Consistency Fine-tuning in generalizing across different classifiers.
classifiers, including ViT-B/16, ResNet56, and WideNer28-10,,
Fine-tuning Classifier vs. Fine-tuning Diffusion Model. A potential concern with Consistency Fine-tuning is the higher certified accuracy and lower training cost associated with Fine-tuning the Classifier (CLS-FT) compared to our approach of Fine-tuning the Diffusion Model (DM-FT). However, experiments, shown in Table indicate that DM-FT does not conflict with CLS-FT, rather, combining these two methods achieves even higher certified accuracy. On another hand, although CLS-FT yield slightly higher certified accuracy than DM-FT, its requirement for fine-tuning a specific classifier compromises the natural property of diffusion purification frameworks with arbitrary off-the-sher classifiers, thus limiting the practical applicability.
Table 6: Certified Accuracy of Consistency Fine- tuning with different classifiers on CIFAR-10. The guidance classifier used in Diffusion Cali- bration is WidcResNNe18-100 10.
Table 7: Certified Accuracy of Fine-tuning the Diffusion Model (DM-FT) compared with Fine- tuning the Classifier (CLS-FT) in diffusion pu- rification frameworks on CIFAR-10.
Accuracy at
Certified Accuracy at c%
Method
Classiffer
DM-FT CLS-FT
0.0
0.25
0.5
0.75
1.0
WTT-B11
90.2
76.4
51.2
42.6
32.4
Diffusion Calihration
WRN28-10
88.2
76.4
59.2
42.0
31.8
90.4
77.2
59.8
42.8
33.2
ResNec56
86.0
72.8
52.6
35.2
25.8
90.2
79.4
62.4
43.8
35.4
VIT-B/16
90.2
79.4
62.4
43,8
35.4
v
90.4
79.8
63.4
44.2
35.2
Condistency min-ttuning
W8N28-10
88.0
42.8
33.0
/
v
90.8
80.0
64.8
44.6
36.8
ResNec56
6 Conclusion
In this paper, we introduced Consistency Purification, a novel framework proposed to enhance certified robustness via randomized smoothing. By incorporating consistency models into diffusion purification approach and further refining them through Consistency Fine-tuning, our empirical experiments have demonstrate the framework's 's ability to achieve high certified robustness efficiently with one single network evaluation for purification.
Limitations. A notable limitation of our study is that our empirical results do not include computing certified robustness of nigh-resolution images such as ImageNet 256x256. This constraint due to the absence of publicly available checkpoints for the consistency model at this resolution. Additionally, training a consistency model for ImageNet x56x256 would require huge computing resources. which are currently beyond our affordability. However, our framework is designed for adaptability and could be easily extended to ImageNet 256 256 once these checkpoints become available. result, empirical evaluations in this paper limited to the CIFAR-10 and ImageNet 64 x 64 datasets.
10

---

References
[1] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-68511 2020.
[2] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[3] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[4] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35:26565-26577 2022.
[5] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[6] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2020.
[7] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In International Conference on Learning Representations, 2020.
[8] Vadim Popov, Ivan Vovk. Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad- tts: diffusion probabilistic model for text-to-speech. International Conference on Machine Learning, pages 1999-8608.. PMLR, 2021.
[9] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633- 8646, 2022.
[10] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao. Alexey Gritsenko, Diederik P Kingma, Ben Poole. Mohammad Norouzi, David J Fleet, al. Imagen video: High definition vidco generation with diffusion models. arXiv preprint arXiv:2210.02303: 2022.
[11] Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2837-2845, 2021.
[12] Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy Mitra. Holodiffusion: Training a 3d diffusion model using 2d images. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 18423-18433 2023.
[13] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
[14] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. In International Conference on Machine Learning (ICML), 2022.
[15] Shutong Wu, Jiongxiao Wang, Wei Ping, Weili Nie, and Chaowei Xiao. Defending against adversarial audio via diffusion model. In The Eleventh International Conference on Learning Representations, 2022.
[16] Jiachen Sun, Jiongxia Wang, Weili Nie, Zhiding Yu, Zhuoqing Mao, and Chaowei Xiao. critical revisit of adversarial robustness in 3d point cloud recognition with diffusion-driven purification. In International Conference on Machine Learning, pages 33100-33114 PMLR, 2023.
11

---

[17] Jinyi Wang, Zhaoyang Lyu. Dahua Lin, Bo Dai, and Hongfei Fu. Guided diffusion model for adversarial purification. arXiv preprint 2.Xi::2205.14969, 2022.
[18] Quanlin Wu, Hang Ye, and Yuntian Gu. Guided diffusion model for adversarial purification from random noise. arXiv preprint arXiv:2206. 2022.
[19] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1310-1320. PMLR, 09-15 Jun 2019.
[20] Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. Advances in Neural Information Processing Systems, 32, 2019.
[21] Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradcep Ravikumar, Cho-Jui Hsich, and Liwei Wang. Macer: Attack-fres free and scalable robust training via maximizing certified radius. arXiv preprint arXiv:2001.02378 2020.
[22] Jongheon Jeong and Jinwoo Shin. Consistency regularization for certified robustness of smoothed classifiers. Advances in Neural Information Processing Systems, 33:10558-10570, 2020.
[23] Miklós Z Horváth, Mark Niklas Müller, Mare Fischer, and Martin Vechev. Boosting randomized smoothing with variance reduced classifiers. arXiv preprint arXiv:2106.06946. 2021.
[24] Jongheon Jeong, Sejun Park, Minkyu Kim, Heung-Chang Lee. Do-Guk Kim, and Jinwoo Shin. Smoothmix: Training confidence--alibratted smoothed classifiers for certified robustness. Advances in Neurul Information Processing Systems, 54:00133-3016,, 2021.
[25] Nicholas Carlini, Florian Tramer, J Zico Kolter, et al. (certified!!) adversarial robustness for free! arXiv preprint arXiv:2206. 10550, 2022.
[26] Chaowei Xiao, Zhongzhu Chen, Kun Jin, Jiongxiao Wang, Weili Nie, Mingyan Liu, Anima Anandkumar, Bo Li, and Dawn Song. Densepure: Understanding diffusion models for ad- versarial robustness. In The Eleventh International Conference on Learning Representations, 2022.
[27] Jiawei Zhang, Zhongzhu Chen, Huan Zhang, Chaowei Xiao, and Bo Li. (DiffSmooth): Certifiably robust learning via diffusion models and local smoothing. In 32nd USENIX Security Symposium (USENIX Security 23), pages 4787-4804, 2023.
[28] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020.
[29] Huanran Chen, Yinpeng Dong, Shitong Shao, Zhongkai Hao, Xiao Yang, Hang Su, and Jun Zhu. Your diffusion model is secretly certifiably robust classifier. arXiv preprint 162 2024.
[30] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference computer vixion and pattern recognition, pages 586-595, 2018.
[31] Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning. 116-6))355-607, 2019.
[32] Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326 1982.
[33] Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.
[34] Huan Zhang, Tei Weng, Pin- Yu Chen, Cho-Jui Hsieh, and Luea Daniel. Efficient neural network robustness certification with general activation functions. In NeurIPS, 2018.
12

---

[35] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[36] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255 leee, 2009.
[37] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference Machine Learning, pages
PMLR, 2023.
[38] Jongheon Jeong and Jinwoo Shin. Multi-scale diffuuson denoised smoothing, In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[39] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
[40] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition. pages 770-778, 2016.
[41] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks, 2017.
[42] Miklós Z Horváth, Mark Niklas Müller, Mare Fischer, and Martin Vechev. Robust and accurate- compositional architectures for randomized smoothing. arXiv preprint arXiv:2204. 00487, 2022.
13

---

A Consistency Purification Algorithm
We provide detailed descriptions of Consistency Purification in the following algorithms. Algorithm| [ presents the function of Consistency Fine-tuning and Consistency Purification respectively. Algo- rithm 2 shows the randomized smoothing algorithm from [9] with applying Consistency Purification to do prediction and compute the certified radius.
Algorithm 1 Consistency Fine-tuning and Consistency Purification
Input: Consistency model purifier De where 0 represents the model parameters. Noise levels used in randomized smoothing {a; ) Arbitrary classification model far. Fine-tuning learning rate
function CONSISTENCYFFINEETUUNGGGCC
2: repeat
3:
sample x E Training Dataset, E {a:)
4: To f- x. +N x2)
5:
t <- GETTIMESTEP(O)
6:
C < LPIPS(x, Da(Za,t"))
7:
+ 0 - VV
8:
until convergence
9: return Do
10: end function
11: function CONISTEENYYPPRIFFAATOOO...
12:
10 + GETTIMESTEP(O)
13: Ers =x+N
14:
Ip DDD (
y f
16:
return y
end function
function GETTIMESTEP(T)
19:
t - (e + N-1
-- (T
e/ry for i f1...., N}
to + find (tilor E (#atth,, 4
21: return ド end function
B Proof of Theorem 3.3
Theorem 3.3. Given the transport Tw, (p) between the data distribution p and the corresponding purified distribution under 9u, then for any O, the probability that the distance between the original sample æ and purified sample a = Te(x) is larger than is upper bounded by Tz,(p)
Proof. We can leverage the Markov's inequality. Because
/
E|i
[]]
x #|| p(x)dxx
/ |
Æll p(x dd
||00 Sr
||0 a|l p(x))d
1|20 Sr
r. pp
|| |r
P([ 21| > r),
14

---

we have
E| 2
P(I - Ø6|| > >))
x me (x)
Tp)(p)
Algorithm/ 2 Randomized Smoothing 19
Input:
Sampling times for prediction n. Sampling times for certification N. Significant confidence level a. Function LOWERCONFBBOUND// n, a)) returns one-sided (1-o) lower confidence interval for the Binomial parameter p given that k ~ Binomial(1,
1: function PREDICT(fa, o,1,0)
2: counts < 0
3: for i 21,22 ... n do
4:
f PONSSSTENC
5: countsly] countsly] +1
6: end for
7: DA ÚB + top two labels in counts
8: nA. ng co co)nntss
9:
if (INOMTEST(RAA n, + nB, 2) S then
10:
return AA
else
12:
return Abstain
13: end
14: end function
16: function
17: countsO +-
18: for 21. n do
19:
20: countsOly] countsOly]
21: end for
22: DA < top label in counts0
23:
counts <- 0 ... 21, N) do
24: for i
25:
countsly] + countsly]+
27:
end for
28:
PA < LOWERCCO
-a)
29: if PA > then
30:
return prediction jA and radius -
else
32:
return Abstain
33:
end
34:
end function
C Training Unconditional Consistency Model for ImageNet-64
We train an unconditional consistency model for ImageNet-64 from the public available conditional version by transiting the class embedding layers to a learnable token, initialization with average class embeddings. For each model forwarding, this token will be combined with the time embeddings for computation. After that, we train the conditional consistency model, initialized with the unconditional model's parameters, ImageNet-64 training set for 120k steps.
15

---

D Purified Images Visualization for ImageNet-64
We present visualization images after diffusion purification by applying onestep-DDPM and Consis- tency Purification for ImageNet-64 under the noise level = 0.25 in Figure
(a) Purified images by onestep- DDsM
(b) Purified images by Consistency Purification
Figure 5: Visualization of purified images after the diffusion purification by applying onestep-DDPP DDPM and Consistency Purification on ImageNet-64 with a = 0.25 noise level. Identical noise patterns are applied to images at corresponding locations. A green border indicates that the purified image is correctly classified, while a red border denotes misclassification by the classifier.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have included our paper's contributions and scope in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the
contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitaitons of our work after Conclusion Section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-speciffiation, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[Yes]
Justification: We provide detailed assumptions in Section 3 and the proof of theorem in Appendix [
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
17

---

All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All experiments in our paper can be reproducible with all disclosed information shown our experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken
to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have submitted our code as the Supplementary Material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips)
pub 1 c guides /odeeSumiiiieeecccccee wsw.nnooiiccoooooooooooo
for more details. While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not
18

---

including code, unless this is central to the contribution (e.g., for new open-soure benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https) //nips. cc/ publ i c/ gu des Codeeu Submi
onPol icy) for more details. The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
reproducible, they
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We have include the detailed experiment settings in Section 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars are not reported because would be too computationally expensive for the large language model fine-tuning process. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text. 8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We include the details about the experiments compute resources in Section
19

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:://///ett http://nerripspp ctt.risspcccppppcco cc/ httppscccccpllpppppccppppppppplp wtt.c///blccccttthceeeeeee c/Ethi caGuide ness
Answer: [Yes]
Justification: We have checked the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics. If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid-
eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of our work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal
impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g.. disinformation. generating fake profiles, surveillance), fairness considerations (e.g., deplommeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and funetioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Justification: Our paper does not have such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring
20

---

that users adhere to usage guidelines or restrictions to access the model or implementing
safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors
should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have included licenses of original owners in code and made citations
in our papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papersw's hcode. com/ data aste has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part of their
submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
21

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing or research with human subjects. Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
22


Reviews:
[{"review_id": "ebMxyX0Hk1", "reviewer": "Reviewer_43Xy", "summary": "This paper introduces Consistency Purification, a novel framework that integrates consistency models with diffusion purification to enhance the certified robustness of deep neural networks, achieving efficient and effective image purification. The framework is further refined through Consistency Fine-tuning with LPIPS loss to ensure semantic alignment between the original and purified images, demonstrating state-of-the-art performance in both certified robustness and efficiency compared to existing methods.", "questions": "See Weakness.", "limitations": "None", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The motivation is agreeable, and the paper is well-structured.\nTheoretical explanations on the advantages of Consistency Purification are provided and clearly written.", "weaknesses": "It's noteworthy that there have been non-diffusion-based prior studies for enhancing certified robustness. A comparative analysis with those works would add depth to this study.\nThe experimental setting, including baseline selection and the selection of evaluation data, is not consistent with the baseline method One-Step DDPM. In this paper, only 100 samples from ImageNet are selected for evaluation. It would be beneficial to evaluate the method across multiple benchmarks and with more test samples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Question A: Comparative analysis with non-diffusion-based pripor studies.\n\nTo compare our consistency purification method with various non-diffusion-based approaches, we conducted additional experiments to compute the certified accuracy under three non-diffusion-based methods [1,2,3]. Cohen et al. [1] first proposed training a classifier with noisy images to ensure certified robustness. Subsequent works [2,3] build on Cohen et al.'s methodology, attempting to enhance the smoothed classifier by adding prediction consistency regularization, or incorporating per-sample bias.\n\n| | 0.0 | 0.25 | 0.50 | 0.75 | 1.00 |\n |-------|------|------|------|------|------| \n| RS [1] | 74.8 | 59.2 | 42.0 | 31.8 | 22.0 |\n| Regularization [2] | 74.4 | 66.0 | 56.2 | 41.4 | 32.8 |\n | ACES [3] | 74.6 | 66.4 | 57.0 | 43.6 | 32.8 |\n| Consistency Purification | **90.4** | 77.2 | 59.8 | 42.8 | 33.2 |\n | + Consistency Finetune | 90.2 | **79.4** | **62.4** | **43.8** | **35.4** |\n\nThe experimental results presented in the table show that our method surpasses all previous non-diffusion-based methods in achieving higher certified accuracy, particularly with a significantly high clean performance at $\\sigma=0.0$.\n\nFurthermore, we would like to highlight that, in contrast to non-diffusion-based methods which incur significant costs by requiring additional fine-tuning of robust classifiers for each specific noise level, our method can be applied directly to any off-the-shelf classifiers. This significantly broadens its practical applications.\n\n[1] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310-1320. PMLR, 2019.\n\n[2] Jongheon Jeong, and Jinwoo Shin. Consistency regularization for certified robustness of smoothed classifiers. Advances in Neural Information Processing Systems 33 (2020): 10558-10570.\n\n[3] Miklós Z. Horváth, Mark Niklas Müller, Marc Fischer, and Martin Vechev. Robust and Accurate--Compositional Architectures for Randomized Smoothing. arXiv preprint arXiv:2204.00487 (2022).\n\n\n> Question B: Experimental setting of evaluation data selection.\n\nAll experiments in our paper utilize the same selection criteria for evaluation data. For the CIFAR-10 dataset, we selected 500 test examples from the 10,000 CIFAR-10 test set, choosing every 20th example in sequence (e.g., the 1st, 21st, 41st, etc.). Similarly, for the ImageNet-64 dataset, we selected 100 test examples from its 50,000  test examples using a fixed interval of 500. This consistent approach ensures that all evaluation datasets, including the baseline method One-Step DDPM, are identical across all experiments, thereby guaranteeing fair comparisons with our proposed method.\n\n> Question C: Limited evaluation examples for ImageNet-64 dataset.\n\nHere we include an additional experiment on ImageNet-64 using 500 samples, selecting every 100th example in sequence from the 50,000 ImageNet-64 test set. We present the certified accuracy of Consistency Purification in comparison with One-Step DDPM in the table below. The results consistently show that our method, with consistency fine-tuning, achieves the highest certified accuracy across the 500 test set of ImageNet-64, demonstrating the effectiveness of our approach.\n\n|                       | 0.0  | 0.05 | 0.15 | 0.25 | 0.35 |\n|-----------------------|------|------|------|------|------|\n| onestep-DDPM          | 55.2 | 44.8 | 33.4 | 15.2 |  8.8 |\n| Consistency Purification | 62.4 | 54.2 | 35.2 | 19.8 | 13.0 |\n| + Consistency Fine-tuning    | **68.6** | **58.0** | **37.4** | **23.4** | **17.4** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mospYS7yro", "reviewer": "Reviewer_wtha", "summary": "The paper introduces the Consistency Model for one-step purification, ensuring the data manifold of the purified samples while maintaining the efficiency of one-step purification. At the same time, the paper proposes Consistency Finetuning to fine tune the Consistency Model to ensure semantic consistency of the purified samples.", "questions": "1. In Example 3.1, the author states in lines 177-178 that if distribution consistency is already being ensured, it should already be a relatively strong constraint. Why can only ensure generation and not enough for denoising?\n2. The paper's statements in 188-191 and the proof in Theorem 3.3 demonstrate the importance of small transport. However, the ideal denoising should be to transform d into x. For a defined transport, it is natural and reasonable to require it to be small. What the author needs to prove more is whether this Gaussian transport lower can be transferred to adversarial attacks, rather than just proving that lower is good?", "limitations": "The paper has addressed the limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper achieves a trade-off between efficiency and effectiveness, realizing Consistency Purification in a one-step manner.\n2. The author conductes both theoretical and experimental verification of the proposed method.\n3. The paper achieves consistent performance improvement in the experiment.", "weaknesses": "1. The paper's innovation is limited, using existing Consistency Models and LPIPS loss to form the diffpure framework.\n2. There are some descriptions not easy to understand. Suggest the author to provide a brief explanation and clarification when transport (Page 3, line 77) first appears in the Introduction. Does Remark 3.4 (line202) need to be bolded?\n3. The paper's validation set on ImageNet-64 is too small, and 100 images are difficult to accurately reflect its validity. Meanwhile, one of the purposes of this paper is to enhance the effectiveness, but the experimental dataset does not align with this original intention.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Concerns about the limited innovation in our paper\n\nWe summarize our contributions and novelty in three points:\n1. Using Consistency Models(CM) are efficient and effective: \n\nWe provide theoretical (section 3, lines 143-174) and empirical support (Table 2) showing that CM enables us to achieve a Pareto-optimal purification framework compared to previous frameworks. First, CM significantly improves purification efficiency through its one-step property, compared to multi-step models like DDPM [1], Score-SDE [2], and EDM [3]. Second, CM produces on-manifold purified samples, enhancing purification effectiveness over other one-step models, such as one-step DDPM [1, 4], which can generate off-manifold samples that have semantic ambiguity (leading to low classifier confidence or misclassification). Third, compared to non-diffusion-based models, we can leverage off-the-shelf CM without requiring additional training, such as adversarial classifiers.\n\n2. Leveraging CM alone is insufficient: \n\nWe provide theoretical (section 3, lines 175-207) and empirical results (Table 1, Figure 2) to show this and further consistency-finetuning with LPIPS loss is necessary. The original CM cannot guarantee semantic alignment and can potentially recover a noisy bookstore image as a market image. Our consistency-finetuning aims to mitigate this issue.\n\n3. Why LPIPS instead of L1 & L2? \n\nWe provide theoretical support (section 3, lines 208-216) and experimental results (Table 1) to explain why typical L1 and L2 losses do not work and why LPIPS loss is appropriate for consistency fine-tuning. In particular, since the LPIPS loss measures the semantic difference rather than just the distance between samples, it ensures correct classification without requiring the purified sample to be identical to the original image. Conversely, minimizing L1 or L2 loss could potentially disrupt the CM structure and lead to off-manifold samples and semantic ambiguity.\n[1] Ho et al. Denoising diffusion probabilistic models. NeurIPS, 2020.\n[2] Song et al. Score-based generative modeling through stochastic differential equations. ICLR, 2021.\n[3] Karras et al. Elucidating the design space of diffusion-based generative models. NeurIPS, 2022.\n[4] Carlini & Tramer et al.(Certified!!) Adversarial Robustness for Free! ICLR, 2023.\n\n> Brief explanation and clarification of the transport definition\n\nAdditionally, our transport (Definition 3.2) aligns with the standard definition in optimal transport theory [1]. Remark 3.4 suggests if the recovered sample distribution is closer to the original and the classifier is more robust, purification performance improves, we can bold it in the final version.\n\n[1] Peyré et al. Computational optimal transport: With applications to data science.\n\n> Small validation set on ImageNet-64\n\nTo accurately evaluate the certified radius for our experiments on ImageNet-64, we expanded our test dataset to include 500 samples. We also present the variance across five subsets of 100 sample results. The results, shown in the following table, illustrate that consistency purification with consistency fine-tuning continues to achieve the highest certified accuracy. Additionally, the small variance observed across the five subsets of 100 samples further demonstrates the validity of our evaluation.\n\n|  | 0.0  | 0.05 | 0.15 | 0.25 | 0.35 |\n|-------------|------|------|------|------|------|\n| onestep-DDPM | 55.2±1.17 | 44.8±0.40 | 33.4±0.75 | 15.2±0.37 | 8.8±0.75 |\n| Consistency Purification | 62.4±1.02 | 54.2±1.16 | 35.2±0.75 | 19.8±0.40 | 13.0±0.63 |\n| + Consistency Fine-tuning| **68.6**±1.20 | **58.0**±1.10 | **37.4**±1.33 | **23.4**±1.20 | **17.4**±1.02 |\n\nAdditionally, while our framework enhances the effectiveness of the one-step purification process, the underlying randomized smoothing method [1] still necessitates extensive sampling times N (e.g. $N=10,000$) for each test example evaluation. This extensive sampling is required to construct a reliable confidence interval for the certified robustness radius statistically. Furthermore, evaluations at various noise levels are required for each line of results presented in our Table 2. Consequently, evaluating the certified accuracy across the entire 50,000 test examples of ImageNet-64 remains a significant time cost.\n\n[1] Cohen et al. Certified adversarial robustness via randomized smoothing. international conference on machine learning. PMLR, 2019.\n\n> Why can only ensure generation is not enough for denoising?\n\nEven if the recovered sample is on-manifold, it can have a different semantic meaning from the original. The on-manifold property benefits generation but not denoising, where semantic consistency is crucial. For example, with a two-point data distribution at $\\{-1, 1\\}$ where $p(x=1)=p(x=-1)=0.5$, and the noisy distribution is similar, if a purification pipeline maps $-1$ to $1$ and $1$ to $-1$, the generation quality is perfect, but denoising fails.\n\n> Concerns about the importance of small transport\n\nAs shown in lines 188-191 and Theorem 3.3, given an attack $\\epsilon$ on a data point $x$, for a purification framework $d$, we obtain the recovered data $\\hat{x} = d(x+\\epsilon)$. The lower the transport from $\\hat{x}$ to $x$, the more likely successful purification is. Figure 2 and Table 1 demonstrate that our purification framework $d$ reduces this transport compared to baselines through fine-tuning. Specifically, Figure 2 shows uniform transport reduction for recovered data across all $\\sigma$ with CM, and consistency fine-tuning further reduces transport compared to the original model.\n\nWe do not claim the transport between the data and the noisy distribution after the attack is small. If such transport is small, the attack is minimal, and successful purification is easier. Our theory applies to attacks causing large transport, such as Gaussian noise attacks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LGQtQsABVA", "reviewer": "Reviewer_aFWW", "summary": "This paper presents an innovative approach to image purification using diffusion models, known as Consistency Purification. Traditional methods like the Denoising Diffusion Probabilistic Model (DDPM) and the Stochastic Diffusion Model face challenges in balancing efficiency and effectiveness. While DDPM provides efficient purification, it fails to ensure that purified images lie on the data manifold. On the other hand, the Stochastic Diffusion Model successfully places images on the data manifold but is computationally intensive. The proposed Consistency Purification leverages a consistency model distilled from the Probability Flow Ordinary Differential Equation (PF-ODE), achieving on-manifold purification in a single step. This approach is further refined through Consistency Fine-tuning with LPIPS loss, enhancing semantic alignment between purified and original images. Comprehensive experiments demonstrate that this framework achieves state-of-the-art certified robustness and efficiency compared to baseline methods.", "questions": "refer to weakness", "limitations": "refer to weakness", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Introducing the consistency model to the purification domain is an interesting idea, because it utilizes the consistency model's feature that maps noised images back to origin. , and experimental results show significant improvements in classification efficiency.\n2. The writing and logical flow are clear and well-structured.", "weaknesses": "1. There is no comparison with multi-step DDPM methods. Providing these results and the corresponding speedup would make the work more solid.\n2. The impact of noise intensity on experimental results is not mentioned. Could you please show this in the rebuttal phase? \n3. It is unclear how the model performs if the noise is not Gaussian. Providing results for different noise distributions would enhance the robustness and applicability of the method.\n4. Including visualizations of the purified images before and after the purification process would make the results more intuitive and compelling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Question A: Comparison with multistep DDPM methods.\n\nWe conducted an experiment with multistep DDPM using 25 sampling steps, and the results are shown in the table below. We found that multistep DDPM has lower certified accuracy than onestep DDPM. This aligns with Carlini et al.'s [1] finding that multistep DDPM sampling adds unwanted details to the image, altering its semantic meaning. Our method provides a higher certified radius and is 25 times faster than multistep DDPM.\n\n|                           | 0.0  | 0.25 | 0.50 | 0.75 | 1.00 |\n|---------------------------|------|------|------|------|------|\n| onestep-DDPM | 87.6 | 73.6 | 55.6 | 39.2 | 39.6 |\n| multistep-DDPM | 86.2 | 72.4 | 55.8 | 40.6 | 31.8 |\n| Consistency Purification | **90.4** | 77.2 | 59.8 | 42.8 | 33.2 |\n| + Consistency Fine-tuning | 90.2 | **79.4** | **62.4** | **43.8** | **35.4** |\n\n\n> Question B: Impact of noise intensity on experimental results.\n\nIn all experiments presented in our paper, we compute the certified radius for each test example at three distinct noise intensities, with $\\sigma \\in {0.25, 0.5, 1.00}$ for CIFAR-10 and $\\sigma \\in {0.05, 0.15, 0.25}$ for ImageNet-64. We then calculate the proportion of test examples whose radius exceeds a specific threshold $\\epsilon$. The highest accuracy among these noise intensities is reported as the certified accuracy at $\\epsilon$. For more detailed results of the certified accuracy under various noise intensities, we have included the results for each $\\sigma$ in the tables below. The results clearly show that our method consistently outperforms other methods across all noise levels.\n\nCertified Accuray under various $\\sigma$ for CIFAR-10\n\n| Method                    | $\\sigma$ | 0.00 | 0.25 | 0.50 | 0.75 | 1.00 |\n|---------------------------|------|------|------|------|------|------|\n | onestep-DDPM          | 0.25 | 87.6 | 73.6 | 55.6 | 37.8 | 0.0  |\n |                           | 0.5  | 73.6 | 61.0 | 49.8 | 39.2 | 29.6 | \n|                           | 1.0  | 49.2 | 40.6 | 33.2 | 26.6 | 21.4 |\n |Consistency Purification | 0.25 | 90.4 | 77.2 | 59.8 | 37.8 | 0.0  | \n|                           | 0.5  | 77.2 | 65.0 | 52.6 | 42.8 | 33.2 |\n|                           | 1.0  | 52.0 | 44.2 | 36.4 | 29.4 | 23.8 | \n| + Consistency Fine-tuning           | 0.25 | **90.2** | **79.4** | **62.4** | **43.2** | 0.0  | \n|                           | 0.5  |**76.4** | **66.4** | **53.8** | **43.8** | **35.4** |\n|                           | 1.0  | **55.4** | **47.2** | **41.0** | **32.0** | **26.0** |\n\nCertified Accuray under various $\\sigma$ for ImageNet-64\n\n| Method                    | $\\sigma$ | 0.00 | 0.05 | 0.15 | 0.25 | 0.35 |\n|---------------------------|------|------|------|------|------|------|\n| onestep-DDPM         | 0.05 | 55.2 | 44.8 | 33.4 | 0.0  | 0.0  |\n|                           | 0.15 | 38.2 | 35.4 | 21.8 | 15.2 | 8.8  |\n|                           | 0.25 | 13.2 | 10.8 | 8.4  | 5.4  | 4.0  |\n| Consistency Purification| 0.05 | 62.4 | 54.2 | 35.2 | 0.0  | 0.0  |\n|                           | 0.15 | 41.8 | 37.2 | 25.4 | 19.8 | 13.0 |\n|                           | 0.25 | 16.2 | 13.8 | 13.0 | 6.2  | 5.8  |\n| + Consistency Fine-tuning         | 0.05 | **68.6** | **58.0** | **37.4** | 0.0  | 0.0  |\n|                           | 0.15 | **51.0** | **43.6** | **32.4** | **23.4** | **17.4** |\n|                           | 0.25 | **18.2** | **15.4** | **13.2** | **8.0**  | **7.2**  |\n\n\n> Question C: Model performances under non-Gaussian noise.\n\nIn the scheme of randomized smoothing, the noise added to the image is chosen to be Gaussian for the simplicity of certified radius calculation, which is the standard practice in previous work [1, 2, 3]. We admit that it will be an interesting direction to conduct non-Gaussian noise to the randomized smoothing approach in future work. \n\n> Question D:  Visualizations of the purified images before and after the purification process\n\nPlease refer to the visualization of images before and after purification via the following anonymous link: https://anonymous.4open.science/r/Consistency-Purification-9B5F/README.md. We have included purified images for CIFAR-10 at a noise level of 0.5 and for ImageNet-64 at a noise level of 0.25. For both datasets, our method produces images that are more detailed and accurate than those processed by One-Step DDPM [1]. This enhancement in image quality partially explains why our method achieves significantly higher certified accuracy compared to the One-Step DDPM method.\n\n[1] Nicholas Carlini et al. (certified!!) adversarial robustness for free! arXiv preprint arXiv:2206.10550, 2022.\n\n[2] Nie, Weili, et al. Diffusion models for adversarial purification. arXiv preprint arXiv:2205.07460 (2022).\n\n[3] Xiao, Chaowei, et al. Densepure: Understanding diffusion models towards adversarial robustness. arXiv preprint arXiv:2211.00322 (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Consistency Purification, a novel framework that integrates consistency models with diffusion purification to enhance the certified robustness of deep neural networks.  The paper provides both theoretical and experimental verification of the proposed method, and is well-written.  Some concerns raised by the Reviewers on the limited novelty and experimental evaluations were successfully addressed by the authors’ rebuttal.  The authors should incorporate all revision and improvement promised in the rebuttal into the final version of the paper.
Decision: Accept (poster)