ADBM: Adversarial Diffusion Bridge Model for Reliable Adversarial Purification
Authors: Xiao Li
Wenxuan Sun
Huanran Chen
Qiongxiu Li
Yingzhe He
Jie Shi
Xiaolin Hu
Venue: ICLR 2025 Poster (2025)
Keywords: diffusion models
adversarial robustness
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: A novel adversarial purification method by building a adversarial diffusion bridge model.

Abstract: Recently Diffusion-based Purification (DiffPure) has been recognized as an effective defense method against adversarial examples. However, we find DiffPure which directly employs the original pre-trained diffusion models for adversarial purification, to be suboptimal. This is due to an inherent trade-off between noise purification performance and data recovery quality. Additionally, the reliability of existing evaluations for DiffPure is questionable, as they rely on weak adaptive attacks. In this work, we propose a novel Adversarial Diffusion Bridge Model, termed ADBM. ADBM directly constructs a reverse bridge from the diffused adversarial data back to its original clean examples, enhancing the purification capabilities of the original diffusion models. Through theoretical analysis and experimental validation across various scenarios, ADBM has proven to be a superior and robust defense mechanism, offering significant promise for practical applications. Code is available at https://github.com/LixiaoTHU/ADBM.

Published as a conference paper at ICLR 2025
ADBM: ADVERSARIAL DIFFUSION BRIDGE MODEL FOR RELIABLE ADVERSARIAL PURIFICATION
Xiao Li',, Wenxuan Sun Huanran Chen Qiongxiu Li Yining Liul.5. Yingzhe He Jie Shi Xiaolin Hulf 'Department of Computer Science and Technology, Tsinghua University
"Peking University *Bejjing Institute of Technology *Aalborg University Harbin Institute of Technology, Weihai 'Huawei Technologies 1ixiao20@mails. ts .edu. sunwenxuan@stu. pku. www.uuu.kkuueuuuu.ooooooooo
huanranchensiit.eduu. en, qilises. 2250301840stu.hit.co wttp:mmaaa.iiiiinaaeemmmmmm
(neyingzhe, shi. shi.jiel
shauijiel@ehuawei.. wauaweeuaawoooommmoo
ABSTRACT
Recently Diffusion- Purification (DiffPure) has been recognized ef- fective defense method against adversarial examples. However, we find DiffPure which directly employs the original pre-trained diffusion models for adversarial purification, to be suboptimal. This is due to an inherent trade-off hetween noise purification performance and data recovery quality. Additionally, the reliability of existing evaluations for DiftPure is questionable, as they rely on weak adaptive attacks. In this work, we propose novel Adversarial Diffusion Bridge Model, termed ADBM. ADBM directly constructs a reverse bridge from the diffused ad- versarial data back to its original clean examples, enhancing the purification capa- bilities of the original diffusion models. Through theoretical analysis and experi- mental validation across various scenarios, ADBM has proven to be superior and robust defense mechanism, offering significant promise for practical applications. Code is available at httpt:://// /tEtts://github com Ltttn//iiiioooOHU ADEM
INTRODUCTION
An intriguing problem in machine learning models, particularly Deep Neural Networks (DNNs), is the existence adversarial examples (Szegedy al. [014 Goodfellow al. 2015). These exam- ples introduce imperceptible adversarial perturbations leading to significant errors, which has posed severe threats to practical applications (Bykholt et al. 2018 Li et al.) 2025). Numerous methods
have been proposed to defend against adversarial examples. But attackers can still evade most early methods by employing adaptive attacks (Athalye et al., 2018 Tramer et al. 2020). Adversarial Training (AT) methods (Madry et al.) 2018 Li et al. 2024c b) are recognized as effective defense methods against adaptive attacks. However, AT typically involves re-training the entire DNNs using adversarial examples, which is impractical for real-world applications. Moreover, the effectiveness of AT is often limited to the specific attacks it has been trained against, making brittle against unseen threats (Zhang et al. 2019 Laidlaw et 2021).
Recently, Adversarial Purification (AP) methods (Shi et al. 2021 Yoon et al. 2021 have gained increasing attention they offer potential solution to defend against unseen threats in plug- and-play manner without retraining the classifiers. These methods utilize the so-called purification module, which exploits techniques such as generative models, as a pre-processing step to restore clean examples from adversarial examples, as illustrated in Figure [[). Recently, diffusion mod- els (Ho al.) 2020), one type of generative model renowned for their efficacy, have emerged as potential AP solutions (Nie et al. (022). Diffusion models learn transformations between complex data distributions and simple distributions like the Gaussian distribution through forward diffusion and reverse prediction processes. In the context of adversarial defense, Diffusion-based Purification
"qqual contribution.
"ooresponndnce Xiaolin Hu.

---

Published as a conference paper at ICLR 2025
(0) Pipeline adveruriial puriftaaiio
Category
(b) DitiPure
Classifier
Adv. Dist.
Diffissed Adv, Dist.
Diffised Cleun Dist.
Cleun Dist.
Punfied Image
ADBM
Purification Module
Adversarial Image
Adv.
Diffused Adv.
Clean Dist.
Figure 1: The inference pipeline of AP (a) and the comparison between DiffPure (b) and ADBM (c). DiffPure relies on the diffused adversaria data distribution (Diffused Adv. Dist.) being close enough to the diffused clean data distribution. ADBM directly builds a reverse process from the diffused adversarial data distribution to clean data distribution.
(DiffPure) (Nic et al. 2022). which tries to purify the adversarial examples by first adding Gaus- sian noise through the forward process with a small diffusion timestep and then recovering clean examples (removing the added Gaussian noise together with adversarial noise) through the reverse process, has achieved superior performance among recent AP methods.
However, find that utilizing DiftPure in the context of AP is suboptimal. The primary reason is that DiffPure directly employs pre-trained diffusion models originally designed for generative tasks, rather than specifically for AP tasks. To delve deeper, the divergence in data distributions between clean and adversarial examples (Stutz et al., 2019 Xie et al. (020)) often leads to noticeable differ- ences in their respective diffused distributions, while DiftPure relies on the assumption that the two diffused distributions are sufficiently close, such that the original reverse process can recover the diffused adversarial data distribution, as depicted in Figure [b). The conflict between the assump- tion and the actual situation compromises the effectiveness of AP. DiffPure attempts to mitigate this conflict by introducing significant noise with a large diffusion timestep. Nevertheless, this solution is impractical; substantial diffusion timestep can severely corrupt the global structure of the input, resulting in a fundamental trade-off between noise purification efficiency and recovery quality. In addition, observe that existing evaluations for DifiPure rely weak adaptive attacks, which may inadvertently give a false sense of security regarding diffusion-baa based purification.
To address the aforementioned problems, this study performs systematic investigation. We first establish a reliable adaptive attack evaluation method for diffusion-basn -based purification. With the rigorous evaluation, our preliminary findings suggest that the robustness of DiffPure is overestimated in existing works (Nie et al.) 2022 Lee & Kim 2023 Chen et al. 2023a). To improve the robustness of diffusion- based purification, we then propose novel Adversarial Diffusion Bridge Model, termed
ADBM. Unlike original diffusion models relying on the similarity between the diffused distributions of clean and adversarial examples for a balanced trade-off, ADBM constructs a direct reverse process (or "bridge") from the diffused adversarial data distribution to the distribution of clean examples, as shown in Figure [T.. The theoretical analysis supports the superiority of ADBM over using original pre-trained diffusion models (DiffPure). In addition, we discuss how to accelerate ADBM for efficient adversarial purification to enhance the practicality of ADBM as a defense mechanism.
Experimental results show that ADBM achieved better robustness than DiffPure under reliable adap- tive attacks. In particular, ADBM achieved a 4.4% robustness gain compared with DifiPure on av- erage on CIFAR- 10 (Krizhevsky al. 2209)). while the clean accuracies kept comparable. ADBM also demonstrated competitive performances with AT methods for seen adversarial threats and stood among recent AP methods. Furthermore, ADBM exhibited much better adversarial robustness than AT methods when facing unseen threats. Additionally, the evaluation results against transfer- based and query-based attacks indicate the practicality of ADBM compared with existing methods.
Our main contributions can be summarized as follows:
We propose ADBM, a novel adversarial diffusion bridge model, as an efficient AP method. Additionally, investigate methods to accelerate the proposed ADBM.
We conduct theoretical analysis to illustrate the superiority of ADBM.
2

---

Published as a conference paper at ICLR 2025
We develop simple yet reliable adaptive attack evaluation method for diffusion- purification methods.
Experiments various settings validate the effectiveness of ADBM, highlighting its relia- bility and potential defense method for practical scenarios when compared with AT.
2 PRELIMINARY AND RELATED WORK
Diffusion models. Given data distribution g(xo), DDPM (Ho al. 2020 constructs discrete-
time Markov chain (xo...., xT) } as the forward process for X ~ ((x)) Gaussian noise is gradually
added to X during the forward process following scaling schedule (8o, 31, ,Br), where 80
O and 3r + 1, such that Xy is near an isotropic Gaussian distribution:
g(x1|xx1-) Nxx: V 1,
(1)
Denote 0t ;= 1 - BB and ā == II: 04 then
(xx,(xx) (o) = -xx:; VaiXo, (1 Le, 6) Varxo + VT āge, V 6
(2)
To generate examples, the reverse distribution (x)) should be learned by a model. But is hard to achieve it directly. In practice, DDPM considers the conditional reverse distribution (x, Xo) and uses xx(X,, as an estimate of Xg. where
X (x,,,) t) == (xt - vI Exco(Xi, t))/ Val
(3)
For given Xo, the training loss Lad of diffusion models thus defined as
L E,, ||c ca(vā + VI ãie, t t)]
To accelerate the reverse process of DDPM, which typically involves hundreds of steps, Song et al. (020)) propose a DDIM sampler based on the intuition that the multiple reverse steps can be per- formed at single step via a non-Markov process. [Song al. (2021) generalize the discrete-time diffusion model to continuous-timg from the Stochastic Differential Equation (SDE) perspective.
Another series of works seemly related ours are diffusion bridges (Zhou et al.. 2024 Zheng al., 2024 Bortoli et al. 2021 Liu et al., 20236), which enlarge the design space of diffusion models by interpolating between two paired distributions. However, ADBM and current diffusion bridge models have distinct motivations and designs. Current diffusion bridge models either map the data distribution simple prior distribution (Bortoli al., 2021) or directly create a Gaussian bridge (Zhou et al. (024) or an optimal transport bridge (Zheng et al. 2024 Liu et al. 2023b) between two data distributions. The former approach (Bortoli et al.[ fundamentally cannot be used for AP, while the latter methods (Zhou et al. 2024 Bortoli et al.] 2021 Liu et al. (023b) tend to theoretical training collapse for the AP task. We give a detailed discussion to these works in Appendix A.T
Diffusion models for adversarial robustness. Recent studies have demonstrated the efficacy of diffusion models (Ho et al.] (2200) in enhancing adversarial robustness in several ways. Some re- searches leverage much data generated by diffusion models to improve the AT performance (Gowal et al. 2021 Wang et al. 2023), , but these AT-based methods do not generalize well under un- seen threat models. DiffPPue (Nie et al. 2022 employs a diffusion model
plug-and-plas pre- processing module to purify adversarial noise. Wang et al. 2022)) improved DiffPure by employing inputs guide the reverse process of the diffusion model to ensure the purified examples are close to input examples. Zhhng et al. (2023) improved DiffPure by incorporating the reverse SDE with mul- tiple Langevin dynamic runs. Zhang et al. (2024) maximized the evidence lower bound of the likeli- hood estimated by diffusion models to increase the likelihood of corrupted images. DiffPure has also shown potential in improving certified robustness within the framework of randomized smoothing (Carlini et al. 2023;; Xiao et al., 2023). Nevertheless, the practicality of randomized smoothing is greatly hindered by the time-consuming Monte Carlo sampling (Cohen et al. 2019). Different from these works, we present an diffusion-based purification method for empirical robustness in practical scenarios. In addition to these works, [hen et al. (2023a) show that a single diffusion model can be transformed into adversarially robust diffusion classifier (RDC) using Bayes' rule. However, compared with diffusion- based purification, RDC requires thousands of times the inference cost and may not scale up, limiting its practical usefulness. We discuss it further in Appendix A.2
3

---

Published as a conference paper at ICLR 2025
Table 1: Accuracies (%) of DiffPure under various evaluation attacks with an loc bound of cae 8/255 on CIFAR-IO. The timestep of DiffPure was 100, following the original implementation.
Table 2: Accuracies (%) of DiflPure on CIFAR-10 with forward step 100, var- ious reverse steps, and different sam- plers. The robust accuracy was evalu- ated under an 1x bound of = 8/255.
Evaluation
Clean Acc
Robust Ace
BPDA (Athalye et al.
2018
90.49 + 0.97
81.40 0.16
Nie el al.
2022
90.07 + 0.97
71.29 0.55
Reverse Step
DDPM
DDIM
Chen ct al.
2023a
90.97
53.52
Clean
Robust
Clean
Robust
Lec & Kmm
(2023)
90.43 + 0.60
51.13 + 0.87
100
90.49
45,83
93.50
41.21
10
84.77
41.02
92.18
41.02
Ours (I:01=20, steps=200)
45.83 E 1.27
Ours (EOT=40,, steps=200)
90.49 + 0.97
45.64 + 1.14
5
68.75
31.25
92.38
42.16
Ours (EOT=20,, steps=400)
46.16 16 + 1.33
29.10
91.79
41.02
1
17.58
91.80
41.41
RELIABLE EVALUATION FOR DIFFPURE
Before delving into the details of ADBM, is important to discuss the white-box adaptive attack for diffusion-based purification first. As the original implementation of DiffPure needs dozens of prediction steps in the reverse process, it is challenging to compute the full gradient of the whole purification process due to memory constraints. To evaluate the robustness of diffusion-basee pu- rification, several techniques have been proposed. But we found that the adaptive evaluations for DiflPure remained insufficient,
detailed Appendix B. We build on these previous insights (Nie et al. 2022 Chen et al. [223aa] Lee & Kim 2023) to develop a straightforward yet effective adaptive attack method against diffusion- based purification. We employ the gradient- based PGD attack (Madry et al. 2018), utilizing the full gradient calculation via gradient-checkpointig and incorporating substantial number EOT (Athalye et al. (018)) and iteration steps. Especially, we set the PGD iteration steps to 200, with 20 EOT samples for each iteration. We note that the cost of reliable evaluation is high yet worthwhile to avoid a false sense of security on DiffPPure, as dis- cussed in Appendix B.2 Tab. sTshowss the results of DiftPure with DDPM sampler under different evaluations Consistent with Nie et al. (2022), we conducted the adaptive attack three times on a subset of 512 randomly sampled images from the test set of CIFAR-10. The results demonstrate that our attack significantly reduced the reported robustness of DiffPure, lowering it from 7 71.29% and 51.13% 45.83% when compared with the originally reported results (Nie et al. 2022 and the recent best practice (Lee & Kim 2023), respectively. We found that increasing the number of iterations EOT steps further did lead higher attack success rates.
With the reliable evaluation method, we further investigated the influence of reverse steps and for- ward steps on the robustness of DiffPure. This exploration offers insights into the effectiveness of duffusion-based purification methods.
Reverse steps. DiffPure suffers from a high inference cost, which greatly hinders its practical ap- plication. The reverse step directly impacts the inference cost of DiffPure. Following Lee & Kim (202). we evaluated both the original DDPM sampler and the DDIM sampler. We present the results in Tab. Jusing different reverse steps and samplers (standard deviation omitted, about 1%). The re- sults show that decreasing the number of reverse steps significantly reduced both clean accuracy and robustness when using DDPM. However, for the DDIM sampler, although clean accuracy slightly decreased with fewer reverse steps, robustness was not significantly affected. This can be attributed to the fact that DDIM does not introduce additional randomness during the reverse process.
Forward steps. We observed a continuous decrease in clean accuracy and a continuous increase in robustness as the number of forward steps increased. Details be found in Appendix B.3
4 ADVERSARIAL DIFFUSION BRIDGE MODEL
ADBM aims to construct a reverse bridge directly from the diffused adversarial data distribution to the clean data distribution. We derive the training objective for ADBM in Sec. 4.1 and explain
'The DDPM sampler represents one discretization version of the reverse VP-SDE sampler (Song et al. 2021). which is originally employed in DiffPure. Their differences become negligible large timestep, allowing VP-SDE and DDPM interchangeably throughout work.
4

---

Published as a conference paper at ICLR 2025
how to obtain the adversarial noise for training ADBM in Sec. 4.2 The AP inference process using ADBM is described in Sec. 4.3 We finally show that ADBM has good theoretical guarantees for AP.
4.1
TRAINING OBJECTIVE
ADBM is diffusion model specifically designed for purifying adversarial noise. It adopts a forward process similar to DDPM, with the difference that ADBM assumes the existence of adversarial noise 6, at the starting point of the forward process during training. This means that the starting point of the forward process is X = Xo + Eg for each Xo- Thus, according to Eq. (2), the forward process can be represented as x v Vā vI â,E, e (O, I), < T. where T denotes the actual forward timestep when performing AP. is typically set to a lower value for AP, egg., 100 in DiffPure, than that used in generative tasks, e-g-, 1,000, to avoid completely corrupting Xo- We discuss how to obtain 6a in Sec. 4.2
In the reverse process of ADBM, the objective is to learn Markov chain her that directly transform from the diffused adversarial data distribution (i.e., xf.) to the clean data distribution (i.e. Xo), as shown in Figure [c). To achieve the goal of purification,
expect that the starting point and the end point of & to be XT := x². and Xy == Xo. respectively. Notably, Xy contains ad advesaaiial noise, while Xo does not. To explicitly align the trajectory of R; with the starting and ending points, we introduce coefficient he, expecting that X, := x x; - C2 for 0 I t S T, with kn 1 and ky With Bayes' rule and the property of Gaussian distribution, we have
(x} | xo) (x Xn) 1(x0-11xx)) (x))
BxQ_1 + exp (-2 +
+C))
(5)
((x,) Xo)
where A is a constant independent of E,. B is the coefficient of x dependent on ca. and C is term without x In inference, given by attackers and e, cannot be decoupled from xij directly. Based on the property Gaussian distribution, eliminating all terms related to Ca in B and C in Eq. ( can be achieved by eliminating all terms related to ca in B. This yields:
expect that ea in Eq. (5) can be eliminated since only x is
VarlVarkit- kz)Ea Ot
(Vōl-1 1a-1)e 0.
(6)
1 04-1
To satisfy Ko kr h Vār
derive hi in Eq. 66
STST.
v Vār - aT)
For detailed derivations, please refer to Appendix
C.I
Following Eq. (3), ADBM uses X (xf t)
,e, (x/ vI arEo (x), to approximate the clean
example Xy. Thus the loss of ADBM can be computed by
h2 V
VI āz
L Ex,r||X0 xx(x;, 1)
Ex,t
Von
Ea
Vor.
ca (x(
(8)
Ee,t
āt
âT ā.
Ca
c
+
ā
(1 0r) )a
1-a. Omitting
in Ho et al.
2202)) for given xo. the final loss Ls of ADBM is given by:
Vaz
aTVI- āi
+e - Ex |
LB Ee,t
(9)
(1 āT) vor
where x = Vi + /1 0rE + or(1-aa) vār(1-ar)
c The training process of ADBM is detailed in the
blue block of Figure
Comparing Eq. and Eq. La and L5 are quite similar except that L, has two additional scaled e, in both the input and prediction objective of eg. Thus in practice, the training of ADBM can fine- tune the pre-trate areitrained diffusion checkpoint with Lg, avoiding training from scratch. As decreases to 0, the coefficient of e, diminishes. This complies with the intuition that e, is gradually eliminated.
5

---

Published as a conference paper at ICLR 2025
ADBM Thaining
Adversarial Noise Generation
Vada vI See
ox(1
L
Outpur E6
%0
Classiffer
v1 de Gr
Figure 2: The illustration of ADBM training. The blue block represents the training objective, and the green block represents the extra module for adversarial noise generation. Black arrows denote the computation of Ta blue arrows denote the computation of The used for generating adversarial noise, and red arrows denote the direction of the gradient flow when calculating Eo.
4.2 ADVERSARIAL NOISE GENERATION
We now proceed to the generation of adversarial noise required for ADBM training. A straightfor- ward method for generating adversarial noise can be maximizing the loss L, of ADBM. However, the idea of deriving adversarial noise directly from the diffusion model itself might not align well with the objectives of the purification task. In this context, the primary goal is to ensure the classi- fier's accuracy on images after they have been purified.
To better align with this goal, we propose to generate adversarial noise with the help of the classifier.
During ADBM training, we input Xn into the classifier, where & = /,. (xy v1 - ae 1o (x( t)). The classification loss Tee is then given by Le(So, (Kn). where y denotes the category label of x and fo, denotes the classifier. Finally, we compute Se, illustrates this process with red arrows indicating the gradient flow direction for maximizing Le- The process of first maximizing T.e then minimizing I., shares a similar procedure with adversarial training, but we note that: 1) The parameters of the classifier should be kept fixed throughout this process, ensuring that ADBM retains the plug-and-play functionality (without modifying the model to be protected) of AP methods; 2) The introduction of e, should align with Eq. ( to effectively address the trade-off in DiffPure between noise removal and data recovery. In addition, although Lin jet al. (2024) designed an impressive AT method named AToP for various AP methods, they claim that AToP ineffective for diffusion models, highlighting the challenges of enhancing diffusion- based purification models.
ôL. obtain C, that maximizes Le- Figure
Any gradient-based attack method generally can be used for maximizing T.e, but in practice some particular proxy attack should be specified. Based on previous experiences, the PGD attack is a popular proxy that has a good generalization ability against various attacks during inference (Athalye et al. 2018 Croce& & Hein 2020 Madry et 2018), and thus we also used such proxy attack in the ADBM training. Our experiments also validate that this proxy can generalize other attacks such as AutoAttack. C&W (Carlimi& & Wagner 2017), and DeepFool (Moosavi-Dezleee Note that during the training of ADBM, both and e in Eq. (9) are randomly sampled for each optimization step. This introduces a significant increase in computational cost when maximizing T with PGD (EOT should used). To address this issue, we propose an alternative method: instead of random sampling, we eliminate the randomness within specific optimization step. In other words, for a given optimization step, we sample t and e once and then fix their values throughout the entire training step (including generating c, using methods like PGD and optimizing cg). The insight behind our approach is that maintaining randomness makes the maximization of Le via PGD challenging without EOT, and eliminating the randomness within a specific optimization step avoids the time-consuming EOT computation and ensures the calculation of adversarial noise that is truly effective for ADBM training. Note that we only eliminate randomness in the ADBM training. During inference, the randomness is kept, as discussed next.
et al. (2016) attacks.
4.3
AP INFERENCE OF ADBM
When using ADBM for AP, both the forward and reverse processes remain unchanged from the original pipeline of DiffPure, as shown in Figure aa). Any reverse samplers developed for diffusion

---

Published as a conference paper at ICLR 2025
models can be directly applied to the AP inference of ADBM without any modification, ADBM only initiates the reverse process from a different starting point compared to traditional diffusion models. Therefore, to improve the practicality of ADBM, we can leverage fast sampling methods such as DDIM to accelerate the reverse process. As demonstrated in Sec. 3 the DDIM sampler efficiently conducts AP, even with single reverse step.
4.4 THEORETICAL ANALYSIS
We provide two theorems to show the superiority of ADBM for adversarial purification.
Theorem 1. Given an adversarial example x6 and assuming the training loss Ls S 6, the distance between the purified example of ADBM and the clean example Xg, denoted || ||, bounded by ô (constant omitted) in expectation when using one-step DDIM sampler. Specifically, we have 1I ò, where (I-ar)T is the constant.
Eg 81000 -xol S
oT
Proof. Please see the full proof in Appendix
Theorem implies that if the training loss of ADBM converges to zero, it can perfectly remove adversarial noises by employing a one-step DDIM sampler. While for DiffPure, we cannot derive such strong theoretical guarantee (The bound provided in Theorem 3.2 of Nie et (2022) is larger than leall || and thus cannot be zero). Moreover, the subsequent theorem demonstrates the superiority of ADBM over DiffPure.
Theorem 2. Denote the probability of reversing the adversarial example to the clean example using
ADBM and DiffPure as P(B) and P(D), respectively. Then P(-) = 1 (xeeD., ).(x))(((x__ & )dx,, where Da denotes the set of adversarial examples. If the timestep is infinite, the following inequality holds:
P(B) > P(D),
wherein
|x varx8ll
for P(B): : p(xolx1) exp
(10)
2(1 - ax))
||x vorxoll
for P(D): : :(xo) X2)) x cxp
(11)
2(1 - (x))
are derived using Bayes' rule, where p(xo & p(xo)p((x(xx) From the perspective of SDE, if timestep is infinite, {x and {xt h-T-0
Proof. (sketch) Eq. 10 and Eq.
follow the same distribution (Song al. 2021). And given that k, < Va, for any 1 < l < T. the inequality P(B)= > P(D) always holds. Please see the full proof in Appendix C.3
Theorem indicates that with infinite reverse timesteps, adversarial examples purified with ADBM are more likely to align with the clean data distribution than those with DiffPure.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS
Datasets and network architectures. We conducted comprehensive experiments on popular datasets, including SVHN (Netzer et al. 2011). CIFAR-10 (Krizhevsky et al. 2009) and Tiny- ImageNet (Le Yang 2015), together with large-scale dataset ImageNet- 100 All these datasets consist of RGB images, whose resolution is 32 x 32 for SVHN and CIFAR-10, 64 64 for Tiny- ImageNet, and 224 x 224 for ImageNet-100. We adopted the widely used WideResNet-28-- 10 (WRN-28-10), WRN-70-16,, WRN-28-10. and ResNet-50 (He et al.) 2016) architectures as clas- sifiers on SVHN, CIFAR-10, 10, Tmay-Ima,,Net, and ImageNet-100, respectively. As for the diffusion
Ihtpp:: / www. kaggle. com httm://aasess//eee heaitss/mmiitttt http:///bbbyyaaaamgeeeeeeetteeee 100 We cannot afford to con- duct experiments on the full ImageNet-1 K (Russakovsky et al. 2015). As alternative, we used ImageNet- 100, curated subset of ImageNet-1 IK featuring 100 categories with the original resolution of ImageNet-1K
7

---

Published as a conference paper at ICLR 2025
models, employed the UNet architecture (Ronneberger et al., 2013) improved by Song et al. (202), specifically, the DDPM++ cont inuous variant. Pre-trained diffusion checkpoints are re- quired for DiffPure. We directly used the checkpoint provided by [Sogg et al. (2021) for CIFAR-10 and we used their code to train the checkpoints for other datasets. These trained checkpoints were used in DiffPure and served baselines for ADBM.
Fine-tuning settings of ADBM. The adversarial noise was computed in the popular norm-ball set- ting ||E, 8/255. When computing Ea, we used PGD with three iteration steps and step size of 8/255, Other settings followed the standard configuration used in Song et al. (2021) The fine- tuning steps were set to 30K, which is about 1/10 the training steps the original diffusion models.
In each fine-tuning step. the value of T in Eq. (9) was uniformly sampled from 100 to 200. Note that when fine-tuning the diffusion models, the parameters of the classifier were kept frozen. Additional settings are provided in Appendix D.1
Defense configurations of ADBM. Unless otherwise specified, the forward diffusion steps were set to be 100 for SVHN and CIFAR-10 and 150 for Tiny-ImageNet and ImageNet-100, respectively. The reverse sampling steps were set to be five. The reverse process used a DDIM sampler. These configurations were also applied to DiffPure for fair comparison.
Table 3: Accuracies (%) of methods under different adaptive attack threats on CIFAR- 10. Average denotes the average accuracies under three attack threats. Vanilla denotes the vanilla model without any defense mechanism. The best results in each column for robust accuracy are highlighted.
Robust Acc
Architecture
Method
Type Cleun Ave
Tac norm
Ii norm
la norm
Average
WRN-70016
Vanilla
97.02
0.00
0.00
0.00
0.00
WRN-70-16
Gowal et al. 2020
91.10
65.92
8.26
27.56
33.91
WRN-70-16
Rebuth el al
2021
88.54
64.26
12.06
32.29
36.20
AT
WRN-70-16
Augment W7 1iill (Gowal al.
2021
88.74
66.18
9.76
28.73
34.89
WRN-70-16
Augment w/ Diff (Wang et
2023
93.25
70.72
8.48
28.98
36.06
MLP-WRN-28-10
Shi et a 2021
91.89
4.56
8.68
7.25
6,83
UNct-WRN-70-16
Yoon ci a. 2021
87.93
37.65
36.87
57.81
44.11
UNeL-WRN-70-16
DilfPure [Wing et
2022
93.16
22.07
28.71
35.74
28.8
UNct-WRN-70-16
Diff-ScoreOpt (Zhang
2024
AP
91.41
13.28
10.94
28.91
17.71
UNtt-WRN-70-16
DiffPure-Langevin ZZhang al
2123
92.18
43.75
39.84
55.47
46.35
UNc--WRN-70-16
DiffPure (Nie ct 2022
92.5 = 0.5
42.2
2.1
44.3
+ 1.3
60.8
+ 2.3
49.1
1.7
UNN--WRN-70-1
ADBM (Ours)
91.9 + 0.8
47.7
2.2
49.6
+
2.2
63.3 - 1.9
53.5
+ 2.1
5.2 ROBUSTNESS AGAINST WHITE-BOX ADAPTIVE ATTACKS
We first evaluate ADBM against the reliable while-box adaptive attacks (Athalye et al. 2018) to show the worst-case adversarial robustness where the attacker has complete knowledge. Note that expect a defense method to be robust not only on threats but also on unseen attack threats. Thus, unless otherwise specified, we evaluated the models on three attack threats: L.., 4, and la. with the bounds 6 8/255, é1 12, e2 1, respectively. Here loc attack is considered the threat as ADBM was trained with loc adversarial noise, while 4 and le attacks can be regarded as unseen threats.
We compared ADBM with SOTA AT and AP methods. All AT models were trained also with loc adversarial examples only, ensuring l and le threats were unseen for these models. We used Au- toAttack (Croce & Hein 2020), which is recognized as the strongest attacks for AT methods, to implement the adversarial threats for AT methods, while we used the attack practice described in Sec. 3 (PGD with 200 iteration steps and 20 EOT samples) to implement the adversarial threats for AP methods. Additional configurations of these attacks can be found in Appendix D.2 Note that this evaluation comparison is justified since Lee & Kim (2023) observed that AutoAttack with EOT yielded inferior attack performance compared to PGD with EOT for stochastic pre- processing de- fenses. Our experiments in Appendix E.T further confirmed that PGD with EOT is the best practice than other attacks such as AutoAttack, C&W, and DeepFool with EOT.
The accuracies under the reliable white-box adaptive attacks CIFAR-10 are shown in Tab. 3 In
this evaluation, we compared ADBM with several competitive methods. Both Gowal et al. 202T and Wang al. 2023) performed AT with about 50M generated images by diffusion models. Addi-
8

---

Published as a conference paper at ICLR 2025
Table 4: Accuracies (%) of methods under three query-based attacks and the cansfer-based attack on SVHN. Average denotes the average accuracies under four attacks. All attacks are performed with the se bound 8/255.
Robust Ace
Architecture
Method
Type
Clean Acc
RayS
Square
SPSA
Transfer
Average
WRN-28-10
Vanilla
98.11
16.89
9.08
13.48
WRN-28-10
RRde et al.
2022
94.46
68.90
58.96
74.22
87.60
76.83
WRN-28-10
Yang et al.
2023
AT
93.00
62.30
60.49
71.78
87.60
75.53
WRN-28-10
Wang
2023
95.56
75.16
67.23
80.39
88.57
81.38
UNet+WRN-28-
DuftPure (Nic ct al.
[0022)
93.98
92.97
92.15
92.19
91.49
92.20
AP
UNN--WRN-28-10
ADBM (Ours)
93.49
93.16
93.32
93.49
92.88
93.21
tionally, compared ADBM with several recent AP methods using generative models, especially diffusion models s By inspecting these results, we can see that despite training with millions of examples, AT methods still exhibited limited robustness against unseen 1 and l2 attacks. In con- trast, ADBM demonstrates a strong defense against I and l2 attacks. Moreover, attempts to enhance the performance of DiffPure by introducing input guidance (DiffPure+Guide, [Wang et al. (2022)) or applying Langevin dynamics (DiffPur+LLaneviin Zhang et al. (2023) occasionally resulted in detrimental effects when assessed through reliable adaptive attack evaluations. Notably, ADBM outperformed DiffPure by achieving average robustness gain of 4.4% on CIFAR-10, 10, while the clean accuracies kept comparable. Similar outcomes were observed for SVHN, Tiny- Inteee,,,,ee, and ImageNet-100, detailed in Appendix E.2
reinforcing similar findings from the CIFAR-10 analyses.
5.3 ROBUSTNESS AGAINST BLACK- BLA ATTACKS
We then considered the more realistic black-box attacks, where the attacker has no knowledge about the defense mechanism, i.e., the purification model, and cannot access the gradients of models. Instead, the attacker can only query the model's output with query-based attacks or use substitute models with brasefer-based attacks.
On the seen threat (i.e. Loo threat). we conducted three query-based attacks: RayS (Chen & Gu 2020), Square (Andriushchenko et al., (200), and SPSA (Uesato et al., 2018)) attacks. Square and RayS are efficient bearch-based attack methods, while SPSA approximates gradients in black- box manner. For Square and RayS, we utilized 5,000 search steps. For SPSA, we set a to 0.001, the number of random samples in each step to 128, and the iteration step to 40. In addition, We implemented a transfer-based attack by generating adversarial examples via the vanilla models. The results under black-box attacks SVHN and CIFAR-10 10 are shown in Tab. 4 and Appendix EE.3 respectively. We can see that these black- box attacks achieved excellent attack performance on the vanilla model. For the SOTA AT method SVHN, these attacks lowered the average accuracy to 81.38%. But surprisingly, all these black-box attacks can hardly lower the accuracies of ADBM. Thus, we can conclude that besides the promising results on white-box attacks, under the realistic black-box attacks, ADBM has advantages over AT models even on the seen threat.
We further conducted experiments on more unseen threats beyond norm-bounded attacks, including patch-like attacks (Gao et al. 2020; Yuan et al. 2022) and recent diffusion fiffusion-bbsed attacks (Chen al., 2023bb] Xue et al.) [2023). ) to evaluate the generalization ability of ADBM. The results shown in Appendix E..fiinicate that ADBM demonstrates better generalization ability than DiffPure and AT methods against these black-box unseen threats.
5.4 ABLATION STUDY
The ablation studies were performed on CIFAR-10. The evaluation followed the setting in Sec. 5.2
Reverse sampling steps. We first investigated the influence of reverse sampling steps on the adver- sarial robustness of ADBM. The number of reverse steps is proportional to the inference cost. We used five reverse steps in the main experiments. Here we evaluated the robustness of ADBM with
RDC (Chen et al. 2023a) was not compared as it required thousands of times the inference cost of DiffPure that cannot afford conduct experiments in our setting. Instead, we discuss in Appendix A.2

---

Published as a conference paper at ICLR 2025
Table 5: Accuracies (%) of methods under dif- ferent adaptive attack threats on C1. The same conventions are used as in Tab. A4
Table 6: Accuracies (%) of ADBM with var- ious adversarial noise generating modes CIFAR-10. Cls indicates whether or not to employ the classifier when generating adver- sarial noise for training.
Reverse Step
Robust Acc
Method
Clean
Loo
l2
Average
5
92.5
42.2
44.3
60.8
49.1
Fixing Fixing x Cls
Loc
l2
Average
DiffPure
92.3
42.7
44.5
60.9
49.4
44.0
44.9
61.4
50.1
1
92.3
43.0
45.2
59.6
49.3
44.3
46.2
60.2
50.2
5
91.9
47.7
49.6
63.3
53.5
43.7
45.6
60.3
50.2
ADBM
91.9
47.7
49.2
63.2
53.3
4
4
47.7
49.6
63.3
53.5
91.5
45.7
50.7
61.9
52.8
fewer steps to investigate whether the number of steps can be further reduced. The results in Tab. 5 show ADBM was more robust than DiffPure regardless of reverse steps. Notably, even with just one reverse step, ADBM maintained its good robustness, We discuss the detailed inference cost related to this further in Sec.
Adversarial noise generation modes. We then analyzed the impact of different adversarial noise generation modes for ADBM. As discussed in Sec. [.2] adversarial noise used for training ADBM can be generated by various modes. We analyzed the contributions of our three key design choices in the generation modes: using the classifier, fixing and fixing x. The results shown in Tab. 6 clearly demonstrate that all of these designs are essential for the success of ADBM. Removing any of these designs hampers the proper computation of adversarial noise for ADBM training.
Effectiveness on new classifiers. To assess the effectiveness of ADBM new classifiers, conducted study to investigate its transferability. Specifically, we utilized the fine-tuned ADBM checkpoint, trained with adversarial noise from WRN-70-16 classifier, the pre-processor for WRN-28-10 model and a vision transformer model (Dosovitskiy et al.. 2021) directly, denoted as ADBM (Transfer). The results shown in (ppendii(EE_jdemoossss. robust accuracies comparable to ADBM directly trained with corresponding classifiers. This finding highlights the practicality of ADBM, as the fine-tuned ADBM model on specific classifier can potentially be directly applied to new classifier without requiring retraining. We guess this may be attributed to the transferability of adversarial noise (Dong al. 2018).
that ADBM (Transfer) achieved
CONCLUSION AND DISCUSSION
In this work, we introduce ADBM, cutting-edge method for diffusion-based adversarial purifica- tion. Theoretical analysis supports the superiority of ADBM enhancing adversarial robustness. With extensive experiments, demonstrated the effectiveness of ADBM across various scenar- ios using reliable adaptive attacks. Notably, ADBM demonstrates significant improvements over prevalent AT methods against unseen threats black-box attacks. In the of foundation models [Bommasani et al.] 2021), training foundation models with AT becomes increasingly challenging due high computational costs. ADBM, on the other hand, provides promising alternative plug-and-play component that can enhance the robustness of existing foundation models without the burdensome of retraining, In addition, our discussion the acceleration of diffusion-based purifica- tion methods may unlock the potential of diffusion- based purification methods in various real-tin -ime applications (see Appendix(F). The social impact of this work is further discussed in Appendix G
Limitation. 1) While ADBM requires quite few reverse steps, it does introduce additional compu- tational parameters due to its reliance on diffusion models compared with AT methods. The current UNet architectures, tailored primarily for generative purposes, are bit large (see Appendix F). Exploring the potential of downsizing these architectures for AP remains an open area. 2) While ADBM requires only about 1/10 training steps of original diffusion models, it does introduce addi- tional fine-tuning cost compared with using the pre-trained diffusion models directly (DiffPure). But note that whether the slight extra training cost is worthwhile depends on the specific problems being addressed. In many security-critica cases, where the training cost is not the primary concern, ADBM offers alternative approach to further enhance robustness with affordable training costs.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was supported by the National Natural Science Foundation of China (No. U2341228).
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: A query- efficient black- box adversarial attack via random search. Eur. Conf. Comput. Vis. (ECCV), volume 12368, pp. 484-501, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Int. Conf. Mach. Learn. (ICML.,, volume 80, pp. 274- 283, 2018.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al. On the opportunities and risks of foundation models. arXiv preprint 200iv:2108.00 07258, 2021.
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with ap- plications score-hased generative modeling. In Adv. Neural Inform. Process. Syst. (NeurIPS), pp. 17695- 17709, 2021.
Nicholas Carlini and David Wagner. Towards evaluating the robustness neural networks, IEEE Symposium on Security and Privacy (S & P). pp. 39-57, 2017.
Nicholas Carlini, Florian Tramèr, Krishnamurthy (Dj) Dvijotham, Leslie Rice, Mingjie Sun, and J. Zico Kolter. (certified!!) adversarial robustness for free! In Int. Conf. Learn. Rep. (ICLR), 2023.
Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang. Chengqi Duan, Hang Su, and Jun Zhu. Robust classification via single diffusion model. arXiv preprint arXiv:2305. 15241,, 2023a.
Jianqi Chen, Hao Chen, Keyan Chen, Yilan Zhang, Zhengxia Zou, and Zhenwei Shi. Diffusion models for imperceptible and transferable adversarial attack. arXiv preprint 20X502335500879, 2023b.
Jinghui Chen and Quanquan Gu. Rays: A ray scarching method for hard-label adversarial attack. In KDD, pp. 1739-1747. ACM, 2020.
Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.0 06174, 2016.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Int. Conf. Mach. Learn. (ICML) pp. 1310-133 1320-, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Jnt. Conf. Mach. Learn. (ICML), volume 119, pp. 2206-2216, 2020.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal A light recipe to train robust vision transformers. In IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). pp. 225-252 253, 2023.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting ad- versarial attacks with momentum. In IEEE Conf. Comput. Vis. Pattern Recog, (CVPR), pp. 9185-9193, 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenhorn al. An image is worth 16x16 words: Transformers for image recognition scale. In Int. Conf. Learn. Rep. (ICLR), 2021.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Ta- dayoshi Kohno, and Dawn Song. Robust physical-worke attacks on deep learning visual classification. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 1625-1634, 2018.
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for faoling deep neural network. In Eur. Conf. Comput. Vis. (ECCV), volume 12373, pp. 307-322, 2020.
Yue Gao, Ilia Shumailov, Kassem Fawaz, and Nicolas Papernot. On the limitations of stochastic pre- processing defenses. In Adv. Neural Inform. Process. Syst. (NeurIPS), 2022.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In fnt. Conf. Learn Rep. (ICLR), 2015.
11

---

Published as a conference paper at ICLR 2025
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy A. Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm- comm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Sven Gowal, Sylvestre-Alviss Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A.
Mann. Improving robustness using generated data. Adv. Neural Inform. Process. Syst. (NeurlPS), pp- 4218-4233, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conf. Comput. Vis. Puttern Recog. (CVPR), pp. 770-778, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Adv. Neural Inform. Process. Syst. (NeurIPS), 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Int. Conf. Learn. Rep. (ICLR), 2015.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. Califorria Institute of Technology, 2009.
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. In Int. Conf. Learn. Rep. (ICLR), 2021.
Ya Le and Xuan Yang. Tiny imagine visual recognition challenge. CS 23/N, Stanford University, 7(7):3, 2015.
Minjong Lee and Dongwoo Kim. Robust evaluation of diffusion- based adversarial purification. In Int. Conf. Comput. Vis. (ICCV), pp. 134-144, 2023.
Xiao Li, Qiongxiu Li, Zhanhao Hu, and Xiaolin Hu. On the privacy effect of data enhancement via the lens of memorization, IEEE Transactions Information Forensics and Security, 2024a.
Xiao Li. Yining Liu, Na Dong, Sitian Qin, and Xiaolin Hu. Partimagenet++ + dataset: Scaling up part-based models for robust recognition. In Eur. Conf. Comput. Vis. (ECCV), pp. 396-414, 2024b.
Xiao Wei Zhang, Yining Liu, Zhanhao Hu, Bo Zhang, and Xiaolin Hu. Language-drinn driven anchors for zero- shot adversarial robustness. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 24686 24686-24695, 2024c.
Xiao Li, Hang Chen, and Xiaolin Hu. On the importance of backbone to the adversarial robustness of object detectors. IEEE Transactions on Information Forensics and Security, 2025.
Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. In Int. Conf. Mach. Learn. (ICMI.), volume 202, pp. 20763-20786, 2023.
Guang Lin, Chao Li, Jianhai Zhang, Toshihisa Tanaka, and Qibin Zhao. Adversarial training purification Conf. Learn. Rep. (ICLR), 2024.
(atop): Advancing both robustness and generalization.
Chang Liu, Yinpeng Dong, Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, and Shibao Zheng. A comprehensive study on robustness of image classification models: Benchmarking and rethinking. arXiv preprint arXiv:2302. 11022.1430 14301, 2023a.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In Int. Conf. Learn. Rep. (ICLR), 2023b.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In Int. Conf. Learn. Rep. (ICLR), 2018
Seyed-Mohsen Moosavi-Dezfoon Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. IEEE Conf. Comput. Vis. Pattern Recog, (CVPR), pp. 2574-2582, 2016.
Yuval Netzer, Tao Wang. Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. NewrIPS Workshop, 2011.
Weill Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anandkumar. Diffusion models for adversarial purification. In Int. Conf. Mach. Learn. (ICML), volume 162, pp. 16805 16827, 2022.
Rahul Rade et al. Reducing excessive margin achieve better accuracy robustness tradc-off. In Int. Conf.
Leam. Rep. (ICLR), 2022.
12

---

Published as a conference paper at ICLR 2025
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A. Mann. Data augmentation can improve robustness, Adv. Neural Inform. Process. Syst. (NeurlPS), pp. 29934, 2021.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAL, pp. 234-241, 2015.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Sathcesh, Sean Ma, et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. (IJV),, (15())2111-2522 2015.
Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-supervised learn- ing. Int. Conf. Learn. Rep. (ICLR), 2021.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Int. Conf. Learn. Rep. (ICLR), 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-hased generative modeling through stochastic differential equations. In Int. Conf. Learn. Rep. (ICLR), 2021.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and generalization. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 6976 6987, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Int. Conf. Learn. Rep. (ICLR), 2014.
Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Ady. Neural Inform. Process, Syst. (NeurlPS), 2020.
Jonathan Uesato, Brendan O'Donoghue, Pushmeet Kohli, and Aäron van den Oord. Adversarial risk and the dangers of evaluating against weak attacks. In Int. Conf. Mach. Learn (ICML), volume 80, pp. 5032-5041, 2018
Jinyi Wang, Zhaoyang Lyu, Dahua Lin, Bo Dai, and Hongfei Fu. Guided diffusion model for adversarial purification. arXiv preprint urXiv:2205. 14969, 2022.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In Int. Conf. Mach. Learn. (ICML), volume 202, pp. 36246-36263, 2023.
Chaowei Xiao, Zhongzhu Chen. Kun Jin, Jiongxiao Wang, Weili Nie, Mingyan Liu, Anima Anandkumar, Bo Li, and Dawn Song. Densepure: Understanding diffusion models for adversarial robustness. In Int. Conf. Learn Rep. (ICLR), 2023.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang. Alan L Yuille, and Quoe V Le. Adversarial examples improve image recognition. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 819 828, 2020.
Haotian Xue, Alexandre Araujo, Bin Hu, and Yongxin Chen. Diffusion-based adversarial sample generation for improved stcalthiness and controllability. Adv. Neural Inform. Process. Syst. (NeurIPS), 2023.
Dongyoon Yang, Insung Kong, and Yongdai Kim. Improving adversarial robustness by putting more regular- izations on less robust samples. In Int. Conf. Mach. Learn. (ICML), volume 202, pp. 39331-39229 39348, 2023,
Jongmin Yoon, Sung Ju Hwang, and Juho Lee. Adversarial purification with basre-based generative models. In Int. Conf. Mach. Learn. (ICML), volume 139, pp. 12062-12072, 2021.
Shengming Yuan, Qilong Zhang, Lianli Gao, Yaya Cheng, and Jingkuan Song. Natural color fool: Towards boosting black-box unrestricted attacks. In Adv. Neural Inform. Process. Syst. (NeurIPS), 2022.
Boya Zhang, Weijian Luo, and Zhihua Zhang. Purify++: Improving diffusion-purificatti with advanced diffusion models and control of randomness. arXiv preprint arXiv:2310. 18762, 2023.
Boya Zhang, Weijian Luo, and Zhihua Zhang. Enhancing adversarial robustness via based optimization. Adv. Neural Inform. Process. Syst. (NeurIPS), 36. 2024.
Huan Zhang, Hongge Chen, Zhao Song, Duane S. Boning, Inderjit S. Dhillon, and Cho-Jui Hsieh. The limita- tions of adversarial training and the blind-spot attack. In Int. Conf. Learn. Rep. (ICLR), 2019.
Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, and Jun Zhu. Diffusion bridge implicit models. arXiv preprins arXiv:2405.15888 2024.
Linqi Zhou, Aaron Lou, Samar Khanna, and Stefano Ermon. Denoising diffusion bridge models. In Int. Conf. Learn. Rep. (ICLR), 2024.
13

---

Published as a conference paper at ICLR 2025
A DISCUSSIONS ON DIFFUSION BRIDGES AND DIFFUSION CLASSIHIERS
A.1 DIFFUSION BRIDGE MODELS
Bortoli et al. (0221) is fundamentally unsuitable for the AP task because it bridges the data dis- tribution with a simple prior distribution, while Zhou et al. (2024); ): [Bortoli et al. (2021): Liu et al. (023b) tend to training collapse for the AP task. Here provide the theoretical explanation: Ad- versarial examples typically reside close to clean examples, particularly in the commonly analyzed -buunded threat settings. This proximity poses significant challenge to applying these bridge models (Zhou et al., 2024] Bortoli et al. 202T Liu et al., 2023b) for AP, as these bridge models' merely predicting the input adversarial image could result in extremely small training loss, finally
incurring a degenerated solution and losing the capability to purify the adversarial perturbations.
Different from these bridge models, ADBM first maps the adversarial data distribution diffused (Gaussian) adversarial data distribution and then creates bridge from the diffused adversarial data distribution to the clean data distribution. This distinct from existing diffusion bridges, effectively solving the problems that previous diffusion bridges face in purification. Our theoretical analysis in Sec. 4.4 consolidates that ADBM guarantees superior performances for AP.
A.2 ROBUST DIFFUSION CLASSIFIESS
Chen al. (2023a) show that a single diffusion model can be transformed into an adversarially robust diffusion classifier (RDC) using Bayes' rule, Although RDC does not require an additional classifier, ADBM has two unique and significant advantages over RDC:
1) ADBM an adversarial purification method, which can function as a preprocessing module in a and-play play manner without retraining the classifiers. This is quite flexible and useful way,
plug-an and pecially in the era of foundation models. For example, we may use ADBM as a preprocessing step to purify the visual inputs of large vision-language model (VLM), without retraining the VLM with adversarial training (huge computational costs are saved). However, RDC does not follow this paradigm, it robust classifier instead of robust pre-processor. RDC is limited to generalize to tasks beyond robust classification, hindering its practical usefulness. Overall, ADBM offers a dis- tinct advantage over RDC by decoupling preprocessing and the specific classification task, allowing each to be optimized independently.
2) While diffusion- based purification requires slightly more parameters (an additional classifier), ADBM requires significantly lower inference costs than RDC. Our following experiments indicate that, even with one reverse step, ADBM can maintain substantial robustness under reliable evalu- ation. In contrast, RDC requires N + T reverse steps, where T is the diffusion steps (e.g., 1000 in DDPM) and is the number of likelihood maximization steps. The inference cost proportional to the reverse steps, and thus the inference cost of RDC is several hundred times that of ADBM.
B EVALUATIONS ON DIFFPURE
B. PREVIOUS EVALUATION METHODS ON DIFFPURE
As the original implementation of DiffPure needs dozens of prediction steps in the reverse process, it is challenging to compute the full gradient of the whole purification process due to memory con- straints. To circumvent computing the full gradient, Nie al. (2022) originally employed the adjoint method (along with Expectation over Transformation (EOT) (Athalye et al.) (018)) to compute an approximate gradient. But recent works have identified to be flawed adaptive attack (Lee & Kim
2023 Chen et al. 2023a). As improved adaptive mechanisms, [ee & Kim 2023) proposed sur-
rogate attack for DiffPure (to approximate the gradient of the iterative reverse procedure better) and
Chen et al. (2023a) utilized the suudient-checkpointiig technique (Chen et al.) 2016) to trade time for memory space and compute the full gradient directly. Despite previous efforts, we find that the adaptive evaluations for DiffPure remained insufficient. Specifically, the surrogate attack failed to compute the full gradient, and the sufficient iteration steps or EOT samples, an issue underscored by Gao et al. (2022) which highlights the importance of adequate steps and samples for evaluating stochastic pre- proceeccssing defenses like DiffPure
attack in Chen et (202a)) employed in-
14

---

Published as a conference paper at ICLR 2025
Table Al: Accuracies of DiffPure on CIFAR-I0 with reverse step varying forward steps, and
using DDIM sampler. The robust accuracy was evaluated under an lo bound of 6o0 = 8/255.
Forward Step
100
110
120
130
140
150
Clean Ace
92.38
91.21
91.01
89.84
89.45
87.89
Robust Acc
42.16
43.75
44.53
47.85
48.43
49.02
B.2 THE COST OF OUR EVALUATION
In our evaluation, we used the full gradient of the whole reverse process and set the PGD iteration steps 200, with 20 EOT samples for each iteration. The cost of the evaluation is quite high, especially in the context of the high reverse steps of the original DiffPure. To give concrete example, for a single input image, the noise prediction model (i.e., Eq) in the original DiftPure implementation with 100 reverse steps needs to be queried total of 400, 000 times.
However, we think that such high efforts are worthwhile as an unreliable evaluation could create false sense of security on defenses, Historical evidence has shown that many defenses initially con- sidered robust, were subsequently breached by more sophisticated and dedicated attacks (Athalye et al. 2018 Tramèr et al. (2020). Our aim is to prevent a similar outcome for diffusion-based purifi cation, advocating for the employment of meticulous and reliable attack evaluation methodology, regardless of the expense involved.
the other hand, with our reliable evaluation, we have investigated the influence of forward steps and reverse steps on the robustness of diffusion-based purification. Our following results in Tab, 2 indicate that reverse steps will not significantly influence the robustness of DiffPure under the re- liable evaluation. This suggests that diffusion- purification techniques might not benefit from increasing the number of reverse steps to complicate the attack process, as such strategies could finally be neutralized by high-cost attacks similar to the reverse steps are reduced (e.g., five steps in our main experiments), the attack cost of our evaluation method is comparable to the most widely used AutoAAtack benchmark (Croce & Hein 2020
have implemented. And when the
B.3 ADDITIONAL. INVESTIGATION ON DIFFPURE WITH OUR EVALUATION
As shown in Tab. we find that the reverse step does not significantly affect the robustness when using the DDIM sampler. Considering the computational cost of attacks, we fixed the reverse step to with the DDIM sampler and focused on investigating the influence of forward steps on robustness. The results are shown in Tab. A1 We observed a continuous decrease in clean accuracy and continuous increase in robustness as the number of forward steps increased. This can be attributed to the introduction of more noise during the forward process.
C DERIVATION OF EQUATIONS AND PROOFS OF THEOREMS
C.I THE DERIVATION OF EQ.
Proof. Following the Bayes' rule, we have
xxx_11xx,, xo) ((x/ x 1, Xo) q(x (xo)
(12)
q(x xx)
Since x =x;- kec, and x² Varx + VI ā,e, there is
x VaxXo + v1 āGE + (Vor ke)ea-
(13)
15

---

Published as a conference paper at ICLR 2025
Based on the property of Gaussian distribution, p(x [x]] xo) also must be Gaussian distribution, thus,
q 1// xo) = q(x; xx x(x11x0-1, xx,, q(x;- 1x00)
x exp( 1/2
1 (xf vaxxi_1 (Vorks 1-kkt- k))))a)
((x] (xo)
2
1 -
(x/_1= - v 1x0 (var-1 1a
(x] V 0x800 (Vāa - -kejea)?
1 01-1
0tt
1 (Ax2 2² Bxx=1 + C(xo, cax
eex
(14)
where
01
1- 8z
A =
1 1-0t
0--1
(1 - (1-a,)( Ō(-1))
(15)
x (Varke-- ke) 2 van-ix0 + (var-1 k1-1)6a
B = -2/ar
Cit
In inference, we expect that cg in Eq. (14 can be eliminated since only x6 is given by attackers and C, cannot be decoupled from x3 directly. Based the property of Gaussian distribution, eliminating all terms related to e, in B and C in Eq. (5) can be achieved by eliminating all terms related to en
in B. This yields:
/a Vapk,
k2)E, (Vor-1 12- 0.
(16)
at
04-1
Although Eq. 16 cannot be directly solved, we can derive its recurrent form. Letus v Vāti.. where 0 5 t S T and To = Pr O. The Eq. (T6) can deduce to
Vai_ (Värn- - vārt) = vāi- Voa-17
(17)
1 -tt-1
Then group the items according timestep (t and t 1):
1
a2
' Oz
(18)
a
a2-1
Now we have a recurrent equation about %e- The equivalence holds for all 0 < t < T. Therefore, by elucidating the relationship between 7 and the initial value Yi, can deduce expression for each 7t. If we reorganize items in Eq. (18). it yields recurrent equation:
1 ãt (M--1 1)
Yt 1
a
āe
1
(n1 - 1)
(19)
1=
ā (1 ā.)
(71 - 1).
ā,(1 (1)
Since "r 0, we have the expression of "1 when t T:
TT 1= ān (1 âT)
(71 1),
āT (1 - ā1)
(20)
ār(( ā1)
71 ān (1 ④T)
and " generated from 7/1 is
2t ār(1 āz)
(21)
ā, âT)
Recalling that k; = Vorate we thus have
ār(1 σ4)
ky = ===vany = Vaz-
T
(22)
Vār(1 aT)
16

---

Published as a conference paper at ICLR 2025
C.2 THE PROOF OF THEOREM
Theorem 1. Given an adversarial example x6 and assuming the training loss Ls S 6, the distance between the purified example of ADBM and the clean example Xo, denoted || I bounded by d in expectation (constant omitted) when using a one-step DDIM sampler. Specifically, we have
Bell - -xol (1-ār)]T 8, where [1-ār)]T is the constant.
ar
Proof. This inequality holds when
the DDIM reverse sampler and set the reverse step According to Song et al. (2020), the reverse process of DDIM is
1.
VT Or, co (8;;
x VōTL-I
VV- a._,, co (X7, Tz).
(23)
Van.
In the discrete case (Ho et al. 2020 (ro.....TT)
a linearly increasing sub-sequence of {O..... T. 0, = T. In the continuous case (Song et al. 2021), (To..... J is a lin-
early increasing sequence in [0. T] c [0.0, 1.0), To 0,0 S Ts = I I. determined in the forward process. Setting the number of reverse steps = 1, then yields:
T.T.- To = 0, and it
a, Co 8r. VI -V-
col,,, + / ar co(
x 1o..var.... ar.--
co Te),
(24)
Vār,
since Rt, 87 X9,, have
VI Crro + VI a.eo (x7, T),
X =āo
(25)
0TT
where since we one-step reverse process, we reuse the notation of the final point of reverse process &n to represent the prediction of x from x7. Since vag = x9 = vārxa + v1 are, xj = Xo Ca. Eq. 25 can be written as
Sn x
(E (x); (xxx;77) T))
VOT
(26)
VoT
En +e Ex(x,, TT).
X0
VoT
âT
Therefore, the distance between X0 and is
vI äT
αT
|80
xoll
En + 6 - (x
Var
④T
(27)
oT
+ 6 60(xx T)
VāT
where the second equivalence holds due to t T and xy X².
kr ca xG. Considering that
Lb
E,,
a
ā
+
<x(xx] t)
Va
1
/1
āi
Ea + (x(xx () 12
(28)
E.
7
14g
āt
< 8,
thus
Ee
ea
+ c
c/
(x( T)
S
(29)
II
Then we have
1- är
/ Vār
12
Ee
1200
x0
= Ee
+
Eo
(x T)
v1 σr
(30)
(1
är) T
8.
āT
17

---

Published as a conference paper at ICLR 2025
C.3 THE PROOF OF THEOREM
Theorem 2. Denote the probability of reversing the adversarial example to the clean example using ADBM and DiffPure as P(B) and P(D), respectively. Then P()) can be computed as
P()= = (xagD, p((x) & d)x, where Da denotes the set of adversarial examples. If the timestep infinite, the following inequality holds:
P(B) > P(D),
wherein
| / 11?
for P(B) p(xo|x)) x exp
(31)
2(1 σc)
|| /oxexoll
for P(D) p(xo X exp
(32)
2(1 a()
Proof. The concept of infinite timestep can be viewed as dividing finite length time into in- finitesimal intervals, which corresponds to the situation of the following SDEs proposed by Song et al. (202T). Denoting w as the standard Wiener process, w as the reverse-time standard Wiener process, and px() the probability density of x,, the forward process be described by
dx fxx, l)dl + g(()dw,
(33)
where f(x, t) and g(t) denote the drift and diffusion coefficients. respectively. Under mild assump- tions, the reverse process can be derived from:
dx (x, t)dt V log (x))dt g())dw.
(34)
In this context, the reverse of a diffusion process is also diffusion process, running backwards in time and given by the reverse-time SDE (Eq, (34)). Therefore, if timestep infinite, (x00: and {&e - as the solutions of Eq. 33 and Eq. respectively, follow the same distribution. And due to the Bayes' rule,
((xo|x+) p(xo) p(xx xx)
p(xx)
p(xo) x xx)
(35)
x(x))
((x))p(xx(xx))
Since x; = x in ADBM and &; == xf in DiftPure, then for all the examples,
P(B) = 1 (x (.xoen.)p((xll dxxo
x / 1 (x0e (xx p xx )xx
(36)
/ 1 (x exp
|xf VoxX0 (Var ke)eall)) p(xo)dxo
2(1 - ((+)
and
P(D) / (xODD, 1xxxe (xxxxo/xx)css
/ 1 (/xoEDD.DDXXO) (xx
(37)
/ /1(MOOED.) exp
||x varxol? 2(1 - at)
p p(xo) )
where the Db represents the set of adversarial examples. Subtract Eq.
36 by Eq.
37 and we have
- ((B)) x / 1(0g22.1((x))
exp (- || Va,Xo (Vot ke)ca exp
|xx / /a.xol13 dxo-
]
2(1 ā.)
2(1 a
(38)
18

---

Published as a conference paper at ICLR 2025
Therefore, we only need to compare |x] V (vār
ke)eall2 and |x= varxoll3 for
comparing P)B) and P(D).
Since x v vI - agE and xij Xo + Ea.
|x / (Vo,- Ca = 1IVa, + v1 -ā, vo,X0 - (Vār ke)eal
(39)
|| Cge k1Eall.
| Vāxxol || || ||Vār + v1 OgE vārxol
(40)
= 1Iv1 āic + callall-
Since < kt < V always holds, thus in expectation we have:
EellvI āGE + krEall
Eel v āre + Vālcall
(41)
Hence,
|xx
v aax0 (vāi Kz) Ke)Eal1
|x
v x
Eg exp
Ex
exp
(42)
2(1 - -(t)
2(1 (i.)
implying that the probability of ADBM reversing the adversarial example to the clean example is higher than that of DiftPure.
D ADDITIONAL SETTINGS
D.1 ADDITIONAL TRAINING SETTINGS OF ADBM
We implemented ADBM based loosely on the original implementations by Song et al. (202 that when fine-tuning the ADBM with the pre-trained checkpoint, following Song al. (202]
1). Note
also adopted the continuous version of Eq. (9. This version is conceptually similar to its discrete counterpart, with the exception that in Eq. 9 represents a continuous value rather than a discrete one. Moreover, we used the Adam optimizer (Kingma & Ba, 2013) and incorporated the expo- nential moving average of models, with the average rate being 0.999. The batch size was set to 128 for SVHN and CIFAR-10, 112 for Tiny-lmageNet, and 64 for ImageNet-100 (due to memory constraints). All experiments were run using PyTorch 1.12.1 and CUDA 11.3 on 4 NVIDIA 3090 GPUs.
D.2 ADDITIONAL ADAPTIVE ATTACK CONHIGURATIONS
When performing the adaptive attacks (PGD with EOT) for the AP methods, we set the step sizes to be 0.007. 0.5, and 0.005 for loc. (1, and le attacks, respectively. For the L attack, we set the sparsity level as 0.95. As shown in Sec. 3 the adaptive evaluating is quite time-consuming. Thus following Nie et al. (2022), we conducted the adaptive attacks three times on a subset of 512 randomly sampled images from the test dataset and reported the mean accuracy along with the standard deviation. We note that sometimes the standard deviation is omitted to present the results more clearly.
Table A2: Accuracies (%) of DiffPure under two attacks using the full gradient on CIFAR-10 -00
Robust Acc
Attack Method
Clean Acc
norm
norm
l2 norm
Average
PGD EOT (Full Grad.)
42.2 = 2.1
44.3 = 1.3
60.8 + 2.3 63.87
49.1 1.7
92.5 + 0.5
AutoAttack + EOT (Full Grad.)
62.70
53.91
60.16
E ADDITIONAL EVALUATION RESULTS
E. COMPARISON BETWEEN DIFFERENT ATTACK METHODS ON CIFAR-10
Similar to Lee & Kim (023), we first performed a comparison between PGD with 20 EOT samples (full gradient) and AutoAt with the same number of EOT samples (the default setting of rand
19

---

Published as a conference paper at ICLR 2025
Table A3: Accuracies (%) of DiffPure and ADBM under different attacks in the l2 setting using the full gradient on 10.. The strongest attack results for each defense method are highlighted.
Method
C&W + EOT
DeepFool BOT
AutoAttack EOT
PGD + EOT (Our Setting)
DiffPure
74.8
78.3
63.9
60.8
ADBM (Ours)
78.3
84.8
66.8
63.3
Table A4: Accuracies (%) of methods under different white-box adaptive attack threats SVHN.
The same conventions are used as in Tab.
Robust Ace
Architecture
Method
Type Clean Acc
T norm
norm
l2 norm
Average
WRN-28-10
Vanilla
98. 11
0.19
0.02
0.03
0.08
WRN-28-10
Rade et al.
2022
94.46
52.65
0.16
6.76
19.86
WRN-28-10
Yang et al.
AT
93.00
52.70
0.04
3.27
18.67
WRN-28-10
(Wan al.
2023
95.56
64.00
0.14
5.07
23.07
UNe++WRN-285 DiffPure (Nie et al.
2022
93.9
-
39.7
2.2
46.
2.1
63.3
0.8
49.7
1.7
AP
UNer+WRN-28--1
ADBM (Ours)
93.5
0.8
0.8
47.9
1.4
51.2
+ 0.6
65.7
+ 1.5
54.9 + 1.1
Table A5: Accuracies (%) of methods under different adaptive attack threats on same conventions are used as in Tab.
The
Robust Acc
Architecture
Method
Type
Clean Acc
norm
norm
12 norm
Average
WRN-28-10
Vanilla
69.49
0.06
2.09
0.08
0.74
WRN-28-10
(Rade al. 2022
50.94
29.13
20.15
24.54
24.61
WRN-28-10
Yang
2023
AT
50.89
27.02
17.79
23.29
22.70
WRNN28-111
(Wang el 2023
57.59
38.41
10.73
27.21
25.45
UNef=WRN-28-10 DiffPure (Nie al.
2022
58.0
24.8
1.8
44.3
0.3
32.9
1.1
34.0
0.8
AP
UNet=WRN-28-10
ADKM (Ours)
1.2 29.3
1.7
46.0
+
0.4
38.1
37.8 + 0.9
version AutoAttack yet with full gradient). The main difference between AutoAttack and vanilla PGD lies in the automatic adjustment of step size by AutoAttack, but the adjustment algorithm may be influenced by the randomness of stochastic pre-processing defense. The comparison results are presented in Tab. A2 PGD with EOT achieved significantly better attack performance across all threat models.
We additionally performed extra attack evaluation with the full-gradient C&W (Carlini & Wag- ner] 2017) and DeepFool (Moosavi-Dezfooli et al.] (016) attacks, which are optimization-base attacks that differ significantly from PGD. Here their original la setting (Carlini & Wagner] [2017] Moosavi-Dezfooli et al., 2016) were evaluated with 1,000 iteration steps and 20 EOT samples. The implementation used the standard corchattack. benchmark" The robustness results are shown in Tab. A3 (along with the PGD and AutoAttack l2 results in Tab. A2). We can see that PGD with EOT is also significantly stronger than other attacks such C&W and DeepFool, which need siderably more iteration steps. Additionally, ADBM is more robust than DiffPure regardless of the evaluation methods.
Thus, considering that AutoAttack is widely recognized as a reliable attack method for AT models, employed AutoAttack for AT evaluation. Conversely, for AP evaluation, we measured the worst- case robustness of each defense method by employing PGD with EOT in our main experiments.
E.2 ADDITIONAL WHITE-BOX RESULTS SEVERAL DATASETS
The accuracies SVHN shown in Tab. A4 ADBM consistently achieved significantly better adversarial robustness than DiffPure across all adversarial threats. On average, ADBM outperformed DiffPure by 5.2%. In addition, although SOTA AT models showed excellent performance on
20

---

Published as a conference paper at ICLR 2025
Table A6: Accuracies (%) of methods under different adaptive attack threats on ImageNet-100. The same conventions are used as in Tab.
Robust Acc
Architecture
Method
Type
Clean Acc
T. norm
11 norm
12 norm
Average
ResNet-50
Liu 2023a)
78.8
47.2
3.6
25.0
25.3
AT
ResNet-50
Debenedett el al.
2023
79.5
50.3
23.8
26.0
UNct+ResNet-50
DifiPure (Nic et al.
2022
77.3
22.2
55.5
58.4
45.4
UNet+ResNet-50
ADBM (Ours)
79.5
25.2
58.6
61.3
48.4
Table A7: Accuracies (%) of methods under three query-based attacks and the transfer-based attack on CIFAR-10. 10. The same conventions are used as in Tab.
Robust Ace
Architecture
Method
Type
Clean Acc
RayS
Square
SPSA Transfer Average
WRN-70-16
Vanilla
97.02
6.64
2.44
1.37
WRN-70-16
Gowal el al.
2020
91.10
79.20
71.62
80.76
89.98
82.33
WRN-70-16 16
Rebufn et al.
2021
88.54
73.91
69.35
75.82
87.64
79.05
AT
WRN-70-16
Gowal ct al.
2021
88.74
78.81
73.93
79.88
87.89
81.85
WRN-70-16
Wang et al. 2023
93.25
81.89
76.30
82.13
90.34
84.78
UNe+-WRN-70-16
Yoon cl al.
2021
87.93
74.22
72.17
68.55
84.28
77.43
UNet+WRN-70-BB 16
DiffPure Nie et al.
2022)
AP
92.51
90.92
90.53
90.33
90.16
90.89
W6N
ADBM (Ours)
91.86
91.31
91.50
90.72
90.39
91.16
attacks, they struggle exhibit robustness against unseen 1 and l2 attacks. On average, ADBM surpasses these AT models by 31.8%.
The white-box adaptive evaluation results Tiny-ImageNet respectively presented in Tab. A5 and Tab. A6 In the Tiny-Iman Inn-lmmgeNet evaluation, we maintained consistent settings with those used for CIFAR-TO, with the exception that for the 4.0 threat, we the bound be e 4/255, which is popular setting for this dataset (Yang et al.
2023 Wang et al. 2023). In the ImageNet-100
evaluation, we set the bounds Eoe 4/255, E1 75, E2 2, which commonly used setting for the high-resolution datasets like ImageNet-IK., IK. For this dataset, we directly compared ADBM with representative ResNet-50 AT checkpoints specified trained on ImageNet- 1 by masking out the 900 output channels not included in 1000eNNet1100. By examining Tab. A5 and Tab. A6 we can draw similar conclusions to those observed on SVHN and CIFAR-10.
E.3 ADDITIONAL RESULTS UNDER BLACK-BOR SEEN THREATS
The evaluation results on CIFAR-10 under black-box box seen threats are presented in Tab. A7 In this evaluation, we maintained consistent settings with those used for SVHN. By examining Tab. and Tab. A7 we can conclude that under the realistic black-box attacks, ADBM has advantages over AT models even on the seen threat.
E.4 RESULTS UNDER BLACK-BEL -LOX UNSEEN THREATS
We further evaluate the generalization ability of ADBM on more unseen threats beyond norm- bounded attacks. Two patch-like attacks, PI-FGSM (Gao et al.) 2020 and NCF (Yuan et al.) 2022 and two recent diffusion-based attacks, DiffAAtack (Chen al. 2023b) and DinPGD (Xue et al. 2023) were evaluated. As the code these methods only natively supports input resolution 224 224, conducted experiments ImageNet-1000 The results are show in Tab. A8 We can conclude that ADBM demonstrates better generalization ability than DiffPure and previous AT methods against these black-box unseen threats.
21

---

Published as a conference paper at ICLR 2025
Table A8: Accuracies (%) of methods under different black-box unseen threats on ImageNet-100.
Architecture
Method
Type
NCF
PI-FGSM
Diff-Attack
Diff-PGD
ResNet-50
Liu et al. 2023a
3.8
26.3
21.5
29.4
AT
ResNet-50
Debenedetl
2023
11.2
29.4
28.5
49.8
UNet+Res.Net-50
DiliPure
38.4
39.8
71.9
AP
UNet+ResNel-500
ADBM (Ours)
33.9
42.7
42.8
72.4
Table A9: Accuracies (%) of different methods on CIFAR-10. 10. Here the classifier used a WRN-28-10 or vision transformer. ADBM (Transfer) was trained with noise from WRN-70-16 16 while ADBM (Direct) was trained with noise from the WRN-28- 10 or the vision transformer directly.
Method
Classifier
1oc
4
lz
Average
DiffPure
42.4
60.3
43.8
48.8
ADBM (Transfer)
WRN-28-10
45.1
61.7
52.0
52.9
ADBM (Direct)
45.3
62.1
51.9
53.1
DiffPure
25.0
41.8
51.6
39.5
ADBM (Transfer)
ViT
28.9
46.7
54.1
43.2
ADBM (Direct)
30.9
48.4
56.1
45.1
Table A10: Inference time (in seconds) for ceffiusion-based purification on different datasets, corre-
sponding to the reverse sampling (RS) steps using an NVIDIA 3090 GPU (with batch size of 1).
Dataset
Network
RS=
2
3
4
5
100 (Original DiffPure)
CIFAR-10 SVHN
UNet+WRN-700 WRN-70-16
0.060
0.099
0.152
0.189
0.248
4.420
UNet+WWR-222
0.052
0.095
0.139
0.182
0.229
4.349
Tiny-ImageNet
UNet+ -0
0.056
0.010
0.149
0.195
0.232
4.407
E.5 RESULTS OF ADBM ON NEW CLASSIFIERS
Here we utilized the fine-tuned ADBM checkpoint, trained with adversarial noise from a WRN-70- 16 classifier, as the pre-processor for WRN-28-10 model and vision transformer (VIT) mode directly, denoted as ADBM (Transfer). The results shown in Tab. A9 demonstrate that ADBM (Transfer) achieved robustness levels comparable ADBM trained directly with WRN-28-10 10 or the ViT (52.9% vs. 53. % on WRN-28-10 and 43.2% vs. 45.1% % on ViT). This finding highlights the practicality of ADBM, the fine-tuned ADBM model on specific classifier can potentially be directly applied to new classifier without requiring retraining. We guess this may be attributed to the transferability of adversarial noise (Dong et al. 2018).
F INFERENCE COST OF ADBM
On one hand, the inference cost of diffusion-based purification can be significantly reduced via fast sampling according our investigation. Tab. gives the actual inference time (in seconds) for diffusson-based purification on different datasets, corresponding to the reverse sampling steps. Given that ADBM has demonstrated effectiveness with a single reverse step (Tab. 2), it has potential applications in various real-time scenarios (only 0.06s is needed for inference in this case, shown in Tab, A10).
On the other hand, similar to DifiPure, ADBM necessitates additional parameters. Taking CIFAR-10 example, the classifier WRN-70-16 16 consists of 267M parameters, while the UNet architecture, specifically the DDFM++ cont nucus used in this work, consists of 104M parameters. remains an open question whether the scale of the UNet architecture, which is currently directly adapted from architectures designed for generative tasks, can be further reduced for adversarial purification. We leave it to future work.
'ee used the checkpoint released by https: / /i thub. com/dqj5 82/vi 82/vit_enn, t__nn_ccinnnnni far_10 From_scratch) which had 90.6% clean accuracy on CIFAR-10.
22

---

Published as a conference paper at ICLR 2025
G BOARDER IMPACT
Our method, which effectively purifies adversarial noise, can potentially hinder the utility certain "adversarial for good" methods. For example, it could cause more difficult copyright protection using adversarial noise (Liang et al. 2023) when abusing our method. In addition, there seems to be a trade-off between adversarial robustness and the risk of privacy leakage (Li et al. 2024a). But in general, we believe the concrete positive impacts on trustworthy machine learning outweigh the potential negative impacts.
23


Reviews:
[{"review_id": "VN5PbznOP3", "reviewer": "Reviewer_hELf", "summary": "This paper trains a purifier based on the theoretical framework of a diffusion model, making it better suited for adversarial purification tasks.", "questions": "please see weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Introduces a comprehensive training process to adapt the diffusion model for adversarial purification tasks.\n- Theoretically and empirically demonstrates superior performance compared to DiffPure.", "weaknesses": "- Unlike DiffPure, the Adversarial Diffusion Model (ADBM) requires reasoning about adversarial perturbations for purification, which may **limit its generalizability to unseen attacks**, similar to challenges faced by adversarial training. For instance, the experimental results in Table 3—beyond different settings of white-box attacks, how does it perform against Square attacks or StAdv?\n- The author mentions the Robust Diffusion Classifier (RDC) method in the paper but **does not compare it with the purification component of RDC, namely Likelihood Maximization (LM)**. To my knowledge, RDC’s LM can handle unseen attacks better than ADBM without needing adversarial perturbation training.\n- In the original diffusion model framework, as the time step approaches infinity, $x_T$ converges to a standard Gaussian distribution. With ADBM losing this property, there may need to be corresponding adjustments in the selection of variables such as $\\bar{\\alpha}$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kIF7wuavUF", "reviewer": "Reviewer_H77B", "summary": "This paper proposes a new method for Diffusion-based Purification, which is named as Adversarial Diffusion Bridge Model (ADBM). Different from previous methods, ADBM directly constructs a reverse bridge from the diffused adversarial data back to its original clean examples. Experiments show the effectiveness of the proposed method.", "questions": "1. What is the performance on more datasets with more classification numbers?\n\n2. What is the performance of the proposed method with various networks?\n\n3. What is the performance of the proposed method towards different types of attack?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed ADBM has the complete theoretical support, and the experiments on the CIFAR10 demonstrate the effectiveness of the proposed method on some chosen datasets.", "weaknesses": "1. The proposed method is only verified on the datasets with small classification number, such as CIFAR10 and SVHN. The experiments on datasets with small sizes can not fully demonstrate the effects of the proposed strategy, since the applications in the real world will have large number of classification number. A more suitable dataset could be ImageNet with at least 1K class number. I think the complicated data distribution will largely influence the performance of diffusion models.\n\n2. The experimental results are mainly built with the network of WRN, and the experimental results with more types of networks are not shown.\n\n3. Compared with other methods, the accuracy on the adversarial examples are not improved a lot, while the accuracy on the clean examples are decreased clearly. The authors should analyze the factor which limits the accuracy on the clean examples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AjI8qHkSsU", "reviewer": "Reviewer_pfC6", "summary": "Adversarial attack is a challenge for the neural network. This paper proposed a diffusion model-based adversarial purification (AP) to defend against adversarial attacks via adversarial training (AT) called ADBM. ADBM alleviates the trade-off between noise purification efficiency and recovery quality by proposing a new loss function to fine-tune the diffusion model via the AT paradigm. ADBM also offers detailed proofs to ensure the adversarial perturbation will be gradually removed. The experimental results show that ADBM achieves the SOTA performance.", "questions": "1. Does ADBM only work for the DDIM sampler? The motivation for this question is that the assumption for the proofs is the DDIM sampler. What happens if we change the DDIM sampler to the original DDPM sampler?\n\nTo sum up, there are two main contributions for ADBM: 1) proposed a full-gradient adaptive attack method. 2) proposed an interesting AT paradigm to enhance AP based on the unconditional diffusion model. All my concerns are listed in Weaknesses and Questions. Considering 1) the lacking the necessary baseline and attack method and 2) the writing problem, I rate it as \"marginally above the acceptance threshold.\" If the author could solve these concerns, I'm willing to increase my rate.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper proposed a new adaptive pipeline based on the diffusion model's full gradient, which can further ensure the diffusion-based AP's robustness. The robustness of the diffusion-based AP is always questionable since it is a computation cost to get the full gradient of the diffusion model. Previous works proposed various gradient estimation methods, such as the adjoint method from DiffPure and the surrogate process from Lee. et al. [1], which remains a question: Is the improvement of such diffusion-based AP due to the estimation error of the gradient? Lee. et al. also prove this question. This paper is the first work to evaluate the robustness of diffusion-based AP by the full gradient to the best of my knowledge. I think this will benefit further study of diffusion-based AP.\n\n2. The AT paradigm for diffusion-based AP proposed in this paper is novel and robust. To establish the loss function for AT, this paper chooses to connect the adversarial samples and the generation target $x_{0}$, which is interesting. Then, the detailed proofs verify that the loss function could ensure the removal of the adversarial perturbation after various times of the reverse process.\n\n3. The experimental results show that ADBM achieves the SOTA performance. Specifically, the experiment also includes unseen attacks to illustrate how the proposed AT paradigm could work on unseen threats.  \n\n[1] Minjong Lee and Dongwoo Kim. Robust evaluation of diffusion-based adversarial purification. In Int. Conf.\nComput. Vis. (ICCV), 2023", "weaknesses": "1. The writing for this paper should be improved. 1) Some math symbols are confused. For example $\\hat{x}\\_0$. It first indicated in 299 lines that $\\hat{x}\\_0$ should be the final point of the reverse process, thus having $\\hat{x}\\_0=x\\_{0}$. But, in 879 lines, $\\hat{x}\\_0$ changed to the prediction of $x_{0}$ for $x_{t}$, thus causing the confusion. There are various similar mistakes, such as $x_{t}^{d}$ in 265 lines.  2) One-step DDIM is confusing since this paper includes the discrete and continuous VP-SDE. Setting $s=1$ is truly the one-step DDIM based on the discrete VP-SDE but is not one-step for continuous VP-SDE since the time sequence of continuous VP-SDE is in $[0.0, 1.0]$. In this condition, $s=1$ means the entire reverse process for continuous VP-SDE instead of the one-step. The authors should explicitly clarify which formulation (discrete or continuous) they use when discussing one-step DDIM and explain the implications for both cases.\n\n2. The experimental results may not be enough. 1) The BPDA [1] should be considered to test the performance of the ADBM since it is a special attack designed for the AP.  Adding additional experiments on CIFAR-10 against BPDA ($\\ell_{\\infty}$,$\\epsilon = 8/255$) will further strength the contribution. 2) AToP should be considered as the baseline, even if only making an ablation study on CIFAR-10 since ADBM is similar to AToP. The only difference is that AToP may be suitable for any generative model, and ADBM is for the diffusion model only.  Adding additional experiments on CIFAR-10 against PGD+EOT ($\\ell_{\\infty}$,$\\epsilon = 8/255$, EOT=20, steps=200) to compare the performance of ADBM and AToP (the generative model chooses GAN, which is the best reported in AToP) will enhance the contribution for this paper. Additionally, adding an ablation study between ADBM and AToP (the generative model chooses VP-SDE) against PGD+EOT ($\\ell_{\\infty}$,$\\epsilon = 8/255$, EOT=20, steps=200) will further show the validity of ADBM.\n\n[1] Minjong Lee and Dongwoo Kim. Robust evaluation of diffusion-based adversarial purification. In Int. Conf.\nComput. Vis. (ICCV), 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)