Why Adversarially Train Diffusion Models?
Authors: Maria Rosaria Briglia
Mujtaba Hussain Mirza
Giuseppe Lisanti
Iacopo Masi
Venue: ICLR 2026 Poster (2026)
Keywords: Robustness
Adversarial training
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Adversarial Training for Diffusion Models makes them extremely robust to outliers, resilient to noise and corrupted data, and finally more secure

Abstract: Adversarial Training (AT) is a known, powerful, well-established technique for improving classifier robustness to input perturbations, yet its applicability beyond discriminative settings remains limited. Motivated by the widespread use of score-based generative models and their need to operate robustly under substantial noisy or corrupted input data, we propose an adaptation of AT for these models, providing a thorough empirical assessment.
We introduce a principled formulation of AT for Diffusion Models (DMs) that replaces the conventional *invariance* objective with an *equivariance* constraint aligned to the denoising dynamics of score matching. Our method integrates seamlessly into diffusion training by adding either random perturbations--similar to randomized smoothing--or adversarial ones--akin to AT.
Our approach offers several advantages: **(a)** tolerance to heavy noise and corruption, **(b)** reduced memorization, **(c)** robustness to outliers and extreme data variability and **(d)** resilience to iterative adversarial attacks.
We validate these claims on proof-of-concept low- and high-dimensional datasets with *known* ground-truth distributions, enabling precise error analysis. We further evaluate on standard benchmarks (CIFAR-10, CelebA, and LSUN Bedroom), where our approach shows improved robustness and preserved sample fidelity under severe noise, data corruption, and adversarial evaluation. Code available at [github.com/OmnAI-Lab/Adversarial-Training-DM](https://github.com/OmnAI-Lab/Adversarial-Training-DM)

Published as a conference paper at ICLR 2026
WHY ADVERSARIALLY TRAIN DIFFUSION MODELS?
Maria Rosaria Briglia'
Mujtaba Hussain Mirzal
Giuseppe Lisanti*
Iacopo Masi!
Sapienza University of Rome
OmnAI Labt
University of Bologna"
ABSTRACT
Adversarial Training (AT) is a known, powerful, well-established technique for improving classifier robustness to input perturbations, yet its applicability beyond discriminative settings remains limited. Motivated by the widespread use of score- based generative models and their need to operate robustly under substantial noisy or corrupted input data, we propose an adaptation of AT for these models, providing a thorough empirical assessment. We introduce a principled formulation of AT for Diffusion Models (DMs) that replaces the conventional invariance objective with an equivariance constraint aligned to the denoising dynamics of score matching. Our method integrates seamlessly into diffusion training by adding either random perturbations-singe to randomized smoothing-44 or adversarial ones akin to AT. Our approach offers several advantages: (a) tolerance to heavy noise and corruption, (b) reduced memorization, (c) robustness to outliers and extreme data variability and (d) resilience to iterative adversarial attacks. We validate these claims proof-of-concept low- and high-dimensional datasets with known ground-truth distributions, enabling precise error analysis. We further evaluate on standard benchmarks (CIFAR-10, CelebA, and LSUN Bedroom), where our approach shows improved robustness and preserved sample fidelity under severe noise, data corruption, and adversarial evaluation. Code available at github.com/OmnAL. Lab/Adderssaril--Triine Tealll-rrinnnn--Da
INTRODUCTION
Large-scale datasets cornerstones the success of generative AI, yet they simultaneously present a significant challenge. Often web-scraped and minimally curated, they frequently contain multiple forms of corruption: inlier noise-subtle perturbations within samples; outlier noise-samples that significantly deviate from the target distribution; missing or corrupted data-commoni, affected by Gaussian noise; and adversarial noise deleeerrelbbally crafted perturbations. While recent approaches have attempted to address training under noisy conditions, they are constrained by restrictive theoreti- cal assumptions. For instance, Daras et (2024c) relies on precise knowledge of noise variance, Daras et al. (2024d) exclusively targets missing data scenarios, and Daras et al. (2024a) presupposes access to both clean and corrupted samples. These approaches fall under the umbrella of "noise-aware training", solving problem with strong assumptions: the methods assume access to clean/noisy sample labels at a sample level, assuming to know the applied noise distribution and its intensity, fully exploiting these assumptions training time. This controlled scenario greatly limits the practical applicability of the proposed methods, as also specified by the authors in the limitation section of their work. The present work aims define the principles of robust training for Diffusion Models, highlighting how this technique enables generative models to be robust against unknown corruption, whether applied to the training data or as inference-time perturbations, such as adversarial attacks. In the classification domain, Adversarial Training (AT) (Szegedy et al., 2014) yields robust classifiers that maintain performance despite input perturbations. Notably, recent research has revealed that AT confers additional capabilities beyond robustness, including generative capabilities (Mujtaba Hussain et al., 2024). Despite its demonstrated efficacy classification tasks, AT has not been systematically extended to other families of deep learning models, particularly generative models. Beyond handling noisy data, generative AI models face further challenges, including data memorization (Jagielski et al., 2023; Somepalli et al.. 2023; Carlini et al., 2023b), leading to information leakage and their propensity to learn spurious correlations from training data that do not reflect real underlying pat- terns. In this work, address this challenge by extending AT to score-based generative models, in

---

Published as a conference paper at ICLR 2026
Proise (x0)
1
Pdata (250)
((xr)
X, 2))
X0
Figure 1: Smooth trajectories. We train the denoising network to follow the score function ie., X2 HX X;-1 using just (o(x,, t), but we also perturb locally as x,++ inside lp ball and then imposing equivariance: x,+6 + Eo (x,,,, t) + 6 a x, This equals adding intermediate step in the Markov Chain, behaving as an additional denoising step, making the model resilient to possible outliers noise in the dataset-Pnoise (xo)-not not proper of (Xo) The local perturbation can be implemented as adversarial or as random (randomized smoothing). Perturbation strength starts large and progressively shrinks when T O. - indicates the forward process: the reverse process.
particular to Diffusion Models (DM) (Ho et al., 2020). Our contribution bridges the gap between AT for classification and the generation paradigm. This extension also reveals new hidden capabilities of AT applied to the generative domain, offering practical and theoretically sound approach to training robust generative models on real, imperfect datasets. Our contribution is then three-fold:
We are the first to reconnect AT to denoising, linking it to Daras et al. (2024d;c;a. Despite some works adversarial aspects in DM training (Yang et al.; Sauer al.), we formally introduce adversarial training for DMs, discussing its practical implications on the learned denoising process.
Inspired by Zhang et al. (2019), we develop an AT algorithm tailored for score-based models. Dif- ferent from classifiers, which require enforcing invariance, score- based models require enforcing equivariance to properly learn the data distribution, as formalized in our key finding in Eq. (14).
We show our method's flexibility in handling noisy data, facing extreme variability like outliers, preventing memorization, and improving robustness. Besides owwdimmesioonl (3D) controlled data, we test our method on CIFAR- 10, CelebA, LSUN and ImageNet, achieving strong performance.
ADVERSARIAL TRAINING SMOOTHS TRAJECTORIES
2.1
PRELIMINARIES
Diffusion Models (DMs) aim to learn a data distribution, Pdata (x) by noising data with a fixed procedure, mapping them to //00, I) using Markov Chain q .x, (xo)
II= (xxx|xxxx)))
where, given a noisy input xt-1, the next state x, is reached through the following gaussian transition:
q(x 1)) = (x /I (t)x: (()):
(1) is the noise scheduler: monotonically decreasing time-varying function chosen s.t. (0) amin- (T)) = max and < m < Omax < 1. The generation is achieved with a learnable "decoding step" that reverts data from noise estimation p(x;-1 ). If the noise scheduler is chosen carefully to take small noising steps, then the approximation x(xx) (x)) se (O, I) and the following equation holds:
(x)) NNxxi v Vaxx-11 (1- aI) where at = II ott)
This means we can encode directly from Xa ト+ X2 as:
X = VoiXo + vT VT-aa az E where E~ NO,
Samples generation is then performed by solving the probability flow ODE (PF-ODE) Song et al. (2021b), from t. T to 0 and starting from xy ~ (O o T), whose solution is learned from the DM. For a given Xo, the training objective LpM reported in Ho et al. (2020) is thus defined as:
LUM Ee~N(O.)) t0,1(
||e eo (x1 c). e]
2

---

Published as a conference paper at ICLR 2026
Input
DDPM
Inv adv
Robusta
Ground-Truit
DDPM
Robustadv
8 ray deviation scheduler
Unifor Outlien. String
0.20
0.00
T
(a)
(b)
(c)
Figure 2: (a) The plot shows lefumost training data either with strong inlier noise (top) or uniform outliers (bottom). The trajectories reveal that DDPM struggles with both, while if you train with invariance (Invadv) the process diverges, Instead, ours (Robustadv) is more robust, avoiding diverging trajectories and better reaching the data centroid. (b) Score vector fields: versors represent the score field, colormap shows magnitude, less more intense. (left) Ground-truth (middle) DDPM; (right) Our Robustadv. AT yields smoother, more consistent scores, better matching the data shape, shrinking variability and increasing field intensity. (c) Perturbation ray. The parameter w controls the slope of - r(I) to shorten the content phase and reduce the curve's steepness in DDPM.
whose objective is to infer the noise applied to the initial image, ensuring that the starting point Xy correctly reconstructed, enabling the model the denoising network e--to to correctly generate in-distribution data during inference. For inference we solve the SDE using eg and the recurrency:
o(t) V1-ai at ca (x(x((0)))) + +0(22, 2 ~~ IO0,I), [0, ,
xx) v1 o(t)
(x((0)
(4)
2.2 MOTIVATION, "IN VITRO" EXPERIMENTS, AND NOISE TYPES
Motivation and overview. Adversarial training has proven highly effective in the classification domain for handling perturbed training data, imposing model invariant response across genuine and adversarially manipulated inputs. Unlike classifiers, its application to DMs requires fundamental reformulation due to their regression-based nature. Our work aims to investigate the properties and applications of adversarially trained DMs, with particular emphasis on the case of corrupted training data. Fig. 2 (a,b) describes the investigated settings of uniform outlier and strong inlier noise, where our approach was demonstrated to be learning the correct data distribution and a smoother score field.
"In vitro" analysis setup. We propose a first analysis of the framework synthetic 3D data, spanning from "linear" and unimodal to more complex multi-modal ones. obl i cuue-plaeeee assumes the data distribution Pdata lives a multi-modal 3D Mixture of Gaussians defined as ( 10, 10], .) + (L20 20, 20). o)
2D subspace with equation y + 2 30, while 3 3-gauss
3 W[10, 30, 30], o) and σ = 0.25. Regarding higher-dimensional data, we built the analysis on the data generated after linearizing, using PCA (Abdi & Williams, 2010), the "Smithsonian Butterflies" image dataset. After fitting a 25 dimensional subspace, retaining 70% of the sample's variance, we sampled data according x' μ E, X U. Sampling stochasticity comes from ~ ~N((0; o). while l R3772 U E 22533772 and X; are the mean, dataset's 's principal components and its singular values. Finally, we discard the real data, fitting the DM on [x'). This allows to perfectly measure distance between the DM generated samples and the lincarized distribution, measuring the closed-form reconstruction error p = |xx(()) UUT (0)( between the data subspace and the generations, where (0( is generated iterating on Eq. (4). We also use the measure Peak Signal-to-Noise Ratio (PSNR) from image processing (Hore & Ziou, 2010).
Noise model tested. We analyze the framework on different noise models. First, we consider inlier noise, implemented by increasing the sampling variance in the case of subspace μ E; A, U, increasing the a. We then include outliers by adding strong noise the ambient space: for 3D data, add point cloud with dense, grid-like, uniform noise; for butterfl fies, we add Gaussian noise on the linearized data as x' + where z ~ A00 oI). Figs. and 3 show the proposed ablations.
hugggi wn cttaaccc.ccc////e ww/.ao///aaaeeeeeeeeeo httggtggtttttttsaaaaaaa hwggangggggggg// smi thson i wutteutttrr suu_st
3

---

Published as a conference paper at ICLR 2026
PSNR 1
(1x) -UU -Uo]xx(0)
Regular dataset
-3
2
00003
DDPM
0 XX
Noisy dataset
0.080
M
15
XX
(
Robustas
13.
8848
Egochs
Epochs
DDPM
Robustar
Robnstalr
Inv
In Inv
Figure 3: (left) On butterflies, report the closed-form reconstruction error From top to bottom: training data, corrupted data, DDPM-generatte samples, and Robustady results. (right) The chart columns display PSNR and closed-fome form reconstruction error measured on clean data (top) and on data corrupted at 90% with Gaussian noise (a 0.1) (bottom). We also include results for invariance regularization with X {0.3, 0.033; these settings prevent the model from properly Pdat-
2.3 ADVERSARIAL TRAINING FOR DIFFUSION MODELS
Diffusion processes rely the denoising function mapping the noisy distribution q the data distribution Pdata. learned from optimizing LDM (see Eq. (3)), which ensures the model to learn the score field correctly, guiding the trajectories toward the data distribution. When defining the AT procedure, applying the standard AT (Szegedy et al., 2014) could hinder the learning process. We, indeed. propose an AT technique that inherits its principles from TRADES (Zhang et al., 2019), and accordingly acts as distribution-leww regularization.
Naïve invariance does not work. As for AT, we aim guarantee that the model maintains constant behavior in its predictions, whether the input sample is corrupted or not. To accom- plish this, standard AT imposes invariance in the classification domain; conversely, for DMs, performing regression task, invariance does not guarantee the same result. Fig. shows that applying classical AT invariance to learn a different distribution than Pduta (x), rooting the generation to produce noisy data. This finding translates also to higher- dimennsoonl data: Fig. (right, bottom row) shows both qualitative samples and quantitative results applying the invariance to but putterfi ies noisy data (90% corrupted samples, 0.1). The model, indeed, only recovers the distribution knowledge the weight À decreases. When moving to real data, applying invariance resulted in worse FID, achieving 356.9 on 50K generated samples from a DM trained on the CIFAR-IO dataset.
Lim - Aarg min |xo + -,t) cx I|
Eq. (5), causes DMs
(5)
Key change is equivariance. Starting from an e- predicting DM, we defined the AT taking into ac- count the need for input sensitivity of the model by enforcing equivariance. The intuition is de- picted in the introductory Fig. and theoretical discussion is given in Section A. Since the main aim is to keep the model rooted to the data distri- bution, despite the additional perturbations 8, the network must learn to correctly recover the previ- ous state x starting from x; + 8. This objective reached by taking into account 5 in the AT loss as arg ming co (x, + t) - [e 1|| While this equation enforces equivariance, it does not yet force smoothness, since two outputs of the network do not interact with each other.
Algorithm 1 AT for Diffusson Models
Input: dataset D, model 0 max timestep T, scheduler ae strength X, ray scheduler (t) repeat
Sample Xu D, E OO, t ~ U ................................ Th), (0,5
U (t),
X; V /xxi vI ā₁E
Compute 6mN using Eq. (8)
adv VāyXa + +VI (5 + c)
レ 0 - VoLx (x,. x Eq. (6)
until convergence
4

---

Published as a conference paper at ICLR 2026
Our Training, In this work, we propose adversarial loss suited for e-predicting DMs. Given timestep t and an initial sample Xg ~ Pdata- we define x; as in Eq. (2), and its perturbed counterpart x, + 6. AT is then defined regularization term, freg. aims to promote local equivariance and smoothness along the regular DM trajectories, which is achieved by locally minimizing the difference between the model's 's prediction on Xr and X2 + 6. The complete loss is given in Eq. (6), where the adversarial component is weighted by a time-dependent coefficient Aς. AT procedure is detailed in Algorithm 1.
regularization of the standard DM objective. The adversarial
LXT(X,, X + 8,t,c) arg min (x.. - -1l2 xoll + o, [ 1) + 87] 5111 (6)
Lom to fit data disir.
Luxe to enforce smoothness
2.4 ADVERSARIAL PERTURBATION THE DIFFUSION PROCESS
Injecting noise in the trajectory space. Learning a diffusion process requires itself to corrupt natural data iteratively during its training. In this context, the adversarial perturbation can be considered as an additional noise component injected into X, during the training. Therefore, defining the sample's adversarial counterpart x, + so that it does not interfere with the diffusion process, requires a proper and careful tuning of the perturbation parameters, accounting for both the DM objective and the intermediate noisy data distributions X2 ~ (xx-1,077)) To ensure compatibility with the diffusion process (Wang & Vastola, 2023; 2024), bound the adversarial noise by time-varying radius r(t) = [|o(() dependent on (t) values, to maintain model stability and avoid mode collapse due to diffusion trajectories merging. Specifically, allowing the ray to grow too large in some diffusion phases, like the content phase of generation (Choi et al., 2022), can lead to data over-smoothing. causing the model not to capture the correct distribution. We define X2 + Ô as:
[=ra(0, ( = (VI az) )0
+7 (7)
+ Varxa + v1 or (E+ + 8), where
where e (O, I) and the exponent
guiding the ray scheduling, whose effect is shown in Fig. 2 (c). Finally, we propose retaining a randomized bias term y B with 3 ~ 24 (0.5, 2),7 E R+, whose aim is to prevent regularization annealing as 0 and avoid data under-smoothing.
Smoothing perturbations. The adopted smoothing perturbation could be either random 6 akin to randomized smoothing Cohen al. (2019), adversarial 5 in AT Goodfellow et al. (2015).
Random: This approach requires the perturbation 8ran to be sampled randomly in a uniform distribution, limited by ri (t), defined as in Eq. (7). Being the ray itself randomized through the variable B, 6 would then be uniform random variable whose standard deviation ra (t)/ /3. proof Section C.3.
Adversarial: In the adversarial setting, we employ the Fast Gradient Sign Method (FGSM) with random start (Kurakin al., 2017). The perturbation is first initialized as 6 then followed by single FGSM step. The resulting perturbation is then projected back onto the foo ball of radius rt))) to ensure ||ōadrliso S (()... The optimization dadv considers the following cost function:
rs(()) v3
[x( 6, = ||co (xx +5, co (xe, t) 1 8mn = Prg(()) 6
8a t))
(8)
where P Præ(t) projects the adversarial perturbation onto the surface of x, 's neighbor loo -bal S is the sign operator and r3 (1)/ /33 is the standard deviation of the attack. Once the attack magnitude is defined, we define the AT regularization strength as X = A.y/3 dependent on the perturbation norm via its standard deviation and global constant À E I
Br(())
3 EXPERIMENTAL RESULTS
Experimental setup. We present results datasets ranging from controlled synthetic 3D data to com- plex, real-world multi-n multi-modal data, presenting results "in vitro" to precisely measure errors in both low- and high- digeesiimeniin settings We further offer results real datasets such as CIFAR-10 (Krizhevsky et al., 2009) (50K images, 32 x 32 pixels), CelebA (Liu et al., 2015) (202K images, 64 x 64 pixels), LSUN Bedroom (Yu et al., 2015) (303K images, 256 x 256 pixels), and ImageNet (1.28M images,
5

---

Published as a conference paper at ICLR 2026
Ho al. (2020), o 0.2
Robustades d 0.2
Ho al. (2020)
Robustadv
L' D fo
DDPM, o 0.1
DDPM, 0.2
Robustad o 0.1
Robustadv dr 0.2
Figure 4: (top-left) Despite 90% of training data being corrupted with Gaussian noise, Robustady generates smooth objects without artifacts, while DDPM retains noise. = 0.2 equals adding 40% of CIFAR-10 variability (Adata = 0.5). (top-right) DDPM generates bedrooms that are irregular and unrealistic propagating the noise whereas Robustadv bedrooms are smooth and neat. (bottom) Results on CelebA. DDPM replicates noise, while ours discards it and produces cleaner faces.
64 64 pixels as in Daras et al. (2024a)), quantitatively evaluating samples, using established metrics such as IS (Salimans et al., 2016) and FID (Heusel et al., 2017). Following Daras et al. (2024c;a), experiment with Gaussian noise corruption Pmoise and only work in challenging settings, testing a percentage of corrupted data of p = 90% with two levels of = {0.1, 0.2}. However, our method does not take into account the distinction between clean and noisy samples nor requires knowledge of the corruption variance a. When computing FID, we always test on the clean dataser despite training with noisy datasets. Our methods are indicated by Robustady when using adversarial perturbation and Robust,an if random. We show additional experiments that support our claims on less memorization, faster sampling, and robustness to attacks. We set w 2, 7 = 8/255 and 0.3: across datasets, we have observed that when raising it to 0.5 we get an over-smoothing effect while low values prevent too much denoising. The adopted DDPM baseline is Nichol & Dhariwal (2021), whose available implementation was adopted as codebase.
3.1 EVALUATION USING DDPM AND DDIM
Controlled Experiments. Fig. 3 (right) shows the results when training on high dim. data living subspace. When training on the clean, regular dataset, the baseline and our Robust DMs perform similarly though Robust,an has slightly better PSNR. When we train the noisy dataset, fx' + + then both Robust DMs offer superior performance (orange and blue curves) with wide gaps compared to the baseline (green curve) in both PSNR and reconstruction error. Specifically Robust appears to be better at noise unlearning. DDPM generations often consist in samples with saturated colors that are unlikely be found in the training set while our method has better fidelity-see Fig. 3(left).
Random or adversarial perturbation? We can dad
DDPM Ho et al. (2020)
Robustady
also reply to this question by ablating and 6 Table (top) shows that the adversar- ial perturbation can guarantee a much stronger denoising effect than random, yet is more expen- sive for training. The impact of our Eq. (6) is remarkable even in the case of random perturba- tion with an FID far below the baselines.
Despite the FID increasing once trained on clean data, images by Robustadv appear smoother and background clutter is removed.
Figure 5:
Resistant to noise by design. Table 2 compares our approach with the baseline DDPM and DDIM on CIFAR-10, CelebA, and LSUN Bedroom, yet corrupted with shite noise. We show that if we apply our method to the original dataset with no noise (p- = 0%), we only get slight increase in the FII. However, if we visually inspect the results, discover that ours is actually

---

Published as a conference paper at ICLR 2026
Table Top: Random vs adv. noise. Bot- tom: Ro allows fewer steps for bet- ter FID. Results on CIFAR-10.
Table 2: Performance under different noise levels on
different real datasets. Values indicate FID / t.
DDPM
Robustad
DDIM
Robusslatr
0.1
0.2
CIFAR-10
metrics- FID
18
FII)
IS
0
0
7.2/
8,95
28.68 7.00
11.62/8.36
31.20/ 6.38
0.9
0.1
58.05 6.93
24.70 7.21
59.28/6.89
25.48 / 6.85
Robuskam
79.21
5.21
68.04
68.
4.34
0.9
0.2
102.68
4,199
24.81 / 7.07
105.43 / /4009
24,93/ 6.69
Robustalv
24.70
7.21
24.81
7.07
CelebA
2.61
19.83
/
0.9
0.1
54,901 2.40
14.54/2.000
41.29 2.48
17.98 1198//2 2.22
steps 300
500
0.9
96.03 /2.65
16.53 2.11
89.28 2.62
20.24/ / 2.20
metrics- +
FID
IS
FID
1S
LSUN Bedroom
DDPM
224.38
3.33
28.07
8.46
0
0
9.90 / /2.31
57.13/ 2.34
27.00 3.15
48. 80/ 2.39
0.9
3.33
44.07
50.53 3.19
48,90 3.96
Robustade
37.89
6.39
24.34
7.53
0.9
0.2
95.85/ 4.08
44.27 2.550
82,20/ 4.39
61.98 3.66
smoothing background features, but still outlines of the objects are visible, shown in Fig. 5 and Section E. When we switch to noisy settings, we have a large improvement over the baseline for both DDPM and DDIM. We highlight that while the baseline FIDs skyrocket to very high values for p = 90%, σ = 0.2, the Robustadv can keep it in a reasonable range, generating images unaffected by the noise. We also provide early results on the ImageNet (Russakovsky et al., 2014) dataset, which comprises 1.28M images, downsampled at a resolution of 64 x 64 pixels following Daras et al. (2024a). The regularization also works effectively on this more complex dataset, resulting in a decrease in FID from 97.6 to 83.8 for p = 90%, a = 0.1 and from 129.4 to 80.3 for 90%, = 0.2. Quantitative evaluations are provided in Table 2, showing major improvement of the regularized training over standard training. Fig. 4 illustrates our method's benefits on the proposed datasets under noisy data conditions. More results and images are available in the appendix.
Time complexity. Training with AT strongly impacts training time due to the overhead of computations needed. DDPM training operations comprehend a single forward pass to get model prediction and a backward pass for weights update. Our regularization adds a backward pass to obtain adversarial loss gradients over the perturbation and doubles the same DDPM operations. The time complexity is x 2.5 for Robustadv. whereas Robustyan is less time-consuming since it does not have to backpropagate for the adversarial perturbation. Despite the training time being higher than the baseline, remarkably, the inference time is the same other methods, and we can attain faster sampling see Section 3.3.
3.2 ROBUST DIFFUSION MODELS MEMORIZE LESS
Following Daras et al. (2024d) we show that Robust DMs are naturally less prone to memorize the training data. We perform an experiment following Somepalli al. (2023): using DDPM and our Robustaiv trained on clean CIFAR-10, we synthesize 50K images from each of them and measure the similarities of those images with the one in the training set, embedding the images with DINO- v2 Oquab et al. (2023). In Daras et al. (2024d) similar experiment was done yet using DeepFloyd IF instead of U-Net DDPM. Although U-Net has much less parameters than DeepFloyd milmiosoons vs billions one could assume that U-Net will overfit less. Fig. 6 (left) shows that still decent amount of generated samples have similarity higher than 0.90. Similarity 2 0.9 roughly corresponds to the same CIFAR image. Robust models have histogram that is drastically shifted on the left and the curve of the histogram in the right part decays more rapidly, having less samples in the region 2 0.9.
3.3 IMPLICATIONS OF SMOOTH DIFFUSION FLOW
Smooth diffusion flow. Fig. 6 (right) shows the diffusion flow from the standard normal distribution to the data distribution. To do so, we use DDPM framework and low-dimensional 3D data, projected to 2D for clarity. In oblique plaane, we can see how Robustadv captures less variability, filtering out noise, while DDPM heatmap is more faded. Moreover, DDPM, misled by the noise, introduces very subtle additional mode, whereas ours maintains a unimodal generation. The same remarks hold for
dataset: in 3- 3-gaussise sisss DDPM's trajectories remain straight, preserving the multi-modal structure (only two modes are visible due to projection). Trade-off analysis on clean and noisy data. This sharpening of the trajectories leads reduction in
distorted by noise, while ours
7

---

Published as a conference paper at ICLR 2026
DDPM Ho al. (2020)
Robustady
Seceed Viswrsis
platee
Rob
36
1000
Count
ob
500-
0
0.50
0.60
0.70
0.80
0.90
1.00
Similarity Score
Figure 6: (left) The histogram shows similarities between generated samples and CIFAR-10, with values above 0.9 indicating near-duplicates. DDPM memorizes more, while Robust reduces near- replicas. (right) Regular training tends to have diverging trajectories w.r.t. the data distribution, while Robustady trades off variability for resilience with trajectories more clustered, sharp, and less faded.
the variance of the generated data, but it does not induce mode collapse. As result, the generated images may lose some high-fi figh-frequency noise and fine details, producing outputs that appear smoother overall. Nevertheless, by applying regularization determined by the parameter A, can effectively modulate its action and thus its smoothing effect. The analysis of the regularization effect, depending on Eq. (6), enables us to define an existing trade-off between image quality and robustness, as well as denoising capabilities, similar to the widely examined trade-off between robust and clean accuracy in robust classifiers. Qualitative examples supporting this are provided in Figs. and 25, where we illustrate how model performance varies when the À is changed and how it affects both generation and denoising capabilities.
LSUN Bedroom
LSUN Bedroom 0.1
LSUN Bedroom 0.2
tF
DB
02
Y -=
À = 0.3
Figure 7: Robustady trained on LSUN Bedroom dataset, with different noisy data (p = 90%, different visible in the image) and varying hyperparameter À =(0.1, 0.2, 0.3).
Faster sampling. Fig. 6 (right) shows that the diffusion flow of Robustaiv is more compact and sharp, less faded. This could imply that the inference process may still recover the right path case the regressed score vector is corrupted or is noisy in case deliberately use fewer steps in Eq. (4) for faster sampling. We tested this hypothesis and the trade-off table of FID in function the number of steps taken is shown in Table (bottom). Even more, if we cross compare Table I (bottom) with Table 2, on clean data Robustaav scores a better FID with 500 steps (24.34) vs 1000 steps (28.68). This experiment supports claim showing that Robust,a, still able generate samples with good
8

---

Published as a conference paper at ICLR 2026
fidelity even if using fewer inference steps. The degradation using less steps is widely more graceful than DDPM especially when we take only 300 steps over 1000.
3.4 ROBUSTNESS TO ADVERSARIAL ATTACKS
Our method is naturally resistant to attacks. Like classifiers, AT enforces robustness adversar- ial perturbations in the diffusion flow. We pro- pose an attack primarily an analytical tool to better understand the fundamental sensitivity of the generative process to perturbations. The attack takes into account the stochastic nature of DM inference and the fundamental hypothe- sis of gaussianity for each diffusion stage. We propose attacking a DM in a white-box setting defining sequence of adversarial perturbations that could maximally disrupt the trajectory at some of the intermediate inference steps, defined as described in Algorithm 2. We also propose procedure determine the range of values of the perturbation in order to maintain the assump- tion of the diffusion process; more information can be found in Section A.2. Fig. 8 shows that our method is much more robust to attacks in diffusion flow: Robustaiv can tolerate up 50% of time step attacked and still generate samples with decent fidelity. Only at 75% time steps attacked, the generation fails for both. The attack illustrated in Algorithm 2 is single-iteration attack. In Section A.3, we extend the pool of considered attacks to include iterative PGD (Madry et al., 2018), and provide the model's performance that setting. In addition, to account for the inherent stochasticity of the diffusion-baat based inference process, evaluate robustness under an Expectation- this framework in the stronger setting of the PGD attack, previously introduced. In particular, the EoT is applied by averaging gradients over five stochastic noise samples. The average gradient is then used update the perturbation. The detailed attack implementation can be found in Section A.4. The table in Fig. 8(b) (bottom) shows that the Robustady model also effectively demonstrates its resilience major disruptions in the diffusion process, as well as robustness EoT-based attacks.
Algorithm 2 DM Trajectory Attack
Input: attack ratio p. max timesteps T. model scheduler (re, a (t), attack strength o; Sample XT ~ ~N(00, for t T to 0 do
X t Ex
&o X+ x Eee x
Vār
(x R0) x var-jo(t)
yarl 10l--rr-11 Xt
1-a;
1-61
x = + +5.6 ~
1.~~N(O NN 22, (1)
x t cx(x),t)
* t xi-vi--neo(xi.
vite
L +
- fx, x6)
x X, a()) sign ( Vx,1)) L.)
X;-1 t E (
end for
Transformation (EoT) (Athalye et al., 2018). We adopt
25%
50%
75%
100%
FID L w/ FGSM
Algorithm: and Section A.2
steps attacked
250
500
750
1000
DDPM
49.8
131.7
190.4
243.4
DDPM
19.29
52.0
90.7
127.7
FID w/ PGD Algorithm and Section A.3
attacked
250
500
750
1000
DDPM
55.7
134.5
200.3
248.1
98.1
Robustan sdt
FID | w/ EoT-PGD
Algaritum 4 and Scction A,4
atacked
250
500
750
1000
DDPM
57.6
132.7
195.99
208.6
Relned,
100.2
(a)
Figure 8: (a) Robustness to Adversarial Attacks. While the baseline DDPM is susceptible to adversarial attacks, Robust DMs better resist them, yielding superior FID and IS for different percentages of time steps attacked (e.g., 25% means 250 out of 1000 DDPM steps attacked). (b) FID under FGSM, PGD and EoT applied to PGD, varying the percentage of attacked timesteps.

---

Published as a conference paper at ICLR 2026
4 RELATED WORK
Diffusion models. Score-based generative models (Song & Ermon, 2019) express the inference process through a Stochastic Differential Equations (SDE) Dhariwal & Nichol (2021). Denoising Diffusion Probabilistic Models (DDPMs) Ho et al. (2020) first introduced diffusion process as score-basee generative framework, becoming standard algorithm in generative modeling on high-dimensional data, overcoming Goodfellow et al. (2020). DMs not only achieve higher fidelity but also provide a more stable training. DMs have been extensively improved: working on the logarithmic likelihood estimate Nichol & Dhariwal (2021), faster sampling Song et al. (2021a), and performing the diffusion process in the latent space Rombach et al. (2022). Karras et al. (2022; 2024) provide insightful clarifications on several DMs design choices, introducing improved U-Net architectures that ensure consistent activation, weight, and update magnitude, achieving state-of- the-art FID on CIFAR and other benchmarks. Lastly, Song et al. (2023) proposed consistency models, a distillation method for one-step inference by directly mapping noise to data. The name consistency arises from the fact that they enforce different noisy versions in the same trajectory to map to the same data. Unlike them, we do not aim to distill a model, but rather to train one enforeing local smoothness of trajectories within the same timestep f, so that their score field remains locally consistent.
Denoising and inverse problems with DMs. The attention to apply DMs on corrupted data has in- creased in recent years (Aali et al., 2023; Xiang et al., 2023; Daras et al., 2024d;a). Given the specific challenges related to training with noisy data, this problem is closely related to inverse problems (Tachella et al., 2024; Kawar et al., 2024). Recently, a line of research focused on applying Stein's Unbiased Risk Estimator (SURE) (Metzler et al., 2020) and its subsequent improvements, including UNSURE (Tachella et al., 2024), GSURE (Kawar et al., 2024), Soft Diffusion (Daras et al.,
2024b), and methods leveraging optimal transport for training with noise (Dao et al., 2024).
Adversarial robustness. This topic is loosely linked to denoising since AT can be seen as a way to remove spurious correlations (Ye et al., 2024) with improved out-of-domain generalization when transferring to a new domain (Ilyas et al., 2019) or related to causal learning (Zhang et al., 2020; 2022). AT variants have been used to improve domain shift (Salman et al., 2020a) and out of distribution (Wang et al., 2022). While it is reasonable say that AT has been extensively studied on classifiers, its application to DMs remains unexplored, except for Sauer et al. (2024), where it is applied for fast sampling, Yang et al. (2024), which investigates the batch samples interconnection. and Lorenz et al. (2024), which found adversarial samples do not align with the learned DM manifold. Finally, Wang et al. (2025) proposed similar equivariant loss to ours, yet they do not demonstrate resilience against attacks or robustness to noisy data.
Adversarial defenses with denoising or randomized smoothing. Several adversarial defenses leverage denoising (Salman et al., 2020b; Carlini et al., 2023a) and randomized smoothing (Cohen et al., 2019), mainly in the context of classifiers. Regarding DMs, Song et al. (2024); Liang et al. (2023); Liang & Wu (2023) have shown that adversarial perturbations, if applied at inference time, can significantly disrupt their generative capabilities, leading to deviations from clean data distributions. Further works introduce the concept of robustness when fine-tuning DMs to make them robust in the context of adversarial purification (Song et al., 2018; Nie et al., 2022; Lin et al., 2024). While these methods differ in the definition of adversarial samples (Li et al., 2025: Liu et al., 2025), they share similar underlying objectives. Our work introduces AT in Diffusion Models to enforce local smoothness in the score field, which may help counteract such deviations during inference. Unlike Guo et al. (2024), our work aims to smooth model trajectories, not embeddings (Section A.5), unlike SmoothDiffusion.
5 CONCLUSIONS AND FUTURE WORK
We presented the first attempt to incorporate AT into DM training, demonstrating that AT for generative modeling entails smoothing the data distribution and can be effectively utilized for denoising the data. We also show that we need to reinterpret it as equivariant property and not invariance. Our method has been proven to be highly robust even under 90% of corrupted data with strong Gaussian noise. In terms of future work, we aim to extend this work to a robust fine-tuning technique that is applicable to larger, new models with reduced training costs. Preliminary results are presented in Section F.2. We also plan to extend our method to work in fully corrupted settings (p 100%) and port our approach to EDM (Karras et al., 2022; 2024) to scale to larger datasets.
10

---

Published as a conference paper at ICLR 2026
Ethies Statement. Based comprehensive analysis, assert that this work does not raise identifiable ethical concerns or foreseeable negative societal consequences within the scope of our study. On the contrary, contributions aim to enhance the robustness of Diffusion Models against attacks.
Reproducibility. To ensure reproducibility, we provide detailed description of our experimental setup in Section including datasets, models, and adversarial attacks, along with their sources. The codebase we adopted for building the AT framework Nichol & Dhariwal (2021), and the our code Traiiiin---iiin-D-..
is released at
LLM Usage. Large language models used exclusively for text polishing and minor exposition refinements. All substantive research content, methodology, and scientific conclusions were developed entirely by the authors
Acknowiedgment. This work was supported by projects PNRR MUR PE0000013-FAIR under the MUR National Recovery and Resilience Plan funded by the European Union- NextGenerationEUU PRIN 2022 project 20227YET9B "AdVVVent" CUP code 353D23012830000 It was also partially supported by Sapienza research projects D2QNeT and BEAT (Better dEep leArning securiTy) bando per la ricerca di Ateneo 2024. Computing was supported by the CINECA clusters under the projects Ge-Di HPIOCRPUVC and RDM HP10C7YY1.2.
REFERENCES
Asad Aali, Marius Arvinte, Sidharth Kumar, and Jonathan I Tamir. Solving inverse problems with score-based generative priors learned from noisy data. In Asilomar Conference on Signals, Systems, and Computers, 2023.
Hervé Abdi and Lynne Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459 2010.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In Proceedings of the 35th International Conference Machine Learning, Proceedings of Machine Learning Research. PMLR, 2018.
Nicholas Carlini, Florian Tramer, Krishnamurthy Dj Dvijotham, Leslie Rice, Mingjie Sun, and J Zico Kolter. (certified!!) adversarial robustness for free! In ICLR, 2023a.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In USENIX Security Symposium, 2023b.
Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11472-11481, 2022.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In ICMI, pp. 1310-1320. PMLR, 2019.
Quan Dao, Binh Ta, Tung Pham, and Anh Tran. A high-quality robust diffusion framework for corrupted dataset. In ECCV, 2024.
Giannis Daras, Yeshwanth Cherapanamjeri, and Constantinos Daskalakis. How much is a noisy image worth? data scaling laws for ambient diffusion. arXiv e-prints, pp. arXiv-2411, 2024a. 202,
Giannis Daras, Mauricio Delbracio, Hossein Talebi, Alex Dimakis, and Peyman Milanfar. Soft diffusion: Score matching with general corruptions. TMLR, 2024b.
Giannis Daras, Alex Dimakis, and Constantinos Costis Daskalakis. Consistent diffusion meets tweedie: Training exact ambient diffusion models with noisy data. In ICML, 2024c.
Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, and Adam Klivans. Ambient diffusion: Learning clean distributions from corrupted data. In NeurlPS, 2024d.

---

Published as a conference paper at ICLR 2026
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In NeurIPS, 2021.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139 144, 2020.
Jiayi Guo, Xingqian Xu, Yifan Pu, Zanlin Ni, Chaofei Wang. Manushree Vasu, Shiji Song, Gao Huang. and Humphrey Shi. Smooth diffusion: Crafting smooth latent spaces in diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge a local nash equilibrium. In NeurIPS, volume 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, volume 33, 2020.
Alain Hore and Djemel Ziou. Image quality metries: Psnr ssim. In 2010 20th international conference on pattern recognition, pp. 2366-2369. IEEE, 2010.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples not bugs, they are features. In NeurlPS, 2019.
Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Guha Thakurta, Nicolas Papernot, et al. Measuring forgetting of memorized training examples. In ICLR, 2023.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In NeurlPS, 2022.
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In CVPR, 2024.
Bahjat Kawar, Noam Elata, Tomer Michaeli, and Michael Elad. Gsure-based diffusion model training with corrupted data. TMLR, 2024.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, CIFAR, 2009.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In ICLR, 2017.
Xiao Li, Wenxuan Sun, Huanran Chen, Qiongxiu Li, Yingzhe He, Jie Shi, and Xiaolin Hu. ADBM:
Adversarial diffusion bridge model for reliable adversarial purification. In ICLR, 2025.
Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A new dataset for deepfake forensics. arXiv preprint arXiv: 0w..:/9999.2000 12962, 2019.
Chumeng Liang and Xiaoyu Wu. Mist: Towards improved adversarial examples for diffusion models. arXiv preprint 11Xx:2305.1268. 2023.
Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song. Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: preventing painting imitation from diffusion models via adversarial examples. In Proceedings of the 40th International Conference Machine Learning (ICML 2023), ICML'23. JMLR.org, 2023.
Guang Lin, Chao Li, Jianhai Zhang, Toshihisa Tanaka, and Qibin Zhao. Adversarial training on purification (ATop): Advancing both robustness and generalization. In The Twelftth International Conference on Learning Representations, 2024. URL https: /ttps/////////ee./ee//ee net/ forum? id=u7559 95w...
12

---

Published as a conference paper at ICLR 2026
Yiming Liu, Kezhao Liu, Yao Xiao, ZiYi Dong. Xiaogang Xu, Pengxu Wei, and Liang Lin. Towards understanding the robustness of diffusion- based purification: A stochastic perspective. In JCLR, 2025.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In ICCV, 2015.
Peter Lorenz, Ricard Durall, and Janis Keuper. Adversarial examples are misaligned in diffusion model manifolds. In 2024 International Joint Conference on Neural Networks (IJCNN, pp. 1-8. IEEE, 2024.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Christopher A Metzler, Ali Mousavi, Reinhard Heckel, and Richard G Baraniuk. Unsupervised learning with stein's 's unbiased risk estimator. arXiv preprint / 10531, 2020.
Mirza Mujtaba Hussain, Briglia Maria Rosaria, Beadini Senad, and Masi Iacopo. Shedding more light on robust classifiers under the lens of energy-based models. In ECCV, 2024.
Andre T Nguyen and Edward Raff. Adversarial Attacks, Regression, and Numerical Stability Regularization. In The AAAI-19 Workshop on Engineering Dependable and Secure Machine Learning Systems, 2019. URL hwtpttts///aaattte //tp/aaaiiv/////////////////aaa. v org/ wwgg/pdf/1810 02885. pdf.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021.
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anandkumar. Diffusion models for adversarial purification. In ICML, 2022.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. TMLR, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, pp. 1 1-42, 2014.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. In NewrIPS, 2016.
Hadi Salman, Andrew Iyas,, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversari- ally robust imagenet models transfer better? In NeurIPS, 2020a.
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and J Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. NeurlPS, 2020b.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In ECCV, 2024.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurIPS, 2019.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffuuion models. In CVPR, 2023,
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021a.
Kaiyu Song, Hanjiang Lai, Yan Pan, and Jian Yin. Mimicdiffusion: Purifying adversarial perturbation via mimicking clean diffusion model. In CVPR, 2024.
13

---

Published as a conference paper at ICLR 2026
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurlPS, 2019.
Yang Song. Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. ICLR, 2018.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In /CLR, 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, and R. Venkatesh Babu. Guided adversarial attack for evaluating and enhancing adversarial defenses. NewrIPS, 2020.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, et al. Towards efficient and effective adversarial training. NeurlPS, 2021.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Julián Tachella, Mike Davies, and Laurent Jacques. Unsure: Unknown noise level stein's unbiased risk estimator. arXiv preprini arXiv:2409.0198; 2024.
Binxu Wang and John Vastola. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications. Transactions on Machine Learning Research, 2024.
Binxu Wang and John J Vastola. Diffusion models generate images like painters: an analytical theory of outline first, details later. CoRR, 2023.
Qixun Wang, Yifei Wang, Hong Zhu, and Yisen Wang. Improving out-of-distribution generalization by adversarial training with structured priors. NeurlPS, 2022.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In ICML,, 2023.
Zekun Wang. Mingyang Yi, Shuchen Xue, Zhenguo Li, Ming Liu, Bing Qin, and Zhi-Ming Ma. Improved diffusion-based generative model with better adversarial robustness. In ICLR, 2025.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Tiange Xiang, Mahmut Yurt, Ali B Syed, Kawin Setsompop, and Akshay Chaudhari. Ddm2: Self-supervised diffusion mri denoising with generative diffusion models. arXiv preprint arXiv:2302.03018, 2023.
Ling Yang, Haotian Qian, Zhilong Zhang, Jingwei Liu, and Bin Cui. Structure-guided adversarial training of diffusion models. In CVPR, 2024.
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma. Xia Hu, and Aidong Zhang. Spurious correlations in machine learning: A survey, arXiv preprint arXiv:2402.1277 127.11 2024.
Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03366 2015.
Cheng Zhang. Kun Zhang, and Yingzhen Li. A causal view on robustness of neural networks. In NeurlPS, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICMI., 2019.
14

---

Published as a conference paper at ICLR 2026
Yonggang Zhang. Mingming Gong, Tongliang Liu, Gang Niu, Xinmei Tian, Bo Han, Bernhard Schölkopf, and Kun Zhang. Adversarial robustness through the lens of causality. In ICLR, 2022.
Yao Zhu, Jiacheng Ma, Jiacheng Sun, Zewei Chen, Rongxin Jiang, Yaowu Chen, and Zhenguo Li. Towards understanding the generative capability of adversarially robust classifiers. In ICCV, 2021.
A APPENDIX
A.1 THEORETICAL CONSIDERATIONS ON ADVERSARIAL TRAINING FOR DIFFUSION MODELS
To craft an appropriate adversarial loss, at first, forward and reverse processes are redefined in light of this further intermediate state. The main aim of performing adversarial training diffusion model is to enhance the robustness capability against adversarial attacks in its reverse process by providing the algorithm with some data that has previously been corrupted. We model this corruption process as an additional chain state, and in this section, we provide theoretical discussion for this assumption.
A.1.1 THE FORWARD PROCESS
The theoretical definition of the DDPM forward process is the following:
a (X1e xo) = II q (x: x)-1), ((x.(x-1) = - -1000)x--00000
where g(xe xx-1) represents the transition probability of the process to move from the state x;-1 at the timestep t - 1 to the state x, at the timestep f. To achieve the aim of integrating the perturbation in the framework, the forward chain can be redefined considering different dynamic of the adversarial forward process. A sample at the time step / is first derived as defined above, and then to it is added an adversarial perturbation do,, that depends on the model's actual state and the value of x,, The overall attack procedure to the model intermediate steps can be represented as concatenation of two transitions. The primary step the ordinary DDPM transition from x;-1 1 X,, which is modeled as q(x, xt-1). The attack transition can be modeled as the step that goes from X, to x; + ô, in the 1-th timestep, being defined similarly as above o' (x; + 5 xz). The two transitions are designed to happen in the same time step t of the chain and, being independent of each other, it is possible to model their interaction as sub-sequence of steps of Markov Chain. This possible since the transition X,-1 + X is already modeled like this and the model (which are constant when crafting the attack, so considerable as constant within the same evaluation) and the value of x, conceived as "previous state". The resulting transition probability o [x; + 6, x4-1) is:
+ 6, depends only on the weights of
d"(x: + 8z xx-1) = d' (xx + 8. xt) g(xe! Xt-1)-
The overall chain can be written
T
d" (xL. + 61.T xo)
II o' (xt + xx) (xe xx-1).
with (xxx(x(-) = (x:; (t) (t)I). Being g(-) a Gaussian transition and being the perturbation addition still modeled Gaussian transition, the DM hypothesis of having only intermediate Gaussian transitions still holds.
A.1.2 REVERSE PROCESS
The reverse process in the diffusion models aims define an algorithm that approximates the forward function and makes it possible to reconstruct the input. In the DDPM formulation, the backward process is defined as:
T
Pe (Xo.T)= =- p (XT) II (Xt-1 xx), Pe (xc-1 1 xx) = N (x--1; p40 (Xr, (), Eg (Xe, 1))
15

---

Published as a conference paper at ICLR 2026
Following the previous substitutions, the desired equivalence when applying perturbations in the forward process would be
xx: + ò, xx-1) = (x: + Ör Xx-1).
that, if considering its approximation, reduces to:
p(x111 x2 + 5τ) p(x; + o, xx-1) p(xx-1)-
This consideration holds also in this case, so if we substitute the objective distributions p(-) with the desired ones we get:
P(X:-1 X2 5;) & p(x:-1) (x + 6, xx-1) p(x:-1) . (x+ + 8, xx) q(x, x(-1).
The above equations hold in case the reverse process is defined in closed form, while in our case the reverse function learned function by pe(-). which is designed and learned properly converge to Paua(X) at a specific timestep 0 of the chain. To properly learn this objective, the network is trained learn regress the amount of noise added in the forward process by minimizing the following simplified objective:
C(xe; 0) - -ox,, 012 where 6 6~ given / E [O,...,T] T..
where q'(x, + ō; xx) represents the transition probability of going from the state the state x Xx + 8, in the same timestep I, the transition from an uncorrupted state to corrupted one modeling available as the distribution depends the kind of through d,- In this case, there attack being performed during the training proces,s but also depends on the state of the model, as the attack is crafted in white box mode:
Sø,t arg max || (x: + 8,1) En (x..
|8
Given the proposed setting, the aim is to define a cost function that allows for modeling the correct Xt-1 when considering the inverted process. The probability distribution that the reverse process needs to learn is:
(x X; + 58,z) x (x
X; + 5g,().
A.1.3 VARIATIONAL LOWER BOUND IN CASE OF PERTURBATION
The Diffusson Models loss function is derived from optimization regarding the variational lower bound. The ELBO defined canonically as:
(x(:))))
log px xx)
log (x))) < E.g
log
q(x:rr xp)
E.g
log(Px,))
Σ
9 x1-1)
=
121
(10)
and the Diffusion Model loss derivation the following:
Po (Xo:))
Eg
log
(X1:T xx)
po (x(-1 xx)
Eg
log (XT)
Σ
log
q (x: x(-1)
Pg (x1-1 xx)
po (xo x1)
Eg
log p (XT)
log
log
q (x, x(-1)
(x1
xo)
Po (x,_1 xx)
9 (x,-1 (xo)
Po (xo xx))
Eg
log (XT)
log q (X(-1
log
X1: X²)
(xi
xo)
9 (x1
xo)
p (Xx))
Po (x+ 1
xx)
E,
log q (XT xo)
log
log Pa (xo x1)
q (xr-1
Xx, Xn)
1>1
16

---

Published as a conference paper at ICLR 2026
= By
DKL (q (XT xo) [lp (xT)) +
Σ
DEL (1 (x--1 xx, xo) lipe (xc-1 xx)) log Pa (xo x1)
t>1
In light of the previous considerations of the forward and backward process, is possible reconsider ELBO derivation as follows:
Po (XxxT)
L Bg
log
9" (x1:T (x0)
Po(xx) [T== ( X; + 5t)
Bgg
log
IT: g(x: x1-1) g'(x + 5z xx.)
Px
Xx + 06)
Eg
((gT(
Σ
log {xx X1-1) g((x; + 8z xx.)
121
po (x xx))
P/ (x
xt + 5z)
Eg
log ((g)()))
log
Σ
log q'(x, + 8 xx)
g(xi x(-1)
121
121
E,
logp(xT)=
Σ log
px
xx)
g(x,-1 X()
a(x(-1 Xp)
<(xe Xo)
1>1
pb(x1
xx + 6c)
g((x xx))
log
g' (xi
Xt + x0)
q'(x + ôz (xo)
t>1
p(XT)
Pa px(x (x:-1 Xx)
log (xT
Σ
log
q
Xo)
g(x 1 X1,
1>1
Px(
X4 + 52)
g'(x: xx))
log
o' (x,
Xt + 81, x d(xx
(x0]
t>1
E. [log po (xo)
xx1= - loog pb (xo
X1 51]]
p(XT)
Po pxxxx11 x,)
Eg
log
Σ
log
gx(x
Xu)
q(x-11 X1, X²)
t>1
px(x1
X1 + 81)
log
log po x1) 1ogob((xo
x1 61)
(11)
g' (xxx
Xt + 8,, xo)
The components to be optimized can be seen as two KL-ddivergences, recalling the formal definition of DDPM optimization. To lower the loss functions the two resulting KL divergences have to be reduced by optimizing both the measure of divergence between the forward x, and the approximated one, by correctly estimating the e and the measure of the noise is added to x; at the timestep This distance measure is represented by the second KL divergence. To transition from the notation q(x, x)-1) to (x,-11 x,, X²) is first necessary to apply Bayes theorem and the chain rule of probability- the exact same reasoning can be used for the second sum.
Start with the conditional probability distribution o' (xe xx-1)-
2. Apply Bayes' theorem express gxx xr-1) in terms of q (x(x11 xx):
g(xe xx-1) q(x,-1 1 xx) q(x.)
((x--1)
3. Now, consider conditioning on an additional variable Xo. According the chain rule of probability, we have:
g(x--1, 1, xx) q(xx-1 xx) g(xx).
17

---

Published as a conference paper at ICLR 2026
4. We want to express ( xx) in terms of well. So, rewrite the joint distribution gxx,_,, X;) as g(xe-1 X1, Xo) q(x,, Xo).
5. Use the chain rule again to break down g(x,, Xp):
g(x,, Xo) (xx Xo) ((x)).
6. Substituting these expressions back into our Bayes' theorem-derived expression, we get:
g(x: xx-1) 9(xx-1
Xx, Xx) g(x, xo)
((xx-1 Xo)
7. Rearrange terms to isolate g(x111 Xx, yielding the desired expression:
g(x: xx-1) q(xx-1 Xo)
q(x,11 Xr, Xo)
g(xe. x0)
A.2 ATTACK FORMULATION
In inference mode, is possible to represent the inverse Markov Chain the sequence of intermediate realizations of Gaussian distributions with fixed parameters regarding mean scaling and variance scaling. From the paper Ho al. (2020) in Eqs. and the 1-th step of the inference can be written as the sampling from the posterior distribution g(xx-1| X,l, xo) = N/ 1x (x,x xo), 3,T), where:
(x,, xo) = var-1o(1) Xo +
aa a1-1) X B. = 1
0 o (t).
This implies that, at each time step, the expected variance and mean of the distribution are defined specific manner. During inference, the value of xo corresponds to the output obtained after the network's prediction. In the context of the DDPM framework, Xo is replaced by the estimated value, which depends on the
network:
X2
(x,)
&
To properly craft the attack and still consider it legitimate, it is essential to scale it to the correct standard deviation to align with the diffusion process. Failing to do so would result in the network's 's inference being affected not by the perturbation itself but by the incorrect range of the perturbation, causing errors due to the inability to maintain the process within its Gaussian assumptions.
In this context, the attack procedure follows the FGSM approach with a random start. However, the perturbation is then scaled to match the appropriate variance timestep f to maintain consistency with the diffusion process. The FGSM attack generates an adversarial example by perturbing the noisy sample x; in the direction of the gradient of cost function with respect to x,- Specifically, the adversarial perturbation is given by:
x = x, + á sign (ign(Vx,44) C(x)))
where e controls the magnitude the perturbation, sign(-) represents the element-wise sign function.
The adversarial attack in this approach is integrated into the diffusion process by leveraging the predictive functions, including a pariance-handling mechanism defined in the model, order to guar- antee concretely adapting the Gaussian hypothesis of the reverse Markov Chain. The adversarial attack begins with perturbing the input x, defining its x/ as:
X X + 8,
a No (t)))
The cost function for the adversarial attack theoretically defined based on the mean prediction:
LFGSM = || (x,, xn) (x'x, xo
where ç4 represents the predicted mean of the diffusion process at time step t, which depends on both the input, respectively the clean sample x, and the adversarial one x', and the original sample
18

---

Published as a conference paper at ICLR 2026
Xp. The optimization goal is to maximize the discrepancy between the predicted means of the clean and adversarial imputs, ensuring that the perturbation effectively disrupts the reverse diffusion process. This cost function, if considered in light of the model's prediction in the e-prediction setting, can be formulated as:
To (x,,
|| (x+ 5,2) Eo (x+, 1)
To compute the adversarial perturbation 6, the gradient of the loss Tg with respect to x is used:
ô = o(t) sign (xx, 6,t))
where (1) scales the perturbation ensure adheres the variance of the Gaussian noise in the reverse diffusion process at the t-th step. This step aligns the adversarial attack with the stochastic nature of the model, ensuring the perturbation remains consistent with the Gaussian hypothesis.
The final adversarial example is then obtained as:
X +
The adversarially perturbed sample x is fed back into the reverse diffusion process, following the recurrence of the inference.
A.3 ITERATIVE ATTACK
In Algorithm 2, we described the attack version that applies single-step attack procedure applied to each and every inference timestep. In this section, we propose a multi-step attack approach based on the PGD iterative attack that, similarly what was described in the previous algorithm, aims to attack model generation at the timestep level. We again highlight that this attack is not intended as practical attack proposed this paper; the main aim of showing this attack approach is provide procedure to assess the abilities of the DM to be resilient against minor perturbations applied to every sampling iteration. In Algorithm 3 we propose the multi-step approach, implemented by applying at every iteration the PGD-20 attack. In this case, being the attack iterative, it is necessary to project at
Algorithm Adversarial Attack on a Diffusson Model.
Input: percentage of attacked timesteps p. total timesteps T, model co, scheduler values a and (t), perturbation strength o, iterations N, the projection operator IP
XT ~ N(0,
for t = T to 0 do
att) t exp(1 log o?)
X;-1 < ED (x),1) 1)
*y - x1 Vl 1a-v
Viee (x( xn) vān-iol!) 2o +
/0. (1.(1- Xt 1-ar
l-a:
6y ~ NO0.00020 (t)
for = 0 to N- 1do
inn + dni
x,,, t cx t)
Xn, + x,n vI axco
Vote
L jx,(x,, xx)
trs (X/ X 112 6,41 = (tt)/N sign V
end for
8 = ((ō, - (t), (t))
x X,
Sample (~ NO0 I)
x f- E0 (x t) + 11x00
end for
the end the perturbation in order keep its values within the range [-o(tt. These values have been chosen following the Gaussianity hypothesis of the intermediate MC states. Diffusion models model intermediate data through intermediate Gaussian distributions where the possible values would
19

---

Published as a conference paper at ICLR 2026
have standard deviation (t). In order not to diverge too much from data distribution and be in suitable range of possible values, we decided to impose as ray of the projection interval the same standard deviation, making it also adaptive to the considered timestep. The table Section 3.4 shows model performance under this PGI)-ike version of diffusion model attack.
A.4 EOT ATTACK
In this section, present PGD-based implementation of the Expectation over Transformation attack (FoT). We build on top of the previously introduced PGD attack (Algorithm 3) in order to define BoT adapted version that would include stochasticity into the optimization of the adversarial noise. We define as the parameter setting the maximum number of samples to approximate the expectation. The implementation adopts the same outline the one adopted by the PGD attack; as
Algorithm 4 EoT Adversarial Attack on Diffusion Model.
Input: percentage of attacked timesteps p, total timesteps T, model Eø, scheduler values or and (1), perturbation strength @, PGD iterations N, the projection operator IP, EoT iterations
XT (), I) for t = T to 0 do 6n (O, ( ( [N,(0
a(t) en (x(x): ore VI-ac
x;-1 Va,
8 ~ (0,,
for n = 0 to N- do
G G-[
x = X; + o
for 1 to do
(~ 10
aade
aade alt)
Es (x(xd ()) + orr
v1-ar
9: Vo. |X, xad ||2
Append to
end for
g Egeg 9
8,t1 P6 + (tt)NN sign(g), -o(t), o(t)
end for
8 5n+1
X Vae + vard
a(t) eo vaid,1) + o.g
vI-o.
end for
consequence, all the previously described details about the attack notation still hold.
A.5 RELATIONSHIP WITH SMOOTH DIFFUSION (GUO AL., 2024)
The method by Guo al. (2024) also introduces method for smoothing the latent space of Diffusion Models. While both papers utilize the term "smothness", we emphasize that the underlying concept of smoothness, its enforcement mechanism, and our primary objectives fundamentally differ. In particular:
Guo et al. (2024) did not demonstrate that their optimization could inherently include being resilient to attacks, focusing more on smooth generation, interpolation, and inverse problems
Guo al. (2024) does not mention any robustness to adversarial attacks
Guo al. (2024) does claim that it could be used train with corrupted data
Table 3 reports the main differences between the two approaches.
20

---

Published as a conference paper at ICLR 2026
Component
Smooth Diffusson
Ours
Arch/Training
Difflc.Diff. + LoRA
UNet with attention + scratch
Equation
|/ (vI ãe #o(e) A2a)
lea ad .t) |co (x, 8]|12
Objective
Reduce gradient norm
Equivariance adversarial under & Resilient to adv. attacks Train corrupted data Faster sampling
Perturbation
normally sampled pix. int. normalized to unit length
Benefit
Smooth Latent
Benefit
Image inversion
Benefit
Stable Interpolation
Table 3: Differences between ours methods and Guo et al. (2024)
B SUPPLEMENTARY MATERIAL
This supplementary material is intended to complement the main paper by providing further motiva- tion for our assumptions and design choices, well additional ablation studies the proposed datasets to demonstrate the effectiivness of our method. is organized into the following sections.
Section C discusses the main differences among the considered approaches, offering deeper analysis that includes both geometrical and empirical motivations behind the adopted design choices. also clarifies the distinction between invariance and equivariance, and presents statisties the adversarial perturbation& o; Section D presents a more detailed analysis of the diffusion flow dynamics by examining the trajectories obtained from low-dimensional datasets under different conditions. Section E provides an extensive qualitative ablation across the real-world datasets introduced in the paper, showcasing a wide variety of samples and comparisons; Section offers additional observations and insights into the proposed approach. We encourage readers to zoom in and compare the results for a better understanding of their quality.
C OBSERVATIONS AND MOTIVATIONS ON OUR ADVERSARIAL TRAINING FRAMEWORK
C.I EQUIVARIANT AND INVARIANT FUNCTIONS FOR ADVERSARIAL TRAINING
Adversarial training in classification has been widely studied over years Goodfellow et al. (2015);
Madry et al. (2018): 8);; Zhang et al. (2019); Wang et al. (2020); Shafahi et al. (2019); Wong et al. (2020); Sriramanan et al. (2021; 2020); Wang al. (2023); Zhu al. (2021); Mujtaba Hussain et al. (2024) in different settings, threat models and under different perspectives. These methods share the objective to enforce invariance in the neural network fo, since the final objective is to enforce the output of the network not to vary in the presence of minor changes in the network input. However, in generative modeling, particularly diffusion models (DMs), enforcing invariance hinders learning the correct distribution, making the model unable to take into account input changes in its prediction. Ignoring the adversarial perturbations applied during a perturbed training leads to deviations in trajectories, resulting in an inaccurate learned distribution. Conversely, training the model to incorporate the negative of the perturbation helps it recognize and manage potential deviations, enabling it to handle noise with broader standard deviations extend the concept of adversarial attacks to regression tasks, even though considering regression tasks on tabular datasets. Their proposed method addresses these attacks by introducing an adversarial training loss based on numerical stability, improving performance under ad adversarial conditions. Even though the latter bridges the concepts of regression and AT, an analysis of implications in the case of randomized and adversarial training applied to the generative model is still topic to cover, particularly with reference to generative models. In this spirit, we propose a new training framework inspired by AT with the aim of shedding light on the concept of adversarial training for DMs, exploiting knowledge from both functional analysis and classification neural networks.
effectively. In Nguyen & Raff (2019), the authors
Formally defining the two properties, we can define both invariance and equivariance. Given function f X + Y. as well as a specified group actions A, f is said to be equivariant with respect to transform E A if and only if
[(aox) = ao [(x), xEX
(12)
21

---

Published as a conference paper at ICLR 2026
Given function f: X Y, as well as specified group action A, f is said to be invariant with respect to an a E A transform if and only
o() ((x), a x X
(13)
In Fig. we extend 1 of the paper and depict what happens at the trajectory level if we enforce invariance instead of equivariance. The vector represents the noise that is added by the diffusion process, represents the added noise by the adversarial training. Finally, we will have two different versions of the noisy point, namely portion of noise equivalent to -e so that is able to correctly go back to Xo- When applying 6, the network's ' objective still has to be the same. The figure shows that if, given the noisy staring point X; + 6, the model is enforced to learn again -c, so if the invariance is applied, the ending point would be some other point in the space different
and X, 6. The model, if unattacked, would like to regress
Xp- On the contrary, if equivariance is applied, the network is forced to regress - (e + 6), making the model able to correctly regress Xo-
Zt + 5
8
INV c (x t)
EQUIV (II,t)
co
370
Figure 9: Not applying equivariance Eg of the space than the desired one, learning then the perturbation that we added as in eNNV. (x..(xx,) ()
EQUIV (x,, t), the model drifts and ends up in different point
C.2 INVARIANCE REGULARIZATION DOES NOT WORK
As empirical evidence of the inconsistency of invariance training in AT for DMs, we prove it on oow-dimeniional data. As proof-of-concee to train on, and then we impose adversarial training, following the same setting as in Algorithm 1, enforcing instead invariance by minimizing the loss function:
consider the oblique-ples 3D dataset as data
LxT(Xe, adv L, E) = arg min
|Eo (x.. <|1 + >ellEa .1)
[EA X1x 211
(14)
8
Com data distrr.
Leep enforce invariance
We decided implement this example on 3D data in order to have the possibility of observing the behavior of 3D trajectories. The plot shows it displaying side-to-side DDPM Ho et al. (2020) and invariance in the same data settings the one displayed in Fig. 2. The model, by enforcing invariance, loses the ability to correctly reconstruct the data manifold, not being able to generate points in the data distribution, whereas the model trained through standard DDPM learns the data distribution but still suffers from learning the noise in case of noisy data. The same behavior can be observed when looking at the trajectories. This analysis clarifies even more what the generation dynamics. The model creates sparse trajectories that do not tend to be clustered, neither at the beginning of the generation nor at the end, thereby causing generated samples to be completely off the data subspace.
A comparison between the plots in Fig. 10 and those in Fig. 11 further emphasizes the benefits of adversarial training with equivariance. The contrast shown in the compared trajectories clearly illustrates how approach consistently produces trajectories that more clustered, sharper, and better aligned with the underlying data manifold, thereby reinforcing the inadequacy of conventional adversarial training methods.
22

---

Published as a conference paper at ICLR 2026
Poor intler nolse
Strong inlier noise
Outliers
Model trajectories
Ho et al (2020)
Inv. adv
Figure 10: The application of invariance on 3D data highlights the incorrect behavior of the training procedure: the learnt data distribution is completely different from the reference one.
C.3 NOTES ON DEFINITION OF THE ADVERSARIAL PERTURBATION
One of the main points of our work is defining a suitable perturbation 5 for unconditional diffusion models that aims at disrupting generation trajectories without relying on acting the model's inputs. In order to craft this kind of attack, we focused on exploiting generation dynamics in order to correctly perturb it each of its steps. Inspired by adversarial attacks with random start (such as R-FGSM Wong et al. (2020)), 5 is first initialized by randomly sampling from uniform distribution, whose bounds are (-r r-]-)]- The initialization distribution is chosen to uniform distribution with varying bounds but always centered at zero. This choice ensures that the perturbation has zero mean, which essential when applied within the diffusion process. A non-zero mean would not only bias the estimation of the noise but also violate the Gaussian transition assumption, which requires the noise to be zero-centered. The parameter is sampled from uniform distribution as follows ~ (0,5, 2]. Its aim is enhance the model's 's robustness trajectory deviations by randomly varying the perturbation's bounds. Once the perturbation bounds are defined, straightforward to calculate the standard deviation of the initialization distribution. The mean value of is given by:
B[d] ((-ra()) (ra())) 0.
2
The variance of the distribution is defined as:
VAR [) (--ra()) (ra(-))1² (2rx(-))² rs(-)²
12
12
3
The variance is consequently defined as a 3-dependent quantity as rescaled batch-wise by this parameter, assuring a random dynamic change of the perturbation bounds.
23

---

Published as a conference paper at ICLR 2026
D COMPREHENSIVE ANALYSIS OF THE DIFFUSION FLOW DYNAMICS
In order to better understand data behavior during the generation procedure, we report in this section further trajectory plots. The plots can only be visualized if the data taken into account is low-dimensional in order to properly track points' behavior in the generation. We exploit the low-dimensional datasets proposed in the paper to further investigate trajectory behavior. Additional qualitative samples of the Diffusion Flow are shown in Fig. 11, supplementing Fig. 4 and Fig. 6, which can be found in the main paper. Unlike this one, here we have the chance to show also the difference between models' behavior when data distribution is affected by strong inlier noise and outliers on both unimodal distribution obl i que- quue-plare ane and 3-gauss ians.
The plot shows that even though the DDPM model reaches the distribution of the final part of the trajectories, those are sparse and, even in the case of inlier noise, they appear not to be densely clus- tered, with some completely diverging from the data distribution. When applying the regularization, particularly in this case, the model is trained with adversarial noise, the density increases in the trajectories, defining sharper and clustered paths, strongly discouraging significant deviations from their central modes. This feature is especially useful when the initial data distribution is noisy, as it helps the model avoid learning erroneous points that stray from the true data distribution, preventing it from capturing the noise present in the starting data. In particular, when outlier noise is present, regularization minimizes its influence, resulting in denser and sharper trajectories that better align with the true data distribution clusters.
E ADDITIONAL. QUALITATIVE SAMPLES UNDER MULTIPLE SETTINGS
E. TRAINED ON CLEAN CIFAR-10 WITH| = 0%
DDPM vs Robustad. In Fig. 12 of this supplementary material, we extend Fig. in the paper and show 300 samples from DDPM vs 300 samples from Robuststv, both trained on the original dataset. Although our method has not been designed to work directly with uncorrupted data, the images that ours generates result in smooth images, the clutter in the background has been canceled, yet the objects and animals are still clearly recognizable, and part of the noise in the background of CIFAR- 10 has been removed. We think that it is reasonable justify the drop with our method denoising action, which is, for example, the removal of part of the characteristic background noise proper of CIFAR-10. This effect can be the reason for the evaluation penalizing us.
have in the FID
500 vs 1000 steps. We expand the current section by including some samples that focus on enriching the paper's 's discussion about faster sampling. In Fig. 13 we offer on the left the results by Robustady with 1000 inference steps trained on uncorrupted data. On the right instead, we show the qualitative samples still with Robustady yet using scheduler with 500 inference steps, thereby cutting 50% of the inference time, Surprisingly, the faster sampling yields better FID. We get 28.68 FID with 1000 steps and 24.34 with 500 steps. In terms of differences, taking more steps generates images with warmer and natural colors, whereas taking fewer steps seems to improve the details of the objects, and the colors look brighter and saturated, probably being closer to the actual CIFAR-10 images.
E.2 TRAINED ON NOISY CIFAR-10 WITH p = 90%, o = 0.1
In Fig. 14 of this supplementary material, provide additional figures not present in the main paper. The figures show 300 samples from DDPM vs Robust, both trained on noisy CIFAR-10 10 with p 90%, inheriting the smoothing effect of the previous setting. In this case, the smoothing action helps absorb the Gaussian noise present in the dataset. This results in improved performance: unlike DDPM (left), ours is able to unleam the noise and keep images still with natural colors.
0.1. The images that ours generates (right) are smooth, similar the one in Fig. 12,
E.3 TRAINED ON NOISY CIFAR-10 WITH p- 90%, 0.2
In Fig. 15 of this supplementary material we extend Fig. 8 of the paper, enriching it with 300 more samples per method yet trained on noisy CIFAR-10 with p 90%, deviation of the added noise, in this case σ = 0.2 represents a very strong one: it means we are adding 40% of the variability that is naturally present in CIFAR-10, being daam 0.5. Despite the
0.2. Looking at the standard
24

---

Published as a conference paper at ICLR 2026
DDPM Ho et al. (2020)
Robustady
Inlier noise
oblique plane
Outlier noise
Inlier nnlse noise
3 g-guussians
Outlier noise
Figure 11: Diffusion flow: DMs vs Robustaav Left column shows the results by Ho al. (2020) under two different types of noise. Regular training tends to incorporate the noise inside the diffusion flow, making more prone to generate undesirable and unexpected results; Right column Robustady that trades off variability for resilience. Indeed, heatmaps on the right are concentrated, clear, and less faded.
25

---

Published as a conference paper at ICLR 2026
clean CIFAR-10 with p 0%
DDPM Ho al. (2020)
Robustady
Figure 12: Trained on clean CIFAR-10 with p 0%. Despite the FID decreases once trained on clean data, generated images by Robustady look smooth, and the clutter in the background has been canceled.
26

---

Published as a conference paper at ICLR 2026
clean CIFAR-10 with p= = 0%
Robustadv w/ 1000 steps (28.68 FID L)
Robustaiv w/ 500 steps (24.34 FID L)
Figure 13: Trained on clean CIFAR-10 with p= 0% but comparing less steps (500) vs the default
DDPM scheduler used for training (1000). Although Robustag with scheduler with fewer steps (500) and do not use it in training, the images on the right with 500 steps have better FID than with the original scheduler on the left.
27

---

Published as a conference paper at ICLR 2026
noisy CIFAR-10 with p 90%, a 0.1 DDPM Ho al. (2020)
Robustady
Figure 14: Trained on noisy CIFAR-10 with p 90%, 0.1. Despite added noise, Robustad images look smooth, and the clutter in the background has been canceled along with the Gaussian noise added. Instea,d DDPM on the left propagates the noise back the output.
28

---

Published as a conference paper at ICLR 2026
strong ambient noise, the images that ours generates (right) smooth similar to the one in Fig. 12 and presence of the strong Gaussian is very rare. Unlike DDPM (left), ours is able to unlearn the noise and keep images still with natural colors.
E.4 TRAINED ON NOISY CELEB-A WITH p 90%, 0.1
We provide a more extensive qualitative analysis on the dataset CelebA Li et al. (2019) in Fig. 16 of this supplementary material. To further motivate the denoising effect, we here show same 300 samples per method yet trained on noisy Celeb-A with generates (right) are smooth, but now instead of absorbing the Gaussian noise present the dataset, unlike DDPM fl), ours is able to unlearn the noise and keep images still with natural colors.
90%,
0.1. The faces that ours
E.5 TRAINED ON NOISY CELEB-A WITH p 90%, 0.2
We provide a more extensive qualitative analysis on the dataset CelebA Li et al. (2019) in Fig. 17 of this supplementary material. To further motivate the denoising effect, we here show same 300 samples per method yet trained on noisy Celeb-A with generates (righf) are smooth but now instead of absorbing the Gaussian noise present in the dataset, unlike DDPM (le ours is able to unlearn the noise and keep images still with natural colors.
90%, = 0.2. The faces that ours
E.6 TRAINED ON NOISY LSUN BEDROOM WITH 90%, = 0.1
We provide a more extensive qualitative analysis on the dataset LSUN Bedroom Yu et al. (2015) in Fig. 19 of this supplementary material. To further motivate the denoising effect, here show same 150 samples per method yet trained on noisy LSUN dataset with 90%, o = 0.1. The generated images by Robustady (righ() result be smoother wrt. the datasets ones and the DDPM generated ones (leff) ones, but the smoothing effect allows absorbing the Gaussian noise present in the dataset: unlike DDPM, ours able to unlearn the noise and keep images still with natural colors.
E.7 TRAINED ON NOISY LSUN BEDROOM WITH p 90%, 0.2
We further enrich the qualitative ablation the dataset LSUN Bedroom Yu et al. (2015) in Fig. 20 of this supplementary material. To further motivate the denoising effect, we here show the same 150 samples per method yet trained on the noisy LSUN dataset with p 90%, images by Robustaa (right) result to be smoother wrt. to the datasets ones and the DDPM generated ones (left) ones, but the smoothing effect allows absorbing the Gaussian noise present in the dataset: unlike DDPM, ours able to unlearn the noise and keep images still with natural colors.
0.2. The generated
29

---

Published as a conference paper at ICLR 2026
noisy CIFAR-10 with p 90%, a 0.2 DDPM Ho al. (2020)
Robustady
Figure 15: Trained on noisy CIFAR-10 with p = 90%, 0.2. Despite added noise, Robustad images look smooth and the clutter in the background has been canceled along with the Gaussian noise added. Instead DDPM on the left propagates the noise back in the output.
30

---

Published as a conference paper at ICLR 2026
noisy Celeb-A with p 90%, a 0.1 DDPM Ho et al. (2020)
Robustadv
Figure 16: Trained on noisy Celeb-A with p = 90%, 0 0.1. Despite added noise, Robustady faces look smooth and the clutter in the background has been canceled along with the Gaussian noise added. Instead, DDPM on the left propagates the noise back the output.
31

---

Published as a conference paper at ICLR 2026
noisy CelebA with p 90%, o 0.2 DDPM Ho al. (2020)
Robustady
Figure 17: Trained on noisy Celeb-A with p = 90%, o a= 0.2. Despite added noise, Robustadv faces look smooth, and the clutter in the background has been canceled along with the Gaussian noise added. Instead, DDPM on the left propagates the noise back in the output.
32

---

Published as a conference paper at ICLR 2026
LSUN Bedroom early stage training
DDPM Ho al. (2020)
Robustady
Figure 18: Trained on clean LSUN Bedroom. Despite the added noise, Robustady produces images that appear smooth and exhibit fewer intricate details. When noise is absent from the training data, this smoothing effect results in the removal fine-graine greegga information from the learned distribution, ultimately reducing data variability.
33

---

Published as a conference paper at ICLR 2026
LSUN Bedroom with p 90%, 0 0.1, early stage training
DDPM Ho et al. (2020)
Robustady
Figure 19: Trained on noisy LSUN Bedroom with y 90%, o= 0.1. Despite added noise, Robustady images look smooth and with fewer intricate details that have been canceled along with the Gaussian noise added. Instead, DDPM on the left propagates the noise back into the output.
34

---

Published as a conference paper at ICLR 2026
LSUN Bedroom with p 90%, o 0.2, early stage training
DDPM Ho et al. (2020)
Robustadv
Figure 20: Trained on noisy LSUN Bedroom with = 90%, σ = 0.2. Despite added noise, Robustady
images look smooth and with less intricate details that have been canceled along with the Gaussian noise added. Instead, DDPM on the left propagates the noise back into the output.
35

---

Published as a conference paper at ICLR 2026
LSUN Bedroom late training
DDPM Ho al. (2020)
Robustadv
98
Figure 21: Trained on clean LSUN Bedroom. Despite added noise, Robustady images look smooth and with less intricate details, even though more detailed than in earlier stages.
36

---

Published as a conference paper at ICLR 2026
LSUN Bedroom with p 90%, 0 0.1, late stage training
DDPM Ho et al. (2020)
Robustadv
Figure 22: Trained on noisy LSUN Bedroom with = 90%, a 0.1. With extended training. Robustaiy not only effectively removes the noise introduced into the dataset contrast DDPM but also restores fine details, resulting in multi-view images with natural colors and enhanced realism.
37

---

Published as a conference paper at ICLR 2026
LSUN Bedroom with p 90%, 0 = 0.2, late stage training
DDPM Ho al. (2020)
Robustady
Figure 23: Trained noisy LSUN Bedroom with p- 90%, 0.2. With extended training, Robustadv effectively removes the noise introduced into the dataset, contrast to DDPM. As result, it produces cleaner images, albeit with less intricate details.
38

---

Published as a conference paper at ICLR 2026
F ADVERSARIAL TRAINING ANALYSIS
This method aims at proposing an AT approach to the diffusion model's training whose design choices have been motivated extensively previous sections well in the main paper. this section, we want to highlight some interesting points we observed during the framework formulation.
F.1 TRAINING DYNAMICS
Adversarial training diffusion models inevitably influences DM training dynamics. Indeed, the proposed regularization acts as smoothing factor for the diffusion process in the trajectory space. In order to evaluate the training dynamics, we propose an ablation on DM generated samples at different training iterations. Fig. 24 is intended show the evolution of generated samples by AT models at different training iterations. The first row shows samples generated by models trained in an early stage, while the second shows generations from models trained for longer. On the right column, the dataset has not been corrupted; the generations, after more training iterations, start losing the bright colors, tending towards more natural-looking colors. Moreover, the generated data starts acquiring its details. The same effects can be seen for models trained on corrupted data, = 0.1 in the middle and a 0.2 on the right (both with p = 0.9%). In those cases, generated samples, which at earlier epochs still resulted in being noisy, are completely denoised. This dynamic suggests that the model first focuses on fitting the overall data model, focusing more on smoothing effect. Once done, the model goes back to learning the details of the data distribution, including some variability, but still not taking into account the noise present the data. Furthermore, clearer picture of the training dynamics can be obtained by examining images Figs. 18 to 23, that effectively compare the robust approach with the DDPM model at both early and late training stages.
also possible see that some
F.2 FINETUNING ANALYSIS
Adversarial training notably introduces training time overhead. In our case, the increased training time is an investment for improved robustness and faster inference, which is particularly relevant in real-world pipelines where inference is repeated continually, while training is performed once for all. This is standard trade-off in modern generative modeling, as seen in classifier guidance, which also increases training and complexity but widely adopted.
In this section, present preliminary results concerning the application of adversarial training in the fine-tuning setting. We point out that fine-tuning does help to alleviate the problem of training cost, so it could be a strong future improvement to allow the application of AT to heavier training pipelines. The table below shows FID and IS results evaluated on the CelebA dataset. For the fine-tuning, the model has been trained for all it training epochs according to the DDPM framework, except for the last 100 ones, when the adversarial regularization loss was applied. The table Table 4 showcases finetuning evaluation results. If compared with results in Table we observe that have similar results to the paper but with a fraction of the computational time.
Configuration ////a
Fine-tuned
From Scratch
DDPM
Robustatv
DDPM
Robustad
0.9
/ 100.68 /2.7
2,13/2.1 25.8/2.1
54.90 2.40
14.54/2.00
0.9
96.03 / 2.65
16.53 2.11
Table 4: Performance comparison (FID/IS) between DDPM and Robustady when finetuning on CelebA
39

---

Published as a conference paper at ICLR 2026
LSUN Bedroom
LSUN Bedroom ø 0.1
LSUN Bedroom 0.2
eazzy st.age wtraiitttaainn training
L.SUN Bedroom
LSUN Bedroom σ 0.1
LSUN Bedroom 0.2
late stagge training
Figure 24: Qualitative results analysis on samples generated by Robustaay at different training stages.
F.3 How À IN EQ. (9) OF THE MAIN PAPER INFLUENCES THE MODEL'S DENOISING CAPABILITY
In the method section, we stress that the choice of the hyperparameter À heavily influences the model's smoothing ability. To further motivate the previous statement, we provide straightforward evidence of this by observing generated samples produced by different models, with the same architectures and minimum regularization ray among all the shown samples. The varying parameter is X, which
40

---

Published as a conference paper at ICLR 2026
LSUN Bedroom
LSUN Bedroom ø 0.1
LSUN Bedroom 0.2
T0
A-o 220
Figure 25: Robustady trained on L.SUN Bedroom dataset, with different noisy data (p 90%, different are visible in the image). The first row sets the regularization hyperparameter À to 0.1, the second to 0.2.
set to the values {0.1, 0.3, Fig. 25 shows the results at an early training stage of the models. Despite being at early stage, the À influence in models' performance already appears clear. When the data is not noisy (first column), increasing its value results in oversmoothing data, losing subject details, due to the smoothing factor introduced by the regularization. When the data becomes noisy, the regularization becomes fundamental learning the correct distribution. the first row, we see that the smoothing action is limited due the small À 0.1, indeed the noise is still present in
41

---

Published as a conference paper at ICLR 2026
the generated samples both in o 0.1 and 0.2, whereas the noise decreases drastically when increasing À to 0.2. In fact, the images shown in the bottom row show a minor presence of noise, which is expected to disappear in later training. On the other side, the increase of the parameter À also causes a loss of details in the image subjects. This phenomenon is due to the smoothing effect, which not only affects noise but also data variability. This smoothing effect becomes when compared to Figs. 18 to 23, all generated with À = 0.3. These comparisons further support the previous observations by extending the analysis across different levels of noise and training stages.
more apparent
G COMPARISON WITH NOISE-AWARE DIFFUSION TRAINING
The primary objective of Daras al. (2024a) develop noise-informed algorithms for training models in the presence of noisy training data. More in detail, the noise-informed training algorithm operates under two fundamental assumptions: Gaussian and prior knowledge of the Gaussian variance, and (ii) identification of the specific training samples affected by noise corruption. To rigorously assess the robustness of this approach under the unknown corruption setting, we developed two distinct training configurations that relax these stringent assumptions and modify the original training framework of Daras et al. (2024a).
the assumption that the noise the dataset is
In Conf. I, the method always knows the exact a level of the noise in the dataset but the assumption which sample is clean x E Xetean and which sample is noisy x E Kuoisy is forced to be correct only (1 - % of the time.
In Conf. 2, the assumption of knowing whether sample is noisy or not is never considered, effectively forcing the same behavior for all the training data, when the data are noised with probability p.
For both configurations, we trained the models on CIFAR-10, considering clean data and noisy data with (0.1, 0.2) and p 90%. Quantitative results summarized in Table 5 while qualitative examples are shown in Fig. 26.
p %
Robustad
Daras et al. (2024a) Conf.
Daras et al. (2024a) Conf. 2
0
28.7
14.0
14.9
0.9
0.1
24.7
94.5
102.7
0.9
0.2
24.8
109.7
105.3
Table 5: Experiments on CIFAR-10 under unknown corruption.
These experiments confirm that relaxing even one the assumptions made in noise-aware solution proposed by Daras et al. (2024a) reduces the method's robustness unknown noise in the data, producing very high FID values. This confirms the practical limitations already highlighted by Daras et al. (2024a). On the contrary, our method is able work in this more challenging setting, where the corruption is unknown, and achieves a stable trend in the FID across different a and without requiring access to clean/noisy labels or corruption parameters.
H LLM USAGE
Large language models were used exclusively for text polishing and minor exposition refinements. All substantive research content, methodology, and scientific conclusions were developed entirely by the authors.
42

---

Published as a conference paper at ICLR 2026
CIFAR-10
CIFAR-I0 = 0.9, = 0.1
CIFAR-10 p = 0.9, = 0.2
Robustad
Daras et al. (2024a) Config
Daras et al. (2024a) Config. 2
Figure 26: Robustadv trained on CIFAR- 10 dataset compared with the two proposed configurations of Daras et al. (2024a) in the unknown noise setting. Training data are either clean (first column) or noised with different noise levels (p 0.9, a (.2),, The images demonstrate that Robustady effectively learns the target data distribution, ignoring the applied noise, even without making any assumptions about the applied perturbation.
43


Reviews:
[{"review_id": "umWpikQ3KK", "reviewer": "Reviewer_Djvf", "summary": "The authors argue that adversarial training (AT) for classifiers, which enforces invariance of predictions under small input perturbations, does not transfer directly to diffusion models (DMs), whose training objective is regression to injected noise. They propose that AT for DMs should instead enforce local equivariance of the noise predictor: when the trajectory state $x_t$ is perturbed by $\\delta$, the predicted noise should change by (approximately) $\\delta$. Formally, they add to the standard DDPM loss an equivariance regularizer (Eq. (6)) and train either with random perturbations or adversarial FGSM‑style perturbations. A time-dependent perturbation radius $r_{\\beta}(t)$ is scheduled to respect the forward‑process variance. The adversarial step uses a variance-aware projection.\n\nEmpirically, on a synthetic 3D and a linearized butterflies dataset, the method produces tighter score fields and trajectories, better PSNR and subspace recovery under corruption. On CIFAR‑10, CelebA, LSUN Bedroom, when 90% of the training data are corrupted with Gaussian noise, Robustadv reduces FID versus DDPM/DDIM. Conversely, when training on clean data, Robustadv degrades FID with smoother but less detailed images. The paper also reports fewer near-duplicates by DINOv2 similarity, better tolerance to trajectory-space attacks and sometimes better FID.", "questions": "1. Can you compute-matched comparisons to noise-aware diffusion training methods on CIFAR-10/CelebA/LSUN for $p \\in \\{ 0.3, 0.6, 0.9 \\}$ and $\\sigma \\in \\{ 0.05, 0.1, 0.2 \\}?\n2. The evaluation in terms of the robustness to adversarial attacks are quite limited and do not match the common practice in the adversarial defense community. Can you evaluate EOT over diffusion stochasticity and adaptive attacks.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper carefully argues that classifier-style invariance is misaligned for diffusion’s noise-regression objective, and instead enforces local equivariance of the score under trajectory perturbations; the formulation is explicit and contrasted with a naive invariance loss, with toy-world visual evidence that invariance drifts off-manifold\n2. The training recipe is simple and can be seen as a plug-in. The proposed regularizer can be easily integrated with standard DDPM training. The time-dependent perturbation radius and adversarial update are demonstrated clearly.\n3. With 90% of training samples noised ($\\sigma = 0.1/0.2$), Robustadv sharply improves FID over DDPM/DDIM on CIFAR-10/CelebA/LSUN, which are supported by qualitative samples as well.", "weaknesses": "1. The paper positions itself against noise-aware diffusion training but does not compare to them experimentally. This leaves competitiveness under “unknown corruption” unestablished.\n2. The penalty on clean data is substantial yet under-analyzed. With $p = 0\\%$ noise, FID worsens markedly. The authors do not provide a trade-off analysis in depth.\n3. The ELBO derivation with adversarial sub‑steps (Appendix A.1.3) is heuristic and does not yield quantitative guarantees in theory; also, assumptions about Gaussian intermediate states under non‑Gaussian perturbations are not formalized.\n4. The perturbation radius $r_{\\beta}(t)$ and scaling of $\\lambda_t$ are primarily hand-crafted. No principled guidance on stability/consistency or dataset-noise dependence, which should have been provided, given that the adversarial training recipe may have altered the optimization landscape of the diffusion model.\n5. The training is ~2.5x slower than standard diffusion model training, which is however already notoriously slow. In practice, this computational overhead may be unacceptable, especially provided with the scale of data used to train a diffusion model successfully.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6VdQruVXN3", "reviewer": "Reviewer_2wFr", "summary": "This paper proposes an adaptation of Adversarial Training (AT) for Diffusion Models, framing it as an equivariance constraint aligned with denoising dynamics. The method aims to improve model robustness against noise, data corruption, and adversarial attacks, while also reducing memorization. The authors provide empirical validation on both low-dimensional synthetic data and standard image benchmarks.", "questions": "1. What is the estimated computational overhead (training time, memory) of your adversarial training method compared to standard diffusion model training?\n\n2. Have you explored the trade-offs between the \"random perturbation\" (smoothing) and \"adversarial perturbation\" variants of your method in terms of robustness gain vs. computational cost?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Originality: Successfully adapts Adversarial Training, a discriminative technique, to the generative setting of Diffusion Models via a novel equivariance constraint.\n\nQuality & Significance: Comprehensive experiments show the method confers clear advantages: robustness to noise/corruption, reduced memorization, and security against adversarial attacks. This is highly relevant for safe and reliable deployment of DMs.", "weaknesses": "- The paper focuses on proof-of-concept and standard benchmarks. Testing on more complex, large-scale datasets (e.g., ImageNet) would further strengthen the claims of general applicability.\n\n- The computational overhead of adversarial training during the diffusion process is not thoroughly discussed.\n\n- related work: The related work section lacks engagement with several key areas of relevant research. This includes: 1) foundational work on robust denoising autoencoders [1], 2) the parallel and distinct line of research on adversarial purification using diffusion models [2], and 3) recent analysis specifically on the nature of adversarial vulnerabilities in DMs [3]. Addressing these would better contextualize the paper's contributions. I think that is not much work for the rebuttal.\n\n\nReferences\n1. \"Time-based Sampling and Reconstruction of Non-bandlimited Signals\" - https://ieeexplore.ieee.org/document/8682626\n2. \"Diffusion Models for Adversarial Purification\" - https://arxiv.org/abs/2205.07460\n3.  ''Adversarial Examples are Misaligned in Diffusion Model Manifolds'' - https://arxiv.org/abs/2401.06637", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SAtWejA4ZK", "reviewer": "Reviewer_PUAf", "summary": "This paper proposes an adversarial training framework for diffusion models that incorporates adversarial regularization directly into the denoising objective to improve robustness against noisy training data and input perturbations during the diffusion process. The proposed method (Robustadv) enforces equivariance by ensuring that small perturbations to noisy samples lead to proportionally consistent responses, while the perturbation magnitude is adaptively bounded to preserve the diffusion process’s stability. Experiments on synthetic and real datasets such as CIFAR-10, CelebA, and LSUN Bedroom demonstrate that the method enhances resistance to corruption, reduces memorization, and achieves smoother diffusion trajectories that allow faster sampling, all while maintaining high image quality even under heavy noise conditions.", "questions": "- Given that the proposed objective enforces local smoothness in the score field, could this formulation be extended to improve adversarial purification or other defense mechanisms?\n- Would the proposed adversarial regularization also improve robustness when training on datasets with natural corruptions?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper introduces a novel perspective on adversarial training for diffusion models by shifting the focus from classifier-level invariance to score-field equivariance. This insight effectively redefines robustness in the context of generative diffusion processes. \n\n2. The proposed method is simple yet well grounded, requiring only a small modification. The time-dependent perturbation bound maintains compatibility with the diffusion process.\n\n3. The paper presents extensive experiments across synthetic and real-world datasets, clearly showing improvements in robustness against noisy training data, reduced memorization, and smoother diffusion trajectories. The analysis of faster sampling and resilience to trajectory attacks further supports the method’s practical value.", "weaknesses": "The paper is overall well executed, with clear motivation, principled formulation, and comprehensive experiments. The only minor weakness is that the evaluation focuses mainly on DDPM and DDIM frameworks, leaving open whether the proposed adversarial regularization generalizes to newer diffusion models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)