NatADiff: Adversarial Boundary Guidance for Natural Adversarial Diffusion
Authors: Max Collins
Jordan Vice
Tim French
Ajmal Saeed Mian
Venue: ICLR 2026 Poster (2026)
Keywords: adversarial attacks
denoising diffusion models
natural adversarial samples
model robustness
test-time errors
Primary area: interpretability and explainable AI
TL;DR: We propose NatADiff, which uses diffusion models to generate realistic, transferable adversarial examples that better resemble real-world model errors than traditional generative attacks.

Abstract: Adversarial samples exploit irregularities in the manifold "learned" by deep learning models to cause misclassifications. The study of these adversarial samples provides insight into the features a model uses to classify inputs, which can be leveraged to improve robustness against future attacks. However, much of the existing literature focuses on constrained adversarial samples, which do not accurately reflect test-time errors encountered in real-world settings. To address this, we propose `NatADiff', an adversarial sampling scheme that leverages denoising diffusion to generate natural adversarial samples. Our approach is based on the observation that natural adversarial samples frequently contain structural elements from the adversarial class. Deep learning models can exploit these structural elements to shortcut the classification process, rather than learning to genuinely distinguish between classes. To leverage this behavior, we guide the diffusion trajectory towards the intersection of the true and adversarial classes, combining time-travel sampling with augmented classifier guidance to enhance attack transferability while preserving image quality. Our method achieves comparable white-box attack success rates to current state-of-the-art techniques, while exhibiting significantly higher transferability across model architectures and improved alignment with natural test-time errors as measured by FID. These results demonstrate that NatADiff produces adversarial samples that not only transfer more effectively across models, but more faithfully resemble naturally occurring test-time errors when compared with other generative adversarial sampling schemes.

Published as a conference paper at ICLR 2026
NATADIFF: ADVERSARIAL BOUNDARY GUIDANCE FOR NATURAL ADVERSARIAL DIFFUSION
Max Collins
Jordan Vice
School Physics, Maths and Computing
School of Physics, Maths and Computing
The University of Western Australia
The University of Western Australia
Perth, WA 6009
Perth, WA 6009 jordan. vice@uwa.
co..coolinne www.ooonnnnnneeooooooooo
uwa edu
Tim French
Ajmal Mian
School of Physics, Maths and Computing
School of Physics, Maths and Computing
The University of Western Australia
The University of Western Australia Perth, WA 6009
Perth, WA 6009
tim. French@uwa.ccommm edu.au
ajmal wwt.miaa@@uu. uwwa edu, au
ABSTRACT
Adversarial samples exploit irregularities in the manifold "learned" by deep learn- ing models to cause misclassifications. The study of these adversarial samples provides insight into the features model uses to classify inputs, which can be leveraged to improve robustness against future attacks. However, much of the existing literature focuses on constrained adversarial samples, which do not aceu- rately reflect test-time errors encountered in real-world settings. To address this, we propose 'NttADiff', an adversarial sampling scheme that leverages denoising diffusion to generate natural adversarial samples. Our approach is based the observation that natural adversarial samples frequently contain structural elements from the adversarial class. Deep learning models can exploit these structural el- ements to shortcut the classification process, rather than learning to genuinely distinguish between classes. To leverage this behavior, guide the diffusion trajectory towards the intersection of the true and adversarial classes, combining time-travel sampling with augmented classifier guidance enhance attack transfer- ability while preserving image quality. Our method achieves comparable white-box attack success rates to current oftte-ofthh-art techniques, while exhibiting signifi- cantly higher transferability across model architectures and improved alignment with natural test-time errors as measured by FID. These results demonstrate that NatADiff produces adversarial samples that not only transfer more effectively across models, but more faithfully resemble naturally occurring test-time errors when compared with other generative adversarial sampling schemes.
INTRODUCTION
Deep learning models can react unpredictably when there is domain difference between training and test data (Szegedy et al. 2014 Goodfellow et al., 2015). Constrained adversarial attacks exploit this vulnerability, adding visually imperceptible pixel-level perturbations deliberately fool models into misclassification (Szegedy et al.) [2014] Goodfellow et al.] 2015 Madry et al. 2018 Croce & Hein 2020). More recently, unconstrained adversarial attacks have been proposed which allow for unrestricted perturbation magnitudes, provided the resulting adversarial image lies sufficiently close to the natural image manifold (Song et al., 2018 Chen et al.) 2023a b).
Defences to these attacks have been proposed (Szegedy et al. 2004 Madry et al. 2018 Gu & Rigazio 2013) Xu et al. 2018: (Samangouei et al. 2018 { Nie et al. (022): however, they largely target attacks formed by adding perturbations natural images-overlookingg the existence of natural adversarial

---

Published as a conference paper at ICLR 2026
samples. Natural adversarial samples more commonly known as test-time errors, and they represent the strongest class of unconstrained adversarial attack. as they are valid (perturbation free and naturally occurring) model inputs that are erroneously classified (Hendrycks et al.] [02IT. The absence of an adversarial perturbation renders many defensive measures ineffective (Agarwal et al.) 2022). Furthermore, natural adversarial samples have been widely studied in the literature, and they have been found to exhibit high transferabiliy-whher multiple classifiers incorrectly classify the same sample (Hendrycks et al., 202T). It is hypothesized that this caused by classifiers independently learning to rely on the same erroneous contextual cues to shortcut classification and reduce training losses without generalising to the underlying task (Hendrycks et al., 2021 Geirhos et al., 2020
Arjovsky et al.) 2020).
Generating natural adversarial samples offers an opportunity to better understand the mechanisms underpinning test-time errors. Prior work has sought to achieve this by using classifier gradients to perturb the sampling process of generative adversarial networks (GANs) and denoising diffusion
models (Song et al. 2018 Chen et al. 2023b Dai et al. 2024 Chen et al. 2025). However, GAN-based approaches lack theoretical justification for perturbing the sample path, and doing so often degrades image quality (Karras et al.) 2019 Abdal et al.) 2019). Alternatively, directly injecting classifier gradients into the diffusion sampling trajectory can result in generating constrained adversarial samples ( (Vaeth et al., 2024, Shen al. 2024) (see Figure (c)). Moreover, existing methods do not account for the link between learned erroneous contextual cues and test-time errors.
We propose NatADifff. highly transferable, diffusion-based (Ho et al. 2020: Song al. 202
adversarial sample generation method. NatADiff leverages the link between contextual cues and test- time errors by guiding the diffusion sampling trajectory towards the intersection of the adversarial and true classes, a technique we define as "adversarial boundary guidance". Additionally, we incorporate classifier augmentations reduce the strength of the constrained adversarial perturbation and to further guide the sampling trajectory towards regions of the image manifold that incorporate features from the adversarial class. We find that NatA Diff--effgeeeertee samples achieve comparable white-box (same target and victim classifier) attack success rates to current state-of of-the-artttt-tt-- adversarial attacks, while exhibiting significantly higher transferability (different target and victim classifier) across models. Furthermore, samples generated using NatADiff align more closely with known test-time errors (with respect to their Fréchet inception distance (FID) (Fréchet 1957)) than those generated through adversarial classifier guidance alone (Dai al. (024). These results demonstrate that NatADiff produces adversarial samples that not only transfer more effectively across models, but more faithfully resemble naturally occurring test-time errors. To summarize our contributions: (i) We propose NatADiff, incorporating classifier transformations, gradient normalization, and time-travel sampling (Shen et al., 2024 Lugmayr al., 2022 Yu et al., (023) improve adversarial classifier guidance and image quality; (ii) We design an adversarial boundary guidance algorithm to reliably navigate the complex, learned manifold, allowing us to generate natural adversarial samples with significantly higher transferability than existing approaches. (iii) We explore how convolution and transformer based classifiers perceive natural adversarial samples, exposing interesting properties of the feature representations learned by deep learning models.
2 DEFINITIONS AND PRELIMINARIES
Constrained, unconstrained, and natural adversarial samples. Broadly speaking there are three categories of adversarial sample: unconstrained, constrained, and natural. Let f Iu + ) be trained image classifier, Tu be the set of allowable image inputs, IN C Iu be the set of natural images, Y' be the set of image classification labels, and O : Ie + ) be an oracle ("perfecct human) classifier. Unconstrained adversarial samples require only that the image is misclassified:
Au & (x E Iu : f(r) + (x))) (Song et al. 2018). Constrained adversarial samples are restricted to an e-neighbourhood about some natural image: AG A {a + 8 E Te x E IN; ||0]|p S e, fxr 5) + Ox + 5)) (Szegedy et al. 2014). Natural adversarial samples are natural images that misclassified: AN & {i E Iv: f(x) 7 ((z)) (Hendrycks al. 2021). Finally, it follows from the above definitions that Av C Ae C Au-
Natural adversarial samples are well-documented phenomenon in deep learning (Hendrycks et al.) 202T). Literature suggests that they typically occur when deep learning models learn to rely on erroneous contextual cues to shorteut classification, opposed truly learning distinguish
2

---

Published as a conference paper at ICLR 2026
X
(a) PGD
(b) Nat. Adv. Samples
(c) Adv. Class. Guid.
(d) NatADiff
Figure 1: A comparison of different types of adversarial samples. Green and red borders indicate
non-adversarial and adversarial samples, respectively. A dotted border denotes artificially generated images, while solid border indicates real-world photographs. (a) Constrained adversarial attacks (PGD (Madry et al.) 2018) used here) add perturbations to clean images. (b) Natural adversarial samples are test-time errors that do contain perturbations. (c) Adversarial classifier guidance (Dail iet al. (2024) produces constrained ad adversarial samples, as the difference between images generated with and without the guidance is minimal- thhir difference amounts to constrained perturbation. (d) Adversarial samples generated with NatADiff diverge from those generated without NatADiff.
between classes (Hendrycks et al. 2021 Geirhos al. 2020; Arjovsky al., 2020) (see Appendix for examples). These cues are typically features within an image that are highly correlated with target class but not indicative the class. For instance, a model that uses oceanic environments cue for predicting "shark" may misclassify image of a shark lying sand. By cxploiting these easy-to-learn cues, models can reduce training loss without correctly Iearning to generalize to the underlying classification task. Additionally, it has been observed that natural adversarial samples are able to bypass common adversarial defences (Agarwal et al.] (022). and they exhibit high transferability i.e., the same image is misclassified by multiple classifiers (Hendrycks al.) 2021). The significant transferability of natural adversarial samples can be attributed to classifiers independently learning to rely on the same contextual cues, which is likely a consequence of shared correlations between cues and class labels across independent datasets (Hendrycks et al.) (021).
Denoising diffusion generative models (Ho et 2020) leverage stochastic differential equation (SDE) to "learn" the space of natural images, allowing for the generation of natural, within-distribution images. The SDE is characterized by the forward process:
day = f(t)a,dt + g(t) dB. VIE [0,7].
where H, R f(t) E and g(t) R are continuous functions of f, and ddB, denotes an Itô integral with respect to the standard multi-dimensional Brownian motion process B, E m and g are chosen such that the forward process progressively "destroys" structure in the image, æn. adding noise until it is approximately marginally Gaussian at termination time, T, i.e., p(æT) Re (0, 1). To generate natural image, the forward process can be reversed, and structure recovered from Gaussian noise using either Anderson reverse- time diffusion (Anderson, 1982) or the flow ODE (Song et al. 2021b) derived from (T.. Both formulations require an estimate of the score function, V æ, log ((x))) which is approximated by neural network, 6n (æt, 1). This network is trained to predict the original image from a noisy version using the objective:
(Pavliotis. 2014b). Functions
min Eno. [|laa (xx (lt)c((z,,t))/(()))
(2)
3

---

Published as a conference paper at ICLR 2026
where a<l) elo f(ujdu and B(1)² aa1)² Jo a(wzzdu. Given an optimal solution, 6Ø- (2t, to the above, the score function is given by (see Theorem P3 in Appendix P
g(u)²
VR, (pg(p((++)) - c. (2;, (1))()) VIE (0, T], Dy E IR..
Additionally, while Eg. (æt, can be used directly estimate vo, it is typically of lower quality than samples generated iteratively from the reverse-time diffusion or flow ODE (Ho et al. 2020 Song
ct al. 2021b Nichol & Dhariwal 2021).
Denoising diffusion class guidance provides finer control over the diffusion process by sam- pling from 2o ~ p(Zoog) y) instead of zo ~ p(zo), where y represents some conditioning infor- mation. To control the strength of class-guided diffusion, the marginal distribution is treated as p(x,,y) (v) x p(yla,)))11) p(=(())))////)))))))(()))))))p //((y), where s E Ry governs how strictly the diffusion adheres to the class constraint. For w > 1, the probability mass of p,y] is "tightened" around the regions of most probable y, while for w < 1, the probability mass is more diffuse. This results in stronger and weaker class adherence, respectively.
Class conditioning is incorporated directly into the diffusion score function in 3 by replacing p(aryy) with p(xx)y) as follows:
Van my log(p(>rlv)) = w m, (gg(p(yl=:)) - 3(1) Ev- (2te, 1)
(4)
1
[weg- (wr,, (,) (1 w)ea (2r2
(5)
3(()
where Eg- (2x,,) and eg- (2x,, t, v) are neural networks trained estimate the noise added in the forward diffusion process when Ep p(æo) and Zo ~ p(zaly)) respectively. The distinction between (4) and (5) illustrates the difference between the two forms of class-guided diffusion: classifier and classifier-free free guidance. In classifier guidance, separate model, po (y(y (a.)), is trained to predict the probability of y given x, (Dhariwal & Nichol (02T). contrast, classifier-free guidance requires training a diffusion model to directly estimate V a, log(p(x;; [2 [y) (Ho & Salimans 2022).
is important to note that (3) does not represent the marginal distribution that arises from applying the diffusion in (I) to æ p(æoly) (Karras et al. 2024). Instead, it is a mechanism that forces the sampling trajectory of Dy into regions with a higher probability of p(y]2z)) and in doing so, deviates from reverse-time diffusion and flow ODE dynamics. However, despite foregoing theoretical guarantees of sampling convergence, class-guided diffusion often exhibits superior sampling quality (Dhariwal & Nichol 2021 Ho & Salimans 2022).
RELATED WORK
Generating unconstrained adversarial samples. Previous work has shown that modern generative models are capable of creating artificial unconstrained adversarial samples (Song et al. [018 Zhao al. 2018 Chen et al. 2023b Dai al. 2024). Initial approaches used GANs as the generative backbone for these attack; however, GANs are sensitive to perturbations to their sampling path, and they lack theoretical justification for such perturbations (Karras et al. 2019: Abdal et al., 2019). Recent methods have leveraged denoising diffusion models (Ho et al. 2020). Diffusion models possess superior generation quality to GANs, and provide theoretical justification for perturbing the sampling path (Dhariwal & Nichol 2021). Dai et al. (2024) leveraged these properties to develop AdvDiff. which treats the true image class, y. and adversarial target, y. as random variables. The joint distribution be decomposed p(æ;, y, Ø) p(yl az, ) (1))))))(((())) az), where it is assumed that y and ij are conditionally independent given the noisy image, Dt Thus, given the forward diffusion in (I) the corresponding diffusion score function (see (4) and (5)) becomes
Væ, log(p(æ; 7)) = ((t)
[weg (xr, L, y) + (1 - 6g (xx, ()] + V log(r())z,)),
(6)
where w and s control the strength of the guidance, eg.
is a network trained to remove noise from æy and the adversarial gradient, V Va, log(p(ū) xx)). is derived from victim classifier that provides class probabilities. Since AdvDiff directly uses the victim classifier gradient, it can be considered a form of classifier guidance, and is therefore susceptible the same issues classifier-guided diffusion.
4

---

Published as a conference paper at ICLR 2026
Classifier-guis guided diffusion requires training model, pa ( to predict the class of image that has been corrupted with Gaussian noise (Dhariwal & Nichol 2021]. which is a form of adversarial
training (He et al. 2019 Li et al. 2019). When a nor-adversarially robust classifier is used instead,
the diffusion process typically generates visually coherent samples that do not adhere to the desired
class conditioning, but erroneously classified as the desired class (Vaeth al. [0244] Shen et al.) 2024). We hypothesize that this phenomenon arises due to constrained adversarial samples frequently
lying within an e-neighborhood of natural samples (Goodiellow et al. 2013 Madry al. 2018
Under this hypothesis, the diffusion model acts as constraint that pushes the sample towards the natural image manifold, while the non-ad adversarially robust classifier introduces a perturbation that directs the sample towards the nearest region containing samples of the desired class. The resulting trade-off between the diffusion model and classifier guidance incentivizes the diffusion trajectory to converge towards the adversarial regions that frequently lie imperceptibly close to the natural image manifold that is, pockets of constrained adversarial samples (Shen et al. 2024).
4
METHODOLOGY
Natural adversarial samples frequently occur when classifiers over-rely on contextual cues to shortcut classification (Hendrycks al. 2021). We incorporate this key observation into our proposed, diffusion- natural adversarial sampling scheme-NatADiff (see Algorithm T. NatADiff lever- ages adversarial boundary guidance to incorporate features from the adversarial class. In addition, we use augmented classifier guidance and time-travel sampling to enhance attack transferability while preserving image quality.
Accounting for sample noise. Classifier-guided diffusion specifically trains classifier to predict the class label of noisy sample æt,. However, in adversarial diffusion guidance, the victim model is typically an "off-the-shelf"" classifier that was never trained on noisy samples. Directly passing to this classifier will likely degrade classification accuracy, leading to inferior diffusion guidance. To address this, take the same approach as (Yu et al. 2023 Bansal et al., 2024 Shen et al. (2024) and use Tweedie's 's formula (Efron 2011) to pass the classifier the current estimate of zn at time t:
(o(x;)) = (x: - ((t) (xt))a--(,,,,,) (x,,,,,)) ()))/a(t).
Reducing adversarial gradient. Constrained adversarial attacks are sensitive to image transforma- tions, with rotations, crops, and translations reducing the success rates of common attack algorithms (Guo et al. 2018). We leverage this by applying differentiable image transforms to reduce the effect of the adversarial gradient that points in the direction of constrained adversarial perturbations. We find that this increases the prevalence of visible adversarial features (see Appendix G. for ablation study). These transformations are similar to the ones used by Shen et al. (2024) to perform training-free classifier Jlassifier-uiided diffusion. The local adversarial signal is "averaged out", reducing the likelihood of generating constrained adversarial samples, and forcing the manifestation of features from the-in case-adversarial class conditioning (Shen et al.) 2024).
Given collection of differentiable image transforms: T (Ti, T2....... we compute the adversarial
classifier gradient as
Væ, log(p((()x)) = g(x:))/llgg))) ((xl)///))) where g(x;) = Va, log (og ( ((TT(((()))))) h RR. + RDI is a function that returns
the victim classifier's 's logit predictions, and 0g R + R is sigmoid function that returns the probability of the target adversarial class.
Adversarial boundary guidance. Initial experiments showed that substituting the improved ad- versarial gradient from (8) into (6) did not steer the diffusion trajectory towards natural adversarial samples (see Appendix G.2 for ablation study). This may occur because classifier augmentations climinate many of the constrained adversarial samples that lie close to the image manifold, but not those further away. Consequently, if the initial sampling point of the diffusion trajectory is too distant from a region of natural adversarial samples, adversarial guidance will push the sample off the image manifold.
To address this, we leverage the connection between natural adversarial samples and the use of contextual cues classification shorteut (Hendrycks et al. 202T Geirhos et al. 2020 Arjovsky et al.
5

---

Published as a conference paper at ICLR 2026
2020). We propose adversarial boundary guidance as method of directing the diffusion trajectory towards samples that incorporate erroneous contextual cues, i.e., features from the adversarial class. We define adversarial boundary guidance
Vx, log(p(zell) 0) 3(t) [ t) (w ww)y + upoynul + a, log(p(ū(zx)), (9)
where w. p.P.S E E μ E [0, By = ca. (xx,,, 2. co. (æt, t), and Vyng co- t,,, ( Cg. (I,, t). w and govern the strength of classifier-free guidance, controls adversarial classifier guidance strength, and μ regulates how strongly the sample tends towards the intersection of the true and adversarial classes. For sufficiently large j4, the sampling trajectory should approach the class intersection, incorporating enough elements from the adversarial class to cause a misclassification, while remaining within the bounds of the true class from human's 's perspective. Note when μ 0, adversarial boundary guidance is equivalent to adversarial classifier guidance (Dai et al. 2024).
To justify (9), we note that the classifier-fres free score function can be rewritten as Vz, log(p(æ:lv)) o a(1) Cg. (xi,t) t) + w'yl where ,y = Cg. (I t, y) - -g.. (zt, t) is a vector that points towards regions of the manifold containing images of class y. Additionally, recall that p p(zrly) is not the marginal density arising from a valid diffusion, rather it is a magnification of the guidance provided by a network that has "learned" the image manifold. To further exploit the information contained in this network, we introduce Vyng. which directs the sampling trajectory towards the class intersection.
Time-travel sampling. Significant disruption to the diffusion sampling path risks degradation in sample quality, or falling off the image manifold (Lugmayr et al. 2022 Yu et al. (2023). To mitigate these issues, we incorporate time-travel sampling into our diffusion scheme, which has been shown
to increase image quality in cases where standard diffusion sampling would otherwise fail (Lugmayr
al. 2022
Yu et al., 2023: Shen et al., 2024).
By injecting additional sampling steps, time-travel sampling allows the diffusion model to explore a wider region of the sample space and recover from suboptimal trajectories. This helps maintain sample quality and prevents the generation process from diverging away from the image manifold. More concretely, given sequence of sampling times [ with f,+1 > t, for all i, time-travel sampling resets the diffusion state at time ( by running the forward process, Lt,,. ~ p( (xte)) and then resampling Ft. using the reverse process (Anderson) 1982, Song et al.] (02Tb)) This
procedure is repeated R times before It, is accepted, after which sampling proceeds to Øt To improve efficiency, time-travel sampling can be applied to a subset of diffusion steps (Yu et al. 2023).
Similarity targeting. Many popular adversarial attacks operate in untargeted setting (Szegedy et al., 2014; Goodfellow et al., 20155: Madry et al.] 2018 Croce & Hein (020), where the only requirement is that the predicted class differs from the true class, i.e., y * y. These attacks often update the adversarial target dynamically during optimization, selecting the most probable incorrect class at each step, and they frequently outperform targeted variants (Croce al. 2020). To extend NatADiff to untargeted settings, we propose similarity targeting (see Algorithm in Appendix FF.
Similarity targeting is based on the assumption that is easier to incorporate adversarial features from classes that are semantically similar to the true class. To heuristically measure this similarity, we leverage the CLIP (Radford al. 202T) text encoder, which maps class labels into a shared image-tex text embedding space. We then select the ad adversarial target as the class most similar to the true class in this embedding space, as measured by cosine similarity. Concretely, given the CLIP text encoder Gene : y -> Ih.", the true class label, Vt. and the set of candidate adversarial labels Veand {y11.... ......... yn Vi. we define the adversarial target as
Cene ) Cenc (v) |Cenc (W/) ||2 Cene (v) 12
ÿ = arg max
(10)
DE Yeand
5 EXPERIMENTS
5.1 EXPERIMENT DETAILS
We evaluate the effectiveness of NatADiff on the ImageNet (Deng al. 2009) classification task, which requires a model to classify an image into one of 1,000 distinct object categories. We target range of off-the-shelf ImageNet classifiers and assess the attack success rates and visual quality

---

Published as a conference paper at ICLR 2026
of the generated samples. All experiments conducted on an NVIDIA RTX 4090 GPU, and each sample takes approximately 103 seconds to generate (see Appendix N for runtime comparisons).
Surrogate and victim models. NatADiff and other comparable attack methods require access to classifier gradients when generating adver- sarial samples. The model whose gradients are used in this way is referred to as the sur- rogate model, and we test ResNet-50 (RN-50) (He et al. 2016). Inception-v3 (Inc-v3) (Szegedy et al.) 2016). and Vision Transformer (ViT-H) (Dosovitskiy et al. 2021 surrogates. We exam- ine the performance of these adversarial sam- ples across RN-50, Inc-v3, ViT-H, adversarially trained ResNet (AdvRes) and Inception (Ad- vInc) (Kurakin et al., 2018), ResNet-152 (RN- 152) (He al.] 2016). Max- VIT (Tu et al. 2022
Algorithm 1 NatADiff
Require: adversarial guidance parameters: w. P. pt, true and adver- sarial classes: 8i victim classifier: h; forward diffusion functions: a(t), _()): stable diftusion model: cgr: VAE decoder: Vasi collec- tion of differentiable image transforms: (7i, Ta. J: sequence sampling sleps with
(t: 1 time tenell parameters: R, ris Fu: adversarial classifier bounds: et. oui number sampling attompts: S: guidance sealers: 6,.66
O, = and f4+1
ET NQ,,
for 1..... ddo for i N...., .......00
in x f, Tu then
R R
else A
for
ft. 1 do
b Time-travel loop
Swin-B (Liu et al. (02T), and DelT (louvron
My = eys (Zt,) hs. v) egt (ze,,
al.
2021) victim models.
Pyng cat (Zly Ni, (() cg- (#),
é = Egx (at, t2) + (w - mw) + upuunn
Diffusion model. We use Stable Diffusion 1.5 (Rombach al.] 2022) (SD1.5) as our base diffusion model for NatADiff and adversarial classifier guidance (Dai al. 2024). SD1.5 is a pretrained latent texi-to-image diffusion model. The diffusion process is performed in a latent space, and a variational autoencoder (VAE), (Kingma
i cu then
-
mn Vas
g Var, log (ss (1+ E A(T,(
Tol
aB(t)g
end if
21,-1 <- reverse diffusion step using é
if then
P Sampling #s; ~y p(*,, 1 1)
Welling, (014). Vaes. is used to decode latent samples into the image space. To facilitate the use of adversarial clas- sifier guidance the VAE must be incorporated into the gradient calculation. Specifically, given sample, 21. from the latent diffusion pro- cess, we introduce the VAE, Vdec. into 7 as (2.) = Vaec ((z, (()) (z,, ()v)))(((()) and take the gradient with respect to 2, instead of x, in (6) and (8). Finally, we use 200 sam- pling steps under the DDIM (Song et al. 202 parameterization, which defines the drift and dif- 1 log(á.)
a(t:)
a(4-1)
8² 3(L,2 (()(()-1)))
21, ~ (amt;-1. 62
end end for end
if argmax
(DVVV (zo))) # $ then
# # 6, #
else break end if
D. Find the search early if sample is frund
for
return Voo(=a)
fusion coefficients in (1) as ftt = 2
and g(1)² dee log(āz), respectively (Han 2024
NatA settinggs We run NatADiff under both targeted and untargeted attack settings. For targeted attacks, assign random adversarial target to each sample. For untargeted attacks, we use similarity targeting from Section 4 During adversarial boundary guidance we use the text prompt "<class name of yr""" " as the conditioning for the true class guidance, y. For intersection guidance, y n ÿ. we use the prompt "<class name of J> and <class name of yy>". We delay adversarial classifier guidance until timestep < 700, i.e., we set conservative value of = 0.2 for all experiments, and select s based on the target classifier (see Appendix 2.2 for ablation study and additional experiment details). We generate 2,000 adversarial samples in each experiment run.
for all 700. Finally, we choose a
Comparison methods. We compare NatADiff to state-of-the-art constrained and unconstrained adversarial attacks: PGD Madry et al. (2018), AutoAttack (AA) Croce & Hein (2020), NCF (Yuan) ct al. 2022), DiffAttack (Chen et al. 2025), ACA (Chen et al.) 2023b), and adversarial classifier guidance (AdvClass) (Dai et al.) (024). All methods use their default parameter settings and for comparison methods that alter a pre-existing "clean" image, we use their suggested ImageNet- compatible dataset our clean baseline (Kurakin et al., 2017). We apply AdvClass under both targeted and untargeted attack settings (using the same similarity targeting as NatADiff). Finally, we acknowledge that although NCE, ACA, and AdyClass classified unconstrained attacks, only
7

---

Published as a conference paper at ICLR 2026
ResNet-50
Inception-v3
VIT-H
True Goldfish Adv': Tia Monkey
True: Mashroom
Tru:: Tining Tabble
Trurc Thimble Adv'; Traffic Light
Trae: Clerds
Tru:: Polamid Camera
Adv : Pocket
Ads' Playypus
Adv Bobsted
Adv Sleeping Bog
True: Bunnet
Trues Carbagg Track
True: Hay Adv Ox
True: Cheeseburger Adx"; Bunana
Trae: Paintteuhh
Truc: Wool
A
Sombrero
Adv"; Snowplow
Adv": Marchstick
Adv": Dultrag
Figure 2: Adversarial samples generated by NatADiff with ResNet-50 (He et al. 2016), Inception-v3 (Szegedy al. 2016), and ViT-H (Dosovitskiy et al. 2021 surrogate models (see column labels). We report the true class and adversarial target for each image. Superscripts T and U denote random and similarity targeted attacks, respectively.
Source
PGD
AA
NCF
DiffAtlack
ACA
AdrClass
NatA
Figure 3: Source and adversarial samples generated by PGD Madry et al.
2018) Auto (AA)
Croce Hein (2020), NCF Yuan al. 2022). DiffAttack (Chen al. 2025), ACA (Chen al.) 2023b), AdvClass (Dai et al. 2024), and NatADiff with ResNet-50 (He et al. 2016 surrogate model. Note true class "burger" and adversarial target for AdvClass and NatADiff is "banana".
NatADiff and AdyClass have a fully unrestricted attack domain, as they are not bound to any initial clean image and are capable of synthesising entirely new images.
Metrics. To assess attack performance, we follow Chen et al. (023b), and report attack success rate (ASR) as the percentage of misclassified samples. To evaluate image quality, use the Inception Score (IS) (Salimans et al. (0166) and Fréchet Inception Distance (FID) (Fréchet 1957). IS provides direct measure of image quality, while FID estimates the similarity between the distributions of generated and real images. We compute FID with respect to both the ImageNet-Val (Deng et al., 2009) and ImageNet-A (Hendrycks et al., 202T) datasets to assess how closely NatADiff samples resemble natural images and known natural adversarial examples, respectively.
5.2 RESULTS
Attack success. NatADiff had comparable white-box ASR current state-of-the-art attacks, but vastly superior transferability across all experiments (see T.. This suggests that NatADiff is able more effectively leverage the mechanisms underpinning natural adversarial samples. Additionally, adversarial training did not offer any meaningful robustness to NatADiff, with both targeted and untargeted attacks transferring adversarially trained ResNet and Inception models.
PGD Madry et al. 2018) and AA (Croce & Hein, 2020) had the lowest transferability. likely because both are perturbation- based attacks, i.c.. they rely on the small pockets adversarial samples that neighbor natural images. These adversarial regions are not guaranteed to align across classifier architectures, especially architectures that are dissimilar, e.g., convolutional vs. transformer. Similarly, the lower transferability of NCF Yuan et al. (2222) and DiffAttack (Chen et al. (025) can be explained by their limited attack surface. NCF is restricted to attacking the color distribution of "clean" source image (see Figure 3. In contrast, DiffAttack crafts adversarial perturbations for
8

---

Published as a conference paper at ICLR 2026
Table 1: Attack success rate (%) and image quality of adversarial samples. Superseripts T and U denote random and similarity targeted attacks, respectively. Bold and underlined values highlight the best and second best scores for each surrogate model. White-box ASR (same surrogate and victim model) is denoted with an Note we do not report image quality for constrained perturbation-baser attacks (these attacks make imperceptible image alterations).
Surrogate
Model
Atack
Vietim Model ASR (3)
Transfomers
Averag
Val
FID-
FID-
RN-50
Inc-33
RN-152
AdvRes
Advlae
VIT-H
Max-ViT VIT
Swin-B
DefT
Clean
3.3
7.6
2.9
3.0
5.8
10.9
3.8
4.5
7.4
5.7
35.0
58.0
94.7
PGD
90.4"
11.3
5.2
4.9
8.1
10.5
4.4
5.5
8.2
17.6
AA
100"
13.3
10.0
3.9
8.8
10.5
5.4
5.6
8.0
18.4
NCF
74.8*
33.4
37.3
28.2
31.2
17.2
24.0
31.7
37.2
35.0
30.4
69.7
85.5
DifAAnack
92.5*
52.5
43.3
42.4
76.8
RN-50
ACA
79.8"
53.3
52.7
40.8
53.1
41.8
46.4
20.3
50.6
52.9
23.9
65.0
77.9
AdvClass
00.6"
35.0
32.1
31.4
33.5
25.8
30.0
30.8
32.88
33.00
38..5
48 9
92-4
AdvClass
99.9"
41.3
38.7
29.7
38.5
50.2
92.7
NaAADie
96.9"
06.5
185.0
17.3
NatADia"
99.3*
68.3
72.1
65.3
08.8
45.3
64.1
65.2
67.0
68.2 2
43.2
51.4
95.9
PGD
10.4
AA
100"
4.9
4.8
12.8
5.7
NCF
31.0
66.7"
23.1 1
29.00
36.3
15.8
18.3
20.4
$0..5
S10.
31.7
69.
85500
DitAAlakk
20.0
74.6"
233-7
3000
30.9
18.9
22..9
26.5
215.8
392.4
355.22
63.7
78:
ACA
67.8"
34.2
43.8
15.1
48.8
78.8
AdvClass
35.1
99.8"
34.5
335.6
39.5
28.8
32.4
34.0
25.7
41.7
333.7
51.0
89.2
AdvClass
38.0
90.01"
38.7
40.4
44.2
300D
36.0
36.6
38.9
44.8
354.7
49.4
93.3
NatADi
53.4
97.9"
4
57.8
62.6
$5.4
44.4
45.1
500.8
56.2
27.7
66.6
78.2
NatADin
67.4
90.4
70.1
60.3
2
47.00
50.1
98.9
PGD
5.8
11.0
3.6
4.0
1.8
96.2"
4.5
5.4
9.2
18.4
AA
6.5
3.9
4.3
8.6
1.00*
4.5
5.9
17.0
NCF
14.8
15.4
18.5
50.6"
20.8
86.4
DifFAttack
20.5
25.0
17.2
18.9
22.4
73.2*
18.1
22.3
20.6
28.5
35.2
63.4
80.0
VITH
ACA
50.5
64.55
48.1
49.1
52.8
75.8*
47.5
40.7
50.5
1/.2
25.5
64.2
80.3
A
34.4
31.4
92.6"
53.4
96.2
AdvClass"
35,2
37.5
35.8
35.2
38.0
98.7*
33.9
34.9
37.7
42.8
39.2
48.5
98.8
NarADife"
73.5
71.3
72.1
98.5*
65.7
88.0
90.5
NatADia
66.8
67.0
65.3
61.9
65.8
99.6"
63.9
65.4
68.6
60.7
31.9
53.9
96.2
source image by perturbing the diffusion latent space subject the constraint that the reconstructed adversarial image must remain sufficiently close to the original (see Figure 3).
Adversarial classifier guidance (Dai et al. 2024) is outperformed by ACA (Chen et al., 2023b) and NatADiff in all experiments. This can be attributed to the limited guidance provided by injecting non-robust classifier gradients into the diffusion sampling trajectory (Shen et al.) 2024). ACA the most comparable to NatA performance-wiss wise; however, ACA alters the semantic structure of source image and thus constrained by the semantics the initial image. contrast, NatADiff has wider attack surface as it is free to generate any image that fools surrogate classifier. Furthermore, NatADiff uses a diffusion model to incorporate adversarial features that are classifier-gnnosti (as seen in Figure 2). and as such, is the only method that does not solely rely on the gradient of surrogate classifier.
VIT-H (Dosovitskiy al. 202 1) is the current state-of-ther the-t-rttttttttt in image classification and is the most resistant to transfer attacks. This is unsurprising, as it uses the modern transformer architecture and is the largest model examined. ViT-H learns more robust feature representation than convolutional and smaller transformer models, which makes it less susceptible to both constrained and natural adversarial samples. However, despite the strengths of the ViT-H architecture, NatADiff is able to reliably generate samples that transfer to ViT-H-albeit at lower ASR than equivalent attacks against all other models.
When comparing NatADiff's targeted attacks with their untargeted counterparts, we see that un- targeted attacks outperform targeted attacks both in terms of victim classifier performance and transfcrability for all classifier except ViT-H. We believe that the discrepancy between targeted and untargeted ViT-H samples can be attributed to the strength of the decision boundary learnt by the ViT-H model. The diffusion model struggles to generate feasible targeted adversarial samples for ViT-H, leading to the introduction of image artifacts (as seen in Figure 18 from Appendix K.5) and supported by the IS and FID-Val scores in Table D.. These artifacts substantially degrade image quality and artificially inflate the attack success rate of the targeted ViT-H samples. Importantly, this de degradation in image quality adversarial targets are easier to achieve than others, which further motivates the use of similarity targeting as method for identifying classifier "weak spots."
limited to targeted ViT-H attacks. This indicates that some

---

Published as a conference paper at ICLR 2026
Image quality. We observe clear disparity in the image quality of targeted and untargeted NatADiff variants (see Table [T). Targeted NatADiff samples exhibit lower FID-A but worse IS and FID-VAL indicating that they closer in distribution to known natural adversarial examples, albeit with lower image quality and less alignment to the ImageNet validation dataset. In contrast, untargeted NatADiff achieves IS and FID--VL comparable to other generative methods, but with higher FID-A, suggesting that overall image quality improves at the expense of alignment with natural adversarial samples. This follows from the known characteristics of natural adversarial samples, which often blend features from disparate classes (Hendrycks et al. 2021 Geirhos et al. 22000; Arjovsky 2020). Replicating such blending places greater demands on the underlying diffusion model to locate plausible points on the image manifold, which can introduce artifacts and degrade image quality. In contrast, similarity targeting blends more related classes, yielding samples with higher visual fidelity but less alignment with natural adversarial distributions. Additionally, NCF (Yuan et al. 2022). DiffAttack (Chen ct al. 2025), and ACA (Chen et al. 2023b) all achieve superior FID-A than AdvClass (Dai et al. 2024) and untargeted NatADiff. This can be attributed to the low FID-A of the clean baseline dataset that NCF, DiffAttack, and ACA use as their source, which causes them to inherit the same distributional properties. Conversely, AdvClass and NatADiff generate artificial samples and are thus constrained both by the distributional tendencies of the underlying diffusion model and the effect of similarity targeting.
Arjovsyy et al.
6 CONCLUSION
We introduce NatADiff, an adversarial sampling scheme that leverages diffusion models to generate highly transferable adversarial samples. Our method is motivated by the observation that natural adversarial samples frequently contain features from the adversarial class, which deep learning models exploit to shortcut the classification processes. To leverage this behavior, we guide the diffusion trajectory towards the intersection of the true and adversarial classes. Our method achieves comparable white-box attack success rates to current state-of-the-at techniques, while exhibiting significantly higher transferability across models. Furthermore, samples generated using NatADiff align more closely with known natural adversarial samples than those generated via adversarial classifier guidance alone. These results demonstrate that NatADiff produces adversarial samples that transfer more effectively than existing attacks, and more faithfully resemble naturally occurring test-time errors than those generated from vanilla adversarial diffusion guidance.
ACKNOWLEDGMENTS
Max Collins is the recipient of the Research Training Program scholarship funded by the Australian Government. Professor Ajmal Mian is the recipient of an Australian Research Council Future Fellowship Award (project number FT210100268) funded by the Australian Government. This research was partially supported by National Intelligence and Security Discovery Research Grants (project number NS220100007), funded by the Department of Defence, Australia.
REFERENCES
Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2StyleGAN: How to embed images into the stylegan latent space? In IEEE/CVF International Conference on Computer Vision, ICCV, pp. 4431 4440. IEEE, 2019.
Akshay Agarwal, Nalini Ratha, Mayank Vatsa, and Richa Singh. Exploring robustness connection between artificial and natural adversarial examples. In IEEF/CVF Conference Computer Vision and Pattern Recognition, CVPR, pp. 179-186, 2022.
Brian D. O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):3133326 1982.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arxiv: /907.0283, 2020.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In 12th International Conference Learning Representations, ICLR, 2024.
10

---

Published as a conference paper at ICLR 2026
Anand Bhattad, Min Jin Chong, Kaizhao Liang, Bo Li, and David A. Forsyth. Unrestricted adversarial examples via semantic manipulation. 8th International Conference on Learning Representations, ICLR, 2020.
Tom B. Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and lan Goodfellow. Unrestricted adversarial examples, 2018.
Jianqi Chen, Hao Chen, Keyan Chen, Yilan Zhang, Zhengxia Zou, and Zhenwei Shi. Diffusion models for imperceptible and transferable adversarial attack. IEEE Trans. Pattern Anal. Mach. Intell., 47(2):9119777, 2025. 2025.
Xinquan Chen, Xitong Gao, Juanjuan Zhao, Kejiang Ye, and Cheng-Zhong Xu. AdvDiffuser: Natural adversarial example synthesis with diffusion models. In IEEE/CVF International Conference Computer Vision, ICCV, pp. 3562-4572, 2023a.
Zhaoyu Chen, Bo Li, Shuang Wu, Kaixun Jiang, Shouhong Ding, and Wenqiang Zhang, Content- based unrestricted adversarial attack. In Advances Neural Information Processing Systems 36, NeurIPS, 2023b.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter-fiee free attacks. In Proceedings of the 37th International Conference on Machine Learning, ICML, volume 119 of Proceedings Machine Learning Research, pp. 2206-2216. PMLR, 2020.
Francesco Croce, Jonas Rauber, and Matthias Hein. Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks. Int. J. Comput. Vis., 128(4))1028-10)) 10466 2020.
G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4):303-314 1989.
Xuelong Dai, Kaisheng Liang, and Bin Xiao. AdvDiff: Generating unrestricted adversarial examples using diffusion models. In European Conference on Computer Vision, ECCV, volume 15104 of Lecture Notes Computer Science, pp. 93-109. Springer, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 248-255, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6): 141-142, 2012.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems 34, NeurlPS, pp. 8780-8794, 2021.
Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR, 2021.
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(496):1602-1614, 2011. ISSN 01621459.
Maurice Fréchet. Sur la distance de deux lois de probabilité. Annales de I'ISP,, V1(3):183-198. 1957.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 11:66)6666666 6735,673.660 2020.
S. Alireza Golestanch, Saba Dadsetan, and Kris M. Kitani. No-reference image quality assessment via transformers, relative ranking, and self-consistency. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV, pp. 3989-3999. IEFF, 2022.

---

Published as a conference paper at ICLR 2026
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, ICLR, 2015.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. In 3rd International Conference on Learning Representations, ICLR, 2015.
Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. 6th International Conference on Learning Representations, ICLR, 2018.
Manhyung Han. DDIM redux: mathematical foundation and some extension. arXiv preprint arXiv:2408.00 07285, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 770- 778. IEEE Computer Society, 2016.
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 588-597. Computer Vision Foundation / IEEE, 2019.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 15262-15271. IEEE/CVF, 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.125%0 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33, NewrlPS, 2020.
Kurt Hornik. Approximation capabilities multilayer feedforward networks. Neural Netw, 4(2): 251-257, 1991.
Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6:695-709, 2005.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 4401 1410. IEEE/CVF, 2019.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems 35, NeurIPS, 2022.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with bad version of itself. In Advances in Neural Information Processing Systems 38, NeurIPS. 2024.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR, 2014.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In 5th International Conference on Learning Representations, ICLR, 2017.
Alexey Kurakin, Ian J. Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang. Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, and Motoki Abe. Adversarial attacks and defences competition. arXiv preprint arXiv:/8044 000970 2018.
Bai Li. Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. In Advances in Neural Information Processing Systems 32, NeurIPS, pp. 9469, 2019.
12

---

Published as a conference paper at ICLR 2026
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical vision transformer using shifted windows. In /EEE/CVF International Conference Computer Vision, ICCV, pp. 9992-10002. IEEE, 2021.
Andreas Lugmayr, Martin Danelljan, Andrés Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR, pp. 1451-111611 IEEE/CVF, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR, 2018.
Leland McInnes and John Healy. UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.
Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assess- ment in the spatial domain. IEEE Trans. Image Process., 21(12):4695-4700 2012.
Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making a "completely blind" image quality analyzer. IEEE Signal Process. Letr., 20(3):209 2222 2013.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8162- 8171. PMLR, 2021.
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anandkumar. Diffuusion models for adversarial purification. In Proceedings of the 39th International Conference on Machine Learning, ICML, volume 162 of Proceedings of Machine Learning Research, pp.
16805-16827. PMLR, 2022.
Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. Cats and dogs. In JEEE Conference Computer Vision and Pattern Recognition, CVPR, 2012.
G. A. Pavliotis. Stochastic processes and applications: diffusion processes, the Fokker-Planck and Langevin equations. Texts in Applied Mathematics. Springer New York, 2014a. ISBN 9781 9781493913237.
G. A. Pavliotis. Stochastic processes and applications: diffusion processes. the Fokker-Plance and Langevin equations, chapter 4, pp. 88-89. Texts in Applied Mathematics. Springer New York, 2014b. ISBN 9781493913237.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 10674-10688 IEEE/CVF, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis., 115(3)2111255) 2015.
Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29, NeurlPS, pp. 2226-2234, 2016.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-Gar Protecting classifiers against adversarial attacks using generative models. In 6th International Conference on Learning Representations, ICLR, 2018.
13

---

Published as a conference paper at ICLR 2026
Vaishaal Shankar, Rebecca Roclofs, Horia Mania, Alex Fang, Benjamin Recht, and Ludwig Schmidt. Evaluating machine accuracy on ImageNet. In Proceedings of the 37th International Conference Machine Learning. ICML, volume 119 of Proceedings of Machine Learning Research, pp. 6634-8644, PMLR, 2020.
Yifei Shen. Xinyang Jiang, Yifan Yang, Yezhen Wang, Dongqi Han, and Dongsheng Li. Understand- ing and improving training-free bass-based diffusion guidance. In Advances in Neural Information Processing Systems 38, NeurlPS, 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR, 2021a.
Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. In Advances in Neural Information Processing Systems 31, NeurlPS, pp. 8322-8333, 2018.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference Learning Representations, ICLR, 2021b.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2818 1826.. IEEE Computer Society, 2016.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML, volume 139 of Proceedings of Machine Learning Research, pp. 10347-10357. PMLR, 2021.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan C. Bovik, and Yinxiao Li. ViT: Multi-axis vision transformer. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), European Conference on Computer Vision, ECCV, volume 13684 of Lecture Notes in Computer Science, pp. 459 479. Springer, 2022.
Philipp Vacth, Alexander M. Fruchwald, Benjamin Paassen, and Magda Gregorova. GradCheck: Analyzing classifier guidance gradients for conditional diffusion sampling. arXiv preprint arXiv:2406. 17399, 2024.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, (186):2579-660) 2008
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computa- lion, 23: 1666416744 2011.
Xiaosen Wang, Kun He, Chuanbiao Song, Liwei Wang, and John E. Hoperoft. AT-GAN: An adver- sarial generator model for non-constrained adversarial examples. arXiv preprint arXiv:1904.07793, 2020.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: state-of-the-an natural language processing. Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNIP,, pp. 38-45, Association for Computational Linguistics, 2020.
Mengda Xie, Yiling He, Zhan Qin, and Meie Fang. RetouchUAA: Unconstrained adversarial attack via realistic image retouching. IEEE Trans. Circuits Syst. Video Technol., 35(3):2586-2602 2025.
14

---

Published as a conference paper at ICLR 2026
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and Distributed System Security Symposium, NDSS, NDSS 2018. Internet Society, 2018.
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free free energy-guided conditional diffusion model. In IEEE/CVF International Conference on Computer Vision, ICCV, pp. 23117-23122 IEEE, 2023.
Shengming Yuan, Qilong Zhang, Lianli Gao, Yaya Cheng, and Jingkuan Song. Natural Color Fool: Towards boosting black-box unrestricted attacks. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35, NeurlPS, 2022.
Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. In 6th International Conference on Learning Representations, ICLR, 2018.
15

---

Published as a conference paper at ICLR 2026
Appendix
Table of Contents
Limitations
17
B Ethics and broader impacts
17
C Reproducibility
17
D certurbation-based attacks and defences
17
Adversarial attacks.
D.2 Defences against adversarial attacks
E Example natural adversarial samples
18
NatADiff similarity attack algorithm
20
NatADiff ablation studies
20
Effect of classifier augmentations
20
Effect of boundary guidance strength
22
G.3
Effect of adversarial boundary guidance prompt structure
22
Selection of adversarial classifier guidance strength
23
H Manifold plots
24
NafA on Oxford Pets
29
NatADiff experiment parameter settings
31
K
Additional DatA samples
33
K.
Mixed Samples
33
K.2
ResNet-50 samples
34
K.3
Inception-v3 samples
................................
35
K4
ViT samples
36
K5
Low-quality targeted ViT samples
L
Additional image quality metrics
37
Resistance to adversarial defences)
37
Runtime comparison
39
User study
40
P Useful denoising diffusion results
41
P.
Conditional forward distribution
41
P2
Conditional forward alternate parameterisation
42
P3
Score-madel link
43
16

---

Published as a conference paper at ICLR 2026
A LIMITATIONS
While NatADiff is effective at generating highly transferable adversarial samples, it remains computa- tionally expensive due to the iterative nature of diffusion and the overhead introduced by adversarial guidance. This is an inherent limitation of diffusion-base change without significant advances in generative modeling or architectural design. Additionally, the use of similarity targeting datasets like ImageNet can lead to subile misclassificationsee, between similar dog brecds-which may diminish the perceived severity of the attack. A potential refinement would be to surface ranked list of similar classes, allowing users select more divergent adversarial targets while retaining the semantic grounding of similarity- -based selection. We also note that we used conservative setting for the adversarial boundary guidance term, μ1, larger values caused generated samples to occasionally include the adversarial class, as discussed in Appendix G.2) Finally, we restrict our evaluation to ImageNet classifiers, as ImageNet offers a diverse label space, which supports varied attack scenarios. Extending NatADiff to more specialized domains remains an avenue for future work.
generative methods, and one unlikely to
B ETHICS AND BROADER IMPACTS
We adhere to the ICLR code of ethics. We acknowledge that this work explores the use of generative models as means of creating highly transferable adversarial samples. While adversarial attacks raise legitimate concerns regarding misuse, our objective is to expose fundamental vulnerabilities of current classifiers and to better understand the structure of natural adversarial samples. By making our models and code publicly available, we aim to support transparency and reproducibility, and we believe that insight into generative adversarial mechanisms a necessary step toward building more secure and interpretable classifiers. We do not use private or sensitive data, and all data and models used are publicly released and broadly studied. In future work, we plan to explore how NatADiff can be extended to detect or defend against naturally occurring adversarial samples.
C REPRODUCIBILITY
We ensure reproducibility by providing detailed descriptions our algorithms (see Algorithms T and 2 and experiment parameter settings (see Tables 7 and [8. Our full codebase is included in the supplementary material, along with all configuration files required to replicate our experiments. Comparison methods are implemented using publicly available repositories, and we follow the authors' recommended hyperparameters unless otherwise stated.
D PERTURBATION-BAN ATTACKS AND DEFENCES
In this section, provide a brief overview of existing constrained perturbation-base attack and defense strategies for image classification models. We focus on the optimization- based formulation of adversarial attacks and highlight the theoretical underpinnings of common training-time defenses.
D.1 ADVERSARIAL ATTACKS
Szegedy al. 2014) were the first demonstrate that imperceptible perturbations to image's pixel values could cause deep learning models to misclassify the image with a high probability (see Figure 1). Mathematically, these constrained adversarial attacks can considered solution the following constrained optimization problem:
min Ca (x + 6; 0,
(11)
SES
where ô E RT is the computed perturbation, æ E m is the vectorized "clean" image, h RT + y is a "ttriind"" classifier model with parameters 0, ÿ E y is the class targeted by the adversarial attack, (+; 0, Ø) is the loss of the classifier with respect to the target adversarial class, and S & R= || p < L} is a convex set of allowable perturbation sizes. Algorithms such as fast gradient sign method (FGSM) (Goodfellow al. 2015), projected gradient descent (PGD) (Madry al. 2018).
17

---

Published as a conference paper at ICLR 2026
and Auto/Ataac (Croce & Hein 2020), have been proposed to efficiently solve the optimization problem in (II).
Other attack methods have relaxed the constraint the magnitude of the adversarial perturbation. These unconstrained adversarial attacks seek to alter the semantic information within an image, resulting in misclassification without visually altering the perceptible class. Additionally, techniques like selective cropping and rotation, texture remapping, color pallette transformations, and generative sampling have all been used to successfully "fool" modern deep learning models (Brown et al. 2018
Bhattad et al.) 2020; Song et al. 2018 Wang et al. 2020
Dai et al. 2024
Xie et al. 2025).
D.2 DEFENCES AGAINST ADVERSARIAL ATTACKS
Several defensive measures have been proposed that aim to purify adversarial inputs (Samangouei et al., 2018; Nie al. 2022). harden model architectures against attacks (Gu & Rigazio 2015
Xu et al. 2018). or improve training procedures (Szegedy et al., 2014, Madry et al. 2018). A key challenge in designing adversarial defences is preventing attackers from crafting that exploit the adapted model. For this reason adversarial training has become one of the most popular defences, as it both addresses the source of the adversarial attack, while providing theoretical guarantees of robustness against all possible perturbation- based adversaries.
attacks
Adversarial training can be formulated the following anddle-point optimization problem:
min (a,a)) max La + 6; 6:0,y) 8ES
(12)
where 'D is the joint distribution of naturally occurring images and classes, and E 3 is the true class label of æ (Szegedy et al. 2014, Madry et al. 2018). The optimization problem in (22) can be thought of minimizing the loss caused by the strongest possible adversarial attack. Thus, any model that minimizes (12) is theoretically guaranteed to be resistant to its strongest possible adversarial perturbations.
E EXAMPLE NATURAL ADVERSARIAL SAMPLES
Figure 4 shows natural adversarial samples from the ImageNet-A dataset (Hendrycks al. 2021) each paired with heatmap of the classifier-guides gradient with respect to the adversarial class, V. log(p( x)). These gradients highlight the image features that contribute misclassification and that would be emphasized during adversarial classifier-guided diffusion (Dai et al. 2024). In the first image, the classifier gradient is concentrated around the school bus and the snowbanks running alongside the road; in the second, it is concentrated on the snail and its shadow; and in the third, it is concentrated on the power switch. This suggests the classifier has "learned" to associate vehicles beside snowbanks with snowplows, dark elliptical objects with cockroaches, and vertical rectangular boxes with pay phones.
18

---

Published as a conference paper at ICLR 2026
School Bus
Snowplow (34%) Snail
Cockronch (43%)
Rahin
Pay- Phone (2255)
log(p(ū(=))
V. (og(p(ū(z))
Figure 4: Top: Natural adversarial samples compiled by Hendrycks et al. (2021) for ImageNet (Deng et al. 2009) classifiers. The green labels denote the ground-truth classes; the red labels the classes assigned by a ResNet-50 classifier (He et al. 2016). Bottom: Heatmap of the ResNet-50 adversarial classifier-uuidance (Dai al. 2024) gradient with respect the adversarial classes, Arrows point features from the adversarial class that affect the ResNet-50 classification.
19

---

Published as a conference paper at ICLR 2026
F NATADIFF SIMILARITY ATTACK ALGORITHM
Algorithm 2 provides the algorithm for the similarity targeted variant NatADiff.
Algorithm 2 NatADiff- -iniiaatSimilarity
Require: adversarial guidance parameters:
troc class: y. candidate adversarial classes: Mead = (ū 92.... J victim classifier: decoder: Vas; encoder: Caz: collection sequence sampling steps with ty T, 4:41 > ta: (r) time-travel parameters: R. r2. Tuc adversarial classifier bounds: et. cu: number sampling attempts: S: guidance scalers: 8y. 6,
forward diffusion functions: a(t). ((t): diffusion model: cg.:
differentiable image transfoms:: (Ti 72.
min TCleetio Cor 7-..aa
2 ~ (N ()
for I..... S do
for i
N..... 1
6
Tu then
R R
R=1
end
for
Rr.
Time travel
By Egr (Ft, Ni Ege (Zt,
Byny cy- (Ze, t (g) Cy (21,)
(m t)) +
uy + pnp
e i cu then
Via att,)
9 Vx, log oo ( 2 ((T)
ool af((t]g
end
2t,-1 + reverse diffusion step using
if
then
P Sampling Ri; ~ -(=2s (14-))))))
v(te))
6 3(t,)² (t
#1, N (ax:; 1 .))
end for if (hegax(((VVaeeee then
= 6,
s=
else
b. End search early sample found
end if
return (xo)
G NATADIFF ABLATION STUDIES
Here we provide ablation studies examining the effect of classifier augmentations, adversarial boundary guidance, and adversarial classifier guidance. We also include visualisation of how these components influence the generated image (see Figure 5). Recall from the main paper that augmented adversarial classifier guidance introduces visual features from the adversarial class (see Appendix G.II. adversarial boundary guidance further increases the amount of adversarial structure added and improves image quality (see Appendix G.2), and time-travel sampling provides an additional improvement in image quality (Lugmayr et al. 2022) (see Figure 5).
G.1 1 EFFECT OF CLASSIFIER AUGMENTATIONS
In the main paper, we argue that classifier augmentations reduce the local adversarial signal of the surrogate classifier gradient and thereby encourage the diffusion model to introduce semantically meaningful adversarial features into the generated image. To verify this claim, we compare NatADiff samples produced with and without classifier augmentations. Using ResNet-50 (He et al. 2016) surrogate classifier, we report Attack Success Rate (ASR), Inception Score (IS) (Salimans et al.) 2016), and Fréchet Inception Distance (FID) (Fréchet 1957). For FID, evaluate with respect
20

---

Published as a conference paper at ICLR 2026
Adv. Class. Guid.
X
Class, Aug. X
Adv, Bound. Guid.
Time-Trae Samp.
X
Image
Res: 33%
Res: TP 83%
Res: TP 98%
Res: TP 32%
Res: TP 100%
Ine: T 77%
Inc: T 64%
Inc: T 72%
Inc: TCC TC%200
Inc: TP 96%
Classification
VTT: 88%
VIT: 83%
VIT: 84%
VIT: 62%
VIT: TP 83%
Res:: T 72%
Res: 74%
ARes: 65%
ARes: TP 32%
ARes: TP 98%
AIne: T 78%
Alne: T 75%
Alne: T 73%
Alne: T 52%
Alne: TP 89%
Figure 5: Effect of adversarial classifier guidance, classifier augmentations, adversarial boundary guidance, and time-travel sampling samples generated by NatADiff. Prompt= "tiger", adversarial target = "toilet paper", surrogate model = ResNet-50 (He et al. 2016). Classification scores are given for ResNet-50, Inception-v3 (Szegedy al. 2016), ViT-H (Dosovitskiy et al. 202T), and adversarially trained ResNer-50 and Inception victim models (Kurakin et al. 2018) Note: "T":
"Tiger", "TP": "Toilet Paper". "TC": "Tiger Cat".
NatA without Classifier Augmentations
NatA Diff with Classifier Augmentations
True: Snake Adv:
True: Cowboy Hat Adv: Mushroom
True: Koala
True: Ambulance
True: Container Ship True: Snowmobile
Bow Tie
Adv: Birdhouse
Adv: Headland
Adv: Kite
Adv: Shower Curtain
Figure 6: Comparison of samples generated by NatADiff under targeted attack settings with and without classifier augmentations, We use a ResNet-50 (He et al. 2016) surrogate model. We report the true class and adversarial target for each image.
both the ImageNet- Val (Deng et al. 2009) and ImageNet-A (Hendrycks et al. (021) datasets to assess how closely NatADiff samples resemble natural images and known natural adversarial examples, respectively. The exact parameter settings for each experiment are provided in Table
From Table 2 we observe that classifier augmentations significantly increase the transferability of
adversarial samples, while retaining comparable white-box ASR (see Table 2. Images generated with
Table 2: Attack success rate (%) and image quality of adversarial samples generated by NatA Diff under targeted attack settings with and without classifier augmentations. We ResNet-50 (He et al. 2016) surrogate model. Bold values highlight the best score. White-box: ASR (same surrogate and victim model) is denoted with an
Attack
CNNs
Victim Model ASR (%)
Transformers
Average
(1)
18
Val (4)
HID.
A
HD.
RN:50
Ine.x3
RN-152
AdvRess
Advine
WITH
Max.VTT VT
Swin-B
DefT
NatADiff
D6.
60.1
56.5
55.3
58
36.8
45
43.0
52.3
56.8
25.0
06.5
77.3
NatADil (No-Aug)
48.5
21

---

Published as a conference paper at ICLR 2026
classifier augmentations have slightly reduced overall image quality (IS and FID- VAL), but improved FID-A. This suggests that classifier augmentations introduce slightly more generative artifacts in an image, but also incorporate more meaningful adversarial features, which produces images that align more closely with known natural adversarial samples (see Figures and
G.2 EFFECT OF BOUNDARY GUIDANCE STRENGTH
The adversarial boundary guidance term, Jr, governs how strongly features from the adversarial class are incorporated into the generated sample. To evaluate the effect of this parameter, we conduct an ablation study across 4 E {0.0, 0.1, 0.2, 0.3, 0.4, 0.53 using ResNect-50 (He al. 2016) surrogate model and applying NatADiff in targeted mode, i.e., with adversarial classes selected at random. We report attack success rate, Inception Score (IS) (Salimans al.) 2016) and Fréchet Inception Distance (FID) (Fréchet 1957). For FID, we evaluate with respect to both the ImageNet- Val (Deng et al.) 2009) and ImageNet-A (Hendrycks et al., 202I) datasets to assess how closely NatA samples resemble natural images and known natural adversarial examples, respectively. The exact parameter settings for each experiment are provided in Table
Table Attack success rate (ASR) and image quality of adversarial samples generated by NatADiff under targeted attack settings with varying adversarial boundary guidance strength, μ.. We use ResNet-50 (He et al. 2016) surrogate model. Bold values highlight the best score. White-box ASR (same surrogate and victim model) is denoted with an ".
Attack
CNNs
Vietim Model (SR (%)
Transfommers
Average ASR
(1)
18
FID- Val (4)
A
FDD-
RN-50
Inc-v3
RN-152
AdvRes
Advinc
WITH
Max-ViT
Swin-H
Deff
NatADiff (++ = 0.0g
95.2"
54.2
49.6
48.7
53.7
32.6
42.4
43.7
48.4
52.1
26.1
67.7
78.9
NatADif 0.11
95.1"
55,2
52.5
51.5
53.9
33.8
11.2
15.0
19.3
53.4
26.6
63.6
78.1
NatADiff (u- 0.2)
96.9"
60.1
55.3
58.9
36.8
45.3
56.8
26.0
T7.3
NarADIff (1+ 0.33
97.4"
62.4
60x0
57.8
61.2
42.6
M0.7
53-4
55.0
60.1
27.6
63.8
77.8
NatADiff 0.4)
98.5"
67.8
65.2
62.5
65.5
19.3
57.1
59.0
60.7
66. 1
28.9
63.4
80.2
NatADif (N= 0.5)
71.6
68.0
70.6
53.7
62.7
80.1
We observe that attack success rate, IS, and FID-Val increase alongside (see Table 3. Interestingly, the lowest FID-A was observed μ 0.2. These quantitative results suggest that larger values of j tend to improve NatADiff performance; however, they do not capture the qualitative shift in sample structure. Large values of и introduce two distinct phenomena: dual class samples, in which both the true and adversarial classes are present in the image (see Figure (a) and (b)), and flipped class samples, in which the original class is entirely overwritten by the adversarial target (see Figure (c) and (d)). Furthermore, as seen in Figure 7 the optimal value of appears to vary across true-adversarial class pairs. Thus, we select a conservative value of μ = 0.2, as manual qualitative investigation found this did not lead to dual and flipped class samples, and experimental results indicate it best aligns with natural adversarial samples as measured by FID-A.
G.3 EFFECT OF ADVERSARIAL BOUNDARY GUIDANCE PROMPT STRUCTURE
Adversarial boundary guidance requires that the true and adversarial classes be encoded into textual prompt. To examine how this intersection prompt shapes attack performance, we compare NatADiff samples generated using different prompt templates. Coneretely, we study four constructions--<caas name>> and <class name>>" ", "A photo of <class name"> and <class name>" ", "<claas name> next <class name>" " and "<class name> behind <class name> "-and evaluate both class orderings, resulting in a total of 8 prompt formats. For each experiment we generate 150 targeted NatADiff samples using a ResNet-50 (He et al., 2016) surrogate model, and we report attack success rate, Inception Score (IS) (Salimans et al. (016) and Fréchet Inception Distance (FID) (Fréchet, 1957). For FID, we evaluate with respect both the ImageNet- Val (Deng et al., 2009) and ImageNet-A (Hendrycks et al. 2021) datasets to assess how closely NatADiff samples resemble natural images and known natural adversarial examples, respectively. The exact parameter settings for each experiment provided in Table
Overall, the prompt structure exerted relatively minimal influence on attack performance or image quality. Table 4 shows that average ASR varied by less than 7.7% across all prompt formats, indicating that adversarial boundary guidance is robust to the particular phrasing used to express the class intersection. The format "A photo of <class name of ÿン> and <class name of y>> yielded the
22

---

Published as a conference paper at ICLR 2026
Strawberry
Crossword
Stove
Ape
Lipstick
Flower
Bagel
Triumphal Arch
/ = 0.0
== 0.1
J = 0.2
= 0.3
J == 0.4
Jl = 0.5
Figure 7: Samples generated by NatADiff using the same random seed while varying p from 0.0 0.5, shown left to right. Green and red labels denote the true and adversarial classes, respectively. Images in (a) and (b) exhibit the dual class phenomenon, where large μ values cause objects from both the true and adversarial classes to appear. Images in (c) and (d) demonstrate the flipped class phenomenon, where large μ values causes the sample to fully adopt the adversarial class, suppressing the original class features.
highest average ASR, but this also coincided with the lowest IS, suggesting that the improved attack transferability may come at the cost of sample fidelity. For this reason, adopt the prompt "<class name of y> and <class name of y> ", which attains the second-highest ASR while maintaining superior image fidelity, and therefore provides and image quality.
favourable balance between attack effectiveness
G.4 SELECTION ADVERSARIAL CLASSIFIER GUIDANCE STRENGTH
The adversarial classifier guidance term, 8, governs the strength of the guidance provided by the surrogate classifier. This is analogous the guidance term used in classifier-uuided diffusion. where a trained classifier steers the sampling trajectory toward a desired class (Dhariwal & Nichol 202Tb. Classifier-guided diffusion is well-understood process that trades sample diversity for class adherence, and the optimal guidance strength varies with the underlying classifier model (Dhariwal &
Nichol 2021
Yu al. 2023 Shen et al. 2024 Dai al. 2024).
To investigate whether our understanding of classifier guided diffusion extends to adversarial classi- fier guidance, conduct ablation study on the influence of the guidance strength term, For
23

---

Published as a conference paper at ICLR 2026
Table 4: Attack success rate (ASR) and image quality of adversarial samples generated by NatADiff under targeted attack settings with varying adversarial boundary guidance prompt templates. We use a ResNet-50 (He al. 2016) surrogate model. Bold values highlight the best score. White-box ASR (same surrogate and victim model) is denoted with an ".
Procupt Stracture
CNNs
Victin Model ASR (8)
Transformers
Avenge ASR
(t)
18
FID. Val (L)
A (
FID-
RN-50
Inc-v3
RN-152
AdvRes
Advinc
VIT-H
Max- VIT
Swin-B
DefT
and
99.3"
59.3
63.3
60.00
56.7
32.0
17.3
51.3
54.0
58.4
9.2
205.0
213.2
and y
98.7"
62.0
63.3
58.0
62.0
40.7
15.7
50.0
54.0
58.2
9.0
201.1
212.0
A photo y and
98.7*
57.3
62.0
58.0
54.0
56.7
212.6
A photo of and
99.3 19.3"
67.3
68.7
63.3
62.1 7
36..7
52.7
63.3
57.3
62.4
8.4
204.1
211.6
97.3"
54.7
52.7
52.0
56.7
33.3
16.0
17.3
52.0
54.7
8.4
205.6
214.6
next y
100.00
59.3
58.7
57.3
36.7
32.7
50.0
18.7
52.0
57.3
8.8
199.1
210.2
behind
98.0"
59.3
60.0
60.0
55.3
4.1.3
48.7
$2.0
65.3
58.9
8.9
205.4
216.3
behind
58.0
57.3
$2.0
58.7
36.7
50.7
56.3
8.4
197.9
each ablation, we generate targeted NatADiff samples using using classifier guidance strengths of = 20, 50, 100, and 200. We report attack success rate (ASR), Inception Score (IS) (Salimans) ctt al. 2016), and Fréchet Inception Distance (FID) (Fréchet 1957) across ResNet-50 (He et al. 2016), Inception-v3 (Szegedy al. 2016), and ViT-H (Dooovisskiy et al. 2021) classifiers using an adversarial boundary guidance strength of μ = 0.2. We take FID with respect to both ImageNet- Val (Deng et al. 2009) and ImageNet-A (Hendrycks al. 2021 assess how closely samples resemble natural images and known natural adversarial examples, respectively. Additionally, we examine the average classifier gradient over the diffusion sampling path. At each sampling time compute the average gradient as 1 2=== av (og(p(y/a:))); The exact parameter settings for each experiment
are provided in Table
From Ta we observe that ASR increases with adversarial classifier guidance strength. Gradient stability also depends on the guidance strength, as illustrated in Figures g 9 guidance strength is low, NatADiff's ASR drops, and the gradient does not smoothly converge to 0 as / + 0. This is likely because the guidance is insufficient to push the sample into regions of the adversarial class on the classifier weak guidance. Conversely, sufficiently large guidance strengths yield substantially higher ASR and gradients that smoothly converge to 0 as the sample enters the desired region of the manifold.
and [O] When the
by low ASR scores coinciding with
We find that FID scores degrade guidance strength increases, while IS remains relatively unaffected in all cases except the ViT-H classifier. This suggests that increasing guidance strength reduces sample diversity, consistent with the conventional understanding of classifier- baassiferr-uidee guided diffusion. However, excessive classifier guidance can also "push" samples off the image manifold, as reflected by the IS degradation observed for VIT-H. This aligns with findings from the main paper, where targeted attacks against ViT-H yielded reduced image quality (see Appendix K.5).
Overall, these results indicate that adversarial classifier guidance operates according to the same underlying mechanisms as standard classifier-guidet diffusion. Following Dhariwal & Nichol (202T) Dai et al. (2024), for each classifier manually tuned by incrementally increasing until NatADiff successfully generated adversarial samples. We additionally implemented an adaptive scaling scheme that increased the guidance strength if sample failed fool the surrogate classifier.
H MANIFOLD PLOTS
In the main paper we claim that adversarial boundary guidance directs the diffusion sampling trajectory towards class intersections. To further support the validity of this claim, we visualise the learned classifier manifolds using both UMAP (Meinnes & Healy, 2018) and t-SNE (van der Maaten & Hinton| 2008), which are dimensionality reduuctioon techniques that allow the image manifolds to be viewed on a 2D plane. UMAP provides a copology-preseryi global structure, while t-SNE preserves local neighbourhood relationships the cost of deforming global structure. We use both methods to provide complementary views that illustrate how NatADiff samples relate to the true-class and adversarial-che distributions.
embedding that tends to maintain
To construct the image manifolds, we use NatADiff to generate adversarial samples for 12 true- adversarial class pairs. All adversarial samples are generated using ResNet-50 (He et al.) 2016)
24

---

Published as a conference paper at ICLR 2026
Table 5: Attack success rate (ASR) and image quality of adversarial samples generated by NatADiff under targeted attack settings with varying adversarial classifier guidance strength, s. We ResNet-50 (He et al. 2016) surrogate model. Bold values highlight the best score. White-box ASR (same surrogate and victim model) is denoted with an
Surrogate
Model
Altick
CNNs
Vistim Model ASR (56)
Tommers
Average ASR
(t)
IS
Val (1)
FID
A (C)
FID
RN-50
Inc-93
RN-152
AevRes
Advinc
VIT-I1
Max-ViT
Swin-B
DelT
NaAADiR
$5.0"
36.0
36.0
365.0
32.01
32.00
30.0
215.0
33.6
4.4
2333.
2280
NatADoff
62.07
46.0
46.0
36.0
9342.4
282.7
100)
50.0
54.0
46.0
50.0
291.3
NatADE is 200)
NatADii 96.0"
62.0
76.0
08.0
60.0
46.0
62.0
06.0
06.0
66.9
4.7
31550
326.2
30.0
46.0
25.0
36.0
34.0
Inc-
NatADif
NHADIIT 100)
30.0"
94.0
56.0
64.0
44.0
46.0
53.8
297.2
NaAADEE is 200)
72.0"
96.0
06.0
68.0
80.0
46.0
54.0
00
56.0
06.4
4.1
319.4
320.8
NatADiit
25.0"
34.0
25.0
22.00
28.0
279
36.0
240.0
296.4
NatADaT 100)
46.0"
46.0
46.0
44.0
1/2.0
42.0
48.0
49.0
45.8
3.9
262-22
279.5
NaAAME ia- 200)
78.0"
78.0
80.0
78.0
78.0
86.0
70.0
72.00
82.0
78.0
2.9
281 3
294.5
(a)
(b)
(6)
Figure 8: ResNet-50 (He al. 2016) classifier gradient with respect o; plotted against sampling time. The solid line denotes the average across the experimental run, while the shaded region is plus/minus standard deviation. The final 400 sampling steps are enhanced and displayed in the second panel of each subfigure. For each subfigure, the classifier guidance strength (s) and white-box ASR are: (a) s 20, ASR 38.0%; (b) 50, ASR 62.0%: (c) s 100, ASR 86.0%: (d) 200, ASR = 96.0%. Please zoom in for improved visibility.
(n)
(b)
(c)
(d)
Figure 9: Inception-v3 (Szegedy et al. 2016) classifier gradient with respect to æ; plotted against sampling time. The solid line denotes the average across all experiment runs, while the shaded region plus/minus standard deviation. The final 400 sampling steps are enhanced and displayed in the second panel of each subfigure. For each subfigure, the classifier guidance strength (s) and white-box
ASR are: (a) s = 20, ASR = 46.0%; (b) = 50, ASR = 70.0%; (c) s = 100, ASR = 94.0%; (d) 200, ASR 96.0%. Please zoom in for improved visibility.
25

---

Published as a conference paper at ICLR 2026
(a)
(b)
(c)
Figure 10: ViT-H (Dosovitskiy et 2021 classifier gradient with respect æ; plotted against sampling time. The solid line denotes the average across the experimental run, while the shaded region plus/minus standard deviation. The final 400 sampling steps enhanced and displayed in the second panel of each subfigure. For each subfigure, the classifier guidance strength (s) and white-bon box ASR are: (a) s 20, ASR 30.0% (b) s 50, ASR 38.0%, (c) 100. ASR 52.0%. (d) 200, ASR = 86.0%. Please zoom in for improved visibility.
surrogate model, and we pass UMAP and t-SNE the penultimate-layee feature embeddings from this classifier. The NatADiff samples are plotted alongside the feature embeddings of clean images from the true and adversarial classes taken from the ImageNet dataset (Deng et al., 2009). The resulting visualisations are provided in Figures and 12
Across both embeddings, NatADiff samples consistently form one or more distinct clusters that fall between the manifolds of the true and adversarial target classes. This positioning directly supports the main paper's s claim: NatADiff guides samples toward class intersections, generating samples that naturally fall near class boundaries. Furthermore, this aligns with the established understanding of natural adversarial samples (lest-time errors), which are believed to arise when classifiers rely on spurious contextual cues rather than truly discriminative features. As sample moves closer to classifier's 's decision boundary, the prevalence of contextual cues from the adversarial class increases, which provides natural explanation for NatADiff's s strong attack performance.
26

---

Published as a conference paper at ICLR 2026
Guldlish
Mushroom Pickat NatACe!
Deing Table Makypun husto
Mantowy
NasAAD
Thimtte Thettic
Cleada NatAner
Polaroid Carrera Sieeping Beg
Sombroro NT
Snowplow
Nartkb
Cheessburger
Pamtbruth
Weot Destrng
Banena
Marchalic
hatAD
Figure 11: UMAP (MeInnes Healy 2018 plots of NatADiff samples alongside the ground truth manifolds. Green and purple samples are real images of the true and adversarial classes taken from the ImageNet dataset (Deng al. 2209). Red samples are generated by NatADiff using ResNet-50 (He et al., 2016) surrogate model. UMAP is applied to embeddings taken from the penultimate layer of ResNet- 50 classifier. Please zoom in for better visual.
27

---

Published as a conference paper at ICLR 2026
Mushroom Packect NasAA
Diving Tool Porspus NozbDer
Moniey
NaSADW!
Nac4Dif
Cheessburger
Sarbage Tuck Sabeepoww NatADa!!
May
Benera
Or
Chesseburgar
Paintrrssh
Distrag
IT
Figure 12: t-SNE (van der Maaten & Hinton 2008 plots of NatADiff samples alongside the ground truth manifolds. Green and purple samples are real images the true and adversarial classes taken from the ImageNet dataset (Deng et al. 2009). Red samples are generated by NatADiff using a ResNet-50 (He et al. 2016) surrogate model. i-SNE applied embeddings taken from the penultimate layer of ResNet-50 classifier. Please zoom in for better visual.
28

---

Published as a conference paper at ICLR 2026
I NATADIFF ON OXFORD PETS
In the main paper, we present results for NatADiff when targeting ImageNet classifiers exclusively. To strengthen the claims made in the main paper, we extend our evaluation to the Oxford Pets dataset (Parkhi et al. 2012). which is more fine-grained and presents unique challenge as opposed to ImageNet. In this setting, the adversarial structural elements introduced by NatADiff will need be more subtle.
We use NatADiff to attack ResNet-50 (He al., 2016), ResNet-152 (He al. 2016), and ViT (Dosovitskiy et al. 2021) classifiers with weights sourced from HuggingFace (Wolf et al. 2020). For each class Oxford Pets, we generate adversarial samples using both targeted (randomly selected) and untargeted Similarity-basee variants of NatADiff. We compare NatADiff against PGD (Madry et al. 2018), AutoAttack (Croce Hein 2020), and adversarial classifier guidance (AdvClass) (Dai et al. 2024). We exclude comparisons with NCF (Yuan et al. 2022). DiffAttack (Chen et al. 2025). and ACA (Chen et al., 2023b), as these methods do not provide variants targeting
Oxford Pets classifiers. We report attack success rate (ASR) and the image quality metrics BRISQUE
Miittal et al. 2012) and TReS (Golestaneh et al. 2022).
observe that NatADiff behaves similarly the ImageNet study, outperforming other methods in terms of attack transferability while maintaining competitive white-box ASR. NatADiff also typically achieves better BRISQUE and close to AdvClass terms of TReS, suggesting that NatADiff produces high-fidelity samples, as shown in Figure [3 Overall, NatADiff generalizes well fine-grained datasets, with adversarial boundary guidance extending minute structural changes. These results are consistent with the strong performance of similarity- -taretted attacks in the ImageNet study, indicating that adversarial boundary guidance particularly effective when classes share similar structural elements.
From Table
Table 6: Attack success rate (%) and image quality of adversarial samples generated for the Oxford Pets dataset (Parkhi et al. 2012). .). Superscripts T and denote random and similarity targeted attacks, respectively. Bold and underlined values highlight the best and second best scores. White-box ASR (same surrogate and victim model) is denoted with an Note we do not report image quality for constrained perturbation- based attacks (these attacks make imperceptible image alterations).
Surrogate
Victim Model ASR (%)
Average ASR
Attack
BRISQUE (L)
TReS (t)
Model
ResNet-50
ResNet-152
VIT
Clean
6.0
8.6
3.8
6.1
PGD
100"
10.3
3.8
38.0
AA
100'
8.1
4.3
37.5
AdyClass
99.5"
17.3
19.5
45.4
5.3
80.9
ResNet-50
AdvClass
98.9"
19.5
16.8
45.0
5.0
81.3
N
97.8*
30.8
27.6
52.1
4.2
78.9
NatADif
96.8*
37.3
35.7
56.6
5.5
80.2
PGD
14.6
99.5*
9.2
41.1
AA
12.4
100*
6.5
39.6
AdvClass
34.6
98.4"
22.2
51.7
7.2
80.3
ResNet-152
AdvClass
37.3
98.9*
26.5
54.2
6.9
80.9
51.9
95.7*
37.3
61.6
5.7
79.4
N
57.8
98.9*
48.6
68.5
5.2
79.6
PGD
13.0
10.8
100"
AA
7.6
8.6
97.3*
37.8
T
28.1
22.2
100*
50.1
6.7
80.2
VIT
AdvClass
24.3
24.3
100"
49.5
6.7
80.1
56.2
29.2
100"
61.8
4.9
80.3
54.6
46.5
100*
67.0
5.3
80.1
29

---

Published as a conference paper at ICLR 2026
ResNet-50
ResNet- 152
VIT
True: Maine Coon
True: Pomeranian
True: Bombay
True: Pit Dall Terrir
Trae: Spbynx Adv': Abvssinia
True: Havanese
Adv
Adv': Havuaese
Ad=1: Russian
Adx": Pumerazian
Adv Newfouddind
True: Birmas Ad Ragdall
True: Esypiian Mau
True: Bengle Adv
True: German Shorthaired Adx Hoand
True: Keeshound Adv": Lembbrger
True: Puz Adv Chilusius
Adv German Shanhaised
Hound
Figure 13: Adversarial samples generated by NatADiff for the Oxford Pets dataset Parkhi et al. using ResNet-50 (He et (016), ResNet-152 (He et al. (016). and ViT (Dosovitskiy et surrogate models (see column labels). We report the true class and adversarial target for each image. Superscripts T and U denote random and similarity targeted attacks, respectively.
2012
2021
30

---

Published as a conference paper at ICLR 2026
J NATADIFF EXPERIMENT PARAMETER SETTINGS
Here we provide the NatADiff parameter values used in our experiments (see Tables 7 and 8
Table 7: Experiment parameters used in diffusion- based adversarial sampling experiments on the ImageNet dataset (Deng et al., 2009). Parameters refer to those defined Algorithms and 2 Experiments were conducted with ResNet-50 (He et al. 2016), Inception-v3 (Szegedy et al. 2016)) surrogate models.
and ViT-H (Dosovitskiy al. 2021
Surrogate Model
NatADiff Parameters
Attack
p
71
Fu
Cu
S
5
6
NatADiff (Aug)
7.5
7.5
0.2
50
5
500
800
0
700
5
0
15
NatA (u= =0.0)
7.5
7.5
0.0
50
500
800
700
5
0
NatADiff (H =011)
7.5
7.5
0.1
50
500
800
700
5
0
15
N (== 0.2)
7.5
7.5
0.2
50
500
800
700
5
0
15
NatADif (m=0.33
7.5
7.5
0.3
50
500
800
700
5
O
NatADif (==0.4)
7.5
7.5
0.4
50
5
500
800
0
700
5
O
15
NatADiffT (#=0.3)
7.5
7.5
500
700
5
RN-50
NatADiffT (Prompt)
7.5
7.5
0.2
50
5
500
800
700
5
O
15
NatADiff (s -20
7.5
7.5
500
700
NatADinTT (s 50)
7.5
7.5
0.2
50
500
800
700
5
O
15
NatADifT (s 100)
7.5
7.5
0.2
100
500
800
700
5
O
15
NatADiff" (s 200)
7.5
7.5
0.2
200
500
800
700
5
D
15
AdvClass
7.5
0.0
0.0
500
0
200
5
O
250
AdvClass
7.5
0.0
500
200
5
0
NatADiff
7.5
7.5
0.2
50
5
500
800
0
700
5
0
15
NatADif
7.5
7.5
50
500
800
700
5
25
NatADiffT (s 20)
7.3
7.5
500
700
5
NatADiff (s 50)
7.5
7.5
0.2
50
5
500
800
700
5
0
20
NatADin (s 100)
7.5
7.5
0.2
100
500
800
700
5
0
20
Inc-v3
NatADinT (s 200)
7.5
7.5
0.2
200
500
800
700
5
0
20
AdvClass
7.5
0.0
0.0
500
200
5
O
250
AdvClass
7.5
0.0
0.0
500
200
5
O
250
NatADinTT
7.5
7.5
0.2
50
5
500
800
0
700
5
O
20
NatADif
7.5
7.5
50
500
700
5
0
NatADiff (s 20)
7.5
7.5
0.2
20
500
800
700
5
0
NatADiffT (s 50)
7.5
7.5
0.2
50
5
500
800
700
5
0
50
NatADifTT (s 100)
7.5
7.5
500
700
VIT-H NatADift (s 200)
7.5
7.5
0.2
200
5
500
800
0
700
5
0
50
AdvClass
7.5
0.0
0.0
500
200
5
0
250
AdvClass
7.5
0.0
0.0
500
0
0
0
200
5
0
250
NatADiffT
7.5
7.5
0.2
100
500
800
0
700
5
0
50
NatADift
7.5
7.5
0.2
100
500
800
700
5
0
50
31

---

Published as a conference paper at ICLR 2026
Table 8: Experiment parameters used in diffusion- bassd adversarial sampling experiments on the
Oxford Pets dataset (Parkhi et al. 2012). Parameters refer to those defined Algorithms and 2
Experiments were conducted with ResNet-50 (He et al. 2016), ResNet-152 (He al. 2016) and ViT (Dosovitskiy et al. 2021) surrogate models.
Surrogate
NatADiff Parameters
Attack
Model
w
8
R
Ti
Fu
Ct
Cu
8
AdvClass
7.5
0.0
0.0
500
0
200
0
250
AdvClass
7.5
0.0
0.0
500
0
0
200
0
250
RN-50
NatADiff
7.5
7.5
0.2
50
5
500
800
0
600
5
0
25
N
7.5
7.5
0.2
50
5
500
800
0
600
0
25
AdvClass
7.3
0.0
0.0
500
0
200
0
250
AdvClass
7.5
0.0
0.0
500
0
0
200
5
0
250
RN-152
NatADiffT
7.5
7.5
0.2
50
5
500
800
0
600
5
0
25
NatADif
7.5
7.5
0.2
50
5
500
800
0
600
0
25
AdvClass
7.5
0.0
0.0
500
0
0
200
0
250
AdvClass
7.5
0.0
0.0
500
0
0
200
5
0
250
VIT
N
7.5
7.5
0.2
50
5
500
800
0
600
5
0
25
U
7.5
7.5
0.2
50
5
500
800
0
600
0
25
32

---

Published as a conference paper at ICLR 2026
K ADDITIONAL NATADIFF SAMPLES
We provide NatADiff samples alongside the classification scores of ResNet-50 (He et al. 2016).
Inception-v3 (Szegedy et al., 2016). ViT-H (Dosovitskiy al.) 2021), and adversarially trained
ResNet-50 and Inception victim models (Kurakin et al. 2018). Samples were generated using
ResNet-50 (He al. 2016), Inception-v3. (Szegedy al. 2016), and ViT-H (Dosovitskiy al. surrogate models.
K.1
MIXED SAMPLES
NesNet-50
Inception-v3
ViT-H
Golffich
Mushmom
Thur:
Thienta
Traec Cicado
Thur Pelanid
Adv:: Plagquc
Adv
Adr':
Ads Shupping Inlamed Camama 78%
Modkey 5215
Res: Paduct
Trafle 23
Bubsled
Ias: lis Monkey 7256
Inc: E-tton 309%
loc: Haryysn 1005;
laxz tnatie Lagle 10005
Leaf Hosts 18%
VI
Guidiiih d875
VTB T-stin 8715
VID Dising Table 100%
WiT Sahoheker 6995
vin Bobiled 10075
VID Slepping Reg 100
AdvRes: Gebltib 60%
AdvResi Baloo 2015
AddRes Parspun 82%
AdvRes: Trafto Ligh Bang
Advitees Cloado 70%
AdoResi Seepiig Bag 977
Moskes
Patspto
Advlla:: liessi
Advlle:
Palaned Canera
Tras: Dunnci
Truc: Glasbazs Track
Trur: Hay
Tras: Cheendnager
Truc: Parsterds
Trur: Wood
"
Ad
Adk":
Adv
Adr Maxtsiik
Adr":
Rei Sunibeero 97%
Roc: Saueplow 01
Rest Hay 64%
Roi Clecselagge L458
Ree Pasedacal 2015 2015.
Ros: Wood 1.0
Iex: Senther 9773
Inc: Xnowglow 577
Inr: Ox 1003
Bar: Bonona 100%
Inc: Masdwaidi 7156
Inc: Clars Amorr 265
VTE Bienet 90%5
VIT: Sasaylooe 66%S
vIth Hay 10005.
MDD Checsebarge 08%
VIB Maccolick 100%
YID Didkng 100.%
AdvRRc Semibeemo we55
AdvRuus Snorspoww
Chanssharger
Piinttaa sal
Liitrry 93%%
Gurhage
100%
Figure 14: Adversarial samples generated using NatADiff with ResNer-50 (He et al, 2016), Inception- v3 (Szegedy et al.) (2166), and ViT-H (Dosovitskiy et al. 2021) surrogate models (see column labels). We report the true class, adversarial target, and classification scores of the surrogate and adversarially trained ResNet-50 and Inception victim models (Kurakin et al. 2018). Superscripts and U denote random and similarity targeted attacks, respectively.
33

---

Published as a conference paper at ICLR 2026
K.2 RESNET-50 SAMPLES
Troe Tegeor
True: Shold
True: Steert Triin
Trae: Poxeegrtacte
Troe: Ox
Trur: Bam
A.d"": Mushed Poiso
Adr Ihopel
Adv-:: Schmouine
Adv Thrre Tred Skoth
Adr Reiaary Sporid
Adc Ring Rinder
Roc Madied Potaso 9695
Rex: Juppod 6436
Rex: Subesaiiz 58%
Roc Thece foed Stodk 61.5
Rec Baitua Sousiel 67%
Rex: Risg Bander 7505
Jsguar
lurpot
Sateuritr
Pemegrazae 80%
Hertany Spania 77%
Riep-Widder 30%
Siguar
Three Teed
Pouzegranze
Speniel
Ban
Traac Botaled
Trur: Candle
Trur: Caroe
Tranz China Calrrd
Trag: Foumiaa
Trues Stove
T Wallicural
Ad-": Pafase
Wase
Adv Kife
Adr Trsewiter
Ads
Mee Biseston Wolfhoordd 1197
Mee: Pertimm 905%
Reez White ilme 1856
Ree thit 98%
Roc Typeerbar 58%6
Ape 7385
Inc: Barsiaa Wolfhooadd 14%
Inc: Perlane 88%
lnc: While Hore 15%
Bac: Rille 8216
Inc: Ditch Oves 8018
Inc: Hougl 07S
WiTe Butsled 1000
VIT's Candle USSC
VIl:Ccmm Cane 84%
Wilt: Chrro Chrra-Caeeze 26%
Vils hosetais 90%%
WTT: Skore 95%9
AdvResi Boboded 5095
AdvBles: Cuodle 76%
AdvRes Caace 44%
AdvRRes: China Cableet 7a58
Adritesi Duxii Over 8338
AdviRes Stove G8%
sieeping Bar
Phntiame
Homm 37%;
Dinuh Ovezz
Bellopood Adr
Cantigon
Boahhose
Hatpper
Dogited
lamppr
67%
NTB 90%
Dogiled 18%
Bulleppoof 90%
Cadigas 83%
AdvRas: Bruibatne SU%
Ads Bac Hamper 241% Adsilaci Hamper 6416
Adviitee Lhspled sa2t Adviliec Bulisled 26%
AdvRte: Crucch 1756
AdvBaa: Caamas 575
Advitee Cantigas 74%;
Advinc Bam 60.6
Advilee Crunch 70%
Adrlnc: Ciines 04%5
Advilen: Kinono 64%
Car Whod
Time: Cowhhey
Trart Leafer
Menqrito
Tin--O--Th--Wooad
Tiurs Airlleer
Petur's Whrzl
Ads Hair Bontl
Adx Prrnds Laod
Ad Window Serees
Ads Gynmin
Ads Spuer Sharts Res: Space Stuzte 85% Airlinze
Res Potte's Wheel 700%
Res: Boei Bottle 62%
Rost Fondd Lost 115
Rest Window Screes 60%
Res Gynontt 98%
Poller's
Wincow Sernen 53%%
Cream 64%
8995
Mosqalto
Aoor Squash
8a%
Brake 26%5
Advlac:
Figure 15: Adversarial samples generated by NatADiff with a ResNet-50 (He al., 2016) surrogate model. We report the true class, adversarial target, and classification scores of ResNet-50 (He et al. 2016), Inception-v3 (Szegedy et al. 2016), ViT-H (Dosovitskiy et al. (202T) and adversarially trained ResNet-50 and Inception victim models (Kurakin al. 2018). Superseripts T and U indicate targeted and untargeted bsiiilariy-bbased) attacks, respectively.
34

---

Published as a conference paper at ICLR 2026
K.3 INCEPTION-V3 SAMPLES
Troe: Geldt1
Troe: Desioup-Crrpo
True: Disk Broke
Truel Fouther Boo
Tres Footull Hellxt
Truc: Foue: Protes Bed
Adv"; Dven
Adr Pendar Car
Adx Preted
Adv Hen Tr- Woods Roy Mailicom 12%
Adv Ram
Adv Degdie
Rac Cor 2009
Rex: Deskoop Compuser 22%
Rex: Protacl 4855
Roc Bus 558
Rex: Digulol 4295
dec 1007s
Pereon 100%
Thn-- The-Woatss 80%
Bux: 100%
Depilad
Deklap Compater
9515
Four
wofk
37
Tras: Far Ciud
Truc: Grund Piazo
Trur: Geambooee
Truc: Janes I-Sint, Toe Start
Truc: Larzpobade
True: Muse
Ady:: Barmeeea
Ads Nastbok
Canoe
A Plaaabrkeen Bei Vestneer 995
Adv Sonk
Adv Wiaer
Roc For Cloz 45%
Rex: Gruad Piaro 22%
Ras: Canoe 315
Roc Lampoladee 26%
Rex: Soupp Boxd 20%
Inc: Banssoouta 10055
Inc: Notebooi 09%5
Ine: Canoe 100%
Iac: Pizzctarion 100%
Bnc: Skank DoCS
Inc: Wates 92%
WT: Cost 100%
VTT Grasd thane wnSc
WTT: Grechvare 90%
WiBs Fnwehall listacl e
Vit: Lampd we56
Vil: Fig 42%)
AdvResi Baracceta 8005
Ads Best Notbook 88%
AdeRes Greeahouse 70%
AdvRes: Ciaara G475.
Adviess Stork 80%
Adriees Plate 8718
Advinc: Harmensta 74%
Nosbbok
Canoe
Advlla Maiios 2055
Advine: Skaret 53%
Ade Ivtter
Waidhain
Lenson
Guscumade
Dapesser
Lenon 10076
Pizza 100'5
Guocutole
10075
Water
100%
WT: Daer 59%
VTs horple 10055
VIT: linooot 80%
VITS Windeer The 90% AdvRes: Bow Ti 9075
VoTl Water Tower 100%
WI:: Wishharn 97%
MdrRes Lemos 51%
Adrieei Paza 8015
AdvResi Gracanole 8a%s
AdvVRes Water lig 4078
Adviess Soap Dipener 100%
Advine: Lemes 80%
Advllne: Ivapse 13%
Adv-lac: Osacamole t186
Advilee: Soo 9296
Advincc Water Jng 1056
Advinc: Soap Unperer 100%
Wardinder
Trie: Telnvisdon
Trur: Stzge
Trur: Shirld
Mexrroin fike
Tra Machole Dover
Ad
Medkcaee Chou
Adr Radio, Waelese
Adc Swiss
Ad Bresspiazz Row: Siuiia 2105
Adv" Tiaden
Adr Deocnut Lobyrtan 22%%
Roc Medicme
Winciox 21%
Sostng 54F
Mouniiin 19%
Mellcise
Radio,
Breasqpiate
Stage
Breasppate
80%
Advlne:
Medictee Ches 87%
Adblirr: Radio, Windles 98%
Advilicc Singe 1896
Advllce Resetplio 80%
Aiviin: Tandon 50%
Advinc: Monhela 319%
Figure 16: Adversarial samples generated by NatADiff with Inception-v3 (Szegedy al. 2016 surrogate model. We report the true class, adversarial target, and classification scores of ResNet- 50 (He et al.) 2016). Inception-v3 (Szegedy et al., 2016). ViT-H (Dosovitskiy al., [021], and adversarially trained ResNet-50 and Inception victim models (Kurakin et al. (018)) Superscripts T and U indicate targeted and untargeted (similarity-baar -baed) attacks, respectively.
35

---

Published as a conference paper at ICLR 2026
K.4
VIT SAMPLES
Troe Chiflieier
Troe: Stooe Will
True: Dan
Trae: House Fiech
Troe: Sleeping Bag
Trurt Saowineile
Tabacco Stop
Ads Aogrur
Ad Plggy Bank
Adv": There Pote
Adr Decart
Adc Shower Canain
Rac Tobacoo Shop .596
Rex: Stoee Wull 1805
Rex:
Pagor Bank 18%
Rac Toaan Polc 455
Ree Stezkdee 2035
Rex: Four Polier
Iec
Step
Jagias
Tocm
Shower Cantaaa
Lespard
Plaay 77%
Teyslop
Dan
Theone
Teur: Stecatairtt
Trur: Mozsith
Trasc Vile Boof
Trusc
Trues Iray
Add::
Ad-T:
Ads Quai
A
Ads Velleball Try 7%%
Mec Puddile 1056
Hee: blakk S1256
Megalnn 996
Rec Apron 985
Roc Piecapple 29%
Inc: Cloyg 8515
Inc: Spenda 25%
lne: Woun Feass 1855
Bac: Aprou 8758
Inc: Plaviaed 121
Ine: Valley bull 85
WiT: Flanwor 766%
VIT Node Bimoe 100tc
vil:: Queal 75%
WiTt Agene RutC
Vit Pracpple 1005
WT: Welleyboll ko%%
AdvResi Woodle Spoce
AdvBlest StU Mask 8818.
AdvRes Chiklaade 81.8
AdvRess Tile Root Dasg
AdrRes: Otter 57%
AdvRees Volleyball D0%
Spaane
Adv
lioe
Adtclin::
Ditling
1.patici
D Boathose
Harpee 71%
Sleapina
Lipetik
fag
30%
Sterping 100%
NTT
100%
MTD Contater Ship 99%
Wite 9ats
AdvRaa:: Uhmos 8456
Ads Bun Burn 2155
AdvRtte Cradle 2155
AdvRte: Cress 65%%
Advlbar Cheuch 2.5
AdviBee Prlliw 46%
Advine: Altar 3077
Advilaer Bum 80%
Adviliec Cralle 736
Advine: Driling Plasfor 54.% Adriner Red Wase 20%6
Adviler: Sleping Rog 3016
Trus: Repesood
Trasc
Croeeborson
Trur Know
Ad
Cralle
Ads Com
Ads Lemon
Adv":: Cillorpot
Ads Aptary
Ads 1ilkow
Ree
Plaze Reck 2495
Res: Lemos 4335
Ros: Repeseed 4375
Ros Coffaged 2195
Res: Malloos 1075
Res: Kine Pull L01S
Lemon 25%
1285
Itans
Colleppor
Aglacy 26%
5779
Advlac:
Chocsinonse
Advllac:
Figure 17: Adversarial samples generated by NatADiff with VIT-H (Dosovitskiy al.) 2021) surrogate model. We report the true class, adversarial target, and classification scores of ResNet- 50 (He et al. 2016), Inception-v3 (Szegedy et al., 2016), ViT-H (Dosovitskiy et al., [02T), and adversarially trained ResNet-50 and Inception victim models (Kurakin et al. (008). Superscripts T and U indicate targeted and untargeted (similarity-brar based) attacks, respectively.
36

---

Published as a conference paper at ICLR 2026
K.5 LOW-QUALITY TARGETED VIT SAMPLES
Targeted ViT-H samples show degraded image quality, as seen in Figure [8 and supported by the IS and FID-Val scores in Table Targeted attacks typically blend features from disparate classes, which places greater demands on the diffusion model to locate feasible point on the image manifold. ViT-H is a strong classifier, and its decision boundaries between unrelated classes appear to more accurate than those of the other classifiers we examined. As a result, the diffusion model struggles to generate feasible targeted adversarial samples for ViT-H, and artifacts are introduced. These artifacts substantially degrade image quality and artificially inflate the attack success rate of the targeted ViT-H samples (as seen Table []. Importantl, these artifacts occur only the ViT-H targeted setting, which is directly observable from the image-quality metrics in Table 1
Trog Geldlisch Adv
Tru:: Hose Carl Ads Dowitcher (Binn)
Trur: Mourzaia Bikz Adv Maichaick
Tran: Parnleush Adv':
True: Pazcitate
Trug: Seviaz Machite
Adr Caeal Purea
Ade Roower
Tro Spocs Car Adr Kimoo
True Steam Locuneiiee
Trurj Mikane Ads Candooo (Plant)
Troe Banhaal Playrr AdV Articolk
Troe Com Adr Liana
Trur Seachore Ads Eppans
Ads Typeneiter
Figure 18: Low-quality adversarial samples generated by NatADiff with VIT-H Dosovitskiy et 202T) surrogate model under targeted attack settings. We report the true class and adversarial target each image. Superscripts and U indicate targeted and untargeted (similarity- -baedd) attacks, respectively.
L ADDITIONAL IMAGE QUALITY METRICS
We use NIQE Miittal et 2013 BRISQUE Mittal al. 2012), and TReS (Golestanch et al. 2022
to provide additional no-reference image quality evaluations of adversarial sampling methods. We assess the image quality of NCF (Yuan al. 2022), DiffAttack (Chen al. 2025 ACA (Chen et 2023b), adversarial classifier guidance (AdvClase Class) DDai et al. 2024 and NatADiff across ResNet-50 (He et al., 2016), Inception-v3 (Szegedy al. (216)) and VITHH (Dosovitskiy al. 202T) surrogate models. These metrics more closely align with human perception of image quality, but they do not address adherence to target data distribution, i.e., how well samples "fit into" the ImageNet dataset.
Adversarial classifier guidance and DiffAttack frequently outperformed other methods NIQE, BRISQUE, and TReS (see Table 9) however, as discussed Section 5.2 this is likely because these methods apply constrained perturbations to source images and clean stable diffusion outputs, respectively. When considering only methods that make structural image alterations, we see that NatADiff outperforms NCF across all image metries, and outperforms ACA NIQE and BRISQUE, with similar TReS scores that slightly favour ACA. This supports the findings from the main paper that NatADiff is able construct visually high-quality adversarial samples.
M RESISTANCE TO ADVERSARIAL DEFENCES
It has previously been shown that perturbation-baser adversarial attacks are sensitive to image transformations such as rotations, crops, and translations whiccc which can substantially reduce attack success rates (Guo al.] 2018). In addition such transformation-based defences, purification approaches aim to remove adversarial noise prior to classification. One such method is DirfPure (Nie et al. 2022), which leverages denoising diffusion model project adversarial samples back
37

---

Published as a conference paper at ICLR 2026
Table 9: Image quality adversarial samples generated using ACA (Chen al. 2023b), DiffAttack (Chen et al. 2025), adversarial classifier guidance (Dai et al. 2024), and NatADift. Bold and underlined values highlight the best and second best scores for each surrogate model. Superscripts and U denote targeted and untargeted attacks, respectively. Note that we report FID with respect to ImageNet- Val (FID-Var Val) and ImageNet-A (FID-A).
Surrogate
Image Quality Metrics
Attack
Model
IS (f)
FID- VID-Val (L)
FID-A (4)
NIQE (L)
BRISQUE (L)
TReS (f)
Clean
55.0
58.0
94.7
4.6
18.1
88.6
NCF
30.4
69.7
85.5
5.5
19.8
68.9
DiffAttack
26.8
64.1
76.8
5.7
17.7
81.8
ACA
23.9
65.0
77.9
6.8
24.4
80.8
RN-50
AdvClass
38.3
48.9
92.4
4.3
11.9
81.8
AdvClass
38.5
50.2
92.7
4.6
12.2
81.3
NatADinT
26.0
66.5
77.3
4.8
12.3
76.0
NatADiff
43.2
51.4
95.9
4.8
12.0
77.9
NCF
31.7
69.1
83.0
4.7
19.0
76.3
DiffAttack
33.2
63.7
78.2
5.8
18.0
81.2
ACA
23.1
68.0
78.8
7.8
28.4
78.7
Inc-v3
AdvClass
33.7
51.0
89.2
4.5
12.3
81.4
AdvClass"
39.7
49.4
93.3
4.5
12.3
81.2
NatADif
27.7
66.6
78.2
4.8
12.4
76.6
NatADiff
47.0
50.5
98.9
4.7
11.7
78.9
NCF
39.8
63.1
86.4
5.6
20.0
70.1
DifTAitack
35.2
63.4
80.0
6.0
18.4
81.3
ACA
25.5
64.2
80.9
7.4
25.5
79.2
ViT-H
AdvClass
38.9
48.5
95.2
4.2
14.8
77.7
AdvClass
39.2
48.5
98.8
4.3
13.5
79.8
NatADif 15.3
88.0
93.5
4.9
20.9
74.7
NatADift
31.9
53.9
96.2
4.6
14.8
78.8
onto the natural image manifold. Given an adversarial image, Zg. the forward diffusion process, p(x, 22g = 0. ). is applied for & T (recall that 7 is the termination time of the forward process), introducing Gaussian noise without fully destroying the original signal. The sample is then passed through the reverse-time diffusion process (Anderson, 1982) or flow ODE (Song et al. 202 (b) to recover purified image. Nie et al. (002) empirically demonstrated and theoretically proved that this procedure effectively removes perturbation-ban basedd adversarial noise, enabling the reverse process to reconstruct "clean" version of the image. Intuitively, the noise injected during forward diffusion overwhelms the adversarial signal, allowing the diffusion model to project the corrupted sample back onto the natural image manifold.
We evaluate the robustness of standard image transformations and DiffPure against adversarial samples generated by PGD (Madry al. 2018). AutoAttack (Croce Hein 2020), NCF (Yuan) et al. 2022), Diff'Attack (Chen et al. (202)), ACA (Chen et al. 2023b), adversarial classifier guidance (AdvClass (Dai al. 2024), and NatADiff. We use ResNet-50 (He al. 2016 surrogate model and test NatADiff and adversarial classifier under both targeted and untargeted modes. Defences are applied as pre-processing step classification; for image transformations, we average classification probabilities over augmented views obtained via cropping, rotation, and grayscale conversion. To quantify defence effectiveness, we report attack success rate (ASR).
Our results show that transform- purification did not meaningfully reduce the efficacy of NatADiff, though it successfully defended against PGD attacks (see Table In contrast, DiffPure provided much stronger defence to most attacks, reducing NatADiff's average ASR by 7.9%. However, DiffPure occasionally degraded overall classifier accuracy, leading to increased ASR for non-surrogate classifiers. This likely occurred when the reverse diffusion process failed to recover the original image, rendering classification unreliable. Compared to other attacks, NatADiff still exhibited superior white-box performance and achieved either best second-best transferability across all vietim
transform-puriffcation
38

---

Published as a conference paper at ICLR 2026
classifiers. Interestingly, NCF showed significant increase in transferability under the DiffPure defence. We hypothesize that this stems from NCF's color-based attacks pushing samples into low-rrobability regions of the manifold during the forward diffusion process. This increases the likelihood that DiffPure fails to recover the original image in the reverse process, thereby degrading classifier accuracy.
Consistent with findings in the main paper, NatADiff achieved best or near-best performance under both transformation and DiffPure defences. Given that natural adversarial samples are known bypass perturbation- brrere defences and image transformations (Agarwal et al. 2022), these results further support our claim that NatADiff generates adversarial examples that are more semantically aligned with naturally occurring test-time errors.
Table 10: Attack success rate of transform and DiffPure- DiffPure-puriffed purified adversarial samples generated
by PGD (Madry al. 2018), AutoAAtack (Croce Hein 2020), NCF (Yuan al. 2022), DiffAttack
(Chen et al. 2025), ACA (Chen et al. 2023b), adversarial classifier guidance (AdyClass (Da et al.
2024), and NatADiff. Samples are generated using ResNet-50 (He al.) 2016) surrogate model.
Bold and underlined values highlight the best and second best scores for each purification method.
Superscripts and U denote targeted and untargeted attacks, respectively. White-box ASR (same
surrogate and victim model) is denoted with
Purification Method
Victim Model ASR (%)
Average ASR
Attack
CNNs
Transformers
RN-50
Inc-v3
RN-152
AdvRes
Advine
VIT-H
Max VIT
Swin B
DefT
Clean
5.3
7.6
2.9
3.0
5.8
10.9
3.8
4.5
7.4
5.7
PGD
99.4"
11.8
5.2
4.9
10.5
4.4
5.5
8.2
17.6
AA
100"
13.3
10.0
3.9
8.8
10.5
5.4
5.6
8.0
18.4
NCF
74.8"
33.4
37.3
28.2
31.2
17.2
24.0
31.7
37.2
35.0
DiffAttack
92.5*
47.1
35.3
43.3
28.4
42.4
38.9
47.2
None
ACA
78.8"
53.3
52.7
19.8
53.1
41.8
46.4
19.3
50.6
52.9
AdvClass
99.6"
35.0
32.1
31.4
33.5
25.8
30.0
30.8
32.8
39.0
AdyClass
99.97
42.5
44.3
38.7
41.1
29.7
37.6
39.1
45.7
NatADir?
96.9"
60.1
56.5
55.3
58.0
36.8
45.3
49.0
52.3
56.8
NatADiff
99.3"
68.3
72.1
65.3
66.8
45.3
64.1
65.2
67.0
68.2
PGD
14.5"
12.1
4.8
4.7
7.4
9.2
3.5
5.2
7.3
7.6
AA
79.4"
12.4
7.9
3.3
10.2
3.4
7.3
15.4
NCF
60.2*
35.1
39.6
28
31.2
16.7
27.1
37.7
34.4
DiffAttack
73.9"
48.6
50.1
339.9
45.8
28.6
43.4
46.7
399.0
46.2
Transform ACA
64.2"
54.8
52.2
50.4
56.4
40.8
47.1
51.99
51.0
52.1
AdvClass
35.2"
33.4
30.4
29.7
31.5
25.5
28.9
31.4
32.1
30.9
AdyClass
65.8"
39.8
38.3
40.0
36.3
37.0
38.4
40.6
NatADiff
85.7"
59.8
55.0
56.8
36.1
16.7
48.6
52.4
55.2
NatADiff
96.6*
68.7
73.3
67.5
65.4
70.3
PGD
21.9"
30.8
20.3
23.3
26.5
21.3
16.7
19.4
20.6
22.3
AA
23.5*
32.5
22.7
28.4
21.7
18.9
23.6
23.6
NCF
68.8"
59.9
60.4
53.9
55.3
46.1
37.5
62.3
59.7
58.2
DiffAttack
45.9"
44.0
39.3
37.3
43.5
38
37.4
41.8
38.7
40.7
DiffPure
ACA
60.8*
57.3
50
55.1
56.5
57.3
AdvClass
35.0"
37.8
34.2
35.2
37.2
30.3
34.3
34.9
36.7
35.1
AdvClass
42.4"
42.6
39.8
39.8
41.9
34.3
38.8
40.3
41.3
40.1
N
56.3"
56.6
53.1
54.8
42.3
51.0
53.4
52.1
NatADiff
71.3*
62.2
61.2
61.0
61.3
47.8
58.5
60.2
61.2
60.5
N RUNTIME COMPARISON
We provide a runtime comparison of PGD (Madry al., 2018). AutoAttack (AA) Croce Hein 2020), NCF (Yuan et al. 2022), DiffAttack (Chen et al., 2025), ACA (Chen et al. 2023b), adversarial classifier guidance (AdvClass Class) DDai et al. 2024) and NatADiff.
clear that the generative approach of NatADiff requires substantially greater runtime: however, this cost yields stronger adversarial samples that transfer more effectively across classifiers (see Table TT. and that are more resistant to ad adversarial purification (see Appendix MM). We argue that the trade-off between runtime and state-of-fe of-the----e-rrtt adversarial strength makes NatADiff compelling attack strategy despite its slower speed.
39

---

Published as a conference paper at ICLR 2026
Table 11: Time comparison adversarial attack methods.
Average Runtime per
Attack
Sample (seconds)
PGD
0.3
AutoAttack
0.7
NCF
6.9
DiffAttack
14.2
ACA
96.8
AdvClass
13.5
NatADiff
103.1
O
USER STUDY
NatADiff's is generative attack method designed to produce adversarial samples that lie near the decision boundary between the true and adversarial classes. However, as discussed in Appendix G.2 the generated image may occasionally fully manifest the adversarial class, resulting in the loss of the intended "true" class. To mitigate this, we conservatively set the adversarial boundary guidance strength μ 0.2. We conduct human study to verify that this choice preserves the human- perceived class.
Accurate classification of ImageNet classes typically requires trained human annotators (Russakovsky
ct al. 2015 Shankar et al. 2020). Previous work has found that untrained annotators often exhibit significant class unawareness, particularly for fine-grate tine-grineed categories such dog breeds (Russakovsky et al. 2015 Shankar et al. 2020). To address these limitations, we restrict our study to a curated subset of 148 easily recognisable ImageNet classe
We present 22 human participants with 60 randomly selected NatADiff samples generated using ResNet-50 surrogate classifier under both targeted and untargeted attack settings. Each participant is asked whether the image contains the true class, the adversarial class, or neither. Participants reported seeing the true class 91% of cases, the adversarial class in 7%, and neither class in 2%. NatADiff's label-flip rate of 9% is comparable to the 10% flip rate reported by Dai al. (2024) for adversarial classifier guidance on the MNIST dataset (Deng 2012). These results indicate that NatADiff reliably produces images that appear belong to the true class for human observers, while fooling classifiers with high probability. Moreover, in practical attack settings, an adversary could manually inspect generated images prior to deployment, further mitigating the impact of occasional class-flipped samples.
"Selected classes: 9, 22, 47, 105, 130, 151, 152, 153, 154, 155, 156, 158, 159, 160, 161, 162, 163, 285,
286, 287, 288, 289, 290, 291, 292, 407, 425, 440, 448, 449, 462, 465, 468, 469, 470,470,44 471, 472, 486 493, 495.
496, 497, 502, 503, 505, 510, 514, 515. 516, 517, 520, 525, 526. 527, 532, 537, 555, 580, 607, 609, 663, 670.
671, 672, 701, 719, 734, 780, 787, 820, 909, 937, 963, 975, 999, 972, 973, 569, 933, 452, 958, 985, 900, 991,
400, 635, 920, 103, 482, 392, 843, 833, 259, 614, 757, 490, 715, 834, 222, 509, 374, 815, 927, 545, 531, 21, 303,
282, 132, 536, 913, 741, 624, 269, 547, 313, 640, 182, 849, 954, 524, 109, 657, 889, 884, 961, 94, 797, 817, 653,
134, 978, 803, 668, 436, 581, 519, 554, 942, 806, 410, 144, 693, 523, 26, 403, and 846.
40

---

Published as a conference paper at ICLR 2026
P USEFUL DENOISING DIFFUSION RESULTS
This section outlines results necessary for working with denoising diffusion models. Citations of original authors are provided where applicable.
P.1 CONDITIONAL FORWARD DISTRIBUTION
The following theorem describes the conditional forward distribution of the denoising diffusion model when conditioned on an arbitrary time т.
Theorem P.I (Conditional Forward Distribution for Denosing Diffusion). Let x, E =R" f()) : R + I. and g(t) + I be continuous functions t, and dBB, denote an Itô integral with respect to the standard multi dimensional Brownian motion process. Then the denoising diffusion model with forward process.
dx, = f())x g(t) dB.,
(13)
admity conditional forward distribution of
X,|X, N (a(T,t)ar, B(T, ((T,t)2((mxmm) VEDT, T,
where a(T,t) = exp (1. f(u)du). B(T,t)² = a(T,t)² f alrayy du, and [(mxm) is the m- dimensional identity matrix. Additionally, is understood that with slight abuse of notation (tt) = a0,, f) and B(t) = t)
alw)
Proof. The diffusion in 13 has Stratonovich representation (see (Pavliotis) 2014a) for treatment of Itô and Stratonovich SDE formulations).
dx; = ft) + g(t) o dB,.
where dB, denotes Stratonovich integral with respect to the standard multi-dimensional Brownian motion process. Thus, the SDE can be solved in the usual manner:
dx; f(t)x;dt + g(t) dB,
dx,
dB.
f()), + g(t dt
dt
-
(e-
ce- S f(u)du
dæ,
(e 1 s(w)au)
1(1)xx
dt
+ (e 1 Ma)du
dB,
g(t) o dx,
Vt>T T
dt
e 1. s(u)du))
g(t)
dB.
f {(udda)
(e- S. (i)du) f(t)x,
dl
dl
g(u) [" (e- f(ujduu) g(() o dBB
S s(u)du)
dB.
du
(e- S. S(u)du)
dI
- (e- Mu)ddu f(u)x,du
(e-
du
dv
n=r Tv (e- 5: s(u)du))
/=1 )}
1 (c- flu)du g(0) o dB. xt ( S Hu)du)
[" g(u) a(T,V)
Zt
o dBB
alT,t)
a(T, t)æ+ + a(T,t) 1) f g(v)
dB.
(14)
a(t,o)
By rewriting (14) in its Itō representation we have
x; a(T,t)x, a(T,t) [
g(v) d.B. a(T,v)
41

---

Published as a conference paper at ICLR 2026
and as S a(1) a(T,u)
dB ~ N (o, for alr,a)z du (mxx)) ) it follows that
g(o?
2:10x N (alt: a(r, L: a(T, u)2 du rlmxm))
g(u)²
P.2 CONDITIONAL FORWARD ALTERNATE PARAMETERISATION
When implementing time-travel sampling (Lugmayr et al. 2022) we require access to the condi- tional forward distribution, p(æ;|22; (2,) However, is frequently the case that diffusion schemes formulated with respect to the full forward distribution, p p( [o)), and some proposed method of sampling the reverse-time -emese diffusion (Anderson 1982), or solving the flow ODE (Song al. 2021 Thus, we provide simple result to derive the conditional forward distribution, p x+), from the parameterisation of the full forward, pl ae (n)).
Lemma P.2 (Conditional Forward Alternate Parameterisation). Given the diffusion formulation in
Theorem P.7 then the conditional forward distribution can alternately be expressed as
X, N (ax,, 6² (2)(mm)) 44A 4IST,
where
Xt|Xo ~ N (a(0, 3(0, 1)2 ((mm))) 24 A4> O,
a(U,7)) = 3(0,))0- (a3(0, and mmxx)) is the m-dimensional identity matrix.
((0.))
Additionally, is understood that with slight abuse of notation ait) a(0, t) and B(t) a00, t)
Proof. We need to show that a a(T.() exp ( ((u)du)) and 6² = B(T, 1)² g(w)² a(T,t)² t)2 f a(r,a)z du as per Theorem P.1 It follows that
a(0,t) t)
a(0, r)
exp (IO (u)ddu)
exp
(Jo f(w)du)
exp S/- [u)ddu - S f())du)
exp J" f(u)du)
a(r,t),
(15)
42

---

Published as a conference paper at ICLR 2026
and
8(0,1)
(a)(0,
B(0,1)² d T))
a(0, [' a u0,
g(u)
a(0,t)² t)²
g(u)²
(0,,))000)) (0,
a(0,u)2 du
g(u)²
g(u)2
=000
a(0, u)
a)0,u)2
g(u)
=(0,
(0, u) du
g(u)²
=(0,t)
du a(0, u) a(0,r)a(T, u) by 15
((,,))a(TT,) u)2
a)0,t)²
g(u)?
u)² du
a(0,7)²
a(T, a(t,u)²
g(u)²
a(?,t)²
a(T, u)² u)
t)²,
P.3 SCORE-MODEL LINK
The following Score-Mode Model Link theorem is based on Karras et al. (Karras et al. 2022) argument. However, we provide a minor extension by conditioning on measurable sets taken from the sigma- algebra of auxiliary random variable, Y. For additional treatments see (Karras et al. 2022
Hyvärinen 2005 Vincent 2011).
Theorem P.3 (Score-Model Link). Let a, E Th f(I): R and g(l) R - R continuous functions of I, and d,B,, denote Itô integral with respect to the standard mulii-dimensional Brownian motion process. Suppose that x; evolves according to the diffusion,
dæ: - f()ax + g((() .d..,
(16)
with observed initial data distribution, Pdana paoly E), where & is taken to be arbitrary element of the sigma-algebra, Y, associated with the random variable
P Y, ie., & E y.
Define
wo(w:,, It ((t))
(17)
a(l)
where all) = exp (Io fw)ddu). 3(1)= a(1) So a(wl a(w du, and eg : 1T x I2 x + R is a model
parameterised by f E with sufficient capacity such that the Universal Approximation Theorem
Cybenko 1989 Hornik 1991) holds for all & E y. Then if B(1)² > 0 v t E (0,T), the following
statements are true:
1 3(t) Eg. (x),, Vt (0, a E ER", & D)
Ve. (og(p(xclvEE))
(18)
2.
B Be Pa (vEE)
1 (a, ((()))))((a 3(0J
Eg- (x,, t,
Vt (0,
(19)
p( E ()
Note that this is a slight abuse of notation. We are assuming that (Sy ,,, P) is probability space and Y: fl + fy random variable such that Y(w) V Py. That is to say, we do not need to take the pre-image when crafting prohahility statements.
43

---

Published as a conference paper at ICLR 2026
3t R" E 3; where
0" A arg min Exo. 2e [1:00 Co 4, 1,
(20)
p(ao, 2z, 4y E {) pp Panta (2olly E E)p((), and p(() - 7.
(21)
Proof. Let (x0)' (2) ..... ....,o%!!!
o and y(() y(() denote observed values of zp and Y.
y0()
Then the data density function given by:
E
8(00 To (1)
Pdata (Foly E E)
(22)
Σ (w(EE))
where 5(-) and (c) denote the Dirac delta and indicator functions, respectivvly. The forward diffusion process in 16 is independent of Y. and thus,
plally <) / p(a ( Pama E E)dro
ΣN 1= p(ac, laro (2)
(23)
Σ/" (U(VEE)
Substituting (3 into the LHS of (18).
Σn p(zz|z0" (1) (
Va, m, log(p(ærly E E)) Va, log
Σ (1(E
Vay log
Σ p(azzlarg (100EE))
-=1 Vn, p ( (*) (3(0)E)
iml p(x (20 (O(WEE)
By Theorem P.I p( N (a(t)xa, [(t)2](mxm))
B(e) (x:
(1))r)))p( Fol. thus,
1 N att) (8)
)p(z)) (")))()))))))))))))))))))))) (2)
V n, log(p(ze/v - -
(24)
ΣN p( (1) (1sesss
44

---

Published as a conference paper at ICLR 2026
Now consider the optimisation problem in (20
[(E: 0)
Exno me am yEE)
20(2 t, 6)16
Exmp(t)
B Bmomp(molVEE))
Em (mm
|1:00 2z, t, E)1
// amz Reoorpeeulee (VEE)
= Rewple)
1::0 - wol 1, 1l2 plan, (xo) daze
= -Empp())
12000 Lo( t, E)12 p( p(auly E E)dxo
Ex= 6(20 x (wEEEe) dxo
( 1200 Lu( t, 2)1 ((
= Exrpp()
ΣN (OWEEs
0 ( ,olas, e, 1.5 plarelag ( dara lotanss
Epwp(()
Σ i== 1 (glNEE)
i=1
Den
Σ plars/at lan (3)) [(ucisE)) | a 1, 5) 11 da
((pp(()
E ((CHEE) Dmm
l'" Das im1
N
Σ place 1200 (8) (((COE))
|0 - 4, E) daade
T Di [(AWEE)
1' Bes 1=1
Tt - 3(t))a(2),, x
p( (((C)))))
dæ,dt
T Ee [OEOEE)
a(l)
((mss,1E:00)
(25)
To minimise 25 with respect to 0, it suffices to find 0 such that ((x,, E; is minimised for each combination of a, and t. That is to say, find the optimal value of (0 i, for each combination of ot and t. Furthermore, (ax, t, E; 0) constitutes a convex optimisation problem with respect to E,( t).. Thus.
0e Deg
0
Σ
N
2,((t)
(1) (uuness) p(z,l=?"))
[ x
2t - (I)) t2 (i)
a(t)
a(t)
iml
N
p x 2 (yiees) (a(t)=)) xz + ((t))c((xe t, >
N
N
(2 ())) )p(az 2 (0)
(E) = Σ ((t), 1, ()p(x,) (=)) (WWEE)
1=1
1 En (x; - aa()) |p(0r/100 (1
t,(x,,,tE))
(26)
B(t)
N p( (1) (EE
> (at)) (1) )((p (2) 1909
B(1)
N mla ( (y0en))
ENn (299
Eegg Pars (eolvE))
B(0) (xt. (o(t))ro)p((aal
(27)
((#))yEE)
A x. E R", E (0,T), E V. As f(t) and g(t) both continuous functions then a(t) and 3(t) are also continuous. follows that (26) is continuous with respect to æ, and t, as it is the
sum of continuous functions and B(1) 0 v E (0, Thus, the Universal Approximation
45

---

Published as a conference paper at ICLR 2026
Theorem
Hornik 1991) holds and there exists 8 O such that Eg. t.C) ( zy 2h E (0,7), E y. Finally, by comparing (24 and 26 we observe that,
Cybenko 1989
log(p(x..l E E)) B(t) co. (x),,,))
which proves (88) and
19 follows from
It is worth noting that Theorem P. unifies the score-model link between conditional and unconditional models. That is say,
Va, log(p(x;)) V s, log(p( sty))
1
B(t) Eg (xe, 1, S2y).
46

---

Published as a conference paper at ICLR 2026
47


Reviews:
[{"review_id": "ibSCOmeP7w", "reviewer": "Reviewer_BqxE", "summary": "This paper proposes NatADiff, a novel diffusion-based sampling scheme designed to generate natural adversarial samples. The method is motivated by the observation that deep learning models often rely on “contextual cues” to perform “shortcut learning”, which is hypothesized to be the cause of natural test-time errors. The core methodology is termed “Adversarial Boundary Guidance”, a technique that guides the diffusion trajectory toward the intersection of the true and adversarial class manifolds, notably implemented by manipulating text prompts (e.g., “True Class and Adversarial Class”). To enhance attack transferability, this method is combined with “augmented classifier guidance”, and “time-travel sampling” is used to preserve image quality. Experimental results demonstrate that while NatADiff achieves comparable white-box ASR to SOTA methods, it exhibits vastly superior transferability. Furthermore, the generated samples prove effective even against adversarially trained models, suggesting the attack targets a more fundamental vulnerability than traditional perturbation-based attacks.", "questions": "Concerns are in the weakness section", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(S1) Vastly Superior Transferability\nThe paper's primary strength lies in its demonstration of vastly superior transferability. As shown in Table 1, the proposed NatADiff significantly outperforms all competitors, including SOTA diffusion-based attacks like AdvClass and DiffAttack, in average transfer ASR. The ability to successfully attack a ViT-H model with samples generated from a CNN (RN-50) at such a high success rate strongly suggests the method identifies fundamental, architecture-agnostic vulnerabilities.\n(S2) Novel and Well-Motivated Methodology\nThis strong empirical result is supported by a novel and well-motivated methodology. The core idea of “Adversarial Boundary Guidance”, which leverages the diffusion model’s text-conditioning to guide the sampling trajectory toward the intersection of the true and adversarial classes (e.g., using an “A and B” prompt), is a clever way to operationalize the “shortcut learning” hypothesis. This is logically combined with “augmented classifier guidance” to suppress non-transferable, pixel-level perturbations and force the model to manifest more robust structural features.\n(S3) Effectiveness Against Adversarial Defenses\nThe effectiveness of NatADiff against adversarially trained models (AdvRes and AdvInc) is a significant finding. These models are specifically designed to resist traditional perturbation-based attacks, and NatADiff’s success against them underscores that it operates via a fundamentally different and more robust attack vector.\n(S4) Insightful Analysis of Attack Variants\nFinally, the paper provides an insightful analysis of the trade-off between the random-targeted (T) and similarity-untargeted (U) variants. The observation that the (T) variant achieves a better FID-A score (alignment with natural errors) at the cost of ASR, while the (U) variant does the opposite, lends support to the hypothesis that natural errors often arise from blending features of disparate classes.", "weaknesses": "(W1) Inconsistent motivation for Similarity Targeting: The authors motivate the use of similarity targeting (U) by stating that it “outperform[s] targeted attacks (T)”. While this holds true for CNN surrogates (RN-50, Inc-v3), the paper fails to acknowledge or analyze the contradictory result from the ViT-H surrogate, where the random targeted attack (T) significantly outperforms the similarity-based untargeted attack (U) in average ASR (73.2% vs 69.7%). This omission weakens the claim that similarity targeting is a universally superior strategy, especially for Transformer-based models.\n(W2) Lack of Direct Evidence for the “Class Intersection” Claim: The paper’s core hypothesis is that NatADiff guides the trajectory towards the “intersection of the true and adversarial classes” 6. However, the evidence provided is indirect (FID-A scores) or qualitative (image samples). The paper would be significantly strengthened by providing direct, quantitative evidence. For example, a t-SNE/UMAP visualization plotting the embeddings of the generated samples against the manifolds of the true and adversarial classes would be necessary to truly validate that the generated samples lie in this hypothesized “intersection”.\n(W3) Limited Scope of Evaluation: The evaluation is confined to the broad, 1000-class ImageNet dataset. It remains unclear how the “contextual cue” hypothesis holds on fine-grained datasets (e.g., Oxford-Pet) where inter-class similarity is high and shortcuts might be far more subtle. While the authors acknowledge this limitation in the appendix, an experiment on a fine-grained dataset would have substantially strengthened the paper's claims of generalizability.\n(W4) In Equation 10, it seems you should use argmax instead of argmin. In this equation, you should specify the class with the highest similarity, not the lowest, as the adversarial target.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WiINPJjouw", "reviewer": "Reviewer_rbeT", "summary": "This paper proposes NatADiff, a diffusion-based method for generating highly transferable natural adversarial samples. It introduces adversarial boundary guidance and classifier augmentations to steer diffusion sampling toward class intersections, mimicking naturally occurring test-time errors. NatADiff achieves strong white-box attack success rates and significantly higher transferability than existing methods, with samples more closely resembling natural adversarial examples.", "questions": "- Why do you only limit to ImageNet?\n- Alg 1: What does the star after $\\epsilon$ mean?\n- Will you provide your code?\n- You use the RTX 4090. You havn't specified the memory size. Maybe only 24 GB. Can you only attack one image after another or more images at once?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel integration of adversarial boundary guidance and classifier augmentations in diffusion models.\n- Strong empirical results: high transferability and competitive white-box performance.\n- Well-motivated by the link between contextual cues and natural adversarial samples.\n- Comprehensive evaluation across multiple architectures and adversarial defenses.", "weaknesses": "- Computationally expensive due to iterative diffusion sampling.\n- Limited to ImageNet; evaluation on more specialized domains is future work.\n- Similarity targeting may lead to subtle misclassifications (e.g., between similar classes).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WGCKFE5f0d", "reviewer": "Reviewer_6oxz", "summary": "This study introduces NatADiff, an adversarial sampling framework that leverages a denoising diffusion model to synthesize \"natural adversarial examples\" by guiding sampled images toward the intersection of the manifold between true and adversarial categories. The authors propose an adversarial boundary guidance method to generate adversarial examples that are more realistic and transferable than baseline attack strategies. Empirical results on ImageNet classifiers and a range of model architectures demonstrate that this approach improves attack transferability and approaches true test-time error.", "questions": "1. Could the authors provide more technical details or ablation results on the construction and effect of the intersection prompt `y ∩ ỹ`? Specifically, how stable are the results under different prompt generation strategies?\n2. Could the authors provide rigorous analysis or experimental results on how the adversarial guidance strength s influences gradient vanishing or explosion phenomena, particularly given its threshold-like behavior and dependence on classifier architectures?\n3. The paper has a great idea. What needs to be done to adapt NatADiff to domains beyond natural images (e.g., audio, tables, or multimodal data)? Are there any inherent limitations?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper connects the concept of adversarial bound guidance with the generation of natural adversarial examples, formalizing the intuition that natural errors occur due to overreliance on contextual cues. This is a well-thought-out approach.\n2. The method demonstrates excellent technical depth and implementation. It combines time-travel sampling, classifier-free guidance, gradient normalization, and boosting; ablation experiments are conducted for each component to evaluate its contribution.\n3. Experimental results demonstrate that NatADiff achieves comparable or even higher attack success rates than state-of-the-art methods in both white-box and transfer settings. The high transferability of its attack is demonstrated across a variety of victim architectures, including convolutional and Transformer-based models.", "weaknesses": "1. The manuscript provides no ablation study or discussion on the robustness of the adversarial boundary guidance to variations in the textual implementation of the intersection prompt `y ∩ ỹ`. The stability of results across different prompt engineering strategies (e.g., varying templates or phrasing) remains entirely unexplored.\n2. The appendix mentions that the adversarial guidance strength was \"manually tuned\" and notes that s behaves close to binarization (the attack succeeds only after reaching a threshold). The paper mentions \"the optimal value of s varied across classifiers\" and reports the specific values in Table 4 but does not verify how s affects gradient stability or vanishing/divergence.\n3. All experiments were conducted only on ImageNet and its commonly used architectures. Whether this method can be extended to non-image data modalities, domain-transfer scenarios, or truly open-world environments has not been discussed or empirically verified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZdsOf98XMb", "reviewer": "Reviewer_n2At", "summary": "This paper presents a novel and well-motivated method for generating natural adversarial samples using diffusion models. The proposed NatADiff framework effectively combines adversarial boundary guidance, classifier augmentations, and time-travel sampling to produce highly transferable and semantically meaningful adversarial examples. \n\nThis paper introduces NatADiff, a diffusion-based method for generating natural adversarial samples. The key contributions are: 1）It proposes a novel technique that guides the diffusion sampling trajectory toward the intersection of the true and adversarial classes, encouraging the inclusion of adversarial features in a semantically plausible manner; 2) It uses differentiable image transformations to reduce the influence of constrained adversarial perturbations and promote more robust and transferable adversarial features; 3) \n It presents an untargeted attack strategy that selects adversarial classes based on semantic similarity using CLIP embeddings, improving attack success and visual quality.", "questions": "1. Why does adversarial boundary guidance improve transferability? Is there a theoretical link between class intersection and model shortcut learning?\n\n2. How do humans perceive NatADiff-generated samples? Do they align with human judgment of class membership?\n\n3. Can NatADiff be adapted to a \"guided editing\" setting, where it modifies a provided source image towards an adversarial target, rather than generating from scratch?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The adversarial boundary guidance mechanism is a novel and intuitive way to incorporate adversarial features without deviating entirely from the true class, leading to more plausible and transferable samples.\n\n2. NatADiff achieves significantly higher transferability than existing methods across a wide range of models, including adversarially trained classifiers.\n\n3. The paper includes thorough ablation studies and visualizations that validate design choices and provide interpretability.", "weaknesses": "1. The methodological novelty is limited; the core components of NatADiff lack fundamental innovation. Classifier augmentations and time-travel sampling are directly borrowed from prior works to address known issues (e.g., constrained perturbations and sample quality degradation). The main proposed contribution, adversarial boundary guidance, is essentially a weighted interpolation between two established concepts: classifier-free guidance and the novel but straightforward idea of guiding towards a class intersection ($y ∩ ȳ$) using a composite text prompt. While the combination of these elements is new and effective, the paper does not introduce a groundbreaking new algorithm or theoretical insight. It is better characterized as a skillful and well-engineered integration of existing techniques towards a new objective.\n\n\n2. NatADiff is computationally expensive (103 seconds per sample), primarily due to the iterative diffusion process and time-travel sampling.\n\n3. Experiments are confined to ImageNet. It is unclear how well NatADiff generalizes to other domains. (eg. CUB-200, Stanford Cars in DiffAttack ).\n\n4.  NatADiff often introduces significant semantic changes to the original content, leading to the generation of implausible or unnatural features (e.g., objects with distorted shapes or unrealistic textures). While this may effectively fool classifiers, it limits the applicability of the attack in real-world scenarios where the adversarial sample must remain a faithful representation of the source semantic content. For instance, in content authentication, medical imaging, or any context where the integrity of specific visual features is paramount, such drastic alterations are not permissible and would be easily flagged by a human observer.\n\n5. The experimental comparison is skewed in favor of NatADiff. Methods like DiffAttack and ACA are reference-based attacks; they are constrained to be similar to a given clean source image. In contrast, NatADiff and AdvClass are generative-based attacks that start from random noise and have the freedom to generate any image on the manifold. This fundamental difference grants NatADiff a significantly larger attack surface, as it is not bound by the semantic content of a specific source image. Comparing these two distinct paradigms under the same \"unconstrained attack\" umbrella is misleading and overstates NatADiff's advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)