DiffAdvMAP: Flexible Diffusion-Based Framework for Generating Natural Unrestricted Adversarial Examples
Authors: Zhengzhao Pan
Hua Chen
Xiaogang Zhang
Venue: ICML 2025 poster (2025)
Keywords: Unrestricted Adversarial Attacks
Diffusion Models
Flexible Unrestricted Adversarial Attacks
Primary area: deep_learning->robustness
TL;DR: A flexible diffusion-based framework for generating natural and effective unrestricted adversarial examples.

Abstract: Unrestricted adversarial examples(UAEs) have posed greater threats to deep neural networks(DNNs)  than perturbation-based adversarial examples(AEs) because they can make extensive changes to images without being restricted in a fixed norm perturbation budget. Although current diffusion-based methods can generate more natural UAEs than other unrestricted attack methods, the overall effectiveness of such methods is restricted since they are designed for specific attack conditions. Additionally, the naturalness of UAEs still has room for improvement, as these methods primarily focus on leveraging diffusion models as strong priors to enhance the generation process. This paper proposes a flexible framework named Diffusion-based Adversarial Maximum a Posterior(DiffAdvMAP) to generate more natural UAEs for various scenarios. DiffAdvMAP approaches the generation of UAEs by sampling images from posterior distributions, which is achieved by approximating the posterior distribution of UAEs using the prior distribution of real data learned by the diffusion model. This process enhances the naturalness of the UAEs. By incorporating an adversarial constraint to ensure the effectiveness of the attack, DiffAdvMAP exhibits excellent attack ability and defense robustness. A reconstruction constraint is designed to enhance its flexibility, which allows DiffAdvMAP to be tailored to various attack scenarios. Experimental results on Imagenet show that we achieve a better trade-off between image quality, flexibility, and transferability than baseline unrestricted adversarial attack methods.

DiffAdvMAP: Flexible Diffusion-Based Framework for Generating Natural Unrestricted Adversarial Examples
Zhengzhao Pan
Hua Chen
Xiaogang Zhang'
Abstract
1. Introduction
Unrestricted adversarial examples(UAEs)) have posed greater threats to deep neural net- works(DNNs) than perturbation-bant examples(AEs) because they can make extensive changes to images without being restricted in a fixed norm perturbation budget. Although cur- rent diffusion-based methods can generate more natural UAEs than other unrestricted attack meth- ods, the overall effectiveness of such methods restricted since they are designed for specific at- tack conditions. Additionally, the naturalness of UABs still has room for improvement, these methods primarily focus on leveraging diffusion models as strong priors to enhance the gener- ation process. This paper proposes a flexible framework named Diffusion based Adversarial Maximum a Posterior(DiffAdyMMAP to generate more natural UAES for various scenarios. Dif- MAPvMAP approaches the generation of UAES by sampling images from posterior distributions, which is achieved by approximating the posterior distribution of UAEs using the prior distribution of real data learned by the diffusion model. This process enhances the naturalness of the UAES. By incorporating an adversarial constraint to en- sure the effectiveness of the attack, DiffAdwMAP exhibits excellent attack ability and defense bustness, A reconstruction constraint is designed to enhance its flexibility, which allows DiffAd- vMAP to be tailored to various attack scenarios. Experimental results on ImageNet show that we achieve a better trade-off between image quality, flexibility, and transferability than baseline unre- stricted adversarial attack methods.
Deep Neural Networks (DNNs) have been prosperous in var- ious vision tasks these years, such as object detection, face recognition, and semantic segmentation. However, many works have shown that DNNs are vulnerable to adversarial examples. Adversarial examples are images intentionally crafted by adding tiny perturbations to natural images. Such modified images can deceive DNNs to make wrong predic- tions while remaining imperceptible to humans, bringing security risks to decision-critical systems. This vulnerabil- ity poses great threats to lots of vision tasks such image classification (Goodfellow et al., 2014), (Carlini & Wagner, 2017), (Madry et al., 2017), segmentation (Li et al., 2023), and tracking (Li et al., 2021b), (Li et al., 2023).
adversarial
Unlike traditional perturbation- based adversarial examples (AEs), which limit perturbations a small range maintain imperceptibilityy unrestricted adversarial examples (UAEs) are generated by applying extensive natural transformations to images, such as color conversion, which significantly reduces the noticeable noise patterns in AEs, UAEs can also generated by training a generative model like AC-GAN (Song et al., 2018), enabling the attacker to produce more natural and unlimited number of UAEs. Such approaches do not require perturbing real images with restricted per- turbations, thus being more concealed and effective than traditional AEs. As result, UAEs have emerged signif- icant area of study in adversarial examples over the past few years due to their potential threat to deep neural networks. Though generative models like GANs and VAEs can learn and sample from data distribution effectively, it's difficult to perform well complex and high-quality datasets like ImageNet (Deng et al., 2009) because of their weak inter- pretability. Diffusion models (Ho et al., 2020) have shown their superiority in synthesizing realistic and high-quality images in recent years, thus, they become powerful com- petitors to GANs and VAEs in generating UAEs. Inspired by this, recent works (Dai et al., 2025), (Chen et al., 2023), (Liu et al., 2023a) explore new methods to generate realistic UAEs complex datasets with diffusion models and obtain better performance than previous works.
"College of Electrical and Information Engineering, Hunan University, Changsha. China CCollege of Computer Science and Electronic Engineering, Hunan University, Changsha, China. Cor- respondence to: Hua Chen <chua @ hnw.edu.cn. Xiaogang Zhang <zhangxg @ whwu.cdu.en..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
However, there still exist some problems that affect the effectiveness and naturalness of UAEs generated by diffu-

---

DiffAdyMAP: Flexible Diffusion-Barne Based Framework for Generating Natural Unrestricted Adversarial Examples
sion models to be considered: 1) As is shown in (Meng et al., 2021), diffusion models tend to add low-level seman- tic information such the layout in the early generation steps while more high-level semantic information in the later steps, modifying the latent code of the early steps in the generation process(Chen et al., 2023) may change the low-level features and take the risk of generating unnatural UAEs. Although some meth et al., (Chen et al., 2024a) generate UAEs by generating adversarial high-level features, they primarily focus on using diffusion models as strong priors to enhance the generation process. This approach does not fully leverage the prior knowledge of real data distributions learned by diffusion models, which may still take the risk of generating unrealistic features. 2) Most existing methods are limited a fixed set of scenarios, as they are designed for specific attacking conditions, such as generating UAEs similar to given reference images or producing UAFs from noise. This narrow focus restricts the overall effectiveness of the adversarial examples.
Our main contributions are summarized as follows:
We propose a flexible diffusion-based framework for generating UAEs named DiffAdvMAP, it can generate UAES under various attacking conditions. We achieve it by approximating the posterior distribution of UAEs using pre-trained diffusion models and sampling from the distribution. This approach leads to better natu- ralness than most diffusion- -ased attack methods.
We design an adversarial constraint and a reconstruc- tion constraint within the Bayesian framework to gen- erate UAEs. The adversarial constraint ensures the effectiveness of UAEs; the reconstruction constraint grants our framework the flexibility to handle various attack conditions.
Experimental results regarding white-box attack suc- cess rate, transferability, and defense robustness demonstrate the effectiivness of DiffAdwMAP. Ad- ditionally. UAEs generated under various attack con- ditions further emphasize its superiority in flexibility and effectiveness over baseline attacks.
To this end, we propose a flexible diffusion- based unre- stricted adversarial attack framework to generate natural UAEs. In our opinion, the posterior distribution of UAES derived from the prior distribution of natural data learned by the diffusion model is more close to natural data distri- bution, we can generate more natural UAEs by sampling from this distribution. We leverage the generation process diffusion model, extending and adjusting
2. Related Works
of a the maximum a posterior(MAA)) method to form our Dif- fAdvMAP framework to generate natural UAEs. Under the Bayesian framework, we first derive the posterior distribu- tion of UAEs based on the real data distribution learned by the diffusion model under the adversarial and reconstruction constraints, the adversarial constraint is used to ensure the effectiveness of the attack, and the reconstruction constraint is used to control the content of generated UAEs. Then we go through the generation process of the diffusion model and sample UAEs from such distribution. Since our framework samples UAEs from the approximated posterior distribution of UAEs, there's no need to go through the whole generation process to remove too many conspicuous adversarial noises. We integrate a destruction and construction method into our framework, which destroys most high-level features of real images by the diffusion process, and regenerates adversarial features via DifIAdvMAP. As a result, our framework can generate UAES with truncated generation process while protecting most lovel features, thus improving the natu- ralness and generation speed of UAES. Finally, when facing different attacking tasks such as generating UABS similar to the given images, generating UAEs from noise, generating UAES via regenerating specified regions of given images, and generating UAES via changing the color or style of given images, the reconstruction constraint in DiffAdy can be customized to such tasks while keeping the naturalness.
2.1. Adversarial Examples
Perturbation-based adversarial attacks are performed by adding small and imperceptible perturbations to natural images such that the target model makes wrong predic- tions. Since (Szegedy et al., 2013) shows the existence of adversarial examples, the security concerns of such attacks are increasing in computer vision and machine learning communities more and more advanced and powerful methods are developed (Moosavi-Dezfonl et al., 201 (Loog et al., 2022). On the other hand, adversarial attacks play important roles in improving contrastive learn- ing(Lee et al., 20 & Nvasconcelos, 2020), image recognition(Xie et al., 2020), privacy protection(Li et al., 2021 a et al., 2023a), and other applications. Attack- ers can easily generate perturbation-bast based adversarial sam- ples by using gradient-based methods such as fast gradi- ent sign method((FGSMM)Goodddlll (Ctack(Carlini & Wagner, 2017), projected gradient de- (MDdayy et al., 2017).
et al., 2014), CW
While most of the perturbation based adversarial attacks that focus on optimizing additive perturbations at the pixel level have achieved good results, it is shown that the restric- tions of perturbations are not accurate in representing the way that humans perceive the differences between similar images (Jia et al., 2022), (Yuan et al., 2022), and thus in- troducing conspicuously noise patterns, such as the global noise introduced by the PGD attack. As a result, researchers
2

---

DiffAdyMAP: Flexible Uiffusion-Basee Framework for Generating Natural Unrestricted Adversarial Examples
Diffusion Process
DiffAdMAA
DifAAVMAP
Generation Process
Infennce PDF
Marimiza
Posterior
PDF
Ri-1 r Po 1R
Figure 1. An overview of the algooi for generating unrestricted adversarial examples
turn to generating AEs using generative models, achieving higher realism than perturbation-based AEs while keeping a high success rate. (Wong & Kolter, 2020) trains a con- ditional VAE to generate a variety of perturbations. (Xiao al., 2018) trains conditional GAN to produce adversarial examples directly. (Song al., 2018) trains an AC-GAN and samples adversarial examples from noise. (Qiu et al., 2020) generates imperceptible AEs by modifying the at- tributes of the natural images using GAN. (Bhattad et al., 2019) perturbs images from the perspective of color and texture by leveraging corresponding pre-trate pre-trained GANs.
sarial samples only, and have not explored approximating the posterior distribution of UAEs yet.
2.2. Diffusion Models
Since (Ho et al., 2020) proposes denoising diffusion proba- bilistic (Ddels and show their superiority in syn- thesizing high-quality and high-diversity images, the appli- cation range of diffusion models is becoming increasingly broad, such as image synthesis(Rombann et al., 20a haria al., 2023)(Zhang et al., 2023), time series predic- al., 2021), video syn-
tion (Tashrro et al., 2021)(Ras) thesis(Harvey et al., 20o et al., 2022), point cloud et al., 2021), adversarial
Diffusion models(Ho al., 2020) are more powerful and stable than GANs and VAËs, and many works have suc- ceeded in generating more realistic UAEs on complex and high-quality datasets. (Chen et al., 2023) is the first to in- vestigate generating UAEs with diffusion models, it adds small adversarial perturbations to each latent code of the generation process and removes unnecessary noise via diffu- sion models to generate natural UAEs, it also leverages the information of original images to preserve semantic impor- tant objects. (Dai et al., 2025) generates realistic UAEs by using the gradient of defending classifiers to guide the latent code during each generation step. (Chen et al., 2024a) gen- erates imperceptible and transferable UAES by optimizing the attention map during the generation process of diffusion models. Furthermore, Diff-PGD (Xue et al., 2023) utilizes diffusion models adapt adversarial examples generated by the PGD (Madry et al., 2017) method align more closely with the real data distribution, resulting in more stealthy adversarial examples. Although it can be applied to vari- ous tasks, it is fundamentally based on the PGD method, a perturbation- attack that depends on global noise pat- terns. Consequently, the naturalness and effectiveness of the adversarial examples remain unsatisfactory, despite the use of diffusion models to alleviate these patterns. To the best of our knowledge, all works far consider using diffusion models as strong priors to enhance the generation of adver-
completion(Lyu et al., perturbations purification(?), etc.
DDPM is defined as a Markov chain comprising forward diffusion steps, Fi-T, which convert an original image x into pure Gaussian noise and series of Gaussian transitions comprising T reverse generation steps. IT:1, which generate high-quality images with pure Gaussian noise input. In each forward diffusion step [1 Tj, Gaussian noise is iteratively added to each latent code x, according to a monotonically increasing noise schedule 31.. Specifically,
((=+(<=-1) = N( V1 11-,,,,,,,330 3.1)
(I)
The reverse generation process, which begins with sampling 2T from Gaussian distribution, generates each latent code -t-1 by removing Gaussian noise from previous latent code 2t and finally generates natural-like data Tig:
(2) In DDPMs, (0( = 1 (2r 1-ar co t)), Ea 1) v Br. Here, Or, = 1- Br, . Cr II aii co(vr, L) is Gaussian noise estimated by model cp-
pe(xx-1lxe) N (e t), (0(
var
VI-ai
(Nichol & Dhariwal, 2021) proposes an improved DDPM that learns the variance schedule to improve quality and effi- ciency. Denoising Diffusion Implicit Models(DDIMss(Song
3

---

DiffAdyMAP: Flexible Diffusion-Ban Based Framework for Generating Natural Unrestricted Adversarial Examples
al., 2020) use non-markovian diffusion process to achieve much faster sampling speed than DDPMs with the same training procedure, whose generation process is:
prior distribution of real data learned by the diffusion model based on the formulas. Afterward, we infer the PDF of the posterior distribution as our objective function. Finally, we follow a greedy optimization procedure to find each adver- sarial latent code 2r: that maximizes the objective function to generate the final UAEs.
Po pa - [xa) = N (Te- 13 V V 120
Ltt v V
(3)
+ 1- 8f
8.
Oz
3.1.1. POSTERIOR DISTRIBUTION DERIVATION
Here, 5, E [0,
1-
is the standard devia-
Given an optional reference real image x, a ground truth label y, diffusion model Gg. and target classifier Fg. our goal is to utilize Ga to generate adversarial examples do that can deviate the decision of F from correct to wrong:
l-a
tion, when 8r = O, the generation process is deterministic,
when 6e = V / I-as
&r.
the generation process is
ae-1"
the same as DDPMs.
(.Atttcck y; if erists)) = Fo( fy
(4)
In addition, efforts are also made improve the qual- ity of conditional image generation. (Dhariwal & Nichol, 2021) leverages a pre-trained noisy classifier to guide class- conditional image synthesis. (Liu et al., 2023b) then extends it to image- and text-based guidance. (Ho & Salimans, 2022) further improves classifier guidance to classifier-free guid- ance, which utilizes an internal latent classifier.
Here, if x exists, 20 must be semantically close to z, and Attack(-) is our attack algorithm. From CW attack(Carlini & Wagner, 2017), we can convert the goal of (0o) # y into the adversarial constraint:
C : Z(do), - mazizy(Z(Bo)) =
(5)
3. Method
Where e S 0 is the confidence level of fooling the classifier, Z(xo); is the logit output of classifier F entry with do as input. For convenience, we denote the logit difference Z(),, (n(u))/y(Z(())) as I(-).
As is shown in Figure 1, DiffAdvMAP is formed with two branches to deal with two main attacking scenarios: whether or not a reference image exists. Suppose a real image is given for reference. In that case, most high-level features of the real image will be destroyed by going through the diffu- sion process for t" steps obtain the latent code x4; if not, noisy image 2т will be sampled from the Standard Gaus- sian distribution, it will go through the original generation process for T - t" steps until the latent code x; is obtained, where T is the total length of the generation process, t" a hyperparameter. Then the UAE is generated similarly for both scenarios with a t ste-seps-o truncated generation process. It approximates the posterior distribution of latent code d given the previous latent code ăt+1 under the adver- sarial and reconstruction constraints, and samples 2 from it iteratively until the final UAE is generated. Going through the truncated generation process, we can generate adversar- ial high-level features by sampling from the approximated posterior distribution to generate more natural UAEs faster.
When generating UAE with a reference image, the differ- ence between the UAE and the reference image is specifi- cally defined in different scenarios, we introduce a recon- struction constraint to control the content of the UAE:
C2 : m o Q(0o) m o R(T
(6)
Here, o means element-wise multiplication and m is the mask used to deal with different kinds of regeneration re- gions. Specifically, when UAEs are generated globally, m is an identity matrix; when UAES are generated in some specified regions, m is the mask that covers such specified regions. Only the regions m covers should be generated when generating regional UAEs. Function ((-) is a cus- tomized function for generating UAEs in different scenarios. For generating image-similar UAEs, 3 is the original refer- ence image, R(x) = x;; for generating style UAEs, 5 is an extra image that contains the target style, 9(-) computes the style score as (Gatys et al., 2016); as for generating color UAEs, ă is the reference image after changing color, 2(-)) converts images from the RGB space into the LAB space.
3.1. Diffusion-Rased Adversarial Maximum a Posterior
We extend and adjust the MAP method to the diffusion- based UAEs generation task. We develop our methods un- der the Bayesian framework, which uses the Bayes formula to derive the posterior distribution of the target data, and samples the target data from this distribution by maximiz- ing the probability density function(PDF) of the posterior distribution. In this section, we first construct the generation problem of UAEs in the form of mathematical formulas, then we derive the posterior distribution of UAEs with the
As result, given the adversarial constraint C₁ and the recon- struction constraint C2, the posterior distribution of UAEs can be represented as po ( ( C2). and since the reverse generation process of DDIMs is a deterministic process that once the input Gaussian noise X is determined, the output 2o is uniquely determined, the generation problem boils down to determine an appropriate X, based on the
4

---

DifTAdyMAP: Flexible Diffusion-Basse Framework for Generating Natural Unrestricted Adversarial Examples
following posterior distribution:
Here, as is shown in equation (3), in DDIMs
pa C1, C2) x (C())))o(((()r)))0(())
2t (a))
ir - Vār--(if + 11
vI at
Then, since the generation process of the diffusion model is a Markov process that can be decomposed as:
(13)
We follow greedy optimization procedure to find each latent code Torr, which samples an 2r by maximizing equa- tion (10), and then samples ēt- given previous latent code St by maximizing equation (12). Note that, each latent code -1-1 in equation (12) is initialized by ft. As for the approx- imation error introduced by the one-step estimation, it will reduce gradually as l reduces and close to zero at the last few steps of the generation process(Zhang et al., 2023), so it has little effect on the quality and effectiveness of UAEs.
T
po(forr(C, C2) po C2) II
po ( C1, C2) t=1
(8)
As a result, derive the posterior distribution of each adversarial latent code as follows:
pe pe( C1,C2) x
(9)
As for the scenario that the reference image is not given, we are supposed to generate the UAE from noise, the gen- erated UAE must contain the object that can be recognized as the predefined label y but be misclassified as another label by the target classifier. So we utilize the conditional diffusion model to generate most low-level features of the target object, and then generate adversarial features with DiffAdvMAP. Please refer to Appendix B for more details.
3.2. Destruction and Construction Method
Since going through the whole generation process of the diffusion model is time-consuming and recent research on real image editing(Mokady et al., 2023)(Couairon et al., 2022)(Kwon & Ye, 2022) show that perturbations can be applied to high-level semantics without compromising im- age realism. And (Meng et al., 202 (CCung et al., 2022) present that generating adversarial features on real images can be regarded special case of real image editing. So we integrate a destruction and construction method with our framework, which allows us to preserve most low-level features of the original images and generate adversarial high- level features while accelerating the generation process.
3.1.2. INFERENCE OF OBJECTIVE FUNCTION
For image-similar UAEs, propose the PDF of the poste- rior distribution of each adversarial latent code è (t [T' 1)) in equation (8), Appendix B shows detailed derivation. The approximation of the log PDF of equation (7) is:
This method is used for obtaining an appropriate intermedi- ate latent code Ia via the diffusion model as follows:
log 19g (iT (ig C2)
1
= |
1|c U(Fo (
[(xel(zo), vn exists (x( ++1 po (a(zx_1(z;) (x:), otherwise
2E77
(10)
It
(14)
1
||
ff
(+T)
2012 25
Here, xp is reference image, q(-) means the diffusion pro- cess, po(-) means the generation process. Then रt in equa- tion (10) is initialized by zt, and DiffAdvMAP is performed with a truncated generation process to generate UAEs. By integrating this method, the generation speed is improved greatly. The pseudo-code is shown in Appendix C.
Where ((i== = 1, is the standard deviation of distribution Po(C,lèr) = 1,2), C" is the normalizing constant, f)()) a one-step estimation used to approximate the final UAE to with intermediate latent code á, to reduce the computing complexity. The one-step estimation is defined as:
io fz(o,) Et
-1 -reooo((
4. Experiments
(11)
Vot
In this section, we evaluate the effectiveness of our frame- work under the black-box settings. This section is organized according to various attack conditions: generating UAEs from noise, global image-similar UAEs generation, regional image-similar UAEs generation, and eustomized UAEs gen- eration. We will evaluate the transferability and robustness against defense methods of our framework in the global image-similar UAES generation and generating UAEs from noise part. We also conduct evaluations under the white-box setting, please refer to Appendix D for more details.
Then the log PDF of the posterior distribution of each adver- sarial latent code from equation (9) can be approximated:
log p/o 1.,, C1, C2)
| - 11
11--1156 ((ft-(((x1))))
207
2
||0 - _/ (ft-1) + C'
26 --1--1
(12)
5

---

DiffAdyMAP: Flexible Diffusion-Basse Framework for Generating Natural Unrestricted Adversarial Examples
4.1. Experimental Settings
Datasets and Metrics. We evaluate the performance of our framework the ImageNet-compatible
AdvDiff
Kataset(Kurakin et al., 2018), consisting of 1,000 images from ImageNet's validation set. In our experiments, we only consider the resolution of 224+ * 224 3. We apply the FID(Heusel et al., 2017) and LPIPS(Zhang et al., 2018) as the image qual- ity metrics for global image-similar UAEs generation, FID, TRESlGtleetanch et al., 2022) and HyperIQA(Su et al., 2020) for generating UAEs from noise. Note that the refer- ence data for computing the FID score is from DiffAtttcc.
DiffAdvMAP
AdvDiff
Models. We adopt the latent diffusion model(?) for gen- erating UAEs from noise, and a pre-trained unconditional DDPM from(Dhariwal & Nichol, 2021) in other attack- ing conditions. We select Inception V3(Inv-v3) (Szegedy et al., 2016), MobileNet V2(Mob- (2)(Sandler al., 2018), Resnet50(Res-50)(HH et al., 2016) and Swin-B(Liu al., 2021) as the surrogate models, and evaluate the trans- ferability of UAEs against each other. In addition, we also take various defense methods into consideration and evaluate the robustness against them: preprocessing meth- ods(DiffPureeNiee et al., 2022), R&P(Xie et al., 2017), and NRP(Naseer al., 2020) and adversarially trained models (Adv-Ine-v3 et al., 2018), Inc-v3ena3- Inc-v3ens4- and IncRes-r2ena (Tramer et al., 2017)).
DiffAdvMAP
Figure 2. UAEs generated from noise using DiffAdyMAP and Ad- DDfff attacking Resnet 50. We can see the eyes of the dog. the windows of the bus, and the body of the butterfly more natural in UAEs generated by DiffAdvMAP.
erate one UAE for each class of the ImageNet dataset, the qualitative results are shown in Figure 2, and we can see that the UAFS generated by our method are more natural. We compare with AdvDiff quantitatively in Table regarding attack success rate, transferabilitt, and image quality. Our framework can generate UAEs with near 100% white-box attack success rate while achieving better naturalness and transferability than the baseline method.
Baseline Attacks. For generating UAEs from noise, we choose Adv et al., 2025) as the baseline method; for Global Image- Similar UAEs Generation, we choose three classical unrestricted attack methoosscAdvvBhatttd al., 2019), ReColorAdv(Laidllw & Feizi, 2019), and NCF(Yuan al., 2022) ), two diffusion-based attack methods PGD(Xue et al., 2023), and DiffAttack et al., 2024a)). We don't 't consider ACA(Chen et al., 2024b) since the official code isn't offered and the method is similar to Attack
4.3. Global Image-Similar UAEs Generation
In this section, conduct experiments regarding transfer- ability, image quality assessment metrics: FID score, LPIPS metric, and defense robustness. Note that there is a trade-off between effectiveness and naturalness: larger perturbations are more likely to be robust against transfer and defense methods but can also diminish naturalness.
Implementation Details. We leverage the DDIM sampling for the generation process. The number of diffusion steps T is respaced to 200,/ L 40, of DiffAdvMAP iterations is set to I = 10 for generating UAEs from noise. For other attacking conditions, T = 100, t t= 20, = 40 and I= 2. We apply an adaptive learning rate with an initial value of Ir = 0.01, E/ = 1,2) in equation (10) is set to 0.1 for all settings. All experiments are done with a single RTX3090 GPU.
-30 and the number
Results on Normally Trained Models. In this part, we evaluate the transferability between four normal DNNs, we select 3 classical unrestricted adversarial attack methods and two crttusioonbbssed attack methods as our bascline. Table 2 shows the quantitative results of the white-box at- tack success rate and transferability. As we can see, our framework achieves near 100% white-box attack success rate, which surpasses most baseline attacks. Meanwhile. in some model transfer experiments, DiffAttack and NCF achieves better transferability than our framework concern- ing the transfer attack success rate. However, our framework surpasses other baselines in all experiments, including the diffusion- based attack method Diff-PGD, which uses the same basic diffusion model as we do. Note that, our frame-
4.2. Generating UAES From Noise
Generating UAEs from noise important for generative model-based adversarial attack methods, attackers can gen- erate an unlimited number of UAEs once such an algorithm is developed. This not only pose a great security chal- lenge to DNNs but also offer enough AEs for adversarial training, thus improving the robustness of DNNs. We gen-
6

---

DiffAdyMAP: Flexible Diffusion-Basse Framework for Generating Natural Unrestricted Adversarial Examples
Table The white-hox attack success rate()), (%), transfer attack success rate (%), image quality metries, as well as the time(sec) of DiftAAdMAAP and AdvDiff' in the task of generating UAEs from noise. Since computing LPIPS score needs reference images, we replace with blind image quality assessment metrics: TRES and HyperIQA.
DEPENDING MODELS
SURROGATE MODELS
FID
TRES(1)
HYPERIQA(()
INC-V3
RES-50
Moa-V2
SWIN-B
ADVDIFF
99.9
10.5
11.3
7.9
43.1
81.4
0.62
14.3
DIFADDMAP(OURS)
99.2
30.0
26.1
21.7
44.3
81.8
0.64
16.4
AnVDIFF
12.8
100.0
9.7
7.1
44.3
81.2
0.62
RES-50
DIFADVMAP(OURS)
29.8
100.0
28.6
18.5
42.8
84.3
0.66
Anvibuy DIFAADYAP(OOUR)
11.2
100.0
MOB-V2
23.7
22.3
99.0
13.8
42,7
83.9
0.65
ADVDEFF
13.6
12.3
SWIN-B
DIFFDUMAP
24.6
25.2
23,9
97.3
43.0
83.2
0.65
work achieves near 50% attack success rate across various transfer models, which highlights the effectiveness of our framework in the transfer-based black-box attack. We also conduct ablation study in terms each module and the adversarial confidence level c, please refer to Appendix E for more details.
Results on Denfense Robustness. We also evaluate the robustness of our framework against three preprocessing de- fense methods and four adversarially trained models. After going through these strategies, we assess the effectiveness by calculating each attack method's white-box attack suc- cess rate. The results are shown Table 3. We can see that DiffAdyMAP keeps top-2 ranking in terms of the robustness against such defense strategies. The satisfactory robustness of Diff AnffAddMPP against such defense methods is due to the design of the adversarial constraint and the approach of sampling from the posterior distribution.
Original Images
Diff-PGD
4.4. Regional Customized UAEs Generation
DiffAttack
Orignaal Images
STOP
10
DiffAdvMAP
Masic
STOP
10
Color UABS
Figure 3. UAEs generated by the three diffusion- based methods, the surrogate model is Inception v3. UAEs generated by Dif- fAdvMAP don't have conspicuous noise patterns and preserve important low-level features. (eg: The face of the statue, the sign on the tail fin of the plane.)
10
Slyfe Imagus
STOP
10
style UAD
Besides, we also evaluate the naturalness of adversarial ex- amples as well as the time cost quantitatively in Table 2. Our framework achieves the best image quality among the attacks, it surpasses other diffusion-based methods includ- ing DiffAttack, which uses the stable diffusion model - powerful model trained on a large set of high-qualit data which provides an optimal trade-off between naturalness and transferability. Compared with Diff-PGD, our frame- work demonstrates significantly better naturalness. In terms of the time cost, DiffAdvMAP still achieves a relatively low time cost. Figure 3 visualizes the adversarial exam- ples generated by the three diffusion-basee attack methods, providing subjective perspective on their naturalness.
Figure 4. Qualitative results regional color UAEs and style UAEs, the surrogate model is the Resnet50, predicted labels are in white.
In some scenarios, UAEs can only be generated by modi- fying some specified regions of the original images. Such UAEs look similar to the original images contain the same objects but differ in some attributes. In this section, we conduct experiments on generating customized UAEs: color UAEs generated by changing the color of reference images, and style UAEs generated by changing the style of reference images toward the target style. The qualitative

---

DiffAdyMAP: Flexible Diffusion-Basee Framework for Generating Natural Unrestricted Adversarial Examples
Table The white-hox attack rate()), transfer attack success (%), image quality metrics, as well as the time(sec) of DiffAdvMAP and baseline methods in the task of generating global image- similar UAEs.
DEPENDING MODELS
SURROCATE MODELS
ATTACK
FID(I)
LPIPS(.)
TIME
INC-V3
RE:-50
MoB- V2
SWIN-B
CADV
91.7
23.1
29.7
14.3
65.7
0.186
18.7
RECOLOBADY
98.4
31.6
39.3
15.0
63.4
0.154
3.86
47.4
$3.8
70.9
0,383
10.45
INC. 3
DIT--GD
83.8
28.2
34.7
10.3
65.9
0.147
9.6
DIFFATTACK
86.1
39.4
42.9
25.4
62.3
0.127
28.2
DIFAADMMAPOOURS)
100.0
42.8
48.6
30.3
61.2
0.127
6.0
CADY
46.8
97.6
57.5
24.7
65.7
0,186
RECOLORADV
47.9
99.2
63.8
28.1
63.4
0.154
NCF
47.4
88.7
69.7
23.2
70.9
0,383
RES-50
DuR-PGD
53.0
95.8
63.8
31.5
66.6
0,170
DITFATTACK
96.3
76.6
56.2
62.6
0.137
DIFAADMAP(OURRS)
100.0
50.5
96.6
68.6
0,211
RECOLORADY
48.7
40.4
99.8
63.3
0.157
NCF
48.1
64.0
92.6
23.9
69.7
0.387
MoB-V2
DUT-PGD
50.1
56.3
94.9
27.3
65.7
0.164
DUEAFTACK
67.8
76.3
98.0
54.2
62.9
0.138
DITAADMMAP(OUS)
64.6
77.0
100.0
54.7
60.0
0.135
CADV
43.2
40.9
46.1
98.4
67.4
0.191
RECOLOBADY
37.6
36.5
42.1
65.7
0.147
NCF
65.5
SWIN-B B
DIR-PGD
46.6
53.1
70.6
0.189
DIFAATTAK
57.7
$6.6
58.4
90.1
65.5
0.138
DIFADVMAP(OURS)
55.6
56,9
63.5
99.1
64.9
0.125
Table 3. Evaluation of the robustness against three defense strategics. A higher white-box attack success rate(%) means better robustness. The surrogate model is Inception
ATTACKS
R&P
NRP
DIFFPURE
ADV-INC-V3
INC-r3ensa
INC-e3ensi
INCRES-D2ent
11.7
53.4
52.5
31.0
37.4
36.4
23.2
RECOLORADV
8.1
57.4
50.6
30.0
32.6
32.5
18.8
NCF
33.6
71.6
67.8
51.2
52.8
$1.0
39.5
DIFE-PGD
30.0
57.3
64.7
31.9
37.0
34.7
19.1
DIFFATTACK
34.5
83.9
72.2
54.0
56.2
56.9
41.7
DIFTADMMAP
46.8
93.3
78.6
51.9
58.2
58.5
44.2
results of changing the color and style of specific objects in the original images are shown in Figure 4. We also conduct experiments on generating regional image-similas UAEs, which more broader perspective, please refer to Ap- pendix F for more details.
optimal trade-off between image naturalness, transferability, runtime, and defense robustness in the black-box setting, which makes it outperform most baseline attacks. Moreover, DiffAdvMAP is flexible enough to generate UAEs under various scenarios, making more effective in various attack conditions, posing a significant challenge to DNNs.
5. Conclusion
Impact Statement
In this paper, we introduce a flexible diffusion-based un- restricted adversarial attack framework, DiffAdyMAP. We generate natural UAEs by sampling adversarial latent code from the approximated posterior distribution of the UAEs. Near 100% white-box attack success rate shows that our framework effectively defeats top-ranked robust models while keeping the naturalness of UAEs. In addition, our framework outperforms current SOTA with more natural- ness and less time cost. DiffAdyMAP also achieves an
This paper presents work to advance the Unrestricted adver- sarial attack. The community has discussed many potential societal consequences comprehensively, none of which we feel must be specifically highlighted here.
8

---

DifIAdyMAP: Flexible Diffusion-Barnn Framework for Generating Natural Unrestricted Adversarial Examples
Acknowledgement
Deng, J., Dong, W., Socher, R., Li, L.-.,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference Computer Vi- sion and Pattern Recognition, pp. 248-255 2009. doi: 10.1 1..109/VRR.0009 200955588...
This work was supported by the National Natural Sci- ence Foundation of China (NSFC) under Grant numbers 62171184, U23A20385, and 62273139. The authors sin- cerely acknowledge the foundation for their financial sup- port, which made this research possible. The authors also acknowledge the constructive feedback of reviewers and the work of ICML'25 program and area chairs..
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8788 8794, 2021.
References
Dosovitskiy, A. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint urXiv:2010. /1999, 2020.
Bao, H., Dong, L., Piao, S., and Wei, F. Beit: Bert pre-training of image transformers. arXiv preprint (XXiv2216.08855: 2021.
Engstrom, L., Ilyas, A., Salman, H., Santurkar, S., and Tsipras, D. Robustness (python library), 2019. URL https: ortps://githus com/ wwl.ccom//MMaa. MadzyLab/ robustness.
Bhattad, A., Chong, M. J., Liang, K., and Li, B. Unrestricted adversarial examples via semantic manipulation. arXiv preprint arXiv: 19044.644,, 2019.
Gatys, A., Ecker, A. S., and Bethge, M. Image style transfer using convolutional neural networks. In 2016 IEEE Conference Computer Vision and Par- tern Recognition (CVPR), pp. 2414-2423, 2016. doi: 10.1 N0.1109/CVPR.20162665
Carlini, N. and Wagner, D. Towards evaluating the robust- ness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp- 39-57. leee, 2017.
Chen, J., Chen, H., Chen, K., Zhang, Y., Zou, Z. and Shi, 7. Diffusion models for imperceptible and transferable adversarial attack. IEEE Transactions on Pattern Analysis and Machine Intelligenee 2024a.
Golestaneh, S. A., Dadsetan, S., and Kitani, K. M. No- reference image quality assessment via transformers, rel- ative ranking, and self-consistency. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp- 1220-1230, 2022.
Chen, X., Gao, X., Zhao, J., Ye, K., and Xu, C.-Z. Ad- vdiffuser: Natural adversarial example synthesis with diffusion models. In Proceedings of the IEEF/CVF In- ternational Conference on Computer Vision (ICCV). pp- 4562-4572, October 2023.
Goodfellow, J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples. urXiv preprint arXiv:1412.6555 2014.
Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., and Zhang, W. Content-based unrestricted adversarial attack. Advances in Neural Information Processing Systems, 36, 2024b.
Harvey, W., Naderiparizi, S., Masrani, V., Weilbach, C., and Wood, Flexible diffusion modeling long videos. Advances in Neural Information Processing Systems, 35: 27953-27965, 2022.
Chung, H., Sim, B., and Ye, J. C. Come-closer-diffusee faster: Accelerating conditional diffusion models for in- verse problems through stochastic contraction. Pro- ceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 12413-12422, 2022.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual leam- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Couairon, G., Verbeek, J., Schwenk, H., and Cord, M. Diffedit: Diffusion- based semantic image editing with mask guidance. arXiv preprint arXiv:2210. 1427, 2022.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter. S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Croce, F., Andriushchenko, M., Sehwag. V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09677 2020.
Ho, C.-H. and Nvasconcelos, N. Contrastive learning with adversarial examples. Advances in Neural Information Processing Systems, 33: 13:17081-17000 2020.
Dai, X., Liang K., and Xiao, B. Advdiff: Generating unrestricted adversarial examples using diffusion models. In European Conference on Computer Vision, pp. 93-109. Springer, 2025.
Ho, and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.15599, 2022.

---

DifTAdyMAP: Flexible Iuffusion-Basee Framework for Generating Natural Unrestricted Adversarial Examples
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
guidance. In Proceedings of the IEEE/CVF winter con- ference on applications of computer vision, pp. 289-299, 2023b.
Ho, J.. Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, J. J. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022.
Liu, Z., Lin, Y., Cao. Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Jia, S., Yin, B., Yao, T., Ding, S., Shen, C., Yang, X., and Ma, C. Adv-attribut: Inconspicuous and transferable ad- versarial attack on face recognition, Advances in Neural Information Processing Systems, 35:34136-34147, 2022.
Long, Y., Zhang, Q.. Zeng, B., Gao, L.,, Liu, X., Zhang, J., and Song, J. Frequency domain model augmentation for adversarial attack. In European conference on computer vision, pp. 549-566. Springer, 2022.
Kurakin, A., Goodfellow, I,, Bengio, S., Dong, Y., Liao, F,, Liang, M., Pang. T., Zhu, J., Hu, X., Xie, C., et al. Adversarial attacks and defences competition. The NIPS' 17 Competition: Building Intelligent Systems, pp. 195-231. Springer, 2018
Lyu, Z., Kong, Z., Xu, X., Pan, L.,, and Lin, D. conditional point diffusiin--reimemmnt paradigm for 3d point cloud completion. arXiv preprint arXiv:2112.035500 2021.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1oXiv:/706.000 706.06688, 2017.
Kwon, G. and Ye, J. C. Diffusion- based image translation using disentangled style and content representation. arXiv preprint arXiv:2209.1 0tt222229..122... 15264, 2022.
Laidlaw, C. and Feizi, S. Functional adversarial attacks. Advances in neural information processing systems, 32, 2019.
Meng, C., He, Y., Song, Y., Song. J., Wu, J., Zhu, J.-Y., and Ermon, S. Sdedit: Guided image synthesis and edit- ing with stochastic differential equations. arXiv preprint arXiv:2108.0100 2021.
Lee, S., Lee, D. B., and Hwang, S. J. Contrastive learn- ing with adversarial perturbations for conditional text generation. arXiv preprint arXiv:2012.07280, 2020.
Mokady, R., Hertz, A., Aberman, K., Pritch, Y., and Cohen- Or, D. Null-text inversion for editing real images us- ing guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6038-6047, 2023.
Li, P., Zhang, Y., Yuan, I.., Zhao, J., Xu, X., and Zhang, X. Adversarial attacks on video object segmentation with hard region discovery. IEEE Transactions Circuits and Systems for Video Technology, 2023.
Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. Deep- fool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2582, 2016. Naseer, M., Khan, S., Hayat, M., Khan, F. S., and Porikli, F. A sulf-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 262-271, 2020.
Li, X., Chen, L., and Wu, D. Turning attacks into protec- tion: Social media privacy protection using adversarial attacks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 208-216. SIAM, 2021a.
Li, Z., Shi, Y., Gao, J., Wang, S., Li, B., Liang, P., and Hu, W. A simple and strong baseline for universal targeted attacks on siamese visual tracking. IEEE Transactions on Circuits and Systems for Video Technology, 32(6): 3880-3894, 2021b.
Nichol, A. Q. and Dhariwal, P. Improved denoising diffu- sion probabilistic models. In International conference on machine learning, pp. 8162-88688 1171-171 PMLR, 2021.
Liu, J., Lau, C. P., and Chellappa, R. Diffprotect: Generate adversarial examples with diffusion models for facial privacy protection. arXiv preprint arXiv:2305.13625, 2023a.
Nie, W., Guo, B., Huang, Y.. Xiao, C., Vahdat, A., and Anandkumar, A. Diffusion models for adversarial purifi- cation. arXiv preprint arXiv:2205.074666 2022.
Liu, X., Park, D. H., Azadi, S., Zhang, G., Chopikyan, A., Hu, Y., Shi, H., Rohrbach, A., and Darrell, T. More con- trol for free! image synthesis with semantic diffusion
Qiu, H., Xiao, C., Yang, L., Yan, X., Lee, H., and Li, B. Se- manticady: Generating adversarial examples via attribute- conditioned image editing. In Computer Vision-ECCV
10

---

DiffAdyMAP: Flexible Diffusion-Ban Based Framework for Generating Natural Unrestricted Adversarial Examples
2020: 16th European Conference, Glasgow, UK, Au- gust 23-28, 2020, Proceedings, Part XIV 16, pp. 19-37. Springer, 2020.
Szegedy, C.,, Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I,, and Fergus, R. Intriguing properties of neural networks, arXiv preprint arXiv: 1rXXiv:3112.699 2013.
Rasul, K., Seward, C., Schuster, I., and Vollgraf, R. Au- toregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International conference on machine learning, pp. 8857-8868. PMLR, 2021.
Szegedy, C., Vanhoucke, V., loffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vi- sion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818 1-2216,666621111 2016.
Tashrro, Y., Song, J., Song, Y., and Ermon, S. Csdi: Con- ditional score-based diffusion models for probabilistic time series imputation. Advances in neural information processing systems, 34:24804-24816, 2021.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffiusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., and McDaniel, P. Ensemble adversar- ial training: Attacks and defenses. arXiv preprint arXiv: 1.XXv:7705.07204 2017.
Saharia, C., Ho, J.. Chan, W., Salimans, T., Fleet, D. J., and Norouzi, M. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 0114713-47666 4 2023. ISSN 19393539. doi: 10. 10.100//PPMM.2222.334444
Wong, E. and Kolter, J. Z. Learning perturbation sets for ro- bust machine learning. arXiv preprint arXiv:2007.084500 2020.
Salman, H., Ilyas, A., Engstrom, L.,, Kapoor, A., and Madry, A. Do adversarially robust imagenet models transfer bet- ter? Advances in Neural Information Processing Systems, 33:3533-3545, 2020.
Xiao, C. Li, B., Zhu, J.-Y., He, W., Liu, M., and Song, D. Generating adversarial examples with adversarial networks. arXiv preprint arXiv: 1ttt:::8000.00010 2018.
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
Xie, C., Wang. J., Zhang. Z., Ren, Z., and Yuille, A. Miti- gating adversarial effects through randomization. arXiv preprint
2017.
Xie, C., Tan, M., Gong, B., Wang. J., Yuille, A. L., and Le, Q. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 819-822 00-82,8 2020.
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explana- tions from deep networks via gradient- localization. In Proceedings of the IEEE international conference on computer vision, pp. 618 626, 2017.
Xue, H., Araujo, A., Hu, B., and Chen, Y. Diffusion- adversarial sample generation for improved stealthiness and controllability. Advances in Neural Information Pro- cessing Systems, 36:2894-2922 2023.
Singh, N. D., Croce, F, and Hein, M. Revisiting adver- sarial training for imagenet: Architectures, training and generalization across threat models. Advances in Neural Information Processing Systems, 36:13931-13955, 2023.
Yuan, S., Zhang, Q., Gao, L., Cheng, Y., and Song. Natu- ral color fool: Towards boosting black-box unrestricted attacks. Advances in Neural Information Processing Sys- tems, 35:7546-7560 2022.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Zhang, G., Ji, Zhang, Y., Yu, M., Jaakkola, T., and Chang, S. Towards coherent image inpainting using denoising diffusion implicit models. In Proceedings of the 40th In- ternational Conference on Machine Learning, ICML'23, 2023.
Song, Y., Shu, R., Kushman, N., and Ermon, S. Constructing unrestricted adversarial examples with generative models. Advances in neural information processing systems, 31, 2018.
Su, S., Yan, Q. Zhu, Y., Zhang, C., Ge, X., Sun, J., and Zhang, Y. Blindly assess image quality in the wild guided by a adlf-adaptive hyper network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3667-3676, 2020.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.

---

DiffAdyMAP: Flexible Diffusion-Basee Framework for Generating Natural Unrestricted Adversarial Examples
Zhou, L.,, Du, Y., and Wu, J. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF international conference computer vision, Pp. 2021.
12

---

DiffAdyMAP:
Flexible Diffusion-Bassn Framework for Generating Natural Unrestricted Adversarial Examples
A. Overview
Here is the overview of the appendix, we will first provide a detailed Bayesian derivation of the approximated posterior distribution as well as the derivation AdvMAP method in Appendix B. Then we put the pseudo-code of DiffAdvMAP and DiffAdvMAP-Regioo in Appendix C. The experimental results under the white-box setting will be evaluated in Appendix D. Appendix E shows the ablation study of each module and the super-parameters of DiffAdvMAP. Appendix F presents the experiments of Regional image-similar UAEs, we also visualize the qualitative results. Appendix G shows more qualitative results of UAEs generated by our framework.
B. Detailed Bayesian Inference and Derivation of DisfAdvMAP
Given the input noise image Ty (0, I) of the diffusion model, the ground truth label y, the adversarial constraint C₁:
G ((o),, (a
(15)
Where S 0 is the confidence level of fooling the classifier, ((Ao); is the logit output of classifier Fg. at entry i with Bgg as input. For convenience, we denote the logit difference Z-)) ma (-())) as 1(-).
The reconstruction constraint C2:
C2 m o (((o)) = m 2())
(16)
Here, means element-wise multiplication and m the mask used to deal with different kinds of regeneration regions. Specifically, when UAEs are generated globally, m is an identity matrix; when UAEs are generated some specified regions, is the mask that covers such specified regions. Only the regions covers should be generated when generating regional UAEs. Function (2(-) is customized function for generating UAEs in different scenarios. For generating image-similar UAEs, ō is the original reference image, M(x)= = for generating style UAEs, . is an extra image that contains the target style, (2)) computes the style score; as for generating color UAEs, ž is the reference image after changing color, 22(.)
converts images from the RGB space into the LAB space.
The posterior distribution image-similar UAEs and UAES generated from noise can derived as follows:
Po(fT, C1,
pe C1, C2)
Pa(Ci, C2)
Po(C1, C2(TT) * pa(TT)
(17)
Po(Ci) po (C2)
(rr) po C2 (Tr) (itr)
(C(Ci) pe (C2)
po M,, Ci)
Po C1)
Poly, C1)
poly. Ci (GT) po(T))
Po(y) * po (C2)
Po ( pe ( ( po
(18)
po(y) * pop(Ci (C1)
Po (o(r))epo(s) (a(Tr
pe p (C)(eT) (0TT
Po (v po (Ci)
po * po (C
pa (C1)
The posterior distribution of latent code 2,- given latent code 2; can be derived:
po |.,, C1, C2) Pe (ār (.x)) po (āt) (C 02(0;-1,
po (āt, C1, C2)
Pa ( (.+.)) Pe ( po (Ci [ 2x) po
(19)
pe (+x) pe C22 (z)) (o()
Po 1 * Pe (C1 1 1)= po (C2 (2)
Pe (C [.., )pu (C2l
13

---

DifTAdyMAP: Flexible Seffusiin-Basce Framework for Generating Natural Unrestricted Adversarial Examples
10(f1-1/21, C1) po pa Dt, y) (o( y) * (po(Ci|@tr1i. Zz,
(3 y, C1)
po -1 |e y) ( x
(20)
po(C1, y [ix) * pa
Pa - |it, y) pe ( (111)
po (C(CCl (2,
Note that since we leverage DDIM, whose generation process is deterministic, as a result, when ir in equation (17)(18)
and - in equation (19) is given, adversarial goal C₁ and reconstruction constraintC2 is independent. Then according to equations (15) and (17), pa (ûT) a Gaussian distribution, and since when ěт is given, So a deterministic function of Tr. Po (1(6)) = c/æx) and pø (m o (8)) = m ())) follows Dirac delta function 5(), then po (C( = po(1(u)) pe pm(m= ()(g)) m o ()))) 6(m (2(oo) = m ()))), which means the probability density is infinite at (2ra)= c, (
cl³r) ((())) c). (c
{))), and 0 elsewhere. The Dirac delta function can be
approximated by Gaussian density function with zero variance. Therefore, if we take the logarithm to equation (17), we can approximate it as:
log Pa (2r
C1, C2)
1
|
1/
||c 1(20)]
|lm o (20o) m (()) +
2
257
20
(21)
~
2
p 2
257
||c ((Go((T))
25
m ( m m M())
where C is the normalizing constant, a and & are the standard deviation the approximated Gaussian distribution of the Dirac delta functions respectively, when & and &2 approaches 0, the approximations go exact.
However, computing Go (âT) needs go through the whole reverse generation process, which is time-consuming. As result, we perform one-step approximation of &o for each step t [T' in DDIM:
24
vI acco (2x, t)
δu fa(e))
(22)
Vār
Then the conditional distributions of (20)) and 2o given ütr can be approximated as Gaussian distributions centered around the one-step approximated value fd (xT):
((()))) N(((6)): [(( (
(23)
Po(m (( (mm o ((žo); m o 2( (2r)),
The approximation of log probability density computed from equation (17) and equation (18) are as follows:
log ( C2)
=log(po (ir)) + log(l6 = clēr) log(p' (m l((r)) m (())2rr) + C'
(24)
= = 12rl 1 2
||c 11.6 ( 2 m 2( (œr)) - m 2(2)
2017
1o [y, Ci)
log(po (v)) + log(ll (1(o)) = c(2r)) + C'
(25)
-
1
1
||c 1(fo (fr)) C
207
Ci 1,2) is the standard deviation of approximated Gaussian (istributtinn)((((t)))) cl²r), which is different from Gi 1, in equation (21), it should be large enough capture the approximation error. Then since the generation process of UAES under C₁ and C2 can be decomposed as:
Polaoo C2) (iT/CC C2) II (6. C1, C2)
(26)
14

---

DiffAdyMAP: Flexible Diffusion-Bawee Framework for Generating Natural Unrestricted Adversarial Examples
(orrly C1) = (ōrly, Ci)
II pb (a, v, 2,, C1)
(27)
We can also approximate the intermediate conditional distributions given latent code é; as:
(.x) N((()))
(28)
Pa (m (2)) (Rt)) = Nm o (((o); m ((f(( 1
Then the reverse generation process given equation (19)(20) can be computed as:
log 1og//((; (i, Ci, C2)
(())) (((())(((()) c(ãr- (x)) + log(pb(m ((()) - ( C'
(og(po(@)-11 log(p' (((oo) c(ã--1)) log(p6(m l((x)) m o ())
(29)
|| | - Del.
- ||c- (
m S(5 (a,-)))) [0-00) - 0(
-
20
211
25 252(-1
log (6:-((0:1 y/, C1)
log(po(6z-1(átt)) + log(r/ c 1, 11 y)) C'
(oy(po(11111 (m:)) + log (l(0)) = c(f2-1)) + C"
(30)
| - iuel1
||c
105b (81-1))112 C
20?
26
note that as is shown in equation (3), in DDIMs,
ft Va,to
f aa- 1a + V 1 -8,
0:-1
V1 āt
(31)
= - + /1 11 at-1
07
v
vā.f%
v1 ā
C. Pseudo-code
The pseudo-code of DiffAdvMAP and DiffAddvMAP-Reiie Region is shown in Alg.1 and Alg. 2 respectively.
15

---

DiffAdyMAP:
Flexible Diffusion-Ban Based Framework for Generating Natural Unrestricted Adversarial Examples
Algorithm 1 DiffAdvMAP
Input: optional reference image z, ground truth label y. diffusion model Eg. target classifier Fs, forward diffusion steps t", random Gaussian noise c, noise schedule 9t.r. MAP iterations I, MAP learning rate in, adversarial confidence level e if exists then
21 V else
for t = T to + 2t-1 = (at-if(((z) + V1 ãe-1 p4- Vo (a(ee)
V1-ai
end for end
Tt. Z'''
for i = to I do
fr. .t- Ir V (6 C2)))
if arg max Folfo (#x-) # then
break end end for
for = t" to 1 do
Ar = vār- (ft) + VI- ar-1 farlftt
VT-ai
-1-1 1 = jM:
for i = 1to I do
- -t-1 lr V(log(ph (2,-) 116e, Ci, C2))
if arg max Fol50 (f+- # y then
break
else
1r=1r*2
end if end for end for return sio
Algorithm 2 DiffAdyMAP-
Input: reference image æ. mask m. ground truth label diffusion model es, target classifier Fa. forward diffusion steps 1", random Gaussian noise noise schedule $1., MAP iterations 7, MAP learning rate Ir, adversarial confidence level e
vār= (1 ā.-)E
ft. m o Zt- + (1 m) or
for 0
do
so m fø (f+- + (1 m) or
f.. -.. +lr Ir V (log(pr
C2)))
if arg max maxFFF(s) (& # y then
break end end for for ts to 1 do De Vōr- v1 dc-1 VI-ai
f,-1 i4 (1 m)
for to do
âp m o f (2.
m)
_1_1 2: Ir * V (log(196 C2))
if arg max F (āo) * y then
break
else
1r=1=*2
end if end for
return śu
16

---

DiffAdyMAP: Flexible Diffusion- Baasdd Framework for Generating Natural Unrestricted Adversarial Examples
D. UAEs Under the White-box Setting
In this part, we compare the naturalness and effectiveness of our framework in terms of image quality and attack success rate against both normal models and robust models. We select the sfate-of-the- white-box liffusion-based unrestricted adversarial attack method: AdvDiffuser (Chen al., 2023) as the baseline method. We evaluate the performance of our framework the identical dataset as AdyDiffuser, it's a subset of the ImageNet test set which contains 1000 randomly selected images, 1 image for each class. We apply the FID score, LPIPS score, and SSIM metric to evaluate the quality of UAES. As for the classifiers, we select a normally trained Resnet50 (He et al., 2016) as the baseline model, and three robust models from the RobustBench leaderboard (Croce et al., 2020) to evaluate the effectiveness of our framework against robust models: a Robust Resnet50 (Salman et al., 2020) B, a Robust Wide-Resmet50-- (Salman et al., 2020) A (these two are current most robust convolutional networks), and an adversarially trained Resnet50 with the PGD attack (Engstrom et al., 2019). We also conduct experiments on vision transformer-basee classifiers: a normally trained vit-b (Dosovitskiy, 2020), Beit (Bao et al., 2021), and robust vit-b (Singh et al., 2023) from RobustBench leaderboard. Note that since the authors of Adv didn't t offer their code, we compare
method with results proposed in their paper, so also compare with AEs generated from Diff- PGD((uu et al., 2023) with our reimplementation. We respace the number of diffusion steps from T = 1000 to T = 400 and set the forward diffusion step of DiffAdvMAP to t = 3. The adversarial confidence level is set to C = -10.
As is depicted in Figure 5, AEs generated by Diff-PGD contain obvious noise patterns when attacking robust models. Meanwhile, though UAEs generated by AdvDiffuser look natural, they change too many low-level features, taking unnatural features to UAEs when compared with the original images, for example, the bird's 's beak in the first column is almost gone. As for our framework, we leverage the prior knowledge of natural data to generate high-level adversarial features instead, which makes UAEs look more natural than AdvDiffuser and Diff-PGD.
DITAUAMAP
AdvDlfluse
DIT PSD
Figure 5. Image- similar UAEs generated by Diff'AdMAP,, AdvDiffuser (Chen et al., 2023) and Diff-PGD (Xue et al., 2023). The defending model is the robust Wide-Resnet50-2 from (Salman al., 2020).
We also conduct quantitative experiments to evaluate the effectiveness and naturalness of our framework. Table 4 presents the quantitative results of Adviiffuser, Diff-PGD, and DiffAdvMAP respectively in terms of attack success rate, runtime for each sample, LPIPS score, SSIM metric, and FID score. As we can see, our framework achieves near 100% white-box attack success rate against both normal models and robust models. Our framework also generates more natural UAEs, represented much lower LPIPS, and FID scores. Meanwhile, the significantly reduced runtime addresses the inherent shortcoming of the slow generation speed of diffusion models, while still preserving the naturalness of UAES.
We also amplify and compare the perturbations added by each method in Figure 6. We can observe that in UAEs generated by our framework, perturbations are tiny and coherent with class-specific semantics, so the prominence areas with low information is greatly reduced. Our observations show that sampling UAEs from the posterior distribution of UAEs effectively improves their naturalness while maintaining a high attack success rate. Moreover, the use of the destruction
17

---

DiffAdyMAP: Flexible Diffusion-Basce Baasod Framework for Generating Natural Unrestricted Adversarial Examples
and construction method allows to regenerate adversarial high-level features through few generation steps, which improves the naturalness and generation speed further. In contrast, though AdvDiffuser achieves better SSIM metric than our framework by introducing the information of original images in each step, it generates UABs from the very beginning of the generation process and perturbs each latent code step by step, which suffers from diffusion models' slow generation speed. also changes too many low-level features completely and only leverages the diffusion model to purify unnecessary perturbations, bringing strangeness and unnaturalness to UA..(e.g. the claw of the chihuahua and the hog's nose and mouth.), and thus appears more noticeable and less imperceptible.
Table Comparing global image- similar unrestricted attacks ImageNet defending models, also include the best-known robustness within loc = 4/255 for cach model.
ATTACKER
AARR %
(OPIP(()/SSSM(C FID(J)
TIME
NORMAL RESNET5O (HE ET AL., 2016)
4x255
100
DIT--GG
98.7
0.180
0.82
57.61
10s
ADVDIEUSEE
100
0.03
0.99
20.9
90s
ADMMADM
100
0.006
0.97
6.83
4.
ROBUST WIDE RESNET50-2 FROM (SALMAN ET AL.. 2020)
4/255
61.9
DIFF-PGD 88.6
0.201
0.82
81.54
ADVDIEFUSE
99.5
0.05
26.7
DIFFADYMAP
99.3
0.011
0.97
12.75
ROBUST RESNET5O FROM (SALMAN
2020)
44/25
65.1
DIFF-PGD 90.5
0.203
0.82
87.03
ADVDIFFUSER
99.8
0.05
0.97
27.2
DIFFADYMAP
99.8
0.007
0.97
9.95
ROBUST RESNET50 FROM (ËNGSTROM ET AL.. 2019)
4x255
70.8
DIEF-PGD 91.5
0.21
0.80
89.26
ADVDIFFUSER
99.4
0.05
0.98
25.9
DIFFADMMP
99.4
0.012
0.97
13.20
NORMAL BEIT (DosOVITSKIY, 2020)
4<245
100
DITT-PG
98.3
0.161
0.82
39.02
ADVDIFFUSER
DIFFADYMAP
100
0.006
0.97
3.87
NORMAL VIT-B (BAO ET AL., 2021)
4/255
100
DIFF-PGD
92.3
0.182
0.82
43.96
ADVDIFFUSER
DIFFADMMAP
100
0.015
0.97
7.64
ROBUST VIT-B B FROM (SINGH AL., 2023)
1x 4/255
45.3
DIFF-PGD 72.1
0.200
0.82
67.67
ADVDIFFUSER
DIRFADYMAP
93.8
0.015
12.41
18

---

Dif:AdyMAP: Flexible Ceffusion-Basce Framework for Generating Natural Unrestricted Adversarial Examples
Figure Perturbations generated by DiffAdvMAP and AdvDifruser respectively. The defending model is robust Wide-Resnet50-2 from (Salman et al., 2020). The predicted labels are shown in white.
E. Ablation Study of Global Image-Similar UAEs Generation
In this section, we use Inception V3 as the surrogate model and analyze the effect of the reconstruction constraint and the forward diffusion step towards the UABs in terms of white-box attack success rate, naturalness, transferability, and generate time. The results are reported in Table 5. As
see, the destruction and reconstruction module makes great contribution towards improving the run time, but it is somewhat detrimental to the image quality; the adversarial constraint ensures the near 100% white-box attack success rate and high transferability, but it will reduce the image quality of UAEs; the reconstruction constraint in generating mmage-similar UAEs can help preserve important semantics of original images, thus improving the image quality, though will reduce the transfcrabiltiy to some extent. In short, we find a better trade-off between transferability, white-box attack success rate and image quality in our DiffAdvMAP framework, and outperform other baseline attacks. We also make an ablation study of adversarial confidence level c, the results are shown in Table 6.
Table 5. Ablation Study of DiffAdvMAP in terms of each module on the Imagenet compatible dataset, the surrogate model is Inception w/o means without the module in the framework.
WHITE-BOX ATTACK
NATURALNESS
TRANSFFRABILITY
SETTINGS
TIME
SUCCESS RATE
FID(()
LPIPS(.)
RES-50
MOB-v2
SWIN-B
W/O RECONSTRUCTION CONSTRAINT
100.0
63.3
0.171
52.5
55.1
40.8
6.0
w/o ADVERSARIAL CONSTRAINT 25.0
58.1
0.056
11.4
16.2
5.7
6.0
w/o DESTRUCTION RECONSTRUCTION
100.0
61.8
0.116
35.9
45.2
24.4
44.5
DIFFADYMAP(OURS)
100.0
61.2
0.127
42.8
48.6
30.3
6.0
As we can see, the adversarial constraint ensures that our framework consistently achieves a near 100% white-box attack success rate, regardless of the adversarial confidence level. As the absolute value of the confidence level increases, the attack strength of the UAEs also increases, which evident in their enhanced transferability. However, the image quality diminishes the absolute value of rises, due to more prominent adversarial features, discussed in the first paragraph of Section 4.3. Nonetheless, the experimental results demonstrate that our framework achieves superior balance between naturalness and attack strength compared to baseline attacks.
19

---

DiffAdyMAP: Flexible Diffusion-Baane Basood Framework for Generating Natural Unrestricted Adversarial Examples
Table Ablation Study of DiffAdvMAP of adversarial confidence level the Imagenet compatible dataset, the surrogate model is Inception V3.
WHITE-BOX ATTACK
NATURALNESS
TRANSFERABILLTY
ADVERSARIAL CONFIDENCE LEVEL
SUCCESS RATE
FID()
LPIPS(()
RES-50
MOB-V2
SWIN-B
-5
98.9
58.5
0.085
16.3
23.8
10.7
-10
99.7
58.5
0.089
20.5
27.6
11.5
-15
99.9
58.5
0.092
23.5
30.5
14.2
-20
100.0
59.0
0.098
26.7
34.1
16.6
-25
100.0
59.5
0.106
30.3
38.2
20.0
-30
100.0
59.9
0.112
35.2
40.9
22.9
-35
100.0
60.4
0.119
38.4
44.5
26.4
-40
100.0
61.2
0.127
42.8
48.6
30.3
F. Regional Image-Similar UAEs Generation
In this section, we leverage random square masks to specify the regions to be perturbed, generating regional image-similar UAEs. We compare the regional adversarial examples with Diff-PGD visually in Figure 4. We select adversarially trained Resnet50 the surrogate model. As we see, in the specified regions, our framework features, while for Diff-PGD, strange textures and noise patterns exist in the image, for instance, in the first column, the white flower displays strange red color; in the fourth column, the noise texture in the left bottom of the fig is quite obvious.
still generate natural adversarial
Original Images
Shoji
DiffAdMMPP
Sliding Door
Asomm
Diff-PGD
Sliding Door
Figure 7. UAFs generated by DiffAdvMAP-Region with random square masks, the defending model is robust Resnet50 from (Engstrom et al., 2019), predicted labels are in white.
We also introduce more qualitative results well the perturbations in Figure 8, the regional UAEs generated against different robust models with two kinds of masks: random square masks and irregular masks generated by Grad- CAM(Selvaraju et al., 2017) method. For random square masks, perturbations become extremely significant, especially when most of the mask doesn't 't include semantic useful information, leading to unnatural adversarial examples or even failure. combined with Grad-CAM to find out the region where the defending model extracts features to predict the ground truth label, the perturbations become coherent with semantics and thus less significant and perceptible.
We also compare our method with Diff-PGD quantitatively, is shown in Table 7, the experiments are done under the white-box setting. As can achieve much higher attack success rate against normal and robust models than Dif-PGD while maintaining a better image quality.
20

---

DifTAdyMAP: Flexible Diffusion-Banne Framework for Generating Natural Unrestricted Adversarial Examples
Figure UAEs specified regions against a robust RedeeResnet50-2 from (Salman al., 2020) with square masks and Grad-CAM masks respectively. We also include corresponding perturbations and predicted labels.
Table 7. Comparing Regional UAEs on ImageNet defending models.
ATTACKER
ASR(%)
LPIPS
SSIM
FID
NORMAL RESNET50
=4/55
100
DIFF-PGD
89.7
0.03
0.97
24.53
DIFFADVMAP
99.8
0.01
0.99
7.05
ROBUT W FROM (SALMAN ET AL.,, 2020)
=x/555 61.9
DIFF-PG 36.5
0.06
0.96 0.98
30.64
DIFFAVMMP
81.6
0.02
14.75
ROBUT RESNET50 FROM (SALMAN AL., 2020)
=4/25
65.1
DIFF-PGD 42.4
0.05
0.96
30.18
DIFFADVMAP 91.8
0.01
0.98
10.38
ROBUT RESNET5O FROM (ENGSTROM ET AL., 2019)
4/2555
70.8
DIFD-PD 45.3
0.06
0.96
32.64
DIFFADVMAP
86.9
0.02
0.98
14.01
G. More Qualitative Results of UAEs Generated by DiffAdvMAP
In this section, we will propose more qualitative results of various UAEs generated by DiffAdwMAP, as shown in Figures 9, 10, 11, and 12.
21

---

Dif:AdyMAP: Flexible Diffusion-Barnn Framework for Generating Natural Unrestricted Adversarial Examples
Inception-v3
Resnet 50
Mobilenet-v2
Swin-B
Figure 9. UAEs generated from noise with four normal DNNs as surrogate models.
22

---

Dif:AdyMAP: Flexible Diffusion-Barnn Framework for Generating Natural Unrestricted Adversarial Examples
BAVIK
Inception-v3
Resnet 50
Mobilenet-v2
Swin-B
Figure 10. Global image-similar UAEs with four normal DNNs surrogate models.
23

---

DiffAdyMAP: Flexible Diffusion-Basse Framework for Generating Natural Unrestricted Adversarial Examples
Original Images
Color UAEs
50
Style Images
Style UAEs
Figure II. Global customized UAEs generated by DiffAdvMAP, the surrogate model is normal Resnet50.
24

---

Dif:AdyMAP: Flexible Diffusion-Barnn Framework for Generating Natural Unrestricted Adversarial Examples
Original Images
50
US MAIL
Masks
Sanbay
50
Color UAEs
US MAIL
50
50
Style Images
50
Style UAEs
US MAIL
Figure 12. Regional customized UAEs generated by DifIAdvMAP, the surrogate model is normal Resnet50.
25


Reviews:
[{"review_id": "yFkWb9BKvo", "reviewer": "Reviewer_58n5", "summary": "This paper introduces DiffAdvMAP, a flexible diffusion-based framework for generating Universal Adversarial Examples (UAEs) under various attack conditions. By approximating the posterior distribution of UAEs using pre-trained diffusion models, DiffAdvMAP generates more natural UAEs compared to existing diffusion-based attack methods. The framework incorporates an adversarial constraint to ensure attack effectiveness and a reconstruction constraint to enhance adaptability across different attack scenarios. Experimental results show that DiffAdvMAP outperforms baselines in white-box success rate, transferability, and defense robustness, demonstrating its superior flexibility and effectiveness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the reviews and kindly reminder. We added the theoretical explanation and the experiments on CLIPure as follows. We wish you can improve the score after reading the response.\n1. Efficiency: The main motivation of the paper is to develop a framework that achieves superior naturalness and adversarial effectiveness while maintaining practical generation efficiency. Due to the inherent generation speed constraints of diffusion-based methods, the efficiency of DiffAdvMAP is not the best among the baseline methods. Still, DiffAdvMAP surpasses most of the baselines in attack efficiency, achieves optimal naturalness, and matches ReColorAdv in generation speed. It achieves the best trade-off between naturalness, efficiency, and effectiveness. \\\nThe theoretical analysis of efficiency:\\\n(1) Generating image-similar UAEs: As introduced in Section 2, compared to other diffusion-based methods(DiffPGD, AdvDiffuser, DiffAttack): Standard diffusion models require multiple U-Net queries for each image generation, causing inherent slowness. DiffAdvMAP accelerates this by generating UAEs via a truncated generation process(20 U-Net queries), achieving 30% faster speeds than DiffPGD (30 queries) and 10× acceleration over AdvDiffuser(400 queries). Even with equivalent query counts to DiffAttack, our method's 1-2 MAP optimizations per step enable 4×faster processing than DiffAttack's attention map computations and optimizations. Compared to non-diffusion-based methods: While GANs have fewer parameters than U-Nets, cAdv requires hundreds of network queries, making it slower than some diffusion approaches. Iterative methods like NCF and ReColorAdv bypass network queries entirely—their speed depends on the objective function complexity and iteration count. \\\n(2) Generating UAEs from noise: For the task of generating UAEs from noise, compared to AdvDiff, both of the methods utilize the same generation process of the latent diffusion model to generate UAEs. Since there’s no real image for reference, we employ a strong class guidance to ensure the naturalness of the UAEs, such that DiffAdvMAP should perform more MAP optimization steps to counteract the strong class guidance in generating adversarial features to ensure enough transferability of UAEs. From Table 1, we can see that DiffAdvMAP has better transferability and naturalness than AdvDiff, requiring only 10% more time.\n2. Effectiveness: As illustrated in Sec.3, we introduce an adversarial constraint to ensure the effectiveness and a reconstruction constraint to control the content of generated UAEs. Such two constraints are considered as two events of the posterior distribution of UAEs. Then the posterior distribution is approximated with the prior knowledge of real data learned by the diffusion model. Unlike previous methods which only use diffusion models as strong denoisers to enhance the generation process, DiffAdvMAP generates adversarial features by sampling from the approximated posterior distribution, which make the adversarial features more coherent to the original features, thus improving the naturalness of UAEs. As a result, DiffAdvMAP have more space to improve effectiveness at the cost of sacrificing naturalness. As illustrated in Tab.6 in Appendix E, the white-box attack success rate is nearly 100%, but the transfer ASR is not enough when the absolute value of the confidence level is small. When the absolute value of the confidence level increases, the transferability increases, and although the naturalness decreases, it is still better than other baselines.\\\n(1) We don’t think DiffAdvMAP overfits to the structure of known classifiers. \\\nWhen performing a transfer-based black-box attack, UAEs are generated with a known surrogate model, eg. Inception V3, then these UAEs are used to attack other unknown classifiers eg. Mobilenet V2, Resnet50 and Swin-Transformer and obtain the transfer attack success rate, the principal of transfer attack results in the decrease of attack success rate(ASR). As shown in Table 2, all of the existing black-box attack methods face an ASR decrease when attacking unknown classifiers. \\\n(2) We complement the experiment on a stronger purifier, CLIPure, with DiffAdvMAP and DiffPGD. The ASRs are as follows:\\\n|Methods/Surrogate Models| Mobilenet V2| Swin-transformer| Incepetion V3 |Resnet50|\\\n|DiffAdvMAP|  48.3  | 57.6  | 39.9  | 49.2|\\\n|DiffPGD      |   31.7 |  52.4|  24.6   |33.8|     \nCLIPure is a zero-shot classifier that classifies images by matching an image with text prompts. It also leverages the CLIP model to purify the adversarial examples in the latent space. The ASR demonstrates that DiffAdvMAP can change the semantic information of images coherently, such that it can resist the semantic purification in the latent space and mislead the text-to-image semantic-based classifier to make wrong predictions in a large proportion.      \n3. Typos: Very thankful for your kindly reminder. We will correct it in revised version.", "claims_and_evidence": "Yes, the claims on the flexibility, naturalness and effectiveness of DiffAdvMAP to generate UAEs are supported by generally clear and convincing evidence. \n\nThe claim of the flexibility of DiffAdvMAP is supported by the design of this method in Constraint 2, i.e., C2 in Equation (2). It allows for different scenarios in UAE generation, including style generation and color generation. The qualitative evaluations in Experiments also indicate the flexibility can be empirically achieved.\n\nThe authors also claimed naturalness and effectiveness of DiffAdvMAP. Intuitively and mathematically, Equations (7), (8), (9) and (14) generally demonstrate the control on realism. Equation (4) controls the effectiveness of DiffAdvMAP to attack classifiers. Results in Tables 1 and 2 demonstrate the naturalness and effectiveness of generated UAEs empirically.\n\nHowever, for the efficiency claimed for DiffAdvMAP, the experimental results in Tables 1 and 2 seem not to strongly support it. Compared to the past methods, time cost of DiffAdvMAP is not the SOTA.", "methods_and_evaluation_criteria": "The proposed DiffAdvMAP is consistent with and able to resolve the motivations via the maximum a posterior. In order to achieve the realism, it is natural to sample adversarial examples directly from an approximated posterior distribution and use the prior knowledge of real data distributions learned by diffusion models, instead of imposing strong prior as diffusion models. \n\nIn terms of the flexibility, the constraint in Equation (6) seems to be sound where $\\Omega(\\cdot)$ allows the change of styles, colors and other scenarios to amend $x$ and generate desired UAEs.\n\nImageNet is a commonly applied dataset to investigate the performance of adversarial example generation. Success rates for attacks can evaluate the basic effectiveness of attackers. FID-score, TRES and HYPERIQA are also proper evaluation criteria, because computing LPIPS score needs reference images and DiffAdvMAP should be studied also for the cases without reference images.", "theoretical_claims": "This work does not have theoretical claims.", "experimental_designs_or_analyses": "As aforementioned, the experimental designs are in general sound. The dataset is ImageNet which is ubiquitous to investigate the performance of adversarial example generation. Success rates are necessary for the basic effectiveness of attackers. FID-score, TRES and HYPERIQA are also proper evaluation criteria, because computing LPIPS score needs reference images. Qualitative results are also demonstrated properly.", "relation_to_broader_scientific_literature": "This work enhances adversarial attackers generation to generate more natural and flexible UAEs. It extends diffusion-based attacks beyond standard settings by allowing UAEs to be generated under various attack conditions (e.g., from noise, region-based modifications, color changes). DiffAdvMAP moves from latent-space perturbation (Chen et al., 2023) to posterior-based sampling, leading to more realistic adversarial images. It is also a more flexible adversarial attack framework than previous diffusion-based approaches.", "essential_references_not_discussed": "For the defense methods against adversarial attacks, a recent and important paper: CLIPure [1] should be mentioned and discussed, because it exploits the semantic information in adversarial examples and achieve the SOTA performance, on top of DiffPure.\n\n[1] MingKun Zhang, Keping Bi, Wei Chen, Jiafeng Guo, and Xueqi Cheng, CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification, ICLR 2025.", "other_strengths_and_weaknesses": "**Strengths**:\n- *Originality*: This paper proposes an original approach to adversarial attack generation by integrating diffusion models with Bayesian inference. Unlike conventional adversarial attacks that rely on fixed perturbations, DiffAdvMAP samples adversarial examples from a learned posterior distribution, leading to more natural unrestricted adversarial examples (UAEs). Flexibility and realism of the generation is also well addressed.\n\n- *Clarity*: The paper is well-organized and clearly written, making it accessible to readers familiar with adversarial attacks and diffusion models. The motivation, methodology, and experiments are explained in a logical flow.\n\n**Weakness**:\n- *Quality*: The paper does not provide a thorough analysis of the computational cost of DiffAdvMAP compared to traditional adversarial attack methods. The time cost of DiffAdvMAP is also not the smallest among the compared past methods in experiments. The biggest limitation of DiffAdvMAP is its decreased effectiveness on unseen classifiers in black-box attack scenarios, although DiffAdvMAP relies on the pretrained diffusion models. It may indicate that DiffAdvMAP overfits to the structure of known classifiers, making it less effective in real-world settings where models are often unknown.", "other_comments_or_suggestions": "List of typos:\n- Page 1 Introduction Line 35-37 \"Such approaches do not require perturbing real images wi0000000000000000000000th restricted perturbations,\" -> \"Such approaches do not require perturbing real images with restricted perturbations,\"", "questions_for_authors": "1. Would the authors provide more insights either theoretically or empirically on why DiffAdvMAP is more effective and efficient compared with past methods?\n\n2. For stronger purifiers such as CLIPure, how effective will the proposed  DiffAdvMAP be to attack them?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UCUhzAEdtq", "reviewer": "Reviewer_Y6Ps", "summary": "This paper proposes DiffAdvMAP, a Bayesian-based framework for generating universal adversarial examples (UAEs) by approximating their posterior distribution. Unlike existing diffusion-based methods, which struggle with low naturalness and limited effectiveness, DiffAdvMAP leverages adversarial and reconstruction constraints to enhance both attack success and adaptability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the reviews and kindly suggestion. According to your opinions, we add experiments on another dataset Celeba-HQ, the experimental results are satisfactory. We wish you can improve the score after reading our response.\\\n1. Dataset Diversity: We use the ImageNet-compatible dataset to evaluate DiffAdvMAP because it contains the most important objects in the real world, and it is widely used in previous works[1][2] as a single dataset. We also evaluate DiffAdvMAP with another subset of ImageNet in Appendix D. DiffAdvMAP is developed based on pre-trained diffusion models, and since it’s hard to obtain open-sourced diffusion models in the area of medical imaging and remote sensing, we supplement experiments on the Celeba-HQ dataset in attacking the face gender classification model. The experimental details are as follows:\\\nWe trained a Resnet18 and a Mobilenet V2 as the gender classifier, and used the Resnet18 as the known surrogate classifier to generate 1000 UAEs, among them, 500 are female faces and 500 are male faces originally. For the evaluation, we propose the clean accuracy of each model, the white-box attack success rate(ASR) towards Resnet18, and transfer ASR towards Mobilenet V2 in the table below.\\\n|Models   | Resnet18| Mobilenet V2|\\\n|Clean Accuracy|98.33%|98.50%|\\\n|ASR|100%|92.3%|\\\nNote that the generation speed is still 6 seconds/image. We think this experiment can show the adaptability of DiffAdvMAP to other datasets to some extent.\\\n[1] Dong Y, Pang T, Su H, et al. Evading defenses to transferable adversarial examples by translation-invariant attacks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 4312-4321.\\\n[2] Gao L, Zhang Q, Song J, et al. Patch-wise attack for fooling deep neural network[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII 16. Springer International Publishing, 2020: 307-322.\n\n2. Optimize efficiency: As introduced in Section 1, diffusion models have a supreme performance in image synthesis. Our DiffAdvMAP method fully utilizes the prior knowledge of real data distribution learned by the diffusion models and manages to generate more effective and natural UAEs than the previous method. We already refined the sampling strategy by leveraging a truncated sampling process. We also propose targeted optimizations on our method to address the inherent generation speed constraints of other diffusion-based methods, keeping a comparable efficiency compared with most attack methods, which is shown in Table 2. Note that the GPU we applied to generate UAEs is RTX 3090 24G. The generation speed can be further improved if we use more advanced hardware. Use a lighter diffusion model to accelerate the generation speed is also included in our future research considerations.\n\n3. Multimodal scenarios: In this paper, we are focusing on generating general UAEs in single-modal cases. Generating UAEs under the multimodal scenarios is an interesting area; we will conduct research in this area in our future work.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "(1) In the Bayesian framework, given the reference image, true label, diffusion model, and target classifier, a mathematical formulation for UAE generation is constructed. The adversarial constraint is derived from the CW attack, and the reconstruction constraint is introduced based on different generation scenarios. The posterior distribution of UAEs is obtained using Bayesian principles, considering the properties of the diffusion model.\n(2) For image-similar UAEs, the posterior distribution's PDF is derived, leading to the objective function through approximation. To balance computational complexity, a one-step estimation method is used, approximating the conditional distribution accordingly. The roles of key parameters, such as standard deviation adjustments, are clarified to ensure the correctness and practicality of the objective function.", "experimental_designs_or_analyses": "(1) Multiple evaluation metrics are employed: FID and LPIPS for global image similarity UAEs, and FID, TRES, and HyperIQA for noise-based UAEs.\n(2) Appropriate baselines are selected for different attack scenarios: AdvDiff for noise-based UAEs and classic unrestricted and diffusion-based attacks for global image similarity UAEs.\n(3) Ablation studies analyze the impact of reconstruction constraints, adversarial constraints, destruction-reconstruction modules, and variations in adversarial confidence levels.", "relation_to_broader_scientific_literature": "Diffusion models excel in image synthesis and time series prediction, but their application in adversarial sample generation often relies on them as priors without fully utilizing their ability to learn real data distributions. DiffAdvMAP integrates adversarial and reconstruction constraints to better leverage learned priors, improving naturalness, attack success, and flexibility, expanding diffusion models’ role in adversarial attack research.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "(1) DiffAdvMAP is primarily evaluated on ImageNet-compatible datasets, but its adaptability to specialized domains like medical imaging and remote sensing remains unexplored.\n(2) While the method improves generation speed, it still relies on complex diffusion models and multiple iterations, leading to high computational costs. This may hinder real-time attacks or large-scale data processing.", "other_comments_or_suggestions": "(1) Expand experiments by incorporating diverse datasets and industrial inspection images, to evaluate DiffAdvMAP’s adaptability. Investigate framework adjustments for specialized data and explore adversarial sample generation in multimodal scenarios.\n(2) Optimize efficiency by improving diffusion model structures, refining sampling strategies, leveraging hardware acceleration, and exploring model compression and quantization to reduce resource consumption.", "questions_for_authors": "(1) Expand experiments by incorporating diverse datasets and industrial inspection images, to evaluate DiffAdvMAP’s adaptability. Investigate framework adjustments for specialized data and explore adversarial sample generation in multimodal scenarios.\n(2) Optimize efficiency by improving diffusion model structures, refining sampling strategies, leveraging hardware acceleration, and exploring model compression and quantization to reduce resource consumption.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No."}}, {"review_id": "6FsBIQuqvj", "reviewer": "Reviewer_nVug", "summary": "This paper introduces Diffusion-based Adversarial Maximum a Posteriori (DiffAdvMAP), a framework that generates unrestricted UAEs by sampling from posterior distributions. It leverages a diffusion model to approximate the prior distribution of real data and incorporates adversarial and reconstruction constraints within a Bayesian framework to enhance both effectiveness and realism. Experimental results on ImageNet demonstrate that DiffAdvMAP achieves a superior trade-off between image quality, flexibility, and transferability compared to existing unrestricted adversarial attack methods.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are sorry that maybe because our writing is not clear enough, you have some misunderstandings about the innovation of our paper. Our responses are as follows, and we hope you can improve the score after reading our responses and paper carefully.\n1. DiffAdvMAP is a flexible framework that can be adapted to different attacking scenarios by sampling UAEs from their approximated posterior distribution under the adversarial constraint and reconstruction constraint. We make specific adaptations to each attacking scenario.\\\n(1) Technical $\\textbf{Difference}$ between generating UAEs from noises and image-similar UAEs:  When generating UAEs from noise, we generate most low-level semantics via the generation process and then generate high-level adversarial features; when generating image-similar UAEs, we destroy the high-level features of the input clean image with several diffusion steps, and regenerate high-level adversarial features, as is shown in Alg.1 in Appendix C. Besides, we leverage different diffusion models for such two tasks: the latent diffusion model and an unconditional diffusion model, we also build different objective functions for each task: we introduce the reconstruction constraint based on the reference image in the task of generating image-similar UAEs, while in the task of generating UAEs from noise, we replace the reconstruction constraint with class condition y, as is introduced in Section 3 and Appendix B. The experimental results in Section 4 also show that DiffAdvMAP can generate UAEs from noise and generate image-similar UAEs. \\\n(2) DiffAdvMAP achieves $\\textbf{innovations}$ in generating regional and customized unrestricted adversarial examples (UAEs): For regional UAEs, it abandons the traditional additive perturbation-based approach applied to masked areas. Instead, it destroys mask-covered regions and reconstructs adversarial features through posterior distribution sampling under dual constraints (adversarial and reconstruction), enabling generative reconstruction of adversarial patterns. For customized UAEs, it embeds specific reconstruction constraints (e.g., style/color consistency) into the posterior probability density function, replacing explicit style/color loss functions. This directly guides probability distribution shaping via reference images to ensure precise semantic alignment.\n2. Table title is $\\textbf{aligned}$ to section content: In section 4, we conduct experiments under the transfer-based black-box setting, the title of Table 2 ‘The white-box attack success rate(%), transfer attack success rate (%), image quality metrics, as well as the run time(sec) of DiffAdvMAP and baseline methods in the task of generating global image-similar UAEs’ means the white-box attack success rate(ASR) of attacking white-box known surrogate models, and transfer ASR of attacking black-box unkown classifiers is included in the table. When performing a transfer-based black-box attack, UAEs are generated with a known surrogate model, eg, Inception V3, then these UAEs are used to attack other unknown classifiers, eg, Mobilenet V2, Resnet50, and Swin-Transformer, and obtain the transfer attack success rate[1]. As a result, the evaluation is conducted under a black-box setting, and the content and the title of the tables in Section 4 are consistent with our claims.\\\n[1] Chen Y, Liu W. A theory of transfer-based black-box attacks: Explanation and implications[J]. Advances in Neural Information Processing Systems, 2023, 36: 13887-13907.\n3. Advdiff and DiffAttack are $\\textbf{different}$: According to the paper and official code of AdvDiff and DiffAttack, AdvDiff leverages the latent diffusion model to generate UAEs from noise, its inputs are random Gaussian noise images, it designs two adversarial guidance techniques to conduct adversarial sampling in the reverse generation process of diffusion models; DiffAttack leverages the stable diffusion model to generate image-similar UAEs, its inputs are clean images, it computes the cross attention map and self attention map of each latent code as the objective fucntion and optimizes the objective function during the generation. As a result, though AdvDiff and DiffAttack are all diffusion-based methods, the principles and corresponding tasks are different, so we compare the performance of our method in different tasks with them in different sections.\n4. Include important paper: Since the authors of AdvDiffuser did’t release their code(though there’s a github repository proposed in the paper, the repository is empty), and it conducts the experiments under the white-box setting, which is different from the setting in our main paper, we have compared DiffAdvMAP with AdvDiffuser in Appendix D under the white-box setting, the evaluation metrics are all from their paper, please refer to it for more details, it’s obvious that the efficiency, effectiveness and naturalness of UAEs generated by our method is better than AdvDiffuser.", "claims_and_evidence": "The claimed technical contributions are overstated. The authors assert that their framework can generate UAEs under various attack conditions, including noise-based and global image-similar UAEs. However, as shown in Fig. 1, generating UAEs from noise simply involves using a pretrained diffusion model to produce a source image, then applying adversarial loss to an intermediate result—making it technically no different from global image-similar UAEs generation. Similarly, regional customized UAEs generation merely applies a mask and/or style loss to the source image, which is not a novel approach.", "methods_and_evaluation_criteria": "In Section 4, the authors claim that the evaluation is conducted under a black-box setting, which does not align with the titles of the subsequent tables.\n\nFurthermore, the classification of UAE generation types (Sections 4.2 and 4.3) appears inconsistent with prior works, where AdvDiff and DiffAttack are typically considered the same type of method. Additionally, the authors fail to include AdvDiffuser [1], an important related work.\n\n[1]AdvDiffuser: Natural Adversarial Example Synthesis with Diffusion Models", "theoretical_claims": "n/a", "experimental_designs_or_analyses": "n/a", "relation_to_broader_scientific_literature": "n/a", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "n/a", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zzAOK27ztQ", "reviewer": "Reviewer_4Qqs", "summary": "In this paper, the authors propose a flexible framework named DiffAdvMAP to facilitate the effective and natural generation of unrestricted adversarial examples (UAEs). The framework is based on the posterior distribution with two constraints of adversary and reconstruction, which is eventually optimized with a gradient method. Experiments validate the effectiveness, efficiency and naturalness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your reviews and kindly reminder. According to your opinions, we add the theoretical explanation of the effetviness of DiffAdvMAP, and explain the reason for the difference of transfer ASRs in Sec. 4.2 and Sec. 4.3, the responses are as follows. We wish you can improve the score after reading them.\n1. The essential factors that make the method more effective: As is illustrated in Section 3, we introduce an adversarial constraint to ensure the effectiveness of UAEs and a reconstruction constraint to control the content of generated UAEs, such two constraints are considered as two events of the posterior distribution of UAEs. Then the posterior distribution is approximated with the prior knowledge of real data learned by the diffusion model, unlike previous methods which only use diffusion models as strong denoisers to enhance the generation process, DiffAdvMAP generates adversarial features by sampling from the approximated posterior distribution, which make the adversarial features more coherent to the original features, thus improving the naturalness of UAEs. As a result, we have more space to improve effectiveness at the cost of sacrificing naturalness. Besides, in order to derive the probability density function(PDF) of the posterior distribution, the events should be deterministic events that are expressed as equations. In this situation, the objective function of CW attack can be converted into the form of an equation, on the right side of the equation is the adversarial confidence level $c$, once $c\\le 0$, the adversarial example can attack the classifier successfully, as for the likelihood function, there’s no exact range indicates that the attack will succeed. As a result, we can control the strength of UAEs more effectively by controlling the value of confidence level c, such that we can achieve a better trade-off between the effectiveness and naturalness of UAEs. As is illustrated in Table 6 in Appendix E, the white-box attack success rate is nearly 100%, but the transfer ASR is not enough when the absolute value of the confidence level is small. When the absolute value of the confidence level increases, the transferability increases, and although the naturalness decreases, it is still better than other baselines.\n2. The reason for transfer ASRs in Sec. 4.2 are different from those in Sec. 4.3: On the one hand, experiments in Section 4.2 and Section 4.3 undertake different tasks with different models: experiments in Section 4.2 generate UAEs from noise with the latent diffusion model[1] and experiments in Section 4.3 generate image-similar UAEs with real images as reference. The input and the generated UAEs are completely different. On the other hand, since there’s no real image for reference in the task of generating UAEs from noise, we employ a strong class guidance to ensure the naturalness of generated UAEs, then it becomes harder for DiffAdvMAP to counteract the strong class guidance to generate adversarial features, thus the effectiveness of generated UAEs decreases when maintianing their naturalness, which is reflected by the transfer attack success rate. \n3. Including important paper: Since the authors of AdvDiffuser don’t release their code(though there’s a github repository proposed in the paper, the repository is empty), and it conducts the experiments under the white-box setting, which is different from the setting in our main paper, we have compared DiffAdvMAP with AdvDiffuser in Appendix D under the white-box setting, the evaluation metrics are all from their paper, please refer to it for more details, it’s obvious that the efficiency, effectiveness and naturalness of UAEs generated by our method is better than AdvDiffuser.\n4. Typos and suggestions: We are sorry we made such typos, the denominator of equaiton(20) should be $p(C_1|x_t,y)$, but it has no impact on the subsequent derivation, we will correct the typos here and in line 36, increase the resolution of Figure in the revised version.", "claims_and_evidence": "Yes. The writing is clear.", "methods_and_evaluation_criteria": "The two constraints adopted in the framework are well-motivated and straightforward to achieve high ASRs and flexibility under different settings. Evaluation on NIPS2017 with diverse metrics like FID, LPIPS in addition to ASR is acceptable.", "theoretical_claims": "The formulation and logic of the theoretical claims seems reasonable.\n\nOne minor question is about the last line of Eq. (20). The equality holds when $p(y|\\hat{x}_t)p(C_1) = p(C_1,y|\\hat{x}_t)$, which is only correct when $C_1$ is independent of $y$ and $\\hat{x}_t$. This is clearly wrong given the definition of $C_1$.", "experimental_designs_or_analyses": "The experiments are extensive with different settings and baselines. Ablations of important modules are conducted. Results are analyzed with sufficient visualizations.", "relation_to_broader_scientific_literature": "This paper provides a unified and flexible theoretical framework for the generation of UAEs with diffusion models.", "essential_references_not_discussed": "AdvDiffuser published in ICCV2023 is suggested to be included as a baseline.", "other_strengths_and_weaknesses": "Strength:\n* The paper formulates the generation of UAEs from a Bayesian perspective, which is interesting.\n* The method is well-motivated to combine two constraints into the framework.\n* The experiments are extensive with sufficient analyses.\n\nWeakness:\n* The method eventually ends up with a loss function with three regularization terms, which is optimized simply with gradient methods. For the term corresponding to the adversarial constraint, it's different from previous methods only because of the utilization of CW attack, rather than original likelihood maximization. There should be more discussions about the practical difference between the proposed method and previous methods and the essential factors that make the method more effective. \n* There is a phenomenon that the transfer ASRs for Sec. 4.2 are lower in general than those in Sec. 4.3. Is there any explanation?", "other_comments_or_suggestions": "* In line 36, a string of \"0\" is injected to the main text.\n* The resolution of Figure 1 should be higher.", "questions_for_authors": "See comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a Bayesian framework to generate unrestricted adversarial examples (UAEs) by sampling from posterior distributions using a diffusion model as a prior. The method integrates adversarial and reconstruction constraints to balance attack effectiveness, image naturalness, and flexibility across diverse attack scenarios (e.g., noise-based or style-transfer). Experiments on ImageNet show superior performance compared to existing methods like AdvDiff. The work received comments from four reviewers. Three reviewers tend to accept the work due to the novel Bayesian framework with Diffusion models for UAE and a comprehensive evaluation. The positive reviewers are also concerned about the computational cost and the lack of theoretical analysis. After the rebuttal, the main concerns are addressed. One reviewer rates the rejection at the first round due to the overstated novelty and omitted baselines. After the rebuttal, the reviewer acknowledges the changes and raises the score to weak reject. The remaining concern is the necessity of separating the generation of UAEs from noise-based and image-similar cases. Upon my own review of the paper, I understand this concern does not significantly detract from the work's originality. Using a single method to address both scenarios offers the practical advantage of reducing the overhead that would come with implementing two independent models. Due to the above facts, I recommend accepting this work.
Decision: Accept (poster)