Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspective
Authors: Andi Zhang
Mingtian Zhang
Damon Wischik
Venue: NeurIPS 2024 poster (2024)
Keywords: Adversarial Examples
Probabilistic Generative Models
Diffusion Models
Energy-based Models
Primary area: safety_in_machine_learning

Abstract: We propose a probabilistic perspective on adversarial examples, allowing us to embed subjective understanding of semantics as a distribution into the process of generating adversarial examples, in a principled manner. Despite significant pixel-level modifications compared to traditional adversarial attacks, our method preserves the overall semantics of the image, making the changes difficult for humans to detect. This extensive pixel-level modification enhances our method's ability to deceive classifiers designed to defend against adversarial attacks. Our empirical findings indicate that the proposed methods achieve higher success rates in circumventing adversarial defense mechanisms, while remaining difficult for human observers to detect.

Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspective
Andi Zhang*
Mingtian Zhang Centre for Artificial Intelligence University College London
Computer Laboratory University of Cambridge az381 cell@eaattb..... ac. uk
zwang@cs....oooo ucl.as ac. .uk
Damon Wischik
Computer Laboratory University of Cambridge djw10050cam. ac. uk
Abstract
We propose probabilistic perspective adversarial examples, allowing embed subjective understanding of semantics as a distribution into the process generating adversarial examples, in principled manner. Despite significant pixel-level modifications compared to traditional adversarial attacks, our method preserves the overall semantics of the image, making the changes difficult for humans to detect. This extensive pixel-level modification enhances our method's s ability deceive classifiers designed to defend against adversarial attacks. Our empirical findings indicate that the proposed methods achieve higher success rates in circumventing adversarial defense mechanisms, while remaining difficult for human observers to detect. Code can be found at htthhtt::/// / htt://gittin githuu...... com/ andiac/
AdvPP
Introduction
The purpose of generating adversarial examples is deceive classifier (which refer to as
victim classifier) by making minimal changes to the original data's semantic meaning. In image
classification, most existing adversarial techniques ensure the preservation of adversarial example semantics by limiting their geometric distance (C, distance) from the original image (39 14] 44] 26). While these methods deceive classifiers using minimal geometric perturbations, they as successful in black-box attack scenarios. Furthermore, the recent surge in adversarial defense methods (26] 33 primarily targets geometric-basee attacks, gradually reducing their effectiveness. In response to these challenges, unrestricted adversarial attacks are gaining traction as potential solution. These methods employ more natural alterations, moving away from the small Cp norm perturbations typical of traditional approaches. This shift towards unrestricted modifications offers more practical approach adversarial attacks.
In this paper, we introduce a probabilistic perspective for adversarial examples. Through this innova- tive lens, both the victim classifier and geometric constraints regarded as distinct distributions: the victim distribution and the distance distribution. Adversarial examples naturally arise as samples from the product of these distributions.
This probabilistic perspective offers an opportunity to transform traditional geometric-based con- straints into semantic constraints. Traditional geometric constraints, when viewed through this
"Conesponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Tomster
Caim Terrirr
0
1
3
H
S
T
Figure 1: Adversarial examples generated by our method. Left: MNIST examples where we injected the subjective semantic understanding that scaling, translation, and distortion preserve digit meaning. The adversarial examples maintain digit interpretability while applying these transformations (sce Figure 33 for comparison with other methods). Right: Adversarial example of hamster image, lever- aging semantic knowledge from pre-trained diffusion models. Despite substantial pixel modifications, the image remains natural- natural-looking (see Figure 4 for comparison with other methods).
probabilistic lens, manifest as simple distance distributions for instance, the squared L2 norm constraint naturally maps to a Gaussian distribution centered at the original image. This reveals that conventional C2 squared constraints implicitly define semantic similarity through Gaussian distribu- tion. Leveraging recent advances in probabilistic generative models (PGMs), we propose replacing this Gaussian distribution (distance distribution) with fitted PGM. This substitution introduces data-driven approach to defining semantic similarity, effectively transforming geometric constraints into semantic ones. To demonstrate the practical implementation of our conceptual framework, we present two approaches for constructing PGMs that capture semantic similarity:
Our first method injects subjective semantic understanding by defining semantic-presse preserving transformations for the original image. We train a PGM model the distribution of these transformed images, thereby learning the manifold of semantically equivalent variations.
Our second method leverages the semantic knowledge embedded in pre-trained PGMs. By fine-tuning PGM the original image, we create localized distribution that capture image-sp specific semantic variations, representing the semantic distance distribution around the original image.
These approaches can also be combined in practice. By employing appropriate PGMs distance distributions, our method generates adversarial examples that appear more natural despite substantial geometric modifications (Figure I. These adversarial examples demonstrate improved transferability in black-box scenarios and higher success rates against adversarial defenses.
Preliminaries
In this section, we present essential concepts related to adversarial attacks, energy-based models, and diffusion models. For detailed information the training and sampling processes of these models, please refer to Appendix B
2.1 Adversarial Examples
The notion of adversarial examples was first introduced by [39]. Let's assume we have classifier C:: 1]" + V, where represents the dimension of the input space and y denotes the label space. Given an image Zoni E [0, 1]" and target label Yuar E V, the optimization problem for finding an adversarial instance Tadv for Zori can be formulated as follows:
min ( Faav)) s.t. C(raav) = Yhar and Tady E [0, 1]"
Here, D is distance metric employed to assess the difference between the original and perturbed images. This distance metric typically relies on geometric distance, which can represented by Li, C2, or Loe norms.

---

However, solving this problem is challenging. (T Vum) 6j (z Tndo) + 02 (Fadv Yur) the optimization problem
339 propose relaxation of the problem: Let
min L(radv Mur) s.t. Tadv E [0, 1]"
(1)
where c1, 02 are constants, and f is an objective function closely tied to the classifie''s 's prediction. For example, in [39], f is the cross-entropy loss function, indicating misclassified direction of the classifier, while [ suggest several different choices for f. [39] recommend solving (T) using
box-constrained L-BFGS.
2.2 Energy-Based Models (EBMs)
involves a non-linear regression function, represented by Eg. with parameter 0. This function is known as the energy function. Given data point, 2, the probability density function (PDF) is given by:
An Ehergy-based Model (EBM)
pa(x) exp(- exp(-Eg(x))
(2)
Za
where Zg = (xp((EE(())) is the normalizing constant that ensures the PDF integrates to 1.
2.3
Diffusion Models
Starting with data zo. we define diffusion process (also known as the forward process) using specific variance schedule denoted by $1,..., ,3.- This process is mathematically represented as:
T
II ((x1)xt-1)
1=1
where each step is defined by
(( = (
: (x / 1- Br x1-1,
where N is the pdf of Gaussian distributions. In this formula, the variance schedule 3r E (0, 1) is selected ensure that the distribution of xT standard normal distribution, g(xT) [xz 0, f).
For convenience, we also define the notation a; := 1- for each 1, and ā = II. ag. By using the property of Gaussian distribution, we have
(<<a|=a) = Mxxs Vā, zo, (1 (1- (1-a:)1)
The reverse process, known the denoising process and denoted by g(<;-1|==), cannot be analytically derived. Thus, we use a parametric model, represented as po (2 = N(xx-1; un (zt, t), Es(x,, t)), to estimate g(xx-1/2)) In practice, (40 and Eg are implemented using a UNet architecture [31] which takes as input a noisy image at its corresponding timestep. For simplicity, within the rest of this paper, we will abbreviate (1 and En as (xx)) and En (xx)) respectively.
3 A Probabilistic Perspective on Adversarial Examples
We propose probabilistic perspective where adversarial examples sampled from adversar- ial distribution, denoted as Padv- This distribution can be conceptualized as a product of expert distributions [17
Padv (Fadv Fari Mar) xx Pvie (aado (Bar) Pais (Fady (Frr))
where Pvic is defined the 'victim distribution", which is based on the victim classifier and the target class Yar- Pais, on the other hand, denotes the distance distribution, where high value of pas indicates significant similarity between Fadv and Zori-
The subsequent theorem demonstrates the compatibility of our probabilistic approach with the conventional optimization problem for generating adversarial examples:
Theorem 1. Given the condition that (ndd Murr) x exp(-c2 I (Tades (Bar)) and paie paic (Fade Fori) ox exp(-cı D(ran, Fads)) the samples drawn from Padv will exhibit the same distribution the adver- sarial examples derived from applying the box-constrained Langevin Monte Carlo method to the optimization problem delineated in equation 7
3

---

x (-Baa))
(Bic)
x ~ Pais Pais(11zuu (-
2 Padte (-Izuri Mlxx)
w.o. adv. train
with adv. train
Pais is Gaussian
t(z) for ~ T
Puis is Gaussian
2 6
4
7
2
b
4
7
2
/
4
9
5 1 4
9 5
0 9 4 0
6
4
9
4
0
6
7
A
9
7
9
0 4 O
5
9
4
0
9 C 0
y g 1
3
9
6 4
5
5
3
7
8
4
0
7
4
7
4
D
+
3
4
244
3
3
4
7
2
1 3
7
7
A
7
.2
(a)
(b)
(c)
(d)
(c)
Figure 2: (a) and (b) display samples drawn from Pvic(- with the victim classifier being non- adversarially trained and adversarially trained, respectively. (c) showcases samples from Pais ( ( when D is the square of L2 norm. (d) illustrates (( for f ~ T, where T represents distribution of transformations, including TPS (see Appendix D.I.. scaling, rotation, and cropping. (e) Samples from Paty (oa, Mar) x exp(-c: D(Fon Tadv )) exp( C2 [T I,r)), where 'D is the L2 norm, is the cross-entropy fCE, Zori are the first 36 images from the MNIST test set, Bhar are set to 1, e₁ is 10-$, and is 10 A green border marks a successful attack, while red denotes failure.
The proof of the theorem can be found in Appendix A Within the context of our discussion, we initially define Pvic and pais to have the same form as described in the theorem. Given this formulation, we can conveniently generate samples from Padv. Pdis- and Poie using LMC. Detailed procedures are provided in Section As we delve further into this paper, we may explore alternative formulations for these components.
The Victim Distribution Pric is dependent on the victim classifier. As suggested by 39 f could be the cross-entropy loss of the classifier. We can sample from this distribution using Langevin dynamics. Figure presents samples drawn from Pvic when the victim classifier is subjected to standard training. exhibiting somewhat indistinct shapes of the digits. This implies that the classifier has learned the semantics of the digits to certain degree, but not thoroughly. In contrast, Figure (b) displays samples drawn from Pvic when the victim classifier undergoes adversarial training. scenario, the shapes of the digits are clearly discernible. This observation suggests that we can obtain meaningful samples from adversarially trained classifiers, indicating that such classifiers depend more on semantics, which corresponds to the fact that an adversarially trained classifier is more difficult to attack. similar observation concerning the generation of images from an adversarially trained classifier has been reported by 33 55).
this
The Distance Distribution Pdis relies D( Tadv), representing the distance between Tadv and Ton- By its nature, samples that are closer to Zoni may yield a higher Pais, which is consistent with the objective of generating adversarial samples. For example, if D represents the square of the La norm, then Pas becomes a Gaussian distribution with a mean of Fon and a variance determined by c1. Figure (c) portrays samples drawn from Pais when D is the square of the L2 distance and eı is relatively large. The samples closely resemble the original images, Tons. This is attributed to the fact that each sample converges near the Gaussian distribution's 's mean, which corresponds the ToriS.
The Product of the Distributions Samples drawn from Padv tend be concentrated in the regions of high density resulting from the product of Prie and Pdis- As is discussed, a robust victim classifier possesses generative capabilities. This means the high-density regions of pvic are inclined generate images that embody the semantics of the target class. Conversely, the dense regions of Pais tend to produce images refiecting the semantics of the original image. If these high-density regions of both Pvic and Pas intersect, then samples from Pady may encapsulate the semantics of both the target class and the original image. As depicted in Figure[ the generated samples exhibit traces of both the target class and the original image. From our probabilistic perspective, the tendency of the generated adversarial samples to semantically resemble the target class stems from the generative ability of the victim distribution. Therefore, it is crucial to construct appropriate Pas and pvic distributions to minimize any unnatural overlap between the original image and the target class. We will focus on achieving this throughout the remainder of this paper.

---

4 Semantics-aware Adversarial Examples
Under the probabilistic perspective, the distance distribution Pdis is not necessarily based on a explicitly defined distance D. Instead, the primary role of Pais ensure that samples generated from (a closely resemble the original data point Zori- With this concept in mind, we gain the flexibility define pdis in various ways. In this work, we construct pais using probabilistic generative model (PGM). By utilizing this data-driven distance distribution and choosing proper victim distribution, we can generate adversarial examples that exhibit more natural transformations in terms of semantics. These are referred to as semantics-aware adversarial examples. Moving forward, given that the distance 'D implicitly learned within Pais, we set C1 for simplicity, and henceforth, use to denote C2.
4.1 Data-driven Distance Distributions
We present two methods to develop the distribution pais(-[Zor) centered on Zorii The first relies on a subjective understanding semantic similarity, while the second leverages the semantic generalization capabilities of contemporary PGMs.
Cemantics-Invarinn Data Augmentation Consider T, set of transformations subjectively believe to maintain the semantics of Zori- We train a PGM on the dataset {t, (Zon) tt (Zon)....) }. where each is sample from T, thereby shaping the distribution pdis- Through T. individuals can incorporate their personal interpretation of semantics into pdis- For instance, one considers that scaling, rotation and TPS distortion (Appendix D.I. do not alter image's semantics, these transformations are included in T.
Fine- Mone-Tuniggg Pretrained PGMs Contemporary PGMs demonstrate remarkable semantic general- ization capabilities when fine-tuned on a single object or image [32 20). Leveraging this trait, we propose fine-tuning the PGM on the given image Toni The distribution of the fine-tuned model then closely aligns with Ton, while still facilitating robust semantic generalization.
4.2 Victim Distributions
The victim distribution Pvic x exp(c f(advv )urr) is influenced by the choice of function f. Let 96 [0. 1]" + 31 be a classifier that produces logits as output with Φ representing the neural network parameters, n denoting the dimensions of the input, and y being the set of labels (the output of 9 are logits). [39] suggested using cross-entropy as the function f. which can be expressed as
fCE( Vaar) (x) + log Σ exp(go(x)lv) log a (g)
where a denotes the softmax function.
4 explored and compared multiple options for f. They found that, empirically, the most efficient choice of their proposed fs is:
fcw(x, Vaar) == max max go (x) (xx [Btar) 0).
the
In this study, we employ few for the MNIST dataset and fce for the ImageNet dataset. A detailed discussion on this choice provided in Section
5 Concrete PGM Implementations
Fundamentally, any probabilistic generative model (PGM) is capable of fitting the distance dis- tribution Pais- However, for efficient sampling from Patv, which the multiplication of Pvic and Pais as introduced in (4) we recommend employing sampling techniques based on the score s log Padv (Fadv (ori, M. Energy-based models and diffusion models are particularly effec- tive in providing these scores. Therefore, in this study, we utilize these models to represent pais-
5

---

5.1 Generating Adversarial Examples Using Energy-Based Models
The distance distribution Pais (- For) can be modeled using mnergy-based models (EBMs). For a given Pori we train or fine-tune an EBM in the vicinity of Ton to represent this distance distribution. Let Eg denote the energy in the EBM. Consequently, the adversarial distribution is expressed as:
Padv (Fady (Toris Yurr) (fre (x(xa))
and the corresponding score is:
Raay PagPadv Padv (Fadv [Tori Ytar) - cV man (Tadv; Mtar) saa (a(da)))
Utilizing Langevin dynamics (Appendix B.1 1) we can sample from this adversarial distribution. The process is detailed in Algorithm (Appendix
5.2 Generating Adversarial Examples Using Diffusion Models
To enhance generation quality and enable the creation of higher resolution images, we frame the con- struction of the adversarial distribution within diffusion model context. Given original image Zori and target class Mur. we employ a diffusion model po,,, ( := (x1-111 Me (Iz),, Ee,, (2%) )) time step 1, where fei denotes the parameters of the model when trained fine-tuned on Tori This diffusion model is used to approximate Pais (-[Zon). For simplicity, we will refer to fon as throughout this paper, assuming no confusion arises from this notation. Then, letting xo Fadv. the adversarial distribution is formulated as:
Pady ( Mori Mtar) x (ri yoar Pais (Fo (on) = ( I p((r))
II (ate (2.)der.... t=1
where p(aT) is N(0, I) and the victim distribution pric (1 (ar) x exp(-ee(xs Yor)). However, sampling To in this form is challenging in the denoising order of diffusion models. Therefore, we incorporate the Pvie term within the product:
f p(oT)) II t=
Pric Pvie )/r" (o
As each denoising step cannot predict zo when sampling It-1, we employ Tweedie's approach 12
2 16 to estimate zo given Zz:
Zole
Vor (z) -I ãe cg(xz))
(5)
with ea (z;) obtainable through the reparametrization trick from 18
v1 (xr ao po (2x))
c0(x))
(6)
3,
This leads practical expression for the adversarial distribution:
f p(xT) II Pvic Pric (BojeeMtar) //T pe (1x
In each denoising step, we sample xz-1 from the distribution Pvic (aojel (x.-1(xx)) The following theorem demonstrates that this distribution approximates Gaussian distribution:
Theorem 2. Let pric(xl exp(-cf(a (x, Ytar)) and po (0 Eo(x:)), have
Pric Mear ) L/T po (0( N(xx-1: + - (oo(x))),Eo()))
where g -V v. ( Ytar lx.. mo(zx) and Zojt is defined in Equation
Building upon Theorem (Zoje: Mhat 1- in line with the assumption made by (10), we introduce A Algorithm 2 (Appendix C). This algorithm designed to sample from the adversarial distribution Pady formulated within the context of diffusion models.
For the proof, refer to Appendix V., ((ãot:: Yar) =
and assuming

---

PGD (Za) Target
Prub
StAdy Target
proposed
Target
Targer
0
0
G
0
O
A O 1
O
0
Q
C
?
4
2
E2
OO
O
O
O
0
D
D
2
2
2
/
C
2
2
Z
2
2
2 Z
2
2
2
2
2
2
2
3
103380 8 9
2 5
8 9
3
A
5
3 3
8
3
3
7
Souce
0
9
E
8
Y F 9
? 9
AV V A
5
5
2
S
5 S
SS
5 5
0
5
5/
6
G
S 8 6
6
6
& 6
6
S
6
C G C
C
G
7
7
7
7 7
7
7
7
7
7 T
T
T
0
3
R
8
6
E
Figure 3: Comparative visual analysis of PGD, Prob CW, StAdv, and our proposed method applied to MNIST. The surrogate classifier used is MadryNet with adversarial training. Images are framed with green border to indicate successful white-box attack, whereas a red border signifies failed attack.
6 Experiments
6.1 MNIST
Setting We use an energy-based model (EBM) to model the distance distribution Pus (-(zoo)))) for a given original image Vori This EBM is specifically trained as {41 (Fori). (. /2 (2 }. where each I represents a sample from the transformation distribution T. This distribution includes variety of transformations such as translations, rotations, and TPS (Thin Plate Spline) transformations (Appendix D.I). Examples of these transformed MNIST images are showcased in Figure (d). To produce high-quality adversarial examples for MNIST, we employ rejection sampling and sample refinement techniques, as detailed in Appendix D
of transformations of wori denoted
For the victim distribution Pvic- choose the adversarially trained Madrynet as our victim (surrogate) classifier. We use few to represent the function I in the victim distri- bution, detailed in Section 4.2
Table 1: Success rate (%) of the methods on MNIST.
PGD
ProbCW
stAdv
OURS
Human Anno.
88.4
89.3
90.1
92.6
White-box Madry Adv
25,3
30.2
29.4
100.0
We benehmark our method against several approaches: PGD (26), ProbCW (which em- ploys a Gaussian distribution for pdis and few for Pvie), and stAdy [46).
Transferability
Madry Net moAdy
15.1
17.4
16.3
61.4
Resnet noAdy
10.2
10.9
12.5
24.3
Adv. Defence
Resnet Adv
7.2
8.8
11.5
18.5
Certified Del
10.7
12.3
20.8
39.2
Quantitative result We select 20 images from each class in the MNIST test set as the original images. For each image, we generate one adversarial example for each target class, excluding the image's ' true class. This yields a total of 20 x 10 x 9 = 1800 adversarial examples for each method. The parameters of each method adjusted ensure approximately 90% of the adversarial examples accurately reflect the original concept of Zon- The effectiveness of our adversarial examples evaluated against the adversarially trained Madrynet under white-box conditions, with results displayed in the 'MadryNet Adv' row of Table Additionally, we task 5 human annotators with classifying these adversarial examples, considering an example to be successfully deceptive if the annotator identifies its original class. The annotators' success rates are shown in the 'Human Anno.' row of Table [ We also assess the transferability and the success rate of the examples against defensive methods, with these outcomes detailed in Table I Notably, the term 'Certified Def denotes the defense method introduced by (45..
Table | demonstrates that our proposed method achieves higher success rate in white-box scenarios and greater transferability other classifiers and defense methods, all while preserving the meaning of the original image. The white-bor box success rate of our method reaches 100% due to the implementation of rejection sampling, as introduced in Appendix D.2
Qualitative result Unlike the quantitative experiment, here we adjust the parameters so that the vast majority of examples can just barely deceive the classifier. The adversarial examples thus generated
7

---

OUES
Gull
OURS
Figure 4: Comparative visual analysis of NCF, cAdv, ACE, ColorFool and proposed method applied to Imagenet. The surrogate classifier used is ResNet50. For additional examples, refer to Appendix
Table Success rate (%) of the methods on Imagenet.
NCF
ColorFool
OURS
OURS 10)
20)
Humna Anno.
2.6
16.9
11.2
6.7
31.2
28.3
25.4
White-bux
Transferability
VGG19
83.7
73.5
72.8
70.8
ResNet 152
71.8
61.1
55.4
54.9
57.3
64.2
67.0
DenseNet
Inception V3
72.4
60.1
57.5
57.4
58.1
64.1
66.9
EfficientNet
72.9
56.3
62.6
60.4
61.6
66.0
Adversarial Defence
Inception V3
EfficientNee B7 Ads
50.3
42.9
34.7
36.1
37.7
40.4
44.2
Ensemble IncRes V2
53.3
36.6
35.6
39.7
42.7
Average
66.2
55.2
49.9
50.3
51.6
55.9
59.6
are displayed in Figure 3 From this figure, it evident that the PGD method significantly alters the original image's meaning, indicating inability preserve the original content. ProbCW and StAdv perform somewhat better, yet they falter, especially when '0' and "I' are the original digits: for '0', the roundness is compromised; for 'I', most adversarial examples take on the form of the target class. Furthermore, ProbCW examples exhibit noticeable overlapping shadows, and the StAdy samples clearly show signs of tampering. In contrast, our method maintains the integrity of the original image's 's meaning the most effectively.
6.2 Imagenet
Setting We employ diffusion model that has been fine-tuned on Voi to approximate the distance distribution Pas (+] Specifically, start with pre-trained diffusion model po(x (2x), and then we finettune on a given Poiis as introduced in section 4.I For the victim distribution, we choose ResNet50 as the surrogate classifier and utilize fcr. the cross-entropy function for S.
To evaluate our method's performance, we compare with several existing approaches: ACE (54)
ColorFool 35 cAdv 2 and NCF [51). Our method is evaluated across three hyperparameter

---

configurations: 5, = 10, and c = 20. We test the transferability of these methods on Inception
V3 [40,, EfficientNet B7 41, VGG19 [36], Resnet 152 [T]] and DenseNet 161 (2T. We also list the attack success rate on the adversarial defence methods such as adversarially trained Inception
V3 2 adversarially trained EfficientNet 47 and ensemble adversarial Inception ResNet v2
42
Quantitative Results We randomly select 1,000 non-human images from the ImageNet dataset to serve as original images Zoris ad- hering to the ethical guidelines outlined in Section [.4, For each method, we then gen- erate one untargeted adversarial example per original image. As with the MNIST exper- iment, Table 2 presents the quantitative re- sults for the ImageNet dataset. In this context, human annotators were presented with pairs consisting of an original image and its corre- sponding adversarial example and were asked to identify the case is considered successful if the annotator mistakenly identifies the original image as the manipulated one. Therefore, 'Human Anno.'
75
Arg. Atk. Success Rate (%)
65
NCF
20
55
ColorFool
ACE
45
24
40
Human Annotation Success Rate (%)
photo. A
Figure Average attack success rate across the blackbox transferability and defence methods v.s. hu- man annotation success rate illustrated in Table 2
success rate around 50% suggests that the adversarial examples are indistinguishable from the original images by human observers.
The data in Table places our proposed method second in terms of transferability across different classifiers and defense methods. Note that the 'Average' line is the average of the transferability lines and the adversarial defence lines. Drawing on data from Table [ Figure S]s plotted, illustrating that proposed method not only secures relatively high attack success rate but also remains minimally detectable to human observers. It's important to mention that while NCF achieves the highest attack success rate in many instances, it is also easily detectable by humans. This observation is supported by the human annotation success rate and further evidenced by our qualitative comparison in Figure 4
Qualitative Result Figure displays adversarial examples generated by our method compared with those from alternative methods, under the parameters used in the quantitative analysis. The images reveal that other methods tend to produce significant color changes to the original image, rendering the alterations easily recognizable by humans. This observation corroborated by the 'Human Amno.' row in Table Mcanwhile, adversarial examples from our method are more subtle and the alterations are less detectable by humans.
Related Work
The term 'unrestricted adversarial attack' refers adversarial attacks that are not confined by geometrical constraints. Unlike traditional attacks that focus minimal perturbations within strict geometric framework, unrestricted attacks often induce significant changes geometric distance while preserving the semantics of the original image. These methods encompass attacks based on spatial transformations [46] [] manipulations within the color space 19 54), the addition of
texture (2), and color transformations guided by segmentation [35] 51] Notably, (38 introduced
concept also termed 'unrestricted adversarial attack'; however, in their context, "unrestricted" signifies
that the attack is not limited by the presence of an original image but rather by original class.
Recent works 7 50 incorporate adversarial attack gradients into the image editing process and utilize contemporary probabilistic generative models diffusion models to create semantic-awan adversarial examples. Our approach distinct. While all methods involve some form combining attack gradients with generative gradients, our method is principled, derived from the original optimization problem introduced Equation (T]. Moreover, we introduce a novel probabilistic perspective adversarial attacks for the first time.

---

8 Discussions
8.1 Contrasting MNIST and ImageNet Experiments
Targeted vs. Untargeted Attacks The MNIST dataset, comprising only 10 classes, allows us to perform targeted attack experiments efficiently. However, ImageNet, with its extensive set of 1,000 classes, presents practical challenges for conducting targeted attacks on each class individually, Consequently, we assess untargeted attack performance, aligning with methodologies in other studies.
Data Diversity Adversarially trained networks like MadryNet for MNIST are difficult to fool, primarily due the limited diversity among handwritten digits. As illustrated in Figure ( (b). the classifier can nearly memorize the contours of each digit, given its impressive generative capabiliitess for such data. In attacking this classifier, we carefully selected the few method for the victim distribution to reduce the influence of the target class's ' 'shadow." In contrast, for ImageNet, the vast diversity and the relatively weaker generative ability of the victim classifier allow for the use of fCE. facilitating higher confidence in target class recognition by the victim classifier.
8.2 Defending This Attack
Adversarial training operates on the principle: 'If know the form of adversarial examples in advance, I can use them for data augmentation during training.' Thus, the success of adversarial training largely depends on foreknowledge of the attack form. Our method bypasses adversarially trained classifiers because the 'semantic- semaatic-preservinng perturbation' we employ is unforeseen by the classifier designers- they use conventional adversarial examples for training.
Conversely, if designers anticipate attacks from our algorithm, they could incorporate examples generated by our method into their training process essentially, new form of adversarial training.
This scenario transforms adversarial attacks and defenses into a game of Rock-Paper- Paper-Scissore where anticipating the type of attack becomes crucial. One might consider training a classifier using all known types of attacks. However, expanding the training data too far from the original distribution typically leads to decreased performance the original classification task, which is undesirable [52]. We believe that investigating the trade-off between this "generalized' adversarial training and accuracy the original task represents promising avenue for future research.
8.3 Limitations
Training or fine-tunin model for each original image Zoi is time-consuming, Recent advancements, such faster fine-tuning methods 220 48). offer potential solutions to mitigate this issue. We see promise in these developments and consider their application an avenue for future research.
8.4 Ethical Considerations in User Studies
As mentioned by (28) the ImageNet dataset contains elements that may be pornographic violate personal privacy. mitigate the exposure of human annotators in our experiments (see Section (6) to such sensitive content, we avoid selecting any images that depict humans for our original images Tori-
Conclusion
This paper offers a probabilistic perspective on adversarial examples, illustrating a seamless transition from 'geometrically restricted adversarial attacks' 'unrestricted adversarial attacks." Building upon this perspective, we introduce two specific implementations for generating adversarial examples using EBMs and diffusion models. Our empirical results demonstrate that these proposed methods yield superior transferability and success rates against adversarial defense mechanisms, while also being minimally detectable by human observers.
10

---

Acknowledgments and Disclosure of Funding
Andi Zhang supported by a personal grant from Mrs. Yanshu Wu. Mingtian Zhang acknowledges funding from the Cisco Centre of Excellence. We thank the Ethics Committee of the Department of Computer Science and Technology, University of Cambridge, for their guidance and advice regarding the user study conducted in this work.
References
[1] R. Alaifari, G. S. Alberti, and T. Gauksson. Adef: an iterative algorithm to construct adversarial deforma- tions. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, OpenReview.net, 2019.
jR5K7
[2] A. Bhattad, M. J. Chong, K. Liang, B. Li, and D. A. Forsyth. Unrestricted adversarial examples via semantic manipulation. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URU/https://oppeeerraae wtt.iiewwwe///oourueeee
id=Sye_DgifFge DgHlFHH
[3] F. L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on pattern analysis and machine intelligence, 11(6):567-585, 1989.
[4] N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. Iece, 2017.
[5] J. Chen, H. Chen, K. Chen, Y. Zhang, Z. Zou, and Z. Shi. Diffusion models for imperceptible and transferable adversarial attack. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
[6] X. Chen, X. Gao, J. Zhao, K. Ye, and C.-Z. Xu. Advdiffuser: Natural adversarial example synthesis with diffusion models. In Proceedings of the IEFF/CVF International Conference Computer Vision, pages 4562-4572, 2023.
[7] Z. Chen, B. Li, S. Wu, K. Jiang, S. Ding, and W. Zhang. Content- unrestricted adversarial attack. Advancex in Neural Information Processing Systems, 36, 2024.
[8] T.SS. Chiang, C.-R. Hwang, and S. J. Sheu. Diffusion for global optimization in r'n. SIAM Journal on Control and Optimization, 25(3) 753, 1987.
[9] Deng, W. Dong, R. Socher, L.. Li, K. Li, and F. Li. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Puttern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pages 248 255. IEEE Computer Society, 2009. doi: 10.1109/CVPR.200 200.55008400..0 URL http:: /tttps://doi..oo org/10 1109/CVPR. 2009. 5206848
[10] P. Dhariwal and A. Q. Nichol. Diffusion models beat gans on image synthesis. M. Ranzato, Beygelz- imer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NewrlPS 2021, December 6-14, 2021, virtual, pages 8780-8794 2021. http://ttss:///ceeeeeeee nww.ggg...nr.p....e.. cc/ wwtpriippp.cppppppeppppepe happp/pppp//2// 2021/ hash/
Abstract. html
[11] Y. Du and Mordatch. Implicit generation and modeling with energy based models. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d' J'Alché-Bus, E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neu- ral Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC. Canada, pages 3603-3613, 2019. URL httpps:/pproceedinngg wwwpeeeeeiinnnee.enneeeeeeee www.eew....ppapeeeeeeeeeeeeeeeee wwtp:ccpppppppepeeee/eee /019/hash/ 379500bB8rb11ee5444444444ee htt.://aeeeee4ed-atrra....... html
[12] B. Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(496): 1602-1610 60141164, 2011.
[13] S. B. Gelfand and S. K. Mitter. Recursive stochastic algorithms for global optimization in r'd. SIAM Journal on Control and Optimization, 2995)9999-1018, 1991.
[14] I. Goodfellow. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA. USA, May 7-9, 2015, Conference Track Proceedings. 2015. URL http://arxiv, org/ aba//bs//111 1412
6572

---

[15] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Lay Vegus, NV, USA, June 27- 30, 2016, pages 770-778. IEEE Computer Society, 2016, doi: 10.1 20.1..09 URL https //doi. org/ 10. www./1119/V.... 2016.90
[16] Y. He, N. Murata, C.-H. Lai, Y. Takida, T. Uesaka, Kim, W.-H. Liao, Y. Mitsufuji. J. Z. Kolter, R. Salakhutdinov, et al. Manifold preserving guided diffusion. ArXiv preprint, abs/2311. 2023. URL URL.ttts::////// //axiiv.//iivv/v/// org/ aba/ 2311. 16424
[17] G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14 (8):1771-1800, 2002.
[18] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ran- zato. R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Sys- tems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, De- cember 6-12, 2020, virtual, 2020. URL Abstract .ttml
heapeediinnnnnuuiiiiie nwwripe cetrrip..ccceeee wttp:ccpppppp/eeeeeeeeeeeeeeeeee /2020/ hash/
[19] H. Hosseini and Poovendran. Semantic adversarial examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1614-1619, 2018.
[20] E. J. Hu, Y. Shen, Wallis, Z. Allen- Y. Li. S. Wang. L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICI. 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. httlitttta/:/ppereereee net/
forum? Cerum?id-ane SetmmmmdddZZeeeeeeF...
[21] G. Huang. Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In 2017 IEEE Conference July 21-26, 2017, pages 2261-2269. IEEE Computer Society, 2017. doi: 10.1109/CVP.22017.225 URL
Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA,
https://doi.... org/ 10. 100.//1R.//P.2 2017 243
[22] K. Kim and J. C. Ye. Noise2score:
Tweedic's approach to self-supervised image denoising without clean images. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34; Annual Conference on Neural Information Processing Systems 2021. NewrlPS 2021, December 6-14, 2021, virtual, pages 864 874, 2021. URL /ttstts::/proceeeeee neurips.cc/
/ttp://000001555511111533333 hat.aaa00011112200111111--1-1- www.eeccc-attttttttt. html
[23] A. Kurakin, I. J. Goodfellow. and S. Bengio. Adversarial machine learning at scale. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings, OpenReview.ne. 2017. URL httpt:ttt////ereeeeseee net.://w.eeeeeeee net: wdw.oornnnnnn.nnaa.a...
[24] A. Lamperski. Projected stochastic gradient langevin algorithms for constrained sampling and non-convex learning. In Conference on Learning Theory, pages 19977, PMLR, 2021.
[25] Y. LeCun, C. Cortes, and C. Burges. Mnist handwritten digit database. ATT Laby [Online]. Available: http://yann. www.n. www.st///////isstt.............. 2, 2010.
[26] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant adversarial attacks. In 6th International Conference on Learning Representations, ICI. 2018, Vancouver BC, Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreviev net Add
[27] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18- 24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8162 8171. PMLR, 2021. URL/tttp://prooeediing
mlr.
ni wwwp//nn.c..i...o/o/c.o html
[28] V. U. Prabhu and A. Birhane. Large datasets: A pyrrhic win for computer vision. ArXiv preprint, abs/2006. 16923, 2020. URL http:/tp////a...orr/////////// wtt.:o////bbb..........0.
[29] M. Raginsky, A. Rakhlin, and M. Telgarsky. Non-coovex learning via stochastic gradient langevin dynamics: nonasymptotic analysis. In S. Kale and O. Shamir, editors, Proceedings of the 30th Conference on Learning Theory, COLT 2017, Amsterdam, The Netherlands, 7-10 July 2017, volume 65 of Proceedings of Machine Learning Research, pages 1674-1703. PMLR, 2017. URL http: httt:://prooceeings. wtt.eeeeeding.m.......... mlr. press//
v65/raginsky17a html
[30] G. O. Roberts and R. L. Tweedie. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, pages 341-363, 1996.
12

---

[31] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assistee IntereeniionMIICAA/ 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer. 2015.
[32] N. Ruiz, Y.I.i, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman. Dreambooth: Fine tuning text-to- image diffusion models for subject-driven generation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 22500-2253 10, 2023.
[33] P. Samangouei, M. Kabkab, and R. Chellappa. Defense-gan gan: attacks using generative models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April May 3. 2018, Conference Track Proceedings. OpenReview.net, 2018. Uttphttps: battap::///peeevieeeee net/ formmmrrmmmmm Bidkimm1-d-
Protecting classifiers against adversarial
[34] S. Santurkar, A. Ilyas, D. Tsipras, L. Engstrom. B. Tran, and A. Madry. Image synthesis with a single (robust) classifier. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d' A'Accéé-Buc, E. B. Fox, and R. Gamett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC. Canada, pages 1260-12711 2019. URL #5228h116ddeaaabb44dddd3333 Abstract. peapeeddiinnnnnuiuiinee neurips wwtpcc/pppppeeeeeeeeeeee /001//hshh///
[35] A. Shamsabadi, R. Matihee-Matilla, and A. Cavallaro. Colorfool: Semantic adversarial colorization. In 2020 IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 1488-1551 IEEE, 2020. doi: 10. 00.10099CVP422260.22000000
URL
sttps://àoi.. wwwoor//////... 10.0 20000///22200000.. 2020 00123
[36] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA. USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL/jtttp:/arzxivorrr
abs/1409. 1556
[37] Y. Song and D. P. Kingma. How to train your energy-based models, ArXiv preprint, 0ats/2111.032885 2021. org/ //axiiv.
032288
[38] Y. Song, R. Shu. N. Kushman, and S. Ermon. Constructing unrestricted adversarial examples with generative models. In S. Bengio, H. M. Wallach, H. Larochelle. K. Grauman, N. Cesa-Bianchi. and R. Garnett, editors, Advances Neural Information Processing Systems 31: Annual Confer- ence on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal. Canada, pages 8322-8338 8333, 2018 URL Itttpe:/ neurips cctpapperccppe////pepppppppp 2018/hashh Abstract. html
[8cea559c47 4744 bdl7 btt.//bbbobbb22222224224000
[39] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, J. Goodfellow, and R. Fergus. Intriguing prop- erties of neural networks. In Y. Bengio and Y. LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings. 2014.
URL /ttp:///://arxx org/ abs/ 1312. 6199
[40] C. Szegedy, Vanhoucke, S. Ioffe. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2818-2826. IEEE Computer Society, 2016. doi: org/10. 1109/
10.1109/CVPR.10 2016..00
1109/CVPR. CVPR. 2016 6.08
[41] M. Tan and Q. V. Lc. Efficientnet: Rethinking model scaling for convolutional neural networks. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 6105-6114 14. PMLR, 2019. URL htt.::t://pooccedingge mlr. press/ v97/tan19a. html
[42] F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. D. McDaniel. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018 Rtll/ttpp://operreeei
[43] B. Tzen, T. Liang, and M. Raginsky. Local optimality and generalization guarantees for the langevin algorithm via empirical metastability. In Bubeck, Perchet, and P. Rigollet, editors, Conference On Learning Theory, COLT 2018, Stockholm, Sweden, 6-9 July 2018, volume 75 of Proceedings of Machine Learning Research, pages 857-875. PMLR, 2018 URL htttp://proceeednggg mlr press// v75/tzen18a, html
13

---

[44] J. Von Neumann. 13. various techniques used in connection with random digits. Appl. Math Ser, 12(36-38):
3, 1951.
[45] F. Wong and J. Z. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference Machine Learning, ICML 2018, Stockholmsmüsson. Stockholm, Sweden, July 10-15. 2018, volume Proceedings of Machine Learning Research, pages 5283-5292. PMLR. 2018. Sasass/v00//ogggi html
URL http:// mlr
[46] C. Xiao, J. Zhu, B. Li, W. Hc, M. Liu, and D. Song. Spatially transformed adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3. 2018, Conference Track Proceedings. OpenReview.net, 2018. httLLatttps///poeeeevv.e.ee
HyyyRRRZC-
[47] C. Xie, M. Tan, B. Gong, J. Wang, A. L. Yuille, and Q. V. Lc. Adversarial examples improve image recognition, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 816-825. IEEE, 2020, doi: 1011199CCVPP22660000000000000000 URL htthhtttpp///ooooo/ org/ 10. 1109 COR00///VP22200022
000900
[48] E. Xie, L. Yao, H. Shi, Z. Liu, D. Zhou, Liu, J. Li, and Z. Li. Difffit: Unlocking transferability of large diffusion models via simple parameter-efficien fine-tuning. ArXiv preprint, abs/2304.06644, 2023, URL https://arxiv org/ abs/ 2304. 06648
[49] P. Xu, J. Chen, D. Zou, and Q. Gu. Global convergence of langevin dynamics based algorithms for nonconvex optimization. In S. Bengio, H. M. Wallach, H. Larochllle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances Neural Information Processing Systems 31: Annual Confer- ence on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal. Canada, pages 3126-3137, 2018 URL Abstract. html
2ee///211/aaaaaae 10/8/hahh// www.:newurrrccccpooo
[50] H. Xue, A. Araujo, B. Hu, and Y. Chen. Diffusion--aase adversarial sample generation for improved stealthiness and controllability. Advances in Neural Information Processing Systems, 36:2894-299 29211 2023.
[51] S. Yuan, Q. Zhang, L. Gao, Y. Cheng, and J. Song. Natural color fool: Towards boosting black-box unrestricted attacks. Advances in Neural Information Processing Systems, 35:7546 7560, 2022.
[52] H. Zhang, Y. Yu, Jiao, E. Xing. L. El Ghaoui, and M. Jordan. Theoretically principled trade-off' between robustness and accuracy. In International conference machine learning, pages 7472-7482 PMLR, 2019.
[53] Y. Zhang, P. Liang, and M. Charikar. A hitting time analysis of stochastic gradient langevin dynamics. In S. Kale and O. Shamir, editors, Proceedings the 30th Conference on Learning Theory, COLT 2017, Amsterdam, Netherlands, 7-10 July 2017, volume 65 of Proceedings of Machine Learning Research, pages 2022, PMLR, 2017. URL, httt::/pprooceeiinngep.oooo.ooep mlr. wwwpppreesssseee 2ataas//////zaag/7/ html
[54] Z. Zhao, Z. Liu, M. A. Larson. Adversarial color enhancement: Generating unrestricted adversarial images by optimizing a color filter. In 31 British Machine Vision Conference 2020, BMVC 2020, Virtual Event, UK, September 7-10, 2020. BMVA Press, 2020. URL htLLhhtts://ww..ee bmwc2000- conf erence com/ wwwaaaaeeseeeeeeeeeeeeaa httpa//ssttppppaaaaeeee hhtpppppppapp//// 0009 pdf
[55] Y. Zhu, J. Ma, J. Sun, Z. Chen, R. Jiang, Y. Chen, and Z. Li. Towards understanding the generative capability of adversarially robust classifiers. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 7728-7737, 2021.
14

---

A Proof of the Theorems
Given the condition that Pric Fadi & exp(-c2 [Tadv Yur)), Pais (Tado Fori) o exp(-o1 D( Fadt)), the samples drawn from Pody will exhibit the same distribution the adver- sarial examples derived from applying the nox-constrained Langevin Monte Carlo method to the
Theorem 1.
optimization problem delineated in equation ①7
Proof. introduced the Projected Stochastic Gradient Langevin Algorithms (PSGLA) address box-constraint optimization problems. By leveraging the PSGLA, we can generate samples close the solution of the optimization problem as stated in Equation IT. This leads to the following update rule:
? 2
VeL(xt, Mue) + c2s): 24 ~ I(0,
To ~ po. T++1 = [o.1] It
where [1, 1 a projection that clamps values within the interval [0, 1J". According to 224 samples generated via this update rule will converge stationary distribution, which can be termed the Gibbs distribution Pritbs:
Pyibis (Tadv (Mar))
) x exp( - -(Wndv Vhar))
x exp( (Toii Tndv) - C2 I (Indvs Ytar)) x exp( -c1 D( Taav)) exp( - 02 (Fadyy Ytar))
X Pais (Xadv (To) Pvic (Bur)
which matches the form of Padv. It is a well-established fact that random variables with identical unnormalized probability density functions share the same distribution.
Theorem 2. Let ( x exp(-of of (2. Ylar)) and po -= = 11 Ex(xx) )), we have
(ooc(oo) ) 1/T pa ( N (po(ze) - [o (o)))) Eg(21)
where g = -V la and øoje is defined in Equation 5
Proof. For brevity, denote / (0(x))) and Σ = Ea (x+). As suggested by [10 with an increasing number of diffusion steps, ||]| - O, allowing us reasonably assume that log Pvic (2oj+ Wur)) has low curvature relative Σ-1. We approximate log Pvic (fojjel using Taylor expansion around
It-1 μ (noting žоμt as function of xx-1) as follows:
log Pvic (Fojs Mar))
R log Pvic ( (Mrr)) 1=, + (xI-1 Var- Poic (2op lx
= C1 + (Ir-1 4) V (-of (ãofe, Vear) (aar))ae 12m-1
= C1 + (It-1 - p)eg
where C1 is constant and g -vz (Bojt
Then
log(psic ( (orr) 1/T Po (o (xx)) = log pa (x+ (xt-1 (rt) +
log Pric (5o Marr)
T
[It-1 - 11)² " -- - 11) { (11-1 11 1)g + C2
d²
(It-1 1 7 Eg) -5----- T
1251-1 11 T [ +
T
Eg C2
2T
(It-1 - # { وو " (I - 1- T )gg) + C23
C
C
which the unnormalized log pdf of a Gaussian distribution with mean + + 5g and variance
E.
D
15

---

B Preliminaries (Continued)
B.1 Langevin Monte Carlo (LMC)
Langevin Monte Carlo (also known as Langevin dynamics) is an iterative method that could be used find near-minimal points of non-convex function !
29
53
43
30
It involves updating the
function as follows:
To Po. Te+1
Tt
V ( + EEt,
2 10
(8)
where po could be uniform distribution. Under certain conditions on the drift coefficient V.g. has been demonstrated that the distribution of z in (8) converges to its stationary distribution [8 30) also referred as the Gibbs distribution
p(x) oc expl (x))) This distribution concentrates around the global minimum of [I3 49 30). If we choose g to be Eg, then the stationary distribution corresponds exactly to the EBM's 's distribution defined (2) As a result, we can draw samples from the EBM using LMC.
B.2 Training/ Fine- Taninn EBM
To train EBM, aim to minimize the minus expectation of the log- ligelikei,oodd, represented by LERM = Ee--rl- log po (a:] Ex-nu [x) - log Zo
where is the data distribution. The gradient is
VoLEBM Ex-pp V Eo))]
V log Ze
Egppp Eg Ex-pp [o)))
(see [37 for derivation, this method is also called contrastive divergence). The first term of V CEMM can be easily calculated as pa the distribution of the training set. For the second term, we can use LMC to sample from Pa [17]
Effective training of an energy-based model (EBM) typically requires the use of techniques such as sample buffering and regularization. For more information, refer to the work of
B.3 Training Fine-Tuuing Diffusion Models
18 presented method to train diffusion models by maximizing the variational lower bounds (VLB), which is expressed through the following loss function:
Lwb = Lo + Ln + TT-I Lrr
where
Lg = log po po (xo)=1), L--1 == DEL ( a. (x)- :r Dxz(((xr<z))) (p(rr)
Assuming Mg obL, where or = 3r or o, 3, (with B, == 1- 1-an
Bi), and using the parametriza-
tion
(o( Vat
It
VI - āi co( _e((x()))
3t
18
proposed a simpler loss function, Lsimple.
Lsimple F [||e c (2t, [)]
which is shown to be effective in practice. Later, (27 suggested that trainable Ee (x t) exp(c log + (1 v) log 3.) could yield better results. Since En is not included siimlee they
introduced new hybrid loss:
Taybrid = Tsipple + AT
In this work, adopt the improved approach suggested by 27
C Pseudocode for Sampling Methods from the Adversarial Distribution
16

---

Algorithm Sampling from Padv by EBM
Input: Trained / finetuned EBM Eg depends Puri, the function corresponding to the victim classifier, target class Yur. total time step T, noise level e, parameter c. Output: Sample x:
~ (0, I).
for 1 1 to T do
2 ~ (0, /)
end for (cV=f(x, Yas) Eo()))
Algorithm 2 Sampling from Padv by diffusion model
Input: Trained / finetuned diffusion model (Ho, Eg), the function f corresponding to the victim classifier, the original image Ton, target class Yur. total time step T, variance schedule .1..... Br and its associate a1..... OrT, 01,, 01,..., Or, parameter c. Output: Sample ro aT (0, I).
for 1 = 7 to do
eø (dit) <1-a;
(xt (oi))(s1))
3:
Σоt: Vac
(xr -1 - acco(zz))
#t-1 (Nuu((x) - +Zi0(z+) Vo (20 Yar) (x()))
end for
D Practical Techniques
This section outlines practical techniques employed in the implementation.
D.I Data Augmentation by Thin Plate Splines (TPS) Deformation
Thin-plate-spline (TPS) (3] is commonly used image deforming method. Given a pair of control points and target points, TPS computes smooth transformation that maps the control points to the target points, minimizing the bending energy of the transformation. This process results in localized deformations while preserving the overall structure of the image, making TPS a valuable tool for data augmentation.
P sau
P
Zori
(yps (Fodis P P(n))
Figure 6: TPS as data augmentation. Left: The original image Fon superimposed with a 5 x 5 grid of source control points Peon- Right: The transformed image overlaid with grid of target control points Pur.
As introduced in Section we aim train an energy-based model on transformations of a single image Tori- In practice, il the diversity of the augmentations of Tori. represented as t(Tor)) is insufficient, the training of the probabilistic generative model is prone to overfitting. To address this issue, we use TPS as a data augmentation method to increase the diversity of t(Fozi). For each Tai- we set a 5 x 5 grid of source control points, Psoo = (() (6)55)
15x5 and defining the target points
(())) + ey (i)) )) where ez ~ NN(0, o²) are random noise added to the with Pun and Pt. as its parameters. This procedure depicted in Figure 6 By setting an appropriate a, we can substantially increase the diversity of the one-image dataset while maintaining its semantic content.
as P... = (() + ez (i)
source control points. We then apply TPS transformation
17

---

D.2
Rejection Sampling
Directly sampling from Padv (- Fodi You) does not guarantee the generation of samples capable of effectively deceiving the classifier. To overcome this issue, we adopt rejection sampling [44], which eliminates unsuccessful samples.
D.3 Sample Refinement
After rejection sampling, the samples are confirmed to successfully deceive the classifier. However, not all of them possess high visual quality. To automatically obtain N semantically valid samples we first generate M samples from the adversarial distribution. Following rejection sampling, we sort the remaining samples and select the top A percent based on the softmax probability the original image's 's class, as determined by an auxiliary classifier. Finally, we choose the top N samples with the lowest energy E, meaning they have the highest likelihood according the energy-basee model.
The entire process of rejection sampling and sample refinement is portrayed in Algorithm
Algorithm 3 Rejection Sampling and Sample Refinement
Input: A trained energy based model (on; based on the original image Tori the victim classifier Ié. an auxiliary classifier Jg, number of initial samples M, number of final samples N, the percentage
Output: N adversarial samples x.
for Si do
Tadv Padv(::20 Toii Vtar)
(Sample from the adversarial distribution.)
if arg maxy 96 (Iadv) = Y/ar then
x = J (Tadv
end if
end for
Sort x by (gg (x:)) (ooe) for E (1,...,/21) |x|} in descent order (x
[Select the first k percent elements from æ.)
Sort x by (xx; Ton) for i {l,.... | in ascent order (x.)
(Select the first N elements from x..
D.4 Adjust the Start Point of Diffusion Process
To preserve more of the original content from Fori while sampling from its adversarial distribution, can initiate the diffusion process at a time step T" that is earlier than T. The details of this approach are outlined in Algorithm 4
II practice, we could select adversarial samples by hand, but we focus on automatic selection here.
Algorithm 4 Sampling from Padv by diffusion model
Input: Trained finetuned diffusion model (Hr, Ee), the function f corresponding to the victim classifier, the original image Fori target class Yta. total time step num denoising step T". variance schedule 31,... Br and its associate a,,..., ar, ā1...., 0r, parameter Output: Sample ap FT N( Ron, (1 ō.)) J)).
Ron,
for t T do
<x(x;) TT
De (xt (o(()
Soje = Va.
(Ie - /1 - āi (x())
TT-1 ((u(Iz)) (Eo(zz)) V (Zo1t, Yar), Ee
end for
18

---

E
Annotator Interface
display the interfaces used by annotators in the user study, as described in Section Note that each case, annotators are given 10 seconds to render their judgment.
Figure 7 and Figure 8
B
proteceed right of
imaee Checss optin oil
MA
on
Figure 7: Annotator Interface for image annotation.
Figure Annotator Interface for comparison.
F Broader Impacts of this work
The present study introduces a novel approach: the semantics-aware adversarial attack. This method provides significant insights into the resilience and vulnerability of sophisticated classifiers.
From an advantageous perspective, highlights the inherent risks associated with robust classifiers. By exposing potential weak points in such systems, the study underscores the necessity for further improvements classifier security. This can pave the way for building more resilient artificial intelligence systems in the future.
Conversely, the work also presents potential pitfalls. There risk that malicious entities might exploit the concepts discussed here for nefarious purposes. It is crucial to take into account the potential misuse of this semantics-awart adversarial attack and accordingly develop preventive
measures to deter its utilization for unethical ends.
19

---

G Datasets and Licenses
We use MNIST 25 and ImageNet g in this work. The MNIST dataset available under the terms of the Creative Commons Attribution-Shan Alike 3.0 license. ImageNet does not have a specific license the moment, but it commonly used dataset in the research community.
H Compute Resource
We conducted our experiments using multiple workstations, each equipped with an NVIDIA RTX 4090 GPU (24GB VRAM) and 64GB of system memory.
20

---

I Qualitative Result (Continued)
Figure 9: Comparative visual analysis of NCE, cAdv, ACE, ColorFool and proposed method applied to Imagenet. The surrogate classifier used ResNet50.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, we confirm that the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, these are provided in Section 8.3
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

Justification:
Yes, these are provided in the preliminary section and in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide the corresponding information in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification:
We provide the link to the GitHub repository in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com htte:/uulllllliiiee/e//eeeuaaasl ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provide the corresponding information in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Since the results contain human study data, cannot duplicate them and attach error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This is provided in Appendix H
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We confirm that the research conform the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
This is discussed in Appendix F
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We filtered the data used in this project as described in Section 8.4
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We discussed this in Appendix G
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not have any new assets report.
Guidelines:
The answer NA means that the paper does release new assets. Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: Yes, we have attached screenshots of the Ul offered to the participants in Appendix [E] The crowdsourcing tasks were entrusted to a company, and the workers were compensated at rate higher than the country's 's minimum wage.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We confirm that the human study passed the ethical review of the Ethies Com- mittee of the Department of Computer Science and Technology, University of Cambridge
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
27


Reviews:
[{"review_id": "MifgUS5zcT", "reviewer": "Reviewer_A1FN", "summary": "This paper introduces a probabilistic framework for generating adversarial examples, focusing on maintaining the semantic integrity of the original images while implementing substantial pixel-level modifications. Unlike conventional adversarial techniques that rely heavily on minimal geometric perturbations, this approach integrates a semantic understanding into the adversarial example generation process, leveraging energy-based models and diffusion models. The core innovation lies in embedding the semantic interpretation as a probabilistic distribution, which guides the adversarial example generation. This allows for effective deception of classifiers, including those equipped with adversarial defenses, while preserving the semantic content to an extent that remains imperceptible to human observers. Empirical evaluations demonstrate that the proposed method outperforms existing techniques in terms of both effectiveness against defenses and undetectability by humans, establishing a new paradigm for constructing robust and stealthy adversarial attacks.", "questions": "Please address the aforementioned concerns and questions.", "limitations": "Paper limitations can be found in the above comments.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clear and well-written, effectively highlighting its contributions with accessible explanations of complex ideas.\n\n2. This paper presents a new probabilistic framework for generating adversarial examples that goes beyond traditional norm-bounded methods by integrating semantic distributions. The approach is theoretically robust, with the theoretical analysis providing a solid foundation that supports the model's effectiveness and introduces innovative concepts to the field of adversarial machine learning.\n\n3. The proposed method significantly outperforms baseline methods, particularly in preserving their semantic integrity.", "weaknesses": "1. The assumption in Equation 4 lacks a detailed derivation, leaving it unclear whether $x_{\\text{ori}}$ and $y_{\\text{tar}}$ need to be independent. Providing a clear derivation and clarifying this assumption would enhance the theoretical rigor of the paper.\n\n2. The training process for the diffusion models is sensitive and requires careful parameter tuning. The paper does not provide enough detail on this sensitivity or potential solutions to mitigate training instability, which impacts the robustness and reproducibility of the method.\n\n3. The paper does not report standard deviations in the performance results. Repeating the experiments is recommended to ensure the reliability and consistency of the findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's time and valuable feedback, which has significantly contributed to improving our work. We address the identified weaknesses as follows:\n\n### About Equation (4)\nOur proposed framework assumes that the adversarial distribution is proportional to the product of the distance distribution and the victim distribution. This assumption separates the distance distribution from the victim distribution, reflecting our understanding that the similarity between objects is not directly related to the classifier being attacked. While this perspective might be debated, we believe it is reasonable within the context of our paper.\n\nTheorem 1 demonstrates that this proposed form of adversarial distribution is consistent with conventional adversarial attacks. In Appendix A, we prove Theorem 1 by starting with the conventional adversarial attack setting (Equation (1)) and deriving the form of the product of $p_\\text{dis}$ and $p_\\text{vic}$. This derivation can be found above line 423 in our paper.\n\nWe appreciate the reviewer's comment and will provide more intuition about separating the distance and victim distributions between lines 71 and 73 to enhance readability.\n\n### About the training process of diffusion models\nPlease refer to the `README.md` file in the ImageNet folder of our attached anonymous GitHub repository (link provided in the abstract). Our code is based on OpenAI's guided diffusion repository, a mature and widely used diffusion implementation. We fine-tune the weights provided by the guided diffusion repository rather than training a diffusion model from scratch.\nThe hyperparameters for fine-tuning are specified in our repository. We maintain most of the original training hyperparameters, with two exceptions:\n\n1. Learning rate: 1e-6 (a commonly used rate for fine-tuning)\n2. Number of fine-tuning steps: 300 (empirically determined)\n\nWe found that after 300 fine-tuning steps on the original image, the fine-tuned diffusion model adequately reflects the original image. These hyperparameters are consistently applied across all images in our evaluation, which we believe is appropriate for our study.\n\nWe thank the reviewer for pointing this out. Indeed, merely including this information in the accompanying code is insufficient. In the final version of the paper, we will add an appendix that provides a detailed discussion of the aforementioned parameter selection and fine-tuning process.\n\n\n### About Repeating the experiments\nTo calculate the standard deviation in repeated experiments, we need to generate multiple sets of adversarial examples for each method and have them annotated by human annotators. Given the time required for annotation, we commit to providing these results in the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QACxlBXW6C", "reviewer": "Reviewer_ggd1", "summary": "This paper proposes a new type of adversarial attack, which generates adversarial examples by solving a box-constrained non-convex optimization problem. Different from the traditional norm-bounded attacks, this paper focuses on unrestricted adversarial attacks by replacing the geometrical distance measure with a semantically-aware distance measure. Specifically, The authors propose using a Langevin Monte Carlo (LMC) technique to sample adversarial examples from a probabilistic distribution. To preserve semantics, the authors use a learned energy function to guide the generation of adversarial samples. Following this, rejection sampling and refinement techniques are employed to select and improve the quality of the generated samples. Experiments show that this attack can fool classifiers while preserving the semantic information compared to baseline methods.", "questions": "Please refer to the **Weaknesses**.", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces an interesting perspective on generating adversarial examples, which is significantly different from the traditional norm-bounded adversarial attacks. \n2. This paper is theoretically sound and the proposed solution is very intuitive.\n3. It is suprising that the proposed attack can achieve a 100% success rate on an adversarially trained model. Adversarial training is often regarded as a SOTA defense method. Therefore, in my view, this work can motivate researchers in this area to design better defense methods. \n4. The proposed method can either outperform baseline methods by a notable margin or significantly improve the quality of the generated adversarial examples in terms of preserving semantic meanings.", "weaknesses": "1. Selecting 20 images from each class in the MNIST test set seems to be too little. I understand that it might be infeasible for human annotators to annotate all adversarial images for the entire MNIST, so I would encourage authors to report the success rate except for human annotations using the entire MNIST. I believe this will make the results more convincing.\n2. This paper is missing ablation studies for rejection sampling and sample refinement techniques. Is it necessary to include these techniques? How would it affect the attack success rate if one of them is removed?\n3. This paper proposes a new attack method but lacks a discussion on how to defend against it. Although it is not compulsory, I am more willing to see how to defend this attack. Can you provide some intuitions on it?\n4. Standard deviations are not reported in this paper. Repeated experiments are encouraged.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's thorough evaluation and acknowledgment of our probabilistic approach and proposed methodology. We are grateful for the identification of weaknesses in our submission. The reviewer's insights have significantly enhanced the quality of this work. We address the review points as follows:\n\n**Success rate except for human annotations**\n\nIn response to the reviewer's suggestion and to enhance the reliability of our results, we conducted an additional experiment using the MNIST test set, without human annotation. The results are as follows:\n\n|                     | PGD  | ProbCW | stAdv | OURS  | OURS (no tech.) | OURS (rej. samp. only) |\n|---------------------|------|--------|-------|-------|-----------------|------------------------|\n| Human Anno.         | N/A  | N/A    | N/A   | N/A   | N/A             | N/A                    |\n| **White-box**       |      |        |       |       |                 |                        |\n| MadryNet Adv        | 26.9 | 30.5   | 30.0  | 100.0 | 35.4            | 100.0                  |\n| **Transferability** |      |        |       |       |                 |                        |\n| MadryNet noAdv      | 15.4 | 18.0   | 15.8  | 60.4  | 17.9            | 60.2                   |\n| Resnet noAdv        | 9.8  | 10.0   | 11.9  | 23.2  | 13.3            | 21.2                   |\n| **Adv. Defence**    |      |        |       |       |                 |                        |\n| Resnet Adv          | 7.3  | 8.5    | 11.2  | 19.4  | 12.9            | 19.4                   |\n| Certified Def       | 11.2 | 12.1   | 22.3  | 40.6  | 24.7            | 40.9                   |\n\nNote that 'OURS (no tech.)' and 'OURS (rej. samp. only)' refers to the ablation study discussed in the subsequent section. \n\n**Ablation study on two techniques**\n\nWe have included the ablation study results for the two techniques in the table above. \n\nRejection sampling, a classical method, aligns naturally with our probabilistic approach to adversarial attacks. By employing rejection sampling, we can generate adversarial examples that achieve a 100% attack success rate in white-box settings. This perfect success rate is possible because we can consistently reject samples that fail to deceive the victim classifier.\n\nOn the other hand, sample refinement primarily affects human annotation while having minimal impact on the attack success rate for classifiers.\n\nThese two techniques are only used in MNIST experiments, because generating targeted adversarial samples for an adversarial trained MNIST classifier is relatively hard: As is shown in Figure 2 (b), the adversarially trained MNIST classifier is already have a strong generation ability, meaning that it already memorize the shape of the each digit.\n\n**Discussion on defending this attack**\n\nWe appreciate the reviewer pointing out this problem. In the final version of the paper, we will include the following discussion:\n\nAdversarial training operates on the principle: 'If I know the form of adversarial examples in advance, I can use them for data augmentation during training.' Thus, the success of adversarial training largely depends on foreknowledge of the attack form. Our method bypasses adversarially trained classifiers because the 'semantic-preserving perturbation' we employ is unforeseen by the classifier designers - they use conventional adversarial examples for training.\n\nConversely, if designers anticipate attacks from our algorithm, they could incorporate examples generated by our method into their training process - essentially, a new form of adversarial training.\n\nThis scenario transforms adversarial attacks and defenses into a game of Rock-Paper-Scissors, where anticipating the type of attack becomes crucial. One might consider training a classifier using all known types of attacks. However, expanding the training data too far from the original distribution typically leads to decreased performance on the original classification task, which is undesirable [1]. We believe that investigating the trade-off between this ‘generalized' adversarial training and accuracy on the original task represents a promising avenue for future research.\n\n**Standard deviation**\n\nTo calculate the standard deviation in repeated experiments, we need to generate multiple sets of adversarial examples for each method and have them annotated by human annotators. Given the time required for annotation, we commit to providing these results in the final version of the paper.\n\n**Reference**\n\n[1] Zhang, Hongyang, et al. \"Theoretically principled trade-off between robustness and accuracy.\" International conference on machine learning. PMLR, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sPgB7MCQM1", "reviewer": "Reviewer_aH4C", "summary": "This paper tackles the field of adversarial image generation by proposing an unrestricted attack method that can be applied to both targeted and untargeted attacks. The innovative approach considers a probabilistic perspective, treating the victim classifier and geometric constraints as distinct distributions. By drawing adversarial examples from the overlap region, the authors ensure that the semantics of the original image are preserved. The efficacy of this proposed approach is convincingly demonstrated through extensive experiments.", "questions": "1. How were the values of c chosen in Table 2?\n2. What are the author's thoughts on the tradeoff between choosing different values of c?\n3. How were the human annotators selected?\n4. How was the set \\tau chosen?", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. I find the probabilistic approach proposed in this paper to be particularly innovative and refreshing. The motivation behind this perspective is clearly articulated, providing a solid foundation for the authors' methodology.\n2. I am impressed by the encouraging experimental results presented in this paper. The inclusion of a human annotation experiment is particularly noteworthy, as it adds an important layer of validation to the authors' claims. Moreover, the study's success in handling both transfer attacks and adversarial defense scenarios further underscores the model's robustness and effectiveness.", "weaknesses": "While the experimental results of the proposed method show promise, I do believe there is room for improvement. Specifically, I think it would be beneficial for the authors to provide more detailed information regarding the human experiment methodology, such as how the five reviewers for the MNIST experiment were selected. Furthermore, I would suggest that the authors consider conducting a follow-up experiment where human annotators are asked to identify perturbed images in the absence of a reference image for the ImageNet experiment, which is a more realistic scenario in an attack setting.\n\nIn addition, I find it intriguing that both NCF and cAdv demonstrated higher success rates in generating adversarial examples compared to the proposed method, as shown in Table 2. This highlights some shortcoming in the proposed approach. While it is expected that NCF would generate images that can be identified as perturbed, I am more surprised that cAdv was able to create perturbed images that are highly similar to the original ones.\n\nLastly, I think it would be beneficial for the authors to explore targeted attacks on ImageNet, given the success of this approach in previous papers such as \"Towards Transferable Targeted Attack\". This could provide valuable insights into the robustness and effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review and your recognition of our probabilistic perspective and proposed method. Thank you for indicating the oversights and shortcomings in our submission. Your feedback has effectively improved the quality of this work. We respond to your questions and concerns as follows:\n\n## Response to weaknesses\n\n### Human Experiments\n\n> more detailed information … such as how the five reviewers for the MNIST experiment were selected. \n\nRegarding the selection of human annotators, please refer to our response to Question 3 below. We have included screenshots of the user interfaces for annotators in Appendix E. We apologize for omitting an important detail in our original submission: we use a voting system to determine the final choice from human annotators. In cases of a draw (which can occur in the MNIST dataset), we randomly select from the tied options. We will incorporate this description in the revised version. Thank you for bringing this to our attention.\n\n> … conducting a follow-up experiment … \n\nWe appreciate your suggestion for a follow-up experiment on ImageNet without reference images. Given the time required for human experiments, we will add this experiment in the final version of our paper.\n\nWe understand your concern regarding the presence of reference images in our experiment. We would like to clarify that this does not compromise our methodology. In our study, annotators are not informed which image is the reference. We realize we inadvertently omitted a crucial detail in Figure 8: the original image and adversarial example are presented in random order, adhering to the A/B test methodology employed in unrestricted adversarial attack papers such as [1] and [2]. We will emphasize this important point in the final version of our paper.\n\n### Comparison with NCF and cAdv\nWe indeed overlooked emphasizing that our method is optimal in balancing the trade-off between human annotation success rate and attack success rate in the paragraph from lines 258 to 264. In the final version, we will explicitly highlight that, according to Figure 5, our method performs best in this trade-off when appropriate parameters are selected.\n\nWe hypothesize that cAdv's lower human annotation success rate may be attributed to the presence of unnatural color spots in the generated images, as illustrated in Figure 4. These artifacts likely make the adversarial examples more noticeable to human annotators.\n\n### Targeted Attacks on ImageNet\nWe acknowledge that targeted attacks on ImageNet pose greater challenges due to the large number of classes, particularly regarding transferability. Our method does not offer specific advantages for this task. In the related work section, we will expand on this issue, discuss the work you recommended, and suggest it as a potential direction for future research based on our probabilistic perspective.\n\n## Answer to questions\n\n**Q1: How were the values of c chosen in Table 2?**\n\n**Answer:** The parameter c controls the influence of the victim distribution. A larger c results in adversarial samples that are more likely to deceive the victim classifier but may deviate more from the original image's semantics. We initially tested the algorithm on a few images, visually assessing the results. Based on these observations, we selected c values of 5, 10, and 20 as they produced distinct yet reasonable outcomes. We then applied these hyperparameters to attack 1000 ImageNet images, yielding the results shown in Table 2 and Figure 5.\n\n**Q2: What are the author's thoughts on the tradeoff between choosing different values of c?**\n\n**Answer:** Figure 5 illustrates the trade-off between different c values, with the x-axis representing human annotation success rate and the y-axis showing attack success rate. As c increases, adversarial examples become more effective at deceiving the victim classifier but are also more easily identified as adversarial by human annotators. We leave the choice of c to the users of our proposed method, depending on their specific requirements.\n\n**Q3: How were the human annotators selected?**\n\n**Answer:** We conducted human participation experiments through a reputable crowdsourcing company. The five annotators were recruited by this company. Our human experiments passed our institute's ethical review of research, taking into account the crowdsourcing company's qualifications, fair compensation, reasonable workload, and potential risks to workers. Due to the anonymous review process, we cannot share specific details here, but we can provide this information to the Area Chairs if required.\n\n**Q4: How was the set \\tau chosen?**\n\n**Answer:** As introduced in Section 4.1, $\\mathcal{T}$ represents a set of transformations that we subjectively believe maintain the semantics of the original image. For MNIST, this includes scaling, rotation, and thin-plate spline (TPS) transformations. For ImageNet, we utilize the distribution of the diffusion model after fine-tuning on the original image, which incorporates some implicit natural transformations learned by the model.\n\nIf we subjectively believe that appropriate color transformations do not affect semantics, we could incorporate such color transformations into $\\mathcal{T}$ when fine-tuning the diffusion model. The primary aim of this work is to propose a probabilistic framework that can embed subjective understanding, rather than to conduct an in-depth exploration of $\\mathcal{T}$ selection. We leave this exploratory task for future work.\n\n**Reference**\n\n[1] Song, Yang, et al. \"Constructing unrestricted adversarial examples with generative models.\" Advances in neural information processing systems 31 (2018).\n\n[2] Bhattad, Anand, et al. \"Unrestricted adversarial examples via semantic manipulation.\" arXiv preprint arXiv:1904.06347 (2019).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their recognition and encouragement of the probabilistic perspective and related approaches we proposed. We are also very grateful for the valuable suggestions made by the reviewers. Your insightful suggestions have significantly contributed to the refinement and improvement of this work.

Meta-review: This paper proposed a way to extend the pixel-level adversarial examples to construct semantics-aware adversarial examples. Authors did good job on rebuttal and all reviewers provide positive scores for this paper. AC read all rebuttal and reviews. AC think all comments are reasonable and think this paper should be accepted to NeurIPS.
Decision: Accept (poster)