Constructing Semantics-Aware Adversarial Examples with Probabilistic Perspective

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:17:37

Abstract: We propose a probabilistic perspective on adversarial examples, allowing us to embed subjective understanding of semantics as a distribution into the process of generating adversarial examples, in a principled manner. Despite significant pixel-level modifications compared to traditional adversarial attacks, our method preserves the overall semantics of the image, making the changes difficult for humans to detect. This extensive pixel-level modification enhances our method's ability to deceive classifiers designed to defend against adversarial attacks. Our empirical findings indicate that the proposed methods achieve higher success rates in circumventing adversarial defense mechanisms, while remaining difficult for human observers to detect.

Main paper: 
 introduction: 
 The purpose of generating adversarial examples is to deceive a classifier by making minimal changes to the original data's meaning. In image classification, most existing adversarial techniques ensure the preservation of adversarial example semantics by limiting their geometric distance from the original image (Szegedy et al., 2013; Goodfellow et al., 2014; Carlini & Wagner, 2017; Madry et al., 2017) . These methods are able to deceive classifiers with a very small geometric based perturbation. However, when targeting robust classifiers trained using adversarial methods, an attack involving a relatively large geometric distance may be necessary. Such alterations have a notable drawback: they can either distort the original image's semantics -going against the fundamental objective of adversarial examples -or lead to discernible changes in the image. Figure 1 illustrates this issue. When applying the PGD attack, constrained by geometric distances, to robust classifiers for digit images, the resulting adversarial images frequently deviate from their original semantic intent. In the case of natural images, even if the core semantics are preserved, the introduced changes can often be readily observed.In this paper, we introduce a probabilistic perspective for understanding adversarial examples. Through this innovative lens, both the victim classifier and geometric constraints are regarded as distinct distributions: the victim distribution and the distance distribution. Adversarial examples emerge as samples drawn from the product of these two distributions, specifically from the regions where they overlap. Notably, the overlap at the tail of the distance distribution accounts for the apparent modifications in the resultant adversarial samples.Based on this probabilistic perspective, we propose a new method for generating semantics-aware adversarial examples. Instead of relying on purely geometrically-induced distance distributions, we transition to a trainable, data-driven distance distribution. This evolution allows for the incorporation of our subjective understanding of semantics. With an appropriate semantic comprehension, our method can produce adversarial examples that are not only semantically aware but also highly effective. As depicted in the right section of Figure 1 , our semantics-aware adversarial attack successfully deceives robust classifiers while largely retaining the original image's semantic essence.  
preliminaries 2.1 adversarial examples: 
 The notion of adversarial examples was first introduced by Szegedy et al. (2013) . Let's assume we have a classifier C : [0, 1] n → Y, where n represents the dimension of the input space and Y denotes the label space. Given an image x ori ∈ [0, 1] n and a target label y tar ∈ Y, the optimization problem for finding an adversarial instance x adv for x ori can be formulated as follows:minimize D(x ori , x adv ) such that C(x adv ) = y tar and x adv ∈ [0, 1] nHere, D is a distance metric employed to assess the difference between the original and perturbed images. This distance metric typically relies on geometric distance, which can be represented by L 0 , L 2 , or L ∞ norms.However, solving this problem is challenging. As a result, Szegedy et al. (2013) propose a relaxation of the problem: minimize L(x adv , y tar ) := c 1 • D(x ori , x adv ) + c 2 • f (x adv , y tar ) such that x adv ∈ [0, 1] n (1)where c 1 , c 2 are constants, and f is an objective function closely tied to the classifier's prediction. For example, in (Szegedy et al., 2013) , f is the cross-entropy loss function, while Carlini & Wagner (2017) suggest several different choices for f . Szegedy et al. (2013) recommend solving (1) using box-constrained L-BFGS.Adversarial training, a widely acknowledged method for boosting adversarial robustness in deep learning models, has been extensively studied (Szegedy et al., 2013; Goodfellow et al., 2014; Huang et al., 2015; Madry et al., 2017) . This technique uses adversarial samples as (part of) the training data, originating from Szegedy et al. (2013) , and has evolved into numerous variations. In this paper, we apply the min-max problem formulation by Madry et al. (2017) to determine neural network weights, denoted as θ. They propose choosing θ to solve:EQUATIONwhere p data represents the data distribution, L CE is the cross-entropy loss, ∥•∥ p denotes the L p norm, and ϵ specifies the radius of the corresponding L p ball. In what follows, we will use the term "robust classifier" to refer to classifiers that have undergone adversarial training.An Energy-based Model (EBM) (Hinton, 2002; Du & Mordatch, 2019) involves a non-linear regression function, represented by E θ , with a parameter θ. This function is known as the energy function.Given a data point, x, the probability density function (PDF) is given by:EQUATIONwhere Z θ = exp(-E θ (x))dx is the normalizing constant that ensures the PDF integrates to 1. For details on sampling and training energy-based models, refer to Appendix A. 
a probabilistic perspective on adversarial examples: 
 We introduce a probabilistic perspective where adversarial examples are sampled from an adversarial distribution, denoted as p adv . This distribution can be conceptualized as a product of expert distributions (Hinton, 2002) :EQUATIONwhere p vic is termed the "victim distribution", reflecting its association with the victim classifier. p dis represents the distance distribution. A substantial value of p dis suggests a close resemblance between x adv and x ori based on the designated distance metric. p pri acts as the prior distribution for x adv .The subsequent theorem demonstrates the compatibility of our probabilistic approach with the conventional optimization problem for generating adversarial examples:Theorem. Given the condition that p vic (x adv ; y tar ) ∝ exp(-c 2 • f (x adv , y tar )), p dis (x adv ; x ori ) ∝ exp(-c 1 • D(x ori , x adv )), and set p pri (x adv ) as a constant, the samples drawn from p adv will exhibit the same distribution as the adversarial examples derived from applying the box-constrained Langevin Monte Carlo method to the optimization problem delineated in equation ( 1).The proof of the theorem can be found in Appendix B. Within the context of our discussion, we initially define p vic , p dis , and p pri to have the same form as described in the theorem. Given this formulation, we can conveniently generate samples from p adv , p dis , and p vic using LMC. Detailed procedures are provided in Appendix C. As we delve further into this paper, we may explore alternative formulations for these components. The victim distribution p vic is dependent on the victim classifier. As suggested by Szegedy et al. (2013) , f could be the cross-entropy loss of the classifier. We can sample from this distribution using Langevin dynamics. Figure 2 (a) presents samples drawn from p vic when the victim classifier is subjected to standard training, exhibiting somewhat indistinct shapes of the digits. This implies that the classifier has learned the semantics of the digits to a certain degree, but not thoroughly. In contrast, Figure 2 (b) displays samples drawn from p vic when the victim classifier undergoes adversarial training. In this scenario, the shapes of the digits are clearly discernible. This observation suggests that we can obtain meaningful samples from adversarially trained classifiers, indicating that such classifiers depend more on semantics, which corresponds to the fact that an adversarially trained classifier is more difficult to attack. A similar observation concerning the generation of images from an adversarially trained classifier has been reported by Santurkar et al. (2019) .The distance distribution p dis relies on D(x ori , x adv ), representing the distance between x adv and x ori . By its nature, samples that are closer to x ori may yield a higher p dis , which is consistent with the objective of generating adversarial samples. For example, if D represents the square of the L 2 norm, then p dis becomes a Gaussian distribution with a mean of x ori and a variance determined by c 1 . Figure 2 (c) portrays samples drawn from p dis when D is the square of the L 2 distance. The samples closely resemble the original images, x ori s. This is attributed to the fact that each sample converges near the Gaussian distribution's mean, which corresponds to the x ori s.The prior distribution p pri encapsulates our assumptions about the distribution of x adv . It is usually set as a constant, which corresponds to a uniform distribution, signifying a noninformative prior.In this paper, while we don't present an intricate choice for p pri , we emphasize that it serves as a principle concept for incorporating prior knowledge about x adv into the sampling process.The product of the distributions Samples drawn from p adv tend to be concentrated in the regions of high density resulting from the product of p vic and p dis , assuming p pri is constant. As is discussed, a robust victim classifier possesses generative capabilities. This means the high-density regions of p vic are inclined to generate images that embody the semantics of the target class. Conversely, the dense regions of p dis tend to produce images reflecting the semantics of the original image. If these highdensity regions of both p vic and p dis intersect, then samples from p adv may encapsulate the semantics of both the target class and the original image. As depicted in Figure 3 (a), the generated samples exhibit traces of both the target class and the original image. From our probabilistic perspective, the tendency of the generated adversarial samples to semantically resemble the target class stems from the generative ability of the victim distribution. 
generating semantics-aware adversarial examples: 
 We propose semantic divergence, denoted by a non-symmetric divergence D sem (x adv , x ori ) := E(x adv ; x ori ), where E( consisting of {t 1 (x ori ), t 2 (x ori ), . . . }. Here, t i ∼ T , and T is a distribution of transformations that do not alter the original image's semantics. In practice, the choice of T depends on human subjectivity related to the dataset. Individuals are able to incorporate their personal comprehension of semantics into the model by designing their own T . For instance, in the case of the MNIST dataset, the transformations could include scaling, rotation, distortion, and cropping, as illustrated in Figure 2(d) .We assume that such transformations do not affect the semantics of the digits in the MNIST dataset. Consequently, our proposed semantic divergence induces the corresponding distance distribution p dis (x adv ; x ori ) ∝ exp(-c 1 • E(x adv ; x ori )), which is exactly the distribution of the energy-based model introduced in formula (3). It is worth noting that the choice of T is subjective, for more details, see section 8 and appendix D.By proposing semantic divergence, we successfully transformed simple distance distributions induced by geometric distances (such as the Gaussian distribution corresponding to the L 2 distance and the Laplace distribution for the L 1 distance) into trainable, data-driven distributions. This data-driven approach enables users to apply data augmentation or integrate supplementary data to characterize semantic distances, based on their personal interpretation of semantics. As a result, it transitions the geometric distance constraints in adversarial attacks to semantic-based constraints.We claim that, given an appropriate T , semantic divergence can surpass geometric distance. Empirically, when attempting to deceive a robust classifier, it's challenging to limit the geometric distance between the adversarial and original images without leaving traces of the adversarial attack, as depicted in Figure 1 and Figure 3 . The attacked images either display a 'shadow' of the target digits or reveal conspicuous tampering traces, such as in Figure 3(c) , where the attacked digit turns gray. This phenomenon was empirically observed and tested by Song et al. (2018) through an A/B test.Conversely, the samples from p adv , as shown in Figure 4 , scarcely display any evident signs of an adversarial attack. While semantic divergence can't entirely prevent the generation of a sample resembling the target class, as shown in Figure 4 (a), we discuss certain techniques to mitigate this issue in Section 5.1.A plausible interpretation is that, when using L 1 or L 2 distances, depending on a geometric distancebased distribution causes p dis (•, x ori ) to cluster tightly around x ori . This results in a minimal overlap between p dis and p vic , with the overlapping region located in the tail areas of the Gaussian or Laplace distribution. Consequently, image samples manifest a combination of Gaussian noise and features from the victim distribution. This mixture often gives rise to discernible "shadows" or evident markers of unnatural alterations. However, when using a distribution based on a proper semantic divergence, the overlap between p dis and p vic expands, occurring in regions that retain the desired semantics, while still effectively deceiving the classifier.  
techniques for deceiving robust classifiers: 
 In this paper, we introduce four techniques that enhance the performance of our proposed method in generating high-quality adversarial examples. Due to space constraints, we detail only two of these techniques in this section. For the remaining two techniques, please refer to Appendix E.The victim distribution p vic ∝ exp(c 2 • f (x adv , y tar )) is influenced by the choice of function f . Let g ϕ : [0, 1] n → R |Y| be a classifier that produces logits as output with ϕ representing the neural network parameters, n denoting the dimensions of the input, and Y being the set of labels (the output of g ϕ are logits). Szegedy et al. (2013) suggested using cross-entropy as the function f , which can be expressed asf CE (x, y tar ) := -g ϕ (x)[y tar ] + log y exp(g ϕ (x)[y]) = -log σ(g ϕ (x))[y tar ]where σ denotes the softmax function.Carlini & Wagner (2017) explored and compared multiple options for f . They found that, empirically, the most efficient choice of their proposed f s is:f CW (x, y tar ) := max(max y̸ =ytar g ϕ (x)[y] -g ϕ (x)[y tar ], 0).From Figure 3 and Figure 4 , we observe that f CW outperforms f CE when the p dis depends on either geometric distance or semantic divergence. A potential explanation for this phenomenon is that, according to its definition, f CW becomes 0 if the classifier is successfully deceived during the iteration process. This setting ensures that the generator does not strive for a relatively high softmax probability for the target class; it simply needs to reach a point where the victim classifier perceives the image as belonging to the target class. Consequently, after the iteration, the victim classifier assigns a relatively low predictive probability to the target class σ(g ϕ (x adv ))[y tar ], as demonstrated in Figure 3 (d) and Figure 4(d) .In this study, we introduce two additional choices for the function f . Although these alternatives are not as effective as f CW , we present them in Appendix I for further exploration.Thin-plate-spline (TPS) (Bookstein, 1989 ) is a commonly used image deforming method. Given a pair of control points and target points, TPS computes a smooth transformation that maps the control points to the target points, minimizing the bending energy of the transformation. This process results in localized deformations while preserving the overall structure of the image, making TPS a valuable tool for data augmentation. P sou P tarx ori t TPS (x ori , P sou , P tar )Figure 5 : TPS as a data augmentation.Left: The original image x ori superimposed with a 5 × 5 grid of source control points P sou . Right: The transformed image overlaid with a grid of target control points P tar .As introduced in Section 4, we aim to train an energybased model on transformations of a single image x ori .In practice, if the diversity of the augmentations of x ori , represented as t(x ori ), is insufficient, the training of the probabilistic generative model is prone to overfitting.To address this issue, we use TPS as a data augmentation method to increase the diversity of t(x ori ). For each x ori , we set a 5 × 5 grid of source control points,P sou = {(x (i) , y (i) )} 5×5 i=1, and defining the target points asP tar = {(x (i) + ϵ (i) x , y (i) + ϵ (i) y )} 5×5 i=1, where ϵ(i)x , ϵ(i)y ∼ N (0, σ 2 ) are random noise added to the source control points. We then apply TPS transformation to x ori with P sou and P tar as its parameters. This procedure is depicted in Figure 5 . By setting an appropriate σ, we can substantially increase the diversity of the one-image dataset while maintaining its semantic content. 
implementation: 
 We implemented our proposed semantics-aware adversarial attack on three datasets: MNIST, SVHN and CIFAR10. For MNIST, the victim classifier we used was an adversarially trained MadryNet (Madry et al., 2017) . For SVHN and CIFAR10, we utilized an adversarially trained ResNet18, in accordance with the methodology outlined by Song et al. (2018) . On the distance distribution side, for every original image x ori , we trained an energy-based model on the training set, which is represented as {t 1 (x ori ), t 2 (x ori ), . . . }. In this case, t i follows a distribution of transformations, T , that do not change the semantics of x ori . For MNIST, we characterized T MNIST as including Thin Plate Spline (TPS) transformations, scaling, and rotation. For SVHN, we defined T SVHN as comprising TPS transformations and alterations in brightness and hue. For CIFAR10, T CIFAR10 contains TPS only. Detailed specifics related to our implementation can be found in Appendix G. MNIST / SVHN Our method generates adversarial samples that can deceive classifiers, but it does not guarantee the preservation of the original image's semantic meaning. As such, we consider an adversarial example successful if human annotators perceive it as having the same meaning as the original label, in line with the approach by Song et al. (2018) . In detail, we randomly select 10 digits, each representing a different class, from the MNIST/SVHN test set to serve as the original image x ori . These are depicted on the left side of Figure 1 . For each x ori , we iterate through the target class y tar ranging from 0 to 9, excluding the class y ori that signifies the ground-truth label of x ori . For every pair of x ori and y tar , we generate N = 100 adversarial examples post sample refinement. The result of each pair is illustrated in Figure 6 . The overall success rate is illustrated in Table 1 . The detail of calculating the success rate is in Appendix F.CIFAR10 As illustrated in Figure 1 , for CIFAR10, both PGD and our approach generate adversarial examples that largely maintain the original image's semantics. To emphasize the superiority of our method, we undertake a human-driven comparison, juxtaposing images from PGD-L 2 and our method, as well as PGD-L ∞ and our method. In detail, by selecting an appropriate value for ϵ, we ensure that in most cases, both PGD attack variants can deceive the classifier, as demonstrated in Figure 1 . Under these circumstances, we extract examples that successfully mislead the classifier. We then pair images produced by the PGD attack with those generated by our method and present them to 10 annotators. They are tasked with determining which image appears more natural (i.e., without signs of computational tampering). The interface used by the annotators can be found in Appendix H. When comparing our method to PGD-L ∞ , annotators found that 96.3% of samples produced by our technique were more similar to real images. Likewise, in the comparison between our method and PGD-L 2 , 87.4% of the time, annotators felt that our method's samples were more akin to real images.Table 2 and Table 3 shows the transferability of our proposed method. We attack adversarially trained Madry Net with our method and feed legitimate unrestricted adversarial examples, as verified by annotators, to other classifiers. To the best of our knowledge, Vinker et al. (2021) were the first to employ TPS as a data augmentation method. They utilized TPS as a data augmentation strategy in their generative model for conditional image manipulation based on a single image. 
discussion: 
 MNIST and SVHN are not 'easy' At first glance, MNIST and SVHN might appear simpler due to their low resolution and basic structure, especially when compared to natural images. Contrary to this initial impression, they pose a greater challenge than natural images. This is because the digit data lacks diversity. Consequently, robust classifiers can easily memorize the distinct shape of each digit, making them particularly resistant to attacks, as illustrated in Figure 2 (b). As depicted in Figure 1 , more significant transformations are required to successfully deceive the classifiers on MNIST and SVHN. There remains a discernible difference in the generation quality between EBMs and other popular probabilistic generative models, such as variational autoencoders and diffusion models. As a result, our current framework struggles to produce adversarial samples for higher-resolution images. However, given the ongoing advancements in deep learning and generative models, we are optimistic that this challenge will be addressed in the near future. 
conclusion: 
 In this work, we present a novel probabilistic perspective on adversarial examples. Building on this probabilistic perspective, we introduce semantic divergence as an alternative to the commonly used geometric distance. We also propose corresponding techniques for generating semantically-aware adversarial examples. Human participation experiments indicate that our proposed method can often deceive robust classifiers while maintaining the original semantics of the input, although not in all cases. We believe that our new perspective and methodology will pave the way for a fresh paradigm in adversarial attacks. 
 
 Appendix: A.1 LANGEVIN MONTE CARLO (LMC)Langevin Monte Carlo (also known as Langevin dynamics) is an iterative method that could be used to find near-minimal points of a non-convex function g (Raginsky et al., 2017; Zhang et al., 2017; Tzen et al., 2018; Roberts & Tweedie, 1996) . It involves updating the function as follows:EQUATIONwhere p 0 could be a uniform distribution. Under certain conditions on the drift coefficient ∇ x g, it has been demonstrated that the distribution of x t in (6) converges to its stationary distribution (Chiang et al., 1987; Roberts & Tweedie, 1996) , also referred to as the Gibbs distribution p(x) ∝ exp (-g(x) ). This distribution concentrates around the global minimum of g (Gelfand & Mitter, 1991; Xu et al., 2018; Roberts & Tweedie, 1996) . If we choose g to be E θ , then the stationary distribution corresponds exactly to the EBM's distribution defined in (3). As a result, we can draw samples from the EBM using LMC. By replacing the exact gradient with a stochastic gradient, we obtain Stochastic Gradient Langevin Dynamics (SGLD) (Welling & Teh, 2011; Teh et al., 2016) .To train an EBM, we aim to minimize the minus expectation of the log-likelihood, represented byL EBM = E X∼p d [-log p θ (X)] = E X∼p d [E θ (X)] -log Z θwhere p d is the data distribution. The gradient isEQUATION(see (Song & Kingma, 2021) for derivation). The first term of ∇ θ L EBM can be easily calculated as p d is the distribution of the training set. For the second term, we can use LMC to sample from p θ (Hinton, 2002) .Effective training of an energy-based model (EBM) typically requires the use of techniques such as sample buffering and regularization. For more information, refer to the work of Du & Mordatch (2019) .Theorem. Given the condition that p vic (x adv ; y tar ) ∝ exp(-c 2 • f (x adv , y tar )), p dis (x adv ; x ori ) ∝ exp(-c 1 • D(x ori , x adv )), and set p pri (x adv ) as a constant, the samples drawn from p adv will exhibit the same distribution as the adversarial examples derived from applying the box-constrained Langevin Monte Carlo method to the optimization problem delineated in equation (1).Proof. Lamperski (2021) introduced the Projected Stochastic Gradient Langevin Algorithms (PS-GLA) to address box-constraint optimization problems. By leveraging the PSGLA, we can generate samples close to the solution of the optimization problem as stated in Equation ( 1). This leads us to the following update rule:EQUATIONwhere Π[0, 1] n is a projection that clamps values within the interval [0, 1] n . According to Lamperski (2021) , samples generated via this update rule will converge to a stationary distribution, which can be termed the Gibbs distribution p gibbs : p gibbs (x adv ; y tar ) ∝ exp(-L(x adv , y tar ))∝ exp(-c 1 • D(x ori , x adv ) -c 2 • f (x adv , y tar )) • 1 ∝ exp(-c 1 • D(x ori , x adv )) • exp(-c 2 • f (x adv , y tar )) • 1 ∝ p dis (x adv ; x ori )p vic (x adv ; y tar )p pri (x adv )which matches the form of p adv . It is a well-established fact that random variables with identical unnormalized probability density functions share the same distribution. In this paper, the distributions we often refer to (e.g., p adv , p vic , p dis , and p pri ) typically take the unnormalized form exp(-c • g(x)) or a constant. Sampling from the product of these distributions can be efficiently done using Langevin Monte Carlo, specifically its box-constraint variant, PSGLA. For instance, consider sampling from a distribution p(x) ∝ exp(-c 1 g 1 (x)) exp(-c 2 g 2 (x)). This can be reformulated as p(x) ∝ exp(-c 1 g 1 (x) -c 2 g 2 (x)). Notably, this distribution mirrors the Gibbs distribution that results from applying PSGLA to the potential L = -c 1 g 1 (x) -c 2 g 2 (x). Thus, samples can be obtained using the update rule in equation ( 8) with L = -c 1 g 1 (x) -c 2 g 2 (x).In Section 4, we mentioned that the distribution of transformations, T , is influenced by human subjectivity. It's worth noting that what one individual perceives as a non-semantically-altering transformation might not be acceptable to another. For instance, as illustrated in Figure 7 , one person might believe that the transformation on the left doesn't hinder their ability to discern the object's semantics and finds this degree of TPS transformation acceptable. However, another individual might perceive the left transformation as overly distorted, and only the more subtle TPS transformation on the right preserves the unchanged semantics. Thus, in practice, it's essential to select a transformation that not only maintains semantics but is also widely accepted by the general populace.E.1 REJECTION SAMPLING Directly sampling from p adv (•; x ori , y tar ) does not guarantee the generation of samples capable of effectively deceiving the classifier. To overcome this issue, we adopt rejection sampling (Von Neumann, 1951) , which eliminates unsuccessful samples and ultimately yields samples from p adv (x adv | arg max y g ϕ (x adv )[y] = y tar ; x ori , y tar ).After rejection sampling, the samples are confirmed to successfully deceive the classifier. However, not all of them possess high visual quality, as demonstrated in Figure 4 (c). To automatically obtain N semantically valid samples 1 , we first generate M samples from the adversarial distribution. Following rejection sampling, we sort the remaining samples and select the top κ percent based on the softmax probability of the original image's class, as determined by an auxiliary classifier. Finally, we choose the top N samples with the lowest energy E, meaning they have the highest likelihood according to the energy-based model.The auxiliary classifier is trained on the data-augmented training set. We do not use the energy of the samples as the sole criterion for selection because some low-visual quality samples may also have a high likelihood. This occurrence is further explained and examined in Appendix J. The entire process of rejection sampling and sample refinement is portrayed in Algorithm 1.Input: A trained energy based model E(•; x ori ) based on the original image x ori , the victim classifier g ϕ , an auxiliary classifier g ψ , number of initial samples M , number of final samples N , the percentage κ. Output: N adversarial samples x.x = ∅ for 0 ≤ i < M do x adv ∼ p adv (•; x ori , y tar ) ▷ Sample from the adversarial distribution. if arg max y g ϕ (x adv )[y] = y tar then ▷ Accept if x adv deceive the classifier.x = x ∪ {x adv } end if end for Sort x by σ(g ψ (x i ))[y ori ] for i ∈ {1, . . . , |x|} in descent order x = (x i ) ⌊κ|x|⌋ i=1 ▷ Select the first κ percent elements from x. Sort x by E(x i ; x ori ) for i ∈ {1, . . . , |x|} in ascent order x = (x i ) N i=1 ▷ Select the first N elements from x.To enhance the signal-to-noise ratio, we assign the same image to five different annotators and use the majority vote as the human decision, as done in (Song et al., 2018) . The screenshot of the annotator's interface is in Appendix H.In detail, we begin with an original image x ori , its label y ori , and a target class y tar . We draw M = 2000 samples from p adv (•; x ori , y tar ), rejecting those that fail to deceive the victim classifier. After sample refinement, we obtain N = 100 adversarial examples, x (i) adv for i ∈ {1, . . . , N }. We express the human annotators' decision as function h and derive the human decision y(i) hum = h(x (i) adv ).As previously mentioned, an adversarial example x(i) adv is considered successful if y (i)hum is equal to y ori . We then compute the success rate s as follows:s = N i=1 1(y (i) hum = y ori ) Nwhere 1 represents the indicator function.In this section, we delve into the specifics of this study's implementation. The related source code has been provided within the supplementary materials. In case of any remaining uncertainties, please refer to the included source code for further clarity.We utilize two distinct classifiers as the victim classifiers, each with a specific dataset: MadryNet (Madry et al., 2017) with MNIST, and ResNet18 (He et al., 2016) with the SVHN and CIFAR10 dataset.We utilize MadryNet (Madry et al., 2017) , a convolutional neural network (CNN), as the victim classifier for the MNIST dataset. The MadryNet model is trained using the Adam optimizer, with a learning rate set at 10 -4 . Our training regimen comprises 14 epochs, with a batch size of 64. Any additional hyperparameters are retained at their default settings as prescribed by PyTorch.During adversarial training on MadryNet, we implement a Projected Gradient Descent (PGD) untargeted attack on the training data, using the parameters ϵ = 0.3, α = 0.036, and 10 steps.In the case of the SVHN and CIFAR10 dataset, we employ ResNet18 (He et al., 2016) , another well-known architecture. The ResNet18 model is trained using the Adam optimizer, with a learning rate set at 10 -4 . Our training regimen comprises 14 epochs, with a batch size of 64. Any additional hyperparameters are retained at their default settings as prescribed by PyTorch. Similarly to the MadryNet, adversarial training on ResNet18 also involves a PGD untargeted attack on the training data, but with different parameters: ϵ = 0.03, α = 0.01, and 10 steps.For the MNIST dataset, we utilize a specialized convolutional neural network with the undermentioned architecture:• The model commences with a 2D convolutional layer employing 64 filters of 5x5 kernel size, a stride of 2, and a larger padding of 4, effectively augmenting the input image size to 32x32. A 'Swish' activation function is then invoked to incorporate non-linearity.• The second layer consists of another convolutional layer using 128 filters of 3x3 size, with a stride of 2 and padding of 1, followed by the 'Swish' activation function.• The third layer is a replica of the previous one but escalates the filter count to 256 while preserving the filter size, stride, and padding, followed by a 'Swish' activation.• The fourth convolutional layer utilizes 256 filters, similar to the third layer, with a 3x3 kernel size, stride of 2, and padding of 1. This is succeeded by a 'Swish' activation function.• Post convolution, the output undergoes flattening to eliminate spatial dimensions.• The flattened output is then passed through a fully connected layer with 256 units, followed by the 'Swish' activation function.• The architecture culminates with a second fully connected layer mapping the 256 units to a determined output size. This size usually correlates to the number of classes in a classification task or the desired output size in regression tasks.The neural network training employs the Adam optimizer with a learning rate of 10 -4 , batch size of 128, and 200 epochs.For the SVHN and CIFAR10 dataset, we utilize the WideResNet structure, specifically, the WRN-28-10 variant. Training the WRN-28-10 also involves the Adam optimizer with a learning rate of 10 -4 , batch size of 128, and spans across 50 epochs.All of our experiments were conducted on a machine equipped with an Intel E5-2680 v3 CPU and an NVIDIA RTX 3090 GPU. During our experiments, we observed that high likelihood samples do not invariably exhibit high visual quality. This phenomenon is showcased in Figure 12 , where, despite being sorted by energy (a parameter proportional to likelihood), the earliest samples do not always deliver high visual quality. Further, we empirically identified a pattern: high likelihood samples that possess low visual quality often correspond to a low softmax probability for class y ori (the label of the original image).Leveraging this observation, we decided to retain only the top few percentile of samples that have the highest softmax probability for class y ori within an auxiliary classifier, and then sort the remaining samples by energy. As currently implemented, EBMs (Du & Mordatch, 2019) do not support high-resolution inputs; therefore, we downscale ImageNet images from their typical size of 224 × 224 pixels to 128 × 128. Our empirical results suggest that this resolution constraint is a limitation of the current EBM implementations, which we discussed in Section 8.To reduce the distortion induced by TPS transformations, we employ the original image as the starting point for the Langevin Monte Carlo process. Surprisingly, this method yields significantly better results than the random noise initialization previously utilized. The visual outcomes are illustrated in Figures 18, 19, and 20 . By comparing these three images, we can discern that our method leaves fewer traces of tampering.Corresponding The present study introduces a novel approach: the semantics-aware adversarial attack. This method provides significant insights into the resilience and vulnerability of sophisticated classifiers.From an advantageous perspective, it highlights the inherent risks associated with robust classifiers. By exposing potential weak points in such systems, the study underscores the necessity for further improvements in classifier security. This can pave the way for building more resilient artificial intelligence systems in the future.Conversely, the work also presents potential pitfalls. There is a risk that malicious entities might exploit the concepts discussed here for nefarious purposes. It is crucial to take into account the potential misuse of this semantics-aware adversarial attack and accordingly develop preventive measures to deter its utilization for unethical ends.In practice, we could select adversarial samples by hand, but we focus on automatic selection here. 


Decision: This paper proposed a way to extend the pixel-level adversarial examples to construct semantics-aware adversarial examples. Authors did good job on rebuttal and all reviewers provide positive scores for this paper. AC read all rebuttal and reviews. AC think all comments are reasonable and think this paper should be accepted to NeurIPS.

Reviews: review: paper_summary: paper_summary: This paper introduces a probabilistic framework for generating adversarial examples, focusing on maintaining the semantic integrity of the original images while implementing substantial pixel-level modifications. Unlike conventional adversarial techniques that rely heavily on minimal geometric perturbations, this approach integrates a semantic understanding into the adversarial example generation process, leveraging energy-based models and diffusion models. The core innovation lies in embedding the semantic interpretation as a probabilistic distribution, which guides the adversarial example generation. This allows for effective deception of classifiers, including those equipped with adversarial defenses, while preserving the semantic content to an extent that remains imperceptible to human observers. Empirical evaluations demonstrate that the proposed method outperforms existing techniques in terms of both effectiveness against defenses and undetectability by humans, establishing a new paradigm for constructing robust and stealthy adversarial attacks.
strength_weakness: strength_weakness: 1. The assumption in Equation 4 lacks a detailed derivation, leaving it unclear whether $x_{\text{ori}}$ and $y_{\text{tar}}$ need to be independent. Providing a clear derivation and clarifying this assumption would enhance the theoretical rigor of the paper.

2. The training process for the diffusion models is sensitive and requires careful parameter tuning. The paper does not provide enough detail on this sensitivity or potential solutions to mitigate training instability, which impacts the robustness and reproducibility of the method.

3. The paper does not report standard deviations in the performance results. Repeating the experiments is recommended to ensure the reliability and consistency of the findings.
questions: questions: Please address the aforementioned concerns and questions.
limitations: limitations: Paper limitations can be found in the above comments.
paper_summary: paper_summary: This paper proposes a new type of adversarial attack, which generates adversarial examples by solving a box-constrained non-convex optimization problem. Different from the traditional norm-bounded attacks, this paper focuses on unrestricted adversarial attacks by replacing the geometrical distance measure with a semantically-aware distance measure. Specifically, The authors propose using a Langevin Monte Carlo (LMC) technique to sample adversarial examples from a probabilistic distribution. To preserve semantics, the authors use a learned energy function to guide the generation of adversarial samples. Following this, rejection sampling and refinement techniques are employed to select and improve the quality of the generated samples. Experiments show that this attack can fool classifiers while preserving the semantic information compared to baseline methods.
strength_weakness: strength_weakness: 1. Selecting 20 images from each class in the MNIST test set seems to be too little. I understand that it might be infeasible for human annotators to annotate all adversarial images for the entire MNIST, so I would encourage authors to report the success rate except for human annotations using the entire MNIST. I believe this will make the results more convincing.
2. This paper is missing ablation studies for rejection sampling and sample refinement techniques. Is it necessary to include these techniques? How would it affect the attack success rate if one of them is removed?
3. This paper proposes a new attack method but lacks a discussion on how to defend against it. Although it is not compulsory, I am more willing to see how to defend this attack. Can you provide some intuitions on it?
4. Standard deviations are not reported in this paper. Repeated experiments are encouraged.
questions: questions: Please refer to the **Weaknesses**.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper tackles the field of adversarial image generation by proposing an unrestricted attack method that can be applied to both targeted and untargeted attacks. The innovative approach considers a probabilistic perspective, treating the victim classifier and geometric constraints as distinct distributions. By drawing adversarial examples from the overlap region, the authors ensure that the semantics of the original image are preserved. The efficacy of this proposed approach is convincingly demonstrated through extensive experiments.
strength_weakness: strength_weakness: While the experimental results of the proposed method show promise, I do believe there is room for improvement. Specifically, I think it would be beneficial for the authors to provide more detailed information regarding the human experiment methodology, such as how the five reviewers for the MNIST experiment were selected. Furthermore, I would suggest that the authors consider conducting a follow-up experiment where human annotators are asked to identify perturbed images in the absence of a reference image for the ImageNet experiment, which is a more realistic scenario in an attack setting.

In addition, I find it intriguing that both NCF and cAdv demonstrated higher success rates in generating adversarial examples compared to the proposed method, as shown in Table 2. This highlights some shortcoming in the proposed approach. While it is expected that NCF would generate images that can be identified as perturbed, I am more surprised that cAdv was able to create perturbed images that are highly similar to the original ones.

Lastly, I think it would be beneficial for the authors to explore targeted attacks on ImageNet, given the success of this approach in previous papers such as "Towards Transferable Targeted Attack". This could provide valuable insights into the robustness and effectiveness of the proposed method.
questions: questions: 1. How were the values of c chosen in Table 2?
2. What are the author's thoughts on the tradeoff between choosing different values of c?
3. How were the human annotators selected?
4. How was the set \tau chosen?
limitations: limitations: N/A

Comments: title: Response to author rebuttal
comment: Thank you for your feedback. We are dedicated to continuously improving the quality of this paper.
Thank you for the feedback and the increased confidence score. We are pleased that our rebuttal has clarified the major concerns.
I thank the authors for responding to my comments and concerns. Some of the questions raised by me have been answered, however not all. Furthermore, I see that other reviewers have raised some valid concerns as well. Thus I have decided not to update my scores.
I would like to thank the authors for their thorough rebuttal. My major concerns have been well-addressed. I am now more confident that this paper should be accepted. I have increased my confidence score from 3 to 4.