Title: Constructing Semantics-Aware Adversarial Examples with Probabilistic Perspective
Venue: NeurIPS 2023
Accepted: no
Keywords: Adversarial examples

Energy-based model
Abstract: In this study, we introduce a novel, probabilistic viewpoint on adversarial examples, achieved through box-constrained Langevin Monte Carlo (LMC). Proceeding from this perspective, we develop an innovative approach for generating semantics-aware adversarial examples in a principled manner. This methodology transcends the restriction imposed by geometric distance, instead opting for semantic constraints. Our approach empowers individuals to incorporate their personal comprehension of semantics into the model. Through human evaluation, we validate that our semantics-aware adversarial examples maintain their inherent meaning. Experimental findings on the MNIST and SVHN datasets demonstrate that our semantics-aware adversarial examples can effectively circumvent robust adversarial training methods tailored for traditional adversarial attacks.

1

Abstract

In this study, we introduce a novel, probabilistic viewpoint on adversarial examples, achieved through box-constrained Langevin Monte Carlo (LMC). Proceeding from this perspective, we develop an innovative approach for generating semantics-aware adversarial examples in a principled manner. This methodology transcends the restriction imposed by geometric distance, instead opting for semantic constraints. Our approach empowers individuals to incorporate their personal comprehension of semantics into the model. Through human evaluation, we validate that our semantics-aware adversarial examples maintain their inherent meaning. Experimental findings on the MNIST and SVHN datasets demonstrate that our semanticsaware adversarial examples can effectively circumvent robust adversarial training methods tailored for traditional adversarial attacks.

1 Introduction

The purpose of generating adversarial examples is to deceive a classifier by making minimal changes 13 to the original data's meaning. In image classification, most existing adversarial techniques ensure the 14 preservation of adversarial example semantics by limiting their geometric distance from the original 15 image [18, 6, 2, 12]. These methods are able to deceive classifiers with a very small geometric based 16 perturbation. However, when targeting robust classifiers trained using adversarial methods, an attack 17 involving a relatively large geometric distance may be necessary. Unfortunately, these considerable 18 distances can be so vast that they ultimately undermine the original image's semantics, going against 19 the core objective of creating adversarial examples. As illustrated in the left portion of Figure 1, when 20 applying the PGD attack [12] constrained by L 2 norm on a robust classifier, the attacked images that 21 successfully deceive the classifier consistently lose their original meaning, which is undesirable. 22

To counter this problem, we propose an innovative approach for generating semantics-aware adver23 sarial examples. Instead of being limited by geometric distance, our approach hinges on a proposed 24 semantic divergence. Specifically, we treat generating adversarial examples as a box-constrained 25 non-convex optimization problem. We employ box-constrained Langevin Monte Carlo (LMC) to 26 find near-optimal solutions for this complex problem. As LMC samples converge to a stationary 27 distribution, we gain a probabilistic understanding of the adversarial attack. Within this probabilistic 28 perspective, the geometric constraint of the adversarial attack can be viewed as a distribution. By 29 replacing this geometric-based distribution with a semantic-based distribution, we can define a 30 semantics-aware adversarial attack in a principled manner. The corresponding divergence induced by 31 the semantic-based distribution is called semantic divergence. Our semantics-aware adversarial attack 32 is capable of deceiving robust classifiers while preserving most of the original image's semantics, as 33 demonstrated in the right section of Figure 1. 34

35

Source

PGD with

L

2

norm

Our method

Target

0

1

2

3

4

5

6

7

8

9

0

1

2

3

4

5

6

7

8

9

Target

0

1

2

3

4

5

6

7

8

9

Source

0

1

2

3

4

5

6

7

8

9

D

7

２３45６7８

2

3

E

2

2

2

Z

2

3

3

3

E

1

D

5

8

3

S

二

9

白

3

y

3

8

S

q

0

3

3

4

6

8

1

3

D

5

6

2

8

2

1

2

8

2

3

5

16

8

54

7.

3

9

6

3

G

S

66666

66分

3

H

2

2 Preliminaries

2.1 Adversarial examples 36

The notion of adversarial examples was first introduced by Szegedy et al. [18]. Let's assume we have 37

label space. Given an image x ori ∈ [0 , 1] n and a target label y tar ∈ Y , the optimization problem for 39

a classifier C : [0 , 1] n →Y , where n represents the dimension of the input space and Y denotes the 38

finding an adversarial instance for x ori can be formulated as follows: 40

images. This distance metric typically relies on geometric distance, which can be represented by L 0 , 42

Here, D is a distance metric employed to assess the difference between the original and perturbed 41

L 2 , or L ∞ norms. 43

However, solving this problem is challenging. As a result, Szegedy et al. [18] propose a relaxation of 44

where c 1 , c 2 are constants, and f is an objective function closely tied to the classifier's prediction. For 46 example, in [18], f is the cross-entropy loss function, while Carlini and Wagner [2] suggest several 47 different choices for f . Szegedy et al. [18] recommend solving (1) using box-constrained L-BFGS. 48

49

2.2 Adversarial training

Adversarial training, a widely acknowledged method for boosting adversarial robustness in deep 50 learning models, has been extensively studied [18, 6, 10, 12]. This technique uses adversarial samples 51 as (part of) the training data, originating from Szegedy et al. [18], and has evolved into numerous 52 variations. In this paper, we apply the min-max problem formulation by Madry et al. [12] to determine 53 neural network weights, denoted as θ . They propose choosing θ to solve: 54

where p data represents the data distribution, L CE is the cross-entropy loss, ‖·‖ p denotes the L p norm, and /epsilon1 specifies the radius of the corresponding L p ball. In what follows, we will use the term 'robust classifier' to refer to classifiers that have undergone adversarial training.

2.3 Energy-based models (EBMs)

An Energy-based Model (EBM) [9, 4] involves a non-linear regression function, represented by E θ , with a parameter θ . This function is known as the energy function. Given a data point, x , the probability density function (PDF) is given by:

2.4 Langevin Monte Carlo (LMC)

where Z θ = ∫ exp( -E θ ( x ))d x is the normalizing constant that ensures the PDF integrates to 1 .

Langevin Monte Carlo (also known as Langevin dynamics) is an iterative method that could be used to find near-minimal points of a non-convex function g [13, 25, 20, 14]. It involves updating the function as follows:

where p 0 could be a uniform distribution. Under certain conditions on the drift coefficient ∇ x g , it has been demonstrated that the distribution of x t in (4) converges to its stationary distribution [3, 14], also referred to as the Gibbs distribution p ( x ) ∝ exp( g ( x )) . This distribution concentrates around the global minimum of g [5, 24, 14]. If we choose g to be -E θ , then the stationary distribution corresponds exactly to the EBM's distribution defined in (3). As a result, we can draw samples from the EBM using LMC. By replacing the exact gradient with a stochastic gradient, we obtain Stochastic Gradient Langevin Dynamics (SGLD) [23, 19].

2.5 Training EBM

To train an EBM, we aim to minimize the minus expected log-likelihood of the data , represented by

where p d is the data distribution. The gradient is

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

(see [16] for derivation). The first term of ∇ θ L EBM can be easily calculated as p d is the distribution of the training set. For the second term, we can use LMC to sample from p θ [9].

Effective training of an energy-based model (EBM) typically requires the use of techniques such as 79 sample buffering and regularization. For more information, refer to the work of Du and Mordatch [4]. 80

(a)

(b)

(c)

(d)

7

2/0

7

2>b

4

6

h

9

5

b

7

5

496

6

6

5

J^

h

13472

3

3 Generating semantics-aware adversarial examples 81

In this section, we introduce a probabilistic approach to understanding adversarial examples. Through 82 this lens, we establish the concept of semantic divergence, offering an alternative to conventional 83 geometric distance. This concept of semantic divergence enables individuals to integrate their unique 84 understanding of semantics into the model, thereby facilitating the creation of semantics-aware 85 adversarial examples. 86

87

3.1 A probabilistic perspective on adversarial examples

LMC and SGLD are not directly applicable to the optimization problem presented in (1) due to their 88 incompatibility with box-constrained optimization problems. To overcome this limitation, Lamperski 89 [11] proposed Projected Stochastic Gradient Langevin Algorithms (PSGLA). By employing PSGLA 90 to generate samples near the solution of the optimization problem specified in (1), we obtain the 91 subsequent update rule: 92

where Π[0 , 1] n is a clamp projection that enforces the constraints within the [0 , 1] n interval. 93 We refer to the stationary distribution of PSGLA as the adversarial distribution p adv ( x ; y tar ) ∝ 94 exp( -L ( x , y tar )) , since samples drawn from this distribution are in close proximity to the optimal 95 value of the optimization problem presented in (1). 96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

The victim distribution p vic is dependent on the victim classifier. As suggested by Szegedy et al. [18], f could be the cross-entropy loss of the classifier. We can sample from this distribution using Langevin dynamics. Figure 2(a) presents samples drawn from p vic when the victim classifier is subjected to standard training, exhibiting somewhat indistinct shapes of the digits. This implies that the classifier has learned the semantics of the digits to a certain degree, but not thoroughly. In contrast, Figure 2(b) displays samples drawn from p vic when the victim classifier undergoes adversarial training. In this scenario, the shapes of the digits are clearly discernible. This observation suggests that we can obtain meaningful samples from adversarially trained classifiers, indicating that such classifiers depend more on semantics, which corresponds to the fact that an adversarially trained classifier is more difficult to attack. A similar observation concerning the generation of images from an adversarially trained classifier has been reported by Santurkar et al. [15].

The distance distribution p dis relies on D ( x ori , x adv ) , representing the distance between x adv and 112 x ori . By its nature, samples that are closer to x ori may yield a higher p adv , which is consistent with 113

Then by definition of L , the adversarial distribution can be represented as a product of expert distributions [9]:

where p vic ( x adv ; y tar ) ∝ exp( -c 2 · f ( x adv , y tar )) denote the victim distribution and p dis ( x adv ; x ori ) ∝ exp( -c 1 · D ( x ori , x adv )) represent the distance distribution.

(a)

(b)

78

.

3

79

.

6

95

.

4

84

.

1

72

.

9

96

.

0

78

.

0

80

.

8

83

.

8

83

.

8

79

.

0

81

.

1

76

.

1

79

.

4

97

.

0

77

.

2

73

.

8

81

.

4

77

.

7

74

.

5

82

.

4

79

.

3

82

.

2

79

.

7

75

.

6

82

.

4

78

.

8

80

.

6

81

.

1

95

.

5

85

.

0

92

.

2

83

.

6

71

.

2

79

.

7

80

.

9

(c)

(d)

38

.

9

27

.

9

94

.

6

17

.

1

23

.

2

95

.

7

28

.

6

24

.

0

20

.

9

27

.

0

17

.

4

34

.

0

21

.

9

16

.

4

96

.

5

20

.

7

22

.

5

36

.

2

27

.

9

25

.

3

31

.

1

21

.

1

40

.

4

30

.

4

26

.

4

16

.

2

33

.

6

27

.

5

18

.

7

94

.

8

40

.

9

90

.

6

38

.

7

17

.

0

43

.

6

38

.

9

the objective of generating adversarial samples. Moreover, if D represents the square of the L 2 114 norm, then p dis becomes a Gaussian distribution with a mean of x ori and a variance determined by 115 c 1 . Figure 2(c) portrays samples drawn from p dis when D is the square of the L 2 distance. The 116 samples closely resemble the original images, x ori s, from the MNIST testset, because each sample is 117 positioned near an optimal point, and these optimal points are the original images, x ori s. 118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

We claim that, given an appropriate T , semantic divergence can surpass geometric distance. Empirically, maintaining the semantics of the original image by limiting the geometric distance between the adversarial image and the original image when deceiving a robust classifier is challenging: as shown in Figure 1 and Figure 3, it is difficult to preserve the semantics of the original images. The attacked images either display a 'shadow' of the target digits or reveal conspicuous tampering traces, such as in Figure 3(c), where the attacked digit turns gray. This phenomenon was empirically observed and tested by Song et al. [17] through an A/B test. Conversely, as depicted in Figure 4, the samples from p adv neither exhibit the 'shadow' of the target digits nor any obvious traces indicating adversarial attack. While semantic divergence can't entirely prevent the generation of a sample resembling the target class, as shown in Figure 4(a), we discuss certain techniques to mitigate this issue in Section 4.1.

A plausible explanation for this is that the utilization of geometric distance causes p dis ( · , x ori ) to 141 overly focus on x ori . However, when applying semantic divergence induced by a suitable T , the 142 density of the distance distribution p dis ( · , x ori ) spreads out relatively more, resulting in a higher 143 overlap between p dis ( · , x ori ) and p vic . This, in turn, provides more opportunities for their product p adv 144 to reach a higher value. 145

3.2 From Geometric Distance to Semantic Divergence

Based on the probabilistic perspective, we propose a semantic divergence, denoted by a non-symmetric divergence D sem ( x adv , x ori ) := E ( x adv ; x ori ) , where E ( · ; x ori ) represents the energy of an energybased model trained on a dataset consisting of { t 1 ( x ori ) , t 2 ( x ori ) , . . . } . Here, t i ∼ T , and T is a distribution of transformations that do not alter the original image's semantics. In practice, the choice of T depends on human subjectivity related to the dataset. Individuals are able to incorporate their personal comprehension of semantics into the model by designing their own T . For instance, in the case of the MNIST dataset, the transformations could include scaling, rotation, distortion, and cropping, as illustrated in Figure 2(d). We assume that such transformations do not affect the semantics of the digits in the MNIST dataset. Consequently, our proposed semantic divergence induces the corresponding distance distribution p dis ( x adv ; x ori ) ∝ exp( -c 1 · E ( x adv ; x ori )) .

146

147

148

149

150

151

152

153

154

155

156

157

(a)

(b)

75

.

0

80

.

5

78

.

7

78

.

6

79

.

2

80

.

5

77

.

9

81

.

3

78

.

2

75

.

5

78

.

4

80

.

3

77

.

7

77

.

5

79

.

3

73

.

7

78

.

1

73

.

1

82

.

1

69

.

6

86

.

0

63

.

5

76

.

4

80

.

7

77

.

4

79

.

9

74

.

3

83

.

9

74

.

6

79

.

6

78

.

7

79

.

2

81

.

5

84

.

2

77

.

0

77

.

6

(c)

(d)

22

.

7

36

.

9

23

.

9

38

.

5

38

.

6

35

.

6

35

.

3

40

.

5

35

.

8

24

.

5

38

.

6

21

.

5

26

.

3

21

.

9

32

.

9

43

.

0

19

.

9

14

.

6

30

.

1

23

.

6

23

.

8

16

.

6

40

.

3

42

.

6

22

.

9

22

.

9

40

.

5

24

.

3

24

.

1

34

.

0

35

.

3

29

.

1

21

.

8

35

.

8

22

.

2

38

.

8

9

9

9

1

7

1

9

4

7

9

1

3

1

9

9

7

7

3

9

7

4

1

9

4

9

1

?

7

9

4

7

1

4 Deceiving robust classifiers

In this section, we present several techniques that enhance the performance of our proposed method in generating high-quality adversarial examples.

4.1 Victim distributions

The victim distribution p vic ∝ exp( c 2 · f ( x adv , y tar )) is influenced by the choice of function f . Let g φ : [0 , 1] n → R |Y| be a classifier that produces logits as output with φ representing the neural network parameters, n denoting the dimensions of the input, and Y being the set of labels (the output of g φ are logits). Szegedy et al. [18] suggested using cross-entropy as the function f , which can be expressed as where σ denotes the softmax function.

Carlini and Wagner [2] explored and compared multiple options for f . They found that, empirically, the most efficient choice of their proposed f s is:

/negationslash

From Figure 3 and Figure 4, we observe that f CW outperforms f CE when the p dis depends on either 158 geometric distance or semantic divergence. A potential explanation for this phenomenon is that, 159 according to its definition, f CW becomes 0 if the classifier is successfully deceived during the iteration 160 process. This setting ensures that the generator does not strive for a relatively high softmax probability 161 for the target class; it simply needs to reach a point where the victim classifier perceives the image as 162 belonging to the target class. Consequently, after the iteration, the victim classifier assigns a relatively 163 low predictive probability to the target class σ ( g φ ( x adv ))[ y tar ] , as demonstrated in Figure 3(d) and 164 Figure 4(d). 165

166

167

In this study, we introduce two additional choices for the function f . Although these alternatives are not as effective as f CW, we present them in Appendix C for further exploration.

168

4.2 Data Augmentation by Thin Plate Splines (TPS) Deformation

Thin-plate-spline (TPS) [1] is a commonly used image deforming method. Given a pair of control 169 points and target points, TPS computes a smooth transformation that maps the control points to the 170 target points, minimizing the bending energy of the transformation. This process results in localized 171 deformations while preserving the overall structure of the image, making TPS a valuable tool for data 172 augmentation. 173

As introduced in Section 3.2, we aim to train an energy174 based model on transformations of a single image x ori . 175 In practice, if the diversity of the augmentations of x ori , 176 represented as t ( x ori ) , is insufficient, the training of the 177 probabilistic generative model is prone to overfitting. 178 To address this issue, we use TPS as a data augmen179 tation method to increase the diversity of t ( x ori ) . For 180 each x ori , we set a 5 × 5 grid of source control points, 181 P sou = { ( x ( i ) , y ( i ) ) } 5 × 5 i =1 , and defining the target points as 182 P tar = { ( x ( i ) + /epsilon1 ( i ) x , y ( i ) + /epsilon1 ( i ) y ) } 5 × 5 i =1 , where /epsilon1 ( i ) x , /epsilon1 ( i ) y ∼ 183 N (0 , σ 2 ) are random noise added to the source control 184 points. We then apply TPS transformation to x ori with 185 P sou and P tar as its parameters. This procedure is depicted 186 in Figure 5. By setting an appropriate σ , we can substan187 tially increase the diversity of the one-image dataset while 188 maintaining its semantic content. 189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

4.3 Rejection Sampling

Directly sampling from p adv ( · ; x ori , y tar ) does not guarantee the generation of samples capable of effectively deceiving the classifier. To overcome this issue, we adopt rejection sampling [22], which eliminates unsuccessful samples and ultimately yields samples from p adv ( x adv | arg max y g φ ( x adv )[ y ] = y tar ; x ori , y tar ) .

4.4 Sample Refinement

After rejection sampling, the samples are confirmed to successfully deceive the classifier. However, not all of them possess high visual quality, as demonstrated in Figure 4(c). To automatically obtain N semantically valid samples 1 , we first generate M samples from the adversarial distribution. Following rejection sampling, we sort the remaining samples and select the top κ percent based on the softmax probability of the original image's class, as determined by an auxiliary classifier. Finally, we choose the top N samples with the lowest energy E , meaning they have the highest likelihood according to the energy-based model.

The auxiliary classifier is trained on the data-augmented training set. We do not use the energy of the samples as the sole criterion for selection because some low-visual quality samples may also have a high likelihood. This occurrence is further explained and examined in Appendix D. The entire process of rejection sampling and sample refinement is portrayed in Algorithm 1.

Algorithm 1 Rejection Sampling and Sample Refinement

Input: A trained energy based model E ( · ; x ori ) based on the original image x ori , the victim classifier g φ , an auxiliary classifier g ψ , number of initial samples M , number of final samples N , the percentage κ .

P

sou

P

tar

x

ori

t

TPS

(

x

ori

,

sou

,

tar

)

P

P

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

Source

99

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

100

97

100

100

100

100

84

100

100

100

100

100

100

100

100

98

96

97

88

80

99

97

94

93

75

87

99

96

89

98

94

89

83

99

100

97

100

100

100

95

100

97

98

100

100

100

100

100

100

100

100

71

94

99

98

96

69

87

99

98

100

93

93

76

99

71

96

100

99

Target

MNIST

0

1

2

3

4

5

6

7

8

9

0

1

2

3

4

5

6

7

8

9

22

65

66

48

19

79

64

57

73

58

98

95

96

99

87

60

100

31

97

98

99

99

98

97

100

98

100

95

99

100

100

99

97

99

97

100

87

99

94

99

98

93

100

97

89

100

100

100

100

100

100

100

100

100

98

94

91

62

97

99

91

96

92

50

100

4

49

93

54

78

34

7

91

92

100

100

99

98

97

99

100

96

97

98

99

100

100

76

99

96

Target

SVHN

0

1

2

3

4

5

6

7

8

9

5 Experiment

5.1 Implementation

We implemented our proposed semantics-aware adversarial attack on two datasets: MNIST and SVHN. For the MNIST dataset, the victim classifier we used was an adversarially trained MadryNet [12]. For the SVHN dataset, we utilized an adversarially trained ResNet18, in accordance with the methodology outlined by Song et al. [17]. On the distance distribution side, for every original image denoted as x ori , we trained an energy-based model on the training set, which is represented as { t 1 ( x ori ) , t 2 ( x ori ) , . . . } . In this case, t i follows a distribution of transformations, T , that do not change the semantics of x ori . For the MNIST dataset, we characterized T MNIST as including Thin Plate Spline (TPS) transformations, scaling, and rotation. For the SVHN dataset, we defined T SVHN as comprising Thin Plate Spline (TPS) transformations and alterations in brightness and hue. Detailed specifics related to our implementation can be found in Appendix A.

5.2 Evaluation

Our method generates adversarial samples that can deceive classifiers, but it does not guarantee the preservation of the original label's semantic meaning. As such, we consider an adversarial example successful if human annotators perceive it as having the same meaning as the original label, in line with the approach by Song et al. [17]. To enhance the signal-to-noise ratio, we assign the same image to five different annotators and use the majority vote as the human decision, as done in [17]. The screenshot of the annotator's interface is in Appendix B.

In detail, we begin with an original image x ori , its label y ori , and a target class y tar . We draw M = 2000 samples from p adv ( · ; x ori , y tar ) , rejecting those that fail to deceive the victim classifier. After sample refinement, we obtain N = 100 adversarial examples, x ( i ) adv for i ∈ { 1 , . . . , N } . We express the human annotators' decision as function h and derive the human decision y ( i ) hum = h ( x ( i ) adv ) . As previously mentioned, an adversarial example x ( i ) adv is considered successful if y ( i ) hum is equal to y ori

. We then compute the success rate s as follows:

where represents the indicator function. 232

✶ We randomly select 10 digits, each representing a different class, from the MNIST/SVHN test set 233 to serve as the original image x ori . These are depicted on the left side of Figure 1. For each x ori , 234 we iterate through the target class y tar ranging from 0 to 9, excluding the class y ori that signifies 235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

the ground-truth label of x ori . As previously described, for every pair of x ori and y tar , we generate N = 100 adversarial examples post sample refinement. The result of each pair is illustrated in Figure 6. The overall success rate is illustrated in Figure 1.

5.3 Results

As depicted in Figure 6 and Table 1, our proposed method often succeeds in fooling robust classifiers, all the while preserving the original semantics of the input. It should be noted, however, that this does not occur in every instance.

6 Related work

Unrestricted adversarial examples Song et al. [17] proposed generating unrestricted adversarial examples from scratch using conditional generative models. In their work, the term 'unrestricted' indicates that the generated adversarial samples, x adv , are not restricted by a geometric distance such as the L 2 norm or L ∞ norm. The key difference between their approach and ours is that their adversarial examples x adv are independent of any specific x ori , while our model generates x adv based on a given x ori . By slightly modifying (7), we can easily incorporate Song's 'unrestricted adversarial examples' into our probabilistic perspective:

where y sou is the source class. It becomes evident that the adversarial examples generated by our p adv ( · ; x ori , y tar ) adhere to Song's definition when x ori is labeled as y sou .

TPS as a Data Augmentation Technique To the best of our knowledge, Vinker et al. [21] were the first to employ TPS as a data augmentation method. They utilized TPS as a data augmentation strategy in their generative model for conditional image manipulation based on a single image.

7 Limitation

This work's foremost limitation pertains to the inherent difficulties in training energy-based models (EBMs), as underscored in the earlier studies by Du and Mordatch [4] and Grathwohl et al. [7]. The EBM training process is notoriously challenging, and a notable gap persists between the generation quality of EBMs and that of other widely-used probabilistic generative models, such as variational autoencoders and diffusion models. Consequently, we are currently unable to generate adversarial samples for images with higher resolution.

8 Conclusion

In this work, we present a probabilistic perspective on adversarial examples by employing Langevin Monte Carlo. Building on this probabilistic perspective, we introduce semantic divergence as an alternative to the commonly used geometric distance. We also propose corresponding techniques for generating semantically-aware adversarial examples. Human participation experiments indicate that our proposed method can often deceive robust classifiers while maintaining the original semantics of the input, although not in all cases.

270

271

272

273

274

275

276

References

F. L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on pattern analysis and machine intelligence , 11(6):567-585, 1989.

N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39-57. Ieee, 2017.

T.-S. Chiang, C.-R. Hwang, and S. J. Sheu. Diffusion for global optimization in rˆn. SIAM Journal on Control and Optimization , 25(3):737-753, 1987.

Y. Du and I. Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint 277 arXiv:1903.08689 , 2019. 278

S. B. Gelfand and S. K. Mitter. Recursive stochastic algorithms for global optimization in rˆd. SIAM 279 Journal on Control and Optimization , 29(5):999-1018, 1991. 280

I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv 281 preprint arXiv:1412.6572 , 2014. 282

W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky. Your classifier is 283 secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263 , 2019. 284

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the 285 IEEE conference on computer vision and pattern recognition , pages 770-778, 2016. 286

G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation , 14 287 (8):1771-1800, 2002. 288

R. Huang, B. Xu, D. Schuurmans, and C. Szepesvári. Learning with a strong adversary. arXiv preprint 289 arXiv:1511.03034 , 2015. 290

A. Lamperski. Projected stochastic gradient langevin algorithms for constrained sampling and non-convex 291 learning. In Conference on Learning Theory , pages 2891-2937. PMLR, 2021. 292

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to 293 adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017. 294

M. Raginsky, A. Rakhlin, and M. Telgarsky. Non-convex learning via stochastic gradient langevin 295 dynamics: a nonasymptotic analysis. In Conference on Learning Theory , pages 1674-1703. PMLR, 2017. 296

G. O. Roberts and R. L. Tweedie. Exponential convergence of langevin distributions and their discrete 297 approximations. Bernoulli , pages 341-363, 1996. 298

S. Santurkar, A. Ilyas, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Image synthesis with a single 299 (robust) classifier. Advances in Neural Information Processing Systems , 32, 2019. 300

Y. Song and D. P. Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288 , 301 2021. 302

Y. Song, R. Shu, N. Kushman, and S. Ermon. Constructing unrestricted adversarial examples with 303 generative models. Advances in Neural Information Processing Systems , 31, 2018. 304

C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing 305 properties of neural networks. arXiv preprint arXiv:1312.6199 , 2013. 306

Y. W. Teh, A. H. Thiery, and S. J. Vollmer. Consistency and fluctuations for stochastic gradient langevin 307 dynamics. Journal of Machine Learning Research , 17, 2016. 308

309

310

B. Tzen, T. Liang, and M. Raginsky. Local optimality and generalization guarantees for the langevin algorithm via empirical metastability. In Conference On Learning Theory , pages 857-875. PMLR, 2018.

311

[21]

Y. Vinker, E. Horwitz, N. Zabari, and Y. Hoshen.

13769-13778, 2021. 313

training sample. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 312

J. Von Neumann. 13. various techniques used in connection with random digits. Appl. Math Ser , 12(36-38): 314 3, 1951. 315

Image shape manipulation from a single augmented

M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of 316 the 28th international conference on machine learning (ICML-11) , pages 681-688, 2011. 317

P. Xu, J. Chen, D. Zou, and Q. Gu. Global convergence of langevin dynamics based algorithms for 318 nonconvex optimization. Advances in Neural Information Processing Systems , 31, 2018. 319

Y. Zhang, P. Liang, and M. Charikar. A hitting time analysis of stochastic gradient langevin dynamics. In 320 Conference on Learning Theory , pages 1980-2022. PMLR, 2017. 321

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach to adversarial attacks that goes beyond traditional norm bounded attacks. Instead, the proposed method focuses on unrestricted attacks that are both effective and capable of preserving the semantic meaning of the input data.\n\nThe method utilizes Langevin Monte Carlo techniques to sample from a distribution of potential attacks. To ensure semantic preservation, a learned energy function is employed, which guides the generation of adversarial samples. Rejection sampling and refinement techniques are then applied to select and further improve the quality of the generated samples.\n\nThe evaluation of the proposed method demonstrates a significant success rate when attacking defended models. By allowing for unrestricted attacks while maintaining semantic integrity, this approach presents a promising advancement in the field of adversarial attacks, showcasing its effectiveness and potential for practical application.\n\nSTRENGTHS:\n1. Interesting work on unrestricted adversarial attack, which is important given that most attacks now are bounded attack.\n\n2. The method is effective in breaking already defended models. Fig 1,2 clearly shows the advantage over norm bounded attacks.\n\nWEAKNESSES:\n1. What is the computation cost of the attack? The paper only evaluates on two toy datasets, MNIST and SVHN, the reviewer is wondering if the method can generalize to larger dataset.\n\n2. Ablation study on the component is missing. Like TPS as data augmentaion, the effect of the choice of the sampling method. Also the method requires specify several hyper parameters, like M. Ablation study is useful.\n\nQUESTIONS:\nIs the TPS augmentation used to capture the energy function of the semantic?\n\nWould TPS augmentation also work for other type of data, say semantic segmentation, where the location of the pixel matters a lot?\n\nSimilar constraint functions are used for defending adversarial attacks, such as [1,2], but they use the constraint for defense. Can the author discuss if their attack can attack the dynamic defense in [1,2], where the defense will reverse the attack to the benign manifold?\n\n[1] Mao et al. Adversarial Attacks are Reversible with Natural Supervision. ICCV 2021.\n\n[2] Mao et al. Robust Perception through Equivariance. ICML 2023.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe adversarial examples generated by classical methods such as PGD have different semantic meaning to the original label, which means that the adversarial examples are easy to be distinguished by human. In this paper, the authors focus on the generalization of adversarial example which preserves the original semantic information. They propose a semantically-aware distance measure to replace the geometrical distance measure. And they use Langevin Monte Carlo method to find the minimal point (adversarial sample) of their proposed loss function. Several techniques that further enhance the performance of the proposed method are presented. From the experimental results, it seems that their generated examples preserve the original semantical imformation.\n\nSTRENGTHS:\n* As far as I know, the proposed adversarial attack method is novel.\n* They proposed a semantical distance measure to generate the semantic-aware examples. Although the idea of semantical measure already exists in many previous work, I think the usage here in adversarial example generalization scenario is interesting and reasonable.\n* Their method is theoretically and experimentally reliable.\n\nWEAKNESSES:\n* One of the limitation of this paper is that, the loss of semantics of adversarial examples only exists in some simple tasks, such as MNIST and SVHN. As the experimental results in previous work shows, the adversarial examples of CIFAR and ImageNet have very little disturbations that cannot be distinguished by human and preserve the semantical information. Hence, I think the significance of this paper is somewhat limited.\n* The motivation of using EBMs and LMC is not very clear to me. In my opinion, we can directly optimize the semantic-aware loss to generate the adversarial examples. The necessity of using the EBMs and LMC should be stated more clearly.\n* In the experiment part, the success rate involves subjective factors. They use human annotators to determine whether the adversarial examples have the same meaning as the original label. Is there a more subjective metric? Otherwise, the experimental results may suffer a credibility crisis.\n* More experiments on CIFAR-10 and CIFAR-100 are necessary.\n* Can you give a more detailed explaination of the training of the energy-based model? I noticed that Section 2.5 includes some brief introduction, but what is the data distribution $p_d$ here? What is the specific training algorithm?\n\nIf the authors can address my concerns well, I will consider raise the score.\n\nQUESTIONS:\n* I am confused when I read Line 69-70. Should $exp(g(x))$ be replaced by $exp(-g(x))$? Otherwise, the distribution $p(x)$ seems to concentrate around the global maximal.\n* If we use the semantric-aware adversarial examples to adversarially train the model, will the model be robust to the semantric-aware adversarial example?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to generate semantics-preserving adversarial examples by framing the construction of adversarial examples as a box-constrained non-convex optimization problem. More specifically, the authors propose a Langevin Monte Carlo (LMC) technique to craft adversarial examples that preserve the meaning of the original inputs they are derived from. With this framing, they cast the generation of adversarial examples as a semantic-based probabilistic distribution. The authors showed that their semantic-aware adversarial attack is capable of fooling robust classifiers while preserving most of the semantics of their source images.\n\nSTRENGTHS:\nThis paper is quite interesting paper and well-written. The problem is well-defined, and the solution quite intuitive. The math is also quite sound. Although the problem of generating semantics-preserving adversarial examples has been studied extensively in the past, it still remains relevant. This paper proposes another interesting perspective on how to approach this problem.\n\nWEAKNESSES:\nAlthough the paper is interesting, the evaluation is quite limited. For instance, the approach is only evaluated on MNIST and SVHN. Evaluating the approach against \"more challenging\" datasets like ImageNet, CIFAR-10, CIFAR-100 would make their contributions more compelling. Also, studying the transferability property of their attacks would strengthen their paper, and give more confidence to the readers about the strength of their attacks. Moreover, I would have liked to see how the magnitude of the noise used in Thin-plate-spine affects the overall performance of their attacks. Finally, the related work section is rather limited. There is a plethora of interesting studies in crafting adversarial examples that are semantics-preserving. For instance, [1] and [2] are quite related to the approach the authors propose, and should be evaluated or discussed further in the related work section. \n\n[1]: Semantics Preserving Adversarial Examples. https://aisecure-workshop.github.io/amlcvpr2021/cr/27.pdf\n[2]: Localized Uncertainty Attacks. https://ui.adsabs.harvard.edu/abs/2021arXiv210609222A/abstract\n\nQUESTIONS:\nI would highly recommend the authors to further experiment with datasets like ImageNet, CIFAR-10, CIFAR-100, etc., to study the transferability property of their adversarial attacks, and to improve the related work section by comparing their approach against relevant approaches that were proposed in the past.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, a probabilistic view of adversarial examples based on the [projected stochastic gradient Langevin algorithm](https://proceedings.mlr.press/v134/lamperski21a.html) is introduced and used as an optimization algorithm instead of the SGD or Adam optimizer for adversarial examples. In addition, the geometric constraint (Lp norms) is replaced by a semantic distance criterion based on an instance-wise energy-based model (i.e., an EBM is trained for each instance, using transformed versions as the training dataset) to ensure semantic/visual proximity to the original input. They improved the adversarial examples using the [CW objective](https://www.computer.org/csdl/proceedings-article/sp/2017/07958570/12OmNviHK8t) and thin-plate splines transformation to create a more diverse training dataset for EBM training. Moreover, they generated a set of successful adversarial attacks (i.e., fooled the classifier) via rejection sampling and proposed a simple selection procedure to select the final adversarial examples based on the softmax probabilities of an auxiliary classifier and the energy of the examples. The experiments show that the proposed method is able to generate adversarial examples that fool the classifier while being visually/semantically indistinguishable to humans.\n\nSTRENGTHS:\n- The proposed method is very detailed and intricate.\n- The Langevin Monte Carlo-based optimization procedure seems to improve the quality of adversarial examples overall.\n- The paper is well-written and clearly structured.\n- Code is provided.\n\nWEAKNESSES:\n- Previous work, e.g., by [Sharma & Chen](https://openreview.net/forum?id=Sy8WeUJPf), has also generated visually similar adversarial examples for the MadryNet while still using a geometric distance ([elastic-net regularization](https://arxiv.org/abs/1709.04114)). This raises questions about the generality of the work’s central claim that it “transcends the restriction imposed by geometric distance, instead opting for semantic constraints” (L4-5) beyond the limitations of the adversarial attack methods shown in the present work.\n- The present work only shows experiments on digit-based datasets (MNIST & SVHN). Applications to datasets with natural images (e.g., CIFAR or ImageNet) are missing. Consequently, the necessity and applicability of the proposed adversarial attack are very unclear, since for natural images the adversarial examples typically remain visually very close to the original inputs; also after adversarial fine-tuning.\n- The work is missing interesting experiments, e.g., what would happen if we use the proposed adversarial attack approach for adversarial training? Does it improve adversarial robustness? Does the adversarial attack also bypass certified defenses? Overall, the experimental section is very short (3 lines of results) and would greatly benefit from, e.g., the aforementioned experiments.\n- The approach requires an instance-wise energy-based model for its semantic distance loss, which must be trained for every sample (on different augmented versions); cf. L122. This may limit its applicability.\n- The proposed attack and problem setup are not quite original, i.e., it combines well-known techniques, or previous work (see first point above) has also already targeted the visual similarity challenge of adversarial examples for adversarially fine-tuned models.\n\nQUESTIONS:\n- Couldn't we just train a single energy-based model for the specific data domain? If so, how do the generated adversarial examples compare to those using an instance-based energy-based model for semantic divergence?\n- Why do the authors refrain from using (currently) better generative methods?\n- Regarding Fig. 2: are the samples generated using only the classifier (neglecting the distance distribution term) for a & b and vice versa for c?\n\n## Suggestions\n\n- The related work could include more discussion on previous works on adversarial examples.\n- For the minimization problem formulations in Sec. 2.1., it’d be good to include that $x_{adv}$ is minimized (even though it’s obvious given the work’s scope).\n- It’d be meaningful to include error bars for the experiments.\n- Results for Song et al in Tab. 1 should be repeated for better comparability, if possible.\n\nLIMITATIONS:\nThe limitations are adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-16"}