Unrestricted Adversarial Examples via Semantic Manipulation
Authors: Anand Bhattad
Min Jin Chong
Kaizhao Liang
Bo Li
D. A. Forsyth
Venue:  (2020)
Keywords: Adversarial Examples
Semantic Manipulation
Image Colorization
Texture Transfer

Abstract: Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation.  Such adversarial perturbations are usually restricted by bounding their $\mathcal{L}_p$ norm such that they are imperceptible, and thus many current defenses can exploit this property to reduce their adversarial impact.  In this paper, we instead introduce "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors - color and texture - in order to generate effective and photorealistic adversarial examples. We show that these semantically aware perturbations are effective against JPEG compression, feature squeezing and adversarially trained model. We also show that the proposed methods can effectively be applied to both image classification and image captioning tasks on complex datasets such as ImageNet and MSCOCO. In addition, we conduct comprehensive user studies to show that our generated semantic adversarial examples are photorealistic to humans despite large magnitude perturbations when compared to other attacks.

Published as a conference paper at ICLR 2020
UNRESTRICTED ADVERSARIAL EXAMPLES VIA SEMANTIC MANIPULATION
Anand Bhattad Min Jin Chong Kaizhao Liang Bo Li
D. Forsyth
University of Illinois at Urbana- Urbana-Champaa Champaign (bhattad2, mchong6, k12, 1bc, daf)@i11inois.co
ABSTRACT
Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with small magnitude of the perturbation. Such adversarial perturbations are usually restricted by bounding their L. norm such that they are imperceptible, and thus many current defenses can exploit this property to reduce their adversar- ial impact. In this paper, we instead introduce "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors color and texture in order to generate effective and photorealistic adversarial examples. We show that these semantically aware perturbations are effective against JPEG compression, feature squeezing and adversarially trained model. We also show that the proposed methods can effectively be applied both image classification and image captioning tasks on complex datasets such as ImageNet and MSCOCO. addition, we conduct comprehensive user studies to show that our generated semantic adversarial examples are photorealistic to humans despite large magnitude perturbations when compared to other attacks.
INTRODUCTION
Machine learning (ML), especially deep neural networks (DNNs) have achieved great success in various tasks, including image recognition (Krizhevsky et al. 2012 He et al. 2016), speech processing (Hinton al. 2012) and robotics training (Levine et al.) 2016). However, recent literature has shown that these widely deployed ML models are vulnerable to adversarial examples carefully crafted perturbations aiming to mislead learning models (Carlini & Wagner 2017 Kurakin al.] 2016 Xiao et al.] 20188). The fast growth of DNNs based solutions demands in-depth studies on adversarial examples help better understand potential vulnerabilities of MI. models and therefore improve their robustness.
To date, a variety of different approaches has been proposed to generate adversarial examples (Goodfellow et al.) 2014b. Carlini & Wagner, 2017 Kurakin et al. 2016 Xiao et al. 2018a); and many of these attacks search for perturbation within a bounded Lp norm in order to preserve their photorealism. However, it is known that the Ep norm distance a perceptual similarity metric is not ideal (Johnson et al., 2016 Isola et al. 2017). In addition, recent work show that defenses trained on Lp bounded perturbation are not robust all against new types of unseen attacks (Kang et al. 2019).
Therefore, exploring diverse adversarial examples, especially those with "unrestricted" magnitude of perturbation has acquired lot of attention in both academia and industries (Brown et 2018
Recent work based on generative adversarial networks (GANs) (Goodfellow et al. 2014a) have introduced unrestricted attacks (Song el al 2018). However, these altacks are limited datasets like MNIST, CIFAR and CelebA, and are usually unable to scale up to bigger and more complex datasets such as ImageNet. Xiao et al. (2018b) directly manipulated spatial pixel flow of an image to produce adversarial examples without Ep bounded constraints on the perturbation. However, the attack does not explicitly control visual semantic representation. More recently, Hosseini & Poovendran (2018) manipulated hue and saturation of image create adversarial perturbations. However, these examples are easily distinguishable by human and are also not scalable to complex datasets.
indicates equal contributions.

---

Published as a conference paper at ICLR 2020
Colorization Attack (cAdv)
Plyea
Clasoitte
cat sitiing on pole bullding
Colorization Model
Color Hints+Maskk Grayscate
Exaple
Captioner
cAde Pentorbation
Texture Attack (tAdv)
Photocopie
Cleciiier
uumbrella is on pole front ota buikiing
Texture Transfor Model
Naamed Niightors
Scrign
Ledi Enample
Capticnnr
IAdv Pertutbalium
Figure 1: An overview of proposed attacks. Top: Colorization attack (cAdv); Bottom: Texture
transfer attack (tAdv). Our attacks achieve high attack success rate via semantic manipulation without any constraints on the Lp norm of the perturbation. Our methods are general and can be used for attacking both classifiers and captioners.
In this work, we propose unrestricted attack strategies that explicitly manipulate semantic visual representations to generate natural-looking adversarial examples that are "far" from the original image in tems of the Cp norm distance. In particular, manipulate color (cAdv) and texture (tAdv) to create realistic adversarial examples (see Fig T. c.Adv adaptively chooses locations an image to change their colors, producing adversarial perturbation that usually fairly substantial, while t.Adv utilizes the texture from other images and adjusts the instance's 's texture field using style transfer.
These semantic transformation-basel adversarial perturbations shed light upon the understanding of what information is important for DNNs to make predictions. For instance, in one of our case studies, when the road recolored from gray blue, the image gets misclassified tench (a fish) although a car remains evidently visible (Fig. 2p). This indicates that deep learning models can easily be fooled by certain large scale patterns. In addition to image classifiers, the proposed attack methods can be generalized to different machine learning tasks such as image captioning (Karpathy & Fei-Fei (2015)). Our attacks can either change the entire caption the target (Chen et 20.7|/1 et al., 2019) take more challenging tasks like changing one or two specific target words from the caption to a target. For example, in Fig. [ "stop sign" of the original image caption is changed to "cat sitting" and "umbrella is" for cAdv and tAdv respectively.
To ensure our "unrestricted" semantically manipulated images are natural, we conducted extensive user studies with Amazon Mechanical Turk. We also tested our proposed attacks several state of the art defenses. Rather than just showing the attacks break these defenses (better defenses will come up), aim show that cAdv and tAAy are able to produce new types of adversarial examples. Experiments also show that our proposed attacks are more transferable given their large and structured perturbations (Papernot et 20166. Our semantic adversarial attacks provide further insights about the vulnerabilities of ML models and therefore encourage new solutions to improve their robustness.
In summary, contributions are: 1) We propose two novel approaches generate "unrestricted" adversarial examples via semantic transformation; 2) We conduct extensive experiments to attack both image classification and image captioning models on large scale datasets (ImageNet and MSCOCO); 3) We show that our attacks are equipped with unique properties such as smooth cAdv perturbations and structured tAdv perturbations. compared to other attacks, our generated adversarial examples appear more natural to humans despite their large perturbations; 5) We test different adversarial examples against several state of the art defenses and show that the proposed attacks are more transferable and harder to defend.
We perform comprehensive user studies show that when
2 COLORIZATION ATTACK (cADV)
Background. Image Colorization is the task of giving natural colorization to a grayscale image. This is an ill-posed problem there are multiple viable natural colorizations given single grayscale
2

---

Published as a conference paper at ICLR 2020
Merganser
Golfcart
Umbrella
Sandbar
GT
Tench
(a]
(b)
Figure 2: Class color affinity. Samples from unconstrained cAdv attacking network weights with zero hints provided. For (a) the ground truth (GT) class is pret zel; and for (b) the GT is car. These new colors added are commonly found in images from target class. For instance, green in Golfeart and blue sea in Tench images.
image. Deshpande et al. (017)) showed that diverse image colorization can be achieved by using an architecture that combines VAE (Kingma Welling (2013) and Mixture Density Network; while Zhang et al. (0117) demonstrated an improved and diverse image colorization by using input hints from users guided colorization process.
Our goal is to adversarially color an image by leveraging pretrained colorization model. We hypothesize that it is possible to find natural colorization that is adversarial for a target model (e.g.. classifier captioner) by searching in the color space. Since a colorization network learns to color natural colors that conform to boundaries and respect short-range color consistency, we can use to introduce smooth and consistent adversarial noise with a large magnitude that looks natural to humans. This attack differs from common adversarial attacks which tend to introduce short-scale high-freyuency artifacts that are minimized to be invisible for human observers.
We leverage Zhang et al. (2016) 2017) colorization model for our attack. In their work, they produce natural colorizations on ImageNet with input hints from the user. The inputs their network consist of the L channel of the image in CIELAB color space X E _ xxxx11 the sparse colored input hints Xab E R #x x2 and the binary mask M B #xxx1 indicating the location of the hints.
cAdv Objectives. There are few ways to leverage the colorization model to achieve adversarial objectives. We experimented with two main methods and achieved varied results.
Network weights. The straightforward approach of producing adversarial colors is to modify /Zhang ett al. (2017) colorization network, C directly. To do so, we simply update C by minimizing the adversarial loss objective Jadv. which in our case, is the cross entropy loss. / represents target class and FF represents the victim network.
0" = arg Jade (F(C(XL Nab, ()
(1)
Hints and mask. We can also vary input hints Xas and mask M produce adversarial colorizations. Hints provides the network with ground truth color patches that guides the colorization, while the mask provides its spatial location. By jointly varying both hints and mask, we are able to manipulate the output colorization. We can update the hints and mask as follows:
M X
arg min Jadu (F(C(X,, Xas, M)))),t M.X.i
(2)
cAdvAttack Methods. Attacking network weights allows the network to search the color space with no constraints for adversarial colors. This attack is the easiest to optimize, but the output colors are not realistic as shown in Fig. 2 Our various strategies outlined below are ineffective as the model learns to generate the adversarial colors without taking into account color realism. However, colorizations produced often correlate with colors often observed in the target class. This suggests that classifiers associate certain colors with certain classes which we will discuss more our case study.
Attacking input hints and mask jointly gives us natural results as the pretrained network will not be affected by our optimization. Attacking hints and mask separately also works but takes a long optimization time and give slightly worse results. For our experiments, we use Adam Optimizer (Kingma& & Ba (2014)) with learning rate of 10 in cAdv. We iteratively update hints and mask until our adversarial image reaches the target class and the confidence change of consecutive iterations does not exceed threshold of 0.05.
3

---

Published as a conference paper at ICLR 2020
GT
k
k-2
k=4
k=6
k=8
Figure 3: Controlling cAdv. We show comparison of sampling 50 color hints from clusters with low-entropy. All images are attacked to misclassify as golf f-catt. Second and fourth row visualize our cluster segments, with darker colors representing higher mean entropy and red dots representing the sampled hints location. Sampling hints across more clusters gives less color variety.
Control over colorization. Current attack methods lack control over where the attack occurs, opting attack all pixels indiscriminately. This lack of control not important for most attacks where the e is small but is concerning in cAdy where making unstructured large changes can be jarring. To produce realistic colorization, we need avoid making large color changes at locations where colors are unambiguous (e.g. roads in general are gray) and focus on those where colors are ambiguous (e.g. an umbrella can have different colors). To do so, we need to segment an image and determine which segments should be attacked preserved.
To segment the image into meaningful areas, we cluster the image's s ground truth AB space using K-Means. We first use Gaussian filter of o = 3 to smooth the AB channels and then cluster them into 8 clusters. Then, we have to determine which cluster's colors should be preserved. Fortunately, Zhang al. (2017) network output per-pixel color distribution for given image which used to calculate the entropy of each pixel. The entropy represents how confident the network is at assigning color that location. The average entropy of each cluster represents how ambiguous their color is. We want to avoid making large changes to clusters with low-entropy while allowing our attack to change clusters with high entropy. One way to enforce this behavior is through hints, which are sampled from the ground truth at locations belonging to clusters of low-entropy. We sample hints from the & clusters with the lowest entropy which we refer cAdvı (e.g. cAdvz samples hints from the 2 lowest entropy clusters).
Number of input hints. Network hints constrain our output to have similar colors as the ground truth, avoiding the possibility of unnatural colorization at the cost of color diversity. This trade-off is controlled by the number of hints given to the network as initialization (Fig. 4).. Generally, providing more hints gives similar colors that are observed in original image. However, having many hints also problematic. Too many hints makes the optimization between drawing adversarial colors and matching local color hints difficult. Since the search space for adversarial colors is constrained because of more hints, we may instead generate unrealistic examples.
Number of Clusters. The trade-off between the color diversity and the color realism is also controlled by the number of clusters we sample hints from as shown in Fig. B. Sampling from multiple clusters gives realistic colors closer to the ground truth image the expense of color diversity.
4

---

Published as a conference paper at ICLR 2020
GT
O Hints
25 Hints
50 Hints
100 Hints
500 Hints
Figure 4: Number of color hints required for cAdv. All images attacked Merganser with ==4 4. When the number of hints increases (from left to right), the output colors are more similar to groundtruth. However, when the number of hints is too high (500), cAdv often generates unrealistic perturbations. This is due to harder optimization for cAdv to both add adversarial colors and match GT color hints. cAdv is effective and realistic with balanced number of hints.
Empirically, from our experiments we find that in terms of color diversity, realism, and robustness of attacks, using & and 50 hints gives us better adversarial examples. For the rest of this paper, we fix 50 hints for all cAdv; methods.
3 TEXTURE ATTACK (tADV)
Background. Texture transfer extracts texture from one image and adds it to another. Transferring texture from one source image another target image has been widely studied in computer vision (Efros& & Freeman (2001 y Gatys et al. (2055). The Convolutional Neural Network (CNN) based texture transfer from Gatys et (2015) led to series of new ideas in the domain of artistic style transfer (Gatys et al. (2016); Huang & Belongie (2017); > Li et al. (2017): ; Yeh et al. (2019)). More recently, Geirhos et al. (2018) showed that DNNs trained on ImageNet et are biased towards texture for making predictions.
Our goal generate adversarial examples by infusing texture from another image without explicit constraints on Cp norm of the perturbation. For generating our tAdv examples, we used a pretrained VGG19 network (Simonyan & Zisserman 2014) to extract textural features. We directly optimize our victim image (I,) by adding texture from target image (T,). A natural strategy to transfer texture is by minimizing within-layer feature correlation statistics (gram matrices) between two images (Gatys) et al, (2015) 2016). Based on Yeh et al. 22099), we find that optimizing cross-layer gram matrices instead of within-layer gram matrices helps produce more natural looking adversarial examples. The difference between the within-layee and the cross-layer gram matrices is that for within-layer the feature's 's statistics are computed between the same layer. For a cross-layer, the statistics are computed between two adjacent layers.
t.Ad Objectives. t.Adv directly attacks the image to create adversarial examples without modifying network parameters. Moreover, there is no additional content loss that is used in style transfer methods (Gatys et al. (2016): Yeh et al. (2019)). Our overall objective function for the texture attack contains texture transter loss (Zz²)) and an cross-entropy loss (Jadu).
L aLL 3Jade (F(Io),
Unlike style transfer methods, we do not want the adversarial examples to be artistically pleasing. Our goal is to infuse a reasonable texture from a target class image to the victim image and fool a classifier or captioning network. To ensure reasonable texture is added without overly perturbing the vietim image too much, we introduce additional constraint the victim image. This constraint helps us to control the image transformation procedure and prevents it from producing artistic images. Let m and n denote two layers of a pretrained VGG-19 19 with a decreasing spatial resolution and C' for number of filter maps in layer our texture transfer loss is then given by
the variation in the gram matrices of
LA (t)
Σ (M.
1 C
Σ 5j
Cmm. ij atd
(T.)
G
(4)
(G (Io))
5

---

Published as a conference paper at ICLR 2020
CL- (L=250
a=500
a=750
a= 1000
Original
Random T
Original
Targel T.
Nearest Target I
Original
Figure 5: tAdv strategies. Texture transferred from random "texture source" (T₂) row I, random target class T, (row 2) and from the nearest target class T, (row 3). All examples misclassified from Beacon to Naut ilus. Images in the last row look photo realistic, while those in the first two rows contain more artifacts the texture weight a increases (left to right).
Let f be feature maps, fd be an upsampled f" that matches the spatial resolution of layer m. The cross layer gram matrices G between the victim image (T,)) and a target image (I,) is given as
C
(1) E [CC( (1)),(0))*
(S)
Texture Transfer. To create tAdv adversarial examples, we need to find images to extract the texture from, which we call "texture source" (T.). A naive strategy is to randomly select image from the data bank as Tg. Though this strategy is successful, their perturbations are clearly perceptible. Alternatively, we can randomly select T3 from the adversarial target class. This strategy produces less perceptible perturbations compared to the random T, method as we are extracting a texture from the known target class. A better strategy to select T, is to find target class image that lies closest to the victim image in the feature space using nearest neighbors. This strategy is sensible as we assure our victim image has similar feature statistics as our target image. Consequently, minimizing gram matrices is easier and our attack generates more natural looking images (see Fig. 5).
For texture transfer, we extract cross-layer statistics in Eq. from the R11, R21, R31, R41, and R51 of pretrained VGG19, We optimize our objective (Eq. using an L-BFGS (Liu & Nocedal ( (989))
optimizer. tAdv attacks are sensitive and if not controlled well, images get transformed into artistic images. Since we do not have any constraints over the perturbation norm, it is necessary to decide when to stop the texture transfer procedure. For successful attack (images look realistic), we limit our L-BFGS to fixed number of small steps and perform two set of experiments: one with only one iteration or round of L-BFGS for 14 steps and another with three iterations of 14 steps. For the three iterations setup, after every iteration, we look at the confidence of our target class and stop if the confidence is greater than 0.9.
Texture and Cross-Entropy Weights. Empirically, we found setting a to be in the range [150, 1000] and B in the range [10-4, 10-3] to be successful and also produce less perceptible tAdy examples. The additional cross-entropy based adversarial objective Jade helps our optimization. We ensure large flow of gradients is from the texture loss and they are sufficiently larger than the adversarial cross- entropy objective. The adversarial objective also helps in transforming victim image to adversarial without stylizing the image. All our tabulated results shown for one iteration, 3 = 10 -3 unless otherwise stated. We use the annotation tte for the rest of the paper to denote the texture method that we using.
250 and

---

Published as a conference paper at ICLR 2020
Method
Model
R50
D121
VGG19
Kurakin et al 2016
R50
100.00
17.33
12.95
Model
R50
D121
VGG19
Wagner 12m
R50
16.50
11.00
Accurucy
76.15
74.65
74.24
Xiao ot at. 201
R50
100
5.23
8,90
Attack
SucceNs
cAdv1
cAdva
99.72
99.78
99.83
99.89
100.00
99.89
cAdv.
D121
R50
18.13
28,56
99.83
31.00
29.43
97.99
99.72
99.50
VGG19
22.94
26.39
100.00
tAdv500
99.27
99.83
99.83
R50
24.51
34,84
tAdv
D121
21.16
99.83
32.56
(a) Whitebox Target Attack
VGG19
20.21
24.40
99.89
Success Rate
(b) Transferability
Table I:
Our attacks are highly successful on ResNe150 (R50), DenseNet121 (D121) and VGG19, In (a), for cAdv, we show results for {1, 43 when attacked with 50 hints. For tAdv we show results for = {250, 500} and 3 = 0.001. In (b) We show the transferability of our attacks. We attack models from the column and test them on models from the rows.
Control over Texture. The amount of texture that gets added to our victim image is controlled by the texture weight coefficient (a). Increasing texture weights improves attack success rate at the cost of noticeable perturbation. When compared within-layer statistics, the cross-layer statisties that we use are not only better at extracting texture, it is also easier to control the texture weight.
4 EXPERIMENTAL RESULTS
In this section, evaluate the two proposed attack methods both quantitatively, via attack success rate under different settings, and qualitatively, based on interesting case studies. We conduct our experiments on ImageNet Deng al. (2009) by randomly selecting images from 10 sufficiently different classes predicted correctly for the classification attack.
We use pretrained ResNet 50 classifier (He et al. (2016)) for all our methods. DenseNet 121 and VGG 19 (Huang et al. Simonyan & Zisserman (2014)) are used for our transferability analysis.
4.1
CADV ATTACK
cAdv achieves high targeted attack success rate by adding realistic color perturbation. Our numbers in Table and Table also reveal that cAdv examples with larger color changes (consequently more color diversity) are more robust against transferability and adversarial defenses. However, these big changes are found to be slightly less realistic from our user study (Table Table
Smooth cAdv perturbations. Fig. g in our Appendix shows interesting properties of the adversarial colors. We observe that cAdv perturbations are locally smooth and are relatively low-frequee owwfrequency. This is different from most adversarial attacks that generate high-frequency noise-like perturbations. This phenomenon can be explained by the observation that colors are usually smooth within object boundaries. The pretrained colorization model will thus produce smooth, low-frequency adversarial colors that conform object boundaries.
Importance of color in classification. From Fig. we can compare how different target class affects our colorization results if we relax our constraints on colors (cAdy on Network Weights, hints), In many cases, the images contain strong colors that are related to the target class. In the case of golf-cart, we get a green tint over the entire image. This can push the target classifier to misclassify the image as green grass is usually overabundant in benign golf-cart images. Fig. 2 shows our attack on an image of a car to tench (a type of fish). We observe that the gray road turned blue and that the colors are tinted. We can hypothesize that the blue colors and the tint fooled the classifier into thinking the image is a tench in the sea.
The colorization model is originally trained to produce natural colorization that conforms to object boundaries. By adjusting its parameters, we are able to produce such large and abnormal color change that is impossible with our attack on hints and mask. These colors, however, show some evidence that colors play a stronger role in classification than we thought. We reserve the exploration of this observation for future works.
While this effect (strong color correlation to target class) is less pronounced for our attack on hints and mask, for all cAdv methods, observe isoluminant color blobs. Isoluminant colors are characterized
7

---

Published as a conference paper at ICLR 2020
Feature Squeezing
Adv
User
Method
Res50
JPEG75
4-bit
5-bit
2x2
3x.3
11-3-4
Res152
Res152
Pref.
Kurakin al. 2016
100
12.73
28. 62.62
86.66
31.28
21.56
29.25
1.08
1.38
0.508
Cartinn & Wagner 2017
99.85
11.30
12.00
30.50
22.00
14.50
18.50
1.08
1.38
0.497
Hosseini Poovendran
2018
1,20
X1a0 et al. (0186
100
17.61
22.51
29.26
28.71
23.51
26.67
4.13
1.39
0.470
cAdvi
100
62.33
47.78
76.17
36, 28
50 50
61.95
12.06
11.62
0.427
cAdva
89
46.61
42.78
72.56
34.28
46.45
59.00
17.39
19.4
0.437
cAAva
99.85
42.61
38.39
69.67
34.34
40.78
54.62
14.13
12.5
0.473
cAdv,
99.81
38.22
36.62
67.06
31.67
37.67
49.17
6.52
10.04
0.476
LAdvioo
99.00
32.89
62.79
89.74
54.94
38.92
40.57
10.9
2.10
0.433
t.Av220
100
36.33
67.68
94.11
58. .2
44.56
15.21
4.6
0.425
LAdv 2000
99.88
31.49
52.69
90.52
51.24
34.85
39.68
19.12
5.59
0.412
L
100
35.23
61.40
93.18
56.31
39.66
45.59
22.28
6.94
0.406
Table 2: Comparison against defense models. Misclassification rate after passing different adver- sarial examples through the defense models (higher means the attack is stronger). All attacks are performed on ResNer50 with whitebox attack. The highest attack success rate is in bold. We also report the user preference scores from AMT for each attack (last column).
by a change in color without a corresponding change in luminance. As most color changes occur along edges in natural images, it is likely that classifiers trained on ImageNet have never isoluminant colors. This suggests that cAdy might be exploiting isoluminant colors to fool classifiers.
4.2 tAdV ATTACK
t.dv successfully fools the classifiers with very small weighted adversarial cross-entropy objective (3) when combined with texture loss, while remaining realistic to humans, As shown in Table [ our attacks are highly successful on white-box attacks tested on three different models with the nearest neighbor texture transfer approach. We also show our attacks are more transferable to other models. In our Appendix, we show ablation results for tAdv attacks along with other strategies that we used for generating tAdv adversarial examples.
Structured tAdv Perturbations. Since we extract features across different layers of VGG, the t.Adv perturbations follow textural pattern. They are more structured and organized when compared to others. Our t.Ay perturbations are big when compared with existing attack methods in Ep norm. They are of high-frequency and yet imperceptible (see Fig. and Fig. 8).
Importance of Texture Classification. Textures are crucial descriptors for image classification and Imagenet trained models can be exploited by altering the texture. Their importance is also shown in the recent work from Geirhos et al. (2018). Our results also shows that even with a small or invisible change in the texture field can break the current state of the art classifiers.
4.3 DEFENSE AND TRANSFERABILITY ANALYSIS
We test all our attacks and other existing methods with images attacked from Resnet50. We evaluate them three defenses JPEG defense (Das et al. 2017). feature squeezing (Xu et al., 2017) and adversarial training. By leveraging JPEG compression and decompression, adversarial noise may be removed. We tested our methods against JPEG compression of 75. Feature squeezing a family of simple but surprisingly effective strategies, including reducing color bit depth and spatial smoothing. Adversarial training has been shown an effective but costly method defend against adversarial attacks. Mixing adversarial samples into training data of a classifier improves its robustness without affecting the overall accuracy. We were able obtain an adversarially pretrained Resnet152 model on ImageNet dataset and hence we tested our Resnet50 attacked images with this model.
Robustness. In general, our attacks are more robust to the considered defenses and transferable for targeted attacks. For cAdv, there is trade-off between more realistic colors (using more hints and sampling from more clusters) and attack robustness. From Table | and 2 we show that as we progressively use more clusters, our transferability and defense numbers drop. A similar trend is observed with the change in the number of hints. cAdv is robust to JPEG defense and adversarial training because of their large and spatially smooth perturbations. For tAdv, increasing texture weight (a) does not necessarily perform well with the defense even though it increases attack success rate, but increasing texture flow with more iterations improves attack's robustness against defenses.
8

---

Published as a conference paper at ICLR 2020
5 HUMAN PERCEPTUAL STUDIES
To quantify how realistic tAdv and cAdv examples are, we conducted user study on Amazon Mechanical Turk (AMT). We follow the same procedure described (Zhang et al., 2016 Xiao et al. 2018b). For each attack, we choose the same 200 adversarial images and their corresponding benign ones. During each trial, one random adversarial-bent -enigg pair appears for three seconds and workers are given five minutes to identify the realistic one. Each attack has 600 unique pairs of images and each pair is evaluated by at least unique workers. We restrict biases this process by allowing each unique user up to rounds of trials and also ignore users who complete the study in less than 30 seconds. total, 598 unique workers completed at least one round our user study. For each image, can then calculate the user preference score as the number of times it is chosen divided by the number of times it is displayed. 0.5 represents that users are unable to distinguish the image is fake. For cAdv and tAdv, indicating that workers have a hard time distinguishing them. The user preferences for all attacks are summarized in Table and their comparison with Ln norm is in Table and Table 5
preferences averages at 0.476 and 0.433 respectively,
ATTACKING CAPTIONING MODEL
Our methods are general and can be easily adapted for other learning tasks. As proof of concept.
we test our attacks against image captioning task. Image captioning is the task of generating sequence of word description for an image. The popular architecture for captioning is Long-Short- Tern-Memory (LSTM) (Hochreiter & Schmidhuber 1997) based models (Karpathy & Fei-Feil [0055] Wang et al., 2017). Recently, (Aneja et al.J (018) proposed a convolutional based captioning model for fast and accurate caption generation. This convolutional based approach does not suffer from the commonly known problems of vanishing gradients and overly confident predictions LSTM network. Therefore, we choose to attack the current state of the art convolutional captioning model. We randomly selected images from MSCOCO (Lin et al. 2014) for image captioning attack.
Attacking captioning models is harder than attacking classifiers when the goal is to change exactly one word in the benign image's caption unlike pixel based attacks (Chen et al., 2017 Xu et al.) (019). We show that attacks successful and have visible artifacts even for this challenging task. In Fig. 6, we change the second word of the caption to dog while keeping the rest of the caption the same. This is challenging targeted attack because, in many untargeted attacks, the resulted captions do not make sense. More examples are in our Appendix.
cAdv
a man holding a red apple in his hand
Attention for man
a dog holding a red apple his hand
Attention for dog
tAdv
woman riding bike down the street
Attention for woman
dog riding bike down the street
Attention for dog
Figure 6: Captioning attack. Top: cAdv; Bottom: tAdv. We attack the second word to dog and
show the corresponding change in attention mask of that word. More examples in Appendix.

---

Published as a conference paper at ICLR 2020
Adversarial Cross-Entropy Objective for Captioning. Let t be the target caption, denote the word position of the caption. F for the captioning model, L for the victim image and Jade for the cross-entropy loss
L capt
Σ Jado ((F()))) bw)
(6)
For cAdv, we give all color hints and optimize to get an adversarial colored image to produce target caption. For tAdv, we add Eqn to to Equ | to optimize the image. We select I's as the nearest neighbor of the victim image from the ones the adversarial target class using ImageNet dataset. We stop our attack once we reach the target caption and the caption does not change in consecutive iterations. Note we do not change the network weights, we only optimize hints and mask (for cAdv) or the victim image (for (Adv) to achieve our target caption.
7 RELATED WORK
Here we briefly summarize existing unrestricted and semantic adversarial attacks. Xiao et al. (2018b) proposed geometric spatial distortion of pixels in image to create adversarial examples. They distort the input image by optimizing pixel flow instead pixel values to generate adversarial examples. While this attack leads to "natural" looking adversarial examples with large L. norm, it does not take image semantics into account. [Song et al (2018) and Dunn et al.) (2019) considered GANs for adversarial attacks. This attack unrestricted in Cp norm but they are restricted simple datasets as it involves training GANs, which have been known to be unstable and computationally intensive for complex datasets like ImageNet (Karras et al. 2017 Brock al., 2018).
Hosseini & Poovendran 2018). changes the hue & saturation of an image randomly to create adversarial examples. It similar cAdv as they both involve changing colors, however, their search space is limited to two dimensions and their images are unrealistic, Appendix (Fig. (O). Also, while this method has a non-trivial untargeted attack success rate, performs extremely poorly for targeted attacks (1. (1.20% success rate in our own experiments on ImageNet). Our work is also related to Joshi (ct al. (2019) and Qiu et al. (2019), who manipulate images conditioned on face dataset attributes like glasses, beard for their attacks. These work focuses on changing single image visual attribute and are conditionally dependent. Our work focuses on changing visual semantic descriptors to misclassify images and are not conditioned to any semantic attributes.
Semantic Based
Explainable
Complex Dataset
Caption
Method
Unrestricted
Photorcalistic
(eg color affinity)
Attack
Kurakin et al. 2016
X
Carlini & Wagner 2017
Hossemu & Poovendran 2018
X
Song al 2018
X
Xıа0 ct 20186
/
cAdV (ours)
tAdv (ours)
Table 3: Summary of the difference in work compared to previous work. Unlike previous attack methods, our attacks are unbounded, semantically motivated, realistic, highly successful, and scales more complex datasets and other MI. tasks. They are also robust against tested defenses.
8 CONCLUSION
Our proposed two novel unrestricted semantic attacks shed light on the role texture and color fields in influencing DNN's predictions. They not only consistently fool human subjects but in general are harder to defend against. We hope by presenting our methods, we encourage future studies on unbounded adversarial attacks, better metries for measuring perturbations, and more sophisticated defenses.
ACKNOWLEDGEMENTS
We thank Chaowei Xiao for sharing their code to compare our methods with Xiao al, (2018b) and helping us setup the user study. We also thank Tianyuan Zhang for providing the AdvRes152 pretrained model. This work supported by NSF Grant No. 1718221 and ONR MURI Award N00014-16-1-2000
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Jyoti Aneja, Aditya Deshpande, and Alexander G Schwing. Convolutional image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5561-5570, 2018.
Andrew Brock. Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 1atv:1800..10000 1009, 2018.
Tom Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and Ian Goodfellow. Unrestricted adversarial examples. arXiv preprint FrXXiv:8809008355 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness neural networks. In 2017 IEFE Symposium on Security and Privacy (SP), pp- 39-57. IEFE, 2017.
Hongge Chen, Huan Zhang, Pin-Yo Chen, Jinfeng Yi, and Cho-Jui Hsieh. Attacking visual language grounding with adversarial examples: A case study on neural image captioning. arXiv preprint
2017.
Nilaksh Das, Madhuri Shanbhogue, Shang- Tse Chen, Fred Hohman, Li Chen, Michael E Kounavis, and Duen Horng Chau. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. arXiv preprint arXiv:1705.02000 2017.
J. Deng, W. Dong, R. Socher, I..J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
Aditya Deshpande, Jiajun Lu, Mao-Chuang Ych, Min Jin Chong, and David A Forsyth. Learning diverse image colorization. CVPR, pp. 2877-2885, 2017.
Isaac Dunn, Tom Melham, and Daniel Kroening. Generating realistic unrestricted adversarial inputs using dual-objective gan training, arXiv preprint 1XXi::1905.02663 2019.
Alexci Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341-346, ACM, 2001.
Leon Gatys. Alexander S Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems, pp. 262-270, 2015.
Leon A Gatys, Alexander Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2414 2423, 2016.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Amagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint orXiv:181. 1. 12231, 2018.
Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp. 2672-2680, 2014a.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 441:::1412.6655 2014b.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, (9))): 82-97, 2012.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 933:1733 1997.
Hossein Hosseini and Radha Poovendran. Semantic adversarial examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1614-1619, 2018.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks.
Xun Huang and Serge J Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, pp. 1510-1519, 2017.
11

---

Published as a conference paper at ICLR 2020
Phillip Isola, Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. 2017.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super- resolution. In European Conference Computer Vision, 2016.
Ameya Joshi, Amitangshu Mukherjee, Soumik Sarkar, and Chinmay Hegde. Semantic adversarial attacks: Parametric transformations that fool deep classifiers. arXiv preprint arXiv: 1e11199440889. 2019.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprini arXiv:1908.000/1 2019.
Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEER conference computer vision and pattern recognition, pp. 3128-3137, 2015.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv: 10..1770...11..... 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6111 2013.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. pp. 1097-1105, 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial cxampees in the physical world. CoRR, abs/1 000.60077.2333 2016. http://pppp//paapo......rrr///// html [tm]]KuraainnB11
Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. 17(39):1-40, 2016.
Yijun Li, Chen Fang, Jimei Yang. Zhaowen Wang, Xin Lu, and Ming- Hsuan Yang. Universal style transfer via feature transforms. In Advances in Neural Information Processing Systems, pp. 386-396, 396, 2017.
Tsung-Yi Yi Lin, Michacl Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitmick. Microsoft coco: Common objects context. European conference on computer vision, pp. 740-755. Springer, 2014.
Dong Liu and Jorge Nocedal. On limited memory bigs method for large scale optimization. Mathematical programming, 45(1-3):503-522 1989.
Nicolas Papernot, Patrick McDaniel, and Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 2016.
Haonan Qiu, Chaowei Xiao, Lei Yang, Xinehen Yan, Honglak Lee, and Bo Li. Semanticadv: Generating adversarial examples via attribute-conditional image editing. arXiv preprint 1rXv.09906007927 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint orXiv:1409 1556, 2014.
Yang Song et al. Constructing unrestricted adversarial examples with generative models. 2018.
Liwei Wang. Alexander Schwing. and Svetlana Lazebnik. Diverse and accurate image description using variational auto-eneoder with an additive gaussian encoding space. In Advances in Neural Information Processing Systems, pp. 7756-5766, 2017.
Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. arXiv preprint vrXivv1801.00 102600,,, 2018a.
Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. arXiv preprint arXiv: 1801 00261222,,222.11..11 2018b.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks, arXiv preprint 1704.01155 2017.
Yan Xu, Baoyuan Wu, Fumin Shen, Yanbo Fan. Yong Zhang, Heng Tao Shen, and Wei Liu. Exact adversarial attack image captioning via structured output learning with latent variables. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 4135-4144, 2019.
12

---

Published as a conference paper at ICLR 2020
Mao-Chuang Yeh, Shuai Tang, Anand Bhattad, Chuhang Zou, and David Forsyth. Improving style transfer with calibrated metries. arXiv preprint arXiv: (XXv:1990000944,, 2019.
Richard Zhang. Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Conference Computer Vision. pp. 649-666. Springer. 2016.
Richard Zhang, Jun- Yan Jun-Yan
Zhu, Phillip Isola, Xinyang Geng, Angela S Lin, Tianhe Yu, and Alexei A Efros. Real-time user- guiddg image colorization with learned deep priors. arXiv preprint arXiv: 020090 2017.
13

---

Published as a conference paper at ICLR 2020
A APPENDIX
A.1 OTHER DETAILS ON HUMAN STUDY
We also chose BIM Kurakin et al.
2016) and CW (Carlini & Wagner, 2017) for comparing our perturbations. Since these attacks are known have low Lp norm, we designed an aggressive version of BIM by relaxing its L.x bound to match the norm of our attacks. We settled with two aggressive versions of BIM with average S = {0.21, 0.347), which we refer to as B1Mg.21 BIM0.34. The average user preferences for BIM drops drastically from 0.497 0.332 when relax the norm to BIM0.311 the decrease in user preferences for tAdv (0.433 to 0,406) and cAdv (0.476 to 0.437) not significant. In Fig. 7 we plot density plot of Co user preference scores.
Prelorence
C= (6) CW
C. [c) BlMa
C. (1) CAdia
(B) BlMca5
(4) Undvise
(6) (Adv]coo
(g) cAdv
Figure 7: Density Plot. Our methods achieve large Loo norm perturbations without notable reduction in user preference. Each plot is density plot between perturbation (Lxo norm) on X axis and Pr(user prefers adversarial image) on Y axis. For ideal systems, the density would be concentrated horizontal line at 0.5. All plots are on the same set of axes. On the left, plots for three baseline methods (Fig a Fig c). Note the very strong concentration on small norm perturbations, which users like. Right 4 plots shows our methods (Fig Fig g). Note strong push into large norm regions, without loss of user preference.
Method
Lo
C2
Loo
Preference
CW
0.587
0.026
0.054
0.506
BIMo.os1
0.826
0.030
0.051
0.497
B
0.893
0.118
0.21
0.355
B BlMp.347
0.892
0.119
0.347
0.332
cAdv2
0.910
0.098
0.489
0.437
cAdva
0.898
0.071
0.448
0.473
cAdvs
0.896
0.059
0.425
0.476
tAdv 250
0.891
0.041
0.198
0.433
1Adv1oo
0.931
0.056
0.237
0.412
tAdv 250
0.916
0.061
0.258
0.425
tAdv 0.946
0.08
0.315
0.406
Table User Study. User preference score and Cp norm of perturbation for different attacks. cAdv and tAdv are imperceptible for humans (score close to 0.5) even with very big L, norm perturbation.
Amack
tarantula
merganser
naurihus
hyena
bencon
golfcart
photoerpier
umbrella
pretzel
sandhar
Mean
Zo
0.82
0.51
0.50
0.58
01.80
0.79
0.88
B.30
01.85
0.54
0.53
BIM
La
0.05
0.02
0.03
0.04
0.072
0.01
0.02
0.022
0.04
0.03
Exe
0.07
0.04
0.06
0.06
0.04
0.03
0.07
0.01
0.04
0.06
0.05
Preference
0.49
0.75
0.44
0.67
0.48
0.43
0.27
0.39
0.38
0.63
0.50
La
0.51
0.55
0.32
0.77
0.18
0.12
0.70
0.:55
0.09
0.71
0.59
CW
Ca
0.05
0.01
0.03
0.03
0.02
0.01
0.04
0.01
0.02
0.04
0.03
0.04
Preferenoe
0.51
0.68
0:43
0.59
0.52
0.51
0.39
0.40
0.12
0.02
0.51
Za
0.91
0.95
0.91
0.94
0.39
0.55
0.39
0.91
0.351
0.55
0.92
tAdv
Ca
0.07
0.08
0.08
0.06
0.05
0.06
0.03
0.05
0.07
0.05
0.06
01227
Preference
0.39
0.58
0:43
0.53
0.46
0.35
0.25
0.40
0.33
0.54
0.13
Eo 0.30
0.30
0.88
0.50
(150)
(150)
0.90
0.89
0.33
0.88
0.30
cAdv
0.33
0.44
0.41
0.31
0.48
0.51
0.55
0.56
0.46
0.39
0.45
Preference
00-46
0.65
D.46
0.58
0.45
0.48
0.30
0.43
0.35
0.59
0.47
Table 5: Class wise Cp norm and user preference breakdown. Users biased and pick few classes quite often (merganser, sandbar) r), and do not like a few classes (photocopier) over others.
14

---

Published as a conference paper at ICLR 2020
Tarantula
Beacon
Golfeart
Nautilus
Photocopler
Pretzel
BIM Perturbations
Perturbations
St-Ady
Perturbations
cAdv
Perturbations
tAdv
Perturbations
Figure 8: Perturbation comparisons. Images are attacked from tarantula to beacon, golf cart, nautilus, photocopier. pretzel from left to right. Our perturbations (cAdy and tAdv) are large, structured and have spatial patterns when compared with other attacks. Perturba- tions from cAdv are low-frequency and locally smooth while perturbations from tAdv are primarily high-frequency and structured. Note gray color indicates no perturbations.
15

---

Published as a conference paper at ICLR 2020
A.2 ADDITIONAL RESULTS
250
500
750
1000
Model
Resnet50
Dense121 74.65
VGG
Accuracy
76.15
74.24
25.00
99.61
98.55
95.92
Random
99.67
99.72
96.16
10-4
99.88
99.61
98.55
95.92
Attack Success
Random Target T.
99.72
99.89
99.94
10 3
97.99
99.27
99.66
99.50
Nearest Target T
97.99
99.72
90.50
10 2
96.26
95.12
96.32
96.59
cAdva 25 hints
99.78
90.83
99.93
tAdv ablation study. Whitebox target
c.Avv, 50 hints
99.78
99.83
100.00
cAdv, 100 hints
99.44
99.50
99.93
success rate with nearest target T. (texture source). In columns, we have increasing texture weight (a) and rows, have increasing adversarial cross-entropy weight (5). All attacks are done on Resnet50.
Whitebox target attack success rate. Our attacks
highly successful on different models across all
strategies. tAdv results are for a = 250, 3
and iter= 1.
Table Ablation Studies.
GT
k=
2=2
k-
k-6
k-8
Figure 9: Additional qualitative examples for controlling cAdv. We show a comparison of sampling 50 color hints from k clusters with low-entropy. All images are attacked to golf f-cart. Even numbered rows visualize our cluster segments, with darker colors representing higher mean entropy and red dots representing the location we sample hints from. Sampling hints across more clusters gives less color variety.
16

---

Published as a conference paper at ICLR 2020
Merganser ->
Umbrella
Sandbar -> Beacon
Hyena > Tarantula
Figure 10: Images attacked with the method of (Hosseini & Poovendran, 2018 are not realistic, as these examples show. Compare Fig [2] showing results our color attack. Similar qualitative results for CIFAR-10 visible in Hosseini & Poovendran (2018).
0=250
a=500
a=750
a=1000
Original
Random
Original
Target s
Original
Nearest larget
Figure 11: Additional qualitative examples for tAdv. Texture transferred from random images (row 1), random images from adversarial target class (row 2) and from the nearest neighbor of victim image from adversarial target class (row 3). All examples are misclassified from Merganser to Umbrells rows contain more artifacts as the texture weight a increases (left to right).
la. a. Images in the last row look photo realistic, while those in the first two
17

---

Published as a conference paper at ICLR 2020
tAdv
banana sitting top of table
Attention for banana
dog sitting top of table
Attention for dog
clock tower the middle of city
Attention for clock
bird the middle a city
Attention bird
cAdv
woman is holding donut her mouth
Attention for woman
dog is holding donut in her mouth
Attention for dog
S Jn
ob
ob S
man sitting on bench with skateboard
boy sitting on bench with skateboard
Attention for
Attention for bird
Figure 12: Captioning attack. We attack the second word of each caption to (dog, bird) and show the corresponding change attention mask of that word. For tAdy we use the nearest neighbor selection method and for cAdv we initialize with all groundtruth color hints.
18

---

Published as a conference paper at ICLR 2020
merganser
na nautilus
hyena
beacon
gelfcart
merganser
nautilus
hyena
beacon
golfcart
merganee
hyena
bescin
goltcar
qulfcart
(a) cAdv,
with 50 hints
(b) cAdv Perturbations
merganser
nautillus
Tryena
beacon
gelicart
merganser
nautilus
hyena
beacon
golicart
megiiner
FINE P0011A0
hyera
beacon
gentcsr
tAdvàso for nearest Target T.
(d) tAdv Perturbations
Figure 13: Randomly sampled, semantically manipulated, unrestricted adversarial examples and their perturbations. (a) Adversarial examples generated by cAdv attacking Hints and mask with hints. (c) Adversarial examples generated by tAdy with a 250, 10 and iter= 1 using nearest target T,. (b) and (d) are their respective perturbations. Note that diagonal images are groundtruth images and gray pixels indicate perturbation.
19


Reviews:
[{"review_id": "rklbmtp6YH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces two new adversarial attacks: one is generating adversarial examples by colouring the original images and the other is by changing textures of the original images. Specifically, the former one minimises the cross-entropy between the output of the classifier and the target label with the network weights of a pre-trained colourisation network. While the latter minimises the cross-entropy as well as the loss that defines the texture differences.\n\nI think the general idea of going beyond perturbations of pixel values in this paper is interesting and the proposed approaches of attacking on colour and textures are intuitive and reasonable. The results seem to be promising with comprehensive experiments including whitebox attack, blackbox attack by transferring, and attacks on defences.\n\nThe paper overall is well-written and easy to follow. But I think the part of attacking for captioning is a bit distracted and there is no comparison with others on this task. I expect existing attacks on pixel can also do this task.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJgAOwI-FB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes cAdv and sAdv, two new unrestricted adversarial attack methods that manipulates either color or texture of an image. To these end, the paper employes another parametrized colorization techniques (and texture transfer method) and proposes optimization objectives for finding adversarial examples with respect to each semantic technique. Experimental results show that the proposed methods are more robust on existing defense methods and more transferrable accross models. The paper also performs a user study to show that the generated examples are fairly imperceptible like the C&W attack. \n\nIn overall, I agree that seeking a new way of attack is important, and the methods are clearly presented to claim a new message to the community: adversarial examples can be even found by exploiting semantic features that humans also utilize, since DNNs tend to overly-utilize them, e.g. colors. These claims are supported by the experiments showing that the generated examples are more transferrable across robust classifiers. Personally, I liked the idea of using another colorization method to design cAdv and the use of K-means clustering to control the imperceptibility. \n\n- Some readers may wonder how the \"averaged-case\" corruption robustness behave for both cAdv and sAdv, e.g. considering random colorization. Would it be worse than the robustness on Gaussian noise?\n- One of my concerns on tAdv is whether the texture added is indeed effective to reduce the accuracy, or its just from the (yet small) beta term in the objective. Adding an ablation of beta=0 case in the result would much help the understanding of the method. \n- Eq 1: I think F should denote the classifier to attack, but the description tells it's the colorization network. As it seems to me that theta is nevertheless for the colorization network, I feel the notation should be refined for better understanding to the readers.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "BJlg1WSTtS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed to generate semantically meaningful adversarial examples in terms of color of texture. In order to make manipulated images photo-realistic, colors to be replaced are chosen by energy values, while textures are replaced with style-transfer technique.\n\nThe paper is written clearly and organized well to understand. The graphs and equations are properly shown. The idea of using color replacement and texture transfer is interesting and novel.\n\nA somewhat weakness is that the discriminator - a pretrained ResNet 50 - is too weak for this scenario. What about a ResNet 50 trained on augmented datasets with color jittering?\nWhat about finetuned ResNet 50 with taking color channel as explicit input, since the attack uses this additional info.\n\nAs tAdv attack seems to manipulate high frequency texture of images, how about applying a Gaussian filter on the images and feed into the discrimimator again? Is that attack still effective or not?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors present adversarial attacks by semantic manipulations, i.e., manipulating specific detectors that result in imperceptible changes in the picture, such as changing texture and color, but without affecting their naturalness. Moreover, these tasks are done on two large scale datasets (ImageNet and MSCOCO) and two visual tasks (classification and captioning). Finally, they also test their adversarial examples against a couple of defense mechanisms and how their transferability. Overall, all reviewers agreed this is an interesting work and well executed, complete with experiments and analyses. I agree with the reviewers in the assessment. I think this is an interesting study that moves us beyond restricted pixel perturbations and overall would be interesting to see what other detectors could be used to generate these type of semantic manipulations. I recommend acceptance of this paper.

Decision: Accept (Poster)