SemanticAdv: Generating Adversarial Examples via Attribute-Conditional Image Editing
Authors: Haonan Qiu
Chaowei Xiao
Lei Yang
Xinchen Yan
HongLak Lee
Bo Li
Venue:  (2020)
Keywords: adversarial examples
semantic attack

Abstract: Deep neural networks (DNNs) have achieved great success in various applications due to their strong expressive power. However, recent studies have shown that DNNs are vulnerable to adversarial examples which are manipulated instances targeting to mislead DNNs to make incorrect predictions. Currently, most such adversarial examples try to guarantee “subtle perturbation" by limiting the Lp norm of the perturbation. In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate “unrestricted adversarial examples". Such semantic based perturbation is more practical compared with the Lp bounded perturbation. In particular, we propose an algorithm SemanticAdv which leverages disentangled semantic factors to generate adversarial perturbation by altering controlled semantic attributes to fool the learner towards various “adversarial" targets. We conduct extensive experiments to show that the semantic based adversarial examples can not only fool different learning tasks such as face verification and landmark detection, but also achieve high targeted attack success rate against real-world black-box services such as Azure face verification service based on transferability. To further demonstrate the applicability of SemanticAdv beyond face recognition domain, we also generate semantic perturbations on street-view images. Such adversarial examples with controlled semantic manipulation can shed light on further understanding about vulnerabilities of DNNs as well as potential defensive approaches.

Under review as a conference paper at ICLR 2020
SEMANTICADV: GENERATING ADVERSARIAL EXAM- PLES VIA ATTRIBUTE-CONDITIONA IMAGE EDITING
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks (DNNss) have achieved great success in various applications due to their strong expressive power. However, recent studies have shown that DNNS vulnerable adversarial examples which manipulated instances targeting to mislcad DNNs to make incorrect predictions. Currently, most such adversarial examples try to guarantee "subtle perturbation" by limiting the Lp norm of the perturbation. In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples". In particular, we propose an algorithm SemanticAdy which leverages disentangled semantic factors to gener- ate adversarial perturbation by altering controlled semantic attributes to fool the learner towards various "adversarial" targets. We conduct extensive experiments to show that the semantic based adversarial examples can not only fool different learning tasks such as face verification and landmark detection, but also achieve high targeted attack success rate against real-world black-box services such Azure face verification service based on transferability. To further demonstrate the applicability of SemanticAdy beyond face recognition domain, we also generate semantic perturbations on srreet-view images. Such adversarial examples with controlled semantic manipulation can shed light on further understanding about vulnerabilities of DNNs as well as potential defensive approaches.
INTRODUCTION
Deep neural networks (DNNs) have demonstrated great successes in advancing the state-of-the-ee
performance discriminative tasks (Krizhevsky et al.] 2012 Goodfellow et al.) [216 He et al.) 2016)
Collobert & Weston 2008 Deng et al. 2013 Silver et al. 2016). However, recent research found that DNNs are vulnerable to adversarial examples which are carefully crafted instances aiming to
induce arbitrary prediction errors for learning systems. Such adversarial examples containing small magnitude of perturbation have shed light on understanding and discovering potential vulnerabilities
of DNNs (Szegedy et al.] 2013; Goodfellow al. 2014b Moosavi-Dezfool. et al. 2016: Papernot
et al. 2016; Carlini & Wagner 2017 Xiao et al. 2018b c a 2019). Most existing work focused constructing adversarial examples by adding Lp bounded pixel-wise perturbations (Goodfellow
ct al. 2014b) or spatially transforming the image (Xiao et al. 2018c| Engstrom et 22.17 (e.g.. in-plane rotation or out-of-plane rotation). Generating unrestricted perturbations with semantically meaningful patterns is an important yet under-explored field.
At the same time, deep generative models have demonstrated impressive performance in learning disentangled semantic factors through data generation in an unsupervised (Radford et al. 2015 Karras
al. 2018 Brock et al. 2019) or weakly-supervised manner based on semantic attributes Yan et al.
2016 Choi et al. 20188) Empirical findings in ( (Yan et al. 2016 Zhu et al. 201 6a Radford et al.
2015) demonstrated that a simple linear interpolation on the learned image manifold can produce smooth visual transitions between pair of input images.
In this paper, we introduce a novel attack SemanticAdv which generates unrestricted perturbations with semantically meaningful patterns. Motivated by the findings mentioned above, we leverage an aetribute-conditionn image editing model (Choi et al. 2018) to synthesize adversarial examples by interpolating between source and target images in the feature-map space. Here, we focus changing a single attribute dimension to achieve adversarial goals while keeping the generated adversarial image reasonably-lookine method, we consider two tasks, namely, face verification and landmark detection, as face recognition field has been extensively explored and the commercially used face models are relatively robust.
(e.g., see Figure Ip. To validate the effectiveness of the proposed attack

---

Under review as a conference paper at ICLR 2020
Reomutratiion Gemertion
Octponal
Heerlt Werfoodion
Mas Alie
Oriaiaal Aorthate
Original
Target
Iaage
Min
Partare mup
Mrr Reb
trage Ut
Sumtt
mage uik
Mardd
Semantichdy,
tottoon bearg dtfeest peodl ooleaa
ppel:
m
Noc Rob
Advesariial
image
Saage LVb.
Imagee
coteitooalll Imagge bilting
Figure 1: Left: Overview of the proposed SemanticAdv. Right: Illustration of our SemanticAdy in the real world face verification platform. Note that the confidence denotes the likelihood that two faces belong to the same person.
since they require a low false positive rate. We conduct both qualitative and quantitative evaluations on CelebA dataset (Liu et al., 2015). To demonstrate the applicability of SemanticAdy beyond face domain, we further extend SemanticAdy to generate adversarial street-view images. We treat semantic layouts as input attributes and use the image editing model (Hong et al., 2018 pre-trained on Cityscape dataset (Cordts al. (2116). Please find more visualization results on the anonymous website: https:://// /tt/sittsss.. www.le.....oooooooo...o...o....o cwm/viewi http://eeeerteeeeeeeeeeeeeeeeeee semant iewpaaattiiddd-examaiae
The contributions of the proposed SemanticAdy three-folds. First, we propose novel semantic- based attack method to generate unrestricted adversarial examples by feature-space interpolation. Second, the proposed method is able generate sutibbute-ooddtiooned modeling. This allows us to analyze the robustness of a recognition system against different types of semantic attacks. Third, side benefit, the proposed attack exhibits high transferability and leads to 65% query-free black-box attack success rate on real-world face verification platform, which outperforms the pixel-wise perturbations in attacking existing defense methods.
perturbations due the
2 RELATED WORK
Semantic image editing. Semantic image synthesis and manipulation is popular research topic in machine learning, graphics and vision. Thanks to recent advances in deep generative models (Kingma
Welling 2014 Goodfellow et al. [2014a] Oord al.) 2016) and the empirical analysis of deep
classification networks (Knihevsky et al. 2012] Simonyan an & Zisserman 2014 Szegedy et al., 2013
past few years have witnessed tremendous breakthroughs towards high-fidlity pure image genera- tion (Radford et al. [2015) Karras et al.) 2018 Brock et al. (2019), attribute-to-iman
generation ( Yan
20.61 Choi et al. 2018), text-to- text-to-image generation MMansimvv et al.) (2015)
Reed Reed al. 2016
Van den Oord et al. [016)] Odena et al. 2017 /Zang et al. 2017 Johnson et al., 2018). and image-
to-image translation (Isola al. 2017 Zhu et al. 2017 Liu et 2017 Wang et al. 2018b Hong
al. 2018).
Adversarial examples. Generating L, bounded adversarial perturbation has been extensively studied recently (Szegedy et al.. 2013 Goodfellow et al.) 20014b; Moosavi-Dezzooll et al., 2016 Papernot al. 2016 Carlini & Wagner 2017]] Xiao et al. 2018b). To further explore diverse adversarial attacks and potentially help inspire defense mechanisms, it important to generate the so- called "unrestricted" adversarial examples which contain unrestricted magnitude of perturbation while still preserve perceptual realism Brown et al. (2018). Recently, Xiao et al. (2018 &c); |Engstrom et al. (2017) propose to spatially transform the image patches instead of adding pixel-wise perturbation, while such spatial transformation does not consider semantic information. Our proposed semanticAdy focuses on generating unrestricted perturbation with semantically meaningful patterns guided by visual attributes.
Relevant to our work, Song et al. (2018) proposed to synthesize adversarial examples with an unconditional generative model. Bhattad et al. (2019) studied semantic transformation in only the color or texture space. Compared to these works, semanticAdy is able to generate adversarial examples in controllable fashion using specific visual attributes by performing manipulation in the
2

---

Under review as a conference paper at ICLR 2020
feature space. We further analyze the robustness of the recognition system by generating adversarial examples guided by different visual attributes. Concurrent to our work, et al. (2019) proposed to generate semantic- mislead the model towards arbitrary adversarial targets. They conduct the manipulation within the attribution space which is less flexible and effective than our proposed feature-space interpolation.
attacks against restricted binary classifier, while our attack is able to
SEMANTIC ADVERSARIAL EXAMPLES
3.1 PROBLEM DEFINITION
Let M be machine learning model trained on dataset D = {(x,y)} consisting of image-label pairs, where x IR MxXWx and y 2 denote the image and the ground-truti label, respectively, Here, H, W, Dr, and D,. denote the image height, image width, number of image channels, and label dimensions, respectively. For each image x, our model M makes a prediction y .Mxx) E Pr. Given a target image-labet pair (x'*t y and y * y' traditional attacker aims synthesize adversarial examples {xx">]] } by adding pixel-wise perturbations to or spatially transforming the original image x such that MM(x"dv) ylsl
In this work, we introduce the concept of semantic attacker that aims at generating adversarial examples by adding semantically meaningful perturbation with conditional generative model g. Compared to traditional attacker that usually produces pixel-wise perturbations, the proposed method able to produce semantically meaningful perturbations.
Semantic image editing. For simplicity, we start with the formulation where the input attribute is represented as a compact vector. This formulation can be directly extended to other input attribute formats including semantic layouts. Let semantic factors (e.g., expression or hair color of a portrait image) of image x, where De indicates the attribute dimension and e; E {0, }} indicates the appearance of i-th attribute. Here, our goal is to use the conditional generator for semantic image editing. For example, given portrait image of girl with black hair and blonde hair as the new attribute, our generator is supposed synthesize new image that turns the girl's 's hair from black to blonde. More specifically, we denote the new attribute as chew E R De such that the synthesized image is given by x=o g(x, c. In the special case when there is no attribute change (c cl the generator simply reconstructs the input: x = g(x, c). Supported by the findings mentioned in (Bengio et al., 2013; Reed et al. 2014). our synthesized image new should fall close to the data manifold if we constrain the change of attribute values to be sufficiently small (e.g., we only update one semantic attribute at a time). In addition, we can potentially generate many such images by linearly interpolating between the semantic embeddings of the conditional generator G using original image x and the synthesized image new with the augmented attribute.
Pe be attribute representation reflecting the
Attribute-space interpolation. We start with simple solution (detailed in Eq. I assuming the adversarial example can be found by directly interpolating in the attribute-space Given a pair of attributes and new we introduce interpolation parameter (0, 1) to generate the augmented attribute vector c* E De (see Eq. I] Given augmented attribute c" and original image x, we produce the synthesized image by the generator G. For our notation purpose, we also introduce delegated function TG as re-parametrization for the generator G. Our formulation is also supported by the empirical results on iitribute-conditione image progression (Yan al. 2016 Radford et al. 2015) that a well-trained generative model has the capability to synthesize a sequence of images with smooth attribute transitions.
x adv Coggiin.(TTTeenn c)ee)) where To x, c, mww) = G(x, c²) and cº = a- c (1 (1-a) a) new
(I)
Feature-map interpolation. Alternatively, we propose to interpolate using the feature map pro- duced by the generator g = Suee o Gens- Here, Gene is the encoder module that takes the image as input and outputs the feature map. Similarly, Gaee is the decoder module that takes the feature map as input and outputs the synthesized image. Let f = Gene E R Hy CF be the feature map of an intermediate layer in the generator, where Hr, Wr and Cr indicate the height, width, and
3

---

Under review as a conference paper at ICLR 2020
number of channels in the feature map.
xd L(Tming
where Tg(a; x, c, c (fc)
f = a o Gead (x, c) + (1 a) 3 Geac c
Compared to suribute-space interpolation which is parameterized by a scalar, we parameterize feature-map interpolation by tensor R xWxxCr (Ott,,,kk (0, where S h Hr, 1 < w < WF, and 1 < k I Cş) with the same shape as the feature map. Compared to linear interpolation over attribute-space, such design introduces more flexibility when interpolating between the original image and the synthesized image. Empirical results in Section critical the adversarial attack success rate.
show our design is
3.2 ADVERSARIAL OPTIMIZATION OBJECTIVES
As we see in Eq. 3 obtain the ad image x by minimizing the objective C(-) with respect to the synthesized image Tg (a:; x, c, che"), ), which is defined in Eq. and Eq.2) respectively. Here, each synthesized image Tg( x, c, c.ec) is produced by the interpolation using the conditional generator G. In our objective function, the first term is the adversarial metric, the second term is a smoothness constraint, and À is used to control the balance between the two terms. The adversarial metric is minimized once the model M has been successfully attacked towards the target image-label pair (xtat, . For identify verification, y' the identity representation of the target image: For structured prediction tasks in our paper, y' either represents certain coordinates (landmark detection) or semantic label maps (semantic segmentation).
ad = argminn,fTooaann c² L(To(a;; c, c Lav (To(;; c M. M, Lsmooth (a))
Identity verification. In the identity verification task, two images are considered to be the same identity the corresponding identity embeddings from the verification model M are reasonably close.
Cadv(To (To x, c, ""e"); M, = max (x, (M (Tp x, c, "="",, x!))
(4)
As we see in Eq. 4 did (:,-) measures the distance between two identity embeddings from the model M, where the normalized Le distance is used in our setting. addition, we introduce the parameter representing the constant related to the false positive rate (FPR) threshold computed from the development set.
Structured prediction. For structured prediction tasks such as landmark detection and semantic segmentation, we use Houdini objective proposed in Cisse al. (2017) as our adversarial metric and select the target landmark (semantic segmentation) target as ys In addition, (m((.-) is a scoring function for each image-label pair and y is the threshold.
Las (Ta((T c. M -Pa~N10.1) ( (,
(To c, "m"),,,,,) vet) < 7 1(y", y
(5)
where y M(Tg(a;x,,, ))w) and 1(y,,'1 is task loss decided by the specific adversarial target.
Interpolation smoothness Lsmooth- As the tensor to be interpolated in the feature-may map space has far more parameters compared to the attribute itself, we propose to enforce smoothness constraint on the tensor used in feature-map interpolation. As we in Eq. the smoothness loss encourages the interpolation tensors to consist of piece-wise constant patches spatially, which has been widely used wixel-wise de-noising objective for natural image processing (Mahendran & Vedaldi 2015 Johnson et al. 2016).
HF -1 WF
HF WF
Ssooohh (a) = Σ
Σ
|a w. - ah 1
Jah,441 - 2n
(6)
h=1 w=1
4

---

Under review as a conference paper at ICLR 2020
4 EXPERIMENTS
In the experimental section, mainly focus analyzing the proposed SemanticAdy in attacking state-of-the-ee face recognition systems on CelebA (Liu et al. 2015) due to its wide applicability extend
(e.g., identification for mobile payment) in the real world. In addition, street scenes with semantic label maps as the condition. We attack the semantic segmentation model DRN-D-22 Yuu et al., 2017) previously trained on Cityscape (Cordts et al. (016) by generating adversarial examples with dynamic objects manipulated (c.g., insert a car into the scene).
attack to urban
The experimental section is organized follows. First, we analyze the quality of generated adversarial examples and qualitatively compare our method with Cg bounded pixel-wise optimization based method (Carlini & Wagner] 2017: Dong et al.) 2018 Xie et al., 2019). Second, we provide both qualitative and quantitative results by controlling each of the semantic attributes at time. In terms of attack transferability, we evaluate our proposed SemanticAdy on various settings and further demonstrate the effectiveness of our method via query-free black-box attacks against online face verification platforms. Third, we compare our method with the baseline against different defense methods on the face verification task. Fourth, we demonstrate that the proposed SemanticAdy also applies to the face landmark detection and street-view semantic segmentation.
4.1 EXPERIMENTAL SETUP
Face identity verification. We select ResNet-50 and ResNet-101 (He et al. 2016) trained on MS-Celeb- M (Guo et al.) 2016) as our face verification models. The models are trained using two different objectives, namely, so ffmax loss (Sun al. 2014 Zhang al. 2018) and cos i ne loss (Wang et al.] 201 8a). For simplicity, we use the notation "R-N-S" to indicate the model with N-layer residual blocks backbone trained using softmax loss, while "R-N-C" indicates the same backbone trained using cosine loss. The distance between face features is measured by normalized L2 distance. For R-101-S model, we decide the parameter x. based on the false positive rate (FPR) for the identity verification task. Three different FPRs have been used: 10 (with k = 1.24),
3 x 10-4 (with x = 1.05), and 10 (with x = 0.60). The distance metrics and selected thresholds are commonly used when evaluating the performance of face recognition model Klare et al. (2015); Beellmacher-Shlizemmee
et al. (2016). Please check the Appendix (see Table B) for more details. To distinguish between the FPR we used in generating adversarial examples and the other FPR used in evaluation, we introduce two notations "Generating FPR (G-FPR)" and "Test FPR (T-FPR)", For the experiment with query-free black-box API attacks, use the online face verification services provided by Face++ (fac) and Ali (ali).
Face landmark detection. We select Face Alignment Network (FAN) (Bulat Tzimiropoulos 2017b) trained on 300W-LP (Zhu et al.) (016b) and fine-tuned on 300-W W (Sagonas et al. 20113) for 2D landmark detection. The network is constructed by stacking Hour-Glass network (Newell et al. 2016) with hierarchical block (Bulat & Tzimiropoulos/ (017a). Given portrait image as input, FAN outputs 2D heatmaps which can be subsequently leveraged to yield 68 2D landmarks.
Semantic attacks on face images. In our experiments, we randomly sample 1, 280 distinct identi- ties form CelebA (Liu et al. 2015). To reduce the reconstruction error brought by the generator (e.g.. A ((x. c)) in practice,
take more step obtain the updated feature map f = Gene Gene c). where x' = argming [[g c) - x|| in feature-map interpolation. In our experiments, we use the last conv layer before upsampl Lingiing
the generator as our as feature-map f given by the attack effectiveness. We also fix the parameter X (c.g., balances the adversarial loss and smoothness constraint in Eq. 3 to be 0.01 for both face verification and landmark detection.
We used the StarGAN (Choi et al. 2018) for ciniibbtt-conditiona image editing. In particular, we re-trained model on CelebA dataset (Liu et al.) (2015) by aligning the face landmarks and then resizing images to resolution 112 x 112. In addition, we select 17 ddentity--reserving attributes our input condition, as such attributes related to facial expression and hair color.
For each distinct identity pair x we perform semanticAdy guided by each of the 17 attributes (e.g., we intentionally add or remove one specific attribute while keeping the rest unchanged). In total, for each image we generate 17 adversarial images with different augmented attributes. In the experiments, we select a pixel-wise adversarial attack method (Carlini & Wagner 2017) (referred as CW) our baseline for comparison. Compared to our proposed method, CW does not require visual attributes as part of the system, as it only generates one adversarial example for each instance. We refer the corresponding attack success rate the instance- wise success rate in which the attack
5

---

Under review as a conference paper at ICLR 2020
success rate is calculated for each instance. For each instance with 17 adversarial images using different augmented attributes, if one the 17 resulting images can attack successfully, we count the attack of this instance one success, vice verse.
Semantic attacks on street-view images. We select DRN-D-22 eYu al. 2017) as our semantic segmentation model and fine-tune the model on image regions with resolution 256 x 256. To synthe- size semantic adversarial perturbations, we consider semantic label maps as the input attribute and leverage generative image manipulation model (Hong et al. 2018) pre-trained on CityScape (Cordts) et al. 2016) dataset. Given an input semantic label map at resolution 256 x 256, we select target object instance (e.g., pedestrian) to attack. Then, by inserting another object instance (e.g., a car) in the vicinity of the target object. Similar to the experiments in the face domain, for both semantic label maps, we used the image manipulation encoder to extract features (with 1, 024 channels at spatial resolution 16 x 16) and conducted feature- space interpolation. We synthesized the final image by feeding the interpolated features to the image manipulation decoder. By searching the interpolation coefficient that maximizes the attack rate, we are able to fool the segmentation model with the synthesized final image.
create manipulated semantic label map
4.2 SemanticAdv ON FACE IDENTITY VERIFICATION
Attribute-space vs. feature-space interpolation. First, we found that both attribute-space and feature-space interpolation could generate reasonable samples (see Figure Appendix). Compared to attribute-space interpolation, generating adversarial examples with feature-space interpolation produced much better quantitative results (see Table Di Appendix). We measured the attack success rate of attribute-space interpolation (with G-FPR- = T-FPR = 10 3): 0.08% on R-101-S, 0.31% on R-101-C, and 0.1 both R-50-S and R-50-C. While feature-space interpolation achieves almost 100% success rate on all those models (see Figure 3). We conjecture that this is because the high dimensional feature space can provide more manipulation freedom.
Source
Tarpot
Semonttcc Adverserial
Advemariaa Boomgles
Sourcs
Torget
Scmanii Adversaria Dooppin
Alversocial Exapples seng
yoing
lparres
"DOuT dighlly
Figure 2: Qualitative comparisons between our proposed SemanticAdy and pixel-wise adversarial examples generated by CW (Carlini & Wagner 2017). Along with the adversarial examples, we also provide the corresponding perturbations (residual) on the right. Perturbations generated by our SemanticAdy (G-FPR- = 100 are unrestricted with semantically meaningful patterns. More results are shown Appendix (see Figure
Overall analysis. Figure the generated adversarial images and corresponding perturbations against R-101-S of SemanticAdv and CW respectively. The text below each figure is the name of augmented attribute, the sign before the name represents "adding" (in red) or "removing" (in blue) the corresponding attribute from the original image. We see that SemanticAdy able to generate perceptually reasonable examples guided by the corresponding attribute. In particular, SemanticAdy able to generate perturbations on the corresponding regions correlated with the augmented attribute, while the perturbations of CW have no specific pattern and are evenly distributed across the image.
Analysis: controlling single attribute. One of the key advantages of SemanticAdv is that we can generate adversarial perturbations in more controllable fashion guided by the semantic attributes. This allows analyzing the robustness of a recognition system against different types of semantic attacks. We group the adversarial examples by augmented attributes in various settings. In Figure 3 we present the attack success rate against two face verification models, namely, R-101-S and R-101-C, guided by different attributes. We highlight the bar with light blue for G-FPR equals to 10-3 and blue for G-FPR equals to 10 10-4, respectively. As we see in this figure, with a larger T-FPR 10-3 our SemanticAdy can achieve almost 100% attack success rate across different attributes. With smaller T-FPR 10 10-4, we find that SemanticAdv guided by some attributes such as Mouth S Slightly Open and Arched Eyebrows achieve less than 50% attack success rate, while the other attributes

---

Under review as a conference paper at ICLR 2020
such Pale Skin and Eyeqlasse SemanticAdy guided by attributes describing the local shape (c.g., mouth, carrings) achieve relatively lower attack success rate compared to attributes relevant to the color (e.g., hair color) entire face region (e.g., skin). This suggests that the face verification models used in our experiments are more robustly trained in terms of detecting local shapes compared colors. Please note that in practice we have the flexibility to select attributes for attacking an image based on the perceptual quality and attack success rate.
relatively less affected. In summary, found that
Figure 4 shows the adversarial examples with augmented semantic attributes against R-101-S model. The attribute names are shown in the bottom. The upper images are 9(x, c"ew) generated by StarGAN with augmented attribute chew where the lower images are the corresponding adversarial images with the same augmented attribute.
Attack Success Rate (%)
Res-101 (softmax loss)
Res-101 (cos loss)
100
100
75
75
50
50
25
25
Pale Skin-
Receding
Hailine
Rosy Cheeks
Blond Hair
Chubby
Young
Bushy Eyebrows
Wavy Hair
Smiling
Wearing
Makeup-
Bags Under Eyes,
Weaning
Earrings
Bangs
Arched Eyebrows
Slightly Open
Pale Skin
Evegiaseeg Harring
Rasy Cheeks
Blond Hair
Chubby
Young
Smiling
Wavy Hair
Heavy Masc
Bushy Eyobrows
Bags Under Eyes
Wearing
Upsiick
Arched Ryebrows
Weariing
Barie
Slighuy Open
Mouth
Mouth
Heavy
Figure 3: Quantitative analysis on the attack success rate with different single-attribute attacks. In each figure, we show the results correspond larger FPR (G-FPR T-FPR 10-3) in skyblue and the results correspond to a smaller FPR (G-FPR T-FPR = 10-4) in blue, respectively.
Target
Generation via attrboteeodiiioo image wditing
Synthesized image with sugiaanned witribute (owi)
Advesaziall Image by opcanicint interpolatto parameter (awwrgmin s
-eyodiasezs
baiubbyy
slishily
Suualyy
susuriag lppetic
amiling
arched eyetemws
Janng
Meaaring
hatrline
Figure 4: Qualitative analysis on single- attribute adversarial attack (G-FPR = 10-3). More results are shown in Appendix (see Figure K
Figure and Figure M)
Analysis: semantic attack transferability. To further understand the property of SemanticAdy, we analyze the transferability of SemanticAdy on various settings. For each model with different FPRs, we select the successfully attacked adversarial examples from Section 4. to construct our evaluation dataset and evaluate these adversarial samples across different models. Table Ta illuttrates transferability of SemanticAdy among different models by using the same FPRs (G-FPR T-FPR = 10- Table lb illustratss the result with different FPRs (G-FPR 10 and T-FPR 10-3) for generation and evaluation. As shown Table Ia] adversarial examples generated against models trained with softmax loss exhibit certain transferability compared to models trained with cosine loss. We conduct the same experiment by generating adversarial examples with CW and found has weaker transferability compared to SemanticAdy (results in brackets of Table TT.
As Table Tb illustrates, the adversarial examples generated against the model with smaller G-FPR 10-0 exhibit strong attack success rate when evaluating on the model with larger T-FPR 10-3 Especially, we found the adversarial examples generated against R-101-S have the best attack performance other models. These findings motivate the analysis of query-free black-box API attack detailed in the following paragraph.
7

---

Under review as a conference paper at ICLR 2020
Mire Map
R-50-S
R-101-S
R-50-C
R-101-C
Mana
MMMe
R-50-5
R-1OT-S
R-50-8
1,000 000)
0.108 (0.007)
0,023 (0.00)
0 (0.002)
R-50-8
(1,000
0.862 (0.530)
R-101-S
0.169 (0.006)
1,000 (1.000)
0.0.0 (0. (0.002)
0.032 (0.003)
R-101-S
0.874 (0.422)
(..001.000)
R-50C
0.166 (0.019)
0.202 (0.025)
1,000 (1.000)
0,048 (0.007)
R-50-C
0.693 (0,347)
0.837 (0.579)
R-10I-C
0.120 (0.015)
0.236 (0.029)
0.040 (0.000)
(L.00)
R-101-C
0.617 (0.218)
0.888 (0.617)
(a)
(b)
Table I: Transferability of SemanticAdy: cell (i, j) shows attack success rate of adversarial examples generated against j-th model and evaluate on i-th model. Results of CW are listed in brackets. Left: Results generated with G-FPR 10-3 and T-FPR 10 10-3; Right: Results generated with G-FPR 10 and T-FPR- 10-3
Query-free black-box API attack. this experiment, we generate adversarial examples against R- 101-S with G-FPR = 10-3 = 1.24), G-FPR = 10 = 0.60), and G-FPR < 10-2 (5: = 0.20), respectively. We evaluate our algorithm on two industry level APIs, namely, Face++ and Yli face verification platform. Since attack transferability has never been explored in concurrent work that generates semantic adversarial examples, we use Lp bounded pixel-wise methods (Carlini & Wagner 2017; Dong et al. 2018) Xie et al. 2019) our baselines. As see Table which shows the best performance of each method, our SemanticAds achieves much higher attack success rate than CW in both APIs with all FPR thresholds (e-g., our adversarial examples generated with G-FPR 10-1 achieves 64.63% attack success rate on Face++ platform with T-FPR- = 10-3) In addition, we found that lower G-FPR can achieve higher attack success rate on APIs within the same T-FPR (see Table in Appendix).
API name
Face++
AliYun
Attacker Evaluation Metric
T-FPR 10
T-FPR = 10
T-FPR 10
T-FPR 10-1
Dong et al. (2018)
30.77
21.03
18,00
6.50
Xie et al. 2019
37.95
25.64
21.50
11.00
CW CG-FFR< < <
41.62
24.37
19.00
12.00
SemanticAdv (G-FPR 10
64.63
42.69
35.50
22.17
Table 2: Quantitative analysis query-free black-box attack. We use ResNet- 101 optimized with fmma loss for evaluation and report the attack success rate(%) (%) on two online face verification platforms. Note that for PGD-based attacks, we adopt MI-FGSM (c = 8) in Dong et al.) (2018) and M-DI--FGSM (e 8) in Xie et al. (2019), respectively.
User study. To measure the perceptual quality of the adversarial images generated by SemanticAdy, we conduct user study on Amazon Mechanical Turk (AMT). We use the adversarial examples generated with G-FPR < 10-4, which is the most strict setting in our experiment, to conduct the user study for both CW and SemanticAdv. In total, we collect 2, 620 annotations from 77 participants. In 39.14-+ 1.66% (close to random guess 50%) of trials the adversarial images generated by SemanticAdv selected reasonably-looking images and in 30.27 + 1.96% of trials, the adversarial images generated by CW are selected as reasonablyylooking It indicates that SemanticAdv can generate more reasonable-looking adversarial examples compared with CW under the most strict setting with G-FPR < 10-4, Qualitative comparisons are shown in Appendix (see Figure H
ResNat-50 (softmax
RecNat-101 (softmax loss)
ResNet-50 loss]
ResNet-101 loss)
1.0
1.0
1.0
1.0
Semar ticAd vo. PPe
Somartica ticde vo. aarr Conorticadi vs. Squeece CM vs, PPG
0.6
0.5
0.6
0.4
0.4
CM
Squacce
0.2
0.7
0.0
10
3-10-1
10.2
10"1
3-10-1
10
10"
3-10-
10
10"1
3'10-4
10-7
Dalee Fondiie Rate
Taxe Fontrice Bate
Faloe Posditue Date
7alte Ponfiiue Rate
Figure 5: Quantitative analysis on attacking several defense methods including JPBG Dziugaate
et al. 2016), Blurring [i Li 2017). and Feature Squeezing (Xu et al. 2017).
SemanticAdy against defense methods. We evaluate the strength of the proposed attack by test- ing against four existing defense methods, namely, Feature squee zing (Xu et al.) (017). Blurring (Li & Li 2017). JPEG (Dziugaite et al. (016)) and AMI (Tao et al., 2018). For AMI (Tao
al. 2018). we first extract attribute witnesses with our aligned face images and then leverage them
8

---

Under review as a conference paper at ICLR 2020
Taske "ontioooregiinn""
SEGERA STAG
Advicant Srugy
SCREN TEERS
SNNrIca BEAD
A
Task: "rotating attack
ditedkon
rnfeeren
refenee
Figure 6: Qualitative results on attacking face landmark detection model.
Task pedestrion removal attack
Syrthmited brage Augreented
Anterooriial Traage oysioiaation Seraghiie
Souree Invige
Seuene (pendittion)
Lapsat
Thrpet Lacous
Maripazated Imagg
Stasipslane
Manipslevdiapps
Task: eyelist pedesrrian attack
Atterorria hrage Optinesttio wraggiin,uu
Source Lapozi (erdiitiial
Laoset Spranddinati
Tucget tnahi
Marigsfatst Image
Verguraard (indinine)
Ventgrbben spore Ggiaandeenii
Figure 7: Qualitative results attacking street-view semantic segmentation model.
construct attribute-steered model. We Ec7 of pretrained VGG (Parkhi et al. 2015 the face representation. AMI yields a consistency score for each face image to indicate whether it a benign image. The score is measured the cos i similarity between the representations from original model and attribute-steered model. With 10% false positives on benign inputs, it only achieves 8% detection accuracy for SemanticAdy and 12% detection accuracy for CW.
Figure Saarressllluee iillustrates SemanticAdy is more robust against these defense methods comparing with CW: The same G-FPR and T-FPR used for evaluation. Under the condition that T-FPR 10-3 both SemanticAdv and CW achieve high attack success rate, while SemanticAdy marginally outperforms CW when FPR goes down 10 While defense methods have proven be effective against CW attacks on classifiers trained with ImageNet (Krizhevsky et al. (012). our results indicate that these methods are still vulnerable in face verification system with small T-FPR.
4.3 SemanticAdy ON FACE LANDMARK DETECTION
We also evaluate the effectiveness of SemanticAdy on face landmark detection. We select attack tasks, namely, "Rotating Eyes" and "Out of Region". For the "Rotating Eyes" task, we rotate the coordinates the eyes the image counter-clockwise by 90". For the "Out of Region" task, we set target bounding box and attempt to push all points out of the box. We summarize the experimental setup and quantitative results the Appendix (see Table D). As we see in Figure 6 our method is applicable to attack landmark detection models.
4.4 SemanticAdy ON STREET-VIEN SEMANTIC SEGMENTATION
We further demonstrate the applicability of our SemanticAdy beyond face domain by generating adversarial perturbations on street-view images. Figure 7 illustrates the adversarial examples on semantic segmentation. In the first example, object instance and insert another car into the scene to attack it. The segmentation model has been successfully attacked to neglect the pedestrian (see last column), while it does exist in the scene (see second-to-lass column). the second example, we insert an adversarial car in the scene by SemanticAdy and the cyclist has been recognized pedestrian by the segmentation model.
select the leftmost the pedestrian the target

---

Under review as a conference paper at ICLR 2020
5 CONCLUSIONS
Overall we presented novel attack method SemanticAdy, which is capable of generating unrestricted adversarial perturbations guided by semantic attributes edition. Compared to existing methods, SemanticAdy works in more controllable fashion. Experimental evaluations on face verification and landmark detection demonstrate several unique properties including attack transferability. We believe this work would open up new research opportunities and challenges in the field of adversarial learning. For instance, how to leverage semantic information to defend against such attacks will lead to potential new discussion.
REFERENCES
Alibaba Cloud Computing Co. Ltd.
53535 html
Megvii Technology Co. Ltd. http:://conslle.facegg.uusssss.. cot.oocucoccccccccccccooos 5679308
Yoshua Bengio, Grégoire Mesnil, Yann Dauphin, and Salah Rifai. Better mixing via deep representa- tions. In ICML., 2013.
Anand Bhattad, Min Jin Chong, Kaizhao Liang, Bo Li, and David A Forsyth. Big but imperceptible adversarial perturbations via semantic manipulation. arXiv preprint 47X1v:1904.06347, 2019.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2019.
Tom B Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and Ian Good- fellow. Unrestricted adversarial examples. arXiv preprint 1809.088800083300 2018.
Adrian Bulat and Georgios Tzimiropoulos. Binarized convolutional landmark localizers for human the IEEE International
pose estimation and face alignment with limited resources. Proceedings the Conference on Computer Vision, pp. 3706-3714 2017a.
Adrian Bulat and Georgios Tzimiropoulos. How far we from solving the 2d & 3d face alignment mroblem?(and a dataset of 230,000 3d facial landmarks). In ICCV, 201
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (S&P). IEEE, 2017.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In CVPR, 2018.
Moustapha Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling deep structured prediction models. In NIPS, 2017.
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedingy of the 25th international conference on Machine learning, pp. 160-167, ACM, 2008
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Ltsc Deng, Jinyu Li, Jui- Ting Huang, Kaisheng Yao, Dong Yu, Frank Seide. Michael L Seltzer, Geoffrey Zweig, Xiaodong He, Jason Williams, al. Recent advances in deep learning for speech research at microsoft. In ICASSP, volume 26, pp. 64, 2013.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018.
10

---

Under review as a conference paper at ICLR 2020
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv: 00803,, 2016.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and translation suffice: Fooling cnns with simple transformations. arXiv preprint arXiv:1712.02779, 2017.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, 2014a.
lan Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume MIT Press, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2014b.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In ECCV. Springer, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Seunghoon Hong. Xinchen Yan, Thomas S Huang, and Honglak Lec. Learning hierarchical semantic image manipulation through structured representations. In NeurlPS, 2018.
Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Seamed-Milles Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in' Real-Life" Images: detection, alignment, and recognition, 2008
Phillip Isola, Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In CVPR, pp. 1125-1134, 2017.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV. Springer, 2016.
Justin Johnson, Agrim Gupta, and Li Fei-Fei. Image generation from scene graphs. In CVPR, pp. 1219-1228, 2018.
Ameya Joshi, Amitangshu Mukherjee, Soumik Sarkar, and Chinmay Hegde. Semantic adversarial attacks: Parametric transformations that fool deep classifiers. arXiv preprint 1wXiv:190400848) 2019.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018
Ira Kemelmaccher-Shizzeree Steven M Seitz, Daniel Miller, and Evan Brossard. The megaface benchmark: million faces for recognition at seale. In CVPR, pp. 4873-4882, 2016.
Diederík Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Diederik Kingma and Max Welling, Auto-encoding variational bayes. In ICLR, 2014.
Brendan F Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K Jain. Pushing the frontiers of unconstrained face detection and recognition: larpa janus benchmark In CVPR, 2015.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In NIPS, 2012.
Xin Li and Fuxin Li. Adversarial examples detection in deep networks with convolutional filter statistics. In Proceedings of the IEEE International Conference Computer Vision, pp. 5764- 5772, 2017.
Ming-V Yu Liu. Thomas Breuel, and Jan Kautz. Unsupervised amage-to-image translation networks. In NIPS, 2017.
11

---

Under review as a conference paper at ICLR 2020
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In CVPR, 2015.
Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov. Generating images from captions with attention. In ICLR, 2015.
Seyed-Mohsen
Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574-2582, 2016.
Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, in-the-wild age database. In CVPR Workshops, pp. 51-59, 2017.
Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose estimation. In European Conference on Computer Vision, pp. 483-499, Springer. 2016.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In JMLR, 2017.
Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In ICML, 2016.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In Security and Privacy (EuroS&P), 2016 IEEE European Symposium on, 2016.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, et al. Deep face recognition. In bmvc, volume 1, pp. 6, 2015.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2015.
Scott Reed, Kihyuk Sohn, Yuting Zhang, and Honglak Lee. Learning to disentangle factors of variation with manifold interaction. In ICML, 2014.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In /CML., 2016.
Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the- wild challenge: The first facial landmark localization challenge. In ICCV Workshop, 2013.
Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. Frontal to profile face verification in the wild. In WACV, pp. 1-9. IEEE, 2016.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schritwieser, loannis Antonoglou, Veda Panneershelvam, Mare Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484, 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. In Advances in Neural Information Processing Systems, pp. 8312-8323, 2018
Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation from predicting 10,000 classes. In CVPR, 2014.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint a3 2013.
12

---

Under review as a conference paper at ICLR 2020
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, et al. Going deeper with convolutions. In CVPR, 2015.
Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretability; Attribute- steered detection of adversarial samples. In NeurlPS, 2018.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. In NIPS, 2016.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In CVPR, 201 8a.
Ting-Chun Wang, Ming- Yu Liu, Lun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and semantic manipulation with conditional gans. In CVPR, 2018b.
Chaowei Xiao, Ruizhi Deng. Bo Li, Fisher Yu, Mingyan Liu, and Dawn Song. Characterizing adversarial examples based on spatial consistency information for semantic segmentation. In ECCV, 2018a.
Chaowei Xiao, Bo Li, Jun- Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. In !JCAI, 2018b.
Chaowei Xiao, Jun-Yan Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. In ICLR, 2018c.
Chaowei Xiao, Dawei Yang, Bo Li, Jia Deng, and Mingyan Liu. Meshadv: Adversarial meshes for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6898-6907, 2019.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.0115: 2017.
Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. Attrbbteeimage: Conditional image generation from visual attributes. In ECCV. Springer, 2016.
Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. Dilated residual networks. In Computer Vision and Pattern Recognition (CVPR), 2017.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stackgan: Text photo-realistic image synthesis with stacked generative adversarial networks. In 2017.
Xingcheng Zhang, Lei Yang, Junjie Yan, and Dahua Lin. Accelerated training for massive classifi- cation via dynamic class selection. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Mun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A Efros. Generative visual manipulation on the natural image manifold. In ECCV. Springer, 2016a. 6a.
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-imane translation using cycle-consistent adversarial networks. In ICCV, 2017.
Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, and Stan Li. Face alignment across large poses: A 3d solution. In CVPR, 2016b.
13

---

Under review as a conference paper at ICLR 2020
A FACE IDENTITY VERIFICATION
Benchmark performance. First, we provide additional information about the ResNet models we used in the experiments. We summarize in Table A the performance on several face identity verifica- tion benchmarks including Labeled Face in the Wild (LFW) dataset (Huang et al. 2008). AgeDB-30 dataset (Moschoglou et al. 2017). and Celebrities in Prontal-Profilee (CFP) dataset (Sengupta et al.
2016
M benchmarks
LFW
AgeDB-30
CFP-FF
CFP-FP
R-50-S
99.27
94.15
99.26
91.49
R-101-S
99.42
95.93
99.57
95.07
R-50-C
99.38
95.08
99.24
90.24
R-101-C
99.67
95.58
99.57
92.71
Table A: The performnace of ResNet models several benchmark datasets.
Identity verification thresholds. Table B shows the threshold values used in our experiments when determining whether two portrait images belong to the same identity or not. The selected FPR thresholds and normalized L2 distance between face features are commonly used when evaluating the performance of face recognition model Klare et al. (2015);
modelKlare
et al. (2016).
FPR/M
R-50-S
R-101-S
R-50-C
R- R-10I-C
10-3
1,181
1.244
1.447
1.469
3 x 10
1.058
1.048
1.293
1.242
10
0.657
0.597
0.864
0.809
Table B: The threshold values for face identity verification.
Implementation details. We use Adam (Kingma & Ba 2015) as the optimizer to generate ad- versarial examples for both SemanticAdv and CW. More specifically, we run optimization for up to 200 steps with fixed learning rate 0.05 for cases when G-FPR < 10-4, Otherwise, we run optimization for up to 500 steps with fixed learning rate 0.01. For pixel-wiss attack method CW, we use additional pixel reconstruction loss with corresponding loss weight to 5. We run optimization for up to 1, 000 steps with fixed learning rate 10
Evaluation metrics. To evaluate the performance of semanticAdv under different attributes, we consider three metrics as follows:
Best: if there is attribute among 17 attributes that can be successfully attacked, we count the attack success rate for this face identity as 1;
we calculate the average attack success rate among 17 attributes for the same face identity;
Average:
Worst: only when all of 17 attributes can be successfully attacked, we count the attack success rate for this person as 1;
Note that, for a fair comparison with CW, we should use the Best metric for our SemanticAdv; as CW is the traditional pixel-wise attack method works regardless of the attribute. In addition, we report the performance using the average and wors! metric, which actually provides additional insights into robustness of face verification models across different attributes. For instance, combining the results from Table C and Figure 3] we understand that the face verification models used in our experiments have different levels of robustness across attributes. For example, face verification models are more robust against local shape variations than color variations, e.g., pale skin has higher attack success rate than mouth open. We believe these discoveries will help the community further understand the properties of face verification models.
Table C shows the overall performance (accuracy) of face verification model and attack success rate of SemanticAdy and CW. As we can see from Table although the face model trained with cosine loss achieves higher face recognition performance, is more vulnerable to adversarial attack compared with the model trained with softmax loss.
14

---

Under review as a conference paper at ICLR 2020
G-FPR
Metrics M
R-50-S
R-TOI-S
R-50-C
R-1OT-C
Verification Accuracy
98.36
98.78
98.63
98.84
0.00
0.00
0.08
0.00
G(x'.c)
0.00
0.00
0.00
0.23
G cee (Best)
0.16
0.08
0.16
0.31
10-3
SemanticAdv (Best)
100.00
100.00
100.00
100.00
SemanticAdy (Worst)
91.95
93.98
99.53
99.77
SemanticAdy (Average)
98.98
99.29
99.97
99.99
CW
100.00
100.00
100.00
100.00
Verification Accuracy
97.73
97.97
97.91
97.85
x C()',c)
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
G(x c..cr)(Bst))
0.00
0.00
0.00
0.00
3 x 10-4
SemanticAdv (Best)
100.00
100.00
100.00
100.00
SemanticAdv (Worst)
83.75
79.06
98.98
96.64
SemanticAdy (Average)
97.72
97.35
99.92
99.72
CW
100.00
100.00
100.00
100.00
Verification Accuracy
93.25
92.80
93,43
92.98
0.00
0.00
0.00
0.00
G(x',c)
0.00
0.00
0.00
0.00
G(x', c. (Best)
0.00
0.00
0.00
0.00
10-4
SemanticAdv (Best)
100.00
100.00
100.00
100.00
SemanticAdv (Worst)
33.59
19.84
67.03
48.67
SemanticAdv (Average)
83.53
76.64
95.57
91.13
CW
100.00
100.00
100.00
100.00
Table C: Quantitative result of identity verification (%). It shows accuracy of face verification model and attack success rate of SemanticAdy and CW. x', G(x', c) and G(x', c(ee) are the intermediate results of our method before adversarial perturbation.
Defense methods. Feature squeezin ig (Xu et al. 2017 is simple but effective method by reducing color bit depth to remove the adversarial effects. We compress the image represented by 8 bits for each channel to 4 bits for each channel to evaluate the effectiveness. For Blurring (Li
& Li 2017). we use 3 x 3 Gaussian kernel with standard deviation to smooth the adversarial perturbations. For JPEG Dziugaite al. (2016), it leverages the compression and decompression remove the adversarial perturbation. We set the compression ratio as 0.75 in our experiment.
B FACE LANDMARK DETECTION
Implementation details. To perform attack on the face landmark detection model, we run opti- mization for up to 2, 000 steps with fixed learning rate 0.05. We set the balancing factor X (see Eq. to value 0.01 for this experiment.
Evaluation Metrics. We apply two different metries for two adversarial attack tasks respectively. For "Rotating Eyes" task, we use well-adopted metric Normalized Mean Error (NME) (Bulat & Tzimiropoulos] 2017b) for experimental evaluation.
1 N
p2ll2 WB B HB
TNME
where Pa denotes the k-th ground-truth landmark, Pk. denotes the k-th predicted landmark and /WB n He is the square-root of ground-truth bounding box, where Wn and H represents the width and height of the box.
For "Out of Region" task, consider the attack is successful if the landmark predictions fall outside a pre-defined centering region on the portrait image. Thus, we introduce metric that reflects the portion of landmarks outside of the pre-defined centering region: Your = Nuss where Now denotes the number of predicted landmarks outside the pre-defined bounding box and Nowa denotes the total number of landmarks.
Nos
15

---

Under review as a conference paper at ICLR 2020
Augmented Attribttes
(Metrics)
Blond Hair
Young
Eyeglasses
Rosy Cheeks
Smiling
Arched Eychrows
Bangs
Fale
Skin
Rotating eyes (rovec) +
28.04
14.03
17.28
8.58
13.24
19.21
23.42
15.99
10.72
reyion (rour) +
Table D: Quantitative results on face landmark detection (%) The two row shows the measured ratios (lower better) for "Rotating Eyes" and "Out Of Region" task, respectively.
We present the quantitative results of SemanticAdy on face landmark detection model in Table D including two adversarial tasks, namely, "Out of Region" and "Rotating Eyes". We observe that our method is efficient to perform attacking landmark detection models. For certain attributes such as "Eyeglasses", "Plae Skin", SemanticAdy is able to achieve reasonably-gon performance.
C USER STUDY
We conducted a user study on the adversarial images of SemanticAdy and CW used the experiment of API-attack and the original images. The adversarial images generated with G-FPR< 10-1 for both methods. We present a pair of original image and adversarial image to participants and ask them to rank the two options. The order of these two images is randomized and the images are displayed for 2 seconds in the screen during each trial. After the images disappear, the participants have unlimited time to select the more cassonably-lookinn each participant, one could only conduct at most 50 trials, and each adversarial image was shown to different participants. Some qualitative results are shown Figure H total, we collect 2, 620 annotations from 77 participants. In 39.14 + 1.96% of trials the adversarial images generated by SemanticAdv are selected as bessonably-lookiing images and 30.27 = 1.96% of trails, the adversarial images generated by CW are selected reasonably-looking images. It indicates that our semantic adversarial examples are more perceptual reasonably-looking than CW. Additionally, we also conduct the user study with larger G-FPR= 10 10-3 3 In 45.42 1.96% of trials, the adversarial images generated by SemanticAdy are selected as reasonably-loow images, which is very close to the random guess (50%).
image according to their perception. For
SemanticAdv
SemanticAdv
Rosy Cheeks
-Smiling
-Pale Skin
Figure H: Qualitative comparisons among ground truth, pixel-widd wise adversarial examples generated by CW, and our proposed SemanticAdv. Here, we present the results from G-FPR 10 so that perturbations are visible.
16

---

Under review as a conference paper at ICLR 2020
D ABLATION STUDY: FEATURE-SPACE INTERPOLATION
We conduct an ablation study on feature-space interpolation by analyzing attack success rates with different feature-maps in the StarGAN network. Table TableB]showa Shows the attack success rate R-101-S. Here, we use f to represent the feature-map after i-th up-sampling operation. fo denotes the feature- map before applying up- up-sampling operation. The result demonstrates that samples generated by interpolating on fu achieve the highest success rate. Since fg is the feature-map before decoder, still well embeds semantic information in the feature space. We adopt fo for interpolation in our experiments.
We also conduct qualitative comparison between attribute-space and feature-space interpolation. As shown in Figure !
images generated by attribute-space and feature-space interpolation are both
reasonably-looking.
T-FPR (G-FPR)
10
(10 3)
3 x
10
(3
10"4)
10
(10
()
Layer (f)
fo
f2
fo
f2
fg
f2
Atlack Success Rate
99.29
98.32
75.62
97.35
94.10
57.15
76.64
67.40
19.63
Table E: Attack success rate by selecting different layer's feature-map for interpolation on R-101-S(%). f, indicates the feature-map after i-th up-sampling operation.
T-FPR (G-FPR)
10
(10
5)
3 x
(3
10
9)
4
(10 +)
Layer (f)
f-
f
f-
f
f
f
Allack Success Rale
49.40
92.09
30.44
81.87
6.66
45.46
Table F: Attack success rate by selecting different layer's s feature-map for interpolation on R-101-
S(%). f-2 indicates the feature-map after the last down-sampling operation and f-2 indicates the feature-map after f-2.
Oratinal image
6o
0.2
04
0J8
2.0
Criaizal Imsaee
50
0.2
0.1
86
1.0
Antiane-qure Inteppaaion
dightyy
spen
Teancrspace Ineepelation
+barp
Figure I: Qualitative comparisons between attribute-space and feature-space interpolation.
E SEMANTIC ATTACK TRANSFERABILITY
In Table G we present the quantitative results of the transferability with G-FPR = 10-4, T-FPR 10-4 We observe that with the strict criterion (Lower T-FPR) of the verification model, the transferability becomes lower cross different models.
To explore whether the improvement transferability in SemanticAdu introduced by semantic editing rather than optimized semantic perturbation in feature space. We add StrawMan [CW + x"e"] baseline which uses a controllable semantic-attribute-base generator to generate semantically different images without any notion of an adversarial attack, and then applies standard L, CW attacks on that generated image. The results are shown in Table H The performance the StrawMan [CW + =ew] is worse than SemanticAdv. This result justifies that our SemanticAdy is able to produce novel adversarial examples which cannot be simply achieved by combining stribute-conditiin image editing model with T.p bounded perturbation.
17

---

Under review as a conference paper at ICLR 2020
Mua /Mas
R-50-S
R-101-S
R-50-C
R-101-C
R-50-S
1.000
0.005
0.000
0.000
R-101-S
0.000
1,000
0.000
0.000
R-50-C
0.000
0.000
1.000
0.000
R-101-C
0,000
0,000
0.000
1.000
Table Transferability of SemanticAdv: cell (4,j) shows attack success rate of adversarial examples generated against j-th model and evaluate on i-th model. Results generated with G-FPR 10
T-FPR 10 10-4.
Must/ Mapt
R-101-S
Must Mops
R-101-S
R-50-S
0.035 (0. 108)
R-50-S
0.615 (0.862)
R-101-S
1.000 (1.000)
R-
1.000 (1.000)
R-50-C
0.145 (0.202)
R-50-C
0.570 (0.837)
R-1OT-C
0.085 (0.236)
R-1OI-C
0.695 (0.888)
(a) G-FPR=10"*, T-FPR=10
(b) G-FPR=10 T-FPR=10-
Table H: Transferability of StrawMan: cell (4,j) shows attack success rate of adversarial examples generated against j-th model and evaluate on i-th model. Results of SemanticAdy are listed in brackets.
F QUERY-FREE BLACK-BOX API ATTACK
API name
Face++
AliYun
Attacker Evaluation Metric
T-FPR =
T-FPR = 10
T-FPR = 10
T-FPR = 10
Original
2.04
0.50
0.00
Generated x
4.21
0.53
0.50
0.00
CW (G-FPR = 10
16.24
3.55
4.50
0.00
StrawMan (CW
10.71
4.08
3,00
0.00
SemanticAdy (G-FPR 30
27.32
7.50
CW (G-FPR = 10
30.61
15.82
12.50
4.50
StrawMan (CW "M"
(0-FPR=1007)
21.32
9.14
7.50
1.50
SemanticAdy (G-FPR
57.22
38,66
29.50
17.50
Dong 2018
30.77
21.03
18.00
6.50
XIe al. 22019
37.95
25.64
21.50
CW (G-FPR < 10 10
41.62
24.37
19.00
12.00
StrawMan (CW nww (G-FPR < 40
27.69
15.38
10.00
3.00
SemanticAdy (G-FPR 40
64.63
42.69
35.50
22.17
Table I: Quantitative analysis on query-free black-box attack. We use ResNet- 101 optimized with softmax loss for evaluation and report the attack success rate(%).
In Table L we present the results of SemanticAdy performing query-free black-box attack on two online face verification platforms. SemanticAdy outperforms CW and StawnMan in both APIs under all FPR thresholds. In addition, we achieve higher attack success rate on APIs using samples generated using lower G-FPR compared samples generated using higher G-FPR with the same T-FPR. Original x and generated x are regarded as reference point of the performance of online face verification platforms.
Figure Sicuuttillsstrtess our SemanticAdy on attacking Microsoft Azure face verification system, which further demonstrate the effectiveness of our approach.
18

---

Under review as a conference paper at ICLR 2020
U
Imgg Ui
Welbuaae now Laai bekery fraet Corldeus scoem
Jees
Inser llc.
Incer
Inser
Mewchhrrm
balaay Seumsiaee hay keng Makerary Semaur Lagel
Fobdic Scars Bur Taay Semoduur Ruler Sppmr Bo
Figure J: Illustration of our SemanticAdy in the real world face verification platform (editing on pale skin). Note that the confidence denotes the likelihood that two faces belong the same person.
19

---

Under review as a conference paper at ICLR 2020
G
MORE VISUALIZATIONS
Sourcee Image
Targer
Generation via ottribun-coondttoone Image editing
Image
Synthesized image with augmeentdd attribute (a-0)
Adversriial image by optimizing interpolation parumeter (a-argmin, (4.))
*blondd hair
-wavy hair
"young
"er reyeellasees
-Jeewy makeup
ryosy cheeks
*chuubyy
-nouth silightly open
bushy eyebrows
-wearing lipatick
-milling
-arched espebrowss
+banga
*wearing earrings
Tbag under
rreceding hairtine
"pale
skin
Symthesized image augmented actribute (a=0)
Adversarial image hy opttimizing Interpolation parameter (omargmin,tAu) (1.))
+blande hair
ewavy hair
-young
+eyeglasses
-heavy makeup
PTosy cheeks
*chuabay
-mouth slightly open
bushy eyebrows
ewaaring lipstick
*smäling
earched eyohrows
-hangs
+wearing earrings
hags under eyes
trecading hairline
*pale skin
Symthesized image sugmented attribute (a-0)
Adversarial image by optimizing interpolation parameter (o (orargmin (.6))
waav hair
"young
eyegglasses
Theavy makoup
'TOBY cheeks
chubby
hair
alightly open
busshy eyebeows
wearing lppstick
samilling
arcced eyebrows
ibangg
*wearing earnings
baggs under eyas
recedding hairline
ppale skin
Synthesized image with augmented attribute (a-0)
Adversarial image
optimizing interpolation parameter (a-argmin (..))
-blonde hair
ewary hair
yyongg
+yyeglasses
-heavy makeup
mmay
echubby
mouth slightly open
bushy eyebrows
*eearing lipstick
smillig
arcced eyebrows
banns
weuring earzings
-bags under eyes
receding hairline
Figure K: Qualitative analysis on single-atrribute adversarial attack (G-FPR= = 100-3
20

---

Under review as a conference paper at ICLR 2020
Synthesized image with auggmented attribute (a+0)
Adversatial image by optimizing interpolation parumeter (orargmin, L..)
*blonde hair
-waaey hair
-young
resoytasses
-heavy makmup
-rosy cheelss
*chubby
moutth silightly open
bushy eyebrews
-tvearing
"smilling
-arched eyebrress
-baings
*wisuring earnings
bags under eyes
rreceding hairtine
palee skin
Synihesized image augmented actribute (a+0)
Arversriill image hy optimizing Interpolation parameter (a-ergmiin ((.))
bblande hair
-wavy hair
"young
eay eeglllssss
heavv makeup
"oosy cheeks
chubbyy
alightly open
busby eyebrows
ewearing lipstick
smallin
sarched eyebrows
-bunge
ewearing earrings
hags under eyes
*receding hairline
"pale
skin
Synthesized Image augmented attribure (a=i))
Adversarial image optimizing interpolation. parameter (o-argmin J.L)
bbaande
-wavy
young
eyegglasses
chubbby
mouth slightly open
makeup
cheeks
bushy eyebrows
"wearing lipsttik
-smiling
*arched eyehrows
+bangs
"wearing earrings
+bage under eyes
Precoding hairline
*pale skin
Synthesized image with augmented attribute
Adversurial optimizing Interpolation parameter (ssargmin, (.D)
-wayy hair
-yongg
*eyoglasses
-heavy cheeles
"chubby
mmouhh slightly open
hair
bushy ayabrows
wearinn lppstick
-millig
arched eyebrows
bags under eyes
recedding hairinna
earrings
Figure Qualitative analysis on single-attribute adversarial attack (G-FPR 10 10-3).
21

---

Under review as a conference paper at ICLR 2020
Synthesized image augmented attribure (a=0)
Adversarial Image optimizing interpolation parameter (o (orargmin (.,)
wwavy hair
"young
reyeglasees
hhevv makrup
Trosy cheeks
ehuubby
allghttl open
Iusshy eyebrows
wearing lipstick
ramilling
aar larched nyebrows
leangg
ewearing eurnings
bags under eyes
recedling helrlinee
pale skin
Synthesized image with augmented attribute
Adversartal image optimizing interpolation parameter (a-argmin J.L.)
-blonnde hair
ewary hair
"younng
-eyeglasses
*heaav makeeup
"chubbyy
cheeeies
sliihuly openn
bussy eyebrows
*reearing lipstick
*smillng
+bangs
"weuring earnings
-bags under eyes
*rocoding hairline
eyebrows
Figure M: Qualitative analysis on single-attribute adversarial attack (G-FPR = 10 10-3).
22

---

Under review as a conference paper at ICLR 2020
Sourcee Image
Torget Image
Mivenaral
Adver aarial Examples using CW
+blonde
skin
ssmiling
'ruceding Suitiiu
*E0g Cheeos
young
smiling
*eyeglas905
+beorr malczup
*0003 under
"pale SKIn
*banng
5Rghoy
"chuboy
#70ST
Ialund Toils
Taangg
smiling
som
Binnde Juir
Toung
+bongs
ewearinn lipetces
pplle
7055
carrings
*reosding hatrline
+bage unDer eyes
1pale Ski
mmothh slgltiy spun
"pal
blonde
Figure N: Qualitative comparisons between our proposed SemanticAdy (G-FPR 10 10-3) and pixel- wise adversarial examples generated by CW. Along with the adversarial examples, we also provide the corresponding perturbations (residual) on the right.
23

---

Under review as a conference paper at ICLR 2020
Our exampple
Our perturbation
CW example
CW perturbationl
editing oe rasy chankk
editing pale
Figure O: Qualitative analysis on single-attribute adversarial attack (SemanticAdv with G-FPR 10-3) by each other. Along with the adversarial examples, we also provide the corresponding perturbations (residual) on the right.
24


Reviews:
[{"review_id": "rkxCcRlYtH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes to generate \"unrestricted adversarial examples\" via attribute-conditional image editing. Their method, SemanticAdv, leverages disentangled semantic factors and interpolates feature-map with higher freedom than attribute-space. Their adversarial optimization objectives combine both attack effectiveness and interpolation smoothness. They conduct extensive experiments for several tasks compared with CW-attack, showing broad applicability of the proposed method.\n\nThe paper is well written and technically sound with concrete experimental results. I'm glad to suggest accepting the paper.\n\nWith the help of attribute-conditional StarGAN, SemanticAdv generates adversarial examples by interpolating feature-maps conditioned on attributes. They design adversarial optimization objectives with specific attack objectives for identity verification and structured prediction tasks. They provide experiments showing the effectiveness of SemanticAdv; analysis on attributes, attack transferability, black-box attack, and robustness against defenses; as well as user study with subjective. The qualitative results also look nice and the code base is open-sourced.\n\nA question out of curiosity, the last conv layer in the generator is used as the feature-map. How is the attack effectiveness of using other layers?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SkxDlf-ttr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors describe a method for adversarially modifying a given (test) example that 1) still retains the correct label on the example, but 2) causes a model to make an incorrect prediction on it. The novelty of their proposed method is that their adversarial modifications are along a provided semantic axis (e.g., changing the color of someone's skin in a face recognition task) instead of the standard $L_p$ perturbations that the existing literature has focused on (e.g., making a very small change to each individual pixel). The adversarial examples that the authors construct, experimentally, are impressive and striking. I'd especially like to acknowledge the work that the authors put in to construct an anonymous link where they showcase results from their experiments. Thank you!\n\nOverall, I think that this is interesting work that can help to broaden the study of adversarial examples and make them more applicable even in non-adversarial settings (e.g., by making models more robust to the changes in semantic attributes that the authors consider). There has been quite a bit of interest in the community in adversarial examples that are not just $L_p$ perturbations, and I believe that the authors' approach will encourage a good deal of follow-up research. \n\nHowever, my main concern with the paper is that in my opinion, it does not sufficiently address why it is important to generate adversarial examples in the way they do. For example:\n\n1) Is the argument that this is a more powerful attack surface, so adversaries should take note (and defenders should figure out how to defend against this)? If that is the case, what is the attack model under which these attacks are realistic? For example, the original $L_\\infty$ attacks are motivated in the sense that the adversarial examples are visually imperceptible, so they might not be noticed by the end-user. What is the equivalent argument for these semantic attacks?\n\n2) Is the argument that these semantic attacks somehow capture a more realistic part of the data distribution over all natural images, and therefore it is good to have models that perform well on these semantic adversarial examples even if we're not concerned about an adversary (e.g., because the model might generalize better to other tasks or be more causally correct)? If that's the case, then I think this needs to be explored more. For example, what about the following straw man baseline: use a controllable semantic-attribute-based generator to generate semantically different images without any notion of an adversarial attack, and then do standard $L_p$ attacks on that generated image? How would that be better or worse than the proposed method?\n\n3) Or is the argument that it is just good to be able to generate examples that models get wrong? If so, why, and why is this method better than other methods?\n\nI think the paper would be significantly stronger if the importance and implications of their work were explicated along the above lines. For this reason, my current assessment is a weak reject, though I'd be open to changing this assessment.\n\n=== Less critical comments, no need to respond or fix right away ===\n\nWhile the overall concept and approach was clear, I generally found the notation and mathematical exposition difficult to follow. Please be more precise. Here is a non-exhaustive list of examples from section 3:\n\na) I'm not sure what's the difference between $x^\\text{tgt}$ and $x^\\text{adv}$, or between $x^\\text{new}$ and  $x^*$. These seem to be used somewhat interchangeably?\n\nb) Equation 3 is the central optimization problem in the paper, and should be written out explicitly using $\\alpha$ as the optimization variable, instead of referring to equations 1 and 2 (in which $x^*$ doesn't even appear).\n\nc) I didn't understand equation 4. What does assuming $M(x^\\text{tgt}) = y^\\text{tgt}$ mean? What happens when that is not true?\n\nd) Equation 5: Why is $y$ in the right hand side by not in the left?\n\ne) Equation 6: $L_\\text{smooth}$ is missing an argument.\n\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "H1lWBP4oFH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes adversarial attacks by modifying semantic properties of the image. Rather than modifying low-level pixels, it modifies mid-level attributes. The authors show that the proposed method is effective and achieves stronger results than the pixel-level attack method (CW) in terms of attacking capability transferring to other architectures. Importantly, the authors show results on a variety of tasks, e.g. landmark detection and segmentation in addition to classification/identification. The most related work is Joshi 2019 and the authors show that the method used in that work (modification in attribute space) is inferior to modification in feature space still via attributes, as the authors proposed. However, I have a few comments and concerns:\n1) The authors mention on page 3 they assume M is an oracle-- what is the impact of this?\n2) The results in Table C don't look good-- the proposed method can *at best* (in a generous setup) equal the results of CW-- maybe I missed something but more discussion would be helpful.\n3) Is there a way to evaluate the merits of semantic modification (beyond attack success) in addition to \"does it look reasonable\"? The authors mention attribute-based modifications are more practical, how can this be evaluated? If attribute-based attacks are better, is there a cost to this? How easy is it to make attribute-based attacks compared to low-level ones?\n4) The authors mention that for their transferrability results, they \"select the successfully attacked...\" (page 7). What is the impact of this, as opposed to selecting non-successfully attacked samples?\n5) Re: behavior with defense methods, is the advantage of the proposed method a matter of training the defense methods in a tailored way, so they're aware of attribute-based attacks?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: I had a little bit of difficulty with my recommendation here, but in the end I don't feel confident in recommending this paper for acceptance, with my concerns largely boiling down to the lack of clear description of the overall motivation.

Standard adversarial attacks are meant to be *imperceptible* changes that do not change the underlying semantics of the input to the human eye. In other words, the goal of the current work, generating "semantically meaningful" perturbations goes against the standard definition of adversarial attacks. This left me with two questions:

1. Under the definition of semantic adversarial attacks, what is to prevent someone from swapping out the current image with an entirely different image? From what I saw in the evaluation measures utilized in the paper, such a method would be judged as having performed a successful attack, and given no constraints there is nothing stopping this.

2. In what situation would such an attack method would be practically useful?

Even the reviewers who reviewed the paper favorably were not able to provide answers to these questions, and I was not able to resolve this from my reading of the paper as well. I do understand that there is a challenge on this by Google. In my opinion, even this contest is somewhat ill-defined, but it also features extensive human evaluation to evaluate the validity of the perturbations, which is not featured in the experimental evaluation here.

While I think this work is potentially interesting, it seems that there are too many open questions that are not resolved yet to recommend acceptance at this time, but I would encourage the authors to tighten up the argumentation/evaluation in this regard and revise the paper to be better accordingly!
Decision: Reject