One Thing to Fool them All: Generating Interpretable, Universal, and Physically-Realizable Adversarial Features
Authors: Stephen Casper
Max Nadeau
Gabriel Kreiman
Venue: ICLR 2022 Submitted (2022)
Keywords: adversaries
interpretablity
generative modeling

Abstract: It is well understood that modern deep networks are vulnerable to adversarial attacks. However, conventional methods fail to produce adversarial perturbations that are intelligible to humans, and they pose limited threats in the physical world. To study feature-class associations in networks and better understand the real-world threats they face, we develop feature-level adversarial perturbations using deep image generators and a novel optimization objective. We term these feature-fool attacks. We show that they are versatile and use them to generate targeted feature-level attacks at the ImageNet scale that are simultaneously interpretable, universal to any source image, and physically-realizable. These attacks can also reveal spurious, semantically-describable feature/class associations, and we use them to guide the design of ``copy/paste'' adversaries in which one natural image is pasted into another to cause a targeted misclassification.

Under review as a conference paper at ICLR 2022
ONE THING TO FOOL THEM ALL: GENERATING INTERPRETABLE, UNIVERSAL, AND PHYSICALLY- REALIZABLE ADVERSARIAL FEATURES
Anonymous authors Paper under double-bling review
ABSTRACT
It is well understood that modern deep networks are vulnerable to adversarial at- tacks. However, conventional methods fail to produce adversarial perturbations that are intelligible to humans, and they pose limited threats in the physical world. networks and better understand the real-
To study feature-class associations world threats they face, we develop feature-level adversarial perturbations using deep image generators and a novel optimization objective. We term these feature- fool attacks. We show that they versatile and use them to generate targeted feature-level attacks at the ImageNet scale that are simultaneously interpretable, universal to any source image, and physicall--realizabll feature/class associations that can be ex-
These attacks also
reveal spurious, semantically-dessribbe ploited by novel combinations of natural objects. We use them to guide the design of "copy/paste" adversaries in which one natural image is pasted into another to cause a targeted misclassification.
INTRODUCTION
State-of-the- neural networks are vulnerable adversarial inputs, which cause the network to fail yet only differ from benign inputs in subtle ways. Adversaries for visual classifiers conventionally take the form of small-norm perturbation to a benign source image that causes misclassification Szegedy et al. 2013) Goodfellow et al. 2014). These are effective, but human, these perturba- noise. As such, analyzing these adversaries does not reveal information about the network relevant to how it will function-ane and how it may fail-when presented with numan-interpretable features. Another limitation with conventional adversaries the limited extent to which they are physically-realizzble.. While they can retain some effectiveness when photographed in a controlled setting (Kurakin et al,, 2016), they are less effective in uncon- trolled settings (e.g Kong et al. 2020 and cannot be created from natural objects.
tions typically appear as random or
Several works discussed in Section 2 have aimed to produce adversarial modifications that are uni- versal to any source image, interpretable, or physically-realizabbe But the best of our knowledge, none exist for accomplishing all three at once. To better understand networks and what threats they face in the real world, we set out to create adversarial examples with all of these desiderata. Fig. gives an example of one of our attacks in which a universal ad adversarial patch depicting crane is physically placed near sunglasses fool network into classifying the image pufferfish.
Because pixel-space optimization produces non-interpretable perturbations, the ability to manipulate images at a higher level needed. We take inspiration from recent advancements in generative modeling (e.g- Brock et al. (2018)) at the ImageNet (Russakovsky et al. 2015) scale. Instead of pixels, we perturb the latent representations inside of a deep generator to manipulate an image in feature-space. In doing so, produce adversarial features which are inserted into source images either by directly modifyyng the latents used to generate them or by inserting a generated patch into natural image. We combine this with loss that uses external discriminator and classifier to regularize the adversarial feature into appearing interpretable and not resembling the target class.
We this strategy to produce what term as feature-fool attacks in which feature-level manip- ulation causes a misclassification yet appears intelligible to a human without resembling the target class. We show that universal attacks more interpretable and better disguised than analo-

---

Under review as a conference paper at ICLR 2022
Sunglasses: 0.8101 Crane: 0.0 Puffer: 0.0
Sunglasses: 0.0
Sunglasses: 0.0107 Crane:
Crane: 0.9206
Puffer: 0.0
Puffer: 0.9783
An example of interpretable, universal, and physically realizable feature-fool attack against a ResNet50. The patch depicts a crane, however, when printed, physically inserted into scene with sunglasses, and photographed. causes misclassification was created by perturbing the latent of generator to manipulate the image in a feature-space and training with loss that jointly optimizes for fooling the classifier and resembling some non-target disguise class.
Figure 1:
pufferfish. The patch
gous pixel-space attacks while being able successfully transfer to the physical world. To further demonstrate their potential for interpretable and physically-realizbble attacks, we also use these ad- versaries to guide the design of copy/paste attacks which one natural image is pasted into another to induce unrelated misclassification. Based on these findings, emphasize the importance of cautious deployment for vision networks and their fortification against these types of feature-level adversarial attacks. The following sections contain related work, methods, experiments, and dis- cussion. For a jargon-free summary our work for readers who are less familiar with research in deep learning, Appendix A.7
RELATED WORK
Conventional adversaries (Szegedy et al. 2013
Goodfellow al. 2014 tend to be non-interpretable pixel-Ievel perturbations and have limited ability to transfer to the physical world. Here, we contex- tualize our approach with other work and natural examples related to overcoming these challenges.
Inspiration from Nature: Mimicry is common in nature, and sometimes, rather than holistically imitating another species' appearance, a mimic will only exhibit particular features. For example, many animals adversarial eyespots stun confuse predators (Stevens & Ruxton 2014 An- other example is the mimic octopus which imitates the patterning, but not the shape, of banded snake. We show in Figure 2 using photo of mimic octopus from Norman et al. (2011 that ResNet50 classifies it as a sea snake.
Generative Modeling: In contrast to pixel-space attacks, method hinges using generator to manipulate images at feature-level. One similar approach has been to train a generator or autoeneoder produce adversarial perturbations that are subsequently applied natural inputs. This has been done by Hayes & Danezis (2018); |opuri et al. (2018a b); Poursaeed et al. (2018); Xiao al. (0118; Hashemi
(2020): Wong & Kolter (200)) to synthesize attacks that are transferable, universal, or efficient to produce. Unlike these, however, we also explicitly focus on physical-raalizabiiity and human-intens craman-interpretabblityy Additionally, rather than training an adversary generator, ours and other related works have skipped this step and simply trained adversarial latent perturbations to pretrained models. Liu et al. (2088) did this with a differentiable image renderer. Song et al. (2018) and Joshi et al. (2019) used deep generative networks, (2020) who aimed to create more semantically understandable attacks by training an autoencoder with a "disentangled" embedding space. However, these works focus small classifiers trained on simple datasets (MNIST (LeCun et al. 2010), SVHN (Netzer et al. 201T). CelebA (Liu et al. 2015) and BDD (Yu al. 2018)). In contrast, work at the ImageNet (Russakovsky et 2015)
well as Wang et al.
ammamticlllyumdddesttaddbblll
2

---

Under review as a conference paper at ICLR 2022
Snake, King Snake, 0.151 Night Snake, 0.081 Ringneck Snake, 0.025
(a)
(b)
Figure 2: Adversarial features in nature. (a) A peacock and butterfly with adversarial eyespots (unfortunately, peacock and ringlet butterfly are ImageNet classes, one cannot meaningfully test how ImageNet networks might be misled by them). (b) A mimic octopus from Norman et al. (2001) is classified sea snake by ResNet50.
scale. Hu al. 202T) do this as well, but only using adversarial patches, while we do so with three types of attacks discussed in the following section. Finally, compared all of the above works, ours is also unique in that we directly regularize adversaries for interpretability and disguise with training objective rather than relying on perturbations in latent space alone.
Physically-Realizabl Attacks: Our first contribution related to physical realizability is inter- pretable attacks that fool classifier even when printed and photographed. This directly relates to the work of Kurakin et al.] (2016) who found that conventional pixel-space adversaries could do this limited extent controlled settings. More recently, (Sharif al. (2016); Brown et al. 20017:: EEkholt et al. (2018) Athalye et al. (018): Liu et al. (0019); Kong et al. (2020) Komkov & Petiushko (2021) used optimization under transformation to create adversarial clothing, stickers, patches, or objects for fooling vision systems. In contrast to each of these, we generate attacks that are not only thiysiaall--realzzan
but also inconspicuous in the sense that they are both interpretable and disguised. Our second contribution involving physically-realzzan attacks is "copy-paste" at- tacks discussed next.
Interpretable Adversaries: In addition to fooling models, our adversaries provide method for discovering semanticalllyddeerrbbbbb feature/class associations learned by network. This relates to work by Geirhos al. (2018) and Leclerc et al. (2021) who debug networks using rendering and style transfer to make a zero-order search over features, transformations, and textural changes in images that cause misclassification. More similar our work are Carter al. (2019) and Mu & An- dreas (2000) who develop interpretations of networks using feature visualization (Olah et al. al. 2017) and network dissection (Bau et al. 2017) respectively. Both find cases in which their interpretations suggest "copy/paste" attack in which natural image of one object is pasted inside another natural image cause a misclassification as a third object. We add this work with new method iden- tify adversarial features for copy/paste attacks, and unlike either previous approach, ours naturally does in a context-come conditioaa fashion.
3 METHODS
3.1 THREAT MODEL
We adopt the "unrestricted" adversary paradigm Song et al. (2018), which requires the network's classification of an adversarial example to differ from some oracle (c.g. human). The adversary's 's goal is to produce a feature that will cause a targeted misclassification without resembling the target class. In particular, we focus on attacks that are universal across a distribution of source images and physically-realizable We assume that the adversary has access to a differentiable image generator, additional auxiliary classifier (optionally), and images from the corresponding discriminator, target classifier's 's training distribution. We also require white-box access to the target classifier, though we present black-box attacks based transfer from ensemble to held-out classifier in Appendix A.2 The adversary is limited in that it can only change a certain portion of either the latent or the image, depending the type of attack use.
3

---

Under review as a conference paper at ICLR 2022
Generator
Classifier
Transformation
Noise
Image Creation
(A, B, or C)
Perturb
A. Patch
B. Region
Generalized Patch
Patch
Patch
Generated Image
Patched Image
Generated Image
Patched Image
Natural Image
Natural Image
Figure 3: Our fully differentiable pipeline for creating patch, region, and generalized patch attacks.
3.2 TRAINING PROCESS
Our attacks involve manipulating the latent representation inside of a single generator layer to pro- duce an adversarial feature. Fig. S outlines our overall approach. We produce three kinds of adver- sarial attacks, patch, region, and generalized patch:
Patch: We use the generator to produce a square patch that inserted in natural image.
Region: We randomly select square portion of the latent representation in generator layer span- ning the channel dimension but not the height or width dimensions and replace with a learned insertion. This analogous
patch of the image in its pixel representation. The modified latent then passed through the rest of the generator, producing the adversarial image.
This method produces patch that be of any shape, hence the name "gen- eralized" patch. First, we generate an image in the same way that we do for region attacks. Second, extract generalized patch. We do this by (I) taking the absolute-valued pixel-level difference between the original and adversarial image, (2) applying a Gaussian filter for smoothing, and (3) creating binary mask from the top decile of these pixel differences. We then apply this mask the generated image to isolate the region of the image that the perturbation altered. We can then treat this as a patch and overlay it onto an image in any location.
Generalized Patch:
Objective: For all attacks, we train perturbation 5 to the latent of the generator to minimize a loss that optimizes for both fooling the classifier and appearing as an interpretable, disguised feature:
arg min E
[xxem (C(A(z, 5, 1)), yusg) + Ieeg [(xx, (...d,.. 6, t))]
with X' distribution over images generator encodings of them (e.g. the validation set gen- eration distribution), T a distribution over transformations, a distribution over insertion locations (which only applies for patch and generalized patch adversaries), C' is the target classifier, A an maage-generating function, T.x emt a targeted crossentropy loss fooling the classifier, Mary the tar- get class, and Leeg regularization loss for interpretability and disguise.
Lreg contains several terms. Our goal is produce features that are interpretable and disguised to human, but absent the ability to scalably or differentiably have human in the loop, instead use Legg as a proxy. All terms in Legg for each type of attack are listed in the following section, but most crucially, includes three types attack, we differentiably resize the patch or the extracted generalized patch and pass through the discriminator and auxiliary classifier. We then add weighted terms to the regularization
calculated using discriminator and auxiliary classifier. For all
4

---

Under review as a conference paper at ICLR 2022
Adv Ring Snake: 0,755 Ing Beer Bottle: 0,9
Adv 8.685 Ing Stinkhorn: 0.999
lbizan Hound: 0.725
Adv Yoommb imp Silky Terriert 0,414
Adv Rousler: 0,678 Ing Padlocks 0.54
Img Yollow Slipperi 0.964
Sic Hustay: 0.009
Sre Running Shoes 0.060
5rc Limoustine 0.060
Src indigo Bunting: 0.003
Sec Springer Spaniel: 0. 0.033
Tgt
Tgt Saftshaker
Tyt Bernese
Scorpians 0.814
Adv Guenen: 8.847
Adv Hartebeart: 0.062
Adr Elaphant:
Border Collie: 0.155
0.092
Figure 4: Examples of universal patch (top), region (middle), and generalized patch (bottom) feature-fool attacks. Each patch and generalized patch labeled with its mean fooling confidence under insertion for random locations and source images (labeled 'Adv') and the confidence with which it is classified as the disguise class (labeled 'Img'). ). Numbers in each patch and generalized patch subfigure title come from different inputs, so they can add to be > 1. The region attacks are labeled with their confidence as their source class ('Src') and the target class ('Tgt').
loss based on (1) the discriminator's (D) logistic loss for classifying the input as fake, (2) the softmax entropy of a classifier's (C") output, and (3) the negative of the classifier's crossentropy loss for classifying the input the attack's 's target class. Thus, we have:
Leeg (a) Lingistic [D(P(a)) + H[C"( Lx.emt (C"(P(a), Mharg)]
Where P(a) returns the extracted and resized patch from adversarial image a, and is place- holder for additional terms explained Section 4.1 These terms encourage the adversarial feature to (1) look real, and (2) look like some specific class, but (3) not the target class of the attack. The choice of what disguise class to use is left entirely to the algorithm.
EXPERIMENTS
4.1 ATTACK DETAILS
We use BigGAN generators off the shelf from Brock al. (0188 using the implementation from Wolf (018), and perturb the Post-ReLU activations of the internal 'GenBlocks'. Notably, due to self attention inside of the BigGAN architecture, for region attacks, the change the output image is not square even though the perturbation to the latent is. By default, we attacked ResNet50 (He) we restrict patch attacks to attacking 1/16 of the image and region and general- ized patch attacks to 1/8. We found that performing our crossentropy and entropy regularization on our patches using adversarially-traine pretable results. Presumably, this relates to how adversarially-traine networks tend to learn more interpretable representations (Engstrom et al. 2019b: Salman et al., 2020 and better approximate the human visual system (Dapello al. 2020 So for crossentropy and entropy regularization, used a 2-network ensemble of an e = La and e = 3 Lax robust ResNet50s from Engstrom et al. 2019a) for regularization. For discriminator regularization, use the BigGAN class-conditional
al. 2016),
auxiliary classifiers produced subjectively more inter-
5

---

Under review as a conference paper at ICLR 2022
discriminator with uniform class vector input (as opposed one-hot vector). For patch ad- versaries, we train under colorjitter, Gaussian blur, Gaussian noise, random rotation, and random perspective transformations to simulate changes that physically-realizate to be robust to. For region and generalized patch ones, we only use Gaussian blurring and horizontal flipping. Also for region and generalized patch adversaries, difference from the original image using the LPIPs perceptual distance (Zhang et al., 2018 So & Durooiannv 2019). Finally, for all adversaries, we apply penalty on the total variation of the patch or change induced on the image to discourage figh-ffeeuency, less- dess-interpretable patterns. All experiments were implemented with PyTorch Paszzee al. 2
adversary would need
promote subtlety by penalizing the
Figure shows examples of universal feature level patch, region, and generalized patch attacks. In particular, the patches on the top row are effective at resembling disguise class to the network. (We also subjectively find that they resemble the disguise class to us as well.) However, when shrunk to the size of a patch and inserted into another image, the network sees them as the target class. This suggests size biases in how networks process features. And to the extent that humans also find these patches to resemble the target, this may suggest similar properties in the human visual system. However, it is key to recognize the framing effects when analyzing these images: recognizing target- class features given the target class versus given no information are different tasks (Hulmmnn & Diakopoulos] 20 Analyzing human perception of feature-level adversaries may be interesting direction for future work.
4.2 INTERPRETABLE, UNIVERSAL, PHYSICALLY- ATTACKS
To demonstrate that ceature-fool adversaries are interpretable and versatile, we generate ad- versarial patches which appear one object to human, cause a targeted misclassifica- tion by the network another, do univer- sally regardless the source image, and are moysicall--realizann
Universal Patch Attacks, Ablation Tests
1.0
We generated feature- fool patches using our approach and compared them to five alternatives. Four of which were ablation tests in which no generator was used in favor of optimization in pixel space and in
Interpretability Conf.
0.6
which each of the three regularization terms
discussed in Section 3.2 were omitted. In the final test, we omitted the generator and all three of the regularization terms, resulting in the same method as Brown et al. (2017). For each test, all else was kept identical includ- ing training under transformation and initializ- ing the patch as an output from the generator. This initialization allowed for the pixel-space and Brown controls to be disguised and was the same as the approach for generating dis- guised pixel-space patch attacks in Brown et al. 2017
Feature Fool
No Gen
0.2
No Disc
No Ent
Brown
0.0
0.0
0.2
0.6
0.8
1.0
Fooling Conf.
Figure 5: Feature-foo patch attacks produce more deceptive and interpretable targeted universal at- tacks than ablation tests. Fooling conf. gives tar- get class confidence. Interpretability conf. shows the auxiliary network's 's label class confidence for the patch. Attacks further up and right are better. Centroids are shown as stars.
In Silico: Before testing in the physical world, did so in silico with 100 universal attacks of each type with random target classes. Fig. 5 plots the results. On the x axis are target class fooling confidences. On the axis are the labeling confidences from the auxiliary classifier which we use as a proxy for human evaluation. For all types of attacks, crafting small patches to be targeted, universal attacks has variable success for both fooling and interpretability. The centroids for all attacks are denoted with stars and suggest that our feature-fools are both better at fooling the classifier and appearing interpretable average than all others with the exception of the adversaries trained without regularization on the entropy of the auxiliary classifier's output. However, the no- entropy regularization ablation adversaries had much lower interpretability confidences.

---

Under review as a conference paper at ICLR 2022
Jean:
Sunglasses: 0.9831
0,8213
Schipperks: 0.9828
Loggerhead: 0.9606
Jean: 0.9811
Sunglesses:
Wombat: 0.4674
Hermit 0.9998
Acorn
Mousetrap: 0.995
Ball Papper: 0.8919
Figure 6: Successful examples of universal. physically-realizable feature-fool attacks (top) and pixel-space attacks (bottom). See Appendix A.6 for full-sized versions of the patches.
The fact that the feature-fool adversaries are more interpretable than controls without the entropy regularization (i.e. appear as some class by the auxiliary classifier) is unremarkable because they were specifically trained better than all but one of the others in mean fooling confidence-partie particularly the tests that did not use a generator. We also subjectively find the feature-fool patches be more interpretable than the Brown controls. See Appendix A.6 for examples of feature-fool and Brown adversarial patches with high fooling confidences. Because they were initialized from generator outputs, some of the Brown patches have vencer-like resemblance to non-target class features, but nonetheless, we find clear from inspection that they contain higher-frequent patterns and are poorly disguised in comparison our feature-fool attacks
so. More notably though, the feature fool attacks fare significantly
In the Physical World:
Next, we compared the physical realizability of our attacks with ones using the methodology of Brown we generated 100 additional adversarial patches for each, selected the 10 with the best mean fooling confidence, printed them, and photographed them next to 9 different ImageNet classes of common household items no patch were correctly classified and analyzed the outputs of the classifier when the adversarial patches were added the physical scene.
I We confirmed that photographs of each object with
Figure G shows successful examples of these physically-realizale feature-fool and Brown patch attacks. Meanwhile, resizable and printable versions of 10 feature-fool and Brown patches are in Appendix AA6 The mean and standard deviation of the fooling confidence for the feature-fool attacks in the physical world were 0.312 and 0.318 respectively (n = 90) while for the Brown attacks, they were 0.474 and 0.423 (n 90). However, due to nonindependence between the results across classes due to the same set of patches being used for each class. These tests the physical world show that the feature-fool attacks were often effective but that there is high variability in this effectiveness. The comparisons to Brown attacks provide some evidence that unlike our results in silico, the feature-fool attacks may be less reliably successful in the real world than the controls. Nonetheless the overall level of fooling success between both groups was comparable.
do not attempt any hypothesis tests
4.3 INTERPRETABILITY AND COPY/PASTE ATTACKS
Using adversarial examples to better interpret networks has been proposed by Dong et al. 2017) and Tomsett et al. 2018). Unlike conventional adversaries, feature-level adversaries reveal feature-che class associations, which are potentially of greater practical interest. For experiments with interpreting a classifier, versions of attacks in without the regularization terms described in Section 3.2
Backkpack, hanana, bath towel, lemon, jeans, spatula, sunglasses, toilet tissue, and toaster.

---

Under review as a conference paper at ICLR 2022
Traffic Light?
Orig
Adv Mean Conf: 0.8661
Indian Afr. Elephant Blue Coloration?
Orig Mean Conf: 0.4368
Adv Mean Conf: 0.7263
Puffer Lionfish Butterfly Wings?
Orig Mean Conf: 0.0195
Adv
Figure 7: Patch, region, and generalized patch adversaries being used to guide three class-universal copy/paste adversarial attacks. Patch adversary example pairs on the left, region adversaries in the middle, and generalized patch adversaries on the right of each odd row. Six successful attack examples are on each even row.
We find that inspecting the resulting adversarial features suggest both useful and harmful feature- class associations. In the Appendix, Fig. | provides simple example of each.
Copy/Paste Attacks: A copy-paste attack one which a natural image inserted into another in order cause an unexpected misclassification. They are more restricted than the attacks in 4.2tion4.21jp because the features pasted into an image must be natural objects rather than ones whose synthesis can be controlled. As result, they are of high interest for developing physically-realizable attacks because they suggest combinations of real objects that could yield unexpected classifications. They also have precedent the real world. For example, feature insertions into pornographic images have been used to evade NSFW content detectors (Yuan et al. 2019
To develop copy/paste attacks, we select source and target class, develop class-universal adver- sarial features, and analyze them for common motifs that resemble natural objects. Then we paste images of these objects into natural images and pass them through the classifier. Two other works have previously developed copy/paste attacks, also via interpretability tools that discover feature- class associations: Carter et al. (2019) and Mu Andreas (020). However, compared to prior approaches, technique may be uniquely equipped to produce germane fooling features. Rather than simply producing features associated with the target class, our adversaries generate fooling fea- tures conditional on the distribution, X, source images (i.e. the source class) with which the
8

---

Under review as a conference paper at ICLR 2022
adversaries trained. This method allows any source/target classes to be selected, but we find the clearest success in generating copy/paste attacks when they are somewhat related (e.g. bee and fly).
Fig. gives thrce illustrative examples. For each attack, we show two example images for each of the patch, region, and generalized patch adversaries. Below these are the copy/paste adversaries with average target class confidence before and after feature insertion for the 6 (out of 50) images for the source class in the ImageNet validation set for which the insertion resulted in the highest target confidence. Overall, little work has been done on copy/paste adversaries, and thus far, methods have always involved a human in the loop. This makes objective comparisons between methods difficult. However we provide examples of in Appendix A.4 to compare with ours.
based tool inspired by Carter et al. (2019)
5 DISCUSSION
By using a generative model to synthesize adversarial features, we contribute more pragmatic understanding of deep networks and their vulnerabilities. As an attack method, our approach is sim- ple and versatile. Across experiments here and in the Appendix, we show that it can be used to pro- duce targeted, interpretable, disguised, universal, physically-realizen attacks the ImageNet level. To the best of our knowledge, we are the first to introduce a method with all of these capabilities. As interpretability method, this approach is also effective targeted means of searching for adversarially exploitable feature-class associations.
black-box, and copy/paste
Conventional adversaries reveal intriguing properties of the learned representations in deep neural networks. However, as a means of attacking real systems, they pose limited threats outside of the digital domain (Kurakin et al. 2016). Given feature-fool attacks, our copy/paste attacks, and related work, a focus on adversarial features and robust, physically-realizable attacks will be key to understanding practical threats. Importantly, even if deep network is adversarially trained to be robust to one class of perturbations, this does not guarantee robustness to others that may be used attack it in deployment. Consequently, we argue for focusing on pragmatic threats, training robust models (e.g. Engstrom et al.] (2019)); Dapello et al. (2020)) ), and the use of caution with deep networks in the real world. As a promising sign, we show in Appendix A.5 that adversarial training is useful against feature-fool attacks.
A limitation is that when more constraints are applied the adversarial generation process (e.g. uni- versality + ing required to find good ones. They also take more time to generate which could be bottleneck to using them for adversarial training. Further still, while we develop disguised adversarial features, we do not generally find them to be innocuous in that they often have somewhat unnatural forms typical of generated images. In this sense, our disguised attacks may nonetheless be detectable. Ultimately, this type of attack is limited by the efficiency and quality of the generator.
+ disguise), attacks are generally less successful, and more screen-
Future work should leverage new advances in generative modeling. One possibly useful technique could be to develop fooling features adversarially against a discriminator which is trained to rec- ognize them from natural features. We also believe that studying human responses to feature-level adversaries and the links between interpretable representations, robustness, and similarity to the pri- mate visual system (Dapello al. 2020) are promising directions for better understanding both net- works and biological brains. Ultimately, given that every single one of the 11 proposals for building safe, advanced AI outlined in Hubinger (200)) directly call for interpretability tools and/or adver- sarial robustness, we believe that continuing this type of work will be highly valuable for building AI systems that are more reliable and trustworthy.
REFERENCES
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pp. 284-293. PMLR, 2018.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. I Proceedings of IEEE conference on computer vision and pattern recognition, pp. -6549, 2017.

---

Under review as a conference paper at ICLR 2022
Anand Bhattad, Min Jin Chong, Kaizhao Liang, Bo Li, and David A Forsyth. Unrestricted adver- sarial examples via semantic manipulation. arXiv preprint arXiv:1904.063477 2019.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 1096, 2018.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprini arXiv: 171..00661..06.3. 2017.
Shan Carter, Zan Armstrong, Ludwig Schubert, Ian Johnson, and Chris Olah. Activation atlas.
Distill, 4(3)=e1,, 2019.
Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David D Cox, and James Di- Carlo. Simulating primary visual cortex at the front of cnns improves robustness to image perturbations. BioRxiv, 2020.
Yinpeng Dong, Hang Su, Jun Zhu, and Fan Bao. Towards interpretable deep neural networks by leveraging adversarial examples. arXiv preprint vrXiv:1708.054999 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.I 11929, 2020.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019a. URL https:://ttttttiit com. http///addlaaabbbbbon how.h/tbrrouuunseeeeee..
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Adversarial robustness as a prior for learned representations. arXiv preprint arXiv: 1904009900 2019b.
Kevin Eykholt, Ivan Eytimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1625-1634, 2018
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint 1rXiv:1871.12233 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 14226677 2014.
Atiye Sadat Hashemi, Andreas Bär, Saced Mozaffari, and Tim Fingscheidt. Transferable universal adversarial perturbations using generative models. arXiv preprint arXiv:2010. /49/9, 2020.
Jamie Hayes and George Danezis. Learning universal adversarial perturbations with generative models. In 2018 IEEE Security and Privacy Workshops (SPW), pp. 43 43-49. IEEE, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Yu-Chih- Ch.h-hTTTaa Hu. Bo-Han Kung, Daniel Stanley Tan, lun-Cheng Chen, Kai-Lung Hua, and Wen- Huang Cheng. Naturalistic physical adversarial patch for object detectors. In Proceedings of the IEFE/CVF International Conference on Computer Vision, pp. 7848-7857, 2021.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708. 2017.
Evan Hubinger. An overview 11 proposals for building safe advanced ai. arXiv preprint arXiv:2012.07532, 2020.
10

---

Under review as a conference paper at ICLR 2022
Jessica Hullman and Nick Diakopoulos. Visualization rhetoric: Framing effects in narrative visual- ization. IEEE transactions on visualization and computer graphics, 17(12):22312222 -24420 2011.
Ameya Joshi, Amitangshu Mukherjee, Soumik Sarkar, and Chinmay Hegde. Semantic adversarial attacks: Parametric transformations that fool deep classifiers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4773 2019.
Lim Kiat. Lucent. http:///t////tttttto/oo///eeeerr
2019.
Stepan Komkov and Aleksandr Petiushko. Advhat: Real-world adversarial attack on areface face id system. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 126.
IEEE, 2021.
Zelun Kong, Junfeng Guo, Ang Li, and Cong Liu. Physgan: Generating physical-world-rssilee adversarial examples for autonomous driving. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2020.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep con- volutional neural networks. Advances in neural information processing systems, 25:1097-1105 2012.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world, 2016.
Guillaume Leclere, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang. et al. 3db: A framework for debugging computer vision models. arXiv preprint arXiv:21066. 03805, 2021.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yam.leccn. http://ww..lc....om///////miii 2, 2010.
Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual-sensitive gan for generating adversarial patches. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 33, pp. 1028-1033 2019.
Hsuch-Tr Ti Derek Liu, Michael Tao, Chun-Liang Li, Derek Nowrouzezahrai, and Alec Jacobson. Beyond pixel norm-balls: Parametric adversaries using analytically differentiable renderer. 2018
arXiv preprint 1
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Luke Melas.
Pyttrchh-retrranndd-vit.
https://ggtthu..com/uukkee
ByTorchh-rretzainee---3 2020.
Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R Venkatesh Babu. Nag: Network for ad- versary generation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 742-751, 20188 2018a.
Konda Reddy Mopuri, Phani Krishna Uppala, and R Venkatesh Babu. Ask, acquire, and attack: Data-free uap generation using class impressions. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 19-34, 2018b.
Jesse Mu and Jacob Andreas. arXiv:2006. 14032, 2020.
Compositional
explanations of neurons.
arXiv preprint
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Mark Norman, Julian Finn, and Tom Tregenza. Dynamic mimicry in an indo-malayan octopus.
Proceedings of the Royal Society of London. Series B: Biological Sciences, 268(1478): 1758, 2001.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11)e77, 2017.
11

---

Under review as a conference paper at ICLR 2022
Nicolas Papernot, Patrick McDaniel, and lan Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Bue -uc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035, Curran Associates, Inc., 2019. URL http:
library.
pdf
Omid Poursaced, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial pertur- bations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4422 4431, 2018.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-255 2015.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better? In ArXiv preprint orXiv:2007.08489 2020.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security. pp. 1528- 1540, 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogmition. arXiv preprint arXiv:1409.1520 2014.
Uchida So and Ihor Durooiaano. Lpips pytorch. https:://githu..com/ -iinno03//
Leips-pytorch 2019.
Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. arXiv preprint arXiv: 180.00880,, 2018
Martin Stevens and Graeme D Ruxton. Do animal eyespots really mimic eyes? Current Zoology. 60(1), 2014.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference computer vision and patlern recognition, pp. 2818-28266 2016.
Richard Tomsett, Amy Widdicombe, Tianwei Xing. Supriyo Chakraborty, Simon Julier, Prudhvi Gurram, Raghuveer Rao, and Mani Srivastava. Why the failure? how adversarial examples can provide insights for interpretable machine learning. In 2018 21 International Conference on Information Fusion (FUSION), pp. 838-845. IEEF, 2018.
Shuo Wang, Shangyu Chen, Tianle Chen, Surya Nepal, Carsten Rudolph, and Marthie Grob- ler. Generating semantic adversarial examples via feature manipulation. arXiv preprint arXiv:2001.02297, 2020.
Thomas Wolf. Pytorch pretrained biggan. pyterch--peetrained---
2018.
Eric Wong and Zico Kolter. Learning perturbation sets for robust machine learning. arXiv preprint arXiv:2007.08450, 2020.
12

---

Under review as a conference paper at ICLR 2022
Chaowei Xiao, Bo Li, Jun- Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adver- sarial examples with adversarial networks. arXiv preprint arXiv: 2018.
Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, and Trevor Darrell. 0dd A diverse driving video database with scalable annotation tooling. arXiv 2(5):6, 2018.
preprint
Kan Yuan, Di Tang, Xiaojing Liao, XiaoFeng Wang. Xuan Feng, Yi Chen, Menghan Sun, Haoran Lu, and Kehuan Zhang. Stealthy porn: Understanding real-world adversarial images for illicit online promotion. In 2019 IEEE Symposium Security and Privacy (SP), pp. 952-966. IEEE, 2019.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings the IEEE conference computer vision and pattern recognition, pp. 586-595, 2018.
13


Reviews:
[{"review_id": "xvAfqeBsa1m", "reviewer": "Reviewer_zy7u", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Strengths]\n\nThe paper is well motivated to generate adversarial patches that are human-interpretable and physically realizable.\n\n[Weaknesses]\n\n1. In Section 3.2, the description of the *region* adversarial attack is confusing and unclear. Does this attack mean that only a square region of the latent representation is perturbed and the other regions of the latent representation are not perturbed? Then how to determine which channels should be perturbed? Besides, it is also confusing that given the generated adversarial image, what is the corresponding original image? Can users generate adversarial images for a given original image, which may be a natural image?\n\n2. The generation process of the binary mask in the *generalized patch* adversarial attack is ad-hoc. Why do the authors apply Gaussian smoothing before computing the binary mask? Why do the authors preserve the pixels with the top 10% pixel differences instead of the top 5% or top 20% pixel differences? The authors should explain the reasons for these settings.\n\n3. In Figure 4, both the adversarial patches and adversarial images do not look like realistic/natural images. Such adversarial patches cannot meet the requirements in real applications. The authors mention that one goal is to make the generated patches look real, but in Figure 4, it is easy to tell that these patches are generated instead of real images. \nBesides, many patches actually look like the target class, *e.g.* the second image in the first line looks like a macaw, and the third image looks like a hound. This conflicts with the disguise requirement.\n\n4. The comparison between the proposed method and other methods is not convincing. First, baselines are not enough. The authors only compare the proposed method with one baseline method in Section 4.2. Second, the test images shown in Figure 6 are too simple to simulate the scenario in real applications, which weakens the validation of the effectiveness of the proposed method. In real applications, objects and scenes will be much more complex. It would be better if authors could test the effectiveness of their method on real-world images of intermediate-level complexity, instead of using extremely simple images.\n\n5. The metric for interpretability in Section 4.2 is questionable. First, the prediction confidence cannot reflect the interpretability of images. For example, some fake OOD images may be classified with a high confidence. Thus, the prediction confidence measures the discriminative power of images, instead of interpretability. To this end, the loss of the discriminator may be more suitable, because the loss of the discriminator reflects whether the image looks real. Second, the comparison based on the prediction confidence is unfair. In the proposed method, the entropy of the classifier’s output is included in the loss function, which boosts the prediction confidence. However, the baseline method does not contain such a regularization term.\n\n6. In Section 4.3, it is claimed that the adversarial patches help people to understand the feature-class associations learned by the DNN. However, the experiment just illustrates that the generated features can successfully attack the DNN, but cannot prove that such features are exactly what the DNN learns for the target class. The feature representation learned by a network needs to be more rigorously analyzed. \n\n7. In odd rows of Figure 8, the generated patches are not realistic, as mentioned in the third weakness. And in even rows of Figure 8, the adversarial images generated using copy-paste attack are not realistic. For example, the traffic light can never appear on top of a bee. The authors are encouraged to add one more penalty term to make the final adversarial images (not the patch itself, but the natural image with the patch pasted) look realistic.\n\n8. Some details of the proposed method are not clear. \n- For three attacks in Section 3.2, I suggest authors use formulas or equations to describe the attacking method. Otherwise, it would result in ambiguities, especially for the generalized patch attack.\n- In the first loss function in Section 3.2, inputs of the image-generating function include an input image $x$. I guess that for patch and generalized patch adversaries, $x$ refers to the natural image. However, for region attacks, what does $x$ refer to? \n- In the second loss function in Section 3.2, there is an ellipsis in the end, which is confusing. What else does the loss include?\n- In Section 4.1, the use of a 2-network ensemble as the auxiliary classifier is ad-hoc. The authors are encouraged to clarify the reason for using an ensemble instead of a single network.\n- In Section 4.1, the authors mention two additional penalty terms, but do not give their explicit formulae, which hurts the reproducibility of the paper. Specifically, the authors mention that “for region and generalized patch adversaries, we promote subtlety by penalizing the difference from the original image using the LPIPs perceptual distance” and “for all adversaries, we apply a penalty on the total variation of the patch or change induced on the image.” However, there are no formulae given for these two penalty terms.\n", "summary_of_the_paper": "This paper focuses on the problem of generating human-interpretable and physically realizable adversarial patches. The authors propose three kinds of adversarial attacks based on feature-level perturbations of the latent representation of generative models. The authors also try to reveal feature-class associations using the proposed attacks. The paper is well-motivated, but many details are not introduced clearly, and experimental results are not convincing. ", "summary_of_the_review": "The paper is well-motivated, but many details are not introduced clearly, and experimental results are not convincing. Some claims are not supported by their experimental results. Thus, I think the paper should be rejected.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "FtqDILeSKm4", "reviewer": "Reviewer_k6Su", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- Their method can synthesize adversarial examples that are universal to any source image, uses interpretable features, and physically realizable, which is novel as far as I know.\n\nWeaknesses:\n- The motivation of this study is unclear. They mention two points: \"better understanding networks\" and \"what threats they face in the real world\", but here are my responses. \n\t- \"Better understanding networks\":\n\t\t- They motivate their method \"as a means of better understanding networks\" but the relevant section (I assume 4.3) only mentions that \"feature-level adversaries reveal feature-class associations, which are potentially of greater practical interest.\"  Existing feature visualization techniques can reveal feature-class associations. What new information do we gain by the proposed method in terms of understanding networks? (For instance, I believe we can use existing feature visualization techniques to produce something similar to examples in Figure 7. )\n\t- \"what threats they face in the real world\"\n\t\t- Why makes it significant to being able to produce adversarial perturbations that are intelligible to humans? They motivate this by saying it poses more threat in the physical world. In my view, the whole reason why adversarial examples are considered to be a threat is that these perturbations are almost imperceptible to humans, which makes it hard to detect even by humans, let alone computer vision systems. If perturbations are intelligible to humans, at least humans can detect these adversaries, which makes  less threatening in some sense, so I fail to understand their argument that intelligible perturbations pose more threat. Can you elaborate on this point?  \n\t\t- They claim that their findings \"emphasize the importance of cautious deployment for vision networks\" but that is something the community is already aware of, so it's not a strong argument to make to convey the significance of their work.  \n\n\n- Nitpick: In the introduction, the authors note that \"they pose limited threats in the physical world\" and \"they are generally ineffective in less controlled ones such as those experienced by autonomous vehicles\", but recently [1] shows that it's easy to attack self-driving cars with adversarial examples in the real world. \n\n[1] : Sato et al \"Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack\"", "summary_of_the_paper": "They propose feature-fool attack, a method to synthesize adversarial examples that are can be placed onto any source image, uses interpretable features, and physically-realizable.\nThey achieve this by perturbing latent representations of large scale generative models. \nIn doing so, they regularize adversarial features so that they appear interpretable but not similar to the attack's target class.\nThey apply their method to copy/paste attacks. ", "summary_of_the_review": "I think this paper has some interesting points, but it could be improved by making the motivation clear. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "63kXUg7ciO", "reviewer": "Reviewer_1mBx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The result that adversarial patches can be created that look like a different class to the target (and are classified as such) is a novel, interesting and important result. \n- The paper includes plenty of visual examples to demonstrate that the attacks are interpretable in the way they claim\n\nWeaknesses:\n- The paper contains very few quantitative experiments. The only comparison to existing methods is to compare the average fooling confidence and interpretability confidence of their adversarial patches against Brown et al patches. No quantitative experiments were provided for \"region\" and \"generalised patch\" attacks. To improve this more quantitative experiments should be included in the paper. Useful experiments would be ablation studies testing different combinations of pixel vs latent, discriminator vs no discriminator and entropy loss vs no entropy loss. Even better would be to include one or two more comparisons from some of the literature cited in Section 2.", "summary_of_the_paper": "This paper introduces an adversarial patch attack in which the patches are classified with high confidence as a different class to the target when classified alone. They are also visually interpretable by humans as this other class, in contrast to conventional adversarial patches, which often can be easily interpreted as the class they are targeting. The core technical contribution of this paper is the introduction of an explicit regularisation to cause the patch to be classified differently to the target class.", "summary_of_the_review": "The core idea of the paper is interesting and novel. I am concerned by the limited experiments, but this does not cause me to doubt the author's main claims. I lean very slightly towards accept but I think the paper might be better suited to a workshop.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5xZRDaPDe8Q", "reviewer": "Reviewer_6FCU", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method involves manipulating the latent representations inside of a single generator layer to produce an adversarial feature. Using these they propose three types of attacks: patch, region and generalized patch. The goal is targeted adversarial attack. There are multiple advantages of designing the attack like this. First, it allows a for humanly interpretable images. Next these can be applied to wide range of images. And importantly, unlike most adversarial work, these images are physically realizable (this is a big strength of this attack). The proposed targeted method also shows impressive performance in the experiments.\n\nMy main concern regarding the proposed method is how would the model perform under object detection and anomaly detection. For example, the attach show in attack 1 can be circumvented using object detection, which would crop two separate part of the image and hence mitigate the attack. The object detection approach might not work on all the showed examples, but then anomaly detection might hamper the progress. Given that the patch is visible easily, I assume this change will be recognizable to an anomaly detection method. That can also possibly render the attack useless. I would like to know the author's thoughts about these concerns. ", "summary_of_the_paper": "The authors propose an interpretable attack on DNNs that is also universal to source images and physically realizable as well. For this purpose they use generative model to synthesize adversarial features which are then added to the source image. The authors then show the efficacy of the method on experimental data.", "summary_of_the_review": "The paper is well written and the ideas are novel. The main strength of the paper is the general applicability of the attack. However, as mentioned in the previous section, my main concern comes from how the method would perform in the presence of object detection and anomaly detection. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this manuacript, the authors develop feature-fool attacks with feature-level adversarial perturbations using deep image generators and a novel optimization objective. They further show that the feature-fool attacks are versatile and can generate targeted feature-level attacks at the ImageNet scale that are simultaneously interpretable, universal to any source image, and physically-realizable.
The reviewers agree that the paper is well-motivated and the authors have addressed some concerns.
However, the reviewers still do not satisfy with some concerns so as to keep the initial scores.
In comparison with the manuscripts I'm handling, I have to recommend to reject it!
Decision: Reject