Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: They propose feature-fool attack, a method to synthesize adversarial examples that are can be placed onto any source image, uses interpretable features, and physically-realizable.
They achieve this by perturbing latent representations of large scale generative models. 
In doing so, they regularize adversarial features so that they appear interpretable but not similar to the attack's target class.
They apply their method to copy/paste attacks. 
main_review: Strengths:
- Their method can synthesize adversarial examples that are universal to any source image, uses interpretable features, and physically realizable, which is novel as far as I know.

Weaknesses:
- The motivation of this study is unclear. They mention two points: "better understanding networks" and "what threats they face in the real world", but here are my responses. 
	- "Better understanding networks":
		- They motivate their method "as a means of better understanding networks" but the relevant section (I assume 4.3) only mentions that "feature-level adversaries reveal feature-class associations, which are potentially of greater practical interest."  Existing feature visualization techniques can reveal feature-class associations. What new information do we gain by the proposed method in terms of understanding networks? (For instance, I believe we can use existing feature visualization techniques to produce something similar to examples in Figure 7. )
	- "what threats they face in the real world"
		- Why makes it significant to being able to produce adversarial perturbations that are intelligible to humans? They motivate this by saying it poses more threat in the physical world. In my view, the whole reason why adversarial examples are considered to be a threat is that these perturbations are almost imperceptible to humans, which makes it hard to detect even by humans, let alone computer vision systems. If perturbations are intelligible to humans, at least humans can detect these adversaries, which makes  less threatening in some sense, so I fail to understand their argument that intelligible perturbations pose more threat. Can you elaborate on this point?  
		- They claim that their findings "emphasize the importance of cautious deployment for vision networks" but that is something the community is already aware of, so it's not a strong argument to make to convey the significance of their work.  


- Nitpick: In the introduction, the authors note that "they pose limited threats in the physical world" and "they are generally ineffective in less controlled ones such as those experienced by autonomous vehicles", but recently [1] shows that it's easy to attack self-driving cars with adversarial examples in the real world. 

[1] : Sato et al "Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack"
summary_of_the_review: I think this paper has some interesting points, but it could be improved by making the motivation clear.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces an adversarial patch attack in which the patches are classified with high confidence as a different class to the target when classified alone. They are also visually interpretable by humans as this other class, in contrast to conventional adversarial patches, which often can be easily interpreted as the class they are targeting. The core technical contribution of this paper is the introduction of an explicit regularisation to cause the patch to be classified differently to the target class.
main_review: Strengths:
- The result that adversarial patches can be created that look like a different class to the target (and are classified as such) is a novel, interesting and important result. 
- The paper includes plenty of visual examples to demonstrate that the attacks are interpretable in the way they claim

Weaknesses:
- The paper contains very few quantitative experiments. The only comparison to existing methods is to compare the average fooling confidence and interpretability confidence of their adversarial patches against Brown et al patches. No quantitative experiments were provided for "region" and "generalised patch" attacks. To improve this more quantitative experiments should be included in the paper. Useful experiments would be ablation studies testing different combinations of pixel vs latent, discriminator vs no discriminator and entropy loss vs no entropy loss. Even better would be to include one or two more comparisons from some of the literature cited in Section 2.
summary_of_the_review: The core idea of the paper is interesting and novel. I am concerned by the limited experiments, but this does not cause me to doubt the author's main claims. I lean very slightly towards accept but I think the paper might be better suited to a workshop.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper focuses on the problem of generating human-interpretable and physically realizable adversarial patches. The authors propose three kinds of adversarial attacks based on feature-level perturbations of the latent representation of generative models. The authors also try to reveal feature-class associations using the proposed attacks. The paper is well-motivated, but many details are not introduced clearly, and experimental results are not convincing. 
main_review: [Strengths]

The paper is well motivated to generate adversarial patches that are human-interpretable and physically realizable.

[Weaknesses]

1. In Section 3.2, the description of the *region* adversarial attack is confusing and unclear. Does this attack mean that only a square region of the latent representation is perturbed and the other regions of the latent representation are not perturbed? Then how to determine which channels should be perturbed? Besides, it is also confusing that given the generated adversarial image, what is the corresponding original image? Can users generate adversarial images for a given original image, which may be a natural image?

2. The generation process of the binary mask in the *generalized patch* adversarial attack is ad-hoc. Why do the authors apply Gaussian smoothing before computing the binary mask? Why do the authors preserve the pixels with the top 10% pixel differences instead of the top 5% or top 20% pixel differences? The authors should explain the reasons for these settings.

3. In Figure 4, both the adversarial patches and adversarial images do not look like realistic/natural images. Such adversarial patches cannot meet the requirements in real applications. The authors mention that one goal is to make the generated patches look real, but in Figure 4, it is easy to tell that these patches are generated instead of real images. 
Besides, many patches actually look like the target class, *e.g.* the second image in the first line looks like a macaw, and the third image looks like a hound. This conflicts with the disguise requirement.

4. The comparison between the proposed method and other methods is not convincing. First, baselines are not enough. The authors only compare the proposed method with one baseline method in Section 4.2. Second, the test images shown in Figure 6 are too simple to simulate the scenario in real applications, which weakens the validation of the effectiveness of the proposed method. In real applications, objects and scenes will be much more complex. It would be better if authors could test the effectiveness of their method on real-world images of intermediate-level complexity, instead of using extremely simple images.

5. The metric for interpretability in Section 4.2 is questionable. First, the prediction confidence cannot reflect the interpretability of images. For example, some fake OOD images may be classified with a high confidence. Thus, the prediction confidence measures the discriminative power of images, instead of interpretability. To this end, the loss of the discriminator may be more suitable, because the loss of the discriminator reflects whether the image looks real. Second, the comparison based on the prediction confidence is unfair. In the proposed method, the entropy of the classifier’s output is included in the loss function, which boosts the prediction confidence. However, the baseline method does not contain such a regularization term.

6. In Section 4.3, it is claimed that the adversarial patches help people to understand the feature-class associations learned by the DNN. However, the experiment just illustrates that the generated features can successfully attack the DNN, but cannot prove that such features are exactly what the DNN learns for the target class. The feature representation learned by a network needs to be more rigorously analyzed. 

7. In odd rows of Figure 8, the generated patches are not realistic, as mentioned in the third weakness. And in even rows of Figure 8, the adversarial images generated using copy-paste attack are not realistic. For example, the traffic light can never appear on top of a bee. The authors are encouraged to add one more penalty term to make the final adversarial images (not the patch itself, but the natural image with the patch pasted) look realistic.

8. Some details of the proposed method are not clear. 
- For three attacks in Section 3.2, I suggest authors use formulas or equations to describe the attacking method. Otherwise, it would result in ambiguities, especially for the generalized patch attack.
- In the first loss function in Section 3.2, inputs of the image-generating function include an input image $x$. I guess that for patch and generalized patch adversaries, $x$ refers to the natural image. However, for region attacks, what does $x$ refer to? 
- In the second loss function in Section 3.2, there is an ellipsis in the end, which is confusing. What else does the loss include?
- In Section 4.1, the use of a 2-network ensemble as the auxiliary classifier is ad-hoc. The authors are encouraged to clarify the reason for using an ensemble instead of a single network.
- In Section 4.1, the authors mention two additional penalty terms, but do not give their explicit formulae, which hurts the reproducibility of the paper. Specifically, the authors mention that “for region and generalized patch adversaries, we promote subtlety by penalizing the difference from the original image using the LPIPs perceptual distance” and “for all adversaries, we apply a penalty on the total variation of the patch or change induced on the image.” However, there are no formulae given for these two penalty terms.

summary_of_the_review: The paper is well-motivated, but many details are not introduced clearly, and experimental results are not convincing. Some claims are not supported by their experimental results. Thus, I think the paper should be rejected.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose an interpretable attack on DNNs that is also universal to source images and physically realizable as well. For this purpose they use generative model to synthesize adversarial features which are then added to the source image. The authors then show the efficacy of the method on experimental data.
main_review: The proposed method involves manipulating the latent representations inside of a single generator layer to produce an adversarial feature. Using these they propose three types of attacks: patch, region and generalized patch. The goal is targeted adversarial attack. There are multiple advantages of designing the attack like this. First, it allows a for humanly interpretable images. Next these can be applied to wide range of images. And importantly, unlike most adversarial work, these images are physically realizable (this is a big strength of this attack). The proposed targeted method also shows impressive performance in the experiments.

My main concern regarding the proposed method is how would the model perform under object detection and anomaly detection. For example, the attach show in attack 1 can be circumvented using object detection, which would crop two separate part of the image and hence mitigate the attack. The object detection approach might not work on all the showed examples, but then anomaly detection might hamper the progress. Given that the patch is visible easily, I assume this change will be recognizable to an anomaly detection method. That can also possibly render the attack useless. I would like to know the author's thoughts about these concerns. 
summary_of_the_review: The paper is well written and the ideas are novel. The main strength of the paper is the general applicability of the attack. However, as mentioned in the previous section, my main concern comes from how the method would perform in the presence of object detection and anomaly detection.
review_title: null

Meta-review: In this manuacript, the authors develop feature-fool attacks with feature-level adversarial perturbations using deep image generators and a novel optimization objective. They further show that the feature-fool attacks are versatile and can generate targeted feature-level attacks at the ImageNet scale that are simultaneously interpretable, universal to any source image, and physically-realizable.
The reviewers agree that the paper is well-motivated and the authors have addressed some concerns.
However, the reviewers still do not satisfy with some concerns so as to keep the initial scores.
In comparison with the manuscripts I'm handling, I have to recommend to reject it!
Decision: Reject