Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper introduces two new adversarial attacks: one is generating adversarial examples by colouring the original images and the other is by changing textures of the original images. Specifically, the former one minimises the cross-entropy between the output of the classifier and the target label with the network weights of a pre-trained colourisation network. While the latter minimises the cross-entropy as well as the loss that defines the texture differences.

I think the general idea of going beyond perturbations of pixel values in this paper is interesting and the proposed approaches of attacking on colour and textures are intuitive and reasonable. The results seem to be promising with comprehensive experiments including whitebox attack, blackbox attack by transferring, and attacks on defences.

The paper overall is well-written and easy to follow. But I think the part of attacking for captioning is a bit distracted and there is no comparison with others on this task. I expect existing attacks on pixel can also do this task.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposed to generate semantically meaningful adversarial examples in terms of color of texture. In order to make manipulated images photo-realistic, colors to be replaced are chosen by energy values, while textures are replaced with style-transfer technique.

The paper is written clearly and organized well to understand. The graphs and equations are properly shown. The idea of using color replacement and texture transfer is interesting and novel.

A somewhat weakness is that the discriminator - a pretrained ResNet 50 - is too weak for this scenario. What about a ResNet 50 trained on augmented datasets with color jittering?
What about finetuned ResNet 50 with taking color channel as explicit input, since the attack uses this additional info.

As tAdv attack seems to manipulate high frequency texture of images, how about applying a Gaussian filter on the images and feed into the discrimimator again? Is that attack still effective or not?
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper proposes cAdv and sAdv, two new unrestricted adversarial attack methods that manipulates either color or texture of an image. To these end, the paper employes another parametrized colorization techniques (and texture transfer method) and proposes optimization objectives for finding adversarial examples with respect to each semantic technique. Experimental results show that the proposed methods are more robust on existing defense methods and more transferrable accross models. The paper also performs a user study to show that the generated examples are fairly imperceptible like the C&W attack. 

In overall, I agree that seeking a new way of attack is important, and the methods are clearly presented to claim a new message to the community: adversarial examples can be even found by exploiting semantic features that humans also utilize, since DNNs tend to overly-utilize them, e.g. colors. These claims are supported by the experiments showing that the generated examples are more transferrable across robust classifiers. Personally, I liked the idea of using another colorization method to design cAdv and the use of K-means clustering to control the imperceptibility. 

- Some readers may wonder how the "averaged-case" corruption robustness behave for both cAdv and sAdv, e.g. considering random colorization. Would it be worse than the robustness on Gaussian noise?
- One of my concerns on tAdv is whether the texture added is indeed effective to reduce the accuracy, or its just from the (yet small) beta term in the objective. Adding an ablation of beta=0 case in the result would much help the understanding of the method. 
- Eq 1: I think F should denote the classifier to attack, but the description tells it's the colorization network. As it seems to me that theta is nevertheless for the colorization network, I feel the notation should be refined for better understanding to the readers.
review_title: Official Blind Review #3

Meta-review: In this paper, the authors present adversarial attacks by semantic manipulations, i.e., manipulating specific detectors that result in imperceptible changes in the picture, such as changing texture and color, but without affecting their naturalness. Moreover, these tasks are done on two large scale datasets (ImageNet and MSCOCO) and two visual tasks (classification and captioning). Finally, they also test their adversarial examples against a couple of defense mechanisms and how their transferability. Overall, all reviewers agreed this is an interesting work and well executed, complete with experiments and analyses. I agree with the reviewers in the assessment. I think this is an interesting study that moves us beyond restricted pixel perturbations and overall would be interesting to see what other detectors could be used to generate these type of semantic manipulations. I recommend acceptance of this paper.

Decision: Accept (Poster)