Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Authors have introduced a new type of adversarial attacks that perturb abstract features of the image. They have shown that pixel space adversarial attack detection and defense techniques are ineffective in guarding against feature space attacks.

I have some concerns about the novelty of the attack and the appropriateness of defenses that have been tested.

- Since the attack is done in the feature space, the defense should also be done in the feature space. For example, adversarial training or smoothing can be done in the feature space. See: https://arxiv.org/abs/1802.03471

- There are attacks that perturb colors or other interpretable features of the image that have not been mentioned in the paper. For example, see https://arxiv.org/abs/1804.00499 and https://arxiv.org/pdf/1906.00001

- If the decoder has a high Lipschitz constant, a small perturbation in the feature scape 'can' lead to a large and visible perturbation in the pixel space. It was not clear to me how this is being controlled in the current method. 
 
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper presents an adversarial attack method, which conducts perturbations in the feature spaces, instead of the raw image space. Specifically, the proposed method firstly learns an encoder that encodes features into the latent space, where style features are learned. At the same time, a decoder is learned to reconstruct the images with the encoded features. To conduct attacks, perturbations are added into the encoded features and attack images are generated with the decoder given the perturbated features. The experiment results look promising, showing that the proposed method achieves better attack performance with realistic adversarial images.

The general idea of perturbating the feature (latent) space is not a novel one, which has been studied in [1]. However, the proposed one is with an autoencoder framework instead of GAN used in [1]. Therefore, the proposed approach is able to construct adversarial examples for specific images. In addition, the training of the encoder is adapted from a style transfer method, which seems to learn good features that capture style features. 

It is a bit unclear on the intuition of the constructions of Eq. (5) and (6). The details may be in Huang & Belongie, 2017. But it is better to provide more intuitive explanation and discussion on why these constructions capture style variation.

The results shown in the paper look promising. But it would be more comprehensive to compare with other pixel attacks in addition to PGD. Moreover, it is unclear whether it is a fair comparison between the proposed approach and pixel attacks, even under the same amount of perturbations. It would be good if the code will be released.

Minor:

Last sentence in the first paragraph of page 3: a missing reference.

[1] Song, Yang, Rui Shu, Nate Kushman, and Stefano Ermon. "Constructing unrestricted adversarial examples with generative models." In Advances in Neural Information Processing Systems, pp. 8312-8323. 2018.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper presents an interesting new adversarial attack technique that attempts to perturb abstract features learned by the target neural net. It is well written and easy to follow.  Its main contribution is the joining of ideas developed in the style transfer literature with those from the adversarial literature.  

The authors establish that they are able to create adversarial attacks that look similar to the original image but are miss classified. These images are not bounded by small epsilons, but are said to be indistinguishable by people. They illustrate a sample of these attacks, but no human study is employed to back up this claim. A simple human evaluation to prove that the attacks are indistinguishable from unperturbed images would strengthen the work (This can be done easily and at low cost by  employing mechanical Turk or an equivalent system for example).  

They make use of a detection mechanism (The Odds are Odd, Rothet al.) to verify that their adversarial attacks are hard to detect, but this particular detection mechanism has already  been broken (https://arxiv.org/pdf/1907.12138.pdf). If there is an as of yet unbroken detection mechanism that could be tested, that would improve the work. Alternatively the authors should acknowledge that there are simpler ways of evading this detection method.    

The attack that they propose targets the feature space, but no feature space detection methods are tested. The work would be improved by testing on a feature based detection methods such as dKNN (https://arxiv.org/pdf/1902.01889.pdf)

Overall the work is interesting and novel, and creatively joins together two otherwise distinct areas of machine learning research to make a modest but novel contribution to the field.
review_title: Official Blind Review #1

Meta-review: This paper provides an improved feature space adversarial attack.

However, the contribution is unclear in its significance, in part due to an important prior reference was omitted (song et al.) 

Unfortunately the paper is borderline, and not above the bar for acceptance in the current pool.
Decision: Reject