Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: 
====== Updates ======
I appreciate the authors' time and effort in the response. I have read the rebuttal, but I am not convinced by the authors' argument on using L2 (or L_\infty) constraints. No matter whether L2 or L_\infty constraint is used, the authors' method is not directly comparable to methods in Song et al. (2018), making the results in Table 2 and Table 3 meaningless and confusing. 

- Song et al. (2018) indeed constraints the search region of latent code to be within a small L2 ball of a randomly sampled anchor latent code. However, this anchor latent code is not directly related to any given image in the dataset, and therefore the generated adversarial examples are not close to any existing image. In contrast, the authors' attack is still basically a norm-bounded attack, which is not directly comparable to the unrestricted attack in Song et al. (2018).

- Song et al. (2018) is a white box attack, while the attack in this paper is black box.

====== Original review =======

This paper proposes to generate semantic preserving adversarial examples by first learning a manifold and then perturbing data along the manifold. In this way the generated adversarial examples can be semantically close to the original clean examples, and the perturbations can be hopefully more natural. For manifold learning, the authors propose to use a similar approach to that proposed in Pu et al. (2017), which uses SVGD to train a VAE. After the VAE is trained, the authors use GBSM to train a model to produce semantic adversarial examples efficiently.

I have many concerns for this paper:

- The approach is not well motivated. It is unclear why using a fully Bayesian framework and employing SVGD to learn the VAE model is preferred for conducting semantic adversarial attacks. Many choices in the algorithm seem to be arbitrary, and there are many approximations in the method whose accuracies have no guarantees. For example, the recognition networks  are used to approximate the updated parameters of the encoder from SVGD. Sampling from the posterior distribution of z is approximated by first doing Monte Carlo over \Theta. For "manifold alignment" another recognition network is used to approximate the updates from SVGD. It is hard to predict how those approximation errors accumulate when all pieces are combined together to form a very complicated algorithm.

- In Equation (6) the authors hard-constrain the generated adversarial example such that they cannot differ from the original data by some pre-specified l_2-norm. This leads to many unfair comparisons in the experiments:

    1. The authors compare their approach to other attacking methods on the success rates of attacking Madry's model and Kolter & Wong's certified model. However, both Madry and Kolter & Wong's model are for attacks using the l_infinity norm. It is unfair that the authors' attack uses l_2 norm. In fact, it is known that models robust to l_infinity norm attacks are generally not robust to attacks using other norms. 

    2. The authors also compare their approach to methods in Song et al. (2018) and Zhao et al. (2018a). However, the two previous approaches did not directly constrain the distance between generated adversarial examples and the corresponding clean inputs. Therefore, when using human evaluation to assess the image quality of generated adversarial examples, the two previous methods are naturally at a huge disadvantage. In stark contrast, the authors' adversarial images are constrained to be close to the corresponding unperturbed images under a small l_2 norm, which naturally have higher image quality.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The paper presents an approach to generating adversarial examples that preserve the semantics of the input examples. To do so, the approach reconstructs the manifold where the input examples lie and then generates new examples by perturbing the elements of the manifold so as to ensure the new elements remain in the manifold to preserve the semantics of the elements.

In the presented system the manifold is learned by means of Stein Variational Gradient Descent, while the perturbation is made by applying the Gram-Schmidt process which ensures that the perturbed elements still reside in the manifold.

To generate adversarial examples the approach presented in the paper considers a scenario in which only the predictions of the classifier are known, to be able to compute and optimize the loss function.

The presented approach has been tested on toy examples regarding images (both numbers from MNIST or SVHN and images from CelebA datasets) and texts (SNLI dataset).

The performance presented in the paper is promising. The results show that the manifold shape is preserved while creating perturbed elements. The system also achieves good results in terms of adversarial success rate, which however I believe should be called "attack success rate", a term widely used in literature.

The paper is well written and seems to me to be mathematically sound. I have very few comments on the paper which does not present any important lack in my opinion.

I suggest moving algorithm 2 to a new page in order to have the whole pseudo-code together.
Moreover, I have found two typos, one on page 4 in the last equation where z'_m I think it is wrongly written as z^'_m (the ' is far from the z), and one on page 6 in the "adversarial examples" paragraph where the word "data" is misspelt as "dat".
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: Major caveat: I have published in the area of adversarial attacks on NLP models, but the specifics of the methods presented in this paper are quite outside of my expertise, and I do not have time to become familiar with them for this review.  I hope there are other reviewers that are more qualified than I am to check the specifics of the methods.

This paper presents a new technique for generating adversarial examples, by first learning the data manifold in an embedding space, then finding an adversarial example that lies on the manifold.  I like this idea, it intuitively seems like a promising method for obtaining semantically meaningful adversarial examples.

As I said above, I do not feel qualified to review whether the method should _theoretically_ accomplish its goals, so my judgment of this paper is on the intuition behind the idea (which I like), and the results that I can see (which are less promising).  In order to have a "semantics preserving" attack, the method needs to (1) remain on the data manifold, and (2) not change the label a human would give to the input.

For (1), this appears to have been accomplished on most datasets, though it seems pretty hard to argue that the artifacts seen in the MNIST examples shown are on the data manifold - there are no such artifacts in any of the inputs, or in the clean reconstruction.  How do the authors claim that this actually did a reasonable job of staying in the data manifold?

For (2), most of the images do indeed look like they should retain their human labels, which is good (but also not hard for adversarial images).  Almost all of the textual examples, however, have correct predictions from the model after the adversarial change to the input.  You can't really argue that these are "semantics preserving", or even "successful attacks", as they change the expected input label.  This is why semantics-preserving attacks are so hard in NLP, and I don't think that this method has accomplished its goal here at all, at least for text.  The authors should consult with experts in NLP before making claims about successfully constructing semantics preserving attacks on NLP models.

I'm pretty on the fence about this paper, as I like the intuition, and the method appears to work reasonably well for vision.  It does not work as claimed for text, however, and that should be fixed before this paper is published (either with softened claims or with better results).  Hopefully people from other perspectives can pipe in and give a more clear picture on this paper.

EDIT: See discussion below for my justification for reducing my score from a 3 to a 1.

EDIT 11/14: The authors' revisions have satisfied my concerns about how the NLP attacks are described.  I'm a little bit nervous about how the examples were changed - it seems that nothing changed about the method itself, so the authors probably cherry-picked better examples - but that's not sufficiently worrying to me to justify rejection.  The pilot study is also quite weak, as the number of inputs that were evaluated was only 20, and the questions presented don't appear to ask about changes in the label.  I don't know how you could get 100% on that given the examples that I saw in the previous version of the paper.  This is all to say that I don't think the NLP attacks are actively problematic anymore, as they were previously, now they are just weak.  The main contribution here is the technical contribution, anyway, so weaker results on one of the datasets tested is not a deal-breaker to me.  Assuming the technical contributions pass muster (which, as I said, I don't really feel qualified to judge), I'm satisfied with this paper as it is now.
review_title: Official Blind Review #1

Meta-review: This paper describes a method for generating adversarial examples from images and text such that they maintain the semantics of the input.

The reviewers saw a lot of value in this work, but also some flaws.  The review process seemed to help answer many questions, but a few remain: there are some questions about the strength of the empirical results on text after the author's updates. Wether the adversarial images stay on the manifold is questioned (are blurry or otherwise noisy images "on manifold"?).  One reviewer raises good questions about the soundness of the comparison to the Song paper.

I think this review process has been very productive, and I hope the authors will agree.  I hope this feedback helps them to improve their paper.
Decision: Reject