Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: Looking at Figure 8 (first version of paper, untargeted FGS classifier attack), it seems that maybe only 10% of MNIST digits are incorrectly reconstructed. Do the authors think this method is a success in achieving what they defined in the problem statement, i.e., "reconstruction looks more like something from another class."?

How is this paper different from the "Adversarial Images for Variational Autoencoders" paper, in terms of both the ideas and the performance? Specially, I think the idea of "latent attack" is very similar in both papers. I think there has to be a section in the paper to explain the difference between these two works.

MNIST dataset is the only dataset that is considered, which is not very convincing.

The paper is much longer than the standard page-limit, which makes it hard to follow.
review_title: pre-review questions

final_score: rating: 6: Marginally above acceptance threshold
review_content: Comments: 

"This contrasts to adversarial attacks on classifiers, where any inspection of the inputs will reveal the original bytes the adversary supplied,
which often have telltale noise"

Is this really true?  If it were the case, wouldn't it imply that training "against" adversarial examples should easily make a classifier robust to adversarial examples (if they all have a telltale noise)?  

Pros: 
  -The question of whether adversarial examples exist in generative models, and indeed how the definition of "adversarial example" carries over is an interesting one.  
  -Finding that a certain type of generative model *doesn't have* adversarial examples would be a really significant result, finding that generative models have adversarial examples would also be a worth negative result.  
  -The adversarial examples in figures 5 and 6 seem convincing, though they seem much more overt and noisy than the adversarial examples on MNIST shown in (Szegedy 2014).  Is this because it's actually harder to find adversarial examples in these types of generative models?  

Issues: 
  -Paper is significantly over length at 13 pages.  
  -The beginning of the paper should more clearly motivate its purpose.  
  -Paper has "generative models" in the title but as far as I can tell the whole paper is concerned with autoencoder-type models.  This is kind of annoying because if someone wanted to consider adversarial attacks on, say, autoregressive models, they might be unreasonably burdened by having to explain how they're distinct from a paper called "adversarial examples for generative models".  
   -I think that the introduction contains too much background information - it could be tightened.  

review_title: null

final_score: rating: 5: Marginally below acceptance threshold
review_content: 
After the rebuttal:

The paper contains an interesting set of results (mainly produced after the initial submission), but novelty is limited, and presentation is suboptimal. 

For me now the biggest problem now is that the title and the content do not correspond. The authors clearly attack deterministic encoder-decoder models (as described in 3.2), which are not at all the same as generative models, even though many generative models make use of this architecture. A small experiment with sampling is interesting, but does not change the overall focus of the paper. This inconsistency in not acceptable. The whole issue could be resolved for example by simply replacing "generative models" by "encoder-decoder networks" in the title. Then I would tend towards recommending acceptance.

------
Initial review:

The paper describes three approaches to generating adversarial examples for deep encoder-decoder generative networks (trained as VAE or VAE-GAN), and shows a comparative analysis of these. While the phenomenon of adversarial examples in discriminative models is widely known and relatively well studied, I am not aware of previous work on adversarial examples for generative networks, so this work is novel (there is a concurrent work by Tabacof et al. which should be cited, though).  The paper has significantly improved since the initial submission; still, I have a number of remarks on presentation and experimental evaluation. I am in the borderline mode, and may change my rating during the discussion phase.

Detailed comments:

1) The paper is 13 pages long - significantly over the recommended page limit of 8 pages. Reviewers have to read multiple papers, multiple versions of each, it is a lot of work. Large portions of the paper should be shortened and/or moved to the appendix. It is job of the authors to make the paper concise and readable. "in our attempts to be thorough, we have had a hard time keeping the length down" is a bad excuse - it may be hard, but has to be done.

2) I intentionally avoided term "generative model" above because it is not obvious to me if the attacks described by the authors indeed attack generative models. To clarify, the authors train encoder-decoders as generative models (VAE or VAE-GAN), but then remove all stochasticity (sampling) and prior on the latent variables: that is, they treat the models as deterministic encoders-decoders. It is not a big surprise that a deterministic deep network can be easily tricked; it would be much more interesting to see if the probabilistic aspect of generative models makes them more robust to such attacks. Am I missing something? I would like the authors to clarify their view on this and adjust the claims in the paper if necessary.

3) The paper is motivated by possible attacks on a data channel which uses a generative network for compressing information. Description of the attack scenario in 3.1 does not look convincing to me. It takes a huge amount of space and I do not think it adds much to the paper. First, experiments on natural images are necessary to judge if the proposed attack could succeed in a realistic scenario and second, I am not aware of any existing practical applications of VAEs to image compression: attacking JPEG would be much more practical. 

4) Experiments are limited to MNIST and, in the latest version, SVHN (which is very nice). While no good generative models of general natural images exist, it is common to evaluate generative models on datasets of faces, so this would be another very natural domain for testing the proposed approach. 

Smaller remarks:
1) Usage of "Oracle" in 3.2 does not look appropriate - oracle typically has access to (part of) ground truth, which is not the case here as far as I understand.
2) Beginning of section 4: "All three methods work for any generative architecture that relies on a learned latent representation z" - "are technically applicable to" would be more correct than "work for". 
3) 4.1: "confidentally"

review_title: Final review

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper considers different methods of producing adversarial examples for generative models such as VAE and VAEGAN. Specifically, three methods are considered: classification-based adversaries which uses a classifier on top of the hidden code, VAE loss which directly uses the VAE loss and the "latent attack" which finds adversarial perturbation in the input so as to match the latent representation of a target input.

I think the problem that this paper considers is potentially useful and interesting to the community. To the best of my knowledge this is the first paper that considers adversarial examples for generative models. As I pointed out in my pre-review comments, there is also a concurrent work of "Adversarial Images for Variational Autoencoders" that essentially proposes the same "latent attack" idea of this paper with both L2 distance and KL divergence.

Novelty/originality: I didn't find the ideas of this paper very original. All the proposed three attacks are well-known and standard methods that here are applied to a new problem and this paper does not develop *novel* algorithms for attacking specifically *generative models*. However I still find it interesting to see how these standard methods compare in this new problem domain.

The clarity and presentation of the paper is very unsatisfying. The first version of the paper proposes the "classification-based adversaries" and reports only negative results. In the second set of revisions, the core idea of the paper changes and almost an entirely new paper with a new co-author is submitted and the idea of "latent attack" is proposed which works much better than the "classification-based adversaries". However, the authors try to keep around the materials of the first version, which results in a 13 page long paper, with different claims and unrelated set of experiments. "in our attempts to be thorough, we have had a hard time keeping the length down" is not a valid excuse.

In short, the paper is investigating an interesting problem and apply and compare standard adversarial methods to this domain, but the novelty and the presentation of the paper is limited.
review_title: Review

final_score: rating: null
review_content: Hello,

An interesting paper! Below are some questions. They are based on the yesterday version of the paper, and I see some of them are addressed in the new version which just appeared. But unfortunately I do not have time to carefully re-read the paper now, so just let me know if some of these do not apply any more.

1. Do you believe the method would be applicable in the attack scenario you described? The added perturbations are very visible.

2. To me the most interesting aspect of this paper is that the proposed method does _not_ work very well, meaning that generative models are "difficult to fool". Do you agree with this intuition? Have you considered this perspective? 

3. Have you experimented with other datasets? Ideally natural images, but at least faces?

4. Why are you only using the fast gradient sign method for generating adversarial examples? Wouldn't you get better results with iterative optimization?

5. Don't you think viewing the figures would be more convenient if you grouped the results by samples, not by methods? As in Figure 7. 

Thanks!
review_title: pre-review questions

final_score: rating: null
review_content: Figure 7 suggests that your adversarial approach is systematically distorting the representation such that the reconstructed images belong to a different class. Have you compared the nearest neighbor of your perturbed embeddings to the embeddings in your training data set? Additionally, have you quantified the degree to which the reconstructed class might be mis-classified as a different class?
review_title: Quantification of the adversarially-perturbed coding

Meta-review: The main idea in this paper is interesting, of considering various forms of adversarial examples in generative models. The paper has been considerably revised since the original submission. The results showing the susceptibility of generative models to attacks are interesting, but the demonstrations need to be more solid and on other datasets to be convincing.
Decision: Reject