Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: paper summary:
The authors claim that likelihood based generative models are not as robust to noise as general consensus claims them to be. To prove this authors make use of adversarial, ambiguous and incorrectly labeled in distribution inputs. Authors address issues regarding robustness in near perfect conditional generative models as well as assess the robustness of the likelihood objective.

Pros of the paper:
1) Authors make well motivated arguments about how a near perfect generative model is also susceptible to attacks by providing examples that are adversarial, and have high likelihood and yet are incorrectly labeled.
2) They also demonstrate how class conditional generative models have poor discriminative power.

Cons:
1) The experiments section is written very poorly. This section relies heavily on the supplement making it hard to read due to the constant back and forth between the results and details of the experiments.
2) Experiments seem largely limited. Comparisons on image  data sets such as MNIST and CIFAR10 alone are not convincing enough to establish generalizability of the proposed theory. For example, the hypothesis could completely fail on text based generative models.


review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Update: I thank the authors' for their response, and have read the other reviews.

This paper demonstrates some theoretical and practical limitations on the use of likelihood based generative models for detecting adversarial examples. They construct a simple counterexample showing that there are adversarial examples for an arbitrarily accurate model (as measured by KL) that are not detectable by diminished likelihood of the model (as the dimension increases). Extending the work of Gilmer et al, this proves that there can be no general robustness guarantee for conditional generative models (Bayes classifiers). They provide compelling empirical evidence that while conditional normalizing flows trained on MNIST can be effective in detecting and defending adversarial attacks, these models trained on CIFAR10 are not. Surprisingly, it is shown that linear interpolations between images of different classes yield higher likelihoods for the CIFAR10 models and that class has little impact on model likelihoods. This goes some way in explaining why the detection is not effective on CIFAR, but questions still remain.

The paper makes fairly modest claims, but does a good job at demonstrating them and shedding some light on the issue. The experiments are thorough and fit into a growing body of evidence that the likelihoods of normalizing flows and other image based likelihood models may not be that informative or well calibrated, where past work has focused on out-of-distribution detection. My only major complaint with the paper is that it is not clear to what extent the theoretical and practical problems are related. As mentioned in the paper, the counterexample construction depends on the geometry of the data rather than the learning model. It could be that for both the MNIST and CIFAR10 datasets, the geometry is such that robustness garauntees are possible, and that the discrepancy in detection and interpolation arises because the normalizing flow has modeled the MNIST distribution much better than the CIFAR10 distribution. In this case we might hope that using conditional likelihood models for adversarial detection can be made effective, but that effort needs to be placed into improving the modeling capability. It's not obvious how to probe this distinction, but it would be good if this was given some thought in the paper. Also it would be good to see the attack detection numbers on BG-MNIST.


Comments:
Difficulty in training conditional generative models:
I believe in the two papers you cite the models do not use the label as input, but rather there is a separate model for each class? The overfitting is likely why the models had slightly lower conditional likelihood. As an aside, there are a couple of other examples of conditional normalizing flow models on images that use a mixture of Gaussians in the latent space [1], [2].

eq. 4: In the paper it is said that the second term in eq 4 is at most log(C), because the uniform distribution would have this value and that therefore this is negligibly small in comparison to the other term. Why exactly is this the case, couldn't the data entropy term be smaller in principle even if it's larger in practice? Or is the argument that the data entropy term scales with the dimensionality, but the label term does not leading to an imbalance? This could use some clarification.

A3: What is meant by ‘While the ground-truth likelihoods for the padded and un-padded datapoints are the same due to independence of the uniform noise and unit density of the noise’ in the appendix section A3? Wouldn't the ground truth negative log likelihoods would increase by the entropy of the uniform noise? Also, then in the bits per dimension calculations is the dimension the number of unpadded dimensions or the padded dimensions?


[1] Izmailov, Pavel, et al. "Semi-Supervised Learning with Normalizing Flows." Workshop on Invertible Neural Nets and Normalizing Flows, International Conference on Machine Learning. 2019.
[2] Atanov, Andrei, et al. "Semi-Conditional Normalizing Flows for Semi-Supervised Learning." Workshop on Invertible Neural Nets and Normalizing Flows, International Conference on Machine Learning. 2019.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Post rebuttal:

Thank you for your response. I appreciate the authors add an experiment on BG-MNIST, which shows the intermediate trend of MNIST and CIFAR-10.

As the authors mentioned, the reweighting scheme could be a simple yet effective way to address the problem of current likelihood-based models. While there is room for improvement to further develop the method, the current version of the paper would be a good contribution to the community.

Hence, I raise my score from 3 to 6.

----------------------------------------

Summary:
This paper investigates some limitations of the conditional generative models (or generative classifiers). First, the authors present a counter-example that a good generative classifier fails to detect adversarial attacks. Second, the authors claim that the marginal and conditional terms of the likelihood objective are the source of the problem. Finally, the authors demonstrate some experiments on adversarial attacks, out-of-distribution (OOD) samples, and noisy labels.

Pros:
- While generative classifiers are believed to be more robust than the discriminative counterparts [1], the authors present a counter-example that it may not be true.
- The authors investigate the marginal and conditional terms of the likelihood objective and demonstrate empirical results that the model fails to capture the outliers.

Cons:

1. The imbalance issue of the likelihood objective is not surprising.

As the data x is far complex than the class y, it is expectable that the penalty from modeling p(x) is larger than the penalty from classifying p(y|x). As mentioned in Table 1 and Appendix A.2, balancing two terms indeed improves the classification performance. However, to meet the high standard of ICLR, the authors should propose an alternative or modification of the likelihood which resolves the existing limitations. For example, [2] decomposes the semantic and background parts to improve the OOD detection using likelihood models.

2. The experiments are not extensively studied.

The authors conduct experiments on two datasets: MNIST and CIFAR-10. The authors may present more results on other datasets (e.g., SVHN or CIFAR-100) and convince if their findings are consistent. Also, some observations seem to be an inheritance of the datasets, e.g., Figure 4 is natural since MNIST has disjoint support and CIFAR-10 has a continuous one.

Minor comments:
- On page 8, ',' should be moved after (Azulay & Weiss, 2018).
- On page 8, (Schott et al.) should be changed to '\citet' format.
- PixelCNN++ is doubly cited.


[1] Li et al. Are Generative Classifiers More Robust to Adversarial Attacks? ICML 2019.
[2] Ren et al. Likelihood Ratios for Out-of-Distribution Detection. NeurIPS 2019.

review_title: Official Blind Review #2

Meta-review: This paper presents theoretical results showing the conditional generative models cannot be robust. The paper also provide counter examples and some empirical evidence showing that the theory is reflected in practice. Some reviewers doubt how much of the theory holds in reality, but still they think that this paper could be a useful for the community. After the rebuttal period, R2 increased their score and it seems that with the current score the paper can be accepted.
Decision: Accept (Poster)