Original: U NDERSTANDING THE L IMITATIONS OF C ONDITIONAL G ENERATIVE M ODELS
Revised: U NDERSTANDING THE L IMITATIONS OF C ONDITIONAL G ENERATIVE M ODELS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts.
Revised: Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts.

Original: They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution.
Revised: They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution.

Original: In this work, we investigate robust classiﬁcation with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises.
Revised: In this work, we investigate robust classiﬁcation with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises.

Original: Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs.
Revised: Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs.

Original: Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classiﬁers.
Revised: Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classiﬁers.

Original: Experimentally, we ﬁnd that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10.
Revised: Experimentally, we ﬁnd that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10.

Original: We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective.
Revised: We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective.

Original: Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classiﬁcation.
Revised: Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classiﬁcation.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Conditional generative models have recently shown promise to overcome many limitations of their discriminative counterparts.
Revised: Conditional generative models have recently shown promise to overcome many limitations of their discriminative counterparts.

Original: They have been shown to be robust against adversarial attacks (Schott et al., 2019; Ghosh et al., 2019;
Revised: They have been shown to be robust against adversarial attacks (Schott et al., 2019; Ghosh et al., 2019;

Original: Song et al., 2018; Li et al., 2018; Frosst et al., 2018), to enable robust classiﬁcation in the presence of outliers (Nalisnick et al., 2019b) and to achieve promising results in semi-supervised learning (Kingma et al., 2014; Salimans et al., 2016; Izmailov et al., 2019; Atanov et al., 2019).
Revised: Song et al., 2018; Li et al., 2018; Frosst et al., 2018), to enable robust classiﬁcation in the presence of outliers (Nalisnick et al., 2019b) and to achieve promising results in semi-supervised learning (Kingma et al., 2014; Salimans et al., 2016).

Original: Motivated by these success stories, we study the properties of conditional generative models in more detail.
Revised: Motivated by these success stories, we study the properties of conditional generative models in more detail.

Original: Unlike discriminative models, which can ignore class-irrelevant information, conditional generative models cannot discard any information in the input, potentially making it harder to fool them.
Revised: Unlike discriminative models, which can ignore class-irrelevant information, conditional generative models cannot discard any information in the input, potentially making it harder to fool them.

Original: Further, jointly modeling the input and target distribution should make it easy to detect out-of-distribution inputs.
Revised: Further, jointly modeling the input and target distribution should make it easy to detect out-of-distribution inputs.

Original: These traits lend hope to the belief that good class-conditional generative models can overcome important problems faced by discriminative models.
Revised: These traits lend hope to the belief that good class-conditional generative models can overcome important problems faced by discriminative models.

Original: In this work, we analyze conditional generative models by assessing them on a spectrum of robustness tasks.
Revised: In this work, we analyze conditional generative models by assessing them on a spectrum of robustness tasks.

Original: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled indistribution inputs.
Revised: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled indistribution inputs.

Original: If a generative classiﬁer is able to perform well on all of these, it will naturally be robust to noisy, ambiguous or adversarially perturbed inputs.
Revised: If a generative classiﬁer is able to perform well on all of these, it will naturally be robust to noisy, ambiguous or adversarially perturbed inputs.

Original: Outlier detection in the above settings is substantially different from general out-of-distribution (OOD) detection, where the goal is to use unconditional generative models to detect any OOD input.
Revised: Outlier detection in the above settings is substantially different from general out-of-distribution (OOD) detection, where the goal is to use unconditional generative models to detect any OOD input.

Original: For the general case, likelihood has been shown to be a poor detector of OOD samples.
Revised: For the general case, likelihood has been shown to be a poor detector of OOD samples.

Original: In fact, often higher likelihood is assigned to OOD data than to the training data itself (Nalisnick et al., 2019a).
Revised: In fact, often higher likelihood is assigned to OOD data than to the training data itself (Nalisnick et al., 2019a).

Original: However, class-conditional likelihood necessarily needs to decrease towards the decision-boundary for the classiﬁer to work well.
Revised: However, class-conditional likelihood necessarily needs to decrease towards the decision-boundary for the classiﬁer to work well.

Original: Thus, if the class-conditional generative model has high accuracy, rejection of outliers from the wrong class via likelihood may be possible.
Revised: Thus, if the class-conditional generative model has high accuracy, rejection of outliers from the wrong class via likelihood may be possible.

Original: Our contributions are:
Revised: Our contributions are:

Original: Provable Robustness We answer: Can we theoretically guarantee that a strong conditional generative model can robustly detect adversarially attacked inputs?
Revised: Provable Robustness We answer: Can we theoretically guarantee that a strong conditional generative model can robustly detect adversarially attacked inputs?

Original: In section 2 we show that even a near-perfect conditional generative model cannot be guaranteed to reject adversarially perturbed inputs with high probability.
Revised: In section 2 we show that even a near-perfect conditional generative model cannot be guaranteed to reject adversarially perturbed inputs with high probability.

Original: Assessing the Likelihood Objective
Revised: Assessing the Likelihood Objective

Original: We discuss the basis to empirically analyze robustness in practice.
Revised: We discuss the basis to empirically analyze robustness in practice.

Original: We identify several fundamental issues with the maximum likelihood objective typically used to train conditional generative models and discuss whether it is appropriate for detecting out-of-distribution inputs.
Revised: We identify several fundamental issues with the maximum likelihood objective typically used to train conditional generative models and discuss whether it is appropriate for detecting out-of-distribution inputs.

Original: Understanding Conﬂicting Results We explore various properties of our trained conditional generative models and how they relate to fact that the model is robust on MNIST but not on CIFAR10.
Revised: Understanding Conﬂicting Results We explore various properties of our trained conditional generative models and how they relate to fact that the model is robust on MNIST but not on CIFAR10.

Original: We further propose a new dataset where we combine MNIST images with CIFAR background, making the generative task as hard as CIFAR while keeping the discriminative task as easy as MNIST, and investigate how it affects robustness.
Revised: We further propose a new dataset where we combine MNIST images with CIFAR background, making the generative task as hard as CIFAR while keeping the discriminative task as easy as MNIST, and investigate how it affects robustness.

Original:  C ONFIDENT M ISTAKES C ANNOT BE R ULED O UT
Revised:  C ONFIDENT M ISTAKES C ANNOT BE R ULED O UT

Original: The most challenging task in robust classiﬁcation is accurately classifying or detecting adversarial attacks; inputs which have been maliciously perturbed to fool the classiﬁer.
Revised: The most challenging task in robust classiﬁcation is accurately classifying or detecting adversarial attacks; inputs which have been maliciously perturbed to fool the classiﬁer.

Original: In this section we discuss the possibility of guaranteeing robustness to adversarial attacks via conditional generative models.
Revised: In this section we discuss the possibility of guaranteeing robustness to adversarial attacks via conditional generative models.

Original: Detectability of Adversarial Examples
Revised: Detectability of Adversarial Examples

Original: In the adversarial spheres work (Gilmer et al., 2018)
Revised: In the adversarial spheres work (Gilmer et al., 2018)

Original: the authors showed that a model can be fooled without changing the ground-truth probability of the attacked datapoint.
Revised: the authors showed that a model can be fooled without changing the ground-truth probability of the attacked datapoint.

Original: This was claimed to show that adversarial examples can lie on the data manifold and therefore cannot be detected.
Revised: This was claimed to show that adversarial examples can lie on the data manifold and therefore cannot be detected.

Original: While (Gilmer et al., 2018) is an important work for understanding adversarial attacks, it has several limitations with regard to conditional generative models.
Revised: While (Gilmer et al., 2018) is an important work for understanding adversarial attacks, it has several limitations with regard to conditional generative models.

Original: First, just because the attack does not change the ground-truth likelihood, this does not mean the model can not detect the attack.
Revised: First, just because the attack does not change the ground-truth likelihood, this does not mean the model can not detect the attack.

Original: Since the adversary needs to move the input to a location where the model is incorrect, the question arises: what kind of mistake will the model make?
Revised: Since the adversary needs to move the input to a location where the model is incorrect, the question arises: what kind of mistake will the model make?

Original: If the model assigns low likelihood to the correct class without increasing the likelihood of the other classes then the adversarial attack will be detected, as the joint likelihood over all classes moves below the threshold of typical inputs.
Revised: If the model assigns low likelihood to the correct class without increasing the likelihood of the other classes then the adversarial attack will be detected, as the joint likelihood over all classes moves below the threshold of typical inputs.

Original: Second, on the adversarial spheres dataset (Gilmer et al., 2018)
Revised: Second, on the adversarial spheres dataset (Gilmer et al., 2018)

Original: the class supports do not overlap.
Revised: the class supports do not overlap.

Original: If we were to train a model of the joint density p θ ( x, y ) (which does not have 100% classiﬁcation accuracy)
Revised: If we were to train a model of the joint density p θ ( x, y ) (which does not have 100% classiﬁcation accuracy)

Original: then the KL divergence KL ( p ( x, y )
Revised: then the KL divergence KL ( p ( x, y )

Original: ||
Revised: ||

Original: p θ ( x, y )) , where p ( x, y ) is the data density, is inﬁnite due to division by zero (note
Revised: p θ ( x, y )) , where p ( x, y ) is the data density, is inﬁnite due to division by zero (note

Original: that KL ( p θ ( x, y ) ||
Revised: that KL ( p θ ( x, y ) ||

Original: p ( x, y )) is what is minimized with maximum likelihood).
Revised: p ( x, y )) is what is minimized with maximum likelihood).

Original: This poses the question, whether small KL ( p ( x, y )
Revised: This poses the question, whether small KL ( p ( x, y )

Original: ||
Revised: ||

Original: p θ ( x, y )) or small Shannon-Jensen divergence is sufﬁcient to guarantee robustness.
Revised: p θ ( x, y )) or small Shannon-Jensen divergence is sufﬁcient to guarantee robustness.

Original: In the following, we show that this condition is insufﬁcient.
Revised: In the following, we show that this condition is insufﬁcient.

Original:  Why no Robustness Guarantee can be Given The intuition why conditional generative models should be robust is as follows: If we have a robust discriminative model then the set of conﬁdent mistakes, i.e. where the adversarial attacks must reside, has low probability but might be large in volume.
Revised: [Figure] Why no Robustness Guarantee can be Given The intuition why conditional generative models should be robust is as follows: If we have a robust discriminative model then the set of conﬁdent mistakes, i.e. where the adversarial attacks must reside, has low probability but might be large in volume.

Original: For a robust conditional generative model, the set of undetectable adversarial attacks, i.e.[Figure] high-density high-conﬁdence mistakes, has to be small in volume.
Revised: For a robust conditional generative model, the set of undetectable adversarial attacks, i.e. high-density high-conﬁdence mistakes, has to be small in volume.

Original: Since the adversary has to be ∆ close to this small volume set, the ∆ area around this small volume set should still be small.
Revised: Since the adversary has to be ∆ close to this small volume set, the ∆ area around this small volume set should still be small.

Original: This is where the idea breaks down due to the curse of dimensionality.
Revised: This is where the idea breaks down due to the curse of dimensionality.

Original: Expanding a set by a small radius can lead to a much larger one even with smoothness assumptions.
Revised: Expanding a set by a small radius can lead to a much larger one even with smoothness assumptions.

Original: Based on this insight we build an analytic counter-example for which we can prove that even if
Revised: Based on this insight we build an analytic counter-example for which we can prove that even if

Original: [Equation]
Revised: [Equation]

Original:  where p = p ( x, y ) is the data distribution, and q = q ( x, y ) is the model, we can with probability ≈ 0 .
Revised: 1) where p = p ( x, y ) is the data distribution, and q = q ( x, y ) is the model, we can with probability ≈ 0 .

Original: 5 take a correctly classiﬁed input sampled from p , and perturb it by at most ∆ to create an adversarial example that is classiﬁed incorrectly and is not detectable.
Revised: 5 take a correctly classiﬁed input sampled from p , and perturb it by at most ∆ to create an adversarial example that is classiﬁed incorrectly and is not detectable.

Original: We note that the probability in every ball with radius ∆ can be made as small as desired, excluding degenerate cases.
Revised: We note that the probability in every ball with radius ∆ can be made as small as desired, excluding degenerate cases.

Original: We also assume that the Bayes optimal classiﬁer is conﬁdent and is not affected by the attack, i.e. we do not change the underlying class but wrongfully ﬂip the decision of the classiﬁer .
Revised: We also assume that the Bayes optimal classiﬁer is conﬁdent and is not affected by the attack, i.e. we do not change the underlying class but wrongfully ﬂip the decision of the classiﬁer .

Original: The counter-example goes as follows:
Revised: The counter-example goes as follows:

Original: Let U ( a, b ) be the density of a uniform distribution on an annulus in dimension d , { x ∈ R d : a ≤ || x || ≤ b } then the data conditional distribution is
Revised: Let U ( a, b ) be the density of a uniform distribution on an annulus in dimension d , { x ∈ R d : a ≤ || x || ≤ b } then the data conditional distribution is

Original: [Equation]
Revised: [Equation]

Original: with p ( y = 0) =
Revised: with p ( y = 0) =

Original: p ( y = 1) = 1 / 2 .
Revised: p ( y = 1) = 1 / 2 .

Original: Both classes are a mixture of two distributions, uniform on the unit sphere and uniform on an annulus, as shown in Fig.
Revised: Both classes are a mixture of two distributions, uniform on the unit sphere and uniform on an annulus, as shown in Fig.

Original: The model distribution is the following:
Revised: The model distribution is the following:

Original: [Equation]
Revised: [Equation]

Original: i.e. for y = 1 the model is perfect, while for y = 

Revised: i.e. for y = 1 the model is perfect, while for y =

Original: we replace the mixture with uniform distribution over the whole domain.
Revised: we replace the mixture with uniform distribution over the whole domain.

Original: If λ 1 (cid:29) λ 2 then points in the sphere with radius 1 should be classiﬁed as class y = 0 with high likelihood.
Revised: If λ 1 (cid:29) λ 2 then points in the sphere with radius 1 should be classiﬁed as class y = 0 with high likelihood.

Original: If λ 2 >> 1 (1+∆) d then the model classiﬁes points in the unit sphere incorrectly with high likelihood.
Revised: If λ 2 >> 1 (1+∆) d then the model classiﬁes points in the unit sphere incorrectly with high likelihood.

Original: Finally if 1 >> λ 1 then almost half the data points will fall in the annulus between 1 and 1 + ∆ and can be adversarially attacked with distance lesser or equal to ∆ by moving them into the unit sphere as seen in Fig.
Revised: Finally if 1 >> λ 1 then almost half the data points will fall in the annulus between 1 and 1 + ∆ and can be adversarially attacked with distance lesser or equal to ∆ by moving them into the unit sphere as seen in Fig.

Original: We also note that these attacks cannot be detected as the model likelihood only increases.
Revised: We also note that these attacks cannot be detected as the model likelihood only increases.

Original: In high dimensions, almost all the volume of a sphere is in the outer shell, and this can be used to show that in high enough dimensions we can get the condition in Eq.
Revised: In high dimensions, almost all the volume of a sphere is in the outer shell, and this can be used to show that in high enough dimensions we can get the condition in Eq.

Original: 1 for any value of (cid:15) and ∆ (and also the conﬁdence of the mistakes δ ).
Revised: 1 for any value of (cid:15) and ∆ (and also the conﬁdence of the mistakes δ ).

Original: The detailed proof is in the supplementary material.
Revised: The detailed proof is in the supplementary material.

Original: This counter-example shows that even under very strong conditions, a good conditional generative model can be attacked.
Revised: This counter-example shows that even under very strong conditions, a good conditional generative model can be attacked.

Original: Therefore no theoretical guarantees can be given in the general case for these models.
Revised: Therefore no theoretical guarantees can be given in the general case for these models.

Original: Our construction, however, does not depend on the learning model but on the data geometry.
Revised: Our construction, however, does not depend on the learning model but on the data geometry.

Original: This raises interesting questions concerning the source of the susceptibility to attacks: Is it the model or an inherent issue with the data?
Revised: This raises interesting questions concerning the source of the susceptibility to attacks: Is it the model or an inherent issue with the data?

Original:  T HE M AXIMUM L IKELIHOOD O BJECTIVE 3. T HE D IFFICULTY IN T RAINING C ONDITIONAL G ENERATIVE M ODELS
Revised:  T HE M AXIMUM L IKELIHOOD O BJECTIVE 3. T HE D IFFICULTY IN T RAINING C ONDITIONAL G ENERATIVE M ODELS

Original: Most recent publications on likelihood-based generative models primarily focus on quantitative results of unconditonal density estimation (van den Oord et al., 2016; Kingma & Dhariwal, 2018;
Revised: Most recent publications on likelihood-based generative models primarily focus on quantitative results of unconditonal density estimation (van den Oord et al., 2016; Kingma & Dhariwal, 2018;

Original: Salimans et al., 2017; Kingma et al., 2016; Papamakarios et al., 2017).
Revised: Salimans et al., 2017b; Kingma et al., 2016; Papamakarios et al., 2017).

Original: For conditional density estimation, either only qualitative samples are shown (Kingma & Dhariwal, 2018), or it is reported that conditional density estimation does not lead to better likelihoods than unconditional density estimation.
Revised: For conditional density estimation, either only qualitative samples are shown (Kingma & Dhariwal, 2018), or it is reported that conditional density estimation does not lead to better likelihoods than unconditional density estimation.

Original: In fact, it has been reported that conditional density estimation can lead to slightly worse data likelihoods (Papamakarios et al., 2017), which is surprising at ﬁrst, as extra bits of important information are provided to the model.
Revised: In fact, it has been reported that conditional density estimation can lead to slightly worse data likelihoods (Papamakarios et al., 2017; Salimans et al., 2017b), which is surprising at ﬁrst, as extra bits of important information are provided to the model.

Original: Explaining Likelihood Behaviour One way to understand this seemingly contradictory relationship is to consider the objective we use to train our models.
Revised: Explaining Likelihood Behaviour One way to understand this seemingly contradictory relationship is to consider the objective we use to train our models.

Original: When we train a generative model with maximum likelihood (either exactly or through a lower bound) we are minimizing the empirical approximation of E x,y ∼ P [ − log( P θ ( x, y ))] which is equivalent to minimizing KL ( P ( x, y )
Revised: When we train a generative model with maximum likelihood (either exactly or through a lower bound) we are minimizing the empirical approximation of E x,y ∼ P [ − log( P θ ( x, y ))] which is equivalent to minimizing KL ( P ( x, y )

Original: ||
Revised: ||

Original: P θ ( x, y )) .
Revised: P θ ( x, y )) .

Original: Consider now an image x with a discrete label y , which we are trying to model using P θ ( x, y ) .
Revised: Consider now an image x with a discrete label y , which we are trying to model using P θ ( x, y ) .

Original: The negative log-likelihood (NLL) objective is:
Revised: The negative log-likelihood (NLL) objective is:

Original: [Equation]
Revised: [Equation]

Original: If we model P θ ( y | x ) with a uniform distribution over classes, then the second term has a value of log( C ) where C is the number of classes.
Revised: If we model P θ ( y | x ) with a uniform distribution over classes, then the second term has a value of log( C ) where C is the number of classes.

Original: This value is negligible compared to the ﬁrst term E x ∼ P [ − log( P θ ( x ))] and therefore the “penalty" for completely ignoring class information is negligible.
Revised: This value is negligible compared to the ﬁrst term E x ∼ P [ − log( P θ ( x ))] and therefore the “penalty" for completely ignoring class information is negligible.

Original: So it is not surprising that models with strong generative abilities can have limited discriminative power.
Revised: So it is not surprising that models with strong generative abilities can have limited discriminative power.

Original: What makes matters even worse is that the penalty for conﬁdent mis-classiﬁcation can be unbounded.
Revised: What makes matters even worse is that the penalty for conﬁdent mis-classiﬁcation can be unbounded.

Original: This may also explain why the conditional ELBO is comparable to the unconditional
Revised: This may also explain why the conditional ELBO is comparable to the unconditional

Original: ELBO (Papamakarios et al., 2017).
Revised: ELBO (Papamakarios et al., 2017).

Original: Another way this can be seen is by thinking of the likelihood as the best lossless compression.
Revised: Another way this can be seen is by thinking of the likelihood as the best lossless compression.

Original: When trying to encode an image, the beneﬁt of the label is at most log( C ) bits which is small compared to the whole image.
Revised: When trying to encode an image, the beneﬁt of the label is at most log( C ) bits which is small compared to the whole image.

Original: While these few bits are important for users, from a likelihood perspective the difference between the correct p ( y | x ) and a uniform distribution is negligible.
Revised: While these few bits are important for users, from a likelihood perspective the difference between the correct p ( y | x ) and a uniform distribution is negligible.

Original: This means that when naively training a class-conditional generative model by minimizing E ( x,y ) ∼ P [ − log( P θ ( x | y ))] , typically discriminative performance as a classiﬁer is very poor.
Revised: This means that when naively training a class-conditional generative model by minimizing E ( x,y ) ∼ P [ − log( P θ ( x | y ))] , typically discriminative performance as a classiﬁer is very poor.

Original: 3. O UTLIER D ETECTION
Revised: 3. O UTLIER D ETECTION

Original: Another issue arises when models trained with maximum likelihood are used to detect outliers.
Revised: Another issue arises when models trained with maximum likelihood are used to detect outliers.

Original: The main issue is that maximum likelihood, which is equivalent to minimizing KL ( P ( x, y )
Revised: The main issue is that maximum likelihood, which is equivalent to minimizing KL ( P ( x, y )

Original: ||
Revised: ||

Original: P θ ( x, y )) , is known to have a “mode-covering” behavior.
Revised: P θ ( x, y )) , is known to have a “mode-covering” behavior.

Original: It has been shown recently in (Nalisnick et al., 2019a) that generative models, trained using maximum likelihood, can be quite poor at detecting out-of-distribution example.
Revised: It has been shown recently in (Nalisnick et al., 2019a) that generative models, trained using maximum likelihood, can be quite poor at detecting out-of-distribution example.

Original: In fact it has been shown that these models can give higher likelihood values, on average, to datasets different from the test dataset that corresponds to the training data.
Revised: In fact it has been shown that these models can give higher likelihood values, on average, to datasets different from the test dataset that corresponds to the training data.

Original: Intuitivily one can still hope that a high accuracy conditional generative model would recognize an input conditioned on the wrong class as an outlier, as it was successfully trained to separate these classes.
Revised: Intuitivily one can still hope that a high accuracy conditional generative model would recognize an input conditioned on the wrong class as an outlier, as it was successfully trained to separate these classes.

Original: In section 4.2 we show this is not the case in practice.
Revised: In section 4.2 we show this is not the case in practice.

Original: While (Nalisnick et al., 2019a) focuses its analysis into dataset variance, we propose this is an inherit issue with the likelihood objective.
Revised: While (Nalisnick et al., 2019a) focuses its analysis into dataset variance, we propose this is an inherit issue with the likelihood objective.

Original: If it is correct then the way conditional generative models are trained is at odds with their desired behaviour.
Revised: If it is correct then the way conditional generative models are trained is at odds with their desired behaviour.

Original: If this is the case, then useful conditional generative model will require a fundamentally different approach.
Revised: If this is the case, then useful conditional generative model will require a fundamentally different approach.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We now present a set of experiments designed to test the robustness of conditional generative models.
Revised: We now present a set of experiments designed to test the robustness of conditional generative models.

Original: All experiments were performed with a ﬂowwith p ( z N ) being a Gaussian mixture model (GMM) with one Gaussian per class, i.e. p ( z N | y ) = N ( µ y , Σ y ) . Assuming p ( y ) is known, then maximizing log( p ( x, y )) is equivalent to maximizing:
Revised: All experiments were performed with a ﬂow model where the likelihood can be computed in closed form as the probability of the latent space embedding (the prior) and a Jacobian correction term; see

Original: [Equation]
Revised: Sec A.1 for a detailed explanation.

Original: At inference time, one can classify by using Bayes rule to obtain p ( y | x ) . We used a GLOW (Kingma &
Revised: Given that we can compute p ( x, y ) for each class, we can easily compute p ( y | x ) and classify accordingly.

Original: Dhariwal, 2018) normalizing ﬂow with 4 levels, afﬁne coupling layers, 1x1 convolution permutations and actnorm in a multi-scale architecture.
Revised: Besides allowing closed-form likelihood computation, the ﬂexibility in choosing the prior distribution was important to conduct various experiments.

Original: We chose 128 channels and 12 blocks per level for MNIST and 256 channels and 16 blocks for CIFAR10. In both MNIST and CIFAR10 experiments we double the number ofchannels with uniforrm(0,1) noise which we scale down to the range [0 , 2 / 256] (taking it into account in the 
Revised: In our work we used a version of the GLOW model; details of the models and training is in the supplementary material sec.

Original: Jacobian term). One major difference is thatwe do the squeeze operation at the end of each level instead of the beginning, which allows us to use 4 levels. The models are optimized using Adam, for 150 epochs. The initial learning rate is 1 e − 3 , decayed by a factor of 10 every 60 epochs.
Revised: B. We note that the results are not unique to ﬂow models, and we veriﬁed that similar phenomenon can be seen when training with the PixelCNN++ autoregressive model (Salimans et al., 2017a) in sec.

Original: For the reweighted optimization the objective is l ( x, y ) = − log( p ( x | y )) /D − log( p ( y | x )) , where D is the data dimension (3x32x32 for CIFAR, 1x32x32 for MNIST).
Revised: E.

Original: 4. T RAINING C ONDITIONAL G ENERATIVE M ODELS
Revised: 4. T RAINING C ONDITIONAL G ENERATIVE M ODELS

Original: Here we investigate the ability to train a conditional generative model with good likelihood and accuracy simultaneously.
Revised: Here we investigate the ability to train a conditional generative model with good likelihood and accuracy simultaneously.

Original: Usually in ﬂow models the prior distribution in latent space z is Gaussian.
Revised: Usually in ﬂow models the prior distribution in latent space z is Gaussian.

Original: For classiﬁcation we used aclass-conditional mixture of 10 Gaussians p ( z | y )
Revised: For classiﬁcation we used aclass-conditional mixture of 10 Gaussians p ( z | y )

Original: =
Revised: =

Original: N ( µ y , σ 2 y )
Revised: N ( µ y , σ 2 y )

Original: We compare three settings: 1) A class-conditional mixture of 10 Gaussians as the prior (Base).
Revised: We compare three settings: 1) A class-conditional mixture of 10 Gaussians as the prior (Base).

Original: A classconditional mixture of 10 Gaussians trained with an additional classiﬁcation loss term (Reweighted).
Revised: A classconditional mixture of 10 Gaussians trained with an additional classiﬁcation loss term (Reweighted).

Original: Our proposed conditional split prior (Split) described in sec.
Revised: Our proposed conditional split prior (Split) described in sec.

Original: A.4 in the supplementary material.
Revised: A.4 in the supplementary material.

Original: Results can be found in table 1.
Revised: Results can be found in table 1.

Original: [Table]
Revised: [Table]

Original: As we can see, especially on CIFAR10, pushing up the accuracy to values that are still far from stateof-the-art already results in non-negligible deterioration to the likelihood values.
Revised: As we can see, especially on CIFAR10, pushing up the accuracy to values that are still far from stateof-the-art already results in non-negligible deterioration to the likelihood values.

Original: This exempliﬁes how obtaining strong classiﬁcation accuracy without harming likelihood estimation is still a challenging problem.
Revised: This exempliﬁes how obtaining strong classiﬁcation accuracy without harming likelihood estimation is still a challenging problem.

Original: We note that while the difference between the split prior and re-weighted version is not huge, the split prior achieves better NLL and better accuracy in both experiments.
Revised: We note that while the difference between the split prior and re-weighted version is not huge, the split prior achieves better NLL and better accuracy in both experiments.

Original: We experimented with various other methods to improve training with limited success, see sec.
Revised: We experimented with various other methods to improve training with limited success, see sec.

Original: C in the supplementary material for furture information.
Revised: C in the supplementary material for furture information.

Original: 4. N EGLIGIBLE I MPACT OF C LASS M EMBERSHIP ON L IKELIHOOD
Revised: 4. N EGLIGIBLE I MPACT OF C LASS M EMBERSHIP ON L IKELIHOOD

Original: [Figure]
Revised: [Figure]

Original: Next we show that even conditional generative models which are strong classiﬁers do not see images with the corrupted labels as outliers.
Revised: Next we show that even conditional generative models which are strong classiﬁers do not see images with the corrupted labels as outliers.

Original: To understand this phenomenon we ﬁrst note that if we want the correct class to have a probability of at least 1 − δ then it is enough for the corresponding logit to be larger than all the others by log( C )
Revised: To understand this phenomenon we ﬁrst note that if we want the correct class to have a probability of at least 1 − δ then it is enough for the corresponding logit to be larger than all the others by log( C )

Original: + log (cid:0) 1 − δδ (cid:1) where C is the number of classes.
Revised: + log (cid:0) 1 − δδ (cid:1) where C is the number of classes.

Original: For C = 10 and δ = 1 e − 5 this is about 6, which is negligible relative to the likelihood of the image, which is in the scale of thousands.
Revised: For C = 10 and δ = 1 e − 5 this is about 6, which is negligible relative to the likelihood of the image, which is in the scale of thousands.

Original: This means that even for a strong conditional generative model which conﬁdently predicts the correct label, the pair { x i , y w (cid:54) =
Revised: This means that even for a strong conditional generative model which conﬁdently predicts the correct label, the pair { x i , y w (cid:54) =

Original: y i } (where w is the leading incorrect class) cannot be detected as an outlier according to the joint distribution. The gap log( p ( x i | y i ))
Revised: y i } (where w is the leading incorrect class) cannot be detected as an outlier according to the joint distribution, as the gap log( p ( x i | y i ))

Original: − log( p ( x i | y w )) is much smaller than the variation in likelihood values.
Revised: − log( p ( x i | y w )) is much smaller than the variation in likelihood values.

Original: In Fig.
Revised: In Fig.

Original: 3 we show this by plotting the histograms of the likelihood conditioned both on the correct class and on the most likely wrong class over the test set.
Revised: 3 we show this by plotting the histograms of the likelihood conditioned both on the correct class and on the most likely wrong class over the test set.

Original: In other words, in order for log( p ( x i | y w )) to be considered an outlier the prediction needs to be extremely conﬁdent, much more than we expect it to be, considering test classiﬁcation error.
Revised: In other words, in order for log( p ( x i | y w )) to be considered an outlier the prediction needs to be extremely conﬁdent, much more than we expect it to be, considering test classiﬁcation error.

Original: 4. A DVERSARIAL ATTACKS AS W ORST C ASE A NALYSIS
Revised: 4. A DVERSARIAL ATTACKS AS W ORST C ASE A NALYSIS

Original: We ﬁrst evaluate the ability of conditional generative models to detect standard attacks, and then try to detect attacks designed to fool the detector (likelihood function).
Revised: We ﬁrst evaluate the ability of conditional generative models to detect standard attacks, and then try to detect attacks designed to fool the detector (likelihood function).

Original: We evalulate both the gradient based Carlini-Wagner L 2 attack (CW- L 2 ) (Carlini & Wagner, 2017b) and the gradient free boundary attack (Brendel et al., 2018).
Revised: We evalulate both the gradient based Carlini-Wagner L 2 attack (CW- L 2 ) (Carlini & Wagner, 2017b) and the gradient free boundary attack (Brendel et al., 2018).

Original: Results are shown in table 2 on the left.
Revised: Results are shown in table 2 on the left.

Original: It is interesting to observe the disparity between the CW- L 2 attack, which is easily detectable, and the boundary attack which is much harder to detect. 
Revised: It is interesting to observe the disparity between the CW- L 2 attack, which is easily detectable, and the boundary attack which is much harder to detect. Attacking Classiﬁcation

Original: [Table]
Revised: [Table]

Original: the classiﬁer, both detected and undetected.
Revised: CIFAR CW − [Table]

Original: Next we modify our attacks to try to fool the detector as well.
Revised: Next we modify our attacks to try to fool the detector as well.

Original: With the CW- L 2 attack we follow the modiﬁcation suggested in (Carlini & Wagner, 2017a) and add an extra loss term (cid:96) det ( x (cid:48) )
Revised: With the CW- L 2 attack we follow the modiﬁcation suggested in (Carlini & Wagner, 2017a) and add an extra loss term (cid:96) det ( x (cid:48) )

Original: =
Revised: =

Original: max { 0 , − log( p ( x (cid:48) ))
Revised: max { 0 , − log( p ( x (cid:48) ))

Original: − T } where T is the detection threshold.
Revised: − T } where T is the detection threshold.

Original: For the boundary attack we turn the
Revised: For the boundary attack we turn the

Original: C -way classiﬁcation into a C + 1 -way classiﬁcation by adding another class which is “non-image” and classify any image above the detection threshold as such.
Revised: C -way classiﬁcation into a C + 1 -way classiﬁcation by adding another class which is “non-image” and classify any image above the detection threshold as such.

Original: We then use a targeted attack to try to fool the network to classify the image into a speciﬁc original class.
Revised: We then use a targeted attack to try to fool the network to classify the image into a speciﬁc original class.

Original: This simple modiﬁcation to the boundary attack will typically fail because it cannot initialize.
Revised: This simple modiﬁcation to the boundary attack will typically fail because it cannot initialize.

Original: The standard attack starts from a random image and all random images are easily detected as “non-image” and therefore do not have the right target class.
Revised: The standard attack starts from a random image and all random images are easily detected as “non-image” and therefore do not have the right target class.

Original: To address this we start from a randomly chosen image from the target class, ensuring the original image is detected as a real image from the desired class.
Revised: To address this we start from a randomly chosen image from the target class, ensuring the original image is detected as a real image from the desired class.

Original: From table 2 (right side) we can see that even after the modiﬁcation CW- L 2 still struggles to fool the detector.
Revised: From table 2 (right side) we can see that even after the modiﬁcation CW- L 2 still struggles to fool the detector.

Original: The boundary attack, however, succeeds completely on CIFAR10 and fails completely on
Revised: The boundary attack, however, succeeds completely on CIFAR10 and fails completely on

Original: MNIST, even when it managed to sometimes fool the detector without directly trying.
Revised: MNIST, even when it managed to sometimes fool the detector without directly trying.

Original: We hypothesize that this is because the area between two images of separate classes, where the boundary attack needs to pass through, is correctly detected as out of distribution only for MNIST and not CIFAR10.
Revised: We hypothesize that this is because the area between two images of separate classes, where the boundary attack needs to pass through, is correctly detected as out of distribution only for MNIST and not CIFAR10.

Original: We explore this further below.
Revised: We explore this further below.

Original: 4. A MBIGUOUS I NPUTS AS A VERAGE C ASE A NALYSIS
Revised: 4. A MBIGUOUS I NPUTS AS A VERAGE C ASE A NALYSIS

Original: To understand why the learned networks are easily attacked on CIFAR but not on MNIST with the modiﬁed boundary attack, we explore the probability density of interpolations between two real images.
Revised: To understand why the learned networks are easily attacked on CIFAR but not on MNIST with the modiﬁed boundary attack, we explore the probability density of interpolations between two real images.

Original: This is inspired by the fact that the boundary attack proceeds along the line between the attacked image and the initial image.
Revised: This is inspired by the fact that the boundary attack proceeds along the line between the attacked image and the initial image.

Original: The minimum we would expect from a decent generative model is to detect the intermediate middle images as “non-image” with low likelihood.
Revised: The minimum we would expect from a decent generative model is to detect the intermediate middle images as “non-image” with low likelihood.

Original: If this was the case and each class was a disconnected high likelihood region, the boundary attack would have a difﬁcult time when starting from a different class image.
Revised: If this was the case and each class was a disconnected high likelihood region, the boundary attack would have a difﬁcult time when starting from a different class image.

Original: Given images x 0 and x 1 from separate classes y 0 and y 1 and for α ∈ [0 , 1] we generate an intermediate image x α = α · x 1 + (1 − α ) x 0 , and run the model on various α values to see the model prediction[Figure] along the line.
Revised: Given images x 0 and x 1 from separate classes y 0 and y 1 and for α ∈ [0 , 1] we generate an intermediate image x α = α · x 1 + (1 − α ) x 0 , and run the model on various α values to see the model prediction along the line.

Original: For endpoints we sample real images that are classiﬁed correctly and are above the detection threshold used previously.
Revised: For endpoints we sample real images that are classiﬁed correctly and are above the detection threshold used previously.

Original: See Fig.
Revised: See Fig.

Original: 1 for interpolation examples from MNIST and CIFAR.
Revised: 1 for interpolation examples from MNIST and CIFAR.

Original: In ﬁgure 4 (a) we see the average results for MNIST for 1487 randomly selected pairs.
Revised: In ﬁgure 4 (a) we see the average results for MNIST for 1487 randomly selected pairs.

Original: As expected, the likelihood goes down as α moves away from the real images x 0 and x 1 .
Revised: As expected, the likelihood goes down as α moves away from the real images x 0 and x 1 .

Original: We also see the probability of both classes drop rapidly as the network predictions become less conﬁdent on the intermediate images.
Revised: We also see the probability of both classes drop rapidly as the network predictions become less conﬁdent on the intermediate images.

Original: Sampling 100 α values uniformly in the range [0 , 1] we can also investigate how many of the  interpolations all stay above the detection threshold, i.e. all intermediate images are considered real by the model, and ﬁnd that this happens only in 0.5% of the cases.
Revised: Sampling 100 α values uniformly in the range [0 , 1] we can also investigate how many of the [Figure] The are interpolations all stay above the detection threshold, i.e. all intermediate images are considered real by the model, and ﬁnd that this happens only in 0.5% of the cases.

Original: On CIFAR images, using 1179 pairs, we get a very different picture (see ﬁg. 4 (b)).
Revised: On CIFAR images, using 1179 pairs, we get a very different picture (see ﬁg. 4 (b)).

Original: Not only does the intermediate likelihood not drop down, it is even higher on average than on the real images albeit to a small degree.
Revised: Not only does the intermediate likelihood not drop down, it is even higher on average than on the real images albeit to a small degree.

Original: In classiﬁcation we also see a very smooth transition between classes, unlike the sharp drop in the MNIST experiment.
Revised: In classiﬁcation we also see a very smooth transition between classes, unlike the sharp drop in the MNIST experiment.

Original: Lastly, 100% of the interpolated images lay above the detection threshold and none are detected as a “non-image” (for reference the detection threshold has 78.6% recall on real CIFAR10 test images).
Revised: Lastly, 100% of the interpolated images lay above the detection threshold and none are detected as a “non-image” (for reference the detection threshold has 78.6% recall on real CIFAR10 test images).

Original: This shows that even with good likelihood and reasonable accuracy, the model still “mashes" the classes together, as one can move from one Gaussian to another without passing through low likelihood regions in-between.
Revised: This shows that even with good likelihood and reasonable accuracy, the model still “mashes" the classes together, as one can move from one Gaussian to another without passing through low likelihood regions in-between.

Original: It also clariﬁes why the boundary attack is so successful on CIFAR but fails completely on MNIST.
Revised: It also clariﬁes why the boundary attack is so successful on CIFAR but fails completely on MNIST.

Original: We note that the basic attack on MNIST is allowed to pass through these low density areas which is why it sometimes succeeds.
Revised: We note that the basic attack on MNIST is allowed to pass through these low density areas which is why it sometimes succeeds.

Original: We note that the results are not unique to ﬂow models, and we veriﬁed that similar phenomenon can be seen when training with the PixelCNN++ autoregressive model (Salimans et al., 2017) in sec. E.
Revised: [Figure]

Original:  In this section, we show that the difference in performance between CIFAR10 and MNIST is difﬁcult toexplain and not just a consequence of how the entropy in the datasets is distributed, i.e how much the uncertainty in the data distribution is reduced after conditioning on the class label.
Revised: Note how the model has learnd to capture digit identity. In this section, we show that the difference in performance between CIFAR10 and MNIST can largely be attributed to how the entropy in the datasets is distributed, i.e how much the uncertainty in the data distribution is reduced after conditioning on the class label.

Original: For MNIST digits, a large source of uncertainty in pixel-space comes from the class label.
Revised: For MNIST digits, a large source of uncertainty in pixel-space comes from the class label.

Original: Given the class, most pixels can be predicted accurately by simply taking the mean of the training set in each class.
Revised: Given the class, most pixels can be predicted accurately by simply taking the mean of the training set in each class.

Original: This is exactly why a linear classiﬁer performs well on MNIST.
Revised: This is exactly why a linear classiﬁer performs well on MNIST.

Original: Conversely on CIFAR10, after conditioning on the class label there still exists considerable uncertainty.
Revised: Conversely on CIFAR10, after conditioning on the class label there still exists considerable uncertainty.

Original: Given the class is “cat,” there still exists many complicated sources of uncertainty such as where the cat is and how it is posed, its surrounding scene and so forth.
Revised: Given the class is “cat,” there still exists many complicated sources of uncertainty such as where the cat is and how it is posed.

Original: In this dataset, a much larger fraction of the uncertainty ispotentially not accounted for after conditioning on the label.
Revised: In this dataset, a much larger fraction of the uncertainty is not accounted for after conditioning on the label.

Original: This is not a function of the domain or the dimensionality of the dataset, it is a function of the dataset itself.
Revised: This is not a function of the domain or the dimensionality of the dataset, it is a function of the dataset itself.

Original: To empirically investigate this, we have designed a dataset which replicates the challenges of CIFAR and places them onto a problem of the same discriminative difﬁculty as MNIST.
Revised: To empirically verify this, we have designed a dataset which replicates the challenges of CIFAR10 and places them onto a problem of the same discriminative difﬁculty as MNIST.

Original: To achieve this, we simply replaced the black backgrounds of MNIST images with randomly sampled (downsampled and greyscaled) images from CIFAR10.In this dataset, which we call background-MNIST (BG-MNIST). [Table]
Revised: To achieve this, we simply replaced the black backgrounds of MNIST images with randomly sampled (downsampled and greyscaled) images from CIFAR10.

Original: [Figure] [Figure] The classiﬁcation problem is identically predictable from the same set of pixels as in standard MNIST but modeling the data density is much more challenging.Note how the model has learned to capture digit identity despite the challenging CIFAR10 structure.
Revised: In this dataset, which we call background-MNIST (BG-MNIST), the classiﬁcation problem is identically predictable from the same set of pixels as in standard MNIST but modeling the data density is much more challenging.

Original: To further control the entropy in a ﬁne-grained manner, we convolve the background with a Gaussian blur ﬁlter with various bandwidths to remove varying degrees of high frequency information.
Revised: To further control the entropy in a ﬁne-grained manner, we convolve the background with a Gaussian blur ﬁlter with various bandwidths to remove varying degrees of high frequency information.

Original: With high blur, the task begins to resemble standard MNIST and conditional generative models should perform as they do on MNIST.
Revised: With high blur, the task begins to resemble standard MNIST and conditional generative models should perform as they do on MNIST.

Original: With low and no blur we expect them to behave as they do on CIFAR10.
Revised: With low and no blur we expect them to behave as they do on CIFAR10.

Original: Table 3 summarizes the performance of conditional generative models on BG-MNIST.
Revised: Table 3 summarizes the performance of conditional generative models on BG-MNIST.

Original: We train models with a “Reweighted” discriminative objective as in Section A.
Revised: We train models with a “Reweighted” discriminative objective as in Section A.

Original: The reweighting allows them to perform well as classiﬁers but the likelihood of their generative component falls to below CIFAR10 levels.
Revised: The reweighting allows them to perform well as classiﬁers but the likelihood of their generative component falls to below CIFAR levels.

Original: As with our MNIST models, when we interpolate between datapoints on these datasets, on average the likliehood is lower on interpolated datapoints than on true data. Interestingly, as the amount of blur decreases (and the data begins to more closely resemble CIFAR10), this difference becomes smaller and is overtaken by the variance in the likelihoods, meaning the likelihood is less successful in detecting these out-of-distribution examples.
Revised: More strikingly, now when we interpolate between datapoints we observe behavior identical to our CIFAR10 models.

Original: These results can be seen in Figure 5.
Revised: This can be seen in Figure 6.

Original: This is in line withour hypothesis that the issue is caused by a combination of class-unrelated entropy andcomplexity of the discriminative task itself.
Revised: Thus, we have created a dataset with the discriminative difﬁculty of MNIST and the generative difﬁculty of CIFAR10. [Table] [Figure]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Despite state of the art performance in many tasks, deep neural networks have been shown to be fragile where small image transformations, (Azulay & Weiss, 2018), or background object transplant (Rosenfeld et al., 2018) can greatly change predictions.
Revised: Despite state of the art performance in many tasks, deep neural networks have been shown to be fragile where small image transformations, (Azulay & Weiss, 2018) or background object transplant (Rosenfeld et al., 2018) can greatly change predictions.

Original: In the more challenging case of adversarial pertubations, deep neural networks are known to be vulnerable to adversarial attacks (Akhtar & Mian, 2018), and while many attempts have been made to train robust models or detect malicious attacks, signiﬁcant progress towards robustness has been made only on MNIST (Schott et al., 2019; Madry et al., 2017).
Revised: In the more challenging case of adversarial pertubations, deep neural networks are known to be vulnerable to adversarial attacks (Akhtar & Mian, 2018), and while many attempts have been made to train robust models or detect malicious attacks, signiﬁcant progress towards truly robust models has been made only on MNIST (Schott et al., 2019; Madry et al., 2017).

Original: Even CIFAR10 remains far from being solved from a standpoint of adversarial robustness.
Revised: Even CIFAR10 remains far from being solved from a standpoint of adversarial robustness.

Original: One common belief is that adversarial attacks succeed by moving the data points off the data manifold, and therefore can possibly be detected by a generative model which should assign them low likelihood values.
Revised: One common belief is that adversarial attacks succeed by moving the data points off the data manifold, and therefore can possibly be detected by a generative model which should assign them low likelihood values.

Original: Although this view has been challenged in (Gilmer et al., 2018), we now discuss how their setting needs to be extended to fully study robustness guarantees of conditional generative models.
Revised: Although this view has been challenged in (Gilmer et al., 2018), we now discuss how their setting needs to be extended to fully study robustness guarantees of conditional generative models.

Original: Recent work (Song et al., 2018; Frosst et al., 2018; Li et al., 2018) showed that a generative model can detect and defend adversarial attacks.
Revised: Recent work (Song et al., 2018; Frosst et al., 2018; Li et al., 2018) showed that a generative model can detect and defend adversarial attacks.

Original: However, there is a caveat when evaluating detectability of adversarial attacks: the attacker needs to be able to attack the detection algorithm as well.
Revised: However, there is a caveat when evaluating detectability of adversarial attacks: the attacker needs to be able to attack the detection algorithm as well.

Original: Not doing so has been shown to lead to drastically false robustness claims (Carlini & Wagner, 2017a).
Revised: Not doing so has been shown to lead to drastically false robustness claims (Carlini & Wagner, 2017a).

Original: In (Li et al., 2018) the authors report difﬁculties training a high accuracy conditional generative model on CIFAR10, and resort to evaluation on a 2-class classiﬁcation problem derived from CIFAR10.
Revised: In (Li et al., 2018) the authors report difﬁculties training a high accuracy conditional generative model on CIFAR10, and resort to evaluation on a 2-class classiﬁcation problem derived from CIFAR10.

Original: While they do show robustness similar to our Carlini-Wagner results, they do not apply the boundary attack which we found to break our models on CIFAR10.
Revised: While they do show robustness similar to our Carlini-Wagner results, they do not apply the boundary attack which we found to break our models on CIFAR10.

Original: This highlights the need to utilize a diverse set of attacks.
Revised: This highlights the need to utilize a diverse set of attacks.

Original: In (Schott et al., 2019) a generative model was used not just for adversarial detection but also robust classiﬁcation on MNIST, leading to state-of-the-art robust classiﬁcation accuracy.
Revised: In (Schott et al., 2019) a generative model was used not just for adversarial detection but also robust classiﬁcation on MNIST, leading to state-of-the-art robust classiﬁcation accuracy.

Original: The method was only shown to work on MNIST, and is very slow at inference time.
Revised: The method was only shown to work on MNIST, and is very slow at inference time.

Original: However, overall it provides an existence proof that conditional generative models can be very robust in practice.
Revised: However, overall it provides an existence proof that conditional generative models can be very robust in practice.

Original: In (Ghosh et al., 2019) the authors also use generative models for detection and classiﬁcation but only show results with the relatively weak FGSM attack, and on simple datasets.
Revised: In (Ghosh et al., 2019) the authors also use generative models for detection and classiﬁcation but only show results with the relatively weak FGSM attack, and on simple datasets.

Original: As we see in Fig.
Revised: As we see in Fig.

Original: 1 and discuss in section 4, generative models trained on MNIST can display very different behavior than similar models trained on more challenging data like CIFAR10.
Revised: 1 and discuss in section 4, generative models trained on MNIST can display very different behavior than similar models trained on more challenging data like CIFAR10.

Original: This shows how success on MNIST may often not translate to success on other datasets.
Revised: This shows how success on MNIST may often not translate to success on other datasets.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work we explored limitations, both in theory and practice, of using conditional generative models to detect adversarial attacks.
Revised: In this work we explored limitations, both in theory and practice, of using conditional generative models to detect adversarial attacks.

Original: Most practical issues arise due to likelihood, the standard objective and evaluation metric for generative models by which probabilities can be computed.
Revised: Most practical issues arise due to likelihood, the standard objective and evaluation metric for generative models by which probabilities can be computed.

Original: We conclude that likelihood-based density modeling and robust classiﬁcation may fundamentally be at odds with one another as important aspects of the problem are not captured by this training and evaluation metric.
Revised: We conclude that likelihood-based density modeling and robust classiﬁcation may fundamentally be at odds with one another as important aspects of the problem are not captured by this training and evaluation metric.

Original: This has wide-reaching implications for applications like out-of-distribution detection, adversarial robustness and generalization as well as semi-supervised learning with these models.
Revised: This has wide-reaching implications for applications like out-of-distribution detection, adversarial robustness and generalization as well as semi-supervised learning with these models.

Original: A T RAINING C ONDITIONAL G ENERATIVE M ODELS A. L IKELIHOOD - BASED G ENERATIVE M ODELS AS G ENERATIVE C LASSIFIERS
Revised: A T RAINING C ONDITIONAL G ENERATIVE M ODELS A. L IKELIHOOD - BASED G ENERATIVE M ODELS AS G ENERATIVE C LASSIFIERS

Original: We present a brief overview of ﬂow-based deep generative models, conditional generative models, and their applications to adversarial example detection.
Revised: We present a brief overview of ﬂow-based deep generative models, conditional generative models, and their applications to adversarial example detection.

Original: Flow based generative models Rezende & Mohamed (2015);
Revised: Flow based generative models Rezende & Mohamed (2015);

Original: Dinh et al.
Revised: Dinh et al.

Original: ; Kingma &
Revised: ; Kingma &

Original: Dhariwal (2018) compute exact densities for complex distributions using the change of variable formula.
Revised: Dhariwal (2018) compute exact densities for complex distributions using the change of variable formula.

Original: They achieve strong empirical results Kingma & Dhariwal (2018) and the closed form likelihood makes them easier to analyze than the closely related VAE Kingma et al.
Revised: They achieve strong empirical results Kingma & Dhariwal (2018) and the closed form likelihood makes them easier to analyze than the closely related VAE Kingma et al.

Original: The main idea behind ﬂow-based generative models is to model the data distribution using a series of bijective mappings z N = f ( x ) =
Revised: The main idea behind ﬂow-based generative models is to model the data distribution using a series of bijective mappings z N = f ( x ) =

Original: f N ◦ f N − 1 ... ◦
Revised: f N ◦ f N − 1 ... ◦

Original: f 1 ( x ) where z N has a known simple distribution, e.g. Gaussian, and all f i are parametric functions for which the determinant of the Jacobian can be computed efﬁciently.
Revised: f 1 ( x ) where z N has a known simple distribution, e.g. Gaussian, and all f i are parametric functions for which the determinant of the Jacobian can be computed efﬁciently.

Original: Using the change of variable formula we have log( p ( x ))
Revised: Using the change of variable formula we have log( p ( x ))

Original: =
Revised: =

Original: log( p ( z N ))
Revised: log( p ( z N ))

Original: + (cid:80)
Revised: + (cid:80)

Original: Ni =1 log( | det( J i ( z i ))
Revised: Ni =1 log( | det( J i ( z i ))

Original: | ) where J i is the Jacobian of f i and z i − 1 = f
Revised: | ) where J i is the Jacobian of f i and z i − 1 = f

Original: i − 1 ... ◦ f 1 ( x ) .
Revised: i − 1 ... ◦ f 1 ( x ) .

Original: The standard way to parameterize such functions f i is by splitting the input z
Revised: The standard way to parameterize such functions f i is by splitting the input z

Original: i − 1 into two z i − 1 = ( z 1 i − 1 , z 2 i − 1 ) and chose
Revised: i − 1 into two z i − 1 = ( z 1 i − 1 , z 2 i − 1 ) and chose

Original: [Equation]
Revised: [Equation]

Original: which is invertible as long as s ( z 1 i − 1 )
Revised: which is invertible as long as s ( z 1 i − 1 )

Original: j = s j (cid:54) = 

Revised: j = s j (cid:54) =

Original: and we have log( | det( J i ( z i − 1 ))
Revised: and we have log( | det( J i ( z i − 1 ))

Original: | )
Revised: | )

Original: = (cid:80) j log( | s j | ) .
Revised: = (cid:80) j log( | s j | ) .

Original: For images the splitting is normally done in the channel dimension.
Revised: For images the splitting is normally done in the channel dimension.

Original: These models are then trained by maximizing the empirical log likelihood (MLE).
Revised: These models are then trained by maximizing the empirical log likelihood (MLE).

Original: A straightforward way to turn this generative model into a conditional generative model is to make p ( z N ) a Gaussian mixture model (GMM) with one Gaussian per class, i.e. p ( z N | y ) = N ( µ y , Σ y ) .
Revised: A straightforward way to turn this generative model into a conditional generative model is to make p ( z N ) a Gaussian mixture model (GMM) with one Gaussian per class, i.e. p ( z N | y ) = N ( µ y , Σ y ) .

Original: Assuming p ( y ) is known, then maximizing log( p ( x, y )) is equivalent to maximizing log( p ( x | y ))
Revised: Assuming p ( y ) is known, then maximizing log( p ( x, y )) is equivalent to maximizing log( p ( x | y ))

Original: =
Revised: =

Original: log( p ( z N | y ))
Revised: log( p ( z N | y ))

Original: + (cid:80) Ni =1 log( | det( J i ( z i )) | ) .
Revised: + (cid:80) Ni =1 log( | det( J i ( z i )) | ) .

Original: At inference time, one can classify by simply using Bayes rule.
Revised: At inference time, one can classify by simply using Bayes rule.

Original: Note that directly optimizing log( p ( x | y ))
Revised: Note that directly optimizing log( p ( x | y ))

Original: results in poor classiﬁcation accuracy as discussed in section 3.
Revised: results in poor classiﬁcation accuracy as discussed in section 3.

Original: This issue was also addressed in the recent hybrid model work Nalisnick et al. (2019a).
Revised: This issue was also addressed in the recent hybrid model work Nalisnick et al. (2019a).

Original: We will now describe some of the various approaches we investigated in order to train the best possible ﬂow-based conditional generative models, to achieve a better trade-off betwen classiﬁcation accuracy and data-likelihood as compared to commonly-used approaches.
Revised: We will now describe some of the various approaches we investigated in order to train the best possible ﬂow-based conditional generative models, to achieve a better trade-off betwen classiﬁcation accuracy and data-likelihood as compared to commonly-used approaches.

Original: We also discuss some failed approaches in the appendix.
Revised: We also discuss some failed approaches in the appendix.

Original: A. R EWEIGHTING
Revised: A. R EWEIGHTING

Original: The most basic approach, which has been used before in various works, is to reweight the discriminative part in eq.
Revised: The most basic approach, which has been used before in various works, is to reweight the discriminative part in eq.

Original: While this can produce good accuracy, it can have an unfavorable trade-off with the NLL where good accuracy comes with severely sub-optimal NLL.
Revised: While this can produce good accuracy, it can have an unfavorable trade-off with the NLL where good accuracy comes with severely sub-optimal NLL.

Original: This tradeoff has also been shown in Nalisnick et al.
Revised: This tradeoff has also been shown in Nalisnick et al.

Original: (2019a) where they train a somewhat similar model but classify with a generalized linear model instead of a Gaussian mixture model.
Revised: (2019a) where they train a somewhat similar model but classify with a generalized linear model instead of a Gaussian mixture model.

Original: A. A RCHITECTURE CHANGE
Revised: A. A RCHITECTURE CHANGE

Original: Padding channels has been shown to increase accuracy in invertible networks Jacobsen et al.
Revised: Padding channels has been shown to increase accuracy in invertible networks Jacobsen et al.

Original: Behrmann et al.
Revised: Behrmann et al.

Original: This helps ameliorate a basic limitation in bijective mappings (see Eq.
Revised: This helps ameliorate a basic limitation in bijective mappings (see Eq.

Original: (6)), by allowing to increase the number of channels as a pre-processing step.
Revised: (5)), by allowing to increase the number of channels as a pre-processing step.

Original: Unlike the discriminative i-RevNet, we cannot just pad zeros as that would not be a continuous density.
Revised: Unlike the discriminative i-RevNet, we cannot just pad zeros as that would not be a continuous density.

Original: Instead we pad channels with uniform(0,1) random noise.
Revised: Instead we pad channels with uniform(0,1) random noise.

Original: In effect we do not model MNIST and CIFAR10 as is typically done in the literature, but rather the zero-padded version of those.
Revised: In effect we do not model MNIST and CIFAR10 as is typically done in the literature, but rather the zero-padded version of those.

Original: By padding the data x, with extra dimensions of uniform noise, our data is now (x, u), this deﬁnes a distribution p(x, u) = p(x)p(u) and since we are using uniform noise p(u)=1, so p(x, u) = p(x). Now, our generative model is modeling p(x, u), so if the marginal p(u) under our model is not perfectly uniform, then this equality no longer holds.
Revised: While the ground-truth likelihoods for the padded and un-padded datapoints are the same due to independence of the uniform noise and unit density of the noise, this is not guaranteed to be captured by the model, making likelihoods very similar but not exactly comparable with the literature.

Original: However, this is not a problem asall models we  compare against each other use the same amount of noise and padding.
Revised: This is not an issue for us, as we only compare models on the padded datasets.

Original: A. S PLIT PRIOR
Revised: A. S PLIT PRIOR

Original: One reason MLE is bad at capturing the label is because a small number of dimensions have a small effect on the NLL.
Revised: One reason MLE is bad at capturing the label is because a small number of dimensions have a small effect on the NLL.

Original: Fortunately, we can use this property to our advantage.
Revised: Fortunately, we can use this property to our advantage.

Original: As the contribution of the conditional class information is negligible for the data log likelihood, we choose to model it in its distinct subspace, as proposed by Jacobsen et al.
Revised: As the contribution of the conditional class information is negligible for the data log likelihood, we choose to model it in its distinct subspace, as proposed by Jacobsen et al.

Original: Thus, we partition the hidden dimensions z
Revised: Thus, we partition the hidden dimensions z

Original: = ( z s , z n ) and only try to enforce the low-dimensional z s to be the logits.
Revised: = ( z s , z n ) and only try to enforce the low-dimensional z s to be the logits.

Original: This has two advantages: 1) we do not enforce class-conditional dimensions to be factorial; and 2) we can explicitly up-weight the loss on this subspace and treat it as standard logits of a discriminative model.
Revised: This has two advantages: 1) we do not enforce class-conditional dimensions to be factorial; and 2) we can explicitly up-weight the loss on this subspace and treat it as standard logits of a discriminative model.

Original: A similar approach is also used by semi-supervised VAEs Kingma et al.
Revised: A similar approach is also used by semi-supervised VAEs Kingma et al.

Original: This lets us jointly optimize the data log-likelihood alongside a classiﬁcation objective without requiring most of the dimensions to be discriminative.
Revised: This lets us jointly optimize the data log-likelihood alongside a classiﬁcation objective without requiring most of the dimensions to be discriminative.

Original: Using the factorization p ( z s , z n | y )
Revised: Using the factorization p ( z s , z n | y )

Original: = p ( z s | y ) · p ( z n | z s , y ) we model p ( z s | y ) as Gaussian with class
Revised: = p ( z s | y ) · p ( z n | z s , y ) we model p ( z s | y ) as Gaussian with class

Original: conditional mean e i = (0 , ..., 0 , 1 , ... 0) and covariance matrix scaled by a constant.
Revised: conditional mean e i = (0 , ..., 0 , 1 , ... 0) and covariance matrix scaled by a constant.

Original: The distribution p ( z n | z s , y ) is modeled as a Gaussian where the mean and variances are a function of y and z n .
Revised: The distribution p ( z n | z s , y ) is modeled as a Gaussian where the mean and variances are a function of y and z n .

Original: B I MPLEMENTATION DETAILS
Revised: B I MPLEMENTATION DETAILS

Original: We pad MNIST with zeros so both datasets are 32x32 and subtract 0.5 from both datasets to have a [-0.5,0.5] range.
Revised: We pad MNIST with zeros so both datasets are 32x32 and subtract 0.5 from both datasets to have a [-0.5,0.5] range.

Original: For data augmentation we do pytorch’s random crop with a padding of 4 and ’edge’ padding mode, and random horizontal ﬂip for CIFAR10 only.
Revised: For data augmentation we do pytorch’s random crop with a padding of 4 and ’edge’ padding mode, and random horizontal ﬂip for CIFAR10 only.

Original: The model is based on GLOW with 4 levels, afﬁne coupling layers, 1x1 convolution permutations and actnorm in a multi-scale architecture.
Revised: The model is based on GLOW with 4 levels, afﬁne coupling layers, 1x1 convolution permutations and actnorm in a multi-scale architecture.

Original: We choose 128 channels and 12 blocks per level for MNIST and 256 channels and 16 blocks for CIFAR10.
Revised: We choose 128 channels and 12 blocks per level for MNIST and 256 channels and 16 blocks for CIFAR10.

Original: In both MNIST and CIFAR10 experiments we double the number of channels with uniforrm(0,1) noise which we scale down to the range [0 , 2 / 256] (taking it into account in the Jacobian term).
Revised: In both MNIST and CIFAR10 experiments we double the number of channels with uniforrm(0,1) noise which we scale down to the range [0 , 2 / 256] (taking it into account in the Jacobian term).

Original: One major difference is that we do the squeeze operation at the end of each level instead of the beginning, which is what allows us to use 4 levels.
Revised: One major difference is that we do the squeeze operation at the end of each level instead of the beginning, which is what allows us to use 4 levels.

Original: This is possible because with the added channels the number of channels is even and the standard splitting is possible before the squeeze operation.
Revised: This is possible because with the added channels the number of channels is even and the standard splitting is possible before the squeeze operation.

Original: The models are optimized using Adam, for 150 epochs.
Revised: The models are optimized using Adam, for 150 epochs.

Original: The initial learning rate is 1 e − 3 , decayed by a factor of 10 every 60 epochs.
Revised: The initial learning rate is 1 e − 3 , decayed by a factor of 10 every 60 epochs.

Original: For the reweighted optimization the objective is
Revised: For the reweighted optimization the objective is

Original: [Equation]
Revised: [Equation]

Original: where D is the data dimension (3x32x32 for CIFAR, 1x32x32 for MNIST).
Revised: where D is the data dimension (3x32x32 for CIFAR, 1x32x32 for MNIST).

Original: For adversarial detection we use a threshold of 1.4 for MNIST (100% of test data are below the threshold) and 4.
Revised: For adversarial detection we use a threshold of 1.4 for MNIST (100% of test data are below the threshold) and 4.

Original: for CIFAR10 (78.6% of test images are below the threshold).
Revised: for CIFAR10 (78.6% of test images are below the threshold).

Original: C N EGATIVE RESULTS
Revised: C N EGATIVE RESULTS

Original: In this work explored many ideas in order to achieve better tradeoff between accuracy with little or no impact.
Revised: In this work explored many ideas in order to achieve better tradeoff between accuracy with little or no impact.

Original: C. R OBUST PRIORS
Revised: C. R OBUST PRIORS

Original: Since the Gaussian prior is very sensitive to outliers, one idea was that conﬁdent miss-classiﬁcations carry a strong penalty which might result in “messing" all the classes together.
Revised: Since the Gaussian prior is very sensitive to outliers, one idea was that conﬁdent miss-classiﬁcations carry a strong penalty which might result in “messing" all the classes together.

Original: A solution would be to replace the Gaussian with a more robust prior, e.g. Laplace or Cauchy.
Revised: A solution would be to replace the Gaussian with a more robust prior, e.g. Laplace or Cauchy.

Original: Another idea we explored is a mixture of Gaussian and Laplace or Cauchy using the same location parameter.
Revised: Another idea we explored is a mixture of Gaussian and Laplace or Cauchy using the same location parameter.

Original: In our experiments we did not see any signiﬁcant difference from the Gaussian prior.
Revised: In our experiments we did not see any signiﬁcant difference from the Gaussian prior.

Original: C. L ABEL S MOOTHING
Revised: C. L ABEL S MOOTHING

Original: Another approach to try to address the same issue is a version of label smoothing.
Revised: Another approach to try to address the same issue is a version of label smoothing.

Original: In this new model the Gaussian clusters are a latent variable that is equal to the real label with probability 1 − (cid:15) and uniform on the other labels with probability (cid:15) .
Revised: In this new model the Gaussian clusters are a latent variable that is equal to the real label with probability 1 − (cid:15) and uniform on the other labels with probability (cid:15) .

Original: Using this will bound the error for conﬁdent miss-classiﬁcation as long as the data is close to one of the Gaussian centers.
Revised: Using this will bound the error for conﬁdent miss-classiﬁcation as long as the data is close to one of the Gaussian centers.

Original: C. F LOW -GAN
Revised: C. F LOW -GAN

Original: As we claimed the main issue is with the MLE objective, it seems like a better objective is to optimize
Revised: As we claimed the main issue is with the MLE objective, it seems like a better objective is to optimize

Original: KL ( p ( x, y ) ||
Revised: KL ( p ( x, y ) ||

Original: p θ ( x, y )) or the Jensen-Shannon divergence as this KL term is highly penalized for miss-classiﬁcation.
Revised: p θ ( x, y )) or the Jensen-Shannon divergence as this KL term is highly penalized for miss-classiﬁcation.

Original: It is also more natural when considering robustness against adversarial attacks.
Revised: It is also more natural when considering robustness against adversarial attacks.

Original: Optimizing this directly is hard, but generative adversarial networks (GANs) Goodfellow et al.
Revised: Optimizing this directly is hard, but generative adversarial networks (GANs) Goodfellow et al.

Original: (2014) in theory should also optimize this objective.
Revised: (2014) in theory should also optimize this objective.

Original: Simply training a GAN would not work as we are interested in the likelihood value for adversarial detection and GANs only let you sample and does not give you any information regarding an input image.
Revised: Simply training a GAN would not work as we are interested in the likelihood value for adversarial detection and GANs only let you sample and does not give you any information regarding an input image.

Original: Since ﬂow algorithms are bijective, we could combine the two objective as was done in the ﬂow-GAN paper Grover et al.
Revised: Since ﬂow algorithms are bijective, we could combine the two objective as was done in the ﬂow-GAN paper Grover et al.

Original: We trained this approach with various conditional-GAN alternatives and found it very hard to train.
Revised: We trained this approach with various conditional-GAN alternatives and found it very hard to train.

Original: GANs are know to be unstable to train, and combining them with the unstable ﬂow generator is problematic.
Revised: GANs are know to be unstable to train, and combining them with the unstable ﬂow generator is problematic.

Original: D A NALYTICAL COUNTER EXAMPLE :
Revised: D A NALYTICAL COUNTER EXAMPLE :

Original: Assume p ( y = 1)
Revised: Assume p ( y = 1)

Original: = p ( y = 0) =
Revised: = p ( y = 0) =

Original: q ( y = 1) =
Revised: q ( y = 1) =

Original: q ( y = 0) = 1 / 2 and
Revised: q ( y = 0) = 1 / 2 and

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: (11) where U ( a, b ) is the uniform distribution on the annulus R d ( a, b )
Revised: (10) where U ( a, b ) is the uniform distribution on the annulus R d ( a, b )

Original: = { x ∈ R d : a ≤ || x || ≤ b } in dimension d .
Revised: = { x ∈ R d : a ≤ || x || ≤ b } in dimension d .

Original: Lemma 1.
Revised: Lemma 1.

Original: For || x || < 1 we have
Revised: For || x || < 1 we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The U ( a, b ) density (when it isn’t zero) is 1 C d ( b d − a d ) where c d is the volume of the ddimensional unit ball.
Revised: The U ( a, b ) density (when it isn’t zero) is 1 C d ( b d − a d ) where c d is the volume of the ddimensional unit ball.

Original: The proof follows by a simple use of Bayes rule.
Revised: The proof follows by a simple use of Bayes rule.

Original: so by having λ 1 >> λ 2 >> 1 (1+∆) d we can have the model switch wrongfully predictions from y = 0 to y
Revised: so by having λ 1 >> λ 2 >> 1 (1+∆) d we can have the model switch wrongfully predictions from y = 0 to y

Original: = 1 when we move x from the annulus R d (1 , 1 + ∆) to R d (0 , 1)
Revised: = 1 when we move x from the annulus R d (1 , 1 + ∆) to R d (0 , 1)

Original: Lemma 2.
Revised: Lemma 2.

Original: If λ 1 > 1 (1+∆) d and λ 1 < 1 − e − (cid:15) then KL ( q ( x, y )
Revised: If λ 1 > 1 (1+∆) d and λ 1 < 1 − e − (cid:15) then KL ( q ( x, y )

Original: || P ( x, y )) ≤ (cid:15)
Revised: || P ( x, y )) ≤ (cid:15)

Original: Proof.
Revised: Proof.

Original: Using the chain rule for KL divergence, KL ( P ( x, y )
Revised: Using the chain rule for KL divergence, KL ( P ( x, y )

Original: || Q ( x, y ))
Revised: || Q ( x, y ))

Original: =
Revised: =

Original: KL ( P ( y ) || Q ( y ))
Revised: KL ( P ( y ) || Q ( y ))

Original: + E y [ KL ( P ( x | y )
Revised: + E y [ KL ( P ( x | y )

Original: || Q ( x | y ))
Revised: || Q ( x | y ))

Original: ] we get that KL ( q ( x, y )
Revised: ] we get that KL ( q ( x, y )

Original: || P ( x, y ))
Revised: || P ( x, y ))

Original: =
Revised: =

Original: KL ( q ( x | y = 0)
Revised: KL ( q ( x | y = 0)

Original: ||
Revised: ||

Original: P ( x | y = 0)) .
Revised: P ( x | y = 0)) .

Original: We now have
Revised: We now have

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Lemma 3.
Revised: Lemma 3.

Original: If 1 >
Revised: If 1 >

Original: [Equation]
Revised: [Equation]

Revised: 1 >

Original: [Equation]
Revised: [Equation]

Revised: log

Revised: [Equation]

Revised: [Equation]

Original: then KL ( P ( x, y ) ||
Revised: then KL ( P ( x, y ) ||

Original: q ( x, y )) ≤ (cid:15)
Revised: q ( x, y )) ≤ (cid:15)

Original: Proof.
Revised: Proof.

Original: Again using the KL chain rule we have
Revised: Again using the KL chain rule we have

Original: [Equation]
Revised: [Equation]

Original: Proposition 1.
Revised: Proposition 1.

Original: For all ( (cid:15), δ, ∆) there is a distribution p and an approximation q in dimension d = 
Revised: For all ( (cid:15), δ, ∆) there is a distribution p and an approximation q in dimension d = ˜ O

Original: [Equation]
Revised: [Equation]

Original: such that
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: but with probability greater then 1 / 3 over samples x ∼ p there is an adversarial example ¯ x satisfying:
Revised: but with probability greater then 1 / 3 over samples x ∼ p there is an adversarial example ¯ x satisfying:

Original: y q ( x ) =
Revised: y q ( x ) =

Original: y p ( x ) with p ( y p ( x ) | x ) and q ( y q ( x ) | x ) greater or equal to 1 − δ .
Revised: y p ( x ) with p ( y p ( x ) | x ) and q ( y q ( x ) | x ) greater or equal to 1 − δ .

Original: The original point is classiﬁer correctly and conﬁdently.
Revised: The original point is classiﬁer correctly and conﬁdently.

Original: y q ( x ) (cid:54) =
Revised: y q ( x ) (cid:54) =

Original: y (¯ x ) , y q (¯ x ) =
Revised: y (¯ x ) , y q (¯ x ) =

Original: y (¯ x ) .
Revised: y (¯ x ) .

Original: We change the prediction without changing the ground-truth label.
Revised: We change the prediction without changing the ground-truth label.

Original: q ( y q (¯ x ) | ¯ x ) < δ , p ( y p (¯ x )
Revised: q ( y q (¯ x ) | ¯ x ) < δ , p ( y p (¯ x )

Original: | ¯ x ) > 1 − δ .
Revised: | ¯ x ) > 1 − δ .

Original: The classiﬁer is conﬁdent in its wrong prediction.
Revised: The classiﬁer is conﬁdent in its wrong prediction.

Original: || x − ¯ x || < ∆ .
Revised: || x − ¯ x || < ∆ .

Original: We make a small change to the inputs.
Revised: We make a small change to the inputs.

Original: The density q (¯ x ) is greater or equal to the median density, making the attack undetectable by observing q ( x ) .
Revised: The density q (¯ x ) is greater or equal to the median density, making the attack undetectable by observing q ( x ) .

Original: For ∆ < 1 the probability in any radius ball can be made as small as desired.
Revised: For ∆ < 1 the probability in any radius ball can be made as small as desired.

Original: The total variation of the distribution can be made as small as desired.
Revised: The total variation of the distribution can be made as small as desired.

Original: The last two conditions exclude degenerate trivial counter-exmaples, one where the whole distribution support is in a ∆ radius ball and ∆ does indeed represent a small pertubation.
Revised: The last two conditions exclude degenerate trivial counter-exmaples, one where the whole distribution support is in a ∆ radius ball and ∆ does indeed represent a small pertubation.

Original: The other condition excludes “pathological" distributions ,e.g. misclassiﬁcation on a dense zero measure set like the rationals.
Revised: The other condition excludes “pathological" distributions ,e.g. misclassiﬁcation on a dense zero measure set like the rationals.

Original: Proof.
Revised: Proof.

Original: In order to satisfy conditions 1-5, using previous lemmas, it is enough that
Revised: In order to satisfy conditions 1-5, using previous lemmas, it is enough that

Original: λ 1 λ 1 + λ 2 ≥ 1 − δ 2.
Revised: λ 1 λ 1 + λ 2 ≥ 1 − δ 2.

Original: + λ 2 (1+∆) d ≤ δ 3.
Revised: + λ 2 (1+∆) d ≤ δ 3.

Original: λ 1 ≤ 1 − e − (cid:15) 4.
Revised: λ 1 ≤ 1 − e − (cid:15) 4.

Original: λ 1 > 1 (1+∆) d 5.
Revised: λ 1 > 1 (1+∆) d 5.

Original: λ 1 < (cid:15) d log(1+∆)
Revised: λ 1 < (cid:15) d log(1+∆)

Original: By setting λ 2 = δ 1 − δ λ 1 we can easily satisfy condition 1.
Revised: By setting λ 2 = δ 1 − δ λ 1 we can easily satisfy condition 1.

Original: It is not hard to see that condition 2 is equivalent to λ 1 ≥ (cid:0) 1 − δ δ (cid:1)1 (1+∆) d which superseeds condition 

Revised: It is not hard to see that condition 2 is equivalent to λ 1 ≥ (cid:0) 1 − δ δ (cid:1)1 (1+∆) d which superseeds condition

Original: when δ < 1 / 2 .
Revised: when δ < 1 / 2 .

Original: Condition 3 can be satisﬁed with λ 1 < (cid:15)/ 2 by using 1 − x ≥ e − 2 x for x < 1 / 2 .
Revised: Condition 3 can be satisﬁed with λ 1 < (cid:15)/ 2 by using 1 − x ≥ e − 2 x for x < 1 / 2 .

Original: This boils down to ensuring d is large enough so that there is a valid λ 1 such as
Revised: This boils down to ensuring d is large enough so that there is a valid λ 1 such as

Original: [Equation]
Revised: [Equation]

Original: Which is true for large enough d as the l.h.s decays exponentially while the r.h.s linearly.
Revised: Which is true for large enough d as the l.h.s decays exponentially while the r.h.s linearly.

Original: Condition 6 is trivial as the radius of the support is ﬁxed so as long as ∆ < 1 the probability in any ∆ radius ball decays exponentially.
Revised: Condition 6 is trivial as the radius of the support is ﬁxed so as long as ∆ < 1 the probability in any ∆ radius ball decays exponentially.

Original: Regarding total variation, we note that from the divergence theorem this can be bounded by a term that depends on the surface area of shperes with ﬁxed radius which decreases to zero as d goes to inﬁnity.
Revised: Regarding total variation, we note that from the divergence theorem this can be bounded by a term that depends on the surface area of shperes with ﬁxed radius which decreases to zero as d goes to inﬁnity.

Original: E P IXEL CNN++
Revised: E P IXEL CNN++

Original: We trained a conditional PixelCNN++ where instead of predicting each new pixel using a mixture of 10 components, we use one mixture component per class.
Revised: We trained a conditional PixelCNN++ where instead of predicting each new pixel using a mixture of 10 components, we use one mixture component per class.

Original: Using reweighting we train using the following objective − log ( p ( x | y )) /dim
Revised: Using reweighting we train using the following objective − log ( p ( x | y )) /dim

Original: + α · − log ( p ( y | x )) .
Revised: + α · − log ( p ( y | x )) .

Original: As one can see from table 4, standard trainig, i.e. α = 0 , results in very poor accuracy, while reweighting the classiﬁcation score results in much better accuracy but worse NLL.
Revised: As one can see from table 4, standard trainig, i.e. α = 0 , results in very poor accuracy, while reweighting the classiﬁcation score results in much better accuracy but worse NLL.

Original: [Table]
Revised: [Table]