Understanding the Limitations of Conditional Generative Models

1 INTRODUCTION . Conditional generative models have recently shown promise to overcome many limitations of their discriminative counterparts . They have been shown to be robust against adversarial attacks ( Schott et al. , 2019 ; Ghosh et al. , 2019 ; Song et al. , 2018 ; Li et al. , 2018 ; Frosst et al. , 2018 ) , to enable robust classification in the presence of outliers ( Nalisnick et al. , 2019b ) and to achieve promising results in semi-supervised learning ( Kingma et al. , 2014 ; Salimans et al. , 2016 ) . Motivated by these success stories , we study the properties of conditional generative models in more detail . Unlike discriminative models , which can ignore class-irrelevant information , conditional generative models can not discard any information in the input , potentially making it harder to fool them . Further , ∗equal contribution jointly modeling the input and target distribution should make it easy to detect out-of-distribution inputs . These traits lend hope to the belief that good class-conditional generative models can overcome important problems faced by discriminative models . In this work , we analyze conditional generative models by assessing them on a spectrum of robustness tasks . ( 1 ) Detection of worst-case outliers in the form of adversarial examples ; ( 2 ) Detection of average-case outliers in the form of ambiguous inputs and ( 3 ) Detection of incorrectly labeled indistribution inputs . If a generative classifier is able to perform well on all of these , it will naturally be robust to noisy , ambiguous or adversarially perturbed inputs . Outlier detection in the above settings is substantially different from general out-of-distribution ( OOD ) detection , where the goal is to use unconditional generative models to detect any OOD input . For the general case , likelihood has been shown to be a poor detector of OOD samples . In fact , often higher likelihood is assigned to OOD data than to the training data itself ( Nalisnick et al. , 2019a ) . However , class-conditional likelihood necessarily needs to decrease towards the decision-boundary for the classifier to work well . Thus , if the class-conditional generative model has high accuracy , rejection of outliers from the wrong class via likelihood may be possible . Our contributions are : Provable Robustness We answer : Can we theoretically guarantee that a strong conditional generative model can robustly detect adversarially attacked inputs ? In section 2 we show that even a near-perfect conditional generative model can not be guaranteed to reject adversarially perturbed inputs with high probability . Assessing the Likelihood Objective We discuss the basis to empirically analyze robustness in practice . We identify several fundamental issues with the maximum likelihood objective typically used to train conditional generative models and discuss whether it is appropriate for detecting out-of-distribution inputs . Understanding Conflicting Results We explore various properties of our trained conditional generative models and how they relate to fact that the model is robust on MNIST but not on CIFAR10 . We further propose a new dataset where we combine MNIST images with CIFAR background , making the generative task as hard as CIFAR while keeping the discriminative task as easy as MNIST , and investigate how it affects robustness . 2 CONFIDENT MISTAKES CAN NOT BE RULED OUT . The most challenging task in robust classification is accurately classifying or detecting adversarial attacks ; inputs which have been maliciously perturbed to fool the classifier . In this section we discuss the possibility of guaranteeing robustness to adversarial attacks via conditional generative models . Detectability of Adversarial Examples In the adversarial spheres work ( Gilmer et al. , 2018 ) the authors showed that a model can be fooled without changing the ground-truth probability of the attacked datapoint . This was claimed to show that adversarial examples can lie on the data manifold and therefore can not be detected . While ( Gilmer et al. , 2018 ) is an important work for understanding adversarial attacks , it has several limitations with regard to conditional generative models . First , just because the attack does not change the ground-truth likelihood , this does not mean the model can not detect the attack . Since the adversary needs to move the input to a location where the model is incorrect , the question arises : what kind of mistake will the model make ? If the model assigns low likelihood to the correct class without increasing the likelihood of the other classes then the adversarial attack will be detected , as the joint likelihood over all classes moves below the threshold of typical inputs . Second , on the adversarial spheres dataset ( Gilmer et al. , 2018 ) the class supports do not overlap . If we were to train a model of the joint density pθ ( x , y ) ( which does not have 100 % classification accuracy ) then the KL divergence KL ( p ( x , y ) ||pθ ( x , y ) ) , where p ( x , y ) is the data density , is infinite due to division by zero ( note that KL ( pθ ( x , y ) ||p ( x , y ) ) is what is minimized with maximum likelihood ) . This poses the question , whether small KL ( p ( x , y ) ||pθ ( x , y ) ) or small Shannon-Jensen divergence is sufficient to guarantee robustness . In the following , we show that this condition is insufficient . Class 1 DensityClass 0 Density Bayes-optimal Classification Model Classification Why no Robustness Guarantee can be Given The intuition why conditional generative models should be robust is as follows : If we have a robust discriminative model then the set of confident mistakes , i.e . where the adversarial attacks must reside , has low probability but might be large in volume . For a robust conditional generative model , the set of undetectable adversarial attacks , i.e . high-density high-confidence mistakes , has to be small in volume . Since the adversary has to be ∆ close to this small volume set , the ∆ area around this small volume set should still be small . This is where the idea breaks down due to the curse of dimensionality . Expanding a set by a small radius can lead to a much larger one even with smoothness assumptions . Based on this insight we build an analytic counter-example for which we can prove that even if KL ( q||p ) < KL ( p||q ) < ( 1 ) where p = p ( x , y ) is the data distribution , and q = q ( x , y ) is the model , we can with probability ≈ 0.5 take a correctly classified input sampled from p , and perturb it by at most ∆ to create an adversarial example that is classified incorrectly and is not detectable . We note that the probability in every ball with radius ∆ can be made as small as desired , excluding degenerate cases . We also assume that the Bayes optimal classifier is confident and is not affected by the attack , i.e . we do not change the underlying class but wrongfully flip the decision of the classifier . The counter-example goes as follows : Let U ( a , b ) be the density of a uniform distribution on an annulus in dimension d , { x ∈ Rd : a ≤ ||x|| ≤ b } then the data conditional distribution is p ( x|0 ) = λ1U ( 0 , 1 ) + ( 1− λ1 ) U ( 1 , 1 + ∆ ) 0 ≤ λ1 ≤ 1 ( 2 ) p ( x|1 ) = λ2U ( 0 , 1 ) + ( 1− λ2 ) U ( 2 , 3 ) 0 ≤ λ2 ≤ 1 with p ( y = 0 ) = p ( y = 1 ) = 1/2 . Both classes are a mixture of two distributions , uniform on the unit sphere and uniform on an annulus , as shown in Fig . 2 . The model distribution is the following : q ( x|0 ) = U ( 0 , 1 + ∆ ) ( 3 ) q ( x|1 ) = λ2U ( 0 , 1 ) + ( 1− λ2 ) U ( 2 , 3 ) i.e . for y = 1 the model is perfect , while for y = 0 we replace the mixture with uniform distribution over the whole domain . If λ1 λ2 then points in the sphere with radius 1 should be classified as class y = 0 with high likelihood . If λ2 > > 1 ( 1+∆ ) d then the model classifies points in the unit sphere incorrectly with high likelihood . Finally if 1 > > λ1 then almost half the data points will fall in the annulus between 1 and 1 + ∆ and can be adversarially attacked with distance lesser or equal to ∆ by moving them into the unit sphere as seen in Fig . 2 . We also note that these attacks can not be detected as the model likelihood only increases . In high dimensions , almost all the volume of a sphere is in the outer shell , and this can be used to show that in high enough dimensions we can get the condition in Eq . 1 for any value of and ∆ ( and also the confidence of the mistakes δ ) . The detailed proof is in the supplementary material . This counter-example shows that even under very strong conditions , a good conditional generative model can be attacked . Therefore no theoretical guarantees can be given in the general case for these models . Our construction , however , does not depend on the learning model but on the data geometry . This raises interesting questions concerning the source of the susceptibility to attacks : Is it the model or an inherent issue with the data ? 3 THE MAXIMUM LIKELIHOOD OBJECTIVE . 3.1 THE DIFFICULTY IN TRAINING CONDITIONAL GENERATIVE MODELS . Most recent publications on likelihood-based generative models primarily focus on quantitative results of unconditonal density estimation ( van den Oord et al. , 2016 ; Kingma & Dhariwal , 2018 ; Salimans et al. , 2017b ; Kingma et al. , 2016 ; Papamakarios et al. , 2017 ) . For conditional density estimation , either only qualitative samples are shown ( Kingma & Dhariwal , 2018 ) , or it is reported that conditional density estimation does not lead to better likelihoods than unconditional density estimation . In fact , it has been reported that conditional density estimation can lead to slightly worse data likelihoods ( Papamakarios et al. , 2017 ; Salimans et al. , 2017b ) , which is surprising at first , as extra bits of important information are provided to the model . Explaining Likelihood Behaviour One way to understand this seemingly contradictory relationship is to consider the objective we use to train our models . When we train a generative model with maximum likelihood ( either exactly or through a lower bound ) we are minimizing the empirical approximation of Ex , y∼P [ − log ( Pθ ( x , y ) ) ] which is equivalent to minimizing KL ( P ( x , y ) ||Pθ ( x , y ) ) . Consider now an image x with a discrete label y , which we are trying to model using Pθ ( x , y ) . The negative log-likelihood ( NLL ) objective is : E ( x , y ) ∼P [ − log ( Pθ ( x , y ) ) ] = Ex∼P [ − log ( Pθ ( x ) ) ] + Ex∼P [ Ey [ − log ( Pθ ( y|x ) ) |x ] ] ( 4 ) If we model Pθ ( y|x ) with a uniform distribution over classes , then the second term has a value of log ( C ) where C is the number of classes . This value is negligible compared to the first term Ex∼P [ − log ( Pθ ( x ) ) ] and therefore the “ penalty '' for completely ignoring class information is negligible . So it is not surprising that models with strong generative abilities can have limited discriminative power . What makes matters even worse is that the penalty for confident mis-classification can be unbounded . This may also explain why the conditional ELBO is comparable to the unconditional ELBO ( Papamakarios et al. , 2017 ) . Another way this can be seen is by thinking of the likelihood as the best lossless compression . When trying to encode an image , the benefit of the label is at most log ( C ) bits which is small compared to the whole image . While these few bits are important for users , from a likelihood perspective the difference between the correct p ( y|x ) and a uniform distribution is negligible . This means that when naively training a class-conditional generative model by minimizing E ( x , y ) ∼P [ − log ( Pθ ( x|y ) ) ] , typically discriminative performance as a classifier is very poor .

Summary: The authors claim that likelihood based generative models are not as robust to noise as general consensus claims them to be. To prove this authors make use of adversarial, ambiguous and incorrectly labeled in distribution inputs. Authors address issues regarding robustness in near perfect conditional generative models as well as assess the robustness of the likelihood objective.