Are Generative Classifiers More Robust to Adversarial Attacks?

1 INTRODUCTION . Deep neural networks have been shown to be vulnerable to adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ) . The latest attack techniques can easily fool a deep neural network with imperceptible perturbations ( Goodfellow et al. , 2014 ; Papernot et al. , 2016b ; Carlini & Wagner , 2017a ; Kurakin et al. , 2016 ; Madry et al. , 2018 ; Chen et al. , 2017a ) , even in the black-box case , where the attacker does not have access to the network ’ s weights ( Papernot et al. , 2017b ; Chen et al. , 2017b ; Alzantot et al. , 2018a ) . Adversarial attacks are serious security threats to machine learning systems , threatening applications beyond image classification ( Carlini & Wagner , 2018 ; Alzantot et al. , 2018b ) . To address this outstanding security issue , researchers have proposed defence mechanisms against adversarial attacks . Adversarial training , which augments the training data with adversarially perturbed inputs , has shown moderate success at defending against recently proposed attack techniques ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Tramèr et al. , 2017 ; Madry et al. , 2018 ) . In addition , recent advances in Bayesian neural networks have demonstrated that uncertainty estimates can be used to detect adversarial attacks ( Li & Gal , 2017 ; Feinman et al. , 2017 ; Louizos & Welling , 2017 ; Smith & Gal , 2018 ) . Another notable category of defence techniques involves the usage of generative models . For example , Gu & Rigazio ( 2014 ) used an auto-encoder to denoise the inputs before feeding them to the classifier . This denoising approach has been extensively investigated , and the “ denoisers ” in usage include generative adversarial networks ( Samangouei et al. , 2018 ) , PixelCNNs ( Song et al. , 2018 ) and denoising auto-encoders ( Kurakin et al. , 2018 ) . These developments rely on the “ off-manifold ” conjecture , that is , that adversarial examples are far away from the data manifold , although Gilmer et al . ( 2018 ) has challenged this observation with a synthetic example . Surprisingly , much less recent work has investigated the robustness of generative classifiers ( Ng & Jordan , 2002 ) against adversarial attacks for multi-class classification , where such classifiers explicitly model the conditional distribution of the inputs given labels . Typically , a generative classifier produces predictions by comparing between the likelihood of the labels for a given input , which is closely related to the “ distance ” of the input to the data manifold associated with a class . Therefore , generative classifiers should be robust to many recently proposed adversarial attacks if the “ off-manifold ” conjecture holds for many real-world applications . Unfortunately , many generative classifiers in popular use , including naive Bayes and linear discriminant analysis ( Fisher , 1936 ) , perform poorly on natural image classification tasks , making it difficult to verify the “ off-manifold ” conjecture and the robustness of generative classifiers with these tools . In recent work , k-nearest neighbors ( Cover & Hart , 1967 ) , a method which shares many similarities with generative classi- 1 Under review as a conference paper at ICLR 2019 fiers , has been significantly improved in handling natural images by leveraging deep feature representations ( Papernot & McDaniel , 2018 ) . To the best of our knowledge , an approach which targets a similar contribution has not yet been proposed for generative classifiers . Are generative classifiers more robust to recently proposed adversarial attack techniques ? To answer this , we improve the naive Bayes algorithm by using conditional deep generative models , and evaluate the conjecture on the proposed generative classifier . In summary , our contributions include : • We propose deep Bayes as an extension of naive Bayes , in which the conditional distribution of an input , given a label , is parameterised by a deep latent variable model ( LVM ) . We learn the LVM with the variational auto-encoder algorithm ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ) , and approximate Bayes ’ rule using importance sampling . • We propose three detection methods for adversarial perturbations . The first two use the learned generative model as a proxy of the data manifold , and reject inputs that are far away from it . The third computes statistics for the classifier ’ s output probability vector on training data , and rejects inputs that lead to under-confident predictions . • We evaluate the robustness of the proposed generative classifier on the MNIST multi-class and CIFAR binary classification tasks . We also improve the robustness of deep neural networks on CIFAR-10 multi-class classification , by fusing discriminatively learned visual feature representations with the proposed generative classifiers . We further compare the generative classifiers with a number of discriminative classifiers , including Bayesian neural networks and discriminative latent variable models . 2 DEEP BAYES : CONDITIONAL DEEP LVM AS A GENERATIVE CLASSIFIER . Denote pD ( x , y ) the data distribution for the input x ∈ RD and label y ∈ { yc|c = 1 , ... , C } , where yc denotes the one-hot encoding vector for class c. For a given x ∈ RD we can define the ground-truth label by y ∼ pD ( y|x ) if x ∈ supp ( pD ( x ) ) . ( 1 ) We assume the data distribution pD ( x , y ) follows the manifold assumption : for every class c , the conditional distribution pD ( x|yc ) has a low-dimensional manifold supportMc = supp ( pD ( x|yc ) ) . Therefore the training dataset D = { ( x ( n ) , y ( n ) ) } Nn=1 is generated as the following : ( x ( n ) , y ( n ) ) ∼ pD ( x , y ) ⇔ y ( n ) ∼ pD ( y ) , x ( n ) ∼ pD ( x|y ) . A ( Bayesian ) generative classifier first builds a generative model p ( x , y ) = p ( x|y ) p ( y ) , and then , in prediction time , predicts the label y∗ of a test input x∗ using Bayes ’ rule , p ( y∗|x∗ ) = p ( x ∗|y∗ ) p ( y∗ ) p ( x∗ ) = softmaxCc=1 [ log p ( x ∗ , yc ) ] , where softmaxCc=1 denotes the softmax operator over the c axis . Therefore , the output probability vector is computed analogously to many deep discriminative classifiers which use softmax activation in the output layer , so many existing attacks can be tested directly . However , unlike discriminative classifiers , the “ logit ” values prior to softmax activation have a clear meaning here , which is the ( approximated ) log joint distribution log p ( x∗ , yc ) of input x∗ conditioned on a given label yc . Therefore , one can also analyse the logit values to determine whether the unseen pair ( x∗ , y∗ ) is legitimate , a utility which will be discussed further in later sections . Naive Bayes is perhaps the most well-known generative classifier ; it assumes a factorised distribution for the conditional generator , i.e . p ( x|y ) = ∏D d=1 p ( xd|y ) . However , this factorisation assumption is inappropriate for e.g . image and speech data . Fortunately , we can leverage the recent advances in generative modelling and apply a deep generative model for the joint distribution p ( x , y ) . We refer to such generative classifiers that use deep generative models as deep Bayes classifiers . In this paper , we consider a deep latent variable model ( LVM ) p ( x , z , y ) , which will be used for classification . In this case , the conditional distribution is p ( x|y ) = ∫ p ( x , z , y ) dz∫ p ( x , z , y ) dzdx . Importantly , this leads to a conditional distribution p ( x|y ) that is not factorised ( even when p ( x|z , y ) is ) , which is much more powerful than naive Bayes . Since maximum likelihood is intractable , we follow 2 Under review as a conference paper at ICLR 2019 Kingma & Welling ( 2013 ) and Rezende et al . ( 2014 ) to introduce an amortised approximate posterior q ( z|x , y ) , and train both p and q by maximising the variational lower-bound : ED [ LVI ( x , y ) ] = 1 N N∑ n=1 Eq [ log p ( xn , zn , yn ) q ( zn|xn , yn ) ] . ( 2 ) After training , the predicted class probability vector y∗ for a future input x∗ is computed by an approximation to Bayes ’ rule with importance sampling : p ( y∗|x∗ ) ≈ softmaxCc=1 [ log 1 K K∑ k=1 p ( x∗ , zkc , yc ) q ( zkc |x∗ , yc ) ] , zkc ∼ q ( z|x∗ , yc ) . ( 3 ) We evaluate the effect of the following factorisation structures on the robustness of the induced classifier from the generative model p ( x , z , y ) ( see Figure 1 ) . p ( x , z , y ) = p ( z ) p ( y|z ) p ( x|z , y ) ( GFZ ) p ( x , z , y ) = pD ( y ) p ( z|y ) p ( x|z , y ) ( GFY ) p ( x , z , y ) = p ( z ) p ( y|z ) p ( x|z ) ( GBZ ) p ( x , z , y ) = pD ( y ) p ( z|y ) p ( x|z ) ( GBY ) p ( x , z , y ) = pD ( x ) p ( z|x ) p ( y|z , x ) ( DFX ) p ( x , z , y ) = p ( z ) p ( x|z ) p ( y|z , x ) ( DFZ ) p ( x , z , y ) = pD ( x ) p ( z|x ) p ( y|z ) ( DBX ) We use the initial character “ G ” to denote generative classifiers and “ D ” to denote discriminative classifiers . Models with the second character as “ F ” have a fully connected graph , while “ B ” models enforce the usage of the latent code z as a bottleneck . The last character of the model name indicates the first node in topological order . Model DFZ is somewhat intermediate , as it builds a generative model for the inputs x but also directly parameterises the conditional distribution p ( y|x , z ) . We do not test other architectures under this nomenclature , as either the graph contains directed cycles ( e.g . x→ y → z → x ) , or z is the last node in topological order ( e.g . x→ y , ( x , y ) → z ) so that the marginalisation of z does not affect classification . 3 DETECTING ADVERSARIAL ATTACKS WITH GENERATIVE CLASSIFIERS . We propose detection methods for adversarial examples using generative classifier ’ s logit values . As an illustrating example , consider a labelled dataset of “ cat ” and “ dog ” images . If an adversarial image of a cat xadv is incorrectly labelled as “ dog ” , then either this image is ambiguous , or , under a perfect generative model , the logit log p ( xadv , “ dog ” ) will be significantly lower than normal . This means we can detect attacks using the logits log p ( x∗ , yc ) , c = 1 , ... , C computed on a test input x∗ , by comparing them with the logits computed on legitimate training inputs . Concretely , the proposed detection algorithms are as follows . We aim to reject both unlabelled input x that have low probability under pD ( x ) , and labelled data ( x , y ) that have low pD ( x , y ) values . • Marginal detection : rejecting inputs that are far away from the manifold . One can select a threshold δ and reject an input x if − log p ( x ) > δ . To determine the threshold δ , we can compute d̄D = Ex∼D [ − log p ( x ) ] and σD = √ Vx∼D [ log p ( x ) ] , then set δ = d̄D + ασD . It is also possible to compute the statistics d̄p , σp on the images generated by the generative model accordingly . • Logit detection : rejecting inputs using joint density . Given a victim model y = F ( x ) , one can reject x if − log p ( x , F ( x ) ) > δy . We can use the mean and variance statistics d̄c , σc computed on log p ( x , yc ) and select δyc = d̄c+ασc . 3 Under review as a conference paper at ICLR 2019 • Divergence detection : rejecting inputs with over- and/or under-confident predictions . Denote p ( x ) as a C-dimensional probability vector outputted by the classifier . For each class c , we first collect the mean classification probability vector pc = E ( x , yc ) ∈D [ p ( x ) ] , then compute the mean d̄c and standard deviation σc on a selected divergence/distance measure D [ pc||p ( x ) ] for all ( x , yc ) ∈ D. A test input x∗ with prediction label c∗ = arg maxp ( x∗ ) is rejected if D [ pc∗ ||p ( x∗ ) ] > d̄c∗ + ασc∗ . Therefore , an example x∗ will be rejected if the classifier is over-confident or under-confident ( ambiguous inputs ) . When D is selected as the KL-divergence , we call this detection method KL detection . Other divergence/distance measures such as total variation ( TV ) can also be used . For better intuition , we visualise the detection mechanisms in Figure 2 with a synthetic “ two rings ” binary classification example . In this case we sample 2× 1000 training data points as : ( x , y ) ∼ pD ⇔ y ∼ Bern ( 0.5 ) , θ ∼ Uniform ( 0 , 2π ) , x|y ∼ N ( x ; cy + ry [ cos ( θ ) , sin ( θ ) ] T , σ2I ) . We consider a generative classifier p ( x , y ) = p ( x|y ) pD ( y ) = N ( x ; µy , σ2I ) Bern ( 0.5 ) with µy = proj ( x , ringy ) = arg minx̂∈R2 , ||x̂−cy||2=ry ||x − x̂||2 . The δ thresholds are selected to achieve 10 % false positive rates on training data . From the visualisations we see that inputs that are far away from the model manifold are rejected by marginal/logit detection . At the same time , logit detection rejects data points that are not on the manifold of the given class . KL/TV detection does not construct manifold-aware acceptance regions , which is as expected since the proposed divergence detection method does not require the classifier to be generative . However , both detection methods have some success in rejecting uncertain predictions , especially for TV , which also rejects ambiguous inputs ( see the ring-cross regions in the last two plots ) . Combining all three methods , we see that the rejected inputs are either far away from the manifold , or are ambiguous . Again we emphasise that a suitable generative model is required to make the detection methods work in practice , since the data distribution pD ( x , y ) is approximated by p ( x , y ) . Detection methods using logit values have also been proposed in Li & Gal ( e.g . 2017 ) ; Feinman et al . ( e.g . 2017 ) . However it is unclear whether the logits values in discriminative classifiers have a clear semantic meaning , while the logit values in deep Bayes represent the log probability of generating the input x given the class label y = yc . Our approach is also distinct from previous “ denoising ” approaches ( e.g . Song et al. , 2018 ; Samangouei et al. , 2018 ; Kurakin et al. , 2018 ) that require training an additional generative model separately . The deep Bayes classifiers share the same generative model as the detection methods , meaning that detected adversarial examples are indeed far away from the classifier ’ s manifold ( which is also an approximation to the data manifold ) . Therefore the claim “ detection neural networks can be bypassed ” ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) does not directly transfer to our approach , and we can use the marginal and logit detection methods to directly verify the “ off-manifold ” adversarial example conjecture .

Summary: This paper aims to test the robustness of generative classifiers [1] w.r.t. adversarial examples, considering their use as a potentially more robust alternative to adversarial training of discriminative classifiers. To achieve this, *Deep Bayes*, a generalization of the Naive Bayes classifier using a latent variable model and trained in a fashion similar to variational autoencoders [2] is introduced, and 7 different latent variable models are compared, covering a spectrum of generative or discriminative classification models, with or without bottlenecks. Their DFX and DBX architectures in particular closely match traditional discriminative classifiers, without and with a latent bottleneck.