Are Generative Classifiers More Robust to Adversarial Attacks?

1 INTRODUCTION . Deep neural networks have been shown to be vulnerable to adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ) . The latest attack techniques can easily fool a deep neural network with imperceptible perturbations ( Goodfellow et al. , 2014 ; Papernot et al. , 2016b ; Carlini & Wagner , 2017a ; Kurakin et al. , 2016 ; Madry et al. , 2018 ; Chen et al. , 2017a ) , even in the black-box case , where the attacker does not have access to the network ’ s weights ( Papernot et al. , 2017b ; Chen et al. , 2017b ; Alzantot et al. , 2018a ) . Adversarial attacks are serious security threats to machine learning systems , threatening applications beyond image classification ( Carlini & Wagner , 2018 ; Alzantot et al. , 2018b ) . To address this outstanding security issue , researchers have proposed defence mechanisms against adversarial attacks . Adversarial training , which augments the training data with adversarially perturbed inputs , has shown moderate success at defending against recently proposed attack techniques ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Tramèr et al. , 2017 ; Madry et al. , 2018 ) . In addition , recent advances in Bayesian neural networks have demonstrated that uncertainty estimates can be used to detect adversarial attacks ( Li & Gal , 2017 ; Feinman et al. , 2017 ; Louizos & Welling , 2017 ; Smith & Gal , 2018 ) . Another notable category of defence techniques involves the usage of generative models . For example , Gu & Rigazio ( 2014 ) used an auto-encoder to denoise the inputs before feeding them to the classifier . This denoising approach has been extensively investigated , and the “ denoisers ” in usage include generative adversarial networks ( Samangouei et al. , 2018 ) , PixelCNNs ( Song et al. , 2018 ) and denoising auto-encoders ( Kurakin et al. , 2018 ) . These developments rely on the “ off-manifold ” conjecture , that is , that adversarial examples are far away from the data manifold , although Gilmer et al . ( 2018 ) has challenged this observation with a synthetic example . Surprisingly , much less recent work has investigated the robustness of generative classifiers ( Ng & Jordan , 2002 ) against adversarial attacks for multi-class classification , where such classifiers explicitly model the conditional distribution of the inputs given labels . Typically , a generative classifier produces predictions by comparing between the likelihood of the labels for a given input , which is closely related to the “ distance ” of the input to the data manifold associated with a class . Therefore , generative classifiers should be robust to many recently proposed adversarial attacks if the “ off-manifold ” conjecture holds for many real-world applications . Unfortunately , many generative classifiers in popular use , including naive Bayes and linear discriminant analysis ( Fisher , 1936 ) , perform poorly on natural image classification tasks , making it difficult to verify the “ off-manifold ” conjecture and the robustness of generative classifiers with these tools . In recent work , k-nearest neighbors ( Cover & Hart , 1967 ) , a method which shares many similarities with generative classi- 1 Under review as a conference paper at ICLR 2019 fiers , has been significantly improved in handling natural images by leveraging deep feature representations ( Papernot & McDaniel , 2018 ) . To the best of our knowledge , an approach which targets a similar contribution has not yet been proposed for generative classifiers . Are generative classifiers more robust to recently proposed adversarial attack techniques ? To answer this , we improve the naive Bayes algorithm by using conditional deep generative models , and evaluate the conjecture on the proposed generative classifier . In summary , our contributions include : • We propose deep Bayes as an extension of naive Bayes , in which the conditional distribution of an input , given a label , is parameterised by a deep latent variable model ( LVM ) . We learn the LVM with the variational auto-encoder algorithm ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ) , and approximate Bayes ’ rule using importance sampling . • We propose three detection methods for adversarial perturbations . The first two use the learned generative model as a proxy of the data manifold , and reject inputs that are far away from it . The third computes statistics for the classifier ’ s output probability vector on training data , and rejects inputs that lead to under-confident predictions . • We evaluate the robustness of the proposed generative classifier on the MNIST multi-class and CIFAR binary classification tasks . We also improve the robustness of deep neural networks on CIFAR-10 multi-class classification , by fusing discriminatively learned visual feature representations with the proposed generative classifiers . We further compare the generative classifiers with a number of discriminative classifiers , including Bayesian neural networks and discriminative latent variable models . 2 DEEP BAYES : CONDITIONAL DEEP LVM AS A GENERATIVE CLASSIFIER . Denote pD ( x , y ) the data distribution for the input x ∈ RD and label y ∈ { yc|c = 1 , ... , C } , where yc denotes the one-hot encoding vector for class c. For a given x ∈ RD we can define the ground-truth label by y ∼ pD ( y|x ) if x ∈ supp ( pD ( x ) ) . ( 1 ) We assume the data distribution pD ( x , y ) follows the manifold assumption : for every class c , the conditional distribution pD ( x|yc ) has a low-dimensional manifold supportMc = supp ( pD ( x|yc ) ) . Therefore the training dataset D = { ( x ( n ) , y ( n ) ) } Nn=1 is generated as the following : ( x ( n ) , y ( n ) ) ∼ pD ( x , y ) ⇔ y ( n ) ∼ pD ( y ) , x ( n ) ∼ pD ( x|y ) . A ( Bayesian ) generative classifier first builds a generative model p ( x , y ) = p ( x|y ) p ( y ) , and then , in prediction time , predicts the label y∗ of a test input x∗ using Bayes ’ rule , p ( y∗|x∗ ) = p ( x ∗|y∗ ) p ( y∗ ) p ( x∗ ) = softmaxCc=1 [ log p ( x ∗ , yc ) ] , where softmaxCc=1 denotes the softmax operator over the c axis . Therefore , the output probability vector is computed analogously to many deep discriminative classifiers which use softmax activation in the output layer , so many existing attacks can be tested directly . However , unlike discriminative classifiers , the “ logit ” values prior to softmax activation have a clear meaning here , which is the ( approximated ) log joint distribution log p ( x∗ , yc ) of input x∗ conditioned on a given label yc . Therefore , one can also analyse the logit values to determine whether the unseen pair ( x∗ , y∗ ) is legitimate , a utility which will be discussed further in later sections . Naive Bayes is perhaps the most well-known generative classifier ; it assumes a factorised distribution for the conditional generator , i.e . p ( x|y ) = ∏D d=1 p ( xd|y ) . However , this factorisation assumption is inappropriate for e.g . image and speech data . Fortunately , we can leverage the recent advances in generative modelling and apply a deep generative model for the joint distribution p ( x , y ) . We refer to such generative classifiers that use deep generative models as deep Bayes classifiers . In this paper , we consider a deep latent variable model ( LVM ) p ( x , z , y ) , which will be used for classification . In this case , the conditional distribution is p ( x|y ) = ∫ p ( x , z , y ) dz∫ p ( x , z , y ) dzdx . Importantly , this leads to a conditional distribution p ( x|y ) that is not factorised ( even when p ( x|z , y ) is ) , which is much more powerful than naive Bayes . Since maximum likelihood is intractable , we follow 2 Under review as a conference paper at ICLR 2019 Kingma & Welling ( 2013 ) and Rezende et al . ( 2014 ) to introduce an amortised approximate posterior q ( z|x , y ) , and train both p and q by maximising the variational lower-bound : ED [ LVI ( x , y ) ] = 1 N N∑ n=1 Eq [ log p ( xn , zn , yn ) q ( zn|xn , yn ) ] . ( 2 ) After training , the predicted class probability vector y∗ for a future input x∗ is computed by an approximation to Bayes ’ rule with importance sampling : p ( y∗|x∗ ) ≈ softmaxCc=1 [ log 1 K K∑ k=1 p ( x∗ , zkc , yc ) q ( zkc |x∗ , yc ) ] , zkc ∼ q ( z|x∗ , yc ) . ( 3 ) We evaluate the effect of the following factorisation structures on the robustness of the induced classifier from the generative model p ( x , z , y ) ( see Figure 1 ) . p ( x , z , y ) = p ( z ) p ( y|z ) p ( x|z , y ) ( GFZ ) p ( x , z , y ) = pD ( y ) p ( z|y ) p ( x|z , y ) ( GFY ) p ( x , z , y ) = p ( z ) p ( y|z ) p ( x|z ) ( GBZ ) p ( x , z , y ) = pD ( y ) p ( z|y ) p ( x|z ) ( GBY ) p ( x , z , y ) = pD ( x ) p ( z|x ) p ( y|z , x ) ( DFX ) p ( x , z , y ) = p ( z ) p ( x|z ) p ( y|z , x ) ( DFZ ) p ( x , z , y ) = pD ( x ) p ( z|x ) p ( y|z ) ( DBX ) We use the initial character “ G ” to denote generative classifiers and “ D ” to denote discriminative classifiers . Models with the second character as “ F ” have a fully connected graph , while “ B ” models enforce the usage of the latent code z as a bottleneck . The last character of the model name indicates the first node in topological order . Model DFZ is somewhat intermediate , as it builds a generative model for the inputs x but also directly parameterises the conditional distribution p ( y|x , z ) . We do not test other architectures under this nomenclature , as either the graph contains directed cycles ( e.g . x→ y → z → x ) , or z is the last node in topological order ( e.g . x→ y , ( x , y ) → z ) so that the marginalisation of z does not affect classification . 3 DETECTING ADVERSARIAL ATTACKS WITH GENERATIVE CLASSIFIERS . We propose detection methods for adversarial examples using generative classifier ’ s logit values . As an illustrating example , consider a labelled dataset of “ cat ” and “ dog ” images . If an adversarial image of a cat xadv is incorrectly labelled as “ dog ” , then either this image is ambiguous , or , under a perfect generative model , the logit log p ( xadv , “ dog ” ) will be significantly lower than normal . This means we can detect attacks using the logits log p ( x∗ , yc ) , c = 1 , ... , C computed on a test input x∗ , by comparing them with the logits computed on legitimate training inputs . Concretely , the proposed detection algorithms are as follows . We aim to reject both unlabelled input x that have low probability under pD ( x ) , and labelled data ( x , y ) that have low pD ( x , y ) values . • Marginal detection : rejecting inputs that are far away from the manifold . One can select a threshold δ and reject an input x if − log p ( x ) > δ . To determine the threshold δ , we can compute d̄D = Ex∼D [ − log p ( x ) ] and σD = √ Vx∼D [ log p ( x ) ] , then set δ = d̄D + ασD . It is also possible to compute the statistics d̄p , σp on the images generated by the generative model accordingly . • Logit detection : rejecting inputs using joint density . Given a victim model y = F ( x ) , one can reject x if − log p ( x , F ( x ) ) > δy . We can use the mean and variance statistics d̄c , σc computed on log p ( x , yc ) and select δyc = d̄c+ασc . 3 Under review as a conference paper at ICLR 2019 • Divergence detection : rejecting inputs with over- and/or under-confident predictions . Denote p ( x ) as a C-dimensional probability vector outputted by the classifier . For each class c , we first collect the mean classification probability vector pc = E ( x , yc ) ∈D [ p ( x ) ] , then compute the mean d̄c and standard deviation σc on a selected divergence/distance measure D [ pc||p ( x ) ] for all ( x , yc ) ∈ D. A test input x∗ with prediction label c∗ = arg maxp ( x∗ ) is rejected if D [ pc∗ ||p ( x∗ ) ] > d̄c∗ + ασc∗ . Therefore , an example x∗ will be rejected if the classifier is over-confident or under-confident ( ambiguous inputs ) . When D is selected as the KL-divergence , we call this detection method KL detection . Other divergence/distance measures such as total variation ( TV ) can also be used . For better intuition , we visualise the detection mechanisms in Figure 2 with a synthetic “ two rings ” binary classification example . In this case we sample 2× 1000 training data points as : ( x , y ) ∼ pD ⇔ y ∼ Bern ( 0.5 ) , θ ∼ Uniform ( 0 , 2π ) , x|y ∼ N ( x ; cy + ry [ cos ( θ ) , sin ( θ ) ] T , σ2I ) . We consider a generative classifier p ( x , y ) = p ( x|y ) pD ( y ) = N ( x ; µy , σ2I ) Bern ( 0.5 ) with µy = proj ( x , ringy ) = arg minx̂∈R2 , ||x̂−cy||2=ry ||x − x̂||2 . The δ thresholds are selected to achieve 10 % false positive rates on training data . From the visualisations we see that inputs that are far away from the model manifold are rejected by marginal/logit detection . At the same time , logit detection rejects data points that are not on the manifold of the given class . KL/TV detection does not construct manifold-aware acceptance regions , which is as expected since the proposed divergence detection method does not require the classifier to be generative . However , both detection methods have some success in rejecting uncertain predictions , especially for TV , which also rejects ambiguous inputs ( see the ring-cross regions in the last two plots ) . Combining all three methods , we see that the rejected inputs are either far away from the manifold , or are ambiguous . Again we emphasise that a suitable generative model is required to make the detection methods work in practice , since the data distribution pD ( x , y ) is approximated by p ( x , y ) . Detection methods using logit values have also been proposed in Li & Gal ( e.g . 2017 ) ; Feinman et al . ( e.g . 2017 ) . However it is unclear whether the logits values in discriminative classifiers have a clear semantic meaning , while the logit values in deep Bayes represent the log probability of generating the input x given the class label y = yc . Our approach is also distinct from previous “ denoising ” approaches ( e.g . Song et al. , 2018 ; Samangouei et al. , 2018 ; Kurakin et al. , 2018 ) that require training an additional generative model separately . The deep Bayes classifiers share the same generative model as the detection methods , meaning that detected adversarial examples are indeed far away from the classifier ’ s manifold ( which is also an approximation to the data manifold ) . Therefore the claim “ detection neural networks can be bypassed ” ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) does not directly transfer to our approach , and we can use the marginal and logit detection methods to directly verify the “ off-manifold ” adversarial example conjecture .

Summary: This work investigates an interesting direction of improving robustness of classifiers against adversarial attacks by using generative models. The authors propose the *deep Bayes classifier*, which is a deep LVM based extension of naive Bayes. Furthermore, the authors extensively explore 7 possible factorisations of the classifier. Thorough experiments are conducted to assess the capability of defending or detecting adversarial examples.  Besides, the authors incorporate discriminative features to generative classifiers and demonstrate clear robustness gain.