A RE G ENERATIVE C LASSIFIERS M ORE R OBUST TO A DVERSARIAL A TTACKS ?
Anonymous authors
Paper under double-blind review
A BSTRACT
There is a rising interest in studying the robustness of deep neural network classiﬁers against adversaries, with both advanced attack and defence techniques being actively developed.
However, most recent work focuses on discriminative classiﬁers, which only model the conditional distribution of the labels given the inputs.
In this paper, we propose and investigate the deep Bayes classiﬁer, which improves classical naive Bayes with conditional deep generative models.
We further develop detection methods for adversarial examples, which reject inputs with low likelihood under the generative model.
Experimental results suggest that deep
Bayes classiﬁers are more robust than deep discriminative classiﬁers, and that the proposed detection methods are effective against many recently proposed attacks.
I NTRODUCTION
Deep neural networks have been shown to be vulnerable to adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2014).
The latest attack techniques can easily fool a deep neural network with imperceptible perturbations (Goodfellow et al., 2014; Papernot et al., 2016b; Carlini & Wagner, 2017a; Kurakin et al., 2016;
Madry et al., 2018; Chen et al., 2017a), even in the black-box case, where the attacker does not have access to the network’s weights (Papernot et al., 2017b; Chen et al., 2017b; Alzantot et al., 2018a).
Adversarial attacks are serious security threats to machine learning systems, threatening applications beyond image classiﬁcation (Carlini & Wagner, 2018;
Alzantot et al., 2018b).
To address this outstanding security issue, researchers have proposed defence mechanisms against adversarial attacks.
Adversarial training, which augments the training data with adversarially perturbed inputs, has shown moderate success at defending against recently proposed attack techniques (Szegedy et al., 2013; Goodfellow et al., 2014; Tram ` er et al., 2017; Madry et al., 2018).
In addition, recent advances in Bayesian neural networks have demonstrated that uncertainty estimates can be used to detect adversarial attacks (Li & Gal, 2017; Feinman et al., 2017; Louizos & Welling, 2017;
Smith & Gal, 2018).
Another notable category of defence techniques involves the usage of generative models.
For example, Gu & Rigazio (2014) used an auto-encoder to denoise the inputs before feeding them to the classiﬁer.
This denoising approach has been extensively investigated, and the “denoisers” in usage include generative adversarial networks (Samangouei et al., 2018), PixelCNNs (Song et al., 2018) and denoising auto-encoders (Kurakin et al., 2018).
These developments rely on the “off-manifold” conjecture, that is, that adversarial examples are far away from the data manifold, although Gilmer et al.
(2018) has challenged this observation with a synthetic example.
Surprisingly, much less recent work has investigated the robustness of generative classiﬁers (Ng &
Jordan, 2002) against adversarial attacks for multi-class classiﬁcation, where such classiﬁers explicitly model the conditional distribution of the inputs given labels.
Typically, a generative classiﬁer produces predictions by comparing between the likelihood of the labels for a given input, which is closely related to the “distance” of the input to the data manifold associated with a class.
Therefore, generative classiﬁers should be robust to many recently proposed adversarial attacks if the “off-manifold” conjecture holds for many real-world applications.
Unfortunately, many generative classiﬁers in popular use, including naive Bayes and linear discriminant analysis (Fisher, 1936), perform poorly on natural image classiﬁcation tasks, making it difﬁcult to verify the “off-manifold” conjecture and the robustness of generative classiﬁers with these tools.
In recent work, k-nearest neighbors (Cover & Hart, 1967), a method which shares many similarities with generative classi- ﬁers, has been signiﬁcantly improved in handling natural images by leveraging deep feature representations (Papernot & McDaniel, 2018).
To the best of our knowledge, an approach which targets a similar contribution has not yet been proposed for generative classiﬁers.
Are generative classiﬁers more robust to recently proposed adversarial attack techniques?
To answer this, we improve the naive Bayes algorithm by using conditional deep generative models, and evaluate the conjecture on the proposed generative classiﬁer.
In summary, our contributions include:
• We propose deep Bayes as an extension of naive Bayes, in which the conditional distribution of an input, given a label, is parameterised by a deep latent variable model (LVM).
We learn the LVM with the variational auto-encoder algorithm (Kingma & Welling, 2013; Rezende et al., 2014), and approximate Bayes’ rule using importance sampling.
• We propose three detection methods for adversarial perturbations.
The ﬁrst two use the learned generative model as a proxy of the data manifold, and reject inputs that are far away from it.
The third computes statistics for the classiﬁer’s output probability vector on training data, and rejects inputs that lead to under-conﬁdent predictions.
• We evaluate the robustness of the proposed generative classiﬁer on the MNIST multi-class and CIFAR binary classiﬁcation tasks.
We also improve the robustness of deep neural networks on CIFAR-10 multi-class classiﬁcation, by fusing discriminatively learned visual feature representations with the proposed generative classiﬁers
We further
compare the generative classiﬁers with a number of discriminative classiﬁers, including Bayesian neural networks and discriminative latent variable models.
D EEP B AYES : CONDITIONAL DEEP LVM AS A GENERATIVE CLASSIFIER
Denote p D ( x , y ) the data distribution for the input x ∈ R D and label y ∈ { y c | c = 1 , ..., C } , where y c denotes the one-hot encoding vector for class c .
For a given x ∈ R D we can deﬁne the ground-truth label by
[Equation]
We assume the data distribution p D ( x , y ) follows the manifold assumption : for every class c , the conditional distribution p D ( x | y c ) has a low-dimensional manifold support M c
= supp ( p D ( x | y c )) .
Therefore the training dataset D = { ( x ( n ) , y ( n ) ) } Nn =1 is generated as the following:
[Equation]
A (Bayesian) generative classiﬁer ﬁrst builds a generative model p ( x , y )
= p ( x | y ) p ( y ) , and then, in prediction time, predicts the label y ∗ of a test input x ∗ using Bayes’ rule,
[Equation]
where softmax Cc =1 denotes the softmax operator over the c axis.
Therefore, the output probability vector is computed analogously to many deep discriminative classiﬁers which use softmax activation in the output layer, so many existing attacks can be tested directly.
However, unlike discriminative classiﬁers, the “logit” values prior to softmax activation have a clear meaning here, which is the (approximated) log joint distribution log p ( x ∗ , y c ) of input x ∗ conditioned on a given label y c .
Therefore, one can also analyse the logit values to determine whether the unseen pair ( x ∗ , y ∗ ) is legitimate, a utility which will be discussed further in later sections.
Naive Bayes is perhaps the most well-known generative classiﬁer; it assumes a factorised distribution for the conditional generator, i.e. p ( x | y ) =
(cid:81)
Dd =1 p ( x d | y ) .
However, this factorisation assumption is inappropriate for e.g. image and speech data.
Fortunately, we can leverage the recent advances in generative modelling and apply a deep generative model for the joint distribution p ( x , y ) .
We refer to such generative classiﬁers that use deep generative models as deep Bayes classiﬁers.
In this paper, we consider a deep latent variable model (LVM) p ( x , z , y ) , which will be used for classiﬁcation.
In this case, the conditional distribution is p ( x | y ) =
[Equation]
[Equation]
Importantly, this leads to a conditional distribution p ( x | y ) that is not factorised (even when p ( x | z , y ) is), which is much more powerful than naive Bayes.
Since maximum likelihood is intractable, we follow
[Figure]
Kingma & Welling (2013) and Rezende et al.
(2014) to introduce an amortised approximate posterior q ( z | x , y ) , and train both p and q by maximising the variational lower-bound:
[Equation]
After training, the predicted class probability vector y ∗ for a future input x ∗ is computed by an approximation to Bayes’ rule with importance sampling:
[Equation]
We evaluate the effect of the following factorisation structures on the robustness of the induced classiﬁer from the generative model p ( x , z , y )
(see Figure 1).
[Equation]
z , y )
=
p D ( x ) p ( z | x ) p ( y | z , x ) (DFX) p ( x , z , y ) =
p ( z ) p ( x | z ) p ( y | z , x ) (DFZ) p ( x , z , y ) =
p D ( x ) p ( z | x ) p ( y | z ) (DBX)
We use the initial character “G” to denote generative classiﬁers and “D” to denote discriminative classiﬁers.
Models with the second character as “F” have a fully connected graph, while “B” models enforce the usage of the latent code z as a bottleneck .
The last character of the model name indicates the ﬁrst node in topological order.
Model DFZ is somewhat intermediate, as it builds a generative model for the inputs x but also directly parameterises the conditional distribution p ( y
| x , z ) .
We do not test other architectures under this nomenclature, as either the graph contains directed cycles (e.g. x → y → z → x ), or z is the last node in topological order (e.g. x → y , ( x , y )
→ z ) so that the marginalisation of z does not affect classiﬁcation.
D ETECTING ADVERSARIAL ATTACKS WITH GENERATIVE CLASSIFIERS
We propose detection methods for adversarial examples using generative classiﬁer’s logit values.
As an illustrating example, consider a labelled dataset of “cat” and “dog” images.
If an adversarial image of a cat x adv is incorrectly labelled as “dog”, then either this image is ambiguous, or, under a perfect generative model, the logit log p ( x adv , “dog” ) will be signiﬁcantly lower than normal.
This means we can detect attacks using the logits log p ( x ∗ , y c ) , c = 1 , ..., C computed on a test input x ∗ , by comparing them with the logits computed on legitimate training inputs.
Concretely, the proposed detection algorithms are as follows.
We aim to reject both unlabelled input x that have low probability under p D ( x ) , and labelled data (
x , y ) that have low p D ( x , y ) values.
• Marginal detection : rejecting inputs that are far away from the manifold.
One can select a threshold δ and reject an input x if − log p ( x ) > δ .
To determine the threshold δ , we can compute ¯ d D = E
x
∼D
− log p ( x )] and σ D =
(cid:112) V x ∼D [log
p x )] , then set δ = ¯ d D + ασ D .
It is also possible to compute the statistics ¯ d p , σ p on the images generated by the generative model accordingly.
• Logit detection : rejecting inputs using joint density.
Given a victim model y = F ( x ) , one can reject x if − log p ( x , F ( x )) > δ y .
We can use the mean and variance statistics ¯ d c , σ c computed on log p ( x , y c ) and select δ y c = ¯ d c + ασ c .
• Divergence detection : rejecting inputs with over- and/or under-conﬁdent predictions.
Denote p ( x ) as a C -dimensional probability vector outputted by the classiﬁer.
For each class c , we ﬁrst collect the mean classiﬁcation probability vector p c = E ( x , y c ) ∈D [ p ( x )] , then compute the mean ¯ d c and standard deviation σ c on a selected divergence/distance measure D[ p c || p ( x )] for all ( x , y c ) ∈ D .
A test input x ∗ with prediction label c ∗ = arg max p ( x ∗ ) is rejected if D[ p c ∗ || p ( x ∗ )]
> ¯ d c ∗ + ασ c ∗ .
Therefore, an example x ∗ will be rejected if the classiﬁer is over-conﬁdent or under-conﬁdent (ambiguous inputs).
When D is selected as the KL-divergence, we call this detection method KL detection .
Other divergence/distance measures such as total variation (TV) can also be used.
For better intuition, we visualise the detection mechanisms in Figure 2 with a synthetic “two rings” binary classiﬁcation example.
In this case we sample 2 × 1000 training data points as: ( x , y ) ∼ p D ⇔ y ∼ Bern (0 . 5) , θ ∼ Uniform (0 , 2 π ) , x | y ∼ N ( x ; c y + r y
[ cos ( θ ) , sin ( θ )] T , σ 2 I ) .
We consider a generative classiﬁer p ( x , y )
=
p ( x | y ) p D ( y ) =
N ( x ; µ y , σ 2 I ) Bern (0 . 5) with µ y =
proj
( x , ring y )
=
arg min ˆ x ∈ R 2 ,
||
ˆ x − c y || 2 =
r y || x − ˆ x || 2 .
The δ thresholds are selected to achieve 10% false positive rates on training data.
From the visualisations we see that inputs that are far away from the model manifold are rejected by marginal/logit detection.
At the same time, logit detection rejects data points that are not on the manifold of the given class.
KL/TV detection does not construct manifold-aware acceptance regions, which is as expected since the proposed divergence detection method does not require the classiﬁer to be generative.
However, both detection methods have some success in rejecting uncertain predictions, especially for TV, which also rejects ambiguous inputs (see the ring-cross regions in the last two plots).
Combining all three methods, we see that the rejected inputs are either far away from the manifold, or are ambiguous.
Again we emphasise that a suitable generative model is required to make the detection methods work in practice, since the data distribution p D ( x , y ) is approximated by p ( x , y ) .
Detection methods using logit values have also been proposed in Li & Gal (e.g. 2017); Feinman et al. (e.g. 2017).
However it is unclear whether the logits values in discriminative classiﬁers have a clear semantic meaning, while the logit values in deep Bayes represent the log probability of generating the input x given the class label y
= y c .
Our approach is also distinct from previous “denoising” approaches (e.g. Song et al., 2018; Samangouei et al., 2018; Kurakin et al., 2018) that require training an additional generative model separately.
The deep Bayes classiﬁers share the same generative model as the detection methods, meaning that detected adversarial examples are indeed far away from the classiﬁer’s manifold (which is also an approximation to the data manifold).
Therefore the claim “detection neural networks can be bypassed” (Carlini & Wagner, 2017a;
Athalye et al., 2018) does not directly transfer to our approach, and we can use the marginal and logit detection methods to directly verify the “off-manifold” adversarial example conjecture.
E XPERIMENTS
We carry out a number of tests on the proposed deep Bayes classiﬁers (3), where q ( z |· ) and p ( z |· ) are factorised Gaussians, and the conditional probability p ( x |· ) , if required, is parameterised by an (cid:96) 2 loss.
Besides the LVM-based classiﬁers, we further train a Bayesian neural network (BNN) with
Bernoulli dropout (dropout rate 0.3), as it has been shown in Li & Gal (2017) and Feinman et al.
(2017) that BNNs are more robust than their deterministic counterparts.
The constructed BNN has 2x more channels than LVM-based classiﬁers, making the comparison slightly “unfair”, as the BNN layers have more capacity.
We use K = 10 Monte Carlo samples for all the classiﬁers.
The adversarial attacks in test include both (cid:96) and (cid:96) 2 untargeted attacks from CleverHans 2.0 library (Papernot et al., 2017a): fast gradient sign method (FGSM, Goodfellow et al., 2014), projected gradient descent (PGD, Madry et al., 2018), momentum iterative attack (MIM, Dong et al., 2017) and Carlini & Wagner (cid:96) 2 (CW, Carlini & Wagner, 2017a).
Two metrics are reported: accuracy of the classiﬁer on crafted adversarial examples, and detection rate on adversarial examples that have successfully caused the classiﬁer to misclassify.
This detection rate is deﬁned as the true positive (TP) rate of ﬁnding an adversarial example, and the detection threshold is selected to achieve a 5% false positive rate on clean training data.
The experiments are performed under various threat model settings.
We further evaluate the transferability of crafted adversarial examples across different classiﬁers in the same way as done in
Papernot et al.
(2016a).
We only provide visualisations in the main text; full table results can be found in the appendix.
Readers are also referred to the appendix for further experiments, including a quantitative analysis of the bottleneck effect on robustness and detection.
4. MNIST
The ﬁrst set of experiments evaluate the robustness of generative classiﬁers on MNIST.
Here the image pixel values are normalised to [0 , 1] , and the LVM-based classiﬁers have dim ( z ) = 64 .
We ﬁrst perform white-box zero-knowledge attacks, i.e. the attacker can differentiate through the classiﬁer to craft adversarial examples, but he/she is not aware of the existence of the detector.
Then we perform white-box perfect-knowledge attacks, where the attacker can differentiate through both the classiﬁer and the detector, and he/she knows the usage of random z samples by the VAE-based classiﬁers (Biggio et al., 2013; Carlini & Wagner, 2017b).
Lastly we consider grey-box and blackbox attacks, and evaluate the robustness of generative classiﬁers against transferred attacks.
White-box attacks (zero-knowledge, (cid:96) ∞ ) Results for (cid:96) ∞
attacks are reported in Figure 3, and in general generative classiﬁers perform better in terms of victim accuracy and minimum perturbation of the attacks.
1 By contrast, DFX & DFZ are not robust to the weakest attack (FGSM) even when (cid:15) = 0 .
2 (where the adversarial examples are still visually close to the original digit “7”).
Interestingly, DBX is the most robust against FGSM & MIM 2 , which agrees with the preliminary tests in Alemi et al.
But DBX is less robust to PGD, and here GBZ is a clear winner.
These results show that the bottleneck is sometimes beneﬁcial for better robustness of MNIST classiﬁers.
For detection, generative classiﬁers have successfully detected the adversarial examples with (cid:15) ≥ 0 .
3 , which is reasonable as the visual distortion is already signiﬁcant.
Importantly, generative classiﬁer’s victim accuracy decreases as the (cid:96) ∞ distortion (cid:15) increases, but at the same time the TP rates attack results.
Here, accuracy (adv) measures classifying the adversarial inputs to the original classes.
[Table]
for marginal and logit detection also increase.
Therefore these (cid:96) ∞ attacks fail to ﬁnd near-manifold adversarial examples that fool both the classiﬁer and the detection methods.
DFZ, as an intermediate between generative and discriminative classiﬁers, has worse robustness results, but has good detection performance for the marginal and logit metrics.
This is because with softmax activation, the marginal distribution p ( x ) is dropped, but in marginal/logit detection p ( x ) is still in use.
White-box attack (zero-knowledge, (cid:96) 2 ) For the CW (cid:96) 2 attack, we performed a hyperparameter search for the loss-balancing parameter c
( (cid:96) 2 distortion increases with larger c ) in { 0 . 1 , 1 , 10 , 100 , 1000 } on the ﬁrst 100 test images, and we found c = 10 returns the highest success rate.
Results are reported in Table 1.
Although being successful on fooling many classiﬁers, CW failed on attacking GBZ.
Also, the mean (cid:96) 2 distortions of the successful attacks on generative classiﬁers are signiﬁcantly larger.
Furthermore, we found the success of the attack on other generative classiﬁers is mainly due to the ambiguity of the crafted adversarial images.
As visualised in Figure(also see Figure D.1 in appendix), the induced distortion from CW leads to ambiguous digits which sit at the perceptual boundary between the original and the adversarial classes.
With the KL detection method, all classiﬁers achieve > 95% detection rates, which is as expected as the default CW attack conﬁguration, by construction, generates adversarial examples that lead to minimal difference between the logit values of the most and the second most probable classes.
White-box attack (perfect-knowledge, (cid:96) ∞ )
The PGD-based perfect-knowledge (cid:96) ∞ attack is designed following (Carlini & Wagner, 2017b): we construct an (approximate) Bayes classiﬁer p k ( y | x ) using (3) for each set of samples { z kc } Cc =1 , and minimize the following with PGD:
[Equation]
The detection statistic Φ( x + η , y ) is − log p ( x + η ) for marginal detection, and δ is the corresponding threshold computed on training data.
For logit/KL detection, the detection statistics and thresholds are constructed accordingly.
We label the three attacks against the marginal, logit and KL based detection schemes ’-PKM’, ’-PKL’ and ’-PKK’ respectively.
When we only have knowledge of the K samples and not the detection method we call this attack ’-PK0’ (ie λ detect =0 in Eq. 4).
Results are visualised in Figure 5.
We see that although the attacker can reduce detection levels, this comes with the trade-off of increasing accuracy, suggesting that a perfect-knowledge adversary cannot break both the classiﬁer and detector working in tandem.
Sanity checks on gradient masking Athalye et al.
(2018) claimed that if a successful defence against white-box attacks is due to gradient masking, then this defence is likely to be less effective against grey-box/black-box attacks, as they do not differentiate through the victim classiﬁer and the defence mechanism (Papernot et al., 2017b).
Therefore, we consider two transfer attacks based on distilling the victim classiﬁer using a “student” CNN which has no gradient masking. The two attacks differ in their threat models: in the grey-box setting the attacker has access to the output probability vectors of the classiﬁers on the training data, while in the black-box setting the attacker has access to queried labels only.
For the latter black-box setting, we follow Papernot et al.
(2017b) to train a substitute CNN using Jacobian-based dataset augmentation,
and we refer to appendix B. for the detailed algorithm.
The grey-box substitutes achieve > 99% agreement with the victims on test data, and the black-box substitutes obtained ∼ 96% accuracy on test data.
[Figure]
Figure 6 shows the accuracy and detection metrics on transferred (cid:96) ∞ attacks crafted on the substitute models, with a comparison to their white-box counterparts. Note that these crafted attacks achieve ∼ 100% success rates on fooling the substitute models when (cid:15) ≥ 0 . 2 (see appendix E).
However, they do not transfer very well to the generative classiﬁers. Importantly, for a ﬁxed (cid:15) setting, the whitebox attacks achieve signiﬁcantly higher success rates (i.e. lower victim accuracies) than their grey- /black-box counterparts, and the gap is at least > 20% for (cid:15) ≤ 0 .
3 (see Table 2). We further present the mean minimum (cid:96) ∞ perturbation in Table 2, and we see that the minimum perturbation obtained by grey-/black-box attacks are signiﬁcantly higher than those obtained by white-box attacks.
All these results suggest that the robustness of generative classiﬁers is unlikely to be caused by gradient masking.
Again on the detection side, the detection rates increase as the perturbation size increase, and they are near 100% for (cid:15)
= 0 .
This means that most of the successfully transferred adversarial images are off the generative classiﬁer’s manifold (as a proxy to the data manifold).
SPSA (evolutionary strategies)
We consider another black-box setting that only assumes access to the logit values of the prediction given an input.
We use the SPSA (cid:96) ∞ attack (Uesato et al., 2018), which is similar to the white-box zero-knowledge attacks, except that gradients are numerically
[Table]
[Table]
estimated using the logit values from the victim classiﬁer.
Results in Table 3 clearly show that SPSA performs much worse on generative classiﬁers when compared to white-box PGD.
Again this means gradient masking is unlikely to be responsible for the improved robustness of generative classiﬁers, as utilising the exact gradient yielded improved results.
Cross-model attack transferability Finally, we report in Figure 7 the transferability results of the crafted adversarial examples between different models (Papernot et al., 2016a).
Here we take adversarial examples crafted in the white-box setting with PGD and MIM ( (cid:15) = 0 . 3 ), and transfer successful attacks to other classiﬁers.
The transfer is effective between generative classiﬁers but not from generative to discriminative (and vice versa).
The attacks crafted on DBX do not transfer in general, while at the same time, DBX is the least robust model in this case.
Furthermore, the generative classiﬁers obtain very high detection rates on all transferred attacks ( > 95% ).
In summary, generative classiﬁers are more robust against the tested transfer attacks across different models.
4. CIFAR PLANE - VS - FROG BINARY CLASSIFICATION
We consider the same set of evaluations on CIFAR-10, in order to validate the robustness of generative classiﬁers on natural images (c.f. Carlini & Wagner, 2017b).
Unfortunately, we failed to train fully generative classiﬁers with comparable test accuracies to discriminative CNNs (typically > 80% ): the clean accuracies for GFZ & GFY are all < 50% .
Even when using the conditional
PixelCNN++ (Salimans et al., 2017) (which uses much deeper networks), the clean accuracy on the test data is 72 .
Instead, we consider a simpler binary classiﬁcation problem and construct a dataset containing CIFAR images from the “airplane” and “frog” categories.
The images in this dataset are scaled to [0, 1].
On this dataset, the generative classiﬁers use dim ( z ) = 128 and obtain > 90% clean test accuracy (see appendix).
The attacks are performed on the test images that all models initially correctly classify, leading to a test set of 1577 instances.
Due to the page limit, we only present white-box zero-knowledge attacks here, and discuss further attacks in the appendix.
White-box attacks (zero-knowledge, (cid:96) ∞ )
We present the white-box (cid:96) ∞ attack results in Figure 8, where the distortion strengths are selected as (cid:15) ∈ { 0 . 01 , 0 . 02 , 0 . 05 , 0 . 1 , 0 . 2 } .
Again, generative classiﬁers are more robust than the discriminative ones, and GBZ is the most robust, much better than the others when (cid:15) ≥ 0 . 05 .
BNN is signiﬁcantly better than other discriminative VAE-based classiﬁers, presumably due to higher randomness.
Detection results are less satisfactory: marginal/logit detection fail to detect attacks with (cid:15)
= 0 .
1 (which attain both high success rate and induce visually perceptible distortion).
KL detection performs better, and interestingly, discriminative classiﬁers dominate in this metric.
These results suggest that the (cid:96) 2 likelihood might not be best suited for modelling natural images (c.f.
Larsen et al., 2016; van den Oord et al., 2016).
Still, the minimum distortion required to fool generative classiﬁers are much higher than that for discriminative ones, indeed the visual distortion of the adversarial examples on generative classiﬁers are more signiﬁcant.
White-box attack (zero-knowledge, (cid:96) 2 )
As the test set is relatively small, we directly perform
CW attacks on the test data with c ∈ { 0 . 1 , 1 , 10 , 100 , 1000 } .
Results are visualised in Figure 9.
The generative classiﬁers are signiﬁcantly more robust than the others (with the best being GBZ), and the mean (cid:96) 2 distortions computed on successful attacks are also signiﬁcantly higher.
The TP rates are low for marginal and logit detection, which is reasonable as the crafted images are visually similar to the clean ones.
Note that the distortion for the attacks on generative classiﬁers is perceptible.
These results indicate that this CW attack is ineffective when attacking generative classiﬁers.
4. F ULL CIFAR-10: COMBINING DEEP B AYES AND DISCRIMINATIVE FEATURES
The ﬁnal experiment examines the robustness of CIFAR-10 multi-class classiﬁers, with the generative classiﬁers trained on discriminative visual features.
To do this, we download a pretrained deep convolutional net 3 with VGG16-like architecture (Simonyan & Zisserman, 2014), and use its feature representation φ ( x ) as the input to the VAE-based classiﬁers: p ( y | x ) = p ( y | φ ( x )) .
The classiﬁers in test include GBZ, GBY and DBX.
We use fully-connected neural networks for these classiﬁers, and select from VGG16 the 9 th convolution layer ( CONV 9) and the ﬁrst fully connected layer after convolution ( FC 1) as the feature layers to ensure ∼ 90% test accuracy (see appendix).
Results on white-box zero-knowledge (cid:96) ∞ attacks are visualised in Figure 10.
Compared with the VGG16 baseline, we see clear improvements in robustness and detection for all VAE-based classiﬁers.
In particular, the generative classiﬁers GBZ and GBY are overall better than DBX.
More importantly, generative classiﬁers based on CONV 9 features are signiﬁcantly more robust than those based on FC 1 features.
In contrast, for DBX, which is discriminative , the robustness results are very similar, indicating that the level of feature representation has little effect.
These results suggest that one can achieve both high clean accuracy and better robustness against adversaries by combining discriminatively learned visual features and generative classiﬁers.
D ISCUSSION
We have proposed deep Bayes as a generative classiﬁer that uses deep latent variable models to model the joint distribution of input-output pairs.
We have given evidence, on multiple classiﬁcation tasks, that generative classiﬁers are more robust to adversarial attacks than discriminative classiﬁers.
Furthermore, the logit in generative classiﬁers has a well deﬁned meaning and can be used to detect attacks, even when the classiﬁer is fooled.
Our results corroborate with the Bayesian neural network literature, in particular (Li & Gal, 2017;
Feinman et al., 2017; Carlini & Wagner, 2017b), in showing that modelling unobserved variables are effective for defending against adversarial attacks 4 .
Concurrent to us, Schott et al.
(2018) also demonstrated the robustness of generative classiﬁers on MNIST, in which the logits are computed by a tempered version of the variational lower-bound.
However, their approach requires thousands of random z samples and tens of optimisation steps to approximate log p ( x | y ) for every input-output pair ( x , y ) , making it less scalable than our importance sampling technique to large datasets and big architectures.
Indeed, we have scaled our approach to CIFAR-10, a natural image dataset, and the robustness results are consistent with those on MNIST.
In addition, we have also shown that the structure of the graphical model has a signiﬁcant impact on robustness: deep LVM-based generative classiﬁers generally outperform the (randomised) discriminative ones.
While we have given strong evidence to suggest that generative classiﬁers are more robust to current adversarial attacks, we do not wish to claim that these models will be robust to all possible attacks.
Aside from many recent attacks being designed speciﬁcally for discriminative neural networks, there is also evidence for the fragility of generative models; e.g. naive Bayes as a standard approach for spam ﬁltering is well-known to be fragile (Dalvi et al., 2004; Huang et al., 2011), and very recently
Tabacof et al.
; Kos et al.
; Creswell et al.
(2017) also designed attacks for (unconditional) VAE-type models.
However, generative classiﬁers can be made more robust too, to counter these weaknesses.
Dalvi et al.
(2004) have shown that generative classiﬁers can be made more secure if aware of the attack strategy, and Biggio et al.
(2011; 2014) further improved naive Bayes’ robustness by modelling the conditional distribution of the adversarial inputs.
These approaches are similar to the adversarial training of discriminative classiﬁers, and efﬁcient ways for doing so with generative classiﬁers can be an interesting research direction.
But even with this note of caution, we believe this work offers exciting avenues for future work.
Using generative classiﬁers offers an interesting way to evaluate generative models and can drive improvements in their ability to tackle high-dimensional datasets, where traditionally generative classiﬁers have been less accurate than discriminative classiﬁers (Efron, 1975; Ng & Jordan, 2002).
In addition, the combination of generative and discriminative models investigated in this paper is a compelling direction for future research.
Overall, we believe that progress on generative classiﬁers can inspire better designs of attack, defence and detection techniques.
A M ODEL ARCHITECTURES
MNIST experiments The VAEs are constructed with convolutional encoders and deconvolutional generators.
More speciﬁcally, the encoder network for q ( z | x , y ) is the same across all VAE-based classiﬁers.
It starts with a 3-layer convolutional neural network with 5 × 5 ﬁlters and 64 channels, with a max-pooling operation after each convolution.
Then, the convolutional network is followed by a MLP with 2 hidden layers, each with 500 units, to produce the mean and variance parameters of q .
The label y is injected into the MLP at the ﬁrst hidden layer, as a one hot encoding (i.e. for
MNIST, the ﬁrst hidden layer has 500+10 units).
The latent dimension is dim ( z ) = 64 .
The p models’ architectures are the following
: GFZ:
For p ( y | z ) we use a MLP with 1 hidden layer composed of 500 units.
For p ( x | y , z ) we used an MLP with 2 hidden layers, each with 500 units, and 4 × 4 × 64 dimension output, followed by a 3-layer deconvolutional network with 5 × 5 kernel size, stride 2 and [64, 64, 1] channels.
GFY: We use an MLP with 1 hidden layer composed of 500 units for p ( z | y ) , and the same architecture as GFZ for p ( x | y , z ) .
DFZ: We use almost the same deconvolutional network architecture for p ( x | z ) as GFZ’s p ( x | y , z ) network, except that the input is z only.
For p ( y | x , z ) we use almost the same architecture as q ( z | x , y ) except that the injected input to the MLP is z and the MLP output is the set of logit values for y .
DFX:
We use the same architecture as G3 for p ( y | x , z ) .
The network for p ( z | x ) is almost identical except that there is no injected input to the MLP, and the network returns the mean and variance parameters for q ( z | x ) .
DBX:
We use GFZ’s architecture for p ( y | z ) and DFX’s architecture for p ( z | x ) .
GBY:
We use GFY’s architecture for p ( z | y ) and DFZ’s architecture for p ( x | z ) .
GBZ: We use GFZ’s architecture for p ( y | z ) and DFZ’s architecture for p ( x | z ) .
The BNN has almost the same architecture as the encoder network q , except that it uses 2x the hidden units/channels, and the last layer is 10 dimensions.
Note that here we used dropout as it is convenient to implement, and we expect better approximate inference methods (such as stochastic gradient MCMC) to return better results for robustness and detection.
CIFAR plane-vs-frog experiments The model architectures are almost the same as used in
MNIST experiments, except that the hidden layer dimensions for the MLP layers are increased to 1000.
For the encoder q , the channels are increased to [64, 128, 256].
For the p models, the deconvolutional networks have different channel values, [128, 64, 3], and the MLP before the deconvolution outputs a 4 × 4 × 256 vector (before reshaping).
The BNN has 2x the channels but still uses 1000 hidden units.
CIFAR-10 experiments The pre-trained VGG16 network is downloaded from https:// github.com/geifmany/cifar-vgg , where the CONV 9 and FC 1 layers correspond to:
CONV 9: https://github.com/geifmany/cifar-vgg/blob/master/ cifar10vgg.py#L
FC 1: https://github.com/geifmany/cifar-vgg/blob/master/ cifar10vgg.py#L
The VAE-based classiﬁers build fully connected networks on top of the extracted features, and use dim ( z ) = 128 for bottleneck.
The encoder q ( z | φ ( x ) , y ) has the network architectures [dim( φ ( x ) )
+ dim( y ), 1000, 1000, dim( z ) × 2], and we use the same encoder architecture across all classiﬁers.
The decoder architectures are as follows:
DBX:
We use an MLP of layers [dim( z ), 1000, dim( y )] for p ( y | z ) and an MLP of layers [dim( φ ( x ) ), 1000, 1000, dim( z ) × 2] for p ( z | φ ( x )) .
GBZ: We use an MLP of layers [dim( z ), 1000, 1000, dim( y )] for p ( y | z ) and an MLP of layers [dim( z ), 1000, 1000, dim( φ ( x ) )] for p ( φ ( x )
| z ) .
GBY:
We use an MLP of layers [dim( y ), 1000, dim( z ) × 2] for p ( z | y ) and GBZ’s architecture for p ( φ ( x ) | z ) .
B A TTACK SETTINGS
We use the Cleverhans package to perform attacks.
We use the default hyper-parameters, if not speciﬁcally stated.
PGD: We perform the attack for 40 iterations with step-size 0.01.
MIM: We perform the attack for 40 iterations with step-size 0.01 and decay factor 1.0.
CW- (cid:96) 2 : We use learning rate 0.01 and conﬁdence 0, and we optimise the loss for 1000 iterations.
SPSA:
We use almost the same hyper-parameters as in Uesato et al.
(2018) except for the number of samples for gradient estimates.
In detail, we perform the attack for 100 iterations with perturbation size 0.01, Adam learning rate 0.01, stopping threshold -5.0 and 2000 samples for each gradient estimate. B. J ACOBIAN - BASED DATASET AUGMENTATION The black-box distillation attack is based on Papernot et al. (2017b), which trains a substitute CNN using Jacobian-based dataset augmentation.
Assume y = F ( x ) is the output one-hot vector of the victim, and p ( x ) is the probability vector output of the substitute model, then at the t th outerloop, we train the substitute CNN on dataset D t = { ( x n , y n ) } with queried y n for 10 epochs, and augment the dataset by [Equation] We initialise D 1 with 200 × 10 datapoints from the MNIST test set, select λ = 0 .
1 , and run the algorithm for 6 outer-loops.
On MNIST, this results in 64 , 000 queried inputs, and ∼ 96% accuracy of the substitute model on test data.
On CIFAR binary classiﬁcation, we use 200 × 2 datapoints for the inital query set D 1 , resulting in 12 , 800 queries in total.
The substitutes achieved almost the same accuracy as their corresponding victim models on clean test datapoints.
C F URTHER EXPERIMENTS C. F URTHER EXPERIMENTS ON CIFAR BINARY CLASSIFICATION
White-box attack (perfect-knowledge, (cid:96) ∞ )
Figure C.1 shows the perfect knowledge attack on the CIFAR binary classiﬁcation task.
Again we see that although the attack is effective for the detection schemes, it comes with the price of decreased mis-classiﬁcation rates.
Interestingly GBY seems to be robust to this attack, where the accuracies on the crafted adversarial examples increase.
Sanity checks on gradient masking We conducted the same sets of transferred (cid:96) ∞ attack experiments, and presents the results in Figure C.2 and Table C.1. Again that these crafted attacks achieve ∼ 100% success rates on fooling the substitute models when (cid:15) ≥ 0 .
Similar to the MNIST experiments, these adversarial examples do not transfer very well to the generative classiﬁers, and for a ﬁxed (cid:15) setting, the white-box attacks achieve signiﬁcantly higher success rates than their grey-/black-box counterparts (with the gap at (cid:15) ≤ 0 . 1 around 30% ).
Furthermore, the minimum perturbation obtained by grey-/black-box attacks are signiﬁcantly higher than those obtained by white-box attacks. For detection, the detection rates are relatively low at (cid:15) ≤ 0 .
1 (where the classiﬁers achieved high accuracy).
But the detection rates increase signiﬁcantly for (cid:15)
= 0 .
2 , where the victim accuracies also drop.
All these results suggest that the robustness of generative classiﬁers is unlikely to be caused by gradient masking. Also most of the successfully transferred adversarial images are off the generative classiﬁer’s manifold (as a proxy to the data manifold).
[Figure]
[Table] [Table] SPSA (evolutionary strategies)
Similarly we perform the SPSA attack (Uesato et al., 2018) on the CIFAR binary classiﬁcation task.
The results for (cid:15) = 0 .
05 are presented in Table C.2, with a comparison to the white-box PGD attacks.
Again we see that SPSA fails to attack generative classiﬁers, and the bottleneck discriminative classiﬁer DBX is signiﬁcantly more robust than the discriminative ones with fully-connected graphical models.
Cross-model attack transferability Finally, we present the CIFAR cross-model attack transferability results in Figure C.3, and here we select (cid:15) = 0 .
1 instead.
Again, transferred attacks are less effective across the victim models.
However, the TP rates for logit detection are signiﬁcantly lower than in the MNIST case (also see Figure 8).
Nevertheless, the detection rates for the “discriminative to generative” transfer are considerably higher.
Combined with the accuracy results, we see that discriminative models as substitutes are ineffective in the transferred attack setting.
C. Q UANTIFYING THE EFFECT OF THE BOTTLENECK LAYER
We see from the main text that classiﬁers with bottleneck structure may be preferred for resisting adversarial examples.
To quantify this bottleneck effect, we train on MNIST models DBX, GBZ and GBY with z dimensions in { 16 , 32 , 64 , 128 } (the main text experiments use dim ( z ) = 64 ).The
clean test accuracy is shown in Table C.3, showing that all models in test perform reasonably well.
[Table]
We repeat the same white-box zero-knowledge (cid:96) ∞ attack experiments as done in the main text, where results are presented in Figure C.4 and Tables E.25, E.26 and E.27.
It is clear that for discriminative classiﬁers, DBX, the models become less robust as the bottleneck dimension dim ( z ) increases.
Interestingly DBX classiﬁers seem to be very robust against FGSM attacks, which agrees with the results in Alemi et al.
For the generative ones, we also observe similar trends (although much less signiﬁcant) of decreased robustness for GBY classiﬁers, and for GBZ the trend is unclear, presumably due to local optimum issues in optimisation.
In summary, GBZ classiﬁers are generally more robust compared to GBY classiﬁers.
More importantly, when the accuracy of [Table] generative classiﬁers on adversarial images decreases to zero, the detection rates with marginal/logit detection increases to 100% .
This clearly shows that the three attacks tested here cannot fool the generative classiﬁers without being detected.
D A DDITIONAL RESULTS
We visualise in Figure D.1 the crafted adversarial images using white-box CW attack.
We present in Table D.1 the clean accuracy on CIFAR plane-vs-frog test images (2000 in total).
We present in Table D.2 the clean accuracy on CIFAR-10 test images.
[Table]
[Figure]
E R ESULTS IN TABLES
We present in tables the full results of the experiments.
See Tables E.1 to E.9 for the white-box attacks.
See Tables E.10 to E.15 for the grey-box attacks.
See Tables E.16 to E.21 for the black-box attacks.
See Tables E.22 to E.24 for CIFAR-10 results with VGG-based classiﬁers.
See Tables E.25 to E.27 for bottleneck effect quantiﬁcation results.
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R
[Table] [Table]
[Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L R [Table] [Table]
[Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L R [Table] [Table]
[Figure] [Table] U nd e r r e v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R [Table] [Table] [Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t I C L R [Table]
[Table]
[Table] [Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t I C L
R
[Table] [Table]
[Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R
[Table] [Table]
[Table]
GBZ
[Table]
[Table]
GBY
[Table]
N
[Table]
A N
[Table]
A N
[Table]
A
N/A
N/A
[Table]
N
[Table]
A N
[Table]
A N
[Table]
N
[Table]
N
[Table]
A
[Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R
[Table] [Table]
[Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R
[Table] [Table]
U nd e r r e
v
i e w a s a c on f e r e n ce p a p e r a t
I C L
R
[Table]