Gradient-Free Adversarial Attacks for Bayesian Neural Networks
Abstract
The existence of adversarial examples, i.e., small perturbations to their inputs that can cause a misclassiﬁcation, underscores the importance of understanding the robustness of machine learning models.
Bayesian neural networks (BNNs), due to their calibrated uncertainty, have been shown to posses favorable adversarial robustness properties.
However, when approximate Bayesian inference methods are employed, the adversarial robustness of BNNs is still not well understood.
In this work, we employ gradient-free optimization methods in order to ﬁnd adversarial examples for BNNs.
In particular, we consider genetic algorithms, surrogate models, as well as zeroth order optimization methods and adapt them to the goal of ﬁnding adversarial examples for BNNs.
In an empirical evaluation on the MNIST and Fashion MNIST datasets, we show that for various approximate Bayesian inference methods the usage of gradient-free algorithms can greatly improve the rate of ﬁnding adversarial examples compared to state-of-the-art gradient-based methods.
1. Introduction
Deep Neural Networks (NN) have shown state-of-the-art performance on many tasks, including image recognition (Krizhevsky et al., 2017) and reinforcement learning (Schulman et al., 2015).
However, the vulnerability of NNs to adversarial examples questions their applicability to safety-critical scenarios, where a failure of a learning model can have catastrophic consequences (Goodfellow et al., 2014).
Because of their principled treatment of uncertainty, Bayesian Neural Networks (BNNs), i e., neural networks with a prior distribution placed over their parameters, have been proposed as a more robust learning model compared to standard SGD-trained NNs (Carbone et al., 2020; Bekasov and Murray, 2018).
Nevertheless, the theoretical results are all limited to the idealised case where the posterior distribution of a BNN is computed exactly via the Bayes’ rule.
In practice, exact computation of the posterior is infeasible and approximate inference methods with a ﬁnite amount of data are employed (Neal, 2012).
As a result, it is still unclear how to best ﬁnd adversarial examples for commonly employed approximate Bayesian inference methods.
In this paper, we start from the observation that for fully trained BNNs the gradient of the loss tends to be uninformative (Carbone et al., 2020), and we adapt gradient-free optimization methods for ﬁnding adversarial attacks on BNNs.
In particular, we consider: zeroth order optimization (Chen et al., 2017), optimization with surrogate gradients (Athalye et al., 2018), and genetic algorithms (Alzantot et al., 2019).
We perform an empirical evaluation of various diﬀerent NN architectures on both the MNIST and FashionMNIST datasets for networks trained with ﬁve diﬀerent commonly employed approximate inference methods for BNNs.
We ﬁnd that the presented algorithms can oﬀer signiﬁcant improvements, ﬁnding adversarial examples for up to 40% of images which remained correctly classiﬁed when using state-of-the-art, ﬁrst-order attacks.
2. Adversarial Attacks for BNNs
Bayesian modelling aims to capture the intrinsic uncertainty of data driven models.
Consider a classiﬁcation problem with n C classes for a neural network f w ( x ) with input x ∈ R m and network parameters (weights and biases) w , then one starts with a prior distribution over the network parameters p ( w ).
The ﬁt of the weights w to the data D is computed through the likelihood p ( D | w ).
Bayesian inference combines likelihood and prior via the
Bayes theorem to obtain a posterior distribution p ( w | D ) ∝ p
( D | w ) p ( w ).
Computing the posterior distribution p ( w | D ) exactly is not possible in general for NNs.
Asymptotically exact samples from p ( w | D ) can be obtained via approximate inference methods such as Hamiltonian Monte Carlo (HMC) (Neal et al., 2011), while approximate samples can be obtained more cheaply via Variational Inference (VI) (Blundell et al., 2015).
Independently of the methods employed to compute the posterior distribution, predictions at a new input x ∗ are obtained from an ensemble of n NNs, each with its individual weights drawn from the approximate posterior distribution p ( w | D ) :
[Equation]
where (cid:104)·(cid:105) p denotes expectation w.r.t.
the distribution p .
Eqn (1) is the expectation of the predictive distribution of the BNN.
Given an input point x ∗ and a strength (i.e. maximum perturbation magnitude) (cid:15) > 0, an adversarial example is a point ˜ x such that | x ∗ − ˜ x | ≤ (cid:15) and [Equation] i
[Equation]
where f w i is the i-th component of f w , and | · | is a suitable similarity metric (taken here to be the l ∞ norm).
As f w is non-linear, solving the above optimization problem exactly is infeasible and several approximate solution methods have been proposed.
Among them gradient-based attacks are arguably the most prominent, eﬃcient, and eﬀective.
In particular, these methods, which include FGSM (Goodfellow et al., 2014) and PGD (Madry et al., 2018), which rely on the gradient of the loss L ( x, w i ) function wrt the input point x .
However, it has been recently shown in (Carbone et al., 2020) that for overpametrised
BNNs in the limit of high accuracy, exact inference, and inﬁnite amount of data it holds that:
[Equation]
Such a vanishing behavior on the gradient (which is known to be not true for SGD-trained
NNs) questions the eﬃcacy of gradient-based attacks for BNNs.
In what follows, we employ several optimization methods which do not directly rely on the gradient of the loss to attack BNNs and empirically evaluate them in Section 3.
2.1. Attacking BNNs without First-Order Information
Attacking deterministic neural networks when the gradient information is missing, unavailable, or uncomputable has been studied in prior works, see e.g., (Papernot et al., 2017;
Athalye et al., 2018; Alzantot et al., 2019).
Whereas several works have studied the robustness of BNNs (Cardelli et al., 2019b; Wicker et al., 2020; Michelmore et al., 2020) and developed more robust training methods (Liu et al., 2018), attacking a BNN when the gradient is uninformative or unavailable has not been considered in detail yet.
Below, we adapt some of the more commonly gradient-free optimization approaches for this setting.
2.1.1. Zeroth Order Adversarial Attacks
As a ﬁrst approach we consider numerical approximations of the gradient via Zeroth order optimization (ZOO) methods (Chen et al., 2017; Ilyas et al., 2018; Wicker et al., 2018).
We consider these methods because it has been shown that for NNs with obfuscated gradients (i.e. when a NN is purposefully made non-diﬀerentiable or their gradients vanish as for
BNNs) numerical approximations of the gradient can still oﬀer useful information for ﬁnding adversarial attacks (Athalye et al., 2018; Papernot et al., 2017).
In particular, these methods can be adapted to BNNs by making various queries to the predictive distribution to compute a ﬁnite diﬀerence approximation of the gradient of the expectation of the loss. The resulting approximated gradient is then feed into standard gradient-based algorithms, such as the fast gradient sign method (FGSM)
(Goodfellow et al., 2014) and projected gradient descent (PGD) (Madry et al., 2018).
2.1.2. Backwards Pass Differentiable Approximations (BPDA)
Backwards Pass Diﬀerentiable Approximation (BPDA) attacks aim to learn a diﬀerentiable surrogate model in order to mimic the decision boundary of the classiﬁer under consideration (Papernot et al., 2017; Athalye et al., 2018).
The BPDA attack can be easily adapted to BNNs by learning a SGD-trained NN on a dataset where the label of each image in the training set is replaced by the predictive distribution of the BNN on that image.
Once this new NN has been learned, one can run standard gradient-based attacks on this network.
2.1.3. Genetic Algorithm (GA) for Adversarial Attacks
Both ZOO and BPDA involve approximating the gradient of the BNN wrt a given input.
This may be limiting for BNNs (see Eqn (2)).
In contrast, in this subsection, we consider a genetic algorithm similar to that proposed in (Alzantot et al., 2019) which proceeds in a completely gradient-free manner.
Given an input point x ∗ the genetic algorithm attack (GA) proceeds by ﬁrst sampling k vectors in an (cid:15) − ball around x ∗ from a uniform distribution.
These vectors are then each added to the original input to create a set of k candidate adversarial examples.
For each candidate, the BNN predictive distribution is queried and each of the k predictions are compared with the original BNN prediction and are scored based on a ﬁtness function.
In our case, this ﬁtness function is taken to be the maximizing the loss function (see Eqn 3 in the Appendix).
Tournament selection is then used to decide which modiﬁcations to keep and which to discard, that is, we randomly draw (with replacement)
j pairs of members from the population and select to keep the member with higher ﬁtness.
New members are made by crossing the winners from the tournament selection and some of the resulting set are randomly mutated before repeating these steps.
After repeating this process m times (each called a generation), the modiﬁcation with the highest ﬁtness is returned as the adversarial perturbation.
A further description and pseudocode are given in the appendix.
3. Empirical Results
We conduct an empirical evaluation of the proposed methods against state-of-the-art gradientbased methods (PGD given here and FGSM in the appendix) for various NN architectures trained with diﬀerent approximate inference methods, including Hamiltonian Monte Carlo (HMC, Neal et al.
(2011)), Bayes by Backprop (BBB, Blundell et al.
, Variational
Online Guass Newton (VOGN, Khan et al.
(2018)), NoisyAdam (NA, Zhang et al.
(2018)), and Stochastic Weight Averaging - Gaussian (SWAG, Maddox et al.
For each of the
BNNs discussed, we select the prior distribution based on the Glorot normal distribution (Sutskever et al., 2013).
For MNIST we ﬁx the strength of the adversarial perturbation (cid:15) = 0 .
1 and for FashionMNIST we instead consider (cid:15) = 0 .
For each PGD attack we consider 5 iterations with 1 restart.
We note that each iteration of PGD requires estimation of the expectation of the gradient via Monte Carlo integration making this setting as computationally expensive as 1000 iterations of PGD in the deterministic case (assuming 100 steps for the Monte Carlo integration).
We perform our experiments on the MNIST and Fashion MNIST data-sets.
We note that in our evaluation, we do not consider bigger and more complex datasets, such as CIFAR-10, because we could not train HMC and BBB on those datasets with good accuracy.
Evaluation of Robust Accuracy on MNIST
We start our evaluation with the MNIST dataset.
In Figure 1 we report the accuracy of the various NNs on clean data.
For this evaluation we consider networks with a single hidden layer of varying width (from 128 to 2048) and we observe that for all the models we obtain an accuracy ≥ 95%.
In Figure 2 (top row) we report the computed robustness to adversarial attacks for all the various networks and methods.
The robustness is measured with the ‘Robust Accuracy’, which is a standard measure of robustness (Szegedy et al., 2013) deﬁned as the percentage of test points that remain successfully classiﬁed after a given attack (i e., the smaller
the
Robust Accuracy, the less robust the network).
From Eqn (2) we expect that, at least for
HMC, which is known to produce asymptotically exact samples from the posterior, all the networks are robust to gradient-based attacks.
This is indeed the case.
However, GA which does not rely on the gradient, is up to 40% more eﬀective in these cases.
Note that the obtained values of robustness are still considerably higher than those commonly reported for SGD-trained NNs (Carbone et al., 2020).
For other approximate inference methods, GA is still more eﬀective than PGD.
However, we observe that the more approximated is the posterior computation, the more eﬀective are gradient-based attacks.
In fact, for all SWAG-trained networks (which in the degenerate case is exactly SGD-training), the robust accuracy to PGD drops from the ∼ 95% of HMC to ∼ 10% (i.e. random guessing).
Finally, we observe that, in the case of HMC, ZOO performs comparably with PGD, while BPDA is considerably better and PGD, even if less eﬀective than EA.
This is due to the fact that at no point during the optimization does
BPDA query the posterior predictive distribution of the BNN under consideration.
This again conﬁrms that for BNNs ﬁnding adversarial attacks by relying on the loss gradient can be misleading.
Evaluation of Robust Accuracy on Fashion MNIST
We continue our evaluation with the Fashion MNIST dataset.
In Figure 1 (centre column) we report the accuracy of the various trained NNs.
For this evaluation we consider networks with 2 hidden layers with same number of neurons for each layer and of varying width (512, 1024, and 2048).
In Figure 2 (bottom row) we report the robust accuracy for all the various networks and methods.
We observe that HMC and BBB trained networks are able to achieve notable success in resisting adversarial examples.
Similar to what is observed in MNIST, we see that for HMC trained networks, GA is the most eﬀective algorithm, showing up to 30% improvements over PGD attacks.
This again conﬁrms how for BNNs gradient-based attacks can be surprisingly ineﬀective compared to black-box optimization methods.
However, when we move to other approximate inference methods we see that these oﬀer less protection against gradient-based attacks.
In fact, in these cases PGD outperforms all the other methods with GA obtaining similar, but slightly worse, results.
As already noticed in (Carbone et al., 2020; Gal and Smith, 2018), this may be due to the fact that Variational Inference (VI) methods, for larger and more complex datasets, may tend to poorly approximate the uncertainty and lead to a posterior that is far from the true distribution (Cardelli et al., 2019a).
As a consequence, the loss gradient may become meaningful and the resulting networks fragile to gradient-based attacks such as PGD.
4. Conclusion
We investigated gradient-free optimization algorithms for ﬁnding adversarial examples on
BNNs.
In an empirical evaluation on the MNIST and FashionMNIST datasets, we showed that gradient-free algorithms can be more eﬀective than gradient-based methods, especially for NNs trained with approximate inference methods that compute a more accurate approximation of the true posterior.
Appendix A. FGSM Results
In Figure 3 we report the attack results for the investigated attacks on the studied posteriors with FGSM.
method against FGSM, FGSM with ZOO approximate gradients, FGSM with
BPDA approximate gradients, GA.
We observe that in all cases GA outperforms other methods.
Appendix B. Genetic Attack Details
The Genetic Attack (GA) follows the conventional structure of a genetic algorithm:
Initialize a population, 2.
compute the ﬁtness of each member of the current population, 3.
(selection) select certain members of the current population to be parents for the next population, 4.
(crossover) cross the selected parents to form children, 5.
(mutation) randomly modify the children and collect them to form the next population, then
The goal of a genetic algorithm is to increase the overall ﬁtness of the population.
In our case, the population consists of a collection of N perturbation vectors which we may add to an image x to turn it into an adversarial image.
We deﬁne the ﬁtness at class c of a member m of the population to be the negative logarithm of the c
th coordinate of the BNN’s prediction for x + m after clipping to make every coordinate in [0 , 1].
[Equation]
The ﬁtness of m at thew correct class measures how adversarial is.
If the Fitness( m, c ) is high, then the BNN predicts that x + m belongs to class c with low probability.
GeneticAttack initializes the population as N random {− ε, ε } -valued vectors with the same dimensions as the input space.
Such vectors are the largest perturbations allowed by our constraint, so intuitively they have the best chance at having a high ﬁtness.
Algorithm B is our selection operator.
It is a standard operator in genetic algorithms called tournament selection Zhong et al., and we use a tournament size of two.
We randomly draw (with replacement) N pairs of members from the population and select from each pair the member with a higher ﬁtness.
Algorithm B is our crossover operation.
It is an adaptation of the standard singlepoint crossover Umbarkar and Sheth (2015) to two-dimensional inputs.
Given two parent images, we randomly select a location to cross the parents horizontally or vertically.
Figure 4 illustrates horizontal crossover.
Algorithm B is our mutation operator.
It decides to mutate each child with probability
R , and mutation consists of ﬂipping the sign on precisely one randomly chosen pixel of that child.
Our mutation operator might be considered a sparse form of ﬂip mutation Soni and Kumar (2014).
In the case of MNIST and Fashion MNIST, we are ﬂipping single coordinates.
In the case of CIFAR10 we are ﬂipping a triple corresponding to perturbations in a pixel’s RGB values.
GeneticAttack stops after G iterations, and it returns the original image x plus the population member with the highest ﬁtness.
Data: BNN, original image x , correct class c , number of generations G , population size N , mutation rate
R , perturbation size ε .
Result: perturbed image x ∗ (hopefully adversarial) /*
Initialize population and fitness.
*/ for i ← 1 , . .
. , N do
P i ← random {− ε, ε } -valued vector end for i ← 1 , . .
. , N do
F i ← Fitness( P i , c ) end /
* Update population and fitness.
*/ for g ← 1 , . .
G do
P ← Mutate(Cross(Select( P, F )) , R ) for i ← 1 , . . .
, N do
F i ← Fitness( P i , c ) end endreturn x + P argmax( F )
Algorithm
Data: Population P , ﬁtnesses F .
Result: Array of parent perturbations.
for i ← 1 , . . .
, size( P ) do select j, k uniformly from { 1 , . .
, size( P ) } if F j > F k then M
i ← P
j else M i ← P k end endreturn M
Algorithm 3 Cross
Data: Array M of parent perturbations.
Result: Array of child perturbations.
for i ← 2 , 4 , 6 , . . .
, size( M ) do p ← P i −q ← P i select t uniformly from { 0 , 1 , . .
, image side length } select u uniformly from { 1 , 2 } if u = 1 then c , d ← cross p , q horizontally at t (Fig. 4) elsec , d ← cross p , q vertically end
C i − 1 ← c C i ← d endreturn C
Data: Array C of child perturbations, mutation rate R .
Result: Array of mutated for i ← 1 , . .
, size( C ) do if r ∼ U c
C i select randomly c j c j end endreturn C