I MPROVED RESISTANCE OF NEURAL NETWORKS TO ADVERSARIAL IMAGES THROUGH GENERATIVE PRETRAINING
Anonymous authors
Paper under double-blind review
A BSTRACT
We train a feed forward neural network with increased robustness against adversarial attacks compared to conventional training approaches.
This is achieved using a novel pre-trained building block based on a mean ﬁeld description of a
Boltzmann machine.
On the MNIST dataset the method achieves strong adversarial resistance without data augmentation or adversarial training.
We show that the increased adversarial resistance is correlated with the generative performance of the underlying Boltzmann machine.
I NTRODUCTION
In an adversarial attack on a classiﬁcation system one seeks inputs that lie very close (according to some norm or measure) to a regular input, but give a completely different classiﬁcation result.
Adversarial attacks on machine learning systems have been studied for over a decade, starting from the study of attacks on spam ﬁltering systems (for a historical review see Biggio and Roli (2018)).
Adversarial attacks on deep learning image recognition models have ﬁrst been discussed by Szegedy et al.
In addition to highlighting security implications adversarial images have acquired a new meaning in this context, presenting us with a fundamental conundrum: How can deep learning systems succesfully generalise and at the same time be extremely vulnerable to minute changes in the input?
The topic of adversarial images in neural nets has been studied intensely with attacks and defences leapfrogging each other (for an overview see Kurakin et al.
All common neural network models suffer from susceptibility to adversarial attacks.
It has even been shown by
Moosavi-Dezfooli et al.
(2017) that adversarial attack patterns can be transferred between different models, suggesting some common underlying mechanism.
Despite the intense study of the subject a solution remains elusive, with Jetley et al.
(2018) suggesting that classiﬁcation performance and vulnerability to adversarial attacks are correlated, while Gilmer et al.
(2018) present evidence for the opposite in a simple model.
The difﬁculty in ﬁnding neural networks robust against adversarial attacks suggest that the standard discriminative neural network training procedure and the resulting networks’ vulnerability are linked.
Making use of generative methods (e.g variational autoencoders, introduced in this context by Gu and Rigazio (2014)) has resulted in creating the most adversarially robust classiﬁcations but with added computational cost at inference time (Schott et al.
Another state of the art approach to increasing adversarial resistance applies denoising to image patches to achieve strong robustness (Moosavi-Dezfooli et al.
In this article we concentrate on one common generative model, Boltzmann machines, and the impact of its use on adversarial resistance.
We examine the adversarial resistance on the standard
MNIST dataset.
Boltzmann machines are a stochastic model with binary units (see e.g. Hinton (2007)).
A number of “hidden” units form a model for the data distribution over the “visible” units.
In a classiﬁcation scenario one would ﬁx the visible units and use the probability distribution over the hidden units for further processing.
In a restricted Boltzmann machine the hidden units are independent and therefore the distribution over the hidden units in a classiﬁcation scenario becomes a product distribution fully characterised by the average values of the individual hidden units.
Pretraining using restricted Boltzmann machines has been used in early neural networks (Hinton et al.
but was abandoned as it became clear that good training can be achieved without it.
Here we follow a slightly different path.
We will use a Boltzmann machine to model image patches only.
We do the modelling in a hierarchical fashion: train one Boltzmann machine for small image patches, then add more hidden units to model the joint distribution over larger patches, similar to the stacked
RBM approach of Norouzi et al.
This forces us to depart from a simple restricted Boltzmann machine model and therefore we cannot obtain the hidden probability distribution as easily as for a restricted Boltzmann machine.
Extended mean ﬁeld methods developed for the description of spin glasses in physics (Thouless et al.
; Georges and Yedidia (1991); Kühn and Helias (2017)) have been applied succesfully to train a restricted Boltzmann machine by Gabrié et al. and Budzianowski (2016). We use these methods to arrive at a description of the hidden units in terms of their average values, where now we replace the single neural network layer resulting from the restricted Boltzmann machine with a many-layer iteration derived from the mean ﬁeld description.
This pre-trained building block can be used as a ﬁxed input layer in a neural network trained in the usual discriminatory fashion.
In summary the contributions of this article are as follows:
• We construct a new neural network building block, based on the mean ﬁeld description of a Boltzmann machine.
The result is a deep feed forward neural network with shared weights and a structure derived from the energy function of the underlying Boltzmann machine.
• We use this building block in a neural network to achieve state of the art resistance to adversarial attacks on MNIST and show a correlation between the loss function of the Boltzmann machine (i
e. it’s generative capability) and the adversarial resistance of the resulting neural network.
• We show that the function mapping visible unit inputs to hidden unit outputs implemented by the new building block is strongly dependent on the input.
A ball of random distortions around a training example gets mapped to a smaller output volume compared to the same mapping starting from a random input.
B OLTZMANN M ACHINES AS F EATURE E XTRACTORS
Let us review the the mean ﬁeld description of a Boltzmann machine (Budzianowski (2016); Gabrié et al.
Boltzmann machines are stochastic, energy based models derived from the statistical physics of interacting spins.
Boltzmann machines are a generative model, the generated probability distribution being the thermal equilibrium probability distribution over all possible spin combinations s
= ( s 1 , s 2 , . . .
, s N ) with s n = ± 1 , as determined by the energy function through its parameters θ = ( b , J )
[Equation]
so that the probability of realising the conﬁguration s is proportional to the Boltzmann factor p
[Equation]
The spins are divided into two sets, the visible units v and the hidden units h .
When training a
Boltzmann machine we want to generate a target probability distribution p 0 ( v ) over the visible units, as deﬁned by example conﬁgurations v (0) j , j = 1 , . . . , n J .
This can be achieved by adjusting the parameters of the energy function to minimise the relative entropy between the marginal distribution over the visible units p ( v , θ ) =
Tr h p ( v , h , θ ) .
and the target distribution:
[Equation]
The gradient of the relative entropy with respect to the parameters of the energy function θ can be expressed in terms of expectation values of spins
[Equation]
[Equation]
where (cid:104) . . .
(cid:105) F denotes the expectation value of the model without constraints (free) and (cid:104) . . .
(cid:105) C ( j ) the expectation value if the visible units are ﬁxed to example v j (clamped).
This gradient can be used in a ﬁrst order optimisation method to arrive at an optimal parameter conﬁguration θ 0 .
While the expression for the gradient looks simple, evaluating the probabilities and expectation values is not, and some approximation scheme needs to be used.
Two schemes are usually used, one based on sampling (Hinton and E. (2002)), the other one based on a mean ﬁeld approximation.
Here we will concentrate on the mean ﬁeld approximation.
The central quantity in mean ﬁeld theory is the free energy.
We can obtain the free energy of our model as an expansion in the order of the coupling J as derived by a number of authors (Georges and Yedidia (1991); Kühn and Helias (2017)).
[Equation]
To lowest order the free energy is given by the entropy of independent spins.
[Equation]
where m i is the magnetisation of spin i .
Up to second order we obtain
[Equation]
where the notation (cid:104) ij (cid:105) denotes summation over unequal indices.
For orders up to four see appendix A. From the free energy we can derive an equation to obtain the mean ﬁeld values m i in the approximation up to order N
[Equation]
with
[Equation]
In particular, the ﬁrst two orders are
[Equation]
and
[Equation]
Higher orders can be found in the appendix A.
In the free case, the transcendental equation, Eq.can be solved by iteration, starting from zero or random magnetisations, until the magnetisations converge.
In the clamped case we keep the visible magnetisations ﬁxed, while iterating over the hidden units.
This results in two sets of distinct magnetisations: the free magnetisations m F and the clamped magnetisations m C ( j ) , where example j is ﬁxed on the visible units.
We also get a set of corresponding free energies F F and F C ( j ) .
The correlation functions can be obtained from the free energies
[Equation]
FC layer, weights 2W, biases b h
FC layer, weights -4J h
FC layer, weights 2J h
W
J J
J J
J J
J J
W
J
Jtanh non-linearity 1-x 2 elementwise elementwise multiply total and the expectation values are given directly by [Equation]
These expectation values can be inserted in the expressions for the gradient, Eqs.
2 and 3, and used to minimise the relative entropy Eq.
Probabilities can be estimated from the free energies [Equation] The training of a Boltzmann machine emphasises the main task of the machine as generating examples from the target distribution, but one can look at the machine from a different point of view: using the hidden units for constructing a model of the example data.
The probability distribution over the hidden units if an example is ﬁxed on the visible units then tells us something about the model required to describe the example.
In a restricted Boltzmann machine (RBM) the probability distribution is a product distribution over the hidden units and features, but one can imagine introducing couplings between hidden units to account for correlations between individual features to arrive at a better/simpler description of the data.
In the remaining article we will discuss machines that have bipartite connectivity between visible and hidden units and full connectivity between hidden units.
During inference we want to obtain the hidden magnetisations given the magnetisations of the visible units.
We split the magnetisations into visible and hidden parts m =
( m v , m h ) .
Similarly we split the magnetic ﬁelds b = (0 , b h ) and the couplings
[Equation]
where W now describes the bipartite connectivity between visible and hidden units and J h the coupling between hidden units.
Keeping terms to second order the mean ﬁeld equations become
[Equation]
Solving the mean ﬁeld equation Eq.
5 requires iteration, e.g. starting from the initial magnetisations m h = 0 .
The iteration can be unrolled into an r layer neural network, where r is greater than the number of iterations necessary for convergence (see Fig. 1).
We will refer to this neural network as the mean ﬁeld Boltzmann network.
T RAINING THE MEAN FIELD B OLTZMANN NETWORK
The Boltzmann machine underlying the mean ﬁeld Boltzmann network is trained on data derived from the standard MNIST training set.
We binarize the MNIST images with a threshold (50/255) and create two datasets.
The ﬁrst dataset contains 4 x 4 patches cut from the binarised images, the second contains 8 x 8 patches.
We start by training a Boltzmann machine for the 4 x 4 patches.
Of all 2 16 possible patches, 679 patches contain over 98% of all occurences in the dataset.
We use these 679 to train a mean ﬁeld Boltzmann machine with 32 hidden units (see Fig. 2).
Four of [Figure] these trained small machines are then combined with 128 hidden units to model the 8 x 8 patches.
The dataset of 8 x 8 patches contains approximately 600000 examples.
For the calculation of the gradient we use batches of 10000 examples.
To calculate the free expectation values we use the mean ﬁeld expansion up to fourth order, for the clamped expectation values it sufﬁces to go to second order.
The magnetisations in the free case are calculated using 100 mean-ﬁeld iterations, for the clamped magnetisation we use 20 iterations.
In the late stage of the training the value for the free partition function is underestimated. This can lead to the sum of individual sample probabilities exceeding one. We adjust the free partition function in calculation of the relative entropy and the gradient so that the probabilities within one batch are normalised to one.
Similar results to the staged training can be obtained by training a Boltzmann machine of identical connectivity directly on the 8x8 patches.
A DVERSARIALLY ROBUST NEURAL NETWORKS
The trained mean ﬁeld Boltzmann network constitutes a new neural network building block that takes in 8 x 8 patches and outputs a vector of 128 hidden magnetisations.
This building block has identical input/output dimensions to a standard convolutional layer with 8 x 8 ﬁlter size and 128 ﬁlters.
We can use this building block as an input layer in an identical fashion to a standard convolutional layer, sweeping it over the two input image dimensions.
Starting from the 28 x
MNIST images, this results in an intermediate representation of 128 x 21 x 21 dimensions.
On top of this representation we build a standard three layer convolutional neural network with ReLU activations.
We add 3 convolution layers (kernel size 3 x 3, stride 2) with 128 ﬁlters, 64 ﬁlters and 32 ﬁlters, respectively, followed by a fully connected layer and a softmax function). The input [Figure]
layer parameters are kept ﬁxed during training and the remaining layers are trained in a standard discriminative fashion.
Let us establish the connection between better generative pre-training and increased adversarial resistance.
We initialize the mean ﬁeld Boltzmann network with parameters from various stages of the generative pre-training.
As a measure for the generative training process we can use the relative entropy of the generated and target probability distribution, Eq. 1.
To assess adversarial resistance we use an iterative gradient based (white box) method with L 2 constraints and record the mean squared error for the ﬁrst image iteration that is misclassiﬁed.
We average the mean squared error for 1000 randomly chosen images from the MNIST test set.
We also calculate the test error for the whole clean MNIST test set for a given neural network.
The results are presented in Fig.
We see that a smaller relative entropy in the pre-training phase correlates with a higher adversarial resistance.
For comparison we apply the same procedure to a standard LeNet network as well as a neural network where the mean ﬁeld Boltzmann network is based on an RBM with 64 visible and 128 hidden units trained in the same mean ﬁeld approximation.
Both LeNet and RBM pre-trained network do not show the strong adversarial resistance observed in the neural network using the mean ﬁeld Boltzmann network as input layer.
On the other hand we see that the classiﬁcation error of the robust neural network on the clean test set, while above the LeNet and RBM error, is independent of the pre-training.
We ﬁnd no evidence for a strong correlation of classiﬁcation error and adversarial resistance in our model.
One reason for the apparent robustness of neural networks are obfuscated gradients (Athalye et al.
Therefore we also evaluate the robust network using a gradient free (black box) method: the Boundary Method (Brendel et al.
(2017)), implemented in Foolbox.
For evaluation we use the network that showed the highest adversarial resistance to the iterated gradient method.
The Bound-
[Table] ary Method, using 1200 iterations, spherical_step=0.1 and otherwise default parameters achieves an average L [Table]
We have added two more attacks to our evaluation. The ﬁrst attack is a transfer attack using a gradient based method on LeNet to obtain a perturbation.
The amplitude of the perturbation is then increased until the network under test misclassiﬁes the image. The second attack is a random Gaussian noise attack, where the variance of the added noise is increased until the network under test misclassiﬁes the image.
Using all four attacks we can construct a compound attack as suggested by
Schott et al.
by considering the attack with the smallest L 2 norm for each image. Combining all four attacks reduces the adversarial robustness below the robustness of any single attack.
Another reason for the apparent robustness of our model could be the very deep (20 layers) preprocessing stage leading to vanishing gradients.
To investigate this further we remove the last, fully connected layer in our neural network and train a second, shallower network with the 32 x 6 xdimensional activation data.
The distilled network consists of 5 convolution layers: (8 x 8, stride 1, 256 ﬁlters), (1 x 1, stride 1, 256 ﬁlters), (3 x 3, stride 2, 128 ﬁlters, padding 1), (3 x 3, stride 2,ﬁlters, padding 1), (3 x 3, stride 1, 32 ﬁlters, padding 1), with ReLU activations.
Putting back the fully connected layer from the full network we can measure the adversarial resistance in the same way as earlier.
We obtain an average L 2 distance of 3.35 and an error rate over the clean dataset of 1.35%, compared to an average L 2 distance of 3.9 for the original network.
Although the distilled network is showing a loss of robustness, the average L 2 distance is still more than twice than the average L 2 distance for LeNet.
This shows that some robustness can be transferred from the network structure used for learning to a relatively simple network and suggests that the robustness is not an artefact of a too deep network structure.
A comparison of our results with currently the most adversarially robust method, the Analysis By
Synthesis (ABS) model of Schott et al.
(2018), is shown in Tab. 1.
For results from other methods see table 1 in Schott et al.
Our model achieves comparable robustness results.
P ROPERTIES OF M EAN F IELD B OLTZMANN N ETWORKS
To understand the emerging adversarial robustness as a function of improved generative capability we examine the response of the mean ﬁeld Boltzmann network around different input vectors.
For one set of input vectors we will consider sample patterns from the training set, for the other set we select random binary patterns.
We then add uniform random noise of amplitude (cid:15) to the input vectors and evaluate the different outputs of the network.
The measure ∆ r/t , presented in Fig.
4, is the Lnorm of the average difference of clean and noisy output of a set of 1000 random and training set patterns respectively with 10000 random noise patterns for each input pattern.
Training set patterns are much more resistant to noise than random patterns and the resistance increases as the Boltzmann machine training progresses, particularly for small amplitude noise. net net norm Δ r a n d o m p a tt e r n t r a i n i n g p a tt e r n clean pattern noisy pattern
D ISCUSSION
We have shown that we can construct a feed forward neural network that shows strong adversarial resistance.
This is achieved by incorporating a generatively pre-trained building block derived from the mean ﬁeld description of a Boltzmann machine, which we called the mean ﬁeld Boltzmann network.
The resulting adversarial resistance strongly correlates with the effectiveness of the generative pre-training.
We believe that the increased adversarial resistance can be traced to how the mean ﬁeld Boltzmann network rejects noise around training example patches.
The noise rejection does not happen for randomly selected examples.
The rejection behaviour is an indication of the local non-linear behaviour of the mean ﬁeld Boltzmann network.
Compare this with the properties of a random matrix as feature extractor, where the restricted isometry property tells us that distances are preserved in the mapping with high probability.
Too weak non-linearity of neural networks has been suggested as one reason for the existence of adversarial images and the increased non-linearity in our model correlates with increased adversarial resistance.
On the level of individual image patches we can still ﬁnd noise patterns that result in a large response when added to a training example, but they are much less likely.
Take this together with the convolutional way that the mean ﬁeld Boltzmann network is deployed.
To get a large adversarial response from a given image region all overlapping image patches have to have a large response to the particular adversarial pattern.
With the probability of large response supressed for one pattern, it becomes even smaller for all the overlapping pattern, resulting in strong adversarial resistance for the full neural network.
Our results, while encouraging, leave a number of questions open.
First we have to ask ourselves if the success is due to the simplicity of the MNIST dataset.
Will we be able to ﬁnd typical image elements and model them with a Boltzmann machine for real image data?
Second, while the adversarial images we ﬁnd show a large deviation from the original image most do not resemble other image classes (for example images see appendix C).
The algorithm still conﬁdently places them in a particular class.
We would prefer a behaviour where images unrecognisable by a human would result in a low conﬁdence of the prediction.
A H IGHER ORDER TERMS IN THE MEAN FIELD EXPANSION
The third and fourth order term in the free energy becomes
[Equation]
The third and fourth order terms for the mean ﬁeld iteration are R (3) i
[Equation]
B H ISTOGRAMS OF L 2 DISTANCES
[Figure]
D M EAN FIELD TRAINING VS . EXACT SOLUTION
For a very small Boltzmann machine we can examine how the mean ﬁeld training procedure used in the main part of the article compares to the exact solution.
We select a Boltzmann machine ofunits, which we divide into 5 visible units and 11 hidden units.
We choose the connectivity to have bipartite connections between visible and hidden units and full connectivity between the hidden units.
To examine the training behaviour we choose two example states out of 32 possible input states, (1, -1, -1, -1, 1) and (-1, -1, 1, 1, -1), to be realised with equal probability.
For the training we use the mean-ﬁeld derived relative entropy and its gradient from section 2.
First, we can examine how the relative entropies from the mean ﬁeld approach and the exact calculation compare during the training procedure.
The results are presented in Fig.
7a.
For most of the training the mean ﬁeld derived relative entropy tracks the exact entropy very closely, only to be underestimated towards the end of the training.
This is due to the underestimation of the partition function / free energy, resulting in the overestimation of probabilities.
The probabilities are presented in Figs.
7b, 7c and their ratio in Fig.
7d.
We see that although the probabilities are overestimated, their ratio is reproduced almost exactly.
D exact
D M
F (a) 1.
D exact p(b) 1.
D exact p(c) 1.
D exact p 1 / p(d)
[Figure]
[Figure]
We can now consider the noise rejection property for the small machine, using the same procedure as in the main article.
For this small machine we can calculate the averages over the input states for all possible combination.
To evaluate the noise rejection we used 1000 random noise patterns for each example.
The results are shown in Fig.
8 and show again the same noise rejection behaviour for the example states.
In contrast, if we use a simple RBM connected machine of the same size
[Figure]
we don’t see any noise rejection behaviour (see Fig. 9).