A Closer Look at the Adversarial Robustness of Information Bottleneck Models
Abstract
We study the adversarial robustness of information bottleneck models for classiﬁcation.
Previous works showed that the robustness of models trained with information bottlenecks can improve upon adversarial training.
Our evaluation under a diverse range of white-box l ∞ attacks suggests that information bottlenecks alone are not a strong defense strategy, and that previous results were likely inﬂuenced by gradient obfuscation.
1. Introduction
The idea of an information bottleneck (IB) (Tishby et al., 1999) is to learn a compressed representation Z of an input
X that is predictive of a target Y .
This leads to the following training objective involving two mutual information terms:
[Equation]
This objective favours a representation Z that retains the minimum amount of information about X while being maximally predictive of Y .
The hyper-parameter β ≥ 0 controls the trade-off between the two losses.
To make this objective practical, Alemi et al.
(2017) used variational techniques to construct the upper bound for the expression in Eq. 1 – also known as the Variational Information Bottleneck (VIB) loss:
[Equation]
where p ( z | x ) is a stochastic encoder distribution, q ( y | z ) is a variational approximation to p ( y | z ) , and q ( z ) is the variational approximation to the marginal p ( z ) .
In a similar way to the Variational Auto-Encoder (VAE) setup (Kingma & Welling, 2014), we can parameterize Gaussian densities p ( z | x ) and q ( y | z ) using neural networks, and ﬁx q ( z ) to be a K -dimensional Gaussian N ( 0 , I ) , where K is the size of the bottleneck layer.
We can then use the reparameterization trick to learn the parameters of the neural networks when optimizing the stochastic estimate of the objective in Eq. 2.
A tighter bound on the IB objective is given by the Conditional Entropy Bottleneck (CEB) (Fischer & Alemi, 2020):
[Equation]
where the second term uses a class-conditional variational marginal q ( z | y ) , and ρ is a hyper-parameter with the same role as β in Eq.2.
CEB parameterizes q ( z | y ) by a linear mapping that takes a one-hot label y as input and outputs a vector µ y representing the mean of the Gaussian q ( z | y ) =
N ( µ y , I ) .
CEB uses an identity matrix for the variance of q ( z | y ) and p ( z | x ) , which is unlike VIB, where the variance of the encoder distribution is not ﬁxed.
Multiple studies suggest that IBs can reduce overﬁtting and improve robustness to adversarial attacks (Alemi et al., 2017;
Fischer & Alemi, 2020; Kirsch et al., 2021).
For example,
Fischer & Alemi (2020) showed that CEB models can outperform adversarially trained models under both l ∞ and l
PGD attacks (Madry et al., 2018) while also incurring no drop in standard accuracy.
However, no clear explanation has been found as to how IB models become more robust to adversarial examples.
Previous works also failed to investigate possible effects of gradient obfuscation which could lead to a false sense of security (Athalye et al., 2018).
In this paper, we continue the analysis into the behaviour of
IB models in the context of adversarial robustness.
Our experiments provide evidence of gradient obfuscation, which leads us to conclude that the adversarial robustness of IB models was previously overestimated.
2. Adversarial robustness
Since the discovery of adversarial examples for neural networks (Szegedy et al., 2014; Biggio et al., 2013), there has been a lot of interest in creating new attacks and defenses.
In this section we brieﬂy review methods for crafting normbounded adversarial examples.
Later, we use these methods to assess the adversarial robustness of IB models.
The Fast Gradient Sign (FGS) attack (Goodfellow et al., 2015) is an l ∞ bounded single-step attack that computes an adversarial example x
adv as x + (cid:15) sign ( ∇ x L ( θ , y, x )) , where x is the original image, y is the true label, L is the cross-entropy loss, and (cid:15) is the perturbation size.
Projected Gradient Descent (PGD) (Madry et al., 2018) is the multi-step variant of FGS.
The l ∞ PGD attack ﬁnds an adversarial example by following iterative updates x t +1 =
Proj B ( x ,(cid:15) )
(cid:0) x t + α sign ( ∇ x L ( θ , y, x )) (cid:1) for some ﬁxed number of steps T .
Here, Proj B ( (cid:15), x ) is a projection operator onto B ( (cid:15), x ) – the l ∞ ball of radius (cid:15) around the original image x .
The attack starts from an initial x 0 sampled randomly within B ( (cid:15), x ) .
The reliability of PGD attacks often depends on the choice of parameters such as the step size α , or the type of loss
L .
Recent PGD variants are designed to be less sensitive to these choices, and it is common to run an ensemble of attacks with different parameters and properties.
AutoAttack (Croce & Hein, 2020) and MultiTargeted (Gowal et al., 2019) are examples of this strategy.
3. Experiments
In this section, we experiment with VIB and CEB models on MNIST and CIFAR-10.
We run a number of diagnostics, which indicate that gradient obfuscation is the main reason why IB models are seemingly robust.
In trying to understand their failure modes, we also look at some toy problems.
Our interpretation of the results is deferred to the next section.
Hyperparameters of all our models and additional plots are included in the appendix.
3.1. MNIST
For VIB experiments on MNIST, we follow the setup of
Alemi et al.
Namely, for the encoder network, we use a 3-layer MLP with the last bottleneck layer of size
K = 256 .
This bottleneck layer outputs the K means and
K standard deviations (after a softplus transformation) of the Gaussian p ( z | x ) .
The decoder distribution q ( y | z ) over 10 classes is parameterized by a linear layer ending with a softmax.
During training, we use the reparameterization trick (Kingma & Welling, 2014) with S = 12 samples from the encoder z ∼ p ( z | x ) when estimating the expectation over p ( z | x ) in Eq.
At test time, we also collect S samples of z , and compute p ( y | x ) as 1 S (cid:80) Ss =1 q ( y |
z s ) .
We refer to this evaluation as “stochastic mode” .
In the “mean mode” , we only use the mean of p ( z | x ) as an input to the decoder.
Our deterministic baseline is an MLP of the same overall structure as the VIB model.
We train it with a cross-entropy loss without any additional regularization.
First, we evaluate our models using the FGS attack.
Figureshows the robust accuracy of VIB models with varying β under the FGS attack with different perturbation sizes (cid:15) .
For the rest of the paper, we assume that input images are in the [0 , 1] range.
Our results slightly differ from those of Alemi et al.
In particular, the performance of our VIB models peaks at β = 0 .
01 instead of β = 0 .
1 as reported previously, and the evaluation in the “mean mode” and “stochastic mode” does not lead to the same results.
Despite these differences, we can still achieve large gains in robust accuracy under the FGS attack for VIB models in comparison to the baseline.
One result that stands out is the unusually high robust accuracy under the attack with (cid:15) = 0 .
Indeed, with this perturbation size, one can design an attack that makes all images solid gray and, as such, the classiﬁer should not do better than random guessing (Carlini et al., 2019).
The obtained robust accuracy above 10% indicates that gradients of VIB models do not always direct us towards stronger adversarial examples.
To check if the improvements in robust accuracy generalize to stronger attacks, we evaluate VIB models with β = 0 .
01 under the
PGD attack with 40 steps, α = 0 .
01 , (cid:15) = 0 .
2 , and a different number of restarts.
Figure 2 shows that we can drive the robust accuracy to zero as we increase the number of restarts.
It is an indication of gradient obfuscation, as the loss landscape cannot be efﬁciently explored by gradientbased methods (Carlini et al., 2019; Croce & Hein, 2020).
3.2. CIFAR-
For CIFAR-10, as our encoder network we use a
PreActivation-ResNet18 (He et al., 2016) followed by an
MLP with the same architecture as the MNIST experiments.
We train this network end-to-end, and only use random crops and ﬂips to augment the data.
As previously, we construct an analogous deterministic model that we do not regularize in any way, thus it overﬁts.
In Figure 3a, we evaluate the adversarial robustness of CEB models on l ∞ PGD attack with 20 steps and α = 0 .
007 (Madry et al., 2018).
It is surprising that some of our deterministic models can outperform an adversariallytrained ResNet from Madry et al.
(2018) with a reported robust accuracy of 45.8%.
This result alone suggests that
PGD attacks should be used with caution when evaluating models that might obfuscate the gradients.
As with MNIST, we can again signiﬁcantly reduce the robust accuracy by increasing the number of restarts as shown in Figure 3b.
To get a better estimate of the robust accuracy in the presence of gradient obfuscation, we use a set of stronger attacks: a mixture of AutoAttack (AA) and MultiTargeted (MT) (Croce & Hein, 2020; Gowal et al., 2019).
We execute the following sequence of attacks: AutoPGD on the cross-entropy loss with 5 restarts and 100 steps, AutoPGD on the difference of logits ratio loss with 5 restarts andsteps, MultiTargeted on the margin loss with 10 restarts and 200 steps.
From Figure 3c, we see that deterministic models have zero robust accuracy, while the performance of CEB models varies across models with different random seeds.
This dependence on the seed could be the consequence of sub-optimal network initialization and difﬁculties related to training IB models.
Some part of the variance in the robust accuracy might still be attributed to having an imperfect attack due to the unreliable gradients.
Finally, in Figure 4 and in the appendix, we show typical loss landscapes produced by the CEB model with ρ =that scored 15.8% accuracy under the AA+MT ensemble of attacks.
These plots are strikingly different from typical smooth non-ﬂat loss landscapes obtained from adversarially trained models (Qin et al., 2019).
The ﬂatness of the plotted landscapes explains why gradient-based attacks with crossentropy loss are not as effective.
Moreover, since IB losses do not explicitly penalize misclassiﬁcation for perturbed inputs within a certain l p -ball, the model is free to choose where to place decision boundaries.
Figure 4 suggests that
CEB models could be robust to much smaller perturbation radii.
3.3. A toy problem
We established that gradient obfuscation makes it harder to understand the robustness properties of IB models on real datasets.
Thus, analysing toy examples can be a useful alternative.
A classiﬁcation task from Tsipras et al.
(2019) is one example that can motivate the use of IBs, where their ability to ignore irrelevant features becomes helpful.
We study this problem in the appendix.
Here, we consider another simple setup where labels y are sampled uniformly at random from {− 1 , 1 } , and two features have the following conditional distributions:
[Equation]
In this example, the label can be predicted from the sign of x 1 , so in the optimal IB case, we need to communicatebit of information about the input.
The ﬁrst feature is also more robust since it requires a larger perturbation before its sign gets ﬂipped.
In practice, we found that a simple VIB classiﬁer does not exclusively focus on x 1 , and so it becomes prone to a rather trivial attack that substracts or adds (cid:15) =to x 2 depending on the label, as shown in Figure 5.
This could be the consequence of SGD training, the approximate nature of the objective function, VIB’s formulation as a combination of competing objectives or other reasons we do not yet understand.
4. Discussion
By re-evaluating adversarial robustness of VIB and CEB models, we have shown that weak adversarial attacks are often unable to provide reliable robustness estimates as these models create highly non-smooth loss surfaces, which are harder to explore with gradients.
Therefore, we believe that previous, as well as future results on the robustness of IB models should include basic checks for gradient obfuscation.
This is especially important when comparing different types of models, e.g. IBs versus adversarial training.
[Figure]
Our experiments were inconclusive as to whether IB models offer adversarial robustness gains relative to the undefended deterministic baseline.
For MNIST, the results under the
FGS attack seemed promising.
However, looking at the performance under the PGD attack with multiple restarts and different perturbation sizes showed a different picture.
For CIFAR-10, some of the CEB models were signiﬁcantly better than the baseline under the strongest attack.
However, we did not identify the exact cause for having excessive variance in the results of models with different random seeds.
Thus, it would be interesting to ﬁnd regimes where
CEB can reliably converge to more robust models.
In this paper, we only considered IB models in discriminative settings.
A generative model related to VIB is βVAE (Higgins et al., 2017).
For auto-encoders, the adversarial attack amounts to ﬁnding inputs that would cause the decoder to reconstruct a visually distinct image, e.g. an object from a different class.
Camuto et al.
(2021) showed that β -VAE for larger values of β is more robust to adversarial attacks.
However, Kuzina et al.
(2021) used a different set of evaluation metrics to challenge this claim.
Cemgil et al.
(2020) attributes the lack of robustness of β -VAE models to the inability of their objective to control the behaviour of the encoder outside of the support of the empirical data distribution.
Namely, without additionally forcing the encoder to be smooth, tuning β alone is not enough for learning robust representations.
Together with our observations for VIB and CEB models, the disagreement about β -VAE’s results corroborates the need for more nuanced evaluation before adversarial robustness claims can be made.
Overall, we believe that using IBs in the context of adversarial robustness is an idea that deserves further exploration.
In this paper, we focused on the empirical evaluation of IB models under standard robustness metrics and illustrating the caveats related to it.
An interesting future research direction would be to understand the properties of IB models, especially in the stochastic regime, from both informationtheoretic and adversarial robustness perspectives.
Another promising direction would be to explore IBs with additional curvature regularization (Moosavi-Dezfooli et al., 2019; Qin et al., 2019) or in combination with adversarial training. Acknowledgements
We would like to thank Taylan Cemgil, Lucas Theis, Hubert
Soyer, Jonas Degrave and the wonderful people from the robustness teams at DeepMind for their help with this project, interesting questions, valuable discussions, and feedback on the paper.
Appendix Toy example
In Figure 6, we plot 1K samples from the data distribution as outlined in Section 3.3.
We train both a deterministic and a VIB model.
For the deterministic model, we used a linear classiﬁer whose weights and biases were initialized with zeros.
For the VIB model, we used the bottleneck of size 2 .
The weights of the encoder were initialized with Xavier uniform scheme (Glorot & Bengio, 2010).
The linear decoder’s weights were initialized to zero.
We use 12 samples from q ( z | x ) during training as well as for the stochastic evaluation mode.
We optimize the parameters of both the linear deterministic and the VIB model using SGD with a learning rate of 0 .
003 , momentum of 0 .
9 and Nesterov updates.
We perform 1000 iterations with a batch size of 1024 (re-sampled from the data distribution each iteration) and the same random seed for both models.
We evaluated clean and robust accuracy on a ﬁxed set of 10K samples.
To this end, considering the discussion in (Stutz et al., 2019;
Tsipras et al., 2019), we create a pre-computed set of adversarial examples by sampling exclusively from the lowdensity regions, cf.
Figure 6.
This emulates adversarial examples directly attacking the feature with weak correlation and is reasonable due to the low dimensionality, i.e., d = 2 .
It also means that we do not consider classical l ∞ constrained adversarial examples.
This is because, even for small (cid:15) , such adversarial examples are not guaranteed to preserve the original label.
This is also complementary to the toy example by Tsipras et al.
(2019) discussed below.
Toy example from Tsipras et al. (2019)
For our second toy problem, we consider a binary classiﬁcation task from Tsipras et al.
(2019), where y ∼ {− [Equation] 1 } uniformly at random, and features are distributed as:
[Equation]
We choose p = 0 .
95 , d = 100 , and η = 0 .
An adversarial attack with (cid:15) = 2 η can shift Gaussian features towards the opposite class, so that x 2 . . .
x d +1 i.i.d ∼ N ( − ηy, 1) .
Thus, it becomes easy to fool a classiﬁer that relies on these features.
Note, however, that this might also change the true label according to the data distribution.
Nevertheless, one might expect that IB models are more robust in this case since the compression cost forces to focus on x 1 – the feature that highly predictive of the label.
Indeed, if we look at
Figure 7, this seems to be the case, but oddly, only for the case of stochastic evaluation.
Note that Tsipras et al.
(2019) constructed this problem to demonstrate that clean and robust accuracy were at odds with each other, and this is what we also see in Figure 7.
There are, however, doubts whether this toy example can reﬂect what happens in realworld scenarios (Yang et al., 2020; Stutz et al., 2019).
On this toy example, we used the same setup as described above for our own toy example.
However, we adapted the VIB bottleneck to be of size 25 due to the increased dimensionality, i.e., d = 100 and only perform 200 update steps.
Architectures and hyperparameters
For MNIST experiments, we based our JAX (Bradbury et al., 2018) implementation on the original VIB code: github.com/alexalemi/vib_demo .
We used the following MLP architecture for the encoder: 1024 - ReLU1024 - ReLU - 2 K , with K = 256 .
The decoder consisted of a single dense layer with a softmax nonlinearity over 10 outputs.
All weights were initialized using the default
Xavier uniform scheme (Glorot & Bengio, 2010), and all biases were initialized to zero.
We used Adam optimizer with an initial learning rate of 10 − 4 and parameters β 1 = 0 .
5 , β 2 = 0 .
We decayed the learning rate by a factor of 0.97 every 2 epochs.
The batch size was set to 100, and we trained the networks for 200 epochs.
Input images within a [0 , 1] range were rescaled inside the network to [ − 1 , 1] range prior to passing them to the ﬁrst dense layer.
We used
Polyak averaging (Polyak & Juditsky, 1992) with a constant decay of 0.999.
For 256 outputs from the bottleneck layer that correspond to the standard deviation of p ( z | x ) , we used the following softplus transformation to make them positive: σ ( x ) =
log(1 + exp( x − 5 . 0)) .
Our deterministic baseline models had the same overall structure as the VIB models, i.e. 1024 - ReLU - 1024 - ReLU - K - 10 - softmax, with all training hyperparameters as above.
For CIFAR-10 experiments, the encoder network was a concatenation of a PreActivation-ResNet18 (He et al., 2016) with the same MLP as in our MNIST setup.
The decoder q ( y | z ) and the backward encoder q ( z | y ) in CEB were again one-layer networks.
We trained everything end-to-end for 1000 epochs.
The batch size was set to 1024, and we used
Adam with an initial learning rate of 0.012 and default β parameters.
The learning rate was multiplied by 0.3 every 250 epochs.
For CEB, we annealed ρ from an initial value of 100 down to its target value during the ﬁrst 4 epochs.
Similarly, for VIB, we increased β from 10 − 8 to its target during the ﬁrst 100 epochs.
Prior to the ﬁrst ResNet layer, input images within a [0 , 1] range were normalized using per-channel means and standard deviations computed across the train set of CIFAR-10.
Additional results on MNIST and CIFAR-
Below, we provide additional ﬁgures for the experiments in Sections 3.1 and 3.2.
For MNIST, Figure 8 shows the results of increasing the number of restarts when we use a
PGD attack with (cid:15) = 0 .
For CIFAR-10, Figure 9 plots the robust accuracy of VIB models under various attacks, and
Figure 10 illustrates cross-entropy loss surface of a CEB model on a couple of test images.
[Figure] [Figure]