F EATURE PRIORITIZATION AND REGULARIZATION IMPROVE STANDARD ACCURACY AND ADVERSARIAL ROBUSTNESS
Anonymous authors
Paper under double-blind review
A BSTRACT
Adversarial training has been successfully applied to build robust models at a certain cost.
While the robustness of a model increases, the standard classiﬁcation accuracy declines.
This phenomenon is suggested to be an inherent trade-off.
We propose a model that employs feature prioritization by a nonlinear attention module and L 2 feature regularization to improve the adversarial robustness and the standard accuracy relative to adversarial training.
The attention module encourages the model to rely heavily on robust features by assigning larger weights to them while suppressing non-robust features.
The regularizer encourages the model to extracts similar features for the natural and adversarial images, effectively ignoring the added perturbation.
In addition to evaluating the robustness of our model, we provide justiﬁcation for the attention module and propose a novel experimental strategy that quantitatively demonstrates that our model is almost ideally aligned with salient data characteristics.
Additional experimental results illustrate the power of our model relative to the state of the art methods.
I NTRODUCTION
Deep learning models have demonstrated impressive performance in a wide variety of applications (Goodfellow et al., 2016; Krizhevsky et al., 2012; Simonyan & Zisserman, 2014; Mnih et al., 2015).
However, recent works have shown that these models are susceptible to adversarial attacks: imperceptible but carefully chosen perturbation added to the input can cause the model to make highly conﬁdent but incorrect predictions (Szegedy et al., 2013; Goodfellow et al., 2015; Kurakin et al., 2016).
Exploring the adversarial robustness of neural networks has recently gained signiﬁcant attention and there is a rapidly growing body of work related to this topic (Kurakin et al., 2016; Tram ` er et al., 2017; Fawzi et al., 2018; Athalye & Sutskever, 2017; Carlini & Wagner, 2017; Kolter &
Wong, 2017; Wong & Kolter, 2018; Madry et al., 2017).
A wide variety of methods are proposed to defend a model against adversarial attacks (Prakash et al., 2018;
Liao et al., 2018; Song et al., 2017; Samangouei et al., 2018).
Despite these advances, many techniques are subsequently shown to be ineffective (Athalye et al., 2018; Athalye & Carlini, 2018), and adversarial training which uses adversarial samples in addition to clean images during the training process has been shown to be able to build relatively robust neural networks (Madry et al., 2017; Athalye et al., 2018; Dvijotham et al., 2018).
With strong adversaries such as the Projected Gradient Descent (PGD) (Madry et al., 2017) or the Iterative Fast Gradient Sign Method (I-FGSM) (Kurakin et al., 2016) adversarially trained models are able to achieve state-of-the-art performance against a wide range of attacks.
Recent advances in the understanding of adversarial training provide insights of its effectiveness. It is shown that standard and robust models depend on very different sets of features (Tsipras et al., 2018; Tanay et al., 2018).
While standard models utilize features including non-robust ones that are weakly correlated with class labels and easily manipulated by small input perturbations, robust models only use robust features that are highly correlated with class labels and invariant to those perturbations. Although adversarial training learns robust features, there are no explicit design components to encourage a model to depend solely on robust features.
Therefore, to further improve the robustness of a model, we propose feature regularization and prioritization schemes. We ﬁrst propose to use attention mechanism to bias a model towards robust features that are highly correlated with the class labels.
We call the learned features at the ﬁnal layer of a network the global features, and the ones at lower level layers the local features.
In our attention module, we use the global features as a way to assign weights to the local features by a non-linear compatibility function.
Since global features are directly used to produce class label prediction, we are effectively assigning weights to local features depending on their correlation to the label. Robust features have higher correlation and therefore will be assigned larger weights which in turn contribute to the model’s robustness. Next, we propose to use feature regularization to learn robust features that are invariant to input perturbations.
We add a L 2 regularization term that penalizes the distance between the learned features of a clean sample x and that of its perturbed adversarial counterpart x (cid:48) to the training objective.
By optimizing this regularizer, we are pushing the model to extract very similar features from the original image and the adversarial image, thus only features that are invariant to the perturbations are learned and we effectively ignore the added noise. From another point of view, a model with small L 2 feature distance maps the two nearby points in the image space to nearby points in the learned high dimensional manifold which is a desirable behavior.
In this paper, we propose an approach that enhances adversarial training with feature prioritization and regularization to improve the robustness of a model.
We use extensive experiments to demonstrate that the attention module focuses on the area of an image which contains the actual object and helps the classiﬁer to only rely on features extracted from those areas.
The background clutter and irrelevant features which could be misleading are suppressed.
The feature regularization further encourages the model to extract robust features that are not manipulated by the adversarial perturbations.
The resulting model has a highly interpretable gradient map that aligns perfectly with salient data characteristics.
The main contributions of this paper include:
• A method based on feature prioritization and regularization, which signiﬁcantly outperforms adversarial training.
Our model is evaluated on the MNIST, CIFAR-10, and CIFAR100 datasets, and demonstrates superior performance relative to both standard classiﬁcation accuracy and adversarial robustness.
• We provide justiﬁcation to show that the attention module helps the model to rely on robust features by assigning larger weights to them.
Through qualitative inspection, we show that the attention maps generated by our non-linear attention estimator focus sharply on the regions of interest while suppressing irrelevant background clutter.
• In addition to qualitative evaluation of the gradient maps, we propose a novel experimental strategy that quantitatively demonstrates the better alignment of the gradient maps generated by our model with salient data characteristics.
R ELATED WORK
Due to the extensive amount of literature in this area and the limited length of this paper, we only review some of the most related works in this section.
For a more comprehensive survey please refer to Akhtar & Mian (2018).
Adversarial training. Kurakin et al.
(2016) use adversarial training as a form of data augmentation where it injects adversarial examples during training.
In every training mini batch, a mixture of clean images and adversarial images generated by one step Fast Gradient Sign Method (FGSM) are used to update the network’s parameters.
It is then improved by Na et al.
by adding adversarial examples generated by iterative methods.
In Madry et al.
(2017), it is proposed to replace all clean images with adversarial images which is a direct result of optimizing a saddle point (min-max) formulation.
By studying the loss landscape of the problem, they suggest that PGD is a universal ﬁrst-order adversary which is then used in their adversary generating process.
L 2 regularization .
A similar idea with feature regularization is proposed in Kannan et al. (2018) which they call logit pairing, to prevent a model from being over-conﬁdent when making predictions.
Compared with logit pairing, feature regularization is more intuitive as it motivates a model to learn very robust features that are invariant to input perturbations, which leads to a robust model. In c [Figure] n
[Figure] ⌃ f c - [Figure]
L globalfeature s o f t m a x c o n v c o n v f c a tt e n t i o n w e i g h t e s t i m a t o r c o n v weightedcombination ⌃ c o n v c o n v f c a tt e n t i o n w e i g h t
e s t i m a t o r cross entropy loss [Figure] addition, we propose to also use attention module to further encourage the model to favor robust features which will improve the robustness.
Attention Models .
Attention in CNN is most commonly deployed for query-based tasks (Seo et al., 2016; Xu et al., 2015; Jetley et al., 2018).
In Jetley et al.
(2018) a method is presented to use a learned representation of the global image as a query to leverage multiple attention maps at different scales, which allows the expression of a complementary focus on different parts of the image.
However, the application of attention to the adversarial robustness aspects has not been seriously explored.
To the best of our knowledge, we are the ﬁrst to employ an attention mechanism in training a robust deep neural network.
In our application, we use a ReLU activated neural network instead of the linear-based method as the attention estimator.
It allows highly non-linear compatibility between the learned global features and the lower-level local features.
A PPROACH
We now present our model that combines the attention module and L 2 feature regularization, and show how it can be applied to enhance the adversarial training to improve the adversarial robustness of a model and its accuracy.
Figure 1 provides an overview of our method.
We adopt the terminology in Jetley et al.
(2018), in particular that local features and global features refer to features extracted at certain layers of a network whose effective receptive ﬁelds are subsets of the image and the entire image.
We start by forwarding each of the clean and adversarial images and computing the attention weights by a non-linear estimator.
Then the individual attention feature is deﬁned to be the weighted combination of the corresponding local features.
Next, we deﬁne an L 2 regularization loss to be the
Euclidean distance between the two sets of learned attention features.
The attention features of the adversarial image are then used to produce the logits, which is followed by softmax layer to produce the cross-entropy loss.
The ﬁnal loss function of our model is a combination of cross-entropy loss and the regularization loss.
3. A DVERSARIAL TRAINING
We adopt the adversarial training described in Madry et al.
(2017) as the basic training approach.
It replaces natural training examples by PGD examples, which is suggested to represent a universal ﬁrst-order adversary.
So far PGD has been shown to represent the strongest attack method (Athalye et al., 2018; Athalye & Carlini, 2018).
A model that is trained with PGD adversaries is also robust against a wide range of other attacks and not yet outperformed by any other approach.
The adversarial training has a saddle point formulation:
[Equation]
where D is the distribution of data x and class labels y , L is the cross-entropy loss function for a model with parameters θ , δ is the additive adversarial perturbation with bound S .
In this paper we consider l ∞ bound as in Madry et al.
Our adversarial samples x (cid:48)
=
x + δ are created by
PGD:
[Equation]
PGD adversaries are computed at each iteration as an approximated optimum of the inner maximization in equation (1) and an update of the parameters θ is made according to the outer minimization formulation.
3. A TTENTION MODEL
As we discussed in Section 1, our goal of attention model is to favor robust features in making predictions.
We propose a non-linear attention model that acts as a feature prioritizing scheme, which is able to put more weight on robust features and less weight on non-robust features to increase the robustness of a classiﬁer.
Let l in be the learned feature vector at layer i ∈ { 1 , 2 , ..., I } of a neural network at spatial location n ∈ { 1 , 2 , ..., N } , and let g be the feature vector of the layer just before the ﬁnal fully connected layer which produces the class label prediction scores (logits).
We use a small ReLU activated neural network to generate compatibility scores between the global feature g and local features l in :
[Equation]
where f is the neural network and the concatenation of g and l in is fed to the network to produce the compatibility scores c in .
We then normalize the scores with a softmax operation to get the attention weights:
[Equation]
Next, we compute the weighted sum of local feature vectors which is the attention feature vector at layer i :
[Equation]
We use the outputs of the last residual block as the local feature for computing attention, and replace the global feature g with the corresponding attention descriptor h i for ﬁnal classiﬁcation.
By using a small neural network instead of the linear alignment models as in Jetley et al.
(2018), we are able to capture non-linear compatibility between the local and global features when producing the attention weights, which is beneﬁcial considering the multiple non-linear function activated layers between the local and global features.
3. F EATURE REGULARIZATION
In addition to the attention mechanism, we also propose to use an L 2 regularization term to encourage the model to extract similar features for the clean image and the corresponding adversarial image.
Denote by G θ the deep neural network, x and x (cid:48) the natural image and adversarial image.
Denote by G θ ( x ) , G θ ( x (cid:48) ) the learned features of the layer just before the ﬁnal fully connected layer (in our case this is the attention weighted global descriptor) which produces the class label prediction scores.
The L 2 regularizer has the following form:
[Equation]
By minimizing the regularization function, the model effectively learns very similar features for the clean sample and the adversarial sample, which are robust features since they are invariant to the adversarial perturbation.
From another perspective, the learned features of a neural network lie on a high dimensional manifold that is linearly separable for different classes because the classiﬁcation layer is a linear classiﬁer followed by a softmax function.
With adversarial training alone, a model only tries to map x and x (cid:48) to the same side of the decision boundary, while with the additional regularization, they are not only on the same side but also mapped to nearby points in the space.
This mapping is a desired behavior considering that, in the original image space, they are very close points representing essentially the same image.
3. M ODEL LOSS
Equipped with the presented methods, the total loss of our model is:
[Equation]
where λ is a hyperparameter that controls the relative weight of the L 2 regularization loss.
E XPERIMENTS AND R ESULTS
In this section, we evaluate our model on the MNIST, CIFAR-10 and CIFAR-100 datasets, and present justiﬁcation to attention module and some quantitative and qualitative results.
4. R OBUSTNESS ON MNIST
First we present the results on the MNIST dataset.
We use a CNN with two convolutional layers with 32 and 64 ﬁlters respectively, each followed by 2 × 2 max-pooling, and a fully connected layer of size 1024.
For the PGD adversary, we run 40 iterations with a step size of 0.01 and l ∞ bound of (cid:15) = 0 .
The settings are the same as in Madry et al.
Since MNIST is a very small scale dataset and the model is very robust with just adversarial training, we do not employ the attention mechanism on MNIST.
We only study the effectiveness of the proposed feature regularization method.
[Table]
The evaluation results on MNIST are presented in Table 1.
Regarding the value of weight λ of the
L 2 regularizer, we ﬁnd that roughly any λ ∈ [0 . 001 , 0 . 05] works well.
The reported results are obtained with λ = 0 .
From Table 1, we can see that a model trained with the proposed feature regularization method is signiﬁcantly more robust against PGD adversary than the baseline model that uses adversarial training alone.
The improvement is nearly 3% for white box attack and 1% for black box attack.
4. R OBUSTNESS ON CIFAR-
Here we present our results on the CIFAR-10 dataset.
For the base network, we use the original
ResNet model that has four residual blocks with [16, 16, 32, 64] ﬁlters and its 3-times wider variant with [16, 48, 96, 192] ﬁlters respectively.
For our attention model, we modify the ResNet by replacing the spatial global average pooling layer after the residual block 4 with a convolutional layer sandwiched between two max-pooling layers to obtain the global feature g .
We use a onehidden-layer neural network with 64 hidden units and ReLU activation function as the non-linear attention weight estimator.
For the PGD adversary, we run 5 iterations with a step size of 2 and l ∞ bound of (cid:15) = 8 / 255 .
In order to isolate and analyze the effectiveness of attention module and implicit denoising independently, we train three models with the following conﬁgurations: AT-reg is an adversarial trained model with feature regularization, AT-att is an adversarial trained model with attention module, and AT-att-reg is the model with both attention and feature regularization. [Table]
[Table]
The evaluation results of aforementioned four models on CIFAR10 are presented in Table 2.
Similar with MNIST, we ﬁnd that roughly any λ ∈ [0 . 01 , 0 . 1] works well for feature regularization.
The reported results are obtained with λ = 0 .
1 for AT-reg and λ = 0 .
01 for AT-att-reg.
From the table, we see that all of the three proposed models have better adversarial robustness over the baseline model that only uses adversarial training, and both models with attention show improvement on the classiﬁcation accuracy on natural examples as well.
By comparing AT-reg with adversarial training (Madry et al., 2017), we note that the feature regularization method provides a signiﬁcant improvement on white box robustness over the baseline method, with a cost of standard accuracy and a slight decline of black box accuracy.
Same trade-off appears in the comparison of AT-att-reg and AT-att.
[Figure]
Next, by comparing the results of models with and without the attention module, we can see that attention contributes to both standard and adversarial accuracy.
The attention structure not only favors robust features, it also relies heavily on features extracted from the spatial area that contains the actual object of concern.
By suppressing the features extracted from the background clutter and misleading perturbations in irrelevant areas, the model with attention module more precisely learns the underlying distribution of the data therefore the better accuracy.
In the next section, we will demonstrate that the attention module assigns larger weights to more robust features, and attention maps sharply focus on the object in the image and ignore the irrelevant background clutter.
4. J USTIFICATION FOR A TTENTION
Now we investigate the attention module and demonstrate how it actually helps to train a more robust model.
As we discussed in Section 1, the intuition for using attention is to assign larger weights on more robust features and smaller weights on less robust features. We now show that this is in fact true by examining the attention weights relative to the feature robustness. Figure 2 shows the relationship between the robustness of a feature and the magnitude of its assigned attention weight.
The robustness measure we use is the L 2 distance between the learned features of a clean and an adversarial image, i.e. the smaller the distance between the features, the more invariant the feature is against input perturbations, therefore the feature is more robust. For each plot in Figure 2, the horizontal axis is the rank of feature robustness, with 0 being the most robust and 63 the least robust, and the vertical axis is the corresponding attention weights. The plot on the left shows the average attention weights for all training images and the plot on the right is for all test images in CIFAR-10. We can see that our proposed attention mechanism indeed assigns larger weights to more robust features and less weights to non-robust features, so the model is more invariant to adversarial perturbation.
Next we show the attention maps of our model in Figure 3 to visualize the attention weights. The attention maps focus sharply on the objects in the images, and the most relevant features like the head and legs of an animal and the wings of an airplane contribute more to the model’s prediction.
4. G RADIENT M AP
In this section we study the gradient maps, which are the gradients of the cross-entropy loss with respect to the input image pixels, for both the baseline and our model on CIFAR-10.
The gradient maps are a direct indicator on how the input features are utilized by a model to produce a ﬁnal prediction.
A large magnitude of the gradient on an input feature signiﬁes a heavy dependence the model has.
Human vision is robust against small input perturbations and the perception of an image reﬂects which input features contribute to the human vision robustness. At the same time, the gradient maps of a robust model also highlight the input features which affect the loss most strongly, therefore more robust models depend on robust features and will be better aligned with human [Figure] [Figure]
vision. So the alignment of gradient maps with the image can be used to evaluate the robustness a model.
Next, we show that the gradient maps generated from our model align better with the salient data characteristics by evaluating the gradient maps both qualitatively and quantitatively.
First we present the qualitative result.
Figure 4 contains the gradient maps from Madry et al.
(2017) and our model.
These are raw gradients with only being clipped and rescaled for visualization.
Overall, we note that both models generate highly interpretable gradient maps that align very well with the image features.
However, upon carefully inspecting the images, it is evident that the gradient maps generated from our model are better than the baseline model.
To point out a few: in columns 2, 3, 6 and 9, our gradient maps have cleaner backgrounds and the gradients only focus on the objects; especially in column 6, the baseline model has large gradients on the text ﬁeld in the background which is irrelevant to the class label (automobile), while in our model gradients in that area are much more suppressed.
In columns 1, 5, and 10 the edges of the faces and heads of the animals are depicted clearer in our model.
Next, we introduce a quantitative evaluation method for gradient maps.
The problem we consider here is to decide how well the gradient maps align with the original images.
The better they align, the more recognizable the gradient images are.
Therefore, in addition to human inspection which could be very subjective, we propose to use a pretrained neural network to classify the gradient maps for all images in the dataset, both the training set and the test set. A standard neural network extracts relevant features from the inputs and make predictions based on the features.
When a gradient map is highly aligned with the original image, the neural net is able to identify more relevant features and thus the classiﬁcation accuracy will be higher.
Therefore, with the classiﬁcation accuracy of gradient maps from both models of all images we are able to quantitatively compare the alignment.
The pretrained model we use is the same ResNet model as in Section 4.2 trained with only natural training data of CIFAR-10.
It achieves an accuracy of 88.79% on the test set.
The classiﬁcation results are presented in Table 4.
To avoid the possible inﬂuence of gradient clipping we also run the evaluation on raw gradients.
As demonstrated by the classiﬁcation accuracy, the gradient maps from our model express signiﬁcantly better alignment with the original images.
[Table]
To summarize [Table] both the qualitative and quantitative results show that the gradient maps from our model have better interpretability and alignment with the original images.
It suggests that our model depends on features that are very closely correlated with the robust features of the input images which explains the improved performance on both standard accuracy and adversarial robustness.
4. R ESULTS ON CIFAR-
Here we present our results on the CIFAR-100 dataset. The experiment setup is the same as CIFAR10 in Section 4.2. [Table] [Table]
White [Table] [Table] [Table]
Black
[Table]
[Table]
C ONCLUSION
In this paper we propose feature prioritization and regularization to enhance the adversarial robustness.
With the non-linear attention module and L 2 feature regularization, a model is improved on both the standard classiﬁcation accuracy and the adversarial robustness over the baseline adversarial training approach.
We provide additional justiﬁcations for the attention module to show that it effectively favors robust features, and study the attention maps to demonstrate that the attention maps focus sharply on the region of interest. We then conduct quantitative and qualitative evaluation on gradient maps and show that they align perfectly with salient data characteristics, and therefore our model heavily relies on the robust features.