Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attacks
Abstract
A common observation regarding adversarial attacks is that they mostly give rise to false activation at the penultimate layer to fool the classiﬁer.
Assuming that these activation values correspond to certain features of the input, the objective becomes choosing the features that are most useful for classiﬁcation.
Hence, we propose a novel approach to identify the important features by employing counter-adversarial attacks, which highlights the consistency at the penultimate layer with respect to perturbations on input samples.
First, we empirically show that there exist a subset of features, classiﬁcation based in which bridge the gap between the clean and robust accuracy.
Second, we propose a simple yet efﬁcient mechanism to identify those features by searching the neighborhood of input sample.
We then select features by observing the consistency of the activation values at the penultimate layer.
1. Introduction
Despite their remarkable performance in a wide range of real world problems, deep neural networks (DNNs) have been shown to be vulnerable to adversarial attacks, where a small perturbation to the input data can fool the network (Bruna et al., 2014; Goodfellow et al., 2015; Carlini & Wagner, 2017).
Consequently, there has been a lot of work in building robust models against these adversarial examples (Madry et al., 2018; Zhang et al., 2019;
Shafahi et al., 2019; Qin et al., 2019; Sehwag et al., 2020; Wu et al., 2020;
Gowal et al., 2020).
The most successful approach to building robust DNN models is based on adversarial training (AT) (Madry et al., 2018), where a network is trained on (approximate) worst-case adversarial examples (often generated by iteratively maximizing some loss function).
the
Accepted by the ICML 2021 workshop on A Blessing in Disguise: The Prospects and Perils of Adversarial Machine success of AT in increasing the robustness, the performance gap between the robust accuracy (adversarial examples as input) and the clean accuracy (non-adversarial inputs) of an adversarially trained model is still quite large (Madry et al., 2018; Zhang et al., 2019;
Tsipras et al., 2019; Yang et al., 2020).
Several modiﬁcations of AT have been proposed to improve robustness, and to bridge the gap between the robust and clean accuracies e.g., by label smoothing and stochastic weight averaging (Chen et al., 2021), by using additional unlabelled data (Carmon et al., 2019), and by employing different activation functions (Xie et al., 2020).
However, these techniques result in marginal increases in robust accuracy, and bridging the gap between robust and clean accuracies has so far remained elusive.
In a parallel line of research, it has been argued that adversarial examples are actually characteristics of the datasets used for training (Ilyas et al., 2019).
That is, the datasets contain samples with both robust and non-robust features, and adversarial examples exist due to the presence of these non-robust features, which are exploited by the classiﬁer to improve its accuracy when trained on the dataset.
It has been further shown that an adversarially trained network (indirectly) limits the effect of these non-robust features.
Based on this observation, minimizing the inﬂuence of non-robust features could pave the way to building a robust model, but until recently very little work is done in this area (Bai et al., 2021; Yan et al., 2021; Xie et al., 2019).
Note that, feature selection is not a new problem in machine learning (Chen et al., 2018a; Gao et al., 2016; Shrikumar et al., 2017; Brown et al., 2012); however, existing schemes are generally used for model interpretability and have not been investigated for robustness against adversarial attacks.
We argue that the key limitation of AT as a defence mechanism is being oblivious to intrinsic properties of the observed sample.
By intrinsic properties we refer to the impact of perturbations in the image domain to the distribution of the activation values at the penultimate layer.
Assuming that each activation value corresponds to a different feature, the change in the distribution of the activation values due to variations within a close neighbourhood of the image provides an insight on their consistency, and helps to identify the common features in the neighbourhood.
Hence, we argue that enforcing the classiﬁer to perform predictions using only these common features will enhance its robustness.
To achieve this, we propose a latent masking approach for feature selection that conditions the classiﬁer, such that, based on the additional side information obtained by searching the close neighborhood of an input sample, the classiﬁer utilizes only a subset of the activation values at the penultimate layer.
The side information acts as a certain consistency measure on the activation values.
We remark here that the majority of existing works approach the feature selection and activation masking problem by either identifying the robust features in the image domain or by analyzing the importance of each feature for the prediction of each class.
Instead, our aim is to search for a robust representation at the penultimate layer for each image, which can be considered as a consensus representation for all the images within the close neighbourhood of the original image.
2. Preliminaries 2.1. Adversarial Training (AT)
Let x ∈ R n denote the input data that we want to classify to a set of labels Y = { 1 , 2 , . .
, Y } .
We deﬁne the classiﬁer as a score function F θ : R n × Y → R , which assigns label ˆ y ∈ arg max y ∈Y F θ ( x , y ) to x , where θ ∈ Θ denotes the parameters of this score function.
With a slight abuse of notation, we also use F to denote the classiﬁer, F ( x ) to denote the label assigned to x , and F ( x , y ) to denote the score of class y for input x .
For a DNN architecture, F consists of L layers, i.e., F = f L ◦ f L − 1 ◦ . . .
◦ f 1 , where the last layer f
L is a fully connected layer.
Let x be a correctly classiﬁed input, i.e., y = F ( x ) is the true label.
Furthermore, let B (cid:15) ( x ) denote the L p -norm ball of radius (cid:15) centered at x , i.e., B (cid:15) ( x ) =
{ ˜ x : (cid:107) ˜ x − x (cid:107) p ≤ (cid:15) } .
An adversarial attack on this classiﬁer F aims to modify the input x to ˜ x ∈ B (cid:15) ( x ) , such that F (˜ x ) (cid:54) =
F ( x ) .
In practice, these adversarial examples are generated by optimizing some loss function L on the classiﬁer (Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016; Chen et al., 2018b;
Laidlaw & Feizi, 2019; Madry et al., 2018).
AT has become de facto defense strategy against adversarial attacks, which can be considered as a two-player game formulated as the following min-max optimization problem:
[Equation]
where ˜ x is the adversarial sample.
Hence, during training, for each clean sample x , ﬁrst an adversarial sample that maximizes the loss L is chosen from the ball B (cid:15) ( x ) , then the network is trained based on the adversarial sample for robustness at inference time.
2.2. Adversarial Examples and Activation Values
The impact of adversarial examples on the activation values in a DNN is previously studied in (Bai et al., 2021; Yan et al., 2021; Xiao et al., 2020).
Let z ( x ) denote the activation values at the penultimate layer for the input sample x , i.e.,
[Equation]
As highlighted in (Bai et al., 2021; Yan et al., 2021), when an adversarial example ˜ x is fed to network, one can observe a signiﬁcant change in the distribution of the activation values in the penultimate layer compared to x .
By detecting and regulating these variations in the distribution of the activation values the robust accuracy can be improved (Bai et al., 2021; Yan et al., 2021).
We want to emphasize that previous works approached this problem by analyzing the class-wise importance , while we consider sample-wise consistency .
To be more precise, previous works try to design an importance mask m for z (˜ x ) , such that m i denotes the importance of the activation valuez (˜ x )
i for the prediction of a certain class.
On the other hand, our objective is to measure the consistency of the activation values and design the mask accordingly.
To clarify, let σ i be deﬁned as σ i :=
| z (˜ x ) i − z
( x ) i | , where ˜ x ∈ B (cid:15) ( x ) , then by sample-wise consistency we refer to a strategy where mask m is designed according to σ i values, particularly by choosing m i inversely proportional to σ i for each image separately, independent from its class prediction.
In the next section, we empirically show how such consistency measure could help to match the clean and robust accuracies in AT.
3. Matching Clean and Robust Accuracies
Assume that there exists an oracle o , which can access the clean sample x and the model θ , and hence, can obtain σ for a sample ˜ x ∈ B (cid:15) ( x ) i.e., σ = o (˜ x ; x , θ ) .
Further, assume that the oracle shares only a partial side information m (˜ x ) , where
[Equation]
Here, S topk ( u ) maps u ∈ R
d to m ∈ { 0 , 1 } d such that | m |
= k
and m i
= 1 if u i is one of the k-largest values in u .
Now, we argue that even side information m (˜ x ) might be sufﬁcient to bridge the gap between the clean and robust accuracies.
To verify this, we consider image classiﬁcation on CIFAR-10 dataset (Krizhevsky, 2009) with the
ResNet-18 model (He et al., 2016) and use side information 1 d − m (˜ x ) , where 1 d is a d -dimensional vector of ones, directly as a mask for the activation values in the penultimate layer z
(˜ x ) .
For AT, we follow (Madry et al., 2018) and use projected gradient descent ( PGD ) attack for 10 steps denoted by PGD 10 ; see Appendix A for full details.
Since classiﬁcation is performed based on ( 1 d − m (˜ x )) ⊗ z (˜ x ) , we refer to this strategy as latent masking (LM) and the one without any side information as proposed in (Madry et al., 2018) as standard adversarial training (SAT).
We take k = 50 in Eq.
(3) for estimating m .
To evaluate the robustness we employ the PGD attack with 20 steps, denoted as PGD 20 , and the results are shown in Table 1.
Note that, for inference we apply the same mask m to both
[Table] [Table]
the clean and adversarial samples.
We observe that when the side information m is used both during AT and the inference phase, robust accuracy increases by 33 − 34% compared to
SAT.
We also observe that clean and robust accuracies match at around 83 − 84% .
This observation empirically supports our claim that using the intrinsic information m on the consistency of the activation values before the classiﬁcation step can help to match the clean and robust accuracies.
Next, we perform a complementary experiment to understand the impact of LM on the training and inference phases separately, and the result is shown in Table 2.
When LM is employed during training but not at inference, we observe a drop in both clean and robust accuracies compared to when
LM is employed for both (cf.
Table 1).
Hence, using LM during training alone is not sufﬁcient for robustness.
However, when we train the network with SAT and apply LM only during inference, we observe that the clean accuracy does not change signiﬁcantly, while the robust accuracy drops by around 8% compared to SAT.
This indicates that with SAT certain latent feature values are useful for the adversarial samples, but may act as noise for clean data.
Hence, when LM is not used during training, the network tries to memorize the non-consistent latent features.
On the other hand, using LM during training enforces the network to focus on the latent features that are correlated between the clean and adversarial data.
4. Side Information with Self-Supervision
We recall that, to obtain LM, we measure the consistency at the penultimate layer
z (˜ x | θ ) by employing an oracle that can access the clean sample z ( x | θ ) as a reference point.
Here we try to address the question whether we can generate a “good” reference sample ˆ x from the observed sample ˜ x to obtain the mask m without an oracle, i.e.,
[Equation]
This leads to two main challenges; deﬁning a good reference sample ˆ x , and generating it from the observed sample ˜ x .
For this, let I c
and I nc represent the set of indices for consistent and non-consistent activation values, respectively, at the penultimate layer of a given clean sample x .
An activation value i is considered consistent, i.e., i ∈
I
c , if
[Equation]
(ˆ
[Equation]
i
[Equation]
for any pair ˆ x , ˜ x ∈ B (cid:15) ( x ) for some small β value.
Further, we assume that we do not know I c
but only its cardinality, which is d − k .
Under these assumptions, if there exists a ˜ x s ∈ B (cid:15) ( x ) such that
[Equation]
then one can directly use z (˜ x s | θ ) instead of masking the latent values in z (˜ x | θ ) .
Even though such an ˜ x s may not exist, it may still be useful to search for an ˜ x s with a sparse representation at the penultimate layer z
(˜ x s | θ ) to use as the reference sample, since adversarial samples increase the non-consistent activation values.
Now, to search for ˜ x s , we deﬁne a loss function L s (˜ x , θ ) =
| z (˜ x | θ ) | 1 and consider a gradient based approach to update the observed sample iteratively, i.e., starting from ˆ x 0 = ˜ x ,
[Equation]
) (ˆ [Equation] [Equation] 7) where Π B (cid:15) (˜ x ) is the projection operator, sgn denotes the sign operator, and α is the step size.
Hence, ˆ x can be generated from ˜ x at inference time by using (7), and we call this procedure as compressive counteradversarial attack (CCA).
By employing CCA, mask m can be obtained by using Eq. 4, and this overall strategy is denoted as LM with CCA (LM-CCA), where during training clean samples are used as reference samples for LM, and during inference a reference sample is obtained by selfsupervision only using the observed sample.
LM-CCA utilizes a ﬁxed masking parameter k for all the samples; however, the number of non-consistent features may depend both on the sample itself and the adversarial attack.
To overcome this issue, we propose to use the sample obtained by CCA, i.e., ˆ x , directly for inference.
We show that this strategy, which we refer to as latent compression with CCA (LC-CCA), can also help to increase the robust accuracy compared to SAT.
Note that, in both LM-CCA and LC-CCA, the sample ˆ x obtained by CCA may not be inside the ball B (cid:15) ( x ) , which may limit its effectiveness.
We investigate this limitation further in Appendix B using the oracle, where we perform additional experiments such that the sample obtained by CCA is always projected to B (cid:15) ( x ) .
Appendix C further extends this to practical scenarios and we observe signiﬁcant improvements in both the clean and robust accuracies, which highlights the advantage of using compressed representation at the penultimate layer.
Note that, in general, cross-entropy loss L CE is used for training DNNs, however it makes the classiﬁer overconﬁdent on the target probabilities (Hein et al., 2019;
M¨uller et al., 2019; Szegedy et al., 2016).
This overconﬁdence is particularly an issue for LM since it corresponds to feature selection and the selected features may
[Table]
change for the samples belonging to same class.
This happens because when we use L CE in LM we enforce the classiﬁer to focus on particular features for each sample.
To mitigate this over-conﬁdence we employ label smoothing , where we use the target probabilities ˜
y , i.e., ˜ y
=
(1 − γ ) y
+ γ ( 1 /c ) , where c is the number of classes, y is a one hot vector of true target class, and γ is the label smoothing parameter.
5. Experimental Evaluation
We train the ResNet-18 model with different conﬁgurations of the LC-CCA and LM-CCA on CIFAR-10 and CIFARdatasets (Krizhevsky, 2009), and similarly to (Rice et al., 2020) we reserve 1000 images from the training set for validation; see Appendix A for full details.
We use PGDattack during training and PGD 20 at inference.
We generate counter adversarial samples using Eq.
Table 3 and Table 4 show the robust and clean accuracies for CIFAR-10 and CIFAR-100 datasets, respectively, for the last training epoch.
We observe that both of the proposed
LM-CCA and LC-CCA schemes can achieve signiﬁcant improvements of 16% and 7 .
3% , respectively, in robust accuracy compared to SAT on CIFAR-10 dataset.
However, we also observe some reduction in the clean accuracy, e.g., 2 . 5% and 2 .
7% drop for LM-CCA and LC-CCA, respectively.
Furthermore, we observe that the label smoothing parameter γ signiﬁcantly impacts both the clean and robust accuracies.
Although ﬁnding an optimal γ is out of the scope of this work, from the results we consider γ = 0 .
1 and γ = 0 .
2 as reasonable choices for LM-CCA and LC-CCA, respectively.
Additional results with different γ values can be found in Appendix D. We also compare our results with a state-of-the-art scheme for suppressing activation values (Yan et al., 2021), denoted by CIFS.
We observe that the proposed schemes surpass CIFS in robustness evaluation.
We also provide additional results for our proposed approaches using early stopping on the validation set in Appendix D.
Figure 1 shows the convergence behaviour of the LMCCA and LC-CCA schemes, respectively, on the CIFARdataset.
We observe that LM directly on the activation values (LM-CCA) can increase the instability of training compared to a natural compression strategy in LC-CCA, and we leave further investigation of this as future work.
[Figure]
[Table]
6. Conclusion
In this work, we showed how a consistency -based LM strategy, which corresponds to feature selection, can signiﬁcantly increase the robust accuracy, and can even match the clean and robust accuracies under supervision of an oracle.
Robust feature selection is not a new concept in the literature, but the novel aspect of our work is the alternative method devised to identify the robust features by analyzing the consistency at penultimate layer with respect to the perturbations to the input sample.
Furthermore, we also introduced the concept of compressive counter-adversarial attack that suppresses the activation values in a self-supervised manner during the inference phase to verify the consistency of the activation values.
A. Experimental Setup
For all experiments, we use ResNet-18 models (He et al., 2016) which are trained using SGD with momentum value of 0.9, weight decay of 5 × 10 − 4 and an initial learning rate of 0.1, which is divided by 10 at the 75 th and 90 th epochs with a total training of 120 epochs.
Clean images are normalized between values of 0 and 1 and augmented with horizontal ﬂip and random crop, and for label smoothing we consider smoothing parameter values γ ∈ { 0 , 0 .
We set the (cid:15) = 8255 in L ∞ norm (maximum perturbation) and step size to 2255 for PGD attack for all the experiments.
For counter adversarial attack, we consider (cid:15) and step size same as PGD attack and number of steps are 10 for all experiments.
B. Additional Results with Oracle
In this section we provide additional results for LM with oracle.
We follow the same experimental setup, but now we try to observe the impact of the parameters k and γ and discover the limits on the performance of the latent masking strategy.
We consider the parameters k = 50 , 75 , 100 and γ = 0 , 0 .
15 and the corresponding results are shown in Table 5 for CIFAR-10 dataset.
We observe that when k = 50 and γ = 0 . 1 , both clean and robust accuracy can be as high as 89% .
Similarly, Table 6 for CIFARdataset shows a high robust accuracy and clean accuracy for γ = 0 and k = 100 .
These results clearly highlights why feature selection is a promising direction for designing defence strategies against adversarial attacks.
We also argue that the key limitation of the latent compression strategy with CCA is that the image obtained with the counter adversarial attack may end up outside the ball B (cid:15) ( x ) and this may limit the prediction accuracy.
To verify that we
[Table] [Table]
can further improve the test accuracy if ˆ x i.e., the sample obtained by CCA can be projected back to ball B (cid:15) ( x ) , we consider an experiment setup with oracle o such that the oracle o projects ˆ x obtained by CCA into B (cid:15) ( x ) , and corresponding results for CIFAR-10 and CIFAR-100 datasets are shown in Table 7 and Table 8, respectively.
It can be seen that as long as one can guarantee that the samples obtained through CCA stays within ball B (cid:15) ( x ) , then the compression strategy can efﬁciently cancel the false activation of the adversarial sample as we can achieve up to 80% robust and 90% clean accuracy simultaneously for CIFAR-10 dataset.
We obtain similar results for CIFAR-100 dataset and these results indicate that the main limitation of the proposed LCCCA scheme is ending up with a sample ˆ x that is outside the ball B (cid:15) ( x ) .
C. Extensions of LC-CCA
Based on superior results that we observed with LC-CCA under the oracle assumption, we propose a further extension
[Table] [Table]
of the LC-CCA scheme
such that during the training and inference phases we use a slightly different implementations of LC-CCA.
More precisely, we only change it during training where we already employ the oracle that utilizes the information from clean sample.
Recall that the counteradversarial attack searches for a sample ˆ x with a sparse representation based on the observed sample ˜ x iteratively in the following manner
[Equation]
starting from ˆ x 0 = ˜ x .
We remark that since ˆ x ∈ B (cid:15) (˜ x ) and ˜ x ∈ B (cid:15) ( x ) , || x − ˆ x || ∞ might be as high as 2 (cid:15) .
Thus, while suppressing the false activation values CCA may also damage useful features.
To overcome this issue, we replace the projection Π B (cid:15) (˜ x ) used in Eq.
(8) with Π B (cid:15) ( x ) during the training phase.
This strategy is particularly helpful for increasing the clean accuracy as shown in Tablefor CIFAR-10 dataset.
We denote this particular strategy with LC-CCA (cid:63) .
The results show that by employing CCA supervised by oracle during training we can improve both the clean and robust accuracy such that LC-CCA (cid:63) achieves 55.43% robust and 89.85% clean accuracy simultaneously.
We also observe that, when the LC-CCA (cid:63) is employed it is more beneﬁcial to use smaller label smoothing parameter γ .
Based on this observation, we further consider a strategy where we use a projection with extra margin parameter δ during the training i.e., we use the projection Π B (cid:15)
+ δ ( x ) in
Eq. 8.
We refer to this approach as LC-CCA (cid:63)δ .
We also perform experiments with LC-CCA (cid:63)δ by taking δ = 2255 and observe that robust accuracy increases around 4% and 12% compared to the LC-CCA (cid:63) and SAT respectively, while losing only 1% in clean accuracy compared to LC-CCA (cid:63) and still 4% higher compared to SAT.
[Table]
D. Additional Results For CIFAR-10 and CIFAR-100 Datasets
In this section [Table] we present additional results for different values of label smoothing parameter γ for both LC-CCA and LM-CCA schemes in Table 10 for CIFAR- [Table]
and in
Table 11 for CIFAR-100, respectively.
We also show the best robust and clean accuracy results for the CIFAR-10 in
Table 12 and CIFAR-100 in Table 13.
Similarly to (Rice et al., 2020) [Table] we measure best accuracy according to the validation set.
From Table 12, both LM-CCA and LC-CCA frameworks’ results show consistent trade off between robust and clean accuracy which is also inline with the Table 3.
[Table]
We also plot the convergence behavior of the proposed LMCCA and LC-CCA strategies for the CIFAR-100 dataset to highlight trade-off for different conﬁgurations in Figure 2 .
One can easily observe that convergence of the LM-CCA strategy exhibits certain instability, while LC-CCA exhibits comparatively smooth convergence behavior.
However, LMCCA has a better classiﬁcation accuracy since LC-CCA may eliminate some useful features as well.
We conjecture that a hybrid approach that combines both strategies such that instead of masking certain values completely, we can instead use their compressed values (to prevent instability that we observe in case of LM-CCA) will work better and
[Table]
[Table]
we leave it as a research direction for future work.
We further demonstrate the robustness of the proposed LCCCA and LM-CCA by comparing against the PGD 20 and
PGD 40 attacks for both CIFAR-10 and CIFAR-100 datasets in Table 14 and in Table 15 respectively.
We can see that accuracy against both PGD 20 and PGD 40 attacks is almost identical for the proposed frameworks.
E. Visualization of the Compressive Counter-Adversarial Attack
To illustrate the impact of the compressive counteradversarial attack, we plot the clean and adversarial images and compare them with image that is obtained by the compressive counter-adversarial attack.
Furthermore, we do the same comparison on the distribution of the activation values at the penultimate layer.
For this, we consider four images from the test set that are misclassiﬁed under adversarial perturbation and classiﬁed correctly when CCA is employed.
This images along-with the activation values at the penultimate layer are shown in Figures 3, 4, 5, 6.
Based on Figures 4 and 5, subjectively, it is not possible to argue visual improvement with CCA.
However, we observe an interesting behavior on the distribution of the activation values on the penultimate layer, that is CCA makes the distribution of the activation values closer to the those observed
[Table]
[Figure]
with clean image.
To show this correlation quantitatively, we measure the cosine similarity between the activation values of clean and adversarial image as well as the clean image and the image obtained after CCA.
We observe that the image with CCA exhibits higher cosine similarity with the clean image.
Hence, the compression strategy on the penultimate layer not necessarily work as a puriﬁcation strategy on the image, but as a regularizer for the distribution of
[Table]
[Table]
the activation values on the penultimate layer by suppressing false activation values.
[Figure] [Figure]
[Figure] [Figure]