D ELVING INTO F EATURE S PACE : I MPROVING A DVERSARIAL R OBUSTNESS BY F EATURE S PECTRAL R EGULARIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
The study of adversarial examples in deep neural networks has attracted great attention.
Numerous methods are proposed to eliminate the gap of features between natural examples and adversarial examples.
Nevertheless, every feature may play a different role in adversarial robustness.
It is worth exploring which feature is more beneﬁcial for robustness.
In this paper, we delve into this problem from the perspective of spectral analysis in feature space.
We deﬁne a new metric to measure the change of features along eigenvectors under adversarial attacks.
One key ﬁnding is that eigenvectors with smaller eigenvalues are more non-robust, i.e., adversary adds more components along such directions.
We attribute this phenomenon to the dominance of the top eigenvalues.
To alleviate this problem, we propose a method called Feature Spectral Regularization (FSR) to penalize the largest eigenvalue, and as a result, the other smaller eigenvalues get increased relatively.
Comprehensive experiments demonstrate that FSR is effective to alleviate the dominance of larger eigenvalues and improve adversarial robustness on different datasets.
Our codes will be publicly available soon.
I NTRODUCTION
It is shown that the performance of Deep Neural Networks (DNNs) decreases dramatically when confronted with adversarial examples (Biggio et al., 2013; Szegedy et al., 2013; Goodfellow et al., 2015).
The vulnerability of DNNs brings potential risk to safety-critical deployments (Finlayson et al., 2019).
To mitigate this vulnerability of DNNs, numerous methods are proposed to improve adversarial robustness (Papernot et al., 2016b; Madry et al., 2018; Xie et al., 2019).
Among these, adversarial training (AT) (Madry et al., 2018) is the most effective approach that achieves state-ofthe-art performance under various attacks (Croce et al., 2020).
Different from standard training, adversarial training trains DNNs on adversarial examples rather than natural examples.
There are numerous methods proposed to eliminate the gap of features between natural examples and adversarial examples (Kannan et al., 2018; Zhang & Wang, 2019; Zhang et al., 2019).
However, they constrain the features on the whole without considering the distinction of contribution from an individual feature.
This may be inappropriate since every feature may play a different role in robustness.
The work of (Ilyas et al., 2019) argued that adversarial examples result from non-robust features (Ilyas et al., 2019), i.e., well-generalizing but brittle features.
This inspires us that we could treat each feature separately in the adversarial setting.
While the work of (Bai et al., 2021; Yan et al., 2021) considered the inﬂuence of different channels on robustness, the problem has not been explored clearly from the perspective of spectral signatures, i.e., the eigenvalues and eigenvectors of feature covariance.
The space is split into many spectral components, and then it is worth analyzing which component is beneﬁcial for adversarial robustness and which is fragile under attack.
In this paper, we show that spectral signatures of deep features have a close connection with adversarial robustness.
Through applying principal component analysis (PCA) to deep features, we could split feature space to various eigenvectors with its eigenvalues.
Our motivation comes from the phenomenon that the standard trained model often results in a sharp distribution of eigenvalues (Yu et al., 2020), i.e., the eigenvalues rapidly become very small as shown in Figure 1.
This property may be beneﬁcial for natural accuracy (Lezama et al., 2018; Papyan et al., 2020), while its
[Figure]
impact on robust generalization is hypothesize sharp distribution of eigenvalues impels models to learn less diverse features and is a cause of the vulnerability of DNNs.
A minority of eigenvalues occupy the overwhelming majority in the sum of eigenvalues, which may make the model pay little attention to features along the eigenvectors with smaller eigenvalues, so features along such directions are not generalized during training.
To verify our hypothesis, we deﬁne a new metric to measure the variation of features along different eigenvectors under attacks, as shown in Figure 4.
Our observation reveals that the adversary tends to add more components along eigenvectors with smaller eigenvalues, and such huge variation could be distinctly alleviated by AT.
Therefore, we propose to improve adversarial robustness by alleviating the sharp distribution of eigenvalues.
Considering that more eigenvalues should occupy the majority, we propose a regularizer named Feature Spectral Regularization (FSR) to penalize the largest eigenvalue of the feature matrix covariance.
Empirical evidence shows that FSR increases the overall eigenvalues relatively, making models focus on more spectral components during training.
We also provide a theoretical explanation with robust linear regression.
Comprehensive experiments conﬁrm that FSR indeed improves the adversarial robustness on several datasets.
Our contributions are summarized as follows:
We ﬁnd a close connection between spectral signatures of features and adversarial robustness.
On one hand, standard trained model presents a sharp distribution of eigenvalues, which is beneﬁcial for natural accuracy while harmful in adversarial setting.
On the other hand, the adversary tends to add more quantity along eigenvectors with smaller eigenvalues.
• We propose Feature Spectral Regularization (FSR) to increase the overall eigenvalues relatively in deep feature space, thus alleviating the sharp distribution of eigenvalues.
Furthermore, we provide a theoretical explanation based on robust linear regression.
• We empirically show that FSR improves adversarial robustness and alleviates the sharp distribution of eigenvalues, through comprehensive experiments.
R ELATED W ORK
Adversarial Defense.
Many defense methods have been proposed to improve adversarial robustness since the discovery of adversarial examples (Papernot et al., 2016b; Xie et al., 2019; Carmon et al., 2019; Zhang et al., 2020).
However, many of them are proven to be noneffective because they highly depend on obfuscated gradients (Athalye et al., 2018) or gradient masking (Papernot et al., 2017).
Among these, adversarial training (Madry et al., 2018) is now regarded as the state-of-theart method (Rice et al., 2020; Pang et al., 2021).
Distinguished form standard training, adversarial training trains DNN on adversarial examples:
[Equation]
(1) where, D is the dataset, the parameters of DNN are denoted as θ , δ means the perturbation within the (cid:15) -ball, and L CE is the cross-entropy (CE) loss.
By introducing a trade-off between robustness and generalization, TRADES (Zhang et al., 2019) is another framework that reaches comparative robustness with AT.
Among the proposed methods based on AT, Adversarial Weight Perturbation (AWP) (Wu et al., 2020) explicitly regularizes the ﬂatness of weight loss landscape, and forms a double-perturbation mechanism, which shows huge improvement on adversarial robustness and alleviates the overﬁtting in AT (Rice et al., 2020).
Spectral Signatures of Feature Representations.
Some studies have revealed that the spectral signatures of features inﬂuence the performance in various learning tasks.
The spectral properties are crucial to detect backdoors (Tran et al., 2018; Hayase et al., 2021).
The eigenvectors corresponding to the larger eigenvalues are found to dominate the transferability of features in adversarial domain adaptation (Chen et al., 2019b).
The work of (Chen et al., 2019a) explores the correlation between negative transfer and the spectral components of features and weights in inductive transfer learning.
By utilizing the principle of Maximal Coding Rate Reduction (MCR 2 ), it is theoretically proven that the larger several singular values of feature matrix for every class should be equal to learn the maximally diverse representation (Yu et al., 2020; Chan et al., 2020).
Different from these studies, we analyse the connection between adversarial robustness and spectral components of deep features.
We aim to explore which components are more fragile under attacks, and propose a method to boost adversarial robustness by constraining spectral properties.
S PECTRAL A NALYSIS IN F EATURE S PACE
In this section, we investigate the connection between spectral signatures and adversarial robustness.
Concretely, we train models by standard training and adversarial training (Madry et al., 2018), and then apply spectral decomposition to attain the spectral signatures.
We ﬁnd that in standard training the eigenvalues reveal a rapidly descending curve while this tendency is alleviated by AT.
Then, we ﬁnd that the adversary tends to add more quantity along eigenvectors with smaller eigenvalues.
3. C URVE OF E IGENVALUES AND A DVERSARIAL R OBUSTNESS
Given a dataset D = { ( x i , y i ) } ni =1 including C classes, x i represents the input data and y i is the label.
DNN is composed of a feature extractor h ( · ) : R D → R d and a linear classiﬁer g ( · ) :
R d → R C .
After centralizing the learned features ( i.e. 1 n (cid:80) ni =1 h ( x i ) = 0 ), we decompose the learned features by spectral decomposition, which is similar to PCA:
[Equation]
where λ j means the eigenvalues with index j and u j ∈ R d represents its eigenvector.
Speciﬁcally, we train ResNet-18 (He et al., 2016) using both standard training and adversarial training on CIFAR-10.
The parameters for AT are the same as (Rice et al., 2020).
We calculate the eigenvalues by applying Eq.
(2), and the eigenvalues are plotted in Figure 2.
The features come from the penultimate layer (512 dimensions).
All the features are extracted from the test set in
CIFAR-10.
A part of the eigenvalues is shown for better visualization.
[Figure]
intrinsic dimension (ID) of features.
The sharp distribution of eigenvalues in standard trained model leads to a lower ID, while ID becomes higher by imposing adversarial training and FSR.
Difference of models in spectral analysis.
As shown in Figure 2(a)(b), the eigenvalues of a standard trained model drop rapidly at some point, while the sharp decrease of eigenvalues is much alleviated by AT.
These sharp spectral signature in standard training makes just a few eigenvalues informative from the opinion of
PCA.
The model fails to explore the inﬂuence of eigenvectors with smaller eigenvalues on classiﬁcation, so the trained model could not recognize the change of features along eigenvectors with smaller eigenvalues.
The eigenvectors which may endow useful features are overly penalized.
Consequently, we propose a hypothesis that the severe dominance of the top eigenvectors is a cause of vulnerability in DNN, and the adversary adds more components in eigenvectors with smaller eigenvalues.
We will verify the proposed hypothesis in the next section.
Connection with intrinsic dimension.
We introduce intrinsic dimension (ID) to quantitatively describe the decreasing tendency in eigenvalues.
ID (i.e., the minimal number of parameters needed to describe a representation) has a close connection with natural accuracy (Ansuini et al., 2019).
It has been found that reduction of ID contributes to an improvement on natural accuracy.
We adopt
PC-ID proposed by (Ansuini et al., 2019) to estimate ID, which is determined by the number of principal components included to describe 90% of the variance.
We mark ID in Figure 2 with solid circles.
The results reveal the ID of a standard trained model is very small, while models obtained by AT achieve a higher ID, which is contrary to the inﬂuence of ID on natural accuracy.
This also veriﬁes that there exists a trade-off between generalization and robustness (Tsipras et al., 2019;
Zhang et al., 2019) from the perspective of ID.
FSR could further increase ID, based on AT.
3. V ARIATION ALONG E IGENVECTORS UNDER A TTACKS
In this section, we aim to verify the hypothesis that adversary adds more components along eigenvectors with smaller eigenvalues in attacking stage.
We deﬁne a metric to quantitatively observe the change of features along different eigenvectors under attack, called variation in this paper.
Deﬁnition 1 (Alignment) Given a dataset D s = { x s,i , y s,i } n i =1 which may be perturbed.
The alignment of D s to the pre-given direction u j is calculated by the expectation over cosine similarity between features extracted by DNN and the direction vector u j :
[Equation]
where the norm (cid:107)·(cid:107) used is Euclidian norm, and u j is calculated by Eq.
The calculation of u j is based on features covariance of natural samples, and u j is ﬁxed.
Deﬁnition 2 (Variation)
Given a dataset D consist of natural examples and its perturbed dataset
D adv .
The variation on direction u j is deﬁned as the ratio between alignment on D adv and D :
[Equation]
The alignment is correlated with the distance between subspace spanned by u j and the actual feature space, so the change of alignment under attack is suitable to describe the inﬂuence of attacks on direction u j , which is called as variation .
Our metric is similar to (Tran et al., 2018; Hayase et al., 2021) used for analyzing backdoors, but we deﬁne the alignment by cosine similarity while the latter uses the inner product.
Compared with inner product, cosine similarity could eliminate the inﬂuence of scale .
In Appendix B.1, we provide a complete procedure to calculate the metrics.
[Figure]
[Equation]
[Figure]
We give an intuitive explanation on why the deﬁned variation is suitable to capture the change of features along different eigenvectors under attacks.
An toy model for illustration is shown in Figure 3.
Suppose the distribution transfers from Figure 3(a) to (b) under attack, we draw an original data x and its shifted data x adv .
Take the ﬁxed direction u 2 as an illustration.
The cosine similarity between x and u 2 is equal to cos ( θ ) , i.e. , cos ( θ )
= (cid:104) x, u 2 (cid:105) / (cid:107) x (cid:107) · (cid:107) u 2 (cid:107) , which is called as alignment above.
If the distribution moves from Figure 3(a) to (b), then the cosine similarity cos ( θ ) increases.
Consequently, the change of the cosine similarity can describe how the features change along a direction under attacks.
For the variation deﬁned in Eq.
(4), r
( D adv , D, u j ) > 1 means the features add more components along direction u j , and vice versa.
We could compare r ( D, D, u j ) with 1 to observe whether the adversary adds or reduces the components along various eigenvectors.
The adversary adds more components along the eigenvectors with smaller eigenvalues.
We visualize the variation of features to different eigenvectors { u 1 , · · · , u 512 } .
The results of CIFAR10 and SVHN are shown in Figure 4.
The attacks used include FGSM (Goodfellow et al., 2015),
[Figure]
PGD (step size (cid:15)/ 10 for 10 steps) (Madry et al., 2018) and C&W attack (Carlini & Wagner, 2017).
We set attack budget (cid:15) = 4 / 255 constrained by (cid:96) ∞ norm.
As observed in Figure 4(a), the variation keeps close or smaller than 1 for the several largest eigenvalues in standard trained model.
However, the variation of smaller eigenvalues is much larger than 1.
This means that FGSM tends to add more components along the eigenvectors u j with smaller eigenvalues in standard training.
A similar phenomenon also exists in PGD and C&W, which veriﬁes the hypothesis in Section 3.1.
We also provide the curve while changing budget (cid:15) and the variation on CIAFR-100 in Appendix B.2.
For models trained by AT, variation of all eigenvectors keeps close to 1, which is distinctly different from standard training.
The high variation of directions with smaller eigenvalues visibly decreases.
It is worth noticing that we apply cosine similarity for deﬁnition, so the inﬂuence from the scale of features is eliminated.
Therefore, the components corresponding to the larger eigenvalues are more robust while the smaller ones are more fragile.
Adopting the opinion of robust features (Ilyas et al., 2019), the features along the eigenvectors with larger eigenvalues are regarded as robust features, and these along the direction with smaller eigenvalues are non-robust features.
The analysis above motivates us to regularize the spectrum signatures to alleviate the rapid decreasing tendency.
If we alleviate the dominance of the top eigenvalues, the adversarial robustness is improved.
F EATURE S PECTRAL R EGULARIZATION
In this section, we propose a method to regularize the eigenvalues, aiming to alleviate the dominance of the top eigenvalues.
We ﬁrst propose our method FSR, and then provide a theoretical analysis.
4. R EALIZATION OF FSR
Through the detailed analysis above, the sharp distribution of eigenvalues weakens the information contained by the smaller eigenvalues, and causes fragility to adversarial attacks.
A promising approach to alleviate such phenomenon is to suppressing the largest eigenvalues, which could mitigate the dominance of the largest eigenvalues relatively.
Another straightforward idea is to increase the small eigenvalues during training.
However, small eigenvalues are usually numerically unstable, which may be easily affected by noise or round-off error in optimization.
In this paper, we propose a method called Feature Spectral Regularization (FSR) by penalizing the largest eigenvalues of feature covariance in a batch of data:
[Equation]
where λ max ( · ) means the largest eigenvalue of a matrix, w ( τ )
=
min { τ/T 0 , 1 } is the weighting function of current epoch τ with a hyper-parameter T 0 , and 1 ∈ R m × m is a constant matrix with each element equal to 1 for calculating the mean.
Moreover, F ∈ R m × d is the feature matrix of a batch composed of row vectors h
( x i ) T , i.e., F = [ h ( x 1 )
| h ( x 2 ) | · · · | h ( x m )] T , and m is the number of samples in a batch of data.
In practice, we can just access the batch of data to approximate the statistics of feature space.
However, the eigenvalues change drastically in the early stages of training, which may cause instability on optimization, so we apply a piecewise linear function w ( τ ) to smooth the training stage.
The λ max ( · ) in Eq.
(5) is equal to the square of the largest singular value in feature matrix ( F − 1 m 1 F ) by Singular Value Decomposition (SVD).
So the realization of FSR is very simple with the help of
Pytorch (Paszke et al., 2019) in a few lines of code.
Summarized objective loss.
Adversarial training (Madry et al., 2018) has been widely proven to be a strong baseline in adversarial defense.
We build and incorporate the proposed FSR into AT training framework for further improving model robustness signiﬁcantly.
The ﬁnal objective is:
[Equation]
(6) where β FSR is a hyper-parameter that controls the trade-off between two items, x adv represents the adversarial example generated from x by PGD (Madry et al., 2018) using cross-entropy loss, F adv is the feature matrix of adversarial examples.
Besides AT, we also apply FSR to defense methods such as TRADES (Zhang et al., 2019) and AT-AWP
(Wu et al., 2020).
We provide a pseudo-code of our method in Algorithm 1 in Appendix A.
Computational complexity.
SVD is an extra computational cost induced by FSR.
For a matrix
F ∈ R m × d , the time complexity of SVD is O (cid:0) min (cid:8) m 2 · d, m · d 2 (cid:9)(cid:1) .
Since the batch size is often small, the excess cost in computation is negligible compared to adversarial training.
4. T HEORETICAL A NALYSIS
Consider a linear regression model ˆ y = (cid:104)
z, θ (cid:105) with (cid:96) 2 perturbation δ based on the feature z ∈ R d , the underlying parameter obtained by minimizing mean square error is θ 0 ∈ R d without perturbation.
We assume the mean of features E ( z ) = 0 and covariance matrix Var( z )
= Σ .
Following the adversarial risk deﬁne by (Xing et al., 2021), R adv is expressed in Eq.
[Equation]
The optimal solution of Eq.(7) denoted as θ adv has an similar formulation with the ridge regression:
[Equation]
(8) where λ can be regarded as a constant (El Ghaoui & Lebret, 1997; Xing et al., 2021).
Given samples { z i , y i } n i =1 , the feature matrix composed of row vector
z Ti is denoted as Z ∈ R n × d , then Σ = 1 n Z T Z . y =
[ y 1 , y 2 , · · · , y n ] T ∈ R n is a column vector composed of real-valued output
y i ( i = 1 , · · · , n ) . Previous methods tend to regularize the classiﬁer θ (El Ghaoui & Lebret, 1997;
Xu et al., 2008).
However, we could directly analyse how the feature Z inﬂuences the robustness of model.
If the feature is more robust for classiﬁer, then whether using the classiﬁer θ 0 or θ adv should have the same prediction.
Thus, we deﬁne the residual risk induced by features Z :
[Equation]
It is essential to normalize the scale of features for comparable representation, so we restrict the norm of Z as used in (Yu et al., 2020).
The SVD of matrix Z ∈ R n × d has the form: Z = UDV T .
Here U =
[ u 1 , · · · , u n ] ∈ R n × n and V ∈ R d × d are orthogonal matrices.
Observe that the constraint for Z only depends on its singular values, i.e., (cid:107) Z (cid:107) 2 F = (cid:80) min { n,d } i =
σ 2 i = s 0 .
Suppose d < n and σ i > 0 , then R res is simpliﬁed to the expression by combining Eq.
(9) and Eq.
[Equation]
[Table] [Table]
using ResNet-18.
The maximum (cid:96) ∞ perturbation is (cid:15) = 8 / 255 .
[Table]
AT 89.
AT + FSR 88.
TRADES 92.
TRADES + FSR 92.
Theorem 1 R res ( σ 1 , · · · , σ d ) is minimum when all the singular values of Z are equal.
As FSR penalizes the largest singular value in feature matrix, it alleviates the dominance of large singular values and helps contribute to equal singular values under the normalized condition, thus helping reduce the residual risk induced by features.
Proof of Theorem 1 is provided in Appendix C.
E XPERIMENTS
In this section, we evaluate the effectiveness of the proposed FSR on CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009) and SVHN (Netzer et al., 2011).
FSR is applied to two baselines: 1) AT (Madry et al., 2018;
Rice et al., 2020); 2) TRADES (Zhang et al., 2019).
It is worth noticing that AT is the most effective method to improve adversarial robustness
(Rice et al., 2020;
Pang et al., 2021; Gowal et al., 2021) in RobustBench (Croce et al., 2020).
Experimental Settings.
For CIFAR-10 and CIFAR-100, we set the (cid:96) ∞ perturbation with (cid:15) = 8 / 255 , the step size of attack 2 / 255 , and the inner iteration steps 10.
The step size is 1 / 255 for
SVHN.
We train ResNet-18 (He et al., 2016) using momentum optimizer with the initial learning rate of 0.1.
For AT, we train 200 epochs and the learning rate decays with a factor of 0.1 at 100 and 150 epochs (Rice et al., 2020).
For TRADES, we train 120 epochs with the learning rate divided by 0.1 at epochs 75, 90, and 100 (Zhang et al., 2019).
The parameter for regularization ( 1 /λ ) is set as 4 for TRADES.
For FSR, we set β FSR = 0 .
Other hyper-parameters keep the same as their original paper.
Considering that the settings have a distinct inﬂuence on robustness (Pang et al., 2021; Gowal et al., 2021), the hyper-parameters remain unchanged while adding our FSR.
5. P ERFORMANCE UNDER WHITE - BOX ATTACKS
To test the adversarial robustness of model in detail, we adopt various white-box adversarial attacks:
FGSM (Goodfellow et al., 2015), PGD-20 (step size (cid:15)/ 4 ) (Madry et al., 2018), CW ( (cid:96) ∞ version optimized by PGD)
(Carlini & Wagner, 2017).
Following the instruction proposed by (Carlini et al.,
[Table]
We report the
[Table]
2019), we use 5 random starts at random offsets away from the initial when applying iterative attacks.
The settings for evaluation promote a quite strong attack.
We report the best checkpoint (the highest robustness under PGD from different checkpoints) as used in (Rice et al., 2020).
The test accuracy is reported in Table 1 ∼ 3.
The results show that FSR can further improve adversarial robustness, compared with the baseline models.
We also test the robustness under AutoAttack (AA) (Croce &
Hein, 2020), which is now regarded as the strongest attack to evaluate adversarial robustness.
As shown in the table, FSR is effective to improve robustness under AA.
The detailed results show that our FSR can indeed boost robustness, rather than depending on obfuscated gradients (Athalye et al., 2018) or gradient masking (Papernot et al., 2017).
The improvement on various datasets reveals that our method has a consistent promotion, rather than relying on a special dataset.
The performance of ﬁnal checkpoint is another essential probe to analyse the adversarial robustness of models.
We also report the performance of ﬁnal checkpoint, as shown in Table 4.
The results show that FSR could also improve the performance in ﬁnal checkpoint on various datasets.
Combination with other methods.
To validate that FSR is an effective module, we try to combine
FSR with Adversarial Weight Perturbation (AWP) (Wu et al., 2020).
AWP helps improve adversarial robustness and effectively alleviates the overﬁtting (Rice et al., 2020) in PGD-based AT.
It is one of the strongest defense methods in RobustBench (Croce et al., 2020), which means it is a high baseline.
The results are listed in Table 5, and the results show that we could even attain improvement on adversarial robustness based on the high baseline AWP , especially under CW attack and AA.
[Table]
Performance under WideResNet.
To explore the performance of our model in large networks, we use WideResNet-34-10 (depth 34 and width 10) (Zagoruyko & Komodakis, 2017) as another backbone network.
In this part, we realize our method based on the well-known effective methods:
AT (Madry et al., 2018;
Rice et al., 2020) and AT-AWP (Wu et al., 2020) on CIFAR-10.
All the parameters of baselines are the same as used in their original paper.
The settings of threaten model are the same as ResNet-18 in the previous section.
We show the results of both best checkpoint and ﬁnal checkpoint in Table 6.
The “Best” means the checkpoint of best performance under PGD , and the “Last” is the performance of ﬁnal checkpoint.
The results verify that our method is also effective in WideResNet, and we especially attain great promotion in ﬁnal checkpoint based on AWP.
5. P ERFORMANCE UNDER BLACK - BOX ATTACKS
Transferability is an intriguing property about adversarial examples, which can be used to implement black-box attack (Papernot et al., 2016a; Liu et al., 2017).
Adversarial examples generated by the
[Figure]
substitute model can be used to attack the target model with the weights and architecture unknown to the adversary.
Transfer attack is beneﬁcial to verify that our method does not rely on gradient masking (Papernot et al., 2017).
As suggested by (Carlini et al., 2019), we adopt another adversarially trained ResNet-18 as the substitute model.
The results are shown in Table 1 ∼ 3, revealing that our methods also improve robustness under black-box attacks.
5. F URTHER UNDERSTANDING OF FSR
In this part, we analyse the inﬂuence of FSR on the eigenvalue spectrum.
The results are shown in Figure 5.
Our motivation is that the top eigenvalues dominate the distribution of eigenvalues, so we use FSR to increase the overall eigenvalues relatively.
To eliminate the inﬂuence from the scale of features, the eigenvalues are divided by the maximum eigenvalue.
As shown in the ﬁgure, FSR increases the eigenvalues relatively, which is consistent with our intention for FSR.
In Appendix D, we show that simply using FSR without AT also promotes a mild improvement on robustness.
5. A BLATION S TUDIES
Sensitivity analysis of β FSR .
We explore how the weight of FSR β FSR inﬂuences the performance of models.
Following the same settings as the previous section, we train our models on
CIFAR-10 using ResNet-18 with the change of β FSR .
The results are reported Table 7.
As observed in Table 7, FSR can signiﬁcantly improve robustness with a wide value range, which veriﬁes the stability and effectiveness of the proposed method.
In this paper, we choose the parameter by considering both generalization and robustness, so we choose β FSR = 0 .
If we focus merely on adversarial robustness, tuning β FSR could still promote a further improvement on robustness.
[Table]
C ONCLUSION
In this paper, we delve into the discrepancy between natural examples and adversarial examples from the perspective of spectral analysis.
The variation of different eigenvectors under adversarial attacks is analysed.
It is shown that the spectral directions with smaller eigenvalues are more fragile under attack, which is induced by the dominance of the top eigenvectors.
Based on the analysis, a method called Feature Spectral Regularization (FSR) is proposed to penalize the largest eigenvalues of the batch covariance matrix, which is numerically stable and could enlarge the overall eigenvalues relatively.
We also provide a theoretical explanation in robust linear regression.
FSR is a simple module readily pluggable into any defense method.
Through comprehensive experiments, we show that FSR can effectively improve adversarial robustness when combined with many defense methods
A A LGORITHM OF FSR Algorithm
B T HE DEFINED VARIATION ON MORE DATASETS
In this part, we provide more experiments about the variation deﬁned by us.
We ﬁrst provide a detailed algorithm to calculate the variation.
More results about the variation are shown following.
B. A LGORITHM FOR C ALCULATING THE V ARIATION Algorithm 2 Calculating the Alignment and Variation
Require: Dataset D = { ( x i , y i ) } i
=1 , ··· ,N (natural examples), the feature extractor h ( · ) .
Ensure: Variation to different eigenvectors r
( D adv , D, u j ) ( j = 1 , · · · , d ) .
1: Centralize
the features h ( x i ) ← h ( x i )
−
1 N
(cid:80) Ni
= h
( x
i
) 2: Calculate
the covariance matrix 1 N (cid:80) Ni =1 h ( x i ) h ( x i ) T
3: Apply eigendecomposition
to the
covariance matrix
to get the
eigenvalues and eigenvectors { λ 1 , · · · , λ d } , { u 1 , · · · , u d } = eig (cid:16) 1 N (cid:80)
Ni =1 h ( x i ) h
( x i ) T (cid:17)
− align
[Equation]
align ( D adv , u j ) = 1 N (cid:80) Ni =1 |(cid:104) h
( x adv,i ) , u j (cid:105)| / (cid:107) h
( x adv,i ) (cid:107) · (cid:107) u j (cid:107)
[Equation]
u
[Equation]
B. M ORE EXPERIMENTS ABOUT THE DEFINED VARIATION
In this part, we show more results about variation to eigenvectors on CIFAR-10, CIFAR-100 and
SVHN.
We compare the models trained on natural examples and adversarial examples.
The adversarially trained models adopt the PGD-AT Madry et al.
(2018) with (cid:96) ∞ perturbation 8/255, and the parameters are the same as Rice et al.
From the curve shown in Figure 6 ∼ 8, we can see that the variation on CIFAR-100 and SVHN reveals the same tendency as CIFAR-10, i.e., adversarial attacks tend to increase the components along the eigenvectors with smaller eigenvalues.
[Table]
[Figure] [Figure]
[Table]
[Figure] [Figure]
v a r i a t i
[Figure]
j
[Figure]
C P ROOF
In this part, we provide the proof of Theorem 1 in Section 4.2.
Consider the linear regression model,
[Equation]
where z ∈ R d means the feature, and θ ∈ R
d represents the parameter of model.
We assume that the features have zero mean, and the covariance is denoted as Σ , i.e. , E ( z ) = 0 , Var( z ) = Σ .
Given samples { z i , y i } ni =1 , the feature matrix composed of row vectors
z Ti is represented as feature matrix Z ∈ R n × d , then Σ = 1 n Z T Z .
Assuming that Z has full column rank, if we minimize the residual sum of squares in Eq.
(11), then the solution is:
[Equation]
12) where y ∈ R n means the real-valued output, i.e. , y T =
( y 1 , y 2 , · · · , y n ) .
When the feature z is noised by (cid:96) 2 perturbation δ , we can deﬁne the adversarial risk induced by the perturbation as (Xing et al., 2021):
[Equation]
The optimal solution to Eq.
(13) denoted as θ adv is
[Equation]
(14) where λ can be regarded as a constant.
The work of (El Ghaoui & Lebret, 1997; Xing et al., 2021) provides more details about how to get λ .
A better feature should lead to the same prediction whether using classiﬁer θ 0 or θ adv , so we deﬁne the residual risk induced by features
[Equation]
We normalize the scale of features by Frobenius norm to avoid the inﬂuence of scale (Yu et al., 2020), which is just correlated to its singular values { σ 1 , · · · , σ d } :
[Equation]
The SVD of the n × d matrix Z has the form:
[Equation]
where U =
[ u 1 , · · · , u n ] ∈ R n × n and V ∈ R d × d are orthogonal matrices.
D =
(cid:20) D 0 0 (cid:21) ∈ R n × d and D
0 = diag ( σ 1 , · · · , σ d ) , σ 1 ≥ σ 2 ≥ · · · ≥ σ d .
After applying SVD, we can simplify the Eq.
(15) to following expression:
[Equation]
According to the property of the subordinate matrix norm, we could derive an upper bound of R res : (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
d (cid:88)
j =
[Equation]
Consider that y can be arbitrary value, so the minimization of R res requires that the ﬁrst item to the right of the inequality sign in Eq.
(19) is as small as possible, i,e,
[Equation]
The minimization of R res is simpliﬁed to the following expression:
[Equation]
Since σ 1 ≥ σ 2 ≥ · · · ≥ σ d , the optimal solution of Eq.
(21) is that all singular values are equal.
D FSR IMPROVES CONFIDENCE CALIBRATION AND ROBUSTNESS
We verify that network trained using only FSR without AT attains a mild improvement on adversarial robustness.
We test the performance of model under FGSM, and the results are listed in Table 8.
[Table]
In addition to robustness, another important but undesirable aspect of DNN is reliability.
When using classiﬁers for prediction, the predictive conﬁdence should be indicative of the actual likelihood of correctness (Guo et al., 2017).
Conﬁdence calibration focuses on the mismatch problem between a model’s conﬁdence and its correctness.
A popular metric used to measure calibration is the Expected
Calibration Error (ECE) (Naeini et al., 2015; Guo et al., 2017), deﬁned as the expected absolute difference between the models’ conﬁdence and its accuracy.
We explore the efﬁcacy of FSR on conﬁdence calibration.
[Figure]
can improve the conﬁdence calibration.
As shown in Figure 9, for the baseline, there is a marked rise in the test ECE at the later period of the training process, which indicates the problem of overconﬁdence.
Remarkably, FSR has an obvious improvement on conﬁdence calibration compared with the baseline, e.g. , reduces the ECE value from 2.97% to 1.18% for CIFAR-10. E F URTHER EXPERIMENTS ABOUT ADVERSARIAL ROBUSTNESS
In this part, we add two experiments about the proposed FSR on adversarial robustness.
We ﬁrst report the results (i.e., mean and standard deviation) when all experiments in Table 1 ∼ 3 are repeated ﬁve times.
Then, the test accuracy of suppressing more eigenvalues is also provided.
E. E XPERIMENTS ABOUT STANDARD DEVIATION
We repeat all the experiments in Table 1 ∼ 3 ﬁve times, and report the results under white-box attack in following Table.
The performance in Table 9 ∼ 11 veriﬁes that our proposed method achieves a steady improvement.
[Table]
[Table]
box attacks using ResNet-18.
The maximum
∞ perturbation is (cid:15) = 8 / 255 .
[Table]
E. S UPPRESSING MORE EIGENVALUES
In the main body of our paper
[Table]
the proposed FSR only penalizes the largest eigenvalue.
In this part, we explore whether penalizing more eigenvalues could achieve a better performance under attack.
The results are listed in Table 12.
As we properly increase the number of suppressed eigenvalues k , the adversarial robustness of models could be further improved.
However, if we keep increasing k , the robustness of the model declines.
We think it is due to that k has some trade-off with the weight of FSR β FSR , i
[Table]
e., a larger k may require a smaller β FSR .
[Table]
F M ORE DISCUSSIONS BETWEEN SPECTRAL SIGNATURES AND ROBUSTNESS
In this section
[Table]
we discuss the connection between our explanation of adversarial examples in the paper and previous studies.
We mainly talk about the connection with interpretations form the perspective of Lipschitz constant and manifold.
F. C ONNECTION BETWEEN FSR AND L IPSCHITZ CONSTANT
In this part, we ﬁrst rederive some basic conclusions about Lipschitz constant in DNN as illustrated in (Yoshida & Miyato, 2017).
For a feature extractor g ( x ) : R D → R d composed of L layers, the weights of each layer in the network are denoted as (cid:8) W l , b l (cid:9)
L l − 1 .
Then the feature extractor g ( x ) =
W L φ L − 1 (cid:0) W L − 1 φ L − 2 ( · · · ) + b L − 1 (cid:1) + b L , where φ l represents the activation function.
We further assume that each activation function is a ReLU.
In this case, φ l act as input-dependent diagonal matrices Φ l X := diag (cid:0) φ l X (cid:1) , where an element in the diagonal φ l X :
= 1 ( (cid:101) x ≥ 0) is one if its pre-activation (cid:101) x
l :=
W l φ l − 1 + b l is positive (Roth et al., 2020).
Then, the global Lipschitz constant of g ( x ) is bounded by the spectral norm of following expression (Yoshida & Miyato, 2017;
Roth et al., 2020; Huang et al., 2021):
[Equation]
For better comparison with our proposed method, we do not further simplify Eq.
(22) like (Yoshida & Miyato, 2017).
With the normalization (Ioffe & Szegedy, 2015) in DNN, the mean of ﬁnal features are always close to zero.
Suppose that the mean of deep features is equal to zero, then
the proposed
FSR denoted as (cid:96) FSR could be simpliﬁed to
[Equation]
(23) where the deep features F = W L φ L − 1 (cid:0) W L −
[Equation]
φ
[Equation]
(cid:0) · · · · · · φ 1 (cid:0) W
[Equation]
x + b
[Equation]
(cid:1) · · · · · · (cid:1)
+ b L − 1 (cid:1)
+ b L , λ max ( · ) is the largest eigenvalue of a matrix, and σ max ( · ) is the spectral norm (largest singular value) of a matrix.
If we compare Eq.
(22) and Eq.
(23), FSR has a very similar form with the upper bound of global
Lipschitz constant.
Since the input data x is a constant, it could be expected that FSR has an effect to decrease the upper bound of the Lipschitz constant.
FSR decreases the Lipschitz constant by constraining features rather than directly regularizing weights.
F. C ONNECTION WITH ADVERSARIAL EXAMPLES ON MANIFOLD
Our observation is consistent with the hypothesis in (Szegedy et al., 2013; Song et al., 2018) that adversarial examples are low-probability (high-dimensional) “pockets” on the manifold from the perspective of spectral signatures.
Our paper reveals that the components along eigenvectors with severely smaller eigenvalues are much rare in standard training (seeing Figure 2).
When confronted with attacks, the adversary tries to add more components along with such directions (seeing Figure 4).
The eigenvectors with smaller eigenvalues could be regarded as “pockets” on the manifold.