Delving into Feature Space: Improving Adversarial Robustness by Feature Spectral Regularization

1 INTRODUCTION . It is shown that the performance of Deep Neural Networks ( DNNs ) decreases dramatically when confronted with adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) . The vulnerability of DNNs brings potential risk to safety-critical deployments ( Finlayson et al. , 2019 ) . To mitigate this vulnerability of DNNs , numerous methods are proposed to improve adversarial robustness ( Papernot et al. , 2016b ; Madry et al. , 2018 ; Xie et al. , 2019 ) . Among these , adversarial training ( AT ) ( Madry et al. , 2018 ) is the most effective approach that achieves state-ofthe-art performance under various attacks ( Croce et al. , 2020 ) . Different from standard training , adversarial training trains DNNs on adversarial examples rather than natural examples . There are numerous methods proposed to eliminate the gap of features between natural examples and adversarial examples ( Kannan et al. , 2018 ; Zhang & Wang , 2019 ; Zhang et al. , 2019 ) . However , they constrain the features on the whole without considering the distinction of contribution from an individual feature . This may be inappropriate since every feature may play a different role in robustness . The work of ( Ilyas et al. , 2019 ) argued that adversarial examples result from non-robust features ( Ilyas et al. , 2019 ) , i.e. , well-generalizing but brittle features . This inspires us that we could treat each feature separately in the adversarial setting . While the work of ( Bai et al. , 2021 ; Yan et al. , 2021 ) considered the influence of different channels on robustness , the problem has not been explored clearly from the perspective of spectral signatures , i.e. , the eigenvalues and eigenvectors of feature covariance . The space is split into many spectral components , and then it is worth analyzing which component is beneficial for adversarial robustness and which is fragile under attack . In this paper , we show that spectral signatures of deep features have a close connection with adversarial robustness . Through applying principal component analysis ( PCA ) to deep features , we could split feature space to various eigenvectors with its eigenvalues . Our motivation comes from the phenomenon that the standard trained model often results in a sharp distribution of eigenvalues ( Yu et al. , 2020 ) , i.e. , the eigenvalues rapidly become very small as shown in Figure 1 . This property may be beneficial for natural accuracy ( Lezama et al. , 2018 ; Papyan et al. , 2020 ) , while its impact on robust generalization is unclear . However , we hypothesize that the sharp distribution of eigenvalues impels models to learn less diverse features and is a cause of the vulnerability of DNNs . A minority of eigenvalues occupy the overwhelming majority in the sum of eigenvalues , which may make the model pay little attention to features along the eigenvectors with smaller eigenvalues , so features along such directions are not generalized during training . To verify our hypothesis , we define a new metric to measure the variation of features along different eigenvectors under attacks , as shown in Figure 4 . Our observation reveals that the adversary tends to add more components along eigenvectors with smaller eigenvalues , and such huge variation could be distinctly alleviated by AT . Therefore , we propose to improve adversarial robustness by alleviating the sharp distribution of eigenvalues . Considering that more eigenvalues should occupy the majority , we propose a regularizer named Feature Spectral Regularization ( FSR ) to penalize the largest eigenvalue of the feature matrix covariance . Empirical evidence shows that FSR increases the overall eigenvalues relatively , making models focus on more spectral components during training . We also provide a theoretical explanation with robust linear regression . Comprehensive experiments confirm that FSR indeed improves the adversarial robustness on several datasets . Our contributions are summarized as follows : • We find a close connection between spectral signatures of features and adversarial robustness . On one hand , standard trained model presents a sharp distribution of eigenvalues , which is beneficial for natural accuracy while harmful in adversarial setting . On the other hand , the adversary tends to add more quantity along eigenvectors with smaller eigenvalues . • We propose Feature Spectral Regularization ( FSR ) to increase the overall eigenvalues relatively in deep feature space , thus alleviating the sharp distribution of eigenvalues . Furthermore , we provide a theoretical explanation based on robust linear regression . • We empirically show that FSR improves adversarial robustness and alleviates the sharp distribution of eigenvalues , through comprehensive experiments . 2 RELATED WORK . Adversarial Defense . Many defense methods have been proposed to improve adversarial robustness since the discovery of adversarial examples ( Papernot et al. , 2016b ; Xie et al. , 2019 ; Carmon et al. , 2019 ; Zhang et al. , 2020 ) . However , many of them are proven to be noneffective because they highly depend on obfuscated gradients ( Athalye et al. , 2018 ) or gradient masking ( Papernot et al. , 2017 ) . Among these , adversarial training ( Madry et al. , 2018 ) is now regarded as the state-of-theart method ( Rice et al. , 2020 ; Pang et al. , 2021 ) . Distinguished form standard training , adversarial training trains DNN on adversarial examples : min θ max ‖δ‖≤ E ( x , y ) ∈D LCE ( x+ δ , y ; θ ) ( 1 ) where , D is the dataset , the parameters of DNN are denoted as θ , δ means the perturbation within the -ball , and LCE is the cross-entropy ( CE ) loss . By introducing a trade-off between robustness and generalization , TRADES ( Zhang et al. , 2019 ) is another framework that reaches comparative robustness with AT . Among the proposed methods based on AT , Adversarial Weight Perturbation ( AWP ) ( Wu et al. , 2020 ) explicitly regularizes the flatness of weight loss landscape , and forms a double-perturbation mechanism , which shows huge improvement on adversarial robustness and alleviates the overfitting in AT ( Rice et al. , 2020 ) . Spectral Signatures of Feature Representations . Some studies have revealed that the spectral signatures of features influence the performance in various learning tasks . The spectral properties are crucial to detect backdoors ( Tran et al. , 2018 ; Hayase et al. , 2021 ) . The eigenvectors corresponding to the larger eigenvalues are found to dominate the transferability of features in adversarial domain adaptation ( Chen et al. , 2019b ) . The work of ( Chen et al. , 2019a ) explores the correlation between negative transfer and the spectral components of features and weights in inductive transfer learning . By utilizing the principle of Maximal Coding Rate Reduction ( MCR2 ) , it is theoretically proven that the larger several singular values of feature matrix for every class should be equal to learn the maximally diverse representation ( Yu et al. , 2020 ; Chan et al. , 2020 ) . Different from these studies , we analyse the connection between adversarial robustness and spectral components of deep features . We aim to explore which components are more fragile under attacks , and propose a method to boost adversarial robustness by constraining spectral properties . 3 SPECTRAL ANALYSIS IN FEATURE SPACE . In this section , we investigate the connection between spectral signatures and adversarial robustness . Concretely , we train models by standard training and adversarial training ( Madry et al. , 2018 ) , and then apply spectral decomposition to attain the spectral signatures . We find that in standard training the eigenvalues reveal a rapidly descending curve while this tendency is alleviated by AT . Then , we find that the adversary tends to add more quantity along eigenvectors with smaller eigenvalues . 3.1 CURVE OF EIGENVALUES AND ADVERSARIAL ROBUSTNESS . Given a dataset D = { ( xi , yi ) } ni=1 including C classes , xi represents the input data and yi is the label . DNN is composed of a feature extractor h ( · ) : RD → Rd and a linear classifier g ( · ) : Rd → RC . After centralizing the learned features ( i.e . 1n ∑n i=1 h ( xi ) = 0 ) , we decompose the learned features by spectral decomposition , which is similar to PCA : 1 n n∑ i=1 h ( xi ) h ( xi ) T = d∑ j=1 ujλju T j ( 2 ) where λj means the eigenvalues with index j and uj ∈ Rd represents its eigenvector . Specifically , we train ResNet-18 ( He et al. , 2016 ) using both standard training and adversarial training on CIFAR-10 . The parameters for AT are the same as ( Rice et al. , 2020 ) . We calculate the eigenvalues by applying Eq . ( 2 ) , and the eigenvalues are plotted in Figure 2 . The features come from the penultimate layer ( 512 dimensions ) . All the features are extracted from the test set in CIFAR-10 . A part of the eigenvalues is shown for better visualization . advori Difference of models in spectral analysis . As shown in Figure 2 ( a ) ( b ) , the eigenvalues of a standard trained model drop rapidly at some point , while the sharp decrease of eigenvalues is much alleviated by AT . These sharp spectral signature in standard training makes just a few eigenvalues informative from the opinion of PCA . The model fails to explore the influence of eigenvectors with smaller eigenvalues on classification , so the trained model could not recognize the change of features along eigenvectors with smaller eigenvalues . The eigenvectors which may endow useful features are overly penalized . Consequently , we propose a hypothesis that the severe dominance of the top eigenvectors is a cause of vulnerability in DNN , and the adversary adds more components in eigenvectors with smaller eigenvalues . We will verify the proposed hypothesis in the next section . Connection with intrinsic dimension . We introduce intrinsic dimension ( ID ) to quantitatively describe the decreasing tendency in eigenvalues . ID ( i.e. , the minimal number of parameters needed to describe a representation ) has a close connection with natural accuracy ( Ansuini et al. , 2019 ) . It has been found that reduction of ID contributes to an improvement on natural accuracy . We adopt PC-ID proposed by ( Ansuini et al. , 2019 ) to estimate ID , which is determined by the number of principal components included to describe 90 % of the variance . We mark ID in Figure 2 with solid circles . The results reveal the ID of a standard trained model is very small , while models obtained by AT achieve a higher ID , which is contrary to the influence of ID on natural accuracy . This also verifies that there exists a trade-off between generalization and robustness ( Tsipras et al. , 2019 ; Zhang et al. , 2019 ) from the perspective of ID . FSR could further increase ID , based on AT . 3.2 VARIATION ALONG EIGENVECTORS UNDER ATTACKS . In this section , we aim to verify the hypothesis that adversary adds more components along eigenvectors with smaller eigenvalues in attacking stage . We define a metric to quantitatively observe the change of features along different eigenvectors under attack , called variation in this paper . Definition 1 ( Alignment ) Given a dataset Ds = { xs , i , ys , i } ni=1 which may be perturbed . The alignment of Ds to the pre-given direction uj is calculated by the expectation over cosine similarity between features extracted by DNN and the direction vector uj : align ( Ds , uj ) = E ( xs , i , ys , i ) ∈Ds |〈h ( xs , i ) , uj〉| ‖h ( xs , i ) ‖ · ‖uj‖ ( 3 ) where the norm ‖·‖ used is Euclidian norm , and uj is calculated by Eq . ( 2 ) . The calculation of uj is based on features covariance of natural samples , and uj is fixed . Definition 2 ( Variation ) Given a dataset D consist of natural examples and its perturbed dataset Dadv . The variation on direction uj is defined as the ratio between alignment on Dadv and D : r ( Dadv , D , uj ) = align ( Dadv , uj ) align ( D , uj ) ( 4 ) The alignment is correlated with the distance between subspace spanned by uj and the actual feature space , so the change of alignment under attack is suitable to describe the influence of attacks on direction uj , which is called as variation . Our metric is similar to ( Tran et al. , 2018 ; Hayase et al. , 2021 ) used for analyzing backdoors , but we define the alignment by cosine similarity while the latter uses the inner product . Compared with inner product , cosine similarity could eliminate the influence of scale . In Appendix B.1 , we provide a complete procedure to calculate the metrics . We give an intuitive explanation on why the defined variation is suitable to capture the change of features along different eigenvectors under attacks . An toy model for illustration is shown in Figure 3 . Suppose the distribution transfers from Figure 3 ( a ) to ( b ) under attack , we draw an original data x and its shifted data xadv . Take the fixed direction u2 as an illustration . The cosine similarity between x and u2 is equal to cos ( θ ) , i.e. , cos ( θ ) = 〈x , u2〉 / ‖x‖ · ‖u2‖ , which is called as alignment above . If the distribution moves from Figure 3 ( a ) to ( b ) , then the cosine similarity cos ( θ ) increases . Consequently , the change of the cosine similarity can describe how the features change along a direction under attacks . For the vari- ation defined in Eq . ( 4 ) , r ( Dadv , D , uj ) > 1 means the features add more components along direction uj , and vice versa . We could compare r ( D , D , uj ) with 1 to observe whether the adversary adds or reduces the components along various eigenvectors . The adversary adds more components along the eigenvectors with smaller eigenvalues . We visualize the variation of features to different eigenvectors { u1 , · · · , u512 } . The results of CIFAR10 and SVHN are shown in Figure 4 . The attacks used include FGSM ( Goodfellow et al. , 2015 ) , PGD ( step size /10 for 10 steps ) ( Madry et al. , 2018 ) and C & W attack ( Carlini & Wagner , 2017 ) . We set attack budget = 4/255 constrained by ` ∞ norm . As observed in Figure 4 ( a ) , the variation keeps close or smaller than 1 for the several largest eigenvalues in standard trained model . However , the variation of smaller eigenvalues is much larger than 1 . This means that FGSM tends to add more components along the eigenvectors uj with smaller eigenvalues in standard training . A similar phenomenon also exists in PGD and C & W , which verifies the hypothesis in Section 3.1 . We also provide the curve while changing budget and the variation on CIAFR-100 in Appendix B.2 . For models trained by AT , variation of all eigenvectors keeps close to 1 , which is distinctly different from standard training . The high variation of directions with smaller eigenvalues visibly decreases . It is worth noticing that we apply cosine similarity for definition , so the influence from the scale of features is eliminated . Therefore , the components corresponding to the larger eigenvalues are more robust while the smaller ones are more fragile . Adopting the opinion of robust features ( Ilyas et al. , 2019 ) , the features along the eigenvectors with larger eigenvalues are regarded as robust features , and these along the direction with smaller eigenvalues are non-robust features . The analysis above motivates us to regularize the spectrum signatures to alleviate the rapid decreasing tendency . If we alleviate the dominance of the top eigenvalues , the adversarial robustness is improved .

Summary: The authors propose a new defense against adversarial attacks by means of a spectral regularization. The defense is based on an inspection of the embedding of training/test data as returned by the penultimate layer of a neural network. The observation is that the relevant subspace of embedded natural data points (covering e.g. 90% of the variance) is quite low-dimensional, where the use of adversarial training increases the dimensionality of this subspace. In addition, the relevant subspace of embedded adversarial examples is in the provided experiments typically a bit higher dimensional than the subspace of natural examples. Based on these observations, the authors propose a regularizing term, which penalizes the variance in the direction of the first principal component (the largest eigenvalue of the feature covariance matrix).  Experiments are conducted on Cifar-10, -100 and SVHN. The proposed regularizer is applied on top of other defense methods, which typically increases the accuracy under attacks by up to 2%.