Delving into Feature Space: Improving Adversarial Robustness by Feature Spectral Regularization

1 INTRODUCTION . It is shown that the performance of Deep Neural Networks ( DNNs ) decreases dramatically when confronted with adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) . The vulnerability of DNNs brings potential risk to safety-critical deployments ( Finlayson et al. , 2019 ) . To mitigate this vulnerability of DNNs , numerous methods are proposed to improve adversarial robustness ( Papernot et al. , 2016b ; Madry et al. , 2018 ; Xie et al. , 2019 ) . Among these , adversarial training ( AT ) ( Madry et al. , 2018 ) is the most effective approach that achieves state-ofthe-art performance under various attacks ( Croce et al. , 2020 ) . Different from standard training , adversarial training trains DNNs on adversarial examples rather than natural examples . There are numerous methods proposed to eliminate the gap of features between natural examples and adversarial examples ( Kannan et al. , 2018 ; Zhang & Wang , 2019 ; Zhang et al. , 2019 ) . However , they constrain the features on the whole without considering the distinction of contribution from an individual feature . This may be inappropriate since every feature may play a different role in robustness . The work of ( Ilyas et al. , 2019 ) argued that adversarial examples result from non-robust features ( Ilyas et al. , 2019 ) , i.e. , well-generalizing but brittle features . This inspires us that we could treat each feature separately in the adversarial setting . While the work of ( Bai et al. , 2021 ; Yan et al. , 2021 ) considered the influence of different channels on robustness , the problem has not been explored clearly from the perspective of spectral signatures , i.e. , the eigenvalues and eigenvectors of feature covariance . The space is split into many spectral components , and then it is worth analyzing which component is beneficial for adversarial robustness and which is fragile under attack . In this paper , we show that spectral signatures of deep features have a close connection with adversarial robustness . Through applying principal component analysis ( PCA ) to deep features , we could split feature space to various eigenvectors with its eigenvalues . Our motivation comes from the phenomenon that the standard trained model often results in a sharp distribution of eigenvalues ( Yu et al. , 2020 ) , i.e. , the eigenvalues rapidly become very small as shown in Figure 1 . This property may be beneficial for natural accuracy ( Lezama et al. , 2018 ; Papyan et al. , 2020 ) , while its impact on robust generalization is unclear . However , we hypothesize that the sharp distribution of eigenvalues impels models to learn less diverse features and is a cause of the vulnerability of DNNs . A minority of eigenvalues occupy the overwhelming majority in the sum of eigenvalues , which may make the model pay little attention to features along the eigenvectors with smaller eigenvalues , so features along such directions are not generalized during training . To verify our hypothesis , we define a new metric to measure the variation of features along different eigenvectors under attacks , as shown in Figure 4 . Our observation reveals that the adversary tends to add more components along eigenvectors with smaller eigenvalues , and such huge variation could be distinctly alleviated by AT . Therefore , we propose to improve adversarial robustness by alleviating the sharp distribution of eigenvalues . Considering that more eigenvalues should occupy the majority , we propose a regularizer named Feature Spectral Regularization ( FSR ) to penalize the largest eigenvalue of the feature matrix covariance . Empirical evidence shows that FSR increases the overall eigenvalues relatively , making models focus on more spectral components during training . We also provide a theoretical explanation with robust linear regression . Comprehensive experiments confirm that FSR indeed improves the adversarial robustness on several datasets . Our contributions are summarized as follows : • We find a close connection between spectral signatures of features and adversarial robustness . On one hand , standard trained model presents a sharp distribution of eigenvalues , which is beneficial for natural accuracy while harmful in adversarial setting . On the other hand , the adversary tends to add more quantity along eigenvectors with smaller eigenvalues . • We propose Feature Spectral Regularization ( FSR ) to increase the overall eigenvalues relatively in deep feature space , thus alleviating the sharp distribution of eigenvalues . Furthermore , we provide a theoretical explanation based on robust linear regression . • We empirically show that FSR improves adversarial robustness and alleviates the sharp distribution of eigenvalues , through comprehensive experiments . 2 RELATED WORK . Adversarial Defense . Many defense methods have been proposed to improve adversarial robustness since the discovery of adversarial examples ( Papernot et al. , 2016b ; Xie et al. , 2019 ; Carmon et al. , 2019 ; Zhang et al. , 2020 ) . However , many of them are proven to be noneffective because they highly depend on obfuscated gradients ( Athalye et al. , 2018 ) or gradient masking ( Papernot et al. , 2017 ) . Among these , adversarial training ( Madry et al. , 2018 ) is now regarded as the state-of-theart method ( Rice et al. , 2020 ; Pang et al. , 2021 ) . Distinguished form standard training , adversarial training trains DNN on adversarial examples : min θ max ‖δ‖≤ E ( x , y ) ∈D LCE ( x+ δ , y ; θ ) ( 1 ) where , D is the dataset , the parameters of DNN are denoted as θ , δ means the perturbation within the -ball , and LCE is the cross-entropy ( CE ) loss . By introducing a trade-off between robustness and generalization , TRADES ( Zhang et al. , 2019 ) is another framework that reaches comparative robustness with AT . Among the proposed methods based on AT , Adversarial Weight Perturbation ( AWP ) ( Wu et al. , 2020 ) explicitly regularizes the flatness of weight loss landscape , and forms a double-perturbation mechanism , which shows huge improvement on adversarial robustness and alleviates the overfitting in AT ( Rice et al. , 2020 ) . Spectral Signatures of Feature Representations . Some studies have revealed that the spectral signatures of features influence the performance in various learning tasks . The spectral properties are crucial to detect backdoors ( Tran et al. , 2018 ; Hayase et al. , 2021 ) . The eigenvectors corresponding to the larger eigenvalues are found to dominate the transferability of features in adversarial domain adaptation ( Chen et al. , 2019b ) . The work of ( Chen et al. , 2019a ) explores the correlation between negative transfer and the spectral components of features and weights in inductive transfer learning . By utilizing the principle of Maximal Coding Rate Reduction ( MCR2 ) , it is theoretically proven that the larger several singular values of feature matrix for every class should be equal to learn the maximally diverse representation ( Yu et al. , 2020 ; Chan et al. , 2020 ) . Different from these studies , we analyse the connection between adversarial robustness and spectral components of deep features . We aim to explore which components are more fragile under attacks , and propose a method to boost adversarial robustness by constraining spectral properties . 3 SPECTRAL ANALYSIS IN FEATURE SPACE . In this section , we investigate the connection between spectral signatures and adversarial robustness . Concretely , we train models by standard training and adversarial training ( Madry et al. , 2018 ) , and then apply spectral decomposition to attain the spectral signatures . We find that in standard training the eigenvalues reveal a rapidly descending curve while this tendency is alleviated by AT . Then , we find that the adversary tends to add more quantity along eigenvectors with smaller eigenvalues . 3.1 CURVE OF EIGENVALUES AND ADVERSARIAL ROBUSTNESS . Given a dataset D = { ( xi , yi ) } ni=1 including C classes , xi represents the input data and yi is the label . DNN is composed of a feature extractor h ( · ) : RD → Rd and a linear classifier g ( · ) : Rd → RC . After centralizing the learned features ( i.e . 1n ∑n i=1 h ( xi ) = 0 ) , we decompose the learned features by spectral decomposition , which is similar to PCA : 1 n n∑ i=1 h ( xi ) h ( xi ) T = d∑ j=1 ujλju T j ( 2 ) where λj means the eigenvalues with index j and uj ∈ Rd represents its eigenvector . Specifically , we train ResNet-18 ( He et al. , 2016 ) using both standard training and adversarial training on CIFAR-10 . The parameters for AT are the same as ( Rice et al. , 2020 ) . We calculate the eigenvalues by applying Eq . ( 2 ) , and the eigenvalues are plotted in Figure 2 . The features come from the penultimate layer ( 512 dimensions ) . All the features are extracted from the test set in CIFAR-10 . A part of the eigenvalues is shown for better visualization . advori Difference of models in spectral analysis . As shown in Figure 2 ( a ) ( b ) , the eigenvalues of a standard trained model drop rapidly at some point , while the sharp decrease of eigenvalues is much alleviated by AT . These sharp spectral signature in standard training makes just a few eigenvalues informative from the opinion of PCA . The model fails to explore the influence of eigenvectors with smaller eigenvalues on classification , so the trained model could not recognize the change of features along eigenvectors with smaller eigenvalues . The eigenvectors which may endow useful features are overly penalized . Consequently , we propose a hypothesis that the severe dominance of the top eigenvectors is a cause of vulnerability in DNN , and the adversary adds more components in eigenvectors with smaller eigenvalues . We will verify the proposed hypothesis in the next section . Connection with intrinsic dimension . We introduce intrinsic dimension ( ID ) to quantitatively describe the decreasing tendency in eigenvalues . ID ( i.e. , the minimal number of parameters needed to describe a representation ) has a close connection with natural accuracy ( Ansuini et al. , 2019 ) . It has been found that reduction of ID contributes to an improvement on natural accuracy . We adopt PC-ID proposed by ( Ansuini et al. , 2019 ) to estimate ID , which is determined by the number of principal components included to describe 90 % of the variance . We mark ID in Figure 2 with solid circles . The results reveal the ID of a standard trained model is very small , while models obtained by AT achieve a higher ID , which is contrary to the influence of ID on natural accuracy . This also verifies that there exists a trade-off between generalization and robustness ( Tsipras et al. , 2019 ; Zhang et al. , 2019 ) from the perspective of ID . FSR could further increase ID , based on AT . 3.2 VARIATION ALONG EIGENVECTORS UNDER ATTACKS . In this section , we aim to verify the hypothesis that adversary adds more components along eigenvectors with smaller eigenvalues in attacking stage . We define a metric to quantitatively observe the change of features along different eigenvectors under attack , called variation in this paper . Definition 1 ( Alignment ) Given a dataset Ds = { xs , i , ys , i } ni=1 which may be perturbed . The alignment of Ds to the pre-given direction uj is calculated by the expectation over cosine similarity between features extracted by DNN and the direction vector uj : align ( Ds , uj ) = E ( xs , i , ys , i ) ∈Ds |〈h ( xs , i ) , uj〉| ‖h ( xs , i ) ‖ · ‖uj‖ ( 3 ) where the norm ‖·‖ used is Euclidian norm , and uj is calculated by Eq . ( 2 ) . The calculation of uj is based on features covariance of natural samples , and uj is fixed . Definition 2 ( Variation ) Given a dataset D consist of natural examples and its perturbed dataset Dadv . The variation on direction uj is defined as the ratio between alignment on Dadv and D : r ( Dadv , D , uj ) = align ( Dadv , uj ) align ( D , uj ) ( 4 ) The alignment is correlated with the distance between subspace spanned by uj and the actual feature space , so the change of alignment under attack is suitable to describe the influence of attacks on direction uj , which is called as variation . Our metric is similar to ( Tran et al. , 2018 ; Hayase et al. , 2021 ) used for analyzing backdoors , but we define the alignment by cosine similarity while the latter uses the inner product . Compared with inner product , cosine similarity could eliminate the influence of scale . In Appendix B.1 , we provide a complete procedure to calculate the metrics . We give an intuitive explanation on why the defined variation is suitable to capture the change of features along different eigenvectors under attacks . An toy model for illustration is shown in Figure 3 . Suppose the distribution transfers from Figure 3 ( a ) to ( b ) under attack , we draw an original data x and its shifted data xadv . Take the fixed direction u2 as an illustration . The cosine similarity between x and u2 is equal to cos ( θ ) , i.e. , cos ( θ ) = 〈x , u2〉 / ‖x‖ · ‖u2‖ , which is called as alignment above . If the distribution moves from Figure 3 ( a ) to ( b ) , then the cosine similarity cos ( θ ) increases . Consequently , the change of the cosine similarity can describe how the features change along a direction under attacks . For the vari- ation defined in Eq . ( 4 ) , r ( Dadv , D , uj ) > 1 means the features add more components along direction uj , and vice versa . We could compare r ( D , D , uj ) with 1 to observe whether the adversary adds or reduces the components along various eigenvectors . The adversary adds more components along the eigenvectors with smaller eigenvalues . We visualize the variation of features to different eigenvectors { u1 , · · · , u512 } . The results of CIFAR10 and SVHN are shown in Figure 4 . The attacks used include FGSM ( Goodfellow et al. , 2015 ) , PGD ( step size /10 for 10 steps ) ( Madry et al. , 2018 ) and C & W attack ( Carlini & Wagner , 2017 ) . We set attack budget = 4/255 constrained by ` ∞ norm . As observed in Figure 4 ( a ) , the variation keeps close or smaller than 1 for the several largest eigenvalues in standard trained model . However , the variation of smaller eigenvalues is much larger than 1 . This means that FGSM tends to add more components along the eigenvectors uj with smaller eigenvalues in standard training . A similar phenomenon also exists in PGD and C & W , which verifies the hypothesis in Section 3.1 . We also provide the curve while changing budget and the variation on CIAFR-100 in Appendix B.2 . For models trained by AT , variation of all eigenvectors keeps close to 1 , which is distinctly different from standard training . The high variation of directions with smaller eigenvalues visibly decreases . It is worth noticing that we apply cosine similarity for definition , so the influence from the scale of features is eliminated . Therefore , the components corresponding to the larger eigenvalues are more robust while the smaller ones are more fragile . Adopting the opinion of robust features ( Ilyas et al. , 2019 ) , the features along the eigenvectors with larger eigenvalues are regarded as robust features , and these along the direction with smaller eigenvalues are non-robust features . The analysis above motivates us to regularize the spectrum signatures to alleviate the rapid decreasing tendency . If we alleviate the dominance of the top eigenvalues , the adversarial robustness is improved .

Summary: By analyzing the spectral difference between the natural and adversarial examples, this paper finds that eigenvectors with smaller eigenvalues are more non-robust and adversary trends to add more components into these directions. To eliminate the dominance of the top eigenvalues, the paper proposes Feature Spectral Regularization (FSR), which adds more penalties to the largest eigenvalues while relatively increasing the smaller ones. Several experimental results demonstrate that FSR can further improve the robustness when combined with other adversarial defenses.