Delving into Feature Space: Improving Adversarial Robustness by Feature Spectral Regularization
Authors: Zhen Cheng
Fei Zhu
Xu-yao Zhang
Cheng-lin Liu
Venue: ICLR 2022 Submitted (2022)
Keywords: adversarial example
adversarial robustness
spectral signature

Abstract: The study of adversarial examples in deep neural networks has attracted great attention. Numerous methods are proposed to eliminate the gap of features between natural examples and adversarial examples. Nevertheless, every feature may play a different role in adversarial robustness. It is worth exploring which feature is more beneficial for robustness. In this paper, we delve into this problem from the perspective of spectral analysis in feature space. We define a new metric to measure the change of features along eigenvectors under adversarial attacks. One key finding is that eigenvectors with smaller eigenvalues are more non-robust, i.e., adversary adds more components along such directions. We attribute this phenomenon to the dominance of the top eigenvalues. To alleviate this problem, we propose a method called \textit{Feature Spectral Regularization (FSR)} to penalize the largest eigenvalue, and as a result, the other smaller eigenvalues get increased relatively. Comprehensive experiments demonstrate that FSR is effective to alleviate the dominance of larger eigenvalues and improve adversarial robustness on different datasets. Our codes will be publicly available soon.

Under review as a conference paper at ICLR 2022
DELVING INTO FEATURE SPACE: IMPROVING ADVER- SARIAL ROBUSTNESS BY FEATURE SPECTRAL REGU- LARIZATION
Anonymous authors Paper under double-bling review
ABSTRACT
The study of adversarial examples deep neural networks has attracted great at- tention. Numerous methods are proposed to eliminate the gap of features between natural examples and adversarial examples. Nevertheless, every feature may play different role in adversarial robustness. It is worth exploring which feature is more beneficial for robustness. In this paper, we delve into this problem from metric
the perspective of spectral analysis in feature space. We define measure the change of features along eigenvectors under adversarial attacks. One key finding is that eigenvectors with smaller eigenvalues are more non-robust, i.e.. adversary adds more components along such directions. We attribute this phe- nomenon the dominance of the top eigenvalues. To alleviate this problem, we propose a method called Feature Spectral Regularization (FSR) to penalize the largest eigenvalue, and as result, the other smaller eigenvalues get increased relatively. Comprehensive experiments demonstrate that FSR is effective to alle- viate the dominance of larger eigenvalues and improve adversarial robustness on different datasets. Our codes will be publicly available soon.
INTRODUCTION
shown that the performance of Deep Neural Networks (DNNs) decreases dramatically when confronted with adversarial examples (Biggio et al., 2013; Szegedy et al., 2013; Goodfellow et al., 2015). The vulnerability of DNNs brings potential risk to safety-critical deployments (Finlayson et al., 2019). To mitigate this vulnerability of DNNs, numerous methods are proposed to improve adversarial robustness (Papernot et al., 2016b; Madry et al., 2018; Xie et al., 2019). Among these, adversarial training (AT) (Madry et al., 2018) 8) is the most effective approach that achieves state-of- the-art performance under various attacks (Croce et al., 2020). Different from standard training, adversarial training trains DNNs adversarial examples rather than natural examples.
There are numerous methods proposed to eliminate the gap of features between natural examples and adversarial examples (Kannan al., 2018; Zhang & Wang, 2019; Zhang al., 2019). However, they constrain the features on the whole without considering the distinction of contribution from an individual feature. This may be inappropriate since every feature may play a different role in robustness. The work of (Ilyas et al., 2019) argued that adversarial examples result from non-robust features (Ilyas al., 2019), i.e., well-generalizing but brittle features. This inspires us that we could treat each feature separately in the adversarial setting. While the work of (Bai et al., 2021; Yan al., 2021) considered the influence of different channels on robustness, the problem has not been explored clearly from the perspective of spectral signatures, i.e., the eigenvalues and eigenvectors of feature covariance. The space split into many spectral components, and then it worth analyzing which component is beneficial for adversarial robustness and which is fragile under attack.
In this paper, we show that spectral signatures of deep features have a close connection with ad- versarial robustness. Through applying principal component analysis (PCA) to deep features, we could split feature space to various eigenvectors with its eigenvalues. Our motivation comes from the phenomenon that the standard trained model often results in a sharp distribution of eigenval- ues (Yu et al., 2020), i.c., the eigenvalues rapidly become very small as shown in Figure 1. This property may be beneficial for natural accuracy (Lezama al., 2018; Papyan al., 2020), while its

---

Under review as a conference paper at ICLR 2022
LCE
standard training
classifier
eigenvalue
AT + FSR
batch
SVD
LFSR
feature extractor
Feature Spectral Regularization
0
eigenvector index
Figure 1: The architecture of DNN with Feature Spectral Regularization (FSR). FSR alleviates the dominance of top eigenvalues and enhances the role of relatively smaller eigenvalues in classifica- tion. When combined with AT, FSR further help learn more diverse features with higher dimensions. In the curve, the solid circles denote the estimated intrinsic dimension of features.
impact on robust generalization unclear. However, we hypothesize that the sharp distribution of eigenvalues impels models to learn less diverse features and is cause of the vulnerability of DNNs. minority of eigenvalues occupy the overwhelming majority in the sum of eigenvalues, which may make the model pay little attention to features along the eigenvectors with smaller cigenvalues, so features along such directions are not generalized during training. To verify our hypothesis, we de- fine a new metric to measure the variation of features along different eigenvectors under attacks, as shown in Figure 4. Our observation reveals that the adversary tends to add more components along eigenvectors with smaller eigenvalues, and such huge variation could be distinctly alleviated by AT.
Therefore, we propose to improve adversarial robustness by alleviating the sharp distribution of eigenvalues. Considering that more eigenvalues should occupy the majority, we propose regular- izer named Feature Spectral Regularization (FSR) to penalize the largest eigenvalue of the feature matrix covariance. Empirical evidence shows that FSR increases the overall eigenvalues relatively, making models focus on more spectral components during training. We also provide a theoretical explanation with robust linear regression. Comprehensive experiments confirm that FSR indeed im- proves the adversarial robustness on several datasets. Our contributions are summarized as follows:
We find close connection between spectral signatures of features and adversarial robust- ness. On one hand, standard trained model presents a sharp distribution of eigenvalues, which is beneficial for natural accuracy while harmful in adversarial setting. On the other hand, the adversary tends to add more quantity along eigenvectors with smaller eigenvalues.
We propose Feature Spectral Regularization (FSR) to increase the overall eigenvalues rela- tively in deep feature space, thus alleviating the sharp distribution of eigenvalues. Further- more, we provide a theoretical explanation based robust linear regression.
We empirically show that FSR improves adversarial robustness and alleviates the sharp distribution of eigenvalues, through comprehensive experiments.
RELATED WORK
Ad Adversarial Defense. Many defense methods have been proposed to improve adversarial robust- ness since the discovery of adversarial examples (Papernot et al., 2016b; Xie et al., 2019; Carmon al.ll.2011 2019; Zhang al., 2020). However, many of them are proven be noneffective because they highly depend on obfuscated gradients (Athalye et al., 2018) or gradient masking (Papernot et al., 2017). Among these, adversarial training (Madry et al., 2018) now regarded the state-of-the- art method (Rice et al., 2020; Pang et al., 2021). Distinguished form standard training, ad adversarial training trains DNN on adversarial examples:
min max ERwDeD LCE (xr 6, v: [|
where, D is the dataset, the parameters of DNN are denoted 0, 8 means the perturbation within the e-ball, and Log is the cross-entropy (CE) loss.
By introducing a trade-off between robustness and generalization. TRADES (Zhang et al., 2019) another framework that reaches comparative robustness with AT. Among the proposed methods based AT, Adversarial Weight Perturbation (AWP) (Wu et al., 2020) explicitly regularizes the flatness weight loss landscape, and forms a curbbe-petturbation mechanism, which shows huge improvement on adversarial robustness and alleviates the overfitting in AT (Rice et al., 2020).
2

---

Under review as a conference paper at ICLR 2022
Spectral Signatures of Feature Representations. Some studies have revealed that the spectral signatures features influence the performance in various learning tasks. The spectral properties are crucial to detect backdoors (Tran et al., 2018; Hayase et al., 2021). The cigenvectors corresponding to the larger eigenvalues are found to dominate the transferability of features in adversarial domain adaptation (Chen al., 2019b). The work of (Chen al., 2019a) explores the correlation between negative transfer and the spectral components of features and weights in inductive transfer learning. By utilizing the principle of Maximal Coding Rate Reduction (MCR²,, it is theoretically proven that the larger several singular values of feature matrix for every class should be equal to learn the maximally diverse representation (Yu et al., 2020; Chan et al., 2020).
Different from these studies, we analyse the connection between adversarial robustness and spectral components of deep features. We aim to explore which components are more fragile under attacks, and propose method to boost adversarial robustness by constraining spectral properties.
3 SPECTRAL ANALYSIS IN FEATURE SPACE
In this section, we investigate the connection between spectral signatures and adversarial robustness. Concretely, we train models by standard training and adversarial training (Madry et al., 2018), and then apply spectral decomposition to attain the spectral signatures. We find that in standard training the eigenvalues reveal rapidly descending curve while this tendency is alleviated by AT. Then, we find that the adversary tends to add more quantity along eigenvectors with smaller eigenvalues.
3.1 CURVE OF EIGENVALUES AND ADVERSARIAL ROBUSTNESS
Given a dataset D = {(.,, yt) P= including C classes, I; represents the input data and yi is the label. DNN composed feature extractor h (.) : IRD + Rd and a linear classifier y (-) : Rd - RC. After centralizing the learned features (i.e. Σ h (xx) 0), we decompose the learned
features by spectral decomposition, which is similar to PCA:
(()) Σ un
where Xy means the eigenvalues with index j and uy E IRd represents its eigenvector.
Specifically, we train ResNet-18 (He et al., 2016) using both standard training and adversarial train- ing on CIFAR-1O. The parameters for AT are the same as (Rice et al., 2020). We calculate the cigenvalues by applying Eq. (2), and the eigenvalues are plotted in Figure 2. The features come from the penultimate layer (512 dimensions). All the features are extracted from the test set in CIFAR-10. A part of the eigenvalues is shown for better visualization.
Difference of models in spectral analysis. As shown in Figure 2(a)(b), the eigenvalues of a standard trained model drop rapidly at some point, while the sharp decrease of eigenvalues is much alleviated by AT. These sharp spec- tral signature in standard training makes just few eigenvalues informative from the opinion of PCA. The model fails explore the influence of eigenvectors with smaller eigenvalues on clas- sification, so the trained model could not rec- ognize the change of features along eigenvec- tors with smaller eigenvalues. The eigenvectors which may endow useful features are overly pe- nalized. Consequently, we propose a hypothe- sis that the severe dominance of the top eigen- vectors is a cause of vulnerability in DNN, and the adversary adds more components in eigen- vectors with smaller eigenvalues. We will verify the proposed hypothesis in the next section.
standard tralning
standard teining
39
0.
AT
08
FSR
07
07
05
D:C
06
05
04
(PC-D:9)
03
1PC 221
02
10 20 30 40 60
70
10 20
(a) Natural examples
(b) Ab)A diversaiial examples
Figure 2: Spectral analysis of models with fea- tures extracted from (a) natural examples and (b) adversarial examples on CIFAR-10. We scale all the eigenvalues that the largest one is "PC- ID" (Ansuini et al., 2019) denotes the estimated intrinsic dimension (ID) of features. The sharp distribution of eigenvalues in standard trained model leads to a lower ID, while ID becomes higher by imposing adversarial training and FSR.
3

---

Under review as a conference paper at ICLR 2022
Connection with intrinsic dimension. We introduce intrinsic dimension (ID) to quantitatively describe the decreasing tendency in eigenvalues. ID (i.e., the minimal number of parameters needed to describe representation) has a close connection with natural accuracy (Ansuini et al., 2019). It has been found that reduction of ID contributes to an improvement on natural accuracy. We adopt PC-ID proposed by (Ansuini et al., 2019) to estimate ID, which is determined by the number of principal components included to describe 90% of the variance. We mark ID in Figure 2 with solid circles. The results reveal the ID of standard trained model is very small, while models obtained by AT achieve a higher ID, which is contrary to the influence of ID on natural accuracy. This also verifies that there exists a trade-off between generalization and robustness (Tsipras al.. 2019; Zhang et al., 2019) from the perspective of ID. FSR could further increase ID, based on AT.
3.2 VARIATION ALONG EIGENVECTORS UNDER ATTACKS
In this section, we aim to verify the hypothesis that adversary adds more components along eigen- vectors with smaller eigenvalues in attacking stage. We define metric to quantitatively observe the change of features along different eigenvectors under attack, called variation in this paper.
Definition (Alignment) Given a dataset D. = {æs.s. 3r.3"== which may be perturbed. The alignment of D, to the pre-given direction uj is calculated by the expectation over cosine similarity between features extracted by DNN and the direction vector uz:
10h (I ug}و
align (2,,u)) (E Il (I+,))) [|ylll
(3)
where the norm |I-I| used is Euclidian norm, and uj is calculated by Eq. (2). The calculation of uj is based on features covariance of natural samples, and uj is fixed.
Definition 2 (Variation) Given dataset D consist of natural examples and its perturbed dataset D Dodv. The variation on direction uj is defined as the ratio between alignment on Dade and D:
(Dadoy D,u;) align (Dadv: Uy)
align (D, Uz))
The alignment is correlated with the distance between subspace spanned by u, and the actual feature space, so the change of alignment under attack is suitable to describe the influence of attacks on direction uj. which called as variation. Our metric is similar to (Tran et al., 2018; Hayase al., 2021) used for analyzing backdoors, but we define the aligmment by cosine similarity while the latter uses the inner product. Compared with inner product, cosine similarity could eliminate the influence of scale. In Appendix B. we provide a complete procedure to calculate the metrics.
We give an intuitive explanation on why the defined vari- ation is suitable to capture the change of features along different eigenvectors under attacks. An toy model for illustration is shown in Figure 3. Suppose the distribu- tion transfers from Figure 3(a) to (b) under attack, we draw original data and its shifted data Fadv- Take the fixed direction U2 as an illustration. The cosine simi- larity between and tiz is equal cos (8), i.e., cos (f) (ar, u2) / ||>:|| ||ue|| which is called as alignment above. If the distribution moves from Figure 3(a) to (b), then the cosine similarity cos (0) increases. Consequently, the change of the cosine similarity can describe how the fea- tures change along a direction under attacks. For the vari- ation defined in Eq. (4), r( r(Dadv: D, uz)) > 1 means the features add more components along direc- tion uj and vice versa. We could compare r(D) D, u;)) with 1 to observe whether the adversary adds or reduces the components along various eigenvectors.
Fade
up
con (
(o)
(a)
(b)
Figure 3: A toy model that demonstrates the validity of the defined variation.
The adversary adds more components along the eigenvectors with smaller eigenvalues. We visualize the variation of features to different eigenvectors {u..... ,uı1))- The results of CIFAR- 10 and SVHN are shown in Figure 4. The attacks used include FGSM (Goodfellow et al., 2015),
4

---

Under review as a conference paper at ICLR 2022
1.6
1.6
1.5
1.6
aahorana taaking
variation rt)
1.4
1.4
144
1.3
vaniatio
1:2
11
2.3
0.9
0.9
08
0.8
0.8
0
50
100
150
200
250 index
300
350
400
450
500
50
100
150
200
250 index
300
350
400
450 500
50
100 150
200
250 300 350 index
400
450 500
(a) FGSM on CIFAR-10
(b) PGD on CIFAR-10
(c) C&W on CIFAR-10
22
whandardmmengg my
1.6
1.8
1.4
1.6
variation
1.4
1.4
12
0.8
0.8
08
0.6
0.6
50 0
100
150
200 250
100 index
350 400 450 500
50 100 150
200 250 300
350 400 450 500
50 100 150
200 250 300 250 400 450 500 index
index
(d) FGSM on SVHN
(c) PGD on SVHN
(f) C&W on SVHN
Figure 4: Variation on different eigenvectors in feature space under various attacks. The results reveal that the adversary adds more components along the eigenvectors with smaller eigenvalues.
PGD (step size e/10 for 10 steps) (Madry et al., 2018) and C&W attack (Carlini & Wagner, 2017). We set attack budget 4/255 constrained by fo norm. As observed in Figure 4(a), the variation keeps close or smaller than for the several largest eigenvalues in standard trained model. However, the variation of smaller eigenvalues is much larger than 1. This means that FGSM tends add more components along the eigenvectors Mj with smaller eigenvalues in standard training. A similar phenomenon also exists in PGD and C& W, which verifies the hypothesis in Section 3.1. We also provide the curve while changing budget and the variation CIAFR- 100 in Appendix B.2.
For models trained by AT, variation of all eigenvectors keeps close to I, which is distinctly different from standard training. The high variation of directions with smaller eigenvalues visibly decreases. is worth noticing that we apply cosine similarity for definition, so the influence from the scale of features is eliminated. Therefore, the components corresponding to the larger cigenvalues are more robust while the smaller ones are more fragile. Adopting the opinion of robust features (Ilyas et al., 2019), the features along the eigenvectors with larger eigenvalues and these along the direction with smaller eigenvalues are non-robust features. The analysis above motivates us regularize the spectrum signatures alleviate the rapid decreasing tendency. If we alleviate the dominance of the top eigenvalues, the adversarial robustness is improved.
regarded as robust features,
4 FEATURE SPECTRAL REGULARIZATION
In this section, we propose a method to regularize the eigenvalues, aiming to alleviate the dominance of the top eigenvalues. We first propose our method FSR, and then provide a theoretical analysis.
4.1 REALIZATION OF FSR
Through the detailed analysis above, the sharp distribution of eigenvalues weakens the information contained by the smaller cigenvalues, and causes fragility to adversarial attacks. A promising ap- proach to alleviate such phenomenon is to suppressing the largest eigenvalues, which could mitigate the dominance of the largest cigenvalues relatively. Another straightforward idea is to increase the small eigenvalues during training. However, small eigenvalues are usually numerically unstable, which may be easily affected by noise or round-off error in optimization. In this paper, propose a method called Feature Spectral Regularization (FSR) by penalizing the largest eigenvalues of feature covariance in a batch of data:
(R- m 1F)
(F--1r)
((r (R-
LFSR (F) = w (T) Amax
(5)
5

---

Under review as a conference paper at ICLR 2022
where Amax (-) means the largest eigenvalue of matrix, w (7) -min (T/To, 1) is the weighting function of current epoch т with a hyper-paramett To. and E Rm== is a constant matrix with each element equal to 1 for calculating the mean. Moreover, F Rmmxd is the feature matrix of batch composed of row vectors (I,) ", i.e., F - [h (x1) (x2) -............ [h (am))", and is the number of samples in a batch of data.
In practice, we can just access the batch of data approximate the statistics of feature space. How- ever, the eigenvalues change drastically in the early stages of training. which may cause instability on optimization, so we apply piecewise linear function w(T) to smooth the training stage, The Amax (-) in Eq. (5) is equal to the square of the largest singular value in feature matrix (F 1 1F) by Singular Value Decomposition (SVD). So the realization of FSR is very simple with the help of Pytorch (Paszke et al., 2019) in a few lines of code.
Summarized objective loss. Adversarial training (Madry et al., 2018) has been widely proven to be strong baseline in adversarial defense. We build and incorporate the proposed FSR into AT training framework for further improving model robustness significantly. The final objective is:
Lude (z, y, 0) = I.CE (Fade: y/; 0) $PSR 1.SRR (Fado)
(6)
that controls the trade-off between two items, Fado represents the adversarial example generated from by PGD (Madry et al., 201 8) using cross-entropy loss, Fade is the feature matrix of adversarial examples. Besides AT, we also apply FSR to defense methods such TRADES (Zhang et al., 2019) and AT-AWP (Wu et al., 2020). We provide pseudo-code of our method in Algorithm in Appendix A.
where BEsR a hhppr--pranee
Computational complexity. SVD is extra computational cost induced by FSR. For matrix F E Rmxxd the time complexity of SVD is O (min { m² d, m mmdmmmd) -d]]). Since the batch size is often small, the excess cost in computation is negligible compared adversarial training.
4.2 THEORETICAL ANALYSIS
Consider a linear regression model 5 = (2,0) with E perturbation 5 based on the feature = E Rd the underlying parameter obtained by minimizing mean square error is Og Rdd without perturbation. We assume the mean of features [B()) = O and covariance matrix Var(z) = Σ. Following the adversarial risk define by (Xing et al., 2021), Redv expressed in Eq. (7):
Radv (0,6) = Eg max
((Zade, 0) (2, 0o))
(7)
a|=ad ==1|s8
The optimal solution of (7) denoted as Oadv has an similar formulation with the ridge regression:
Ondv (E A E0,,
where À can be regarded as constant (El Ghaoui & Lebret, 1997; Xing et al., 2021).
Given samples {2,, yi) V the feature matrix composed of row vector 2." is denoted as 2 E Ruxd, E R is a column vector composed of real-valued output 9: (i 1, ... n). Previous methods tend regularize the classifier 0 (El Ghaoui & Lebret, 1997; Xu et al., 2008). However, we could directly analyse how the feature Z influences the robustness of model. If the feature is more robust for classifier, then whether using the classifier On or Oado should have the same prediction. Thus, we define the residual risk induced by features Z:
then Σ- nT = [/1, ........ yn]'
min Rrea (Z) || 200l12, s.t. || 50-
is essential to normalize the scale of features for comparable representation, so we restrict the norm of Z as used in (Yu al., 2020). The SVD of matrix Z E R has the form: Z UDVT. Here U = [u1,.. [n.] E [Rax and V E Rdxd are orthogonal matrices. Observe that the constraint for Z only depends on its singular values, i.e., | = Ei=1
min(n.d) of sp- Suppose d and
o; > 0, then Rres is simplified to the expression by combining Eq. (9) and Eq. (8):
d
An
d
min Ryes (o)...., = min (r))).)))
Σ j=1
Uj
u y
s.t. 1=1
o = Sp- (10)
(ar A
0 +
An

---

Under review as a conference paper at ICLR 2022
Table 1:
Test accuracy (%) on CIFAR-10 under thite-box/black bbx box attacks using ResNet-18. The maximum lo perturbation is 8/255. The best results are boldfaced for highlight.
White-box Attack
Black-box Attack
Defense
Natural
FGSM
PGD-20
CW pc
FGSM
PGD-20
CW
AA
AT
82.02
57.08
51.63
50.28
63.63
61.12
61.36
48.13
AT + FSR
82.18
57.45
52.24
51.25
63.74
61.25
61.63
48.71
TRADES
83.80
57.93
51.02
49.53
65.34
62.94
62.68
48.12
TRADES FSR
84.64
58.54
51.66
50.00
65.98
63.60
63.40
48.66
Table 2: Test accuracy (%) on CIFAR-100 under mhite-box/black-boor attacks using ResNet-18 18. The
maximum &or perturbation is c = 8/255.
White-box Attack
Black-box Attack
Defense
Natural
FGSM
PGD-20
CW
FGSM
PGID-20
CW
AA
AT
55.57
30,86
27.62
25,93
41.50
40.94
42.27
23.74
AT + FSR
54.57
32.16
29.18
26.90
41,93
41.42
42.69
24.79
TRADES
55.38
29.77
26.31
23.58
40.76
39.98
41.31
22.65
TRADES FSR
57.55
32.16
28.49
24.99
42.57
41.91
43.22
23.82
Table 3: Test accuracy (%) on SVHN under seatte-bxx/black-ae box attacks using ResNet-18. The
maximum lor perturbation is 8/255.
White-box Attack
Black-box Attack
Defense
Natural
FGSM
PGD-20
CW nc
FGSM
PGD-20
CW oc
AA
AT
89.95
59.41
47.68
44.84
67.24
62.09
63.57
41.83
AT + FSR
88.56
59.02
51.17
47.72
67.39
62.72
64.54
44.14
TRADES
92.42
68.60
58.27
55.31
73.13
68.90
70.11
52.54
TRADES FSR
92.69
69.82
59.22
55.69
73.35
69.32
70.60
52.79
Theorem
Ryes (o1,""",0d) .a)) is minimum when all the singular values of Z are equal.
As FSR penalizes the largest singular value in feature matrix, it alleviates the dominance of large singular values and helps contribute to equal singular values under the normalized condition, thus helping reduce the residual risk induced by features. Proof of Theorem is provided in Appendix C.
EXPERIMENTS
In this section, we evaluate the effectiveness of the proposed FSR on CIFAR-10 (Krizhevsky al., 2009), CIFAR-10 100 (Krizhevsky et al., 2009) and SVHN (Netzer et al., 2011). FSR is applied to two baselines: AT (Madry al., 2018; Rice et al., 2020); 2) TRADES (Zhang al., 2019). It worth noticing that AT is the most effective method to improve adversarial robustness (Rice et al., 2020; Pang et al., 2021;
Gowal et al., 2021) in RobustBench (Croce et al., 2020).
Experimental Settings. For CIFAR-10 and CIFAR-100, set the Eo perturbation with 8/255, the step size of attack 2/255, and the inner iteration steps 10. The step size is 1/255 for SVHN. We train ResNet-18 (He et al., 2016) using momentum optimizer with the initial learning rate of 0.1. J. For AT, we train 200 epochs and the learning rate decays with a factor of 0.1 at 100 and 150 epochs (Rice al., 2020). For TRADES, we train 120 epochs with the learning rate divided by 0.1 at epochs 75, 90, and 100 (Zhang et al., 2019). The parameter for regularization (1/A) is set for TRADES. For FSR, we set 8ESR 0.01. Other tlype--paaameted keep the same as their original paper. Considering that the settings have a distinct influence on robustness (Pang et al., 2021; Gowal et al., 2021), the hyper- pypem-parameters remain unchanged while adding our FSR.
PERFORMANCE UNDER WHITE-BOX ATTACKS
To test the adversarial robustness of model in detail, adopt various white-box adversarial attacks: FGSM (Goodfellow et al., 2015), PGD-20 (step size e/4) (Madry et al., 2018), CW (Exe version optimized by PGD) (Carlini & Wagner, 2017). Following the instruction proposed by (Carlini al.,
7

---

Under review as a conference paper at ICLR 2022
Table 4: Test accuracy (%) across different datasets based on final checkpoint under white-box
attacks using ResNet-18. The maximum &o perturbation is 8/255.
CIFAR-I0
CIFAR-100
--00
SVHN
Defense
PGD-20
CW
PGD-20
CW
PGD-20
CW
AT AT + FSR
43.65
44.17
19.94
20.46
41.94
43.35
44.91
44.79
22.46
21.12
44.04
44.25
TRADES
50.85
49.66
26.32
23.44
57.99
55.05
TRADES FSR
51.15
49.54
28.43
24.68
59.11
59.13
Table 5: Test accuracy (%) CIFAR-10 10 under white-hox attacks using ResNet- 18 based on AWP.
We report the results based on best checkpoint. The maximum &or perturbation is e = 8/255.
Defense AT-AWP
Natural
FGSM
PGD-20
CW AA
80.23
59.03
55.13
51.38
49.48
AT-AWP + FSR
80.63
59.11
55.13
51.99
49.93
2019), use random starts at random offsets away from the initial when applying iterative attacks. The settings for evaluation promote quite strong attack. We report the best checkpoint (the highest robustness under PGD from different checkpoints) as used in (Rice al., 2020). The test accuracy is reported in Table 1~33. The results show that FSR can further improve adversarial robustness, compared with the baseline models. We also test the robustness under Auto.Ataak (AA) (Croce & Hein, 2020), which is now regarded as the strongest attack to evaluate adversarial robustness. As shown in the table, FSR is effective to improve robustness under AA. The detailed results show that our FSR can indeed boost robustness, rather than depending on obfuscated gradients (Athalye et al., 2018) or gradient masking (Papernot et al., 2017). The improvement on various datasets reveals that our method has consistent promotion, rather than relying on special dataset.
The performance of final checkpoint another essential probe to analyse the adversarial robustness of models. We also report the performance of final checkpoint, shown in Table 4. The results show that FSR could also improve the performance final checkpoint on various datasets.
Combination with other methods. To validate that FSR is an effective module, we try combine FSR with Adversarial Weight Perturbation (AWP) (Wu et al., 2020). AWP helps improve ad adversarial robustness and effectively alleviates the overfitting (Rice et al., 2020) PGD-based AT. is one of the strongest defense methods in RobustBench (Croce et al., 2020), which means it is high baseline. The results are listed in Table 5, and the results show that we could even attain improvement on ad adversarial robustness based the high baseline AWP, especially under CW attack and AAA.
Table 6: Test accuracy (%) on CIFAR-10 under white-box attacks using 14deResNNet33-10.
FGSM
PGD-20
CW
Defense
Best
Last
Best
Last
Best
Last
AT
61.85
57.69
55.10
47.46
53.54
48. 12
AT FSR
62.22
59.12
54.93
47.89
53.98
48.32
AT-AWP
63.64
63.96
58.17
57.09
55.81
55.11
AT-AWP FSR
64.19
63.59
58.23
58.01
55.99
56.25
Performance under WideResNet. To explore the performance of our model in large networks, we use WideResNet-34-10 (depth 34 and width 10) (Zagoruyko & Komodakis, 2017) as another backbone network. this part, we realize our method based on the well-known effective methods: AT (Madry et al., 2018; Rice et al., 2020) and AT-AWP (Wu et al., 2020) CIFAR-10. 10. All the parameters of baselines are the same as used in their original paper. The settings of threaten model the same ResNet-18 18 in the previous section. We show the results of both best checkpoint and final checkpoint in Table 6. The "Best" means the checkpoint of best performance under PGD, and the "Last" is the performance of final checkpoint. The results verify that method is also effective in WideResNet, and we especially attain great promotion in final checkpoint based on AWP.
5.2 PERFORMANCE UNDER BLACK-BOX ATTACKS
Transferability is an intriguing property about adversarial examples, which can be used to implement black-box attack (Papernot al., 201 60; Liu et al., 2017). Adversarial examples generated by the
8

---

Under review as a conference paper at ICLR 2022
0.9
AT AT+FSR
0.9
AT
0.9
AT
AT+FSR
AT
0.8
0.8
0.7
0.6
0.5
0.6
0.5
0.5
0.5
nomalteed
0.4
commella sanelted
0.4
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0
6
12
10
20
30
40
index J
index
Index
(a) CIFAR-10
(b) CIFAR-100
(c) SVHN
Figure 5: Spectral analysis of FSR. (a) normalized eigenvalues (max-normalized) on CIFAR-10;
(b) normalized eigenvalues on CIFAR- 100; (c) normalized eigenvalues on SVHN.
substitute model can used to attack the target model with the weights and architecture unknown to the adversary. Transfer attack is beneficial to verify that method does not rely on gradient masking (Papernot et al., 2017). As suggested by (Carlini et al., 2019), we adopt another adversari- ally trained ResNet-18 18 as the substitute model. The results shown in Table 1~ 3, revealing that our methods also improve robustness under black-box attacks.
5.3 FURTHER UNDERSTANDING OF FSR
In this part, we analyse the influence of FSR on the eigenvalue spectrum. The results are shown in Figure 5. Our motivation is that the top eigenvalues dominate the distribution of eigenvalues, so we use FSR to increase overall eigenvalues relatively. To eliminate the influence from the scale of features, the eigenvalues are divided by the maximum eigenvalue. As shown in the figure, FSR increases the eigenvalues relatively, which is consistent with our intention for FSR. In Appendix D, show that simply using FSR without AT also promotes mild improvement robustness.
5.4 ABLATION STUDIES
Sensitivity analysis of 3FSR- We explore how the weight of FSR fFSR influences the perfor- mance of models. Following the same settings as the previous section, we train models on CIFAR-10 using R8sNet-18 with the change 3FSR- The results are reported Table 7. As ob- served in Table 7, FSR significantly improve robustness with wide value range, which verifies the stability and effectiveness of the proposed method. In this paper, we choose the parameter by considering both generalization and robustness, adversarial robustness, tuning frsn could still promote a further improvement on robustness.
choose 3rsn 0.01. If we focus merely
Table 7: Test accuracy (%) under white-box attacks for different 8F5R-
Brsn
0
0.005
0.010
0.020
0.060
Natural
82.02
81.96
82.18
81.84
81.74
PGD-20
51.63
52.42
52.24
52.77
52.21
CW
50.28
51.04
50.77
51.21
50.75
6 CONCLUSION
In this paper, we delve into the discrepancy between natural examples and adversarial examples from the perspective of spectral analysis. The variation of different eigenvectors under adversarial attacks is analysed. It is shown that the spectral directions with smaller eigenvalues are more fragile under attack, which induced by the dominance the top eigenvectors. Based on the analysis, a method called Feature Spectral Regularization (FSR) is proposed to penalize the largest eigenvalues of the batch covariance matrix, which numerically stable and could enlarge the overall eigenvalues rela- tively. We also provide theoretical explanation in robust linear regression. FSR is simple module readily pluggable into any defense method. Through comprchensive experiments, we show that FSR effectively improve adversarial robustness when combined with many defense methods.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. 3,
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity; Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283, PMLR, 2018. 2, 8
Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang. Improving adversarial robustness via channel-wise activation suppressing. In International Conference Learning Representations, 2021.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Sindić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 387- 402. Springer, 2013. 1
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 Ieee Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017. 5,7
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint 100.:0:900.06705 2019. 7,9
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, and John Duchi. Unlabeled data improves adversarial robustness. Advances in Neural Information Processing Systems, 2019. 2
Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma. Deep networks from the principle of rate reduction. arXiv preprint 1rXiv:2010.14765, 2020. 3
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019a. 3
Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discriminabil- ity: Batch spectral penalization for adversarial domain adaptation. In International Conference on Machine Learning, pp. 1091-1090. PMLR, 2019b. 3
Francesco Croce and Matthiss Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fie free attacks. In International Conference on Machine Learning, pp. 2206- 2216. PMLR, 2020. 8
Francesco Croce, Maksym Andriushchenko. Vikash Sehwag. Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness bench- mark. arXiv preprint arXiv:2010..0 000007,, 2020. 1, 7,8
Laurent EI Ghaoui and Hervé Lebret. Robust solutions to least-squares problems with uncertain data. SIAM Journal on Matrix Analysis and Applications, 18(4):1035-104, 1997. 6, 17
Samuel G Finlayson, John D Bowers, Joichi Ito, Jonathan L Zittrain, Andrew Beam, and Isaac Kohane. Adversarial attacks medical machine learning. Science, 363 2019.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015. 1, 4,
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference Machine Learning, pp. 1321-1330, 2017. 18
10

---

Under review as a conference paper at ICLR 2022
Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. Defense against backdoor attacks via robust covariance estimation. In International Conference on Machine Learning, pp- 4129-4133 PMLR, 2021. 3,4
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp- 770-778, 2016. 3,7
Yujia Huang, Huan Zhang, Yuanyuan Shi, Zico Kolter, and Anima Anandkumar. Training certifi- ably robust neural networks with efficient local lipschitz bounds. Advances Neural Information Processing Systems, 2021. 20
Andrew Ilyas, Shibani Santurkar, Logan Engstrom, Brandon Tran, and Aleksander Madry. Ad- versarial examples are not bugs, they are features. Advances in Neural Information Processing Systems, 32, 2019. 5.5
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456.
PMLR, 2015. 20
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint 1803.06333. 2018. 1
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
José Lezama, Qiang Qiu, Pablo Musé, and Guillermo Sapiro. Ole: Orthogonal low-rank embedding- plug and play geometric loss for deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8109-8118, 2018.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial ex- amples and black-box attacks. In International Conference on Learning Representations, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. 2, 5, 6, 7, 8, 14
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015. 18
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011. 7
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag tricks for adversarial training. In International Conference on Learning Representations, 2021. 2.7
Nicolas Papernot, Patrick McDaniel, and lan Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016a.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEE, 2016b. 1,2
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-516699999 519, 2017. 2, 8,9
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40): 14652-24663 2020.
11

---

Under review as a conference paper at ICLR 2022
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32: 8026-8037, 2019. 6
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference Machine Learning, pp. 1993-8104. PMLR, 2020. 2, 3, 7, 8, 14
Kevin Roth, Yannic Kilcher, and Thomas Hofmann, Adversarial training is a form of data-dependent operator regularization. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin in Neural Information Processing Systems, volume 33, pp. 14973-14985. Curran Associates, Inc., 2020. 20
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Ad- versarially robust generalization requires more data. S. Bengio, Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianee and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In Inter- national Conference Learning Representations, 2018. 20
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6199, 2013.
1, 20
Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. In Advances in Neural Information Processing Systems, volume 31, Curran Associates, Inc., 2018. 3,4
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. International Conference on Learning Representa- fions, 2019. 4
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gen- eralization. In Advances in Neural Information Processing Systems, volume 33, pp. 1958-2966 Curran Associates, Inc., 2020. 2, 6,
Cihang Xie, Yuxin Wu, Laurens der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 501-509, 2019. 1,2
Yue Xing, Ruizhi Zhang, and Guang Cheng. Adversarially robust estimate and risk analysis in linear regression. In International Conference on Artificial Intelligence and Statistics, pp. 514- 522. PMLR, 2021. 6, 17
Huan Xu, Constantine Caramanis, and Shie Mannor. Robust regression and lasso. In Proceedings of the 21st International Conference on Neural Information Processing Systems, pp. 1801-1808, 2008. 6
Hanshu Yan, Jingfeng Zhang, Gang Niu, Jiashi Feng, Vincent Tan, and Masashi Sugiyama. Cifs: Improving adversarial robustness of cnns via channel-wise importance-based feature selection. In International Conference on Machine Learning, pp. 11693-111111 11703. PMLR, 2021.
Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning, 2017. 19, 20
Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in Neural Information Processing Systems, 33, 2020. 3, 6, 17
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks, 2017. 8
12

---

Under review as a conference paper at ICLR 2022
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering- bassd adversarial training. Advances Neural Information Processing Systems, 32:1831-18 1841, 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning. pp. 7472-7482. PMLR, 2019. 1, 2, 4, 6,
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning, pp. 11278- 1287. PMLR, 2020.
13

---

Under review as a conference paper at ICLR 2022
A
ALGORITHM OF FSR
Algorithm Robust Training with FSR
Input: Training data D ((z yt) hi-1,-... NN DNN composed of feature extractor h (-) and linear
classifier 9 (.), maximum training steps T
Output: Robust model { (-)
for in 21,2.....
do
2: Read mini-batch ( [
3: Generate adversarial robustness using PGD
4: Compute the FSR loss in Eq. (5) using adversarial examples
5: Compute the overall loss in Eq. (6)
6: Optimize all parameters of model using gradient descent
7: end for
B THE DEFINED VARIATION ON MORE DATASETS
In this part, we provide more experiments about the variation defined by us. We first provide detailed algorithm to calculate the variation. More results about the variation are shown following.
B. ALGORITHM FOR CALCULATING THE VARIATION
Algorithm 2 Calculating the Alignment and Variation
Require: Dataset D {(1,, ui) NN (natural examples), the feature extractor h (-).
Ensure: Variation to different cigenvectors r (Dudes D, uz) (j 1,. d).
1: Centralize the features h (x:) < (zz) & h (x;)
2: Calculate the covariance matrix N Σm l (x;))
3: Apply eigendecompositiin the covariance matrix get the eigenvalues and eigenvectors
(Ai,... Aa) {ui,... ua} eig N E (r;)h (z:)²)
4: Generate the adversarial example Fado, from I; by attack (i = 1,,...,,
5: Centralize the features of adversarial examples l (Fadv,s) + h (Fadv,)) 2 Ex h (Fodva))
6: for j 1,... do
7: Calculate the cosine similarity between features and eigenvectors align (D, ug) + MN (h (æ:) u/)l// ||h (a:))l ||ull|
align ( (Dudey U2)
Li=1 (Radv.;) :): uz>l/ [|h (Fade,)) [|uyll
8: Calculate the variation different eigenvectors
T (Dadu, D,u, align (Dadv, uz) /align (D, uz)
9: end for
B.2 MORE EXPERIMENTS ABOUT THE DEFINED VARIATION
In this part, we show more results about variation to eigenvectors on CIFAR-I0, CIFAR-100 and SVHN. We compare the models trained natural examples and adversarial examples. The adver- sarially trained models adopt the PGD-AT Madry et al. (2018) with foo perturbation 8/255, and the parameters are the same Rice al. (2020). From the curve shown in Figure 6~-8, the variation on CIFAR- 100 and SVHN reveals the same tendency as CIFAR-10, i.c, adversarial attacks tend to increase the components along the eigenvectors with smaller eigenvalues.
that
14

---

Under review as a conference paper at ICLR 2022
1,06
2
FOOM (or 09851
1.05
1.04
18
1.03
vanation
varation
1.02
1.4
1.01
1.2
0.99
0.8
0.97
FOSM (1-4)256)
0.6
0.96
201
251
301
351
51
102
151
201
251
301
351
4011
451
501
index
index
(a) Variation FGSM (STD)
(b) Variation to FGSM (AT)
1.6
1.05
1.5
POD (5-0225)
1.04
(146725)
1.03
1.3
1.02
Vanation
variation
1.2
1.1
(1111225)
0.9
PGD (L-4)256)
0.8
0.96
101
151
201
253
index
index
(c) Variation to PGD (STD)
(d) Variation to PGD (AT)
1.04
1.5
Cant (m00786)
1.03
1.4
1.02
variation
1.2
variation
1.01
1.1
Colf (H-2209)
0.98
0.8
0.97
51
101
151
201
251 index
301
351
401
451
501
51
101
151
201
251 index
301
351
401
451
501
(e) Variation to C&W (STD)
(l) Variation C&W (AT)
Figure 6: Variation of all eigenvectors in feature space to adversarial attacks on CIFAR-10. 'STD' means the model is trained on natural examples. 'AT' means the model is trained on adversarial examples. It should be noticed that the range of ordinate values for 'STD' is different from 'AT'.
1.02
1.015
variaton (
12
1.01
1.1
variaton
1
0.995
0.9
FOSM juet
0.99
FOSM
(u-2255)
0.985
0.98
151
201
251
301
351
401
451
501
51
101
151
201
251
301
351
401
451
501
index
index
(a) Variation FGSM (STD)
(b) Variation to FGSM (AT)
1.3
1.02
1.035
variation r.)
1.2
1.07
veriation
0.995
0.9
PCR (1020000)
0.99
12330) (2-4220)
PCD (1950255
0.985
0.8
101
151
201
251
301
351
501
101
151
201
251
301
351
401
451
501
index
Index
(c) Variation to PGD (STD)
(d) Variation to PGD (AT)
15

---

Under review as a conference paper at ICLR 2022
1.2
variation (()
variation r()
1.01
1.005
GAW (1-2299)
O6W (L-4299))
0.99
CAN (144788))
0.8
0.985
101
151
201
251
301
351
451
501
101
151
201
251
301
351
401
451
501
index
index
(e) Variation to C&W (STD)
Variation C&W (AT)
Figure 7: Variation of all eigenvectors in feature space to adversarial attacks on CIFAR-100.
1.06
variation r
14
variation
1.02
FOOM FG2M (2-4225))
POSM (C-41256)
0.8
0.96
101
151
201
251
301
351
401
451
501
51
101
151
201
251
301
351
401
451
501
index
index
(a) Variation FGSM (STD)
(b) Variation to FGSM (AT)
2
variation (
1.8
(e222255) (e-4255)
1.04
variation
12
PCD 250)
0.96
0.6
0.92
151
201
251
301
351
401
451
501
1
51
101
151
201
251
301
351
401
451
501
index
index
Variation PGD (STD)
(d) Variation PGD (AT)
1.1
1.8
1
(2462255)
Vanation
1.4
variation
1.2
1
0.95
CAW (mn1
0.8
CBW (L-4256)
0.6
351
index
index
(c) Variation to C&W (STD)
(f) Variation to C&W (AT)
Figure 8: Variation of all eigenvectors in feature space adversarial attacks SVHN.
C
PROOF
In this part, we provide the proof of Theorem in Section 4.2.
Consider the linear regression model,
i = (2,6)
(11)
where 2 E dd means the feature, and 0 E Rd represents the parameter of model. We assume that the features have mean, and the covariance is denoted as Σ, ie.. [[2) = 0, Var(z) E.
16

---

Under review as a conference paper at ICLR 2022
Given samples (2t, Vi 1= the feature matrix composed of row vectors 2. is represented feature matrix Z R then Σ 1 zzz. Assuming that Z has full column rank, if we minimize the n residual sum of squares in Eq. (11), then the solution is:
0o (Z" 7" y.
(12)
where y E I means the real-valued output, i.e., y² (y1, .2. :.))
When the feature 2 is noised by le perturbation 8, we can define the adversarial risk induced by the perturbation as (Xing et al., 2021):
Radi (0,6) 6) = EE max ((Zadu: 0) - (z, 00)))
(13)
I| =||5a
The optimal solution to Eq. (13) denoted as Oddo is
Bddv (2 (2+ x1) -00
(14)
where À can be regarded as constant. The work of (EI Ghaoui & Lebret, 1997; Xing et al., 2021)
provides more details about how get
A better feature should lead to the same prediction whether using classifier fg or oodv so we define the residual risk induced by features
min Ryes (Z) = min |20 - Z0oll2: 8.l. 1 2
80-
(15)
2
We normalize the scale of features by Frobenius norm to avoid the influence of scale (Yu et al., 2020), which is just correlated to its singular values {a1, (" aa)):
1 trace (Z²Z)=
0j 50-
(16)
j=1
The SVD of the matrix Z has the form:
2 = UDV,
(17)
Do 0
where [M,,'.. un] E Raxn and V E Raxd are orthogonal matrices. D
Raxd
and Da diag ("i, Ja), 02 dd.
After applying SVD, we can simplify the Eq. (15) to following expression:
An
min Rrea (o1,.... .a)) min (01e-(00)) (0a)
U
uj
y
s.t.
>
o
80-
(18)
An
j=1
According to the property of the subordinate matrix norm, we could derive an upper bound of R
d
An
d
An
U 0 + An v
Σ 4 + Xn
llvll
(19)
Consider that y can be arbitrary value, so the minimization of Ryes requires that the first item the right of the inequality sign in Eq. (19) is as small as possible, i,e,
An
min (or Ra)
Σ
An uy
min (ompnraa
(20)
03 + An
The minimization of Rres is simplified to the following expression:
arg min Rres (01, .aa) arg min (ap.".a (a))
An (IT." .ra) o + An"
s.t.
Σ
0j
Sg.
(21)
j=1
Since ai 2 02
Fa, the optimal solution of Eq. (21) is that all singular values equal.
17

---

Under review as a conference paper at ICLR 2022
D FSR IMPROVES CONFIDENCE CALIBRATION AND ROBUSTNESS
We verify that network trained using only FSR without AT attains a mild improvement on adversarial robustness. We test the performance of model under FGSM, and the results are listed in Table 8.
Table 8: Test accuracy (%) under FGSM on CIFAR- 10 using ResNet-15
Attack Strength
1/255
2/255
3/255
4/255
5/255
ERM
65.74
54.61
49.55
46.96
44.69
ERM + FSR
70.01
59.19
53.68
49.71
46.61
In addition to robustness, another important but undesirable aspect of DNN is reliability. When using classifiers for prediction, the predictive confidence should indicative of the actual likelihood of correctness (Guo et al., 2017). Confidence calibration focuses on the mismatch problem between model's s confidence and its correctness. A popular metric used to mcasure calibration is the Expected Calibration Error (ECE) (Nacini et al., 2015; Guo et al., 2017), defined the expected absolute difference between the models' confidence and its accuracy. We explore the efficacy of FSR on confidence calibration.
8
10
7
Eest (%)
+SR
3
TestECE (%)
6
2
test
3
41
61
121 training epochs
101
141
161
181
201
ERM
+FSR
ERM
+FSR ResNet-18
CIFAR-10
ResNet-18
CIFAR-100
(a)
(b)
Figure 9: (a) Comparison of test ECE (lower is better) curves when training ResNet-18 18 on CIFAR-
10. (b) FSR can improve the confidence calibration.
As shown in Figure 9, for the baseline, there is a marked rise in the test ECE at the later period of the training process, which indicates the problem of overconfidence. Remarkably, FSR has obvious improvement on confidence calibration compared with the baseline, e,g., reduces the ECE value from 2.97% 1.18% for CIFAR-IO.
E FURTHER EXPERIMENTS ABOUT ADVERSARIAL ROBUSTNESS
In this part, we add two experiments about the proposed FSR on adversarial robustness. We first report the results (i.e., mean and standard deviation) when all experiments Table 1~33 are repeated five times. Then, the test accuracy of suppressing more eigenvalues is also provided.
E.1 EXPERIMENTS ABOUT STANDARD DEVIATION
We repeat all the experiments in Table 1~3 five times, and report the results under white-box attack following Table. The performance in Table 9~~11 verifies that our proposed method achieves a steady improvement.
18

---

Under review as a conference paper at ICLR 2022
Table Test accuracy (%) on CIFAR-10 under white-box attacks using ResNet-18. The maximum Lo perturbation is 8/255.
Defense
Natural
FGSM
PGD-20
C&W
AA
AT AT FSR
0.224
57.38+0.37
51.52-0.19
50.52+#0.28
48.07=0014
82.57=00.6
58.02:10.52
52.12-0.16
51.36=0.17
48.91=0.17
TRADES
83.73=0.06
58.09=0.14
51.10=0.10
49.67=0.13
48.18=0.13
TRADES + FSR
84.08=0.48
58.43-10.10
51.66=0.05
50.00=0.008
48.62-0.17
Table 10: Test accuracy (6) on CIFAR-100 under white-box attacks using ResNet-18. The maxi-
mum loo perturbation is
8/255.
Defense
Natural
FGSM
PGD-20
C&W
AA
AT
55.63:10.06
30.87:10.12
27.6
26.
23.93:10.17
AT FSR
5..5
32.16+0.14
29.01
26.38
24.76+
TRADES
56.03=10.65
29.04:10.07
23.83::0.32 26.22.10225
22.81
TRADES + FSR
57.64=0.16
32.10=0.008
28.38=0.15
25.14+:021
23.85=0017
Table 11: Test accuracy (%) on SVHN under white-box attacks using ResNet-18. The maximum Lo perturbation is 8/255.
Defense AT AT + FSR
Natural
FGSM
PGD-20
C&W
AA
9..16+0.21
59.94
0.28
45.24
42.00=
09.5
59.79=2.777
5.38
47.85:10224
44.41=0.24
TRADES
92.48-0.11 68.7320.15
58.82=0.34
55.48
52.56=0.06
TRADES + FSR
92.39-00.55
69.72:10.29
59.04
55.64
52.78-0.28
E.2
SUPPRESSING MORE EIGENVALUES
the main body our paper, the proposed FSR only penalizes the largest eigenvalue. In this part, we explore whether penalizing more eigenvalues could achieve better performance under attack. The results are listed in Table 12. As we properly increase the number of suppressed eigenvalues k the adversarial robustness of models could be further improved. However, if we keep increasing k the robustness of the model declines. We think it is due to that & has some trade-off with the weight of FSR BrSR. i.e., larger ls may require smaller 9TSR-
Table 12: Test accuracy (%) CIFAR-10 10 under AutoAttack (AA) using ResNet- 18 while suppress- ing the largest eigenvalues. The maximum loo perturbation is 8/255.
k
1
2
4
8
12
16
AA
48.71
48.95
49.02
49.17
48.55
48.14
F MORE DISCUSSIONS BETWEEN SPECTRAL SIGNATURES AND ROBUSTNESS
In this section, we discuss the connection between our explanation of adversarial examples in the paper and previous studies. We mainly talk about the connection with interpretations form the perspective of Lipschitz constant and manifold.
F.1 CONNECTION BETWEEN FSR AND LIPSCHITZ CONSTANT
In this part, we first rederive some basic conclusions about Lipschitz constant in DNN as illustrated in (Yoshida & Miyato, 2017). For feature extractor g(x)) RD IRd composed of L layers, the weights of each layer in the network are denoted as {w², b'} Then the feature extractor g (z) = Wh (W L- 1-----1 (...) + = + 6%, where o' represents the activation function. We further assume that each activation function a ReLU. In this case, o' act as input-dependent diagonal matrices Φ'x diag (ox), where element in the diagonal dy = 1 1 0) is one
19

---

Under review as a conference paper at ICLR 2022
if its pre-activation 3' := Wl b' is positive (Roth et al., 2020). Then, the global Lipschitz constant of g(x) is bounded by the spectral norm of following expression (Yoshida & Miyato, 2017; Roth et al., 2020; Huang et al., 2021):
Lglob S Wh xx' wh.-l.kk 2 Ø 1
(22)
For better comparison with proposed method, we do not further simplify Eq. (22) like (Yoshida & Miyato, 2017). With the normalization (loffe & Szegedy, 2015) in DNN, the mean of final features always close zero. Suppose that the mean of deep features is equal to zero, then the proposed FSR denoted as EFSR could be simplified to
Ersn Ammax (F
mmax (WL- (WI--10' (....... o (W x b1) ......) + +b" +b²g²
(23)
-|| (Wh-1 (W (Wh-1 (......6' (W x b1) ......) + 1) + 6
=llWL xx Wh-1 $x ... ... W =e12
where the deep features W (WL- (WL-1 (....... of (W + bi) ....... + (---))++ +b+ Amax (.) the largest eigenvalue of a matrix, and Tmax ()) is the spectral norm (largest singular value) of matrix.
we compare Eq. (22) and Eq. (23), FSR has very similar form with the upper bound of global Lipschitz constant. Since the input data is a constant, it could be expected that FSR has effect
to decrease the upper bound the Lipschitz constant. FSR decreases the Lipschitz constant by constraining features rather than directly regularizing weights.
F.2 CONNECTION WITH ADVERSARIAL EXAMPLES ON MANIFOLD
Our observation is consistent with the hypothesis in (Szegedy et al., 2013; Song et al., 2018) that adversarial examples are low-probayility (high-dimensional) "pockets" on the manifold from the perspective of spectral signatures. Our paper reveals that the components along cigenvectors with severely smaller eigenvalues are much rare standard training (seeing Figure 2). When confronted with attacks, the adversary tries to add more components along with such directions (seeing Fig- ure 4). The eigenvectors with smaller eigenvalues could be regarded as "pockets" on the manifold.
20


Reviews:
[{"review_id": "n8LAEJx9vA8", "reviewer": "Reviewer_kop5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The major contribution of this paper is the demonstrations of the connections between adversary and feature spectrum, and the development of a new regularizer applicable in adversarial training. The theoretical analysis on the other hand also supports the proposal of flattening the spectrum.\n\nThe paper is well written with adequate supports from many quantitative and qualitative results, especially those for the feature spectrums.\nOne of the major issues with this paper is the marginal improvement over the SOTA adversarial defense methods, which can be seen in Table 1-5. \n\nAs the SOTA adversarial defense algorithms also have the potential to flatten the spectrum of features (and many other benefits), it is unclear if the FSR will always benefit the defense, as the experiment results showed.\n\nFSR only suppresses the largest eigenvectors, and it seems there is a gap between this operation and goal of adjusting feature spectrums. Why not suppress more eigenvalues? Some explanations would help here.\n\nAnother factor worth further discussion is the basic CNN model. It is unclear whether a strong CNN model will react differently to attack and proposed defense methods, and this could be considered in the experiments.\n\nThe dataset considered only includes low-resolution images, e.g., 32x32. It would be interesting to see if the defense is effective on high-dimensional data, i.e., 224x224 where the adversarial attack is more significant.\n", "summary_of_the_paper": "The paper has presented a new method to improve the robustness of features under adversarial attacks. Authors developed a new metric for the change of features subject to attacks and key findings are eigenvectors of small eigenvalues are more inclined to change under adversarial attacks, i.e., non-robust. Authors believe the dominance of large eigenvalues and eigenvectors are the primary reasons and a way of flattening the spectrum of features would help mitigate this issue. Authors propose to suppress the largest eigenvalues during training, i.e., spectral regularization, which show positive results in adversarial defense when working together with SOTA defense models.", "summary_of_the_review": "In brief, the paper reveals the connections between feature spectrum and non-robust features under adversarial attack and proposes an interesting regularization term for robust training. However, the new method does offer significant improvement over SOTA and was not comprehensively evaluated on high-dimensional data or using different backbone networks.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ERi4GRnVj0", "reviewer": "Reviewer_wNJf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The analysis of connections between the sensitivity to attacks and the subspace of the computed embedding is very interesting. I believe that the authors might have struck here a very interesting research direction, which could fuel further research and lead to new insights.\n\nUnfortunately, the present paper does not go beyond the incidental observations. There is no theoretical guarantee or connection which indicates how the dimensionality of the embedding influences the robustness to attacks or how low-variance subspaces can be exploited by attack methods. The authors do provide a theoretical analysis for robust linear regression models, but the connection/applicability to DNNs is unclear and I also come to the opposite conclusion from the authors, regarding the eigenvalues of the most robust regression model (see below).   \n\nThe experiments show that the proposed regularizer can improve the robustness by a few percent, but we do not observe that a more homogeneous distribution of eigenvalues really prevents adversarial attacks from happening. In this respect, I doubt that the proposed method would be used in practice. Employing the regularizer requires tuning its weight, and even when the authors could show that for Cifar-10, -100 and SVHN a weight around 0.01 is suitable, this might look very different for other datasets. Hence, a theoretical guarantee/connection from the embedding PCA to robustness would be needed in order to estimate in which cases the proposed approach can help and to what extend.\n\n### Theorem 1\nTheorem 1 states that the design matrix $Z$ of a linear regression model yields the most robust predictions if all eigenvalues of $Z$ are equal. The robustness is measured by $\\lVert Z\\theta_a -Z\\theta_0\\rVert^2$, where $\\theta_a = (\\Sigma+\\lambda I )^{-1}\\Sigma\\theta_0$. Using the SVD of $Z=UDV^\\top$, we can rewrite\n$$\n\\begin{align*}\n\\lVert Z\\theta_a -Z\\theta_0\\rVert^2 &= \\lVert \\mathrm{diag}\\left(\\frac{n\\lambda\\sigma_i}{\\sigma_i^2+n\\lambda}\\right)V^\\top \\theta_0\\rVert^2\\\\\n&= \\sum_i \\left( \\frac{n\\lambda\\sigma_i}{\\sigma_i^2+n\\lambda} V_{\\cdot i}^\\top \\theta_0\\right)^2\\\\\n&= \\sum_i  \\frac{n^2\\lambda^2\\sigma_i^2}{(\\sigma_i^2+n\\lambda)^2} (V_{\\cdot i}^\\top \\theta_0)^2\n\\end{align*}\n$$\nThe last term attains its minimizer $Z$ s.t. the constraint $\\lVert Z\\rVert=s_0$ if $\\sigma_1=s_0$, $V_{\\cdot 1}$ orthogonal to $\\theta_0$ and the rest of the singular values being zero. In your proof, you fix $y=z^\\top \\theta_0$, but since you optimize over $Z$ and since $y$ is a function of $z$ I don't see how you can treat $y$ like a constant.\n\n### Minor issues:\n* Use maybe grammarly or another tool to capture the most obvious language errors\n* The space in which PCA is applied and whose eigenvalues are inspected in this work remains largely unclear until Section 3.\n* Is there any connection of the PCA indicators to known robustness influencers, like the Lipschitz constant of the DNN function for example?\n* Can we infer something about the manifold hypothesis (natural examples are on a manifold in the embedded space and adversarial examples are outside of that manifold) from the PCA of the embedding? How can your observation be placed in the scope of the related work on adversarial examples on manifolds?\n* How can we transfer the information about robust linear regression models to DNNs?\n* Why is FSR not evaluated without an additional defense method?\n\n  \n\n ", "summary_of_the_paper": "The authors propose a new defense against adversarial attacks by means of a spectral regularization. The defense is based on an inspection of the embedding of training/test data as returned by the penultimate layer of a neural network. The observation is that the relevant subspace of embedded natural data points (covering e.g. 90% of the variance) is quite low-dimensional, where the use of adversarial training increases the dimensionality of this subspace. In addition, the relevant subspace of embedded adversarial examples is in the provided experiments typically a bit higher dimensional than the subspace of natural examples. Based on these observations, the authors propose a regularizing term, which penalizes the variance in the direction of the first principal component (the largest eigenvalue of the feature covariance matrix).\n\nExperiments are conducted on Cifar-10, -100 and SVHN. The proposed regularizer is applied on top of other defense methods, which typically increases the accuracy under attacks by up to 2%.", "summary_of_the_review": "The authors make an interesting observation about adversarial robustness of DNN classifiers based on a PCA of the embedded data points. However, the observations are not backed up by theoretical insights and the empirical evidence is not very conclusive. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2Umup7Q47m9", "reviewer": "Reviewer_gwLs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) The overall presentation of the paper is clear and easy to follow;\n2) In addition to the empirical experiments, the paper also provides theoretical analysis;\n\nWeaknesses:\n1) The curve trend of standard training (ST) vs. AT in Figure 1 and Figure 2 is not quite consistent?  In Figure 1, the ST curve is almost under the AT curve; while in Figure 2, the AT curve is almost under the ST curve.\n\n2) The paper claims that FSR can further improve the robustness of adversarial defenses, however, if we look at the results, the improvements are very marginal in most cases. So I am wondering: (1) whether this marginal improvement is worth well given we need extra computation on the SVD; (2) if we repeat the experiments many times (say 10 rounds), will these results still statistically hold?\n\n3) How will this method work for a large-scale dataset like ImageNet? Will it cause significant computational overhead due to the SVD? And will it be efficient enough for practical settings?\n\n4) in Figures 6, 7 in the Appendix, the trend for some of the cases seems not as obvious as Figure 4. For example, Figure 6 (e), Figure 7 (d)(f), are all close to 1. I am therefore wondering about the stability of this finding.\n\n5) It is not clear that how it is related to the PCA; need more explanations on this part.\n\n\nMinor issue: under Figure 3, there is a sentence “We could compare r(D, D, u_j)…” should be “r(D_{adv}, D, u_j)”.\n\n\n\n\n\n\n\n#-------------------------------------------------------------------------------------------------------------------------------------------#\n\n#---------------------------------------------------------------- After rebuttal --------------------------------------------------------#\n\n#-------------------------------------------------------------------------------------------------------------------------------------------#\n\nThanks so much for addressing my previous concerns. \n\nBased on the feedback and the available reviews and discussions, I will keep my current score because 1) this method might not be practical to handle large-scale dataset; 2) According to some of the updated experimental results when there are no adversarial attacks in the dataset (and in some types of adversarial attacks), the proposed can actually hurt the performance (compared with the AT). However, in a practical setting, one never knows whether there are adversarial attacks or not in the dataset. \n", "summary_of_the_paper": "By analyzing the spectral difference between the natural and adversarial examples, this paper finds that eigenvectors with smaller eigenvalues are more non-robust and adversary trends to add more components into these directions. To eliminate the dominance of the top eigenvalues, the paper proposes Feature Spectral Regularization (FSR), which adds more penalties to the largest eigenvalues while relatively increasing the smaller ones. Several experimental results demonstrate that FSR can further improve the robustness when combined with other adversarial defenses.  ", "summary_of_the_review": "The overall presentation of this paper is clear and the authors have conducted experiments on different datasets, adversarial attacks, and different adversarial defenses. However, I still have several concerns as I list above, so I will give it a “6: marginally above the acceptance threshold”.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Based on the observation that the eigenvectors with smaller eigenvalues are more non-robust (i.e., adversary adds more components along such directions), the authors propose a method called Feature Spectral Regularization (FSR) to penalize the largest eigenvalue, and as a result, the other smaller eigenvalues get increased relatively.
In this paper, in addition to FSR, theoretical analysis along with experimental results on different datasets and models were presented.
Although the proposed FSR has some merits, the major concerns from the reviewers include (1) impractical use on large-scale datasets and (2) lack of significant improvement over SOTA.
Compared with other submissions I'm handling, I have to reject this manuscript.
Decision: Reject