Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper has presented a new method to improve the robustness of features under adversarial attacks. Authors developed a new metric for the change of features subject to attacks and key findings are eigenvectors of small eigenvalues are more inclined to change under adversarial attacks, i.e., non-robust. Authors believe the dominance of large eigenvalues and eigenvectors are the primary reasons and a way of flattening the spectrum of features would help mitigate this issue. Authors propose to suppress the largest eigenvalues during training, i.e., spectral regularization, which show positive results in adversarial defense when working together with SOTA defense models.
main_review: The major contribution of this paper is the demonstrations of the connections between adversary and feature spectrum, and the development of a new regularizer applicable in adversarial training. The theoretical analysis on the other hand also supports the proposal of flattening the spectrum.

The paper is well written with adequate supports from many quantitative and qualitative results, especially those for the feature spectrums.
One of the major issues with this paper is the marginal improvement over the SOTA adversarial defense methods, which can be seen in Table 1-5. 

As the SOTA adversarial defense algorithms also have the potential to flatten the spectrum of features (and many other benefits), it is unclear if the FSR will always benefit the defense, as the experiment results showed.

FSR only suppresses the largest eigenvectors, and it seems there is a gap between this operation and goal of adjusting feature spectrums. Why not suppress more eigenvalues? Some explanations would help here.

Another factor worth further discussion is the basic CNN model. It is unclear whether a strong CNN model will react differently to attack and proposed defense methods, and this could be considered in the experiments.

The dataset considered only includes low-resolution images, e.g., 32x32. It would be interesting to see if the defense is effective on high-dimensional data, i.e., 224x224 where the adversarial attack is more significant.

summary_of_the_review: In brief, the paper reveals the connections between feature spectrum and non-robust features under adversarial attack and proposes an interesting regularization term for robust training. However, the new method does offer significant improvement over SOTA and was not comprehensively evaluated on high-dimensional data or using different backbone networks.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: By analyzing the spectral difference between the natural and adversarial examples, this paper finds that eigenvectors with smaller eigenvalues are more non-robust and adversary trends to add more components into these directions. To eliminate the dominance of the top eigenvalues, the paper proposes Feature Spectral Regularization (FSR), which adds more penalties to the largest eigenvalues while relatively increasing the smaller ones. Several experimental results demonstrate that FSR can further improve the robustness when combined with other adversarial defenses.  
main_review: Strengths:
1) The overall presentation of the paper is clear and easy to follow;
2) In addition to the empirical experiments, the paper also provides theoretical analysis;

Weaknesses:
1) The curve trend of standard training (ST) vs. AT in Figure 1 and Figure 2 is not quite consistent?  In Figure 1, the ST curve is almost under the AT curve; while in Figure 2, the AT curve is almost under the ST curve.

2) The paper claims that FSR can further improve the robustness of adversarial defenses, however, if we look at the results, the improvements are very marginal in most cases. So I am wondering: (1) whether this marginal improvement is worth well given we need extra computation on the SVD; (2) if we repeat the experiments many times (say 10 rounds), will these results still statistically hold?

3) How will this method work for a large-scale dataset like ImageNet? Will it cause significant computational overhead due to the SVD? And will it be efficient enough for practical settings?

4) in Figures 6, 7 in the Appendix, the trend for some of the cases seems not as obvious as Figure 4. For example, Figure 6 (e), Figure 7 (d)(f), are all close to 1. I am therefore wondering about the stability of this finding.

5) It is not clear that how it is related to the PCA; need more explanations on this part.


Minor issue: under Figure 3, there is a sentence “We could compare r(D, D, u_j)…” should be “r(D_{adv}, D, u_j)”.







#-------------------------------------------------------------------------------------------------------------------------------------------#

#---------------------------------------------------------------- After rebuttal --------------------------------------------------------#

#-------------------------------------------------------------------------------------------------------------------------------------------#

Thanks so much for addressing my previous concerns. 

Based on the feedback and the available reviews and discussions, I will keep my current score because 1) this method might not be practical to handle large-scale dataset; 2) According to some of the updated experimental results when there are no adversarial attacks in the dataset (and in some types of adversarial attacks), the proposed can actually hurt the performance (compared with the AT). However, in a practical setting, one never knows whether there are adversarial attacks or not in the dataset. 

summary_of_the_review: The overall presentation of this paper is clear and the authors have conducted experiments on different datasets, adversarial attacks, and different adversarial defenses. However, I still have several concerns as I list above, so I will give it a “6: marginally above the acceptance threshold”.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a new defense against adversarial attacks by means of a spectral regularization. The defense is based on an inspection of the embedding of training/test data as returned by the penultimate layer of a neural network. The observation is that the relevant subspace of embedded natural data points (covering e.g. 90% of the variance) is quite low-dimensional, where the use of adversarial training increases the dimensionality of this subspace. In addition, the relevant subspace of embedded adversarial examples is in the provided experiments typically a bit higher dimensional than the subspace of natural examples. Based on these observations, the authors propose a regularizing term, which penalizes the variance in the direction of the first principal component (the largest eigenvalue of the feature covariance matrix).

Experiments are conducted on Cifar-10, -100 and SVHN. The proposed regularizer is applied on top of other defense methods, which typically increases the accuracy under attacks by up to 2%.
main_review: The analysis of connections between the sensitivity to attacks and the subspace of the computed embedding is very interesting. I believe that the authors might have struck here a very interesting research direction, which could fuel further research and lead to new insights.

Unfortunately, the present paper does not go beyond the incidental observations. There is no theoretical guarantee or connection which indicates how the dimensionality of the embedding influences the robustness to attacks or how low-variance subspaces can be exploited by attack methods. The authors do provide a theoretical analysis for robust linear regression models, but the connection/applicability to DNNs is unclear and I also come to the opposite conclusion from the authors, regarding the eigenvalues of the most robust regression model (see below).   

The experiments show that the proposed regularizer can improve the robustness by a few percent, but we do not observe that a more homogeneous distribution of eigenvalues really prevents adversarial attacks from happening. In this respect, I doubt that the proposed method would be used in practice. Employing the regularizer requires tuning its weight, and even when the authors could show that for Cifar-10, -100 and SVHN a weight around 0.01 is suitable, this might look very different for other datasets. Hence, a theoretical guarantee/connection from the embedding PCA to robustness would be needed in order to estimate in which cases the proposed approach can help and to what extend.

### Theorem 1
Theorem 1 states that the design matrix $Z$ of a linear regression model yields the most robust predictions if all eigenvalues of $Z$ are equal. The robustness is measured by $\lVert Z\theta_a -Z\theta_0\rVert^2$, where $\theta_a = (\Sigma+\lambda I )^{-1}\Sigma\theta_0$. Using the SVD of $Z=UDV^\top$, we can rewrite
$$
\begin{align*}
\lVert Z\theta_a -Z\theta_0\rVert^2 &= \lVert \mathrm{diag}\left(\frac{n\lambda\sigma_i}{\sigma_i^2+n\lambda}\right)V^\top \theta_0\rVert^2\\
&= \sum_i \left( \frac{n\lambda\sigma_i}{\sigma_i^2+n\lambda} V_{\cdot i}^\top \theta_0\right)^2\\
&= \sum_i  \frac{n^2\lambda^2\sigma_i^2}{(\sigma_i^2+n\lambda)^2} (V_{\cdot i}^\top \theta_0)^2
\end{align*}
$$
The last term attains its minimizer $Z$ s.t. the constraint $\lVert Z\rVert=s_0$ if $\sigma_1=s_0$, $V_{\cdot 1}$ orthogonal to $\theta_0$ and the rest of the singular values being zero. In your proof, you fix $y=z^\top \theta_0$, but since you optimize over $Z$ and since $y$ is a function of $z$ I don't see how you can treat $y$ like a constant.

### Minor issues:
* Use maybe grammarly or another tool to capture the most obvious language errors
* The space in which PCA is applied and whose eigenvalues are inspected in this work remains largely unclear until Section 3.
* Is there any connection of the PCA indicators to known robustness influencers, like the Lipschitz constant of the DNN function for example?
* Can we infer something about the manifold hypothesis (natural examples are on a manifold in the embedded space and adversarial examples are outside of that manifold) from the PCA of the embedding? How can your observation be placed in the scope of the related work on adversarial examples on manifolds?
* How can we transfer the information about robust linear regression models to DNNs?
* Why is FSR not evaluated without an additional defense method?

  

 
summary_of_the_review: The authors make an interesting observation about adversarial robustness of DNN classifiers based on a PCA of the embedded data points. However, the observations are not backed up by theoretical insights and the empirical evidence is not very conclusive.
review_title: null

Meta-review: Based on the observation that the eigenvectors with smaller eigenvalues are more non-robust (i.e., adversary adds more components along such directions), the authors propose a method called Feature Spectral Regularization (FSR) to penalize the largest eigenvalue, and as a result, the other smaller eigenvalues get increased relatively.
In this paper, in addition to FSR, theoretical analysis along with experimental results on different datasets and models were presented.
Although the proposed FSR has some merits, the major concerns from the reviewers include (1) impractical use on large-scale datasets and (2) lack of significant improvement over SOTA.
Compared with other submissions I'm handling, I have to reject this manuscript.
Decision: Reject