Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work studies the intriguing layerwise properties of robustness overfitting and reveals that adversarial overfitting typically happens in the deep layers of networks. Based on the observations, two regularizations are utilized for the latter layers to tackle the overfitting. Extensive experiments with different datasets, attack methods, and model architectures verify the proposed method.

strength_and_weaknesses: Pros:

1. This paper is well-written and easy to follow.

2. An in-depth analysis is performed to support the hypothesis.

3. Significant performance improvement in many settings.

Cons:

1. It would be better if the authors could offer some comparison with SOTA methods.

Question:

1. In Section 3.1, for AT-fix-param-[XXX], does fixing mean the parameters are fixed as random initialization? And do you still update the fc layer?
clarity,_quality,_novelty_and_reproducibility: Clarity: Good. 

Quality: Good.

Novelty: Good. 

Reproducibility: Fair. (Releasing code can further improve reproducibility)
summary_of_the_review: The analysis of this paper is rigorous and well-supported. I would lean to accept. If the authors can offer more comparisons with SOTA methods on addressing adversarial overfitting, I would like to increase my score further. Releasing code can also be a bonus.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In this work, the authors study the phenomenon of robust overfitting during adversarial training (AT) of deep neural networks — specifically about the deviations that arise due to optimization at different layers of these networks. The paper demonstrates that if the deeper layers of the network are frozen or optimized with a lower learning rate during AT, robust overfitting is significantly reduced, albeit the accompaniment of reduction in test robustness as well, in contrast to that seen if the same is performed with the earlier layers. A similar phenomenon is seen with the Adversarial Weight Perturbation (AWP) regularizer applied to different layers, where it is seen that latter layers contribute more to robust overfitting. 
strength_and_weaknesses: Strengths:
1) The paper is well-written, and presents a sequence of empirical evaluations in a clear, concise manner to help understand the effect of optimization of different layers on robust overfitting. 
2) The proposed methods RAT-LR and RAT-AWP are fairly simple: RAT-LR applies a fixed learning rate to the deeper layers, while RAT-AWP in a similar vein applies AWP solely to the deeper layers, to impose additional regularization for the latter part of the network alone.


Weaknesses:
1) The contributions and novelty of the techniques presented are fairly limited, given that the two primary methods proposed are to either keep learning rate fixed or apply AWP for latter layers, instead of applying the same to the entire network as a whole. Furthermore, the deeper layers/blocks are known to have many more parameters (especially for ResNet based models), thus it is not entirely surprising that the latter layers contribute more to overfitting. This aspect could have been analyzed in much more detail in the paper, to study of depth vs parameter count on robust overfitting. 
2) The analysis presented is also fairly focussed to the very specific case of step-decay of learning rate as well, when in practice several recent works on robust defenses utilize other learning rate schedules such as cyclic, cosine, annealed cosine etc. Furthermore, the effect of the magnitude of the learning rate drop, the spacing of drop points, effect on deeper layers, only FC layer etc. could have been studied. Since the paper does not present theoretical viewpoints of the problem (which is certainly fine on its own), the extent of empirical study could have been significantly broadened to better understand robust overfitting in slightly more generalized settings.   
3) This is further reinforced given the fact that AWP based regularization on its own is seen to drastically reduce robust overfitting, and appears to be the best performing method as seen from Figure 3 (a) and 3(c). Thus the contributions of the paper become slightly unclear in this setting, over and above AWP based training alone.
4) Given the excellent performance of AWP-AT and its effect in mitigating robust overfitting, the baseline AWP method needs to be included in the main empirical evaluations as presented in Tables-1,2, since it is well known that far out-performs the standard AT baseline presented. From the original AWP paper, WideResNet-34-10 models trained on CIFAR-10 with AWP-AT achieves 54.04% and AWP-TRADES achieves 56.17% AutoAttack accuracy, while the proposed RAT-AWP-AT achieves 54.46%. Thus, the improvements in robust performance, or noteworthy contributions over the AWP method  are not clearly seen.
5) Further, the clean accuracy of models need to be reported alongside the robust accuracies in the same tables, due to the well-known robustness-accuracy tradeoff. Thus, it is difficult to judge if a given method with slightly higher robust accuracy is inherently better, if it is accompanied by a large decrease in clean accuracy.

clarity,_quality,_novelty_and_reproducibility: The paper is presented overall in a clear, concise manner. The contributions and novelty are however fairly limited, given that the primary methods proposed are limited to training with a fixed learning rate or the application of AWP for latter layers alone.

summary_of_the_review: As mentioned in the weaknesses section, the contributions of the paper are fairly limited, and the proposed method thus does not present significant benefits over the prior work AWP, which this paper seeks to incorporate. While the overall idea is well presented, the empirical evaluations could be improved to better understand the phenomenon of robust overfitting.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper investigates the impact of different DNN layers impact the performance of adversarial training (AT). The ablation experiments reveal that the latter (deeper) layers are more influential to the robust generalization gap and the final performance. Two techniques were then proposed to improve the performance of standard AT method via adaptive learning rate or adaptive weight perturbation.
strength_and_weaknesses: Strengths:
1. The student of an interesting phenomenon, robust overfitting, in adversarial training.
2. The key findings are interesting and appear quite new to me.
3. The two proposed defenses are validated on multiple datasets.

Weaknesses:
1. Missing important analysis and discussion to existing work [1], where a detailed grid search was applied to WideResNet to find that the deeper layers are also more impactful to the final performance of AT. The conclusion is the same as in this paper. And they find that this is because the deep layers are overparameterized and can be simply reduced to mitigate the robust overfitting. This work did a similar analysis as in [1] but using different techniques, i.e., training or not training, hyperparameter ablation, adaptive lr etc. An in-depth analysis should be conducted to connect this work to [1].

2. Incomplete robustness evaluation with different AT methods. In table 1/2/3, the standard AT is compared with the proposed method. It is thus not clear to me whether other AT methods like TRADES, MART, and AWP are also suffering from robust overfitting, and how the proposed method performs with these AT methods. How the proposed techniques work with data augmentation strategies in [2]. 

3. Why latter layers are so special? Any theoretical insights or an in-depth analysis? How can we solve this issue completely? Is it guaranteed to improve the robust accuracy if the latter layers are treated differently, can it improve the current SOTA robustness shown in https://robustbench.github.io/


[1] Huang, Hanxun, et al. "Exploring architectural ingredients of adversarially robust deep neural networks." Advances in Neural Information Processing Systems 34 (2021): 5545-5559.
[2] Rebuffi, Sylvestre-Alvise, et al. "Fixing data augmentation to improve adversarial robustness." arXiv preprint arXiv:2103.01946 (2021).
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to read. The proposed methods are somewhat novel. The proposed methods can be easily reproduced.
summary_of_the_review: An interesting finding for adversarial training, and is confirmed from different perspectives by different techniques. However, the findings need deeper analysis and the robustness of the proposed mitigation methods needs a complete comparison.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper investigates robust overfitting, an important phenomenon that occurs in the adversarial training. In contrast to previous approach, the paper does not regard the neural network as a black-box model but divide a DNN into a series of layers and investigate the effect of different network layers on robust overfitting. It finds that the latter layer has larger impact on the robust overfitting. To this end, the paper proposes two regularization methods on the latter layers of the networks, namely $RAT_{LR}$ and $RAT_{AWP}$, and shows that two these methods help to mitigate robust overfitting.
strength_and_weaknesses: Strength

1. The paper studies an important problem of robust overfitting in the adversarial training. It is highly motivated to divide the DNN into several layers instead of regarding it as a black-box model.

2. The experiments are intensive, showing that the proposed methods, RAT, help to mitigate robust overfitting.

Weakness

1. My major concern is about the number of parameters that the paper fix in the experiments of Section 3. Different blocks have different number of parameters. Therefore, fixing different blocks will result in the models with different number of effective parameters so that they will have different capacity. If the model that fixs the latter part has smaller number of parameters than the model that fixs the former part, it cannot prove the latter part has larger impact on the robust overfitting. Instead, the model capacity would be more important. In addition, adding more analyses beyond the architecture of ResNet would help to strengthen the conclusion.

2. As mentioned by the paper, AWP helps to mitigate robust overfitting. What is the performance of applying AWP to all layers? If this one has better performance than $RAT_{AWP}$, then the proposed method seems to have small novelty.

3. In the experiments of Section 4, as AWP is shown to improve the performance. $RAT_{AWP}$ should not only be compared with AT with AWP as well.

4. In the experiment of Section 4, the final accuracy of $RAT_{LR}$ is lower than AT in many cases. Therefore, the empircal application of this method seems very limited.

Minors

How are the last fully connected layers treated in the experiments? Are they fixed?
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to read. Although the code is not provided, I think the algorithm is relatively simple and the experiments are relative standard. Therefore, I think the results may be easy to reproduce.
summary_of_the_review: Although the findings and directions of paper is novel, I think the main claim of the paper is not well supported.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work studies the intriguing layerwise properties of robustness overfitting and reveals that adversarial overfitting typically happens in the deep layers of networks. The authors proposed two regularizations and reported extensive experiments with different datasets, attack methods, and model architectures.

Reviewers in general appreciate the clear writing, extensive experiments and often remarkable performance gains. They are however less excited about the limited innovation, and the ad-hoc selection (e.g.,  to either keep the learning rate fixed or apply AWP for latter layers, without a formalized study why latter layers are "special").

Due to the limited rebuttal presented by the authors, the reviewers did not find the ground to recommend this paper positively. However, given this paper's excellent empirical performance, AC would like to encourage the authors to dig deeper into "why latter layers" beyond hand-waving, which should grant this paper with a neat chance in the next venue opportunity. Another possible way is to think from the angle of "efficient adversarial training" and to consider layer selection as the trade-off knob.
justification_for_why_not_higher_score: See above. No sufficient rebuttal was provided to address reviewer concerns
Decision: Reject