Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary
=======
The paper leverages two methods for improving generalization in standard training, logit smoothing and stochastic weight averaging, and show that these results can mitigate robust overfitting and improve generalization for adversarial training methods. 

Overall, the paper was clear and easy to follow. There are a number of ablation studies showing the marginal effects of the two methods, as well as experiments demonstrating how the approaches vary with certain choices in methodology. My initial impression is positive, though there are certain changes described below which would help solidify the paper and its claims. 


Comments for discussion
=======================
By improving upon the results in Rice et al. 2020, the authors purport to have state of the art results. However, there's be a plethora of new work since then which have improved these numbers even further. Fortunately, since the submitted work handles the standard CIFAR10 setting, there are a number of public benchmarks that can be used here. It would be great if the authors could train a comparable model and evaluate it using one of these benchmarks (e.g. the autoattack framework at https://github.com/fra31/auto-attack). 

To be clear, since a number of these approaches on the benchmark are quite recent, I am not requesting that the authors directly compare to these new methods in their work. However, the baseline that they do compare to (e.g. Rice et al. 2020) is evaluated in this framework (and had a not-insignifcant drop in robust accuracy), so it would be of significant utility to also evaluate the approach using the improved attack. Performing this evaluation would serve two purposes: 

1. This should alleviate most concerns on the validity of the result 
2. This makes the work easily comparable for future work 

Note that reaching the top of the benchmark is not a requirement for publication. As long as it is consistent with the claims of the paper, that the approach reduces robust overfitting for PGD training and improves upon the PGD baseline within this benchmark, then this is fine. If the authors can report how their approach performs under this improved evaluation or a comparable alternative, then I am happy to adjust my score accordingly. However, the authors probably shouldn't claim state-of-the-art performance without doing this evaluation first. 


Minor comments
==============
+ In section 3.2, it is mentioned that Table 2 supposedly shows differences when a robust self-teacher is pretrained, but this does not seem to be the case. 


Update
======
I have looked through the response and edited version. The updated evaluation looks solid and provides a potential solution to a robust overfitting problem. Although the work is primarily empirical in nature, it may inspire directions for future work to look into more theoretical explanations of robust overfitting. 
review_title: Studies smoothing approaches from standard training to reduce robust overfitting

final_score: rating: 7: Good paper, accept
review_content: #########################################################################

Summary:
This paper uses the existing tricks that can enhance the standard training, to show that combining some of those tricks (in this paper, labels/logits smoothing and weight averaging) can improve adversarial training. 

#########################################################################

Pros: 
1 Compared with existing weight manipulation AT methods, this paper first utilizes stochastic weight averaging (SWA) (averaging multiple checkpoints along the training trajectory) without incurring computational overhead. 

2 This paper conducted experiments across four different datasets.

#########################################################################

Cons: 
1 The paper’s novelty is marginal. Specifically, first, label/logit smoothing has been demonstrated effective in adversarial training due to the better separation of different classes. For example, to my knowledge, three papers got accepted with the shared philosophy but slightly different techniques/decorations [1, 2, 3] 
Second, as the authors mentioned, manipulating model weights is also shown effective [4].
Therefore, this paper's conceptual improvements are marginal.  

[1] Metric Learning for Adversarial Robustness, NeurIPS 2019\
[2] Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness, ICLR 2020\
[3] Boosting Adversarial Training with Hypersphere Embedding, NeurIPS 2020\
[4] Revisiting loss landscape for adversarial robustness, NeurIPS 2020

2 This paper hypothesizes “one source of robust overfitting might lie in that the model ‘overfits’ the attacks generated in the early stage of AT and fails to generalize or adapt to the attacks in the late stage.”  It is not clear to me why this hypothesis is valid. Would you explain more justifications?

3. In Figure 3, are there any experimental results for SWA SA and SWA RA? 
Besides, more robustness evaluations are needed, e.g., CW attack, AA attack, Guided Adversarial Margin Attack. 
More adversarial training on different network structures are needed, e.g., Wide ResNet. 

review_title: This paper uses logit smoothing and weight averaging to enhance adversarial training. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper studies a method for mitigating robust overfitting. Rice et al., and others have observed that when training a neural network robustly on say CIFAR10, then the robust test error often overfits, i.e., it has a U-shaped curve as a function of training epochs. Rice et al. demonstrated that early stopping the robust training enables state-of-the-art robust performance. However, to realize this performance, it is necessary to find a good early stopping point, which can be difficult (but can be found with testing on a validation set). The paper proposes an alternative to early stopping: smoothing the logits and smoothing the weights, by using  two existing techniques, namely self-training and stochastic weight averaging. The paper finds that smoothing mitigates robust overfitting, and reports even a slight improvement over early stopping at the optimal point.

Strength:
- The paper proposes two existing techniques to prevent robust overfitting:  self-training and stochastic weight averaging, and shows that combining those two approaches is very effective in preventing robust overfitting for ResNet-18.
- The paper is well written and easy to follow.

Weaknesses:
- All experiments are carried out with one network only: ResNet-10. To validate the claim that learned label smoothing can mitigate robust training, is important to test the label smoothing method on a variety of setups and models.
- The improvement are tiny relativ to early stopping, and another submission to this workshop (https://openreview.net/pdf?id=Xb8xvrtB8Ce) has shown that the baseline the paper under review is comparing to (the setting of Rice et al.) is quite brittle relative to choices of hyperparameters such as slight differences in weight decay. Therefore, the gains obtained by the paper under review could very well be subsumed by slightly tuning early stopping.

Summary: It is an important problem to study methods that mitigate robust overfitting, and the paper proposes a combination of two smoothing techniques and demonstrates its effectiveness through extensive experiments. I'm therefore leaning to recommend acceptance of this paper, however, as mentioned, the paper's results might not generalize to other models and the slight gains over early stopping might be void by slightly tuning ES better. Therefore, I would not be upset if this paper were rejected. In any case, the paper's results would be more convincing if it would contain results for different models (e.g., VGG and other deep nets but also simple baseline models such as random feature models), and if it would contain a simple theoretical statement to provide intuition why label smoothing should help. 

---- 
UPDATE: Thanks; I have read the response, kept my score, and responded below.
review_title: An interesting potential approach to prevent robust overfitting

Meta-review: This paper focuses on the problem of robust overfitting. The philosophy behind sounds quite interesting to me, namely, injecting 
more learned smoothening during adversarial training. This philosophy leads to two simple yet effective methods: one leveraging knowledge distillation and self-training to smooth the logits, and the other performing stochastic weight averaging to smooth the weights.

The clarity and novelty are above the bar of ICLR. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please carefully address all 
comments in the final version.
Decision: Accept (Poster)