Robust Overfitting may be mitigated by properly learned smoothening
Authors: Tianlong Chen
Zhenyu Zhang
Sijia Liu
Shiyu Chang
Zhangyang Wang
Venue: ICLR 2021 Poster (2021)
Keywords: Robust Overfitting
Adversarial Training
Adversarial Robustness

Abstract: A recent study (Rice et al.,  2020) revealed overfitting to be a dominant phenomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements. This intriguing problem of robust overfitting motivates us to seek more remedies. As a pilot study, this paper investigates two empirical means to inject more learned smoothening during AT: one leveraging knowledge distillation and self-training to smooth the logits, the other performing stochastic weight averaging (Izmailov et al., 2018) to smooth the weights. Despite the embarrassing simplicity, the two approaches are surprisingly effective and hassle-free in mitigating robust overfitting. Experiments demonstrate that by plugging in them to AT, we can simultaneously boost the standard accuracy by $3.72\%\sim6.68\%$ and robust accuracy by $0.22\%\sim2 .03\%$, across multiple datasets (STL-10, SVHN, CIFAR-10, CIFAR-100, and Tiny ImageNet), perturbation types ($\ell_{\infty}$ and $\ell_2$), and robustified methods (PGD, TRADES, and FSGM), establishing the new state-of-the-art bar in AT. We present systematic visualizations and analyses to dive into their possible working mechanisms. We also carefully exclude the possibility of gradient masking by evaluating our models' robustness against transfer attacks. Codes are available at https://github.com/VITA-Group/Alleviate-Robust-Overfitting.

Published as a conference paper at ICLR 2021
ROBUST OVERFITTING MAY BE MITIGATED BY PROP- ERLY LEARNED SMOOTHENING
Tianlong Chen', Zhenyu Zhang", Sijia Liu Shiyu Chang', Zhangyang Wang' Uniieessity of Texas at Austin, UUniversity of Science and Technology of China 'Michigan State University, "MIT-IBM Watson AI Lab, IBM Research [tianleng. chen, atlaswang) @utexas..................... edu, zzyl 99699969999 996.9m91611mmmmm il uste edu liusi j150msu....mmm edu, chang com com
ABSTRACT
A recent study (Rice et al., 2020) revealed overfitting to be dominant phe- nomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements. This intriguing problem of robust overfitting motivates to seek more remedies. As pilot study, this paper in- vestigates two empirical means to inject more learned smoothening during AT: one leveraging knowledge distillation and self-training to smooth the logits, the other performing stochastic weight averaging (Izmailov et al. 2018) to smooth the weights. Despite the embarrassing simplicity, the two approaches are sur- prisingly effective and hassle-free in mitigating robust overfitting. Experiments demonstrate that by plugging them to AT, we can simultaneously boost the standard accuracy by 3.72% 6.68% and robust accuracy by 0.22% 2.03%, across multiple datasets (STL-10, SVHN, CIFAR- 10, CIFÁR-100, and Tiny Ima- geNet), perturbation types (Loo and &2), and robustified methods (PGD, TRADES, and FSGM), establishing the new state-of-the-ard bar AT. We present system- atic visualizations and analyses to dive into their possible working mechanisms. We also carefully exclude the possibility of gradient masking by evaluating our models' robustness against transfer attacks. Codes are available httpp: //github. wtt.:uuub..ooo//IA-rr---a.
INTRODUCTION
Baseline
Our Methods
Standard Test Error
Adversarial training (AT) (Madry al. 2018), i.c.,
0.6
training a deep network to minimize the worst-case training loss under input perturbations, is recognized as the current best defense method to adversarial at-
0.4
tacks. However, one of its pitfalls was exposed by
a recent work (Rice et al. 2020): in contrast to the
50
150
200
Robust Test Error
commonly-hede belief that overparameterized deep networks hardly overfit in standard training (Zhang
0.7
et 2016 Neyshabur et al., 2017 Belkin et al.)
0.6
2019 overfitting turns out to be dominant phe-
0.5
nomenon in adversarially robust training of deep net- works. After certain point in AT, e.g., immediately after the first learning rate decay, the robust test er- rors will only continue substantially increase with further training (see Figure [ bottom for example).
0
50
100
150
200
Epochs
Figure 1: The standard (top) and robust (bottom) test eтrоr comparison, between the vanilla PGD-AT baseline (Madry al. 2018), and PGD- AT with our proposed weight/label smoothening techniques applied, on CIFAR-I0 with ResNet- 18. Our methods effectively mitigates the robust overfitting (Rice et al.) (020) even when trained to 200 epochs, while maintaining the same high standard/robuus accuracies compared to the best early-stop checkpoint of the baseline.
That surprising phenomenon, termed "robust overfilting". " has been prevalent on many datasets and models. As Rice et al. (2020) pointed out, it poses serious challenges to assess recent algorithmic
"Equal Contribution.

---

Published as a conference paper at ICLR 2021
advances upon AT: by just using an earlier checkpoint, the performance of AT be drastically boosted to match the more recently reported saee--of-hhe-rrree (Yang et al. 2019b Zhang et al., 2019b). Even worse, Rice et al. (020) tested several other implicit and explicit regularization methods, including weight decay, data augmentation and semi-supervised learning; they reported that none of those al- ternatives seem combat robust overfitting (stably) better than simple early stopping. The authors thus advocated using the validation set to select a stopping point, although the manual picking would inevitably trade off between selecting either the peak point of robust test accuracy or that of standard accuracy, which often do not coincide (Chen et al. 2020a).
Does there exist more principled, hands-off. and hassle-free mitigation for this robust overfitting, for us to further unleash the competency of AT? This paper explores two options along the way, that draw two more sophisticated ideas from enhancing standard deep models' generalization. Both could be viewed as certain types of learned smoothening, and are directly plugged into AT:
Our first approach to smooth the logits in AT via self-training, using knowledge distilla- tion with the same model ire-trained as a self-teacher. The idea is inspired by two facts: (1) label smoothening (Szegedy et al. 2016) can calibrate the notorious overconfidence of deep networks (Hein et al. 2019), and that was found to improve their standard generaliza- tion; (2) label smoothening can be viewed as a special case of knowledge distillation (Yuan) ct al. 2020), and self-training can produce more semantic-aware and discriminative soft
label "self-teachers" than naive label smoothening (Chen et al. 2020b; Tang et al. 2020).
Our second approach is to smooth the weights in AT via stochastic weight averaging (SWA) (Tzmailov et al.] 2018), a popular training technique that leads to better standard generaliza- tion than SGD, with almost computational overhead. While SWA has not yet be applied to AT, it is known to find flatter minima which are widely believed to indicate stronger ro- bustness (Hein & Andriushchenko 2017 Wu et al.] [2020a). Mcanwhile, SWA could also be interpreted as a temporal model ensemble, and therefore might bring the extra robustness of ensemble defense (Tramer et al. 2018 Grefenstette al. (018)) with the convenience of a single model. Those suggest that applying SWA is natural and promising for AT.
To be clear, neither have been utilized in standard training to alleviate (standard) overfitting and improve generalization, by fixing over-confidence and by finding flatter solutions, respectively. By introducing and adapting them to AT, our aim is to complement the existing study, demonstrating that while simpler regular- izations were unable fix robustness overfitting as Rice et al. (2020) found, our learned logit/weight smoothening could effectively regularize and mitigate it, without needing early stopping.
nor SWA was invented by this paper: they
Experiments demonstrate that by plugging in the two techniques to AT, we can simultaneously boost the standard accuracy by 3. 72% ~ 6. 6.68% and robust accuracy by 0.22% ~ 2.03%, across multiple datasets (0T SVHN, CIFAR- 10, CIFAR-100, 100, and Tiny ImageNet), perturbation types (foe and f2), and robustified methods (PGD, TRADES and FSGM), establishing the new state-of-the-art in AT. As shown in Figure I example, our method eliminates the robust overfitting phenomenon in AT, even when training up to 200 epochs. Our results imply that although robustness overfitting is more challenging than standard overfitting, its mitigation is still feasible with properly-chosen, advanced regularizations that were developed for the latter. Overall, our findings join (Rice et al. 2020) in re-establishing the competitiveness of the simplest AT baseline.
1.1
BACKGROUND WORK
Deep networks are easily fooled by imperceivable adversarial samples. To tackle this vulnerability,
numerous defense methods were proposed (Goodfellow et al., 2015] Kurakin et al., 2016 Madry
al.
2018). yet many of them (Liao et al.) 2018] Guo et al. [2018] Xu et al. 2017;] Dziugaite
et al. 2016 Dhillon et al. 2018 Xie et al., 2018: Jiang et al. 2020) were later found to result from
training artifacts, such as obfuscated gradients (Athalye et al. 2018) caused by input transformation
or randomization. Among them, adversarial training (AT) (Madry et al. 2018) remains one of the most competitive options. Recently more improved defenses have been reported (Dong et al. 2018
Yang al. 2019b; Mosbach al. 2018 Hu et al., 2020 Wang al.) 2020a Dong et al. 2020
Zhang et al. 2020a b). with some of them also being variants of AT, e.g. TR.
DES (Zhang et al.
2019b) and AT with metric learning regularizers (Mao et al. 2019 Pang al.
2019 2020).
While overfitting has become less a practical concern in training deep networks nowadays, it was not yet noticed nor addressed in the adversarial defense field until lately. An overfitting phenomenon was
2

---

Published as a conference paper at ICLR 2021
first observed in few fast adversarial training methods (Zhang al. 2019a; Shafahi et al. 2019b
Wong et al., 2020) based on FGSM (Goodfellow et al., 2015). e.g. sometimes the robust accuracy against PGD adversary suddenly drop to nearly zero after some training. (Andriushchenko &
Flammarion 2020) suggested it to be rooted in those methods' local linearization assumptions of the
loss landscape in those "fast" AT. The recently reported robust overfitring (Rice al. 2020) seems to raise a completely new challenge for the classical AT (not fast); the model starts to irreversibly lose robustness after training with AT for period, even the double-descent generalization curves still seemed to hold (Belkin et al. 2019; Nakkiran et al. (019). Among various options tried in Rice al. (2020), early-stopping was so far the only effective remedy found.
METHODOLOGY
2.1 LEARNING TO SMOOTH LOGITS IN AT
AT enforces models robust against adversarial attacks of specific type and cer- tain magnitudes. However, it has been shown to "overfit" the threat model "seen" during training (Kang et al. 2019; Maini al., 2019 Stutz et al. 2020), and its gained robustness does not ex- trapolate to larger perturbations nor unseen attack types. Stutz et al. (2020) hypothesized this to be an unwanted consequence of enforcing high-confidence predictions on adversarial examples since high-confidence predictions are difficult to extrapolate to arbitrary regions beyond the seen examples during training. We generalize this observation: during AT, the attacks generated at every iteration can be naturally considered as continuously varying/evolving. along with the model training. There- fore, we hypothesize one source of robust overfitting might lie in that the model "overfits" the attacks generated in the early stage of AT and fails to generalize or adapt to the attacks in the late stage.
Rationale: Why
To alleviate the overconfidence problem, we adapt the label smoothening (LS) technique in standard training (Szegedy et al. 2016). LS creates uncertainty in the one-hot labels, by computing cross- entropy not with the "hard" targets from the dataset, but with a weighted mixture of these one-hot targets with the uniform distribution. This uncertainty helps to tackle alleviate the overconfidence problem Hein et al. (2019) and improves the standard generalization. The idea of LS was previously investigated in other detense methods (Shafahi et al., 2019a| Goibert & Dohmatob] 2019), but much of the observed robustness gains were later attributed to obfuscated gradients (Athalye et al. 20188). Two recent works (Stutz et al. 2020 Cheng et al. 2020) have integrated LS with AT to inject label uncertainty: [Stutz et al. (2020) used a convex combination of uniform and one-hot distributions as target for the cross-entropy loss in AT, which resembles the LS regularizer, while Cheng et al. (2020)
concurrently used an LS regularizer for AT.
However, there is one pitfall of the naive LS in (Szegedy et al., 2016): over-smoothening labels in a data-blind way could cause loss of information in the logits, and hence weakened discriminative power of the trained models (Müller et al. (019)). That calls for a careful and adaptive balance be- tween discriminative capability and confidence calibration of the model. In the context of AT, Stutz) et al. (2020) crafted a perturbation-deeendeer
parameter, to explicitly control the transition from one-hot to the uniform distribution when the attack magnitude grows from small to large. To iden- tify more automated and principled means, we notice another recent work (Yuan et al. 2020). who explicitly connected knowledge distillation (KD) (Hinton al. 2015) to LS. The authors pointed out that I.S equals a special case of KD using a virtual and hand-crafted teacher; on the contrary, the conventional KD provides data-driven soften labels rather than simply mixing one-shot and uniform vectors. Together with many others (Furlanello et al. [0018] 8: Chen et al., 2020b), these works demon- strated that using model-based and learned soft labels supplies much superior confidence calibration and logit geometry compared to the naive LS (Tang et al. 2020).
Furthermore, (Furlanello al. 2018 Chen et al.) 2020b Yuan et al.) (220) unanimously revealed that another strong teacher model with extra privileged information
Label Smoothing
Tt-D reg
Standrad Teacher
Probability lit
Robust Teacher
is NOT critical to the success of KD. Yuan et al. (2020) shows that even a poorly-trained teacher with much lower accuracy can still improve the student. Moreover, Chen jet al. ( Yuan et al. (020)) find self-teacher to be sufficienlly effective for KD that is, using soft-logit out- puts from the student or designed manually as the KD regularization train itself (also called teacher-free KD
Class
Figure 2: Comparing the logit distribution of different LS/KD means CIFAR-10 10 ing ResNet-18 18 (C7 is the correct label).
3

---

while Published as a conference paper at ICLR 2021
(Tf-KD) in (Yuan et al., 2020)). These observations make the main cornerstone for our learned logit smoothening approach next.
Approach: How We follow (Chen et al. 2020b Yuan et al., 2020) to use self-training with the same model, but introduce one specific modification. The one model could be trained with at least two different ways: standard training, or robust training (AT or other cheaper ways; see ablation experiments). That can yield two self-teachers. We assume both to be available; and let be the input, the one-hot ground truth label, â the adversarial perturbation bounded by lp norm ball with radius e, and 0,,00, the weights of the state-ssttadardttrrnnnee self-teachers, respectively. Note the two self-teachers share the identical network architecture and training data with our target model. Our self self-training smoothed loss function expressed below (AΙ and X2 are two hyperparameterss:
min LE (xx)) {(1 \1 22) max (x y)+ (x
0
ADD udv (f 8), f 8)) + X2 KDD ().
where Lxx is robustified cross-entropy loss adopted in the original AT; ICDadv adv and KDDand are the Kullback-Leible divergence loss with the robust-trained and standard-trainee self-teachers, respec- tively. \ 0.5 and 22 0.25 are default in all experiments. More details are in Appendix A2.1
Figure 2 visualizes an example of logit distributions, generated by naive LS (Szegedy et al., 2016), the TF-KD regularizer using manually-designed self-teacher in (Yuan al. 2020). well our standard- and robust-trained teachers, respectively. We observe both standard and robust self- teachers are more discriminative than the other two baseline smoothenings, while the robust self- teacher is relatively more conservative as one shall expect.
2.2 LEARNING TO SMOOTH WEIGHTS IN AT
Rationale: Why Another measure that is often believed to indicate the standard generalization is the flatness: the loss surface the final learned weights for well-generalizing models is relatively "flat". Similarly, Wu et al. (2020a) advocated that a flatter adversarial loss landscape shrinks the robustness generalization gap. This is aligned with (Hein & Andriushchenko 2017) where the authors called it local Lipschitz and proved that the Lipschitz constant can be used to formally measure the robustness of machine learning models. The flatness preference of robust model has been echoed by many empirical defense methods, such as hessian/curvaatur-bbane
regularization (Moosavi- iieosvv-DDezfooli et al., 2019), gradient magnitude penalty (Wang & Zhang 2019), smoothening with random noise (Liu et al. (2018), or entropy regularization (Jagatap et al. (020)). However, all those methods will incur (sometimes heavy) computational or memory overhead: and many can cause standard accuracy drops, c.g., hessian/curvattue-bbee based methods (Gupta al. 2020)).
Stochastic weight averaging (SWA) (Izmailov et al.) 2018) was proposed to enforce the weight smoothness, by simply averaging multiple checkpoints along the training trajectory. SWA is known to find much flatter solutions than SGD, is extremely easy to implement, improves standard gen- eralization, and has almost no computational overhead. SWA has been successfully adopted in semi-supervised learning (Athiwaratkun et al. 2018), Bayesian inference (Maddox et al.) (019). and low-precision training (Yang et al.] (209a). In this paper, we introduce SWA to AT for the first time, in order to smooth the weights and find flatter minima that may improve the adversarially robust generalization. Note that we choose SWA mainly due to its simplicity for groofs-of-concep extensively comparing alternative "flatness" regularizations is beyond our current work's 's scope.
One additional bonus of adopting SWA in AT is the temporal ensemble effect of SWA. It has been widely observed (Tramer et al. 2018 Grefenstette et al.) 20188] Wu et al.) [020bb] Wang et al.) 2021) that training model with the attack transferred from another could reduce "trivial robustness" caused by locally nonlinear loss surfaces, and therefore constructed model ensembles for stronger defense. SWA was interpreted as approximating the fast geometric ensembling (Garipov et al. 2018), by aggregating multiple checkpoint weights at different training time. Applying SWA to AT therefore may lead to stronger and more transferable attacks, and consequently stronger defense due to ensembling, with the convenience of single model.
Approach: How Following (Izmallov et al. 2018), applying SWA to AT is straightforward:
W&WA- = WWWA x n + WT WT = WT-1 + AWT
(2)
4

---

Published as a conference paper at ICLR 2021
GIFAR-1 10
CIFAR-100
STL-10
Tiny ImageNet
80
Acturacy 10
40-
50
64
0
25
25
75
300 125
150 175 202
25
52
75
100 125
153
175
200
22
50
75
300 Epochs
125
150
175
200
20
40
60
82
100
Epochs
Epochs
Epochs
Figure 3: Results of testing accuracy over epochs for ResNet-18 trained on CIFAR-10, CIFAR- 100, S0L,-10, and Tiny ImageNet. Dash / Solid lines show the standard accuracy (SA) / robust accuracy (RA). Blue, Green, Black and Orange curves represent the performance of Baseline, KD, SWA and KD&SWA respectively.
where T indexs the training epoch, n the number of past checkpoints be averaged. WswA the averaged network weight, W the current network weight, and AW the SGD update.
3 EXPERIMENT AND ANALYSIS
Datasets We consider five datasets in our experiments: CIFAR-10, CIFAR- 100 (Krizhevsky & Hinton 2009). SVHN (Netzer et al.) 2011), STL-10 (Coates al. 201 1 and Tiny-ImageNet (Deng et al., 2009). In all experiments, we randomly split the original training set into one training and one validation sets with 9:1 ratio. Due to the limited space, we place the SVHN results in Appendix A1.3 The ablation studies and the visualizations are mainly on CIFAR-10 10 and CIFAR-100.
We consider three representative attacks: FGSM (Goodfellow et al. 2015). PGD (Madry et al., 2018). and TRADES (Zhang et al., 2019b). All of them are applied with (E2,c 128/255) or (C... 8/255) setting in (Madry et al. 2018), to generate adversarial samples. We use DSGM-I/PGD-I0NT 10TTRDES--- for training and PGD-20 for testing as the default setting, following Madry et al. (2018); Chen et al. (2020a). In addition, we use Auto-Attack (Croce & Hein, 2020 and CW Attack (Carlini & Wagner 2017) for a more rigorous evaluation. More details are provided in the Appendix A2.2
Attack Methods
Training and Evaluation Details For all experiments, we by default use ResNet-18 (He et al. 2016), with the exception of VGG-16 (Simonyan & Zisserman 2014) and Wide-ResNet (Zagorukko & Komodakis 2016) adopted in Table For training,
adopt SGD optimizer with momentum 0.9 and weight decay of 5 x 10 14-4 for a total of 200 epochs, with a batch size of 128. The learning rate starts from 0.1 (0.01 for SVHN (Rice et al. 2020)), decay to one-tenth epochs 50 and 150 respectively. For Tiny-ImageNet, we train for 100 epochs, and the learning rate decay at epochs 50 and 80 with other settings unchanged. The self-training KD regularization applied throughout the entire training, and SWA is employed after the first learning rate decay (when the robust overfitting usually starts to occur). We evaluate two common metrics that are widely adopted (Zhang et al. 2019b, Chen et al.) 2020a): Standard Testing Accuracy (SA), and Robust Testing Accuracy (RA), which are the classification accuracies on the original and the attacked test sets, respectively.
3.1 TACKLING ROBUST OVERFITTING
Superior Performance Across Datasets Table demonstrates our proposal on STL-10, CIFAR- 10, CIFAR-100, and Tiny-ImageNet. We consider PGD-AT (Madry al. 2018 Baseline; and denote our two training techniques as +KCDstdkady (KD with standard and robust self self-teachers), and +SWA, respectively. To numerically show the gap of robust overfitting, we also report the best RA values when early stopping during training, the final RA in the last epoch, and as the difference between final minus best. For reference, we also report the corresponding SA for the same best-RA checkpoint (not the best SA value throughout training). the final epoch SA, and their difference.
We first observe that the robust overfitting prevails in all Baseline cases, with RA differences be- tween final and best early-stopping values as large as 9.34% (CIFAR-10). In comparison, SA stays stable (with negative gaps on STL-10 and CIFAR- 10) or continues to improve along with more train- Fortunately, the gaps were
ing epochs (with small positive gaps on CIFAR- 100 and significantly reduced by +KD std&adv and further diminished to only 0.4% to 0.6% when SWA is also applied.
5

---

Published as a conference paper at ICLR 2021
Table 1: Performance showing the occurrence of robust overfitting across datasets and the effectiveness of proposed remedies with ResNet-18. The difference between best and final robust accuracy indicates degra- dation in performance during training. We pick the checkpoint which has the best robust accuracy the validation set. The best results and the smallest performance differences are marked bold.
Robust Accuracy (RA)
Standard Accuracy (SA)
Datasct
Settings
Best
Final
Diff.
Best
Final
Diff.
Baseline
36.24
32.20
4.04
56.50
63.69
-7.19
STL-10
Bascline + ICD
36.39
34.83
1.56
62.44
64.81
-2.37
Baseline + ICD DDtaladd + SWA
36.46
35,90
0,56
62.76
64.24
-1.48
Baseline
50.72
41.38
9.34
80.78
82.44
-1.66
CIFAR-10
Baseline + 1CD sttdkdaddv
50.89
48.26
2.63
83.67
85.25
-1.58
Baseline + I.D + SWA
52.14
51.53
0.61
84.65
85.40
-0.75
Baseline
27.32
19.84
7,48
53.90
53.56
0.34
CIFAR- 100
Baseline + ICD stdkaay
27.56
26.02
1.54
57.42
60.34
-2.22
Baseline + D.Dataa + SWA
28,28
27.69
0.59
60.58
60.85
-0.27
Baseline
19.81
13.43
6.38
45.85
45.58
0.27
Tiny-ImageNet
Baseline + KDDddaadd
21.45
19.25
2.20
48.98 98
51.15
-2.17
Baseline + 1C + SWA
21.84
21.45
0.39
50.57
51.38
-0.81
Further, we observe our methods to push the best RA higher by 0.22% ~ 2.03%. For example, the best RA on Tiny-ImageNet rises from 19.81% 21.84%. Meanwhile, since there is no longer robust overfitting early in training, the best RA checkpoints become to select late epochs (often close to the end). Consequently, the SA values of the selected best RA models are all substantially improved. For example on CIFAR- 100, the standard accuracy of our methods (best RA checkpoint) surpasses the baseline's 's best RA checkpoint by 6. 6.68%, and by 7.29% for the final checkpoint.
Figure further plots the RA and SA curves during training, from which we can clearly observe the diminishing of robust overfitting, after applying stDatakaddy SWA and a combination of two meth- ods. The training curves robustly improve until the end, without compromising the best achievable RA results, and further leads to a much-improved trade-off between RA and SA by avoiding early stopping (e.g., selecting early checkpoint for RA, when SA might still be half-baked).
Across Perturbations and Robustified Methods Our success can extend beyond PGD-AT. Ta- ble presents more results in different perturbations (i.e. 82, loo) and diverse robustified meth- ods (1.c. FSGM in (Wong et al. 2020). TRADES in (Zhang et al. (019bb). Consistent obser- vations can be made: almost eliminated robust overfitting gaps, and significant gains on RA (by 0.61% ~ 3. 11%) and SA (by 1.80% 4.22%).
We also compare with previous state-of-the-ee results in (Rice et al. 2020) under the same setting. As shown in Table (Appendix), our methods shrink the gap between the RA best checkpoint and the final epoch RA from 5.70% to 0.17% and simultaneously improve 4.50% by RA and 3.04% by SA. More results can be found in Appendix AI
Across Architectures and Improved Attacks Table demonstrates the effectiveness of our meth- ods across different architectures, including VGG-16, Mide-ResNet-34-4 and Mide-ResNNet34--11
Specifically, our methods reduce the drop of robust accuracy from 5.83% to 0.06% with VGG-16 on CIFAR-10 while achieving an extra robust accuracy improvement of 2.57%, 1.69% and 1.23% with VGG-16, verify the improvements achieved by our methods, we conduct extra evaluations under improved attacks. As shown in Table after applying the combination of KD and SWA, the overfitting prob- lem is largely mitigated under both Auto-Attack (Croce & Hein 2020) and CW attack (Carlini & Wagner 2017]. Take CIFAR-10 foo adversary as an example, our approaches shrink the drop of ro- bust accuracy from 7.04% -0.09% under Auto-Attack, and 14.96% to 0. 7.79% under CW attack, when comparing the best model to the eventually converged model. These results indicate that our methods generalize to different architectures and improved attacks.
and Nide-RessNet334110 on CIFAR1O. respectively. To further
Excluding Obfuscated Gradients An often argued "counterfeit" of improved robustness is caused by less effectiveness of generated adversarial examples due to obfuscated gradients (Athalye show that methods maintain improved robustness
et al. 2018). To exclude this possibilityy under unseen transfer attacks. To start with, the left figure in shows the transfer testing per- formance on unseen robust model (here we use separately robustified ResNet-50 with PGD-10

---

Published as a conference paper at ICLR 2021
Table 2: Controlled experiments CIFAR-10. The difference between best and final robust accuracy indicates
degradation in performance during training. We pick the checkpoint which has the best robust accuracy on the validation set. The best results and smallest performance difference are marked bold.
Robust
Accuracy (RA)
Standard Accuracy (SA)
Adversary
Norm
Radius
Settings
Best
Final
Diff.
Best
Final
Diff.
Bascline
67.26
65.93
1.33
88.78
88.86
-0.08
PGD
2
128
255
Our Methods
70.37
70.23
0.14
90.58
90.48
0.10
Baseline
66.48
64.12
2.36
88.77
89.40
-0.63
FGSM
&
255
Our Methods
69.53
69.49
0.04
91.11
91.15
-0.04
Baseline
43.14
37.09
6.05
85.77
86.51
-0.74
FGSM
000
Our Methods
43.75
43.74
0.01
88.53
88.72
-0.19
Baseline
69.07
64.81
4.26
85.43
85.61
-0.18
TRADES
8
128
255
Our Methods
70.30
69.60
0.70
89.65
89.55
0.10
Baseline
51.07
47.32
3.75
79.67
81.81
-2.14
TRADES
200
255
Our Methods
52.92
52.28
0.64
82.95
83.31
-0.36
Table 3: Controlled experiments across different architecture on CIFAR-10/100 under foo adversary. The difference between best and final robust accuracy indicates degradation in performance during training. We pick the checkpoint which the best robust accuracy on validation set. The best results and smallest performance difference are marked in bold.
Robust Accuracy (RA)
Standard Accuracy (SA)
Architecture
Dataset
Settings
Best
Final
Diff.
Best
Final
Diff.
Baseline
46.42
40.59
5.83
75.29
79.54
-4.25
VGG-16
CIFAR-10
Our Methods
48.99
48.93
0.06
79.00
79.69
-0.69
Baseline
21.64
17.43
4.21
39.26
45.84
-6.58
VGG-16
CIFAR-100
Our Methods
24.79
24.73
0.06
48.20
49.00
-0.80
Baseline
52.59
43.06
9.53
81.53
83.28
-1.75
WRN-34-4
CIFAR-10
Our Methods
54.28
53.90
0.38
85.17
85.50
-0.33
Baseline
28.02
20.61
7.41
53.19
53.63
-0.44
WRN-34-4
CIFAR-100
Our Methods
30.10
29.80
0.30
57.23
58.05
-0.82
Baseline
54.27
47.12
7.15
84.
85.72
-1.56
WRN-34-10
CIFAR-10
Our Methods
55.50
55.34
0.16
86.81
87.06
-0.25
Baseline
29.95
24.02
5.93
56.56
56.42
0.14
WRN-34-10
CIFAR-100
Our Methods
31.93
31.51
0.42
60.86
61.78
-0.92
Table Evaluation under improved attacks on CIFAR- 10TAR--0/100 with ResNet-18. The difference between best and final robust accuracy indicates degradation in performance during training. We pick the checkpoint which has the best robust accuracy under PGD-20 attack on the validation set. The best results and the smallest performance difference marked in bold.
Auto-Attack
CW Attack Final
Dataset
Norm
Radius
Settings
Best
Final
Diff.
Best
Diff.
CIFAR-10
l
128 c 255
Baseline
67.18
64.29
2.89
73.80
53.77
20.03
Our Methods
68.87
68.90
-0.03
73.89
73.79
0.10
Baseline
47.00
39.96
7.04
75.48
60.52
14.96
CIFAR-10
t00
253
Our Methods
49.35
49.44
-0.09
77.83
77.04
0.79
CIFAR-100
Eg
128
Baseline
37.16
33.43
3.73
48.43
37.73
10.70
255
Our Methods
40.56
40.61
-0.05
51.02
50.90
0.12
Baseline
22.73
18.11
4.62
45.89
37.76
8.13
CIFAR-100
600
8
255
Our Methods
25.42
25.35
0.07
49.46
49.07
0.39
on CIFAR-100), using attacks generated by the different epoch checkpoints of PGD-AT Baseline, Baseline + stData&ravv and Baseline JCDaid&eady + SWA. A higher robust accuracy on the unseen methods consistently yield stronger
robust model corresponds to weaker attack. Apparently, and more transferable attacks, while the attacking quality generated by the baseline quickly drops with deteriorated transferability. Similarly, the right figure of Figure 4 transfers the attack from
7

---

Published as a conference paper at ICLR 2021
Transfer Attack on Unseen Robust Model
Transfer Attack from Unseen Robust Model
44
Baseline
Robust Accuracy (%)
Robust Accuracy (%)
42
KD
35
KD&SWA
40
30
38
25-
Baseline
36
KD
20
KD&SWA
34
0
50
100
150
200
50
100
150
200
Epochs
Epochs
Figure 4: (Left) Transfer attack performance on an unseen robust model, where attacks are generated by Baseline, KD, and KD&SWA's different epoch checkpoints. (Right) Transfer attack performance on models from Baseline, KD. and KD&SWA, where attacks are generated by an unseen robust model. The unseen robust model is ResNet-50 trained by 10.-10. All experiments are conducted on the CIFAR-100 100 dataset.
Table 5: Ablation studies on CIFAR-I0 with ResNet-18. Compared with Baseline (PGD-AT) methods, the performance improvements and degradations by adding each component are reported in red and blue numbers.
Robust Accuracy (RA)
Standard Accuracy (SA)
Settings
Best
Final
Diff.
Best
Final
Diff.
Baseline
50.72
41.38
9.34
80.78
82.44
-1.66
KCDaud
48.72(1 2.00)
43,08(* 1.70)
5,64
83.40(1
2.62)
84,80(1 2.36)
-1.40
K"Dadv
51.01 0.29)
47.72(1 6.34)
3.29
82.18(1 1.40)
83.51(1 1.07)
-1.33
SWA
51.65(1 0.93)
48.21( 6.83)
3.44
83.42(1 2.64)
84.12(f 1.68)
-0.70
KCD KDuddaadv
50.89(1 0.17)
48.26(1 6.88)
2.63
83.67(1 2.89)
85.25(1 2.81)
-1.58
stakkaddy + SWA
52.14(1 1.42)
51.53(1 10.15)
0.61
84.65(1 3.87)
85.40(1 2.96)
-0.75
an unseen robust model to the above three methods, while our methods consistently defend better. Those empirical pieces of evidence suggest that our RA gains are not a result of gradient masking.
3.2 ABLATION STUDY AND VISUALIZATION
adD ICDatd and SWA We study the effectiveness of each component in logit and weight smoothening. We also specifically decompose stData into two ablation methods: ICData (by setting \2 0 in Eqn. (T), and K.Dadv (by setting \ = 0), respectively. Table5]shows that I.Daad Raa and SWA all substantially contribute to suppressing the robust overfitting and enhancing the SA-RA trade-off. We notice that while ICD seems to (understandably) sacrifice the best RA bit for improving TA, combining it with ICD adv brings the RA compromise back and boosts them both.
Naive LS versus learned logit smoothening As ICD could be viewed as a learned version of LS (Yuan et al., 2020), we next quan- tify the benefit of using IDDeddeaad. compared to naive LS (Szegedy et al.] 2016), and the teacher-free knowl- edge distillation regularization (Tf- KDreg) in (Yuan et al.) 2020), all incorporated with PGD-AT on CIFAR-10. Table show that both naive LS and II-KDrey reg also reduce robust overfitting to some extent, but far less competitive than K.D stdddddd Moreover, the robustness gains of naive LS and Tf-KD longer hold under transfer attacks, implying that they are susceptible to obfuscated gradients. Further visualization in Figure 5 demonstrates that our methods smooth the logits without compromising the class-wise discriminative information, while naive LS and Tf-KD might suffer from weaker gradients here.
Table 6: Ablation of label smoothing versus ACD on CIFAR-10.
Robust Accuracy (RA) Best
Standard Accuracy (SA)
Settings
Final
Diff.
Best
Final
Diff.
Label Smoothing
50.75
44.73
6.02
81.28
83.54
-2.26
TI-KD mg
50.79
45.65
5.14
79.63
83.56
-3,93
KDD Adlkaav-FOOO FGSM
50.81
48.
2.49
84.79
85.59
-0.80
adkeady PGD10
50.89
48.26
2.63
83.67
85.25
-1.58
A.D. Rukaad PGD100
50.92
48.47
2.45
83.31
84.64
-1.33
Quality of Teachres An extra price for our learned logit smoothening the pre-training of self-teachers, although this is already quite common in similar literature (Chen et al., 2020a b). To further reduce this burden, we explore whether high-quality and more expensive pre-training necessary for us, and fortunately find that is not the case. For example, Table shows only marginal performance difference when the robust self-teacher praited using FGSM or PGD-10/100.
8

---

Published as a conference paper at ICLR 2021
Label smoothing
TF-DD Regularization
Methods
Adv
Figure 5: 1-SNE results different logits smoothing approaches on CIFAR-IO. Dots stars represent clean and adversarial images respectively. Orange, Blue and Green represent classes A, and C respectively. For each class, we visualize all testing images and their corresponding adversarial images from PGD-20.
Baseline
Our Methods
-337
18
Figure 6: Comparison of loss landscapes of models trained by baseline (the first row) and our methods (the second row). Loss plots in each column generated from the same original image randomly chosen from the CIFAR-100 test dataset.
denotes the loss value. Following the settings in (Engstrom et al. (208)), we plot the loss landscape function: z loss (ar r1 y r2). where rı sigu V, f()) and ra Rademachur(0.55) Here denostes the original image. and f() a trained model whose inputs are scaled to [0, 1].
Visualizing Flatness and Local Linearity We expect SWA find flatter minima for AT to improve its generalization, and we show to indeed happen by visualizing the loss landscape w.r.t. both input and weight spaces. Figure (6 shows that our methods notably flatten the rugged landscape w.r.t. the input space, compared to the PGD-AT baseline, which aligns with the robust generalization claims in (Moosavi-Dezfooli et al.
Robust Testing Loss
8
Baseline
6
Our Methods
4
2
2019 Wu et al.) 2020a). Figure 7 follows (Izmailov et al.
0.0
0.5
1.0
1.5
2.0
2018 perturb the trained model in the weight space and show how the robust testing loss changes over the perturbation radius. We perturb 10 different random directions at each dif- ferent &a distance. Our methods present better weight smooth- ness around the achieved local minima too, which suggests im- proved generalization (Dinh et al. 2017 Petzka et al., (019).
Distance
Figure 7:
Robust testing loss as function of perturbed weight dis- tance, starting from models trained hy PGD-AT baseline and our methods on CIFAR-100.
We additionally look the local linearity measurement proposed in (Andriushchenko Flammar ion 2020), which originally addresses catastrophic overfitting in fast AT. As shown in Figure All our methods also achieve consistently better local linearity.
CONCLUSION
This paper takes one more step towards addressing the recently discovered robust overfitting is- sue in AT. We present two empirical solutions to smooth the logits and weights respectively; both are motivated by successful practice in improving standard generalization, and we adapt them for AT. While Rice et al. (2020) found simpler regularizations unable to fix robustness overfitting, our learned smoothening regularization seems to largely mitigate that. Extensive experiments show our proposal to establish new state-of-tha performance made, the underlying cause of robust overfitting is not yet fully explained. Our future work will connect more theoretical understandings of this issue (Wang al. 2019 2020b).
AT. While promising progress has been

---

Published as a conference paper at ICLR 2021
REFERENCES
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast ad adversarial training. arXiv preprint arXiv:2007.0266 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: h 1802.004200 2018.
Ben Athiwaratkun, Mare Finzi, Pavel Izmailov, and Andrew Gordon Wilson. There are many consis- tent explanations of unlabeled data: Why you should average. arXiv preprint arXiv:186..00
2018.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine- learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116(32): 15849-144444 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Tianlong Chen, Sijia Liu,, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supervid pre-training to fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 699-708, 2020a.
Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, and Zhangyang Wang. Self-pu: Self boosted and calibrated positive-unlabeles training. arXiv preprint arXiv:2006. / /280, 2020b.
Minhao Cheng, Qi Lei, Pin- Yu Chen, Inderjit Dhillon, and Cho-Jui Hsich. Cat: Customized adver- sarial training for improved robustness. arXiv preprint arXiv:2002.06789, 2020.
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223, 2011.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In International Conference on Machine Learning, pp. 2206- 2216. PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
Icee, 2009.
Guneet S. Dhillon, Kamyar Azizzadenesheli, Jeremy D. Bernstein, Jean Kossaifi, Aran Khanna, Zachary C. Lipton, and Animashree Anandkumar. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations, 2018. URL
https://openrevii. net htt.:/oormmmii-HeRGGSS
Gavin Weiguang Ding, Luyu Wang, and Xiaomeng Jin. AdverTorch v0.1: An adversarial robustness toolbox based on pytorch. arXiv preprint urXiv:190..07622 2019.
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Proceedings of the 34th International Conference Machine Catuning-Volumee 70, pp. 1019-1028. JMLR. org, 2017.
Minjing Dong, Yanxi Li, Yunhe Wang, and Chang Xu. Adversarially robust neural architectures. arXiv preprint arXiv:2009.00902 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun 7hu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression adversarial images. arXiv preprint arXiv:1608 00853, 2016.
10

---

Published as a conference paper at ICLR 2021
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint 0rXiv:1877.10272 2018.
Chaohao Fu, Hongbin Chen, Na Ruan, and Weijia Jia. Label smoothing and adversarial robustness. arXiv preprini arXiv:2009.0825: 2020.
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In International Conference on Machine Learning, pp. 1607-1616, 2018.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. In Advances in Neural Information Processing Systems, pp. 8789-8798, 2018.
Morgane Goibert and Elvis Dohmatoh. Adversarial robustness via adversarial label-smoothing. arXiv preprint arXiv: /906. 1567, 2019.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015. URL http://
Edward Grefenstette, Robert Stanforth, Brendan O' O'Donoghue, Jonathan Uesato, Grzegorz Swirszez, and Pushmeet Kohli. Strength in numbers: Trading-off robustness and computation via adversarially-trained ensembles. arXiv preprint arXiv: 1. 1.09300, 2018.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. In International Conference Learning Representations, 2018. URL htttp:://operrevieee. net/ wwt.://eee..t...eeeeeee.o... fow.foomdddyyy.y.cm..m
Vipul Gupta, Santiago Akle Serrano, and Dennis DeCoste. Stochastic weight averaging in parallel: Large-batch training that generalizes well. arXiv preprint arXiv:2001. 02312, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems, pp. 2266-2276, 2017.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 41-50, 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv: 15Xv::1503.02531 2015.
Ting- Kuei Hu, Tianlong Chen, Haotao Wang, and Zhangyang Wang. Triple wins: Boosting curacy, robustness and efficiency together by enabling input-adaptive inference. arXiv preprint arXiv:2002.10025, 2020.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. arXiv preprint mrXiv:1803.05407, 2018.
Gauri Jagatap, Animesh Basak Chowdhury, Siddharth Garg, and Chinmay Hegde. Adversarially robust learning via entropic regularization. arXiv preprint arXiv:2008. 2tt.:20008./3338,, 2020.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. arXiv preprint arXiv:2010.13332 2020.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv: arXiv:1908. (aae::908..00001 2019.
11

---

Published as a conference paper at ICLR 2021
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. 2016.
arXiv preprint
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-29477 2017.
Liao, M. Liang, Y. Dong. T. Pang. X. Hu, and Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In 2018 IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1778-1777 1787, 2018.
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. Towards robust neural networks via random self-ensemble. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 369-385, 2018.
Wesley Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. In Advances in Neural Information Processing Systems, pp. 13153- 13164, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. In International Conference on Learn- ing Representations, 2018. URL ht httl:tttss///opeeer///e/
net http://foormidddddde wwwoooommmddrrioaaaaae
Pratyush Maini, Eric Wong, and J Zico Kolter. Adversarial robustness against the union of multiple perturbation models. arXiv preprint arXiv:1909. 99,004668 2019.
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. In Advances in Neural Information Processing Systems, pp. 480-491, 2019.
Seyed-Mohsen Moosavi-Dezfooll Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Ro- bustness via curvature regularization, and vice versa. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9078-9086, 2019.
Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, and Dietrich Klakow. Logit pairing methods can fool pradient-based attacks. arXiv preprint arXiv:/810.. /810.1202, 2018.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, pp. 4694-4703, 2019.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. arXiv preprint arXiv:1912.0222 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning 2011, 2011. URL http://uflal. sta..oodd..saa.o..de.....
housennuberrs/rips2001 hww.seuuee.huuueeerrrr.s... pdf
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring general- ization in deep learning. In Advances in neural information processing systems, pp. 5947-5956,
2017.
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross-entropy loss for adversarial robustness. arXiv preprint 1905:1995..22... 2019.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Hang Su, and Jun Zhu. Boosting adversarial training with hypersphere embedding. arXiv preprint arXiv:2002.08679, 2020.
Henning Petzka, Linara Adilova, Michael Kamp, and Cristian Sminchisescu. A reparameterization invariant flatness measure for deep neural networks, arXiv preprint arXiv: /9/2. 00050 2019.
12

---

Published as a conference paper at ICLR 2021
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. arXiv preprint arXiv:2002.11566 2020.
Jérôme Rony, Luiz G Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient gradient-based adversarial attacks and defenses. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4322-4330, 2019.
Ali Shafahi, Amin Ghiasi, Furong Huang, and Tom Goldstein. Label smoothing and logit squeezing: A replacement for adversarial training? arXiv preprint arXiv:1910.11585 2019a.
Ali Shafahi, Mahyar Najbi,, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3358-3369, 2019b. 2019b..
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint
2014.
David Stutz, Matthias Hein, and Bernt Schiele. alizing unseen attacks. In Proc. of the International Conference on Machine Learning (ICML), 2020.
adversarial training: Gener-
C. Szegedy, V. Vanhoucke, Ioffe, Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818-288888888 2826, 2016.
Jiaxi Tang, Rakesh Shivanna, Zhe Zhao, Dong Lin, Anima Singh, Ed H Chi, and Sagar Jain. Under- standing and improving knowledge distillation. arXiv preprint arXiv:2002.03532, 2020.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In International Conference Learning Representations, 2018.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for- all adversarial training: In-situ tradeoff between robustness and accuracy for free. arXiv preprint arXiv:2010. 18288, 2020a.
Jannu Wang and Haichao Zhang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In Proceedings of the IEEE International Conference on Computer Vision, pp. 6629-6638, 2019.
Xin Wang, Jie Ren, Shuyun Lin, Xiangming Zhu, Yisen Wang, and Quanshi Zhang. A unified approach interpreting and boosting adversarial transferability. In ICLR, 2021.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020b.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020. URL http:ttt//:////eeeeeee
Dongxian Wu, Yisen Wang, and Shu-tao Xia. Revisiting loss landscape for adversarial robustness. arXiv preprint arXi::2004.0588. 2020a.
Dongxian Wu, Yisen Wang. Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR, 2020b.
Cihang Xie, Jianyu Wang. Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 2018.
htt.httss::// wttp://ewww.eeeooomooooooosss//
13

---

Published as a conference paper at ICLR 2021
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017.
Guandao Yang, Tianyi Zhang, Polina Kirichenko, Junwen Bai, Andrew Gordon Wilson, and Chris De Sa. Swalp: Stochastic weight averaging in low precision training. In International Conference on Machine Learning, pp. 7015-7024, 2019a.
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. ME-Net: Towards effective adversarial robust- ness with matrix estimation. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019b.
Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng. Revisiting knowledge distillation via label smoothing regularization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3903-333333 391139 2020.
Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146, 2016.
Wide residual networks.
arXiv preprint
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv: /61 1.0350,, 2016.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227-238, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning (ICML), pp. 7472-7482, 2019b.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference Machine Learning, pp. 11278-12 11287. PMLR, 2020a.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweightee adversarial training. arXiv preprint arXiv:001.011366 2020b.
14

---

Published as a conference paper at ICLR 2021
A1
MORE EXPERIMENT RESULTS
A1.1 STATE-OF-THE-ARY BENCHMARK ON CIFAR-100 100
We implement our methods with exactly the same setting (Rice al. 2020 and compare it with the baseline result reported from the original paper. As shown in Table A7 and Figure A8 our methods achieve great improvements both on robust accuracy and standard accuracy (1.64% RA and 3.78% in SA for co. 4.50% in RA and 3.04% in SA for (2), which establish a new state-of-the- art bar.
Table A7: Comparative Experiment on CIFAR-100 -OFAR- we follow the same setting and compare with the baseline result from (Rice al. 2020). Best refers the model with best robust accuracy during training and Final is average of accuracy over last 5 epochs.
Robust Accuracy (RA)
Standard Accuracy (SA)
Adversary
Norm
Radius
Settings
Best
Final
Diff.
Best
Final
Diff.
PGD
E2
128
Baseline Our Methods
43.20
37.5011.009
5.70
62.50
60.101:0.23
2.40
255
47.70
47.5311000
0.17
65.54
65.5610.01
-0.02
Baseline
28.10
21.40:10.39
6.70
$2.70
54.10:10.23
-1.40
PGD
t0
255
Our Methods
29.74
29.40=10.22
0.34
56.48
57.69:10.03
-1.21
CIFAR-100 A.
CIFAR-100 8
60
60
50
50
Accuracy (96)
Accuracy (96)
40
30
30
20
Baseline
Baseline
20
Baseline SA
Baseline
SA
KD&SWA SA
KD&.SWA SA
KD&SWA
KD&SWA SA
10
50
100
150
200
50
100
150
200
Epochs
Epochs
Figure A8: Results testing accuracy over epochs for ResNet-18 trained on CIFAR- with the same setting as Rice et al. (2020). Dash lines show the standard accuracy (SA); solid lines represent the robust accuracy (RA). Blue, Green and Orange curves represent the performance of Baseline, KD and KD&SWA respectively.
A1.2 T-SNE RESULT ON CIFAR-100
We visualize the learned feature space with all training images and their corresponding adversarial images from PGD-10 10 on CIFAR-100. As shown in Figure A9] our learned features have a larger dis- tance between classes while being more clustered within the same class. The more distinguishable feature embedding justifies the improvement of both robust and standard accuracy.
A1.3 SUPERIOR PERFORMANCE ON SVHN
We conduct our experiments on SVHN with ResNet-18 18 (He et al., 2016 architecture and adopt an SGD optimizer with a momentum of 0.9 and a weight decay of 5 x 10-4 for 80 epochs in total with a batch size of 128. The learning rate starts from 0.01 and follows a cosine annealing schedule. The result can be found best checkpoint for &oo is improved from 52.60% to 53.65 33.65% and robust overfitting is alleviated by 6.30%. In the meantime, standard accuracy has also been improved by 2.47%. The superior performance on SVHN aligns with results on other datasets, which shows the effectiveness of our methods.
Table A8 and Figure A10 As we can see, the robust accuracy of the
A15

---

Published as a conference paper at ICLR 2021
Baseline
Our Methods
20
-10
-80
-20 40 -02
-20
Clean
Clean
Clean
Clean
Clean
Figure A9: I-SNE results of different models trained on CIFAR-I00, Dots and stars represent for clean and adversarial images respectively. Red, Blue and Green represent classes A, B and C respec- tively. For each class, we visualize all training images and their corresponding adversarial images from PGD-10. The left figure is Baseline; the right figure is Our Methods.
Baseline SA
Baseline RA
KD
SWA SA
SWA RA
KD&SWA SA
KW& RA
Figure A10: Results of testing accuracy over epochs ResNet- trained on SVHN. Dash lines show the standard accuracy (SA); solid lines represent the robust accuracy (RA). Blue, Green, Black and Orange curves represent the performance of Baseline, KD, SWA and KD&SWA respectively.
Table A8: Performance showing the occurrence robust overfitting and effectiveness of our proposed reme- dies with ResNet-18 18 on SVHN. The difference between best and final robust accuracy indicates degradation in performance during training. We pick the checkpoint which has the best robust accuracy on validation dataset. The best results and the minimum performance difference marked in bold.
Robust Accuracy (RA)
Standard Accuracy (SA)
Dataset
Settings
Best
Final
Diff.
Best
Final
Diff.
Baseline
52.60
43.30
9.30
87.93
89.94
-2.01
SVHN
Bascline + &CD
52.93
48.46
4.47
87.62
91.36
-3.74
Bascline + ICDstd&add + SWA
53.65
50.65
3.00
90.40
91.70
-1.30
A1.4 LOCAL LINEARITY
As proposed by (Andriushchenko & Flammarion 2020). the catastrophic overfitting problem is mainly due to the local linearity reduction when adversarial training with FGSM et al. 2020). So we borrow this measurement in the robust overfitting scenario, which calculates the expectation of the cosine similarity of the gradient between the original input and randomly perturbed one with a uniform distribution, shown in Eqn. The result shown in figure A1l indicaats that methods help to slow the decline of local linearity and the maintenance of local linearity is also helpful for preventing robust overfitting.
A16

---

Published as a conference paper at ICLR 2021
E B(x.))ED.nE((((s0))
(Vx))((0))),)) (
The Local Linearity
Baseline
KD
0.96
KD&SWA
Cosine Similarity arit
0.94
0.92
0.90
0.86
0
50
100
150
200
Epochs
Figure A11: The local linearity is calculated from all test images CIFAR-100 with models from each period of the training process. Blue, Green and Orange curves represent the local lincarity of Baseline, KD and KD&SWA respectively.
A1.5 ABLATION OF TRANSFER ATTACK
With the purpose of fully comparing the effects of label smoothing and knowledge distillation, we introduce transfer attack with an unseen non-robust model with the same architecture, follow the same setting as (Fu et al. 2020). A higher accuracy on the unseen model indicates a weaker attack generated by the corresponding setting while higher accuracy from the unseen model means better robustness. As shown in Table A9 only knowledge distillation shows significant improvement with both accuracies, compared with basDDine(PGD-AT) methods. The strength of generated adversarial images is improved by 4.32% and the robustness is improved by 2.91 % for the best model. We also experiment with an unseen robust model and get consistent improvement. This improvement indi- cates that knowledge distillation introduces more discriminating information from teacher models, which is better than manually designed label smoothing methods.
Table A9: Ablation of Transfer attack. The accuracy on unseen model is the accuracy of unseen model with adversarial images generated source models from different settings and the accuracy from unseen model means the opposite. We generated adversarial images for all test images on CIFAR-10 with lx PGD-20. Baseline represents the PGD-AT methods.
Accuracy on unseen model
Accuracy from unseen model
Settings
Best
Final
Best
Final
Baseline
69.87
80.43
79.72
81.77
Label smoothing
70.94
81.24
78.44
82.82
Tf-XCDreg
73.29
82.21
79.47
82.71
K"Dstakady-PGGn PGD10
65.55
72.63
82.63
84.01
A1.6
SWA VERSUS ISWA
One possible extension of SWA is to replace WT-1 with SwA in Eqn. 2 iSWA and compare with the original SWA in Table Both weight smoothing techniques can mitigate robust overfitting, and iSWA performs slightly better on RA while sacrificing some SA.
We name this variant as
A17

---

Published as a conference paper at ICLR 2021
Table A10:
Ablation of SWA CIFAR-10.Bes refers the model selected with best robust
accuracy on validation dataset and Final is the model at the end of training process.
Robust Accuracy (RA)
Standard Accuracy (SA)
Settings
Best
Final
Diff.
Best
Final
Diff.
KCD stddedddy--GDDD SWA
52.14
51.53
0.61
84.65
85,40
-0.75
PGDD ISWA
52.36
52.33
0.03
83.17
83.54
-0.37
A2
MORE METHODOLOGY AND IMPLEMENTATION DETAILS
A2.1
KNOWLEDGE DISTILLATION
We state ICD as follows:
KD(Y.0) -H(I(y), = Σ t(y) log (0);
where t(y);
(//) (/,
T 2 in our case, following the standard setting in (Hinton et al. 2015
Li & Hoiem 2017
A2.2 ADVERSARIAL TRAINING
Adversarial training incorporates generated adversarial examples into the training process and sig- nificantly improves the robustness of networks. In our paper, we implemented three different ad- versarial training schemes: FGSM, PGD and TRADES, which can be described as the optimization problem below: Eqn/4 for FGSM and PGD, Eqn for for TRADES.
min (Xy)ED max C(f(0, 8).9) (x
(4)
min (XajeD ((f(0.x),)) + + max x) 8xB,(x)
(5)
As for the maximization process, FGSM perturbs the input with single step in the direction of the sign of the gradient and PGD is the iterative form of FGSM with random restarts, which works as follows.
81+1=
= ( +0),
(6)
81+1
a (
(7)
TRADES (Eqn 7 replaces the cross entropy loss in PGD with the Kull-back-Leibler divergence of network output for the clean input and the adversarial input. Where is the network with parameters 0, (x, y) is the data. a is the step size and 8' is the adversarial perturbation after times iterations. The perturbation is constrained in an Ep norm ball, i.e.|68ll S which is realized by projection.
2 for PGD and consider both Eoo and l2 in our paper. For foo adversary, we use e 255 and a
255
TRADES with 10 steps in training and 20 steps in testing, while using 253 for FGSM during training. As for & adversary, use all three attack methods.
128 and 15 with the same steps loc adversary in
255
For comprehensive evaluation, consider two improved attacks, ie.., Auto-Attack (Croce & Hein 2020) and CW Attack (Carlini Wagner (017). We use the official implementation and default settings for Auto-Attack (e with 23 and la with 255
128 and the implementation
A18

---

Published as a conference paper at ICLR 2021
from AdverTorch (Ding et al. 2019) for CW attack with the same setting Rony al. (2019).
specifically, search step on C with an initial constant of 0.1, with 100 iterations for each search step and 0.01 learning rate. Detailed links are provided below:
The Official Repository:
The Leaderboard: tttps://robustbecch..... oithrmb.o.......oo..
A19


Reviews:
[{"review_id": "u_PP25tvI", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies a method for mitigating robust overfitting. Rice et al., and others have observed that when training a neural network robustly on say CIFAR10, then the robust test error often overfits, i.e., it has a U-shaped curve as a function of training epochs. Rice et al. demonstrated that early stopping the robust training enables state-of-the-art robust performance. However, to realize this performance, it is necessary to find a good early stopping point, which can be difficult (but can be found with testing on a validation set). The paper proposes an alternative to early stopping: smoothing the logits and smoothing the weights, by using  two existing techniques, namely self-training and stochastic weight averaging. The paper finds that smoothing mitigates robust overfitting, and reports even a slight improvement over early stopping at the optimal point.\n\nStrength:\n- The paper proposes two existing techniques to prevent robust overfitting:  self-training and stochastic weight averaging, and shows that combining those two approaches is very effective in preventing robust overfitting for ResNet-18.\n- The paper is well written and easy to follow.\n\nWeaknesses:\n- All experiments are carried out with one network only: ResNet-10. To validate the claim that learned label smoothing can mitigate robust training, is important to test the label smoothing method on a variety of setups and models.\n- The improvement are tiny relativ to early stopping, and another submission to this workshop (https://openreview.net/pdf?id=Xb8xvrtB8Ce) has shown that the baseline the paper under review is comparing to (the setting of Rice et al.) is quite brittle relative to choices of hyperparameters such as slight differences in weight decay. Therefore, the gains obtained by the paper under review could very well be subsumed by slightly tuning early stopping.\n\nSummary: It is an important problem to study methods that mitigate robust overfitting, and the paper proposes a combination of two smoothing techniques and demonstrates its effectiveness through extensive experiments. I'm therefore leaning to recommend acceptance of this paper, however, as mentioned, the paper's results might not generalize to other models and the slight gains over early stopping might be void by slightly tuning ES better. Therefore, I would not be upset if this paper were rejected. In any case, the paper's results would be more convincing if it would contain results for different models (e.g., VGG and other deep nets but also simple baseline models such as random feature models), and if it would contain a simple theoretical statement to provide intuition why label smoothing should help. \n\n---- \nUPDATE: Thanks; I have read the response, kept my score, and responded below.", "extra_scores": {}, "extra_text": {}}, {"review_id": "in8QAuRcMU", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "#########################################################################\n\nSummary:\nThis paper uses the existing tricks that can enhance the standard training, to show that combining some of those tricks (in this paper, labels/logits smoothing and weight averaging) can improve adversarial training. \n\n#########################################################################\n\nPros: \n1 Compared with existing weight manipulation AT methods, this paper first utilizes stochastic weight averaging (SWA) (averaging multiple checkpoints along the training trajectory) without incurring computational overhead. \n\n2 This paper conducted experiments across four different datasets.\n\n#########################################################################\n\nCons: \n1 The paper’s novelty is marginal. Specifically, first, label/logit smoothing has been demonstrated effective in adversarial training due to the better separation of different classes. For example, to my knowledge, three papers got accepted with the shared philosophy but slightly different techniques/decorations [1, 2, 3] \nSecond, as the authors mentioned, manipulating model weights is also shown effective [4].\nTherefore, this paper's conceptual improvements are marginal.  \n\n[1] Metric Learning for Adversarial Robustness, NeurIPS 2019\\\n[2] Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness, ICLR 2020\\\n[3] Boosting Adversarial Training with Hypersphere Embedding, NeurIPS 2020\\\n[4] Revisiting loss landscape for adversarial robustness, NeurIPS 2020\n\n2 This paper hypothesizes “one source of robust overfitting might lie in that the model ‘overfits’ the attacks generated in the early stage of AT and fails to generalize or adapt to the attacks in the late stage.”  It is not clear to me why this hypothesis is valid. Would you explain more justifications?\n\n3. In Figure 3, are there any experimental results for SWA SA and SWA RA? \nBesides, more robustness evaluations are needed, e.g., CW attack, AA attack, Guided Adversarial Margin Attack. \nMore adversarial training on different network structures are needed, e.g., Wide ResNet. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ha8AU0FU9oP", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n=======\nThe paper leverages two methods for improving generalization in standard training, logit smoothing and stochastic weight averaging, and show that these results can mitigate robust overfitting and improve generalization for adversarial training methods. \n\nOverall, the paper was clear and easy to follow. There are a number of ablation studies showing the marginal effects of the two methods, as well as experiments demonstrating how the approaches vary with certain choices in methodology. My initial impression is positive, though there are certain changes described below which would help solidify the paper and its claims. \n\n\nComments for discussion\n=======================\nBy improving upon the results in Rice et al. 2020, the authors purport to have state of the art results. However, there's be a plethora of new work since then which have improved these numbers even further. Fortunately, since the submitted work handles the standard CIFAR10 setting, there are a number of public benchmarks that can be used here. It would be great if the authors could train a comparable model and evaluate it using one of these benchmarks (e.g. the autoattack framework at https://github.com/fra31/auto-attack). \n\nTo be clear, since a number of these approaches on the benchmark are quite recent, I am not requesting that the authors directly compare to these new methods in their work. However, the baseline that they do compare to (e.g. Rice et al. 2020) is evaluated in this framework (and had a not-insignifcant drop in robust accuracy), so it would be of significant utility to also evaluate the approach using the improved attack. Performing this evaluation would serve two purposes: \n\n1. This should alleviate most concerns on the validity of the result \n2. This makes the work easily comparable for future work \n\nNote that reaching the top of the benchmark is not a requirement for publication. As long as it is consistent with the claims of the paper, that the approach reduces robust overfitting for PGD training and improves upon the PGD baseline within this benchmark, then this is fine. If the authors can report how their approach performs under this improved evaluation or a comparable alternative, then I am happy to adjust my score accordingly. However, the authors probably shouldn't claim state-of-the-art performance without doing this evaluation first. \n\n\nMinor comments\n==============\n+ In section 3.2, it is mentioned that Table 2 supposedly shows differences when a robust self-teacher is pretrained, but this does not seem to be the case. \n\n\nUpdate\n======\nI have looked through the response and edited version. The updated evaluation looks solid and provides a potential solution to a robust overfitting problem. Although the work is primarily empirical in nature, it may inspire directions for future work to look into more theoretical explanations of robust overfitting. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on the problem of robust overfitting. The philosophy behind sounds quite interesting to me, namely, injecting 
more learned smoothening during adversarial training. This philosophy leads to two simple yet effective methods: one leveraging knowledge distillation and self-training to smooth the logits, and the other performing stochastic weight averaging to smooth the weights.

The clarity and novelty are above the bar of ICLR. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please carefully address all 
comments in the final version.
Decision: Accept (Poster)