Double Descent in Adversarial Training: An Implicit Label Noise Perspective
Authors: Chengyu Dong
Liyuan Liu
Jingbo Shang
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial training
Robust overfitting
Double descent
Label noise

Abstract: Here, we show that the robust overfitting shall be viewed as the early part of an epoch-wise double descent --- the robust test error will start to decrease again after training the model for a considerable number of epochs. Inspired by our observations, we further advance the analyses of double descent to understand robust overfitting better. In standard training, double descent has been shown to be a result of label flipping noise. However, this reasoning is not applicable in our setting, since adversarial perturbations are believed not to change the label. Going beyond label flipping noise, we propose to measure the mismatch between the assigned and (unknown) true label distributions, denoted as \emph{implicit label noise}. We show that the traditional labeling of adversarial examples inherited from their clean counterparts will lead to implicit label noise. Towards better labeling, we show that predicted distribution from a classifier, after scaling and interpolation, can provably reduce the implicit label noise under mild assumptions. In light of our analyses, we tailored the training objective accordingly to effectively mitigate the double descent and verified its effectiveness on three benchmark datasets.


Under review as a conference paper at ICLR 2022
DOUBLE DESCENT IN ADVERSARIAL TRAINING: AN IMPLICIT LABEL NOISE PERSPECTIVE
Anonymous authors Paper under double-blind review
ABSTRACT
Here, we show that the robust overfitting shall be viewed as the early part of an epoch-wise double descent the robust test error will start to decrease again after training the model for considerable number of epochs. Inspired by our observations, we further advance the analyses of double descent to understand robust overfitting better. In standard training, double descent has been shown to be a result of label flipping noise. However, this reasoning is not applicable our setting, since adversarial perturbations are believed not to change the label. Going beyond label flipping noise, we propose to measure the mismatch between the assigned and (unknown) true label distributions, denoted implicit label noise. We show that the traditional labeling of adversarial examples inherited from their clean counterparts will lead implicit label noise. Towards better labeling, we show that predicted distribution from a classifier, after scaling and interpolation, can provably reduce the implicit label noise under mild assumptions. In light of our analyses, we tailored the training objective accordingly to effectively mitigate the double descent and verified its effectiveness on three benchmark datasets.
INTRODUCTION
In adversarial training, a typical phenomenon is that after a certain training epoch, the robust test error will start to increase constantly with further training, despite the robust training error continues decrease (Rice 2020). This phenomenon, known as robust overfitting, believed to be separated from double descent (Belkin et al.) 2019) in the literature (Rice et al. 2020).
Here, we find that robust overfitting shall be viewed as the early part of an epoch-wise double de-
80
WRN-28-114 1/4
Robust Test Error (%6)
WRN-26
scent (Nakkiran et al.) (020). As shown in Figure []]
WRN-28-5
for relatively large models, the robust test error in-
creases only transiently and will eventually decrease again after considerable number of training epochs. Morcover, we observe consistent and similar patterns across various training settings (e.g., different sample sizes, optimizers, learning rate schedulers, and neural architectures), which further verifies our intuition on the connection between the robust overfitting and the double descent (See Appendix B.I.
70-
65
55-
10a
10a
102
202
Epochs
Figure 1: Robust overfitting can be viewed
Inspired by this observation, we further analyze the epoch-wise double descent to better understand the robust overfitting. In standard training, the double descent shown to be related to the label flipping noise (Nakkiran et al., 2020) Yang et al. (020b). However, adversarial perturbations are believed to not change underlying labels, which is further con-
as an early part of the epoch-wise double de- scent. Here we employ PGD training (Madry) et al. 2018) on CIFAR- (Krizhevsky (2009) with Wide ResNet (WRN) (Zagoruyko & Ko modakis 2016 and fixed learning rate.
firmed empirically from our manual inspection. Therefore, the popular label flipping noise explanation is not applicable here. On the other hand, it is reasonable to believe that adversarial perturbation distorts the label distribution since the examples do become more ambiguous after perturbation. Thus, focus the implicit label noise which measures the mismatch between the assigned label

---

Under review as a conference paper at ICLR 2022
distribution and (unknown) true label distribution. Extensive analyses of such implicit label noise reveal the intriguing dependency of robust overfitting on perturbation size (Dong et al., 2021b) and data quality (Dong al., 2021a).
Guided by our analyses, we design a searetically-rronndee method to mitigate the robust overfitting. The key idea is to resort to alternative labeling of the adversarial examples. We show that the predictive label distribution of a probabilistic classifier adversarially trained as usual, but after being properly scaled and interpolated, can be utilized as a better labeling of the adversarial examples and provably reduce the implicit label noise. This echoes the recent empirical practice of incorporating knowledge distillation (Hinton al.) 2015) into adversarial training (Chen et al., (02T). While previous works intuitively select fixed scaling and interpolation parameters for knowledge distillation, we show that it is possible to fully unleash the potential of knowledge distillation by automatically determining the set of parameters that maximally reduces the implicit label noise with strategy similar to confidence calibration (Guo al., (017). Such strategy can further mitigate robust overfitting to minimal amount without additional human tuning effort. Extensive experiments on different datasets, training methods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.
In summary, our findings and contributions are as follows.
We show that robust overfitting shall be viewed as the early part of an epoch-wis wise double descent, extending the common belief in adversarial training.
We show that double descent in adversarial training may originate from the implicit label noise introduced by improper labeling of adversarial examples in adversarial training practice.
We show an alternative labeling of the adversarial examples can be established to provably reduce the implicit label noise and mitigate the robust overfitting.
The remainder of this paper is organized as follows. In Section 2 we briefly review the existing
works that explore robust overfitting and double descent. In Section 3] we theoretically explore the origin of double descent in adversarial training from an implicit label noise perspective. In Section 4 we propose mitigate the double descent in adversarial training by alternative labeling based on our understanding. Section 3 demonstrates the effectiveness of our method on realistic datasets. Conclusions and further implications are discussed in Section 6
2 RELATED WORK
Robust overfitting and double descent in adversarial training. Double descent refers to the phenomenon that overfitting by increasing model complexity will eventually improve test set performance (Neyshabur et al. 2017 Belkin et al. 2019). This appears to conflict with the robust overfitting phenomenon in adversarial training, where increasing model complexity by training longer will impair test set performance constantly after a certain point during training. It is thus believed in the literature that double descent and robust overfitting are separate phenomena (Rice et al. 2020).
Towards a more complete understanding of robust overfitting, in this work, conduct adversarial training for exponentially more epochs than the typical practice. We find that robust overfitting shall be viewed as the early part of an epoch-wise double descent. And increasing the model architecture size, another way to increase the model complexity, can modulate the epoch-wise double descent curve such that either the overfitting curve shown, or the entire double descent curve revealed within the same number of training epochs as shown in Figure]] [] Therefore, robust overfitting will not go beyond modern generalization theory as an exception and should be adequately explained by the origin of double descent such as label noise.
recent work also considers different notion of double descent that defined with respect to the perturbation size YYu et al. 2021). Such double descent might be more related to the robustness- accuracy trade-off problem (Papernot et al. 2016 Su et al. 2018: Tsipras et al. 2019 Zhang et al.. 2019) rather than the classic picture of double descent based model complexity.
Understand double descent in adversarial training. In standard training, double descent is often attributed to increased variance, with label noise being common source. Definitions of label noise vary in literature. poooreiicall-ggounne analyses of double descent focus on additive label
noise (Advani& & Saxe 2020 Mei & Montanari 2019
Hastie et al., 2019 Belkin et al. 2020 d' d'Ascoli
2

---

Under review as a conference paper at ICLR 2022
et al. 2020), but only applicable to regression problems. Theoretical results double descent scarce on classification problems, with a few works introducing noise by randomly masking the feature vector (Deng et al., 2019 Kini & Thrampoulidis, 2020). Analyses of double descent classification problems are more common in empirical studies, where a typical way to induce double descent is to inject label flipping noise, namely the labels of a random fraction of training examples are flipped to other labels (Nakkiran et al., 2020; Yang et al. (020b). However, such a definition of label noise cannot properly fit the scenario in adversarial training, where labels are not likely to be flipped due to small adversarial perturbation.
To understand the double descent in adversarial training within the existing picture of double descent, we introduce implicit label noise which is incurred by the mismatch between the assigned label distribution and true label distribution of the adversarial example. Such label noise can be interpreted as an sistance-dependntt class dependent label noise in reference to the systematic studies on the taxonomy of label noise (Frénay & Verleysen 2014 double descent in adversarial training.
Song et al.) 2020), thus can be the origin of
Mitigate robust overfitting, Robust overfitting hinders the practical deployment of adversarial training methods as the final performance is often sub-optimal. Various regularization methods including classic approaches such as l and &g regularization and modern approaches such as cutout (Devries & Taylor (017) and mixup (Zhang et al. 2018) have been attempted to tackle robust overfitting, whereas they are shown to perform no better than simply early stopping the training on validation set (Rice et al., 2020). However, early stopping raises additional concern as the best checkpoint of the robust test accuracy and that of the standard accuracy often do not coincide (Chen et al., 2020). thus inevitably sacrificing the performance on either criterion. Various regularization methods specifically designed for adversarial training are thus being proposed to outperform early stopping, including regularization the flatness of the weight loss landscape (Wu) et al. 2020 (Stutz et al.) [202T) introducing curruuraerre activation functions (Singla et al., (202)) and adopting stochastic weight averaging (Izmailov et al. (018)) and knowledge distillation (Hinton) et al. 2015) (Chen et al. 202T). These methods are likely to suppress the implicit label noise, with the self-distillation framework (i.c. the teacher shares the same architecture as the student model) introduced by (Chen et al.) 2021) supervision is almost equivalent to the alternative labeling inspired by our understanding of the origin of label noise in adversarial training.
particular example since introducing teacher's 's outputs as
3 DOUBLE DESCENT FROM AN IMPLICIT LABEL NOISE PERSPECTIVE
In this section, we present a novel perspective to understand the double descent in adversarial training. All proofs in the remainder of this paper are provided in the appendix.
3.1 PRELIMINARIES
define the label space. Considering a classification problem with training set D = {(x y)). where x X is an input and E y is its assigned label. We denote p(Y = =]]lz) as the distribution of the assigned labels of given by a group of annotators. Typically y = arg max, p(Y jlæ²). When there is no ambiguity, we will denote p(Y = j|z)) as p(y]æz) with a slight abuse of notation. Let y" y be the true label of x and p(y" (x) be its distribution, which can be given by a group of experts. Note y" = arg max, p(Y" = j ())
Let X' C R2d define the input space equipped with a norm || : - R+ and y (1,,
Assumption 3.1. We assume there is no label noise in D, namely y y² and ply x)) = p(y" (z). Definition 3.1 (Data quality). Given an example x X, we define its data quality as max, p(Y* jlæ), namely the maximum class probability of the true label distribution.
Assumption 3.2. We assume the original clean dataset D contains mostly high-quality data, namely max, p(y =](x)= R 1.
We now briefly review adversarial training. Let f(::0) X' + 3 be a probabilistic classifier and f(: 0)₂ be its j-th class probability. Adversarial training can be viewed as an data augmentation technique that trains the classifier on set of adversarial examples D3 ((18, ys)), namely
0" = arg min Ep, {(f(x( 0), ys).
(1)
3

---

Under review as a conference paper at ICLR 2022
True Label Assigned Label
airpiane
automobule
bird
cat
Harle
4-5
Traditional Adversarial
Label
Rectified Model Probability
Figure 2: The illustration of the origin of implicit label noise in adversarial training. The traditional adversarial label introduces the implicit label noise by inducing a distribution mismatch between the assigned label distribution and true label distribution of the adversarial example. Our rectified model probability as an alternative labeling can provably reduce this distribution match.
Here, 2g + 8 is the adversarial example of produced by the inner maximization, where = =ar max f(J(x + 6; 0)
[Se
and ys E y is the assigned label of xg. Finally, we denote vī E y as the true label of 25-
Remark 3.1. In adversarial training, it is the common practice that copies the label of a clean example to its adversarial counterpart, namely ya + y and plys (z8) p(y]z).
We denote 5s and its associated distribution as the Iruditional adversarial label. We believe traditional adversarial label the key problem that induces double descent adversarial training.
3.2 TRADITIONAL. ADVERSARIAL LABEI. DOES NOT INTRODUCE LABEL FLIPPING NOISE
In standard learning, it often necessary manually inject label noise make the double descent evident for modern neural architectures (Nakkiran et al. 2020 Yang et al. 20206). We wish to check if the traditional adversarial label produces any label flipping noise, a typical type of label noise, namely if ÿs F yī for any adversarial example zz. This is equivalent to checking if y" * yg- since Ts = y by the construction the traditional adversarial label (Remark 331] and = y" by the construction of our clean dataset (Assumption) (.I). Note that y² + V means the semantics of the adversarial example distorted significantly such that its true label is now different from the true label of its clean counterpart.
We first visually check the adversarial examples created by the inner maximization in adversarial training. We are specifically interested in those examples with the lowest data quality they are mostly likely to become label noise after perturbation. We estimate the data quality based on model ensemble as shown in Appendix E. One can find that the adversarial counterparts of those low-quality examples still match their original labels, albeit being slightly more ambiguous.
We have also found some evidence from the literature. Yang et al. (2020a) draw the conclusion by studying the distance between examples the input space. In the CIFAR-10 training set, they found that the minimum distance between any two examples from two different classes respectively is around 50 50/255 in terms of foe norm, while the perturbation radius typically used in adversarial training is around 8/255, which is significantly smaller. It is thus reasonable to claim that a significant amount of label flipping noise will not be produced by the inner maximization, and subsequently it cannot explain the double descent observed in adversarial training.
3.3 TRADITIONAL ADVERSARIAL LABEL INTRODUCES IMPLICIT LABEL NOISE
Although traditional adversarial label is unlikely to directly produce label flipping noise, we show that it can produce label noise implicitly.
An adversarial example often contains salient characteristics of classes other than the original label from human perspective (Tsipras et al, 2019; ]lyas et al. 2019). This implies that the true label distribution of adversarial example is likely to be different from its clean counterpart, which means
4

---

Under review as a conference paper at ICLR 2022
p(vā(=s) # p(y"lz). Note this will not conflict with vă y" discussed in Section 3.2 since the latter only implies arg max; p(Yg = = = arg max, p(Y =)))).
Now show the traditional adversarial label is improper in terms of the label distribution. As illustrated in Figure 2. we have p (28) = p(ylz) by Remark 3.1 and p(y|2) = ply by Assumption 3.1 which together with the effect of adversarial perturbation p(uz as) F p(v" [=)
imply that p (2)) F p(uš 2.5). This means there is a distribution mismatch between the true label
distribution and the assigned label distribution of the adversarial example. Such distribution mismatch introduces implicit label noise in the training set Dj employed in adversarial training.
Definition 3.2 (Implicit label noise). Given an example the probability that its assigned label is different from its true label, namely p(Y + Y*)x).
X, we define the implicit label noise as
Remark 3.2. Implicit label noise is equivalent to instance-depender and class-dependent label
noise, since p(Y + Y*1x) -1- -1-2, p( = =j j)z)pp(Y* = =](r), and p(Y + j|Y
j. x) = 1 p(Y j|y = j. a typical definition of label noise. It can easily seen that if p(Y # Y x)) O, p(Y # = j.x) > for some j.
Before showing the connection between the distribution mismatch and the implicit label noise, we need quantify the distribution mismatch first. Here adopt the total variation distance.
Definition 3.3 (Total variation (TV) distance). Let J be a subset of the label sample space y. For two discrete probability distributions ply) and p(y/) where y.y of V, the total variation distance between them can be defined as
(pp()) - p(y') max
Σrly = - = JEJ
JEJ
We are now ready to present our main result.
Theorem 3.1 (Traditional adversarial label incurs implicit label noise). The implicit label noise incurred by the distribution mismatch between the traditional adversarial label and the true label of the adversarial example is howrr-bounded, namely
plūs + y xxs) 2. [|p(ūs(=s) ply3l=s)]lzv [lp(y"(z) p(uā(=>))l
(4)
An informal proof can be sketched from a frequentist's view and help the understanding of implicit la- bel noise. Say there are M identical copies of æ5 in the training set D3. with their true labels and tradi- tional adversarial labels distributing according to p(v (z)) and p(ū6 (ass), respectively. The number of
copies that have the same true label and assigned label is M Σ3 min ( s = jlæs), p(Y5 j|
The fraction of label noise exists in D3 is thus ΣS min(p(Vs = j(ara) = =j (xx) [pp(0s(za) p(yğ [rr by the definition of the total variation distance.
3.4 DEPENDENCE OF IMPLICIT LABEL NOISE IN ADVERSARIAL TRAINING
We now show the implicit label noise in adversarial training depends on the perturbation size and the data quality, which will subsequently affect the double descent curves.
We consider optimal adversarial perturbation for simplicity, which is defined as
8" = arg max
((f 8)6),
(5)
where (-) denotes the optimal probabilistic classifier such that (*)x (x:). With e being the cross-entropy loss, the above equation can he rewritten as
arg min fa(x + 8;0)
which implies the optimal perturbation will reduce the probability mass of the true label. With the optimal adversarial perturbation, the minimum implicit label noise can be solved follows.
Note here the assigned label is random variable rather than an outcome. See Theorem 3.1 for hetter understanding of implicit label noise.
5

---

Under review as a conference paper at ICLR 2022
Theorem 3.2 (Minimum implicit label noise under optimal perturbation). Assume the true label distribution ply" (x) is locally convex around x and can be asymppooicaalyy described as
v p(Y - ا|احا| x ppppp = j]ax) p(Y" =
we have
min pLñ6 + v8 (36) x s(1 max pa = j|z)),
where min means the lower bound of the minimum label noise.
In Appendix A.
show that Assumption (7) holds true for Gaussian mixture model. The above theorem shows that, when the data quality of the clean example is relatively high, the implicit label noise introduced by adversarial perturbation proportional to (I) the perturbation radius (2) the data quality. We conduct controlled experiments and empirically verify this correlation in Appendix B.2
Interestingly, the above analysis implies that if the perturbation does reduce the probability mass of the true label, it will not cause implicit label noise even with an extremely large perturbation radius. We demonstrate this detail and empirically verified it for Gaussian noise in Appendix [..3]
3.5 IMPLICIT LABEL NOISE INDUCES DOUBLE DESCENT
In standard training, the effect of label noise on dou-
ble descent has been rigorously studied based upon
Standard Test Error (%)
both analytical settings (Mei & Montanari 2019
Hastie et al.) 2019; Deng al. 2019| Belkin et
60
2020) and bias- variacee analyses (Jacot et al. 2020
Yang et al. 2020b; Ascoli et al. 2020). Since im-
plicit label noise is just special case of label noise
40
(Remark 3.2). and adversarial training can be viewed as standard training on an augmented dataset (Equa- tion (.. it can be inferred that implicit label noise will inerease the variance and make evident dou- ble descent in adversarial training. To demonstrate this a straightforward way, in Figure [ we employ standard training on dataset augmented by fixed ad- versarial perturbation and show can indeed produce double descent.
8/255
16/255
Epochs
Without explicitly injecting label noise, standard training can also produce dou- ble descent if conducted on dataset aug- mented by fixed and small adversarial per- turbation, which can be properly explained by our implicit label noise perspective. De- tailed experiment settings can be found in Appendix F3
Figure 3:
Since implicit label noise modulates the double de- scent, and by Theorem 3.2 it depends on the pertur- bation radius and data quality, the double descent in adversarial training should strongly correlate with the perturbation radius and data quality. Indeed, has been observed respectively that small perturbation radius will not induce robust overfitting (Dong et al. 20216), and high-quality data will not induce robust overfitting (Dong et al. 2021a).
MITIGATE DOUBLE DESCENT IN ADVERSARIAL TRAINING
In this section, focus on suppressing the implicit label noise in adversarial training mitigate double descent from both theoretical and practical perspectives. Since the implicit label noise is incurred by the traditional adversarial label which improperly labels the adversarial example and causes a distribution mismatch between its assigned label and true label, we wish to find an alternative label (distribution) for the adversarial example to reduce such distribution mismatch.
4.1 RECTIFY MODEL PROBABILITY TO REDUCE DISTRIBUTION MISMATCH
We show that it is possible reduce the distribution mismatch by utilizing the predictive probability of classifier trained on traditional adversarial labels, which we will refer as the model probability for simplicity. We now provide a theoretical guarantee to show that, with temperature scaling (Hinton et al. 2013) Guo et al. (2017) enabled in the softmax function, model probability induces distribution mismatch provably smaller than the traditional adversarial label.
Guo

---

Under review as a conference paper at ICLR 2022
Theorem 4.1 (Model probability induces smaller distribution mismatch than the traditional adver- sarial label), Let fxx;8TT) denote the predictive probability of a hypothesis classifier scaled by temperature T, namety
exp(2,/T)
f(x; (a;; E3 exp(2;/T))
where 2 is the logits of the classifier from x.
Let æg be adversarial example correctly classified by classifier i.e. arg max, f(zs; vz- then there exists T, such that
[1 8, T) plyš (2))llr & (Ip(ūs(z)) p(v xr) [2))ll
One possible way to further reduce the distribution mismatch is to interpolate between the model probability and the traditional adversarial label.
Theorem 4.2 (Interpolation can further reduce the distribution mismatch), Let vs be an adver- sarial example incorrectly classified by classifier J, i.e. arg max, f(raa; ((nx;;,T) 0,T); + vz. Assume max; p(Yg (aaa) 1/2, then there exists interpolation ratio X, such that
|| f( T) + - X) p(ūa|aaa p(yō l I (1ss 0, T) p(uš
Note the above theorem focus incorrectly classified examples and thus can be regarded complement to Theorem 4.1
As summarization, to reduce the distribution mismatch, we propose the following distribution as the assigned label the adversarial example in adversarial training instead the traditional adversarial label.
(os (x8) = X f(xs; 0, 7) + (1- -1--)) A)- ((ūslaa))
We refer this label distribution the Rectified model probability.
In Appendix D we show that the optimal hyper-parae syper-parameters (i.e. and X) of almost training examples concentrate the same set of values by empirically studying synthetic dataset with known true label distribution. Therefore is possible to find an universal set of hyper-parameten that reduce the distribution mismatch for all adversarial examples.
4.2 DETERMINE THE OPTIMAL TEMPERATURE AND INTERPOLATION RATIO
The set of temperature and interpolation ratio in the rectified model probability that maximally reduces the distribution mismatch is straightforward find as the true label distribution of the adversarial example is unknown in reality. Fortunately, given a sufficiently large validation dataset as whole, it is possible to measure the overall distribution mismatch in frequentist's view without knowing the true label distribution of every single example. A popular metric adopted here is the negative log- likeeiioehood (NLL) loss, which is known as proper scoring rule (Gneiting & Raftery 2007) and is also employed in the confidence calibration of deep networks (Guo et al. [2017). By Gibbs's inequality is easy to show that the NLL loss will only be minimized when the assigned label distribution matches the true label distribution (Hastie et al. (001), namely
IF (es (EE log PogTe, (Y, = 1/((13) 2 EF DDy (ms)) log
And here y is known since the adversarial example should share the same argmax label with its clean counterpart as shown in Section 3.2
Therefore, we propose to find the optimal T and À in Equation g as
T, = arg min E (ee log (Ys = yg 22)
(10)
T.A
One may note that Equation 100) cannot be directly optimized since the traditional adversarial label only defined on the example the training set and cannot be simply generalized to the validation set. A reasonable solution is using the nearest neighbour classifier to find the closest traditional adversarial label for every example in the validation set. However, to speed up the optimization we propose to employ the classifier overfitted by the traditional adversarial labels on the training set as an surrogate (see Appendix E.2), which works well in practice. Such process incurs almost no simply obtain the logits of surrogate classifier the validation set.
additional computation
7

---

Under review as a conference paper at ICLR 2022
4.3 RECTIFIED MODEL PROBABILITY MITIGATES DOUBLE DESCENT
We now work on realistic dataset (CIFAR-10) to demonstrate the rectified model probability proposed in Equation ( can effectively mitigate the robust overfittin, equivalently the epoch-wise double descent in adversarial training. The outer minimization of adversarial training (Equation ! now becomes
0 = arg min ED, e (f 0), Po x (W6(2)))
(11)
0
where Trad denotes the parameters of classifier trained the traditional adversarial label beforehand. The details of the experimental setting are available in Appendix F.
As shown in Figure 4 adversarial training on rectified model probability can best mitigate the robust overfitting when the temperature 7 and interpolation ratio À are optimal. Such optimal hyperparameters perfectly aligns with the ones automatically determined by Equation (10).
1.34
NLL (Val)
1.33
NLL
155
10
10
0.5
0.5
0.7
0.8
0.8
1.0
Robust Test Acc (%)
Robust Test Acc (%)
49.0
48.5
42-
47.0
46-
Chockpoint Chockpolnt
46.5
Chockpoint
10
0.5
0.8
Figure 4: (Upper) NLL loss obtained on the validation set for different T and x. (Bottom) Robust test accuracy at the best and last checkpoint by adversarial training with the rectified model probability with different T and X. X = 0.8 for grid search on 7 (Left) and 7= for grid search on À (Right). Orange dashed lines in the left and right columns indicate the temperature and the interpolation ratio automatically determined by Equation (IO), respectively.
EXPERIMENTS
Experiment setup. We consider three datasets, CIFAR-10, CIFAR-100 (Krizhevsky 2009) and
Tiny-ImageNe (Le & Yang 2015). We consider robustness against lo norm-bounded ad adversarial attack with perturbation radius 8/255, and employ AutoAttack (Croce & Hein 2020) for reliable evaluation. We conduct PGD training on pre-activation ResNet-18 (He et al. (016) in this section.
Appendix includes results on additional adversarial training methods (e.g.. TRADES (Zhang et
2019). FGSM (Goodfellow et al. 2015)), neural architectures (e.g.. VGG (Simonyan & Zisserman
2015). WRN) and evaluation metries (e.g., PGD-1000, Square Attack (Andriushchenko et al. 2020).
RayS Chen & Gu 22200)) More setup details can be found in Appendix
Our method essentially adversarial training with self-distillation equipped with algorithm automatically searching for the optimal papem-paraettes. which now denote as KD-AT-Auto, We compare KD-AT-Auto with two baselines: regular adversarial training traditional adversarial label (AT), and adversarial training combined with self-distillation (KD-AT) and interpolation ratio 0.5 as suggested by Chen et al. (202)).
Results & Discussions.
with fixed temperature
As shown in Figure 5 method effectively mitigate robust overfitting for all datasets, with both standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training. In Table T. we measure the difference between the RA the best checkpoint (Best) and at the last
8

---

Under review as a conference paper at ICLR 2022
CIFAR-10
CIFAR-100
Tiny-ImageNet
(a]
(b)
(c)
Test Test Accuracy (%%
Test Accuracy (1%
Test Accuracy
30
80
20-
(TGRGD-ATI
RA (PSD-ATI
(GD-AT)
(RSD-ATI
(NDDATII (KD-AT)
RA (PGD-71
(DD-AD
RA
KD-AT
JD-ND
20
(D AT--An-
RA (KD-ATT-aat))
XD-ATiuuta
(KD WO--A--uun))
(KG-AT-Aut))
ID-ATTAutto
-10
15
100
125
150
75
100
125
150
40
Epochs
Epochs
Epochs
Figure 5: Our method can effectively mitigate robust overfitting for different datasets.
checkpoint (Last) clearly show the overfitting gap. Our method can reduce the overfitting gap to less than 0.3% for all datasets. One may note that self-distillation with fixed syper-parameters is in fact inferior in terms of reducing robust overfitting, while effectiveness can be significantly improved with the optimal verifies our understanding of robust overfitting.
automatically determined by our method, which further
Compared with self-distillation with fixed hyper-parameters, our method can also boost both RA and SA the best checkpoint for all datasets.
Our method can further be combined with orthogonal techniques such as Stochastic Weight Averaging (SWA) (Izmailov et al. 2018) and additional standard teachers as mentioned in previous work (Chen
al. 2021 to achieve better performance. More results and discussion can be found in Appendix C.2
Robust Acc.
(%)
Standard Acc.
(%)
Dataset
Setting
T
Best
Last
Diff.
Best
Last
Diff.
AT
47 35
41.42
5.93
82.67
84.91
-2.24
CIFAR-10
KD-AT
2
0.5
48.76
46.33
2.43
82.89
85.49
-2.60
KD-AT-Auto
1.47"
0.8"
49.05
48.80
0.25
84.26
84.47
-0.21
AT
24.79
19.75
5.04
57.33
57.42
-0.09
CIFAR-100
KD-AT
0.5
25.77
23.58
2.19
57.24
60.04
-2.80
KD-AT-Auto
1.53*
0.83
26.36
26.24
0.12
58.80
59.05
-0.25
17.20
15.40
1.80
47.72
47.62
0.10
Tiny-ImageNet
KD-AT
2
0.5
17.86
17.18
0.68
47.73
48.28
-0.55
KD-AT-Auto
1.23*
0.85
18.29
18.39
-0.10
47.46
47.56
-0.10
Table 1: Performance of our method on different datasets. determined by our method.
denotes the hyper-parameter automatically
CONCLUSION AND DISCUSSIONS
In this paper, extend the understanding of robust overfitting and show that it is the early part of an epoch-wise double descent in adversarial training. Our further analyses show that the double descent may originate from the implicit label noise introduced by the improper labeling of the adversarial examples. Based on our understanding, we propose an alternative labeling of adversarial examples by rectifying model probability, which can effectively mitigate robust overfitting without any manual hyper-parameter tuning.
By showing robust overfitting and epoch-wise double descent are fact the same phenomena, our work eases the difficulty in understanding overfitting in modern generalization theory. Existing theories on double descent can be readily applied to adversarially robust learning setting, and potentially consolidate the theoretical grounding of robust learning.
Beyond the double descent in adversarial training, our definition of implicit label noise from a distribution mismatch perspective may also exist in variety of scenarios, especially in roal-world tasks where labeling is often complicated and ambiguous. Our method to rectify such label noise should thus be applicable well. This would be interesting future work.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBLITY STATEMENT
We conduct experiments on public benchmark. We will release implementations for all methods and scripts for all experiments on GitHub, under the Apache-2.0 license.
ETHIC STATEMENT
In this paper, we explore the double descent phenomenon in adversarial robust learning from the implicit label noise perspective. We experiment three benchmark datasets that publicly available. Our analyses offer more in-depth understandings about adversarial training and can better improve the robustness of neural network models. Therefore, we believe our work is ethically on the right side of spectrum and has no potential for misuse, and cannot harm any vulnerable population.
REFERENCES
Madhu S. Advani and Andrew M. Saxe. High-dimensional dynamics of generalization error in neural networks. Neural Networks, 132:428 - 446, 2020.
Maksym Andriushchenko, Croce, Nicolas Flammarion, and M. Hein. Square attack: query- efficient black-box adversarial attack via random search. ArXiv, abs/1912.00049, 2020.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine- learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116: 15849 15854, 2019.
Mikhail Belkin, Daniel J. Hsu, and J. Xu. Two models of double descent for weak features. SIAM J. Math. Data Sci., 2:1167-1180, 2020.
Y. Carmon, Aditi Raghunathan, L. Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. ArXiv, abs/1905. 13736, 2019.
J. Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. Proceed- ings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning. 2020 IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pp. 696-705, 2020.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. International Conference on Learning Representations, volume 2021.
Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
Stéphane d' Ascoli, Maria Refinetti, G. Biroli, and F. Krzakala. Double trouble in double descent: Bias and variance(s) in the lazy regime. ArXiv, abs/2003.01054, 2020.
Jia Deng, W. Dong, R. Socher, L. Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Zeyu Deng, A. Kammoun, and Christos Thrampoulidis. A model of double descent for high- dimensional binary linear classification. ArXiv, abs/1911.08 1.05822, 2019.
Terrance Devries and Graham W. Taylor. Improved regularization convolutional neural networks with cutout. ArXiv, abs/1 00/117088045522 2017.
Chengyu Dong, Liyuan Liu, and Jingbo Shang. Data profiling for adversarial training: On the ruin of problematic data. ArXiv, abs/2102.074377, 2021a.
10

---

Under review as a conference paper at ICLR 2022
Yinpeng Dong, Ke Xu, Xiao Yang. Tianyu Pang. Zhijie Deng, Hang Su, and Jun Zhu. Exploring memorization adversarial training. ArXiv, abs/2106.01606, 2021b.
Benoît Frénay and M. Verleysen. Classification in the presence of label noise: A survey. IEEE Transactions on Neural Networks and Learning Systems, 25:845-800 869, 2014.
T. Gneiting and A. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102:359- 378, 2007.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/ 141.6647, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy A. Mann, and Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. ArXiv, abs/2010.03593, 2020.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. ArXiv, abs/1706.04599, 2017.
T. Hastie, A. Montanari, S. Rosset, and R. Tibshirani. Surprises in high-dimensional ridgeless least squares interpolation. ArXiv, abs/1903.0886/ 2019.
Trevor J. Hastie, Robert Tibshirani, and Jerome H. Friedman. The elements of statistical learning. 2001.
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ArXiv, abs/1 hbss/6603.0507,, 2016.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. ArXiv, 12261,
Geoffrey E. Hinton, Oriol Vinyals, and Dean. Distilling the knowledge in a neural network. ArXiv, 15,,55 2015.
Andrew Ilyas, Shibani Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. NeurIPS, 2019.
Pavel Izmailov, Dmitrii Podoprikhin, T. Garipov, D. Vetrov, and A. Wilson. Averaging weights leads to wider optima and better generalization. ArXiv, 0tts/8003.05400 2018.
Arthur Jacot, Berfin Simsek, F. Spadaro, Clément Hongler, and F. Gabriel. Implicit regularization of random feature models. In ICML., 2020.
Ganesh R. Kini and Christos Thrampoulidis. Analytic study of double descent in binary classification: The impact of loss. 2020 IEEE International Symposium Information Theory (ISIT), pp. 2527-2532, 2020.
A. Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. 2015.
I. Loshchilov and F. Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv: Learning. 2017.
A. Madry, Aleksandar Makelov, L. Schmidt, D. Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ArXiv, abs/ 7od.0600666683, 2018.
Song Mei and A. Montanari. The generalization error of random features regression: Precise asymptotics and double descent curve. arXiv: Statistics Theory, 2019.
Prectum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, B. Barak, and llya Sutskever. Deep double descent: Where bigger models and more data hurt. ArXiv. abs/1912.02292. 2020.
Yuval Netzer, Tiejic Wang, Adam Coates, A. Bissacco, Bo Wu, and A. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
11

---

Under review as a conference paper at ICLR 2022
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. Exploring general- ization deep learning. In NIPS, 2017.
Nicolas Papernot, P. McDaniel, Arunesh Sinha, and Michael P. Wellman. Towards the science of security and privacy machine learning. ArXiv, abs/1611.038/ 14, 2016.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timo- thy A. Mann. Fixing data augmentation to improve adversarial robustness. ArXiv, abs/2103.01946, 2021.
Leslie Rice, Eric Wong, and J. Kolter. Overfitting adversarially robust deep learning. ArXiv, abs/2002. 11569, 2020.
K. Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556. 2015.
Vasu Singla, Sahil Singla, David Jacobs, and Soheil Feizi. Low curvature activations reduce overfitting in adversarial training. ArXiv, abs/2102.00861,, 2021.
Leslie Smith. Cyclical learning rates for training neural networks. 2017 IEEE Winter Conference Applications of Computer Vision (WACV), pp. 464 472, 2017.
Hwanjun Song, Minseok Kim, Dongmin Park, and Lee. Learning from noisy labels with deep neural networks: A survey. ArXiv, abs/2007.08199 2020.
David Stutz, Matthias Hein, and B. Schiele. Relating adversarially robust generalization to flat minima. ArXiv, abs/2104.04448, 2021.
D. Su, Huan Zhang, Chen, Jinfeng Yi, P. Chen, and Yupeng Gao. Is robustness the cost of accuracy? comprehensive study the robustness of 18 deep image classification models. In ECCV, 2018.
D. Tsipras, Shibani Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. arXiv: Machine Learning, 2019.
Jonathan Uesato, Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and P. Kohli. Are labels required for improving adversarial robustness? ArXiv, hbss/1905.13722 2019.
Dongxian Wu, Shutao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generaliza- tion. arXiv: Learning, 2020.
Y. Yang, Cyrus Rashtchian, Hongyang Zhang, R. Salakhutdinov, and K. Chaudhuri. A closer look at accuracy robustness. arXiv: Learning, 2020a.
Zitong Yang, Yaodong Yu, Chong You, Steinhardt, and Y. Ma. Rethinking bias-variance trade-off for generalization of neural networks. In ICML, 2020b.
Yaodong Yu, Zitong Yang, E. Dobriban, J. Steinhardt, and Yi Ma. Understanding generalization in adversarial training via the bias-variance decomposition. ArXiv, abs/2103.09947,, 2021.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. ArXiv, abs/1605.07146, 2016.
Hongyang Zhang, Yaodong Yu, J. Jiao, E. Xing, L. Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. ArXiv, abs/1901.0857.. 2019.
Hongyi Zhang, Moustapha Cissé, Yann Dauphin, and David Lopez-Paz, mixup: Beyond empirical risk minimization. ArXiv,
2018.
12

---

Under review as a conference paper at ICLR 2022
A PROOFS
A.1 PROOFS AND REMARKS FOR IMPLICIT LABEL NOISE
Total variation distance.
For two discrete probability distributions p(y) and p(y') where y, v E Vr the total variation distance between them can be equally defined as
(l{()) - p(1 = max
Σ p(v= - -ppl =)
JEJ
jEJ
Σ
ply = - ply =]D)
Proof of Theorem 3.1
This theorem can be easily proven by the coupling inequality.
Proof of Theorem
Proof. For simplicity, we consider the adversarial perturbation generated by FGSM. Other adversarial perturbation can be viewed as a Taylor series such perturbation.
max, ply" =j(xz)
8 E
(12)
V max; ply" = =j j|xx)]
Now let j = arg max ply = jlær). Recall (* = p(y) (xrs) p(ubles) and p(ualur))
ply")x) (x). We have
min plus # vıl vg(
||p(l" x:) p(u3)((110000 (3
Σ lps" = j|z) p(uš = =](0s)
TV distance
2
2
ply" -²'xx) - -pus = -'((ss]
2
1
(x) f( + 8)z-1
2
1
(x)) f' 8)5.1
Adversarial perturbation
2
1
(T), 50
v2 (z)z))
2
1
M ||11|3
-
(x))
Local convexity
2
M
8
2 |0:
2
(z)
1
M
8
2 11012
2
IV
114
8
マ 11013
where the last inequalities leverage the fact that
|| 11:0 - 1112
With the assumption
(1-1"(4))
[*(x); +
IV ('(x,,l
(p(2))
f*(2), +
13

---

Under review as a conference paper at ICLR 2022
then have
min p(ù # y/ + 5) x s(1 - ppy j" lx)); where min means the lower bound of the minimum label noise.
Sanity check of Assumption 7 Gaussian mixture model (GMM).
Here check if Assumption 7 reasonable by studying
The conditional distribution in Gaussian mixture model can be formulated
(3,a) Σ, (11, Σ, (1))(())
dggg (x)
ply = j|æ) = (x)x)
where
1 - (x - oo (x - mi) exp
(x) / //de((2mgg
The gradient can be derived as
3 (z)
E, Vo(()
V (x)
(1) (1))
(1)
-200(Z)0o (x - pb)
+0)g,(z) E, ())((0)07 (x me)
E, (11
[ (x))
(1)))(())
10 -x-1))) -11) EioT (s pi) (x)
(1g((())
E, (2g((())
= (()))
-0 (x - (1))))))+ + Σ fr (x))0 (a - mi)
When x - M4 + 0. (z) + and (x), - 0, for +
( - 1)07 (xx 14)
Therefore
x1 (x),
A.2 PROOFS FOR MITIGATING DOUBLE DESCENT
Proof of Theorem 4.1
Proof. Let j = argmax plys = )وزs)) and thus p(ys = j=1xs) E [1/c, 1]. Let g(T) == (1,50, which is continuous function defined on [0, oo The condition j arg max; f(z;; 0,T),, ensures that g(T) E [1/c, 1], where e is the number classes. By the intermediate value theorem, there exists T such that g(T*) p(ys = j*(xs).
Let T = T", we have
(f(;;; T,T) - (ollls))llr
1
Σ
(1s3: - -ly- = j (28)
2
)
1
Σ
|s(ss; 0, - plu3 = j|x0)|
2
1
Σ
f(x8; 0, T) + Σ
plus = j(as)
2
1
p(ys
14

---

Under review as a conference paper at ICLR 2022
where the inequality holds by the triangle inequality. Meanwhile, we have
[[ æz) [|p() p(yå l2s)
[]((v) - p(u8 2s)llrv
1 - plus - -y(xo)) +
Σ plu3 - w/(08)
1 ply3 = y)(z5)
2 1 p(u3 = j (x))
Therefore, it can that for T T
[1(( p(u3 (s))l |Ip(ūs(xs) p(ut [s)]
Proof Theorem
Lemma A.1. Let 2g be an example incorrectly classified by classifier f in terms of the true label distribution pluš j|ar8), namely
arg max O,T); 35"
where j = arg max, p(už = j/æ8). Assume ply. = =;*(xs)) 2 1/2, then
(, T) S plus (((08).
Proof. We prove it by contradiction. Assume f(x;; O(Ts; plut jr(0s), we have ((as;0,7), T' > pluš = j=((z)) 2 1/2. Therefore,
|ax;; 0,T); < Σ ((Es;,,T), =1 ((s;;,,T) < 1/2, Vj $":
which means f(z;; 0 T) < f(rx;;,T) which contradicts our condition.
0, 0,T) Vj # jr This leads to j = arg max, f(zs; 0 T)
Now we prove Theorem 4.2
Proof. First let p(yalza) = p(y(z)) Ro (y). This is our assumption. But the approx here would be problem, need exactly one-hot.
Let = arg max; p(uğ = j (.8)) By Lemma A.I we have f(xs; 0, T)-- I p(yž = (z I 1.
Then there exists X* > 0, such that X ff value theorem.
(1-X*) = p(yž = j" j((xx) by the intermediate
15

---

Under review as a conference paper at ICLR 2022
Let À X", we have
2 IllA f( + (1- X)
T)
2 III f( 0 T) + (1 (1- X) 11y) p(v - 11ffl1z:00000000 0.T) p(vv [rv]
E IA 8 0, + +11 - -). = y) plus =]/z)|
Σ (x 0T - plyă = -|
Σ IA (11-
+ (11 A) 1 = y²) - pp03 = -|>0|| - Σ (f (z 0,T); - pluă = j|(rs)
E AA- A(z( 0,T), - plu3 = -|(z)|]-
Σ
1f(zs:0, plu² -|aas] [f 0, T)5- =pvō = =>"[ms)
iati"
játj"
Σ (--((z;;, f(x;; 0, T) [f(z; 0, T) ply3 [xs)
$04$
Σ [f(z6;0, 0 T) x- f(z;; 8 T)y lo(u5 = = - (ars;B,T)s
1373"
Σ [s(ss;0, 0, T) - x. s(08; 0, T)y) - [X A.(sss; 0, O), + (1 2) f(x8; 0, 7-7)
$341
EMar 0,T); - Σ (x,; 0,T); A1
=0 0.
B MORE EMPIRICAL ANALYSES
B. EPOCH-WISE DOUBLE DESCENT IS UBIQUITOUS IN ADVERSARIAL TRAINING
In this section, we conduct extensive experiments with different optimizers, sample sizes, model architectures, and learning rate schedulers to verify the connection between robust overfitting and epoch-wise double descent. The default experiment settings are listed in Appendix in detail.
Optimizer. Similar to the setting employed in Nakkiran et al. 2020). we conduct the adversarial training using both the Adam optimizer and SGD. As already shown in Figure [ double descent can be observed for both optimizers, although Adam may be inferior compared to SGD.
Sample size. We randomly sample desired number of examples without replacement from the
original training set in CIFAR-10. As shown in Figure 6 for both optimizers, increasing sample size will shrink the area under the double descent curve, but will not significantly distort its shape. Similar observation is also made for double descent curve under standard training (Nakkiran et al. 2020
Adam
SGD
90
(Robust) Test Error (%6)
(%6)
as 80
Sample Size (x10²)
30-
75-
75
70
65
(Robust)
65
10
Epochs
Epochs
Figure 6: Varying sample size will shrink the area under the epoch-wise double descent curve, but will not significantly distort its shape.
Model capacity. We modulate the capacity of the deep model by varying the widening factor of the Wide ResNet. To extend the lower limit of the capacity, we allow the widening factor be less
16

---

Under review as a conference paper at ICLR 2022
than 1, in which case the number of channels in each residual block is scaled similarly but rounded. The number of channels the first convolutional layer will be reduced accordingly to ensure the width monotonically increasing through the forward propagation. To accelerate the training with an extremely large model, we randomly sample a training set of size 5000 and employ the Adam optimizer, since the sample size will not significantly distort the shape of the double descent as shown above. Figure (7a shows that the double descent will gradually become more complete as the model capacity increases and the model translates from under-parameterized wver-parameteriee regime (Nakkiran et al. 2020
Model capacity
Model architecture
100
WWN-28-118
WRN 28
Robust Test Error (%)
WRN-28-1
Robust Test Error (%6
85-
pre activation ResNet 18 VGG-11 11
95
WRN-28-5
WIN--88-2
80-
70-
65
73
55-
70
30
Epochs
Epochs
(a) Similar to Figure in the introduction, but in- clude more widening factors to show the gradual transition of the model from paramttarameterizze to paramearaeterii regime. The training curves are smoothed by a window of 5.
(b) Different model architectures will affect the double descent curve. In particular, VGG-11 11 will have the second descent delayed due to its inferior performance compared residual architectures.
Figure 7: Effect of model on the epoch-wise double descent curve
Model architecture. We also experiment on model architectures other than Wide ResNet, including pre-activation R8 (He et al. 2016) and VGG-11 (Simonyan & Zisserman 2015). We select these configurations to ensure approximately comparable model capacities As shown in Figure 7 different model architectures may produce slightly different double descent curves. The second descent of VGG- 11 in particular will be delayed due to its inferior performance compared to residual architectures.
Learning rate scheduler. A specific learning rate scheduler may shape the robust overfitting differently as suggested by Rice et al. (2020). We consider the following learning rate schedulers in our experiments.
Piecewise decay: The initial learning rate rate is set as 0.1 and is decayed by a factor of 10 at the 100th and 500th epochs within a total of 1000 epochs.
initially proposed by /mmth (2017) and has been popular in adversarial training. We set the maximum learning rate to be 0.2, and the learning rate will linearly increase from 0 to 0.2 for the initial 400 epochs and decrease 0 for the later 600 epochs.
Cyelic: This scheduler
Cosine: This scheduler was initially proposed by Loshchilov & Hutter (2017). The learning rate starts at 0.1 and gradually decrease following a cosine function for a total of 1000 epochs.
Experiments on various learning rate schedulers show the second descent can be widely observed except the piecewise decay, where the appearance of second descent might be delayed due to extremely small learning rate in the late stage of training. This further demonstrates the connection between robust overfitting and epoch-wise double descent.
B.2 DEPENDENCE OF DOUBLE DESCENT IN ADVERSARIAL TRAINING
In this section, we show that the double descent in adversarial training is conditioned on the data quality. As models are trained adversarial examples, data properties in adversarial training are
WWRN-28-5, pre-activation ResNet- and VGG-11 have 9.13 10 11.17 10 and 9.233 respectively.
and9.23 x 10 parameters,
17

---

Under review as a conference paper at ICLR 2022
Consland learning rate
Robust Test Error (%6
Piecevise
Cosine
10a
10a
10
10
Epochs
Figure 8: The effect of the learning rate scheduler on the epoch-wise double descent curve in adversarial training. Modulating the model capacity can produce training curves with diverse behaviors. Different model architectures may produce slightly different double descent curves. The training curve is smoothed by moving average with a window of 5.
concerned with the quality of the clean examples, and the adversary employed generate the perturbation, which further depends on the perturbation radius and the number of attack iterations We show that the double descent adversarial training has strong dependence the data quality and the perturbation radius, but almost no dependence on the number of attack iterations.
Perturbation radius. Overfitting has been shown dominate in adversarial training, but rarely appear in standard training (Rice et al. 2020). This suggests the overfitting, or more generally double descent, is conditioned on the perturbation radius adversarial training, since standard training and standard accuracy can be viewed as adversarial training and robust accuracy with zero perturbation radius, respectively. By modulating perturbation radius, we show that such correlation is gradual.
Perturbation Radius (/255)
Quality Rank (x10²)
100
95
(Robust) Test Error (%)
Robust Test Error (%))
90-
90-
80-
70-
85
60
80
78
70
10a
Epochs
Epochs
Figure 9: (Left) Dependence of double descent on the perturbation radius. E = 0/255 indicates the standard training where no double descent occurs. (Right) Dependence of double descent on the data quality. The curves are smoothed by a window of epochs to reduce overlapping.
As shown in Figure the double descent emerges and exacerbates the perturbation radius increases, indicating a strong correlation between perturbation radius and double descent in adversarial training. In particular, when the perturbation radius around 4/255, second minimum can be observed which is equivalently good or even better than the first minimum. This again validates the strong connection between robust overfitting and double descent, and suggests longer training can still be helpful for adversarial training.
Data quality. Previous works have shown that some datasets such as ImageNet (Deng et al. 2009) may produce significantly stronger robust overfitting (Rice et al. 2020). has also been observed that the low-quality data in the same dataset causes the robust overfitting (Dong et al. 202 This
Thhe best step size in terms of the model performance can often be determined from the combination of perturbation radius and attack iterations (Madry al. 2018)
18

---

Under review as a conference paper at ICLR 2022
implies that the double descent in adversarial training hinges the quality of the data. We measure the data quality using the predictive probability of a classifier (see Appendix B.2 details), and sample training sets with different levels of data quality controlled by threshold of the quality-based rank. Figure [ shows that as the quality of the training set degrades, the double descent gradually emerges and exacerbates, indicating strong correlation between the data quality and the double descent in adversarial training. One may again note that when the quality of the training data is relatively high, a second minimum can be observed which is equivalently good as the first minimum.
The number of attack iterations. We have shown that the double descent in adversarial training strongly depends on the perturbation radius. In this section we conduct experiments to show whether it also depends on the strength of the adversary.
In Figure I0 we fix the perturbation radius as 4/255 where the double descent is relatively complete and vary the number of attack iterations of the PGD attack employed in the inner maximization. One may find that as long as the model capacity is reasonably large, the number of attack iterations will not significantly affect the double descent curve, both for epoch-wise one and model-wise one. From the analysis implicit label noise, this is easy to understand as more attack iterations will not reduce the probability corresponding to the true label much more-it is widely observed more iterations in PGD attack only marginally increase the attack successful rate. Consequently, the distribution mismatch between the true label distribution and the assigned label distribution that induces the implicit label noise will not expand significantly.
Epoch-wise Double Descent
Model-wise Double Descent
Robust Test Error (%6
Robust Test Error (%)
74.
72-
85
iterationn
80-
68
25-
66
-10
64-
62
13f
101
10
10*
10t
101
Epochs
Widening Factor
Figure 10: Dependence of double descent on the number of attack iterations. As more iterations are employed in the inner maximization, neither epoch-wise nor model-wise double descent changes significantly except for an extremely small model (WRN-28-1/8), For the model-wise double descent, only the test error at the best checkpoint is shown to avoid overlapped curves since the last checkpoint achieves similar error as the best checkpoint in this training setting.
B.3 GAUSSIAN NOISE
One may expect adversarial training and adversarial augmentation (see Section 3.5) produce double descent simply by perturbing the inputs, thus increasing the variance. To verify the implicit label noise induced by a mismatch between the true label distribution and the assigned label distribution essential to produce double descent, we select one type of common corruption, Gaussian noise, to perturb the image inputs. We first briefly show that Gaussian noise does not cause a distribution mismatch.
We follow the notation introduced in Section 3.1 Under a I-st order approximation of [r), the mismatch between the true label distribution and label distribution can be derived as (see proof of Theorem 3.2 in Appendix A.
where 8 E Id now refers to a Gaussian perturbation.
[|p(z (2) p(ūs x5) 1
[l (2s) - p(y|=) = Σ (ef*(z), 8
8" fz
2
(13)
where 5 /N(0,,0000000 Ia). Since the image inputs span a -ow-dimension subspace X' and the dimension-
ality d is large (3072 for 10), is highly likely that 81''' which means V,, (x); - 0 and
8T (2)) = 0 for all j. One can also empirically verify that a Gaussian perturbation is almost always orthogonal to the difference between any two images, while an adversarial perturbation is not.
19

---

Under review as a conference paper at ICLR 2022
We experiment dataset perturbed by Gaussian noise and verify intuition. Similar to adversarial augmentation, we apply Gaussian noise to the training set only once, and then conduct standard training the perturbed training set. Aligned with the setting of adversarial augmentation (See Appendix F.3 we employ Adam to train a WRN-28-5 on randomly selected 5000 examples for 1000 epochs. As shown in Figure [1 Gaussian noise with perturbation radius as high as 30/255, which will reduce the accuracy of a classifier to the same level as an adversarial attack will, does not produce significant double descent. We note that similar observation has been made on common curruption benchmark CIFAR-100 - (Hendrycks & Dietterich, 2019) in previous work (Yang et al. 2020b).
80/255
Test Error (%)
V
10"
107
10a
10
Epochs
Figure 11: Even with extremely high Gaussian noise corrupting the training set, significant double descent can be observed. This shows the input perturbation not essential to produce double descent.
C MORE EXPERIMENT RESULTS
C.I ADVERSARIAL TRAINING METHIODS, NEURAL ARCHITECTURES AND EVALUATION METRICS
In this section conduct extensive experiments with different adversarial training methods, neural architectures and robustness evaluation metrics to verify the effectiveness of our method.
Robust Ace, (%)
Standard Ace, (%)
Method
Setting
T
X
Best
Last
Diff.
Best
Last
Diff.
48.50
45.33
2.97
82.79
82.68
0.11
TRADES
KD-AT
2
0.5
48.74
47.32
1.22
82.30
83.03
-0.73
KD-AT-Auto
1.12"
0.82"
48.75
48.39
0.36
82.44
82.80
-0.36
AT
41.96
35.39
6.57
85.91
87.20
-1.29
FGSM
KD-AT
2
0.5
42.82
41.61
1.21
86.69
87.93
-1.24
KD-AT-Auto
2.18
0.78*
44.11
43.75
0.36
87.38
87,66
-0.28
Table 2: Performance of our method with different adversarial training methods.
20

---

Under review as a conference paper at ICLR 2022
Robust Acc. (%)
Standard Acc.
(%)
Architecture
Setting
Best
Last
Diff.
Best
Last
Diff.
AT
42.21
39.12
3.09
73.95
80.45
-6.50
VGG-19
KD-AT
0.5
43.59
42.69
0.90
74.30
77.80
-3.50
KD-AT-Auto
1.28
0.79*
44.27
44.24
76.41
76.79
-0.38
AT
49.85
42.89
6.96
84.82
85.87
-1.05
WRN-28-5
KD-AT
2
0.5
51.08
48
2.68
85.36
86.88
-1.52
KD-AT-Auto
1.6
0.82
51.47
51.10
0.37
86,05
86.24
-0.19
AT
52.29
46.04
6.25
86.57
86.75
-0.18
WRN-34-10
KD-AT
2
0.5
53.11
50.97
2.14
86.41
88.06
-1.65
KD-AT-Auto
1.6
0.83*
54.17
53.71
0.46
87.69
88.01
-0.32
Table 3: Performance of our method with different neural architectures.
Robust Acc.
(%)
Attacks
Setting
À
Best
Last
Diff.
AT
50.64
43.00
7.64
PGD-1000
KD-AT
0.5
51.79
48.43
3.36
KD-AT-Auto
1.47
0.8*
52.05
51.71
0.34
AT
53.47
48.90
4.57
Square Attack
KD-AT
0.5
54.39
52.92
1.47
KD-AT-Auto
L.28
0.79*
55.23
55.17
0.06
AT
55.76
51.63
4.13
RayS
KD-AT
0.5
56.59
55.50
1.09
KD-AT-Auto
1.6"
0.82"
57.74
$7.54
0.20
Table 4: Performance of our method under different adversarial attacks. PGD-1000 refers to PGD attack with 1000 attack iterations, with step size fixed as 2/255 as recommended by Croce & Hein (2020).
C.2 COMBINED WITH ADDITIONAL TECHNIQUES
Here, we show that combined with the additional techniques proposed (Chen et 2021) our method can achieve better performance.
We note that our proposed method is essentially the baseline knowledge distillation for adversarial training with robustly trained self-teacher, equipped with algorithm that automatically finds its optimal hyperparameters (i.e. the temperature T and the interpolation ratio A). Stochastic Weight Averaging (SWA) and additional standard teachers employed in (Chen et al. 2021) orthogonal contributions. KD-AT-Auto can certainly be combined with SWA and KD-Std to achieve better performance. As shown in Table 5 on CIFAR-10, KD-AT KD-Std + SWA (Chen et al., 2021) can already reduce the overfitting gap (difference between the best and last robust accuracy) to almost O, while KD-AT-Auto KD-Std + SWA maintains overfitting gap close O. Interestingly, on the SVHN dataset (Netzer et al. 2011). where KD-AT + KD-Std + SWA still produces a high overfitting KD-AT-Auto KD-Std SWA can further push
gap (also see Appendix A1.3 3.3 in (Chen al. 2021 this gap to close to 0.
Here, the interpolation ratio of the standard teacher fixed as 0.2 and the SWA starts the first learning rate decay for all experiments. We employ PGD-AT (Madry et al. 2018) as the base adversarial training method and conduct experiments with a pre-activation ResNet-18. The robust accuracy is evaluated with AutoAttack. Other experiment details are in line with Appendix F.1
Furthermore, we note that (Chen et al. 2021 shows SWA and KD-Std are essential components to mitigate robust overfitting on top of KD-AT. while we show that KD-AT itself can mitigate robust overfitting by proper parameter tuning. We are thus able to separate these components and allow a more flexible selection of hyperparameters in diverse training scenarios without fear of overfitting, In particular, although (Chen et al. 202I) suggests SWA starting at the first learning rate decay (exactly when the overfitting starts) mitigates robust overfitting, the effectiveness of SWA mitigating
21

---

Under review as a conference paper at ICLR 2022
Robust Acc. (%)
Standard Acc.
(%)
Dataset
Setting
T
Best
Last
Diff.
Best
Last
Diff.
AT
47.35
41.42
5.93
82.67
84.91
-2.24
CIFAR-10
KD-AT KD-Std SWA KD-AT-Auto + KD-Std + SWA
2
49.98
49.89
0.09
85.06
85.52
-0,46
1.47*
0.8*
50.03
50.05
-0.02
84.69
84.91
-0.22
AT
47.83
39.77
8.06
90.18
91.11
-0.93
SVHN
KD-AT + KD-Std + SWA
2
0.5
47.88
46.46
1.42
91.59
91.76
-0.17
KD-AT-Auto + KD-Sid SWA
1.53*
0.83
50.58
50.09
0.49
90.54
90.76
-0.22
Table 5: Performance of our method combined with SWA and additional standard teacher on different datasets.
overfitting may strongly depend on its hhper-paramenn parameter selection including Su. i.e., the starting epoch and r. i.e., the decay rate") which is also mentioned in recent work (Rebuffi al. 2021 We also did some additional experiments CIFAR-10 10 following the SWA setting in (Rebuffi et al., 2021 to demonstrate the wide applicability hyperparameters of SWA, KD-AT + KD-Ste SWA cannot consistently mitigate robust overfitting, while KD-AT-Aul Auto + KD-Std + SWA can maintain an overfitting gap close to 0 and achieve better robustness as well.
our method. As shown by Table 6 when changing the
Robust Acc.
(%)
Standard Acc.
(%)
Setting
Best
Last
Diff.
Best
Last
Diff.
KD-AT + KD-Std + SWA
80
0.999
49.00
48.04
0.96
84.04
86.11
-2.07
KD-AT-Auto + KD-Stt Std SWA
80
0.999
49.35
49.25
0.1
85.38
85.91
-0.37
KD-AT + KD-Std + SWA
0
0.999
49.01
48.01
1.0
83.78
86.20
-2.12
KD-AT-Auto + KD-Std SWA
0
0.999
49.32
49.25
0.07
84.78
85.48
-0.7
Table Performance of our method combined with SWA with different pyper-parameters
D STUDY ON SYNTHETIC DATASET WITH KNOWN TRUE LABEL DISTRIBUTION
Synthetic Dataset. Since the true label distribution is typically unknown for adversarial examples in real-worlh datasets, simulate the mechanism of implicit label noise in adversarial training from a feature learning perspective. Specifically, we adapt mixup (Zhang al.) (018) for data augmentation CIFAR-10, For every example in the training set, we randomly select another example x' in a different class and linearly interpolate them by ratio namely == pa (1 p)z', which essentially perturbs with features from other classes. Therefore, the true label distribution arguably y p- (2) + (1-p) 1(y/). Unlike mixup, we intentionally set the assigned label as !j ~ 1 (y), thus deliberately create a mismatch between the true label distribution and the assigned label distribution. We refer this strategy as mixup augmentation and only perform it once before the training. this way, the true label distribution of every example in the synthetic dataset is fixed.
horse automobile
Figure 12: Example mixup augmentation.
Concentration of optimal temperature and interpolation ratio of individual examples. In Section 4.. we have shown that in terms of individual examples. the rectified model probability can provably reduce the distribution mismatch between the assigned label distribution and true label distribution of the adversarial example. However, since the true label distribution is unknown in realistic scenarios, it is not possible to directly follow Theorems 4. and 4.2 and calculate the optimal for each example in the training set. The best we can do to employ
set of
*SWA can be implemented using exponential moving average 0' of the model parameters 0 with decay 7, namely 8' 0' + r) at each training step (Rebuffi al. 2021).
22

---

Under review as a conference paper at ICLR 2022
validation set and determine universal of tlype--parameter based the NLL loss, which expects all training examples to share similar optimal temperatures and interpolation ratios. Here, based on the synthetic dataset where true label distribution is known, empirically verify this assumption reasonable.
3500
5000
3000
2500
Counts
2000
3000
1500
2000
1000
1000
0.5
1.0
1.5
2.5
0.5
0.7
0.9
1.0
T
Figure 13: The histograms of optimal temperature (left) and interpolation ratio (right) of individual examples.
In Figure 13 left, solve the optimal temperature for each correctly classified training example based on Theorem [4.1] with the interpolation ratio fixed as 1.0. One can find that the individual optimal temperatures mostly concentrate between 0.5 and 1.5. In Figure 13 right, solve the optimal interpolation ratio for each incorrectly classified training example based on Theorem 4.2 with the temperature fixed as 1.0. One can find that the individual optimal interpolation ratio mostly concentrate between 0.5 and 0.7.
E TOOLKIT
E. ESTIMATION OF THE DATA QUALITY
We use the predicative probabilities of classifiers trained on CIFAR-10 to score its training data. Similar strategy is employed in previous works to select high-qualin unlabeled data to improve adversarial robustness (Uesato al., 2019 Carmon et al.) 2019 Gowal et al.) (020). Slightly adversarially trained instead of regularly trained models to measure the quality of in-distribution data, since under standard training almost all training examples will be overfitted and gain overwhelmingly high confidence. Specifically, we adversarially train a pre-activation ResNet-18 with PGD and select the model at the
deviating from these works focusing out-of-distribution data,
the best checkpoint in terms of the robustness. The quality of example is estimated model probability corresponding to the true label without adversarial perturbation and random data augmentation (flipping and clipping). We repeat this process 10 times with random initialization to obtain a relatively accurate estimation.
E.2 DETERMINE THE OPTIMAL HYPER-PARAMETER
We employ a model overlitted on the training set to generate approximate traditional adversarial label of the adversarial example in the validation set. Such overfitted model is typically the model the final checkpoint when conducting regular adversarial training for sufficient epochs. Mathematically, our final method to determine the optimal temperature and interpolation ratio in rectified model probability can be described as
T. A arg min (Zs.B1))D.. (() f(æs; 0, TT) (1 A) f(xs;; . T). 96). T.A
(14)
where fxx;; 8", T) denotes the predictive probability a surrogate model scaled by temperature on T...
23

---

Under review as a conference paper at ICLR 2022
F EXPERIMENTAL DETAILS
F.1 SETTINGS FOR MAIN EXPERIMENT RESULTS
Dataset. We include experiment results on CIFAR-IO, CIFAR-100, Tiny-ImageNet and SVHN.
Training setting. We employ SGD as the optimizer. The batch size is fixed to 128. The momentum and weight decay are set to 0.9 and 0.0005 respectively. Other settings are listed as follows.
we conduct the adversarial training for 160 epochs, with the learning rate starting at 0.1 and reduced by factor of 10 at the 80 and 120 epochs.
Tiny-ImageNet: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0.1 and reduced by factor of 10 at the 40 and 60 epochs.
SVHN: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0.01 (as suggested by (Chen al.
) and reduced by factor of 10 the 40 and 60 epochs.
Adversary setting. We conduct adversarial training with for norm-bounded perturbations. We employ adversarial training methods including PGD-AT, TRADES and FGSM. We set the perturbation radius to be 8/ 8/255. For PGD-AT and TRADES, the step size is 2/ 2/255 and the number of attack iterations is 10.
Robustness cvaluation. We consider the robustness against lor norm-bounded adversarial attack with perturbation radius 8/255. We employ AutoAttack for reliable evaluation. We also include the evaluation results again PGD-1000, Square Attack and RayS.
Neural architectures. We include experiments results on pre-activation ResNet-18, WRN-28-5, WRN-34-10 and VGG-19.
Hardware.
We conduct experiments NVIDIA Quadro RTX A6000.
F.2
SETTINGS FOR ANALYZING DOUBLE DESCENT IN ADVERSARIAL TRAINING
Dataset. We conduct experiments the CIFAR-10 dataset, without additional data.
Training setting. We conduct the adversarial training for 1000 epochs unless otherwise noted. By default we use the Adam optimizer with the learning rate fixed 0.0001, since it requires minimal hyper-parameter tuning. For SGD the learning rate starts at 0.1, and will not be changed unless otherwise noted. The batch size will be fixed to 128, and the momentum will be necessary. No regularization such as weight decay is used. These settings are mostly aligned with the empirical analyse of double descent under standard training (Nakkiran et al. 2020).
0.9 wherever
To reduce the computation load demanded by exponential training epochs In individual cases, we reduce the size of the training set by randomly sampled a subset of size 5000 from the original training set without replacement. This will linearly shift the double descent curve but will not significant distort its shape as shown Appendix B. We adopt this setting for extensive experiments such as the analyses of the dependence of double descent on perturbation radius, data quality and the number of attack iterations. Note that in the experiment associated with data quality, randomly sampled the training subset from those examples with quality lower than threshold. The sampled subset is restricted to class- balanced.
Sample size.
Adversary setting. We conduct adversarial training with foo norm- boundeeunde perturbations. We employ standard PGD training with the perturbation radius set to 8/255 unless otherwise noted. The number of attack iterations is fixed as 10, and the perturbation step size is fixed 2/255.
Robustness evaluation. We consider the robustness against la norm-bounded adversarial attack with perturbation radius 8 8/255. We use PGD attack with 10 attack iterations and step size set to 2/255.
Neural architecture. By default experiment on Wide ResNet (Zagoruyko & Komodakis 2016) with depth 28 and widening factor 5 (WRN-28-5) to speed up training.
Hardware. We conduct experiments NVIDIA Quadro RTX A6000.
24

---

Under review as a conference paper at ICLR 2022
F.3 SETTINGS FOR ADVERSARIAL AUGMENTATION
We generate adversarial examples with PGD attack on the model obtained at the best checkpoint through pratical adversarial training (see Appendix [.] for details). The number of attack iterations is fixed as 10 and the step size fixed as 2/255. The adversarial examples of all training examples along with their labels are then grouped into new training set, where we conduct standard training for 1000 epochs. Other settings are same as those listed in Appendix F.2 except no adversary is employed.
25


Reviews:
[{"review_id": "rpzFqlcm62J", "reviewer": "Reviewer_JBGf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written and the proposed technique to mitigate the overfitting behavior for adversarial training is clearly motivated in sections 3 and 4. However, I have a few concerns regarding the experimental studies and comparative methods:-\n\n1. Writing: The experimental setup and in general the comparative model is not very well written. \n\n\n2. Comparative methods: \nChen et al. (2021) [1] proposed to apply a self-training smoothed loss function using both robust-/standard-trained self-teachers (in Eq. 1 of [1]). That required two hyperparameters i.e., $\\lambda_1$ and $\\lambda_2$. Next, they apply stochastic weight averaging (SWA). \n\n--- However, the authors compared with \"PGD training combined with self-distillation (KD-AT) with fixed temperature $T = 2$ and interpolation ratio $\\lambda$ = 0.5 as suggested by Chen et al. (2021).\" To the best of my understanding, this is a variant of their proposed technique and not the technique proposed in [1]. Can you please comment on this?\n\n\n3. Comparative results in Table 1:\nChen et al. (2021) already demonstrated to reduce the best vs last robust accuracy to almost close to 0\\% (using Auto-attack and CW attack in Table 4 of Chen et al. [1]). However, these results are significantly better than the reported results for comparative models in Table 1.\n\nA few minor comments:-\n\n1. (Section 3.3) \"It has been widely observed that adversarial examples transfer between different classifiers even with distinct architectures \"\n\n-- I don't think that this comment is true for adversarially trained robust models. It may be better to remove this line.\n\n2. Theorem 4.1 and 4.2: Please mention that the proofs are provided in the appendix.\n\n3. Figure 5(c): why didn't you use the same number of training epochs for TinyImagenet and CIFAR-10/CIFAR-100 datasets?\n\n4. Appendix should have been submitted along with the main paper, after reference.\n\n[1] Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations, volume 1, 2021", "summary_of_the_paper": "This paper demonstrates that \"robust overfitting\" during adversarial training is an early part of an epoch-wise double descent phenomenon for relatively large models. The authors also demonstrate that the overfitting behavior is observed as we select larger perturbations during training.\n\nThe authors explained this \"double descent\" phenomenon using \"implicit label noise\" and finally demonstrated that by temperature scaling and knowledge distillation, we can significantly mitigate the overfitting behavior for adversarial training.\n\n================= Post Rebuttal =================\nThe authors have answered most of my concerns. Hence, I have increased my score to 6.\nPlease find additional comments to improve the organization of the paper and experimental results in the following.", "summary_of_the_review": "Pros:- This paper demonstrates that \"robust overfitting\" during adversarial training is an early part of an epoch-wise double descent phenomenon for relatively large models. Proposed a novel technique motivated by this analysis.\n\nCons:- Comparative results are not convincing enough. \n\nOverall, I liked the paper. I am giving an initial score of 5 due to my concerns about comparative models in experiments.  If the authors can clarify my concerns, I shall increase my score towards acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "oX7iJH_vF9q", "reviewer": "Reviewer_YNwA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrength:\n1. as far as I am concerned, the idea of implicit label noise is interesting and can potentially encourage further investigations that connect the fields of adversarial robustness and learning with noisy data.\n2. the author proposed a method using temperature scaling and interpolation to reduce the implicit label noise, hence, mitigating the robust overfitting.\n3. Both theoretical investigations and numerical experiments are provided, showing the proposed method can help mitigate robust overfitting on realistic datasets.\n\n\nWeakness/Concerns:\n\n1. From my viewpoint, claims like \"label noise is often essential to explain double descent in standard learning for modern neural architectures\" are overstated and misleading. \n\n   In fact, the corresponding citations in the paper (Nakkiran et al., 2020; Yang et al., 2020b) do NOT explain double descent as a result of label noise. Instead, Nakkiran et al., 2020 explain double descent in terms of model misspecification, and Yang et al., 2020b explain double descent in the context of the bias-variance trade-off together with the unimodal-behavior of variance. \n\n   Another claim of such kind is at the beginning of section 3.5, where the authors state, \"the effect of label noise on double descent has been rigorously studied based upon both analytical settings ...\" After briefly checking the citations, none of them recognize label noise as the cause of double descent (correct me if I'm wrong; I could make a mistake here since I only quickly go through those papers.).\n\n   While adding label noise makes the double descent phenomenon more evident, the key concept is still the interplay between the complexity of the model and data. The author should rephrase relevant statements to make them more rigorous.\n\n2. The way of presenting definition 3.2 could potentially cause confusion, and I was confused when I read this part first time. The terminology true label and assigned label reminded me $\\arg\\max_jP(Y_\\delta^*=j|x_\\delta)$ and $\\arg\\max_jP(Y^*=j|x)$, which, according to section 3.2, should be the same, thus, making me wonder why there is a need to consider $P(\\tilde{y}_\\delta \\ne y^*_\\delta|x_\\delta)$. On the other hand, if I understand correctly, the phrase \"its assigned label is different from its true label\" in definition 3.2 refers to a \"potential difference\" that resulted from the mismatch between the true label and assigned label distributions.\n\n\n\n\n\n", "summary_of_the_paper": "This paper proposes identifying the robust overfitting as the early part of epoch-wise double-descent, which is caused by implicit label noise derived from the mismatch between the true label and assigned label distributions on the adversarial examples. The authors further provide a method that combines temperature scaling and interpolation to mitigate robust overfitting. Both theoretical and experiments are included to show the effectiveness of the proposed method on realistic datasets.\n", "summary_of_the_review": "This paper studied the cause of robust overfitting and proposed a method to mitigate it. Both theoretical investigation and experiments on realistic datasets are included for demonstrating the effectiveness of the method. The paper is overall well written. However, the presentation of key concepts needs to be modified, and overstated claims should be rephrased. The key idea of implicit label noise is novel and insightful and could inspire works that connect the fields of adversarial robustness and learning with noisy data. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "_L2ADe-5VQn", "reviewer": "Reviewer_mzDh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strength\n\n* The connection between robust overfitting and model-wise double descent was already observed in the deep double descent paper [1]. But the discussion in the original paper was brief and not intended to make a complete point. In this paper, the authors further connect the epoch-wise double descent to robust overfitting, which is relatively novel.\n\n* The experiments conducted in this paper are extensive. The experiments show that the epoch-wise double descent prevails across different hyperparameters including learning rates, optimizers, architectures and, so on.  \n\n* The implicit label noise hypothesis is interesting. The experiments on the perturbation radius, gaussian noise, and data quality help understand the theoretical results. The method motivated to mitigate implicit label noise is sound. \n\n### Weakness\n\n* On the theoretical results: the authors attribute the double descent to the implicit label noise - a distribution shift of the ground truth labels between the adversarial and clean samples. Given that humans cannot distinguish adversarial examples, it is questionable how substantial such a shift can impact. Comparing Figure 1 against [1], the test loss trend of adversarial training with $l_\\infty$ norm attack of perturbation $8/255$ is similar to the trend of ERM training with 5-10% label noise, which is quite surprising. Does the implicit noise label really have the same impact with such a level of label noise or is there other fundamental differences that makes the adversarial training more challenging? It would be useful to somehow quantify the implicit label noise and understand how severe it has caused.\n\n* On the experimental results: apart from the vanilla adversarial training, the authors mainly compare their proposed method against the baseline proposed in [2]. However, it seems that the authors didn't compare with a complete version of the method as the stochastic weight averaging seems not to be used in their experiment. Since this is not explicitly mentioned in the paper, please correct me if I miss anything. \n\n* Minor: $J$ is not defined in eq. (3). \n\n[1] Nakkiran, Preetum, et al. \"Deep Double Descent: Where Bigger Models and More Data Hurt.\" International Conference on Learning Representations. 2019.\n\n[2] Chen, Tianlong, et al. \"Robust overfitting may be mitigated by properly learned smoothening.\" International Conference on Learning Representations. 2020.", "summary_of_the_paper": "This paper studies the double descent phenomenon in the adversarial training, with an aim of explaining the wide observation of robust overfitting. The authors made the connection by showing that with a large enough model, the robust overfitting can be seen as the early stage of an epoch-wise double descent. This observation is confirmed with extensive experiments and analyzed by a proposed metric called the implicit label noise. A method is proposed to mitigate such noise and the consequent overfitting.", "summary_of_the_review": "This paper offers some novel understanding about robust overfitting, but the findings are not surprising as similar observations have been made in the previous literature on the model-wise double descent. I'm also not fully convinced by the equivalent impact of label noise and implicit label noise on the overfitting. The proposed method seems also not significant and the comparison is not very clearly stated in the paper. For these reasons, I'm currently on the negative side.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PNEjHgNmXM", "reviewer": "Reviewer_nwPt", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths :  \n1. This work provides a novel and interesting analysis for explaining the robust overfitting problem of adversarial training. In this work, they make a connection between the double descent in learning with noisy labels and the robust overfitting in adversarial training by introducing “implicit noisy labels”. I believe the novel analysis will provide a new perspective for the community to explain some other phenomenons.\n2. This paper is well-written and easy-to-follow. The authors give a clear analysis from the perspective of implicit noisy labels and further propose a simple method based on the analysis.\n3. The evaluation of the proposed method is conducted with Autoattack, which makes this work more convincing.\n\nWeaknesses :\nAlthough the analysis sounds pretty interesting, the support for the explanation is somewhat weak. In particular, the authors claim that double descent in adversarial training is caused by implicit noisy labels and empirically verify it by showing that training with static adversarial examples would also encounter the double descent phenomenon. In my opinion, the causal relationship between double descent and implicit noisy labels is not clearly supported.", "summary_of_the_paper": "In this work, the authors aim to show that double descent in adversarial training might be caused by implicit label noise, that is, the distribution mismatch between the true label distribution and the assigned label distribution of the adversarial examples. They empirically support this claim by showing that training with static adversarial examples would also encounter the double descent phenomenon. To further solve this problem, they propose to apply temperature scaling and interpolation to create a soft label for each adversarial sample in adversarial training. Experiments on CIFAR10/100 and Tiny-imagenet are conducted to validate the efficacy of the proposed method.", "summary_of_the_review": "Overall, the analysis in this work is novel and interesting, although there exists a weakness in supporting the claim. In this concern, I recommend a score marginally above the acceptance threshold", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper suggests that robust overfitting could be viewed as the early-part of a double descent phenomenon for adversarial training. The authors identify implicit label noise, i.e. the label distribution mismatch between the true example and the generated adversarial example as a possible explanation for this phenomenon in adversarial training. This claim is empirically supported by experiments using static adversarial examples. The authors propose a method using temperature scaling and interpolation to mitigate the effects caused by implicit label noise for robust overfitting. This method is evaluated on CIFAR 10/100 and tiny-Imagenet. Concerns have been raised in the reviews about sufficient justification for the claim that implicit label noise leads to adversarial overfitting. The rebuttal answers this question to some extent. Concerns have also be raised about the writing and whether sufficient details of the experimental setup are present in the main paper. While I acknowledge the difficulty of fitting all details within page limits, I would think that these details are crucial given that primary support for the claims made are from empirical observations.
Decision: Reject