Double Descent in Adversarial Training: An Implicit Label Noise Perspective

1 INTRODUCTION . In adversarial training , a typical phenomenon is that after a certain training epoch , the robust test error will start to increase constantly with further training , despite the robust training error continues to decrease ( Rice et al. , 2020 ) . This phenomenon , known as robust overfitting , is believed to be separated from double descent ( Belkin et al. , 2019 ) in the literature ( Rice et al. , 2020 ) . Here , we find that robust overfitting shall be viewed as the early part of an epoch-wise double descent ( Nakkiran et al. , 2020 ) . As shown in Figure 1 , for relatively large models , the robust test error increases only transiently and will eventually decrease again after a considerable number of training epochs . Moreover , we observe consistent and similar patterns across various training settings ( e.g. , different sample sizes , optimizers , learning rate schedulers , and neural architectures ) , which further verifies our intuition on the connection between the robust overfitting and the double descent ( See Appendix B.1 ) . Inspired by this observation , we further analyze the epoch-wise double descent to better understand the robust overfitting . In standard training , the double descent is shown to be related to the label flipping noise ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . However , adversarial perturbations are believed to not change underlying labels , which is further con- firmed empirically from our manual inspection . Therefore , the popular label flipping noise explanation is not applicable here . On the other hand , it is reasonable to believe that adversarial perturbation distorts the label distribution since the examples do become more ambiguous after perturbation . Thus , we focus on the implicit label noise which measures the mismatch between the assigned label distribution and ( unknown ) true label distribution . Extensive analyses of such implicit label noise reveal the intriguing dependency of robust overfitting on perturbation size ( Dong et al. , 2021b ) and data quality ( Dong et al. , 2021a ) . Guided by our analyses , we design a theoretically-grounded method to mitigate the robust overfitting . The key idea is to resort to an alternative labeling of the adversarial examples . We show that the predictive label distribution of a probabilistic classifier adversarially trained as usual , but after being properly scaled and interpolated , can be utilized as a better labeling of the adversarial examples and provably reduce the implicit label noise . This echoes the recent empirical practice of incorporating knowledge distillation ( Hinton et al. , 2015 ) into adversarial training ( Chen et al. , 2021 ) . While previous works intuitively select fixed scaling and interpolation parameters for knowledge distillation , we show that it is possible to fully unleash the potential of knowledge distillation by automatically determining the set of parameters that maximally reduces the implicit label noise with a strategy similar to confidence calibration ( Guo et al. , 2017 ) . Such strategy can further mitigate robust overfitting to a minimal amount without additional human tuning effort . Extensive experiments on different datasets , training methods , neural architectures and robustness evaluation metrics verify the effectiveness of our method . In summary , our findings and contributions are as follows . • We show that robust overfitting shall be viewed as the early part of an epoch-wise double descent , extending the common belief in adversarial training . • We show that double descent in adversarial training may originate from the implicit label noise introduced by improper labeling of adversarial examples in adversarial training practice . • We show an alternative labeling of the adversarial examples can be established to provably reduce the implicit label noise and mitigate the robust overfitting . The remainder of this paper is organized as follows . In Section 2 , we briefly review the existing works that explore robust overfitting and double descent . In Section 3 , we theoretically explore the origin of double descent in adversarial training from an implicit label noise perspective . In Section 4 , we propose to mitigate the double descent in adversarial training by alternative labeling based on our understanding . Section 5 demonstrates the effectiveness of our method on realistic datasets . Conclusions and further implications are discussed in Section 6 . 2 RELATED WORK . Robust overfitting and double descent in adversarial training . Double descent refers to the phenomenon that overfitting by increasing model complexity will eventually improve test set performance ( Neyshabur et al. , 2017 ; Belkin et al. , 2019 ) . This appears to conflict with the robust overfitting phenomenon in adversarial training , where increasing model complexity by training longer will impair test set performance constantly after a certain point during training . It is thus believed in the literature that double descent and robust overfitting are separate phenomena ( Rice et al. , 2020 ) . Towards a more complete understanding of robust overfitting , in this work , we conduct adversarial training for exponentially more epochs than the typical practice . We find that robust overfitting shall be viewed as the early part of an epoch-wise double descent . And increasing the model architecture size , another way to increase the model complexity , can modulate the epoch-wise double descent curve such that either the overfitting curve is shown , or the entire double descent curve is revealed within the same number of training epochs as shown in Figure 1 . Therefore , robust overfitting will not go beyond modern generalization theory as an exception and should be adequately explained by the origin of double descent such as label noise . A recent work also considers a different notion of double descent that is defined with respect to the perturbation size ( Yu et al. , 2021 ) . Such double descent might be more related to the robustnessaccuracy trade-off problem ( Papernot et al. , 2016 ; Su et al. , 2018 ; Tsipras et al. , 2019 ; Zhang et al. , 2019 ) , rather than the classic picture of double descent based on model complexity . Understand double descent in adversarial training . In standard training , double descent is often attributed to increased variance , with label noise being a common source . Definitions of label noise vary in literature . Theoretically-grounded analyses of double descent focus on additive label noise ( Advani & Saxe , 2020 ; Mei & Montanari , 2019 ; Hastie et al. , 2019 ; Belkin et al. , 2020 ; d ’ Ascoli et al. , 2020 ) , but only applicable to regression problems . Theoretical results on double descent are scarce on classification problems , with a few works introducing noise by randomly masking the feature vector ( Deng et al. , 2019 ; Kini & Thrampoulidis , 2020 ) . Analyses of double descent on classification problems are more common in empirical studies , where a typical way to induce double descent is to inject label flipping noise , namely the labels of a random fraction of training examples are flipped to other labels ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . However , such a definition of label noise can not properly fit the scenario in adversarial training , where labels are not likely to be flipped due to small adversarial perturbation . To understand the double descent in adversarial training within the existing picture of double descent , we introduce implicit label noise which is incurred by the mismatch between the assigned label distribution and true label distribution of the adversarial example . Such label noise can be interpreted as an instance-dependent , class dependent label noise in reference to the systematic studies on the taxonomy of label noise ( Frénay & Verleysen , 2014 ; Song et al. , 2020 ) , thus can be the origin of double descent in adversarial training . Mitigate robust overfitting . Robust overfitting hinders the practical deployment of adversarial training methods as the final performance is often sub-optimal . Various regularization methods including classic approaches such as ` 1 and ` 2 regularization and modern approaches such as cutout ( Devries & Taylor , 2017 ) and mixup ( Zhang et al. , 2018 ) have been attempted to tackle robust overfitting , whereas they are shown to perform no better than simply early stopping the training on a validation set ( Rice et al. , 2020 ) . However , early stopping raises additional concern as the best checkpoint of the robust test accuracy and that of the standard accuracy often do not coincide ( Chen et al. , 2020 ) , thus inevitably sacrificing the performance on either criterion . Various regularization methods specifically designed for adversarial training are thus being proposed to outperform early stopping , including regularization the flatness of the weight loss landscape ( Wu et al. , 2020 ; Stutz et al. , 2021 ) , introducing low-curvature activation functions ( Singla et al. , 2021 ) and adopting stochastic weight averaging ( Izmailov et al. , 2018 ) and knowledge distillation ( Hinton et al. , 2015 ) ( Chen et al. , 2021 ) . These methods are likely to suppress the implicit label noise , with the self-distillation framework ( i.e . the teacher shares the same architecture as the student model ) introduced by ( Chen et al. , 2021 ) as a particular example since introducing teacher ’ s outputs as supervision is almost equivalent to the alternative labeling inspired by our understanding of the origin of label noise in adversarial training . 3 DOUBLE DESCENT FROM AN IMPLICIT LABEL NOISE PERSPECTIVE . In this section , we present a novel perspective to understand the double descent in adversarial training . All proofs in the remainder of this paper are provided in the appendix . 3.1 PRELIMINARIES . Let X ⊂ Rd define the input space equipped with a norm ‖ · ‖ : X → R+ and Y = { 1 , . . . , c } define the label space . Considering a classification problem with a training set D = { ( x , y ) } , where x ∈ X is an input and y ∈ Y is its assigned label . We denote p ( Y = j|x ) as the distribution of the assigned labels of x given by a group of annotators . Typically y = argmaxj p ( Y = j|x ) . When there is no ambiguity , we will denote p ( Y = j|x ) as p ( y|x ) with a slight abuse of notation . Let y∗ ∈ Y be the true label of x and p ( y∗|x ) be its distribution , which can be given by a group of experts . Note y∗ = argmaxj p ( Y ∗ = j|x ) . Assumption 3.1 . We assume there is no label noise in D , namely y = y∗ and p ( y|x ) = p ( y∗|x ) . Definition 3.1 ( Data quality ) . Given an example x ∈ X , we define its data quality as maxj p ( Y ∗ = j|x ) , namely the maximum class probability of the true label distribution . Assumption 3.2 . We assume the original clean dataset D contains mostly high-quality data , namely maxj p ( Y ∗ = j|x ) ≈ 1 . We now briefly review adversarial training . Let f ( · ; θ ) : X → Y be a probabilistic classifier and f ( · ; θ ) j be its j-th class probability . Adversarial training can be viewed as an data augmentation technique that trains the classifier f on a set of adversarial examples Dδ = { ( xδ , yδ ) } , namely θ∗ = argmin θ EDδ ` ( f ( xδ ; θ ) , yδ ) . ( 1 ) Here , xδ ≡ x+ δ is the adversarial example of x produced by the inner maximization , where δ = argmax δ , ‖δ‖≤ ` ( f ( x+ δ ; θ ) , y ) , ( 2 ) and yδ ∈ Y is the assigned label of xδ . Finally , we denote y∗δ ∈ Y as the true label of xδ . Remark 3.1 . In adversarial training , it is the common practice that copies the label of a clean example to its adversarial counterpart , namely yδ ← y and p ( yδ|xδ ) ← p ( y|x ) . We denote ỹδ ≡ y and its associated distribution as the traditional adversarial label . We believe traditional adversarial label is the key problem that induces double descent in adversarial training . 3.2 TRADITIONAL ADVERSARIAL LABEL DOES NOT INTRODUCE LABEL FLIPPING NOISE In standard learning , it is often necessary to manually inject label noise to make the double descent evident for modern neural architectures ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . We wish to check if the traditional adversarial label produces any label flipping noise , a typical type of label noise , namely if ỹδ 6= y∗δ for any adversarial example xδ . This is equivalent to checking if y∗ 6= y∗δ , since ỹδ = y by the construction of the traditional adversarial label ( Remark 3.1 ) and y = y∗ by the construction of our clean dataset ( Assumption 3.1 ) . Note that y∗ 6= y∗δ means the semantics of the adversarial example is distorted significantly such that its true label is now different from the true label of its clean counterpart . We first visually check the adversarial examples created by the inner maximization in adversarial training . We are specifically interested in those examples with the lowest data quality as they are mostly likely to become label noise after perturbation . We estimate the data quality based on model ensemble as shown in Appendix E.1 . One can find that the adversarial counterparts of those low-quality examples still match their original labels , albeit being slightly more ambiguous . We have also found some evidence from the literature . Yang et al . ( 2020a ) draw the same conclusion by studying the distance between examples in the input space . In the CIFAR-10 training set , they found that the minimum distance between any two examples from two different classes respectively is around 50/255 in terms of ` ∞ norm , while the perturbation radius typically used in adversarial training is around 8/255 , which is significantly smaller . It is thus reasonable to claim that a significant amount of label flipping noise will not be produced by the inner maximization , and subsequently it can not explain the double descent observed in adversarial training . 3.3 TRADITIONAL ADVERSARIAL LABEL INTRODUCES IMPLICIT LABEL NOISE Although traditional adversarial label is unlikely to directly produce label flipping noise , we show that it can produce label noise implicitly . An adversarial example often contains salient characteristics of classes other than the original label from human perspective ( Tsipras et al. , 2019 ; Ilyas et al. , 2019 ) . This implies that the true label distribution of an adversarial example is likely to be different from its clean counterpart , which means p ( y∗δ |xδ ) 6= p ( y∗|x ) . Note this will not conflict with y∗δ = y∗ as discussed in Section 3.2 , since the latter only implies argmaxj p ( Y ∗ δ = j|xδ ) = argmaxj p ( Y ∗ = j|x ) . Now we show the traditional adversarial label is improper in terms of the label distribution . As illustrated in Figure 2 , we have p ( ỹδ|xδ ) = p ( y|x ) by Remark 3.1 and p ( y|x ) = p ( y∗|x ) by Assumption 3.1 , which together with the effect of adversarial perturbation p ( y∗δ |xδ ) 6= p ( y∗|x ) imply that p ( ỹδ|xδ ) 6= p ( y∗δ |xδ ) . This means there is a distribution mismatch between the true label distribution and the assigned label distribution of the adversarial example . Such distribution mismatch introduces implicit label noise in the training set Dδ employed in adversarial training . Definition 3.2 ( Implicit label noise ) . Given an example x ∈ X , we define the implicit label noise as the probability that its assigned label 1 is different from its true label , namely p ( Y 6= Y ∗|x ) . Remark 3.2 . Implicit label noise is equivalent to instance-dependent and class-dependent label noise , since p ( Y 6= Y ∗|x ) = 1 − ∑ j p ( Y = j|Y ∗ = j , x ) p ( Y ∗ = j|x ) , and p ( Y 6= j|Y ∗ = j , x ) = 1 − p ( Y = j|Y ∗ = j , x ) is a typical definition of label noise . It can easily seen that if p ( Y 6= Y ∗|x ) > 0 , p ( Y 6= j|Y ∗ = j , x ) > 0 for some j . Before showing the connection between the distribution mismatch and the implicit label noise , we need to quantify the distribution mismatch first . Here we adopt the total variation distance . Definition 3.3 ( Total variation ( TV ) distance ) . Let J be a subset of the label sample space Y . For two discrete probability distributions p ( y ) and p ( y′ ) where y , y′ ∈ Y , the total variation distance between them can be defined as ‖p ( y ) − p ( y′ ) ‖TV = max J ∣∣∣∣∣∣ ∑ j∈J p ( y = j ) − ∑ j∈J p ( y′ = j ) ∣∣∣∣∣∣ ( 3 ) We are now ready to present our main result . Theorem 3.1 ( Traditional adversarial label incurs implicit label noise ) . The implicit label noise incurred by the distribution mismatch between the traditional adversarial label and the true label of the adversarial example is lower-bounded , namely p ( ỹδ 6= y∗δ |xδ ) ≥ ‖p ( ỹδ|xδ ) − p ( y∗δ |xδ ) ‖TV = ‖p ( y∗|x ) − p ( y∗δ |xδ ) ‖TV . ( 4 ) An informal proof can be sketched from a frequentist ’ s view and help the understanding of implicit label noise . Say there areM identical copies of xδ in the training setDδ , with their true labels and traditional adversarial labels distributing according to p ( y∗δ |xδ ) and p ( ỹδ|xδ ) , respectively . The number of copies that have the same true label and assigned label is M ∑ j min { p ( Ỹδ = j|xδ ) , p ( Y ∗δ = j|xδ ) } . The fraction of label noise exists in Dδ is thus 1 − ∑ j min { p ( Ỹδ = j|xδ ) , p ( Y ∗δ = j|xδ ) } = ‖p ( ỹδ|xδ ) − p ( y∗δ |xδ ) ‖TV by the definition of the total variation distance .

Summary: This paper demonstrates that "robust overfitting" during adversarial training is an early part of an epoch-wise double descent phenomenon for relatively large models. The authors also demonstrate that the overfitting behavior is observed as we select larger perturbations during training.  The authors explained this "double descent" phenomenon using "implicit label noise" and finally demonstrated that by temperature scaling and knowledge distillation, we can significantly mitigate the overfitting behavior for adversarial training.  ================= Post Rebuttal ================= The authors have answered most of my concerns. Hence, I have increased my score to 6. Please find additional comments to improve the organization of the paper and experimental results in the following.