Double Descent in Adversarial Training: An Implicit Label Noise Perspective

1 INTRODUCTION . In adversarial training , a typical phenomenon is that after a certain training epoch , the robust test error will start to increase constantly with further training , despite the robust training error continues to decrease ( Rice et al. , 2020 ) . This phenomenon , known as robust overfitting , is believed to be separated from double descent ( Belkin et al. , 2019 ) in the literature ( Rice et al. , 2020 ) . Here , we find that robust overfitting shall be viewed as the early part of an epoch-wise double descent ( Nakkiran et al. , 2020 ) . As shown in Figure 1 , for relatively large models , the robust test error increases only transiently and will eventually decrease again after a considerable number of training epochs . Moreover , we observe consistent and similar patterns across various training settings ( e.g. , different sample sizes , optimizers , learning rate schedulers , and neural architectures ) , which further verifies our intuition on the connection between the robust overfitting and the double descent ( See Appendix B.1 ) . Inspired by this observation , we further analyze the epoch-wise double descent to better understand the robust overfitting . In standard training , the double descent is shown to be related to the label flipping noise ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . However , adversarial perturbations are believed to not change underlying labels , which is further con- firmed empirically from our manual inspection . Therefore , the popular label flipping noise explanation is not applicable here . On the other hand , it is reasonable to believe that adversarial perturbation distorts the label distribution since the examples do become more ambiguous after perturbation . Thus , we focus on the implicit label noise which measures the mismatch between the assigned label distribution and ( unknown ) true label distribution . Extensive analyses of such implicit label noise reveal the intriguing dependency of robust overfitting on perturbation size ( Dong et al. , 2021b ) and data quality ( Dong et al. , 2021a ) . Guided by our analyses , we design a theoretically-grounded method to mitigate the robust overfitting . The key idea is to resort to an alternative labeling of the adversarial examples . We show that the predictive label distribution of a probabilistic classifier adversarially trained as usual , but after being properly scaled and interpolated , can be utilized as a better labeling of the adversarial examples and provably reduce the implicit label noise . This echoes the recent empirical practice of incorporating knowledge distillation ( Hinton et al. , 2015 ) into adversarial training ( Chen et al. , 2021 ) . While previous works intuitively select fixed scaling and interpolation parameters for knowledge distillation , we show that it is possible to fully unleash the potential of knowledge distillation by automatically determining the set of parameters that maximally reduces the implicit label noise with a strategy similar to confidence calibration ( Guo et al. , 2017 ) . Such strategy can further mitigate robust overfitting to a minimal amount without additional human tuning effort . Extensive experiments on different datasets , training methods , neural architectures and robustness evaluation metrics verify the effectiveness of our method . In summary , our findings and contributions are as follows . • We show that robust overfitting shall be viewed as the early part of an epoch-wise double descent , extending the common belief in adversarial training . • We show that double descent in adversarial training may originate from the implicit label noise introduced by improper labeling of adversarial examples in adversarial training practice . • We show an alternative labeling of the adversarial examples can be established to provably reduce the implicit label noise and mitigate the robust overfitting . The remainder of this paper is organized as follows . In Section 2 , we briefly review the existing works that explore robust overfitting and double descent . In Section 3 , we theoretically explore the origin of double descent in adversarial training from an implicit label noise perspective . In Section 4 , we propose to mitigate the double descent in adversarial training by alternative labeling based on our understanding . Section 5 demonstrates the effectiveness of our method on realistic datasets . Conclusions and further implications are discussed in Section 6 . 2 RELATED WORK . Robust overfitting and double descent in adversarial training . Double descent refers to the phenomenon that overfitting by increasing model complexity will eventually improve test set performance ( Neyshabur et al. , 2017 ; Belkin et al. , 2019 ) . This appears to conflict with the robust overfitting phenomenon in adversarial training , where increasing model complexity by training longer will impair test set performance constantly after a certain point during training . It is thus believed in the literature that double descent and robust overfitting are separate phenomena ( Rice et al. , 2020 ) . Towards a more complete understanding of robust overfitting , in this work , we conduct adversarial training for exponentially more epochs than the typical practice . We find that robust overfitting shall be viewed as the early part of an epoch-wise double descent . And increasing the model architecture size , another way to increase the model complexity , can modulate the epoch-wise double descent curve such that either the overfitting curve is shown , or the entire double descent curve is revealed within the same number of training epochs as shown in Figure 1 . Therefore , robust overfitting will not go beyond modern generalization theory as an exception and should be adequately explained by the origin of double descent such as label noise . A recent work also considers a different notion of double descent that is defined with respect to the perturbation size ( Yu et al. , 2021 ) . Such double descent might be more related to the robustnessaccuracy trade-off problem ( Papernot et al. , 2016 ; Su et al. , 2018 ; Tsipras et al. , 2019 ; Zhang et al. , 2019 ) , rather than the classic picture of double descent based on model complexity . Understand double descent in adversarial training . In standard training , double descent is often attributed to increased variance , with label noise being a common source . Definitions of label noise vary in literature . Theoretically-grounded analyses of double descent focus on additive label noise ( Advani & Saxe , 2020 ; Mei & Montanari , 2019 ; Hastie et al. , 2019 ; Belkin et al. , 2020 ; d ’ Ascoli et al. , 2020 ) , but only applicable to regression problems . Theoretical results on double descent are scarce on classification problems , with a few works introducing noise by randomly masking the feature vector ( Deng et al. , 2019 ; Kini & Thrampoulidis , 2020 ) . Analyses of double descent on classification problems are more common in empirical studies , where a typical way to induce double descent is to inject label flipping noise , namely the labels of a random fraction of training examples are flipped to other labels ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . However , such a definition of label noise can not properly fit the scenario in adversarial training , where labels are not likely to be flipped due to small adversarial perturbation . To understand the double descent in adversarial training within the existing picture of double descent , we introduce implicit label noise which is incurred by the mismatch between the assigned label distribution and true label distribution of the adversarial example . Such label noise can be interpreted as an instance-dependent , class dependent label noise in reference to the systematic studies on the taxonomy of label noise ( Frénay & Verleysen , 2014 ; Song et al. , 2020 ) , thus can be the origin of double descent in adversarial training . Mitigate robust overfitting . Robust overfitting hinders the practical deployment of adversarial training methods as the final performance is often sub-optimal . Various regularization methods including classic approaches such as ` 1 and ` 2 regularization and modern approaches such as cutout ( Devries & Taylor , 2017 ) and mixup ( Zhang et al. , 2018 ) have been attempted to tackle robust overfitting , whereas they are shown to perform no better than simply early stopping the training on a validation set ( Rice et al. , 2020 ) . However , early stopping raises additional concern as the best checkpoint of the robust test accuracy and that of the standard accuracy often do not coincide ( Chen et al. , 2020 ) , thus inevitably sacrificing the performance on either criterion . Various regularization methods specifically designed for adversarial training are thus being proposed to outperform early stopping , including regularization the flatness of the weight loss landscape ( Wu et al. , 2020 ; Stutz et al. , 2021 ) , introducing low-curvature activation functions ( Singla et al. , 2021 ) and adopting stochastic weight averaging ( Izmailov et al. , 2018 ) and knowledge distillation ( Hinton et al. , 2015 ) ( Chen et al. , 2021 ) . These methods are likely to suppress the implicit label noise , with the self-distillation framework ( i.e . the teacher shares the same architecture as the student model ) introduced by ( Chen et al. , 2021 ) as a particular example since introducing teacher ’ s outputs as supervision is almost equivalent to the alternative labeling inspired by our understanding of the origin of label noise in adversarial training . 3 DOUBLE DESCENT FROM AN IMPLICIT LABEL NOISE PERSPECTIVE . In this section , we present a novel perspective to understand the double descent in adversarial training . All proofs in the remainder of this paper are provided in the appendix . 3.1 PRELIMINARIES . Let X ⊂ Rd define the input space equipped with a norm ‖ · ‖ : X → R+ and Y = { 1 , . . . , c } define the label space . Considering a classification problem with a training set D = { ( x , y ) } , where x ∈ X is an input and y ∈ Y is its assigned label . We denote p ( Y = j|x ) as the distribution of the assigned labels of x given by a group of annotators . Typically y = argmaxj p ( Y = j|x ) . When there is no ambiguity , we will denote p ( Y = j|x ) as p ( y|x ) with a slight abuse of notation . Let y∗ ∈ Y be the true label of x and p ( y∗|x ) be its distribution , which can be given by a group of experts . Note y∗ = argmaxj p ( Y ∗ = j|x ) . Assumption 3.1 . We assume there is no label noise in D , namely y = y∗ and p ( y|x ) = p ( y∗|x ) . Definition 3.1 ( Data quality ) . Given an example x ∈ X , we define its data quality as maxj p ( Y ∗ = j|x ) , namely the maximum class probability of the true label distribution . Assumption 3.2 . We assume the original clean dataset D contains mostly high-quality data , namely maxj p ( Y ∗ = j|x ) ≈ 1 . We now briefly review adversarial training . Let f ( · ; θ ) : X → Y be a probabilistic classifier and f ( · ; θ ) j be its j-th class probability . Adversarial training can be viewed as an data augmentation technique that trains the classifier f on a set of adversarial examples Dδ = { ( xδ , yδ ) } , namely θ∗ = argmin θ EDδ ` ( f ( xδ ; θ ) , yδ ) . ( 1 ) Here , xδ ≡ x+ δ is the adversarial example of x produced by the inner maximization , where δ = argmax δ , ‖δ‖≤ ` ( f ( x+ δ ; θ ) , y ) , ( 2 ) and yδ ∈ Y is the assigned label of xδ . Finally , we denote y∗δ ∈ Y as the true label of xδ . Remark 3.1 . In adversarial training , it is the common practice that copies the label of a clean example to its adversarial counterpart , namely yδ ← y and p ( yδ|xδ ) ← p ( y|x ) . We denote ỹδ ≡ y and its associated distribution as the traditional adversarial label . We believe traditional adversarial label is the key problem that induces double descent in adversarial training . 3.2 TRADITIONAL ADVERSARIAL LABEL DOES NOT INTRODUCE LABEL FLIPPING NOISE In standard learning , it is often necessary to manually inject label noise to make the double descent evident for modern neural architectures ( Nakkiran et al. , 2020 ; Yang et al. , 2020b ) . We wish to check if the traditional adversarial label produces any label flipping noise , a typical type of label noise , namely if ỹδ 6= y∗δ for any adversarial example xδ . This is equivalent to checking if y∗ 6= y∗δ , since ỹδ = y by the construction of the traditional adversarial label ( Remark 3.1 ) and y = y∗ by the construction of our clean dataset ( Assumption 3.1 ) . Note that y∗ 6= y∗δ means the semantics of the adversarial example is distorted significantly such that its true label is now different from the true label of its clean counterpart . We first visually check the adversarial examples created by the inner maximization in adversarial training . We are specifically interested in those examples with the lowest data quality as they are mostly likely to become label noise after perturbation . We estimate the data quality based on model ensemble as shown in Appendix E.1 . One can find that the adversarial counterparts of those low-quality examples still match their original labels , albeit being slightly more ambiguous . We have also found some evidence from the literature . Yang et al . ( 2020a ) draw the same conclusion by studying the distance between examples in the input space . In the CIFAR-10 training set , they found that the minimum distance between any two examples from two different classes respectively is around 50/255 in terms of ` ∞ norm , while the perturbation radius typically used in adversarial training is around 8/255 , which is significantly smaller . It is thus reasonable to claim that a significant amount of label flipping noise will not be produced by the inner maximization , and subsequently it can not explain the double descent observed in adversarial training . 3.3 TRADITIONAL ADVERSARIAL LABEL INTRODUCES IMPLICIT LABEL NOISE Although traditional adversarial label is unlikely to directly produce label flipping noise , we show that it can produce label noise implicitly . An adversarial example often contains salient characteristics of classes other than the original label from human perspective ( Tsipras et al. , 2019 ; Ilyas et al. , 2019 ) . This implies that the true label distribution of an adversarial example is likely to be different from its clean counterpart , which means p ( y∗δ |xδ ) 6= p ( y∗|x ) . Note this will not conflict with y∗δ = y∗ as discussed in Section 3.2 , since the latter only implies argmaxj p ( Y ∗ δ = j|xδ ) = argmaxj p ( Y ∗ = j|x ) . Now we show the traditional adversarial label is improper in terms of the label distribution . As illustrated in Figure 2 , we have p ( ỹδ|xδ ) = p ( y|x ) by Remark 3.1 and p ( y|x ) = p ( y∗|x ) by Assumption 3.1 , which together with the effect of adversarial perturbation p ( y∗δ |xδ ) 6= p ( y∗|x ) imply that p ( ỹδ|xδ ) 6= p ( y∗δ |xδ ) . This means there is a distribution mismatch between the true label distribution and the assigned label distribution of the adversarial example . Such distribution mismatch introduces implicit label noise in the training set Dδ employed in adversarial training . Definition 3.2 ( Implicit label noise ) . Given an example x ∈ X , we define the implicit label noise as the probability that its assigned label 1 is different from its true label , namely p ( Y 6= Y ∗|x ) . Remark 3.2 . Implicit label noise is equivalent to instance-dependent and class-dependent label noise , since p ( Y 6= Y ∗|x ) = 1 − ∑ j p ( Y = j|Y ∗ = j , x ) p ( Y ∗ = j|x ) , and p ( Y 6= j|Y ∗ = j , x ) = 1 − p ( Y = j|Y ∗ = j , x ) is a typical definition of label noise . It can easily seen that if p ( Y 6= Y ∗|x ) > 0 , p ( Y 6= j|Y ∗ = j , x ) > 0 for some j . Before showing the connection between the distribution mismatch and the implicit label noise , we need to quantify the distribution mismatch first . Here we adopt the total variation distance . Definition 3.3 ( Total variation ( TV ) distance ) . Let J be a subset of the label sample space Y . For two discrete probability distributions p ( y ) and p ( y′ ) where y , y′ ∈ Y , the total variation distance between them can be defined as ‖p ( y ) − p ( y′ ) ‖TV = max J ∣∣∣∣∣∣ ∑ j∈J p ( y = j ) − ∑ j∈J p ( y′ = j ) ∣∣∣∣∣∣ ( 3 ) We are now ready to present our main result . Theorem 3.1 ( Traditional adversarial label incurs implicit label noise ) . The implicit label noise incurred by the distribution mismatch between the traditional adversarial label and the true label of the adversarial example is lower-bounded , namely p ( ỹδ 6= y∗δ |xδ ) ≥ ‖p ( ỹδ|xδ ) − p ( y∗δ |xδ ) ‖TV = ‖p ( y∗|x ) − p ( y∗δ |xδ ) ‖TV . ( 4 ) An informal proof can be sketched from a frequentist ’ s view and help the understanding of implicit label noise . Say there areM identical copies of xδ in the training setDδ , with their true labels and traditional adversarial labels distributing according to p ( y∗δ |xδ ) and p ( ỹδ|xδ ) , respectively . The number of copies that have the same true label and assigned label is M ∑ j min { p ( Ỹδ = j|xδ ) , p ( Y ∗δ = j|xδ ) } . The fraction of label noise exists in Dδ is thus 1 − ∑ j min { p ( Ỹδ = j|xδ ) , p ( Y ∗δ = j|xδ ) } = ‖p ( ỹδ|xδ ) − p ( y∗δ |xδ ) ‖TV by the definition of the total variation distance .

Summary: This paper proposes identifying the robust overfitting as the early part of epoch-wise double-descent, which is caused by implicit label noise derived from the mismatch between the true label and assigned label distributions on the adversarial examples. The authors further provide a method that combines temperature scaling and interpolation to mitigate robust overfitting. Both theoretical and experiments are included to show the effectiveness of the proposed method on realistic datasets.