Original: Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
Revised: Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overﬁtting

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: We show that label noise exists in adversarial training.
Revised: We show that label noise exists in adversarial training.

Original: Such label noise is due tothe mismatch between the true label distribution of adversarial examples and thelabel inherited from clean examples – the true label distribution is distorted by theadversarial perturbation, but is neglected by the common practice that inherits labelsfrom clean examples.
Revised: Such label noise is due tothe mismatch between the true label distribution of adversarial examples and thelabel inherited from clean examples – the true label distribution is distorted by theadversarial perturbation, but is neglected by the common practice that inherits labelsfrom clean examples.

Original: Recognizing label noise sheds insights on the prevalence ofrobust overfitting in adversarial training, and explains its intriguing dependenceon perturbation radius and data quality.
Revised: Recognizing label noise sheds insights on the prevalence ofrobust overﬁtting in adversarial training, and explains its intriguing dependenceon perturbation radius and data quality.

Original: Also, our label noise perspective alignswell with our observations of the epoch-wise double descent in adversarial training.
Revised: Also, our label noise perspective alignswell with our observations of the epoch-wise double descent in adversarial training.

Original: Guided by our analyses, we proposed a method to automatically calibrate the labelto address the label noise and robust overfitting.
Revised: Guided by our analyses, we proposed a method to automatically calibrate the labelto address the label noise and robust overﬁtting.

Original: Our method achieves consistentperformance improvements across various models and datasets without introducingnew hyper-parameters or additional tuning.
Revised: Our method achieves consistentperformance improvements across various models and datasets without introducingnew hyper-parameters or additional tuning.

Original:  Introduction
Revised:  Introduction

Original: Adversarial training (Goodfellow et al., 2015; Huang et al., 2015; Kurakin et al., 2017; Madry et al.,2018) is known as one of the most effective ways (Athalye et al., 2018;
Revised: Adversarial training (Goodfellow et al., 2015; Huang et al., 2015; Kurakin et al., 2017; Madry et al.,2018) is known as one of the most effective ways (Athalye et al., 2018;

Original: Uesato et al., 2018) to enhancethe adversarial robustness of deep neural networks (Szegedy et al., 2014; Goodfellow et al., 2015).
Revised: Uesato et al., 2018) to enhancethe adversarial robustness of deep neural networks (Szegedy et al., 2014; Goodfellow et al., 2015).

Original: Itaugments training data with adversarial perturbations to prepare the model for adversarial attacks.
Revised: Itaugments training data with adversarial perturbations to prepare the model for adversarial attacks.

Original: Despite various efforts to generate more effective adversarial training examples (Ding et al., 2020;
Revised: Despite various efforts to generate more effective adversarial training examples (Ding et al., 2020;

Original: Zhang et al., 2020), the labels assigned to them attracts little attention.
Revised: Zhang et al., 2020), the labels assigned to them attracts little attention.

Original: As the common practice, theassigned labels of adversarial training examples are simply inherited from their clean counterparts.
Revised: As the common practice, theassigned labels of adversarial training examples are simply inherited from their clean counterparts.

Original: In this paper, we argue that the existing labeling practice of the adversarial training examplesintroduces label noise implicitly, since adversarial perturbation can distort the data semantics (Tsipraset al., 2019; Ilyas et al., 2019).
Revised: In this paper we argue that the existing labeling practice of the adversarial training examplesintroduces label noise implicitly, since adversarial perturbation can distort the data semantics (Tsipraset al., 2019; Ilyas et al., 2019).

Original: For example, as illustrated in Figure 1, even with a slight distortion ofthe data semantics (e.g., more ambiguous), the label distribution of the adversarially perturbed datamay not match the label distribution of the clean counterparts.
Revised: For example, as illustrated in Figure 1, even with a slight distortion ofthe data semantics (e.g., more ambiguous), the label distribution of the adversarially perturbed datamay not match the label distribution of the clean counterparts.

Original: Such distribution shift is neglectedwhen assigning labels to adversarial examples, which are directly copied from the clean counterparts.
Revised: Such distribution shift is neglectedwhen assigning labels to adversarial examples, which are directly copied from the clean counterparts.

Original: We observe that distribution mismatch caused by adversarial perturbation along with improperlabeling practice will cause label noise in adversarial training .
Revised: We observe that distribution mismatch caused by adversarial perturbation along with improperlabeling practice will cause label noise in adversarial training .

Original: It is a mysterious and prominent phenomenon that the robust test error would start to increase afterconducting adversarial training for a certain number of epochs (Rice et al., 2020), and our labelnoise perspective provides an adequate explanation for this phenomenon.
Revised: It is a mysterious and prominent phenomenon that the robust test error would start to increase afterconducting adversarial training for a certain number of epochs (Rice et al., 2020), and our labelnoise perspective provides an adequate explanation for this phenomenon.

Original: Specifically, from a classicbias-variance view of model generalization, label noise that implicitly exists in adversarial trainingcan increase the model variance (Yang et al., 2020) and thus make the overfitting much more evidentcompared to standard training.
Revised: Speciﬁcally, from a classic bias-variance view of model generalization, label noise that implicitly exists in adversarial trainingcan increase the model variance (Yang et al., 2020) and thus make the overﬁtting much more evidentcompared to standard training.

Original: Further analyses of label noise in adversarial training also explain theintriguing dependence of robust overfitting on the perturbation radius (Dong et al., 2021b) and dataquality (Dong et al., 2021a) presented in the literature.
Revised: Further analyses of label noise in adversarial training also explain theintriguing dependence of robust overﬁtting on the perturbation radius (Dong et al., 2021b) and dataquality (Dong et al., 2021a) presented in the literature.

Original: [Table]
Revised: [Table]

Original: True
Revised: True

Original: [Table]
Revised: [Table]

Original: /sB6/
Revised: /sB6/

Original: wEAyZJB</latexit>
Revised: wEAyZJB</latexit>

Original: P (˜ Y 0 | x 0 ) <latexit sha1_base64="AhqicuZyvRZ3dmjd8VBTevD9huM=">AAAB+XicbVDLSsNAFJ34rPUVdaebYJHWTUlEUHcFNy5ctGAf0oYymUzaoZNJmLkpltgv8BfcuFDErZ/gH7jzL/wEp4+Fth64cDjnXu69x4s5U2DbX8bC4tLyympmLbu+sbm1be7s1lSUSEKrJOKRbHhYUc4ErQIDThuxpDj0OK17vcuRX+9TqVgkbmAQUzfEHcECRjBoqW2a5UILGPdpejvM39/lj9tmzi7aY1jzxJmSXMmsfH9c7z+U2+Zny49IElIBhGOlmo4dg5tiCYxwOsy2EkVjTHq4Q5uaChxS5abjy4fWkVZ8K4ikLgHWWP09keJQqUHo6c4QQ1fNeiPxP6+ZQHDupkzECVBBJouChFsQWaMYLJ9JSoAPNMFEMn2rRbpYYgI6rKwOwZl9eZ7UTorOafGiotOw0QQZdIAOUQE56AyV0BUqoyoiqI8e0TN6MVLjyXg13iatC8Z0Zg/9gfH+A+rBlig=</latexit> [Figure]
Revised: P (˜ Y 0 | x 0 ) <latexit sha1_base64="AhqicuZyvRZ3dmjd8VBTevD9huM=">AAAB+XicbVDLSsNAFJ34rPUVdaebYJHWTUlEUHcFNy5ctGAf0oYymUzaoZNJmLkpltgv8BfcuFDErZ/gH7jzL/wEp4+Fth64cDjnXu69x4s5U2DbX8bC4tLyympmLbu+sbm1be7s1lSUSEKrJOKRbHhYUc4ErQIDThuxpDj0OK17vcuRX+9TqVgkbmAQUzfEHcECRjBoqW2a5UILGPdpejvM39/lj9tmzi7aY1jzxJmSXMmsfH9c7z+U2+Zny49IElIBhGOlmo4dg5tiCYxwOsy2EkVjTHq4Q5uaChxS5abjy4fWkVZ8K4ikLgHWWP09keJQqUHo6c4QQ1fNeiPxP6+ZQHDupkzECVBBJouChFsQWaMYLJ9JSoAPNMFEMn2rRbpYYgI6rKwOwZl9eZ7UTorOafGiotOw0QQZdIAOUQE56AyV0BUqoyoiqI8e0TN6MVLjyXg13iatC8Z0Zg/9gfH+A+rBlig=</latexit> overﬁtting can be viewed as an early part of the epochwise double descent.

Original:  employ PGD training (Madry et al., 2018) on CIFAR10 (Krizhevsky, 2009) with Wide ResNet (WRN) (Zagoruyko & Komodakis, 2016) and a fixed learning rate.
Revised: We employ PGD training (Madry et al., 2018) on CIFAR10 (Krizhevsky, 2009) with Wide ResNet (WRN) (Zagoruyko & Komodakis, 2016) and a ﬁxed learning rate.

Original: WRN- 28 - k refers to WRN with depth 28 and widen factor k .
Revised: WRN- 28 - k refers to WRN with depth 28 and widen factor k .

Original: Providing the label noise in adversarial training, one canfurther expect the existence of double descent basedon the modern generalization theory of deep neuralnetworks.
Revised: Providing the label noise in adversarial training, one canfurther expect the existence of double descent basedon the modern generalization theory of deep neuralnetworks.

Original: Epoch-wise double descent refers to the phenomenon that the test error will first decrease and thenincrease as predicted by the classic bias-variance tradeoff, but it will decrease again as the training continues.
Revised: Epoch-wise double descent refers to the phenomenon that the test error will ﬁrst decrease and thenincrease as predicted by the classic bias-variance tradeoff, but it will decrease again as the training continues.

Original: Such phenomenon is only reported in standard training of deep neural networks, often requiring significantlabel noise in the training set (Nakkiran et al., 2020).
Revised: Such phenomenon is only reported in standard training of deep neural networks, often requiring signiﬁcantlabel noise in the training set (Nakkiran et al., 2020).

Original: As the label noise intrinsically exists in adversarialtraining, such epoch-wise double descent phenomenonalso emerges when the training goes longer.
Revised: As the label noise intrinsically exists in adversarialtraining, such epoch-wise double descent phenomenonalso emerges when the training goes longer.

Original: Indeed,as shown in Figure 2, for a relatively large model suchas WRN-28-5, on top of the existing robust overfittingphenomenon, the robust test error will eventually decrease again after 1 , 000 epochs .
Revised: Indeed,as shown in Figure 2, for a relatively large model suchas WRN-28-5, on top of the existing robust overﬁttingphenomenon, the robust test error will eventually decrease again after 1 , 000 epochs .

Original: Following Nakkiranet al.
Revised: Following Nakkiranet al.

Original: (2020), we further experiment different modelsizes.
Revised: (2020), we further experiment different modelsizes.

Original: One can find that a medium-sized model willfollow a classic U-curve, which means only overfittingis observed; and the robust test error for a small modelwill monotonically decrease.
Revised: One can ﬁnd that a medium-sized model willfollow a classic U-curve, which means only overﬁttingis observed; and the robust test error for a small modelwill monotonically decrease.

Original: These are well aligned with the observations in standard training regime.
Revised: These are well aligned with the observations in standard training regime.

Original: This again consolidates our understanding of label noise in adversarial training.
Revised: This again consolidates our understanding of label noise in adversarial training.

Original: In light of our analyses, we design a theoretically-grounded method to mitigate the label noisein adversarial training automatically.
Revised: In light of our analyses, we design a theoretically-grounded method to mitigate the label noisein adversarial training automatically.

Original: The key idea is to resort to an alternative labeling of theadversarial examples.
Revised: The key idea is to resort to an alternative labeling of theadversarial examples.

Original: We show that the predictive label distribution of an adversarially trainedprobabilistic classifier can approximate the true label distribution with high probability.
Revised: We show that the predictive label distribution of an adversarially trainedprobabilistic classiﬁer can approximate the true label distribution with high probability.

Original: Thus it can beutilized as a better labeling of the adversarial examples and provably reduce the label noise.
Revised: Thus it can beutilized as a better labeling of the adversarial examples and provably reduce the label noise.

Original: We alsoshow that with proper temperature scaling and interpolation, such predictive label distribution canfurther reduce the label noise.
Revised: We alsoshow that with proper temperature scaling and interpolation, such predictive label distribution canfurther reduce the label noise.

Original: This echoes the recent empirical practice of incorporating knowledgedistillation (Hinton et al., 2015) into adversarial training (Chen et al., 2021).
Revised: This echoes the recent empirical practice of incorporating knowledgedistillation (Hinton et al., 2015) into adversarial training (Chen et al., 2021).

Original: While previous worksheuristically select fixed scaling and interpolation parameters for knowledge distillation, we showthat it is possible to fully unleash the potential of knowledge distillation by automatically determiningthe set of parameters that maximally reduces the label noise, with a strategy similar to confidencecalibration (Guo et al., 2017).
Revised: While previous worksheuristically select ﬁxed scaling and interpolation parameters for knowledge distillation, we showthat it is possible to fully unleash the potential of knowledge distillation by automatically determiningthe set of parameters that maximally reduces the label noise, with a strategy similar to conﬁdencecalibration (Guo et al., 2017).

Original: Such strategy can further mitigate robust overfitting to a minimalamount without additional human tuning effort.
Revised: Such strategy can further mitigate robust overﬁtting to a minimalamount without additional human tuning effort.

Original: Extensive experiments on different datasets, trainingmethods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.
Revised: Extensive experiments on different datasets, trainingmethods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.

Original: In summary, our findings and contributions are: 1) we show that the labeling of adversarial examplesin adversarial training practice introduces label noise implicitly; 2) we show that robust overfittingcan be adequately explained by such label noise, and it is the early part of an epoch-wise doubledescent; 3) 2e show an alternative labeling of the adversarial examples can be established to provablyreduce the label noise and mitigate the robust overfitting.
Revised: In summary, our ﬁndings and contributions are: 1) we show that the labeling of adversarial examplesin adversarial training practice introduces label noise implicitly; 2) we show that robust overﬁttingcan be adequately explained by such label noise, and it is the early part of an epoch-wise doubledescent; 3) 2e show an alternative labeling of the adversarial examples can be established to provablyreduce the label noise and mitigate the robust overﬁtting.

Original: Related Work
Revised: Related Work

Original: Robust overfitting and double descent in adversarial training.
Revised: Robust overﬁtting and double descent in adversarial training

Original: Double descent refers tothe phenomenon that overfitting by increasing model complexity will eventually improve test setperformance (Neyshabur et al., 2017; Belkin et al., 2019).
Revised: Double descent refers tothe phenomenon that overﬁtting by increasing model complexity will eventually improve test setperformance (Neyshabur et al., 2017; Belkin et al., 2019).

Original: This appears to conflict with the robustoverfitting phenomenon in adversarial training, where increasing model complexity by training longerwill impair test set performance constantly after a certain point during training.
Revised: This appears to conﬂict with the robustoverﬁtting phenomenon in adversarial training, where increasing model complexity by training longerwill impair test set performance constantly after a certain point during training.

Original: It is thus believed inthe literature that robust overfitting and epoch-wise double descent are separate phenomena (Riceet al., 2020).
Revised: It is thus believed inthe literature that robust overﬁtting and epoch-wise double descent are separate phenomena (Riceet al., 2020).

Original: In this work we show this is not the complete picture by conducting adversarial trainingfor exponentially more epochs than the typical practice.
Revised: In this work we show this is not the complete picture by conducting adversarial trainingfor exponentially more epochs than the typical practice.

Original: A recent work also considers a different notion of double descent that is defined with respect to theperturbation size (Yu et al., 2021).
Revised: A recent work also considers a different notion of double descent that is deﬁned with respect to theperturbation size (Yu et al., 2021).

Original: Such double descent might be more related to the robustnessaccuracy trade-off problem (Papernot et al., 2016;
Revised: Such double descent might be more related to the robustnessaccuracy trade-off problem (Papernot et al., 2016;

Original: Su et al., 2018; Tsipras et al., 2019;
Revised: Su et al., 2018; Tsipras et al., 2019;

Original: Zhang et al.
Revised: Zhang et al.

Original: ,2019), rather than the classic understanding of double descent based on model complexity.
Revised: ,2019), rather than the classic understanding of double descent based on model complexity.

Original: Mitigate robust overfitting.
Revised: Mitigate robust overﬁtting.

Original: Robust overfitting hinders the practical deployment of adversarialtraining methods as the final performance is often sub-optimal.
Revised: Robust overﬁtting hinders the practical deployment of adversarialtraining methods as the ﬁnal performance is often sub-optimal.

Original: Various regularization methodsincluding classic approaches such as ℓ 1 and ℓ 2 regularization and modern approaches such ascutout (Devries & Taylor, 2017) and mixup (Zhang et al., 2018) have been attempted to tackle robustoverfitting, whereas they are shown to perform no better than simply early stopping the training ona validation set (Rice et al., 2020).
Revised: Various regularization methodsincluding classic approaches such as ` 1 and ` 2 regularization and modern approaches such ascutout (Devries & Taylor, 2017) and mixup (Zhang et al., 2018) have been attempted to tackle robustoverﬁtting, whereas they are shown to perform no better than simply early stopping the training ona validation set (Rice et al., 2020).

Original: However, early stopping raises additional concern as the bestcheckpoint of the robust test accuracy and that of the standard accuracy often do not coincide (Chenet al., 2021), thus inevitably sacrificing the performance on either criterion.
Revised: However, early stopping raises additional concern as the bestcheckpoint of the robust test accuracy and that of the standard accuracy often do not coincide (Chenet al., 2021), thus inevitably sacriﬁcing the performance on either criterion.

Original: Various regularizationmethods specifically designed for adversarial training are thus proposed to outperform early stopping,including regularization the flatness of the weight loss landscape (Wu et al., 2020; Stutz et al., 2021),introducing low-curvature activation functions (Singla et al., 2021), data-driven augmentations thatadds high-quality additional data into the training (Rebuffi et al., 2021) and adopting stochastic weightaveraging (Izmailov et al., 2018) and knowledge distillation (Hinton et al., 2015)
Revised: Various regularizationmethods speciﬁcally designed for adversarial training are thus proposed to outperform early stopping,including regularization the ﬂatness of the weight loss landscape (Wu et al., 2020; Stutz et al., 2021),introducing low-curvature activation functions (Singla et al., 2021), data-driven augmentations thatadds high-quality additional data into the training (Rebufﬁ et al., 2021) and adopting stochastic weightaveraging (Izmailov et al., 2018) and knowledge distillation (Hinton et al., 2015)

Original: (Chen et al., 2021).
Revised: (Chen et al., 2021).

Original: These methods are likely to suppress the label noise in adversarial training, with the self-distillationframework (i.e. the teacher shares the same architecture as the student model) introduced by (Chenet al., 2021) as a particular example since introducing teacher’s outputs as supervision is almostequivalent to the alternative labeling inspired by our understanding of the origin of label noise inadversarial training.
Revised: These methods are likely to suppress the label noise in adversarial training, with the self-distillationframework (i.e. the teacher shares the same architecture as the student model) introduced by (Chenet al., 2021) as a particular example since introducing teacher’s outputs as supervision is almostequivalent to the alternative labeling inspired by our understanding of the origin of label noise inadversarial training.

Original:  Preliminaries
Revised:  Preliminaries

Original: A statistic model of label noise (Frénay & Verleysen, 2014).
Revised: A statistic model of label noise (Frénay & Verleysen, 2014).

Original: Let X ⊂ R d define the input space 111 equipped with a norm ∥ · ∥ : X → R + and Y=
Revised: Let X ⇢ d deﬁne the input spaceequipped with a norm k · k : X ! + and Y

Original:  [ K ] := { 1 , 2 , . .
Revised: = [ K ] := { 1 , 2 , . .

Original:  , K } define the label space.
Revised: . , K } deﬁne the label space.

Original: We 112 introduce four random variables to describe noisy labeling process.
Revised: Weintroduce four random variables to describe noisy labeling process.

Original: Let X ∈ X denote the input,
Revised: Let X 2 X denote the input,

Original: Y ∈ Y denote the true label of the input, ˜ Y ∈ Y denote the assigned label of an input provided by 114 an annotator, and finally E denote the occurrence of a label error by this annotator.
Revised: Y 2 Y denote the true label of the input, ˜ Y 2 Y denote the assigned label of an input provided by 114 an annotator, and ﬁnally E denote the occurrence of a label error by this annotator.

Original: E is a binaryrandom variable with value 1 indicating that the assigned label is different from the true label for agiven input, i.e., E = 

Revised: E is a binaryrandom variable with value 1 indicating that the assigned label is different from the true label for agiven input, i.e., E =

Original: ( ˜ Y ̸ = Y ) .
Revised: ( ˜ Y 6 = Y ) .

Original: We study the case where the label error depends on both the input
Revised: We study the case where the label error depends on both the input

Original: X and the true label Y .
Revised: X and the true label Y .

Original: For a classification problem, a training set consists of a set of examples thatare sampled as D = { ( x i , ˜ y i ) } i ∈ [ N ] .
Revised: For a classiﬁcation problem, a training set consists of a set of examples thatare sampled as D = { ( x i , ˜ y i ) } i 2 [ N ] .

Original: Definition 3.1 (Label noise) .
Revised: Deﬁnition 3.1 (Label noise) .

Original: We define label noise p e in a training set D as the empirical measure ofthe label error, namely p e ( D ) = 1 /N (cid:80) i ∈
Revised: We deﬁne label noise p e in a training set D as the empirical measure ofthe label error, namely p e ( D ) = 1 /N P

Original: [ N ] 1 (˜ y i
Revised: [Equation]

Original: ̸

Original:  = y i ) .
Revised: i 6 = y i ) .

Original: Assumption 3.1.
Revised: Assumption 3.1.

Original: We assume the annotation of a clean dataset involves no label error, namely
Revised: We assume the annotation of a clean dataset involves no label error, namely

Original: P ( E = 1 | Y =
Revised: P ( E = 1 | Y =

Original: y, x ) = 0 .
Revised: y, x ) = 0 .

Original: This directly implies P ( ˜ Y | x ) =
Revised: This directly implies P ( ˜ Y | x ) =

Original: P ( Y | x ) (see proof in the Appendix).
Revised: P ( Y | x ) (see proof in the Appendix).

Original: Definition 3.2 (Data quality) .
Revised: Deﬁnition 3.2 (Data quality) .

Original: Given a training set D , we define its data quality as q ( D ) =124 E ( x,y )
Revised: Given a training set D , we deﬁne its data quality as q ( D ) =

Original: ∈D P ( Y = y | x )
Revised: E ( x,y ) 2 D P ( Y = y | x )

Original: Adversarially augmented training set.
Revised: Adversarially augmented training set.

Original: Let f : X → Y be a probabilistic classifier and f ( · ) j 126 be its predictive probability at class j .
Revised: Let f : X ! Y be a probabilistic classiﬁer and f ( · ) jbe its predictive probability at class j .

Original: The adversarial example of x generated by f is obtained bysolving the maximization problem x ′ = arg max z ∈B ε ( x ) ℓ ( f ( z ) , y ) .
Revised: The adversarial example of x generated by f is obtained bysolving the maximization problem x 0 = arg max z 2 B " ( x ) ` ( f ( z ) , y ) .

Original: Here ℓ can be a typical lossfunction such as cross-entropy.
Revised: Here ` can be a typical lossfunction such as cross-entropy.

Original: And B ε ( x ) denotes the norm ball centered at x with radius ε , i.e.,
Revised: And B " ( x ) denotes the norm ball centered at x with radius " , i.e.,

Original: B ε ( x ) ={ z ∈ X : ∥
Revised: B " ( x ) =

Original:  z − x ∥ ≤ ε } .
Revised: { z 2 X : k z  x k  " } .

Original: Following previous notations, we denote Y ′ as a random variable representing the true label of x ′and ˜ Y ′ as a random variable representing the assigned label of x ′ .
Revised: Following previous notations, we denote Y 0 as a random variable representing the true label of x 0and ˜ Y 0 as a random variable representing the assigned label of x 0 .

Original: We refer D ′{+
+}
Revised: We refer D {+
+}

Original: = { ( x ′ , ˜ y ′ ) } as the 132 adversarially augmented training set.
Revised: = { ( x 0 , ˜ y 0 ) } as theadversarially augmented training set.

Original:  training.
Revised: Adversarial training.

Original: Adversarial training can be viewed as a data augmentation technique thattrains the parametric classifier f θ on the adversarially augmented training set (Tsipras et al., 2019),namely
Revised: Adversarial training can be viewed as a data augmentation technique thattrains the parametric classiﬁer f ✓ on the adversarially augmented training set (Tsipras et al., 2019),namely

Original: [Equation]
Revised: [Equation]

Original:  Label noise implicitly exists in adversarial training
Revised:  Label noise implicitly exists in adversarial training

Original: In this section, we demonstrate the implicit existence of label noise in the adversarially augmentedtraining set.
Revised: In this section, we demonstrate the implicit existence of label noise in the adversarially augmentedtraining set.

Original: We first consider a simple case where the adversarial perturbation is generated basedon an ideal classifier that predicts the true label distribution.
Revised: We ﬁrst consider a simple case where the adversarial perturbation is generated basedon an ideal classiﬁer that predicts the true label distribution.

Original: Under such a case we prove that thelabel noise in the adversarially augmented training set is lower-bounded.
Revised: Under such a case we prove that thelabel noise in the adversarially augmented training set is lower-bounded.

Original: We then show that inrealistic cases an adversarially trained classifier can approximate the true label distribution with highprobability.
Revised: We then show that inrealistic cases an adversarially trained classiﬁer can approximate the true label distribution with highprobability.

Original: Therefore, additional error terms will be required to lower bound the label noise.
Revised: Therefore, additional error terms will be required to lower bound the label noise.

Original: Allproofs for the remainder of this paper are provided in the appendix.
Revised: Allproofs for the remainder of this paper are provided in the appendix.

Original: 4.1 When adversarial perturbation is generated by the true probabilistic classifier
Revised: 4.1 When adversarial perturbation is generated by the true probabilistic classiﬁer

Original: We first consider an ideal case where the adversarial perturbation is generated by the true probabilisticclassifier f ( x ) :
Revised: We ﬁrst consider an ideal case where the adversarial perturbation is generated by the true probabilisticclassiﬁer f ( x ) :

Original: = P ( Y | x ) , namely the classifier producing the true label distribution on any input x .
Revised: = P ( Y | x ) , namely the classiﬁer producing the true label distribution on any input x .

Original: The true label distribution is distorted by adversarial perturbation.
Revised: The true label distribution is distorted by adversarial perturbation.

Original: We quantify the mismatchbetween two probability distributions using the total variation (TV) distance .
Revised: We quantify the mismatchbetween two probability distributions using the total variation (TV) distance .

Original: Definition 4.1 (TV distance) .
Revised: Deﬁnition 4.1 (TV distance) .

Original: Let A be a collection of the subsets of the label sample space Y .
Revised: Let A be a collection of the subsets of the label sample space Y .

Original: The
Revised: The

Original: TV distance between two probability distributions P ( Y ) and P ( Y ′ ) can be defined as ∥ P ( Y ) −
Revised: TV distance between two probability distributions P ( Y ) and P ( Y 0 ) can be deﬁned as k P ( Y )

Original: P ( Y ′ )∥
Revised: P ( Y 0 )

Original:  TV = sup J ∈A | P ( Y∈ J ) − P ( Y
Revised: k TV = sup J 2 A | P ( Y

Original: [Equation]
Revised: [Equation]

Original: J )  | .
Revised: J )  P ( Y 0 2 J ) | .

Original: We now show that adversarial perturbation generated by the true probabilistic classifier can inducea mismatch between the true label distributions of clean inputs and their adversarial examples.
Revised: We now show that adversarial perturbation generated by the true probabilistic classiﬁer can inducea mismatch between the true label distributions of clean inputs and their adversarial examples.

Original: For simplicity we consider adversarial perturbation based on FGSM and cross-entropy loss, namelyx ′{+
+}
Revised: For simplicity we consider adversarial perturbation based on FGSM and cross-entropy loss, namelyx {+
+}

Original: = x − ε ∥∇ f ( x )y ∥
Revised: = x  " kr f ( x )

Original: − 1 ∇ f ( x ) y .
Revised: y k  1 r f ( x ) y .

Original: The distribution mismatch induced by such adversarial perturbation 156 can be lower bounded.
Revised: The distribution mismatch induced by such adversarial perturbation 156 can be lower bounded.

Original: Lemma 4.1.
Revised: Lemma 4.1.

Original: Assume f
Revised: Assume f

Original: ( x ) y is L -locally Lipschitz around x with bounded Hessian.
Revised: ( x ) y is L -locally Lipschitz around x with bounded Hessian.

Original: Let σ m ={+=inf z 2 B " ( x )  min ( r 2 f ( z ) y ) > 
+}
Revised: Let  m {+=inf z 2 B " ( x )  min ( r 2 f ( z ) y ) > 
+}

Original: inf
Revised: and

Original: z ∈B ε ( x ) σ min ( ∇ 2 f ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f ( z ) y ) > 0 .
Revised:  M = sup z 2 B " ( x )  max ( r 2 f ( z ) y ) > 0 .

Original: Here σ min and σ maxdenote the minimum and maximum eigenvalues of the Hessian, respectively.
Revised: Here  min and  maxdenote the minimum and maximum eigenvalues of the Hessian, respectively.

Original: We then have
Revised: We then have

Original: [Equation]
Revised: [Equation]

Original: One can find that the right-hand side is positive as long as the upper bound of the Hessian norm is nottoo large, which is reasonable as previous works have shown that small hessian norm is critical toboth standard (Keskar et al., 2017) and robust generalization (Moosavi-Dezfooli et al., 2019).
Revised: One can ﬁnd that the right-hand side is positive as long as the upper bound of the Hessian norm is nottoo large, which is reasonable as previous works have shown that small hessian norm is critical toboth standard (Keskar et al., 2017) and robust generalization (Moosavi-Dezfooli et al., 2019).

Original: Assigned label distribution is unchanged.
Revised: Assigned label distribution is unchanged.

Original: Despite the fact that the true label distribution isdistorted by adversarial perturbation, we note that the assigned label distribution of adversarialexamples is still the same as their clean counterparts.
Revised: Despite the fact that the true label distribution isdistorted by adversarial perturbation, we note that the assigned label distribution of adversarialexamples is still the same as their clean counterparts.

Original: Remark 4.1.
Revised: Remark 4.1.

Original: In adversarial training, it is the common practice that directly copies the label of aclean input to its adversarial counterpart, namely ˜ y ′{+
+} = 
Revised: In adversarial training, it is the common practice that directly copies the label of aclean input to its adversarial counterpart, namely ˜ y {+
+} = ˜

Original: ˜ y and P ( ˜ Y ′{+
+}
Revised: y and P ( ˜ Y {+
+}

Original: | x ′ ) =
Revised: | x 0 ) =

Original: P ( ˜ Y | x ) .
Revised: P ( ˜ Y | x ) .

Original: Distribution mismatch indicates label noise.
Revised: Distribution mismatch indicates label noise.

Original: We show that a mismatch between the true labeldistribution and the assigned label distribution in a training set will always indicate the existence oflabel noise.
Revised: We show that a mismatch between the true labeldistribution and the assigned label distribution in a training set will always indicate the existence oflabel noise.

Original: Lemma 4.2.
Revised: Lemma 4.2.

Original: Given a training set D = { ( x i , ˜ y i ) } i ∈ [ N ] , the label noise is lower-bounded by the 172 mismatch between the true label distribution and the assigned label distribution.
Revised: Given a training set D = { ( x i , ˜ y i ) } i 2 [ N ] , the label noise is lower-bounded by the 172 mismatch between the true label distribution and the assigned label distribution.

Original: Specifically, with [Figure] augmented  255 ) can also produce prominent over-
Revised: Speciﬁcally, with versarially augmented training set (e.g. " = 16 / 255 ) can also produce prominent overﬁtting.

Original: In contrast on the  without adversarial  applied  = 0 ) observed.[Figure]
Revised: In contrast, on the original training set without adversarial perturbation applied ( " = 0 ), no overﬁtting is observed.

Original:  probability 1 − δ , we have
Revised: reduce the probability 1   , we have

Original: [Equation]
Revised: [Equation]

Original: Label noise implicitly exists in adversarial training.
Revised: Label noise implicitly exists in adversarial training.

Original: In the adversarially augmented training set
Revised: In the adversarially augmented training set

Original: D ′ , such distribution mismatch exists exactly.
Revised: D 0 , such distribution mismatch exists exactly.

Original: By Remark 4.1 we have P ( ˜ Y ′ | x ′ ) 
Revised: By Remark 4.1 we have P ( ˜ Y 0 | x 0 ) =

Original: = P ( ˜ Y | x ) and byproperty of the clean dataset (Assumption 3.1) we have P ( ˜ Y | x ) =
Revised: P ( ˜ Y | x ) and byproperty of the clean dataset (Assumption 3.1) we have P ( ˜ Y | x ) =

Original: P ( Y | x ) , which together means
Revised: P ( Y | x ) , which together means

Original: P ( ˜ Y ′ | x ′ ) =
Revised: P ( ˜ Y 0 | x 0 ) =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: However, Lemma 4.1 shows that P ( Y ′ | x ′ ) {+
+}
Revised: However, Lemma 4.1 shows that P ( Y 0 | x 0 ) {+
+}

Original: ̸

Original: = P ( Y | x ) , which implies thatP ( ˜ Y ′ | x ′ )
Revised: = P ( Y | x ) , which implies that

Original: ̸ = P ( Y ′ | x ′ ) .
Revised: P ( ˜ Y 0 | x 0 ) 6 = P ( Y 0 | x 0 ) .

Original: This indicates that label noise exists in the adversarially augmented training 179 set.
Revised: This indicates that label noise exists in the adversarially augmented trainingset.

Original: We now have the following theorem, which is our main result.
Revised: We now have the following theorem, which is our main result.

Original: Theorem 4.3.
Revised: Theorem 4.3.

Original: Assume f
Revised: Assume f

Original: ( x ) y is L -locally Lipschitz around x with Hessian bounded below.
Revised: ( x ) y is L -locally Lipschitz around x with Hessian bounded below.

Original: Instantiatethe same notations as in Lemma 4.1.
Revised: Instantiatethe same notations as in Lemma 4.1.

Original: With probability 1 − δ , we have
Revised: With probability 1   , we have

Original: [Equation]
Revised: [Equation]

Original: The above results suggest that as long as a training set is augmented by adversarial perturbation, butwith assigned labels unchanged, label noise emerges.
Revised: The above results suggest that as long as a training set is augmented by adversarial perturbation, butwith assigned labels unchanged, label noise emerges.

Original: We demonstrate this by showing that standardtraining on a fixed adversarially augmented training set can also produce overfitting.
Revised: We demonstrate this by showing that standardtraining on a ﬁxed adversarially augmented training set can also produce overﬁtting.

Original: Specifically, foreach example in a clean training set we apply adversarial perturbation generated by a adversariallytrained classifier.
Revised: Speciﬁcally, foreach example in a clean training set we apply adversarial perturbation generated by a adversariallytrained classiﬁer.

Original: We then fix such an augmented training set and conduct standard training on it.
Revised: We then ﬁx such an augmented training set and conduct standard training on it.

Original: Weexperiment on CIFAR-10 with WRN-28-5.
Revised: Weexperiment on CIFAR-10 with WRN-28-5.

Original: A training subset of size 5k is randomly sampled to speedup the training.
Revised: A training subset of size 5k is randomly sampled to speedup the training.

Original: More details about the experiment settings can be found in the appendix.
Revised: More details about the experiment settings can be found in the appendix.

Original: Figure 3shows that prominent overfitting (as well as epoch-wise double descent) can be observed when theperturbation radius is relatively large.
Revised: Figure 3shows that prominent overﬁtting (as well as epoch-wise double descent) can be observed when theperturbation radius is relatively large.

Original: On the other hand, if a training set is augmented by perturbation that will not distort the true labeldistribution, there will not be label noise.
Revised: On the other hand, if a training set is augmented by perturbation that will not distort the true labeldistribution, there will not be label noise.

Original: We demonstrate this by showing that standard trainingon a training set augmented with Gaussian noise will not induce overfitting.
Revised: We demonstrate this by showing that standard training ona training set augmented with Gaussian noise will not induce overﬁtting.

Original: As shown in Figure 4,even with a extremely large radius of Gaussian perturbation, no overfitting is observed.
Revised: As shown in Figure 12,even with a extremely large radius of Gaussian perturbation, no overﬁtting is observed.

Original: This alsodemonstrates that input perturbation not necessarily leads to overfitting.
Revised: This alsodemonstrates that input perturbation not necessarily leads to overﬁtting.

Original: Intuitive interpretation of label noise in adversarial training.
Revised: Intuitive interpretation of label noise in adversarial training.

Original: We introduce a simple exampleto help understand the emergence of label noise in adversarial training.
Revised: We introduce a simple exampleto help understand the emergence of label noise in adversarial training.

Original: Example 4.1 (Label noise due to a symmetric distribution shift) .
Revised: Example 4.1 (Label noise due to a symmetric distribution shift) .

Original: Let D =
Revised: Let D =

Original: ( x i , y i ) } i ∈ [ N ] be a 199 clean labeled training subset where all inputs x i = x are identical and have a one-hot true labeldistribution, i.e., P ( Y | x ) = 1 y .
Revised: ( x i , y i ) } i 2 [ N ] be a 199 clean labeled training subset where all inputs x i = x are identical and have a one-hot true labeldistribution, i.e., P ( Y | x ) = 1 y .

Original: We now construct an adversarially augmented training subset D ′ ={+
+}
Revised: We now construct an adversarially augmented training subset D {+
+}

Original:  { ( x ′ i , ˜ y ′ i ) } i ∈ [ N ] , where ˜y ′
Revised: = { ( x 0 i , ˜ y 0 i ) } i 2 [ N ] , where ˜

Original: ={+y 
+}
Revised: {+y 
+}

Original:  y and x ′ is generated based on adversarial perturbation that distorts the true label distribution symmetrically.
Revised: = y and x 0 is generated based on adversarial perturbation that distorts the true label distribution symmetrically.

Original: Specifically,
Revised: Speciﬁcally,

Original: [Equation]
Revised: [Equation]

Original: Then by Lemma 4.2 we have p e ( D ′ ) ≳ η .
Revised: Then by Lemma 4.2 we have p e ( D 0 ) & ⌘ .

Original: One can find that there is indeed η faction of noisy labels in D ′ .
Revised: One can ﬁnd that there is indeed ⌘ faction of noisy labels in D 0 .

Original: This is because if we sample thelabels of x ′ based on its true label distribution, we expect 1 − η faction of x ′ are labeled as y , while ηfraction of x ′ are labeled to be other classes.
Revised: This is because if we sample thelabels of x 0 based on its true label distribution, we expect 1  ⌘ faction of x 0 are labeled as y , while ⌘fraction of x 0 are labeled to be other classes.

Original: However, in D ′ , all x ′ are assigned with label y , whichmeans η fraction of x ′ are labeled incorrectly.
Revised: However, in D 0 , all x 0 are assigned with label y , whichmeans ⌘ fraction of x 0 are labeled incorrectly.

Original: In realistic datasets we can consider inputs with similarfeatures for such reasoning.
Revised: In realistic datasets we can consider inputs with similarfeatures for such reasoning.

Original: The above example also shows that label noise in adversarial training may be stronger than one’simpression.
Revised: The above example also shows that label noise in adversarial training may be stronger than one’simpression.

Original: Even a slight distortion of the true label distribution, e.g. η = 0 .
Revised: Even a slight distortion of the true label distribution, e.g. ⌘ = 0 .

Original: 1 , will be equivalent toat least 10% noisy label in the training set.
Revised: 1 , will be equivalent toat least 10% noisy label in the training set.

Original: This is because the true label distribution of every traininginput is distorted, resulting in significant noise in the population.
Revised: This is because the true label distribution of every traininginput is distorted, resulting in signiﬁcant noise in the population.

Original: Dependence of label noise in adversarial training.
Revised: Dependence of label noise in adversarial training.

Original: Theorem 4.3 shows that the label noise inadversarial training is proportional to (1) the perturbation radius (2) the data quality.
Revised: Theorem 4.3 shows that the label noise inadversarial training is proportional to (1) the perturbation radius (2) the data quality.

Original: Considering labelnoise can be an important source of variance in the generalization of deep neural networks (Nakkiranet al., 2020; Yang et al., 2020), such dependence of label noise explains the intriguing observationsin the literature that robust overfitting (or epoch-wise double descent) in adversarial training willvanish with small perturbation
Revised: Considering labelnoise can be an important source of variance in the generalization of deep neural networks (Nakkiranet al., 2020; Yang et al., 2020), such dependence of label noise explains the intriguing observationsin the literature that robust overﬁtting (or epoch-wise double descent) in adversarial training willvanish with small perturbation

Original: radii (Dong et al., 2021b) or high-quality data (Dong et al., 2021a).
Revised: radii (Dong et al., 2021b) or high-quality data (Dong et al., 2021a).

Original: We conduct more controlled experiments to verify this correlation empirically, as shown in Figure 5, 
Revised: We conduct more controlled experiments to verify this correlation empirically, as shown in Figure 5, on the data quality with a ﬁxed perturbation radius ( " = 8 / 255 ). To construct a training subset with high data quality, we ﬁrst calculate the predictive probability based on an ensemble of multiple models. We then rank all training examples based on the predictive probability and select the top-k ones. The curves are smoothed by a window of 5 epochs to reduce overlapping.

Original: [Figure]
Revised: Here we conduct PGD training on CIFAR-10 with WRN-28-5. More experiment details can be found in the Appendix.

Original: 4.2 Adversarial perturbation generated by a realistic classifier
Revised: 4.2 Adversarial perturbation generated by a realistic classiﬁer

Original: We now consider a realistic case where the adversarial perturbation is generated by a probabilisticclassifier f θ .
Revised: We now consider a realistic case where the adversarial perturbation is generated by a probabilisticclassiﬁer f ✓ .

Original: Approximation of the true label distribution.
Revised: Approximation of the true label distribution.

Original: We show that after sufficient adversarial training,the predictive label distribution of f θ can approximate the true label distribution with high probability.
Revised: We show that after sufﬁcient adversarial training,the predictive label distribution of f ✓ can approximate the true label distribution with high probability.

Original: Lemma 4.4.
Revised: Lemma 4.4.

Original: Denote S =
Revised: Denote S =

Original: { x : ( x, y ) ∈ D} as the collection of all training inputs.
Revised: { x : ( x, y ) 2 D} as the collection of all training inputs.

Original: Let ρ ≥ 1 and
Revised: Let ⇢  1 and

Original: C be an ρε -external covering of S with covering number N ρε .
Revised: C be an ⇢" -external covering of S with covering number N ⇢" .

Original: Let f θ be a probabilistic classifier 225 that minimizes the adversarial empirical risk (1).
Revised: Let f ✓ be a probabilistic classiﬁerthat minimizes the adversarial empirical risk (1).

Original: Assume f θ is L θ -locally Lipschitz continuous ina norm ball of radius ρε around x ∈ C .
Revised: Assume f ✓ is L ✓ -locally Lipschitz continuous ina norm ball of radius ⇢" around x 2 C .

Original: Let κ ≥ 1 and ˆ S be a subset of S with cardinality at least(1 [-− 
 /κ-] +1 / ( κN ρε ))
Revised: Let   1 and ˆ S be a subset of S with cardinality at least(1 [-− 
 /κ-] 1 /  +1 / (  N ⇢" ))

Original: N .
Revised: N .

Original: Let N ε ( ˆ S ) denote the neighborhood of the set ˆ S , i.e. N ε ( ˆ S ) =(cid:83) [Equation]
Revised: Let N " ( ˆ S ) denote the neighborhood of the set ˆ S , i.e. N " ( ˆ S ) =

Original: ∈ ˆ 
Revised: S x 2 ˆ S B " ( x ) .

Original: [Equation] ε [Equation] Then for any x ∈ N ε ( ˆ S ) , with probability at least 1 − δ ,
Revised: Then for any x 2 N " ( ˆ S ) , with probability at least 1   ,

Original: [Equation]
Revised: [Equation]

Original: Label noise in adversarial training with a realistic classifier.
Revised: Label noise in adversarial training with a realistic classiﬁer.

Original: Adversarial perturbation generatedby a realistic classifier f θ will distort its predictive label distribution by gradient ascent.
Revised: Adversarial perturbation generatedby a realistic classiﬁer f ✓ will distort its predictive label distribution by gradient ascent.

Original: Subsequently,the true label distribution will also be distorted with high probability since the predictive labeldistribution of a realistic classifier f θ can approximate the true label distribution.
Revised: Subsequently,the true label distribution will also be distorted with high probability since the predictive labeldistribution of a realistic classiﬁer f ✓ can approximate the true label distribution.

Original: Specifically, by thetriangle inequality we have
Revised: Speciﬁcally, by thetriangle inequality we have

Original: [Equation]
Revised: [Equation]

Original: where the last two terms are the approximation error of true label distribution on both clean andadversarial examples, which are guaranteed to be small.
Revised: where the last two terms are the approximation error of true label distribution on both clean andadversarial examples, which are guaranteed to be small.

Original: To conclude, we have the following result.
Revised: To conclude, we have the following result.

Original: Theorem 4.5.
Revised: Theorem 4.5.

Original: Instantiate the notations of Lemma 4.4.
Revised: Instantiate the notations of Lemma 4.4.

Original: For any x ∈ N ε ( ˆ S ) , with probability at least1 − 3 δ , we have
Revised: For any x 2 N " ( ˆ S ) , with probability at least 237 3  , we have

Original: [Equation]
Revised: [Equation]

Original: where ξ = 1 + √ 4 κN s K .
Revised: where ⇠ = 1 + p 4  N s K .

Original:  Mitigate Label Noise in Adversarial Training
Revised:  Mitigate Label Noise in Adversarial Training

Original: Since the label noise is incurred by the mismatch between the true label distribution and assignedlabel distribution of adversarial examples in the training set, we wish to find an alternative label(distribution) for the adversarial example to reduce such distribution mismatch.
Revised: Since the label noise is incurred by the mismatch between the true label distribution and assignedlabel distribution of adversarial examples in the training set, we wish to ﬁnd an alternative label(distribution) for the adversarial example to reduce such distribution mismatch.

Original: We’ve already shownthat the predictive label distribution of a classifier trained by conventional adversarial training, whichwe denote as model probability in the following discussion, can in fact approximate the true labeldistribution.
Revised: We’ve already shownthat the predictive label distribution of a classiﬁer trained by conventional adversarial training, whichwe denote as model probability in the following discussion, can in fact approximate the true labeldistribution.

Original: Here we show that it is possible to further improve the predictive label distribution andreduce the label noise by calibration.
Revised: Here we show that it is possible to further improve the predictive label distribution andreduce the label noise by calibration.

Original: 5.1 Rectify model probability to reduce distribution mismatch
Revised: 5.1 Rectify model probability to reduce distribution mismatch

Original: We show that it is possible to reduce the distribution mismatch by temperature scaling (Hinton et al.
Revised: We show that it is possible to reduce the distribution mismatch by temperature scaling (Hinton et al.

Original: Guo et al., 2017) enabled in the softmax function.
Revised: Guo et al., 2017) enabled in the softmax function.

Original: Theorem 5.1 (Temperature scaling can reduce the distribution mismatch) .
Revised: Theorem 5.1 (Temperature scaling can reduce the distribution mismatch) .

Original: Let f θ ( x ; T ) denote the predictive probability of a probabilistic classifier scaled by temperature T , namely f θ ( x ; T ) 
Revised: Let f ✓ ( x ; T ) denote the predictive probability of a probabilistic classiﬁer scaled by temperature T , namely f ✓ ( x ; T ) j = exp( z j /T ) / (

Original: j = exp( z j /T ) / ( (cid:80) j exp( z j /T )) , where z is the logits of the classifier from x .
Revised: P j exp( z j /T )) , where z is the logits of the classiﬁer from x .

Original: Let x ′ be an adversarial example correctly classified by a classifier f θ , i.e. arg max j f θ ( x ′ )
Revised: Let x 0 be an adversarial example correctly classiﬁed by a classiﬁer f ✓ , i.e. arg max j f ✓ ( x 0 )

Original: j [Equation] y ′ , then there exists T , such that
Revised: j = y 0 , then there exists T , such that

Original: [Equation]
Revised: [Equation]

Original: Another way to further reduce the distribution mismatch is to interpolate between the model probability and the one-hot assigned label.
Revised: Another way to further reduce the distribution mismatch is to interpolate between the model probability and the one-hot assigned label.

Original: We show that the interpolation works specifically for incorrectlyclassified examples and thus can be viewed as a complement to temperature scaling.
Revised: We show that the interpolation works speciﬁcally for incorrectlyclassiﬁed examples and thus can be viewed as a complement to temperature scaling.

Original: Theorem 5.2 (Interpolation can further reduce the distribution mismatch) .
Revised: Theorem 5.2 (Interpolation can further reduce the distribution mismatch) .

Original: Let x ′ be an adversarial example incorrectly classified by a classifier f θ , i.e. arg max j f θ ( x ′ ; T ) j ̸ = y ′ .
Revised: Let x 0 be an adversarial example incorrectly classiﬁed by a classiﬁer f ✓ , i.e. arg max j f ✓ ( x 0 ; T ) j 6 = y 0 .

Original: Assumemax j P ( Y ′ =
Revised: Assumemax j P ( Y 0 =

Original: j | x ′ ) ≥ 1 / 2 , then there exists an interpolation ratio λ , such that
Revised: j | x 0 )  1 / 2 , then there exists an interpolation ratio  , such that

Original: [Equation]
Revised: [Equation]

Original: x [Equation] where f θ ( x ′ ; T, λ ) =
Revised: x 0 ) k TV , where f ✓ ( x 0 ; T,  ) =

Original: λ · f θ ( x ′ ; T ) + (1 − λ ) · P ( ˜ Y ′ | x ′ ) .
Revised:  · f ✓ ( x 0 ; T ) + (1   ) · P ( ˜ Y 0 | x 0 ) .

Original: As a summarization, to reduce the distribution mismatch, we propose to use f θ ( x ′ ; T, λ ) as theassigned label of the adversarial example in adversarial training, which we refer as the rectified modelprobability .
Revised: As a summarization, to reduce the distribution mismatch, we propose to use f ✓ ( x 0 ; T,  ) as theassigned label of the adversarial example in adversarial training, which we refer as the rectiﬁed modelprobability .

Original: In Appendix E, we show that the optimal hyper-parameters (i.e. T and λ ) of almost all trainingexamples concentrate on the same set of values by studying on a synthetic dataset with known truelabel distribution.
Revised: In Appendix E, we show that the optimal hyper-parameters (i.e. T and  ) of almost all trainingexamples concentrate on the same set of values by studying on a synthetic dataset with known truelabel distribution.

Original: Therefore it is possible to find an universal set of hyper-parameters that reduce thedistribution mismatch for all adversarial examples.
Revised: Therefore it is possible to ﬁnd an universal set of hyper-parameters that reduce thedistribution mismatch for all adversarial examples.

Original: 5.2 Determine the optimal temperature and interpolation ratio
Revised: 5.2 Determine the optimal temperature and interpolation ratio

Original: The set of temperature and interpolation ratio in the rectified model probability that maximallyreduces the distribution mismatch is not straightforward to find as the true label distribution of theadversarial example is unknown in reality.
Revised: The set of temperature and interpolation ratio in the rectiﬁed model probability that maximallyreduces the distribution mismatch is not straightforward to ﬁnd as the true label distribution of theadversarial example is unknown in reality.

Original: Fortunately, given a sufficiently large validation dataset asa whole, it is possible to measure the overall distribution mismatch in a frequentist’s view withoutknowing the true label distribution of every single example.
Revised: Fortunately, given a sufﬁciently large validation dataset asa whole, it is possible to measure the overall distribution mismatch in a frequentist’s view withoutknowing the true label distribution of every single example.

Original: A popular metric adopted here is thenegative log-likelihood (NLL) loss, which is known as a proper scoring rule (Gneiting & Raftery,2007) and is also employed in the confidence calibration of deep networks (Guo et al., 2017).
Revised: A popular metric adopted here is thenegative log-likelihood (NLL) loss, which is known as a proper scoring rule (Gneiting & Raftery,2007) and is also employed in the conﬁdence calibration of deep networks (Guo et al., 2017).

Original: By
Revised: By

Original: Gibbs’s inequality it is easy to show that the NLL loss will only be minimized when the assignedlabel distribution matches the true label distribution (Hastie et al., 2001), namely 
Revised: Gibbs’s inequality it is easy to show that the NLL loss will only be minimized when the assignedlabel distribution matches the true label distribution (Hastie et al., 2001), namely [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, we propose to find the optimal T and λ as
Revised: Therefore, we propose to ﬁnd the optimal T and  as

Original: [Equation]
Revised: [Equation]

Original:  5.3 Rectified model probability mitigates double descent[Figure] [Equation]
Revised: [Equation] 5.3 Rectiﬁed model probability mitigates double descent

Original: [Figure] [Equation] [Figure] We now work on a realistic dataset (CIFAR-10)to demonstrate the rectified model probabilitycan effectively mitigate the robust overfitting, orequivalently the epoch-wise double descent inadversarial training.
Revised: We now work on a realistic dataset (CIFAR-10) to demonstrate the rectiﬁed model probabilitycan effectively mitigate the robust overﬁtting, orequivalently the epoch-wise double descent inadversarial training.

Original: The outer minimization ofadversarial training (Equation (1)) now becomes
Revised: The outer minimization ofadversarial training (Equation (1)) now becomes

Original: [Equation]
Revised: [Equation]

Original: where ˆ θ denotes the parameters of a classifieradversarially trained beforehand.
Revised: where ˆ ✓ denotes the parameters of a classiﬁeradversarially trained beforehand.

Original: The detailsof the experimental setting are available in the
Revised: The detailsof the experimental setting are available in the

Original: Appendix.
Revised: Appendix.

Original: As shown in Figure 6, adversarial training onrectified model probability can  mitigate the robust overfitting when the temperature T and interpolation ratio λ are optimal.
Revised: As shown in Figure 6, adversarial training onrectiﬁed model probability can best mitigate therobust overﬁtting when the temperature T andinterpolation ratio  are optimal.

Original: Such optimalhyperparameters perfectly aligns with the onesautomatically determined by Equation (9).
Revised: Such optimalhyperparameters perfectly aligns with the onesautomatically determined by Equation (9).

Original:  ExperimentsExperiment setup.
Revised:  ExperimentsExperiment setup.

Original: We conduct experiments on three datasets including CIFAR-10, CIFAR100 (Krizhevsky, 2009) and Tiny-ImageNet (Le & Yang, 2015).
Revised: We conduct experiments on three datasets including CIFAR-10, CIFAR100 (Krizhevsky, 2009) and Tiny-ImageNet (Le & Yang, 2015).

Original: We conduct PGD training on preactivation ResNet-18 (He et al., 2016) with 10 iterations and perturbation radius 8 / 255 by default.
Revised: We conduct PGD training on preactivation ResNet-18 (He et al., 2016) with 10 iterations and perturbation radius 8 / 255 by default.

Original: Weevaluate robustness against ℓ ∞ norm-bounded adversarial attack with perturbation radius 8 / 255 , andemploy AutoAttack (Croce & Hein, 2020) for reliable evaluation.
Revised: Weevaluate robustness against ` 1 norm-bounded adversarial attack with perturbation radius 8 / 255 , andemploy AutoAttack (Croce & Hein, 2020) for reliable evaluation.

Original: Appendix D.2 includes results onadditional model architectures (e.g., VGG (Simonyan & Zisserman, 2015), WRN), adversarial trainingmethods (e.g., TRADES (Zhang et al., 2019), FGSM (Goodfellow et al., 2015)), and evaluationmetrics (e.g., PGD-1000 (PGD attack with 1000 iterations), Square Attack (Andriushchenko et al.
Revised: Appendix D.1 includes results onadditional model architectures (e.g., VGG (Simonyan & Zisserman, 2015), WRN), adversarial trainingmethods (e.g., TRADES (Zhang et al., 2019), FGSM (Goodfellow et al., 2015)), and evaluationmetrics (e.g., PGD-1000 (PGD attack with 1000 iterations), Square Attack (Andriushchenko et al.

Original: ,2020), RayS (Chen & Gu, 2020)).
Revised: ,2020), RayS (Chen & Gu, 2020)).

Original: More setup details can be found in Appendix G.
Revised: More setup details can be found in Appendix G.

Original: Results & Discussions.
Revised: Results & Discussions.

Original: Our method is essentially the baseline adversarial training with a robusttrained self-teacher, equipped with an algorithm automatically deciding the optimal hyper-parameters,which we now denote as KD-AT-Auto.
Revised: Our method is essentially the baseline adversarial training with a robusttrained self-teacher, equipped with an algorithm automatically deciding the optimal hyper-parameters,which we now denote as KD-AT-Auto.

Original: We compare KD-AT-Auto with two baselines: regularadversarial training (AT), and adversarial training combined with self-distillation (KD-AT) with fixedtemperature T = 2 and interpolation ratio λ = 0 .
Revised: We compare KD-AT-Auto with two baselines: regularadversarial training (AT), and adversarial training combined with self-distillation (KD-AT) with ﬁxedtemperature T = 2 and interpolation ratio  = 0 .

Original: 5 as suggested by Chen et al.[Figure] ImageNet [Figure]
Revised: 5 as suggested by Chen et al.

Original: Dataset

Original: Setting

Original: [Table]

Original: CIFARAT -

Original: [Table][Table]
Revised: [Table]

Original: KD
Revised: As shown in Figure

Original: [Table][Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] KD
Revised: our method can effectively mitigate robust overﬁtting for all datasets, withboth standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training.

Original: [Table] AT [Table]
Revised: In Table

Original: Auto 1 .

Original: [Table][Table] [Table]
Revised: [Table]

Original: CIFARAT - - [Table] As shown in Figure 7, our method can effectively mitigate robust overfitting for all datasets, withboth standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training. In Table 1, we measure the difference between the RA at the best checkpoint (Best) and at the lastcheckpoint (Last) to clearly show the overfitting gap.
Revised: we measure the difference between the RA at the best checkpoint (Best) and at the lastcheckpoint (Last) to clearly show the overﬁtting gap.

Original: Our method can reduce the overfitting gap to lessthan 0 .
Revised: Our method can reduce the overﬁtting gap to lessthan 0 .

Original: 5% for all datasets.
Revised: 5% for all datasets.

Original: One may note that self-distillation with fixed hyper-parameters is in factinferior in terms of reducing robust overfitting, while its effectiveness can be significantly improvedwith the optimal hyper-parameters automatically determined by our method, which further verifiesour understanding of robust overfitting.
Revised: One may note that self-distillation with ﬁxed hyper-parameters is in factinferior in terms of reducing robust overﬁtting, while its effectiveness can be signiﬁcantly improvedwith the optimal hyper-parameters automatically determined by our method, which further veriﬁesour understanding of robust overﬁtting.

Original: Compared with self-distillation with fixed hyper-parameters,our method can also boost both RA and SA at the best checkpoint for all datasets.
Revised: Compared with self-distillation with ﬁxed hyper-parameters,our method can also boost both RA and SA at the best checkpoint for all datasets.

Original: Our method can further be combined with orthogonal techniques such as Stochastic Weight Averaging(SWA)
Revised: Our method can further be combined with orthogonal techniques such as Stochastic Weight Averaging(SWA)

Original: (Izmailov et al., 2018) and additional standard teachers as mentioned in previous work (Chenet al., 2021) to achieve better performance.
Revised: (Izmailov et al., 2018) and additional standard teachers as mentioned in previous work (Chenet al., 2021) to achieve better performance.

Original: More results and discussion can be found in Appendix D.3.
Revised: More results and discussion can be found in Appendix D.2.

Original:  Conclusion and Discussions
Revised:  Conclusion and Discussions

Original: In this paper, we show that label noise exists implicitly in adversarial training due to the mismatchbetween the true label distribution and the assigned label distribution of adversarial examples.
Revised: In this paper, we show that label noise exists implicitly in adversarial training due to the mismatchbetween the true label distribution and the assigned label distribution of adversarial examples.

Original: Suchlabel noise can explain the dominant overfitting phenomenon.
Revised: Suchlabel noise can explain the dominant overﬁtting phenomenon.

Original: Based on a label noise perspective, wealso extend the understanding of robust overfitting and show that it is the early part of an epoch-wisedouble descent in adversarial training.
Revised: Based on a label noise perspective, wealso extend the understanding of robust overﬁtting and show that it is the early part of an epoch-wisedouble descent in adversarial training.

Original: Finally, we propose an alternative labeling of adversarialexamples by rectifying model probability, which can effectively mitigate robust overfitting withoutany manual hyper-parameter tuning.
Revised: Finally, we propose an alternative labeling of adversarialexamples by rectifying model probability, which can effectively mitigate robust overﬁtting withoutany manual hyper-parameter tuning.

Original: The label noise implicitly exists in adversarial training may have other important effects on adversarially robust learning.
Revised: The label noise implicitly exists in adversarial training may have other important effects on adversarially robust learning.

Original: This can potentially consolidate the theoretical grounding of robust learning.
Revised: This can potentially consolidate the theoretical grounding of robust learning.

Original: For instance, since label noise induces model variance, from a model-wise view, one may need toincrease model capacity to reduce the variance.
Revised: For instance, since label noise induces model variance, from a model-wise view, one may need toincrease model capacity to reduce the variance.

Original: This may partially explain why robust generalizationrequires significantly larger model than standard generalization.
Revised: This may partially explain why robust generalizationrequires signiﬁcantly larger model than standard generalization.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .•
Revised: [Yes] See Section ?? .•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section B(c)
Revised: [Yes] See Section B(c)

Original: Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeSection B(d)
Revised: [Yes] SeeSection B(d)

Original: Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]2.
Revised: [Yes]2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?
Revised: of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: Board (IRB) approvals, if applicable?[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: applicable?

Original: [N/A]A Proofs
Revised: [N/A]

Original: A.1 Preliminaries

Original: Proof in Assumption 3.1.

Original: Here we prove that if there no label error in the clean dataset, then

Original: P ( ˜ Y | x ) =

Original: P ( Y | x ) .

Original: Proof.

Original: First, we note that

Original: [Equation]

Original: Since P ( E = 1 | Y =

Original: [-j, x ) = 
-]
Revised: [-j, x ) = 
-]

Original: we have,

Original: P

Original: [Equation]

Original: Therefore we have for all j ′ ,

Original: P ( ˜ Y =

Original: j ′ | x

Original: [Equation]

Original: =

Original: P ( Y =

Original: j ′ | x ) .

Original: Total Variation distance for discrete probability distributions.

Original: For two discrete probability distributions P ( Y ) and P ( Y ′ ) where Y, Y ′ ∈ Y , the total variation distance between them can be equally defined as

Original: [Equation]

Original: [-A.2 Proofs in Section 4.
-]
Revised: [-A.2 Proofs in Section 4.
-]

Original: Proof of Lemma 4.1.

Original: Proof.

Original: For simplicity, we consider the adversarial perturbation generated by FGSM.

Original: Other adversarialperturbation can be viewed as a Taylor series of such perturbation.

Original: [Equation]

Original: First, we bound the distribution mismatch by gradient norm.

Original: [Equation]

Original: TV distance

Original: [Equation]

Original: Bounded Hessian

Original: [Equation]

Original: Now if ∥ · ∥ = ∥ · ∥ 2 , we have

Original: [Equation]

Original: If ∥ · ∥ = ∥ · ∥ ∞ , we can utilize the fact that ∥ · ∥ ∞ ≤ ∥ · ∥ 2 ≤ √ d ∥ · ∥ ∞ , thus

Original: [Equation]

Original: Second, we bound the gradient norm by the L -local Lipschitzness assumption.

Original: Let x ∗ be a closest input that achieves the local maximum on the predicted probability at y , namely x ∗ = arg min z ∈ X,f ( z ) y

Original: =1 ∥ x − z ∥ .

Original: Because x ∗ is the local maximum and f is continuously differentiable, ∇ f ( x ∗ )

Original: y = 0 , thus ∇ f ( x )

Original: [Equation]

Original: Therefore we have

Original: [Equation]

Original: Plug this into Equation (12) or Equation (13) we then obtain the desired result.

Original: Proof of Lemma 4.2.

Original: Proof.

Original: First, we show that the expectation of the label error is lower bounded by the mismatchbetween the true label distribution and the assigned label distribution.

Original: [Equation]

Original: Second, given a sampled training set D = { ( x i , ˜ y i ) } i ∈

Original: [ N ] , the empirical measure of label error E should converge to its expectation almost surely, namely

Original: [Equation]

Original: Using standard concentration inequality such as Hoeffding’s inequality we have, with probability 1 − δ ,

Original: [Equation]

Original: This implies

Original: [Equation]

Original: Since P ( E = 1) =

Original: E x P ( E = 1 | x ) , we have, with probability 1 − δ ,

Original: [Equation]

Original: which means p e ( D ) > 0 as long as N is large.

Original: Proof of Theorem 4.3.

Original: Proof.

Original: First, by the fact that P ( ˜ Y ′ | x ′ )

Original: =

Original: P ( ˜ Y | x ) and P ( ˜ Y | x ) =

Original: P ( Y | x ) we have P ( ˜ Y ′

Original: | x ′ ) =

Original: P ( Y | x ) .

Original: Therefore, apply Lemma 4.2 to an adversarially augmented training set we have with probability 1 − δ ,

Original: [Equation]

Original: δ

Original: [Equation]

Original: Further, apply Lemma 4.1 and the definition of data quality, we have with probability 1 − δ ,

Original: [Equation]

Original: A.3 Proofs in Section 4.

Original: Proof of Lemma 4.4.

Original: Proof.

Original: Let D = ( x, y ) be the adversarially augmented training set. Let S = { x : ( x, y ) ∈ D} be 511 the collection of all training inputs.
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: First we note that the set of all training inputs can be groupedinto several subsets such that the inputs in each subset possess similar true label distribution.

Original: Moreformally, Let C

Original: =

Original: { ¯ x j } N ρε j =1 be an ρε -external covering of S with minimum cardinality, namely

Original: S ⊆ (cid:83) x ∈C { x ′ | ∥ x ′ − x ∥ ≤ ρε } , where we refer ¯ x j as the covering center, and N ρε is the covering 515 number.

Original: Let {S j } N ρε j =1 be any disjoint partition of S such that S j ⊆ { x ′ |∥ x ′ − ¯ x j ∥ ≤ ρε } .

Original: We show that

Original: S j attains a property that the true label distribution of any input x in this subset will not be too far from the sample mean of one-hot labels ¯y j =

Original: |S j | − 1 (cid:80) x ∈S j 1 y in this subset.

Original: Specifically, letp ( x ) =

Original: P ( Y | x ) .

Original: We have with probability 1 − δ ,

Original: [Equation]

Original: To prove this property we first present two lemmas.

Original: Lemma A.1 (Lipschitz constraint of the true label distribution) .

Original: Let S j be a subset constructed aboveand ¯y j

Original: =

Original: |S j | − 1 (cid:80) x ∈S j 1 y . Then for any x ∈ S j we have, [Equation] Proof.
Revised: Board (IRB) approvals, if applicable?

Original: First, since x ∈ S j , we have ∥ x − ¯ x j ∥ 1 ≤ ρε , which implies ∥ p ( x ) − p (¯ x j ) ∥ 1 ≤ Lρε by the 524 locally Lipschitz continuity of p .

Original: Then for any x, x ′ ∈ S j , we will have ∥ p ( x ) − p ( x ′ )

Original: ∥ ≤ 2 Lρε by 525 the triangle inequality.

Original: Let N

Original: S =

Original: |S j | .

Original: Therefore,

Original: [Equation]

Original: Further, the linearity of the expectation implies

Original: [Equation]

Original: Therefore ∥ p ( x ) − E [ ¯y j ] ∥ ≤ 2 Lρε .

Original: Lemma A.2 (Concentration inequality of the sample mean) .

Original: Let S be a set of x with cardinality N .

Original: Let ¯y =

Original: N − 1 (cid:80) x ∈S 1 y be the sample mean.

Original: Then for any p -norm ∥ · ∥ and any ε > 0 , we have with 530 probability

Original: [Equation]

Original: [Equation]

Original: Proof.

Original: Note that ¯y obeys a multinomial distribution, i.e. ¯y ∼ N − 1 multinomial

Original: N, E

Original: [ ¯y ]) .

Original: Thislemma is thus the classic result on the concentration properties of multinomial distribution based onℓ 1 norm Weissman et al.

Original: ; Qian et al.

Original: One can see that Lemma A.1 bounds the difference between true label distribution of individualinputs and the mean true label distribution, while Lemma A.2 bounds the difference between thesample mean and the mean true label distribution.

Original: Therefore the difference between the true labeldistribution and the sample mean is also bounded, since by the triangle inequality we have withprobability 1 − δ ,

Original: [Equation]
Revised: [N/A]

Original: We now show that given the locally Lipschitz constraint established in each disjoint partition weconstructed above, the prediction given by the empirical risk minimizer will be close to the samplemean.

Original: As an example, we focus on the negative log-likelihood loss, namely ℓ ( f θ ( x ) , y )

Original: = − 1 y ·log f θ ( x ) .

Original: Other loss functions that are subject to the proper scoring rule can be investigated in asimilar manner.

Original: First, we regroup the sum in the empirical risk based on the partition constructedabove, namely

Original: [Equation]

Original: where ˆ R ( f θ , S j ) = −|S

Original: j | − 1 (cid:80) |S

Original: j | i =1 1 y i · log f θ ( x i ) is the empirical risk in each partition.

Original: Since we 546 are only concerned with the existence of a desired minimizer of the empirical risk, we can view f θ as able to achieve any labeling of the training inputs that suffices the local Lipschitz constraint.

Original: Thus theempirical risk minimization is equivalent to the minimization of the empirical risk in each partition.

Original: The problem can thus be defined as, for each j = 1 , · · · , N

Original: [Equation]

Original: [Equation]

Original: 22) where the constraint is imposed by the locally-Lipschitz continuity of f θ .

Original: By the following lemma,we show that the minimizer of such problem is achieved only if f θ (¯ x j ) is close to the sample mean.

Original: Lemma A.3.

Original: Let ¯ y =

Original: |S j | − 1 (cid:80) x ∈S j 1 y .

Original: The minimum of the problem (22) is achieved only if

Original: [Equation]

Original: Proof.

Original: We note that since the loss function we choose is strongly convex, to minimize the empirical risk, the prediction of any input x must be as close to the one-hot labeling as possible.

Original: Therefore the problem (22) can be formulated into a vector minimization where we can employ

Original: Karush–Kuhn–Tucker (KKT) theorem to find the necessary conditions of the minimizer.

Original: Let p i :

Original: =

Original: f θ ( x i ) and ˜ ε =

Original: Lρε for simplicity.

Original: We rephrase the problem (22) asmin

Original: [Equation]

Original: [Equation]

Original: Case I.

Original: We first discuss the case when p k + ˜ ε < 1 for all k .

Original: First, we observe that for any p ,the minimum of the above problem is achieved only if p y i i

Original: = p y i + ˜ ε .

Original: Because by contradiction, ifp y i

Original: i < p y i + ˜ ε , we will have − log p y i i > − log( p y i + ˜ ε ) , and p y i + ˜ ε belongs to the feasible set,which means p y i i does not attain the minimum.
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: The above problem can then be rephrased as

Original: [Equation]

Original: where we have neglected the condition associated with p k ̸

Original: =

Original: y i

Original: i , since they do not contribute to theobjective, they can be chosen arbitrarily as long as the constraints are sufficed, and clearly theconstraints are underdetermined.

Original: Let N k

Original: = (cid:80) i 1( y i = k ) , we have (cid:80) i log( p y i + ˜ ε ) =

Original: (cid:80) k N k log( p k + ˜ ε ) .

Original: Therefore the aboveproblem is equivalent to

Original: [Equation]

Original: where ¯y ≡ [ N 1 /N, · · · , N k /N ]

Original: T is equal to the sample mean N − 1 (cid:80)

Original: [Equation]

Original: y i

Original: [Equation]

Original: To solve the strongly convex minimization problem (25) it is easy to employ KKT conditions to show that

Original: [Equation]

Original: Case II.

Original: We now discuss the case when ˆp is the minimizer of (23) and there exists k ′ such thatˆp k ′ + ˜ ε ≥ 1 .

Original: And ˆp ̸ = p , where p = ¯y (1 + K ˜ ε ) − ˜ ε is the form of the minimizer in the previouscase.

Original: Considering a non-trivial case p ∗ k ′ < 1 − ˜ ε .

Original: Otherwise the true label distribution is already close tothe one-hot labeling, which is the minimizer of the empirical risk.

Original: Therefore by (cid:80) k ̸ = k ′ p k > ˜ ε wehave the condition

Original: [Equation]

Original: Now considering the minimization objective R ( p ) =

Original: − N − 1 (cid:80)

Original: i 1 y i · log p i .

Original: For all i with y i

Original: =

Original: k ′ , wemust have p

Original: y i

Original: i = 1 , otherwise the optimal cannot be attained by contradiction. Then the minimizationproblem can be rephrased as [Equation] where the first constraint is imposed by ˆp k ′ ≥ 1 − ˜ ε .
Revised: spent on participant compensation?

Original: Employ KKT conditions similarly we can have ˆp k = ¯y k /λ − ˜ ε where λ is a constant.

Original: By checkingthe constraint we can derive λ ≥ (cid:80) k ¯y k / ( K ˜ ε ) .

Original: However, the minimization objective

Original: [Equation]

Original: requires λ to be minimized.

Original: Therefore λ = (cid:80)

Original: k ̸ = k ′ ¯y k

Original: [Equation]

Original: ˜ ε ) , which implies

Original: [Equation]

Original: Now since ˆp = arg min p R ( p ) and ˆp ̸ = p , we must have R ( ˆp ) <

Original: R ( p ) .

Original: This means−

Original: [Equation]

Original: 29) which is reduced to

Original: [Equation]

Original: [Equation]

Original: But this is contradict to our assumption.

Original: We are now be able to bound the difference between the predictions of the training inputs produced bythe empirical risk minimizer and the sample mean in each S j .

Original: To see that we have for each x ∈ S j .

Original: [Equation]

Original: By Equation (15) we then have for any x ∈ S j , with probability 1 − δ ,

Original: [Equation]

Original: [Equation]

Original: which means the difference between the predictions and the true label distribution is also bounded.

Original: Step III:

Original: Show the disjoint partition is non-trivial.

Original: In (32), we have managed to boundthe difference between the predictions yielded by an empirical risk minimizer and the true labeldistribution based on the cardinality of the subset |S j | , namely the number of inputs in j -partition.

Original: However |S j | is critical to the bound here as if |S j | = 1 , then (32) becomes a trivial bound.

Original: Here we 594 show |S

Original: j | is non-negligible based on simple combinatorics.

Original: Lemma A.4.

Original: Let {S j } N ρε j =1 be a disjoint partition of the entire training set S .

Original: Denote S ( x ) as thepartition that includes x . Let N ( x ) =

Original: |S ( x ) | and N = |S| .

Original: Then for any κ ≥ 1 ,(cid:12)(cid:12)(cid:12)(cid:12)(cid:26) x | N ( x ) ≥

Original: N [Equation]
Revised: [N/A]

Original: (cid:27)(cid:12)(cid:12)(cid:12)(cid:12) ≥ (cid:18) 1 − κ

Original: [Equation]

Original: Proof.

Original: We note that the problem is to show the minimum number of x such that N ( x ) ≥ N/ ( κN ρε ) .

Original: This is equivalent to find the maximum number of x such that N ( x ) ≤ N/ ( κN ρε ) .

Original: Since we only have

Original: N ρε subsets, the maximum can be attained only if for N ρε − 1 subsets S , |S| = N/ ( κN ρε ) .

Original: Otherwise, 600 if for any one of these subsets |S| < N/ ( κN ρε ) , then it is always feasible to let |S| = N/ ( κN ρε ) 601 and the maximum increases.

Original: Similarly, if the number of such subsets is less than N ρε − 1 , then it is 602 always feasible to let another subset subject to |S| = N/ ( κN ρε ) and the maximum increases.

Original: We can 603 then conclude that at most N ( N ρε − 1) / ( κN ρε ) inputs can have the property N ( x ) ≤ N/ ( κN ρε ) .

Original: The above lemma basically implies when partitioning N inputs into N ρε subsets, a large fraction ofthe inputs will be assigned to a subset with cardinality at least N/ ( κN ρε ) .

Original: Here N ρε is the coveringnumber and is bounded above based on the property of the covering in the Euclidean space.

Original: Apply

Original: Lemma A.4 to (32), and use the fact that ∥ · ∥ TV = ∥ · ∥ 1 / 2 for category distributions, we then arrive 609 at Lemma 4.4.

Original: Proof of Theorem 4.5.

Original: Proof.

Original: First, we show that adversarial perturbation generated by a realistic classifier can change itspredictive distribution.

Original: Considering adversarial perturbation based on FGSM and cross-entropy loss,namely x ′ = x − ε ∥∇ f θ ( x ) y ∥

Original: − 1 ∇ f θ ( x ) y , we can obtain a result similar to Lemma 4.1.

Original: Lemma A.5.

Original: Assume f θ ( x ) y is L θ -locally

Original: Lipschitz around x with bounded Hessian.

Original: Let σ m =

Original: inf

Original: z ∈B ε ( x ) σ min ( ∇ 2 f θ ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f θ ( z ) y ) > 0 .

Original: Here σ min and σ maxdenote the minimum and maximum eigenvalues of the Hessian, respectively.

Original: We then have

Original: [Equation]

Original: Second, We prove that the true label distribution will be distorted by the adversarial perturbationgenerated by a realistic classifier.

Original: This is guaranteed if the predictive distribution of a realisticclassifier can approximate the true label distribution.

Original: Specifically, by utilizing Lemma A.5 and

Original: Lemma 4.4, we have with probability 1 − 2 δ ,

Original: [Equation]

Original: Finally, we show that such distribution mismatch induces label noise in the adversarially augmentedtraining set.

Original: Similar to the proof for the true classifier, by the common labeling practice of adversarialexamples we have P ( ˜ Y ′

Original: | x ′ )

Original: =

Original: P ( ˜ Y | x ) =

Original: P ( Y | x ) .

Original: By utilizing Lemma 4.2 1 we then have withprobability 1 − 3 δ ,

Original: [Equation]

Original: where ξ = 1 + (cid:112) 4 κN ρε K .

Original: [-A.4 Proofs in Section 
-]
Revised: [-A.4 Proofs in Section 
-]

Original: Proof of Theorem 5.1.

Original: Proof.

Original: Let j ∗ =

Original: argmax P ( Y ′ =

Original: j | x ′ ) and thus P ( Y ′ =

Original: j ∗ | x ′ ) ∈

Original: [1 /c, 1] .

Original: Letg ( T ) :

Original: = f ( x ′ ; θ, T ) j ∗ , which is a continuous function defined on [0 , ∞ ] .

Original: The condition j ∗ =arg max j f ( x ′ ; θ, T ) j ensures that g ( T ) ∈

Original: [1 /c, 1] , where c is the number of classes.

Original: By the intermediate value theorem, there exists T ∗ , such that g ( T ∗ ) =

Original: P ( Y ′ =

Original: j ∗ | x ′ ) .

Original: Let T

Original: =

Original: T ∗ , we have

Original: [Equation]

Original: where the inequality holds by the triangle inequality.

Original: Meanwhile, we have

Original: [Equation]

Original: | x

Original: [Equation]

Original: Therefore, it can seen that for T

Original: =

Original: T ∗ ,

Original: [Equation]

Original: Proof of Theorem 5.2.

Original: Lemma A.6.

Original: Let x ′ be an example incorrectly classified by a classifier f

Original: in terms of the true label distribution P ( Y ′ =

Original: j | x ′ ) , namely

Original: [Equation]

Original: where j ∗ = arg max j P

Original: ( Y ′ =

Original: j | x ′ ) .

Original: Assume P

Original: ( Y ′ =

Original: j ∗ | x ′ ) ≥ 1 / 2 , then

Original: [Equation]

Original: [Equation]

Original: Proof.

Original: We prove it by contradiction.

Original: Assume f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =

Original: j ∗ | x ′ ) , we have f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =

Original: j ∗ | x ′ ) ≥ 1 / 2 .

Original: Therefore,

Original: [Equation]

Original: which means f ( x ′ ; θ, T )

Original: j < f ( x ′ ; θ, T ) j ∗ , ∀ j ̸

Original: = j ∗ .

Original: This leads to j ∗ = arg max j f ( x ′ ; θ, T ) j , 638 which contradicts our condition.

Original: [-Now we prove Theorem 5.
-]
Revised: [-Now we prove Theorem 5.
-]

Original: Proof.

Original: First let P ( Y ′ | x ′ )

Original: =

Original: P ( y | x ) ≈ 1 ( y ) .

Original: Let j ∗

Original: = arg max j P

Original: ( Y ′ =

Original: j | x ′ ) .

Original: By Lemma A.6 wehave f ( x ′ ; θ, T ) j ∗ ≤ P ( Y ′∗ =

Original: j ∗ | x ′ ) ≤ 1 .

Original: Then there exists λ ∗ > 0 , such that λ ∗ · f ( x ′ ; θ, T ) j ∗ +(1 − λ ∗ )

Original: =

Original: P ( Y ′ =

Original: j ∗ | x ′ ) by the intermediate value theorem.

Original: Let λ = λ ∗ , we have

Original: [Equation]

Original: Y

Original: [Equation]

Original: j

Original: [Equation]

Original: |

Original: [Equation]

Original: |

Original: [Equation]

Original: T

Original: [Equation]

Original: j

Original: [Equation]

Original: ( Y

Original: [Equation]

Original: j

Original: [Equation]

Original: |

Original: [Equation]

Original: j

Original: [Equation]

Original: j ̸

Original: [Equation]

Original: [ f (

Original: [Equation]

Original: θ

Original: [Equation]

Original: T

Original: [Equation]

Original: · f (

Original: [Equation]

Original: T

Original: [Equation]

Original: [Equation]

Original: [ P ( Y ′

Original: [Equation]

Original: j

Original: [Equation]

Original: | x ′ )

Original: [Equation]

Original: f ( x ′

Original: [Equation]

Original: θ

Original: [Equation]

Original: j

Original: [Equation]

Original: [Equation]

Original: (cid:88)

Original: j

Original: [Equation]

Original: j ̸

Original: [Equation]

Original: j

Original: [Equation]

Original: [ f (

Original: [Equation]

Original: T

Original: [Equation]

Original: · f ( x

Original: [Equation]

Original: θ

Original: [Equation]

Original: T

Original: [Equation]

Original: j ]

Original: [Equation]

Original: [ λ · f ( x ′

Original: [Equation]

Original: θ

Original: [Equation]

Original: T )

Original: j

Original: [Equation]

Original: + (1 − λ ) − f ( x ′ ; θ

Original: [Equation]

Original: T

Original: [Equation]

Original: j

Original: [Equation]

Original: [Equation]

Original: =

Original: [Equation]

Original: B Limitations

Original: We note that alternative labeling of adversarial examples proposed in this paper is based on thefact that the predictive distribution of a classifier trained with empirical risk minimization canapproximate the true label distribution of training examples.

Original: However, such approximation maynot be accurate especially if the classifier is not carefully regularized during training.

Original: Post-trainingconfidence calibration techniques such as temperature scaling and interpolation can only improvethe approximation in terms of the entire training set, but cannot improve it in a sample-wise manner.

Original: How to learn the true label distribution of adversarial training examples during adversarial trainingmore accurately remains an open problem.

Original: Also, such alternative labeling also requires to train another independent classifier beforehand, whichinduces additional training cost.

Original: C More empirical analyses

Original: C.1 Epoch-wise double descent is ubiquitous in adversarial training

Original: In this section, we conduct extensive experiments with different model architectures, and learningrate schedulers to verify the connection between robust overfitting and epoch-wise double descent.

Original: The default experiment settings are listed in Appendix G.2 in detail.

Original: Model capacity.

Original: We modulate the capacity of the deep model by varying the widening factor ofthe Wide ResNet.

Original: To extend the lower limit of the capacity, we allow the widening factor to be lessthan 1 .

Original: In such case, the number of channels in each residual block is scaled similarly but rounded,and the number of channels in the first convolutional layer will be reduced accordingly to ensure thewidth monotonically increasing through the forward propagation.

Original: Model architecture.

Original: We also experiment on model architectures other than Wide ResNet, includingpre-activation ResNet-18 (He et al., 2016) and VGG-11 (Simonyan & Zisserman, 2015).

Original: We select

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: these configurations to ensure comparable model capacities 2 .

Original: As shown in Figure 8, different modelarchitectures may produce slightly different double descent curves.

Original: The second descent of VGG-11in particular will be delayed due to its inferior performance compared to residual architectures.

Original: Learning

Original: rate scheduler

Original: A specific learning rate scheduler may shape the robust overfittingdifferently as suggested by Rice et al.

Original: We consider the following learning rate schedulers inour experiments.

Original: • Piecewise decay : The initial learning rate rate is set as 0 .

Original: and is decayed by a factor of 10 at the100 th and 500 th epochs within a total of 1000 epochs.

Original: • Cyclic : This scheduler was initially proposed by Smith (2017) and has been popular in adversarialtraining.

Original: We set the maximum learning rate to be 0 . 2 , and the learning rate will linearly increase

Original: from 0 to 0 .

Original: 2 for the initial 400 epochs and decrease to 0 for the later 600 epochs.

Original: • Cosine : This scheduler was initially proposed by Loshchilov & Hutter (2017).

Original: The learning ratestarts at 0 .

Original: 1 and gradually decrease to 0 following a cosine function for a total of 1000 epochs.

Original: Experiments

Original: on various learning rate schedulers show the second descent can be widely observedexcept the piecewise decay, where the appearance of second descent might be delayed due toextremely small learning rate in the late stage of training.

Original: D More experiment resultsD.1 Training longer

Original: As shown in Figure 9, We show that our method can maintain the robust test accuracy with moretraining epochs.

Original: Here, we follow the settings in Figure 7 except we train for additional epochs up to400 epochs for each dataset.

Original: D.2 Adversarial training methods, neural architectures and evaluation metrics

Original: In this section we conduct extensive experiments with different neural architectures, adversarialtraining methods and robustness evaluation metrics to verify the effectiveness of our method.

Original: [Figure]

Original: Tiny

Original: [Figure]

Original: ImageNet

Original: [Figure]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: KD

Original: [Table]

Original: Auto

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: WRN-28AT - -

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: KD

Original: [Table]

Original: AT

Original: [Table]

Original: KD

Original: [Table]

Original: AT-

Original: [Table]

Original: [Table]

Original: [Table]

Original: [-D.
-]
Revised: [-D.
-]

Original: Combined with additional orthogonal techniques

Original: We note that motivated from our theoretical analyses, our proposed method (KD-AT-Auto) is essentially the baseline knowledge distillation for adversarial training (KD-AT) with a robustly trained self-teacher, equipped with an algorithm that automatically finds its optimal hyperparameters (i.e. thetemperature T and the interpolation ratio λ ).

Original: Stochastic Weight Averaging (SWA) and additional stan-

Original: [Table]

Original: dard teachers (KD-Std) employed in (Chen et al., 2021) are orthogonal contributions.

Original: KD-AT-Autocan certainly be combined with SWA and KD-Std to achieve better performance.

Original: [Table]

Original: As shown in Table 5, on CIFAR-10, KD-AT + KD-Std + SWA

Original: (Chen et al., 2021) can already reducethe overfitting gap (difference between the best and last robust accuracy) to almost 0 .

Original: It is thus hard

Original: [Table]

Original: to see any further reduction by combining our method.

Original: To this end, we introduce an extra dataset

Original: [Table]

Original: SVHN (Netzer et al., 2011).

Original: As shown in Table 5, on SVHN, KD-AT + KD-Std + SWA still produces

Original: [Table]

Original: a high overfitting gap (also see Appendix A1.3 in (Chen et al., 2021)), whereas by combining withour algorithm to automatically find the optimal hyper-parameters (KD-AT-Auto + KD-Std

Original: + SWA),the overfitting gap can be further reduced to almost 0 .

Original: This demonstrates the effectiveness and wide

Original: [Table]

Original: applicability of our principle-guided method on mitigating robust overfitting.

Original: [Table]

Original: [Table]

Original: [-Here, the interpolation ratio of the standard teacher is fixed as 0 . 
-]
Revised: [-Here, the interpolation ratio of the standard teacher is fixed as 0 . 
-]

Original: and the SWA starts at the firstlearning rate decay for all experiments.

Original: We employ PGD-AT (Madry et al., 2018) as the baseadversarial training method and conduct experiments with a pre-activation ResNet-18.

Original: The robustaccuracy is evaluated with AutoAttack.

Original: Other experiment details are in line with Appendix G.1.

Original: Furthermore, we note that (Chen et al., 2021) shows SWA and KD-Std are essential components tomitigate robust overfitting on top of KD-AT, while we show that KD-AT itself can mitigate robustoverfitting by proper parameter tuning.

Original: We are thus able to separate these components and allow amore flexible selection of hyperparameters in diverse training scenarios without fear of overfitting.

Original: Inparticular, although (Chen et al., 2021) suggests SWA starting at the first learning rate decay (exactlywhen the overfitting starts) mitigates robust overfitting, the effectiveness of SWA on mitigatingoverfitting may strongly depend on its hyper-parameter selection including s 0 , i.e., the starting epochand τ , i.e., the decay rate 3 , which is also mentioned in recent work (Rebuffi et al., 2021).

Original: We alsodid some additional experiments on CIFAR-10 following the SWA setting in (Rebuffi et al.

Original: 2021)to demonstrate the wide applicability of our method.

Original: As shown by Table 6, when changing thehyperparameters of SWA, KD-AT + KD-Std + SWA cannot consistently mitigate robust overfitting,while KD-AT-Auto + KD-Std + SWA can maintain an overfitting gap close to 0 and achieve betterrobustness as well.

Original: [Table]

Original: E Study on a synthetic dataset with known true label distribution

Original: [Figure]

Original: Synthetic

Original: Dataset.

Original: Since the true label distribution is typically unknownfor adversarial examples in real-world datasets, we simulate the mechanism ofimplicit label noise in adversarial training from a feature learning perspective.

Original: Specifically, we adapt mixup (Zhang et al., 2018) for data augmentation on

Original: CIFAR-10.

Original: For every example x in the training set, we randomly selectanother example x ′ in a different class and linearly interpolate them by aratio ρ , namely x := ρx + (1 − ρ ) x ′ , which essentially perturbs x withfeatures from other classes.

Original: Therefore, the true label distribution is arguablyy ∼ ρ · 1 ( y )+(1 − ρ ) · 1 ( y ′ ) .

Original: Unlike mixup, we intentionally set the assigned 733 label as ˆ y ∼ 1 ( y ) , thus deliberately create a mismatch between the true labeldistribution and the assigned label distribution.

Original: We refer this strategy asmixup augmentation and only perform it once before the training.

Original: In thisway, the true label distribution of every example in the synthetic dataset is fixed.

Original: Concentration of optimal temperature and interpolation ratio of individual examples.

Original: In

Original: Section 5.1 we have shown that in terms of individual examples, the rectified model probability canprovably reduce the distribution mismatch between the assigned label distribution and true labeldistribution of the adversarial example.

Original: However, since the true label distribution is unknown inrealistic scenarios, it is not possible to directly follow Theorems 5.1 and 5.2 and calculate the optimalset of hyper-parameters for each example in the training set.

Original: The best we can do is to employ avalidation set and determine a universal set of hyper-parameters based on the NLL loss, which expectsall training examples to share similar optimal temperatures and interpolation ratios.

Original: Here, based onthe synthetic dataset where a true label distribution is known, we empirically verify this assumptionis reasonable.

Original: [Figure]

Original: In Figure 11 left, we solve the optimal temperature for each correctly classified training examplebased on Theorem 5.1 with the interpolation ratio fixed as 1 . 0 .

Original: One can find that the individualoptimal temperatures mostly concentrate between 0 . 5 and 1 . 5 .

Original: In Figure 11 right, we solve theoptimal interpolation ratio for each incorrectly classified training example based on Theorem 5.2with the temperature fixed as 1 . 0 .

Original: One can find that the individual optimal interpolation ratio mostlyconcentrate between 0 . 5 and 0 .

Original: F Method details

Original: [Figure]

Original: F.1 Determine the optimal hyper-parameters

Original: [Figure]

Original: One may note that Equation (9) cannot be directly optimized since the traditional adversarial label isonly defined on the example in the training set and cannot be simply generalized to the validationset.

Original: A reasonable solution is using the nearest neighbour classifier to find the closest traditionaladversarial label for every example in the validation set.

Original: However, to speed up the optimization, wepropose to employ the classifier overfitted by the traditional adversarial labels on the training set asan surrogate, which works well in practice.

Original: Specially, we employ a model overfitted on the trainingset to generate approximate traditional adversarial label of the adversarial example in the validationset.

Original: Such overfitted model is typically the model at the final checkpoint when conducting regular adversarial training for sufficient epochs.

Original: Mathematically, our final method to determine the optimaltemperature and interpolation ratio in rectified model probability can be described as

Original: [Equation]

Original: 37) where f θ s ( x ′ ; T ) denotes the temperature-scaled predictive probability of a surrogate model on x ′ .

Original: Here the validation set is constructed by applying adversarial perturbation generated by f θ to the cleanvalidation set.

Original: For adversarial perturbation we utilize PGD attack with 10 iterations, the perturbationradius as 8 / 255 and the step size as 2 / 255 .

Original: Note that Such process incurs almost no additionalcomputation as we simply obtain the logits of a surrogate classifier.

Original: G Experimental detailsG.1 Settings for main experiment resultsDataset.

Original: We include experiment results on CIFAR-10, CIFAR-100, Tiny-ImageNet and SVHN.

Original: Training setting.

Original: We employ SGD as the optimizer.

Original: The batch size is fixed to 128.

Original: The momentumand weight decay are set to 0 .

Original: 9 and 0 . 0005 respectively.

Original: Other settings are listed as follows.

Original: • CIFAR-10/CIFAR-100: we conduct the adversarial training for 160 epochs, with the learning ratestarting at 0 . 1 and reduced by a factor of 10 at the 80 and 120 epochs.• Tiny-ImageNet: we conduct the adversarial training for 80 epochs, with the learning rate starting at0 . 1 and reduced by a factor of 10 at the 40 and 60 epochs.• SVHN: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0 .

Original: 01 (assuggested by (Chen et al., 2021)) and reduced by a factor of 10 at the 40 and 60 epochs.

Original: Adversary setting.

Original: We

Original: conduct

Original: adversarial training with ℓ ∞ norm-bounded perturbations

Original: Weemploy adversarial training methods including PGD-AT, TRADES and FGSM.

Original: We set the perturbationradius to be 8 / 255 .

Original: For PGD-AT and TRADES, the step size is 2 / 255 and the number of attackiterations is 10 .

Original: Robustness evaluation.

Original: We consider the robustness against ℓ ∞ norm-bounded adversarial attackwith perturbation radius 8 / 255 .

Original: We employ AutoAttack for reliable evaluation.

Original: We also include theevaluation results again PGD-1000, Square Attack and RayS.

Original: Neural architectures.

Original: We include experiments results on pre-activation ResNet-18, WRN-28-5,

Original: WRN-34-10 and VGG-19.

Original: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.2 Settings for analyzing double descent in

Original: adversarial training

Original: Dataset

Original: We conduct experiments on the CIFAR-10 dataset, without additional data.

Original: Training setting.

Original: We conduct the adversarial training for 1000 epochs unless otherwise noted.

Original: By default we use SGD as the optimizer with a fixed learning rate 0 . 1 .

Original: When we experiment on asubset (see below) we use the Adam optimizer to improve training stability, where the learning rate isfixed as 0 . 0001 .

Original: The batch size will be fixed to 128 , and the momentum will be set as 0 .

Original: 9 wherevernecessary.

Original: No regularization such as weight decay is used.

Original: These settings are mostly aligned with theempirical analyse of double descent under standard training (Nakkiran et al., 2020).

Original: Sample size.

Original: To reduce the computation load demanded by an exponential number of trainingepochs, we reduce the size of the training set by randomly sampled a subset of size 5000 fromthe original training set without replacement.

Original: We adopt this setting for extensive experiments foranalyzing the dependence of epoch-wise double descent on the perturbation radius and data quality(i.e.

Original: Figure 5)..

Original: Adversary setting.

Original: We conduct adversarial training with ℓ ∞ norm-bounded perturbations.

Original: Weemploy standard PGD training with the perturbation radius set to 8 / 255 unless otherwise noted.

Original: Thenumber of attack iterations is fixed as 10 , and the perturbation step size is fixed as 2 / 255 .

Original: Robustness evaluation.

Original: We consider the robustness against ℓ ∞ norm-bounded adversarial attackwith perturbation radius 8 / 255 .

Original: We use PGD attack with 10 attack iterations and step size set to2 / 255 .

Original: Neural architecture.

Original: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016)with depth 28 and widening factor 5 (WRN-28-5) to speed up training.

Original: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.3 Estimation of the data quality

Original: In this section we elaborate on the calculation of data quality for analyzing the dependence on labelnoise in adversarial training.

Original: We use the predicative probabilities of classifiers trained on CIFAR-10 to score its training data.

Original: Similar strategy is employed in previous works to select high-quality unlabeled data to improveadversarial robustness (Uesato et al., 2019; Carmon et al., 2019; Gowal et al., 2020).

Original: Slightlydeviating from these works focusing on out-of-distribution data, we use adversarially trained insteadof regularly trained models to measure the quality of in-distribution data, since under standardtraining almost all training examples will be overfitted and gain overwhelmingly high confidence.

Original: Specifically, we adversarially train a pre-activation ResNet-18 with PGD and select the model atthe best checkpoint in terms of the robustness.

Original: The quality of an example is estimated by themodel probability corresponding to the true label without adversarial perturbation and random dataaugmentation (flipping and clipping).

Original: We repeat this process 10 times with random initialization toobtain a relatively accurate estimation.

Original: G

Original: 4 Settings for standard training on fixed augmented training sets

Original: [-G.4.
-]
Revised: [-G.4.
-]

Original: General settings for both adversarial augmentation and Gaussian augmentation

Original: Dataset.

Original: We conduct experiments on the CIFAR-10 dataset, without additional data.

Original: Training setting.

Original: We conduct the standard training for 1000 epochs.

Original: We use Adam as the optimizerwith a fixed learning rate 0 . 0001 to improve training stability with a small training set (see below).

Original: The batch size will be fixed to 128 , and the momentum will be set as 0 .

Original: 9 wherever necessary.

Original: Noregularization such as weight decay is used.

Original: Sample size.

Original: To reduce the computation load demanded by an exponential number of trainingepochs, we reduce the size of the training set by randomly sampled a subset of size 5000 from theoriginal training set without replacement.

Original: Neural architecture.

Original: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016)with depth 28 and widening factor 5 (WRN-28-5).

Original: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.4.2 Construction of the training set

Original: Adversarial

Original: augmentation

Original: We first obtain a robust model by conduct PGD training with preactivation ResNet-18 on CIFAR-10.

Original: We use early stopping to obtain the most robust model on avalidation set.

Original: The specific settings are aligned with Section G.1.

Original: Using this model, we then generate adversarial examples with PGD attack on the 5000 examplesrandomly sampled from CIFAR-10 training set.

Original: The number of attack iterations is fixed as 10 and thestep size is fixed as 2 / 255 .

Original: The adversarial examples along with their original labels are then groupedinto a training set for adversarial augmentation experiments.

Original: Gaussian augmentation.

Original: We apply Gaussian noise to the 5000 examples randomly sampled from

Original: CIFAR-10 training set.

Original: The perturbed examples along with their original labels are then grouped intoa training set for Gaussian augmentation experiments.