Original: Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
Revised: Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We show that label noise exists in adversarial training.
Revised: We show that label noise exists in adversarial training.

Original: Such label noise is due tothe mismatch between the true label distribution of adversarial examples and thelabel inherited from clean examples – the true label distribution is distorted by theadversarial perturbation, but is neglected by the common practice that inherits labelsfrom clean examples.
Revised: Such label noise is due to the mismatch between the true label distribution of adversarial examples and the label inherited from clean examples – the true label distribution is distorted by the adversarial perturbation, but is neglected by the common practice that inherits labels from clean examples.

Original: Recognizing label noise sheds insights on the prevalence ofrobust overfitting in adversarial training, and explains its intriguing dependenceon perturbation radius and data quality.
Revised: Recognizing label noise sheds insights on the prevalence of robust overfitting in adversarial training, and explains its intriguing dependence on perturbation radius and data quality.

Original: Also, our label noise perspective alignswell with our observations of the epoch-wise double descent in adversarial training.
Revised: Also, our label noise perspective aligns well with our observations of the epoch-wise double descent in adversarial training.

Original: Guided by our analyses, we proposed a method to automatically calibrate the labelto address the label noise and robust overfitting.
Revised: Guided by our analyses, we proposed a method to automatically calibrate the label to address the label noise and robust overfitting.

Original: Our method achieves consistentperformance improvements across various models and datasets without introducingnew hyper-parameters or additional tuning.
Revised: Our method achieves consistent performance improvements across various models and datasets without introducing new hyper-parameters or additional tuning.

Original:  Introduction
Revised:  Introduction

Original: Adversarial training (Goodfellow et al., 2015; Huang et al., 2015; Kurakin et al., 2017; Madry et al.,2018) is known as one of the most effective ways (Athalye et al., 2018;
Revised: Adversarial training (Goodfellow et al., 2015; Huang et al., 2015; Kurakin et al., 2017; Madry et al., 2018) is known as one of the most effective ways (Athalye et al., 2018;

Original: Uesato et al., 2018) to enhancethe adversarial robustness of deep neural networks (Szegedy et al., 2014; Goodfellow et al., 2015).
Revised: Uesato et al., 2018) to enhance the adversarial robustness of deep neural networks (Szegedy et al., 2014; Goodfellow et al., 2015).

Original: Itaugments training data with adversarial perturbations to prepare the model for adversarial attacks.
Revised: It augments training data with adversarial perturbations to prepare the model for adversarial attacks.

Original: Despite various efforts to generate more effective adversarial training examples (Ding et al., 2020;
Revised: Despite various efforts to generate more effective adversarial training examples (Ding et al., 2020;

Original: Zhang et al., 2020), the labels assigned to them attracts little attention.
Revised: Zhang et al., 2020), the labels assigned to them attracts little attention.

Original: As the common practice, theassigned labels of adversarial training examples are simply inherited from their clean counterparts.
Revised: As the common practice, the assigned labels of adversarial training examples are simply inherited from their clean counterparts.

Original: In this paper, we argue that the existing labeling practice of the adversarial training examplesintroduces label noise implicitly, since adversarial perturbation can distort the data semantics (Tsipraset al., 2019; Ilyas et al., 2019).
Revised: In this paper we argue that the existing labeling practice of the adversarial training examples introduces label noise implicitly, since adversarial perturbation can distort the data semantics (Tsipras et al., 2019; Ilyas et al., 2019).

Original: For example, as illustrated in Figure 1, even with a slight distortion ofthe data semantics (e.g., more ambiguous), the label distribution of the adversarially perturbed datamay not match the label distribution of the clean counterparts.
Revised: For example, as illustrated in Figure 1, even with a slight distortion of the data semantics (e.g., more ambiguous), the label distribution of the adversarially perturbed data may not match the label distribution of the clean counterparts.

Original: Such distribution shift is neglectedwhen assigning labels to adversarial examples, which are directly copied from the clean counterparts.
Revised: Such distribution shift is neglected when assigning labels to adversarial examples, which are directly copied from the clean counterparts.

Original: We observe that distribution mismatch caused by adversarial perturbation along with improperlabeling practice will cause label noise in adversarial training .
Revised: We observe that distribution mismatch caused by adversarial perturbation along with improper labeling practice will cause label noise in adversarial training .

Original: It is a mysterious and prominent phenomenon that the robust test error would start to increase afterconducting adversarial training for a certain number of epochs (Rice et al., 2020), and our labelnoise perspective provides an adequate explanation for this phenomenon.
Revised: It is a mysterious and prominent phenomenon that the robust test error would start to increase after conducting adversarial training for a certain number of epochs (Rice et al., 2020), and our label noise perspective provides an adequate explanation for this phenomenon.

Original: Specifically, from a classicbias-variance view of model generalization, label noise that implicitly exists in adversarialtrainingcan increase the model variance (Yang et al., 2020) and thus make the overfitting much more evidentcompared to standard training. Further analyses of label noise in adversarial trainingalso explain theintriguing dependence of robust overfitting on the perturbation radius (Dong et al., 2021b) and dataquality (Dong et al., 2021a) presented in the literature.
Revised: Specifically, from a classic bias-variance view of model generalization, label noise that implicitly exists in adversarial training

Original: [Table]
Revised: [Table]

Original: True
Revised: True

Original: [Table]
Revised: [Table]

Original: /sB6/
Revised: /sB6/

Original: wEAyZJB</latexit>
Revised: wEAyZJB</latexit>

Original: P (˜ Y 0 | x 0 ) <latexit sha1_base64="AhqicuZyvRZ3dmjd8VBTevD9huM=">AAAB+XicbVDLSsNAFJ34rPUVdaebYJHWTUlEUHcFNy5ctGAf0oYymUzaoZNJmLkpltgv8BfcuFDErZ/gH7jzL/wEp4+Fth64cDjnXu69x4s5U2DbX8bC4tLyympmLbu+sbm1be7s1lSUSEKrJOKRbHhYUc4ErQIDThuxpDj0OK17vcuRX+9TqVgkbmAQUzfEHcECRjBoqW2a5UILGPdpejvM39/lj9tmzi7aY1jzxJmSXMmsfH9c7z+U2+Zny49IElIBhGOlmo4dg5tiCYxwOsy2EkVjTHq4Q5uaChxS5abjy4fWkVZ8K4ikLgHWWP09keJQqUHo6c4QQ1fNeiPxP6+ZQHDupkzECVBBJouChFsQWaMYLJ9JSoAPNMFEMn2rRbpYYgI6rKwOwZl9eZ7UTorOafGiotOw0QQZdIAOUQE56AyV0BUqoyoiqI8e0TN6MVLjyXg13iatC8Z0Zg/9gfH+A+rBlig=</latexit> 
Revised: P (˜ Y 0 | x 0 ) <latexit sha1_base64="AhqicuZyvRZ3dmjd8VBTevD9huM=">AAAB+XicbVDLSsNAFJ34rPUVdaebYJHWTUlEUHcFNy5ctGAf0oYymUzaoZNJmLkpltgv8BfcuFDErZ/gH7jzL/wEp4+Fth64cDjnXu69x4s5U2DbX8bC4tLyympmLbu+sbm1be7s1lSUSEKrJOKRbHhYUc4ErQIDThuxpDj0OK17vcuRX+9TqVgkbmAQUzfEHcECRjBoqW2a5UILGPdpejvM39/lj9tmzi7aY1jzxJmSXMmsfH9c7z+U2+Zny49IElIBhGOlmo4dg5tiCYxwOsy2EkVjTHq4Q5uaChxS5abjy4fWkVZ8K4ikLgHWWP09keJQqUHo6c4QQ1fNeiPxP6+ZQHDupkzECVBBJouChFsQWaMYLJ9JSoAPNMFEMn2rRbpYYgI6rKwOwZl9eZ7UTorOafGiotOw0QQZdIAOUQE56AyV0BUqoyoiqI8e0TN6MVLjyXg13iatC8Z0Zg/9gfH+A+rBlig=</latexit> can increase the model variance (Yang et al. ) and make the much more compared to noise in explain the intriguing dependence of robust overfitting on the (Dong et al., 2021b) and data quality (Dong et al., 2021a) presented in the literature

Original: [Figure]
Revised: [Figure]

Original: employ PGD training (Madry et al., 2018) on CIFAR10 (Krizhevsky, 2009) with Wide ResNet (WRN) (Zagoruyko & Komodakis, 2016) and a fixed learning rate.

Original: WRN- 28 - k refers to WRN with depth 28 and widen factor k .

Original: Providing the label noise in adversarial training, one canfurther expect the existence of double descent basedon the modern generalization theory of deep neuralnetworks.
Revised: Providing the label noise in adversarial training, one can further expect the existence of double descent based on the modern generalization theory of deep neural networks.

Original: Epoch-wise double descent refers to the phenomenon that the test error will first decrease and thenincrease as predicted by the classic bias-variance tradeoff, but it will decrease again as the training continues.
Revised: Epoch-wise double descent refers to the phenomenon that the test error will first decrease and then increase as predicted by the classic bias-variance tradeoff, but it will decrease again as the training continues.

Original: Such phenomenon is only reported in standard training of deep neural networks, often requiring significantlabel noise in the training set (Nakkiran et al., 2020).
Revised: Such phenomenon is only reported in standard training of deep neural networks, often requiring significant label noise in the training set (Nakkiran et al., 2020).

Original: As the label noise intrinsically exists in adversarialtraining, such epoch-wise double descent phenomenonalso emerges when the training goes longer.
Revised: As the label noise intrinsically exists in adversarial training, such epoch-wise double descent phenomenon also emerges when the training goes longer.

Original: Indeed,as shown in Figure 2, for a relatively large model suchas WRN-28-5, on top of the existing robust overfittingphenomenon, the robust test error will eventually decrease again after 1 , 000 epochs .
Revised: Indeed, as shown in Figure 2, for a relatively large model such as WRN-28-5, on top of the existing robust overfitting phenomenon, the robust test error will eventually decrease again after 1 , 000 epochs .

Original: Following Nakkiranet al.
Revised: Following Nakkiran et al.

Original: (2020), we further experiment different modelsizes.
Revised: (2020), we further experiment different model sizes.

Original: One can find that a medium-sized model willfollow a classic U-curve, which means only overfittingis observed; and the robust test error for a small modelwill monotonically decrease.
Revised: One can find that a medium-sized model will follow a classic U-curve, which means only overfitting is observed; and the robust test error for a small model will monotonically decrease.

Original: These are well aligned with the observations in standard training regime.
Revised: These are well aligned with the observations in standard training regime.

Original: This again consolidates our understanding of label noise in adversarial training.
Revised: This again consolidates our understanding of label noise in adversarial training.

Original: In light of our analyses, we design a theoretically-grounded method to mitigate the label noisein adversarial training automatically.
Revised: In light of our analyses, we design a theoretically-grounded method to mitigate the label noise in adversarial training automatically.

Original: The key idea is to resort to an alternative labeling of theadversarial examples.
Revised: The key idea is to resort to an alternative labeling of the adversarial examples.

Original: We show that the predictive label distribution of an adversarially trainedprobabilistic classifier can approximate the true label distribution with high probability.
Revised: We show that the predictive label distribution of an adversarially trained probabilistic classifier can approximate the true label distribution with high probability.

Original: Thus it can beutilized as a better labeling of the adversarial examples and provably reduce the label noise.
Revised: Thus it can be utilized as a better labeling of the adversarial examples and provably reduce the label noise.

Original: We alsoshow that with proper temperature scaling and interpolation, such predictive label distribution canfurther reduce the label noise.
Revised: We also show that with proper temperature scaling and interpolation, such predictive label distribution can further reduce the label noise.

Original: This echoes the recent empirical practice of incorporating knowledgedistillation (Hinton et al., 2015) into adversarial training (Chen et al., 2021).
Revised: This echoes the recent empirical practice of incorporating knowledge distillation (Hinton et al., 2015) into adversarial training (Chen et al., 2021).

Original: While previous worksheuristically select fixed scaling and interpolation parameters for knowledge distillation, we showthat it is possible to fully unleash the potential of knowledge distillation by automatically determiningthe set of parameters that maximally reduces the label noise, with a strategy similar to confidencecalibration (Guo et al., 2017).
Revised: While previous works heuristically select fixed scaling and interpolation parameters for knowledge distillation, we show that it is possible to fully unleash the potential of knowledge distillation by automatically determining the set of parameters that maximally reduces the label noise, with a strategy similar to confidence calibration (Guo et al., 2017).

Original: Such strategy can further mitigate robust overfitting to a minimalamount without additional human tuning effort.
Revised: Such strategy can further mitigate robust overfitting to a minimal amount without additional human tuning effort.

Original: Extensive experiments on different datasets, trainingmethods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.
Revised: Extensive experiments on different datasets, training methods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.

Original: In summary, our findings and contributions are: 1) we show that the labeling of adversarial examplesin adversarial training practice introduces label noise implicitly; 2) we show that robust overfittingcan be adequately explained by such label noise, and it is the early part of an epoch-wise doubledescent; 3) 2e show an alternative labeling of the adversarial examples can be established to provablyreduce the label noise and mitigate the robust overfitting.
Revised: In summary, our findings and contributions are: 1) we show that the labeling of adversarial examples in adversarial training practice introduces label noise implicitly; 2) we show that robust overfitting can be adequately explained by such label noise, and it is the early part of an epoch-wise double descent; 3) 2e show an alternative labeling of the adversarial examples can be established to provably reduce the label noise and mitigate the robust overfitting.

Original:  Related Work
Revised:  Related Work

Original: Robust overfitting and double descent in adversarial training.
Revised: Robust overfitting and double descent in adversarial training.

Original: Double descent refers tothe phenomenon that overfitting by increasing model complexity will eventually improve test setperformance (Neyshabur et al., 2017; Belkin et al., 2019).
Revised: Double descent refers to the phenomenon that overfitting by increasing model complexity will eventually improve test set performance (Neyshabur et al., 2017; Belkin et al., 2019).

Original: This appears to conflict with the robustoverfitting phenomenon in adversarial training, where increasing model complexity by training longerwill impair test set performance constantly after a certain point during training.
Revised: This appears to conflict with the robust overfitting phenomenon in adversarial training, where increasing model complexity by training longer will impair test set performance constantly after a certain point during training.

Original: It is thus believed inthe literature that robust overfitting and epoch-wise double descent are separate phenomena (Riceet al., 2020).
Revised: It is thus believed in the literature that robust overfitting and epoch-wise double descent are separate phenomena (Rice et al., 2020).

Original: In this work we show this is not the complete picture by conducting adversarial trainingfor exponentially more epochs than the typical practice.
Revised: In this work we show this is not the complete picture by conducting adversarial training for exponentially more epochs than the typical practice.

Original: A recent work also considers a different notion of double descent that is defined with respect to theperturbation size (Yu et al., 2021).
Revised: A recent work also considers a different notion of double descent that is defined with respect to the perturbation size (Yu et al., 2021).

Original: Such double descent might be more related to the robustnessaccuracy trade-off problem (Papernot et al., 2016;
Revised: Such double descent might be more related to the robustnessaccuracy trade-off problem (Papernot et al., 2016;

Original: Su et al., 2018; Tsipras et al., 2019; Zhang et al. ,2019), rather than the classic understanding of double descent based on model complexity.
Revised: Su et al., 2018; Tsipras et al., 2019; Zhang et al., 2019), rather than the classic understanding of double descent based on model complexity.

Original: Mitigate robust overfitting.
Revised: Mitigate robust overfitting.

Original: Robust overfitting hinders the practical deployment of adversarialtraining methods as the final performance is often sub-optimal.
Revised: Robust overfitting hinders the practical deployment of adversarial training methods as the final performance is often sub-optimal.

Original: Various regularization methodsincluding classic approaches such as ℓ 1 and ℓ 2 regularization and modern approaches such ascutout (Devries & Taylor, 2017) and mixup (Zhang et al., 2018) have been attempted to tackle robustoverfitting, whereas they are shown to perform no better than simply early stopping the training ona validation set (Rice et al., 2020).
Revised: Various regularization methods including classic approaches such as ℓ 1 and ℓ 2 regularization and modern approaches such as cutout (Devries & Taylor, 2017) and mixup (Zhang et al., 2018) have been attempted to tackle robust overfitting, whereas they are shown to perform no better than simply early stopping the training on a validation set (Rice et al., 2020).

Original: However, early stopping raises additional concern as the bestcheckpoint of the robust test accuracy and that of the standard accuracy often do not coincide (Chenet al., 2021), thus inevitably sacrificing the performance on either criterion.
Revised: However, early stopping raises additional concern as the best checkpoint of the robust test accuracy and that of the standard accuracy often do not coincide (Chen et al., 2021), thus inevitably sacrificing the performance on either criterion.

Original: Various regularizationmethods specifically designed for adversarial training are thus proposed to outperform early stopping,including regularization the flatness of the weight loss landscape (Wu et al., 2020; Stutz et al., 2021),introducing low-curvature activation functions (Singla et al., 2021), data-driven augmentations thatadds high-quality additional data into the training (Rebuffi et al., 2021) and adopting stochastic weightaveraging (Izmailov et al., 2018) and knowledge distillation (Hinton et al., 2015)
Revised: Various regularization methods specifically designed for adversarial training are thus proposed to outperform early stopping, including regularization the flatness of the weight loss landscape (Wu et al., 2020; Stutz et al., 2021), introducing low-curvature activation functions (Singla et al., 2021), data-driven augmentations that adds high-quality additional data into the training (Rebuffi et al., 2021) and adopting stochastic weight averaging (Izmailov et al., 2018) and knowledge distillation (Hinton et al., 2015)

Original: (Chen et al., 2021).
Revised: (Chen et al., 2021).

Original: These methods are likely to suppress the label noise in adversarial training, with the self-distillationframework (i.e. the teacher shares the same architecture as the student model) introduced by (Chenet al., 2021) as a particular example since introducing teacher’s outputs as supervision is almostequivalent to the alternative labeling inspired by our understanding of the origin of label noise inadversarial training.
Revised: These methods are likely to suppress the label noise in adversarial training, with the self-distillation framework (i.e. the teacher shares the same architecture as the student model) introduced by (Chen et al., 2021) as a particular example since introducing teacher’s outputs as supervision is almost equivalent to the alternative labeling inspired by our understanding of the origin of label noise in adversarial training.

Original:  Preliminaries
Revised:  Preliminaries

Original: A statistic model of label noise (Frénay & Verleysen, 2014).
Revised: A statistic model of label noise (Frénay & Verleysen, 2014).

Original: Let X ⊂ R d define the input space111 equipped with a norm ∥ · ∥ : X → R + and Y =
Revised: Let X ⊂ R d define the input space equipped with a norm ∥ · ∥ : X → R + and Y =

Original: [ K ] := { 1 , 2 , . .
Revised: [ K ] := { 1 , 2 , . .

Original: , K } define the label space.
Revised: , K } define the label space.

Original: We112 introduce four random variables to describe noisy labeling process.
Revised: We introduce four random variables to describe noisy labeling process.

Original: Let X ∈ X denote the input,
Revised: Let X ∈ X denote the input,

Original: Y ∈ Y denote the true label of the input,
Revised: Y ∈ Y denote the true label of the input,

Original: ˜ Y ∈ Y denote the assigned label of an input provided by114 an annotator, and finally E denote the occurrence of a label error by this annotator.
Revised: ˜ Y ∈ Y denote the assigned label of an input provided by an annotator, and finally E denote the occurrence of a label error by this annotator.

Original: E is a binaryrandom variable with value 1 indicating that the assigned label is different from the true label for agiven input, i.e., E = [-
-] ( ˜ Y ̸ = Y ) .
Revised: E is a binary random variable with value 1 indicating that the assigned label is different from the true label for a given input, i.e., E = [-
-]1 ( ˜ Y ̸ = Y ) .

Original: We study the case where the label error depends on both the input
Revised: We study the case where the label error depends on both the input

Original: X and the true label Y .
Revised: X and the true label Y .

Original: For a classification problem, a training set consists of a set of examples thatare sampled as D = { ( x i , ˜ y i ) } i ∈
Revised: For a classification problem, a training set consists of a set of examples that are sampled as D = { ( x i , ˜ y i ) } i ∈

Original: [ N ] .
Revised: [ N ] .

Original: Definition 3.1 (Label noise) .
Revised: Definition 3.1 (Label noise) .

Original: We define label noise p e in a training set D as the empirical measure ofthe label error, namely p e ( D ) = 1 /N (cid:80) i ∈
Revised: We define label noise p e in a training set D as the empirical measure of the label error, namely p e ( D ) = 1 /N (cid:80) i ∈

Original: [ N ] 1 (˜ y i
Revised: [ N ] 1 (˜ y i

Original: ̸
Revised: ̸

Original: = y i ) .
Revised: = y i ) .

Original: Assumption 3.1.
Revised: Assumption 3.1.

Original: We assume the annotation of a clean dataset involves no label error, namely
Revised: We assume the annotation of a clean dataset involves no label error, namely

Original: P ( E = 1 | Y =
Revised: P ( E = 1 | Y =

Original: y, x ) = 0 .
Revised: y, x ) = 0 .

Original: This directly implies P ( ˜ Y | x ) =
Revised: This directly implies P ( ˜ Y | x ) =

Original: P ( Y | x ) (see proof in the Appendix).
Revised: P ( Y | x ) (see proof in the Appendix).

Original: Definition 3.2 (Data quality) .
Revised: Definition 3.2 (Data quality) .

Original: Given a training set D , we define its data quality as q ( D ) =124 E ( x,y ) 
Revised: Given a training set D , we define its data quality as q ( D ) = E ( x,y ) ∈D P

Original: ∈D P ( Y =
Revised: [Equation] Y =

Original: y
Revised: y

Original: | x )
Revised: | x )

Original: Adversarially augmented training set.
Revised: Adversarially augmented training set.

Original: Let f : X → Y be a probabilistic classifier and f ( · ) j126 be its predictive probability at class j .
Revised: Let f : X → Y be a probabilistic classifier and f ( · ) j be its predictive probability at class j .

Original: The adversarial example of x generated by f is obtained bysolving the maximization problem x ′ =
Revised: The adversarial example of x generated by f is obtained by solving the maximization problem x ′ =

Original: arg max z ∈B ε ( x ) ℓ ( f ( z ) , y ) .
Revised: arg max z ∈B ε ( x ) ℓ ( f ( z ) , y ) .

Original: Here ℓ can be a typical lossfunction such as cross-entropy.
Revised: Here ℓ can be a typical loss function such as cross-entropy.

Original: And B ε ( x ) denotes the norm ball centered at x with radius ε , i.e.,
Revised: And B ε ( x ) denotes the norm ball centered at x with radius ε , i.e.,

Original: B ε ( x ) = { z ∈ X : ∥
Revised: B ε ( x ) = { z ∈ X : ∥

Original: z − x ∥ ≤ ε } .
Revised: z − x ∥ ≤ ε } .

Original: Following previous notations, we denote Y ′
Revised: Following previous notations, we denote Y ′

Original: as a random variable representing the true label of x ′and ˜ Y ′ as a random variable representing the assigned label of x ′ .
Revised: as a random variable representing the true label of x ′ and ˜ Y ′ as a random variable representing the assigned label of x ′ .

Original: We refer D ′
Revised: We refer D ′

Original: = { ( x ′ , ˜ y ′ ) } as the132 adversarially augmented training set.
Revised: = { ( x ′ , ˜ y ′ ) } as the adversarially augmented training set.

Original:  training.
Revised: Adversarial training.

Original: Adversarial training can be viewed as a data augmentation technique thattrains the parametric classifier f θ on the adversarially augmented training set (Tsipras et al., 2019),namely
Revised: Adversarial training can be viewed as a data augmentation technique that trains the parametric classifier f θ on the adversarially augmented training set (Tsipras et al., 2019), namely

Original: [Equation]
Revised: [Equation]

Original:  Label noise implicitly exists in adversarial training
Revised:  Label noise implicitly exists in adversarial training

Original: In this section, we demonstrate the implicit existence of label noise in the adversarially augmentedtraining set.
Revised: In this section, we demonstrate the implicit existence of label noise in the adversarially augmented training set.

Original: We first consider a simple case where the adversarial perturbation is generated basedon an ideal classifier that predicts the true label distribution.
Revised: We first consider a simple case where the adversarial perturbation is generated based on an ideal classifier that predicts the true label distribution.

Original: Under such a case we prove that thelabel noise in the adversarially augmented training set is lower-bounded.
Revised: Under such a case we prove that the label noise in the adversarially augmented training set is lower-bounded.

Original: We then show that inrealistic cases an adversarially trained classifier can approximate the true label distribution with highprobability.
Revised: We then show that in realistic cases an adversarially trained classifier can approximate the true label distribution with high probability.

Original: Therefore, additional error terms will be required to lower bound the label noise.
Revised: Therefore, additional error terms will be required to lower bound the label noise.

Original: Allproofs for the remainder of this paper are provided in the appendix.
Revised: All proofs for the remainder of this paper are provided in the appendix.

Original: 4.1 When adversarial perturbation is generated by the true probabilistic classifier
Revised: 4.1 When adversarial perturbation is generated by the true probabilistic classifier

Original: We first consider an ideal case where the adversarial perturbation is generated by the true probabilisticclassifier f ( x ) :
Revised: We first consider an ideal case where the adversarial perturbation is generated by the true probabilistic classifier f ( x ) :

Original: = P ( Y | x ) , namely the classifier producing the true label distribution on any input x .
Revised: = P ( Y | x ) , namely the classifier producing the true label distribution on any input x .

Original: The true label distribution is distorted by adversarial perturbation.
Revised: The true label distribution is distorted by adversarial perturbation.

Original: We quantify the mismatchbetween two probability distributions using the total variation (TV) distance .
Revised: We quantify the mismatch between two probability distributions using the total variation (TV) distance .

Original: Definition 4.1 (TV distance) .
Revised: Definition 4.1 (TV distance) .

Original: Let A be a collection of the subsets of the label sample space Y .
Revised: Let A be a collection of the subsets of the label sample space Y .

Original: The
Revised: The

Original: TV distance between two probability distributions P ( Y ) and P ( Y ′ ) can be defined as ∥ P ( Y ) −
Revised: TV distance between two probability distributions P ( Y ) and P ( Y ′ ) can be defined as ∥ P ( Y ) −

Original: P ( Y ′ )
Revised: P ( Y ′ )

Original: ∥ TV = sup J ∈A | P ( Y ∈J ) − P
Revised: ∥ TV = sup J ∈A | P ( Y ∈

Original:  ( Y 
Revised: [Equation] ) − P ( Y ′ ∈

Original: [Equation]
Revised: [Equation]

Original: J ) | .
Revised: ) | .

Original: We now show that adversarial perturbation generated by the true probabilistic classifier can inducea mismatch between the true label distributions of clean inputs and their adversarial examples.
Revised: We now show that adversarial perturbation generated by the true probabilistic classifier can induce a mismatch between the true label distributions of clean inputs and their adversarial examples.

Original: For simplicity we consider adversarial perturbation based on FGSM and cross-entropy loss, namelyx ′ =
Revised: For simplicity we consider adversarial perturbation based on FGSM and cross-entropy loss, namely x ′ =

Original: x − ε ∥∇ f ( x ) y ∥
Revised: x − ε ∥∇ f ( x ) y ∥

Original: − 1 ∇ f ( x ) y .
Revised: − 1 ∇ f ( x ) y .

Original: The distribution mismatch induced by such adversarial perturbation156 can be lower bounded.
Revised: The distribution mismatch induced by such adversarial perturbation can be lower bounded.

Original: Lemma 4.1.
Revised: Lemma 4.1.

Original: Assume f
Revised: Assume f

Original: ( x ) y is L -locally Lipschitz around x with bounded Hessian.
Revised: ( x ) y is L -locally Lipschitz around x with bounded Hessian.

Original: Let σ m =
Revised: Let σ m =

Original: inf
Revised: inf

Original: z ∈B ε ( x ) σ min ( ∇ 2 f ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f ( z ) y ) > 0 .
Revised: z ∈B ε ( x ) σ min ( ∇ 2 f ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f ( z ) y ) > 0 .

Original: Here σ min and σ maxdenote the minimum and maximum eigenvalues of the Hessian, respectively.
Revised: Here σ min and σ max denote the minimum and maximum eigenvalues of the Hessian, respectively.

Original: We then have
Revised: We then have

Original: [Equation]
Revised: [Equation]

Original: One can find that the right-hand side is positive as long as the upper bound of the Hessian norm is nottoo large, which is reasonable as previous works have shown that small hessian norm is critical toboth standard (Keskar et al., 2017) and robust generalization (Moosavi-Dezfooli et al., 2019).
Revised: One can find that the right-hand side is positive as long as the upper bound of the Hessian norm is not too large, which is reasonable as previous works have shown that small hessian norm is critical to both standard (Keskar et al., 2017) and robust generalization (Moosavi-Dezfooli et al., 2019).

Original: Assigned label distribution is unchanged.
Revised: Assigned label distribution is unchanged.

Original: Despite the fact that the true label distribution isdistorted by adversarial perturbation, we note that the assigned label distribution of adversarialexamples is still the same as their clean counterparts.
Revised: Despite the fact that the true label distribution is distorted by adversarial perturbation, we note that the assigned label distribution of adversarial examples is still the same as their clean counterparts.

Original: Remark 4.1.
Revised: Remark 4.1.

Original: In adversarial training, it is the common practice that directly copies the label of aclean input to its adversarial counterpart, namely ˜ y ′ =
Revised: In adversarial training, it is the common practice that directly copies the label of a clean input to its adversarial counterpart, namely ˜ y ′ =

Original: ˜ y and P ( ˜ Y ′
Revised: ˜ y and P ( ˜ Y ′

Original: | x ′ )
Revised: | x ′ )

Original: =
Revised: =

Original: P ( ˜ Y | x ) .
Revised: P ( ˜ Y | x ) .

Revised: [Figure]

Revised: augmented training set (e.

Revised: 16 / 255 ) can also produce prominent overfitting.

Revised: In contrast, on the original training set without adversarial perturbation applied ( ε = 0 ), no overfitting is observed.

Original:  Distribution mismatch indicates label noise.
Revised: [Figure] will not produce overfitting. Here we select extremely large perturbation radius to reduce the test error to be comparable to the Distribution mismatch indicates label noise.

Original: We show that a mismatch between the true labeldistribution and the assigned label distribution in a training set will always indicate the existence oflabel noise.
Revised: We show that a mismatch between the true label distribution and the assigned label distribution in a training set will always indicate the existence of label noise.

Original: Lemma 4.2.
Revised: Lemma 4.2.

Original: Given a training set D = { ( x i , ˜ y i ) } i ∈
Revised: Given a training set D = { ( x i , ˜ y i ) } i ∈

Original: [ N ] , the label noise is lower-bounded by the172 mismatch between the true label distribution and the assigned label distribution.Specifically, with [Figure] augmented
Revised: [ N ] , the label noise is lower-bounded by the mismatch between the true label distribution and the assigned label distribution.

Original: 255 ) can also produce prominent over-

Original: In contrast on the without adversarial applied

Original: = 0 ) observed. [Figure] probability 1 − δ , we have
Revised: Specifically, with probability 1 − δ , we have

Original: [Equation]
Revised: [Equation]

Original: Label noise implicitly exists in adversarial training.
Revised: Label noise implicitly exists in adversarial training.

Original: In the adversarially augmented training set
Revised: In the adversarially augmented training set

Original: D ′ , such distribution mismatch exists exactly.
Revised: D ′ , such distribution mismatch exists exactly.

Original: By Remark 4.1 we have P ( ˜ Y ′
Revised: By Remark 4.1 we have P ( ˜ Y ′

Original: | x ′ )
Revised: | x ′ )

Original: =
Revised: =

Original: P ( ˜ Y | x ) and byproperty of the clean dataset (Assumption 3.1) we have P ( ˜ Y | x ) =
Revised: P ( ˜ Y | x ) and by property of the clean dataset (Assumption 3.1) we have P ( ˜ Y | x ) =

Original: P ( Y | x ) , which together means
Revised: P ( Y | x ) , which together means

Original: P ( ˜ Y ′
Revised: P ( ˜ Y ′

Original: | x ′ )
Revised: | x ′ )

Original: =
Revised: =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: However, Lemma 4.1 shows that P ( Y ′ | x ′ )
Revised: However, Lemma 4.1 shows that P ( Y ′ | x ′ )

Original: ̸
Revised: ̸

Original: = P ( Y | x ) , which implies that
Revised: = P ( Y | x ) , which implies that

Original: P ( ˜ Y ′
Revised: P ( ˜ Y ′

Original: | x ′ )
Revised: | x ′ )

Original: ̸
Revised: ̸

Original: = P ( Y ′
Revised: = P ( Y ′

Original: | x ′ ) .
Revised: | x ′ ) .

Original: This indicates that label noise exists in the adversarially augmented training179 set.
Revised: This indicates that label noise exists in the adversarially augmented training set.

Original: We now have the following theorem, which is our main result.
Revised: We now have the following theorem, which is our main result.

Original: Theorem 4.3.
Revised: Theorem 4.3.

Original: Assume f
Revised: Assume f

Original: ( x ) y is L -locally Lipschitz around x with Hessian bounded below.
Revised: ( x ) y is L -locally Lipschitz around x with Hessian bounded below.

Original: Instantiatethe same notations as in Lemma 4.1.
Revised: Instantiate the same notations as in Lemma 4.1.

Original: With probability 1 − δ , we have
Revised: With probability 1 − δ , we have

Original: [Equation]
Revised: [Equation]

Original: The above results suggest that as long as a training set is augmented by adversarial perturbation, butwith assigned labels unchanged, label noise emerges.
Revised: The above results suggest that as long as a training set is augmented by adversarial perturbation, but with assigned labels unchanged, label noise emerges.

Original: We demonstrate this by showing that standardtraining on a fixed adversarially augmented training set can also produce overfitting.
Revised: We demonstrate this by showing that standard training on a fixed adversarially augmented training set can also produce overfitting.

Original: Specifically, foreach example in a clean training set we apply adversarial perturbation generated by a adversariallytrained classifier.
Revised: Specifically, for each example in a clean training set we apply adversarial perturbation generated by a adversarially trained classifier.

Original: We then fix such an augmented training set and conduct standard training on it.
Revised: We then fix such an augmented training set and conduct standard training on it.

Original: Weexperiment on CIFAR-10 with WRN-28-5.
Revised: We experiment on CIFAR-10 with WRN-28-5.

Original: A training subset of size 5k is randomly sampled to speedup the training.
Revised: A training subset of size 5k is randomly sampled to speed up the training.

Original: More details about the experiment settings can be found in the appendix.
Revised: More details about the experiment settings can be found in the appendix.

Original: Figure 3shows that prominent overfitting (as well as epoch-wise double descent) can be observed when theperturbation radius is relatively large.
Revised: Figureshows that prominent overfitting (as well as epoch-wise double descent) can be observed when the perturbation radius is relatively large.

Original: On the other hand, if a training set is augmented by perturbation that will not distort the true labeldistribution, there will not be label noise.
Revised: On the other hand, if a training set is augmented by perturbation that will not distort the true label distribution, there will not be label noise.

Original: We demonstrate this by showing that standard trainingon a training set augmented with Gaussian noise will not induce overfitting.
Revised: We demonstrate this by showing that standard training on a training set augmented with Gaussian noise will not induce overfitting.

Original: As shown in Figure 4,even with a extremely large radius of Gaussian perturbation, no overfitting is observed.
Revised: As shown in Figure 4, even with a extremely large radius of Gaussian perturbation, no overfitting is observed.

Original: This alsodemonstrates that input perturbation not necessarily leads to overfitting.
Revised: This also demonstrates that input perturbation not necessarily leads to overfitting.

Original: Intuitive interpretation of label noise in adversarial training.
Revised: Intuitive interpretation of label noise in adversarial training.

Original: We introduce a simple exampleto help understand the emergence of label noise in adversarial training.
Revised: We introduce a simple example to help understand the emergence of label noise in adversarial training.

Original: Example 4.1 (Label noise due to a symmetric distribution shift) .
Revised: Example 4.1 (Label noise due to a symmetric distribution shift) .

Original: Let D =
Revised: Let D =

Original: ( x i , y i ) } i ∈ [ N ] be a199 clean labeled training subset where all inputs x i = x are identical and have a one-hot true labeldistribution, i.e., P ( Y | x ) = 1 y .
Revised: ( x i , y i ) } i ∈ [ N ] be a clean labeled training subset where all inputs x i = x are identical and have a one-hot true label distribution, i.e., P ( Y | x ) = 1 y .

Original: We now construct an adversarially augmented training subset D ′ =
Revised: We now construct an adversarially augmented training subset D ′ =

Original: { ( x ′ i , ˜ y ′ i ) } i ∈ [ N ] , where ˜ y ′
Revised: { ( x ′ i , ˜ y ′ i ) } i ∈ [ N ] , where ˜ y ′

Original: =
Revised: =

Original: y and x ′ is generated based on adversarial perturbation that distorts the true label distribution symmetrically.
Revised: y and x ′ is generated based on adversarial perturbation that distorts the true label distribution symmetrically.

Original: Specifically,
Revised: Specifically,

Original: [Equation]
Revised: [Equation]

Original: Then by Lemma 4.2 we have p e ( D ′ ) ≳
Revised: Then by Lemma 4.2 we have p e ( D ′ ) ≳

Original: η .
Revised: η .

Original: One can find that there is indeed η faction of noisy labels in D ′ .
Revised: One can find that there is indeed η faction of noisy labels in D ′ .

Original: This is because if we sample thelabels of x ′ based on its true label distribution, we expect 1 − η faction of x ′ are labeled as y , while ηfraction of x ′ are labeled to be other classes.
Revised: This is because if we sample the labels of x ′ based on its true label distribution, we expect 1 − η faction of x ′ are labeled as y , while η fraction of x ′ are labeled to be other classes.

Original: However, in D ′ , all x ′ are assigned with label y , whichmeans η fraction of x ′ are labeled incorrectly.
Revised: However, in D ′ , all x ′ are assigned with label y , which means η fraction of x ′ are labeled incorrectly.

Original: In realistic datasets we can consider inputs with similarfeatures for such reasoning.
Revised: In realistic datasets we can consider inputs with similar features for such reasoning.

Original: The above example also shows that label noise in adversarial training may be stronger than one’simpression.
Revised: The above example also shows that label noise in adversarial training may be stronger than one’s impression.

Original: Even a slight distortion of the true label distribution, e.g. η = 0 .
Revised: Even a slight distortion of the true label distribution, e.g. η = 0 .

Original: 1 , will be equivalent toat least 10% noisy label in the training set.
Revised: 1 , will be equivalent to at least 10% noisy label in the training set.

Original: This is because the true label distribution of every traininginput is distorted, resulting in significant noise in the population.
Revised: This is because the true label distribution of every training input is distorted, resulting in significant noise in the population.

Original: Dependence of label noise in adversarial training.
Revised: Dependence of label noise in adversarial training.

Original: Theorem 4.3 shows that the label noise inadversarial training is proportional to (1) the perturbation radius (2) the data quality.
Revised: Theorem 4.3 shows that the label noise in adversarial training is proportional to (1) the perturbation radius (2) the data quality.

Original: Considering labelnoise can be an important source of variance in the generalization of deep neural networks (Nakkiranet al., 2020; Yang et al., 2020), such dependence of label noise explains the intriguing observationsin the literature that robust overfitting (or epoch-wise double descent) in adversarial training willvanish with small perturbation
Revised: Considering label noise can be an important source of variance in the generalization of deep neural networks (Nakkiran et al., 2020; Yang et al., 2020), such dependence of label noise explains the intriguing observations in the literature that robust overfitting (or epoch-wise double descent) in adversarial training will vanish with small perturbation

Original: radii (Dong et al., 2021b) or high-quality data (Dong et al., 2021a).
Revised: radii (Dong et al., 2021b) or high-quality data (Dong et al., 2021a).

Original: We conduct more controlled experiments to verify this correlation empirically, as shown in Figure 5,
Revised: We conduct more controlled experiments to verify this correlation empirically, as shown in Figure 5,

Original: [Figure]
Revised: [Figure]

Original: 4.2 Adversarial perturbation generated by a realistic classifier
Revised: 4.2 Adversarial perturbation generated by a realistic classifier

Original: We now consider a realistic case where the adversarial perturbation is generated by a probabilisticclassifier f θ .
Revised: We now consider a realistic case where the adversarial perturbation is generated by a probabilistic classifier f θ .

Original: Approximation of the true label distribution.
Revised: Approximation of the true label distribution.

Original: We show that after sufficient adversarial training,the predictive label distribution of f θ can approximate the true label distribution with high probability.
Revised: We show that after sufficient adversarial training, the predictive label distribution of f θ can approximate the true label distribution with high probability.

Original: Lemma 4.4.
Revised: Lemma 4.4.

Original: Denote S =
Revised: Denote S =

Original: { x : ( x, y ) ∈ D} as the collection of all training inputs.
Revised: { x : ( x, y ) ∈ D} as the collection of all training inputs.

Original: Let ρ ≥ 1 and
Revised: Let ρ ≥ 1 and

Original: C be an ρε -external covering of S with covering number N ρε .
Revised: C be an ρε -external covering of S with covering number N ρε .

Original: Let f θ be a probabilistic classifier225 that minimizes the adversarial empirical risk (1).
Revised: Let f θ be a probabilistic classifier that minimizes the adversarial empirical risk (1).

Original: Assume f θ is L θ -locally Lipschitz continuous ina norm ball of radius ρε around x ∈ C .
Revised: Assume f θ is L θ -locally Lipschitz continuous in a norm ball of radius ρε around x ∈ C .

Original: Let κ ≥ 1 and ˆ S be a subset of S with cardinality at least(1 − 

Revised: Let κ ≥ 1 and ˆ S be a subset of S with cardinality at least (1 −

Original: /κ +1 / ( κN ρε ))
Revised: /κ +1 / ( κN ρε ))

Original: N .
Revised: N .

Original: Let N ε ( ˆ S ) denote the neighborhood of the set ˆ S , i.e. N ε
Revised: Let N ε ( ˆ S ) denote the neighborhood of the set ˆ S , i.e. N ε

Original: ( ˆ S )
Revised: ( ˆ S )

Original: = (cid:83) [Equation] ∈ ˆ 
Revised: = (cid:83) x ∈ ˆ S B ε ( x ) .

Original: [Equation] ε [Equation] Then for any x ∈ N ε ( ˆ S ) , with probability at least 1 − δ ,
Revised: Then for any x ∈ N ε ( ˆ S ) , with probability at least 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: Label noise in adversarial training with a realistic classifier.
Revised: Label noise in adversarial training with a realistic classifier.

Original: Adversarial perturbation generatedby a realistic classifier f θ will distort its predictive label distribution by gradient ascent.
Revised: Adversarial perturbation generated by a realistic classifier f θ will distort its predictive label distribution by gradient ascent.

Original: Subsequently,the true label distribution will also be distorted with high probability since the predictive labeldistribution of a realistic classifier f θ can approximate the true label distribution.
Revised: Subsequently, the true label distribution will also be distorted with high probability since the predictive label distribution of a realistic classifier f θ can approximate the true label distribution.

Original: Specifically, by thetriangle inequality we have
Revised: Specifically, by the triangle inequality we have

Original: [Equation]
Revised: [Equation]

Original: where the last two terms are the approximation error of true label distribution on both clean andadversarial examples, which are guaranteed to be small.
Revised: where the last two terms are the approximation error of true label distribution on both clean and adversarial examples, which are guaranteed to be small.

Original: To conclude, we have the following result.
Revised: To conclude, we have the following result.

Original: Theorem 4.5.
Revised: Theorem 4.5.

Original: Instantiate the notations of Lemma 4.4.
Revised: Instantiate the notations of Lemma 4.4.

Original: For any x ∈ N ε ( ˆ S ) , with probability at least1 − 3 δ , we have
Revised: For any x ∈ N ε ( ˆ S ) , with probability at least 1 − 3 δ , we have

Original: [Equation]
Revised: [Equation]

Original:  where ξ = 1 + √ 4 κN s K .
Revised: 2 − K (cid:19) L θ + L (cid:19)(cid:21) − ε 2 σ M 4 − ξ (cid:114)2 N log δ , (7) where ξ = 1 + (cid:112) 4 κN ρε K .

Original:  Mitigate Label Noise in Adversarial Training
Revised:  Mitigate Label Noise in Adversarial Training

Original: Since the label noise is incurred by the mismatch between the true label distribution and assignedlabel distribution of adversarial examples in the training set, we wish to find an alternative label(distribution) for the adversarial example to reduce such distribution mismatch.
Revised: Since the label noise is incurred by the mismatch between the true label distribution and assigned label distribution of adversarial examples in the training set, we wish to find an alternative label (distribution) for the adversarial example to reduce such distribution mismatch.

Original: We’ve already shownthat the predictive label distribution of a classifier trained by conventional adversarial training, whichwe denote as model probability in the following discussion, can in fact approximate the true labeldistribution.
Revised: We’ve already shown that the predictive label distribution of a classifier trained by conventional adversarial training, which we denote as model probability in the following discussion, can in fact approximate the true label distribution.

Original: Here we show that it is possible to further improve the predictive label distribution andreduce the label noise by calibration.
Revised: Here we show that it is possible to further improve the predictive label distribution and reduce the label noise by calibration.

Original: 5.1 Rectify model probability to reduce distribution mismatch
Revised: 5.1 Rectify model probability to reduce distribution mismatch

Original: We show that it is possible to reduce the distribution mismatch by temperature scaling (Hinton et al. Guo et al., 2017) enabled in the softmax function.
Revised: We show that it is possible to reduce the distribution mismatch by temperature scaling (Hinton et al., 2015; Guo et al., 2017) enabled in the softmax function.

Original: Theorem 5.1 (Temperature scaling can reduce the distribution mismatch) .
Revised: Theorem 5.1 (Temperature scaling can reduce the distribution mismatch) .

Original: Let f θ ( x ; T ) denote the predictive probability of a probabilistic classifier scaled by temperature T , namely f θ ( x ; T )
Revised: Let f θ ( x ; T ) denote the predictive probability of a probabilistic classifier scaled by temperature T , namely f θ ( x ; T )

Original: j = exp( z j /T ) /
Revised: j = exp( z j /T ) /

Original: ( (cid:80) j exp( z j /T )) , where z is the logits of the classifier from x .
Revised: ( (cid:80) j exp( z j /T )) , where z is the logits of the classifier from x .

Original: Let x ′ be an adversarial example correctly classified by a classifier f θ , i.e. arg max j f θ ( x ′ )j [Equation]
Revised: Let x ′ be an adversarial example correctly classified by a classifier f θ , i.e. arg max j f θ ( x ′ )

Original:  y ′ , then there exists T , such that
Revised: j = y ′ , then there exists T , such that

Original: [Equation]
Revised: [Equation]

Original: Another way to further reduce the distribution mismatch is to interpolate between the model probability and the one-hot assigned label.
Revised: Another way to further reduce the distribution mismatch is to interpolate between the model probability and the one-hot assigned label.

Original: We show that the interpolation works specifically for incorrectlyclassified examples and thus can be viewed as a complement to temperature scaling.
Revised: We show that the interpolation works specifically for incorrectly classified examples and thus can be viewed as a complement to temperature scaling.

Original: Theorem 5.2 (Interpolation can further reduce the distribution mismatch) .
Revised: Theorem 5.2 (Interpolation can further reduce the distribution mismatch) .

Original: Let x ′ be an adversarial example incorrectly classified by a classifier f θ , i.e. arg max j f θ ( x ′ ; T ) j ̸ = y ′ .
Revised: Let x ′ be an adversarial example incorrectly classified by a classifier f θ , i.e. arg max j f θ ( x ′ ; T ) j ̸ = y ′ .

Original: Assumemax j P ( Y ′ =
Revised: Assume max j P ( Y ′ =

Original: j | x ′ ) ≥ 1 / 2 , then there exists an interpolation ratio λ , such that[Equation] x
Revised: j | x ′ ) ≥ 1 / 2 , then there exists an interpolation ratio λ , such that

Original: [Equation]
Revised: [Equation]

Original: where f θ ( x ′ ; T, λ )
Revised: where f θ ( x ′ ; T, λ )

Original: =
Revised: =

Original: λ · f θ ( x ′ ; T ) + (1 − λ ) · P ( ˜ Y ′
Revised: λ · f θ ( x ′ ; T ) + (1 − λ ) · P ( ˜ Y ′

Original: | x ′ ) .
Revised: | x ′ ) .

Original: As a summarization, to reduce the distribution mismatch, we propose to use f θ ( x ′ ; T, λ ) as theassigned label of the adversarial example in adversarial training, which we refer as the rectified modelprobability .
Revised: As a summarization, to reduce the distribution mismatch, we propose to use f θ ( x ′ ; T, λ ) as the assigned label of the adversarial example in adversarial training, which we refer as the rectified model probability .

Original: In Appendix E, we show that the optimal hyper-parameters (i.e. T and λ ) of almost all trainingexamples concentrate on the same set of values by studying on a synthetic dataset with known truelabel distribution.
Revised: In Appendix E, we show that the optimal hyper-parameters (i.e. T and λ ) of almost all training examples concentrate on the same set of values by studying on a synthetic dataset with known true label distribution.

Original: Therefore it is possible to find an universal set of hyper-parameters that reduce thedistribution mismatch for all adversarial examples.
Revised: Therefore it is possible to find an universal set of hyper-parameters that reduce the distribution mismatch for all adversarial examples.

Original: 5.2 Determine the optimal temperature and interpolation ratio
Revised: 5.2 Determine the optimal temperature and interpolation ratio

Original: The set of temperature and interpolation ratio in the rectified model probability that maximallyreduces the distribution mismatch is not straightforward to find as the true label distribution of theadversarial example is unknown in reality.
Revised: The set of temperature and interpolation ratio in the rectified model probability that maximally reduces the distribution mismatch is not straightforward to find as the true label distribution of the adversarial example is unknown in reality.

Original: Fortunately, given a sufficiently large validation dataset asa whole, it is possible to measure the overall distribution mismatch in a frequentist’s view withoutknowing the true label distribution of every single example.
Revised: Fortunately, given a sufficiently large validation dataset as a whole, it is possible to measure the overall distribution mismatch in a frequentist’s view without knowing the true label distribution of every single example.

Original: A popular metric adopted here is thenegative log-likelihood (NLL) loss, which is known as a proper scoring rule (Gneiting & Raftery,2007) and is also employed in the confidence calibration of deep networks (Guo et al., 2017).
Revised: A popular metric adopted here is the negative log-likelihood (NLL) loss, which is known as a proper scoring rule (Gneiting & Raftery, 2007) and is also employed in the confidence calibration of deep networks (Guo et al., 2017).

Original: By
Revised: By

Original: Gibbs’s inequality it is easy to show that the NLL loss will only be minimized when the assignedlabel distribution matches the true label distribution (Hastie et al., 2001), namely
Revised: Gibbs’s inequality it is easy to show that the NLL loss will only be minimized when the assigned label distribution matches the true label distribution (Hastie et al., 2001), namely

Original: [Equation]
Revised: [Equation]

Original: Therefore, we propose to find the optimal T and λ as
Revised: Therefore, we propose to find the optimal T and λ as

Original: [Equation]
Revised: [Equation]

Original: 5.3 Rectified model probability mitigates double descent
Revised: 5.3 Rectified model probability mitigates robust overfitting

Original: [Figure] [Equation] 
Revised: [Equation] [Figure]

Original: [Figure]
Revised: N LL ( V a l ) 1010 1 T R o b u s t T e s t A cc ( % )

Original: [Equation]
Revised: Best Checkpoint Last Checkpoint 0.5 0.6 0.7 0.8 0.9 1.0 46.

Original: [Figure]
Revised: R o b u s t

Original:  We now work on a realistic dataset (CIFAR-10)to demonstrate the rectified model probabilitycan effectively mitigate the robust overfitting, orequivalently the epoch-wise double descent inadversarial training.
Revised: T e s t A cc ( % ) Best Checkpoint Last Checkpoint We now work on a realistic dataset (CIFAR-10) to demonstrate the rectified model probability can effectively mitigate the robust overfitting, or equivalently the epoch-wise double descent in adversarial training.

Original: The outer minimization ofadversarial training (Equation (1)) now becomes
Revised: The outer minimization of adversarial training (Equation (1)) now becomes

Original: [Equation]
Revised: [Equation]

Original: where ˆ θ denotes the parameters of a classifieradversarially trained beforehand.
Revised: where ˆ θ denotes the parameters of a classifier adversarially trained beforehand.

Original: The detailsof the experimental setting are available in the
Revised: The details of the experimental setting are available in the

Original: Appendix.
Revised: Appendix.

Original: As shown in Figure 6, adversarial training onrectified model probability can mitigate the robust overfitting when the temperature T and interpolation ratio λ are optimal.
Revised: As shown in Figure 6, adversarial training on rectified model probability can mitigate the robust overfitting when the temperature T and interpolation ratio λ are optimal.

Original: Such optimalhyperparameters perfectly aligns with the onesautomatically determined by Equation (9).
Revised: Such optimal hyperparameters perfectly aligns with the ones automatically determined by Equation (9).

Original:  ExperimentsExperiment setup.
Revised:  Experiments

Revised: Experiment setup.

Original: We conduct experiments on three datasets including CIFAR-10, CIFAR100 (Krizhevsky, 2009) and Tiny-ImageNet (Le & Yang, 2015).
Revised: We conduct experiments on three datasets including CIFAR-10, CIFAR100 (Krizhevsky, 2009) and Tiny-ImageNet (Le & Yang, 2015).

Original: We conduct PGD training on preactivation ResNet-18 (He et al., 2016) with 10 iterations and perturbation radius 8 / 255 by default.
Revised: We conduct PGD training on preactivation ResNet-18 (He et al., 2016) with 10 iterations and perturbation radius 8 / 255 by default.

Original: Weevaluate robustness against ℓ ∞ norm-bounded adversarial attack with perturbation radius 8 / 255 , andemploy AutoAttack (Croce & Hein, 2020) for reliable evaluation.
Revised: We evaluate robustness against ℓ ∞ norm-bounded adversarial attack with perturbation radius 8 / 255 , and employ AutoAttack (Croce & Hein, 2020) for reliable evaluation.

Original: Appendix D.2 includes results onadditional model architectures (e.g., VGG (Simonyan & Zisserman, 2015), WRN), adversarial trainingmethods (e.g., TRADES (Zhang et al., 2019), FGSM (Goodfellow et al., 2015)), and evaluationmetrics (e.g., PGD-1000 (PGD attack with 1000 iterations), Square Attack (Andriushchenko et al.
Revised: Appendix D.2 includes results on additional model architectures (e.g., VGG (Simonyan & Zisserman, 2015), WRN), adversarial training [Figure] Tiny [Figure]

Revised: ImageNet

Revised: [Figure]

Original: ,2020), RayS (Chen & Gu, 2020)).
Revised: [Table] CIFARAT - [Table] methods (e.g., TRADES (Zhang et al., 2019), FGSM (Goodfellow et al., 2015)), and evaluation metrics (e.g., PGD-1000 (PGD attack with 1000 iterations), Square Attack (Andriushchenko et al., 2020), RayS (Chen & Gu, 2020)).

Original: More setup details can be found in Appendix G.
Revised: More setup details can be found in Appendix G.

Original: Results & Discussions.
Revised: Results & Discussions.

Original: Our method is essentially the baseline adversarial training with a robusttrained self-teacher, equipped with an algorithm automatically deciding the optimal hyper-parameters,which we now denote as KD-AT-Auto.
Revised: Our method is essentially the baseline adversarial training with a robusttrained self-teacher, equipped with an algorithm automatically deciding the optimal hyper-parameters, which we now denote as KD-AT-Auto.

Original: We compare KD-AT-Auto with two baselines: regularadversarial training (AT), and adversarial training combined with self-distillation (KD-AT) with fixedtemperature T = 2 and interpolation ratio λ = 0 .
Revised: We compare KD-AT-Auto with two baselines: regular adversarial training (AT), and adversarial training combined with self-distillation (KD-AT) with fixed temperature T = 2 and interpolation ratio λ = 0 .

Original: 5 as suggested by Chen et al.
Revised: 5 as suggested by Chen et al.

Original: [Figure]

Original: ImageNet

Original: [Figure]

Original: Dataset

Original: Setting

Original: [Table]

Original: CIFARAT -

Original: [Table]

Original: [Table]

Original: KD

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: KD

Original: [Table]

Original: AT

Original: [Table]

Original: Auto 1 .

Original: [Table]

Original: [Table]

Original: [Table] CIFARAT - - [Table] As shown in Figure 7, our method can effectively mitigate robust overfitting for all datasets, withboth standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training.
Revised: As shown in Figure 7, our method can effectively mitigate robust overfitting for all datasets, with both standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training.

Original: In Table 1, we measure the difference between the RA at the best checkpoint (Best) and at the lastcheckpoint (Last) to clearly show the overfitting gap.
Revised: In Table 1, we measure the difference between the RA at the best checkpoint (Best) and at the last checkpoint (Last) to clearly show the overfitting gap.

Original: Our method can reduce the overfitting gap to lessthan 0 .
Revised: Our method can reduce the overfitting gap to less than 0 .

Original: 5% for all datasets.
Revised: 5% for all datasets.

Original: One may note that self-distillation with fixed hyper-parameters is in factinferior in terms of reducing robust overfitting, while its effectiveness can be significantly improvedwith the optimal hyper-parameters automatically determined by our method, which further verifiesour understanding of robust overfitting.
Revised: One may note that self-distillation with fixed hyper-parameters is in fact inferior in terms of reducing robust overfitting, while its effectiveness can be significantly improved with the optimal hyper-parameters automatically determined by our method, which further verifies our understanding of robust overfitting.

Original: Compared with self-distillation with fixed hyper-parameters,our method can also boost both RA and SA at the best checkpoint for all datasets.
Revised: Compared with self-distillation with fixed hyper-parameters, our method can also boost both RA and SA at the best checkpoint for all datasets.

Original: Our method can further be combined with orthogonal techniques such as Stochastic Weight Averaging(SWA) (Izmailov et al., 2018) and additional standard teachers as mentioned in previous work (Chenet al., 2021) to achieve better performance.
Revised: Our method can further be combined with orthogonal techniques such as Stochastic Weight Averaging (SWA) (Izmailov et al., 2018) and additional standard teachers as mentioned in previous work (Chen et al., 2021) to achieve better performance.

Original: More results and discussion can be found in Appendix D.3.
Revised: More results and discussion can be found in Appendix D.3.

Original:  Conclusion and Discussions
Revised:  Conclusion and Discussions

Original: In this paper, we show that label noise exists implicitly in adversarial training due to the mismatchbetween the true label distribution and the assigned label distribution of adversarial examples.
Revised: In this paper, we show that label noise exists implicitly in adversarial training due to the mismatch between the true label distribution and the assigned label distribution of adversarial examples.

Original: Suchlabel noise can explain the dominant overfitting phenomenon.
Revised: Such label noise can explain the dominant overfitting phenomenon.

Original: Based on a label noise perspective, wealso extend the understanding of robust overfitting and show that it is the early part of an epoch-wisedouble descent in adversarial training.
Revised: Based on a label noise perspective, we also extend the understanding of robust overfitting and show that it is the early part of an epoch-wise double descent in adversarial training.

Original: Finally, we propose an alternative labeling of adversarialexamples by rectifying model probability, which can effectively mitigate robust overfitting withoutany manual hyper-parameter tuning.
Revised: Finally, we propose an alternative labeling of adversarial examples by rectifying model probability, which can effectively mitigate robust overfitting without any manual hyper-parameter tuning.

Original: The label noise implicitly exists in adversarial training may have other important effects on adversarially robust learning.
Revised: The label noise implicitly exists in adversarial training may have other important effects on adversarially robust learning.

Original: This can potentially consolidate the theoretical grounding of robust learning.
Revised: This can potentially consolidate the theoretical grounding of robust learning.

Original: For instance, since label noise induces model variance, from a model-wise view, one may need toincrease model capacity to reduce the variance.
Revised: For instance, since label noise induces model variance, from a model-wise view, one may need to increase model capacity to reduce the variance.

Original: This may partially explain why robust generalizationrequires significantly larger model than standard generalization.
Revised: This may partially explain why robust generalization requires significantly larger model than standard generalization.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .•
Revised: [Yes] See Section ?? .

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data are proprietary.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section B(c)
Revised: [Yes] See Section B (c)

Original: Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeSection B(d)
Revised: [Yes] See Section B

Original:  Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Proofs 
Revised: A Proofs A.1 Preliminaries

Original: A.1 Preliminaries

Original: Proof in Assumption 3.1.
Revised: Proof in Assumption 3.1.

Original: Here we prove that if there no label error in the clean dataset, then
Revised: Here we prove that if there no label error in the clean dataset, then

Original: P ( ˜ Y | x ) =
Revised: P ( ˜ Y | x ) =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: Proof.
Revised: Proof.

Original: First, we note that
Revised: First, we note that

Original: [Equation]
Revised: [Equation]

Original: Since P ( E = 1 | Y =
Revised: Since P ( E = 1 | Y =

Original: j, x ) = 

Revised: j, x ) =

Original: we have,
Revised: we have,

Original: P

Original: [Equation]
Revised: [Equation]

Original: Therefore we have for all j ′ ,
Revised: Therefore we have for all j ′ ,

Original: P ( ˜ Y =

Original: j ′ | x

Original: [Equation]
Revised: [Equation]

Original: =

Original: P ( Y =

Original: j ′ | x ) .

Original: Total Variation distance for discrete probability distributions.
Revised: Total Variation distance for discrete probability distributions.

Original: For two discrete probability distributions P ( Y ) and P ( Y ′ ) where Y, Y ′ ∈ Y , the total variation distance between them can be equally defined as
Revised: For two discrete probability distributions P ( Y ) and P ( Y ′ ) where Y, Y ′ ∈ Y , the total variation distance between them can be equally defined as

Original: [Equation]
Revised: [Equation]

Original: A.2 Proofs in Section [-4.
-]
Revised: A.2 Proofs in Section [-4.
-]4.

Original: Proof of Lemma 4.1.
Revised: Proof of Lemma 4.1.

Original: Proof.
Revised: Proof.

Original: For simplicity, we consider the adversarial perturbation generated by FGSM.
Revised: For simplicity, we consider the adversarial perturbation generated by FGSM.

Original: Other adversarialperturbation can be viewed as a Taylor series of such perturbation.
Revised: Other adversarial perturbation can be viewed as a Taylor series of such perturbation.

Original: [Equation]
Revised: [Equation]

Original: First, we bound the distribution mismatch by gradient norm.
Revised: First, we bound the distribution mismatch by gradient norm.

Original: [Equation]
Revised: [Equation]

Original: TV distance
Revised: TV distance

Original: [Equation]
Revised: [Equation]

Original: Bounded Hessian
Revised: Bounded Hessian

Original: [Equation]
Revised: [Equation]

Original: Now if ∥ · ∥ = ∥ · ∥ 2 , we have
Revised: Now if ∥ · ∥ = ∥ · ∥ 2 , we have

Original: [Equation]
Revised: [Equation]

Original: If ∥ · ∥ = ∥ · ∥ ∞ , we can utilize the fact that ∥ · ∥ ∞ ≤ ∥ · ∥ 2 ≤ √ d ∥ · ∥ ∞ , thus
Revised: If ∥ · ∥ = ∥ · ∥ ∞ , we can utilize the fact that ∥ · ∥ ∞ ≤ ∥ · ∥ 2 ≤ √ d ∥ · ∥ ∞ , thus

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Second, we bound the gradient norm by the L -local Lipschitzness assumption.
Revised: Second, we bound the gradient norm by the L -local Lipschitzness assumption.

Original: Let x ∗ be a closest input that achieves the local maximum on the predicted probability at y , namely x ∗ = arg min z ∈ X,f ( z ) y
Revised: Let x ∗ be a closest input that achieves the local maximum on the predicted probability at y , namely x ∗ = arg min z ∈ X,f ( z ) y

Original: =1 ∥ x − z ∥ .
Revised: =1 ∥ x − z ∥ .

Original: Because x ∗ is the local maximum and f is continuously differentiable, ∇ f ( x ∗ )
Revised: Because x ∗ is the local maximum and f is continuously differentiable, ∇ f ( x ∗ )

Original: y = 0 , thus∇ f ( x )
Revised: y = 0 , thus

Original: [Equation]
Revised: [Equation]

Original: Therefore we have
Revised: Therefore we have

Original: [Equation]
Revised: [Equation]

Original: Plug this into Equation (12) or Equation (13) we then obtain the desired result.
Revised: Plug this into Equation (12) or Equation (13) we then obtain the desired result [Equation]

Original: Proof of Lemma 4.2.
Revised: Proof of Lemma 4.2.

Original: Proof.
Revised: Proof.

Original: First, we show that the expectation of the label error is lower bounded by the mismatchbetween the true label distribution and the assigned label distribution.
Revised: First, we show that the expectation of the label error is lower bounded by the mismatch between the true label distribution and the assigned label distribution.

Original: [Equation]
Revised: [Equation]

Original: Second, given a sampled training set D = { ( x i , ˜ y i ) } i ∈
Revised: Second, given a sampled training set D = { ( x i , ˜ y i ) } i ∈

Original: [ N ] , the empirical measure of label error E should converge to its expectation almost surely, namely
Revised: [ N ] , the empirical measure of label error E should converge to its expectation almost surely, namely

Original: [Equation]
Revised: [Equation]

Original: Using standard concentration inequality such as Hoeffding’s inequality we have, with probability 1 − δ ,
Revised: Using standard concentration inequality such as Hoeffding’s inequality we have, with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: This implies
Revised: This implies

Original: [Equation]
Revised: [Equation]

Original: Since P ( E = 1) =
Revised: Since P ( E = 1) =

Original: E x P ( E = 1 | x ) , we have, with probability 1 − δ ,
Revised: E x P ( E = 1 | x ) , we have, with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: which means p e ( D ) > 0 as long as N is large.
Revised: which means p e ( D ) > 0 as long as N is large.

Original: Proof of Theorem 4.3.
Revised: Proof of Theorem 4.3.

Original: Proof.
Revised: Proof.

Original: First, by the fact that P ( ˜ Y ′ | x ′ )
Revised: First, by the fact that P ( ˜ Y ′ | x ′ )

Original: =
Revised: =

Original: P ( ˜ Y | x ) and P ( ˜ Y | x ) =
Revised: P ( ˜ Y | x ) and P ( ˜ Y | x ) =

Original: P ( Y | x ) we have P ( ˜ Y ′
Revised: P ( Y | x ) we have P ( ˜ Y ′

Original: | x ′ ) =
Revised: | x ′ ) =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: Therefore, apply Lemma 4.2 to an adversarially augmented training set we have with probability 1 − δ ,[Equation] δ
Revised: Therefore, apply Lemma 4.2 to an adversarially augmented training set we have with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: Further, apply Lemma 4.1 and the definition of data quality, we have with probability 1 − δ ,
Revised: Further, apply Lemma 4.1 and the definition of data quality, we have with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: A.3 Proofs in Section 4.
Revised: A.3 Proofs in Section 4.

Original: Proof of Lemma 4.4.
Revised: Proof of Lemma 4.4.

Original: Proof.
Revised: Proof.

Original: Let D =
Revised: Let D =

Original: ( x, y ) be the adversarially augmented training set.
Revised: ( x, y ) be the adversarially augmented training set.

Original: Let S
Revised: Let S

Original: = { x : ( x, y ) ∈ D} be511 the collection of all training inputs.
Revised: = { x : ( x, y ) ∈ D} be the collection of all training inputs.

Original: First we note that the set of all training inputs can be groupedinto several subsets such that the inputs in each subset possess similar true label distribution.
Revised: First we note that the set of all training inputs can be grouped into several subsets such that the inputs in each subset possess similar true label distribution.

Original: Moreformally, Let C
Revised: More formally, Let C

Original: = { ¯ x j } N ρε j =1 be an ρε -external covering of S with minimum cardinality, namely
Revised: = { ¯ x j } N ρε j =1 be an ρε -external covering of S with minimum cardinality, namely

Original: S ⊆ (cid:83) x ∈C { x ′ | ∥ x ′ − x ∥ ≤ ρε } , where we refer ¯ x j as the covering center, and N ρε is the covering515 number.
Revised: S ⊆ (cid:83) x ∈C { x ′ | ∥ x ′ − x ∥ ≤ ρε } , where we refer ¯ x j as the covering center, and N ρε is the covering number.

Original: Let {S j } N ρε j =1 be any disjoint partition of S such that S j ⊆ { x ′ |∥ x ′ − ¯ x j ∥ ≤ ρε } .
Revised: Let {S j } N ρε j =1 be any disjoint partition of S such that S j ⊆ { x ′ |∥ x ′ − ¯ x j ∥ ≤ ρε } .

Original: We show that
Revised: We show that

Original: S j attains a property that the true label distribution of any input x in this subset will not be too far from the sample mean of one-hot labels ¯y j =
Revised: S j attains a property that the true label distribution of any input x in this subset will not be too far from the sample mean of one-hot labels ¯y j =

Original: |S j | − 1 (cid:80) x ∈S j 1 y in this subset.
Revised: |S j | − 1 (cid:80) x ∈S j 1 y in this subset.

Original: Specifically, letp ( x ) =
Revised: Specifically, let p ( x ) =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: We have with probability 1 − δ ,
Revised: We have with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: To prove this property we first present two lemmas.
Revised: To prove this property we first present two lemmas.

Original: Lemma A.1 (Lipschitz constraint of the true label distribution) .
Revised: Lemma A.1 (Lipschitz constraint of the true label distribution) .

Original: Let S j be a subset constructed aboveand ¯y j
Revised: Let S j be a subset constructed above and ¯y j

Original: = |S j | − 1 (cid:80) x ∈S j 1 y .
Revised: = |S j | − 1 (cid:80) x ∈S j 1 y .

Original: Then for any x ∈ S j we have,
Revised: Then for any x ∈ S j we have,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: First, since x ∈ S j , we have ∥ x − ¯ x j ∥ 1 ≤ ρε , which implies ∥ p ( x ) − p (¯ x j ) ∥ 1 ≤ Lρε by the524 locally Lipschitz continuity of p .
Revised: First, since x ∈ S j , we have ∥ x − ¯ x j ∥ 1 ≤ ρε , which implies ∥ p ( x ) − p (¯ x j ) ∥ 1 ≤ Lρε by the locally Lipschitz continuity of p .

Original: Then for any x, x ′ ∈ S j , we will have ∥ p ( x ) − p ( x ′ )
Revised: Then for any x, x ′ ∈ S j , we will have ∥ p ( x ) − p ( x ′ )

Original: ∥ ≤ 2 Lρε by525 the triangle inequality.
Revised: ∥ ≤ 2 Lρε by the triangle inequality.

Original: Let N
Revised: Let N

Original: S =
Revised: S =

Original: |S j | .
Revised: |S j | .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: Further, the linearity of the expectation implies
Revised: Further, the linearity of the expectation implies

Original: [Equation]
Revised: [Equation]

Original: Therefore ∥ p ( x ) − E [ ¯y j ] ∥ ≤ 2 Lρε .
Revised: Therefore ∥ p ( x ) − E [ ¯y j ] ∥ ≤ 2 Lρε .

Original: Lemma A.2 (Concentration inequality of the sample mean) .
Revised: Lemma A.2 (Concentration inequality of the sample mean) .

Original: Let S be a set of x with cardinality N .
Revised: Let S be a set of x with cardinality N .

Original: Let ¯y =
Revised: Let ¯y =

Original: N − 1 (cid:80) x ∈S 1 y be the sample mean.
Revised: N − 1 (cid:80) x ∈S 1 y be the sample mean.

Original: Then for any p -norm ∥ · ∥ and any ε > 0 , we have with530 probability
Revised: Then for any p -norm ∥ · ∥ and any ε > 0 , we have with probability

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Proof.
Revised: Proof.

Original: Note that ¯y obeys a multinomial distribution, i.e. ¯y ∼ N − 1 multinomial
Revised: Note that ¯y obeys a multinomial distribution, i.e. ¯y ∼ N − 1 multinomial

Original: N, E
Revised: N, E

Original: [ ¯y ]) .
Revised: [ ¯y ]) .

Original: Thislemma is thus the classic result on the concentration properties of multinomial distribution based onℓ 1 norm Weissman et al.
Revised: This lemma is thus the classic result on the concentration properties of multinomial distribution based on ℓ 1 norm Weissman et al.

Original: ; Qian et al.
Revised: ; Qian et al.

Original: One can see that Lemma A.1 bounds the difference between true label distribution of individualinputs and the mean true label distribution, while Lemma A.2 bounds the difference between thesample mean and the mean true label distribution.
Revised: One can see that Lemma A.1 bounds the difference between true label distribution of individual inputs and the mean true label distribution, while Lemma A.2 bounds the difference between the sample mean and the mean true label distribution.

Original: Therefore the difference between the true labeldistribution and the sample mean is also bounded, since by the triangle inequality we have withprobability 1 − δ ,
Revised: Therefore the difference between the true label distribution and the sample mean is also bounded, since by the triangle inequality we have with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: We now show that given the locally Lipschitz constraint established in each disjoint partition weconstructed above, the prediction given by the empirical risk minimizer will be close to the samplemean.
Revised: We now show that given the locally Lipschitz constraint established in each disjoint partition we constructed above, the prediction given by the empirical risk minimizer will be close to the sample mean.

Original: As an example, we focus on the negative log-likelihood loss, namely ℓ ( f θ ( x ) , y )
Revised: As an example, we focus on the negative log-likelihood loss, namely ℓ ( f θ ( x ) , y )

Original: = − 1 y ·log f θ ( x ) .
Revised: = − 1 y · log f θ ( x ) .

Original: Other loss functions that are subject to the proper scoring rule can be investigated in asimilar manner.
Revised: Other loss functions that are subject to the proper scoring rule can be investigated in a similar manner.

Original: First, we regroup the sum in the empirical risk based on the partition constructedabove, namely
Revised: First, we regroup the sum in the empirical risk based on the partition constructed above, namely

Original: [Equation]
Revised: [Equation]

Original: where ˆ R ( f θ , S j ) = −|S
Revised: where ˆ R ( f θ , S j ) = −|S

Revised: [Equation]

Original: j | − 1 (cid:80) |S
Revised: | − [Equation] S

Original: j | i =1 1 y i · log f θ ( x i ) is the empirical risk in each partition.
Revised: [Equation] | [Equation] log f θ ( x i ) is the empirical risk in each partition.

Original: Since we546 are only concerned with the existence of a desired minimizer of the empirical risk, we can view f θ as able to achieve any labeling of the training inputs that suffices the local Lipschitz constraint.
Revised: Since we are only concerned with the existence of a desired minimizer of the empirical risk, we can view f θ as able to achieve any labeling of the training inputs that suffices the local Lipschitz constraint.

Original: Thus theempirical risk minimization is equivalent to the minimization of the empirical risk in each partition.
Revised: Thus the empirical risk minimization is equivalent to the minimization of the empirical risk in each partition.

Original: The problem can thus be defined as, for each j = 1 , · · · , N [Equation]
Revised: The problem can thus be defined as, for each j = 1 , · · · , N ρε ,

Original: [Equation]
Revised: [Equation]

Original: 22) where the constraint is imposed by the locally-Lipschitz continuity of f θ .
Revised: where the constraint is imposed by the locally-Lipschitz continuity of f θ .

Original: By the following lemma,we show that the minimizer of such problem is achieved only if f θ (¯ x j ) is close to the sample mean.
Revised: By the following lemma, we show that the minimizer of such problem is achieved only if f θ (¯ x j ) is close to the sample mean.

Original: Lemma A.3.
Revised: Lemma A.3.

Original: Let ¯ y =
Revised: Let ¯ y =

Original: |S j | − 1 (cid:80) x ∈S j 1 y .
Revised: |S j | − 1 (cid:80) x ∈S j 1 y .

Original: The minimum of the problem (22) is achieved only if
Revised: The minimum of the problem (22) is achieved only if

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We note that since the loss function we choose is strongly convex, to minimize the empirical risk, the prediction of any input x must be as close to the one-hot labeling as possible.
Revised: We note that since the loss function we choose is strongly convex, to minimize the empirical risk, the prediction of any input x must be as close to the one-hot labeling as possible.

Original: Therefore the problem (22) can be formulated into a vector minimization where we can employ
Revised: Therefore the problem (22) can be formulated into a vector minimization where we can employ

Original: Karush–Kuhn–Tucker (KKT) theorem to find the necessary conditions of the minimizer.
Revised: Karush–Kuhn–Tucker (KKT) theorem to find the necessary conditions of the minimizer.

Original: Let p i :
Revised: Let p i :

Original: =
Revised: =

Original: f θ ( x i ) and ˜ ε =
Revised: f θ ( x i ) and ˜ ε =

Original: Lρε for simplicity.
Revised: Lρε for simplicity.

Original: We rephrase the problem (22) asmin
Revised: We rephrase the problem (22) as

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Case I.
Revised: Case I.

Original: We first discuss the case when p k + ˜ ε < 1 for all k .
Revised: We first discuss the case when p k + ˜ ε < 1 for all k .

Original: First, we observe that for any p ,the minimum of the above problem is achieved only if p y i i
Revised: First, we observe that for any p , the minimum of the above problem is achieved only if p y i i

Original: = p y i + ˜ ε .
Revised: = p y i + ˜ ε .

Original: Because by contradiction, ifp y i i < p
Revised: Because by contradiction, if p y i i < p

Original: y i + ˜ ε , we will have − log p
Revised: y i + ˜ ε , we will have − log p

Original: y i i > − log( p y i + ˜ ε ) , and p y i + ˜ ε belongs to the feasible set,which means p y
Revised: y i i > − log( p y i + ˜ ε ) , and p y i + ˜ ε belongs to the feasible set, which means p y

Original: i i does not attain the minimum.
Revised: i i does not attain the minimum.

Original: The above problem can then be rephrased as
Revised: The above problem can then be rephrased as

Original: [Equation]
Revised: [Equation]

Original: where we have neglected the condition associated with p k ̸
Revised: where we have neglected the condition associated with p k ̸

Original: =
Revised: =

Original: y i
Revised: y i

Original: i , since they do not contribute to theobjective, they can be chosen arbitrarily as long as the constraints are sufficed, and clearly theconstraints are underdetermined.
Revised: i , since they do not contribute to the objective, they can be chosen arbitrarily as long as the constraints are sufficed, and clearly the constraints are underdetermined.

Original: Let N k
Revised: Let N k

Original: = (cid:80) i 1( y i = k ) , we have (cid:80) i log( p y i + ˜ ε ) =
Revised: = (cid:80) i 1( y i = k ) , we have (cid:80) i log( p y i + ˜ ε ) =

Original: (cid:80) k N k log( p k + ˜ ε ) .
Revised: (cid:80) k N k log( p k + ˜ ε ) .

Original: Therefore the aboveproblem is equivalent to
Revised: Therefore the above problem is equivalent to

Original: [Equation]
Revised: [Equation]

Original: where ¯y ≡ [ N 1 /N, · · · , N k /N ]
Revised: where ¯y ≡ [ N 1 /N, · · · , N k /N ]

Original: T is equal to the sample mean N − 1 (cid:80)
Revised: T is equal to the sample mean N − 1 (cid:80)

Original: [Equation]

Original:  y i [Equation]
Revised: i 1 y i .

Original: To solve the strongly convex minimization problem (25) it is easy to employ KKT conditions to show that
Revised: To solve the strongly convex minimization problem (25) it is easy to employ KKT conditions to show that

Original: [Equation]
Revised: [Equation]

Original: Case II.
Revised: Case II.

Original: We now discuss the case when ˆp is the minimizer of (23) and there exists k ′ such thatˆp k ′ + ˜ ε ≥ 1 .
Revised: We now discuss the case when ˆp is the minimizer of (23) and there exists k ′ such that ˆp k ′ + ˜ ε ≥ 1 .

Original: And ˆp ̸ = p , where p = ¯y (1 + K ˜ ε ) − ˜ ε is the form of the minimizer in the previouscase.
Revised: And ˆp ̸ = p , where p = ¯y (1 + K ˜ ε ) − ˜ ε is the form of the minimizer in the previous case.

Original: Considering a non-trivial case p ∗ k ′ < 1 − ˜ ε .
Revised: Considering a non-trivial case p ∗ k ′ < 1 − ˜ ε .

Original: Otherwise the true label distribution is already close tothe one-hot labeling, which is the minimizer of the empirical risk.
Revised: Otherwise the true label distribution is already close to the one-hot labeling, which is the minimizer of the empirical risk.

Original: Therefore by (cid:80) k ̸ = k ′ p k > ˜ ε wehave the condition
Revised: Therefore by (cid:80) k ̸ = k ′ p k > ˜ ε we have the condition

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Now considering the minimization objective R ( p ) =
Revised: Now considering the minimization objective R ( p ) =

Original: − N − 1 (cid:80)
Revised: − N − 1 (cid:80)

Original: i 1 y i · log p i .
Revised: i 1 y i · log p i .

Original: For all i with y i
Revised: For all i with y i

Original: =
Revised: =

Original: k ′ , wemust have p
Revised: k ′ , we must have p

Original: y i
Revised: y i

Original: i = 1 , otherwise the optimal cannot be attained by contradiction.
Revised: i = 1 , otherwise the optimal cannot be attained by contradiction.

Original: Then the minimizationproblem can be rephrased as
Revised: Then the minimization problem can be rephrased as

Original: [Equation]
Revised: [Equation]

Original: where the first constraint is imposed by ˆp k ′ ≥ 1 − ˜ε .
Revised: where the first constraint is imposed by ˆp k ′ ≥ 1 − ˜

Revised: [Equation]

Original: Employ KKT conditions similarly we can have ˆp k = ¯y k /λ − ˜ ε where λ is a constant.
Revised: Employ KKT conditions similarly we can have ˆp k = ¯y k /λ − ˜ ε where λ is a constant.

Original: By checkingthe constraint we can derive λ ≥ (cid:80) k ¯y k / ( K ˜ ε ) .
Revised: By checking the constraint we can derive λ ≥ (cid:80) k ¯y k / ( K ˜ ε ) .

Original: However, the minimization objective
Revised: However, the minimization objective

Original: [Equation]
Revised: [Equation]

Original: requires λ to be minimized.
Revised: requires λ to be minimized.

Original: Therefore λ = (cid:80)
Revised: Therefore λ = (cid:80)

Original: k ̸ = k ′ ¯y k 
Revised: k ̸ = k ′ ¯y k / ( K ˜ ε ) , which implies

Original: [Equation]
Revised: [Equation]

Original: ˜ ε ) , which implies

Original: [Equation]
Revised: [Equation]

Original: Now since ˆp = arg min p R ( p ) and ˆp ̸ = p , we must have R ( ˆp ) <
Revised: Now since ˆp = arg min p R ( p ) and ˆp ̸ = p , we must have R ( ˆp ) <

Original: R ( p ) .
Revised: R ( p ) .

Original: This means−
Revised: This means −

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 29) which is reduced to 
Revised: which is reduced to (cid:88)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: But this is contradict to our assumption.
Revised: But this is contradict to our assumption.

Original: We are now be able to bound the difference between the predictions of the training inputs produced bythe empirical risk minimizer and the sample mean in each S j .
Revised: We are now be able to bound the difference between the predictions of the training inputs produced by the empirical risk minimizer and the sample mean in each S j .

Original: To see that we have for each x ∈ S j .
Revised: To see that we have for each x ∈ S j .

Original: [Equation]
Revised: [Equation]

Original: By Equation (15) we then have for any x ∈ S j , with probability 1 − δ ,
Revised: By Equation (15) we then have for any x ∈ S j , with probability 1 − δ ,

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which means the difference between the predictions and the true label distribution is also bounded.
Revised: which means the difference between the predictions and the true label distribution is also bounded.

Original: Step III:
Revised: Step III:

Original: Show the disjoint partition is non-trivial.
Revised: Show the disjoint partition is non-trivial.

Original: In (32), we have managed to boundthe difference between the predictions yielded by an empirical risk minimizer and the true labeldistribution based on the cardinality of the subset |S j | , namely the number of inputs in j -partition.
Revised: In (32), we have managed to bound the difference between the predictions yielded by an empirical risk minimizer and the true label distribution based on the cardinality of the subset |S j | , namely the number of inputs in j -partition.

Original: However |S j | is critical to the bound here as if |S j | = 1 , then (32) becomes a trivial bound.
Revised: However |S j | is critical to the bound here as if |S j | = 1 , then (32) becomes a trivial bound.

Original: Here we594 show |S j | is non-negligible based on simple combinatorics.
Revised: Here we show |S j | is non-negligible based on simple combinatorics.

Original: Lemma A.4.
Revised: Lemma A.4.

Original: Let {S j } N ρε j =1 be a disjoint partition of the entire training set S .
Revised: Let {S j } N ρε j =1 be a disjoint partition of the entire training set S .

Original: Denote S ( x ) as thepartition that includes x . Let N ( x ) =
Revised: Denote S ( x ) as the partition that includes x . Let N ( x ) =

Original: |S ( x ) | and N = |S| .
Revised: |S ( x ) | and N = |S| .

Original: Then for any κ ≥ 1 ,(cid:12)(cid:12)(cid:12)(cid:12)(cid:26) x | N ( x ) ≥ N [Equation] (cid:27)(cid:12)(cid:12)(cid:12)(cid:12) ≥ (cid:18) 1 − κ
Revised: Then for any κ ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We note that the problem is to show the minimum number of x such that N ( x ) ≥ N/ ( κN ρε ) .
Revised: We note that the problem is to show the minimum number of x such that N ( x ) ≥ N/ ( κN ρε ) .

Original: This is equivalent to find the maximum number of x such that N ( x ) ≤ N/ ( κN ρε ) .
Revised: This is equivalent to find the maximum number of x such that N ( x ) ≤ N/ ( κN ρε ) .

Original: Since we only have
Revised: Since we only have

Original: N ρε subsets, the maximum can be attained only if for N ρε − 1 subsets S , |S| = N/ ( κN ρε ) .
Revised: N ρε subsets, the maximum can be attained only if for N ρε − 1 subsets S , |S| = N/ ( κN ρε ) .

Original: Otherwise,600 if for any one of these subsets |S| < N/ ( κN ρε ) , then it is always feasible to let |S| = N/ ( κN ρε )601 and the maximum increases.
Revised: Otherwise, if for any one of these subsets |S| < N/ ( κN ρε ) , then it is always feasible to let |S| = N/ ( κN ρε ) and the maximum increases.

Original: Similarly, if the number of such subsets is less than N ρε − 1 , then it is602 always feasible to let another subset subject to |S| = N/ ( κN ρε ) and the maximum increases.
Revised: Similarly, if the number of such subsets is less than N ρε − 1 , then it is always feasible to let another subset subject to |S| = N/ ( κN ρε ) and the maximum increases.

Original: We can603 then conclude that at most N ( N ρε − 1) / ( κN ρε ) inputs can have the property N ( x ) ≤ N/ ( κN ρε ) .
Revised: We can then conclude that at most N ( N ρε − 1) / ( κN ρε ) inputs can have the property N ( x ) ≤ N/ ( κN ρε ) .

Original: The above lemma basically implies when partitioning N inputs into N ρε subsets, a large fraction ofthe inputs will be assigned to a subset with cardinality at least N/ ( κN ρε ) .
Revised: The above lemma basically implies when partitioning N inputs into N ρε subsets, a large fraction of the inputs will be assigned to a subset with cardinality at least N/ ( κN ρε ) .

Original: Here N ρε is the coveringnumber and is bounded above based on the property of the covering in the Euclidean space.
Revised: Here N ρε is the covering number and is bounded above based on the property of the covering in the Euclidean space.

Original: Apply
Revised: Apply

Original: Lemma A.4 to (32), and use the fact that ∥ · ∥ TV = ∥ · ∥ 1 / 2 for category distributions, we then arrive609 at Lemma 4.4.
Revised: Lemma A.4 to (32), and use the fact that ∥ · ∥ TV = ∥ · ∥ 1 / 2 for category distributions, we then arrive at Lemma 4.4.

Original: Proof of Theorem 4.5.
Revised: Proof of Theorem 4.5.

Original: Proof.
Revised: Proof.

Original: First, we show that adversarial perturbation generated by a realistic classifier can change itspredictive distribution.
Revised: First, we show that adversarial perturbation generated by a realistic classifier can change its predictive distribution.

Original: Considering adversarial perturbation based on FGSM and cross-entropy loss,namely x ′ = x − ε ∥∇ f θ ( x ) y ∥
Revised: Considering adversarial perturbation based on FGSM and cross-entropy loss, namely x ′ = x − ε ∥∇ f θ ( x ) y ∥

Original: − 1 ∇ f θ ( x ) y , we can obtain a result similar to Lemma 4.1.
Revised: − 1 ∇ f θ ( x ) y , we can obtain a result similar to Lemma 4.1.

Original: Lemma A.5.
Revised: Lemma A.5.

Original: Assume f θ ( x ) y is L θ -locally
Revised: Assume f θ ( x ) y is L θ -locally

Original: Lipschitz around x with bounded Hessian.
Revised: Lipschitz around x with bounded Hessian.

Original: Let σ m =
Revised: Let σ m =

Original: inf
Revised: inf

Original: z ∈B ε ( x ) σ min ( ∇ 2 f θ ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f θ ( z ) y ) > 0 .
Revised: z ∈B ε ( x ) σ min ( ∇ 2 f θ ( z ) y ) > 0 and σ M = sup z ∈B ε ( x ) σ max ( ∇ 2 f θ ( z ) y ) > 0 .

Original: Here σ min and σ maxdenote the minimum and maximum eigenvalues of the Hessian, respectively.
Revised: Here σ min and σ max denote the minimum and maximum eigenvalues of the Hessian, respectively.

Original: We then have
Revised: We then have

Original: [Equation]
Revised: [Equation]

Original: Second, We prove that the true label distribution will be distorted by the adversarial perturbationgenerated by a realistic classifier.
Revised: Second, We prove that the true label distribution will be distorted by the adversarial perturbation generated by a realistic classifier.

Original: This is guaranteed if the predictive distribution of a realisticclassifier can approximate the true label distribution.
Revised: This is guaranteed if the predictive distribution of a realistic classifier can approximate the true label distribution.

Original: Specifically, by utilizing Lemma A.5 and
Revised: Specifically, by utilizing Lemma A.5 and

Original: Lemma 4.4, we have with probability 1 − 2 δ ,
Revised: Lemma 4.4, we have with probability 1 − 2 δ ,

Original: [Equation]
Revised: [Equation]

Original: Finally, we show that such distribution mismatch induces label noise in the adversarially augmentedtraining set.
Revised: Finally, we show that such distribution mismatch induces label noise in the adversarially augmented training set.

Original: Similar to the proof for the true classifier, by the common labeling practice of adversarialexamples we have P ( ˜ Y ′
Revised: Similar to the proof for the true classifier, by the common labeling practice of adversarial examples we have P ( ˜ Y ′

Original: | x ′ )
Revised: | x ′ )

Original: =
Revised: =

Original: P ( ˜ Y | x ) =
Revised: P ( ˜ Y | x ) =

Original: P ( Y | x ) .
Revised: P ( Y | x ) .

Original: By utilizing Lemma 4.2 1 we then have withprobability 1 − 3 δ ,
Revised: By utilizing Lemma 4.2 1 we then have with probability 1 − 3 δ ,

Original: [Equation]
Revised: [Equation]

Original: where ξ = 1 + (cid:112) 4 κN ρε K .
Revised: where ξ = 1 + (cid:112) 4 κN ρε K .

Original: A.4 Proofs in[-Section 
-]
Revised: A.4 Proofs in[-Section 
-]

Original: Proof of Theorem 5.1.
Revised: Proof of Theorem 5.1.

Original: Proof.
Revised: Proof.

Original: Let j ∗ =
Revised: Let j ∗ =

Original: argmax P ( Y ′ =
Revised: argmax P ( Y ′ =

Original: j | x ′ ) and thus P ( Y ′ =
Revised: j | x ′ ) and thus P ( Y ′ =

Original: j ∗ | x ′ ) ∈
Revised: j ∗ | x ′ ) ∈

Original: [1 /c, 1] .
Revised: [1 /c, 1] .

Original: Letg ( T ) :
Revised: Let g ( T ) :

Original: = f ( x ′ ; θ, T ) j ∗ , which is a continuous function defined on [0 , ∞ ] .
Revised: = f ( x ′ ; θ, T ) j ∗ , which is a continuous function defined on [0 , ∞ ] .

Original: The condition j ∗ =arg max j f ( x ′ ; θ, T ) j ensures that g ( T ) ∈
Revised: The condition j ∗ = arg max j f ( x ′ ; θ, T ) j ensures that g ( T ) ∈

Original: [1 /c, 1] , where c is the number of classes.
Revised: [1 /c, 1] , where c is the number of classes.

Original: By the intermediate value theorem, there exists T ∗ , such that g ( T ∗ ) =
Revised: By the intermediate value theorem, there exists T ∗ , such that g ( T ∗ ) =

Original: P ( Y ′ =
Revised: P ( Y ′ =

Original: j ∗ | x ′ ) .
Revised: j ∗ | x ′ ) .

Original: Let T
Revised: Let T

Original: =
Revised: =

Original: T ∗ , we have
Revised: T ∗ , we have

Original: [Equation]
Revised: [Equation]

Original: where the inequality holds by the triangle inequality.
Revised: where the inequality holds by the triangle inequality.

Original: Meanwhile, we have
Revised: Meanwhile, we have

Original: [Equation]

Original: | x

Original: [Equation]
Revised: [Equation]

Original: Therefore, it can seen that for T
Revised: Therefore, it can seen that for T

Original: =
Revised: =

Original: T ∗ ,
Revised: T ∗ ,

Original: [Equation]
Revised: [Equation]

Original: Proof of Theorem 5.2.
Revised: Proof of Theorem 5.2.

Original: Lemma A.6.
Revised: Lemma A.6.

Original: Let x ′ be an example incorrectly classified by a classifier f
Revised: Let x ′ be an example incorrectly classified by a classifier f

Original: in terms of the true label distribution P ( Y ′ =
Revised: in terms of the true label distribution P ( Y ′ =

Original: j | x ′ ) , namely
Revised: j | x ′ ) , namely

Original: [Equation]
Revised: [Equation]

Original: where j ∗ = arg max j P
Revised: where j ∗ = arg max j P

Original: ( Y ′ =
Revised: ( Y ′ =

Original: j | x ′ ) .
Revised: j | x ′ ) .

Original: Assume P
Revised: Assume P

Original: ( Y ′ =
Revised: ( Y ′ =

Original: j ∗ | x ′ ) ≥ 1 / 2 , then 
Revised: j ∗ | x ′ ) ≥ 1 / 2 , then f ( x ′ ; θ, T

Original: [Equation]
Revised: [Equation]

Revised: j ∗ ≤ P

Revised: Y

Original: [Equation]
Revised: [Equation]

Revised: = j ∗ | x ′ ) .

Original: Proof.
Revised: Proof.

Original: We prove it by contradiction.
Revised: We prove it by contradiction.

Original: Assume f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =
Revised: Assume f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =

Original: j ∗ | x ′ ) , we have f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =
Revised: j ∗ | x ′ ) , we have f ( x ′ ; θ, T ) j ∗ > P ( Y ′ =

Original: j ∗ | x ′ ) ≥ 1 / 2 .
Revised: j ∗ | x ′ ) ≥ 1 / 2 .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original:  which means f ( x ′ ; θ, T )
Revised: = [Equation] ∗ f ( x ′ ; θ, T ) j = 1 − f ( x ′ ; θ, T ) j ∗ < 1 / 2 , ∀ j ̸ = j ∗ , which means f ( x ′ ; θ, T )

Original: j < f ( x ′ ; θ, T ) j ∗ , ∀ j ̸
Revised: j < f ( x ′ ; θ, T ) j ∗ , ∀ j ̸

Original: = j ∗ .
Revised: = j ∗ .

Original: This leads to j ∗ = arg max j f ( x ′ ; θ, T ) j ,638 which contradicts our condition.
Revised: This leads to j ∗ = arg max j f ( x ′ ; θ, T ) j , which contradicts our condition.

Original: Now we prove Theorem [-5.
-]
Revised: Now we prove Theorem [-5.
-]5.

Original: Proof.
Revised: Proof.

Original: First let P ( Y ′ | x ′ )
Revised: First let P ( Y ′ | x ′ )

Original: =
Revised: =

Original: P ( y | x ) ≈ 1 ( y ) .
Revised: P ( y | x ) ≈ 1 ( y ) .

Original: Let j ∗
Revised: Let j ∗

Original: = arg max j P
Revised: = arg max j P

Original: ( Y ′ =
Revised: ( Y ′ =

Original: j | x ′ ) .
Revised: j | x ′ ) .

Original: By Lemma A.6 wehave f ( x ′ ; θ, T ) j ∗ ≤ P ( Y ′∗ =
Revised: By Lemma A.6 we have f ( x ′ ; θ, T ) j ∗ ≤ P ( Y ′∗ =

Original: j ∗ | x ′ ) ≤ 1 .
Revised: j ∗ | x ′ ) ≤ 1 .

Original: Then there exists λ ∗ > 0 , such that λ ∗ · f ( x ′ ; θ, T ) j ∗ +(1 − λ ∗ )
Revised: Then there exists λ ∗ > 0 , such that λ ∗ · f ( x ′ ; θ, T ) j ∗ + (1 − λ ∗ )

Original: =
Revised: =

Original: P ( Y ′ =
Revised: P ( Y ′ =

Original: j ∗ | x ′ ) by the intermediate value theorem.
Revised: j ∗ | x ′ ) by the intermediate value theorem.

Original: Let λ = λ ∗ , we have
Revised: Let λ = λ ∗ , we have

Original: [Equation]

Original: Y

Original: [Equation]

Original: j

Original: [Equation]

Original: |

Original: [Equation]
Revised: [Equation]

Original: |

Original: [Equation] T [Equation] j 
Revised: | f ( x ′ ; θ, T ) j ∗ − P

Original: [Equation]

Original: ( Y [Equation] j [Equation]
Revised: ( Y ′ =

Original: |

Original: [Equation] j [Equation] j  ̸
Revised: j ∗ | x ′ ) | ≤ (cid:88) j,j ̸

Original: [Equation]

Original: [ f ( [Equation] θ [Equation]
Revised: = j ∗ | λ · f ( x ′ ; θ, T ) j − f ( x ′ ; θ, T ) j | − | f ( x ′ ; θ, T ) j ∗ − P

Original: T [Equation] · f ( [Equation]
Revised: ( Y ′ =

Original:  T [Equation] [Equation] [ P ( Y ′ 
Revised: j ∗ | x ′ ) | = (cid:88) j,j ̸ = j ∗ [ f ( x ′ ; θ, T ) j − λ · f ( x ′ ; θ, T ) j ] − [ P ( Y ′ =

Original: [Equation]

Original: j [Equation] | x ′ )
Revised: j ∗ | x ′ )

Original: [Equation]

Original:  f ( x ′ [Equation] θ [Equation]
Revised: − f ( x ′ ; θ, T ) j ∗ ] =

Original: j

Original: [Equation]

Original: [Equation]

Original: (cid:88) j [Equation] j ̸
Revised: (cid:88) j,j ̸

Original: [Equation]

Original: j

Original: [Equation]

Original: [ f (

Original: [Equation]

Original: T

Original: [Equation]

Original: · f ( x

Original: [Equation]

Original: θ

Original: [Equation]

Original: T

Original: [Equation] j ] [Equation] [ λ · f ( x ′ 
Revised: = j ∗ [ f ( x ′ ; θ, T ) j − λ · f ( x ′ ; θ, T ) j ] − [ λ · f ( x ′ ; θ, T ) j ∗ + (1 − λ ) − f ( x ′ ; θ, T ) j ∗ ] = (cid:88)

Original: [Equation]

Original: θ [Equation] T ) j 
Revised: j f

Original: [Equation]

Original: + (1 − λ ) − f ( x ′ ; θ

Original: [Equation]

Original: T

Original: [Equation]
Revised: [Equation]

Original: j

Original: [Equation] [Equation] = [Equation]
Revised: = 0 .

Original: B Limitations
Revised: B Limitations

Original: We note that alternative labeling of adversarial examples proposed in this paper is based on thefact that the predictive distribution of a classifier trained with empirical risk minimization canapproximate the true label distribution of training examples.
Revised: We note that alternative labeling of adversarial examples proposed in this paper is based on the fact that the predictive distribution of a classifier trained with empirical risk minimization can approximate the true label distribution of training examples.

Original: However, such approximation maynot be accurate especially if the classifier is not carefully regularized during training.
Revised: However, such approximation may not be accurate especially if the classifier is not carefully regularized during training.

Original: Post-trainingconfidence calibration techniques such as temperature scaling and interpolation can only improvethe approximation in terms of the entire training set, but cannot improve it in a sample-wise manner.
Revised: Post-training confidence calibration techniques such as temperature scaling and interpolation can only improve the approximation in terms of the entire training set, but cannot improve it in a sample-wise manner.

Original: How to learn the true label distribution of adversarial training examples during adversarial trainingmore accurately remains an open problem.
Revised: How to learn the true label distribution of adversarial training examples during adversarial training more accurately remains an open problem.

Original: Also, such alternative labeling also requires to train another independent classifier beforehand, whichinduces additional training cost.
Revised: Also, such alternative labeling also requires to train another independent classifier beforehand, which induces additional training cost.

Original: C More empirical analyses 
Revised: C More empirical analyses C.1 Epoch-wise double descent is ubiquitous in adversarial training

Original: C.1 Epoch-wise double descent is ubiquitous in adversarial training

Original: In this section, we conduct extensive experiments with different model architectures, and learningrate schedulers to verify the connection between robust overfitting and epoch-wise double descent.
Revised: In this section, we conduct extensive experiments with different model architectures, and learning rate schedulers to verify the connection between robust overfitting and epoch-wise double descent.

Original: The default experiment settings are listed in Appendix G.2 in detail.
Revised: The default experiment settings are listed in Appendix G.2 in detail.

Original: Model capacity.
Revised: Model capacity

Original: We modulate the capacity of the deep model by varying the widening factor ofthe Wide ResNet.
Revised: We modulate the capacity of the deep model by varying the widening factor of the Wide ResNet.

Original: To extend the lower limit of the capacity, we allow the widening factor to be lessthan 1 .
Revised: To extend the lower limit of the capacity, we allow the widening factor to be less than 1 .

Original: In such case, the number of channels in each residual block is scaled similarly but rounded,and the number of channels in the first convolutional layer will be reduced accordingly to ensure thewidth monotonically increasing through the forward propagation.
Revised: In such case, the number of channels in each residual block is scaled similarly but rounded, and the number of channels in the first convolutional layer will be reduced accordingly to ensure the width monotonically increasing through the forward propagation.

Original: Model architecture.
Revised: Model architecture.

Original: We also experiment on model architectures other than Wide ResNet, includingpre-activation ResNet-18 (He et al., 2016) and VGG-11 (Simonyan & Zisserman, 2015).
Revised: We also experiment on model architectures other than Wide ResNet, including pre-activation ResNet-18 (He et al., 2016) and VGG-11 (Simonyan & Zisserman, 2015).

Original: We select
Revised: We select

Original: [Figure]
Revised: [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: these configurations to ensure comparable model capacities 2 .
Revised: these configurations to ensure comparable model capacities 2 .

Original: As shown in Figure 8, different modelarchitectures may produce slightly different double descent curves.
Revised: As shown in Figure 8, different model architectures may produce slightly different double descent curves.

Original: The second descent of VGG-11in particular will be delayed due to its inferior performance compared to residual architectures.
Revised: The second descent of VGGin particular will be delayed due to its inferior performance compared to residual architectures.

Original: Learning rate scheduler
Revised: Learning rate scheduler.

Original: A specific learning rate scheduler may shape the robust overfittingdifferently as suggested by Rice et al.
Revised: A specific learning rate scheduler may shape the robust overfitting differently as suggested by Rice et al.

Original: We consider the following learning rate schedulers inour experiments.
Revised: We consider the following learning rate schedulers in our experiments.

Original: • Piecewise decay : The initial learning rate rate is set as 0 .
Revised: • Piecewise decay : The initial learning rate rate is set as 0 .

Original: and is decayed by a factor of 10 at the100 th and 500 th epochs within a total of 1000 epochs.
Revised: and is decayed by a factor of 10 at the 100 th and 500 th epochs within a total of 1000 epochs.

Original: • Cyclic : This scheduler was initially proposed by Smith (2017) and has been popular in adversarialtraining.
Revised: • Cyclic : This scheduler was initially proposed by Smith (2017) and has been popular in adversarial training.

Original: We set the maximum learning rate to be 0 . 2 , and the learning rate will linearly increase from 0 to 0 .
Revised: We set the maximum learning rate to be 0 . 2 , and the learning rate will linearly increase from 0 to 0 .

Original: 2 for the initial 400 epochs and decrease to 0 for the later 600 epochs.
Revised: 2 for the initial 400 epochs and decrease to 0 for the later 600 epochs.

Original: • Cosine : This scheduler was initially proposed by Loshchilov & Hutter (2017).
Revised: • Cosine : This scheduler was initially proposed by Loshchilov & Hutter (2017).

Original: The learning ratestarts at 0 .
Revised: The learning rate starts at 0 .

Original: 1 and gradually decrease to 0 following a cosine function for a total of 1000 epochs.
Revised: 1 and gradually decrease to 0 following a cosine function for a total of 1000 epochs.

Original: Experiments
Revised: Experiments

Original: on various learning rate schedulers show the second descent can be widely observedexcept the piecewise decay, where the appearance of second descent might be delayed due toextremely small learning rate in the late stage of training.
Revised: on various learning rate schedulers show the second descent can be widely observed except the piecewise decay, where the appearance of second descent might be delayed due to

Revised: extremely small learning rate in the late stage of training.

Original: D More experiment resultsD.1 Training longer
Revised: D More experiment results D.1 Training longer

Original: As shown in Figure 9, We show that our method can maintain the robust test accuracy with moretraining epochs.
Revised: As shown in Figure 9, We show that our method can maintain the robust test accuracy with more training epochs.

Original: Here, we follow the settings in Figure 7 except we train for additional epochs up to400 epochs for each dataset.
Revised: Here, we follow the settings in Figure 7 except we train for additional epochs up to 400 epochs for each dataset.

Original: D.2 Adversarial training methods, neural architectures and evaluation metrics
Revised: D.2 Adversarial training methods, neural architectures and evaluation metrics

Original: In this section we conduct extensive experiments with different neural architectures, adversarialtraining methods and robustness evaluation metrics to verify the effectiveness of our method.
Revised: In this section we conduct extensive experiments with different neural architectures, adversarial training methods and robustness evaluation metrics to verify the effectiveness of our method.

Original: [Figure]
Revised: [Figure]

Original: Tiny
Revised: Tiny

Original: [Figure]
Revised: [Figure]

Original: ImageNet
Revised: ImageNet

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: KD
Revised: KD

Original: [Table]
Revised: [Table]

Original: Auto
Revised: Auto

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: WRN-28AT - -
Revised: WRN-28AT - -

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: KD
Revised: KD

Original: [Table]
Revised: [Table]

Original: AT
Revised: AT

Original: [Table]
Revised: [Table]

Original: KD
Revised: KD

Original: [Table]
Revised: [Table]

Original: AT-
Revised: AT-

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: [-D.
-] Combined with additional orthogonal techniques
Revised: [-D.
-]D.3 Combined with additional orthogonal techniques

Original: We note that motivated from our theoretical analyses, our proposed method (KD-AT-Auto) is essentially the baseline knowledge distillation for adversarial training (KD-AT) with a robustly trained self-teacher, equipped with an algorithm that automatically finds its optimal hyperparameters (i.e. thetemperature T and the interpolation ratio λ ). Stochastic Weight Averaging (SWA) and additional stan-
Revised: We note that motivated from our theoretical analyses, our proposed method (KD-AT-Auto) is essentially the baseline knowledge distillation for adversarial training (KD-AT) with a robustly trained self-teacher, equipped with an algorithm that automatically finds its optimal hyperparameters (i

Original: [Table]
Revised: [Table]

Revised: e. the temperature T and the interpolation ratio λ ).

Original: dard teachers (KD-Std) employed in (Chen et al., 2021) are orthogonal contributions.
Revised: Stochastic Weight Averaging (SWA) and additional standard teachers (KD-Std) employed in (Chen et al., 2021) are orthogonal contributions.

Original: KD-AT-Autocan certainly be combined with SWA and KD-Std to achieve better performance.
Revised: KD-AT-Auto can certainly be combined with SWA and KD-Std to achieve better performance.

Revised: As shown in Table

Original: [Table]
Revised: [Table]

Original: As shown in Table 5, on CIFAR-10, KD-AT + KD-Std + SWA
Revised: on CIFAR- [Table] KD-AT + KD-Std + SWA

Original: (Chen et al., 2021) can already reducethe overfitting gap (difference between the best and last robust accuracy) to almost 0 .
Revised: (Chen et al., 2021) can already reduce the overfitting gap (difference between the best and last robust accuracy) to almost 0 .

Original: It is thus hard[Table] to see any further reduction by combining our method.
Revised: It is thus hard to see any further reduction by combining our method.

Original: To this end, we introduce an extra dataset
Revised: To this end, we introduce an extra dataset

Original: [Table]

Original: SVHN (Netzer et al., 2011).
Revised: SVHN (Netzer et al., 2011).

Original: As shown in Table 5, on SVHN, KD-AT + KD-Std + SWA still produces[Table] a high overfitting gap (also see Appendix A1.3 in (Chen et al., 2021)), whereas by combining withour algorithm to automatically find the optimal hyper-parameters (KD-AT-Auto + KD-Std + SWA),the overfitting gap can be further reduced to almost 0 .
Revised: As shown in Table 5, on SVHN, KD-AT + KD-Std + SWA still produces a high overfitting gap (also see Appendix A1.3 in (Chen et al., 2021)), whereas by combining with our algorithm to automatically find the optimal hyper-parameters (KD-AT-Auto + KD-Std + SWA), the overfitting gap can be further reduced to almost 0 .

Original: This demonstrates the effectiveness and wide[Table] applicability of our principle-guided method on mitigating robust overfitting.[Table]
Revised: This demonstrates the effectiveness and wide applicability of our principle-guided method on mitigating robust overfitting.

Original: [Table]
Revised: [Table]

Original: Here, the interpolation ratio of the standard teacher is fixed as 0 . 

Revised: Here, the interpolation ratio of the standard teacher is fixed as 0 .

Original: and the SWA starts at the firstlearning rate decay for all experiments.
Revised: and the SWA starts at the first learning rate decay for all experiments.

Original: We employ PGD-AT (Madry et al., 2018) as the baseadversarial training method and conduct experiments with a pre-activation ResNet-18.
Revised: We employ PGD-AT (Madry et al., 2018) as the base adversarial training method and conduct experiments with a pre-activation ResNet-18.

Original: The robustaccuracy is evaluated with AutoAttack.
Revised: The robust accuracy is evaluated with AutoAttack.

Original: Other experiment details are in line with Appendix G.1.
Revised: Other experiment details are in line with Appendix G.1.

Original: Furthermore, we note that (Chen et al., 2021) shows SWA and KD-Std are essential components tomitigate robust overfitting on top of KD-AT, while we show that KD-AT itself can mitigate robustoverfitting by proper parameter tuning.
Revised: Furthermore, we note that (Chen et al., 2021) shows SWA and KD-Std are essential components to mitigate robust overfitting on top of KD-AT, while we show that KD-AT itself can mitigate robust overfitting by proper parameter tuning.

Original: We are thus able to separate these components and allow amore flexible selection of hyperparameters in diverse training scenarios without fear of overfitting.
Revised: We are thus able to separate these components and allow a more flexible selection of hyperparameters in diverse training scenarios without fear of overfitting.

Original: Inparticular, although (Chen et al., 2021) suggests SWA starting at the first learning rate decay (exactlywhen the overfitting starts) mitigates robust overfitting, the effectiveness of SWA on mitigatingoverfitting may strongly depend on its hyper-parameter selection including s 0 , i.e., the starting epochand τ , i.e., the decay rate 3 , which is also mentioned in recent work (Rebuffi et al., 2021).
Revised: In particular, although (Chen et al., 2021) suggests SWA starting at the first learning rate decay (exactly when the overfitting starts) mitigates robust overfitting, the effectiveness of SWA on mitigating overfitting may strongly depend on its hyper-parameter selection including s 0 , i.e., the starting epoch and τ , i.e., the decay rate 3 , which is also mentioned in recent work (Rebuffi et al., 2021).

Original: We alsodid some additional experiments on CIFAR-10 following the SWA setting in (Rebuffi et al. 2021)to demonstrate the wide applicability of our method.
Revised: We also did some additional experiments on CIFAR-10 following the SWA setting in (Rebuffi et al., 2021) to demonstrate the wide applicability of our method.

Original: As shown by Table 6, when changing thehyperparameters of SWA, KD-AT + KD-Std + SWA cannot consistently mitigate robust overfitting,while KD-AT-Auto + KD-Std + SWA can maintain an overfitting gap close to 0 and achieve betterrobustness as well.
Revised: As shown by Table 6, when changing the hyperparameters of SWA, KD-AT + KD-Std + SWA cannot consistently mitigate robust overfitting, while KD-AT-Auto + KD-Std + SWA can maintain an overfitting gap close to 0 and achieve better robustness as well.

Original: [Table]
Revised: [Table]

Original: E Study on a synthetic dataset with known true label distribution
Revised: E Study on a synthetic dataset with known true label distribution

Original: [Figure]
Revised: [Figure]

Original: Synthetic 
Revised: Synthetic Dataset.

Original: Dataset.

Original: Since the true label distribution is typically unknownfor adversarial examples in real-world datasets, we simulate the mechanism ofimplicit label noise in adversarial training from a feature learning perspective.
Revised: Since the true label distribution is typically unknown for adversarial examples in real-world datasets, we simulate the mechanism of implicit label noise in adversarial training from a feature learning perspective.

Original: Specifically, we adapt mixup (Zhang et al., 2018) for data augmentation on
Revised: Specifically, we adapt mixup (Zhang et al., 2018) for data augmentation on

Original: CIFAR-10.
Revised: CIFAR-10.

Original: For every example x in the training set, we randomly selectanother example x ′ in a different class and linearly interpolate them by aratio ρ , namely x := ρx + (1 − ρ ) x ′ , which essentially perturbs x withfeatures from other classes.
Revised: For every example x in the training set, we randomly select another example x ′ in a different class and linearly interpolate them by a ratio ρ , namely x := ρx + (1 − ρ ) x ′ , which essentially perturbs x with features from other classes.

Original: Therefore, the true label distribution is arguablyy ∼ ρ · 1 ( y )+(1 − ρ ) · 1 ( y ′ ) .
Revised: Therefore, the true label distribution is arguably y ∼ ρ · 1 ( y ) +(1 − ρ ) · 1 ( y ′ ) .

Original: Unlike mixup, we intentionally set the assigned733 label as ˆ y ∼ 1 ( y ) , thus deliberately create a mismatch between the true labeldistribution and the assigned label distribution.
Revised: Unlike mixup, we intentionally set the assigned label as ˆ y ∼ 1 ( y ) , thus deliberately create a mismatch between the true label distribution and the assigned label distribution.

Original: We refer this strategy asmixup augmentation and only perform it once before the training.
Revised: We refer this strategy as mixup augmentation and only perform it once before the training.

Original: In thisway, the true label distribution of every example in the synthetic dataset is fixed.
Revised: In this way, the true label distribution of every example in the synthetic dataset is fixed.

Original: Concentration of optimal temperature and interpolation ratio of individual examples.
Revised: Concentration of optimal temperature and interpolation ratio of individual examples.

Original: In
Revised: In

Original: Section 5.1 we have shown that in terms of individual examples, the rectified model probability canprovably reduce the distribution mismatch between the assigned label distribution and true labeldistribution of the adversarial example.
Revised: Section 5.1 we have shown that in terms of individual examples, the rectified model probability can provably reduce the distribution mismatch between the assigned label distribution and true label distribution of the adversarial example.

Original: However, since the true label distribution is unknown inrealistic scenarios, it is not possible to directly follow Theorems 5.1 and 5.2 and calculate the optimalset of hyper-parameters for each example in the training set.
Revised: However, since the true label distribution is unknown in realistic scenarios, it is not possible to directly follow Theorems 5.1 and 5.2 and calculate the optimal set of hyper-parameters for each example in the training set.

Original: The best we can do is to employ avalidation set and determine a universal set of hyper-parameters based on the NLL loss, which expectsall training examples to share similar optimal temperatures and interpolation ratios.
Revised: The best we can do is to employ a validation set and determine a universal set of hyper-parameters based on the NLL loss, which expects all training examples to share similar optimal temperatures and interpolation ratios.

Original: Here, based onthe synthetic dataset where a true label distribution is known, we empirically verify this assumptionis reasonable.
Revised: Here, based on the synthetic dataset where a true label distribution is known, we empirically verify this assumption is reasonable.

Original: [Figure]
Revised: [Figure]

Original: In Figure 11 left, we solve the optimal temperature for each correctly classified training examplebased on Theorem 5.1 with the interpolation ratio fixed as 1 . 0 .
Revised: In Figure 11 left, we solve the optimal temperature for each correctly classified training example based on Theorem 5.1 with the interpolation ratio fixed as 1 . 0 .

Original: One can find that the individualoptimal temperatures mostly concentrate between 0 . 5 and 1 . 5 .
Revised: One can find that the individual optimal temperatures mostly concentrate between 0 . 5 and 1 . 5 .

Original: In Figure 11 right, we solve theoptimal interpolation ratio for each incorrectly classified training example based on Theorem 5.2with the temperature fixed as 1 . 0 .
Revised: In Figure 11 right, we solve the optimal interpolation ratio for each incorrectly classified training example based on Theorem 5. with the temperature fixed as 1 . 0 .

Original: One can find that the individual optimal interpolation ratio mostlyconcentrate between 0 . 5 and 0 .
Revised: One can find that the individual optimal interpolation ratio mostly concentrate between 0 . 5 and 0 .

Original: F Method details 
Revised: F Method details F.1 Determine the optimal hyper-parameters

Original: [Figure] F.1 Determine the optimal hyper-parameters

Original: [Figure] One may note that Equation (9) cannot be directly optimized since the traditional adversarial label isonly defined on the example in the training set and cannot be simply generalized to the validationset.
Revised: One may note that Equation (9) cannot be directly optimized since the traditional adversarial label is only defined on the example in the training set and cannot be simply generalized to the validation set.

Original: A reasonable solution is using the nearest neighbour classifier to find the closest traditionaladversarial label for every example in the validation set.
Revised: A reasonable solution is using the nearest neighbour classifier to find the closest traditional adversarial label for every example in the validation set.

Original: However, to speed up the optimization, wepropose to employ the classifier overfitted by the traditional adversarial labels on the training set asan surrogate, which works well in practice.
Revised: However, to speed up the optimization, we propose to employ the classifier overfitted by the traditional adversarial labels on the training set as an surrogate, which works well in practice.

Original: Specially, we employ a model overfitted on the trainingset to generate approximate traditional adversarial label of the adversarial example in the validationset.
Revised: Specially, we employ a model overfitted on the training set to generate approximate traditional adversarial label of the adversarial example in the validation set.

Original: Such overfitted model is typically the model at the final checkpoint when conducting regular adversarial training for sufficient epochs.
Revised: Such overfitted model is typically the model at the final checkpoint when conducting regular adversarial training for sufficient epochs.

Original: Mathematically, our final method to determine the optimaltemperature and interpolation ratio in rectified model probability can be described as
Revised: Mathematically, our final method to determine the optimal temperature and interpolation ratio in rectified model probability can be described as

Original: [Equation]
Revised: [Equation]

Original: 37) where f θ s ( x ′ ; T ) denotes the temperature-scaled predictive probability of a surrogate model on x ′ .
Revised: [Equation] where f θ s ( x ′ ; T ) denotes the temperature-scaled predictive probability of a surrogate model on x ′ .

Original: Here the validation set is constructed by applying adversarial perturbation generated by f θ to the cleanvalidation set.
Revised: Here the validation set is constructed by applying adversarial perturbation generated by f θ to the clean validation set.

Original: For adversarial perturbation we utilize PGD attack with 10 iterations, the perturbationradius as 8 / 255 and the step size as 2 / 255 .
Revised: For adversarial perturbation we utilize PGD attack with 10 iterations, the perturbation radius as 8 / 255 and the step size as 2 / 255 .

Original: Note that Such process incurs almost no additionalcomputation as we simply obtain the logits of a surrogate classifier.
Revised: Note that Such process incurs almost no additional computation as we simply obtain the logits of a surrogate classifier.

Original: G Experimental detailsG.1 Settings for main experiment resultsDataset.
Revised: G Experimental details G.1 Settings for main experiment results

Revised: Dataset.

Original: We include experiment results on CIFAR-10, CIFAR-100, Tiny-ImageNet and SVHN.
Revised: We include experiment results on CIFAR-10, CIFAR-100, Tiny-ImageNet and SVHN.

Original: Training setting.
Revised: Training setting.

Original: We employ SGD as the optimizer.
Revised: We employ SGD as the optimizer.

Original: The batch size is fixed to 128.
Revised: The batch size is fixed to 128.

Original: The momentumand weight decay are set to 0 .
Revised: The momentum and weight decay are set to 0 .

Original: 9 and 0 . 0005 respectively.
Revised: 9 and 0 . 0005 respectively.

Original: Other settings are listed as follows.
Revised: Other settings are listed as follows

Original: • CIFAR-10/CIFAR-100: we conduct the adversarial training for 160 epochs, with the learning ratestarting at 0 . 1 and reduced by a factor of 10 at the 80 and 120 epochs.• Tiny-ImageNet: we conduct the adversarial training for 80 epochs, with the learning rate starting at0 . 1 and reduced by a factor of 10 at the 40 and 60 epochs.• SVHN: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0 .
Revised: • CIFAR-10/CIFAR-100: we conduct the adversarial training for 160 epochs, with the learning rate starting at 0 . 1 and reduced by a factor of 10 at the 80 and 120 epochs. • Tiny-ImageNet: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0 . 1 and reduced by a factor of 10 at the 40 and 60 epochs. • SVHN: we conduct the adversarial training for 80 epochs, with the learning rate starting at 0 .

Original: 01 (assuggested by (Chen et al., 2021)) and reduced by a factor of 10 at the 40 and 60 epochs.
Revised: 01 (as suggested by (Chen et al., 2021)) and reduced by a factor of 10 at the 40 and 60 epochs.

Original: Adversary setting.
Revised: Adversary setting.

Original: We conduct adversarial training with ℓ ∞ norm-bounded perturbations
Revised: We conduct adversarial training with ℓ ∞ norm-bounded perturbations.

Original: Weemploy adversarial training methods including PGD-AT, TRADES and FGSM.
Revised: We employ adversarial training methods including PGD-AT, TRADES and FGSM.

Original: We set the perturbationradius to be 8 / 255 .
Revised: We set the perturbation radius to be 8 / 255 .

Original: For PGD-AT and TRADES, the step size is 2 / 255 and the number of attackiterations is 10 .
Revised: For PGD-AT and TRADES, the step size is 2 / 255 and the number of attack iterations is 10 .

Original: Robustness evaluation.
Revised: Robustness evaluation.

Original: We consider the robustness against ℓ ∞ norm-bounded adversarial attackwith perturbation radius 8 / 255 .
Revised: We consider the robustness against ℓ ∞ norm-bounded adversarial attack with perturbation radius 8 / 255 .

Original: We employ AutoAttack for reliable evaluation.
Revised: We employ AutoAttack for reliable evaluation.

Original: We also include theevaluation results again PGD-1000, Square Attack and RayS.
Revised: We also include the evaluation results again PGD-1000, Square Attack and RayS.

Original: Neural architectures.
Revised: Neural architectures.

Original: We include experiments results on pre-activation ResNet-18, WRN-28-5,
Revised: We include experiments results on pre-activation ResNet-18, WRN-28-5,

Original: WRN-34-10 and VGG-19.
Revised: WRN-34-10 and VGG-19.

Original: Hardware.
Revised: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.
Revised: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.2 Settings for analyzing double descent in 
Revised: G.

Original:  adversarial training
Revised: Settings for analyzing double descent in adversarial training

Original: Dataset
Revised: Dataset.

Original: We conduct experiments on the CIFAR-10 dataset, without additional data.
Revised: We conduct experiments on the CIFAR-10 dataset, without additional data.

Original: Training setting.
Revised: Training setting

Original: We conduct the adversarial training for 1000 epochs unless otherwise noted.
Revised: We conduct the adversarial training for 1000 epochs unless otherwise noted.

Original: By default we use SGD as the optimizer with a fixed learning rate 0 . 1 .
Revised: By default we use SGD as the optimizer with a fixed learning rate 0 . 1 .

Original: When we experiment on asubset (see below) we use the Adam optimizer to improve training stability, where the learning rate isfixed as 0 . 0001 .
Revised: When we experiment on a subset (see below) we use the Adam optimizer to improve training stability, where the learning rate is fixed as 0 . 0001 .

Original: The batch size will be fixed to 128 , and the momentum will be set as 0 .
Revised: The batch size will be fixed to 128 , and the momentum will be set as 0 .

Original: 9 wherevernecessary.
Revised: 9 wherever necessary.

Original: No regularization such as weight decay is used.
Revised: No regularization such as weight decay is used.

Original: These settings are mostly aligned with theempirical analyse of double descent under standard training (Nakkiran et al., 2020).
Revised: These settings are mostly aligned with the empirical analyse of double descent under standard training (Nakkiran et al., 2020).

Original: Sample size.
Revised: Sample size.

Original: To reduce the computation load demanded by an exponential number of trainingepochs, we reduce the size of the training set by randomly sampled a subset of size 5000 fromthe original training set without replacement.
Revised: To reduce the computation load demanded by an exponential number of training epochs, we reduce the size of the training set by randomly sampled a subset of size 5000 from the original training set without replacement.

Original: We adopt this setting for extensive experiments foranalyzing the dependence of epoch-wise double descent on the perturbation radius and data quality(i.e. Figure 5)..
Revised: We adopt this setting for extensive experiments for analyzing the dependence of epoch-wise double descent on the perturbation radius and data quality (i.e. Figure 5)..

Original: Adversary setting.
Revised: Adversary setting.

Original: We conduct adversarial training with ℓ ∞ norm-bounded perturbations.
Revised: We conduct adversarial training with ℓ ∞ norm-bounded perturbations.

Original: Weemploy standard PGD training with the perturbation radius set to 8 / 255 unless otherwise noted.
Revised: We employ standard PGD training with the perturbation radius set to 8 / 255 unless otherwise noted.

Original: Thenumber of attack iterations is fixed as 10 , and the perturbation step size is fixed as 2 / 255 .
Revised: The number of attack iterations is fixed as 10 , and the perturbation step size is fixed as 2 / 255 .

Original: Robustness evaluation.
Revised: Robustness evaluation.

Original: We consider the robustness against ℓ ∞ norm-bounded adversarial attackwith perturbation radius 8 / 255 .
Revised: We consider the robustness against ℓ ∞ norm-bounded adversarial attack with perturbation radius 8 / 255 .

Original: We use PGD attack with 10 attack iterations and step size set to2 / 255 .
Revised: We use PGD attack with 10 attack iterations and step size set to 2 / 255 .

Original: Neural architecture.
Revised: Neural architecture.

Original: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016)with depth 28 and widening factor 5 (WRN-28-5) to speed up training.
Revised: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016) with depth 28 and widening factor 5 (WRN-28-5) to speed up training.

Original: Hardware.
Revised: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.
Revised: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.3 Estimation of the data quality
Revised: G.3 Estimation of the data quality

Original: In this section we elaborate on the calculation of data quality for analyzing the dependence on labelnoise in adversarial training.
Revised: In this section we elaborate on the calculation of data quality for analyzing the dependence on label noise in adversarial training.

Original: We use the predicative probabilities of classifiers trained on CIFAR-10 to score its training data.
Revised: We use the predicative probabilities of classifiers trained on CIFAR-10 to score its training data.

Original: Similar strategy is employed in previous works to select high-quality unlabeled data to improveadversarial robustness (Uesato et al., 2019; Carmon et al., 2019; Gowal et al., 2020).
Revised: Similar strategy is employed in previous works to select high-quality unlabeled data to improve adversarial robustness (Uesato et al., 2019; Carmon et al., 2019; Gowal et al., 2020).

Original: Slightlydeviating from these works focusing on out-of-distribution data, we use adversarially trained insteadof regularly trained models to measure the quality of in-distribution data, since under standardtraining almost all training examples will be overfitted and gain overwhelmingly high confidence.
Revised: Slightly deviating from these works focusing on out-of-distribution data, we use adversarially trained instead of regularly trained models to measure the quality of in-distribution data, since under standard training almost all training examples will be overfitted and gain overwhelmingly high confidence.

Original: Specifically, we adversarially train a pre-activation ResNet-18 with PGD and select the model atthe best checkpoint in terms of the robustness.
Revised: Specifically, we adversarially train a pre-activation ResNet-18 with PGD and select the model at the best checkpoint in terms of the robustness.

Original: The quality of an example is estimated by themodel probability corresponding to the true label without adversarial perturbation and random dataaugmentation (flipping and clipping).
Revised: The quality of an example is estimated by the model probability corresponding to the true label without adversarial perturbation and random data augmentation (flipping and clipping).

Original: We repeat this process 10 times with random initialization toobtain a relatively accurate estimation.
Revised: We repeat this process 10 times with random initialization to obtain a relatively accurate estimation.

Original: G 4 Settings for standard training on fixed augmented training sets
Revised: G.4 Settings for standard training on fixed augmented training sets

Original: [-G.4.
-]
Revised: [-G.4.
-]G

Original:  General settings for both adversarial augmentation and Gaussian augmentation
Revised: 1 General settings for both adversarial augmentation and Gaussian augmentation

Original: Dataset.
Revised: Dataset.

Original: We conduct experiments on the CIFAR-10 dataset, without additional data.
Revised: We conduct experiments on the CIFAR-10 dataset, without additional data.

Original: Training setting.
Revised: Training setting.

Original: We conduct the standard training for 1000 epochs.
Revised: We conduct the standard training for 1000 epochs.

Original: We use Adam as the optimizerwith a fixed learning rate 0 . 0001 to improve training stability with a small training set (see below).
Revised: We use Adam as the optimizer with a fixed learning rate 0 . 0001 to improve training stability with a small training set (see below).

Original: The batch size will be fixed to 128 , and the momentum will be set as 0 .
Revised: The batch size will be fixed to 128 , and the momentum will be set as 0 .

Original: 9 wherever necessary.
Revised: 9 wherever necessary.

Original: Noregularization such as weight decay is used.
Revised: No regularization such as weight decay is used.

Original: Sample size.
Revised: Sample size.

Original: To reduce the computation load demanded by an exponential number of trainingepochs, we reduce the size of the training set by randomly sampled a subset of size 5000 from theoriginal training set without replacement.
Revised: To reduce the computation load demanded by an exponential number of training epochs, we reduce the size of the training set by randomly sampled a subset of size 5000 from the original training set without replacement.

Original: Neural architecture.
Revised: Neural architecture.

Original: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016)with depth 28 and widening factor 5 (WRN-28-5).
Revised: By default we experiment on Wide ResNet (Zagoruyko & Komodakis, 2016) with depth 28 and widening factor 5 (WRN-28-5).

Original: Hardware.
Revised: Hardware.

Original: We conduct experiments on NVIDIA Quadro RTX A6000.
Revised: We conduct experiments on NVIDIA Quadro RTX A6000.

Original: G.4.2 Construction of the training set
Revised: G.4.2 Construction of the training set

Original: Adversarial augmentation
Revised: Adversarial augmentation.

Original: We first obtain a robust model by conduct PGD training with preactivation ResNet-18 on CIFAR-10.
Revised: We first obtain a robust model by conduct PGD training with preactivation ResNet-18 on CIFAR-10.

Original: We use early stopping to obtain the most robust model on avalidation set.
Revised: We use early stopping to obtain the most robust model on a validation set.

Original: The specific settings are aligned with Section G.1.
Revised: The specific settings are aligned with Section G.1.

Original: Using this model, we then generate adversarial examples with PGD attack on the 5000 examplesrandomly sampled from CIFAR-10 training set.
Revised: Using this model, we then generate adversarial examples with PGD attack on the 5000 examples randomly sampled from CIFAR-10 training set.

Original: The number of attack iterations is fixed as 10 and thestep size is fixed as 2 / 255 .
Revised: The number of attack iterations is fixed as 10 and the step size is fixed as 2 / 255 .

Original: The adversarial examples along with their original labels are then groupedinto a training set for adversarial augmentation experiments.
Revised: The adversarial examples along with their original labels are then grouped into a training set for adversarial augmentation experiments.

Original: Gaussian augmentation.
Revised: Gaussian augmentation.

Original: We apply Gaussian noise to the 5000 examples randomly sampled from
Revised: We apply Gaussian noise to the 5000 examples randomly sampled from

Original: CIFAR-10 training set.
Revised: CIFAR-10 training set.

Original: The perturbed examples along with their original labels are then grouped intoa training set for Gaussian augmentation experiments.
Revised: The perturbed examples along with their original labels are then grouped into a training set for Gaussian augmentation experiments.