Understanding the Interaction of Adversarial Training with Noisy Labels
Authors: Jianing Zhu
Jingfeng Zhang
Bo Han
Tongliang Liu
Gang Niu
Hongxia Yang
Mohan Kankanhalli
Masashi Sugiyama
Venue: ICLR 2022 Submitted (2022)
Keywords: noisy labels
adversarial training

Abstract: Noisy labels (NL) and adversarial examples both undermine trained models, but interestingly they have hitherto been studied independently. A recent adversarial training (AT) study showed that the number of projected gradient descent (PGD) steps to successfully attack a point (i.e., find an adversarial example in its proximity) is an effective measure of the robustness of this point. Given that natural data are clean, this measure reveals an intrinsic geometric property---how far a point is from its nearest class boundary. Based on this breakthrough, in this paper, we figure out how AT would interact with NL. Firstly, we find if a point is too close to its noisy-class boundary (e.g., one step is enough to attack it), this point is likely to be mislabeled, which suggests to adopt the number of PGD steps as a new criterion for sample selection to correct NL. Secondly, we confirm that AT with strong smoothing effects suffers less from NL (without NL corrections) than standard training, which suggests that AT itself is an NL correction. Hence, AT with NL is helpful for improving even the natural accuracy, which again illustrates the superiority of AT as a general-purpose robust learning criterion.

Under review as a conference paper at ICLR 2022
UNDERSTANDING THE INTERACTION OF ADVERSAR- IAL TRAINING WITH NOISY LABELS
Anonymous authors Paper under double-blind review
ABSTRACT
Noisy labels (NL) and adversarial examples both undermine trained models, but interestingly they have hitherto been studied independently. A recent adversarial training (AT) study showed that the number of projected gradient descent (PGD) steps to successfully attack a point (i.e., find an adversarial example in its proximity) is an effective measure of the robustness of this point. Given that natural data are clean, this measure reveals an intrinsic geometric property- hhow far a point is from nearest class boundary. Based on this breakthrough, in this paper, we figure out how AT would interact with NI.. Firstly, we find if a point is too close to its noisy-class boundary (e.g., one step enough to attack it), this point is likely to be mislabeled, which suggests adopt the number of PGD steps as a new criterion for sample selection to correct NL. Secondly, we confirm that AT with strong smoothing effects suffers less from NL (without NL corrections) than standard training. which suggests that AT itself is an NL correction. Hence, AT with NL is helpful for improving even the natural accuracy, which again illustrates the superiority of AT as a general-purpos robust learning criterion.
INTRODUCTION
In practice, the process of data labeling is usually noisy. Thus, seems inevitable to learn with noisy labels (Naaarjaan et al.) 2013). To combat noisy labels, researchers have designed robust -oise learning methods, such as sample selection (Jiang et al. 2018) and loss/label correction (Patrini et al.) Nguyen Nguyen et al. 2019). Meanwhile, safety-critical areas (e.g.. medicine and finance) require deep neural networks to be robust against adversarial examples (Szegedy et al. 20144) [gguyen et al.) 2015). To combat vdvesaaiaal examples, adversarial training methods empirically generate adversarial data on the fly for updating the model (Madry et al. 2018 Zhang et al. 2019a).
An interesting fact is that, the research community is exploring label-noise learning and adversarial training independently. For example, Ding et al. (2020) and ZZhang et al. (2021b) demonstrated that the non-robust data that close the nearest class boundary easy be attacked: their adversarial variants easily cross over the decision boundary. To fine-tune the decision boundaries for adversarial robustness, Ding et al. (2020) adaptively optimized small margins for non-robust data, while Zhang et al. (2021b) gave more weights on them. However, both methods in adversarial training explored the adversarial robustness with an implicit assumption that data have clean labels. Obviously, it is not realistic in practice. To this end, we figure out the interaction of adversarial training with noisy labels.
We discover that when noisy labels occur in adversarial training (the right panel of Figure incorrect data (square points) are more likely be non-robust (i.e., the predicted labels of their adversarial variants disagree with the given labels). Specifically. Figure compares the difference between standard training (ST (Zhang al. 2017)) and adversarial training (AT (Madry et al. 2018) with noisy labels. Commonly, small number of incorrect data (square points) surrounded by large number of correct data (round points). In ST, deep networks shape two small clusters (the left panel of Figure T around the two incorrect data due to memorization effects ZZang al., 2017). In contrast, AT has strong smoothing effects, i.e., smoothing out the small clusters around incorrect data and letting incorrect data alone (the right panel of Figure
To explain the above phenomenon in AT, we believe that the adversarial counterparts generated by (majority) correct data can help smooth the local neighborhoods of correct data, which encourages

---

Under review as a conference paper at ICLR 2022
Neighbochuokk incorrect data
225
2,00
Avergg
1.00
cate (4:)
0.23
Kaiie saer 6 (57)
0.30
(A7)
ST
AT
Epochs
Figure 1: The results of ST and AT on a binary dataset with noisy labels. Dots denote correct data; squares denote incorrect data. The color gradient represents the predic- tion confidence: the deeper color represents the higher prediction confidence. Left panel: A deep network shapes two small clusters (red and blue ones in cross-over areas) around two incorrect data due to memorization effects in ST. Right panel: These clusters have been smoothed out in AT. Boxes represent the unit-norm ball of AT.
Figure 2: The average entropy of mod- els trained by ST and AT. This value is calculated on 100 points in each neighborhood of incorrect data, us- ing CIFAR-IO with symmetric-fippinng noise. Both solid and dashed lines rep- resent ST and AT, respectively. Note that ST learns incorrect data more de- terministically than AT.
deep networks to be locally constant within the neighborhood (Papernot et al.) (016). Therefore, in AT, becomes difficult for deep networks to form small but separated clusters around each incorrect data. Consequently, these incorrect data are non-robust, which echos the parallel findings that robust training avoids memorization of label noise (Sanaal et al. 2021).
Furthermore, make quantitative comparisons between ST and AT in the presence of label noise. Zhang et al. (017)) showed that ST indeed overfits noisy labels, which definitely degrade the generalization performance of deep networks. From Figure it can be seen that the training accuracy of deep networks on incorrect data obviously lower than that on correct data in AT. Nonetheless, the performance gap totally disappeared in ST. Therefore, compared ST, AT can always distinguish correct data and incorrect data. Observing Figure 4 the test accuracy of deep networks first increases then decreases in ST. Nonetheless, such a trend has been largely alleviated totally eliminated in AT. Therefore, AT can mitigate negative effects of noisy labels, since the smoothing effects of AT can prevent memorizing such incorrect data.
Moreover, under noisy labels, we realize that AT provides a new measure-how how difficult it is to attack data to generate adversarial variants whose predictive labels are different from the given labels- which can distinguish correct/incorred data (Figures ((a) and and typical/rare data (Figure 8) well. This new measure can be approximately realized by the number projected gradient descent (PGD) steps (Madry et al. 2018)) i.e., how many PGD iterations we need to generate misclassified adversarial variants. Compared with the commonly used measure, i.c., the loss value (Jiang et al., 2018; Han et al.) 20188), we find that the number of PGD steps could be alternative even better measure in AT (Figures [[a] and (()).. In addition, we discover that this new measure can easily pick up rare (atypical) data among typical data (Figure R). where modern datasets often follow long-tailed distributions (Feldman & Zhang 2020).
Main contributions.
To sum up, our contributions can be summarized in three aspects as follows.
1. We explore the in-depth interaction of AT with noisy labels. Namely, we take a closer look at the smoothing effects of AT under label noise (Section [3]. Subsequently, we conduct quantitative comparisons: compared with ST, AT can always distinguish correct and incorrect data and mitigate negative effects of label noise (Section
2. We realize that AT naturally provides new measure called the number of PGD steps, i.c., how many PGD iterations are needed to generate misclassified adversarial examples. Such a new measure can clearly differentiate the correct/incorrect data and typical/rare data 55
3. We provide two simple examples of the applications of our new measure: a) we develop a robust annotator, which can robustly annotate unlabeled (U) data considering that U data could be adversarially perturbed (Section 1): b) our new measure could be an alternative to the predictive probability for providing the confidence of annotated labels (eecion(6.2)
2

---

Under review as a conference paper at ICLR 2022
2 BACKGROUND AND RELATED WORK
Adversarial training (AT). As one of the primary defenses against adversarial examples (Good
fellow et al., 2015) Carlini & Wagner 2017] Athalye et al. 2018). AT has been widely studied to
improve the adversarial robustness of deep neural networks (DNNs) (Cai et al. 2018 Wang et al. 2020b a Jiang et al., 2020b; Wu et al., chen
Chen et al. 2020 Bai et al., 2021, Chen et al., [022]]
Tian et 20.1). The key objective of AT is to minimize the training loss on the adversarial variants of training data. We review the details of AT (Madry et al. 2018 used in this paper.
Let (X' , dox)) denote the input feature space X' with the infinity distance metric dir( x²) ) ||
x' oooo and [x[x]] [ {x' E X dint dint(z, x') S }} be the closed ball of radius e > 0 centered at æ in X. S = ( y1) Ji-1 is dataset and (I.,, yt) are i.i.d. from an underlying distribution, where 2; E X,
Y: E y = f0, 10.1.... ,....... 1}, and C denotes the number of classes. The objective function of AT is
min JoEF
E(lfo(z.....)
(1)
where é; is an adversarial variant of input data 2; within the c-ball centered at x and fo()) : :X RC is score function. L RC x y - IR is a loss function which is a composition of a base loss ln : AC-1 y R (e-g., the cross-entropy loss) and an inverse link function 4. IRC - A c-1 (e.g., the soft-max activation), in which A c is the corresponding probability simplex- in other words, {(ff()). 3) (f
To generate the adversarial variants 전 for natural data æ, AT employs the PGD method (Madry et al. 2018 Given a starting point z00) E X' and step size a > 0, PGD works as follows:
(1+1) 1 (ax(1) + a sign V 201 efo (z:( ),))),
until certain stopping criterion is satisfied. In the above equation, t N. E is the loss function, (0) refers to natural data or natural data perturbed by a small Gaussian or uniform random noise, y is the
corresponding label for natural data æ, (+() is an adversarial data point at step f, and [=) :) is the projection function that projects the adversarial data back into the e-ball centered at x (0) if necessary.
is common to use PGD generate adversarial variants ă in AT methods (Wang et al.) 2019; Zhang et al. 2020). Recently, Zhang et al.) (202Tb) explored adversarial robustness by giving more weights on the non-robust data with the assumption that all labels are correct. Specifically, the non-robust data are geometrically close to the class boundaries, which can easily go across the class boundaries by a small perturbation. To approximate the distance between the data and the class boundaries, they proposed the geometry-awan projected gradient descent (GA-PGD) to calculate the geometry value K, which is the least number of iterations that PGD needs to find misclassified adversarial variants of input data, In this paper, we utilize the geometry value k to represent our proposed measure (i.e., the number of PGD steps); we further explore its applications such as selecting correct/incorrect and typical/rare data (Section 5), assisting to develop robust annotator (Section 6.1) and providing the annotation confidence (Section (.2).
Label-noise learning. We consider a training set with X' = (x1,.... .N) and its associated labels y (y1,-..,WN) UN), where yi y is the one-hot label for the instance v; and (xi, v:) drawn i.i.d. from some unknown distribution. In the setting of label noise, we observe noisy labels 5 = (JI..... DN) where J E 5 might be different from the corresponding ground-truth -ruth label Yi y. In this paper, we mainly focus on typical class-conditional noise: 1) symmetric-ftippinn noise (Van Rooyen et al. 2015), where noisy labels are corrupted at random with the uniform distribution; 2) pair-flipping noise (Han et al. 2018). where noisy labels are corrupted between adjacent classes that are prone to be mislabeled. Note that pair-flipping noise is extremely hard case of esymmetric-ffipiinn noise (Patrini et al. (017).
To combat noisy labels, researchers have designed robust label-noise learning methods, such as sample selection (Malach & Shalev-Shwarz 2017 Jiang et al., 2020a Han et al. 2020a), loss correction (Han et al. |020b Liu & 2020), and label correction (Wang al. (018)). Among them, sample selection is emerging due to its simplicity. The key idea of sample selection is to back-propagate clean samples (regarded as correct data) during training. Since DNNs learn simple
3

---

Under review as a conference paper at ICLR 2022
Natural training data ST
Necural training data in AT
Natural training data in ST
Natural training data in AT
100
Starken sostran (i
Stankeed aocurnyy
Standard sccuEARy
Standan aacuraary
75
21.99
300
41
103
Epede
Riurch
(a) CIFAR-10
(b) MNIST
Figure 3: The standard accuracy of ST and AT on correct/incorrect training data using CIFAR-10 noise. Solid lines denote the accuracy of correct training data,
and MNIST with symmetric-flippin while dashed lines correspond to that of incorrect training data. Compared with ST, there is large performance gap in the standard accuracy of correct/incoreet training data in AT.
patterns first Zhang et al. 2017; Arpit et al. 2017), the loss value is used as a general criterion for selecting clean samples (Han et al. 2018 Yao et al. 2020). Specifically, the data with small-loss values are considered as clean samples, which are used to update the model. contrast, the data with large-loss values are considered as noisy samples, which should be discarded utilized in another way (Han et al. (020a). Note that this paper proposes a new criterion the number of PGD steps-for for sample selection (Section 5
Co-existence of adversarial examples and noisy labels (NL). Some work has previously studied adversarial examples and noisy labels (NI.) jointly. Alayrac et al. (2019) empirically showed that NL negatively affects AT's performance, but AT's robust accuracy apparently suffers less than AT's natural accuracy. Sanyal et al. (202I) empirically found that AT can avoid the memorization of NL. Damodaran al. 2019) proposed to use Wasserstein adversarial regularization to combat NL for benefiting ST's generalization. In contrast, we advocate the AT's smoothing effect by making quantitative comparisons between AT and ST under NL. Specifically, ST has memorization effect that gradually memorizes NL and degrades generalization in the end, while AT has a smoothing effect that avoids the memorization of NL and combats NL for benefiting generalization. In addition, we propose a new measure called the number of PGD steps for sample selection, which can differentiate the correct/incorrect data and typical/rare data, and we provide two exemplar applications for our new measure. Furthermore, in Appendix A we provide extensive comparisons between our study and the existing literature.
SMOOTHING EFFECTS OF ADVERSARIAL TRAINING
In this section, we take a closer look at the smoothing effects AT with NL. At a high level, we conduct experiments
synthetic dataset with incorrect labels, which explicitly show the smoothing effects of AT (Figure []. We then use a real-world dataset, CIFAR-10 10 (Krizhevsky) 2009). with incorrect labels, which further validates the smoothing effects of AT (Figure 2). As a key result, we find that AT can smooth out the small clusters around incorrect data (the right panel of Figure [ which leads to incorrect data being non-robust in AT, i.c., they can be easily attacked to flip labels. The setup and more results can be found in Appendix B
In detail, we empirically confirmed that DNNs can memorize random noise in ST (the left panel of Figure []. which has been found in previous works (Zhang et al. 2017: Arpit et al. 2017). However, a recent study (Sanyal et al., (02T) claimed that AT can avoid the memorization of incorrect data through analyzing model predictions. Going beyond their analysis, we further investigated AT with noisy labels and provided an in-depth explanation, namely smoothing effects. Specifically, AT prevents incorrect data from forming small clusters, which should be the primary reason for avoiding the memorization of incorrect data.
To justify our smoothing effect, we performed series of comparison experiments using ST and AT on synthetic dataset with incorrect labels. In Figure [ the model trained by ST can overfit the incorrect data (yellow and black squares), and thus have incorrect predictions (red and blue clusters in cross-over areas) around incorrect data. While in AT (with smoothing effects), such clusters have obviously disappeared. The reason is due the smoothing effects from the adversarial variants
4

---

Under review as a conference paper at ICLR 2022
Natural text date in ST
Natural tost data in AT
Natural tost data in $77
Notural tost date in AT
Stunkal conmey (8)
Suadkal wontany (95)
Stonkant neuuaay (85)
Stondan sonnayy
2
Epocho 100
Epocts
Lpochs
Eporta
(a) CIFAR-10
(b) MNIST
Figure 4: The standard accuracy of ST and AT natural test data using CIFAR- 10 and MNIST with caanmetrrc-fiipinng noise for training. Note that the larger noise rate causes the test accuracy of ST dropping more seriously due memorization effects in deep learning, while AT alleviates such negative effects.
generated from correct data. Namely, the number of correct data is larger than that of incorrect data. Thus, it is difficult for incorrect data to smooth their neighborhood.
Further, calculated the entropy values of the model predictions the CIFAR-IO dataset, which aims to validate the smoothing effect in practice. Specifically, we randomly selected 100 points in each neighborhood (within small c-ball) of the incorrect data and calculated their average entropy values in training (Figure 2 As a measure of uncertainty (Dai Chen 2012), the entropy value was calculated by the following formula:
H(IX)) = Σ Σ FEX DEY
p(z, v) logp(ylz).
The smaller value represents the higher certainty of model prediction (and vice versa), which indicates that the model learns the data more deterministically. Thus, the higher certainty leads to the higher possibility of incorrect data forming small clusters in their neighborhoods.
We compared the entropy values of ST and AT. During the training process, under the same noise rate, the entropy value of AT is always higher than that of ST. After epoch 60, the entropy value of ST drops very fast, while that of AT remains high. It clearly shows that smoothing effects in AT prevent the model from learning incorrect data with their neighborhoods deterministically, which further confirms that harder for incorrect data to form small clusters. By observing Figures] [ and 2 we confirmed that it is difficult for incorrect data in AT to form small clusters due to the smoothing effects from the adversarial variants of correct data.
4 KNOCK-ON EFFECTS OF ADVERSARIAL TRAINING
In this section, explore knock-on effects of AT comprehensively. We show the quantitative differences between ST and AT with noisy labels. First, in terms of training accuracy, we show that correct/incorrect data can be always distinguishable in AT (Figure 3. Second, in terms of test accuracy, we demonstrate that AT alleviates negative effects of incorrect data and then improves the model generalization (Figure 4. Note that display the experimental results the CIFAR-1O and MNIST datasets (LeCun et al. 1998) with symmetric-fipping noise in this section. More results (e.g.. pair-flipping noise, the CIFAR-100 dataset, the loss values, and different networks) can be found in Appendix((
4.1 DISTINGUISHABLE CORRECT/INCORRES DATA
In Figure we plotted the standard accuracy of natural training data in ST and AT. In the early stage of training, there is a clear performance gap between the standard accuracy of ST on correct/incorreet training data. However, after 60 epochs, the standard accuracy of ST on incorrect training data increases rapidly, while that of AT on incorrect training data rises relatively slowly. When the training comes to epoch 100, the standard accuracy of ST on correct/incorrect training data merged together. Nonetheless, there still large performance gap in AT. Compared to the results on CIFAR-I0 10, such gap is more obvious MNIST.
5

---

Under review as a conference paper at ICLR 2022
4.2 ALLEVIATION OF MEMORIZATION EFFECTS
In Figure 4 we plotted the standard accuracy of natural test data in ST and AT. It shows that AT can alleviate the negative effects of label noise and then improve the model generalization. Specifically, the larger noise rate causes the test accuracy of ST to drop more seriously, i.c., memorization effects (Arpit et al.) 2017). However, AT reduces such negative effects. By checking the standard accuracy of natural test data, we find that there is no obvious overfitting phenomenon in AT. We also visualized the loss landscape (Li et al. 2018) of models trained by ST and AT to further substantiate the alleviation, which can be found in Appendix
worthwhile to observe the results on MNIST: simply using AT can make the model obtain performance similar noise-free training. However, on more complex CIFAR-1O, incorrect data still have a certain negative impact on the model trained by AT. To reduce such an impact, a simple yet effective method is to sample selection to filter correct/incorrect data for training (Jiang et al. 2018 Cheng et al. 2021). Therefore, it is critical to have a measure which can provide the stratification for correct/ncorrect data. Normally, the loss value
be good candidate in ST. However, in AT, we can find a better measure such as the number of PGD steps (i.e., geometry value x). Since the smoothing effects in AT can make incorrect data be non-robust, the geometry value k-how difficult it is to attack data to let them go across the decision boundary-could be naturally used measure for this task.
Noiee
Content
Ubrret
Coerrer data
Geomelry valuue A
Incerrect data
10-
Epochs
Epodis
0800 0200
Noise nate: 0.2
Nokso rate: 0.4
00.0
COC0030
Glemmniy 1.
N.
00009
Loss value
101
Epuclic
Epedls
Figure 5: Comparisons of correct/incorrec data in terms of the loss value (top panel) and the geometry value x (bottom panel) on CIFAR-1O with symmetric- flipping noise in AT. We calculate the mean values in each epoch. We clearly demonstrate that the value has similar trend as loss value in AT; both can be used for differentiating correct/incorret data in AT.
Figure 6: We choose the model trained by AT using CIFAR-IO with 20% symmetric- flipping noise. We jointly analyze the ge- ometry value x and the loss value, which shows that the value к can provide fine stratification on typical (i.e., larger k)/rare (i.e., smaller s:) data.
NEW MEASURE: GEOMETRY VALUE K
In this section, we show the geometry value λ could new measure for the data stratification. First, the geometry value can differentiate correct/incorreet data in AT (Figures 5 (a) and 7
Compared with the loss value, which has been widely used in sample selection (Jiang et al. 2018
Han al.) 2018 Yu al. 2019), show that the geometry value filter incorrect data with different noise types. Second, we demonstrate that the geometry value x can provide finer stratification typical/rare data (Figures 6 and (8. We also discussed with different configurations (e.g., PGD step number or the e-ball) and more results can be found in Appendix [
have a better performance to
5.1 GEOMETRY VALUE VS. LOSS VALUE
To combat NL, sample selection methods are very effective. As a common measure in sample selection, the loss value is used filter incorrect data. For example, small-loss data be regarded

---

Under review as a conference paper at ICLR 2022
03
-5505
-8505 65
Epeatoo
Cpect0)
Spec560
Loss natural
Densttt
tse900
Eaxtet
ErsiMb
frecldo
Geometry value In
Geometry value AT
(a) 20% Symmetric-flipping noise
(b) 40% Pair-flipping noise
Figure 7: The density of AT on correct/incoreet data using CIFAR-I0 with (a) 20% symmetric- flipping noise and (b) 40% pair-flipping noise. Top panels: the loss value in AT. Bottom panels: the geometry value K. in AT. The geometry value x has a better distinction on correct/incorrect data.
as "correct" data. However, there are two limitations using the loss value as a measure. First, we need to adjust different thresholds to obtain better selection effect, when the dataset has different noise rates and types YYao al., 2020). Second, for pair-lipppng noise, the loss value cannot distinguish correct/incoreet data well (the top panel of Figure ((b)..
we compared the geometry value and loss value of correct/incorrect data in the training process. We found that the value can be used to differentiate incorrect data from correct data, since has a similar trend to the loss value in AT. To further compare two measures distinguishing correct/incorrect data, we plotted the density maps of two measures on the CIFAR-10 dataset with different noise types in Figures ((a) and 7(b) To compare the two measures in a mcaningful way, performed the min-max normalization TTax & Duin 2000) on both the loss value and geometric value K, which scales the range of values in [0, 1J.
In Figure []
For symmetric- cemmetric-fliping noise (Figure (aa) although the loss value distinguish correct data from incorrect data during the training process, the geometric value λ has a better distinction between correct and incorrect data. Specifically, the top panels of Figure 7(a) show that there a large number of correct/incorred data with the same loss value, which requires a carefully designed threshold select the correct data from incorrect data. In contrast, correct/incorrect data can be well divided using the value k in the bottom panels of Figure 7(a) We can easily select correct/incorreet data with high purity. More obviously, for pair-flipping noise, the loss value of correct/incorrect data overlaps in the top panels of Figure 7(b However, the value in the bottom panels of Figure 7(b) still provides good discrimination on correct/incorrect data.
In addition, we found that the geometry value к can provide fine stratification on typical/rare data. First, we jointly analyzed the value 6s and the loss value in AT (Figure (). where we stratified correct data via K. Secondly, by inspecting the semantic information with different k, we found that the value x can represent whether the data is relatively typical or rare (Figures ((a) and [(b)). Moreover, plotted bivariate graph of the loss value and the value in Figure 6 In this figure, we mainly focused on the correctly classified data (blue scattered dots), since the wrongly classified data (orange scattered dots) had been clearly discriminated by big loss values. Note that, for small-loss (correct) data, the value x can further subdivide such data into typical and rare types.
5.2 DISTINGUISHABLE TYPICAL/RARE DATA
From the macro perspective, the loss value can be regarded as measure to classify correct and incorrect data (Jaag et al. 2018). Namely, small-loss data can be regarded as correct data, and vice versa. However, such stratification a bit rough, which motivates us to seek a micro measure called the geometry value x. (the number of PGD steps) in AT. To justify our findings in Figure O we visualized the semantic information of CIFAR- 10 (Figure 8(a)) and MNIST (Figure 8(b) under different K. We found that images with large k (rightmost) are prototypical and easier to recognize from the viewpoint of human perception, while images with small x (or K = 0) seem to be rarer (or incorrect). These rare images have atypical semantic information, such as some strange shapes ("8" with k = 14 in Figure ((b) or confusing backgrounds ("deer" with A = 2 in Figure B(a)). More results about the images with different be found in Appendix D
7

---

Under review as a conference paper at ICLR 2022
plane
9
3
3
3
N:
K: K: K:
K:
6:
K:: 29
K: 40 8
$
3
deer
Kc
3
NC 10
U
<st 14
K: 29
KC 40
(a) CIFAR-10
(b) MNIST
Figure 8: The geometry value i w.r.t. images in CIFAR-1O and MNIST with 20% and 10% symmetric- flipping noise. The leftmost of each subfigure the given label (i.e., deer and plane or 3 and of all images on the right. We randomly select four examples with the different in each class. As the geometric value increases from left (k 0) to right (x 10 or 40), the semantic information of images is more typical and recognizable.
Algorithm 1: Robust Annotation Algorithm.
Input network fe. training dataset S {a y1) y:) learning rate n, number of epochs batch size m, number of batches M, threshold for geometry value K, threshold for loss value
Output :robust annotator fo- for epoch = I....... T do
for mini-batch l...., M do
Sample: mini-batch {<x; y1) ) from S.
for i= Im..., (in parallel) do
Calculate: K2 and & of (24.4)).
if K4 < K and & > L then Update: y4 arg max; fo(x).
end Generate: adversarial data 3. by PGD method.
end
Update: 0 0 (V((((((
end
end
APPLICATIONS OF GEOMETRY VALUE K
In this section, we provide two applications of our measure the geometry value (the number of PGD steps). Since the value k can differentiate correct/incorrect data in AT (Section 5.T). in the presence of label noise, we can use it detect noisy labels and correct labels (Figure D) Meanwhile, as it can have a fine stratification for typical/rare data (Section 5.2)) we can provide the confidence of annotated labels according to the value (Figure 10)
Regardless of ST or AT, high-quality training data are always essential for acquiring a good model (Deng et al., 2009). but the labeling process of high-quality data requires lot of human resources. To deal with such problem, many methods used ST to facilitate a standard annotator to annotate large-scale unlabeled (U) data (Carmon al., 2019 Alayrac et al., 2019). However, this standard annotator fails when U data are adversarially manipulated.
In practice, label-noise issues widely exist in wol-w training datasets, and learning with NI seems inevitable. Meanwhile, the existence of adversarial examples (Szegedy et al.) 2014 Goodfellow et al.) 2015) also poses a threat to annotate U data. Therefore, we design a robust annotation algorithm [Algorithm assign reliable labels for U data even in the presence of adversarial manipulations and noisy traming labels (Section 6.1. Compared to human beings, the standard annotator cannot give the information whether the label assignment for U data is reliable. Nonetheless, new measure could be an alternative to the predictive probability for providing the confidence of annotated labels (Section 6.2p. The detailed experimental setups can be found in Appendix E
8

---

Under review as a conference paper at ICLR 2022
The test cata with PCD-20 attack
Zocen-oi noulta
The dota
The test data
Comennyy postineed
Winnab proliced
Aesumry (75
Aveune
Acconry
Nem
48
Raticol achoooodd coenepted dats
oatin sommpttd data
Goometay value
Goonuctry valle
The accuracy of four approaches as- signing correct labels to adversarial U data from CIFAR-10. Left panel: the full results. Right panel: the zoom-out results (without standard annotator), Our robust annotator has a satisfac- tory performance on assigning reliable labels.
Figure 9:
The accuracy (left panel) and num- ber (right panel) of correctly predicted U data w.r.t. the geometry value s. We randomly select 2000 test data in CIFAR-1O as unlabeled data. The larger A corresponds to the higher prediction accuracy.
Figure 10:
6.1 ROBUST ANNOTATOR
We can construct a robust annotator assign labels for U data. Here, consider a real-world scenario, namely, existence of label noise in training data and adversarial manipulations in U data. Our robust annotator has better labeling performance than the standard annotator, since value x and the loss value jointly to select incorrect training data. We re-annotate high-quality pseudo labels for these incorrect data, and adversarially train on the whole data. Then, our robust annotator can reliably assign labels.
the
In Figure we tested the accuracy of assigning correct labels U data in the presence of adversarial manipulations. We compared four methods, namely, our robust annotator with 20% symmetric- flipping noise (red line), the PGD- annotator with 20% symmetric-flipiinne noise (orange line), the PGD-based annotator without noise (oracle, black dashed line), and the standard annotator without noise (blue line). On normal U data (i.e., zero adversarial ratio), the standard annotator has better performance of labeling. However, when U data is subject to certain adversarial manipulations (i.e., ratio above 0.2), the labeling quality of the standard annotator decreases sharply, but that of our robust annotator still remains satisfactory. An extreme case is that, when all U data (ratio 1.0) are added to adversarial manipulations, labels assigned by the standard annotator become completely unreliable, but our labels assigned by the robust annotator are still better than the PGD-based annotator with 20% commetric--lipiing noise.
6.2 CONFIDENCE SCORES
For given data point, the geometry value can provide a confidence score, which represents the reliability of label annotations. The measure value д. can distinguish between typical data (correctly labeled with high probability) and rare data (wrongly labeled with high probability) in U data. In the left panel of Figure 10 we plotted the accuracy of correctly predicted data with the value The larger k corresponds to higher prediction accuracy, which shows that the value k can indeed represent the reliability of label annotations. In the right panel of Figure 10 we further investigated the number of correctly predicted data with the value K. Most of the data have the value λ 10, which corresponds
high prediction accuracy. Meanwhile, small part of the data have the value k E [0, 6), which corresponds to a low prediction accuracy. Since the number of data with value E [7, 9] small, the standard deviation of the accuracy large.
CONCLUSION
In this paper, we explored the interaction of adversarial training (AT) with noisy labels. We took closer look at smoothing effects of AT, and further investigated positive knock-on effects of AT. As result, AT can distinguish correct/incorrect data and alleviate memorization effects in deep networks. Since smoothing effects can make incorrect data non-robust, the geometry value (i.c., the number of projected gradient descent steps) could be a new measure to differentiate correct/incorrect and typical/rare data. Moreover, we gave two applications of our new measure, i.e., the robust annotator and confidence scores. With the robust annotator, we can assign reliable labels for adversarial U data. With confidence scores, we can know the reliability of label annotations.

---

Under review as a conference paper at ICLR 2022
8 ETHICS STATEMENT
This paper does not raise any ethics concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, iasrriiinationbiis/faaannn concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of experimental results, we will provide a link for an anonymous repository about the source codes of this paper in the discussion forums.
REFERENCES
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NeurIPS, 2019.
Devansh Arpit, Stanistaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In ICMI., 2017.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018
Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang. Improving adversarial robustness via channel-wise activation suppressing. In ICLR, 2021.
Qi-Zhi Cai, Chang Liu, and Dawn Song. Curriculum adversarial training. In IJCAI, 2018.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness neural networks. In Symposium on Security and Privacy (SP), 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. In NewrlPS, 2019.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training fine-tuning. In CVPR, 2020.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In ICLR, 2021.
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance- dependent label noise: sample sieve approach. In ICLR, 2021.
Wei Dai and Xiaowei Chen. Entropy of function of uncertain variables. Mathematical and Computer Modelling, 2012.
Bharath Bhushan Damodaran, Kilian Fatras, Sylvain Lobry, Rémi Flamary, Devis Tuia, and Nicolas Courty. Wasserstein adversarial regularization (war) on label noise. arXiv preprint arXiv: 1XXi::9904.03366, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. In NeurIPS, 2020.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
10

---

Under review as a conference paper at ICLR 2022
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, 2018.
Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang. and Masashi Sugiyama. Sigua: Forgetting may make learning with noisy labels more robust. In ICML, 2020a.
Kai Han, Yunhe Wang, Yixing Xu, Chunjing Xu, Enhua Wu, Chang Xu. Training binary neural networks through learning with noisy supervision. In ICML, 2020b.
Lu Jiang, Zhengyuan Zhou, Thomas Leung. Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks corrupted labels. In ICML, 2018.
Lu Jiang, Di Huang, Mason Liu, and Weilong Yang. Beyond synthetic noise: Deep learning on controlled noisy labels. In ICML, 2020a.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. In NeurIPS, 2020b.
Takuhiro Kaneko, Yoshitaka Ushiku, and Tatsuya Harada. Label-noise robust generative adversarial networks. In CVPR, 2019.
Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):278--2224 1998.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In NeurlPS, 2018.
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In ICML, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Eran Malach and Shai Shalev-Shwartz. Decoupling" when to update" from" how to update". In NeurIPS, 2017.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML., 2010.
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. In NeurlPS, 2019.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K. Ravikumar, and Ambuj Tewari. Learning with noisy labels. In NeurIPS, 2013.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
Due Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo, Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. Self: Learning to filter noisy labels with self-ensembling, In ICLR, 2019.
Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the science of security and privacy machine learning. arXiv:6611.0381. 2016.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In CVPR, 2017.
Amartya Sanyal, Puneet Dokania, Varun Kanade, and Philip H. Torr. How benign is benign overfitting? In ICLR, 2021.
11

---

Under review as a conference paper at ICLR 2022
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
DM Tax and RP Duin. Feature scaling in support vector data descriptions. 2000.
Qi Tian, Kun Kuang, Kelu Jiang, Fei Wu, and Yisen Wang. Analysis and applications of class-wise robustness in adversarial training. KDD, 2021.
Brendan Van Rooyen, Aditya Menon, and Robert C Williamson. Learning with symmetric label noise: The importance of being unhinged. In NeurIPS, 2015.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for- all adversarial training: In-situ tradeoff between robustness and accuracy for free. 2020a.
Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia. Iterative learning with open-set noisy labels. In CVPR, 2018.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020b.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. NeurlPS, 33, 2020.
Quanming Yao, Hansi Yang, Bo Han, Gang Niu, and James Tin- Yau Kwok. Searching to exploit memorization effect in learning with noisy labels. In ICML, 2020.
Xingrui Yu, Bo Han, Jiangehao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? ICML, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. 1rxivv1605.000 07..00716,,.0 2016.
Chiyuan Zhang, Samy Bengio. Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. ICLR, 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019a.
Jingfeng Zhang, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Where is the bottleneck of adversarial learning with unlabeled data? arXiv preprint 19/10699 2019b.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Jingfeng Zhang, Xilie Xu, Bo Han, Tongliang Liu, Gang Niu, Lizhen Cui, and Masashi Sugiyama. Noilin: Does noisy labels always hurt adversarial training? arXiv preprini, 2021a.
Jingfeng Zhang, Jianing Zhu, Gang NIu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighted adversarial training. In ICLR, 2021b.
12

---

Under review as a conference paper at ICLR 2022
A DETAILED DISCUSSIONS ON THE DIFFERENCES WITH THE EXISTING STUDIES
This section discusses the difference between our work and other related studies that focused on either improving adversarial training (AT) learning with noisy labels (NL) in ST. Our work focuses on figuring out the in-depth interaction of (canonical) adversarial training with (generalized) noisy
labels.
Improving adversarial training (AT). Some studies focused improving the AT's performance by leveraging additional unlabelled (U) data (Carmon et al.] 2019 Najañ et al., 2019) Allayrac et al.) 2019; Zhang et al., 2019b). The main points are leveraging additional U data in AT that can achieve both higher robust accuracy and higher standard accuracy. The ablation study of (Alayrac et al. 2019) provided an analysis on the impact of ceametric-flipping noise for model robustness, which simulates the unreliable annotation of U data. By comparison, our study claims that AT itself an NL correction. We conduct experiments on the generalized setting of NL, i.e. with different kinds of label noise (e.g., symmetric-flipiine noise and pair-flippe hair-fliping noise) and different noise rates (e.g.. (0.1,0.4)).
For benefiting adversarial robustness, Zhang et al. (2021a) proposed to inject NL over the training process. They assume that the training set is noise-free, and they inject NL on the fly as AT's regularization method. By comparison, our settings and motivations are different. We assume the training set is label-noisy and find that AT can naturally mitigate the negative effect of NL in the training set. We focus on the understanding of AT's smoothing effects on NL.
Learning with noisy labels (NL) in ST. Damodaran et al., (2019) focused learning with NL in ST and designed a Wasserstein Adversarial Regularization (WAR) as a correction method. They added the WAR into standard training (ST) to combat with the NI.. The authors also provided analysis for their proposed correction method. Specifically, they explained WAR as label interpolation that uses the prediction of the adversarial data to interpolate the original label for the natural data, By comparison, we identified and illustrated the inherent reason for AT's NL correction, i.e., the smoothing effect of the AT. Due AT's smoothing effect, the predictions of the adversarial data are more trustworthy and can be used to interpolate the noisy labels.
Kaneko et al. (2019) focused on robust label noise learning in the generation task. They proposed a robust GAN to combat with NL. Specifically, they introduced a noise transition model as an auxiliary classifier for discriminator, which similar to the forward correction method in label noise learning. Their proposal conduct adversarial learning (AT) for generating images. By comparison, we leverage AT for defending adversarial examples and meanwhile correcting NL.
Sanyal et al. (2021) found NL in ST causes the significant adversarial vulnerability. Besides, they found NL widely exists even in some standard datasets (e.g., MNIST and CIFAR-10) and identified NL as one of the causes for adversarial vulnerability in ST. In terms of AT, they empirically found that AT can avoid the memorization of NL by conducting the experiments using fymmetric-flipping noise on AT. By comparison, smoothing effect. Furthermore, we conduct extensive experiments across different noise types (c.g.. asymmetric-flipping noise) on AT and make comprehensive comparisons with ST.
figure out why AT avoid the memorization of NI., i.e., AT's
To sum up. our main point is understanding the interaction of AT with NL, which is different from the previous studies. Specifically, we have shown the AT's smoothing effects on NL and identified that AT itself an NL correction. Therefore, compared with ST, AT can avoid the memorization of NL and make correct (clean) data and incorrect (noise) data always distinguishable. Besides the discoveries. we proposed new measure- -PD steps that can stratify correct/incorrect and typical/rare data in AT, which may provide a new perspective for sample selections.
B THE SMOOTHING EFFECTS OF ADVERSARIAL TRAINING
In this section, We provide the detailed setup and more results the synthetic binary dataset and the real-world dataset (CIFAR- 10) with noisy labels, which demonstrate the smoothing effects of adversarial training (AT).
13

---

Under review as a conference paper at ICLR 2022
ST
AT (PGD-1)
AT (PGD-2)
AT (GGD3)
AT (PGD-4)
Figure 11: The results of standard training (ST) and adversarial training (AT) on binary dataset with noisy labels. Dots denote correct data, while squares denote incorrect data. The color gradient represents the prediction confidence: the deeper color represents higher prediction confidence. In the leftmost panel, deep networks shapes two small clusters (red and blue ones cross-over areas) around two incorrect data due to memorization effects in ST. As the number of PGD iterations increases, the smoothing effects in AT gradually strengthens, and two small clusters gradually shrink until they disappear in the rightmost panel. Namely, these clusters have been smoothed out in AT (PGD-4). Boxes represent the norm ball of AT.
Experimental setup. To construct synthetic binary dataset, we randomly generate 23 points (i.c., (a., b), where (0, and b (0, 1)) with binary labels (i.e., "0" and "1") two-dimensional plane. Among all data, we choose two points to assign incorrect labels. For the binary classification, build simple network contains linear layers and ReLU (Nair & Hinton 2010 layers. We train the simple network in ST and AT using Adam with the initial learning rate=0.001 for 1000 iterations. In AT, we the perturbation bound 0.08 and the PGD step size 0.02.
Result. In Figure 11 We plot the classification results in the two-dimensional plane for both ST and AT. We use different PGD iterations to generate adversarial examples, which shows the smoothing process dynamically. In ST, deep network will shape two small clusters around two incorrect data due to memorization effects. While AT, these small clusters will gradually shrink until they disappear, as the smoothing effects in AT strengthens (i.e., from PGD-1 to PGD-4).
Neighborboods of incorrect data
Neighbor of incorrect data
2.25
2.25
2.00
2.00
Awerge HDW
1.75
1.75
1.50
1.50
1.25
Averuge
1,25
1.00
Notse (AT)
1.00
0.75
Norre rate 0.4 (ST)
0.25
0.50
Notoe rate
0.4 (AT)
0.50
20
40
(
80 100
Eporhs
Epochs
(a) Random direction
(b) Adversarial direction
The average entropy of models trained by ST and AT. This value is calculated on 100 points in each neighborhood of incorrect data, using CIFAR- 10 with symmetric- flipping noise. Both solid and dashed lines represent ST and AT, respectively. Note that ST learns incorrect data more deterministically than AT.
Figure 12:
Result. In Figure 12 we plot the entropy values of the model predictions on the CIFAR-10 dataset. In Figure 12(a) we randomly select 100 points each neighborhood (within a small c-ball, where c= 0.031) of the incorrect data and calculate their average entropy values in training using the models trained by ST and AT. In Figure 12(b we generate the adversarial variant for each incorrect data by PGD-1 attack with = 0.031 and calculate the average entropy values in training. The detailed training settings can found in Appendix C.I On the whole, compared with ST, the entropy values in AT always higher. It demonstrates that AT did not learn incorrect data with their neighborhoods deterministically. which confirms that the smoothing effects in AT prevent incorrect data from forming small clusters during training.
14

---

Under review as a conference paper at ICLR 2022
C KNOCK-ON EFFECTS OF ADVERSARIAL TRAINING
In this section, we provide more complementary experiments and analysis for the positive knock-on effects of AT. First, we show the results of standard training and test accuracy on CIFAR- and MNIST datasets with different noise rates and types (Appendix C.I). Second, we show the analysis of the natural data and adversarial data in AT (Appendix C.2e). Third, we use different networks to investigate positive knock-on effects of AT with noisy labels (Appendix C.3). Finally, we show the results of loss value with different noise rates and types (Appendix
C.I TRAINING ACCURACY AND TEST ACCURACY
Natural training data in ST
Notural traizing cata AT
Natural terining 2a ST
Notural traiingg čata in
IE0
Stasted oosuney (K;
Stander ooxuery [SC
Stander acouney (SC.
Stoodord acomery (SC,
8%
6
55
30
-00
62
30
100
12
Equrta 100
100
Epocto
Fyech
Epoda
(a) CI
(b) MNIST
Figure 13: The standard accuracy ST and AT on correct/incorrect training data using CIFAR-II and MNIST with coametriccfllppinng noise. Solid lines denote the accuracy of correct training data, while dashed lines correspond to that of incorrect training data. Compared with ST, there is large performance gap in the standard accuracy of correct/incorrett training data in AT.
Natural training data in ST
Notural traizing ćata AT
Natural terining data
Notural traming cata in
Skaddal accuaa (5)
S0
Standand axcrnn (S]
Standad secunee (5)
18
Stanked accuney (50
5
5
103
100
20 Epoche
100
100
Epoche
Epoche
Epochs
(a) CIFAR-10
(b) MNIST
Figure 14: The standard accuracy of ST and AT on correct/incorreet training data using CIFAR-1O 10 and MNIST with fair-flipping noise. Solid lines denote the accuracy of correct training data, while dashed lines correspond to that of incorrect training data. Compared with ST. there a large performance gap in the standard accuracy of correct/incorrect training data in AT.
Experimental setup. We conduct our experiments two datasets with different noise rates (e.g.. [0.1, 0.4)) and different noise types (e.g.. symmetric-fiipping noise and pair-flipping noise). We use the method in (Han et al., 2018) generate noisy training data. For the CIFAR-10 dataset, we normalize it into [0, IJ: Each pixel is scaled by 1/255. We perform the standard CIFAR-10 data augmentation: a random 4 pixel crop followed by a random horizontal flip. For the MNIST dataset, we normalize it into [o. 1]. We train ResNet-18 18 in ST and AT using SGD with 0.9 momentum for 100 epochs on CIFAR-10 dataset. The initial learning rate is 0.1 divided by 10 at Epoch 30 and 60 respectively. The weight decay=0.0005. For MNIST dataset, 2019a), and set the initial learning rate as 0.01. The rest of the settings remain the same as training on CIFAR-1O. In AT, we set the perturbation bound
use SmalICNN (Zhang et al.)
0.031, the PGD step size a 0.007, and PGD step numbers K = 10. For standard evaluation, we obtain standard accuracy on natural training data according correct/incorrect labels and natural test data with all correct labels. For the robust
15

---

Under review as a conference paper at ICLR 2022
evaluation, obtain robust accuracy on adversarial training and adversarial test data. The adversarial test data are generated by PGD-20 attack with the same perturbation bound c = 0.031 and the step size 0.031/4, which keeps the same as Wang et al. (2019). All PGD generation have a random start, i.e, the uniformly random perturbation of -, 6 added to the natural data before PGD iterations.
Result 1. In Figures 13 and 14 plot the standard accuracy of correct/incorrect training data with different noise rates and types. On the whole, compared with ST, correct/incorret training data can be always distinguishable in AT regardless of noise rates and types. Compared with symmetric-flipinng noise, AT has better performance on distinguishing correctincorreet data with pair-fipppigg noise. Note that, under the same noise rates, the standard accuracy on incorrect training data AT with pair-flipping noise is lower than that with symmetric-flipp flipping noise.
Natural tost data in ST
Naturl tet data in AT
Naturrl cest data in ST
Naturl text data in AT
Stanked wenney (6)
Stadiad acorary (8)
Stardadd (4)
Stadiad woneyy (6)
8
Side nabi 1.2
2
15
2
8
Epochs
Rgodbs
Spotis
Epodis
(a) CIFAR-10
(b) MNIST
Figure 15:
The standard accuracy of ST and AT on natural test data, where training data using CI and MNIST with symmetric-ffipping noise. Note that the larger noise rate causes the test accuracy of ST dropping more seriously due to memorization effects in deep learning, while AT alleviates such negative effects.
Natural tor data in ST
Naturl tet dats in AT
Natural nest data in ST
Naturl tett data in AT
Stadded scoeary 10
Stadded A000De 40
R 8 8
Sander woneey (ig
Stardad scnnecy
60
5 et Et 1 15
Sidae rabli 122
8 a a
15
63
Egudlss
Rgodbs
Sperise
Rgodis
(a) CIFAR- 10
(b) MNIST
Figure 16: The standard accuracy of ST and AT on natural test data, where training data using CIFAR-10 and MNIST with pair-flipping noise. Note that the larger noise rate causes the test accuracy of ST dropping more seriously due to memorization effects in deep learning, while AT alleviates such negative effects.
Result 2. In Figures 15 and T6 we plot the standard accuracy of natural test data with different noise rates and types. On the whole, AT can alleviate the negative effects of label noise due to memorization effects in deep learning. Under each noise type, as the noise rate increases, standard accuracy of ST on natural test data drops more seriously in the later stage of training (c.g.. after 60 epochs in CIFAR- 10). In AT, we only observe that the larger noise rate causes the lower standard accuracy on natural test data, while the overfitting phenomenon is not obvious. Compared with CIFAR-IÓ. both symmetire- symmetire-fipping flipping and pair-flipping noise have more serious negative effects on MNIST, while simply using AT can alleviate these effects to greater extent. In Figure 17 we also visualize the loss landscape (Li et al. 2018) of models trained by ST and AT on CIFAR-10. Such visualization can further substantiate that AT mitigates negative effects of label noise via the lens of the model generalization. Namely, the loss landscape w.r.t. weight space an adversarially trained model (i.c., AT') is smoother and flatter than that of model using ST.
16

---

Under review as a conference paper at ICLR 2022
(a) ST
(b) AT
Figure 17: The loss landscape weight space of models trained by ST and AT using CIFAR-10 1- with 20% symmetric-flipingg noise. The red/blue colors denote large/small values, which reflect the relative position in the loss landscape. Note that the loss landscape of a model trained by AT is smoother and flatter than that by ST, which reflects the better model generalization by AT.
We also conduct experiments on a harder dataset, i.e., CIFAR-100. The smoothing effect of AT is not sigmificant as that on CIFAR-
MNIST but can also be found on this dataset. Specifically, there is a larger performance gap in the standard accuracy of correct/incorrec training data in AT (e.g. 24% -30%) than ST (e.g., 0.15% -0.25%), and the test accuracy of ST dropping more seriously (e.g., 2% -16%) than AT (e.g., 1 1% -7%).
C.2 NATURAL DATA AND ADVERSARIAL DATA
Natural training AT
PGD-10 trvining
Robos (50
Standed aecasv (R3
47.0
Soler
an.
200
1000
100
10
01
200
0000
10
30
K0
70
90
100
Fpocha
Spartox
(a) Training accuracy
(b) accuracy
Figure 18: The standard/robust accuracy AT on natural training data, adversarial training data (PGD-10), adversarial test data (PGD-20) using the CIFAR-1O dataset with symmetric-flipping noise.
we plot the standard and robust accuracy on natural data and adversarial data PGD-10 training data and PGD-20 test data) using CIFAR-IO with symmetire-fipping noise. Different from ST. each natural training data will generate corresponding adversarial data in AT. We also check the difference robust accuracy between correct and incorrect adversarial data during training. We found that AT can also distinguish correct/incorrec adversarial data over the whole training process. However, we find that the difference between correct and incorrect adversarial data (right panel in Figure 19(a) is smaller than that between incorrect and correct natural data (left panel Figure 19(a.
Result. In Figure 18
C.3 DIFFERENT NETWORKS
Result. In Figure 19 plot the standard accuracy on natural training/test data using ResNet- 10. ResNet- 18, 2e and ResNet-34 trained by ST and AT. We conduct the experiments using CIFAR-10 dataset with 20% symmetric-fipping noise. The training settings keep the same as Appendix C.1 We find that, using different networks, AT still has a better performance on distinguishing correct/incorrect data compared with ST and can alleviate the negative effects of label noise.
Result. Figure 20 we plot the standard accuracy on natural test data using a large deep network, Wide-ResNet (e.g. WRN-32-10 (Zagoruyko & Komodakis 2016)), trained by ST and AT. We
17

---

Under review as a conference paper at ICLR 2022
Natural
Natural traiting
Naturaal tost ST
Natural test data AL
100
160
aecmeeyy 290
Standard oonmeyy 199)
accurey
Standord ooumer
Rt
(a) Training accuracy
(b) Test accuracy
Figure 19: The standard accuracy of AT natural training/test data using the CIFAR-10 dataset with 20% cramnttrccflipping noise. We conduct the experiments using ResNet-10 10, ResNet-18, ResNet-26 and ResNet-34.
110
Stealan accunes (25)
88
Stedland acceues E
accezey 5
Steddan
303
131
22
43
12)
300
130
20
300
1.51
20
41
121
120
Epoda
Rgoda
(a) symmetire-fipping noise
(b) pair-flipping noise
Figure 20: The standard accuracy of AT on natural test data using the CIFAR-10 dataset with symmetric-flippine flipping and pair-fippping noise. We conduct the experiments using WRN-32-10.
conduct the experiments using the CIFAR-IO dataset with different noise rates and types. We train the network for 120 epochs and the weight decay=0.0000 the rest of the settings keep the same as Appendix C.I We find that AT can still alleviate negative effects of label noise due to memorization effects of deep networks. Particularly, compared with the coametric-flipinng noise, AT has better performance on avoiding memorization of pair-flipping noise, which can be confirmed by Figures) 15 and
C.4 THE LOSS VALUE
Result. In Figures 21 and 22 check the loss value of conrect/incorrect training data with different noise rates and types. On the whole, compared with ST, correct/incorreet training data can also be more distinguishable in AT using the loss value, regardless of noise rates and types.
18

---

Under review as a conference paper at ICLR 2022
Naturi sraining date in ST
Natural training daza AT
Natuoai training datc. ST
Natural training date AT
Los voloe
Los valoo
Epocts
Dpodh
Epodao
Epodas
(a) CIFAR-10
(b) MNIST
Figure 21: The loss value of ST and AT on correct/incorreet training data using CIFAR-IO and MNIST with symmetric-ffipping noise. Solid lines denote the loss value of correct training data, while dashed lines correspond to that of incorrect training data. Compared with ST, there is large gap in the loss value of correct/incorret training data in AT.
Natural trimingg čata in ST
Natural training date AT
Notural training data ST
Naturl training data m AT
wine
Lon velte
Loos Wille
3.0
0.7
0.0
Epodis
Roodh
Eporis
Epodis
(a) CIFAR-10
(b) MNIST
Figure 22: The loss value of ST and AT on correct/incorrect training data using CIFAR-1O and MNIST with pair-flppiig noise. Solid lines denote the loss value of correct training data, while dashed lines correspond to that of incorrect training data. Compared with ST, there is a large gap in the loss value of correctiiccrrect training data in AT.
19

---

Under review as a conference paper at ICLR 2022
D NEW MEASURE: GEOMETRY VALUE K
In this section, we provide more experimental results of the geometry value x. vs. that of the loss value, and provide more visualization about the specific semantic information corresponds our new measure. First, we calculate the loss value and geometry value k of correct/incorrect data in AT with different noise rates and types (Appendix D.I). Second, we display more visualization results on CIFAR-10 10 and MNIST datasets to show the relationship between the geometry value K and image data (Appendix D.2 Moreover, we also discuss the geometry value & with different PGD configurations (Appendix D.3
D.1
GEOMETRY VALUE vs. LOss VALUE
Result. In Figures and 24 we plot the density maps of two measures on CIFAR- 10 dataset with symmetric-flipping and pair-flipping noise. We calculate the loss value of natural data and the geometry value in AT using 5 checkpoints at different epochs (e.g., Hpoch20, Epoch40,, Epoch60, Epoch80, E0oc which trained with the same settings in Appendix[C.1] We perform the min-max normalization TTxx & Duin, 2000) on both loss value and geometric value K, which scales the range of values in [0, 1j. On the whole, is clear that the geometry value x has a stable performance of distinguishing correct/incorrect data under different noise rates and types. Specifically, under the pair-flipping noise with the large noise rate (e.g., Noise rate: 0.4), the loss value cannot differentiate correct/incorrect data well, while the geometry value K can still have a satisfied distinguishing performance.
D.2 DISTINGUISH RARE AND TYPICAL DATA
visualize more results about the semantic information of images corresponding to different using CIFAR-10 and MNIST datasets. For obtaining the geometry value N, we use the GA-PGD method proposed by (Zhang al., 2021b), which calculates the least number of iteratiins that PGD needs to find the mis-classified adversarial variants of input data. On CIFAR-1O. it is calculated by PGD-10 attack with the perturbation bound e 0.031 and the step size a = 0.007. On MNIST, the geometry value is calculated by PGD-40 attack with the perturbation bound = 0.3 and the step size a = 0.01. In general, the geometry value k can represent whether the data is relatively typical or rare.
Result. In Figures 25 and 26
About Rare and Incorrect Data. In AT, the incorrect data are actually far away from the decision boundary, because AT's smoothing effect prevents memorizing the incorrect data (see Figure I Therefore, it nearly requires 0 steps. Rare data are near the decision boundary and requires a few steps (e.g., 1 or 2) find its misclassified adversarial variants.
D.3 IMPACT OF PGD CONFIGURATIONS
with Specifically, we plot more results as Figure with different step size Figure 27 and the e-ball 28 To be specific, we keep the same setting adopted in Madry et al. (2018) (i.e., step size a = 2.5 x e/ n, where is the step number). With the same e-ball, we can find that the smaller step size (i.e., with the larger step number n) can provide a nuanced stratification compared with the larger one, which means that the data with similar loss values can have more different values. With the same step size a, a small ball radius can not well stratify the data since the PGD attack may never be able to successfully attack some examples.
Back to the proposed geometry value k, the key idea behind it is to approximate the distance from a specific sample to the decision boundary. A small step size can provide nuanced approximation of the distance. Thus, it may be able to provide a nuanced stratification. Since the λ value is captured by the adversarial attacks, large radius e-ball is needed provide the sufficient attacking strength for successful adversarial attack to reach the decision boundary. Our previous experimental results have shown that the parameters of PGD in Madry et al.) (2018) (widely considered in related literature) is an appropriate choice to realize the correct/incoreet or typical/atypical data stratification in the benchmarked MNIST and CIFAR- 10 datasets.
20

---

Under review as a conference paper at ICLR 2022
Donsky
-6.53m
35
65
Densty
-0.500 05
-0505 25 L0
-6590 05 10 -0583 15
15
-6500 65 10
Spodnn
Geometry wilue k
(a) Noise rate: 0.1
Density
Ipech20
Eocet4:
Spechd0
Raceh80
Escer102
Donaky
-0.533 0z 14 15
-0563 35 10
15
-0500 61 34 15
-0503 15 14 15
-5500 at 14
Geometry value
(b) Noise rate: 0.2
Density
Spocie0
spoin0
Eaucres
voss natura data
Denaity
0.500 10 15
0503 10 13
0.500 05 30 15
0502 15 15
5.500 03 10 15
Geometry
(c) Noise rate: 0.3
Density
Spolhzg
Sodee
Bausies
Escc1107
Loss value of natural data AT
Denarty
-5500
Facetec
Trerhin
(d) Noise rate: 0.4
Figure 23: The density of AT correct/incorrect data using CIFAR-JO with symmetric-flippin noise. Top panels: the loss value in AT. Bottom panels: the geometry value k. in AT. Note that the geometry value k has better distinction on correct/incorret data.
21

---

Under review as a conference paper at ICLR 2022
Densiry
Spochzo
Exse145
Spoin0
Escch103
Loss value of Fraauraal data AT
Densty
-03.0
00
Spech20
Boce140
Spochd0
Raceh102
Geometry
(a) Noise rate: 0.1
Cerect
Danskt
-0.5an
-4583 16 -4.500
-0562
35
-55000 65 1/
Eperhoa
Eateté:
Spochin
Epor765
Eare1103
Loss value of natural data in AT
Denstty
Specno0
Epochso
Epicres
Earcht05
Geometry wilte k
(b) Noise rate: 0.2
Denstty
-0
-0500 0.
-0502
-2.000 65
Spoch20
Coseh45
Spech50
Epcer8:
Coseh102
Loss value of natural data in AT
Densty
Spooh20
Eaceh45
Specido
Eaceres
Esseh105
Goometry value k AT
(c) Noise rate: 0.3
Comect
-0.5.6 0. 1.0 1.5 -6500 65 13 1.
-0536 05 1.0 1.5 -5500 5.5
10 155
-053.6 3,0 15
value
Esccia))
spouis
Geometry value An
(d) Noise rate: 0.4
Figure 24: The density of AT on conrectincorrect data using CIFAR-IO with pair-flipping noise. Top panels: the loss value in AT, Bottom panels: the geometry value x in AT. Note that the geometry value & has better distinction on correct/incoreect data.
22

---

Under review as a conference paper at ICLR 2022
plane
KC: 0-5
K: 5-10
AS:
N:
K: (15
K: 5-10
AC: 10
bird
cat
0
AC: 0-5
K: 5-10
K: 10
Kt: 0
K:: 0-5
K:: 5-10
10
deer
dog
K:: 0-5
K: 5-10
10
N: 0
(1-
5- 6-10
10
frog
horse
PC: 0-5
Pc: 5-10
K:
10: 5-10
K::
ship
truck
K: 0
AC: 0-5
:C: 5-10
K: 10
R:
K:. 015
KC: 5-10
ACC 10
Figure 25: The geometry value к w.r.t. images in CIFAR-10 with 20% symmetric-ftippiin noise. The lefumost the given label all images on the right. We randomly selected four examples with the different (k = 0, E (0, 5), K E (5, 10), = 10) in each class. As the geometric value increases from left (x = 0) right (K 10), the semantic information of images more typical and recognizable.
23

---

Under review as a conference paper at ICLR 2022
d
6
6
1
3
0-20
20-40
40
0-20
20-40
40
2
2
9
7
3
3
0-20
H::
R:
R:
R: 20-40
7
4
4
5
S
S
K: Q
K: 0-20
K: 20-40
K: 40
A:
K: 0-20
N:: 20-40
K:: 40
H
b
6
4
1
4
7
0
0-20
20-10 10
K: 40
K: 0
K: ()-20
N: 20-10
40
3
8
7 R:
a
9
0-20
K::
R:
R: 20-40
Figure 26: The geometry value x w.r.t. images in MNIST with 10% symmetrie-flipingg noise. The leftmost is the given label of all images on the right. We randomly selected four examples with the different x (K = 0, K E (0, 20). x E (20, 40). x = 40) in each class. As the geometric value k increases from left (K 0) to right (N 40), the semantic information of images is more typical and recognizable.
24

---

Under review as a conference paper at ICLR 2022
Step number: 2
Step number
Step number:
Step number:
Correct data
Correct date
Correct datt
Correct datte
cort 6303
incorrect da5a
Incomrect dota
Incorrect data
Gooonnc valle
Geomeny volue
Loss value//
Loss value'
Step number: 10
number:
numbar:
number:
Ocomodry voUr
Correct ciota Incoreect data
HEROWOED
Loss value
value
Figure 27: The geometry value k under the different PGD step numbers n with the 8/255 and the step size a 2.5 x e/n. Note that the smaller step size a (i.e., with the larger step number n) can provide a nuanced stratification compared with the larger one.
== 8/255
== 12/255
Correct data
Correct data Incorrect data
Incorrect
Geometry value
Geomeetr valuee
12
Loss
== 16/255
- 20/255
Correct data Incorract
Correct data Incorrect data
Geometry value
Geometry value
12
09
18.00
0:00
(86300
Loss value
Loss value
Figure 28: The geometry value x under the different e-ball with the step numbers n = 8 and the step size a 2.5 e/n. Note that small ball radius can not well stratify the data since the PGD attack may never be able to successfully attack some examples (i.e., even steps it can not find misclassified variant).
25

---

Under review as a conference paper at ICLR 2022
E APPLICATIONS OF GEOMETRY VALUE K
In this section, we provide the detailed experimental setups for robust annotator and confidence scores, First, we provide the detailed version of Algorithm (i.e. Algorithm (2) and the details to implement the experiment in Figure 9 experiment Figure TO (Appendix
(Appendix E. Second, we provide the details to implement
Motivation. Both the two applications are of much significance for obtaining high-quality labeled data. First, since the co-existence of noisy labels and adversarial examples is very realistic, we need to consider training the annotator from noisy training data, and make it robust to adversarial manipulation in unlabeled data. Second, since the label amnotations are not always correct, need the confidence score to show whether the annotation is trustworthy or not.
E.1 ROBUST ANNOTATOR
In Figure 9 we compare four methods on the CIFAR-1O 10 dataset, namely, our robust annotator with 20% symmetric-flippin noise, the PGD-based annotator with 20% symmetric-flppiine noise, the PGD-based annotator without noise, and the standard annotator without noise.
Algorithm 2: Robust Annotator Algorithm (in detail).
Input :newwork fo, training dataset S {[ 34) 10)1-11- learning rate n, number of epochs batch size m, number of batches M, threshold for geometry value K, threshold for loss value
Output:robust annotator fo- for epoch= = ........7. do
for mini-batch I..... M do Sample: a mini-batch (x.;,,)) )m from S.; for i I...., (in parallel) do
Calculate: ni and l of (24.3t by GA-PGD method (Zhang al. 2021b) and ((fo(xz), Yt), respectively: K; < K and & > then Update: y arg max; fo(x).: end
Generate: adversarial data ãc by PGD method (Madry et al. 2018
end
Update: 0 --V-
end
end
Choice of Thresholds. As for the choice of the thresholds in learning with noisy labels, we can some existing methods to estimate the noise of dataset and then we the dynamic thresholds based on the two values of the training data (e.g., greater than or equal to the top 20% largest values).
Experimental setup. To generate the noisy training data, randomly assign the wrong label for a part of correct training data using the method in (Han et al. 2018). For the CIFAR-1O dataset, we normalize into [0, 1]: Each pixel is scaled by 1/255. We perform the standard CIFAR-10 data augmentation: a random 4 pixel crop followed by a random horizontal flip. For all annotators, we train WRN-32-10 (Zagoruyko & Komodakis 2016) for 120 epochs using SGD with 0.9 momentum. The initial learning rate is 0.1 reduced to 0.01, 0.001 and 0.0005 at epoch 60, 90 and 110. The weight decay is 0.0002. For standard annotator, we use natural data to update the model. For our robust annotator and the PGD-based annotator, we generate the adversarial data to update the model, the perturbation bound Gtrain = 0.031, the PGD step is fixed to 10, and the step size is fixed to 0.007. All PGD generation have random start, i.e, the uniformly random perturbation of [-c, added to the natural data before PGD iterations. For our robust annotator, we use the same generation method with PGD-based annotator as previous mentioned before Epoch 40. After that,
Algorithm 2
26

---

Under review as a conference paper at ICLR 2022
train robust annotator. We set the threshold for geometry value K 2 and the threshold for loss value L to the loss value 20% m largest natural data in each mini-batch, where the m = 128 is batch size. We use the model predictions of the selected natural data their new label to generate the adversarial data. As for the evaluations, we select a part of natural test data on the test set of CIFAR-/O /0 to add adversarial manipulations by PGD-20 attack. The perturbation bound étest 0.031, the step number is 20, and the step size = éteat 4, which keeps the same as Wang et al. (2119) We use the natural and adversarial test data to check the performance of annotators on assigning correct labels for the U data. Note that, the experiments are conducted using Tesla V100-SXM2, which takes about 8 hours for each individual trial.
E.2 CONFIDENCE SCORES
In Figure 10
plot the accuracy and number of correctly predicted U data w.r.t the geometry value
6.
Experimental setup. We train ResNet-18 model in AT with 20% symmetric-flippnng noise on the CIFAR-1O dataset. The training settings keep the same as Appendix C.I] We use the model checkpoint at Epoch 35 for assigning labels and we randomly select 2000 test data in CIFAR- /0 as unlabeled data. We run the test with 5 repeated times with different random seeds for selecting different test data, In the left panel of Figure [0] we calculate the mean and standard deviation value of accuracy. In the right panel of Figure 10 we show the number of correctly/wrongl predicted data one of the experiments.
27


Reviews:
[{"review_id": "rU4Rdxfy9C", "reviewer": "Reviewer_FKyS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper studies the usefulness of adversarial training to mitigate the impact of incorrect labelling. I believe this is an interesting topic worth further investigation. However, I have the following concerns regarding the correctness, rigor, novelty, presentation of the paper.\n\n[Part A] Some claims or designs are not rigorous or reasonable.\n\nA1: In the last paragraph of page 4 and the second paragraph of page 5, the authors compare the cases of AT in MNIST and CIFAR10, but I do not believe such comparison is appropriate. I agree that CIFAR10 dataset is more difficult than MNIST, however, we use a much larger adversarial budget on MNIST (0.3) than CIFAR10 (8/255). AT on MNIST is not necessarily easier than CIFAR10.\n\nA2: In Section 3, the authors claim that AT have the smooth effect to prevent the model from fitting the incorrectly labelled data. In this regard, why don't the authors calculate the local Lipchitz constant in the neighbourhood of the incorrectly labelled data? From my point of view, local Lipschitz constant better depicts the smoothness. The Shannon entropy in Equation (3) only demonstrate how confident the model is to the prediction.\n\nA3: In Section 5, the authors claim that the geometry value k is a better metric to distinguish the correct and incorrect labels, while in Algorithm 1, both k and the loss values l are utilised to filter the incorrect labels. This seems contradictory. In addition, it would be difficulty to set an appropriate threshold value L in Algorithm 1. I think the author should conduct ablation study on the condition (the \"if\" line in Algorithm 1) to filter incorrect labels and show the geometry value is a better metric than loss value.\n\nA4: In Figure 9, the proposed \"Robust annotator\" have only very marginal improvement when the ratio of adversarial corrupted U data is big. When this ratio is smaller than 0.7, which seems a more reasonable setting, the proposed method is worse than PGD-based annotator.\n\n[Part B] I don't think some discoveries of this paper is novel. Some findings are already included in previous works and is not surprising. Contribution of this paper is limited.\n\nB1: Ref[a] already pointed out ReLU network tends to give overconfident predictions under standard training (ST). In addition, Ref[a] also finds that adversarial training is a kind of calibration to mitigate this overconfident prediction issue. Their findings are very similar, if not identical, to the findings in Section 3 and Figure 2 in this paper. \n\nRef[a]: Hein, M., Andriushchenko, M., & Bitterwolf, J. (2019). Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 41-50).\n\nB2: The authors point out AT can better preserve the standard test accuracy than ST when the training data has label noise. This finding is of little practical usage. For example, ST can utilise a label noise-free validation set, which is much smaller than the training set, to do early stopping. Through this way, ST can quickly achieve nice clean test accuracy and is faster than AT.\n\nB3: The contributions of this paper is limited on top of Ref[b]. Ref[b] theoretically and empirically demonstrate that networks that converge to data of label noise have high adversarial vulnerability. The equivalent claim of this is network that is adversarially robust should have larger loss on the training data of label noise. This is exactly that Figure 1 and Section 3 in this paper discuss. Ref[b] has theoretical justifications while this paper only provides empirical evidence.\n\nRef[b]: Sanyal, A., Dokania, P. K., Kanade, V., & Torr, P. (2020, September). How Benign is Benign Overfitting?. In International Conference on Learning Representations.\n\nB4: This paper lacks theoretical justifications and quantitative study on this phenomenon in terms of the perturbation bound $\\epsilon$. We know ST is a special case of AT with $\\epsilon = 0$. With the increase of $\\epsilon$, ST can gradually shift to AT. I believe adversarial training with small values of $\\epsilon$ would demonstrate a more similar behaviour to ST. A quantitative analysis on this needs to be conducted.\n\n[Part C] The presentation and writing of some part need improvement.\n\nC1: I think Equation (3) is the definition of Shannon entropy, why it is $p(x, y) log p(y|x)$ instead of $p(y|x) log p(y| x)$. Note that $\\sum_y p(y|x) = 1$ but $\\sum_y p(x, y) \\neq 1$. We do not know $p(x, y)$ or estimate it in practice, because $p(x, y)$ is the joint distribution.\n\nC2: Some wordings need improvement. For example, in the first point of the contribution: \"AT can always distinguish ...\" I do not think AT can ALWAYS does so. There are several similar mistakes in other sections.", "summary_of_the_paper": "This paper studies the adversarial training in the context of label noises. Specifically, it is discovered that adversarial training can prevent the model from overfitting to the label noises, leading to a more smooth landscape. In addition, the authors point out that the number of PGD step sizes can be considered as a useful metric to distinguish instances with correct or incorrect labels.", "summary_of_the_review": "Based on the questions and concerns above. Unfortunately, I cannot recommend acceptance of this manuscript to ICLR. I welcome the author to post rebuttals and will do a re-evaluation after the our discussion ends.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "UIKVfg6I8Nv", "reviewer": "Reviewer_cfM4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- The question they study is certainly well-motivated: label noise and adversarial robustness appear to touch on two separate notions of robustness, yet it seems plausible that adversarial training might avoid running into the issues that vanilla training has with overfitting noisy labels in the training data. This paper explores this potential connection from a variety of different angles beyond just the basic question of whether AT can improve generalization in the presence of label noise. \n- The \"geometry value\" seems like a useful proxy for discerning incorrectly labeled points from correctly labeled points.\n\nWeaknesses:\n- As the paper mentions, the general question of whether AT can avoid memorizing noisy labels has been studied in previous work, e.g. [Sanyal et al. 2021], and the geometry value was introduced in a previous work [Zhang et al. 2021b] as a proxy for the distance to the decision boundary. My understanding is that on the former front, the present paper mainly differs from [Sanyal et al. 2021] in that it provides a candidate explanation for this phenomenon via the so-called ``smoothing effect.'' I found the description of the \"smoothing effect\" vague, but more importantly, I wasn't very convinced that the experiments in the smoothing effect section were sufficiently meaningful. Certainly the 2D synthetic example is too limited in scope to draw major conclusions. I was also confused by the average entropy experiment because it doesn't seem to be related to what the 2D experiment in Figure 1 is suggesting. E.g. take the top-right incorrectly labeled point in Figure 1. In the left plot, the distribution over labels in a neighborhood of that point is concentrated around yellow (corresponding to red), and in the right plot, it's concentrated around black (corresponding to blue). So the entropies in the two cases appear to be comparable? I might just be missing something basic, but it would be helpful at least for me to have more discussion in Section 3 clarifying this point. A related confusion I had about the meaning of the \"smoothing effect\" is that [Sanyal et al. 2021] observed that \"adversarial training does produce more complex decision boundaries,\" whereas the qualitative picture I get out of Section 3 seems to be the opposite?\n- While the geometry value experiments in Section 5 were interesting, it wasn't clear from Figure 10 that it's a useful proxy for annotation confidence. There, because there are so few points with geometry value between 6 and 9, the lower end of the confidence intervals in the left plot in Figure 10 are essentially the same for values 4 to 9.\n- A minor comment: while the abstract makes clear that geometry value was introduced in prior work, various parts of the intro (e.g. P. 2) give off the impression that the definition of geometry value is novel to this submission, so perhaps the wording in these places could be fixed?", "summary_of_the_paper": "This submission empirically studies the efficacy of adversarial training for mitigating the effect of label noise in training data. Their findings are as follows:\n1) \"Smoothing effect\" of adversarial training:\n\ta) on a 2-dimensional synthetic binary classification dataset where two points are incorrectly labeled, they show that vanilla training yields a classifier that memorizes the bad labels by forming \"clusters\" around the incorrectly labeled points, whereas adversarial training does not yield such clusters\n\tb) for CIFAR injected with 20/40% random label noise, they ran vanilla and adversarial training on the noisy data and found that if you look at the distribution over labels within the neighborhood of a random incorrectly labeled point, on average the entropy of that distribution is higher for the classifier obtained by adversarial training than by vanilla training\n2) For vanilla training on CIFAR (also MNIST) injected with label noise, the gap between accuracy on the correctly labeled training data and the incorrectly labeled data closed over the course of training, whereas this gap does not close or seems to close much more slowly for adversarial training.\n3) Over these same noisy datasets, adversarial training seems to mitigate the impact of noisy training data on (clean) test accuracy, unlike vanilla training for which generalization degrades as label noise increases\n4) They consider a quantity they call the \"geometry value\" of a data point (x,y), which corresponds to the number of PGD steps needed to find a differently labeled point in the neighborhood of x. This quantity was originally introduced by [Zhang et al. 2021b], and in the present paper they find that:\n\ta) compared to loss(x,y), it appears to be a more effective way to effective way to distinguish correctly labeled data from incorrectly labeled training data, as well rare data from typical data\n5) They propose a \"robust annotator\" for labeling unlabeled data that has possibly been subject to adversarial perturbations. The algorithm repeatedly alternates between 1) identifying training data points with high loss and low geometry value and re-labeling them according to the current classifier and 2) running a step of adversarial training. It appears to do slightly better than a PGD-based annotator baseline when trained over CIFAR injected with label noise. They also note that the geometry value can provide some kind of \"confidence score\" to go along with the label annotations.", "summary_of_the_review": "The experiments on geometry value were interesting, and the paper asks an important question in trying to relate label noise and adversarial robustness. That said, for the results on adversarial training mitigating overfitting of noisy labels, I'm not sure I fully grasp what the key new conceptual insights are relative to prior work as I wasn't particularly convinced by the definition of or the experiments on the \"smoothing effect\" of adversarial training. So I'm a bit on the fence and leaning towards weak reject, though I'm happy to update my score if the authors could help clarify my confusions above.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PwJCYW0kkRt", "reviewer": "Reviewer_sbre", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths:** The use of _PGD step number_ is intuitive and this intuition is verbally justified in my opinion. Detailed analysis is conducted on CIFAR-10 to empirically justify the benefit of _PGD step number_ over the loss value. The _robust annotator_ method proposed by the authors seems to be bringing improvement to the dataset in consideration (CIFAR-10).\n\n**Weaknesses:** The main weakness of this paper is the lack of sufficient evidence to support the claims and the use of _PGD step number_.  In detail,\n\n* The claims made in the paper are often overgeneralizing, see [1] for some examples. **Question:** I have difficulty understanding how these claims are made so confidently. What evidence are they based on? I understand the intuition from the decision boundary perspective, but I do not think only CIFAR-10 experiments in a very specific setting (pair-flipping noise) are enough to support all these claims. Can the authors clarify?\n\n* Similarly as above, the justification is made neither through a rigorous analysis nor an extensive set of experiments. It does not have to be a theoretical analysis, but only CIFAR-10 experiments with the robust annotator and confidence scores are very limited. **Question:** when will the use of PGD step number break? what are the limitations? what happens for instance when the data has a large number of classes or in more extended noise settings? There is some mention of it in the appendix (CIFAR-100, asymmetric-flipping), but it is either not sufficient enough to get an answer to this in my opinion (CIFAR-100) or must be moved to the main body as needing to read 27 pages (3x page limit) should be prevented in order to see sufficient amount of justification. I think more than half of the main body is giving background; the paper could be written such that the contributions are more richened and highlighted. \n\n[1]  Main contributions on pg. 2, the first paragraph of Section 4\n\n\n**Note to the authors:** I'm willing to review my assessment based on the authors' answers.", "summary_of_the_paper": "This paper focuses on understanding adversarial training in the presence of label noise by conducting empirical studies. Based on their observations, the authors propose to use _PGD step number_ of adversarial training as a new measure for sample selection to correct noisy labels. Moreover, they present two use-cases, namely 1. a _robust annotator_ algorithm to label unlabeled instances, and 2. PGD step number as a _confidence score_ for the labeling of unlabeled instances. Empirical observations are primarily made on CIFAR-10 images.", "summary_of_the_review": "Although the use of _PGD step number_ seems promising, there is only very limited evidence to support the claims in the paper. That is, there is neither a rigorous analysis/formulation nor a comprehensive empirical study to justify the use of _PGD step number_. Hence, I believe significant improvement is needed for this paper to back up its claims, leading my decision to a rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-Yi6NQ7dNjJ", "reviewer": "Reviewer_BbnD", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "> Strength of this paper: this paper addresses an interesting and novel problem connecting adversarial training and noisy labels. The writing and clear and easy to follow. The proposed method is intuitive yet effective. Extensive experimental studies were conducted.\n\n> Main concern: The results are highly dependent on the hyper-parameter values, for example, epsilon (adversarial attack budget), and learning rate alpha in the PGD attack. Intuitively, lower epsilon and lower alpha can both lead to a higher number of iteration in the PGD attack.\n\n> In Figure 7, the overlap between correct and incorrect data's density of AT is large. I am wondering if there is a confidence value or statistics number to measure the distributional difference? I am not sure if merely mean/std can capture this difference.\n", "summary_of_the_paper": "This paper studies the connection between noisy labels (NL) and adversarial training (AT). The contribution of this paper is two-fold. The first one is to adopt the number of PGD attack steps as a criterion for sample selection to correct noisy labels. The second one is that adversarial training can serve as a way to correct noisy labels. These two contributions indicate that adversarial training can be applied to more general model robustness problems.\n\n", "summary_of_the_review": "An interesting connection is studied and the contribution is clear. But the work lacks theoretical insight and quantitative results. So I would recommend only a borderline accept.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the proposed idea looks interesting but the paper is seriously lacking in the definition of its scope: there is no quantitative result, experiments are quite limited, and there is not enough discussion of the limitations. With more work this could become a very interesting paper.
Decision: Reject