Data Quality Matters For Adversarial Training: An Empirical Study
Authors: Chengyu Dong
Liyuan Liu
Jingbo Shang
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial training
Data quality
Robust overfitting
Robustness overestimation
Robustness-accuracy trade-off

Abstract: Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose great challenges to both reliable evaluation and practical deployment. Here, we empirically show that these problems share one common cause --- low-quality samples in the dataset. Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness. We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training. We find that when low-quality data is removed, robust overfitting and robustness overestimation can be largely alleviated; and robustness-accuracy trade-off becomes less significant. These observations not only verify our intuition about data quality but may also open new opportunities to advance adversarial training. 

Under review as a conference paper at ICLR 2022
DATA QUALITY MATTERS FOR ADVERSARIAL TRAIN- ING: AN EMPIRICAL STUDY
Anonymous authors Paper under double-blind review
ABSTRACT
Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose great challenges to both reliable evaluation and practical deployment. Here, we empirically show that these problems share one common cause-low- quality samples in the dataset. Specifically, we first propose strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental the adversarial robustness. We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training. We find that when low-quality data is removed, robust overfitting and robustness overestimation can be largely alleviated; and -accuurrsssacuuracy trade-off becomes less significant. These observations not only verify our intuition about data quality but may also open new opportunities to advance adversarial training.
INTRODUCTION
Adversarial training (Goodfellow et al. 2015 Huang et al. 2015 Kurakin et al. 2017 Madry
al. 2018) is arguably the most effective way to establish the robustness of deep neural networks against adversarial perturbations. Nevertheless, several intriguing problems and phenomena hover in adversarial training practice, including (1) robust overfitting (Rice et al. 2020). (2) robustness
overestimation (esato et al. 22018: Mosbach et al. [018 Croce & Hein 2020b Chen & Gu 2020).
and (3)
trade-off (Papernot et al. 2016)] Su et al., 2018 Tsipras et al. 2019
Zhang et al. 2019)..
In this work, we focus on the data in adversarial training and show that the three aforementioned problems are all interconnected with the quality of the data employed in adversarial training. Specifi- cally, we measure the data quality based on their stability of being learned through training and show that unstably learned examples are of low quality to adversarial training. We show that low- fow-quality data can be one common cause of the three aforementioned problems. To the best of our knowledge, this is the first time the effect of the data quality being systematically studied in adversarially robust learning. This is also the first time that all these problems are investigated in a holistic manner.
As demonstration, we partition the training set of CIFAR-10 (Krizhessky 2009) into two exclusive subsets with equal size and balanced classes, while ensuring all the examples in one subset have higher quality than all the examples in the other one in a class-wise manner. We then conduct both standard training and adversarial training with Projected Gradient Descent (PGD-AT) (Madry et al. 2018). The model is fixed as pre-activation ResNet-18 18 (He et al., 2016). As shown Figure I adversarial training on the high-quality half yields high robustness, almost robust overfitting, genuine robustness against the popular PGD attack (Madry et al.] (018), and minor robustness- accuracy trade-off that is only noticeable in the late stage of training, which is in sharp contrast to that on the low-quality half.
We further conduct controlled experiments three real-world datasets, CIFAR-IO, CIFAR- 100 Krizhevsky 2009) and Tiny- ImageNet (Le & Yang 2015), to explore the interconnection
RRbustnesss overestimation refers to the problem that the adversarial robustness may be spuriously high against certain types of adversaries.

---

Under review as a conference paper at ICLR 2022
Robust Overfitting
Robustness Overestimation
Standard Test Accuracy (%)
Robustness-accuracy Trade-off
Robust Test Accuracy (%)
(a)
High-qua By halrerGp trainin)
(b)
High-qusl by half 06p 10 attick
200
[c)
Lsw Bu hal 1P00 trainng)
Robust Test Accuracy
High-quat by half (uuoteracc)
50-
40-
60
30-
20-
+igh-qus By hart iStanda trainng idpp-qusiity hal (PGD trainigl insequality crandard maining!
10-
Inveqqual by UPC3-10 wTteer
Lnuequaa by (hunsttrrccs
insequallty by IPGD maining!
25
100
125
100
125
150
25
50
75
100
125
150
Epochs
Epochs
Epochs
Figure 1: Despite being equal-size and class- balanced, the high-quality half and low-quality half of the CIFAR-10 10 training set have drastically different behaviors in adversarial training.
between data quality and those three problems in adversarial training. Our experiments cover vari- ous sample sizes, training methods (e.g., PGD-AT and TRADES (Zhang et al. (219)). and neural architectures (e.g., pre-activation ResNet and Wide ResNet (WRN) (Zagoruyko Komodakis 2.1
Our major findings are summarized as follows.
Low-quality data may not be useful or even detrimental to adversarial training.
Low-quality data causes robust overfitting in adversarial training.
Adversarial training with low-quality data can lead to overestimated robustness against certain adversarics such as the PGD attack.
bobbuttness-accuaay trade-off is only prominent when the dataset contains low-quality data. When using high-quality data only, the standard accuracy achieved by adversarial training and standard training is comparable.
The remainder of this paper is organized as follows. In Section 2 we briefly review the existing works that most related to our data-centric investigation of adversarial training. In Section 3 propose strategy to estimate the data quality based on learning stability, and show that unstably- learned data are of low quality to adversarial training. Sections 4.1 4.2 and 4.3 investigate the interconnection between data quality and major problems in adversarial training including robust overfitting, robustness overestimation, and robustness-acquu accuraacy trade-off respectively. Conclusions and further implications are discussed in Section
2 RELATED WORK
Increasing attention has recently been paid to analyzing and improving the quality of the (training) data. Compared with standard learning, only limited number of all learnable features would be beneficial to the adversarial robustness (Ilyas et al.) 2019), thus making data quality more important adversarially robust learning. Existing works have been focusing on the effect of sample size and data distribution in adversarial training. [Schmidt et al. (018) show that adversarially robust learning requires a much higher sample complexity than standard learning. They also show that such sample complexity requirement is highly sensitive the data distribution. Ding et al. (2019) further observe that adversarial robustness achieved by adversarial training is sensitive to semantically- distribution shift. (Shafahi al. (2019) show that the adversarial robustness achieved by any classifier is fundamentally limited by the properties of the data distribution. Our study is distinguished from previous works by investigating the datasets with the same size and within the same distribution. We also systematically study the other important problems in adversarial training beyond the performance.
Towards advancing adversarial robustness, substantially more works have been devoted to customizing the inner maximization or outer minimization in adversarial training based on the properties and behaviors of training data. Representative works including Wang et al. (2020), which shows that misclassified examples have greater impacts on adversarial robustness, and Zhang et al.) (2020b),
which observes that some training data are more robust to adversarial attack and will significantly improve the robustness against certain types of attacks if they are emphasized in adversarial training. Following a curriculum adversarial training framework (Cai et al. 2018), Wang et al. (2019) proposes to gradually increase the convergence quality of adversarial examples through training by observing that training on adversarial examples with high convergence quality in the later stages leads to better robustness. Selecting high-quality data may be more crucial when including extra data into training
2

---

Under review as a conference paper at ICLR 2022
that are either unlabeled (Uesato al. 2019 Carmon et al. 2019 Gowal et al. 2020
generated
by generative models (Sehwag et al., 2021 Gowal et al., 2021). where the predictive probability of model is comomly used to measure the data quality. We review more works the different measures of data quality and their applications works by both methodology and objective. In terms of the methodology, we identify the low-quality examples based on their behaviors of being learned throughout the adversarial training. Our approach to measure data quality yields more fine-grained results than misclassification distinguish the individual examples, and more consistent results than the resistance to perturbation or predictive probability among different training methods and neural architectures (see Appendix D). In terms of the objective, we are not aiming to compete with the state-of-the-an methods, but rather to reveal the important influence data quality on the problems adversarial training through rigorously controlled experiments and reliable robustness evaluation. Such comparative and systematic analyses
Appendix [ Our study is distinguished from these
are lacking in the current adversarially robust learning research.
Due to the space constraint, we will discuss more works specifically related to the benefit of data in adversarially robust learning, robust overfitting, robustness overestimation, and robustness-accuracy trade-off when it comes to Sections 3.2
4.2 and 4.3] respectively. More broadly related work regarding data quality in standard and robust learning will be discussed in Appendix
3 DATA QUALITY IN ADVERSARIAL TRAINING
3.1 DATA QUALITY MEASURE: LEARNING STABILITY AS AN EXAMPLE
We propose to measure the quality of the data in adversarial training based on the frequency of events that an example predicted correctly under adversarial perturbation throughout the training, which we will refer as the learning stability in the rest of the paper. Specifically, given an example x, the model f and the optimizer w, the learning stability s(x; S.w) can formulated as
s(x; f,w) 1 [(t)f(t, x +6) = y(x), t t
(1)
where 5 is a too norm- bounded adversarial perturbation, f(t,-) denotes the classifier at epoch t, (z)) denotes the true label of example x, and T is the total number of epochs the model will be trained. A similar measure in standard training has been utilized by Moon et al. (2020) regularize the confidence estimates of deep neural networks, and is shown be roughly proportional to the probability that example will be correctly predicted by a model.
We find that, although the absolute value of the learning stability can vary a lot (see Appendix C.2). the ranking of examples based on the learning stability is consistent across different experimental setups, such as random initializations, training epochs, training methods, and neural architectures as shown in Figure 2 This suggests that the learning stability can reflect the intrinsic "hardness" of an example in a relative manner.
TIUPE
VIS AWA
3.50
p=0 33
p
2000
2--
p=
(a) Random initializations
(b) Training epechs (c) Different methods (d) Different models
Figure 2: Correlation between the rankings of the examples (scaled by the total number of examples) based on learning stability estimated by different training settings. Spearman's 's rank coefficient (p) is annotated at the upper left of each subfigure. Here we consider adversarial training methods including PGD-AT, TRADES and MART (Wang et al., 2020), and neural architectures including pre-activation ResNet- 18, VGG-19 (Simonyan & Zisserman, 2015) and WRN-16-10.
Towards more accurate estimation of such hardness, calculate the ranking of the learning stability based on the ensemble of multiple experiments (see Appendix C.I). Figure B3 and Figure 3b show samples of the examples with the lowest learning stability and the examples with the highest
learning stability, respectively, in the CIFAR- 10 training set. One can find that unstably-learned examples appear more ambiguous and may not align well with their given labels. Under adversarial
3

---

Under review as a conference paper at ICLR 2022
airplane
automobile
bird
Trog
ship
airplane
automobile
bird
frog
ship
bird
ship
cat
ship
airplane
airplane
automobile
bird
frog
airplane
(a) Examples with the lowest learning stability (b) Examples with the highest learning stability
Figure 3: Examples selected from CIFAR-10 training set by learning stability. The top row shows the original images annotated with their true labels and the bottom row shows the corresponding adversarially-pertue images annotated with their labels predicted by model.
perturbation with relatively small size, they cannot easily be identified even by humans. This aligns with our intuition that unstably-learnet examples may be intrinsically hard.
In standard learning, hard examples are often essential to the model performance, which is manifested in real-world applications such as hard example mining (Kumar et al.) 2010] (hhivvsttaaa al., 2016) and ablation studies the effect of removing hard examples (Toneva et al. (2019). Empirical evidence can also found in our Figure Tc) where standard training on the unstably-learned half yields higher standard accuracy than that the stably-lcarned half.
In adversarial training, rather surprisingly, we observe an opposite case. Unstably-leareed examples often not useful and may even hurt the robustness achieved by adversarial training (see Section (.2)) Therefore, the rest of the paper, we refer those unstably-learne examples as low-quality examples, and measure the data quality relatively using the learning stability. It is worth mentioning that, in Appendix D we show that it is possible to estimate the data quality based on other behaviors and properties of the data (e.g., predictive probability, minimum perturbation, and learning order) and all these estimations will yield consistent findings.
3.2 UNSTABLY-LEARNED DATA IS OF LOW-QUALIN FOR ADVERSARIAL TRAINING
In this section, we conduct controlled experiments on three real-world datsets and show that low- quality examples are not useful and may even be detrimental to adversarial training.
Related work. In adversarially robust learning, solely increasing sample size will not always yield better performance and sometimes even be detrimental. [Din et al. (2019) observes that the robustness achieved by adversarial training plateaus after training size is sufficiently large on MNIST (LeCun al. 1998). Introducing additional data examples may not always help the adversarial robustness if they are unrelated to the original dataset (Uesato et al. 2019 Gowal et al. 2020). Yang et al.) (2020a) shows that proper pruming of the dataset will benefit the adversarial robustness for non- non-parametric classifiers. A recent study also shows that adversarial vulnerability may originate from the label noise in the dataset (Sanyal et al.) 2020), which apparently hurts the adversarial robustness. However, note that low-quality data is not simply label noise after manually inspecting the samples as those in Figure 3a] Moreover, in-depth studies show that the fractions of label noise in benchmark datasets are only marginal (Northcutt et al. 202T).
Experimental setup of controlled experiments. We conduct controlled experiments dissect the effect of training data on adversarial training. Specifically, we gradually remove more examples from the training set and record the robustness achieved by adversarial training from scratch on an increasingly smaller training subset, while all other experiment settings remain the same. The training examples are removed either randomly, or in an ascending order of data quality, i.e., low-quality data will be removed first. By removing the same number of random data and low-quality data, it should be able to climinate the effect of sample size in the comparison and reveal the effect data quality.
We consider the robustness against foo norm-bounded adversarial attack with perturbation radius 8/255 throughout the paper. We evaluate the adversarial robustness against AutoAttack (AA) (Croce & Hein 2020b), one of the adversarial attacks known to be relatively reliable. We employ popular adversania training methods including PGD-AT and TRADES combined with early stopping (Rice)
al. 2020), which have been shown to attain the improvements of almost all other adversarial training
4

---

Under review as a conference paper at ICLR 2022
variants (Croce & Hein 2020b; Chen & Gu 2020 Pang al. 2021). We conduct experiments with pre-activation ResNet- 18 in the main paper and Wide ResNet in the appendix. We present results on three datasets including CIFAR-10. 10, CIFAR-100 100 and Imy-ImageNet. Other experiment details can be found in Appendix The same experimental setup will be applied to Sections 4, 5 and 6 as well.
CIFAR-10
CIFAR-100
Tiny-lmageNet
30
25
PGD-AT
Robust Test Accuracy (%)
20
15
35
Remove randomly
10-
Remove low-quality first
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
30
50-
25
TRADES
45
20
40-
15
10
5-
0.0
Fraction of training examples removed
Figure 4: Best robust test accuracy obtained when removing different fractions of training examples either randomly or in an ascending order of data quality.
Figure shows that removing random examples from the training set degrades the robustness achieved by adversarial training rapidly. In contrast, when removing low-quality examples first, the achieved robustness maintains or even increases. For instance, by PGD-AT, up to 40%, 60%, and 50% low-quality training examples can be removed without reducing the robustness significantly (< 1%) on CIFAR-10, CIFAR-100 100, and Tiny-ImageNet, respectively. More surprisingly, removing examples with the lowest data quality can achieve robustness even higher than that achieved the entire training set for both PGD-AT and TRADES CIFAR-10, and TRADES for other datasets, although the training set is actually becoming smaller. We also validate the above observations with different perturbation sizes and neural achitectures in Appendix B.2
Results & Discussions.
4 DATA QUALITY AND PROBLEMS IN ADVERSARIAL TRAINING
In this section, we investigate the interconnection between data quality and problems in adversarial training by conducting extensive controlled experiments.
4.1 ROBUST OVERHITING
The problem and related work. Robust overfitting is prevalent phenomenon in adversarial training (Rice et al.) 2020). Specifically, the robust test accuracy will constantly decrease after a certain point in adversarial training, resulting in an inferior final performance. Such overfitting occurs consistently across different datasets, training settings, adversary settings, and neural architectures, and cannot be completely eliminated other than using early stopping (Rice et al. 2020
Low-quality data causes robust overfitting. Our controlled experiments suggest that the robust overfitting results from the low-quality examples. Figure already shows that adversarial training on the high-quality half induces almost no robust overfitting. This suggests that robust overfitting is against our conventional understanding of overfitting in the sense that it can be mitigated by smaller training sample size. Figure 5 further shows that when a particular fraction of low-quality examples removed from the training set (c.g.. on CIFAR- 10, about 30% for PGD-AT and 50% for TRADES), the robust overfitting would vanish. The smaller number of low-quality examples are removed from the training set, the more severe the robust overfitting will be. In contrast, when the examples are randomly removed from the training set, the robust overfitting is often prominent consistently across different training set sizes. This suggests that low quality examples cause the robust overfitting, and more low-quality examples will make it more severe.
5

---

Under review as a conference paper at ICLR 2022
CIFAR-10
CIFAR-100
Liny-ImageNet
50
20
45
Robust Test Accuracy (%)
15
PGD-AT
20
35
10
30-
Beroone mindority deed medipeatt
Remrve Lask dheckgorct
5-
20
5
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
50-
15
45
TRADES
20
40
10
35
30
10
25
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
Fraction of training examples removed
Figure 5: Robust test accuracy obtained at the best and last checkpoints by adversarial training when removing training examples either randomly or in an ascending order data quality. The robust overfitting can be captured by the visual gap between the dashed and solid curves of the same color.
Motivated from the feature learning perspective, we conjecture that low-quality examples cause robust overfitting because robust features might be forgotten when low-quality examples are adversarially learned. More detailed explanation can be found in Appendix G.2
Mitigate robust overfitting. Based on the above understanding, to mitigate robust overfitting, one may need to intentionally prevent the model from learning low-quality examples. Removing low-quality examples from the training set and the early stopping (Rice et al. (2020) are the most straightforward methods to prevent such learning. Recent works to mitigate robust overfitting including regularizations of the flatness of the weight loss landscape (Wu et al.) 2020 Stutz et al.) 202T), introducing low-curvature activation functions (Singla et al.] (2021) and adopting stochastic weight averaging (Izmailov al.) 2018) and knowledge distillation (Hinton al. 2015) (Chen et al..
2021). These methods are likely to be consistent with our data-centric understanding as various regularization techniques may suppress the learning of low-quality examples.
4.2 ROBUSTNESS OVERESTIMATION
The problem and related work. We refer robustness overestimation as the problem that the robustness of adversarial training method may appear spuriously high against certain types of adversaries, but be significantly diminished by stronger adversaries (Chen & 2020). Such problem might result from a significantly more difficult inner maximization induced by highly non-linear loss landscape (Qin et al. 2019), thus is more subtle than the conventional gradient masking problem
where the gradients of the model completely not useful (Papernot al. 2017 Tramèr et al. 2018
Athalye et al., 2018 Uesato et al. 2018 Engstrom et al. 2018).
We focus on the PGD attack as it is the most popular method to evaluate adversarial robustness. Previous works have already shown that PGD attack with insufficient iterations (Mosbach et al. 2018) and fixed step size (Croce & Hein 2020b) may yield misleading robustness. Existing adversarial training methods may suffer from unreliable evaluation against PGD attack as mentioned in recent
comprehensive studies (Croce& & Hein 2020b Chen
Gu 2020 Pang et al. 2022].
Low-quality data can cause robustness overestimation. show that low-quality data can cause overestimated robustness if evaluated against PGD attack by controlled experiments. We report the best robust test accuracy throughout the training rule out the effect of robust overfitting. We employ AutoAttack as a stronger adversary to evaluate the adversarial robustness. Although Auto/ttack not necessarily reflects the true robustness (Tramer et al., 2020), it is more reliable than PGD attack and is sufficient to reveal the significance overestimation.
For adversarial training methods, we
We denote the overestimation gap as the difference between the robust test accuracy evaluated against PGD-10 10 attack and Attok. Figure shows that the overestimation gap shrinks significantly when low-quality data is removed from the training For PGD-AT on all datasets, the robustness

---

Under review as a conference paper at ICLR 2022
CIFAR-10
CIFAR-100
Liny-ImageNet
55
25
Robust Test Accuracy (9)
PGD-AT
45-
20
15
Beroone mindority (903 SITUSI
35
5-
Remrve
30
5
0
0.0
0.2
0.4
0.6
0.6
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
50-
20
TRADES
45
20
15
35
10
5-
30
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
Fraction of training examples removed
Figure Best robust test accuracy obtained and evaluated against PGD attack and AA when removing training examples either randomly or in an ascending order of data quality, The overestimation gap can be captured by the visual gap between the dashed and solid curves of the same color.
obtained on the entire training set is often higher than that obtained on a smaller training set with low-quality data removed if evaluated against PGD attack, but will instead be close to even lower than the latter if evaluated against AutoAttack. In contrast, when the data is removed randomly from the training set, the overestimation gap largely maintains and also consistently larger than that produced by removing an equal amount of low-quality data. These evidences show that low-quality data in the training set is an important source of the robustness overestimation.
In Appendix B.3 evaluate the robustness against variety of black-box attacks including Square Attack (Andrishchenkoo al. 2020), RayS (Chen & Gu 2020 and transfer attack from a separately trained model, and show consistent correlation between data quality and robustness overestimation.
From a geometrical perspective, we suspect low-quality data causes robustness overestimation as they lie close to the decision boundary (see Appendix D.2p. The perturbation balls of cow-quality data from different classes may overlap, thus twisting the local loss landscape and complicating the maximization problem in gradient-basse based attacks (See more discussion in Appendix G.3
Table I: Robust test accuracy achieved by various training methods on the entire training set, evaluated by various attack methods. "." indicates the transfer attack from a surrogate model.
White-box Attack
Black-box Attack
Dataset
PGD- 10 (%)
PGD- RGD-10001 (%)
AA (%)
Square (%)
RayS (%)
PGD-1000" (%)
PGD
52.07
51.04
48.12
56.19
56.86
62.24
GAIRAT
59.51
59.33
32.34
41.98
44.17
58.06
Diff
+7.44
+8.29
-15.78
-14.21
-12.69
-4.18
TRADES
52.56
51.75
48.58
55.20
55.63
63.19
MART
53.81
53.39
46.57
52.76
53.45
60.73
Diff
+1.25
+1.64
-2.01
-2.44
-2.18
-2.46
Implications. Since low-quality data can cause robustness overestimation, it can be intentionally exploited to create spuriously high robustness against weak adversaries such as PGD attack. We find that this problem is ubiquitous among sophisticated adversarial training variants. We select two recently published methods GAIRAT (Zhang et al. 2020b) and MART and show that they suffer from robust overestimation due to the emphases on low-quality data in their training objectives (See Appendix II. As shown Table [ these methods trained with recommended settings (see Appendix
FD can achieve significant improvements compared to their corresponding baselines when evaluated against white-box PGD attack, even with large number of attack iterations. However, when evaluating against stronger adversary such as AutoAttack, black-box attacks, the performance of these methods are instead inferior. Similar observations have been made for these methods in recent works (Gowal et al. 2020 Hitaj et al. 2021). These results imply that emphasizing the
7

---

Under review as a conference paper at ICLR 2022
100
CIFAR-100
Tiny-lmageNet
100
80
70
Standard Test Accuracy (%)
80
60
50
PGD-AT
60
mesiomll Sepvoden vna oi
40
30
20
los quality fire undvrr ourod mrsinin
40
20
10
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.0
0.0
0.2
0.4
0.6
0.0
100
80
70
80-
60
50-
TRADES
60-
40
30-
40
20
10
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
0.8
Fraction training examples removed
Figure 7: Standard test accuracy obtained by standard training and adversarial training when removing different fractions of training examples either randomly or in ascending order of data quality. The goossgennrraliztiinn gap can be captured by the visual gap between the dashed and solid curves of the same color.
low-quality data in adversarial training may not be appropriate and it is important to validate the robustness against a wide variety of evaluation metrics as more methods are focusing on customizing the training for individual examples.
4.3 ROBUSTNESS-ACCURERE TRADE-OFF
The problem and related work. is widely observed that adversarially training the model comes at the cost of standard accuracy, which is typically referred as the robustmess-accwran trade- off (Papernot et al. problems, this trade- off might be ievitable either because no optimal classifier exists TTipras et al.
2016 Su al. 2018 Tsipras et al., 2019 Zhang et 2019 For certain learning
2019: Zhang al.) 2019) or the hypothesis classifier is not expressive enough (Nakkiran (219)). Meanwhile, this trade-off is data-sensitive. Specifically, Ding et al. (2019) showed that the level of trade-off depends on the data distribution, and Tsipras al. (2019) observed that the adversarial training is beneficial to standard accuracy when the training data insufficient to train the model.
Robustness- accuraacy trade-off is correlated with the data quality. We show that the robustness- accuracy trade-off is sensitive to the data quality within the same dataset. On high-quality data, this trade-off not significant; however, on low-quality data, it is prominent.
We present the correlation between data quality and cross-eeneralization gap by controlled experi- ments. Following previous work (Chen et al. 2020), we define the cross- gosss-enneralization gap as the standard test accuracy difference between standard training and adversarial training on the same dataset, which reflects the degree of robustness-accuracy trade-off induced by adversarial training on specific dataset. Note here we report the standard test accuracy at the last checkpoint for all settings in case the selected checkpoint varies due to robust overfitting. As in Figure 7 the cross-generalle gosssgennrallization gap maintains the same or even enlarges as more randomly selected examples are removed from the training set. In contrast, the cross- generalizztirinnn gap gradually shrinks as more low-quality examples are removed from the training set. For example, on all three datasets, adversarial training (either PGD-AT or TRADES) and standard training yield comparable standard accuracy, when using the 10% highest quality examples only (i.e., 90% lowest quality ones removed).
Implications. Our identified correlation between data quality and robustness- accuracy trade-off complements the existing observations of the impact of data distribution on robustness. Besides the training set size (Tsipras et al. 2019) and distribution shift (Ding et al. 2019), the quality of the training data can also influence this trade-off. In Figure | the cross gonnerasalaliiiii gap induced by PGD-AT is negative when using the 10% most high-quality examples on CIFAR- 10 and CIFAR- 100, which means that adversarial training is actually beneficial to the standard accuracy. Similar phenomena were previously only observed MNIST or extremely small training sets
8

---

Under review as a conference paper at ICLR 2022
(~ 100 examples) (Tipras al. 2019). Meanwhile, low-quality examples dominate the trade-off potentially because they are intrinsically ambiguous as shown in Figure Bal where robust features may be difficult to learn. This is aligned with the understanding of trade-off off from the feature learning perspective (Tsipras et al. 2019 Ilyas et al. 2019). More detailed explanations motivated by this perspective will be discussed in Appendix G.1
Based on the above understanding, to mitigate the robustness-accuracy trade-off one may resort properly deal with low-quality data in adversarial training. Multiple existing works are likely to implicitly perform this by customizing either the
Mitigate robustness-accuracy trade-off.
inner maximization (Balaji et al., (2019; Ding al. [2020] Cheng et al., 2020; Zhang et al.) (020a) or outer minimization (Huang et al. 2020 Wang et al. 2020; Zhang et al. (0206) for individual
examples (see more Appendix D). Although these methods can mitigate the trade-off by increasing either robustness or accuracy while maintaining the other, the effectiveness may be unclear under
oate-of-theeart robust evaluation methods (Croce & Hein 2020b) as shown in Section 4.2
CONCLUSION AND DISCUSSIONS
In this work we systematically study the difficulties in adversarial training from a data quality perspective. We show that the data quality is an important but previously neglected aspect for adversarial training. is strongly correlated with the current problems in adversarial training including robust overfitting, robustness overestimation and robustmess-accraae trade-off.
Towards understanding the difficulty of robust learning, our analyses of the data quality complements the existing analyses on the impact of data properties such distribution and sample size. Robust learning through adversarial training can exhibit diverse levels of difficulty on individual examples in the same benchmark dataset. From human perspective, low-quality data is often ambiguous, which implies that adversarial training might be susceptible to complex and indistinct image conditions, thus challenging the feasibility of adversarial training in a realistic scenario where ambiguous data may pre- vail. One may note that achieving robustness through adversarial training on ImageNet (Russakovsky al. 2015) already shown to be much harder (Rozsa et al. 2016 Kurakin et al. 2017).
In practice, our analyses of the data quality provides new methodology to analyze the effectiveness of adversarial training methods. The learning behavior of method on low-quality data reveals the potential reason why it may work or not work. For example, TRADES employs an alternative adversarial loss that penalizes the difference between the predictive probability from an adversarial example and its clean counterpart, instead of the true label, which will impose weaker supervision on low-quality examples since low-quality examples typically have low predictive probabilities (see Appendix D.I.. As a result, TRADES underfit low-quality data thus potentially gaining robustness compared to PGD-AT under the same perturbation setting, which resembles the effect of directly removing low-quality data.
Furthermore, our analyses encourage a new perspective to utilize the benchmark dataset for adversarial training. The fact that removing low-quality data maintains or even improves robustness implies that adversarial training cannot achieve the optimal robustness on the entire dataset, especially when the model capacity is relatively low. A promising direction to boost the robustness is to design methods able to learn low-quality data properly such that it can also contribute to the robustness. Nevertheless, We empirically find that no existing adversarial training method, without resort to larger model capacity, can achieve reliable robustness improvement through proper learning of low-quality data. Whether or not such a method exists remains an interesting problem.
Lastly, as it becomes increasingly popular to introduce additional data into adversarial training either labeled, unlabeled or generated (Hendrycks et al.| [2019a] Uesato et al., 2019 Carmon et al., 2019) Najafi et al. [0019] Sehwag et al. [202]] Gowal et a 2021 ) extra effort needs to be paid to picking high-quality data. The design of an effective and efficient data quality measurement is an important future work to excel in the performance.
Limitations. We emphasize that removing low-quality examples only serves as a demonstration of our intuition of data quality in adversarial training, and should not be advocated as a competing method. Although removing low-quality examples can potentially advance the best robustness, it will inevitably cost standard accuracy (see Appendix E., since those hard examples are of high quality to standard training.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBLITY STATEMENT
We conduct experiments on public benchmark. We will release implementations for all methods and scripts for all experiments on GitHub, under the Apache-2.0 license.
ETHIC S STATEMENT
In this paper, conduct series of controlled experiments to investigate the interconnections between data quality and problems in adversarial training. We experiment on three benchmark datasets that publicly available. We have shown that low-quality examples are helpful for standard training but may hurt the adversarial robustness. We emphasize that removing low low-quality examples only serves as a probing method demonstrate our intuition of data quality in adversarial training. and should not be advocated as a competing method. Therefore, we believe our work is ethically on the right side of spectrum.
REFERENCES
Maksym Andriushchenko, Croce, Nicolas Flammarion, and M. Hein. Square attack: query- efficient black-box adversarial attack via random search. ArXiv, abs/1912.00004, 2020.
D. Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron C. Courville, Yoshua Bengio, and S. Lacoste-Julien. A closer look at memorization in deep networks. ArXiv, abs/1706.05394, 2017.
Anish Athalye, Nicholas Carlini, and D. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. ArXiv, 0oss/802.00420 2018.
D. Ayala. The theory and practice of item response theory. 2008.
Y. Balaji, T. Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. ArXiv, abs/1910.08051. 2019.
Yoshua Bengio, J. Louradour, Ronan Collobert, and Weston. Curriculum learning. In ICML '09, 2009.
Qi-Zhi Cai, Min Du, Chang Liu, and Dawn Xiaodong Song. Curriculum adversarial training. ArXiv, abs/1 805.04807, 2018.
Nicholas Carlini and D. Wagner. Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57, 2017.
Y. Carmon, Aditi Raghunathan, L. Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. ArXiv, abs/1905.13777 2019.
Haw-Shiuan Chang, E. Learned-Miller, and A. McCallum. Active bias: Training more accurate neural networks by emphasizing high variance samples. In NIPS, 2017.
J. Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. Proceed- ings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
Lin Chen, Yifei Min, Mingrui Zhang, and Amin Karbasi. More data can expand the generalization gap between adversarially robust and standard models. ICML, 2020.
Chen, Yash Sharma, Huan Zhang. Jinfeng Yi, and Hsieh. Ead: Elastic-net attacks to deep neural networks via adversarial examples. ArXiv, 0Ss/1709.041144 2018.
Tianlong Chen, Zhenyu (Allen) Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In ICLR, 2021.
Minhao Cheng, Qi Lei, Pin-Yo Chen, I. Dhillon, and C. Hsich. Cat: Customized adversarial training for improved robustness. ArXiv, Abs/2000200688, 2020.
10

---

Under review as a conference paper at ICLR 2022
F. Croce and M. Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In ICML, 2020a.
F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020b.
Jiequan Cui, Shu Liu, L. Wang, and J. Jia. Learnable boundary guided adversarial training. ArXiv, abs/201 1144, 2020.
G. W. Ding, Kry Yik Chau Lui, Xiaomeng Jin, Luyu Wang, and Ruitong Huang. On the sensitivity of adversarial robustness to input data distributions. ArXiv,
2019.
G. W. Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin adversarial (mma) training: Direct input space margin maximization through adversarial training. ArXiv, abs/1812.02637, 2020.
Elvis Dohmatob. Limitations of adversarial robustness: strong no free lunch theorem. ArXiv, abs/1810.04065, 2018.
Embretson and S. Reise. Item response theory for psychologists. 2000.
L. Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. ArXiv, abs/1 807. 10272, 2018.
Y. Freund and R. Schapire. A decision-theoretie generalization of on-line learning and an application to boosting. In EuroCOLT, 1995.
Ian J. Jonathonn Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6557, 2015.
Jonathon
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy A. Mann, and Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. ArXiv, abs/2010.03593, 2020.
Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy Mann. Improving robustness using generated data. ArXiv, 0/5/2110.094660 2021.
Guy Hacohen, Leshem Choshen, and D. Weinshall. Let's agree to agree: Neural networks share classification order on real datasets. arXiv: Learning, 2019.
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ArXiv, abs/1 103/1603.5 2016.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-tran pee-training can improve model robustness and uncertainty. In JCML, 2019a.
Dan Hendrycks, K. Zhao, Steven Basart, J. Steinhardt, and D. Song. Natural adversarial examples. ArXiv, abs/1907.07774, 2019b.
Geoffrey F. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, abs/1 503.02531, 2015.
Dorjan Hitaj. Giulio Pagnotta, Iacopo Masi, and L. Mancini. Evaluating the robustness of geometry-
aware instance- -eweeghhee adversarial training. ArXiv, abs/2103.01914, 2021.
Lang Huang. C. Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk
minimization. ArXiv, abs/2002. 10319, 2020.
Ruitong Huang, B. Xu, Dale Schuurmans, and Csaba Szepesvari. Learning with strong adversary. ArXiv, 00//1511.03034 2015.
Andrew Ilyss, Shibani Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. In NeurlPS, 2019.
Pavel Izmailov, Dmitrii Podoprikhin, T. Garipov, Dmitry P. Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. ArXiv, abs/l 003.05407, 2018.
11

---

Under review as a conference paper at ICLR 2022
A. Javanmard, M. Soltanolkotabi, and H. Hassani. Precise tradeoff's in adversarial training for linear regression. In COLT, 2020.
Harini Kannan, A. Kurakin, and lan Goodfellow. Adversarial logit pairing. ArXiv, abs/1 10//0063363 2018.
G. Katz, C. Barrett, D. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. ArXiv, 702.01 135, 2017.
A. Krizhevsky. Learning multiple layers of features from tiny images. 2009.
M. Kumar, Ben Packer, and D. Koller. Self-paced learning for latent variable models. In NIPS, 2010.
A. Kurakin, Ian J. Goodfellow, and S. Bengio. Adversarial machine learning at scale. ArXiv, 1bs/166 2017.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. 2015.
Y. LeCun, L. Bottou, Yoshua Bengio, and P. Haffner. Gradient-basee learning applied to document recognition. 1998.
Mingchen Li, M. Soltanolkotabi, and S. Oymak. Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. ArXiv, abs/1903. 1680, 2020.
A. Madry, Aleksandar Makelov, L. Schmidt, D. Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ArXiv,
2018.
Takeru Miyato, S. Maeda, Masanori Koyama, and S. Ishii. Virtual adversarial training: A regulariza- tion method for supervised and semi-supervised learning. IEEE Transuctions on Pattern Analysis and Machine Intelligence, 11:1979-1993, 2019.
Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. Confidence-aware learning for deep neural networks. In /CML, 2020.
Seyed-Mohsen Moosavi-Dezfoolli Alhussein Fawzi, and P. Frossard. Deepfool: A simple and accurate method to fool deep neural networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2574-2582, 2016.
Marius Mosbach, Maksym Andriushchenko, T. A. Trost, M. Hein, and D. Klakow. Logit pairing methods can fool gradient-based attacks. ArXiv, abs/1810 12042, 2018.
A. Najafi, S. Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. ArXiv, abs/1905. 11199005.322. 13021, 2019.
Preetum Nakkiran. Adversarial robustness may be at odds with simplicity. ArXiv, 2019.
Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial robustness toolbox v1.2.0. CoRR, 1807.01069. 2018. URL https
Curtis G. Northcutt, Anish Athalye, and Jonas Mueller. Pervasive label errors in test sets destabilize machine learning benchmarks. ArXiv, abs/2103. 14749, 2021.
Tianyu Pang. Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. In International Conference on Learning Representations, 2021. URL htttps:///perrevilerr
net/forum: 1d=Xb8xvit3Bee
Nicolas Papernot, P. McDaniel, Arunesh Sinha, and Michael Wellman. Towards the science of security and privacy in machine learning, ArXiv, abs/161 1.038103814 14..0.81.12.. 2016.
Nicolas Papernot, P. McDaniel, Ian J. Goodfellow, S. Jha, Z. Y. Celik, and A. Swami. Practical black-box attacks against machine learning. Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, 2017.
12

---

Under review as a conference paper at ICLR 2022
Camilo Pestana, N. Akhtar, W. Liu, David Glance, and A. Mian. Adversarial perturbations prevail in the y-channel of the yeber color space. ArXiv, abs/2003.00883, 2020a.
Camilo Pestana, Wei Liu, David Glance, and A. Mian. Defense- friendl images in adversarial attacks: Dataset and metrics for perturbation difficulty. ArXiv, (bbs22111.02673 2020b.
R. Prudêncio, J. Hemmáneez-Orallo, and A. Martinez-Usó. Analysis of instance hardness in machine learning using item response theory. 2015.
Chongli Qin, J. Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Kohli, Adversarial robustness through local linearization. In NeurIPS, 2019.
Aditi Raghunathan, Sang Michael Xie, F. Yang, John C. Duchi, and P. Liang. Understanding and mitigating the tradeoff between robustness and accuracy. In ICML. 2020.
Leslie Rice, Eric Wong, and J. Kolter. Overlitting adversarially robust deep learning, ArXiv, abs/2002. 11569, 2020.
Kevin Roth, Yannic Kilcher, and T. Hofmann. Adversarial training is form of data-dependent operator norm regularization. arXiv: Learning, 2020.
Andras Rozsa, M. Günther, and T. Boult. Are accuracy and robustmess correlated. 2016 15th IEEE International Conference Machine Learning and Applications (ICMLA), pp. 227-232, 2016.
Olga Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Zhiheng Huang, A. Karpathy, A. Khosla, Michael Bernstein, A. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211-252, 2015.
2015.
Amartya Sanyal, P. Dokania, Varun Kanade, and P. Torr. How benign is benign overfitting? ArXiv, abs/2007.04028, 2020.
L. Schmidt, Shibani Santurkar, D. Tsipras, Kunal Talwar, and A. Madry. Adversarially robust generalization requires more data. In NeurlPS, 2018.
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mitral. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? 2021.
A. Shafahi, W. Huang, Christoph Studer, S. Feizi, and T. Goldstein. Are adversarial examples inevitable? ArXiv, 0bs/1809.02104, 2019.
Abhinay Shrivastava, Abhinav Gupta, and Ross B. Girshick. Training region-based object detectors with online hard example mining. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 761-769, 2016.
K. Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CORR, abs/1409, 409. 1556, 2015.
Vasu Singla, Sahil Singla, David Jacobs, and Soheil Feizi. Low curvature activations reduce overfitting in adversarial training. ArXiv, abs/2102.07861, 2021.
M. Smith and T. Martinez. Improving classification accuracy by identifying and removing instances that should be misclassified. The 2011 International Joint Conference on Neural Networks, pp. 2690-2697, 2011.
M. Smith, T. Martinez, and C. Giraud-Carrier. An instance level analysis of data complexity. Machine Learning, 95:225-256, 2013.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, and R. Venkatesh Babu. Guided adversarial attack for evaluating and enhancing adversarial defenses. ArXiv, 1.s/20011.4969 2020.
David Stutz. M. Hein, and B. Schiele. Disentangling adversarial robustness and generalization. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6969-6988 2019.
13

---

Under review as a conference paper at ICLR 2022
David Stutz, Matthias Hein, and B. Schiele. Relating adversarially robust generalization to flat minima. ArXiv, abs/2104.04448, 2021.
D. Su, Huan Zhang, H. Chen, Jinfeng Yi, P. Chen, and Yupeng Gao. Is robustness the cost of accuracy?- - a comprehensive study on the robustness of deep image classification models. In ECCV, 2018.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and G. Gordon. An empirical study of example forgetting during deep neural network learning. ArXiv, abs/1812.05159, 2019.
Florian Tramèr, A. Kurakin, Nicolas Papernot, D. Boneh, and P. McDaniel. Ensemble adversarial training: Attacks and defenses. ArXiv, abs/1705.07200 2018.
Florian Tramèr, N. Carlini, W. Brendel, and A. Madry. On adaptive attacks to adversarial example defenses. ArXiv, abs/2002.08347, 2020.
D. Tsipras, Shibani Santurkar, L. Engstrom, A. Turner, and A. Madry. There is no free lunch in adversarial robustness (but there are unexpected benefits). ArXiv, 10111222.122,, 2018.
D. Tsipras, Shibani Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. arXiv: Machine Learning, 2019.
Jonathan Uesato, Brendan O' O'Dongguee, A. Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. ArXiv, abs/1802.05666, 2018.
Jonathan Uesato, Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and P. Kohli. Are labels required for improving adversarial robustness? ArXiv, Abs/1905.11722. 2019.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Yuxin Wen, Shuai Li, and Kui Jia. Towards understanding the regularization of adversarial robustness on neural networks. ArXiv, abs/201 1.07478, 2020.
Tsui-Wei Wei Weng, Huan Zhang, P. Chen, Jinfeng Yi, D. Su, Yupeng Gao, C. Hsieh, and L. Daniel. Evalu- ating the robustness neural networks: An extreme value theory approach. ArXiv, abs/1801.10577 2018.
Dongxian Wu, Shutao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generaliza- tion. arXiv: Learning, 2020.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, A. Yuille, and Quoc Le. Adversarial examples improve image recognition. 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 816-825, 2020.
Y. Yang, Cyrus Rashtchian, Yizhen Wang, and K. Chaudhuri. Robustness for non-parametric classification: generic attack and defense. In AISTATS, 2020a.
Y. Yang, Cyrus Rashtchian, Hongyang Zhang, R. Salakhutdinov, and K. Chaudhuri. A closer look at accuracy vs. robustness. arXiv: Learning, 2020b.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. ArXiv, abs/1605 .0.6005..1100 146, 2016.
Hongyang Zhang, Yaodong Yu, J. Jiao, E. Xing, L. Ghaoui, and Michael I. Jordan. Theoretically principled trade-off berween robustness and accuracy. ArXiv, abs/ abs/1991.eg 11990108573. 2019.
Jingfeng Zhang, Xilie Xu, B. Han, Gang Niu, Li zhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. ArXiv, abs/2002.1 1242, 2020a.
14

---

Under review as a conference paper at ICLR 2022
Jingfeng Zhang, Jianing Zhu, Gang Niu, B. Han, M. Sugiyama, and M. Kankanhalli. Geometry-aware instance-reweigee adversarial training. ArXiv, abs/2010.01736, 2020b.
Tianyi Zhou, S. Wang, and J. Bilmes, Curriculum learning by dynamic instance hardness. In NeurIPS, 2020.
15

---

Under review as a conference paper at ICLR 2022
A MORE RELATED WORK
A.1 ROBUSTNESS OVERESTIMATION
We discuss two recently published methods where the emphases on problematic data can be easily identified from their training objectives (See Table 2 compared to the bascline methods.
GAIRAT (hhag et al. 2020b) is based upon PGD-AT with an additional sample-wiss weight in the loss function. Examples that are farther from the decision boundary will be assigned with larger weights, which amounts to emphasis the problematic data since the problematic score can be alternatively characterized by the distance to the decision boundary (see Appendix D.2).
MART (Wang et al. 2020) can be largely viewed as a variant TRADES with an additional term promoting the loss of those examples with low output probabilities, which also amounts to an emphasis on the problematic data based on the probability characterization of the problematic score (sce Appendix D.I.). D.I.
Table 2: Training objectives in the outer minimization in various adversarial training methods, where ((:.)) indicates the cross-entropy loss, f()) indicates the probabilistic prediction of model and f indicates the probability corresponding to class y, 5 indicates the adversarial perturbation of x generated by the inner maximization.
Method
Training objective
PGD-AT
miny En.y [e(fx + y)]
GAIRAT
miny Ex.y (I
TRADES
miny Ex.y
( (x))
MART
minj En.y
A.2 ROBUSTNESS CBBUSTNESS-ACCURACY TRADE-OFF
We note that there exists an abundant body of works that focuses on the cobustness-accuracy trade-off in robust learning. In additional to the related works discussed in the main paper, here we review some works that attack this problem from other perspectives. This is by no means an exhaustive review.
has been argued that the robustness cabustness-accuracy trade-off inherent to the data distribution and is thus inevitable for any classifier. Tsipras et al. (2018) 2019) and Zhang al. (011) theoretically show that no optimal classifier can achieve both robustness and accuracy on toy problems. Dohmmtob (2018) formalizes this into "No Free Lunch" problem, and further proves the inevitability of trade-off under mild assumptions of the data distribution. Nakkiran (2019) shows that the trade-off is inevitable because the hypothesis class is not expressive enough. Javanmard al. (2020) shows that the adversarial training may improve generalization in an wver-parameteriaa regime, but hurt it in under-parannn madee-parameterized regime.
On the contrary, some works argue that the robustness- accuracy trade-off is not necessarily inevitable in a realistic setting. RRaghunathan et al. (2020) shows that the trade-off stems from the over- parameterization of the hypothesis class. Robust self-training (Carmon et al. 2019: Najafi et al.) 2019 [Uesato al., 2019). overcoming the sample complexity leveraging additional unlabeled data, thus can effectively mitigate the trade-off. Yang et al. (2020b) shows that the trade-off in practice is result of either the model failing to impose local Lipschiztness, or not generalizing sufficiently, WWen et al. (2020) and Roth al. (2020) show that adversarial training is essentially form of operator norm regularization, thus hurting the generalizability if not properly configured.
There more works implying that the robustness and accuracy may not be contradict by showing that adversarial examples can benefit generalization either through different perturbation generation strategies (Stutz et al. 2019) different adversarial training strategies (Xie al. 2020).
16

---

Under review as a conference paper at ICLR 2022
A.3 DATA PROFILING IN STANDARD LEARNING
In classical machine learning, data quality is important because algorithms may be sensitive to noise and outliers. By measuring the degree of class overlapping and skewness dataset, |mmihh et al.) (2013) proposes generic definition of instance hardness representing how likely an example will be misclassified. |rrudencio et al. (2015) motivates from item response theory (IRT) (Embretson & Reise 2000; Ayala 2008) to characterize instance hardness. [Smith & Martinez (2011) shows that properly removing hard examples in the dataset improves performance for a variety learning algorithms
including but not limited to Decision Tree and Support Vector Machine.
In deep learning regime, models with large capacity are typically more robust to outliers. Nevertheless, data examples can still exhibit diverse levels of difficulties, Arpit et al. (2017) finds that data examples are not learned equally when injecting noisy data into training. Tooeva et al. (2019) shows that certain examples forgotten frequently during training, which means that they can be first classified correctly then incorrectly. Model performance can be largely maintained when removing those least forgettable examples from training. Zhoou et al. (2020) proposes dynamically estimate instance hardness during training and encourage the model to focus on those hard examples from a curriculum learning (Bengio al. 2009) perspective, which can improve both the performance and efficiency for a wide range of datasets and neural architectures. More generally, under self- self-paced learning (Kumar 2010) framework, diverse methods have been proposed to mine hard examples on the fly (Chang 2017).
al.
et al.
A.4
DATA PROFILING IN ROBUST LEARNING
In robust learning regime, the model is required to learn features that are robust to perturbations. Such task is generally more difficult, where the data examples are thus more likely to differentiate in terms of their behaviours in learning or contribution to the model performance. Plenty of works have analyzed the diverse behaviours of data during adversarial training, and proposed variety of methods that treating the data examples differently. A detailed review has been made in Section 2 and Section 4.3 4.1 that focus on existing problems in adversarial training specifically.
Here we mainly review works that investigate adversarial examples from data perspective. Hendrycks et al. (2019b) collects a set of unperturbed images, known as the "natural adversarial examples" that significantly degrades the performance of trrte-rrt image classifiers. Pestana (ct al. (2020a) observes that the adversarial perturbations concentrate in the Y-channel of the YCbCr space, which is believed to contain more shape and texture related information. [Pestana et al, (2020b) reports the existence of a set of images that are particularly robust to adversarial perturbations. Including such data in validation significantly limits the reliability of the robustness evaluation.
B MORE EXPERIMENT RESULTS
B.
ADDITIONAL NEURAL ARCHITECTURES
Robust Overfitting
Robustness Overestimation
Robustness-accuracy Trade-off
Robust Test Acc. (%))
55
Standard lest Acc. (%)
90
Robust Test Acc.
50
45
45
40
40
70
35
Samova randemy ibaa shacoponki Samove cualty fire (larr checkppiin
NatHove sandarty (P30 stacki
Berroce re serdrmly (Stanzard trsiing! randovily
35
60
30
Ramone
50
Remove fime (Adweradds tra ning:
0.0
0.2
0.4
0.5
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
Fraction of training examples removed
Figure 8: Correlation between data quality and three problems in adversarial training shown by conducting controlled experiments on WRN-28-10.
In this section we show the interconnection between data quality and those three problems in adversarial training including robust overfitting, robustness overestimation and robustness robustness-accuracyy accuracy trade-off also holds true for other neural architectures such as WRN. Here conduct experiments
17

---

Under review as a conference paper at ICLR 2022
WRN-28-10 10 and employ PGD training CIFAR-10 example. As shown in Figure & as low- quality examples are removed from the training set, the robust overfitting would vanish, robustness overestimation and robustness-accute accuracy trade-off would become less significant, in contrast to the observations when removing randomly selected examples.
B.2 DATA QUALITY AND ROBUSTNESS
Experiments on different models and perturbation sizes. For larger perturbation radii and smaller models, removing certain group of low-quality examples yields more robustness improve- ment as shown in Figure 9a and Figure 9b respectively. Note that in Figure (6 the robustness improvement yielded by removing 20% examples with the lowest quality diminishes as the model capacity increases. For considerably large models such as WRN-34- 10 it will eventually hurts. These analyses match our intuition that adversarial examples produced by large perturbation are hard to learn; and smaller models are hard to learn adversarial robustness.
Robust Test Accuracy (%)
Robust Test Accuracy (%)
53
48-
52
46-
Original training set
Remove 20% examples with lowest quality
Original training
50
Remove 20% examples with lowest quality
(/255)
16
22
28
34
Depth
(a) Removing a set of qualiuy examples improves the robustness consistently for different training per- turbation radii The robustness improvement is more prominent for larger perturbation radii (2 10/255). Here we remove 20% examples with the lowest qual- ity. We conduct adversarial training with PGD-AT. The model is fixed pre-activation ResNet-18. Other details of the experimental settings mentioned in Appendix
(b) The rohustness improvement by removing of low-quality examples is more prominent for smaller models, and gradually vanishes as the model capacity increases. Here we control the model capacity by mod- ulating the depth of Wide ResNet, where the width is fixed 10. We experiment on PGD-AT with the train- ing perturbation radius yiclding the best robustmess (12/255).
B.3 DATA QUALITY AND ROBUST OVERESTIMATION
Experiments on more robustness evaluation methods. To exclude the possibility that AutoAt- tack itself is sensitive to the data property due to its sophisticated design, we take CIFAR-10 as an example and select 20% as a representative fraction low-quality examples being removed where the overestimation gap shrinks significantly. We further evaluate the robustness obtained by removing 20% low-quality examples against a variety of black-box attacks including Square Attack, RayS and transfer attack from separately trained surrogate model (See Appendix for details). We conduct repeated experiments (5 times) to reduce the statistical bias. We find that compared to the training set with 20% low-quality examples removed, the robustness obtained on the entire training set can be significantly higher (~ 2%) when evaluated against PGD-IC 10 and PGD-1000 i.c. PGD attack with up 1000 iterations. However, the improvement actually not clear if evaluated against Square Attack and RayS in that the average difference is less than the standard deviation, and may even be negative for AutoAttack and transfer attack. contrast, compared to the training set with 20% randomly selected examples removed, the robustness obtained on the entire training set is consistently higher for all evaluation attacks. This demonstrates that the low-quality data can cause difficulty for PGD attack to reliably evaluate the robustness, which cannot be avoided even with substantially large number of iterations. This also implies PGD attack is likely to be suboptimal with fixed parameters (Mosbach et al. 2018 Croce & Hein 2020b).
18

---

Under review as a conference paper at ICLR 2022
C MORE ABOUT DATA QUALITY
C.I CALCULATION OF THE DATA QUALITY RANK
To output a relatively accurate estimation of the data quality rank, we synthesize the results of multiple experiments. Specifically, we adversarially train pre-activation ResNet-18 using PGD-10 on the entire CIFAR-I0 training set for 160 epochs with learning rate initialized as 0.1 and decayed at epoch 80 and 120 with a factor of 10. We repeat the training 10 times and average the data quality ranks calculated on each example, which yields a relatively stable estimation
C.2 DISTRIBUTION OF LEARNING STABILITY
In Section 3.2 we estimate the data quality relatively by calculating the ranking of the examples based on learning stability. Directly using learning instability to measure the data quality might be troublesome as its magnitude varies greatly across different training settings (e.g. number of training epochs).
Figure 10 shows the distribution of the learning stability with different training epochs. One may observe that, the overall magnitude of the learning stability varies greatly as the training setting changes, which is reasonable since the model complexity hinges on the training settings and all training examples will eventually be overfitted given sufficient training epochs.
Epoch 100
Epoch 500
# Epoch 1000
Fraction of exampies
Fraction of examples
Fraction of examples
0.15
0.3
0.4
0.3
0.2
0.2
0.05
0.1
0.2
0.6
1.0
0.0
0.4
0.6
1.0
0.4
1.0
Learning stability
Learning stability
Learning stability
Figure 10: The distribution of learning stability given different training epochs.
D MORE METHODS TO MEASURE THE DATA QUALITY
In this section, we show that it is possible to estimate the data quality motivated from multiple measurements including prediction probability, minimum perturbation and learning order. Each measurement is itself consistent across different training settings, and is also correlated with the data quality rank estimated in Section 3.1 We also briefly review existing adversarial training methods leveraging these measurements to customize the inner maximization or outer minimization for individual examples in their designs, which implics that their improved performance is potentially due to treating examples of different data quality differently.
D.1 PREDICTION PROBABILITY
Based on the standard adversarial training Madry et al., 2018), multiple variants pivot on the utilization of soft output in the adversarial loss function. Here we specifically refer the soft output as the either the output before the softmax function, namely logit, or that after the softmax function, namely prediction probability. Adversarial logit pairing (ALP) (Kannan al., 2018) is method explicitly penalizing the difference between the logits from an clean example and its adversarial counterpart on top of the standard cross-entropy loss for adversarial training. BGAT (Cui et al. 2020 instead collects the logits of clean examples from an auxiliary clean model. GAT (Sriramanan et al. 2020) adopts similar loss function penalizing probability difference instead of logit difference.
"Due to computational constraints, estimation process only provides about 1600 unique values for data quality ranks, which are not enough to differentiate all the 5 x 10 training examples in CIFAR-10. 10. However, in above analyses, we at most partition the training set into 10 subsets with different quality levels, which only results in about 0.6% indistinguishable examples between any two adjacent subsets. One can seek more accurate estimation of data quality rank by incorporating the results from more experiments.
19

---

Under review as a conference paper at ICLR 2022
VAT (Miyato et al. 2019) and TRADES (Zhang et al. 2019) propose loss function matching the probability from an adversarial example with its clean counterpart, instead of the true label. Self-adaptive training (Huang et al. 2020) and MART (Wang et al. 2020) use the probabilities from clean examples to weight the examples in the loss function, although the former focuses more on examples with high probabilities while the latter focuses more on examples with low probabilittes.
Prediction probability is also related to whether an example is correctly classified or not. Low probability of the true label indicates an example is likely be misclassified, which might be troublesome in adversarial training because adversarial examples of misclassfied examples are "undefined" (Wang et al., 2020). Several methods are thus motivated to treat correctly classified and misclassified examples differently. MMA (Ding et al. (2020) employs adverarial training only on correctly classified examples, leaving misclassífied examples to standard training. MART (Wang et al., 2020) introduces a term associated with the probability of the true label in the loss function to encourage the learning on misclassified examples. A summary of the loss functions employed in these methods can be found in Table 1 by Wang et al. (2020).
Recently, adversarial training with additional data becomes increasingly popular. Prediction probabil- ity is used to identify high-quality data that is more relevant to the original distribution (Gowal et al. 2020
19
Data Quality flank
Data
Date
05
62 04
Sa 644
12-1
12"
140
30
Prediction Probabillyy
Minimum Perturbation
Learned Epoch
(a) Prediction probability
(b) Minimum perturbation
(c) Learning order
Figure 11: Correlation between data quality rank and other measurements
Here we show that the prediction probability of an example is correlated with its data quality rank. These sophisticated methods are thus likely to achieve robustness gain by treating examples with different quality levels differently. We specifically refer the prediction probability the probability corresponding to the true label from a clean input, and use the best model terms of robustness throughout training to estimate it. For every example, we average the probabilities obtained by the same 10 experiments introduced in Section C.I] As shown in Figure IIa prediction probability is inversely correlated with data quality rank. A high-quality example inclined be correctly classified by the model with high probability. We do not estimate the data quality based on whether an example will be misclassified or not since the misclassification rate varies greatly across different training settings.
D.2 MINIMUM PERTURBATION
Standard adversarial training often sets a perturbation radius universal all training examples. However, it has been widely noticed that individual examples may have different levels of robustness against adversarial attacks. It might be helpful to customize the perturbation for each example during adversarial training. MMA (Ding et al. 2020) proposes a method to estimate proper individual perturbation for each example based on its distance to the decision boundary. The perturbation is determined by line search along the perturbation direction initialized by norm-constrained PGD attack. In a similar vein, IAAT (Balaji et al.) 2019) performs a dynamic update of individual perturbation throughout the training, CAT (Cheng et al.) 2020) further incorporates individual label smoothing based on the estimated perturbation level. Instead of customizing individual perturbation radius for each example, FAT (Zhang al., 2020a) customizes the number of attack iterations for each example, such that the perturbation is just enough to fool the model. GAIRAT (Zhang et al. 2020b) further adopts weighted loss function based such individual attack iterations to focus
20

---

Under review as a conference paper at ICLR 2022
more those examples far from the decision boundary. Nevertheless, none of these works detailedly analyzes the impact of such individual perturbation on the adversarial training.
Here, we show that profiling the data based individual perturbation radius leads to similar result as that motivating from learning stability introduced in Section[3.] We denote the minimum perturbation of example as the smallest perturbation radius required to change model's prediction on it. Ideally, the minimum perturbation of an example amounts to its minimum distance to the decision boundary. We use an untargeted attack based I-FGSM (Iterative Fast Gradient Sign Method) (Goodfellow it al. 2015; Kurakin et al. 2017) with step size 1/255 based on the implementation in Adversarial Robustness Toolbox (ART) (Nioolae et al. 2018). We empirically found the step size 1/255 is often small enough to ensure converged minimum perturbation. One can also employ other attacks including but not limited to DeepFool (Moosavi-Dezfooli et al. 2016), CW (Carlini Wagner 2017). EAD (Chen et al., (018) and FAB (Croce & Hein 2020a) in estimation. But note that all these methods cannot obtain the true minimum perturbation, but only the upper bound of it (Weng) al. 2018). 8). Estimation of the minimum perturbation through optimization is known as NP-hard
problem Katz et al. 2017).
In Figure we use the best model in terms of robustness obtained in training to estimate the minimum perturbation, and average the results obtained by the same 10 experiments mentioned in Section One can find that the minimum perturbation is inversely correlated with the data quality rank, which means that low-quality example is more likely to reside near the decision boundary. This suggests the sophisticated methods mentioned above likely treat examples with different qualities differently. It also suggests that examples with different amounts of minimum perturbation will influence the adversarial training differently in terms of the contributions robustness and aforementioned problems.
D.3 LEARNING ORDER
Learning order refers to the phenomenon that Deep Neural Networks (DNNs) learn the examples in similar order, which is shared by different random initializations and neural architectures. Such phenomenon is observed widely in standard training and training with noisy inputs and labels (Arpit
al. 2017 Li al. 2020). It demonstrated that the learning order originates from the coupling between DNNs and benchmark datasets (Hacohen et al. 2019), since DNNs learn synthetic datasets without specific order and classifiers other than DNNs such as AdaBoost (Freund & Schapire 1995)
leam benchmark datasets without a specific order.
We show that learning order still exists in adversarial training. We denote the 1s learned epoch as the first epoch when a training example is classified correctly under adversarial attack. We show there is correlation between the 1st learned epochs an example across different training settings (see Section D.4.. Furthermore, we show that learning order is correlated with data quality rank. In Figure Ic we average the 1st learned epochs an example obtained by the same experiments as mentioned in Section C.1 One can find the 1s learned epoch is positively correlated with its quality rank, which means qualiuylity examples are likely to be learned late during training.
Note that if we pick the best epoct as boundary and partition the training examples based on their 1st learned epochs, the resulting two subsets correspond to exactly the correctly classified and misclassified examples. This implies that those adversarial training methods treating examples differently based on whether they are correctly classified or not, as mentioned in Section [D]]] are likely to be special case of treating examples differently based on their quality, from yet another perspective.
D.4 CONSISTENCY OF THE MOTIVATION
We show that each motivation mentioned above is itself consistent across different training settings. Therefore is possible to estimate the data quality rank similarly from each motivation. In Figure 12
13 14
we use the same experiments mentioned in Figure 2 to show that the prediction probability, minimum perturbation and learning order are all consistent across random initializations, different training methods, and neural architectures. Nevertheless, one may find that these estimations are
'The epoch when the best model in terms of robustness is obtained
21

---

Under review as a conference paper at ICLR 2022
relatively less consistent compared the data quality rank estimated from learning stability, which is the major reason that we estimate the data quality rank from learning stability in the main text.
Dxpenmeen-) Exseiment
Saper meettil expermettd
TRADES
(a) Random initializations
(b) Different methods
(c) Different models
Figure 12:
Scatter plots of the quality ranks training examples based on prediction probabilities obtained by different training settings. The prediction probability of an example is consistent across random initializations, different methods and models.
Deseriment-1 3 Espetmenn) Sppelment-1 Exparment
PGD
TTADES
WRN
MARi TRADeS
(a) Random initializations
(b) Different methods
(c) Different models
Figure 13: Scatter plots of the quality ranks of training examples based on minimum perturbations obtained by different training settings. The minimum perturbation of example is consistent random initializations, different methods and models.
Exseriment-2 3 Eapetment))
Eape Went--- Experiente2
PGD MAAT
-RRADES
FCD
PGD KRN
Saper Sipermeni1 Eppmmet3
42
(a) Random initializations
(b) Different methods
(c) Different models
Figure 14: Scatter plots of the quality ranks of training examples based on 1 learned epochs obtained by different training settings. The 1x learned epoch refers to the first epoch when example is classified correctly under adversarial attack, which is consistent across random initializations, different methods and models.
22

---

Under review as a conference paper at ICLR 2022
E LIMITATIONS OF REMOVING LOW-QUALITY DATA
We note that removing low-quality training data cannot be advocated as completing method. Although removing low-quality data can benefit the robustness certain scenarios, will inevitably impair the standard accuracy, since those hard examples are of high-quality to standard training. As shown in Table 3 removing the 20% examples with the lowest quality can improve the best robustness consistently for different training methods and perturbation radii, but will also diminish the standard accuracy for most settings (except PGD training with perturbation radius 12/255)
Table 3: Performance of adversarially training pre-activation ResNet-18 CIFAR-10 using PGD and TRADES with different training perturbation radii.
# Training samples removed (%)
Method
(/255)
Standard Acc (%)
Robust Ace (%)
PGD
8
83.92
48,12
PGD
8
20%
82.26
48.24
PGD
12
76.08
48.95
PGD
12
20%
77.85
49.22
TRADES TRADES
8
81.73
48.18
8
20%
78.16
49.35
TRADES TRADES
12
76.56
45.40
12
20%
73.53
46.85
F EXPERIMENT DETAILS
We adopt the following setting for all experiments unless otherwise noted.
Robustness evaluation. We consider the robustness against foo born-bounded adversarial attack with perturbation radius 8/255. Throughout the paper, the following attacks are employed.
AutoAttack: This is currently the strongest adversarial attack to the best of our knowledge. No parameter setting is required.
PGD-10: PGD with 10 attack iterations and step size fixed to 2/255.
PGD-1000: PGD with 1000 attack iterations. The step size fixed to 2/255, which is the best value suggested by Croce & Hein (202bb).
Square Attack: This is strong score-based black-box attack and is also built into the standard AutoAttack. We list it separately here to eliminate the possibility that the adaptive design in AutoAttack can be sensitive the data. We adopt 1 restart with maximally 5000 queries.
RayS: This is strong decision- based black-box attack that only requires the final prediction the model. We set the maximum number of queries to be 10000.
Transfer attack: We use PGD-AT to adversarially train a WRN-28-10 on the CIFAR-10 training set. The model at the best checkpoint terms the robustness will then serve as surrogate model to generate the adversarial attack.
Adversary setting. We conduct adversarial training with foo norm-bounded perturbations. We employ PGD-AT and TRADES as base methods, and two sophisticated variants GAIRAT and MART. We fix the perturbation radius 8/255 unless otherwise noted. The number of attack iterations is fixed as 10, and the perturbation step size is fixed as 2/255. We adopt early stopping (Rice et al. 2020) default strategy and report the best robustness obtained throughout the training. Additional settings specific to each method are listed as follows.
TRADES: The regularization hyperparameter is fixed as 1/A = 6.0 as recommended in the official implementation.
23

---

Under review as a conference paper at ICLR 2022
GAIRAT: A sample-wis wise weighting based the distance to the decision boundary is employed in GAIRAT. The distance to the decision boundary is approximated by the minimum number of iterations k(z, to generate adversarial perturbation that successfully fools the model in the inner maximization. The sample-wise weighting is given by
w r, y) = 1+ tanh (A 5 (1 2 k(x, ()l,)))KK))
2
where K is the maximum number of iterations. We fix À 0 throughout the training. w( y) will be normalized ensure Σ; w(z, y)/n = I, where n the total number of training examples.
MART: We adopt the version that employ the boosted cross entropy (BCE) loss in the outer minimization. The regularization hyperparameter is fixed as 1/A 6.0 to be aligned with TRADES.
Training setting. We employ SGD the optimizer. The momentum and weight decay are set as 0.9 and 0.0005 respectively, which are aligned with the common practice. For Wide ResNet, we conduct the training for 120 epochs, with the learning rate starting 0.1 and reduced by factor 10 at epoch 100 and 110. For other neural architectures, we conduct the training for 160 epochs, with the learning rate starting at 0.1 and reduced by factor of 10 at epoch 80 and 120.
We conduct experiments on three datasets including CIFAR- 10, CIFAR-100 and Tiay-ImageNet without additional data.
Dataset.
Neural architecture.
We conduct experiments with pre-activation ResNet-18, 18, VGG and Wide
ResNet.
Hardware.
We conduct experiments on NVIDIA Quadro RTX A6000..
G EXPLANATION
In this section, we move one step further to probe into the effect of data quality in adversarial training. In previous works, it has been proven that standard classifier and robust classifier learn fundamentally different features (Tsipras et al. 2019). Useful features prevail in the dataset, but are not necessarily robust and comprehensible human (Ilyss et al. 2019). In light of such analyses of robust/non- robust features, we motivate from a feature learning perspective and try to uncover the potential mechanisms of how low-quality data is interconnected with the existing difficulties in adversarial training including Under the assumption that similar features will be recognized in a specific class of examples, we selectively conduct the adversarial training either one or a few classes of examples. In this way, we can isolate the impact of each set of similar features and analyze the effects of low-quality data on the learning of features.
-ccuuracy trade-off, robust overfitting and robustness overestimation.
G.1 DOBUSTNESS-ACCURAEE TRADE-OFF
Towards understanding the correlation between robustness- accuracy trade-off and data quality, we conjecture that low-quality data will cause the loss of useful features in adversarial training.
As shown in Figure 3a low-quality examples are intrinsically ambiguous from a human perspective. Therefore, the perturbation radius relatively large, the adversarial attack may generate reasonable images of classes other than the true class. Indeed, as shown in 3a the adversarial counterparts of low-quality examples catch salient characteristics of the classes that the model predicts. contrast, the adversarial counterparts of high-quality examples are still explicit images of the true classes. 16/255 based
Here, we adversarially attack an example using PGD-20 with perturbation radius on the best model obtained through PGD-AT. The perturbation radius 16/ /25 is exactly twice the perturbation radius commonly used in adversarial training, which implies that adversarial counterparts generated on the low-quality examples during the training might be marginal cases from human perspective.
Based on the above observation, suspect that the adversarial training might not be suitable for low-quality examples. is explained in llas et al. (2019) that adversarial training works because the adversary can exploit non-robust features of classes other than the true class, thus forcing the model
24

---

Under review as a conference paper at ICLR 2022
to rely only the robust features of the true class. However, due to the ambiguity, such "distracting" features generated on a low-quality example might be too prominent such that it overwhelms the regular features of other classes when it is forced to be classified into the true class of this example, thus significantly damage the recognizability of other classes.
All
High-quality Subset
airplnne
orpbne
ZU00mAD l2
oulo mo
80
slep
TestAcc
TestAcc
20
Brplana
ship
Blck
Figure 15: Fine tune the model obtained by standard training with adversarial examples generated by targeted attack either all the training examples or the high-quality subset among them. When the attack in adversarial training is primarily targeted to "Airplane", the test accuracy of "Airplane" after fine-tuning significantly lower, while the test accuracies of other classes are comparable. Instead, when the attack is targeted to "Automobile", the test accuracy of "Automobile" after fine-tuning is significantly lower. In contrast, when fine tuning the model only on the high-quality subset, such difference is not significant.
We manifest such loss of recognizability by robustly fine tuning the model obtained by standard @ training. Instead of untargeted attack, we use targeted attack in adversarial training to isolate the effect of recognizabilityy loss. Specifically, for the PGD attack employed in adversarial training, we replace the inner maximization by minimization towards a target class c, except for those examples have class e as their true labels, where the target is directed to another class c". We refer this special case c" adversarial training. Figure 15 shows the average standard test accuracy produced by fine-tuning for 30 epochs using Airplane-Truck, Automobile-Tie -Trcck and Ship-Truck adversarial training. These classes are selected because the model produces highest standard test accuracy on them. One can find that, when the attack in adversarial training is primarily targeted to "Airplane", the test accuracy of "Airplanne" after fnn-tuning is significantly lower, while the test accuracies of other classes are comparable. Instead, when the attack is targeted to "Automobile", the test accuracy of "Automobile" after fine-tuning is significantly lower. In contrast, when fine tuning the model only on the high-quality examples, such difference is not significant. This reflects the detail of how qualquality examples in adversarial training hurts the learning of useful features.
G.2 ROBUST OVERFITTING
Here, we further show that the low-quality examples will also damage the robust recognizability. In adversarial training, the gradient- based adversary may generate robust features of another class on low-quality examples due to their intrinsic ambiguity, especially when the model is already relatively robust. Consequently, the model may forget the robust recognizability of other classes because it is forced to classify such robust feature to the original class, which leads to robust overfitting.
We manifest the loss of recognizability by fine-tuning the best model obtained by regular adversarial training. Similarly, we use targeted attack to isolate the effect. accuracy produced by Truck and Ship-Truck adversarial training. These classes are selected because the model produces highest robust test accuracy on them. One can find similar results that, when the attack in adversarial training is primarily targeted to "Airplane", the robust test accuracy of "Airplane" after fine-tuning is significantly lower. Instead, when the attack is targeted to "Automobile", the robust test accuracy of "Automobile" after fin-ttnning is significantly lower. In contrast, when fine tuning the model only on the high-quality data, such difference is not prominent.
[6] the average robust test the best model for 30 epochs using Airplane- Truck, Automobile-
'I is not common use targeted attack in adversarial training, only for demonstration here.
25

---

Under review as a conference paper at ICLR 2022
All
High-quality subset
arplans bubo ship
60-
Robust Test Acc
Robust Act
50
60-
60
20
Applans
Back
Figure 16: Fine tune the best model obtained in adversarial training with targeted attack on either all the training examples or the high-quality subset among them. When the attack in adversarial training is primarily targeted to "Airplane", the robust test accuracy of "Airplane" after fine-tuning is significantly lower, while the performance of other classes are comparable. We can observe similar patterns when the attack is targeted to "Automobile" or "ship". In contrast, when fine tuning the model only on the high-quality subset, the difference is insignificant.
G.3
ROBUSTNESSS OVERESTIMATION
+Low-quality
+ighh-quliity
2:00
1.75
High-qullity set
1.75
High-ouallyy set
sAimbane
*Alralane
1.50
+5hip
+Shp
Overestimation
1:25
+hiplnne& Ship
1.25
Mirplene Ship
1:00
0.75
0.75
0.50
0.50
0.25
0.00
0.00
Airpione
Automobiirdd
Car
Dear
Dog
Free
Hone
Ship
Track
Airplane
Autombiiffe
Cit
Denr
Deg
Freg
Home
Ship
Truc
Figure 17: The overestimation gap, namely the difference between P0 and Auto Attack evalua- tion, generated by adding examples of two competing classes into a high-quality subset.
+Airplane& & Horse
+5 hip & Horse
2.01
2.00
High-qually
High-qually sec
1.75
Airpane
Ship
1.50
+Horse
1.50
*Horee
Overestimation tion
1.25
AArplane horse
1.25
Shhp & Horse
1:00
1.00
0.75
0.75
0.30
0.25
0.25
0.00
Arplarne
AutomatiBad
Cat
Dest
Feg
Home
Stip
Treck
A4
Alplane
Automotifaa
Bin
Cat
Dear
Ong
ring
Home
Ship
Trick
Figure 18: The overestimation gap generated by adding examples of two non-competing classes into a high-quality subset.
We show that the low-quality data causes robustness overestimation through a mechanism which we refer as "competing". In Figure 17 we sample a subset by adding 500 additional low-quality example a class-balanced high-quality subset of size 10', adversarially train the model on
$10% of all the training examples in one class
26

---

Under review as a conference paper at ICLR 2022
it, and show the overestimation gap. One find that when we only add examples of one class either "Airplane" or "Ship", the overestimation gap increases, but not significantly compared to the overestimation gap of the original high-quality subset. However, if we add the examples of two "competing" classes "Airplane" and "Ship" at the same time, the overestimation gap increases substantially, and mostly attributes the class "Ship". Here, "competing" classes means these two classes contain images that are likely to have similar feature The overestimation gap is not significant if the additional examples are not from "competing" classes, as shown in Figure [8] where we add additional examples from either "Airplane" and "Horse", or "Ship" and "Horse". The overestimation gap also not significant if the additional examples are not low-quality, shown in Figure 17 where we add additional high-quality examples, even if they are from competing classes,
Towards understanding this mechanism, we focus on the inner maximization since overestimation mainly indicates weakened generation of adversarial perturbation, while previously we assume this process is ideal. Recall that adversarial attack works because in the inner maximization, the adversary optimizes towards classes other than the original class and thus can exploit distracting features to fool the model (yyas et al. 2019). However, since the ambiguous low-quality examples from different classes tend contain similar features, the adversary may optimize towards different classes starting from similar features, subsequently damage its capability to exploit distracting features of these classes.
'One refer the sample images we showed in Figure
27


Reviews:
[{"review_id": "oy7S6KRkWQ", "reviewer": "Reviewer_Rp8o", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**:\n* The paper has a clear message that adversarial training and nominal training do not benefit from the same type of samples.\n* Extensive experiments on CIFAR-10 to analyze the robust performance of \"high quality\" data (selected by their proposed criterion) under three angles commonly discussed in the Lp-norm community. \n* The paper is well written and very clear.\n\n**Weaknesses/Suggestions/Questions**:\n1) **Main Weakness**. It really lacks of an analysis in the extra data case. This case should be actually perfectly suited for the message of the paper as extra data for CIFAR-10 is known to be of lesser quality (as per the references in the paper). Similarly, recent works [1, 2] have proposed to use generated data in the Lp-norm case. The proposed quality criterion in the paper would be great to analyze the quality of the generated images compared to the original CIFAR-10 images.\n2) In Figure 3, \"with their labels predicted by a model\", which model? This is a bit vague.\n3) Figures 4 to 7. First, please put in the caption how the robust test accuracy is obtained. Against PGD-10, PGD-40? It is unclear. Second, all the first rows are labeled \"PGD\". I think you mean \"Adversarial Training\" as introduced by [3]. PGD is the optimization procedure and it also used on TRADES. So the naming \"PGD\" is misleading.\n4) Typo: \"can also be found in our Figure 1(a)\" -> \"can also be found in our Figure 1(c)\" on page 3.\n\n**References**:\n1) Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, Timothy Mann. *Fixing data augmentation to improve adversarial robustness*. 2021.\n2) Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, Prateek Mittal. *Improving adversarial robustness using proxy distributions*. 2021.\n3) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu. *Towards deep learning models resistant to adversarial attacks*. 2017.", "summary_of_the_paper": "**Few sentences summary**: the paper proposes an empirical study on how data quality impacts robust performance in the Lp-norm setting under three angles: robust overfitting (i.e. the difference in accuracy between the best and last checkpoints), robustness evaluation (i.e. consistency of the robust performance when trying various robust evaluations like AutoAttack) and clean/robust accuracy trade-off (i.e. the gap in performance between the clean and robust accuracies). They show that \"high quality\" data (selected with their proposed criterion) performs better than \"low quality\" data for these three performance metrics.\n\nRegarding the **results** and **contributions**:\n* The authors propose a quantitative definition of \"quality\"  based on the average training robust accuracy over epochs per sample.\n* They show that \"low quality\" samples are better for standard training whereas \"high quality\" samples are better for adversarial training.\n* Studies on 1) robust overfitting, 2) robustness evaluation and 3) clean/robust accuracy trade-off on the datasets CIFAR-10/100 and TinyImageNet for Adversarial Training and TRADES.", "summary_of_the_review": "Empirical study paper with an interesting message that adversarial training and nominal training do not benefit from the same type of samples. The paper could have gone more in depth by studying the extra data case which is commonly used in the literature and gets the SOTA results. Especially, this extra data case normally fits the message of the paper so it is a missing element in the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "YNieYIy79db", "reviewer": "Reviewer_mNG6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- Interesting direction- I find this work very exciting since it contributes to our understanding of adversarial training.  Many works have demonstrated that adversarial training requires more data.  This work expands along this direction by observing how the quality of data impacts generalization of adversarial training.\n- Good scope in experiments, significant results- The authors perform experiments on multiple datasets and adversarial training methods and the trends look consistent across methods.  Patterns discovered between data quality and properties like robustness, robust overfitting, and robustness estimation also look significant.\n\nWeaknesses:\n- In the main text, it say that results for WideResNet architecture are in the appendix, but when I looked at the appendix I couldn't find figures for WRN corresponding to the experiments in the main text.  I think adding some figures for WRN with gradually removing lowest quality vs random would also help show the generalizability of the observed trends.\n- In section 3.2 related works, the authors state that prior works show that more data can be detrimental to adversarial training, but I don't agree that is the case.  I think prior works demonstrate that adversarial training benefits a lot from additional data.  For instance, Gowal et al. 2020 is cited to support the statement that additional data unrelated to the original dataset can hurt adversarial robustness, but Gowal et al. demonstrate that additional data always improves over the no additional data case.  There is also discussion of label noise in this section, but it isn't clear how label noise is connected to the idea of additional data and data quality.  Additionally, I think discussion of other works which measure notions of hardness and data quality for adversarial training can also be added to related works.  For instance, methods of measuring difficulty of adversarial examples used for curriculum adversarial training [1,2,3].  Recently there has also been a line of works discussing ways to measure the data quality of additional data generated by generative models for use with adversarial training [4, 5].\n\n\n[1] Wang, Yisen, et al. \"On the Convergence and Robustness of Adversarial Training.\" ICML. Vol. 1. 2019.\n[2] Zhang, Jingfeng, et al. \"Attacks which do not kill training make adversarial learning stronger.\" International Conference on Machine Learning. PMLR, 2020.\n[3] Cai, Qi-Zhi, Chang Liu, and Dawn Song. \"Curriculum adversarial training.\" Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018.\n[4] Sehwag, Vikash, et al. \"Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?\" arXiv preprint arXiv:2104.09425 (2021)\n[5] Gowal, Sven, et al. \"Improving Robustness using Generated Data.\" arXiv preprint arXiv:2110.09468 (2021).", "summary_of_the_paper": "The paper investigates the impact of data quality, measured by the proportion of epochs in which the model classifies a specific input correctly during training, on the robustness, generalization, and robustness-accuracy tradeoff of adversarially trained models.  Unlike with standard training, the authors find that more difficult inputs (lower quality inputs) can hurt adversarially trained models.  They find that compared to randomly removing data during training, removing low quality data can lead to higher robustness, less robust overfitting, less robustness overestimation, and less robustness-accuracy tradeoff.", "summary_of_the_review": "I vote to accept this paper since I find the empirical contributions significant; the authors rigorously study the impact of data quality on various properties of adversarially trained models including robustness, robust overfitting and robustness overestimation, and robustness-accuracy tradeoff.  I think the discussion of related works can be improved though.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "OBfkOafsATy", "reviewer": "Reviewer_VxQb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(Edited score post rebuttal)\n\nStrengths: \n(i) This paper studies an important problem on the effect of data quality on different aspects of robust training. \n(ii) The paper starts with the intuitive idea that data quality should matter and attempts to make a systematic empirical investigation across multiple datasets. \n(iii) Overall, the paper is easy to read and the experiment details/results are clearly presented. \n(iv) The observation that different sets of points improve robust vs standard accuracy is quite interesting to me (though that wasn't the focus of the paper). \n\nWeaknesses: While the paper makes an interesting investigation, I worry that the conclusions/implications of the study are unclear and some inferences seem problematic to draw. \n(i) One claim the paper makes is that low quality data can be detrimental to robust accuracy. Concretely, if we remove points from the dataset, that would improve robustness. Looking at Figure 4, the cases where this happens are: CIFAR-100 with TRADES and Tiny Imagenet with TRADES. I am confused why the conclusions vary between TRADES and Adversarial training? How is the \\beta parameter set in the TRADES experiments. The fact that TRADES and adversarial training differ in their trends suggests that the beta parameter that controls accuracy vs robustness is important. The paper mentions that the hardness is an \"intrinsic\" measure since the relative ordering doesn't vary much across methods. So is it true that removing the *same* points leads to different performance with TRADES and adversarial training?\n(ii) Other than the (slightly questionable) claim that removing low quality data improves performance, I didn't find the other implications on robustness particularly surprising. Even in standard training, it has been shown that there are some \"support vectors\" (or prototypes) which effectively dictate the final model learnt, and the rest of the dataset does not matter as much. The paper seems to mainly replicate this for robust training. \n(iii) Computing the data quality measure: If I understand correctly, you need to compute the robust accuracy across the training checkpoints. Which attack do you use to compute this? Does it matter how strong the attack is?\n(iv) What's an actionable takeaway from this work? Aside from the caveat of point (i) above on whether there is a reliable takeaway, I wonder how to actually operationalize the idea of using higher quality data? The data quality of each point seems to depend on the remaining points (since it comes from some ordering of robust accuracy), and computing this requires training the model on all the data. Can the authors please comment with any thoughts here?\n(iv) Computing data quality for extra data: Can you use checkpoints trained on CIFAR-10 to evaluate data quality of additional data (for e.g. from Carmon et al. 2019) not present in the training set? Basically, this would involve computing the robustness at the new point (not in training) across different training epochs. If the authors could show that this improves performance over adding all the mined extra points, I would be convinced that there is something interesting here. Relatedly, how does this measure of data quality empirically differ from just logits for confidence (which was what was used to mine the extra data in Carmon et al. 2019). \n", "summary_of_the_paper": "This paper studies the effect of data quality on adversarial robustness. Specifically, they focus on one measure of data quality (number of times there is a perturbation that is misclassified across training iterations). They study the effect of data quality on robust overfitting, robustness-accuracy tradeoffs and \"robustness overestimation\" (gap between strong and weak attacks). The main conclusions reported are that data quality as measured by their metric plays an important role in all three aspects, and a suggested takeaway is that we need data of higher quality to improve robustness. ", "summary_of_the_review": "This paper studies an interesting question about the effect of data quality on robustness. Unfortunately, I find the conclusions for robustness as emphasized in the paper either unsubstantiated (removing low quality data improves robustness) or difficult to make actionable. Nevertheless, there are some interesting empirical observations in the paper such as the difference between important points for standard and robust accuracy, the effect of data quality on robustness-accuracy tradeoffs which were not known previously. I apologize for the late review - there was some issue at my end and I realized the review didn't get submitted. I would really appreciate if the authors could respond to my questions above. Thanks!", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2yTTrzYRWS1", "reviewer": "Reviewer_soLM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1.\tThis paper evaluates the learning stability of instances under various settings, which indicates that the estimation of quality for each data is almost accurate and consistent.\n\n2.\tThis paper is well organized. The authors analyze the influences of low-quality data on the three issues separately, which empirically demonstrates removing the low-quality data can mitigate the issues.\n\nWeakness:\n\n1.\tCalculating the quality of data is time-consuming and computationally expensive. The quality of data is closely related to a particular dataset. That is to say, when the dataset is updated such as adding some new data into the dataset, the quality of data in the new dataset needs to be recalculated. It should be a limitation for the practical use of low-quality data.\n\n2.\tRemoving the low-quality data sometimes marginally improves adversarial robustness, but more often hurts the adversarial robustness and natural generalization. Therefore, the significance of the proposed low-quality data seems minor.\n", "summary_of_the_paper": "This paper proposes a metric for evaluating the learning stability of data and point out that unstably-learned instances are of low-quality for adversarial training. Through extensive controlled experiments, this paper investigates the impact of low-quality data on three issues in adversarial training, i.e., robust overfitting, robustness overestimation, and robustness-accuracy trade-off. The experimental results show that removing the low-quality instances can mitigate the issues.", "summary_of_the_review": "Overall, this paper systematically study the influences of data quality on the three problems in adversarial training. However, I am skeptical about the practical values of the proposed metric for data quality.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the effect of data quality on adversarial robustness. It focuses on a single measure of data quality (number of times there is a perturbation that is misclassified across training iterations). The authors study the effect of data quality on robust overfitting, robustness-accuracy tradeoffs and "robustness overestimation" (gap between strong and weak attacks). The main conclusions reported are that data quality as measured by their metric plays an important role in all three aspects, and a takeaway is that data of higher quality may improve robustness. While the reviewers appreciated the premise of this work, some concerns remain post rebuttal. For example, few reviewers remain skeptical of the universality of the notion of "data quality" as measured in the paper because different training methods behave differently and the data quality measured in the paper is tailored to a particular training algorithm. Some reviewers also opined that at least one of the practical implications discussed in the rebuttal should be systematically investigated and that it is important to study the effectiveness of different data quality measures, especially for the extra data. Given all this, we are unable to recommend acceptance at this time. We hope the authors find the reviewer feedback helpful.
Decision: Reject