NoiLin: Improving adversarial training and correcting stereotype of noisy labels 
Authors: Jingfeng Zhang
Xilie Xu
Bo Han
Tongliang Liu
Lizhen Cui
Gang Niu
Masashi Sugiyama
Venue: Accepted by TMLR ()

Abstract: Adversarial training (AT) formulated as the minimax optimization problem can effectively enhance the model's robustness against adversarial attacks. The existing AT methods mainly focused on manipulating the inner maximization for generating quality adversarial variants or manipulating the outer minimization for designing effective learning objectives. However, empirical results of AT always exhibit the robustness at odds with accuracy and the existence of the cross-over mixture problem, which motivates us to study some label randomness for benefiting the AT. First, we thoroughly investigate noisy labels (NLs) injection into AT's inner maximization and outer minimization, respectively and obtain some observations on when NL injection benefits AT. Second, based on the observations, we propose a simple but effective method---NoiLIn that randomly injects NLs into training data at each training epoch and dynamically increases the NL injection rate once robust overfitting occurs. Empirically, NoiLIn can significantly mitigate the AT's undesirable issue of robust overfitting and even further improve the generalization of the state-of-the-art AT methods. Philosophically, NoiLIn sheds light on a new perspective of learning with NLs: NLs should not always be deemed detrimental, and even in the absence of NLs in the training set, we may consider injecting them deliberately.


Published in Transactions on Machine Learning Research (06/2022)
NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels
Jingfeng Zhang'
RIKEN Center for Advanced Intelligence Project (ATP)
Xilie Xu"
muniiie@@comp.mus.
School of Computing, National University of Singopore
Bo Han Department of Computer Science, Hong Kong Baptist University
bhonml@comp.hkbu. www.://wmmpp.b.e.....o
Tongliang Liu
tongliang, liu Bww.yynnyy.eee edu. on
Trustworthy Machine Learning Lab, University of Sydney
Lizhen Cui
cl2@edu. edu.cn
School of Software & Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University
Gang Niu
gong.niu@riken..jp
RIKEN Center for Advanced Intelligence Project (AIP)
Masashi Sugiyama
sugi@k. wtokyyy.ttkkoo...o ac.
RIKEN Center for Advanced Intelligence Project (ATP)
Graduate School of Frontier Sciences, the Universily of Tokyo
Reviewed on OpenReview: https: // opt.//veeeer.......mm net/ form?? id= =
Abstract
Adversarial training (AT) formulated as the minimax optimization problem can effectively enhance the model's robustness against adversarial attacks. The existing AT methods mainly focused on manipulating the inner maximization for generating quality adversarial variants or manipulating the outer minimization for designing effective learning objectives. However, empirical results of AT always exhibit the robustness odds with accuracy and the existence of the cross-over mixture problem, which motivates us study some label random- ness for benefiting the AT. First, we thoroughly investigate noisy labels (NLs) injection into AT's inner maximization and outer minimization, respectively and obtain the observations on when NL injection benefits AT. Second, based on the observations, we propose simple but effective method No that randomly injects NLs into training data at each training epoch and dynamically increases the NL injection rate once robust overfitting occurs. Em- pirically, NoiLIn can significantly mitigate the AT's undesirable issuc of robust overfitting and even further improve the generalization of the state-of-the-art AT methods. Philosoph- ically, NoiLIn sheds light new perspective of learning with NLs: NLs should not always be deemed detrimental, and even in the absence of NLs in the training set, may consider injecting them deliberately. Codes are available in https:/ /ttg/ggttthhooooo hwt.b///e////tt////////m Siffheart/NoiL...
1 Introduction
Security-related areas require deep models to be robust against adversarial attack (Szegedy et al. 2014). To obtain adversurial robustness, adversarial training (AT) (Madry el. al. 2018) would be currently the most
"Equal contribution

---

Published in Transactions on Machine Learning Research (06/2022)
Epoch 1
Epoch 2
Epoch Epoch
(a) Existing
Legend
AT frameworks
AT on training set
Clean training set
S
Noisy training set Square data (Label: Red)
S
Triangle data (Label: Green)
(b) Our NoiL.I gradually increuses NL injoction
Round (Label: Orange)
into frameworks
Figure 1: Comparisons between the existing AT framework (panel (a)) and our AT-NoiLIn framework (panel (b)). At each training epoch, AT-NoiLIn randomly flips labels of a portion of data (marked in the black edges), generating noisy-label training set S for the learning.
effective defense that has far not been comprehensively compromised (Athalye et al 2018 in which AT is formulated as a minimax optimization problem with the inner mazimization to generate adversarial data within small neighborhoods of their natural counterparts and the outer minimization to learn from the generated adversarial data.
However, AT empirically shows a severe tradcoff between the natural accuracy of natural test data and the robust accuracy of adversarial test data [Tsipras et al.) 2019). Besides, Zhang al.) (2020b) showed AT has cross-over mirture problem, where the adversarial variants of the natural data cross the decision
boundary and fall in the other-class areas. Even, [Sanyal et al. (2021) showed the benchmark dataset (such
CIFAR (Krizhevsky] 2009)) contains some noisy data points. Furthermore. Donhauser et al. (2021) showed
an unorthodox way to yield an estimator with a smaller robust risk, i.e., flipping the labels of a fixed fraction of the training data.
The above facts motivate us to explore manipulating labels for benefits that were largely overlooked by the existing AT studies. To enhance adversarial robustness further, the existing AT methods have mainly focused manipulating the inner maximization the outer minimization. For example, by manipulating the inner maximization for generating quality adversarial data, Zhang et al. (2020b) generated friendly adversarial data that are near the decision boundaries but are wrongly classified. On the other hand, by manipulating the outer minimization for designing the model's loss functions for the learning, Zhang et al. (2019b) designed TRADES that has two loss terms, i.e., the cross entropy loss on the natural data and the Kullback-Leibl (KL) divergence loss on the adversarial data. Since NLs were often deemed to hurt the training (Angluin Laird 1988), little efforts has been made to investigate noisy labels (NLs) in AT
To correct this negative stereotype of NLs, firstly, this paper thoroughly investigates NL injections in AT's every component. (i) We inject NLs in inner maximization. In each training minibatch, we randomly choose a portion of data whose adversarial variants are generated according to the flipped labels; the labels in outer minimization are intact for the learning. The label-flipped adversarial variants are far from the decision boundaries, thus the label-flipped adversarial data become no longer adversarial to the current model. In other words, label-flipped adversarial data serve more like natural data. a result, when AT learns label- flipped adversarial data, the deep model's generalization gets increased but the robustness gets decreased. We inject NLs in outer minimization. We generate the adversarial data in each training minibatch according to the intact labels but randomly flip portion of labels for the learning. NL injection in outer minimization leads to high data diversity for the learning, i.c., the deep model learns from different labels sets over the training process. From another perspective, NL injection in outer minimization implicitly averages deep models trained with different label sets. Therefore, NLs serve as regularization that can largely alleviate the AT's undesirable issue of robust overfitting (Rice et al. (020)) and even slightly enhances AT's best- spoch-checkpoing robustness. (iii) We inject NLs in both inner maximization and outer minimization. In each training epoch, we randomly choose portion of data to flip their labels; then, conduct standard
2

---

Published in Transactions on Machine Learning Research (06/2022)
adversarial training (SAT (Madry et al. 2018 on the noisy set. In other words, adversarial variants of this portion of data are generated according to the flipped labels, and their flipped labels are also used for the learning. The empirical results are very similar to case (ii).
Inspired by the above three observations, secondly, we propose a simple method "NoiLIn" that dynamically adjusts NL injection into AT (see Figure | for illustrations). Tn each training epoch, we first randomly flip portion of labels of the training set, and then execute an AT method (such as SAT, TRADES (Zhang et al. 2019b), WAP (Wu et al., 2020)) using the noisy-label set. As the training progresses, increase the flipping portion if there occurs robustness degradation (evaluated on validation set).
Our contributions are summarized as follows. (a) We study NLs from a novel perspective, i.e., injecting
NLs into the AT's training process; we observe that NI. injection can even benefit. AT. (b) Based on our observations, we propose to gradually increase the NL injection rate over the training process (i.e., NoiLIn). NoiLIn can significantly relieve the issuc of robust overfitting of some AT methods such as SAT and TRADES. Using large-scale Wide ResNet. Zagoruyko & Komodakis (2016) on the CIFAR-10 dateset Krizhevsky (2009), NoiLIn can even improve the generalization of current state-out-of-art (SOTA) TRADES-AWE Wu et al. (2020) by surprisingly 2.2% while maintaining its robustness (measured by auto attack (AA) Croce & Hein (2020g) (c) Philosophically, we should not always consider NLs to be detrimental. Even in the absence of NTs in the training set, we may consider injecting them deliberately.
2 Background and Related Work
There are many ways to improve the model's adversarial robustness, such as robustness certification
Hein
& Andriushchenko 2017 Weng et al. 2018 Wong & Kolter 2018 Mirman et al.,
2018
Tsuzuku et al. 2018
Lécuyer al. 2019 Xiao et al.) 2019 Cohen et al.) 2019 Balunovie & Vechev 2020
Zhang et al.]
2020a
Singla & Feizi
2020;
Zhang et al. 2021a Liptschiz regularization (Cisse et al.
2017
Moosavi-
et a..
2019 Qin et al. 2019). incorporating attack modules
Yan et al.) 2018).
and detecting adversarial
data
Metzen et al. 2017] Li & Li 2017: Carlini & Wagner 2017b Tian et al.
2018
Ma et al.) 2018 Lee
let al.
2018
Pang et al.] 2018
Smith & Gal 2018 Roth el al.] 2019
Liu el al.
2019
Yin & Rohde 2020
Sperl
et al. 2020 Cohen et al.
2020 Sheikholeslami et al. 2021 Chen et al.,
2021a
Yang et al.
2020b Qin
jet al.]
2020:
Tian et al. 2021
Wu et al.) 2021b). Nevertheless, adversarial training (AT) still stands out
as the most effective defense against the adaptive and strong attacks, attracting significant attention. This
section briefly reviews the background of AT and provides related work.
Adversarial Training (AT) AT aims to a) classily the natural data correctly, and b) make the decision boundaries "thick" so that no data are encouraged to fall nearby the decision boundaries. AT's learning objective is as follows.
Let (X, Loo) be the input feature space X' with the infinity distance metric Lo((E, = || - a'lloo, and
[x] = {a' E X' (x, sc
(1)
be the closed ball of radius e > centered at in X. Given a dataset S ((2, y:) where x; E X' and Vi y = 10,1...., C 1}, AT's learning objective is formulated as a minimax optimization problem, i.e,
min fej
Σ (((a)))))) B arg max (((())
(2)
inner maxiimization
outer minimizatiin
Eq. (2 implies the AT's realization with one step of inner macimization on generating adversarial data Zz and one step of outer minimization on fitting the model f to the generated adversarial data (ă,, v1).
Projected Gradient Descent (PGD) To generate adversarial data, SAT Madry et al. 2018 uses PGD to approximately solve the inner maximization. SAT formulates the problem of finding adversarial data as a
Throughout this paper, use AT denate adversarial training methods general and use SAT to specify the standard AT method by Madry et al. (2018).
3

---

Published in Transactions on Machine Learning Research (06/2022)
constrained optimization problem. Namely, given a starting point x²()² E X' and step size a > 0, PGD works as follows:
x (1+1) = Mm (z + a sign( (V 2(t) v))) 20
(3)
until certain stopping criterion is satisfied. Commonly, the criterion can be a fixed number of iterations
K. In Eq. equation 3 E is the loss function in Eq. equation 2 . refers to natural data or natural data
corrupted by a small Gaussian or uniform random noise; y is the corresponding true label; xx()) is adversarial data at step t; and (oa(() is the projection function that projects the adversarial data back into the c-ball centered al (0 if necessary.
Broad Studies on AT Existing literature
AT in many aspects such as customizing the
inner maximization for simulating the better adversary (Goodfellow al. 2015 Shafahi el. al. 2019b
Zhang
al. et
2019a
Wong et al.
2020
Vivek & Babu
2020 Cai al.] 2018 Wang a..
2019: Zhang al.,
2020b;
Sriramanan et al. 2020
the outer minimization
for designing better loss functions /taooon///reuuuu rreullaizerr
Farnia
ct
al.]
2019
Song et al.
2019
Zhang et al.]
2019b
Ding et al.
2020
Wang et al.) 2020
Wu et al.
2020;
Zhang
et. al.
2021b; Pang et al.
2021 Chen et al.
2021b) designing /earrciiiaragnnngg robust netw structures (Xie
cct al.
2020
Xie & Yuille 2020
Guo et al. 2020 Schwag ct al. 2020
Yan ct al.] 2021 Du et al.)
2021 Li
iet al.
2021
Huang et al.
2021
Li et al.] 2021). employing multiple models (Pang el al. 2019 Yang et al.
2020a),
augmenting the training data (Tramèr et al. 2018: Schmidt al. 2018 Alayrac et al., 2019 Carmon
et al.
2019
Najafi et al.
2019 Song et al.
2020 Lee et al.) 2020; Gong et al.) 2021 Gowal et
2021b;
Wu et al. 2021a), and analyzing AT's intriguing properties Tsipras et al.) 2019 lyaas et al.
2019 Stutz
et al.
2019
Yin et al. 2019 Rice et al.; 2020; Zhhang & Zhu 2019: Raghunathan el al. 2020
Yang el al.
2020c..
g. Besides, recent studies showed AT could benefit other domains such as pre-training Chen et al.
2020
Jiang et al. 2020), out-of-distribution generalization (Yi ct al. (021). inpainting (Khachaturov ct al.
2021
interpretability (Ross& & Velez 2018 fairness (Xu et al. 2021 and so on.
Interaction between NLs and AT To avoid confusion, we clarify the differences between this paper and the existing studies of AT on NLs. NLs practically exist in the training set (Natarajan et al., 2013
and therefore, some studies have explored the effects of AT on NLs.. Alayrac et al. (2019) showed that NLs
degraded both generalization and robustness of AT, but robustness suffers less from NLs than generalization.
Sanyal et al.] (221)) showed robust training avoids memorization of NLs. Furthermore, Zhu et al. (2021) showed AT has a smoothing effect that can naturally mitigate the negative effect of NTs. Nevertheless, those studies assumed NLs exist in the training set, which is detrimental to AT. In comparison, we assume the training set is noise-free. We treat NLs as friends and deliberately inject NLs to benefit. AT in terms of
relieving robust overfitting of SAT (Mdryy et al. 2018 and TRADES (Zhang et al. 2019b), even improving TRADES-AWP (Wu et al. 2020 s generalization while maintaining its peak robustness.
Relation with DisturbLabel (Xie et al., 2016) DisturbLabel, the most relevant work to NoiLIn, also randomly selects a small subset of data in each training epoch and then sets their ground-truth labels to be incorrect. Xie et al. (2016) studied only standard training (ST) for generalization. Differently, our work focuses on AT's robustness and generalization. Besides, compared with DisturbLabel that injects small ratio NLs (20% or less) to alleviate ST's overfitting, AT needs injecting larger NLs ratio (e.g., 40% more) to alleviate the AT's robust overfitting (see experiments in Figure 4 Compared with ST, AT encounters worsen situations of data overlaps of different classes, thus requiring stronger label randomness.
Relation with Memorization in AT (Dong et al., 2022 The independent and concurrent work by Dong et al. 2022) explored various AT methods (i.e., training stability of SAT and TRADES methods) under completely random labels. Besides, Dong et al. (022) proposed to mitigate SAT's and TRADES's issue of robust overfitting via adding temporal ensembling (TE) (Laine & Aila 2017 as additional learning objective to penalize the overconfident predictions. This shares a similar idea to injecting Icarned smoothing (Chen el al. 2021b) (i.e., adding several additional learning objectives) for mitigating the robust overfitting We argue that our NoiLIn is simpler but is no worse than the above peer methods. TE and smoothness injection methods (Dong et al., 2022: Chen et al., (0021b) both need add additional lcarning objectives, which introduce more hyperparameters than NoiLIn's for finetuning the performance. Besides, without introducing additional learning objectives, No saves computational resources.
4

---

Published in Transactions on Machine Learning Research (06/2022)
Natural test data
AA
Geveration trace of PGD adversarial data
Natura test arentreyy (36)
Robust tost. accuracy (%))
Class
Clasr2
20
02
0.65
022
201
Labeka Midditel Middihell
Class:D
50
60
401
129
Epoch
Epoch
(a) CIFAR-10 dataset
(b) Supporting schematics
Figure 2: Figure ((a) shows the learning curves of injecting various levels of NL in inner maximization. (The number in the legend represents NT. injection rate y/; the black dash line represents SAT.) Figure 2(b) shows the generation trace of adversarial data on a synthetic ternary classification. (The white round denotes natural data. The blue trace is generated with correct label; both yellow and blue traces are generated with noisy label (NL). The color gradient represents the prediction confidence: the deeper color represents higher prediction confidence.) NL injection in inner maximization prevents generating effectively adversarial data.
Relation with Label Manipulations Benefiting Robustness. There debates whether label smoothing and logit squeezing (Shafahi et al. 2019a), and logit pairing (Engstrom et al. 2018 Mosbach jtt al. (018)) genuinely benefit adversarial robustness or mask gradients for overly reporting robustness. This paper avoids such a debate by evaluating our NoiLIn method using the strongest AutoAttack (AA) (Croce & Hein 2020), which should thwart the concerns of gradient. obfuscations (Athalye et al. 2018
3 A Closer Look at NL injection in AT
Before introducing our algorithm NoiLIn, we get started with a thorough understanding of NL injection in AT. Specifically, AT (see Eq. (2) has two label-related parts: one is inner maximization for generating adversarial data; another is outer minimization for learning the generated adversarial data, In this section, inject NLs in the two parts scparately and obtain some intriguing observations.
3.1 NL Injection in Inner Maximization
Experiment details We conducted experiments of injecting NLs in inner maximization on the CIFAR-10 dataset Krizhevsky (2009). In Figure 2(a] we compare AT with NLs in inner maximization (red lines with different color degrees) and SAT (black dashed lines). We inject cymmetric-flipiing NLs Van Rooyen et al. (2015), where labels y are flipped at random with the uniform distribution. In each training mini-batch, we randomly flip 7) portion of labels of training data; then, the adversarial data are generated according to the flipped labels. The noise rate is chosen from {0, 0.05, 0.1, 0.2, 0.3, 0.5). Note that it is exactly SAT when = 0. The perturbation bound is set to Grain = 8/255; the number of PGD steps is set to K = 10, and the step size is set to a 2/255. The labels of outer minimization are intact for the learning. We trained ResNet-18 He et al. (2016) using stochastic gradient descent (SGD) with 0.9 momentum for 120 epochs with the initial learning rate of 0.1 divided by 10 at Epochs 60 and 90, respectively.
We evaluate the robust models based on the two evaluation metrics, i.e., natural test accuracy on natural test data and robust test accuracy on adversarial data generated by AutoAttack (AA) Croce & Hein (0200) respectively. The adversarial test data are bounded by Lox perturbations with Étest 8/255.
For completeness, in Appendix A we demonstrate other attacks (such as PGD and CW attacks), and we also inject pair-flipping NLs Han et al. (2018), where labels y are flipped between adjacent classes that are prone to be mislabeled.
5

---

Published in Transactions on Machine Learning Research (06/2022)
Natural data
0.35
Natural tnst atenrveyy 196)
90
Robust test accuracy (9)
Divenstty uf zi gradicut
SAT with ND Rijectios
0.30
40
0:20-
23.
0.13
92
20
60
100
100
110
120
120
15
20
40
00
100
150
130
123
120
Epoch
Epoch:
Divensity of atg's onter loees
(a) CIFAR-10 datasct
(b) Supporting schematics
Figure 3: Figure 3(a) shows the learning curves of injecting various levels of NL in outer minimization. (The number in the legend represents NL injection rate 1,. The black dash line represents SAT.) Figure 3(b) shows data diversity comparisons between SAT (black dots) and SAT with NL injection (right dots). (Red dots are scattered at upper-right corner, which justifies that NL injection in outer minimization leads to high data diversity over the training epochs.) NL injection in outer minimization serves as a strong regularization that prevents robust overfitting.
Observation (i) NL injection in inner mazimization improves AT"s generalization but degrades AT's robustness. As shown in Figure [[a]] with the increasing of n, AT's generalization consistently increases (left panel), and AT's robustness consistently decreases (right panel).
Figure 2(b) illustrates the effects of NLs on generating adversarial data. We draw the generation traces of PGD adversarial data on two-dimensional ternary classification dataset. We randomly choose start point (white dot) whose correct label is "0" and plot the generation traces of adversarial data using correct label "0" (green trace), wrong label "1" (blue trace), and wrong label "2" (yellow trace), respectively. We find that NLs (wrong label "1" or "2") mislead the generation traces that should have pointed at the decision boundary. Instead, generation traces with NLs fall in the internal areas of class "0" Therefore, the generated
NL-adversarial data are similar to the natural data of class "0". To further justify the above phenomenon, Figure 10 (in Appendix A uses the CIFAR-10 dataset to corroborate that label-flipped adversarial data are similar to natural data.
Therefore, the reasons Observation (i) are as follows. When PGD (see Eq. 3) generates adversarial data, NLs mislead PGD to find wrong directions to the decision boundary. Then, the label-Olpped adversarial data becomes no longer adversarial the current model, and they serve more like their natural counterparts. NL injection in inner maximization at each minibatch equals randomly replacing a part of adversarial data with natural data. Consequently, AT learning more natural data and less adversarial data leads to better generalization and worse robustness.
3.2
NL Injection in Outer Minimization
Experiment details Figure 3(a) shows the results of series of experiments with various NL levels ny injected in outer minimization. Adversarial data are generated according to the intact labels. At every training minibatch, we randomly choose an portion of adversarial data to flip their labels for the learning. The other settings (e.g., the learning rate schedule, optimizer, network structures, dataset, erain and ftsst: K, a) are kept the same as Section 3.1 Note that we do not show 7/ > 0.3 because injecting too much noise (especially at the early training stage) significantly destabilizes and even kills the training.
In Figure 3(a) we show the learning curves of injecting symmetric-fliping NLs Van Rooyen et al. (0155) For completeness, in Appendix B we show other attacks (such as the PCD and CW allacks) and also show mair-flippin NL Han et al. (2018) injection. In Appendix B we find an intriguing phenomenon that pair-flipping NL injection obfuscates gradients of the CW attack. Specifically, pair-fippping NL injection significantly boosts the model's robustness against the CW attack.

---

Published in Transactions on Machine Learning Research (06/2022)
Observation (i)) NL injection in outer minimization alleviates robust overfitting. As shown in Figure 3(a) we find that at several last epochs, NL. injection leads to consistently better robustness than SAT (black dashed line). Besides, the robustness at the final epoch improves with the increase of noise rate 1). This shows that NL injection in outer minimization can significantly alleviate the undosirable issue of AT's robust overfitting Rice el. al. (2020).
We interpret the Observation (ti) from data augmentation perspective. NL injection in outer minimization forces AT to learn the adversarial data ā with the flipped label & occasionally. Considering an adversarial data point (ă. and its flipped label y. the objective loss function is (() z)). Taking the MSE loss an example, we can rewrite the loss {() = Ilf())) bll as [1(f(z) - 5 + y) - vll. in which we view f(i) : f(ž) - ÿ + y as noisy output. Through solving arg g ll.(()) [(Taug)]l, we basically project noisy output onto input space and augment original adversarial data i with Zaug. Therefore, NL injection could increase the data diversity: at each training epoch, the model comes across different versions of data.
We manifest the above perspective through conducting a proof-of-concept experiment on the CIFAR-10 dataset. We compare SAT and SAT with NL injection. We randomly select 30 training data and collect the statistics of each data point at each epoch (120 epochs in total). Each data point 2; corresponds to a 120-dimensional vector. zz's data diversity is reflected by the variance of its 120-dimensional vector.
Figure 3(b] illustrates the two statistics of each data point: 1) xy's outer loss (horizontal axis) the loss of the generated adversarial data, i.e., ((f(ā,) ū.) and 2) 2;'s gradients (vertical axis) the fz-norm of the weight gradient on Bt), i.c., (V))(f())), 80ll.. For notational simplicity, over the training process, is is always equal to y/ for SAT but not always equal to y, for SAT with NL. injection (because y4 is sometimes flipped to ÿ)-
Figure 3(b) shows that red rounds are scattered at the upper-right corner, and black squares are clustered at the lower-left corner, which justifies that SAT with NL injection has higher data diversity than SAT. Figure 3(b] echocs the empirical observations in Figure 3(a) which implics that the high data diversity may impede the the undesirable issue of robust overfitting.
NL Injection in both Inner Maximization and Outer Minimization
Observation (iii) NL injection in data has similar effect as Observation (ii). In cach training minibatch, we randomily choose a portion of data to flip their labels. Then, we conduct SAT on them. We empirically find the performance is very similar to Observation (ii), with even a slightly better robustness. To avoid repctition and save space, we leave those results in Appendix C
Observation (iv) There exists another case where NL injection in inner maximization mismatches that in outer minimization. Specifically, at every minibatch, we choose a portion of data to flip labels in inner maximization and another portion of data to flip labels in outer minimization. In this setting, we observe slight degradation of robustness because label-flippdd adversarial data are no longer serve the regularization purpose and are no longer adversarial as well to enhance robustness. The results can be found in Appendix D
4
Method
The four observations in Section 3 give us some insights on designing our own methods: Obs (i) and Obs. (iv) actually harms AT's robustness, but robustness is usually AT's main purpose. Obs. (ii) and Obs. (iii) have the similar effects that effectively mitigate robust overfitting. Inspired by the above Obs. (ii) Obs. (iii), we propose our Algorithm [ (i.e., NoiL.I) that dynamically increases NT. injection rate once the robust overfitting occurs. The simple strategy such SAT (Madry el al., 2018), TRADES (Zhang et al.] 2019b), TRADES-AWP (Wu el al.] (020). As a result, NoiLIn can fix robust overfitting issues of SAT and TRADES and further enhance generalization of TRADES-AWE? (see experiments parts in Section 5.2).
be incorporated into various effective AT methods
2TRADES- AWP does not have robust overfitting issues because the AWP method has already fixed Besides, AWP has further enhanced TRADES's robustness, thus TRADES AWP becomes the state-of-the-art, AT method.
7

---

Published in Transactions on Machine Learning Research (06/2022)
Algorithm NoiLIn: automatically increasing Noisy Labels Injection rate in AT methods
Input: network fe. training set Strain, validation set Svalid, total epochs E, initial noise Mmmn maximal noise rate Mmax sliding window size boosting rate y
Output: adversarially robust network fe
") Mmmin
for Epoch e= l, do
Randomly flip 1 portion of labels training dataset Strain to get Š Update So using S by an AT method (such as SAT, TRADES, TRADES-AWP) Obtain robust validation accuracy Ar using Svalid
if -i=e-,A Ai "5=e-r -i then
7 = m=mnn( (1 + 7), (maxx) // Boost NL injection rate nj if robust overfitting occurs.
end if
end for
CIRAR
CEPAR
SVIN
bobist tes Meccano (i)
Bolox (E)
Dokaar Ires aeremiy (90)
Nateal acrine
Natrrll and AYTHNEE
Seae
Fpoch
Fyuch
Fpoch
CIFAR-10
CTPAS 100
SVHN
corrsood (5))
Notmn setrrvee
lood anreary
Rehive aed ammmry
Notra soe
Figure 4: Evaluations on ResNet-18. We compare generalization (red lines) and robustness (blue lines, evaluated by the strongest AA attack) between our NoiLIn (solid lines) and two AT methods such as SAT and TRADES (dash lines) on three datasets CIFAR-10, CIFAR-100, and SVHN. The yellow histogram below each figure reflects the change of NT. injection rate over the training process. The shaded color reflects the standard deviations over the five repeated trails. To maintain figures' neatness, we put the training accuracy curves in Appendix D (Figure 17
As stated in Algorithm at the beginning of each epoch, randomly flip n portion of labels of the training dataset Strain to get a noisy dataset 8; then, we execute an existing AT method (c.g., SAT and TRADES) on the noisy dataset S. We monitor the training process using a clean validation set. We increase noise injection rate by a small ratio once the robust overfitting occurs.
5
Experiment
5.1 NoiLin can Alleviate Robust Overfitting
Robust overfitting Rice al. 2020 is an undesirable phenomenon in some AT methods particularly such SAT and TRADES, where after the first learning rate decay, the model's robustness stops increasing but begins to decrease.
In Figure 4 we incorporate our NoiLIn using symmetric-lipping NLs into two typical AT methods, i.e., SAT and TRADES. We adversarially train ResNet-18 on three datasets, ie., CIFAR-10, CIFAR-101 100 and SVHN. We took 1000 data from training set for validation and took the remaining data for the training. The robust validation accuracy (Ae in Algorithm [ was evaluated on PGD-10 attack for the consideration of speed. For the hyperparameter setting of Ao-NoiL.n, the noise rate 7 initialized mmin 0.05 with
8

---

Published in Transactions on Machine Learning Research (06/2022)
Table 1: Evaluations on Wide ResNet. We reported the test accuracy of the best checkpoint and that of the last chockpoint well as the gap between
/last:tstd. (gap)".
Defeuse
Natural
~100
AA
SAT (Madry of al. 2018
87.00 17.....5///8.1.....0 (+0.13)
(1.723311/40.76pp00
(-6.72)
51.06 m/
947
(-577)
+AT+TE Dong
2022
85.84 886 25 (+0.41)
55.91
(-2.68)
52.21 to
(-1889)
NAT-NoiLin
85.9210 / (+0.51)
$5.99_0.352/
000.0..///33.7999 19.577
(-2.20)
52.50.0.27M /0.0/500...000 10.55s
(- (-1.88
TRADES Zhang et al.
2019b
81.92 85.16=n..02 (+0.25)
63.8920054 19. 101
(-3.80)
52. 62.54:0.313/ 47 45
(-4.99)
TRADES +T Dong ot al 2022
25, a0. 78.78 (+0.53)
63.30
42:0.002
(- (-8888
52. 50.60=a... 4m 450 00.000+ 341
22477)
TRADES-NE No
81.39=0.142/85.88 0009:0..20/888000.00o (+1.50)
(22688)
53.14=0.520 50.15.0....00.
-2888)
TRADFS-AWP Wu 2020
/84.99...0/ (+0.48)
68 68.88
/67.00 191 (- 1.28)
65.88 /54
(.997)
BRADES-AWP-NOT
18. (+-0.44)
59.98=0.319 /
(-0.44)
56.12=o. 130/ ///555.88 30.
(-0233)
7max = 0.6, r = 10 and y= = 0.1. For the hyperparameter setting of MAAAEES-NoLLIM we set 7min = 0.05, 7max 0.4, 7= == 10 and 7 = 0.05. We trained ResNet-18 using SGD with 0.9 momentum for 120 epochs with the initial learning rate of 0.1 (on CIFAR datasets and 0.01 on SVHN) and divided by 10 at Epochs 60 and 90, respectively. We compare the generalization evaluated on natural test data (red lines) and robustness evaluated on AA adversarial data (Croce & Hein 2020) (blue lines) between typical AT methods (such as SAT and TRADES; dash lines) and NoiLIn counterparts (such as SAT-NoiLIn and TRADES-NoiLLN solod lines). Besides, at the bottom of each panel, we also demonstrate the dynamic schedule of noise rate v using the yellow histogram.
Figure 4 empirically validates the efficacy of NoiLIn to alleviate robust overfitting. When the lcarning rate decays, the robust test accuracy of NoiLIn keeps steady and even rises slightly instead of immediately dropping. Besides, observe that when the robust overfitting occurs, the noise rate (yellow pillars) gradually rises to deliver stronger regularization to combat robust overfitting.
5.2 Performance Evaluations on Wide ResNet
To manifest the power of NoiLIn, adversarially train Wide ResNet Zagoruyko & Komodakis 2016 on CIFAR-10 by incorporating our NoiLIn strategy using symmetric-llipping NLs to three common and effective AT methods, i.e., SAT, TRADES and TRADES-AWP (adversarial weight perturbation). In Table we used WRN-32-10 for AT-NoiLIn that keeps same as Madry et al. (2018) and used WRN-34-10 10 for TRADES-NoiLIn and TRADES-AWP-NOLLI that keeps same as Zhhang et al. (2019b); Wu et al. (0200)
The hyperparameters of AWP exactly follows Wu et al. (002)). Other training settings optimizer, the hyperparameters for scheduling noise rate) keeps the same as Section 5.1 In Table we also
(e.g., learning rate,
compare independent and contemporary work, AA. TE (Lemporal ensembing) Dong et al. 2022).
On the current SOTA method TRADES-ANE our NoiLIn could further enhance its natural test accuracy by surprisingly 2% while maintaining (even slightly improving) its robustness. Besides, on natural test accuracy, NoiLIn seems to have a bigger negative impact on SAT than TRADES. In terms of learning objectives, SAT has one cross-entropy loss on adversarial data; TRADES has two loss, i.e., cross-entropy loss on natural data + 6xKL divergence loss between natural data and adversarial data, in which KL loss does not involve labels and only cross-entropy loss involves labels. Therefore, NoiLIn has less negative impact on TRADES than AT on natural accuracy.
5.3 Ablation Study
In this section, we conduct ablation studies. We show NoiLIn is more effective in larger perturbation bounds G, is less sensitive to weight decay. Besides, we compare NoiLIn with label smoothing (LS), and conduct extensive ablation studies such as NoiLIn with various learning rate schedulers and various noisy-label training set.
NoiLIn under larger e. NoiLIn is more elfective for larger e. Robust overfitting becomes even worse when e becomes larger. We compare SAT and SAT-NoiLIn under larger e using ResNet-18 on CIFAR-10 10 datasot. The training scttings of SAT and SAT-NoiLIn keep same as Soction 5. except c. We choose Grain from (8/255,10/ 12//222225,,14////// 2255.. The step size a for PGD was e/4. Robust test accuracy was evaluated on adversarial data bounded by Too perturbations with éest étrain-
9

---

Published in Transactions on Machine Learning Research (06/2022)
Natural tee dota
AA
Natural test Cata
A4
Natural test aneumnyy (X)
(96)
(hreeste)) (Poa)
Natura accuracy
Boleatt bost aucounicy
6050
12/20
10/20
9/20
20/50
10.00
1030
12200
Epodl
Ipoch
(a) Robust overfitting is more severe under larger = 16/255.
(b) NoiLIn enhances robustness further under larger C.
Figure 5: Figure ((a) shows the learning curves of SAT-NoiLII under Ctrain 16/255. (Robust test accuracy was evaluated on AA attacks with étest = 16 255). Figure 5(b) shows comparisons between SAT and SAT- Nn under larger étrain- (We reported the test accuracy evaluated at the best checkpoint according to the best robust accuracy. All figures keep étrain = Esest.) We abbreviate SAT-NoiL using symmetric-fipping NLs as "SAT-NoiLIn (Symmetric)" and SAT-NoiLIn using pair-flippine NLs as "SAT-NoiLIn (Pair)".
Figure 5(a) illustrates the learning curves of SAT and SAT-NoiLIIn under étrain = frrst = 16/255. Figure 5(b) reports performance of the best checkpoints under different e. The fest-checkpoints are chosen according the best robust accuracy. Under larger c, SAT has more severe issue of robust overfitting and the robustness becomes lower. It is because AT fit. data's neighborhoods, and the neighborhood becomes exponentially larger w.r.t. data's dimension even if become slightly larger. This will cause larger overlap issues between classes, which requires larger label randomness for the learning. Fortunately, under larger e, NoiLIn is more effective relieving robust overfitting and even further improves adversarial robustness.
NoiLIn's sensitivity to weight decay. Pang et al. 2021 empirically found different values of weight decay can largely affect AT's adversarial robustness. In the ablation study, we explore the sensitivity of No to weight decay. Figure t shows NoiLIn can reduce AT methods sensitivity on weight decay.
Natural test dara
Robest tost accuroec
SXT. Sath SSeretto
SeT Nattin (VV)
Weight: decay (x10-l)
Weight decay (x10-)
Figure 6: Test accuracy of robust models trained under different values of weight decay.
In Figure 6 we conduct Algorithm | under different values of weight decay and compared the SAT-NoiLIn with SAT. We use R8sNot-18 and evaluate robustness at the best checkpoint. The settings keep the same as Pang et al. (2021). From the right panel of Figure g both red and blue lines are higher and flatter than black lines across different values of weight decay. It signifies that noise injection can enhance adversarial robustness of SAT and meanwhile make SAT less sensitive to varying values of weight decay.
NoiLIn's relation with label smoothing (LSS.. When taking the expectation w.r.t. all training epochs, NL injection is similar employing the tochnique of label smoothing (Szegeyy et al. (2016); Muller et al. (2019) into AT (AT-LS) (Cheng al., 2020 Pang et al.] 2021). Despite the mathematical similarity, we empirically found that NoiLIn can relieve robust overfitting while LS cannot.
In Table 2 we reported robustness evaluations of AT with LS and AT-NoiLIn using ResNet-18 on CIFAR-10 dataset and set the training seltings same as Section 5.1 AT-LS uses the p level of smoothed label ÿ in outer minimization while using the one-hot label y in inner maximization. The p level smoothed label y is a C-dimensional vector with the y-th dimension being 1 - p and all others being dh. We implement L.S with p = 0.1 only in outer minimization ("AT-LS-outer") and LS with p = 0.1 in both inner maximization and outer minimization (*AT-LS-both"). Further, we even conduct LS in both inner maximization and
10

---

Published in Transactions on Machine Learning Research (06/2022)
AT-L.-adaptivee and SAT-NoiLIn (Symmetric). We report the test accuracy of the best chockpoint and that of the last checkpoint as well the gap between them (hest/last (gap)".
Table 2: Comparisons between SAT, AT-LS-outer (p 0.1), AT-LS-both (p (p= 0.1).
Defense
Natural
C&W
AA
SAT Madry ct al. (2018
81.97/84.76
(+2.79)
49.53/45.12
(-4.71)
48.09/43.30
-4.79)
AT-LS-outer Pang et al. 20021
82.76/85.15
(+2.39)
50.06/45.40
(-4.66)
48.60 48.60/43.44 ( (-5.16)
AT-LS-both
82.28/84.34 (+2.06)
49.79/46.17
(-3.62)
48.44/44.59
(-3.85)
ATa-S-adap
22.68/85.20
(+2.52)
50.03/46.72 (-3.31)
48.68/45.22
(-3.45)
SAT-NoiLIn
82.97/83.80 (+0.89)
51.21/50.36 (-0.85)
48.74/47.88 (-0.81)
outer minimization with dynamic LS level p ("AT-LS-dynamicc). The schedule of p is set exactly same as that of 17 in AT-NoiLIn. We report the test accuracy at the best/last epoch and the gap between them in Table 2 Extensive comparisons between AT-LS-outer under other different P and AT-NoiLIn are reported in Appendix D.1.
We found none of LS strategies can effectively relieve the robust overfitting. Table 2 shows that compared with variants of AT-LS, AT-NoiLIn can largely relieve robust overfitting, while LS fails to make any effect. The reasons may be that AT-LS uses fixed smoothed labels, which is similar to AT using fixed one-hot labels, thereby having lower data diversity during the training process; by contrast, NoiLIn randomly flips labels at each epoch, leading to higher data diversity and thus effectively preventing robust overfitting (see Section 3.2).
In addition, compared with LS, No saves (C-1) log operations for each data. For each adversarial train- ing data z whose prediction is p(5), the calculation of the cross-entropy loss of AT-LS is ΣS Vj (, while that of SAT-NoiLIn is - (g)).
NoiLIn under different learning rate (LR) schedulers. Rice et al. 2020 showed robust overfitting ubiquitously occurs in AT under different LR schedulers. We conducted SAT-NoiLIn under different LR schedulers using ResNet-18 on CIFAR-10 dataset and kept all the training settings, such as the optimizer and hyperparaneters of noise rate, exactly same as Section 5.1 except the LR scheduler. We report the
Table 3: Different learning rate schedule
Defense
LR scheduler
Natural
C& 100
AA
SAT
Madry et (2018)
piecewise
81.97/84.76 (+2.79)
49.53/45.12
(471))
48.09/43.30
(44799
piecewise
82.97/83.86 (+0.89)
51.21/50.36
(-0.85)
18.74/17.93
(-0.81)
SAT-NoiLIn
miltiple docay
81.74/83.35 (+1.61)
49.42/48.59
(-0.83)
47.50/46.22
(-1.28)
(Symmetric)
cosine
84.86/85.00
(+0.16)
00.83/50.59
(-0.24)
48.24/18.08
(-0.16)
cyclic
83.48/83.48
(+0.00)
61.41/51.41 (+0.00)
49.05 49.05/49.05 (+0.00)
test accuracy of the best checkpoint and that of the last checkpoint as well as the gap between them in Table 3 and show the learning rate schduler in Figure [8] We observe the gap between test accuracy of the best checkpoint and that of the last checkpoint largely narrows with automatic NL injection, which clearly indicates No can mitigate robust overfitting under all different LR schedulers. Further, we demonstrate the learning curves of SAT-NoiLIn under various LR schedulers in Figure 18 (Appendix D.2). which again validates NL injection can relieve overfitting.
NoiLIn under noisy-label training set. Readers may foel curious about what happens when NoiLIn meets noisy-label set because training set is often noisy-labeled Xiao et al. (2015). We compare An-NoiLIn and SAT noisy CIFAR-10 dataset using ResNet--1. We use symmetric-fippingg flipping NL to construct noisy CIFAR-10. The fraction of NI. is chosen from f0%, 5%, 10%, 20%, 30%, 50%). Note that when the fraction of NL is 0%, the training set is exactly the clean CIFAR-10 dataset. The training settings of SAT and AT- No kept same as Section 5.1 In Figure 7 we show the performance of SAT (black lines) and AT-NoiLIn (red lines) on noisy-labed training set that contains different fractions of NL.
We find red lines are always above black lines, which indicates that even with noisy-label training set, our Nn is still an effective method to improve the AT methods.
11

---

Published in Transactions on Machine Learning Research (06/2022)
Natural test data (Best docckoiitt)
AA (Best checkpoinn
Natural test data (Last checipoont)
AA (Last checkpoiitt
Robnet tod secutcey (%)
Notural tox necureey
Natural test aecurayy
Rohnst
NL (5)
Fractson (%)
Proction (8)
Fouctin (7))
Figure 7: Comparisons between SAT and SAT-NoiLIn on noisy-label training set containing different frac- tions of NL.
NoiLIn with extra unlabeled training data Carmon et al. (2019) and Gowal ct al. (2021a) showed that additional unlabeled data can enhance adversarial robust Therefore, we conduct Nn with additional training unlabeled data provided by Carmon et al. (2019). We train WRN-28- 10 with the cosine decay learning rate schedule. We set the ratio of labeled-to-umlabellled data per batch to 3:7 following Gowal et al. 2021a). The total training epoch is 200 and the batch size is 256. Other hyper-parameters for injecting In Table we compare the natural test, accuracy and robust test accuracy (evaluated by AA attack) between NoiLIn and Robust Self-training (Carmon et al.) 2019p. Note that the results of Robust Solf-training with extra data Carmon et al.. (2019) are copied from AutoAttack leaderboard (Croce et al.) 2020 Surprisingly, Table shows that Nn can enhance both natural and robust test accuracy, which validates the superiority of the NoiLII method.
label noise are the same as Section 5
Table 4: Evaluations of NoiLIn with additional unlabeled data on WRN-28-10.
Defense
Natural AA
Robust Self-teraining by Carmon et al. (0019)
89.69
59.53
NoiLIn with additional unlabeled data
91.11
59.91
6 Conclusion
In this paper, we claim that NL injection can benefit AT. We have explored the positive effects of NL injection in inner maximization and in outer minimization, respectively. Our observations have motivated us to propose a simple but effective strategy, namely "NoiLIn", to combat the issue of robust overfitting and even enhance robustness further (e.g., especially under larger e).
There are two limitations of our current work. 1) still difficult to explicitly, not intuitively, explain why NL injection improves robustness. 2) We do not know
We empirically verified the benefits of NoiLIn, but is
the optimal rate of NL injection at each training epoch, except leveraging validation set. In the future, we will attempt to address these limitations.
Acknowledgment
J7. was supported by JST Strategic Basic Rescarch Programs, ACT-X, Grant No. JPMJAX21 and JSPS Grants-in-Aid for Scientific Research (KAKENHI), Early-Career Scientists, Grant No. 22K17955, Japan.
BH was supported by the RGC Early Career Scheme No. 22200720, NSFC Young Scientists Fund No. 62006202, Guangdong Basic and Applied Basic Research Foundation No. 2022A1515011652, RIKEN Col- laborative Research Fund and HKBU CSD Departmental Incentive Grant. LTL was partially supported by the Australian Research Council Project DP180103424, DE190101473, IC190100031, DP220102121. LC was supported by the National Key R&D Program of China No. 2021YFF0900000, NSFC No.91846205, SDNSFC No. 20.222009,,2O00, Shandong Provincial Key Rescarch and Development Program (Major Scien- tific and Technological Innovation Project) No. 20210X0000 MS was supported by JST AIP Acceler- ation Research Grant Number JP- MJCR20U3 and the Institute for AT and Beyond, UTokyo
12

---

Published in Transactions on Machine Learning Research (06/2022)
References
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NewrIPS, 2019.
Dana Angluin and Philip Laird. Learning from noisy examples. Machine Learning, 2(4):343 370, 1988.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Mislav Baluovicc and Martin Vechev. Adversarial training and provable defenses: Bridging the gap. ICLR, 2020.
Qi-Zhi Cai, Chang Liu, and Dawn Song. Curriculum adversarial training. In LJCAI, 2018.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy (SP), 2017a.
Nicholas Carlini and David A. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Securily, AlSec 2017b.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. In NeurIPS, 2019.
Kejiang Chen, Yuefeng Chen, Hang Zhou, Chuan Qin, Xiaofeng Mao, Weiming Zhang, and Nenghai Yu. Adversarial examples detection beyond image space. In JEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2021a.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning. In CVPR, 2020.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In ICLR, 2021b.
Minhao Cheng, Qi Lei, Pin-Yu Chen, Inderjit Dhillon, and Cho-Jui Hsieh. Cat Customized adversarial
training for improved robustness. arXiv:2002.0678) 2020.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usumier. Parseval networks: Improving robustness to adversarial examples. In ICML, 2017.
Gilad Cohen, Guillermo Sapiro, and Raja Giryes. Detecting adversarial samples using influence functions and nearest neighbors. In CVPR, 2020.
Jeremy M. Cohen, Flan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing, In ICML, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint Xiv:20000. 0.0000, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icce, 2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chan Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. Tn ICIR, 2020.
13

---

Published in Transactions on Machine Learning Research (06/2022)
Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, and Jun Zhu. Exploring memoriza- tion in adversarial training. In TCLR, 2022.
Konstantin Donhauser, Alexandru Tifrea, Michael Aemni, Reinhard Heckel, and Fanny Yang, Interpolation can hurt robust generalization even when there no noise. In NeurIPS, 2021.
Xuefeng Du, Jingfeng Zhang, Bo Han, Tongliang Liu, Yu Rong, Gang Niu, Junzhou Huang, and Masashi Sugiyama. Learning diverse-structured networks for adversarial robustness. In ICML, 2021.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adver- sarial logit pairing. arXie preprint or 1807. 10272, 2018.
Farzan Farnia, Jesse M. Zhang, and David Tse. Generalizable adversarial training via spectral normalization. In ICLR, 2019.
Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. Maxup: Lightweight adversarial training with data augmentation improves neural network training. In CVPR, 2021.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmect Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. OrXiv:2010.05500 2021a.
Sven Gowal, Aivrestre-Aviie Rebuff,, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy Mann. Improving robustness using generated data. In NewrIPS, 2021b.
Minghao Guo, Yuzhe Yang, Rui Xu, Ziwei Liu, and Dahua Lin. When nas meets robustness: In search of robust architectures against adversarial attacks. In CVPR, 2020.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NewrIPS, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of classifier against adversarial manipulation. In NewrIPS, 2017.
Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, and Xingiun Ma. Ex- ploring architectural ingredients of adversarially robust deep neural networks. In NeurIPS, 2021.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry, Adversarial examples are not bugs, they are features. In NewrIPS, 2019.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-trainiee training by adversarial contrastive learning. In NewrIPS, 2020.
David Khachaturov, Ili Shumailov, Yiren Zhao, Nicolas Papernot, and Ross Anderson. Markpainting: Adversarial machine learning meets inpainting. In ICML, 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In ICLR, 2017.
Mathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In Symposium on Security and Privacy (SP), 2019.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of- distribution samples and adversarial attacks. In NewrIPS, 2018.
14

---

Published in Transactions on Machine Learning Research (06/2022)
Sachyung Lee, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adversarially robust generalization. In CVPR, 2020.
Xin Li and Fuxin Li. Adversarial examples delection in deep networks with convolutional filter statistics. In ICCV, 2017.
Yanxi Li, Zhaohui Yang, Yunhe Wang, and Chang Xu. Neural architecture dilation for adversarial robustness. In NewrIPS, 2021.
Jiayang Liu, Weiming Zhang, Yiwci Zhang, Dongdong Hou, Yujia Liu, Hongyue Zha, and Nenghai Yu. Delection based defense against adversarial examples from the steganalysis point of view. In CVPR, 2019.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi N. R. Wijewickrema, Grant Schoenebeck, Dawn Song, Michael F. Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In ICLR, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial pertur- bations. In ICLR, 2017.
Matthew Mirman, Timon Gehr, and Martin T. Vechev. Disferentiable abstract interpretation for provably robust neural networks. In ICML, 2018.
Seyed-Mohsen Moosavi-Dezffolli Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robustness via curvature regularization, and vice versa. In CVPR, 2019.
Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, and Dictrich Klakow. Logit pairing methods can fool gradient-based attacks. arXiv preprint XrXi:18810.1204 2018.
Rafacl Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural informalion processing systems, 32, 2019.
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyalo. Robustness to adversarial perturba- tions in learning from incomplete data. In NewrIPS, 2019.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In NewrIPS, 2013.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial examples. NewrIPS, 2018
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversily. In ICML, 2019.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. TCLR, 2021.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishman, Krishnamurthy Dvijotham, Albussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In NewrIPS, 2019.
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison W. Cottrell, and Geoffrey E. Hinton. Detecting and diagnosing adversarial images with class-conditional capsule reconstructions. In ICLR, 2020.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. In ICML, 2020.
Leslie Rice, Eric Wong, and J Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
15

---

Published in Transactions on Machine Learning Research (06/2022)
Andrew Slavin Ross and Finale Doshi-Velez, Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In AAAT, 2018.
Kevin Roth, Yannic Kilcher, and Thomas Hofmann. The odds odd: A statistical test for detecting adversarial examples. In ICML, 2019.
Amartya Sanyal, Puneet K. Dokania, Varun Kanade, and Philip Torr. How benign is benign overfitting ? Tn TCLR, 2021.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry, Adversarially robust generalization requires more data. In NeurIPS, 2018.
Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust neural networks. In NewrIPS, 2020.
Ali Shafahi, Amin Ghiasi, Furong Huang, and Tom Goldstein. Label smoothing and logit squeezing: a replacement for adversarial training? arXiv preprint 1rXiv:1910.1158. 2019a.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurIPS, 2019b.
Fatemeh Sheikholeslami, Ali Lotfi, and J. Zico Kolter. Provably robust classification of adversarial examples with detection. In ICLR, 2021.
Sahil Singla and Soheil Feizi. Second-order provable defenses against adversarial attacks. In JCML, 2020.
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detection. In UAI, 2018.
Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. Improving the generalization of adversarial training with domain adaptation. In ICLR, 2019.
Chuanbiao Song, Kun He, Jiadong Lin, Liwei Wang, and John E. Hopcroft. Robust local features for improving the generalization of adversarial training. In ICLR, 2020.
Philip Sperl, Ching Yu Kao, Peng Chen, Xiao Lei, and Konstantin Böttinger. DLA: layse-layer-anallsis for adversarial example detection. In JEEE European Symposium on Security and Privacy, EuroSEP, 2020.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, and Venkatesh Babu R. Guided adversarial attack for evaluating and enhancing adversarial defenses. In NewrIPS, 2020.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and generalization. In CVPR, 2019.
Christian Szegedy, Wojeiech Zaremba, Tlya Sutskever, Joan Bruna, Dumitru Erhan, Tan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Christian
Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pallern recognilion, pp. 2818 2826, 2016.
Jinyu Tian, Jiantao Zhou, Yuanman Li, and Jia Duan. Detecting adversarial examples from sensitivity inconsistency of awatial-rransfon domain. In AAAI, 2021.
Shixin Tian, Guolei Yang, and Ying Cai. Detecting adversarial examples through image transformation. In AAAI, 2018.
Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. Tn ICIR, 2019.
16

---

Published in Transactions on Machine Learning Research (06/2022)
Yusuke Tsuzuku, Issci Sato, and Masashi Sugiyama. Sipschitz-Marghi training: Scalable certification of perturbation invariance for deep neural networks. In NewrIPS, 2018.
Brendan Van Rooyen, Aditya Krishna Menon, and Robert C Williamson. Learning with symmetric label noise: The importance of being unhinged. In NewrIPS, 2015.
B. S. Vivek and R. Venkatesh Babu. Single-step adversarial training with dropout scheduling. In CVPR, 2020.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Tsui- Weng, Huan Zhang, Pin-Yo Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. In ICLR, 2018.
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In ICML, 2018.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? In NearIPS, 2021a.
Dongxian Wu, Shu-Tao Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. NewrIPS, 2020.
Yuhang Wu, Sunpreet S. Arora, Yanhong Wu, and Hao Yang. Beating attackers at their own games: Adversarial example detection using adversarial gradient directions. In AAAI, 2021b.
Kai Yuanqing Xiao, Vincent Tjeng, Nur Muhammad (Mahi) Shafiullah, and Aleksander Madry, Training for faster adversarial robustness verification via inducing relu stability. In ICLR, 2019.
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In CVPR, 2015.
Cihang Xie and Alan L. Yuille. Intriguing properties of adversarial training at scale. In ICLR, 2020.
Cihang Xie, Mingxing Tan, Boqing Gong, Alan Yuille, and Quoc V Le. Smooth adversarial training. arXiv preprint Xrx::2006. 14536, 2020.
Lingxi Xie, Jingdong Wang, Zhen Wei, Meng Wang, and Qi Tian. Disturblabel: Regularizing cnn on the loss layer. In CVPR, 2016.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In ICML, 2021.
Hanshu Yan, Jingfeng Zhang, Gang Niu, Jiashi Feng, Vincent Tan, and Masashi Sugiyama. Cifs: Improving adversarial robustness of cmns via whan importance-based feature selection. In ICML, 2021.
Ziang Yan, Yiwen Guo, and Changshui Zhang. Deep defense: Training dnns with improved adversarial
robustness. In NewrIPS, 2018.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: Diversifying vulnerabilities for enhanced robust gener- ation of ensembles. In NeurIPS, 2020a.
Puyudi Yang, Jianbo Chen, Cho-Jui Hsich, Jano-Ling Wang, and Michael I. Jordan. ML-LOO: detecting adversarial examples with feature attribution. Tn AAAI, 2020b.
17

---

Published in Transactions on Machine Learning Research (06/2022)
Yao- Yuo-Yunn Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R. Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy vs. robustness. In NewrIPS, 2020c.
Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhi-Ming Ma. Improved ood generalization via adversarial training and pre-training. In ICML, 2021.
Dong Yin, Kannan Ramchandran, and Peter L. Bartlett. Rademacher complexity for adversarially robust generalization. In JCML, 2019.
Xuwang Yin and Soheil Kolouri Gustavo K. Rohde. GAT: generative adversarial training for adversarial example detection and robust classification. In ICLR, 2020.
Sergey Zagoruyko and Nikos Komodakis, Wide residual networks. arXic:6605.07146 2016.
Bohang Zhang, Tianle Cai, Zhou Lu, Di He, and Liwei Wang. Towards certifying I-infinity robustness using neural networks with l-inf-dist neurons. ICML, 2021a.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In NewrIPS, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In JCML, 2019b.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards stable and efficient. training of verifiably robust neural networks. In ICLR, 2020a.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020b.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry- aware reweihte-e adversarial training. In ICLR, 2021b.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. Tn ICML, 2019.
Jianing Zhu, Jingfeng Zhang, Bo Han, Tongliang Liu, Gang Niu, Hongxia Yang, Mohan Kankan- halli, and Masashi Sugiyama. Understanding the interaction of adversarial training with noisy labels. arxiv:21 03482, 2021.
18

---

Published in Transactions on Machine Learning Research (06/2022)
Natural data
PGD-10
CRW
AA
Rolust arcarrry 1%6
iccarary (96)
mocaaay (96)
Nahurd test accurrayy
0:3
(3)
211
61
100
00
100
120
20
20
401
241
100
100
122
16
20
40
601 Epoch
50
11X
50
100
120
10
20
40
50
40
3000
50
100
120
Epech
Epoch
Epoch
Figure 8: The learning curves of injecting various levels of symmetric-flipping NL in inner maximization. The number in the legend represents noise rate 77.
Natural test data
PGD-10
CRW
AA
201
Rohust arraaay (96)
mecmmay (96)
Rolaut accumnry (96)
in.
Natural test nocuracy
0.3
18
30)
21
60
60)
100
100
120
20
40
E01 Epoch
100
00
100
123
16
20
40
60 Epoch
10C
50
100
121
15
20
40
60 Epoch
80
100
50
160
120
Epoch
Figure 9: The learning curves of injecting various levels of pair-flipping NL in inner maximization. The number in the legend represents noise rate 77.
A NL Injection in Inner Maximization
Figure is symmetric-fipping NL injection in inner maximization; Figure 9 is pair-flipping NL injection in inner maximization. We evaluated the robust models based on the four evaluation metrics, i.e., natural test accuracy on natural test data, robust test accuracy on adversarial data generated by PGD-40 Carmon et al. (2019), C&W -30 (Lx version C&W Carlini & Wagner (2017)) optimized by PGD-30), and AutoAt- fack Croce Hein 2020) (AA), respectively. The adversarial Lest data are bounded by Loo perturbations with éteat 8/255. Figures and g consistently show NL injection in inner maximization improves AT's generalization and degrades AT's robustness. To further justify the above phenomenon in Figure 2(b) in Figure 10 we used real-world dataset CIFAR-10 to reveal the similarity among "PGD with NL" (PGD adversarial data with wrong labels), "PGD with CT." (PGD adversarial data with correct labols), and "Nat- ural" (natural data). We conducted a standard adversarial training for 120 epochs and saved the model's checkpoint at every epoch. At every chockpoint, we randomly selected 1000 training data and generated "PGD with NL." and <PGD with CT," of these data. We used model's Kullback-Lad (2019b) as the similarity metric (the smaller value means larger similarity). In the left panel of Figure 10. we compared similarity among "PGD with NL", "PGD with CL", and "Natural". Besides, in the right panel of Figure 10 we used the correct labels to calculate the cross-entropy loss of "PGD with NL "PGD with CL", and "Natural", respectively.
(KT.) loss Zhang al.
The left panel of Figure 10 shows that the value of KL(PGD with NL, Natural) (red solid line) is apparently lower than that of KL((GD with CL, Natural) (blue solid line). Besides, the right panel of Figure I0 shows that the cross-entropy loss of "PGD with NL" is almost identical that of "Natural". Compared with "PGD with CL" "PGD with NL is more similar to "Natural" (natural data). Therefore, babel-flipped adversarial data similar to natural data.
19

---

Published in Transactions on Machine Learning Research (06/2022)
Simclarity
Croes-cutropy loes
KLPGD wal NI, Nateed! Ni,
M
Nanual
üivergeezee
Loss valtue
04:35
0.12
0.15
20
00
200
133
0
40
110)
120
Epoch
Epoch
Figure 10: The similarity among PGD with CL (adversarial data generated with correct label), PGD with symmetric-flipinn NT. (adversarial data generated with noisy label), and Natural data on CIFAR-10 dataset.
Natural data
PG
C&W
AA
Natural accureyy (%6)
Rohust les accorrics (6
Robusl acenrncyy (%))
Robust leat acturacy
20
60
60
0L1G
x
330
10
100
116
170
120
30
10
60)
1191
103
18
125
15
20
40
60
50
1801
106
110
130
120
10
30
40
1141
100
120
150)
Epoch
Epoch
Epoch
Eporc
Figure 11: The learning curves of injecting various levels of symmetric- NL in outer minimization. The number the legend represents noise rate 17
B NL Injection in Outer Minimization
Figure symmetric-flipinng NL injection in outer minimization; Figure 12 is pair-flipping NL injection in outer minimization. The evaluation metries keep the same as Appendix
From the third panel of Figure 12 observe that NL in outer minimization can significantly improve the model's robustness against the C&& attack, especially injecting pair- mair-fipping label noise.
Natural data.
PCD-10
C&W
AA
Nabaral test occararry (96)
Rolust comrray (%)
nomarayy (%))
uranyy (%)) 2
16.
20
m
KLIG
gt
8
331
111
(1
111
1000
120
20
20
101
141
50
100
100
15
20
40
601 Epoch
501
110
130
15
20
400
000 Epoch
80
1001
150
Epech
Еpoch
Figure 12: The learning curves of injecting various levels of pair-flipping NL in outer minimization. The number in the legend represents noise rate
20

---

Published in Transactions on Machine Learning Research (06/2022)
SAT
AT with semmtriio-lipinn NL
with puistlipping
Logit margin
20
40
50 100 120
Epoch
We compare the median logit margin of natural test data and its standard deviation over the training process of SAT (black dashed line), that of AT with symmetric-fipping NL in outer minimization (rod line), and that of AT with pair-flpping NL in outer minimization (blue line). We find NL in outer minimization reduces logit margins, thus obfuscating gradients of the C& W allack Carlini & Wagner 2017a
Figure 13:
Natural data
PGD-10
CL-W
AA
Robood areurracy (36)
Roivnd uennary (%6)
Robust accurrry (%)
E
Natural arcuracy
15
RH
60
E
3
B
8
g
-30
211
181
90
100
99
100
120
30
20
400
201
501
102
101
110
133
15-
20
40
60
601
70
50
10X
301
1101
130
10
20
40
50
80
300
130
Epoch
Epoch
Epoch
Epoch
Figure 14: The learning curves of injecting fymmetric-flipping NL into both inner maximization and outer minimization. The number in the legend represents noise rate 17-
To figure out the reason for this interesting phenomenon, at every training epoch, we illustrated the median logit margin (x) - maxj få (x)) over all natural test data and its standard deviation in Figure 13 We compare SAT (black dashed line) with AT with NL. (the red line for injecting symmetric noise and the blue line for injecting pair flipping noise). As shown in Figure 13 injecting NL in outer minimization leads to a small logit margin, especially injecting pair-fippping NL (the blue line).
Note that the implementation of the C&W attack follows
arg max lx]
[max(max fd (z) (x) k. 0)), i
(4)
where is positive constant value that aids the optimization. When the logit margin is small and close to 0, the starting point of optimization of the inner loss of the C&W attack (i.e., (max ( (ø)-x, 0)) in Eq. equation 4 is also small and close to O. This will incur the gradient vanishing problem and hurdle the optimization for finding the C-adverssaiial data. Therefore, NL in outer minimization obfuscates gradients of C& W allacks by reducing logit margins.
C Injecting NL in Both Inner Maximization and Outer Minimization
Observation (iii) We report the learning curves of ResNet-18 trained by AT with NL injection in both inner maximization and outer minimization on CIFAR-10 dataset in Figure 14 (symmetric-fiipping NL) and Figure 15 flipiig NL). The noise rate 7) chosen from {0.1, 0.3). The detailed training settings
(e.g., the optimizer and learning schedule) the same as Section 3.1
21

---

Published in Transactions on Machine Learning Research (06/2022)
Natural data
PGD-10
Rolust arcarary 1%6
accararr (96)
mccraay (96)
Nahurd iecurray
15
m
01
8
18
211
611
181 Epoch
150
100
120
20
20
401
141 Epoch
100
100
122
16
20
40
60 Epoch
50
10K
120
10
20
40
50
60
3003
120
Epoch
The learning curves of injecting pair-flipping NL into both inner maximization and outer mini- mization. The number in the legend represents noise rate n).
Figure 15:
Natural test dsta
PGD-40
C&W
AA
Nahurl test nosuruuc (2)
Rolned nccuracy (96)
46.
Rohned nomtrayy
Bohnsd nomunny
16
40
1
$4.5
$1.0
20
30
3
170
110
199
Epoch
Epoch
Epoch
Epoch
Figure 16: The learning curves of ResNet-18 trained by AT with different NL injection in inner maximization and outer minimization. The number in the legend represents noise rate 17.
Figures 14 and 15 show that, with the increasing of 17 (the color gradually becomes deeper), the natural test accuracy (the leftmost pancls) on natural test data decreases; while the robust test accuracy (the right three panels) on adversarial data evaluated at the last checkpoint and at the best checkpoint increases simultaneously. For example, when equals to 0.1 (blue lines with different shades) in Figure 14 the natural test accuracy at Epoch 120 comparable with SAT (black dashed line), and the robust test accuracy at Epoch 120 is obviously above the black dashed line. Such observation manifests that injecting NL in both inner maximization and outer minimization alleviate robust overfitting and even improve the best-checkpoint robustness.
Observation (iv) Figure 16 demonstrates the learning curves of ResNet-18 trained by AT with different NL injection in inner maximization and outer minimization. In details, at every minibatch, we choose a portion of data to flip labels in inner maximization and another portion of data to flip labels in outer minimization. In Figure 16. NL is generated by symmetric flipping. The noise rate r7 keeps same in both inner maximization and outer minimization, and 7 E {0.1, 0.2, 0.3). The detailed training settings (e.g., the optimizer and learning schedule) are the same as Section 3.1 Figure 16 shows that adversarial robustness is slightly degraded, which indicates that label-flipped adversarial data no longer serve the regularization purpose and are no longer adversarial as well to enhance robustness.
D Extensive Experimental Details
Computing resources. All experiments were conducted on a machine with Intel Xcon Gold 5218 CPU, 250GB RAM and six NVIDIA Tesla V100 SXM2 GPU, and three machines with Intel Xeon Silver 4214 CPU, 128GB RAM and four GeForce RTX 2080 Ti GPU.
22

---

Published in Transactions on Machine Learning Research (06/2022)
CIFAR-
CIFAR.
SVHV
100)
100
100
Dia-Inaao
501 See Bert
Robust acceL10CC (16)
Natural accuraey (16)
1541 (Naa Tuet)
Robaust acc (96)
Natural accuracy (9%)
552T Dhat Tert
Robudi at EUELLO (96)
5001-Sell (bat/
SCE-Sichii (Sab-2ott
Natural accuracy
20
3
Tod-Tron))
5AB (Ral-Taaa))
tholl 5xT (itbk Tidi
5A2
SAT-- (iddToo
SCT-Soil (Bab-aa))
101
101
Niiio
60
100
120
01
150
130
100
120
Epoch
Epoch
Epoch SVHN
CIFAR 10
CIFAR 100
TRLLES
TRADES
60
TRADES
Natural arcuracy (76)
Robust aueutracy (16)
Natural accuracy (96)
THAADES
Roloted al curaayy (%)
Natural accurrey (96)
60
195AUR25-KLL (Nat-
Robust arruary
50
TRADES (hu--Tea))
TRADES Buc-Thea
TRAtes (hulThua)
SRALES Meilla IodkInak
TRADe Satlis Hule Iakal
(Pak-Tok)
31
20-
TRabes
1-
Nois
Neie
Noko
00
20
100
120
100 120
20
40
06
50
100
120
Epoch
Epoch
Epoch
Figure 17: Evaluations R88Net-18. We compare generalization (green/red lines for training/ter data) and robustness (purple/blue bluupee//lueeee lines for training/ staiingg///ttttt test data, evaluated by the strongest AA attack) between NoiLIn (solid lines) and two AT methods such as SAT and TRADES (dash lines) three datascts-CIFAR. 10, CIFAR-100, and SVHN. The yellow histogram below each figure reflects the change of NL injection rate over the training process.
Selection criterion for the best checkpoint. We selected the best checkpoint based on robust test accuracy on AA Croce & Hein (2020) adversarial data when using ResNet-18. When using Wide ResNet, we use PGD-10 adversarial data for the consideration of speed. Note that the best checkpoints appear
Figure Figure 5(b] Figure 7 and all tables.
D.1 Relation with Label Smoothing
2020 Pang el al. 2021 has been comprehensively investigated among different smooth- AT-LS Cheng et al.
ing levels Pang et al. 2021 pointed out AT with LS only in outer minimization (*AT-I.S-outer)) under a mild smoothing level (e.g., p 0.1) can slightly improve adversarial robustness. Therefore, we sampled p from {0.05, 0.1, 0.2, 0.33. Training details (e.g., the optimizer and learning rate schedule) of SAT, AT-LS-outer under different p. and SAT-NoiL.In (Symmetric) kept. same as Section 5.1 We compared the performance of SAT, AT-LS-outer under different p. and SAT-NoiLIn (Symmetric) using RosNet-18 on CIFAR-10 datasct in Table We only reported AT-LS-outer (p = 0.1) in Table 2 for the reason that AT-LS-outer (p 0.1) achieves the most robustness against AA attacks at the best checkpoint among AT-LS-outer under different p.
AT-NoiLIn relieves robust overfitting while AT-LS does not. Compared with AT-LS under different P, AT-NoiLIn indeed alleviates robust overfitting. The absolute value of the robust test accuracy gap (in the parentheses the right two columns of Table 2 obtained by AT-NoiLIn is consistently smaller than that obtained by AT-L.S under different, p. In addition, observed AT-NoiL.In achieves better performance
23

---

Published in Transactions on Machine Learning Research (06/2022)
Table 5: Comparisons between SAT, AT-LS-outer Pang et al. (2021) under different p and SAT-NoiL.In
(Symmotric). We reported the test accuracy of the best checkpoint and that of the last checkpoint as well the gap between them "best/last (gap)".
Defense
Natural
AA
SAT Madry et al.
2018
81.97/84.76 (+2.79)
49.53/45.12
(-4.71)
48.09/43.30
(-4.99)
AT- -S-Lter (p- 0.05)
62.38/85.19 (+2.81)
49.60/44.95
(-4.65)
47.86/43.01
(-4.79)
AT-L.S-otte (p- = 0.1)
82.76/85.15
(+2.39)
50.06/45.40
(-4.66)
48.60/43.44
(-5.16)
AT-LS-outer (p- = 0.2)
92.80/85.19 (+2.39)
49.87/45.7
(-4.16)
48.42/43.96
(-4.46)
AT-LS-outer (p= = 0.3)
82.45/85.70
(+3.25)
49.70/ 45.33
(-4.37)
48.26/43.
43.99
(-4.33)
SAT-NoiT.In (Symmetric)
82.97 /83.6/ (+0.89)
51.21/50.55
(-0.85)
48.74 47.99
(-0.81)
of robustness evaluations bsed on C&W attacks and AA attacks at the best epoch than AT-LS under all different p.
AT-NoiLIn is more computationally efficient than AT-LS. We have stated that NoiLIn saves (C-1) log operations for each data compared with L.S. However, when it comes to empirical verification, we found AT-NoiLIn uses the comparable training time with AT-LS. Both AT-NoiLIn and AT-LS using ResNet-18 on CIFAR-10 dataset spent about 256 seconds per training epoch evaluated on a GeForce RTX 2080 Ti GPU. It is owing to that the GPU can parallelly conduct the log operation for each data. That. is, for each adversarial data ã, the GPU only needs to conduct one log operation on all clements of p(ă) simultancously.
But when AT-LS conducted on the dataset which contains more classes (C is larger), such Ima- geNet Deng et al. (2009) dataset, the GPU may need to conduct at least more than one log operation for each data. Since the whole tensor of p(ž) is too large for the limited GPU memory, p(ž) has to be segmented into several small tensors that satisfying GPU memory for log operation. By contrast, AT-NoiLIn only needs to conduct one log operation on the ÿ-th element p()) in this case. Therefore, NoiLIn framework better than AT-LS from the perspective of computational efficiency.
D.2 NoiLIn under Different Learning Rate Schedulers
Nacural toa data
PCD 40
CLW
Boboat tod semuaa (9)
LB abolder
Natra
Notaan tesk date.
PCD-
C.AW
Dolroe oue conney 189)
2
Baods
Epecl
Figure 18: The leftmost panel shows the learning rate w.r.t. epoch under different LR schedulers. The right panels demonstrate the learning curves of AT-NoiLIn under various LR schedulers using symmetric-fiiping NL (the upper panels) and pair-flipping NL (the lower panels), respectively.
We conducted SAT-NoiLIn using pair-flipping NL under different LR schedulers using ResNet-18 on CIFAR- datasct. All the training settings, such as the optimizer and hyperparameters of noise rate, kept exactly same as Section 5.1 except the LR scheduler. The learning rate w.r.t. epoch under different LR schodulers is shown in the leftmost. panel of Figure We demonstrated the learning curves of AT-NoiLIn (Symmetric) in
24

---

Published in Transactions on Machine Learning Research (06/2022)
Table 6: Evaluations of SAT-NoiLIn with pair-fipping NL using ResNet--18 on CIFAR-10 10 dataset under
various LR schedulers. We reported the test accuracy of the best checkpoint and that of last checkpoint
well as the gap between them "best/last (gap)".
Defense
LR scheduler
Natural
C&W 100
AA
SAT Madry et [2018]
piecewise
91.97/84.76 (+2.79)
49.53/45.12
(-4.71)
48.09/43.30
(-4.79)
piecewise
81.29/81.26 (+2.97)
56.27/66.27 (+1.92)
48.30/ 16.59
(-1.71)
SAT-NoiLIn
multiple docay
81.21/82.75 (+1.54)
50.87/54.01
(+3.14)
46.28/45.14
(-1.14)
(Pair)
cosine
81.69/84.59
(+2.90)
53.09/56.06
(+2.97)
46.75/45.31
(-1.4)
cyclic
84.34/84.34 (+0.00)
55.58/55.50 /5.588 (+0.00)
47.75/47.75 (400000)
the upper row of Figure 18 and AT-NoiLIn (Pair) in the lower of Figure 18 under various LR schedulers. Figure 18 clearly indicates NoiL.I can mitigate robust overfitting under all different LR schedulers.
Further, we reported the test accuracy of the best checkpoint and that of the last checkpoint as well as the gap between them achieved by NoiLIn using pair-flipping NL in Table g We observed the gap between test accuracy the best checkpoint and that of the last checkpoint largely narrows with automatic NL injection, which validates NoiLIn can relieve overfitting.
Moreover, Table 3 and Table show AT-NoiL.In under cyclic LR decay simply obtains the best checkpoint the last epoch (the gap is exactly +0.00), which suggests AT-NoiLIn under cyclic LR scheduler can help save the time for selecting the best checkpoint. Note that for selecting the best checkpoint, it is time-consuming to gain the robust test accuracy based on AA attack for ResNet-18 and PGD-10 attack for Wide ResNet over all training epochs.
D.3 NoiLIn with Batches that Consist of both Natural and Adversarial Examples
We evaluate the performance of ResNet-18 trained with batches composed of both natural and adversarial examples via NoiLIn (namely, SAT-NNiLIn-Mix). At each epoch, the model parameters are updated via minimizing the sum of natural data's cross-entrope loss and adversarial data's cross-entropy loss. We keep other training configurations (e.g., batch size) same as NoiLIn in Section 5 We repeat the experiments 3 times with different random seeds. In Figure 19 we show the learning curve of SAT-NoiLIn-Mix:
For one thing, Figure 19 shows that the natural test accuracy of SAT-NoiLIn-Mix is consistently above that of SAT and SAT-NoiLIn. This could be attributed to incorporating natural data into each batch, thus helping to improve natural accuracy. For another thing, we find that NoiLIn still can mitigate robust overfitting when each batch is composed of natural and adversarial data, although the robust test accuracy of SAT-NoiLIn- Mix is significantly lower than that of NoiLIn. Therefore, to achieve better robustness, we recommend NoiLIn with batches that consist of only adversarial examples.
Natural test data
Natural test aceurwyy (96)
30
Robard test accarney (95)
So.
So.
s0.
SXY SAT-KolLh SAT-NNilll-Mii
40
80
100
20
100
120
Figure 19: Comparisons between SAT, SAT-NoiLIn, SAT-NoiLIn with batches composed both natural and adversarial examples (SAT-NoiT.n-Mixx.
25


Reviews:
[{"review_id": "lncQ9s6UuJ", "reviewer": "Reviewer_8SzM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Some of the particularly good points about the paper, in my opinion, are as follows\n\n* I found the discussion of the role of the noise in the outer and inner maximisation problem interesting, intuitive, and borne out by experiments.\n* I think the authors have made a very good attempt at discussing/mentioning relevant existing literature.\n* The visualisations and the notations are also clear and helpful. I appreciated showing magnified views of the training curves as this shows, in greater detail, the most relevant part of the training.\n* I think the experimental evaluation is also sufficient. CIFAR10, CIFAR100, and SVHN are three commonly used datasets in adversarial training literature and the authors have used all of them. The paper uses AutoAttack to measure robustness, which is now a standard and a powerful method.\n\n\nWeakness\n* The experimental advantage of NoiLin is minor and insignificant. It is minor because the difference from not using NoiLin is often less than 1% and insignificant because it does not have any kind of error bars. For example in table 1, for AutoAttack NoiLin is always within 1% of its competitor. Similar results are seen in its comparison with label smoothing.  Therefore, I am not convinced that NoiLin has any actual advantage.\n* I found the discussion of the importance of noisy labels in the introduction very far-fetched and unsupported by experimental evidence. In essence, the main argument the paper makes is that \"noisy labels are helpful for AT\" as it allows for AT to account for the overlap in the input domain between different classes. However, there is no experimental evidence to show i) that this overlap exists in noiseless datasets ii) NoiLin actually helps avoid robust overfitting by this exact overfitting. Therefore, this discussion in the introduction seems to unrelated to the rest of the paper and potentially incorrect.\n* I am confused about the implication of Figure 3(b).  Why is having a larger variance in the loss of training data points more helpful and indicative of _helpful_ diversity ? Maybe I am missing the argument here and would encourage the authors or clarify the importance of this figure.\n* Section 4.4 and Figure 6 in Donhauser et. al have a very similar observation that adding noisy labels help avoid robust overfitting for adversarial training and they do this by avoiding the $\\ell_2$-max margin interpolator. How is the conclusion of this paper different from theirs and, according to the authors, is the underlying phenomenon the same in both cases ?\n\n\n\nDonhauser, Konstantin, et al. \"Interpolation can hurt robust generalization even when there is no noise.\" Advances in Neural Information Processing Systems 34 (2021).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper discusses the use of noisy labels in adversarial training. Noisy label can be introduced in the outer or the inner optimisation problem of PGD-type training. The paper discusses both of them and how they impact clean and adversarial accuracy. Further, the paper proposes an algorithm to introduce even more noise in adversarial training as a method to further improve adversarial accuracy by preventing robust overfitting.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "* Please repeat the experiments to show **confidence intervals** for all results.\n* Figure 5(b) left's y axis label is probably incorrect. Shouldn't it be  _Natural test accuracy_ ?\n* Please discuss the importance of Figure 3(b) --- why is the \"diversity\" as defined in the paper useful and how does it add to the contribution of the paper ?\n* Please look at modifying the introduction especially the part that discusses the role label noise plays in adversarial training. I find the current explanation that label noise somehow allows for overlap between different classes to not have a detrimental effect during AT to be unsatisfactory as it is not supported by any experiments. Rather I find the discussion in Donhauser et. al. that label noise is a natural regulariser as it avoids the max $\\ell_2$ margin interpolator to be more reasonable. Could I please ask the authors to discuss this ?", "broader_impact_concerns": "There are not broader ethical or impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CoITphumV7", "reviewer": "Reviewer_TrgE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths: \n* The paper includes solid experimental results. The observations related to noisy label injection in inner maximization / outer minimization are accompanied with intuitive explanations that are easy to follow. \n* The proposed algorithm is a \"meta\" algorithm that can be combined with existing algorithms due to the fact that state-of-the-art algorithms are not exploiting noisy labels. \n\n# Weaknesses\n* Writing requires more proofread and more mathematical rigour. Also, section 3.3 seems somewhat redundant.\n* Missing literature: The fact that more than half page discussing NoiLIn's relation with label smoothing while the original label smoothing paper is not cited is a bit concerning to me.\n* Correcting stereotype of noisy labels in the title is a bit misleading here: At least the label smoothing paper (https://arxiv.org/pdf/1910.11585.pdf) proposes using label noise as regularization to achieve adversarial robustness, which is intuitively very natural. Therefore, the novelty of the paper given that \"there may be no stereotype in using label noise for adversarial training\" is relatively small. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new empirical method NoiLIn to improve robustness in adversarial training: inject noisy labels into the adversarial training procedure's outer minimization, and the level of noise introduced in each epoch gets increased when robust overfitting happens. The method is motivated by their empirical observation that having noisy labels in the outer minimization helps reduce robust overfitting. However, they also observed that injecting too much noise destabilizes the training, which I believe is the main reason that NoiLIn gradually increases the level of noise along with training epoch. The experiment section shows that NoiLIn improves SOTA results, and comparison with label smoothing method is provided. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Adding the related paper and explaining that $\\textbf{the label noise has been used in a positive way for adversarial training}$ is critical.  \n\nAll the proposed adjustments below would strengthen the work: \n\n* a few grammatical errors (note that this list is not holistic since the reviewer was not reading word by word): \n  * P1: \"in labels for the learning the neighborhoods\"\n  * P1: \"from a unknown distribution\"\n  * P7: \"NL injection in data at has similar effect\"\n  * P9: \"The training settings of ... keeps \"\n  * P9: \"NoiLIn have ...\"\n* some abbreviations appear before their explanations. For example, NoiLIn (Pair) appears in Figure 5, but was first explained in Figure 6's caption. \n\n* More mathematical rigour would be appreciated: for example, when mentioning \"labels $y$ are flipped at random with the uniform distribution\", does the support set include the true label or not? As another example, condition for determine robust overfitting is stated as \"$\\sum_{i=e-\\tau}^{e}\\mathcal{A}\\_i<\\sum_{j=e-\\tau-1}^{e-1}\\mathcal{A}\\_j$\", why not just say $\\mathcal{A}\\_e < \\mathcal{A}\\_{e-\\tau-1}$?\n\n* Other writing related (suggestions): \n  * avoid using \"etc\" in academic paper writing. \n  * The second and third paragraph under Observation (i): it is better to illustrate the effects using Figure 2(b) first. Explaining Figure 2(b) in the third paragraph seems redundant/in the wrong order because one may already understand the key points through the second paragraph. ", "broader_impact_concerns": "There are no concerns on the ethical implications. Defending adversarial attack is important to safely applying machine learning techniques. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BwNvRHiyPZ", "reviewer": "Reviewer_4KSq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method consistently reduces adversarial overfitting.\n\n- The evaluation of adversarial robustness uses multiple attacks.\n\n- The authors conduct a range of ablation studies to empirically understand the behavior of their method.\n\n- The paper is well written.\n\n\nWeaknesses:\n- Overall the gains of the proposed method are modest. Compared to relevant baselines on CIFAR-10, the additional robustness is around 1 percentage point and the increase in standard (non-adversarial) accuracy of the same model is about 2 percentage points.\n\n- It is unclear if the proposed method also helps on larger datasets such as ImageNet. However I understand adversarial training on ImageNet requires substantially more compute and may not be feasible for the authors.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the effect of adding label noise in adversarial training to increase the robustness of neural networks. The authors experiment with both adding noise to the inner maximization problem of adversarial training (finding an adversarial example), and the outer minimization problem (minimizing the empirical risk on the adversarial examples). Based on exploratory experiments, the authors focus on adding noise to the outer minimization problem as they find that injecting noisy labels there helps reduce adversarial overfitting. Adversarial overfitting is an effect highlighted in earlier work where the adversarial test accuracy noticeably decreases as training progresses. To address adversarial overfitting most effectively via the injection of noisy labels, the authors propose a scheme where the fraction of noisy labels in each batch is dynamically adjusted based on the amount of adversarial overfitting on a validation set.\n\nAfter narrowing the scope of investigation to noisy labels in the outer minimization, the authors test their method on three datasets (CIFAR-10, CIFAR-100, and SVHN) and find a small but consistent reduction in adversarial overfitting. The authors then compare to multiple baselines and conduct a range of ablation studies to measure the empirical performance of their method in more detail.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- As an additional baseline, it could be helpful if the authors also compare to training a network with batches that consist of both standard examples and adversarial examples.\n\n- A summary results table similar to Table 1 but for CIFAR-100 and SVHN would make it easier to compare the effect of the proposed method to relevant baselines.\n\n- Why do only some of the plots and tables show both the \"symmetric\" and \"pair\" version of NoiLIn?\n\n- It would be good to know if the method also improves when additional training data is used as extra unlabeled data has been shown to increase adversarial robustness. The following paper provides a good overview of the currently best methods in terms of adversarial robustness: https://arxiv.org/abs/2010.03593  Ideally the advantages of NoiLIn would be additive with the currently best variants of adversarial training and yield a new state-of-the-art.\n\n\nSmall changes:\n\n- \"In Table , we also\" - table number missing near the bottom of Page 8.\n\n- Typo \"abalation study\" at the beginning of Section 5.3.\n\n- Figure 5 b: The y-axis label on the left plot should be \"Natural test accuracy (%)\".\n\n- The difference between the abbreviations \"AT\" and \"SAT\" is unclear.\n\n- The third sentence near the top of Page 11 seems to have a spurious \"kept\".\n", "broader_impact_concerns": "The paper offers incremental improvements for a problem that has been studied extensively in the literature over the past few years (adversarial robustness on image classification datasets like CIFAR-10). Hence the paper raises no new ethical concerns and from my perspective does not require a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a noisy label injection method to improve adversarial training. All reviewers acknowledged the merits of this work and unanimously recommend acceptance. In the rebuttal phase, there were several issues regarding the explanations and intuitions of the proposed method, as well as some ambiguous descriptions of the presented results. In the revised version, the authors have successfully addressed all reviewers' concerns. Although the performance improvement over standard adversarial training can sometimes be marginal, the improvement is consistently observed on all datasets. Therefore, the technical claims are solid. I recommend accepting the submission as is.
Decision: Accept (poster)