Annealing Self-Distillation Rectification Improves Adversarial Training
Authors: Yu-Yu Wu
Hung-Jui Wang
Shang-Tse Chen
Venue: ICLR 2024 poster (2024)
Keywords: Adversarial training
Adversarial robustness
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We propose a data-driven label rectification technique to mitigate robust overfitting.

Abstract: In standard adversarial training, models are optimized to fit invariant one-hot labels for adversarial data when the perturbations are within allowable budgets. However, the overconfident target harms generalization and causes the problem of robust overfitting. To address this issue and enhance adversarial robustness, we analyze the characteristics of robust models and identify that robust models tend to produce smoother and well-calibrated outputs. Based on the observation, we propose a simple yet effective method, Annealing Self-Distillation Rectification (ADR), which generates soft labels as a better guidance mechanism that reflects the underlying distribution of data. By utilizing ADR, we can obtain rectified labels that improve model robustness without the need for pre-trained models or extensive extra computation. Moreover, our method facilitates seamless plug-and-play integration with other adversarial training techniques by replacing the hard labels in their objectives. We demonstrate the efficacy of ADR through extensive experiments and strong performances across datasets.

Published as a conference paper at ICLR 2024
ANNEALING SELF-DISTILLATION RECTIFICATION IM- PROVES ADVERSARIAL TRAINING
Yu- Yu Wu, Hung-Jui Wang, Shang-Tse Chen National Taiwan University (r10922018 8, r1 -1092206, stchen) @cs eecieee.... ntu. eduutedd.tt
ABSTRACT
In standard adversarial training, models are optimized to fit invariant one-hot la- bels for adversarial data when the perturbations are within allowable budgets. However, the overconfident target harms generalization and causes the problem of robust overfitting. To address this issue and enhance adversarial robustness, we analyze the characteristics of robust models and identify that robust models tend to produce smoother and well-calibrated outputs. Based on the observation, we propose a simple yet effective method, Annealing ceff-Distillation Rectification (ADR), which generates soft labels as a better guidance mechanism that reflects the underlying distribution of data. By utilizing ADR, labels that improve model robustness without the need for pre-trained models or extensive extra computation. Moreover, our method facilitates seamless plug-and play integration with other adversarial training techniques by replacing the hard labels in their objectives. We demonstrate the efficacy of ADR through extensive experiments and strong performances across datasets.
obtain rectified
INTRODUCTION
Deep Neural Network (DNN) has been shown exhibit susceptibility to adversarial attacks (Szegedy et al.) 2014), wherein intentionally crafted imperceptible perturbations introduced into the original input cause the model's predictions to be altered. Among various defense methods (Ku- rakin et al. 2017: Liao et al. 2018 WWon & Kolter 2018)) Adversarial Training (AT) (Madry et al.
2018) stands out as one the most effective techniques (Athalye et al., 2018 Uesato et al. 2018)
to enhance DNN's adversarial robustness. However, while AT has proven effective in countering adversarial attacks,
not immune to the problem of robust overfitting (Rice et al. 2020). AT constrains the model to generate consistent output when subjected to perturbations within an at- tack budget. However, manually assigned hard labels are noisy (Dong et al. 2022b for AT since they fail to reflect shifts in the data distribution. Minimizing the adversarial training loss results in worse generalization ability on the test data. To address this issue, several approaches have been proposed, including label smoothing (Pang al. (021), consistency regularization (Dong et al., 2 Tarvainen & Valpola| 2017), and knowledge distillation (Chen et al.) 2021 Cui et al. 2021 Goldblum al. 2020al Zhu al. 2022 Zi et 2022 Zhao al. 2022)) which create a smoother objective function to alleviate the problem of robust overfitting. However, these approaches often assume uniform label noise (Pang et al. 2021), require consistency in the model's output over time (Dong et al. 2022b; Tarvainen & Valpola| 2017), or depend on an additional robust model trained in advance (Chen al.) [021] Cui et al.) 2021 Goldblum et al., 2020a). None of these methods directly focus on designing a well-rectified target without the need for pre-trained models.
To enhance AT, we investigate the characteristics that distinguish robust models from non-robust ones by analyzing the disparity in output distributions. Our findings indicate that robust models should possess good calibration ability, which manifested by a lower average confidence level when it is likely to make errors. In addition, robust models' output distribution should remain consistent for the clean data and its adversarial counterpart. Based on this observation, we propose novel approach called Annealing Self-Distillation Rectification (ADR), which interpolates the outputs from model weight's 's momentum encoder with one-hot targets to generate noise-aware labels that reflect the underlying distribution of data. The intuition behind this method is that if an image

---

Published as a conference paper at ICLR 2024
is similar to the other classes once, we should assign higher probability to those classes but still maintain the dominant probability for the actual class to ensure the final prediction is not altered.
The weight momentum encoding scheme, also known Mean Teacher, is a widely used technique in semi-supervised (arvainen & Valpola, 2017) and self-supervised (Grill et al. 2020 Caron et al. 2021 learning that involves maintaining exponential moving average (EMA) of weights the trained model. The self-distillation EMA also serves as a Weight Average (WA) (Garipov et al.
2018) method, which smoothes the loss landscape to enhance robustness (Izmailov al.) 2018 Chen et al. 202T] Gowal et al. 2020). To ensure our model generates well-calibrated results that reflect inter-class relations within examples, we introduce softmax temperature (Hinton et al. 2015) to scale outputs from the EMA model. Initially, the temperature is set to larger value producing uniform distribution but gradually decreases following a cosine annealing schedule. This dynamic adjustment of temperature is undertaken in recognition of the EMA model's s capacity to represent inter-class relationships improves over the course of training. We introduce an interpolation strategy to ensure the true class label consistently maintains the highest probability in targets. Notably, the interpolation factor experiences progressive increments as a reflection of our growing confidence in the precision and robustness of the momentum encoder. In summary, our contributions are:
We conduct a comprehensive analysis of the robust models' output properties. Our inves- tigation confirms the calibration ability of robust models. Additionally, we observe that robust models maintain output consistency on benign data and its adversarial counterpart, which motivates us to design a unified rectified label to enhance the efficacy of adversarial defense. For samples within the lp norm neighborhood of given input, they should be associated with a single smooth and rectified target in adversarial training.
Rectification (ADR), a simple yet effective tech- nique that leverages noise-aware label reformulation to refine the original one-hot target. Through this method, we obtain well-calibrated results without requiring pre-trained mod- els extensive extra computational resources. Additionally, ADR can be incorporated into other adversarial training algorithms at ease by substituting the hard label in their objec- tives, thus enabling seamless plug-and-play integration.
We propose Annealing
Our experimental results across multiple datasets demonstrate the efficacy of ADR in im- proving adversarial robustness. Substitute the hard labels with well-calibrated ones gener- ated by ADR alone can achieve remarkable gains in robustness. When combined with other AT tricks (WA, AWP), ADR further outperforms the state-of-the-arr results on CIFAR-100 and TinllmageNet-2000 datasets with various architectures.
RELATED WORK
Adversarial training (AT) has been demonstrated to be effective in enhancing the white box ro- bustness of DNN (Croce et al. 2021 PGD-AT (Madry et al.] 2018), which introduces worst-case inputs during training. has been the most popular approach for improving robustness. An alterna- tive AT method, TRADES (Zhang et al.] 2019), provides systematic approach to regulating the trade-off between natural accuracy and robustness and has yielded competitive results across multi- ple datasets. Despite the efficacy, AT often suffers from robust overfitting (Rice et al. 2020). Below, we summarize works that address the issue of robust overfitting by reforming the label. We also provide extra survey on works aiming to mitigate the issue with other methods in Appendix A
Rectify labels in AT. AT can smooth the predictive distributions by increasing the likelihood of the target around the e-neighborhood of the observed training examples [Lakshminarayana et al.] 2017). A recent study by Grabinski et al. (2022) has shown that robust models produced by AT tend to exhibit lower confidence levels than non-ro models, even when evaluated on clean data. Due to the substantial differences in output distributions between robust and standard models, using one-hot labels, which encourage high-confidence predictions on adversarial examples, may not be optimal. Dong al. (2022b) and Dong et al. (2022a) have demonstrated that one-hot labels are noisy in AT, as they are inherited from clean examples while the data had been distorted by attacks. The mismatch between the assigned labels and the true distributions
exacerbate overfitting com- pared to standard training. Rectifying labels is shown to be effective in addressing the issue of robust overfitting in AT (Dong et al. 2022a). Label Smoothing (I.S) (Szegedy et al., 2016) is technique
2

---

Published as a conference paper at ICLR 2024
that softens labels by combining one-hot targets and uniform distribution. By appropriately choos-
ing the mixing factor, mild LS can enhance model robustness while calibrating the confidence of
the trained model (Pang al., 2021 Stutz al. 2020). However, overly smoothing labels in
data-blind manner can diminish the discriminative power of the model (Müller et al. 2019] Paleka
& Sanyal| (022) and make it susceptible gradient masking (Athalye et al. 2018). Prior works
(Chen et al.) 2021: Cui et al.) 2021] Goldblum et al. [2020a] Zhu et al. [022] Zi et al. 2021 Zhao
al. 2022) have utilized Knowledge Distillation (KD) generate data driven soft labels, outper-
forming baseline approaches. Temporal Ensembling (TE) (Dong et al. (022b) and Mean Teacher
(MT) (Zhang et al.) 2022) have applied consistency loss into training objectives, thus preventing overconfident predictions through consistency regularization. More recently, Dong et al. (2022a) have employed pre-trained robust model to reform training labels, addressing label noise in AT.
PRELIMINARIES
3.1 ADVERSARIAL TRAINING (AT)
Given a dataset D= {(x, w)] where x, E Rd is a benign example, yt E {l....,C) is ground truth label often encoded as a one-hot vector y; E (0.1) and C is the total number of classes, PGD-AT (Madry et al. 2018) can be formulated as the following min-max optimization problem:
min
max {(fo(x), (ES(X,)
where fo is a model with parameter 0. is the cross-entropy loss function, and S(x) = fx' : ||x' x|Ip S { is an adversarial region centered at with radius e > 0 under 1n-norm threat model.
The adversarial example x can be obtained by projected gradient descent (PGD) to approximate the inner maximization in adversarial training, which randomly initializes a point within S(x,) and iteratively updates the point for K steps with:
IIs(x:) (xf + a sign (Vx((f(((()
(2)
where IT(.) is the projection, a is the attack step size, denotes iteration count, and x/
x
3.2 DISTRIBUTIONAL DIFFERENCE IN THE OUTPUTS OF ROBUST AND NON- ROBUST MODEL
The standard training approach encourages models to generate confident predictions regardless of the scenario, leading overconfident outcomes when the testing distribution changes. In contrast, robust models, when compared to their standardly trained counterparts, possess superior calibration properties that exhibit low confidence in incorrect classified examples (Grabinski et al.) 2022). In addition, study conducted by Qin et al.] (2021) has revealed that poorly calibrated examples are more vulnerable to attacks. The interplay between robustness and confidence calibration motivates us to enhance the information inherent in labels. Therefore, we initiate our analysis by examining the differences in output distribution between robust and normal models.
Robust model generates a random output on OOD data When there significant distribution shift in the testing data, a well-calibrated model is expected to display uncertainty in its predictions by assign- ing uniformly random probabilities to unseen examples. To analyze the difference in output distributions when predicting out-of-distribution (OOD) data, we follow the approach by Snock et al. (2019); Qin et al. (2021). Specifically, we compare the histogram of the output entropy of the models. We use ResNet-18 to train the models on the CIFAR- 10 (in-distribution) training and evaluate the CIFAR- 100 (OOD) testing set. Given that most categories in CIFAR- 100 were not present during training, we expect the models to reveal suspicion. As shown in Figure the non-robust model has low entropy (high confidence) on OOD data, while the robust model exhibits high uncertainty on average.
CIPVB. (10k:2004///T6--000000 200
Rabea: Vodal
1060
Encroyy
Figure 1: Output dis- tribution on OOD data. Both models are trained on CIFAR-10 and tested on CIFAR-100 100.
3

---

Published as a conference paper at ICLR 2024
Non Made!
Robust Made!
Non Rustt Model
Robust Model
Distr bution shift after PGD
cantrils desitee
ccecat casorie
wargg clastles
Entropy
Entmpy
Entopy
(a)
(h)
(c)
(d)
(c)
Figure 2: (a) and (b) are entropy distributions on the correctly classified and misclassified examples on the standard and robust model respectively. (c) and (d) are entropy distributions on the clean and adversarial examples on the standard and robust model respectively. (e) shows histograms of JS
divergence for output distribution shift under the PGD-10 10 attack.
Robust models are uncertain on incorrectly classified examples To investigate potential distri- butional differences in the behavior of standard and robust models when encountering correctly or incorrectly classified examples, we consider the model's confidence level in its predictions. Lower confidence levels typically associated with higher error rates if the model is well-calibrated. To this end, we experiment by using a ResNet-18 18 model trained and evaluated on the CIFAR-10 dataset to demonstrate the distributional differences between correctly and incorrectly classified examples for both standard and robust models. Specifically, Figure 2a illustrates that the standard model ex- hibits low entropy levels for correctly classified examples, but relatively uniform entropy levels for incorrectly classified ones. Higher confidence in the prediction does not guarantee better perfor- mance. On the other hand, Figure 2b shows that the robust model tends to exhibit relatively high entropy levels (low confidence) for misclassified examples. We can infer that when the robust model confident its prediction, the classification accuracy likely to be high.
Output distribution of models on clean or adversarial examples are consistent Several prior studies (Zhao et al., 2022, Cui et al. 2021 have suggested learning clean images' representation from the standard model and adversarial example's s representation from the robust model to improve robustness while maintaining accuracy. However, the underlying assumption that the robust model exhibits comparable representations of clean images to those generated by the standard model has not been thoroughly examined. Therefore, we investigate whether these models show comparable behavior when presented with clean and adversarial (PGD-10) examples on the CIFAR-10 dataset.
We demonstrate that the standard model exhibits low entropy in both scenarios (Figure 2c whereas the robust model yields high entropy on average (Figure 2d). Additionally, Figure 2e reveals two models' histograms of JS divergence, representing the extent of output distribution shift when input is attacked. We can observe that even if robust models are attacked successfully, the change of output distribution measured in JS divergence still small compared to standard models. The robust models show higher consistency (low JS divergence), while the standard models make drastic output changes. Therefore, promoting learning from standard models on normal examples may not be ideal for robust models, as robust models do not generate high confidence output on clean data.
4 METHODOLOGY
4.1 MOTIVATION: RECTIFY LABELS IN NOISE-AWARE MANNER
Based on previous analysis, robust models should satisfy three key properties: first, they generate nearly random probability on OOD data; second, they demonstrate high uncertainty when it is likely to make a mistake; and third, they exhibit output consistency for both clean examples and their ad- versarial counterparts. However, the one-hot label used in AT does not provide sufficient guidance to reflect real-world distribution. Restricting the output to fix hard labels can be toxic to ad vverraaiaal training (Dong et al., 2022a), as causes the model to memorize the labels (Dong et al. 2022b) to minimize training loss, but at the expense of losing generalization ability on the testing set. In recent study conducted by Paleka & Sanyal (2022), it was found that uniform label noise has a sim- ilar degree of adverse impact as worst-case data poisoning. They also provide empirical evidence that real-world noise is less harmful than uniform-labe noise. Specifically, the noise introduced by human annotators poses lower adversarial risk than uniform label noise. Therefore, designing a label-softening mechanism that gradually approaches the true underlying distribution instead of assigning uniform noise label smoothing is essential to improve robustness. Building on these in-
4

---

Published as a conference paper at ICLR 2024
1. Get Recilied Distribttion P(x)
Ground TrufhGGaase
Cosine Annzaling
2. Adversarial Attack
Cosine Annealing
3. Adversarial Training
4. Teacher
8
Epoch
Epocc
Teacher 6
f
1. Rectftie
Distribution P(x)
Teachar
P((x))
9, 7 (3--))-- 7).8.
Losa (xe'), P(x))
Student 8,
P,(x')
3. Update Student via SGD
2. Update at' via Multi-step PGD
Figure 3: Overview of ADR.
sights, we propose data-driven scheme, Annealing Self-Distillatiin Rectification (ADR), to rectify labels in a noise-aware manner that mimics the behavior of human annotators.
4.2 ANNEALING SELF-DIISTILLATION RECTIFICATION
To be specific, let Ag represent the trained model's parameter to optimize, and 6, be the EMA of 0., which is updated by 0, = y 0r + (1 7) - 0g where y the decay factor. (x(x,) EMA's output distribution on input x,, and Pt (x,))((c) serves as EMA's predicted probability for the class c. To obtain a rectified label, we first calculate the EMA's softened distribution (x(x;) with temperature T, which follows a cosine annealing from high to low. Since Se, cannot provide sufficient knowledge at the beginning of training, the high temperature encourages the EMA's output approach uniform noise. As fe, becomes more accurate and robust, we anneal the temperature to make the distribution more descriptive of the inter-class relations. The smoothed distribution P: (x;) for fo, is follows,
(x,, softmax(foo (x4./T)
However, fe, does not always classify correctly, especially when training is insufficient. To ensure the correct class has the highest probability so the target is unchanged, we interpolate the predicted distribution of fe., P. (x;), with ground-truth one-hot ys, which is built from ys- by interpolation ratio A. X follows an increasing cosine schedule, allowing us to trust EMA more over time. For each X1, we also adjust À to A ensure the true class has the highest probability across the distribution.
A = clip (A 10:---Pxxxxxp (P.(x.) - (x))))(
where vi exhibits the EMA's predicted class and ys represents the ground truth class. When EMA makes correct prediction, that is P.(xx) = (x.))((*)) there is need to adjust the interpo- lation rate, otherwise, we decrease À by the amount that the EMA model makes mistake on x; and then clip to the [0, 1] range. Finally, the rectified distribution Pxx.) used for adversarial attack and training is carried out as
P(x,) A, Px(x,) (1 (1-1x)) Ax) yy
We use the rectified label P(x;) to replace the ground truth label y; in Equation I and Equation to conduct adversarial training. Similarly, the softened P(x,) can be applied in other adversarial training algorithms, c.g. TRADES (Zhang et al. 2019),, by replacing the hard labels. We illustrate the overview of ADR in Figure 3 and summarize the pseudo-code in Appendix B
EXPERIMENTS
In this section, we compare the proposed ADR to PGD-AT and TRADES Table [ We fur-
ther investigate the efficacy of ADR in conjunction with model--weghttsppace smoothing techniques Weight Average (WA) (Izmailov et al. 2018 Gowal et al.) 2020) and Adversarial Weight Pertur- bation (AWP) (Wu et al. 2020) with ResNet18 (He al. 2016a) in Table 2 Experiments
conducted on well-established benchmark datasets, including CIFAR-10, CIFAR- (Krizhevsky)
al. 2009). and Tent (Le & Yang 2015 Deng et al. 2009).
We highlight that
5

---

Published as a conference paper at ICLR 2024
Table 1: Test accuracy (%) of ADR compared to PGD-AT and TRADES with ResNet18. Best refers to the checkpoint with the highest robust accuracy on the evaluation set under PGD-10 evaluation. Final is the last checkpoint, and Diff is the difference of accuracy between Best and Final. The best results and the smallest performance differences are marked in bold.
AutoAttack
Standard Acc.
Dataset
Method
Best
Final
Diff.
Best
Final
Diff.
AT
48.81
43.19
5.62
82.52
83.77
-1.25
AT + ADR
50.38
47.18
3.20
82.41
85.08
-2.67
CIFAR-10
TRADES
50.1
48.17
1.93
82.95
82.42
-0.53
TRADES ADR
51.02
50.4
0.62
83.4
83.76
-0.36
AT AT + ADR
24.95
19.66
5.29
55.81
56.58
-0.77
26.87
24.23
2.64
56.1
56.95
-0.85
CIFAR- 100
TRADES
24.71
23.78
0.93
56.37
56.01
0.36
TRADES + +ADR
26.42
25.15
1.27
56.54
54.42
2.12
AT AT ADR
18.06
15,86
2.2
45.87
49.35
-3.48
19.46
18.83
0.63
48.19
47.65
0.54
Tet-
TRADES 17.35
16.4
0.95
48.49
47.62
0.87
TRADES + ADR
19.17
18.86
0.31
51.82
50.87
0.95
defending against attacks datasets with enormous classes is more difficult the model's de- cision boundary becomes complex. We also provide experiments on Wide ResNet (WRN-34-10) (Zagoryyko & Komodakis (016)) with additional data compared other state-of-the-an methods reported on RobustBench (Croce et al. 2021) in Table 3 Our findings reveal that ADR outperforms methods across different architectures, irrespective of the availability of additional data.
5.1 TRAINING AND EVALUATION SETUP
We perform adversarial training with perturbation budget 8/255 under l -nom in all experi- ments. In training, we use the 10-step PGD adversary with step size a = 2/2555 We adopt 3 for TRADES as outlined in the original paper. The models are trained using the SGD optimizer with Nesterov momentum of 0.9, weight decay 0.0005, and a batch size of 128. The initial learn- ing rate is set to 0.1 and divided by 10 50% and 75% of the total training epochs. Simple data augmentations include 32 x 32 random crop with 4-pixel padding and random horizontal flip (Rice 2020 Gowal et al. 2020 Pang al. 202 1) are applied in all experiments. Following Wu et al.
al.
22020; Gowal et al. (2020). we choose radius 0.005 for AWP and decay rate = 0.995 for
WA. For CIFAR-IO/IO0 we use 200 total training epochs, X follows cosine scheduling from 0.7 to 0.95, and F is annealed with cosine decreasing from 2.5 to 2 on CIFAR- 10 and 1.5 to on CIFAR-
100, respectively. As for TinylmageNet-299 200, we crop the image size 64 x 64 and use 80 training epochs. We adjust À from 0.5 to 0.9 and r from 2 to 1.5 on this dataset.
During training, we evaluate the model with PGD-10 and select the model that has the highest robust accuracy on the validation set with early stopping (Rice et al. (2020). For testing, we use AutoAttack (Croce Hein| 2020b) which comprises an ensemble of attacks including APGD-CE (Croce & Hein 2020b), A APGD-DLR (Croce & Hein 2020b), FAB (Croce & Hein 2020a) and Square attack (Andriuscchenko et al. 2020) for rigorous evaluation. To climinate the possibility of gradient obfus- cations (Athalye et al. 2018),
provide sanity checks in Appendix E Unless otherwise specified, the robust accuracy (RA) is computed under AutoAttack to demonstrate the model's generalization
ability on unseen attacks. The computation cost analysis is attached in Appendix G
5.2 SUPERIOR PERFORMANCE ACROSS ROBUSTIFIED METHODS AND DATASETS
Table demonstrates the results of ADR combined with PGD-AT and TRADES on 10, CIFAR- and TinylmageNet-200. Our initial observations reveal that robust overfitting exists in all baselines, with differences between final and best early-stopping robust accuracies as large as 5.62% CIFAR-10 while the standard accuracy (SA) remains stable with more training epochs.

---

Published as a conference paper at ICLR 2024
Test accuracy (%) of ADR combining with WA and AWP. The best results are marked in bold. The performance improvements and degradation are reported in red and blue numbers.
Table 2:
ResNet-18
WRN-34-10
Method
AutoAttack
Standard Acc.
AutoAttack
Standard Acc.
AT
48.81
82.52
52.12
85.15
+ WA
49.93 (+ 1.12)
83.71 (+ 1.19)
53,97 (+ 1.85)
83.48 (- 1.67)
WA + AWP
50.72 (+ 1.91)
82.91 (+ 0.39)
55.01 (+2)
87.42 (+ 2.27)
+ ADR
50.38 (+ 1.57)
82.41 0.11)
53.25 (+ 1.13)
84.67 (- 0.48)
+ WA + ADR
50.85 (+ 2.04)
82.89 (+ 0.37)
54.10 (+ 1.98)
82.93 (- 2.22)
WA + AWP + ADR
51.18 (+ 2.37)
83.26 (+ 0.74)
55.26 (+ 3.14)
86.11 (+ 0.96)
(a) CIFAR-10
ResNet-18
WRN-34-10
Method
AutoAttack
Standard Acc.
AutoAttack
Standard Acc.
AT
24.95
55.81
28.45
61.12
+ WA
26.27 (+ 1.32)
53.54 (- 2.27)
30.22 (+ 1.77)
60.04 (- 1.08)
+ WA + AWP
27.36 (+ 2.41)
59.06 (+ 3.25)
30.73 (+ 2.28)
63.11 (+ 1.99)
+ ADR
26.87 (+ 1.92)
56.10 (+ 0.29)
29.35 (+ 0.90)
59.76 (- 1.36)
+ WA + ADR
27.51 (+ 2.56)
58.30 (+ 2.49)
30,46 (+ 2.01)
57.42 (- 3.70)
WA + AWP ADR
28.50
(+ 3.55)
57.36 (+ 1.55)
31.60 (+ 3.15)
62.21 (+ 1.09)
(b) CIFAR- 100
R8
WRN-34-10
Method
AutoAttack
Standard Acc.
AutoAttack
Standard Acc.
AT
18.06
45.87
20.76
49.11
+ WA
19.30 (+ 1.24)
49.10 (+ 3.23)
22.77 2+01)
53.21 (+ 4.10) 53.35 (+ 4.42)
+ WA + AWP
19.58 (+ 1.52)
48.61 (+ 2.74)
23.22 (+46)
+ ADR
19.46 (+ 1.40)
48.19 2.32)
21.85 (+ 1.09)
51.52 (+ 2.41)
+ WA + ADR
20.23 (+ 2.17)
48.55 (+ 2.68)
23.01 (+
51.03 (+ 1.92)
WA + AWP ADR
20.12 (+ 2.06)
48.27 2.40)
23.33 (+5))
51.44 (+ 2.33)
(c) TinylmageNet-2000
When combined with our approach, we observe consistent improvements across datasets, with re- duced robust overfitting gaps from 5.62% to 3.2% CIFAR- 10, 5.29% to 2.64% on CIFAR- 100, and 2.2% to 0.63% on TinylmageNet-200.. Furthermore, the robust accuracy improves by 0.92% to 1.92% across experiments. By alleviating robust overfitting, the best checkpoints end of the training, thereby improving the SA in most settings. Our findings indicate that ADR can effectively enhance the RA-SA trade-off by improving robustness while achieving higher standard accuracy. We also present performance variation across multiple reruns in Appendix H
closer to the
5.3 COMBING WITH WEIGHT SPACE SMOOTHING TECHNIQUES AND LARGER ARCHITECTURE
The proposed ADR can be integrated with other AT techniques to boost robustness further (Table Additional experiments with TRADES are presented in Appendix C WA (Gowal al. 2020 and AWP (Wu et al.) 2020) are the model- smoothing techniques that improve the stability and performance of AT. In our case, we can acquire WA result by evaluating 6, as it maintains the EMA of the trained model's weight. Combining ADR with AWP and WA, we obtain large gains in RA ranging from 1.12% to 3.55% and 0.37% to 3.23% in SA compared to the ResNet-18 baselines.
Following prior works (Chen et al. 2021 Addepalli et al. 2022). we additionally use Wide ResNet (WRN-34-10) to demonstrate that ADR scales to larger architectures and improves RA and SA. The result shows that our method effectively enhances robust accuracy up to 3.14% on CIFAR- 3.15% on CIFAR-106 100, and 2.57% on Tiny InylmmggNet-200 use the same X, ResNet-18 for WRN-34- 10, which might not be optimal, to reduce the cost of hyperparameter searching. Therefore, we observe a slight drop in standard accuracy in some WRN cases. Nevertheless, ADR still outperforms baselines in robustness without tuning paramerrs
compared to each of its baselines. Notably, we
7

---

Published as a conference paper at ICLR 2024
Table 3: Comparison of ADR with related works on CIFAR-100.
Architecture
Method
Extra Data
AutoAttack.
Standard Acc.
Zhang el al. (2022
26.03
58.17
Dong et al.
2022b
26.30
56,45
ResNet18
Dong cl al. 2022a
26.36
58.80
Addepall et al.
2022
27.62
66.69
A1+ADR (Ours)
28.50
57.36
Rebulli al. 202
DDPM
28.50
56.87
Preact-ResNer18
Rade & Moosavi- Deztooli
2022
DDPM
28.88
61.50
AT+ADR (Ours)
DDPM
29.59
57.88
Chen & Lee (2021)
30.59
64.07
et al. [2022
30.77
64.89
Sehwag et al. (2022
DDPM
31.15
65.93
WRN-34-10
Cui al. (202T
31.20
62.99
Addepalli al. [022
31.30
68.74
AT+ADR (Ours)
31.60
62.21
AT+ADR (Ours)
DDPM
32.19
59.60
5.4
COMPARISON WITH RELATED WORKS AND USE ADDITIONAL DATA CIFAR-100
Table 3 compares our proposed ADR defense against related works on a more challenging CIFAR-
100 dataset. We select leading methods (Rade & Moosavi-Dezfoolij 2022 Gowal al.
2020
Rebulli et al.
a Addepalli et al. 2022, Cui et al.) 2021] Sehwag et al. 2022 Jia et al. 2021 on RobustBench and methods similar ours which introduce smoothing in
2022
Chen & Lee
training labels (Dong et al. 2022b a Zhang et al. (022) to make fair comparison. Since knowledge
distillation also promotes learning from the soft target, we discuss the benefits of ADR over those methods in Appendix[] [ The reported numbers are listed in their original papers on RobustBench. We also provide a similar comparison on TinylmageNet-200 in Appendix
D Given the observed benefits of incorporating additional training data to promote robust generalization (Schmidt et al. 2018), we employ a DDPM (Ho et al. 2020) synthetic dataset (Gowal et al. 2020 Rebulfi et al, 202 composed of million samples. Detailed experiment setup can be found in Appendix
Our experimentation with AT AT-WA-WW-ADR on ResNet- 18 yields a robust accuracy of 28.5% and standard accuracy of 57.36%, comparable Rebuffi et al. (2021a) that utilizes additional 1M DDPM data on Preact-ResNNet18, which yields an RA of 28.5% and SA of 56.87%. Remarkably, model attains equal robustness and superior standard accuracy without using additional data when employing a similar-sized model. Similarly, we achieve an RA of 31.6% on WRN-34-10, while Schwag et al. (2022) scores only 31.15% with additional data. Additionally, adding DDPM data in the training set leads to further improvement in robust accuracy for ADR, by 1.09% and 0.59% for R8 and WRN-34-10, respectively. In both cases, ADR achieves new state-of- the-art performance, both with and without additional data, on the CIFAR-100 benchmark. It is worth noting that some methods introduce extra auxiliary examples when training (Dong et al.) 20226 Rade & Moosavi-Dezfoon 2022). and some bring complex augmentation into AT (Rade & Moosavi-Dezfoolla 2222] Rebuffi et al. 2021a). and so might obtain superior SA compared to ADR. However, regarding the optimal robustness to achieve, empirical findings provide compelling evidence that rectifying training labels with a realistic distribution is a valuable approach.
5.5 ACHIEVING FLATTER WRIGHT LOSS LANDSCAPE
Landacape
Several studies (Wu et al. 2020 Stutz et 2021 have found that flatter weight loss landscape leads to a smaller robust generalization gap
when the training process sufficient. Many methods (Wu et al. 2020) Chen et al.] 2021 Gowal et al. 2020; Zhang et al. 2019) addressing ro-
bust overfitting issues predominantly find flatter minima. We visualize the weight loss landscape by plotting the loss change when moving the weight w along a random direction d with magnitude a. The direction sampled from Gaussian distribution with filter nonmalization (Li al. 2018). For each perturbed model, we generate adversarial examples on the fly with PGD-10 and calculate the mean loss across the testing set.
Figure 4: Model weight loss landscape compari- son for AT and ADR.
8

---

Published as a conference paper at ICLR 2024
Clean Accuncy
Robust Accuracy
66:11
0022
L224
1.00
2:668
6:400
8252
6.7-2030
8222
6622
0:24
0332
6.30
0.7-3.450
6422
0.43
3:44
444
6.000
68175
63-2014
0.22
2.26
8.315
18150
0.3-2462
0.802
0A42
6401
Amadin
88325
Amnuiing
15800
Arraaary
0.422
Figure 5: Effectiveness of different temperature 7 and label interpolation factor À of ADR.
Figure 4 compares the weight loss landscape between AT and ADR on CIFAR- 10. ADR achieves a flatter landscape, implying better robust generalization ability. While smooth the label for ADR in training. we use the one-hot label as ground truth to calculate the cross-entropy loss in this exper- iment, so the model trained by ADR has a higher loss value than AT on average. Additionally, we visualize the loss landscape around the data point in Appendix and observe similar phenomenon that ADR produces flatter loss landscape.
Accuracy Robustness Tade off
5.6 ACCURACY VS. ROBUSTNESS TRADE-OFF
$1.0
Mitigating the trade-off between accuracy and robustness has been chal- lenging the realm of adversarial training. To investigate if ADR is capable of reducing such trade-off, we combine ADR with TRADES and adjust the trade-off parameter to demonstrate the performance dif- ference in terms of different values of /3. The result is represented in Figure 6 We decrease 3 from left to right, where a higher value of 3 gives better robustness. We can clearly observe that TRDDES+ADR achieves a better trade-off compared to that of using the hard label alone.
Cean Ace
Figure 6: Comparison of the trade-off for ADR with TRADES.
5.7 ABLATION STUDY ON EFFECTIVENESS OF TEMPERATURE AND INTERPOLATION FACTOR
To disclose the impact of temperature and interpolation factor À on the proposed ADR, we conduct ablation studies to exhibit grid search outcomes of SA and RA in Figure 5 Throughout experiments, and À are held constant unless explicitly specified "Annealing." " to scrutinize the effects of vary- ing parameter values. Furthermore, we include Label Smoothing (LS) this study, which can be viewed as special case where approaches infinity, evaluate how data-driven smoothness im- proves performance. Our analysis reveals that in terms of clean accuracy, choosing a smaller and suitable temperature moderately large À with appropriate r can ensure that the training labels inherent enough inter-class relationship. Therefore, selecting a proper value of parameters is vital to maintaining clean accu- racy while enhancing robustness. Our experiment also reveals that annealing in both temperature and interpolation factors is heneficial improve robustness, which shows the efficacy of gradually increasing reliance on the EMA model.
3 can achieve the best performance. As for the robustness, using
CONCLUSION
In this paper, we characterize the key properties that distinguish robust and non-robust model out- put. We find that robust model should exhibit good calibration and maintain output consistency on clean data and its adversarial counterpart. Based on this observation, we propose data-driven label softening scheme ADR without the need for pre-trained resources or extensive computation over- head. To achieve this, we utilize the self-distillation EMA model to provide labeling guidance for the trained model, with increasing trust placed in the EMA as training progresses. Comprehensive ex- periments demonstrate that ADR effectively improves robustness, alleviates robust overfitting, and obtains a better trade-off in terms of accuracy and robustness. However, we note that the algorithm's 's optimal temperature and interpolation ratio depend on the dataset, and improper selection of these parameters can limit performance improvements. The automatic determination of optimal parame- ters in training will be important future research direction that can further boost the robustness.

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
Adversarial training has the potential to improve the security, and reliability of machine learning sys- tems. In practical settings, adversarial attacks be employed by malevolent actors in attempt deceive machine learning systems. This phenomenon can engender grave consequences for domains such autonomous driving vehicles and facial recognition. To enhance the security and reliabil- ity of machine learning systems, adversarial training can be employed to produce more dependable models. However, it worth noting that robust models can also exploited by ill-intentioned users. In the context of the CAPTCHA, ad adversarial perturbations can be added to images to distinguish be- tween humans and robots since the robots are expected to be fooled by the adversarial data. If robots can attain robust models, they would not be susceptible to adversarial examples and could supply accurate answers. The advancement of model robustness may inspire people to formulate a better strategy to differentiate between humans and robots.
REPRODUCIBILITY
We describe the detailed experiment settings and hyperparameters in section 5.1 and Appendix F Furthermore, the source code can be found in the supplementary materials to ensure the reproducibil- ity of this project.
ACCKNOWLEDGGEEENT
This work was supported in part by the National Science and Technology Council under Grants MOST 110-2634-F002-051, MOST 110-222-F-002-014MMYY, NSTC 113-2923-E-002- 010-MY2, DTCC-112--634---02-00222 CC-112L891001 and by Center of Data Intelligence: Technologies. Applications, and Systems
by National Taiwan University under Grant NTU-
under Grant NTU- 121900903.
REFERENCES
Sravanti Addepalli, Samyak Jain, and R. Venkatesh Babu. Efficient and effective augmentation strategy for adversarial training. CoRR, abs/2210.153133 2022.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square tack: A query-efficient black-box adversarial attack via random search. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020 16th Eu- ropean Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, volume 12368 of Lecture Notes in Computer Science, pp. 484-501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICMI, 2018, Stockholmsmässsan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 274-283. PMLR, 2018
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902.067000 2019.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In Hanna M. Wallach, Hugo Larochelle. Alina Beygelzimer, Florence Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NewrlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 11190-1 12011 2019.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10- 17, 2021, pp. 6630-96400 IEFF, 2021.
10

---

Published as a conference paper at ICLR 2024
Erh-Chung Chen and Che-Rung Lee. LTD: low temperature distillation for robust adversarial train- ing. CoRR, abs/21 11.02331, 2021.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 2196-2205. PMLR, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensem- ble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learning. ICML 2020. 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 2216- PMLR, 2020b.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flam- marion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversar- ial robustness benchmark. Joaquin Vanschoren and Sai-Kit Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021.
Jiequan Cui, Shu Liu, Liwei Wang, and Jiaya Jia. Learnable boundary guided adversarial training- In 2021 IEEE/CVF International Conference Computer Vision, ICCV 2021, Montreal, QC. Canada, October 10-17, 2021, pp. 15701-15710. IEEE, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Puttern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pp. 248-255. IEEE Computer Society, 2009.
Chengyu Dong, Liyuan Liu, and Jingbo Shang. Label noise in adversarial training: A novel perspec- tive study robust overfitting. In Advances in Neural Information Processing Systems, 2022a.
Yinpeng Dong. Ke Xu, Xiao Yang, Tianyu Pang. Zhijie Deng. Hang Su, and Jun Zhu. Exploring memorization in adversarial training. In The Tenth International Conference on Learning Repre- sentations, ICLR 2022, Virtual Event, April 25-29, 2022. Open 2022b.
Logan Engstrom, Andrew Ilass, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. CoRR, abs/ ht///807..10 180111807722 10272, 2018.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, and Andrew Gordon Wil- son. Loss surfaces, mode connectivity, and fast ensembling of dnns. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Infor- mation Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 8803-8812, 2018
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In The Thirty-Fourlh AAAI Conference on Artificial Intelligence, AAAI 2020, The -Seooy- Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Sympo- sium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, Febru- ary 7-12, 2020, pp. 3903-4003. AAAI Press, 2020a.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 9996-4003 2020b.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy A. Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against boun-bounded adversarial examples. CoRR, abs/2010.03593, 2020.
Julia Grabinski, Paul Gavrikov, Janis Keuper, and Margret Keuper. Robust models are less over- confident. CoRR, 0bs/2210.05938 2022.
11

---

Published as a conference paper at ICLR 2024
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your latent A new approach to self-supervised learning. In Hugo I Larochelle, Mare' Aurelio Ranzato, Raia Had- sell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Process- ing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 770-778. IEEE Computer Society, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision ECCV 2016 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV, volume 9908 of Lecture Notes in Computer Science, pp. 630-645. Sprineer 2016b.
Matthias Hein and Maksym Andriushchenko. Formal guarantees the robustness of classi- fier against adversarial manipulation. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Gamett (eds.). Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Process- ing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 2266-2276, 2017.
Dan Hendrycks and Kevin Gimpel. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1 606.08415, 2016.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. 2015.
CoRR,
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Mare Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. In Amir Globerson and Ricardo Silva (eds.), Proceedings of the Thirty-Fourth Conference on Uncertainty Artificial telligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pp. 876-885, AUAI Press, 2018.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. LAS-AT: adversarial training with learnable attack strategy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 13388-13398. IEEE, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Ad- vances in Neural Information Processing Systems 30: Anmual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6402-64111 6413, 2017.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
12

---

Published as a conference paper at ICLR 2024
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 6391-64000 2018.
Lin Li and Michael W. Spratling. Data augmentation alone can improve adversarial training. CoRR, abs/2301 0.9899, 2023.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In 20/8 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1778-1787 Computer Vision Foundation IEEE Computer Society, 2018.
Iyy Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Confer- ence Track Proceedings. OpenReview.net, 2018.
Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buu, Emily B. Fox, and Roman Gameet (eds.), Advances in Neural Information Processing Systems 32: Annual Con- ference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 4696-4705, 2019.
Daniel Paleka and Amartya Sanyal. A law of adversarial risk, interpolation, and label noise. CoRR, abs/2207.03922 2022.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Yao Qin, Xuezhi Wang, Alex Beutel, and Ed H. Chi. Improving calibration through the relationship with adversarial robustness. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 14358-14369) 2021.
Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve better accuracy vs. robustness trade-off. In The Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Sylvestre-Alvis Rebuffi, Sven Gowal, Dan A. Calian, Florian Stimberg, Olivia Wiles, and Timothy A. Mann. Fixing data augmentation to improve adversarial robustness. CORR, abs/2103.01946, 2021a.
Alvivse Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Data augmentation can improve robustness. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021b.
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. In Proceedings of the 37th International Conference on Machine Learning, ICMI. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 8093-8106 PMLR, 2020.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Ad- versarially robust generalization requires more data. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa- Baisa-,,ncch,,,,, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems, pp. 5019-5031, 2018.
13

---

Published as a conference paper at ICLR 2024
Vikash Sehwag. Saced Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Robust learning meets generative models: Can proxy distributions improve ad- versarial robustness? In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Leslie N. Smith and Nicholay Topin. Super-convergence: Very fast training of residual networks using large learning rates. CoRR, abs/1708.0711 020000 2017.
Jasper Snock, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sebastian Nowozin, D. Scul- ley, Joshua V. Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems, pp. 13969-13980, 2019.
David Stutz, Matthias Hein, and Bernt Schiele. Confidence adversarial training: Gen- eralizing to unseen attacks. In Proceedings of the 37th International Conference Machine Learning, ICMI. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 9155-9166. PMLR, 2020.
David Stutz, Matthias Hein, and Bernt Schiele. Relating adversarially robust generalization to flat minima. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC. Canada, October 10-17, 2021, pp. 7787-7797. IEEE, 2021.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2818- 2826. IEEE Computer Society, 2016.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve semi-supervised deep learning results. In Sth International Conference on Learning Representations, ICLR 2017, Toulon, France, April 26-26, 2017, Workshop Track Proceedings. OpenReview.net, 2017.
Antonio Torralba, Robert Fergus, and William T. Freeman. 80 million tiny images: large data set for nonparametric object and scene recognition. JEEE Trans. Pattern Anal. Mach. Intell., 30(11) 1): 1978-1970, 2008
Jonathan Uesato, Brendan O' O'Donoghue, Pushmeet Kohli, and Aäron van den Oord. Adversar- ial risk and the dangers of evaluating against weak attacks. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning. ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 5032-5041. PMLR, 2018
Eric Wong and J. 7ico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th Inter- national Conference on Machine Learning, ICML. 2018, Stockholmsmässon, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 5283-5292. PMLR, 2018.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tice Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh, Youngjoon Yoo, and Junsuk Choe. Cutmix: Regularization strategy to train strong classifiers with localizable features. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), Oc- lober 27 November 2, 2019, pp. 6022-6031. IEFE, 2019.
14

---

Published as a conference paper at ICLR 2024
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Richard C. Wilson, Edwin R. Hancock, and William A. P. Smith (eds.), Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016. BMVA Press, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael Jor- dan. Theoretically principled trade-off between robustness and accuracy. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), International conference on machine learning, pp. 7472-7482. PMLR, 2019.
Shudong Zhang, Haichang Gao, Tianwei Zhang, Yunyi Zhou, and Zihui Wu. Alleviating robust overfitting of adversarial training with consistency regularization. CoRR, abs/2205. 1744, 2022.
Shiji Zhao, Jie Yu, Zhenlong Sun, Bo Zhang, and Xingxing Wei. Enhanced accuracy and robustness via multi-teacher adversarial distillation. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IV, volume 13664 of Lecture Notes in Computer Science, pp. 585 602. Springer, 2022.
Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. The Tenth International Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distil- lation: Robust soft labels make student better. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pp. 16423-16432. IEEE, 2021.
15

---

Published as a conference paper at ICLR 2024
A ADDITIONAL RELATED WORK: MITIGATE ROBUST OVERFITTING
The phenomenon of robust overfitting Rice et al. (020)) represents significant challenge in AT, motivating researchers explore various avenues for mitigation. One such approach is use heuristic-driven augmentations Rebufii et al. (2021b), such as CutMix Yun et al. (2019); [Rebufii] et al. (022a), DAJAT Addepalli Addepali et al. (2022), and CropShift [i & Spratling (202). which employ
sets of augmentations carefully designed to increase data diversity and alleviate robust overfitting. Another strategy involves the expansion of the training set, which offers a direct means to address overfitting. By incorporating additional unlabeled data Carmon et al. (2019) high-quality gen- erated images via deep diffusion probabilistic models (DDPM) Rebuffi et al. (2021a); Gowal et al. images from 80M-TI Torralba et al. (2008) eliminates the occurrence of robust overfitting Rebuffi et al. (2021a). Despite the demonstrated effectiveness of extra data, increasing the size of the training set com- putationally expensive, rendering AT infeasible for larger datasets.
(2020): Sehwag et al. (2022). the introduction of an extra of 500K pseudo-labelle
Early stopping straightforward method for producing robust models Rice et (2020) How- ever, due to the fact that the checkpoint of optimal robust accuracy and that of standard accuracy
frequently do not align Chen al. (202T), utilizing either of these measures can result in a com-
promise of overall performance. Weight Average (WA) Izmailov et al. (2018); Chen et al. (2021); Gowal et al. (2020) tracks the exponential moving average of model weights, thereby promoting flatter minima and increased robustness Hein & Andriushchenko/ (2017). Another effective regu- larization technique Adversarial Weight Perturbation (AWP) Wu et al. (2020), which serves to promote flatness within the weight loss landscape and yields enhanced generalization capabilities Stutz et al. (2021).
B ALGORITHM FOR ADR
Algorithm Annealing Self- Soff-Distilltionn Rectification (ADR)
Input: Training set {(x,, y1))
Parameter: A classifier f()) with learnable parameters 0,; 0g is exponential moving average of 0, with decay rate y; Batch size m; Learning rate vy: Total training iterations E; Attack radius c, attack step size o, number of attack iteration K; Temperature r; Interpolation ratio A.
1: Randomly initialize the network parameters 0,, 0, + 0,
2: for e = to F do
3: Calculate Te according to the current iterations.
4: Calculate A according to the current iterations.
5: Sample a mini-batch ((xj, y;) )m from D
6: for j 1 to m (in parallel) do
7:
(x(x,) t softmax(ff) fo. (x))/Te)
>> Calculate rectified label
8:
(Px (0b) (x(x))(()))) X5 = clip
9:
P(x,) + A (1- 21)
10:
X3 x te.d, ō, where ~ Uniform(-1, (1
Construct adversarial example
11:
for = to K do
12:
x,- II (xx
sign(( P(x;))))
end for end for
14:
15:
0g + 0.
n ΣØ V (x.((((o((())))
Update model parameters
m
16:
0e + 7 02 + (1 () 0
17: end for
C TEST ACCURACY OF TRADES + ADR COMBING WITH WA AND AWP
In this study, investigate the impact of combining TRADES and ADR with other adversarial training techniques, namely WA and AWP, on ResNet-18, 18, as outlined in Table Our experimental results demonstrate that leveraging soft target generated by ADR yields exceptional robustness and
16

---

Published as a conference paper at ICLR 2024
Table Test accuracy (%) ADR + TRADES combining with WA and AWP on ResNet-18. The best results are marked in bold. Robust Accuracy (RA) is evaluated with AutoAttack and Standard Accuracy (SA) refers to the accuracy of normal data.
CIFAR-10
CIFAR-I00
TinylmageNet-200
Method
RA
SA
RA
SA
RA
SA
TRADES
50.10
82.95
24.71
56.37
17.35
48.49
WA
51.10
81.77
25.61
57.93
17.69
49.51
WA + AWP
51.25
81.48
26.34
58.40
17.66
49.21
+ ADR
51.02
83.40
26.42
56.54
19.17
51.82
WA + ADR
51.28
82.69
27.11
58.58
19.17
51.99
+ WA + AWP + ADR
50.59
80.84
27.63
57.16
19.48
51.38
Table 5: Comparison of ADR with other related works TinylmageNet-200. The best result for each architecture is marked in bold
Architecture
Method
AutoAttack.
Standard Acc.
AT
18.06
45.87
R1
Rade Moosavi-Dezfooll
2022
18.14
52.60
Dong et al.
2022a
18.29
47.46
AI+ADR Ours)
20.23
48.55
WRN-28-10
Rebuffi et al.
(021bb)
21.83
53.27
AT ATuADR(Ours)
20.76
49.11
WRN-34-10
23.33
51.44
standard accuracy improvement, thereby achieving superior trade-off. Specifically, ADR results in 1.18%, 2.92%, and 2.13% RA improvement and 0.45%, 2.21%, and 3.5% SA improvement on the baseline performance of CIFAR-10, 10, CIFAR-100 and TinylmageNet-200 respectively.
However, we observe that the robustness improvement saturates or even slightly deteriorates when combining TRADES+ADR with weight smoothing techniques CIFAR-10. This is attributed to the fact that TRADES already promotes learning and attacking adversarial data on a softened target, making the additional soft objective by ADR less effective. Nevertheless, the TRADES+ADR approach remains beneficial when dealing with more challenging datasets that contain greater number target classes combined with WA and AWP.
D TEST ACCURACY (%) COMPARED WITH RELATED WORKS ON TINYIMAGHNET-200.
8); Rade & Moosavi-Deztooll 2002);; Dong et al. (2022)): Rebuffi et al. (20216) in Table on the TinylmageNet-2 dataset, which is a more challenging robustness benchmark than CIFAR- 10 CIFAR-100 due to its larger class size and higher-resolution images, using the original numbers from their respective papers. A larger class dataset often results in
We present ADR evaluated against related works Madry et al. (2018):
complex decision boundaries, which increases the likelihood of an attacker identifying vulnerabilities in the model. Our experimental results demonstrate that ADR achieves state-of-the-arr performance, improving RA by 1.94% to 2. 1.1%% when using ResNet-18 18, and achieving remarkable 2.57% improvement over the baseline on WRN-34-18 10. In summary, we observe that ADR stands out in the challenging multi-class scenario.
model trained
E SANITY CHECK FOR GRADIENT OBFUSCATION
Athalye et al. (018) argued that some defenses improving the robustness by obfuscated gradient, which is introduced intentionally through non-differentiable operations unintentionally through
17

---

Published as a conference paper at ICLR 2024
Robust Accuracy vs. number of steps K of PGD
Robust Accuracy under Different Attack Radius
ADR
0.8
ADR
0.80
PGD Robust Acc.
0.75
0.6
0.70
PGD 100
0.4
0.65
0.60
0.2
0.55
0.0
8
16
32
64
128
256
512
1024
16
32
128
255
Attack Radius
Attack Steps
(a)
(h)
Figure 7: The changes of robust accuracy against different attack radius
(7a
and attack steps 7b
10.0
10.0
0.75
5.0
5.0
0.50
0.25
0.04
0.04
0.04
0 0.04
-n.0% 97.0%. FF..0o...0..
0.00
o.86o80 2.b.gge..00. b.0%.00 5.04
0.00
1..h 8.0.00 6.04
0.80
0.08 o.6.. 8.0.000
0.00
0.04
-0.0 -0.04
0.04
0.04
0.04
Non-robust model
2.00
1.00
3.75
2.0
2.0
0.8
1.50
1.5
1.00
1.25
1.0
0.4
0.25
0.04
0.02
0.02
0.02
0.00
-0.1 1...%
00
-0.06 08.98
-6.0 2o.0 2.00
-008
0.00
0.0%
-0. 04
6.0% -0.04
-0.04
1.02
0.006
(b) PGD-AT
1.00
2.2
2.2
2.0
18
2.0
1.6
0.75
0.50
0.07
0.02
0.02
0.00
-0.26 100g 2...888.....oo .6.4
0.02
96
08.0
00.02
26 Roo
0.02
-006
98.0% 1.0%)...
0.002
6.04
6.04
(c) ADR
Figure 8: Comparison of loss landscapes for the non-robust model (the first row), PGD-AT model (the second row), and ADR model (the third row). Loss plots in each column are generated from the same image chosen from the CIFAR-10. Following the same setting as Engstrom al. (0018 Chen et al. (2021), we plot the loss landscape function and v22 ~ Rademacher (0.5). The x and y axes represent the magnitude of the perturbation added in each direction and the axis represents the loss. i denotes the input and f'(.) is the trained model. The color represents the gradient magnitude the loss landscape clipped [0, 1] range, which conveys the smoothness of the surface.
(oss(x-77) + y-22), where
sign( V (i))
18

---

Published as a conference paper at ICLR 2024
numerical instability, be cireumvented. Though already AutoAttack Croce & Hein (2020b) which has been shown to provide a reliable estimation of robustness as an adversary throughout the experiments, bility of gradient obfuscation. Following the guidelines by Carlini et al. (2019), we examine the impact of changes the robustness with ResNet-18 trained by AT+ADR on CIFAR-10 against loc. perturbation.
conduct additional evaluations with ADR to eliminate the possi-
Unbounded PGD attack The unbounded PGD adversary should reduces model robustness 0%. Figure 7a shows the changes of PGD-100 robust accuracy of AT+ADR with different attack radius The robust accuracy for AT+ monotonically drops to 0% as the attack radius e increases.
Increasing PGD attack steps Increasing the number of attack iterations should only marginally lower robust accuracy. Figure (hjswwss shows the changes in robust accuracy AT+ADR with differ- ent steps of PGD attack. We can observe that the robust accuracy almost converges after K 16, more steps attack do not lead to lower robust accuracy.
Inspect loss landscape around inputs Figure g shows the loss landscape of Non-Robust, AT, and AT+ADR model around randomly selected examples from CIFAR-IO. Compared to the Non- Robust model, the adversarially trained models (both PGD-AT and AT+ADR) have flattened the rugged landscape, which does not exhibit the typical patterns of gradient obfuscation Engstrom notable that despite both PGD-AT and ADR having smooth landscapes, ADR has lower gradient magnitude (dark blue color in the figure), which implies the loss changes for AT+ADR is smaller than AT when small perturbations added to the input.
al. (2018).
F EXPERIMENT SETUP WHEN UTILIZING ADDITIONAL DATA
Following Gowal et ) Rebuffi et (202 we train Preact-ResNet15S (He al. 2016b) and WRN-34-10 with SiLU (Hendrycks & Gimpel 2016) as the activation function when utilizing synthetic data. We adopt rigorous experimental design tollowing (Gowal et al. 2020 Rebuffi al., 2021a), training the
8 (He et al. 2016b) and WRN-34- 10 architectures with SiLU (Hendrycks Gimpel. (0166) as the activation function when utilizing synthetic data. We leverage cyclic learning rates (Smith & Topin 2017) with cosine annealing (Loshchilov & Hutter] 2017) by setting the maximum learning rate 0.4 and warmup period of epochs, ultimately training for total of 400 CIFAR-100 equivalent epochs. Our training batch size is set to 1024, with 75% of the batch composed of the synthetic data. We maintain consistency with other experiment details
outlined in section 5.1
G COMPUTATION COST ANALYSIS
A standard 10 step AT includes 10 forwards and backward to find the worst-case perturbation when given a normal data point x. After we generate the adversarial data x', requires an additional forward and backward pass to optimize the model. We will need 11 forward and backward pass per iteration to conduct PGD-10 adversarial training. When introducing ADR to rectify the targets, an additional forward is needed for the EMA model. We need 12 forward and 11 backward pass in total.
We provide time per epoch for adversarial training in Table The experiment is reported by running each algorithm on a single NVIDIA RTX A6000 GPU with batch size 128. From the table, we can infer that the computation cost introduced by ADR is relatively small (1.83% ResNet-18 4.45% on WRN-34-10 on average) while the overhead brought by AWP is a lot higher (12.6% on ResNet- 18, 12.8% WRN-34-10 on average). We can achieve similar robustness improvement AWP with ADR with less computation cost required.
H VARIANCE ACROSS RERUNS
Table presents the results of five repeated runs for AT and the proposed defenses AT+ADR on 10 with R8sNet-18. Our findings indicate that the proposed ADR approach consistently
19

---

Published as a conference paper at ICLR 2024
Table 6: Computational cost analysis for ADR combining with AT techniques on CIFAR-10,
CIFAR-100 and mangeeageNet-2000 with ResNet-18 and WRN-34-10. We report the time required per epoch in seconds tested on a single NVIDIA RTX A6000 GPU.
Dataset
Architecture
Method
Time/epoch (sec)
AT
81
+ADR
83
ResNet-18
+AWP
92
AWP+ADR
93
CIFAR-I0
575
+ADR
610
WRN-34-10
+AWP
650
*WP+ADR
695
AT
81
+ADR
83
ResNet-18
+AWP
91
AWP+ADR
93
CIFAR-100
AT
586
+ADR
598
WRN-34-10
+AWP
661
+WW+ADD
672
584
+ADR
593
ResNet-18
+AWP
654
AWP+ADR
662
AT
4356
+ADR
4594
WRN-34-10
+AWP
4904
+WW++DDR
5127
Table 7: Variation in performance (%) of AT and ADR on ResNet-18 across reruns on CIFAR-10.
The robust accuracy is evaluated under the PGD-100 attack.
AT
AT+ADR
Robust Ace.
Standard Acc.
Robust Ace.
Standard Ace.
Run-1
52.80
82.52
55.13
82.41
Run-2
52.28
82.41
54.88
82.21
Run-3
52.74
82.39
54.92
82.18
Run-4
52.55
82.30
54.91
82.68
Run-5
52.63
82.31
54.73
82.31
Average 52.60
82.38
54.91
82.36
Standard Deviation
0.182
0.079
0.128
0.180
Table 8: Comparison of ADR with knowledge distillation methods CIFAR-100 with ResNet- 18.
Method
AutoAttack
Standard Acc.
ARD Goldblum et al.
2020b
25.65
60.64
RSLAD ZZi et al. 2021
26.70
57.74
IAD Zhu et 2022
25.84
55.19
MT (Zhang et al
2022
26.03
58.10
AT+ADR
28.50
57.36
20

---

Published as a conference paper at ICLR 2024
(TAAR-0//NIICIAA 100/OUT)
ADR
ADR
PGD AT
cornectty clasariee classifed
clean ssamples axamples
4000
Non Model
3.00
104
Count
107
2003
10
1000
0.0
0.5
1.0
1.5
2.0
2.175 2:200 2:225 2,200 2.275 2,300 Entropy
2.175 2.200 2.225
2.200
2.275 2.300
Entropy
Entropy
(a) Histogram output entropy on OOD data. All models are trained on CIFAR-10 and tested on CIFAR-100.
(b) Histogram of entropy distri- butions on the correctly classi- fied and misclassified examples for PGD-AT ADR model.
(c) Histogram of entropy distri- butions on the clean and adver- sarial examples for PGD-AT ADR model.
Figure 9: ADR model's output distribution exhibits similar properties as AT robust models
outperforms AT as evidenced by a higher mean (54.91% % for AT+ADR compared to 52.6 for AT) and lower standard deviation in robust accuracy (0.128% for AT+ADR compared 0.182% for AT). This suggests that ADR provides superior performance and greater stability in terms of robustness. While there is a larger standard deviation observed in standard accuracy when ADR combined with AT, we consider the variance to be within an acceptable range (0.18%). Our results demonstrate that ADR yields stable performance that is independent of random seeds or other initialization states.
I DISCUSSION ABOUT DIFFERENCE WITH KNOWLEDGE DISTILLATION BASED METHODS
Knowledge distillation in adversarial training generally requires pre-trained resources teacher models, on the other hand, we do not acquire anything in advance when using ADR. Therefore, is great advantage that ADR can achieve satisfactory results without additional resources. Fur- thermore, the phase of training a teacher model for knowledge distillation, the hard label still encourages the teacher to exhibit nearly one-hot output and may lead to overconfident results. It challenging to control the softness of the target when distillate to the student model because the teacher might output nearly one-hot distribution for some examples that have high confidence and rather smooth distribution for others. The bias and confidence from the teacher model might be in- correctly inherited by the student model. Instead of forcing the model to learn from one-hot ground truth, the noise-aware ADR label promotes the model to adapt label noise from the early training stage. We control the smoothness of the learning target by manipulating the À and T, as À con- trols the fraction of the ground truth class and т decides the smoothness of the noise composed in the rectified label. We provide additional results Table to validate that ADR superior to the knowledge distillation based methods.
J LIMITATIONS
In this work, we proposed ADR that employs a self-distillate EMA model to generate a finely cal- ibrated soft label enhance the robustness of models against adversarial attacks. However, we observe that the optimal parameters for each dataset vary. Thus, selecting appropriate parameters that suit the current training state crucial to ensure optimal performance. that while ADR demonstrates its efficacy in improving the performance of TRADES, the extent of improvement saturates when combined with other adversarial training techniques (WA, AWP) on fewer class datasets e.g. CIFAR-10. This outcome could be attributed to the fact that TRADES already promotes attacking and learning the data with soft targets generated by the trained model itself, and these additional techniques further smooth the model weight space. Thus, when the tar- get class is fewer, the improvement provided by ADR, which also emphasizes a smooth objective, becomes indistinguishable when all techniques are employed simultaneously.
also noteworthy
21

---

Published as a conference paper at ICLR 2024
0.55
0.50
Standard Acc.
ACC.
0.7
0.45
0.40
0.6
0.35
0.5
0.30
AT
0.4
+ADR
(standard pre-trained)
ATT ADR (standard pre-trained)
0.2
AT +DDR (AT pre trained)
0.20
AT ADR pre-traihedi
Epoch
Epoch
(a) Standard accuracy.
(b) Robust accuracy
Figure 10: The standard and robust accuracy training curve when transferring ADR from standard and adversarial mr-iriieed models.
AutoAttack
Standard Acc.
AT AT+ADR AT+ADR standard model initialized
48.81
82.52
50.38
82.41
49.45
82.22
AT+ADR + AT model initialized
50.11
83.66
Table 9; Transferring ADR from pre-trained checkpoints as initialization. The standard model is trained with the normal classification objective, and the AT model is pre-trained with PGD-AT.
K A CLOSER LOOK AT ADR MODELS' OUTPUT DISTRIBUTION
we see that the ADR model trained on CIFAR-10 is more uncertain than the model trained with PGD-AT when encountering out-of-distribution data (CIFAR-100), supports our claim that the robust model should not be over-confident when seeing something it has never seen before. From Figure (b]and and Figure 9c we can observe that the ADR-trained model exhibits similar output distribution as the model trained with PGD-AT (Figure 25 and Figure 2d), except that the ADR-trained model exhibits higher entropy levels general. The reason behind this phenomenon is that ADR builds sofer target as training labels, the output of the ADR model will also be smoother, resulting in higher entropy levels on average.
From Figure 9a
L ADR TRAINING TRANSFERRING FROM PRE-TRAINED INITIALIZATION
In this experiment, we leverage pre-trained models for the initialization of the ADR process. Specif- ically, we employ ResNet-18 on the CIFAR-I0 dataset, adhering all other parameter settings out- lined in Section 5.1 The key variation lies in the initialization of the model with either a standard or PGD-AT pre-trate pre-trained model, followed by subsequent ADR training. Our reporting in Table is based on the optimal checkpoint obtained during our comprehensive evaluation. Moreover, to offer a more comprehensive view of the observed trends, we present line charts depicting the performance trajectories with both clean and robust accuracy in Figure TO
The experimental findings reveal a consistent trend wherein ADR consistently outperforms the base- line in terms of robust accuracy, regardless of whether it is initialized with a pre-trained model or not. The utilization of either a standard model or a PGD-AT pre-trained weight as an initialization fails to further augment the maximum robustness achievable through ADR. Nevertheless, it is note- worthy that employing an AT pre-trained checkpoint results in a notable enhancement of standard accuracy by 1.25% when compared training with random initialization. This outcome underscores the potential for mitigating the accuracy-robustned tradeoff, indicating the feasibility of achieving improved performance by utilizing AT pre-trained model during the initialization phase.
22


Reviews:
[{"review_id": "uTeCsr53cW", "reviewer": "Reviewer_NJvp", "summary": "This paper proposes Annealing Self-Distillation Rectification (ADR), an improved adversarial training (AT) method that emphasizes the rectification of the labels used in AT. It is found that the outputs of robust and non-robust models are distributionally different in several aspects, and it is argued that labels rectified in a noise-aware manner can better reflect the output distribution of a robust model. Hence, the proposed ADR uses the interpolation between the one-hot labels and the outputs of an EMA teacher to produce the rectified distributions, which replace the one-hot labels used in existing AT methods. Experimental results suggest that ADR can achieve state-of-the-art robust accuracy.", "questions": "1. Are the robust models trained via ADR more conformed to the empirical findings in Section 3.2, as compared with vanilla AT?\n\n2. Should the rectified labels be assigned to adversarial images only, or both clean and adversarial images? Considering that different AT methods use different targets for clean and adversarial images (e.g., PGD-AT and TRADES), this can be an important question when one would like to apply ADR to other AT methods.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method ADR is intuitive and can be easily integrated into different AT methods.\n\n2. Experimental results suggest that ADR can achieve significant improvement over the baseline and superior robust accuracy to existing methods.\n\n3. The details of the method and the experiments are clearly stated, and the source code is also provided.", "weaknesses": "1. While Section 3.2 provides some insightful observations, it is not very clear how they are reflected by the design of the proposed ADR. Particularly, Section 4.1 mentions that robust models should \"generate nearly random probability on OOD data\" and \"demonstrate high\nuncertainty when it is likely to make a mistake\". However, there is no direct evidence (analytical or empirical) for how ADR may help achieve these properties. Instead, it seems that the motivation for ADR mostly comes from the previous works (like those cited in Section 4.1) that suggested the importance of label rectification in AT.\n\n2. In Table 3, it is shown that using DDPM data can improve the robust accuracy of WRN-34-10 trained via AT+ADR, but at the cost of a significant decrease in standard accuracy. This may be undesired since augmenting the training set with DDPM data can improve both clean and robust accuracy for AT according to (Rebuffi et al., 2021a). There should be some explanations or discussions on this issue.\n\n3. The texts in Figure 2 may be too small, which can be difficult to read when printed out.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j7GK7zComL", "reviewer": "Reviewer_EopP", "summary": "The paper investigates the phenomenon of robust overfitting and reports that robust models exhibit outputs more calibrated compared to standard models. It further proposes a label smoothing scheme for mitigating overfitting in adversarial training via employing model weight averaging, annealed interpolation and softmax temperature. Experimental results indicate robustness gains and a reduction of the severity of robust overfitting.", "questions": "- In Table 2, test accuracies of ADR combined with WA and AWP are presented. From the results it seems that each method contributes a (roughly) similar amount of robustness gain. Could the authors comment on whether they consider ADR, WA, AWP to be complementary?\n- In the motivation (subsection 4.1.) it is stated that the aim is 'to design a label-softening mechanism that properly reflects the true distribution'. In the presented approach the teacher network provides the probability distribution with which the one-hot label is interpolated. Can the authors comment on why the teacher network (model weight average of student) reflects the 'true distribution'?\n- The results presented in Figure 5 show the results of a hyperparameter study for the interpolation and temperature parameter. It is concluded that annealing both parameters is beneficial for robustness. Do the authors consider the differences (e.g. in the row on $\\lambda$ annealing) significant enough to draw such conclusions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The experiments performed are extensive and incorporates several baseline methods for comparison.\n- Modifying self-distillation EMA (weight averaging) with annealed interpolation and softmax temperature is an interesting idea.", "weaknesses": "- While the method is shown to increase the robustness and mitigates overfitting, other experimental results seem to lack significance for drawing conclusions (weight loss lanscape, effectiveness of temperature and interpolation parameters).\n- The relation to other similar methods investigated is not clear (see questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2LVt7hrlCH", "reviewer": "Reviewer_YNYz", "summary": "This work proposes a novel adversarial training scheme where the ground-truth labels are rectified with an EMA teacher network. The experimental results show that the proposed method achieves a better accuracy-robustness trade-off with smaller overfitting gaps than the baselines. The proposed method can also be integrated with other AT approaches and brings further robustness boost.", "questions": "Did you use any pretrained models for initialization? I wonder how ADR works when using pretraining models.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- I appreciate the simplicity of the proposed method. It should be easy to implement and it can be used with other AT techniques.\n    \n- The paper presents a clear motivation for the proposed method with experimental results. I also find the idea behind inspiring as it is somewhat consistent with my understanding of AT, i.e., easier training (less adversarial/weaker signal) leads to better results.\n    \n- It is shown that the technique brings small additional training time.\n    \n- The paper is well-written with nice figures and a clear structure.", "weaknesses": "- Considering the simplicity of the method, I think the experiments are not very extensive. More datasets (e.g., ImageNet), attacks, and especially baseline AT methods should be considered.\n    \n- The self-distillation brings high memory cost, which I believe would be a main limitation for the practical use of the proposed method.\n    \n- It seems that this work is not the first to reveal that robust models are better calibrated, yet the authors conclude this finding as one of the major contributions of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nhQEleA5O1", "reviewer": "Reviewer_VvcZ", "summary": "The authors have proposed a self-distillation method to generate soft labels in AT. The soft labels are crafted by the natural logits of the teacher model and the ground-truth. They clip the weight of soft labels to keep the correct classes having the highest confidence. Besides, Annealing and temperature are introduced to adjust the labels adaptive. Empirical evaluation on  three benchmark datasets shows its improvement on robustness based on baselines.", "questions": "Because it has little novelty, its performance is now the most significant evaluation indicator. Its performance on WRN is not competitive enough. I will increase my score if better results come out.\n\n=========After rebuttal=============\nThe authors' response address most of my concerns. I thus am willing to increase the rating to 6", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. A lot of experiments have been done to show its effectiveness on robustness compared with other knowledge distillation methods, which makes it credible.\n2. The paper is well-organized and easy to understand.\n3. The paper has shown abunbant experimental details, with good reproducibility.", "weaknesses": "1. The motivation is too easy and there seem no new insights in the manuscript.\n2. The results on Table 2 show that on WRN on three datasets, AT+ADR has a weaker performance compared with AT+WA, and AT+WA+ADR also has a weaker performance compared with AT+WA+AWP. It may demonstrte ADR doesn't work well compared with WA and AWP on large models.\n3. The comparison with RobustBench in the paper is not fair, results on AT+ADR rather than AT+WA+AWP+ADR should be used. In this comparison, ADR achieves atate-of-the-art robust performance on ResNet-18 but has a poor performance on WRN.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)