Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training
Authors: Shruthi Gowda
Bahram Zonooz
Elahe Arani
Venue: ICLR 2024 poster (2024)
Keywords: Adversarial training
Adversarial Robustness
Generalization
Robustness
Robust overfitting
Selective training
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: A selective adversarial training method that enhances the trade-off between standard and robust generalization while also mitigating robust overfitting.

Abstract: Adversarial training improves the robustness of neural networks against adversarial attacks, albeit at the expense of the trade-off between standard and robust generalization. To unveil the underlying factors driving this phenomenon, we examine the layer-wise learning capabilities of neural networks during the transition from a standard to an adversarial setting. Our empirical findings demonstrate that selectively updating specific layers while preserving others can substantially enhance the network's learning capacity. We, therefore, propose CURE, a novel training framework that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights. Importantly, CURE is designed to be dataset- and architecture-agnostic, ensuring its applicability across various scenarios. It effectively tackles both memorization and overfitting issues, thus enhancing the trade-off between robustness and generalization and additionally, this training approach also aids in mitigating "robust overfitting". Furthermore, our study provides valuable insights into the mechanisms of selective adversarial training and offers a promising avenue for future research.

Published as a conference paper at ICLR 2024
CONSERVE-UPDDAT-REEIIS TO CURE GENERALIZA- TION AND ROBUSTNESS TRADE-OFF IN ADVERSARIAL TRAINING
Shruthi Gowda'2, 12, BBhhra Zonooz 2.3 & Elahe Arani" 2.4
Navinfo Europe Eindhovve University of Technology Toomoom *Wayve www.ggoau......m. nn, b. p.zonooz@tue.n.. nn, e a.arani eraani@gmail.com com
ABSTRACT
Adversarial training improves the robustness of neural networks against adversarial attacks, albeit at the expense of the trade-off between standard and robust gener- alization. To unveil the underlying factors driving this phenomenon, the layer-wise learning capabillitiss of neural networks during the transition from a standard to adversarial setting. Our empirical findings demonstrate that selec- tively updating specific layers while preserving others can substantially enhance the network's learning capacity. We therefore propose CURE, novel training framework that leverages gradient prominence criterion to perform selective conservation, updating, and revision of weights. Importantly, CURE is designed to be dataset- and architecture-agnosii scenarios. It effectively tackles both memorization and overfitting issues, thus enhancing the trade-off between robustness and generalization and additionally, this training approach also aids in mitigating "robust overfitting". Furthermore, our study provides valuable insights into the mechanisms of selective adversarial training and offers a promising avenue for future research.
examine
ensuring its applicability across various
INTRODUCTION
The susceptibility of deep neural networks (DNNs) to adversarial attacks (Szegedy et al., 2014
Goodfellow et al. 2015) continues to present a substantial challenge in the field. Adversarial training has emerged as a promising strategy to enhance the robustness of DNNs against adversarial attacks (Madry et al. 2018 Zhang et al.) 2019 Tramer et al. 20188) Wang et al. 2019). However, transitioning from standard training with natural images to adversarial training introduces distinct behavior patterns.
Despite the benefits of adversarial training in improving robustness, it often results in compromised performance on clean images, creating a noticeable trade-off between standard and adversarial generalization (Raghunathan et al. 2019). Another intriguing observation is that, in contrast to the standard setting, longer durations of adversarial training can paradoxically lead to reduced test performance. This generalization gap in robustness between training and testing data, commonly referred to as robust overfitting (Rice et al. (220)), is prevalent in adversarial training. Therefore, is imperative to gain a deeper understanding of the underlying factors driving these behaviors to advance the development of reliable and trustworthy Al systems.
Few studies have attempted to understand learning behavior in an adversarial setting. Nakkiran (2019) show that the capacity of the network needs to be increased for better generalization, while others have found that this is not always the case (Huang et al. (202T) Additionally, suggestions include the need for larger amounts of data, labeled (Schmidt et al. 2018) or unlabeled (Carmon et al.) 2019 Najafi et al.) 2019). for better performance. Other works have revealed that the discord between natural and ad versarial accuracy is caused by differently trained feature representations (Tsipras et al. 2018 llyas et al. 2019). However, numerous aspects of neural network learning behavior when confronted with adversarial samples remain unexplored. Exploring how networks adapt to both natural and adversarial data can yield valuable insights into their learning dynamics and capabilities.
"Contributed equally.
'The code is available at: httpp::/// http:////gggtt.b.cc/////l/ll-l//

---

Published as a conference paper at ICLR 2024
Our study empirically investigates neural network learning capabilities during the transition from standard to adversarial settings. Rather than considering the entire network as a unified entity, we seek to identify fixable weights that maintain performance for learning adversarial data without a significant accuracy drop in the other distribution. Layer- wise analysis of network properties regarding weight updates demonstrates that not all weights need modification for effective learning. Selectively updating specific weights while preserving information in others can achieve a better performance balance on standard and adversarial distributions.
natural data and updatable weights
Inspired by the findings of the empirical analysis, we propose novel scheme for adversarial training characterized by an adaptive and dynamic parameter update mechanism. By assessing the significance of gradients, our method enables the network to retain valuable information in specific weights while updating only those necessary for the data distribution. We introduce novel Robust Gradient Prominence which guides the decision-making process regarding which parameters to update and which to keep fixed. Our proposed method, CURE, consists of conservation (maintaining subset parameters for retention of natural knowledge), updation (modifying a subset of parameters learn adversarial knowledge), and revision (consolidating knowledge periodically).
CURE (Ours)
87
FAT
86
Natural Acc
AAS-AT
85
ATRADES
84
MART
CURE achieves better trade-off between standard and robust generalization compared to the state-of-the-art (SOTA) meth- ods on different architectures and datasets (Figure D]. Notably, two existing research areas focused on enhancing the trade-off and mitigating robust overfitting often do not synergize. For in- approaches like TRADES (Zhang al. 2019), intended to enhance the trade-off, still contend with robust overfitting issues. Conversely, adversarial weight pertur- bation methods (Wu et al., 2020). designed to mitigate robust overfitting, exhibit lower natural accuracy, thus indicating reduced overall generalization. In contrast, CURE not only achieves an im- proved trade-off but also effectively reduces overfitting, surpassing methods explicitly designed to address this issue. Additionally, models trained using CURE exhibit higher robustness against adversarial attacks, requiring stronger and more semantically meaningful attacks to deceive the network than other methods (Figure [8. Overall, our proposed approach offers promising direction for understanding the broader concepts of generalization and robustness in DNNs and paves the way for improved model performance in real-world scenarios.
83
IAD
48
52
56
58
Robust Acc (C&W)
stance, regularization-basse
Figure I: Generalization robust- ness trade-off NeteResNet- 34-10 and CIFAR-10. CURE displays
better trade-off between standard and robust (C&W) performance.
2 BACKGROUND
Standard Training: Given natural dataset D with input and label pairs (x, E D, standard training (ST) involves minimizing the objective function with respect to the model parameters 8. Deep neural networks trained on clean or natural data exhibit high generalization However, the network generates incorrect predictions if the natural input is imperceptibly modified.
test data.
ST:
min E( Co y)
Adversarial examples are perturbed inputs that result in misclassifications. The perturbations are carefully crafted imperceptible noise that, when added to the inputs, visually results similar image but can fool the neural network. For an input and label y, an adversarial sample can be generated by adding a perturbation 8 such that,
8 = arg max Co(a + 5,
where A is usually a simple convex set, such as an Ep ball.
Adversarial Training: AT has emerged one of the effective techniques to make neural networks more robust and less susceptible to such attacks. Adversarial examples are generated during training to improve predictions these images to produce robust network. Standard Adversarial Training
2

---

Published as a conference paper at ICLR 2024
100
B-1 B-2
B-3
B-4
Natural Acc
Robust Acc
80
60
40
20
0
ST
2
3
12
23
123 234 SAT
Retained (Nat)
Updated (Adv)
Updated blocks (U-)
(a) The four blocks of ResNet-18 are considered for the layer-wise study. All layers are trained with natural images (ST) first. The second shows the example architecture for U-23, where the first and last blocks are frozen and the second and third are updated while training adversarially. (b) Standard generalization and robustness of different blocks of ResNet- 18 on CIFAR-10 dataset
Figure 2:
(AT) et al. 2018) is formulated min-max
max optimization problem,
AT: 0" = (nin) max Lo(x + 6.
BEA
where the inner maximization generates strong adversarial examples for each input during training, and the outer minimization trains the model on these adversarial examples to improve robustness.
Several AT techniques have been proposed that include both natural and adversarial images during training. Adversarial regularization methods (Goodfellow et al. (015)) include regularization ob- jectives in addition to classification loss. TRADES (Zhang al. 2019) introduced a regularization objective between the probabilities of natural and their adversarial counterparts and MART (Wang al. 2019) adds an emphasis on the influence of miselassified examples. Multi-model approaches utilize more than one network or ensemble-hase scmi-/unsupervised
training techniques to improve the trade-off. Many haaming-based AT techniques (Schmidt et al. 2018)) have emerged. These tech- niques leverage additional data, increased network capacity, or network ensemble strategies and do not examine the learning patterns to perform adaptive training.
Robust Overfitting: In contrast to ST, where prolonged training leads to near-zero training errors and decreased test loss, adversarial training exhibits distinct behavior characterized by an increase in test errors beyond specific threshold, referred to as "robust overfitting" (Rice et al.) 2020). This challenging issue has proven resistant to various conventional mitigation methods, including regularization and data augmentation. Notably, recent advances have introduced several strategies for alleviating robust overfitting in DNNs, including early stopping Rice et al. (2020). semi-supervised learning (Carmon et al., 2019), as well as data interpolation techniques (Lee al. 2020 Chen et al.)
2021). Furthermore, alternative approaches leverage sample re-weighting (Zhang et al. 2021). weight
perturbation (Wu et al. 2020) and weight smoothing techniques (Chen et 2020
Hwang al.
2021 to combat robust overfitting.
3 LAYER- WASE CONSERVATION AND UPDATION: EMPIRICAL ANALYSIS
Despite extensive research efforts addressing adversarial training, a persistent knowledge gap ham- pers our comprehension of the network's learning behavior during the transition from standard to adversarial training. This limited understanding hinders the efficacy of proposed solutions, often neglecting the unique learning capacities of network layers. Understanding how weight initialization and updates affect the loss landscape and convergence is critical, particularly as the network adapts to diverse data distributions. A layer-wise network analysis can help identify which weights are suited for retaining information and which possess greater learning capacity. Re-initialization has been explored in ST (Li et al. 2020 Alabdulmohssnn et al., 2021). but its effect on AT has not been thoroughly investigated. We perform selective freezing, re- re-initialization, and update of specific weights to gain deeper insight into the network's 's behavior. By examining these factors, we can uncover valuable insights that further enhance the effectiveness of training strategies.
3

---

Published as a conference paper at ICLR 2024
ST
NonRobusst-Rtus
NonRobust- Robust
0.8
0.6
04
U-34
U-123
U-234
02
0.0
Layer
Layer
Figure 3: Representation similarity between robust and nob-robust features in ResNet- 18 trained on the CIFAR-10 dataset.
Setup:
We investigate the layer-wise properties of a network by first training the standard way on natural images and then transitioning to the adversarial setting. We reinitialize different layers in each experiment while keeping the rest of the network fixed. We utilize ResNet- as the network, with four blocks and an additional linear classifier layer. The notation U-b represents the update of block "b" while keeping the rest of the network frozen. We create 12 different combinations (U-1, U-2, U-3, U-4, U-12, U-23, U-34, U-123, U-234). The example of U-23 is shown in Figure More details of the empirical study are provided Appendix, Section
3.1
ADVERSARIAL ROBUSTNESS
The natural and adversarial accuracy of multiple models on the CIFAR-IO dataset is shown in Figure 2(b). The "ST" and 'SAT" represent the accuracy of standard training and adversarial training (Madry al. 2018) respectively. Our study on the learning behavior of neural networks during the transition from ST to AT uncovers intriguing properties of the layers. The results show that training only the early layers of a network adversarially can cause reduced performance on both data distributions. As new distribution, the existing information undergoes a certain degree compromise, characterized as overwriting. Notably, we observe that updating specific layers exert more pronounced impact accuracy. Updating the early layers (U-1) results in decrease in accuracy, indicating that enhancing the trade-off is not facilitated by updating these layers. In contrast, making later layers trainable leads to milder decline in accuracy. This is attributed to the fixed representations learned in the earlier layers aiding the subsequent layers to adapt to the new distribution, reflecting increased learning capacity. These findings suggest that training the entire network without selectively updating the parameters may not be optimal for training.
the network acquires knowledge
3.2 REPRESENTATION ALIGNMENT
Visualizing the features learned on natural and adversarial data and examining the differences helps to understand the representation alignment. We employ representation similarity metric, Centered Kernel Alignment (CKA) (Korbblth et al. 2019). Figure visually depicts the similarity between natural and robust representations for each model. In ST. observe significant dissimilarity in representations, indicating the inherent challenge for naturally trained models to perform well on adversarial data. In contrast, AT results in similar representations in the early layers but divergence in the final layer, driven by perturbations designed to alter decision boundaries. Models that selectively update primarily the final layers exhibit greater similarity and fewer block structures in their represen- tations. Optimizing parameters to promote alignment and learn generic representation between the distributions holds promise for achieving better balance.
3.3 ROBUST OVERHITTING
Analysis of adversarial test accuracy over the course of long training reveals consistent trend of declining accuracy beyond a certain point. Figure 4 Iillustrates the test accuracy curves for all models. selectively update early layers,
The base (AT) model prominently exhibits overfitting. When observe a reduction in overfitting tendencies, albeit at the cost of lower overall accuracy (U-1, U-2, and U-12). The middle layers play important role in balancing between reducing overfitting and
4

---

Published as a conference paper at ICLR 2024
50
Test Acc (Adv)
10
40
60
30
30
30
AT
U:22
AT
U/234
15
0-3
15
U-34
15
U-345
200
100
Epochs
Epochs
Epachs
Figure 4: Adversarial test accuracy during training after layer-wise updation and conservation of ResNet-18 18 blocks on CIFAR-10 dataset.
having higher test performance, and the U-23 model stands out with relatively stable test accuracy. This suggests that selectively updating the layers plays role in mitigating robust overfitting while maintaining competitive performance levels.
The empirical findings presented in this study underscore the significance of dynamic updation in neural networks and its impact on overall performance, thus opening up new possibilities for developing more effective AT methods.
PROPOSED METHOD
Hypothesis: Our empirical study on neural networks exposed to adversarial examples has led to several key insights. We postulate that the traditional approach of training the entire network unit and updating all weights is suboptimal when learning different data distributions. Instead, selectively updating certain weights and conserving others can lead more effective utilization of the network's 's learning capabilities, By considering the retention and learning capabilities of the network, we can achieve better balance between natural and adversarial robustness, reduce robust overfitting, and promote increased representation similarity between robust and non-robust features. Based on these findings, we propose new AT method incorporating these considerations for enhanced performance.
Conserve-Update-Ree Revise: Our method is based on three key components: Conservation (of knowl- edge from natural data), Updation (of knowledge from adversarial data), and REvision (of consol- idated knowledge) CURE. Instead of manually fixing and updating the layers, as in our initial empirical analysis, we devise a technique to dynamically update the weights of the network each layer based importance criterion.
Consider a network f and classifier parameterized by 0. (Xmatt v) D represents the natural images and labels. The network is trained natural images using the standard training schema. We use this pre-trained network and switch to the adversarial training setting. Adversarial samples Fade generated during training for each clean sample naa by adding perturbation 8*. For training in the adversarial setting, we utilize a classification loss to improve the performance on natural images and adversarial regularization loss to encourage the network produce similar probabilities for both natural and adversarial samples. As seen in Figure 3 a better feature similarity between natural and adversarial distributions can result in robust models. To minimize the distance between the predictions of these distributions, we utilize the KL-divergence objective;
8* = max D (p(#nat; ()]lp( +5; 5:01)]
(4)
SEA
Lado LCE (Xnati 0) +
(D))+DILppppee D ()(pp(aadv. 0)).
(5)
Instead of updating all the weights, we want to update a subset with the most significant impact on accuracy. Conserving some of the weights allows for more learning capacity, thus helping avoid memorization and overwriting and achieving a better trade-off. To increase performance on both distributions, introduce novel "Robust Gradient Prominence" objective to decide the weights to be updated. The idea is that the feature representations that have the most impact on the model's predictions will have greater influence the gradients of the model parameters. Thus, the gradient
5

---

Published as a conference paper at ICLR 2024
prominence score can help identify the essential weights whose activation representations can impact the predictions. When dealing with two different distributions, gradient prominence can measure which weights contribute more to the joint distribution of both natural and adversarial accuracy and which can skew the results.
Robust Gradient Prominence (RGP) aims to achieve this by utilizing the gradient importance of both natural and adversarial samples. RGP dynamically determines which weights to update and which ones freeze in each layer of the network.
RGP(w) = =x 0C (Inat; 0) + aa)
d ( 0)
Ow
aw
where [( 0) and L(Iadu 0) are the cross-entropy losses on natural and adversarial examples, respectively. helps balance the importance between natural vs. adversarial distributions.
Based on the RGP score, a gradient mask is created by removing a fraction (p% of weights with a low prominence measure. The gradient mask is applied update only the most important weights. RGP ensures that we (1) conserve past knowledge by preventing fraction of weights from updating, (2) learn new knowledge by updating weights that have the capacity to learn without overwriting, and (3) utilize the network's 's capacity efficiently to achieve balanced performance.
During training with natural and adversarial images, the model's performance may lean towards one of the distributions at different points in the training process. We introduce the revision stage to consolidate knowledge across all these patterns and obtain more balanced performance. A revision model is updated with stochastic momentum update (SMU) of the training model a revision rate r and a revision decay factor r, Oree f- SM d).
frev d' Orev + (1 d) 0 if sample (0, <.
Stochastic updates to the revision help accumulate knowledge throughout the training, and this consol- idated information is shared with the training model. The adversarial training receives additional boost from this knowledge revision the accumulated knowledge. A consistency regularization objective regularizes the knowledge flow from the revision model to the training model.
LCR = Drc. (p(Enat 0rec)) (pp(tai) 0)) + Dr.. (p(Eadu Orew)) Oreu))p((Zndo) 0))-
(8)
The overall training objective is L Ledo + CCR- Algorithm is detailed in Appendix, Section A
CURE improves adversarial training by conserving weights with lower RGP scores, updating weights that are more accountable for better performance on both natural and adversarial data, and additionally incorporating a knowledge revision from the consolidated and stable model.
5 RESULTS
We benchmark our method on various architectures, datasets, and attacks. We introduce the Natural-Robustnes Ratio (NRR) metric to quantify the trade-off between generalization and robustness. All experimental details, hyperparameters, metrics, and analysis details are provided
Experimental Sctup:
in Appendix Sections and
The performance of CURE against various SOTA methods on CIFAR-10 dataset on two different architectures is tabulated in Table 'Nat" denotes accuracy on the clean set, and evaluations include PGD20, C&W, and Auto/Atacks (AA). The table also features methods that incorporate more than one model in their training framework (the second part). Notably, CURE achieves the highest NRR among all methods, showcasing superior performance in balancing robustness and natural accuracy. For instance, when considering the WideResNer-344 -4-10 model, CURE outperforms TRADES by 2.5% PGD, FAT by 10% on AA, and mullipple-tacche--aseee technique LAD by 2.3% on C& W attacks, all while maintaining the highest natural accuracy.
Further, we present the performance of the ResNet-18 network both CIFAR-100 and SVHN datasets in Table 2 CURE outperforms all other methods, even in the face challenging attacks like PGD with higher steps and C& W. When subjected to the C&W attack SVHN, CURE exhibits relative improvement of 10% over TRADES, 22% over MART, and 8% over ST. while maintaining the highest natural accuracy, surpassing TRADES by 3%. MART by 5% and ST by 2.3%.

---

Published as a conference paper at ICLR 2024
Table I: Performance comparisons on the CIFAR-I0 dataset encompassing various architectures and attacks. The second part highlights methods that employ multiple networks in their approach. NRR quantifies the trade-off between natural performance and C&W attack results.
WideResNet-34
ResNet-18
Method
Nat
PGD20
AA
C&W
NRR
Nat
PGD20
A.A
C&W
NRR
AT
ICLR118
85.17
55.08
44.04
52.91
65.27
82.78
$1.30
44.63
49.72
62.12
TRADES
ICML'19
84.73
56.82
52.95
54.29
66.17
82.41
52.76
48.37
50.43
MART
ICLR
83.62
56.74
51.23
53.16
64.99
80.70
54.02
47.49
49.35
61.24
ICML/20
86.60
49.86
47.48
49.35
62.87
87.72
46.69
43.14
49.66
63.41
ST-AT
ICLR'23
84.92
57.73
53.54
83.10
54.62
50.50
51.43
63.53
ACT
20
87.10
54.77
84.33
55.83
ARD
AAAU20
85,18
53.79
82.84
51.41
IAD
ICLR'22
83.06
56.17
52.68
53.99
65.44
80.63
53.84
50.17
51,60
62.92
LAS-AT
CVPR'22
85.24
57.07
53.58
55.45
67.19
82.39
53.70
49.94
51.96
63.72
CURE
87.05
58.28
52.10
55.25
67.60
86.76
54.92
49.69
52.48
65.04
Table 2: Performance comparisons encompass various datasets and attacks on the ResNet-18 architec- ture, evaluating both natural and robust performances. NRR quantifies the trade-off between natural performance and C&W W attack results.
CTFAR 100
SVHN
Method
Nat
PGD20
PGDI00
C&W
NRR
Nat
PGD20
PGD100
C&W
NRR
AT
ICLR'18
57.27
26.66
26.29
24.89
34.69
89.21
51.18
50.35
48.39
62.74
TRADES
57.94
29.25
29.10
25.88
35.77
90.20
54.49
54.18
52.09
66.04
MART
ICLR'20
55.03
28.25
28.10
26.60
35.86
88.10
54.70
54.13
48.95
61.25
FAT
ICML220
61.61
18.35
17.98
19.31
29.40
HAT
ICLR/21 21
58.73
27.92
24.60
34.67
92.06
$7.35
54.77
68.67
ST-AT
ICLR'23
58.44
30,53
30.39
26.70
36.65
90.68
56.35
$6.00
52.75
66.69
CURE
60.72
30.81
29.82
27.95
38.27
92.77
61.56
58.73
57.34
70.87
CURE employs an innovative sparse updating technique, which not only augments the network's learning capacity but also facilitates the learning of common representation for both natural and adversarial data distributions. By retaining and selectively updating weights, we strategically avoid excessive overwriting, and ensure the preservation of previously learned natural information, thus enabling a better adaptation to adversarial data. Further, the brief revision stage in our method further improves performance by stochastically sharing stable consolidated knowledge during training. These compelling findings underscore the remarkable efficacy of CURE in striking an optimal trade-off between robustness and natural accuracy.
Generalization and Robustness Trade-Off. The trade-off between generalization and robustness is illustrated in Figures 5]). Traditional approaches tend to prioritize robustness the expense of natural accuracy. In contrast, CURE presents a balanced method that facilitates learning from standard and perturbed data. By selectively conserving weights and freeing up network capacity for learning on the current distribution, CURE ensures that natural accuracy does not significantly decline while learning adversarial samples. This approach effectively maintains better equilibrium between robustness and generalization, offering more desirable solution for robust training.
Attack Strengths. To evaluate the effectiveness of CURE under varying levels of perturbation, we conducted experiments using a PGD-20 attack with increasing perturbation strengths. The value ranges from 0.25/ /55 to 8/ 8/255. In Figure (b), the performance of AT and TRADES monotonically decreases and drops at higher perturbation, while MART exhibits relatively stable performance but also experiences a decline at higher perturbation levels. CURE consistently outperforms all other methods across all perturbation strengths, demonstrating its superior robustness.
Gradients Analysis. We gain insights into the retention and updating processes by visualizing gradients across multiple layers of the network during training. Figure (oa) presents the mean of the gradients of the "conv2" layer within each of the four blocks of the ResNet-18 architecture. Remarkably, after the application of CURE, the gradients exhibit significantly lower magnitudes. This
7

---

Published as a conference paper at ICLR 2024
Adversarial (validation)
55
86
Standard Acc
85-
Acc
50
Adversaria
70
45
84
Adver Porrsa
40
60
AT
.48
TRADES
MART
35
MART
CURE
30
-46
AT
TRADES MART CURE
0.25
0.5
1.0
2.0
4.0
25 50 75
150175000
(b)
(c)
Figure 5: (a) Generalization and robustness trade-off; (b) Performance across different perturbation strengths; (c) Robust overfitting, on CIFAR-10 dataset. CURE achieves a hetter trade-off, and shows consistent robustness across increasing levels, while also mitigating robust overfitting.
Layer1.1 Conv2
Layer2.1 Conv2
Intitial training stage
M
1c:03
1e:03
60
CURE
Gradlent magnitude (mean)
Be:04
be:04
Se-04
gradients updated
20
36.04
Da-00
0c+00
L.yer3.1 Conv2
Layer4.1 Conv2
Final training stage
3e:04
60
Be-04
04
5c 04
1c 04
20
0e100
De400
500 299
Training iterations
Training iterations
Figure 6: (a) Mean of the gradient magnitudes while training baseline and CURE: (b) Percentage of gradients updated in each layer (layer'x' "o'n'""oo''"""" "onv'y'''weighh" during initial and final phases of training.
suggests that the training mechanism entails less abrupt gradient updates at each iteration, resulting in smoother training that avoids overfitting to individual perturbed and noisy samples. Figure (
illustrates the percentage of gradient updates per layer during the initial and final stages of training. In the early stages of training, substantial number of weights undergo updates when exposed to adversarial data. However, as training progresses, the RGP metric identifies the weights that need to be fixed prevent overwriting. Interestingly. it highlights the significance of certain middle and final layers, aligning with our empirical observations. This controlled training allows for the update of only relevant weights, maintains the knowledge learned from natural data, and adapts acquire more stable representation for both natural and adversarial data distributions. The strategy of sparse updating has ultimately led enhanced trade-off and also reduced robust overfitting.
ROBUST OVERFITTING
Robustness against overfitting crucial concern when it comes adversarial training. As training progresses, the network's 's robust performance on test data declines, in contrast to the typical behavior observed in standard training (Rice al.) 2020). This implies that the network memorizes adversarial examples instead of generalizing them to unseen examples. This phenomenon can be seen in Figure 5tc), where methods are trained for an extended period (the same as ST). traditional methods, such AT or TRADES, the test accuracy for adversarial data increases to certain point and then decreases. However, CURE effectively mitigates this overfitting issue, resulting in a consistently increasing accuracy, similar to ST.
These observations are further supported by the results presented Table 3 CURE achieves the highest test accuracy the end of training, indicating A value of 0, which signifies no decline in performance. outperforms all other SOTA methods (with a negative A), which are specifically designed address the issue of robust overfitting. The sparse update technique in CURE not
8

---

Published as a conference paper at ICLR 2024
Table 3: Robust Overfitting comparison against several methods the CIFAR-10 dataset.
PreActResNNt-18
WideResNet 34-10
Method
Natural
AA
Natural
AA
Best
Last
A
Best
Last
Best
Last
Best
Last
AT
ICLR'18
82.50
83.99
1.49
48.21
42.46
-5.75
85,90
86.63
0.74
53.42
48.22
-5.20
TRADES
ICML719
81.19
82.48
1.29
49.03
46.80
-2.23
84.65
53.08
AWP
NeuriPS'20 20
83.33
84.39
1.06
50.57
49.95
-0.62
85.57
54.04
KD
IC
84.51
85.40
0.89
49.87
49.72
-0.15
86.81
87.06
0.25
TE
ICLR/22
82.35
82.79
0.44
50.59
49.62
-0.97
IDBI
ICLR/23
83.96
84.92
0.97
50.74
49.99
-0.75
88.61
89.12
0.52
55.83
54.01
-1.82
CURF
86.54
86.54
0.00
50.23
50.23
0.00
87.05
87.05
0.00
52.10
52.10
0.00
only achieves an improved trade-off but also offers the remarkable outcome of reducing robust overfitting. The selective conservation and updation approach allows for better learning of the two data distributions and reduces overfitting to the training data. This is significant advantage, as robust overfitting can be major hindrance when training on complex datasets that require longer training. Unlike methods, that rely on time-consuming validation testing and early stopping techniques to achieve optimal accuracy, our approach offers not only enhances the network's 's robustness but also substantially reduces robust overfitting, effectively addressing the two primary challenges associated with adversarial training.
effective training process. In summary, CURE
ROBUSTNESS AGAINST NATURAL CORRUPTIONS
DNNs are also susceptible to natural corruptions, which include distortions in input data that occur in real-world scenarios (Hendrycks & Dietterich 2018). The inconsistent predictions under such conditions can lead undesirable outcomes, par- ticularly in safety-critical applications. To evaluate the robustness of our models against these corrup- tions, we simulate dataset by introducing vari- ous types of corruptions (fifteen different types).
60
AT
TRADES
55
MART
CURE
Robustness
50-
45
Figure 7 demonstrates the efficacy of CURE in addressing various types of corruption, including blur, digital, noise, and weather. CURE achieves improved resistance to images affected by multiple conruptions. Our method focuses striking bal- ance between retaining acquired knowledge from one distribution and learning a different distribu- tion, thereby enhancing the stability and robustness of the model. These findings highlight the poten- tial of CURE as a promising solution to enhance robustness of neural networks the presence of diverse corruptions.
40
35
30
Blur
Digital
Noise
Weather
Figure 7: Robustness against various natural corruptions (at severity level 3) on CIFAR-10 dataset.
CONCLUSION
We present novel method, denoted CURE, for adversarial training that addresses the challenge of achieving robustness without compromising generalization. Through a selective weight preservation and updating strategy, CURE enables neural networks to learn effectively from both natural and adversarial data distributions. Empirical evaluations across diverse datasets, architectures, attack types, and strengths consistently establish the superiority of CURE, showcasing its enhanced performance in terms of robustness and standard generalization. Importantly, it effectively mitigates the issue of robust overfitting, a longstanding challenge in adversarial training, while also demonstrating greater robustness natural data corruption. In essence, work not only sheds light the intricate mechanisms of adversarial training but also propels the field towards an intriguing avenue selective training schemes.

---

Published as a conference paper at ICLR 2024
REFERENCES
Ibrahim Alabdulmohsin, Hartmut Maennel, and Daniel Keysers. The impact of reinitialization on generalization in convolutional neural networks. arXiv preprint arXiv:2109.000 2021.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pp. 484-501. Springer, 2020.
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Adversarial concurrent training: Optimizing robustness and accuracy trade-off of deep neural networks. In The 31 British Machine Vision Virtual Conference, 2020.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium security and privacy (sp), pp. 39-57. Icce, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John Duchi, and Perey Liang. Unlabeled data improves adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
Chen Chen, Jingfeng Zhang, Xilie Xu, Tianlei Hu, Gang Niu, Gang Chen, and Masashi Sugiyama. Guided interpolation for adversarial training. arXiv preprint arXiv:2102.07327, 2021.
Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, and Jingjing Liu. Efficient robust training via backward smoothing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 6222-6230, 2022.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang. and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. International Conference on Learning Representations, 2020.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference Machine Learning, pp. 2195-2205. PMLR. 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustmess with an ensemble of diverse parameter-Fi attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020b.
Yinpeng Dong, Ke Xu, Xiao Yang. Tianyu Pang. Zhijie Deng, Hang Su, and Jun Zhu. Exploring memorization in adversarial training. In International Conference on Learning Representations, 2022. /
n for.mpp.-=77E999/CP..
Jonathan Frankle, David J Schwab, and Ari S Morcos. Training batchnorm and only batchnorm: On the expressive power of random features in enns. In International Conference Learning Representations, 2020.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3996-4003,, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. stat, 1050:20, 2015.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 201
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual net- works. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings. Part IV 14. pp. 630-645. Springer, 2016b.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2018.
10

---

Published as a conference paper at ICLR 2024
Hanxun Huang, Yisen Wang. Sarah Erfani, Quanquan Gu, James Bailey, and Xingjun Ma. Explor- ing architectural ingredients of adversarially robust deep neural networks. Advances in Neural Information Processing Systems, 34:5545-5555 5559, 2021.
Joong-wor won Hwang, Youngwan Lee, Sungchan Oh, and Yuseok Bae. Adversarial training with stochastic weight average. In 2021 IEEE International Conference on Image Processing (ICIP), pp. 814-818. IEFE, 2021.
Andrew Hyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: Adversarial training with learnable attack strategy. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 13398-13408, 2022.
Simon Korbblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International Conference on Machine Learning, pp. 3519- 3529, PMLR, 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Sachyung Lee, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adversari- ally robust generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 272-281, 2020.
Lin Li and Michael W. Spratling. Data augmentation alone can improve adversarial training. In The Eleventh International Conference on Learning Representations, 2023. URL https://
openreview. net nett/forumidd=/AuuNNNC
Qizhang Li, Yiwen Guo, Wangmeng Zuo, and Hao Chen. Squeeze training for adversarial robustness.
In The Eleventh International Conference on Learning Representations, 2023. URL https
/oopenreview net. htt////ormmmd/d/// tmm
Xingjian Li, Haoyi Xiong, Haozhe An, Cheng-Zhong Xu, and Dejing Dou. Rifle: Backpropagation in depth for deep transfer learning through re-initializing the fully-connected layer. In International Conference on Machine Learning, pp. 6010-6019. PMLR, 2020.
Jiang Liu, Chun Pong Lau, Hossein Souri, Soheil Feizi, and Rama Chellappa. Mutual adversarial training: Learning together is better than going alone. IEEE Transactions on Information Forensics and Security, 17:2364-2377, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. Advances in Neural Information Processing Systems, 32, 2019.
Preetum Nakkiran. Adversarial robustness may be at odds with simplicity. arXiv preprint 1rX1v:1901.00532. 2019.
Rahul Rade and Seyed-Mohsen Moosavi-Dezfoolii Reducing excessive margin to achieve better accuracy vs. robustness trade-off. In International Conference on Learning Representations, 2021.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang. John Duchi, and Percy Liang. Adversarial training can hurt generalization. In ICML 2019 Workshop on Identifying and Understanding Deep Learning Phenomena, 2019.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark the 2017.
robustness of machine learning models. arXiv preprint
11

---

Published as a conference paper at ICLR 2024
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pp. 8093-8104 8104. PMLR, 2020.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Ad- versarially robust generalization requires more data. Advances in neural information processing systems, 31, 2018.
Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Sat: Improving adversarial training via aurriculum-based loss smoothing. In Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, pp. 23-36, 2021.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Damiel. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representations, 2018.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Ro- bustness may be at odds with accuracy. In International Conference Learning Representations, 2018.
Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In International Conference Learning Representations, 2021.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 33:2958 33:2958-2969, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. International conference on machine learning, pp. 11278 1287. PMLR, 2020.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighted adversarial training. In International Conference on Learning Representations, 2021.
Jianing Zhu, Jiangchao Yao, Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. In International Conference on Learning Representations, 2021.
12

---

Published as a conference paper at ICLR 2024
A CURE ALGORITHM
Algorithm CURE: Caaeive--ppdtt-R-e
Input: Dataset D, Batch size m
Initialize: Model f parameterized by 0 trained on natural images (Inats y) ~ D
while Not Converged do
2: Sample mini-batch: (11.81),, (Im. Vm) ~ D
3: Sample perturbation from a set of allowed perturbations S bounded by
4:
Optimize the adversarial perturbation:
8" = arg max (ets(1 + 8; 0) + aDxL (1()))(((++))
D Eq.
5:
Compute both adversarial and natural losses:
Lade Lets(Minnt 0) + Dr.. ((())maa)) (
Egq.
Lmat (Tntt 0)
6: Incorporate the RGP metric:
RGP(w)
(0(anat)))
+ a) @C( |
Eq.
7: Revision Stage Stochastically update semantic memory:
Sample 10,
8:
if 8 < then Oren = -drev + +1 -d -0
>> Eq.
9:
Consistency regularizaton loss:
CCR = DxL (( (ree) [I((xnat: 0)) + D KL (p(Fadvi Orev plee)) 0)
Eq.
The overall training objective: L Lade LCR Masking: compute stochastic gradients for each layer /
11:
Set the gradients lower than the p percentile RGP to
12: Update the parameter 0
B CURE ALTERNATIVE VERSION
We also introduce an efficient version of CURE, denoted as CURFER CURFER does not require pretrained model on natural images; instead, it undergoes a short warm-up phase where it is trained on natural samples before transitioning to adversarial training. This entire training process, including initial natural training and adversarial training, is completed in the same number of epochs. The results presented in Table 4 demonstrate that CUREER performs nearly as well as CURE, with only 1 %% lower performance.
Table 4: Performances of CURE and CUREEM on ResNet-18 network on CIFAR datasets across various attacks. The second highest is underlined.
CIFAR-10
CIFAR-100
Method
Nat
PGD20
C&W
NRR
PGD20
AA
C&W
NRR
AT 82.78
51.30
44.63
49.72
62.12
$7.27
26.66
23.60
24.89
34.69
TRADES
82.41
52.76
48.37
50.43
62.57
$7.94
29.25
24.71
25.88
35.77
MART
80.70
54.02
47.49
49.35
61.24
55.03
28.25
25.13
26.60
35.86
FAT
87.72
46.69
43.14
49.66
63.41
61.61
18.35
17.98
19.31
29.40
CURE
86.76
54.92
49.69
52.48
65.04
60.72
30.81
25.12
27.95
38.27
CURE
83.51
54.90
48.50
51.94
64.04
58.15
30.53
24.78
26.12
36.04
C
ADDITIONAL RESULTS
In this section, we provide additional results more datasets and architectures.
C.1 EFFECT OF PRETRAINING
As CURE utilizes model trained natural images, also provide comparison where the baselines utilize trained model as well. [5ble the accuracy of using pre-trained models for the two baselines. The results of the baselines do not improve solely due the use of pretrained
13

---

Published as a conference paper at ICLR 2024
network. This demonstrates that the value of CURE lies in how we utilize the information already trained and how we retain while learning a new data distribution.
Table 5: Performance comparisons on the CIFAR-I0 dataset using pre-trained ResNet-18 architecture for all methods.
CIFAR10
CIFAR 100
Method
Nat
PGD20
C&W
NRR
Nat
PGD20
C&W
NRR
SAT
84.37
43.92
44.14
57.95
54.33
20.36
19.96
29.19
TRADES
80.90
50.84
49.04
61.06
53.25
25.50
23.08
32.20
CURE
86.76
54.90
52.48
65.04
60.72
30.81
27.95
38.27
C.2 GENERALIZATION TO LARGER ARCHITECTURES
To illustrate the efficacy of CURE on larger architectures, we present results for ResNer-50 and ResNet- 101 Table 6 This trend of high performance extends larger architectures as well. Note that, for a fair comparison across methods, we maintained consistency by employing the hyperparameters of ResNet- for all three methods and trained them on the larger architectures.
Table 6: Performance comparisons on CIFARIO dataset on larger ResNet architectures.
ResNet-50
ResNet-101
Method
Nat
PGD20
C&W
NRR
Nat
PGD20
C&W
NRR
SAT
72.86
37.54
36.92
49.00
72.22
35.49
34.25
46.64
TRADES
73.06
40.33
37.56
49.61
72.28
39.16
36.69
48.67
CURE
74.58
43.85
41.04
52.94
73.67
42.80
38.90
50.91
C.3 ABLATION STUDY
Through an ablation study, our aim is to provide insights into the individual contributions of two key components in CURE: RGP (Robust Gradient Prominence) and the Revision stage. The RGP plays a pivotal role in identifying weights that hold significant influence during the training process. It contributes to enhancing the network's ability adapt to adversarial data distribution. However, to strike a balance, the revision stage is introduced, leveraging consolidated and stable information. It serves as crucial factor in achieving equilibrium towards natural accuracy as well. The synergistic effect of including both RGP and Revision, as evident in the last row, leads to optimal performance, showeasing the effectiveness of their collective contributions.
Table Ablation study on CURE components' impact on model performance trained on CIFARIO dataset and ResNet18 architecture. NRR is calculated w.r.t. AA.
RGP
Rev
Nat
PGD20
C&W
AA
NRR
X
X
82,41
52.76
50.43
48.37
60.95
/
X
81.27
53.20
51.33
49.06
61.18
X
83.28
51.57
50.96
47.35
60.37
86.76
54.90
52.48
49.69
63.19
C.4 NRR: ACCURACY AND ROBUSTNESS TO AUTOATTACK TRADE-OFF
Table SUM and NRR metrics used to evaluate trade-off. NRR quantifies the trade-off between natural performance and AA attack results.
Method
WideRessNet34-110
ResNet-18
Nat
PGD20
AA
C&W
SUM
NRR
Nat
PGD20
AA
C&W
SUM
NRR
AT 1CL
85.17
55.08
44.04
52.91
129.21
58.05
82.78
51.30
44.63
49.72
127.41
57.99
TRADES
ICM1119
84.73
56.82
52.95
54.29
137.68
65.17
82.41
52.76
48.37
50.43
130.78
60.95
MART
83.62
56.74
51.23
53.16
134.85
63.53
80.70
54.02
47.49
49.35
128.19
59.79
FAT
K3M120
86.60
49.86
47.48
49.35
134.08
61.33
87.72
46.69
43,14
49.66
130.86
57.83
ST-AT
13.
84.92
57.73
53.54
138.46
65.67
83.10
54.62
50.50
51.43
133.60
62.82
ACT
BMVC/20
87.10
54.77
84.33
55.83
ARD
AAAF20
85.18
53,79
82.84
51.41
IAD
ICLR
83.06
56.17
52.68
53,99
135.74
64.47
80.63
53.84
50.17
$1.60
130,80
61.85
LAS-AT
85.24
57.07
53.58
55.45
138.79
65.79
82.39
53.70
49.94
51.96
132.33
62.18
CURE
87.05
58.28
52.10
$5.25
139.15
65.19
86.76
$4.92
49.69
$2.48
136.45
63.18
14

---

Published as a conference paper at ICLR 2024
Orig
automobile
horse
deer
ship
dog
truck 0.079)
Uur 0.066)
horse (urt 0.024)
bird (fur 0.040)
horse (lw 0.043)
TRADES
truck 0.075)
Our 0,071)
frog (2w 0.018)
frog 0.045)
cat (1 0.043)
MART
truck
dog (ur 0.092)
frog (2w 0.016)
frog 0.047)
horse (4w 0.054)
CURE
ship (lur 0.106)
dog (Aw 0.106)
frog (Rw 0.055)
(4wr 0.063)
horse (far 0.057)
Figure 8: Minimum perturbation required fool the models on ResNet-18 CIFAR-10 dataset.
D ADDITIONAL ANALYSES
D.1 ADVERSARIAL PERTURBATION
To comprehensively evaluate the robustness of our adversarially trained models, we assess their robustness against attacks and quantify the level of diffieulty in deceiving them with adversarial perturbations. This assessment is performed using the foolbox library (Rauber et al. 2017), employing the Lint distance metric to calculate the minimum perturbation necessary for successful attack. To visually depict the minimum perturbations required to fool each model, we present comprehensive
visualization in Figure g
The visualizations provide clear comparison of the minimum perturbations required to fool each of the robust models. The original image is shown the first row, whereas the subsequent rows depict the minimum perturbations needed to deceive each model. In particular, the CURE model exhibits higher level of robustness, since requires a larger perturbation (values displayed below the image) to make an incorrect prediction compared to other methods. Furthermore, the visualizations highlight that the semantic features of the object are more preserved in CURE. contrast to other methods, the models trained with CURE exhibit higher level of sensitivity to perturbations. While perturbing the background or irrelevant pixels may suffice to fool the models trained with other methods, the models trained with CURE require modifications the semantically meaningful pixels in order to be deceived. This demonstrates the effectiveness of CURE in preserving the integrity important object features and further highlights its robustness against adversarial attacks.
15

---

Published as a conference paper at ICLR 2024
100
Method
SAT
TRADES
MART
CURE
80
Robustness
60
40
20
0
Brightness
Contrast Fog
Frost
Snow
Gaussian Noise
Shot Noise
Impulse Noise Motion Blur
Defocus Blur
Glass Blur
Zoom Blu
Gaussian glu Pixelate
Elastic Transform JPEG Compression
Speckle Noise
Figure 9: Robustness against 15 various natural corruptions (at severity level 3) on CIFAR-10 dataset.
D.2 ROBUSTNESS AGAINST NATURAL CORRUPTIONS
There are 17 distinct types corruptions. In the main paper, we have classified them into four groups (following Hendrycks & Dietterich (): Blur, Digital, Noise, and Weather, as depicted in Figure to facilitate easier comparison among methods' performance. Within the Blur group, we incorporate Defocus, Glass, Motion, Gaussian and Zoom blur effects. Additionally,
explore the Digital category. introducing variations in Contrast, Elastic Transformation, JPEG Compression, and Pixelation. To simulate diverse Weather conditions, introduce Brightness, Fog, Frost, and Snow. The Noise category encompasses Gaussian, Impulse, Speckle and Shot noise. These corruptions are systematically applied at five different severity levels, ranging from (indicating less severe corruption) to 5 (representing the most severe corruption). Figure illustrates the accuracy against all 15 forms of corruption.
D.3 GRADIENT ANALYSIS
Figure T0 displays the updates convolution layers, with higher magnitude of update observed in the middle layers. Further, Figure 6 we observe higher updates in BatchNorm layers compared to convolutional layers. This observation underscores the significant role of batchnorm statistics in our scenario, where we initiate training from a natural model and expose it to adversarial samples with distinct data distribution. Few works delve into exploring the influence of batch normalization in training. Notably, (Frankle et al. (020) showcases the potential to achieve remarkably high accuracy by exclusively training the affine parameters of BatchNorm, while keeping all other parameters frozen at their original initializations. This intriguing behavior prompts further investigation into the impact of batch normalization selective and adversarial training domains.
E RELATED WORKS
We describe more of the SOTA methods in the extended related works. SAT (Sitawarin et al. 2021
proposes a surriculum-based loss smoothing to enhance its robustness. BS (Chen et al. 2022 introduces a new initialization strategy coupled with backward smoothing to reduce computation cost and enhance the efficiency of training. FAT (Zhang et al. 2020) investigates benign attacks that do not halt training and identifies their regularization effect on the training. ST-AT (Li et al. 2023) combines adversarial examples and collaborative examples, which exhibit significantly lower losses than their neighbors, into the training process. formulates a joint optimization objective that minimizes the prediction discrepancies between these examples. HAT (Rade & Moosavi-Deztooli
16

---

Published as a conference paper at ICLR 2024
Intitial training stage
Final training stage
70
30
% of gradients updated
60
25
50
40
20
10
10.cl
c1
02.0c
12.1
12.1.
B.0.cl
130
B.1cl
13.1.22
14.0cl
c.00c2
141.cl
M.1.c2
1.0c1
11:0
11.1cl
1.
12.0.1
c2.0.22
12.1.c
12.1.22
1300c
c2.0
B3.
12.1.c2
14.0
c4.
M.1.cl
M.1.c2
11.0
12.0
11
Figure 10: Percentage of gradients updated in each convolution layer (layer'x .convyy 'weight')
during initial, middle and final phases of training.
2021) reduces the excessive directional margin by incorporating additional wrongly labeled examples during training.
Several approaches adopt multi-network strategies during training, such as ACT (Arani et al. 2020) which concurrently trains networks,
natural and the other adversarial samples. ARD
(Goldblum et al. 2020) transfers knowledge from a robust teacher network to a more compact and resilient student network. Multiple models mutually trained together in MAT (Liu et al., 2022)
and self-ensemble techniques are employed in SEAT (Wang & Wang 2021). IAD (Zhu et al. 2021
utilizes introspective adversarial distillation improve the reliability of knowledge transfer the presence of noisy or adversarial teacher models. LAS-AT (Jia et al. 2022) automatically generates sample-dependee adversarial attacks dynamically by utilizing an additional network.
Robust overfitting: In AWP (Wu et al., 2020) adversarial perturbations, computed through an optimization process to maximize the worst-case loss, are applied the model weights, resulting in improved robustness against adversarial attacks. KD (Chen et al., 2020) proposes two approaches to mitigate robust overfitting: one using knowledge distillation and self-training to smooth the logits and the other applying stochastic weight averaging (SWA) to smooth the weights. Dong et al. (2022) explores the phenomenon of memorization and explores convergence and generalization issues through random labels. The approach integrates the temporal ensembling (TE) approach into the AT framework. Li & Spratling (2023) introduces an augmentation method called Cropshift, enhancing diversity and hardness control, and proposes augmentation scheme called Improved Diversity and Balanced Hardness (IDBH).
F IMPLEMENTATION DETAILS AND METRICS
We evaluate the performance of our approach multiple architectures, including ResNer-18 (He ct al. 2016a), WideResNet- 10 (Zagoruyko & Komodakis 2016). and PreActResNet (He et al. 2016b). Datasets used in study include CIFAR-10. CIFAR- (Krizhevsky) 2009) and SVHN. For comparison, we consider the baseline performances as reported the respective base and comparative works. We also establish Ratural-Robusthie Ratio (NRR) metric to better measure the trade-off between natural and robust performance F2 In the following, more details about all hyperparameters and metrics provided.
F.1 HYPERPARAMETERS
For our method, all models are trained using the SGD optimizer with momentum of 0.9. The augmentations include basic random crop and random flip operations. Projected Gradient Descent (PGD) is used to generate adversarial images. For adversarial training, PGD with step 10 is considered with perturbation strength e 8 and step size e/4. tabulatee the other hyperparameters used in our method. Our revision stage occurs stochastically with only 20%probability. Moreover, as
17

---

Published as a conference paper at ICLR 2024
Table 9: Hyperparameters used for CURE. The learning rate is 0.1, the number of epochs is 200 and the weight decay is 5e 3 The revision rate r and decay factor d for the revision stage are set to 0.2 and 0.999 for all the experiments.
Architecture
Dataset
p
o
CIFAR-10
30
0.1
1.0
ResNet-18
CIFAR-100
30
0.1
SVHN
30
0.2
1.0
CIFAR-10
50
0.3
WideResNet:34-10
CIFAR-100
50
0.3
0.5
PreActResNet-18
CIFAR-10
20
0.2
1.0
Table 10: Hyperparameters sensitivity analysis in CURE. The hyperparameters remain stable across various settings and datasets. Among them, "p" and "a" are important and are intuitive, as they determining the trade-off between natural and adversarial accuracy.
Sensitivity to
Sensitivity to
Sensitivity to a
p
0
Nat
Rob
Y
Nat
Rob
p
Y
Nat
Rob
30
0.1
86.76
54.92
1.0
86.76
54.92
30
0.1
1.0
86.76
54.92
0.5
86.25
53.13
10
85.15
55.65
0.2
86.95
54.17
0.5
88.52
53.67
training advances, gradually decay the revision rate minimize revisions when the network stabilizes.
In the evaluation setting, examined several white-box attacks and hybrid attack that combines both white- and black-box methods. White-box attacks have full access to the model parameters and limited by maximum perturbation of e. On the other hand, black box attacks are created by attacking surrogate model that has the same architecture as the targeted model, using natural images as input.
PGD (Projected Gradient Descent) is considered to be one of the strongest first-order attacks, as it is able find the maximum possible perturbation that will cause the model misclassify an input. The PGD attack starts by generating a random initial perturbation and then repeatedly takes gradient steps in the direction that increases the model's classification error, until the maximum perturbation limit is reached. This is done by projecting the perturbation back onto the e-ball after each step, which ensures that the perturbation does not exceed the maximum allowed value. The PGD attack can be customized
different values of step size and number of steps. In the evaluation discussed in the previous information, we are using three variations of the step size (10, 20, and 50) and various perturbation strengths of C&W (Carlini & Wagner 2017) is also known be of the strongest white-box attacks. The C& W attack is based on an optimization problem that tries to minimize the 1.2 distance between the original input and the adversarial example, while also maximizing the model's classification error. AutoAAttcc (AA) (Croce & Hein 2020b). is an automated attack method that is designed to be both reliable and strong. AA consists of white-box attacks: APGD-CE, APGD-DLR (Croce& & Hein 2020b), FAB (Croce & Hein 2020a) and black box attack: Square (Andriushchenko al., 2020). The APGD-CE (AutoAttack with PGD and Cross-Entron Eatsssy loss) and APGD-DLR (AutoAttack with PGD and Data-Limited-quee
loss) methods are based on the Projected Gradient Descent (PGD) attack, and they use different loss functions (Cross- Ectors-Entropy and Data-Limited-query loss, respectively) to optimize the perturbation. The square attack is black-box attack that leverages the ability the model to classify images squares and rectangles in order to generate adversarial examples. The 'Fast and Accurate Black-Box Attack' (FAB) is also included in AutoAttack.
Sensitivity: : The hyperparameters are stable across settings and datasets and also complement each other. Therefore, CURE does not require extensive fine-tuning across different datasets and settings. Some hyperparameters, like the revision rate (r) and decay factor (d) for the revision stage, are set universally (r 0.2, d 0.999) across all experiments. The revision rate (r) determines the
18

---

Published as a conference paper at ICLR 2024
stochasticity of the revision stage, with a 20% probability of occurrence. This is more important during the initial stage of training and controlled reduction in revisions occurs as training progresses, as the network stabilizes.Thr hyperparameter 7 which is associated with the consistency regularization term, is generally stable and is set to 1.0.
The main hyperparameters of interest are "p" and "a", which quite intuitive. The parameter "a" influences the importance given to the adversarial distribution in the learning process. Higher values of a emphasize natural accuracy. Setting a to lower value, such 0.1 or 0.2, is preferable, especially when the network is pre-trained in the standard setting. The p parameter is associated with estimating the percentile in gradients below which the gradients are set to zero. The choice of p influences the balance between retaining previous knowledge and learning new information. More details are provided in Table TO
Computation cost and limitations: In terms of computation cost, CURE utilizes a single network for inference, similar other methods. During the training process, an additional pass is performed to calculate and choose the relevant weights, and during the revision stage, stochastic momentum update of the training model is utilized.
F.2 NATURAL-ROBUSTNESS RATIO
In the field of deep neural networks, it is often crucial to strike balance between two fundamental aspects of model performance: natural accuracy and robust accuracy. Natural accuracy refers to a model's ability to perform well standard, unaltered data that closely resembles the training data distribution. On the other hand, robust accuracy assesses the model's performance when exposed to perturbed or adversarial data, which may deviate significantly from the training distribution.
The NRR -ouuuale-ooussesss Ratio) metric is introduced as means to quantitatively measure this trade-off between natural aceuracy and robust accuracy. considers both aspects of model performance when dealing with standard and perturbed data.
NRR 2 x Natural Robust
NaturalAccuracy + RobustAccuracy
The NRR metric combines these two accuracy measurements into a single value. high NRR value indicates that the model excels in both natural and robust accuracy simultaneously, achieving a well-balanced trade-off. Conversely, a lower NRR value suggests that the model might prioritize one aspect (natural robust accuracy) over the other, potentially sacrificing performance in the less prioritized dimension.
F.3 CKA
Central Kernel Alignment (CKA) method for measuring the similarity between two sets of features, such as the features of neural network's intermediate layers. It is commonly used to analyze the representations learned by neural networks and to understand how different architectures or training methods affect the representations.
K1 K2
CKA(FI, F2) |Krl |K2lr
(10)
where Fi and F2 are the two sets of features being compared, K₁ and K₂ are the corresponding kernel matrices, and |r denotes the Frobenius norm.
CKA is calculated by first computing the dot product of the two sets of features, then squaring the result, and finally normalizing by the product of the norms the two sets of features. The result is scalar value between -1 and 1, where a value close indicates high alignment, value close to -1 indicates low alignment, and a value close to 0 indicates no alignment.
G EMPIRICAL ANALYSIS DETAILS
Balancing the trade-off between generalization and robustness remains an enduring challenge in the field of AT, exacerbated by the intricacies of robust overfitting. Furthermore, while standard test.
19

---

Published as a conference paper at ICLR 2024
100
Natural Acc
Robust Acc
80
60
20
0
ST
3
12
23
34
123
234
SAT
Updated blocks (U-)
Figure 11: Standard generalization and robustness of different blocks of ResNet-18 on CIFAR-10 dataset without re-initializing layers.
accuracy typically exhibits upward trajectory plateaus during ST, it experiences a decline in the adversarial setting, signaling underfitting. And the decrease in accuracy on clean images suggests memorization. Fixing the parameters of naturally trained model before adversarial training can provide insights into the learning behavior of the network, which is crucial for achieving a balance between generalization and robustness. Hence, advocate for incorporating layer-wise properties into developing training schemes.
Re-initialization, or the process of randomly initializing the weights of a neural network during training, can provide detailed insights into what each layer of the network is learning. Especially, when training data on different distributions, re-initialization network performss on both distributions. Re- Re-initialization has been explored in standard training (transfer learning and fine-tuning) but not in adversarial setting. Here, we investigate the layer-wise properties of network through the lens of retention and updation.
prove important learn how the
We use a pre-trained model (on the original CIFAR- 10 dataset) as the base network. We freeze the parameters and perform (M(Madry et al., 2018) on CIFAR-10 dataset. The notation U-b represents the update of block "b" while keeping the rest of the network frozen. We create different combinations (U-1, U-2, U-3, U-4, U-12, U-23, U-34, U-123, U-234). The example of U-23 is shown Figure[(a). a).
Setup:
Adversarial Robustness: In Figure b).).. the "ST" and "SAT" represents the accuracy of the ST and AT (Madry et al., 2018). The label "I" ("U-1) represents the freezing of all layers except the first block, which is trained and updated adversarial images. Similarly, "23" represents freezing block 1. 4, and the final layer 5, while re-initializing and updating the weights of the second and third block of the network. Among the models trained with combinations of layers, freezing the first and last layers and updating the weights of the second and third proves beneficial in increasing both natural and adversarial performance (compared the AT baseline). This shows that updating the parameters of the whole network is not an optimal approach to train the two data distributions.
Figure [I displays similar analysis, albeit without re-initializing the layers; other words, the blocks are solely fine-tuned on the adversarial data. While the results show slightly higher performance in certain settings, the overall pattern remains consistent with the re-initialization analysis.
Figure 3 shows the similarity between natural and robust representations for each of the models. For the nine models, trained with different retention and update strategies, pattern of "block structures" is observed. Block structures indicate a high level of intra-layer similarity and lower inter-layer similarity. For instance, in the U-1 model, the first block displays considerable dissimilarity with the fourth block, while the last three blocks exhibit higher similarity. Notably, the block structure becomes less pronounced in later layers. Moreover, models that selectively update only the final layers demonstrate more similarity in the representations (e.g.. U-4, U-34, U-234). The representation similarity between different layers in the DNNs trained by CURE is visualized using the CKA metric in Figure 12 Compared to standard training (ST) and adversarial training (AT), CURE lies in the middle, leaning towards enhanced robust accuracy. demonstrates more balanced and nuanced performance, with overall superior trade-off.
Representation Alignment:
20

---

Published as a conference paper at ICLR 2024
NonRobust-Robust
1.0
30
0.8
Layer
0.6
20
0.4
Figure 12: Representation similarity be- tween robust and nob-robust features in ResNet- 18 trained on the CIFAR-10 dataset using CURE.
0.0
0
10
30
Layer
Robust Overfitting: Robust overfitting is common issue in adversarial training, where test accuracy tends to decrease while training accuracy increases plateaus, indicating overfitting. Figure 4 displays the test accuracy curves for all eight models. In the first graph, freezing layer! and layer2 individually reduces overfitting but leads to decrease in overall accuracy. In the second graph, updating the last two layers (U-34) exhibits overfitting behavior, while updating the first two layers (U-12) and the middle two layers (U-23) perform better.
21


Reviews:
[{"review_id": "tMT4tXPLQv", "reviewer": "Reviewer_DhtN", "summary": "This paper focuses on the trade-off between standard and robust generalization. To this end, this paper proposes CURE that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights, which can tackle both memorization and overfitting issues.", "questions": "In this paper, many conclusions are not supported authors' experimental results, i.e., we cannot infer these conclusions just based on existing experimental results.\nThus, many conclusions in Section 3 are over-claimed.\nDetails are stated in weaknesses.", "limitations": "", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The problem authors focused on is very interesting.", "weaknesses": "1. Experimental results in Fig. 2 cannot support authors' claim that \".. cause reduced performance on both data due to overwriting of learned information..\" Specifically, just a comparison of accuracy in Fig. 2  cannot reflect the overwriting of learned information. Authors should design new solid experiments to support this conclusion.\n2. Authors did not clarify how to disentangle robust and non-robust features, which still presents a significant challenge. Hence, Fig.3 is in doubt.\n3. I wonder why \"a subset with the most significant impact on accuracy\" equals to the subset of weights that \"contribute more to the joint distribution of both natural and adversarial accuracy.\" Can you prove it or explain it?\n4. A algorithm flowchart will help readers better understand how weights are updated in each epoch. In each epoch, are different or same subsets of weights updated?\n5. What does \"sample∼U(0,1)<r\" in Eq. 7 mean?\n6. Experimental results cannot verify the effectiveness of the proposed CURE method, since authors just conducted experiments on resnet18 and resnet34. Please conduct more experiments on more classic DNNs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZXnyh7akQA", "reviewer": "Reviewer_HvAu", "summary": "This paper presents a novel approach to improving the adversarial robustness of DNNs while maintaining the performance on natural samples. The authors first conduct empirical studies to discover that updating weights in all layers in AT may be not good for the generalization of DNNs in both natural and adversarial samples. Thus, they propose an adaptive method to selectively update a subset of weights in the DNN. The proposed method is simple but empirically effective.", "questions": "- Is CURE used by training the model from scratch with the loss function in equation (5) or finetuning a pre-trained network? If the CURE is adopted on a pre-trained model, I am concerned that it may be unfair to compare it with other training-from-scratch methods. Besides, it also increases the computational cost to train the model twice (pre-train with ST and then finetune with CURE). \n- How stable is CURE when using different hyper-parameters ($\\alpha,\\beta$ and $p$)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This paper presents a simple yet effective method of improving both robustness and generalization of DNNs.\n\nThis study discovers some interesting phenomena, e.g., updating middle layers is beneficial to standard and robustness generalization, and adversarial training increases the similarity between features of different layers.\n\nThe paper is well-written and easy to follow.", "weaknesses": "- The authors repeatedly use “overwritten” and “learning”, but I cannot grasp the essential difference between them. What kind of update of weights is referred to as “overwritten” and “learning”, respectively? Can you provide a clearer definition for the two terms? Besides, how do the authors come to the conclusion about “overwritten” and “learning” from the accuracy drop in Figure 2(a)? \n- In Section 2, the authors update the weights of selected layers by re-initializing and updating them. Have you ever tried not re-initializing but directly finetuning them with adversarial examples? Learning weights from random noises, especially weights in shallow layers may lead to a significant drop in performance, but finetuning them will not.\n- I’m a little confused about Figure 3. If the weights of deep layers (e.g. U-34) are updated while shallow layers (e.g. blocks 1 and 2) are fixed, the features in shallow layers are also supposed to be frozen. Thus, the similarity between shallow layers in U-34 should be the same as in the ST model. However, the similarities between shallow layers in Figure 3 are all different.\n- Although this paper presents some differences in performance between updating different layers, it does not clearly “disentangle clean and adversarial representations” or “disentangle robust and non-robust features”. Also, this paper does not strictly define and extract layers with “greater learning capacity”. I suggest the authors use such claims carefully and seriously.\n- Figure 4 indicates that updating high layers may cause robustness overfitting, but Figure 6 (b) shows a large ratio of updated gradients in high layers. I expect the authors to discuss more about such a misalignment.\n- How about the representation similarity between different layers in the DNNs trained by CURE?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lsjYtFJ6iu", "reviewer": "Reviewer_i2U9", "summary": "This paper unveils the underlying factors of the trade-off between standard and robust generalization in adversarial training by examining the layer-wise learning capabilities of neural networks during the transition from a standard to an adversarial setting. The paper demonstrates that selectively updating specific layers while preserving others can substantially enhance the network's learning capacity empirically, and proposes a method to leverage a gradient prominence criterion to perform selective conservation, updating, and revision of weights named CURE. The paper verified the effectiveness of CURE on various dataset and architecture, which verifying the effect in enhancing the trade-off between robustness and generalization and alleviating robust overfitting empirically.", "questions": "1.Is the proposed method still works well on larger dataset, for example ImageNet?\n2.There are so many hyperparameters such as α,β,γ,r,d, how to mediate so many hyperpapameters effectively?\n3.For deeper neural networks, such as resnet-101, is the proposed method still works well?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper has good originality, high quality and clear expression. The paper unveils the trade-off between standard and robust generalization in adversarial training in the perspective of layer-wise learning capabilities of neural networks and proposes a new method to alleviate robust overfitting.", "weaknesses": "The analysis of selective adversarial training are empirically not theoretically.It's better to provide theoretically analysis of selective adversarial training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kcznp290kN", "reviewer": "Reviewer_PH92", "summary": "This paper proposes a new method to improve the trade-off between robustness and generalization in adversarial training.\nFirst, this paper investigates the difference between adversarial and clean representations layer-wisely,\nand next, it investigates overfitting in adversarial training when parameters of only some layers are selectively updated.\nBased on the observation that some layers tend to suffer from overfitting, this paper proposes CURE that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights.\nTo evaluate the trade-off, this paper establishes a new metric: Natural-Robustness Ratio which is calculated by using accuracy against C&W and natural accuracy.\nCURE is evaluated in terms of this metric, robustness against several attacks including AutoAttack, and robustness against natural corruption.", "questions": "- How is the performance if we use only RGP? If it is not good, are equations (4), (5), (7), and (8) relevant to the layer-wise analysis?\n- Is there a rational explanation as to why the C&W is used in the evaluation of trade-off? Is there any reasonable explanation for Natural-Robusthess Ratio? Why is eq.(9) suit to evaluate the trade-off?\n- How does NRR become if using AutoAttack?\n- What is the difference between TRADES and eqs.(4) and (5)?\n- Why does Table 1 not contain the result of HAT, and does Table 2 not contain ACT, ARD, LAD, LAS-AT?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This paper addresses an important problem in adversarial training: Overfitting and the trade-off between natural accuracy and robustness.\n- The detailed investigation of layer-wise learning phenomena in adversarial learning is novel and provides interesting insights.\nRevealed layer-wise properties might inspire researchers in this area and might cause new defense methods.\n- Gradient-based selective update for adversarial training is a new and interesting idea. \nThe figure of gradients in the training (Fig. 6) intuitively shows how the proposed method works by using the information of gradients well.\n- CURE is evaluated by using various attacks and architectures. However, baselines are not consistent and results might be cherry-picked.", "weaknesses": "- This paper lacks an ablation study. Although layer-wise analyses are interesting, the proposed method contains several components besides selective updating. How is the performance if we use only RGP? \nIf it is not good, are equations (4), (5), (7), and (8) relevant to the layer-wise analysis?\nIf other parts than the selective updating contribute to the performance, the layer-wise analysis may not be very worthwhile.\n\n- This paper does not present a fair and honest evaluation and presentation of the results of the experiments.\nThe trade-off metric is intentionally designed to make the proposed method look overly good. Is there a rational explanation as to why the C&W is used in the evaluation of trade-off, even though AutoAttack performs better than C&W in terms of attack success rate? I suspect that C&W is chosen because the numbers of metrics are not better in the case of AutoAttack. In fact, robustness against AutoAttack of the proposed method is not always greater than baselines.\nAdditionally, the vertical and horizontal scales in Figure 1 are not aligned, which can be misleading.\nNatural corruptions are selectively used from CIFAR10C. Their results may be cherry-picked. I would like to see the results against all natural corruptions in CIFAR10C. Baseline methods are not consistent over expreiments.\nWhy does Table 1 not contain the result of HAT, and does Table 2 not contain ACT, ARD, LAD, and LAS-AT?\n\n- The boundary between the proposed and existing methods is described ambiguously. Eqs. (4) and (5) seem to be an objective function of TRADES. Why are these equations written in the section of the proposed method?\nAdditionally, SMU seems to be exponential moving average (EMA) with a stochastic parameter. SEAT (Wang & Wang, 2021) and other recent methods also use EMA, which is sometimes called weight averaging. Unlike them, the proposed method uses averaged parameters in the regularization term. I would like to see the comparison between averaging weights directly and using averaged weights for regularization.\n- Minor issues\n    - [a] might be related work that addresses the trade-off and focuses on the difference between the representations of clean data and adversarial examples. The method in [a] outperforms LAS-AT in terms of the trade-off. Since it seems to be concurrent work, I think that it is not necessary to compare.  \n    [a] Suzuki S et al. \"Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff.\" ICCV 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)