On the Over-Memorization During Natural, Robust and Catastrophic Overfitting
Authors: Runqi Lin
Chaojian Yu
Bo Han
Tongliang Liu
Venue: ICLR 2024 poster (2024)
Keywords: overfitting
natural overfitting
robust overfitting
catastrophic overfitting
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Overfitting negatively impacts the generalization ability of deep neural networks (DNNs) in both natural and adversarial training. Existing methods struggle to consistently address different types of overfitting, typically designing strategies that focus separately on either natural or adversarial patterns. In this work, we adopt a unified perspective by solely focusing on natural patterns to explore different types of overfitting. Specifically, we examine the memorization effect in DNNs and reveal a shared behaviour termed over-memorization, which impairs their generalization capacity. This behaviour manifests as DNNs suddenly becoming high-confidence in predicting certain training patterns and retaining a persistent memory for them. Furthermore, when DNNs over-memorize an adversarial pattern, they tend to simultaneously exhibit high-confidence prediction for the corresponding natural pattern. These findings motivate us to holistically mitigate different types of overfitting by hindering the DNNs from over-memorization training patterns. To this end, we propose a general framework, $\textit{Distraction Over-Memorization}$ (DOM), which explicitly prevents over-memorization by either removing or augmenting the high-confidence natural patterns. Extensive experiments demonstrate the effectiveness of our proposed method in mitigating overfitting across various training paradigms.

Published as a conference paper at ICLR 2024
ON THE OVER-MEMORIZATIO DURING NATURAL, ROBUST AND CATASTROPHIC OVERFITTING
Runqi Lin
Chaojian Yu
Sydney AI Centre, The University of Sydney rlin0511e (iin0511@syddey edu. au
Sydney AI Centre, The University of Sydney chyu8051 @eeeeesyyyyeeeeeeeee eu
Bo Han
Tongliang Liu"
Hong Kong Baptist University phanm1@comp hkbu. edu. hk
Sydney Al Centre, The University of Sydney tongliang liuusyddey edd. au
ABSTRACT
Overfitting negatively impacts the generalization ability of deep neural networks (DNNs) in both natural and adversarial training. Existing methods struggle consistently address different types of overfitting, typically designing strategies that focus separately either natural adversarial patterns. In this work, we adopt a unified perspective by solely focusing on natural patterns to explore dif- ferent types of overfitting. Specifically, we examine the memorization effect in DNNs and reveal shared behaviour termed over-meemorization, which impairs their generalization capacity. This behaviour manifests as DNNs suddenly be- coming bigh-com confidenne in predicting certain training patterns and retaining a persistent memory for them. Furthermore, when DNNs over- werrmee an adver- sarial pattern, they tend to simultaneously exhibit high-confidence prediction for the corresponding natural pattern. These findings motivate us to holistically miti- gate different types of overfitting by hindering the DNNs from over-memorization training patterns. To this end, we propose a general framework, Distraction Over- Memorization (DOM), which explicitly prevents over- memorization by either removing or augmenting the high-confidence natural patterns. Extensive experi- ments demonstrate the effectiveness of our proposed method in mitigating over- fitting across various training paradigms. Our implementation can be found at https: /ttpthu.ttt/tttg/////////t/oot wwwmcmmmmmmmmccccccccccccm 1 ab/ 2024 _CL__DOM.
INTRODUCTION
In recent years, deep neural networks (DNNs) have achieved remarkable success in pattern recognition tasks. However, overfitting, a widespread and critical issue, substantially impacts the generalization ability of DNNs. This phenomenon manifests DNNs achieving exceptional performance on training patterns, but showing suboptimal representation ability with unseen patterns.
Different types of overfitting have been identified in various training paradigms, including natural overfitting (NO) in natural training (NT), as well as robust overfitting (RO) and catastrophic overfitting (CO) in multi-step and single-step adversarial training (AT). NO (Dietterich, 1995) presents as the model's generalization gap between the training and test patterns. On the other hand. RO (Rice et al., 2020) is characterized by gradual degradation in the model's 's test robustness as training progresses. Besides, CO (Wong et al., 2019) appears as the model's 's robustness against multi-step adversarial attacks suddenly plummets from a peak nearly 0%.
In addition to each type of overfitting having unique manifestations, previous research (Rice et al., 2020: Andriushchenko & Flammarion, 2020) suggests that directly transferring remedies from one type of overfitting to another typically results in limited or even ineffective outcomes. Consequently, most existing methods are specifically designed handle each overfitting type based on characteristics
"oorresooniigg author

---

Published as a conference paper at ICLR 2024
associated with natural or adversarial patterns. Despite the significant progress in individually addressing NO, RO and CO, a common understanding and solution for them remain unexplored.
In this study, we take unified perspective, solely concentrating on natural patterns, to link overfitting in various training paradigms. More specifically, we investigate the DNNs' memorization effect con- cerning each training pattern and reveal a shared behaviour termed weermmemorization This behaviour manifests as the model suddenly exhibits high-confidence in predicting certain training (natural or adversarial) patterns, which subsequently hinders the DNNs' generalization capabilities. Additionally, the model persistent a strong memory for these over-memorization patterns, retaining the ability to predict them with high-confidence, even after they'v 've been removed from the training process. Furthermore, we investigate the DNNs' prediction between natural and adversarial patterns within single sample and find that the model exhibits similar memory tendency in over-memorization samples. This tendency manifests as, when the model over-memorizes certain adversarial patterns, will simultaneously display high-confidence predictions for the corresponding natural patterns. Leveraging this tendency, we are able to reliably and consistently identify vver-memorization samples by solely examining the prediction confidence on natural patterns, regardless of the training paradigm.
Building this shared behaviour, we aim to holistically mitigate different types of overfitting by hindering the model from over- memorization training patterns. To achieve this goal, we propose a general framework named Distraction Dver-Memorization (DOM), that either removes or applies data augmentation to the high-confidence natural patterns. This strategy is intuitively designed to weaken the model's 's confidence in ver-memorizzaion patterns, thereby reducing its reliance them. Extensive experiments demonstrate the effectiveness of our proposed method in alleviating overfitting across various training paradigms. Our major contributions are summarized as follows:
We reveal shared behaviour, overroeemiization, across different types of overfitting: DNNS tend to exhibit sudden high-confidence predictions and maintain persistent memory for certain training patterns, which results in a decrease in generalization ability.
We discovered that the model shows similar memory tendency in over-memorization samples: when DNNs over- werrmeeorize certain adversarial patterns, they tend to simultaneously exhibit high-confidence in predicting the corresponding natural patterns.
Based on these insights, we propose general framework DOM to alleviate overfitting by explicitly preventing over-memorization. We evaluate the effectiveness of our method with various training paradigms, baselines, datasets and network architectures, demonstrating that proposed method can consistently mitigate different types of overfitting.
2 RELATED WORK
2.1 MEMORIZATION EFFECT
Since Zhang al. (2021) observed that DNNs have the capacity to memorize training patterns with random labels, a line of work has demonstrated the benefits of memorization in improving generalization ability (Neyshabur et al., 2017; Novak et al., 2018; Feldman, 2020; Yuan et al., 2023). The memorization effect (Arpit et al., 2017; Bai et al.. 2021; Xia et al., 2021; 2023; Lin et al., 2022; 2023b) indicates that the DNNs prioritize learning patterns rather than brute-force memorization. In the context of multi-step AT, Dong et al. (2021) suggests that the cause of RO can be attributed to the model's 's memorization of one-hot labels. However, the prior studies that adopt unified perspective to understand overfitting across various training paradigms are notably scarce.
2.2 NATURAL OVERFITTING
NO (Dietterich, 1995) typically shown as the disparity in the model's performance between training and test patterns. To address this issue, two fundamental approaches, data augmentation and regularization, are widely employed. Data augmentation artificially expands the training dataset by applying transformations Mixup (Zhang et al., 2018), AutoAugment (Cubuk et al., 2018) and RandomErasing (Zhong et al., 2020). On the other hand, regularization methods introduce explicit constraints mitigate NO, including dropout ( (Wan et al., 2013; Ba & Frey, 2013; Srivastava et al., 2014), stochastic weight averaging (Izmailov et al., 201 8), and stochastic pooling (Zeiler & Fergus, 2013).
the original patterns, such Cutout (DeVries & Taylor, 2017),
the DNNs to
2

---

Published as a conference paper at ICLR 2024
2.3 ROBUST AND CATASTROPHIC OVERFITTING
DNNs are known to be vulnerable to adversarial attacks (Szegedy et al., 2014), and AT has been demonstrated be the most effective defence method (Athalye al., 2018; Zhou et al., 2022). AT is generally formulated as a min-max optimization problem (Madry et al., 2018; Croce et al., 2022). The inner maximization problem tries to generate the strongest adversarial examples to maximize the loss, and the outer minimization problem tries to optimize the network to minimize the loss on adversarial examples, which can formalized as follows:
min R W)
max f(x + 6, y: SEA
where (ar, y) is the training dataset from the distribution D, ((r, y; 0) is the loss function parameterized by 0, is the perturbation confined within the boundary shown as: A {6 ||8|p S c).
For multi-step and single-step AT, PGD (Madry et al., 2018) and RS-FGSM (Wong et al., 2019) are the prevailing methods used to generate adversarial perturbations, where the II denotes the projection:
Uniform(-e,4,
OPD = [[oo (n + sign V V2-nttr-1ke =((1 _-_- .):0)),
ÖRS-FGSM [[,, ( + sign (Vz+n E(x + 11 0))
With the focus on DNNs' robustness, overfitting has also been observed in AT. An overfitting phenomenon known as RO (Rice et al., 2020) has been identified in multi-step AT, which manifests as gradual degradation in the model's test robustness with further training. Further investigation found that the conventional remedies for NO have minimal effect on RO (Rice al., 2020). As a result, lot of work attempts to explain and mitigate RO based on its unique characteristics. For example, some research suggests generating additional adversarial patterns (Carmon et al., 2019; Gowal et al., 2020), while others propose techniques such adversarial label smoothing (Chen et al., 2021; Dong et al., 2021) and adversarial weight perturbation (Wu et al., 2020; Yu et al., 2022a:b). Meanwhile, another type of overfitting termed CO (Wong al., 2019) has been identified in single-step AT, characterized by the model's robustness against multi-step adversarial attacks will abruptly drop from peak to nearly 0%. Recently studies have shown that current approaches for addressing NO and RO are insufficient for mitigating CO (Andriushchenko & Flammarion. 2020; Sriramanan et al., 2021). To eliminate this strange phenomenon, several approaches have been proposed, including constraining the weight updates (Golgooni et al., 2023; Huang et al., 2023a) and smoothing the adversarial loss surface (Andriushchenko & Flammarion, 2020; Sriramanan et al., 2021; Lin al., 2023a).
Although the aforementioned methods can effectively address NO, RO and CO separately, the understanding and solutions for these overfitting types remain isolated from each other. This study reveals a shared DNN behaviour termed over-memooization. Based on this finding, we propose the general framework DOM aiming to holistically address overfitting across various training paradigms.
3 UNDERSTANDING OVERFITTING IN VARIOUS TRAINING PARADIGMS
In this section, we examine the model's memorization effect on each training pattern. We observe that when the model suddenly becomes high-confidence predictions in certain training patterns, its generalization ability declines, which we term as over-memorization (Section 3.1). Furthermore, notice that memorizooizttion also occurs in adversarial training, manifested by the DNNs simultancously becoming high-confidence in predicting both natural and adversarial patterns within single sample (Section 3.2). To this end, we propose a general framework Distraction Over- Memorization (DOM) to holistically mitigate different types of overfitting by preventing over- memorization (Section 3.3). The detailed experiment settings can be found in Appendix A.
3.1 OVERRMEMMOIIZATION IN NATURAL TRAINING
To begin, we explore the natural overfitting (NO) by investigating the model's memorization effect. As illustrated in Figure (left), we can observe that shortly after the first learning rate decay (150th epoch), the model occurs NO, resulting in a 5 5% performance gap between training and test patterns.
3

---

Published as a conference paper at ICLR 2024
100
5.57
Generalization Gap (%
Transformed
Pattern Proportion
80%
Accuracy (%)
60%
Original
4.84
40%
[1.0, sc)
4.63
[0.6, 0.8)
4.42
20%
Natural Training Natural Testing
[0.2, 0.4)
0% 50 100
[0.0,
50 100 150 200
250 300 Epoch
150 200 Epoch
250 300
Baseline Remove Remove Remove
AII-HC Ori-HCTranssHIC
Figure 1. Left Panel: The training and test accuracy of natural training. Middle Panel: Proportion of training patterns based on varying loss ranges. Right Panel: Model's generalization gap after removing different categories of high-confidence (HC) patterns.
Then, we conduct statistical analysis of the model's training loss on each training pattern, as depicted in Figure 1 (middle). We observe that aligned with the onset of NO, the proportion of the model's 's observation prompted us consider whether the decrease in DNNs' generalization ability linked to the increase in high-confidence training patterns. To explore the connection between high-confidence patterns and NO, we directly removed these patterns (AII-HC) from the training process after the first learning rate decay. As shown in Figure 1 (right), there is a noticeable improvement (4%) in the model's s generalization capability, with the gencralization gap shrinking from 4.84% to 4.63%. This finding indicates that continuous learning these high-confidence patterns may not only fail to improve but could actually diminish the model's generalization ability.
(loss range 0-0.2) prediction patterns suddenly increases by 20%. This
To further delve into the impact of high-confidence patterns on model generalization, divide them into two categories: the "original" that displays small-loss before NO, and the "transformed" that becomes small-loss after NO. Next, we separately remove these categories to investigate their individual influence, as shown in Figure/ 1 (right). We can ob- serve that only removing the original high-confidence (Ori-HC) pat- terns negatively affects the model's 's generalization (5.57%), whereas only removing the transformed high-confidence (Trans-HC) pat- terns can effectively alleviate NO (4.42%). Therefore, the primary decline in the model's generalization can be attributed to the leamn- ing of these transformed high-confidence patterns. Additionally, we note that the model exhibits an uncommon memory capacity for transformed high-confidence patterns, as illustrated in Figure 2. Our analysis suggests that, compared to the original ones, DNNs show notably persistent memory for these transformed high-confidence patterns. This uncommon memory evidenced by a barely in- crease (0.01) in training loss after their removal from the training process. Building on these findings, we term this behaviour as over-memorization, characterized by DNNs suddenly becoming nigh-confidence predictions and retaining persistent memory for certain training patterns, which weakens their generalization ability,
0.09
Training Loss
0.07
0.05
0.03
Trans-HC Ori-HC
0.01
150
200
250
300
Epoch
Figure 2. The loss curves for both original and transformed high-confidence (HC) patterns after removing all HC patterns.
3.2
OVER-MEMORIZATION IN ADVERSARIAL TRAINING
In this section, we explore the over-memorization behaviour in robust overfitting (RO) and catas- trophic overfitting (CO). During both multi-step and single-step adversarial training (AT), we notice that similar to NO, the model abruptly becomes high-confidence in predicting certain adversarial patterns with the onset of RO and CO, as illustrated Figure (Ist and 2nd). Meanwhile, directly removing these high-confidence adversarial patterns can effectively mitigate RO and CO, de- tailed in Section 4.2. Therefore, the combined observations suggest a shared behaviour that the meremimmoizttion of certain training patterns impairs the generalization capabilities of DNNs.
Besides, most of the current research on RO and CO primarily focuses on the perspective of adversarial patterns. In this study, investigate the AT-trained model's memorization effect on natural patterns,
4

---

Published as a conference paper at ICLR 2024
100
100%
100%
Adversotial Paltern
Overlap Rate (%)
-Grup (IC)
Accuracy (%)
2
60%
60%
60
-Grup3
40%
40%
Natural Traning Nataral Tosins
20%
2.5) (1.)
151
20%
1255 00.
2.5
20
PGD Tosting
1.05
[0.5. 12) (0.,
20
Grou
100 Epoch
150
200
100
150
Epoch
Epach
Epoch
(a) Multi-step adversarial training.
100
100%
100%
-Grosp (BC)
Ascuncy (P6
80
Adversarial Pattien
80%
Natural Pattern
Overlap Rate (%)
-Grup
-noupp
60
60%
60%
60
Narural
20
HESM Testing
Nauml Testing
Tealig
20%
(15.228)
(6..
(.0.
1.5
20%
10.5,100 0.00
125.00
12
20
Guop (LC)
50 Epoch
75
100
Epoch
Epoch
Epoch
(b) Single-step adversarial training.
Figure 3. 1st Panel: The training and test accuracy of adversarial training. 2nd/3rd Panel: Proportion of adversarial/naatural patterns based varying training loss ranges. 4th Panel: The overlap rate between natural and adversarial patterns grouped by training loss rankings.
as illustrated in Figure 3 (3rd). With the onset of RO and CO, observe a sudden surge in high- confidence prediction natural patterns within the AT-trained model, similar to the trend seen in adversarial patterns. Intriguingly, the AT-trained model never actually encounters natural patterns, only interacts with the adversarial patterns generated from them. Building on this observation, hypothesize that the DNNs' memory tendency is similar between the natural and adversarial pattern for a given sample. To validate this hypothesis, we ranked the natural patterns by their natural training loss (from high-confidence to low-confidence), and subsequently divided them into ten groups, each containing 10% of the total training patterns. Using the same approach, we classify the adversarial patterns into ten groups based on the adversarial training loss as the ranking criterion. From Figure 3 (4th), we can observe significantly high overlap rate (90%) between the high-confidence predicted natural and ad- versarial patterns. This observation suggests that when the model over- wer-memorizes an adversarial pattern, it tends to simultaneously exhibit high-confidence in predicting the corresponding natural pat- tern. We also conduct the same experiment in TRADES (Zhang et al., 2019), which encounters natural patterns during the training process, and reaches the same observation, as shown in Appendix B. To further validate this similar memory tendency, detect the high-confidence adversarial pattern solely based on their corresponding natural training loss. From Figure clearly distinguish the high-confidence and low-confidence adver- sarial patterns by classifying their natural training loss. Therefore, by leveraging this tendency, we can reliably and consistently iden- tify the meeorieeooizttoon pattern by exclusively focusing on the natural training loss, regardless of the training paradigm.
Adversarial Training Loss
Natuural Pattern
-Other Natural Patter
attempt
are able
50
100
150
200
Epoch
Figure 4. The average loss of adversarial pattern grouped by natural training loss.
3.3 PROPOSED APPROACH
Building on the above findings, we propose a general framework, named Distraction Over- Memorization (DOM), which is designed to proactively prevent the model from over-memorization training patterns, thereby eliminating different types of overfitting. Specifically, we first establish a fixed loss threshold to identify ver-memorizzaion patterns. Importantly, regardless of the training paradigm, DOM exclusively compares the natural training loss with this established threshold. Subse- quently, framework employs two mainstream operations to validate our perspective: removal and
5

---

Published as a conference paper at ICLR 2024
Algorithm 1: Distraction Over-Memorization (DOM)
Input: Network fe. epochs E, mini-batch M, loss threshold T, warm-up epoch K., data argumentation operate D.A. data argumentation strength 3, data argumentation iteration ?.
for t 1... F; ... ...
ENT f(z, 0);
if DOMRE and K. then if Natural Training then
0 0 Ve (ENNT(ENN > T));
if Adversarial Training then EAT f(a D.v:);;
== Vo (Exz > T));
else if DOMDA and K. then
while n <= do
if (TDA (Z v)) > then
2DATENT <T) (x((T( < T)) and break;
else
TeNT < <) ulÊNT < 7) +1 (1- 3) + DA < <T)) +
if Natural Training then
EDA NT a. 2: 0); 8
DA-NNN):
if Adversarial Training then
(DA-AT f(EDA ÂDA: (DA:;;0); 0; # - V (T)
clse
# Standard optimize network parameter 0 according to training paradigm.
data augmentation denoted as DOMRE and DOMDA. respectively, For DOMRE. we adopt a straight- forward approach to remove all high-confidence patterns without distinguishing over-memorization and normal- memaalmmemrizztion. This depends on the observation that DNNs exhibit a significantly per- sistent memory for wer-memorizztion patterns, evidenced in Figure 2. As training progresses, we expect the loss of normal-memooization patterns to gradually increase, eventually surpassing the threshold and prompting the model relearn. In contrast, the loss for over-mmmooization patterns is unlikely to notably increase with further training, hindering their likelihood of being relearned.
On the other hand, DOMDA utilizes data augmentation techniques to weaken the model's confidence in over-memorization patterns. Nonetheless, research by Rice et al. (2020); Zhang et al. (2022) have shown that the ability original data augmentation is limited for mitigating RO and CO. From the perspective of mer-iiemoizttion, we employ iterative data augmentation on high-confidence patterns to maximization reduce the model's reliance on them, thereby effectively mitigating overfitting. The implementation of the proposed framework DOM is summarized in Algorithm 1.
EXPERIMENTS
In this section, we conduct extensive experiments to verify the effectiveness DOM, including exper- iment settings (Section 4.1), performance evaluation (Section 4.2), and ablation studies (Section 4.3).
4.1 EXPERIMENT SETTINGS
Data Argumentation. The standard data augmentation techniques random cropping and horizontal flipping are applied in all configurations. For DOMDA, we use two popular techniques, AUG- MIX (Hendrycks et al., 2019) and RandAugment (Cubuk et al., 2020).
Adversarial Paradigm. We follow the widely-used configurations, setting the perturbation budget as = 8/255 and adopting the threat model as Loe. For adversarial training, we employ the default PGD-10 (Madry al., 2018) and RS-FGSM (Wong al., 2019) to generate the multi-step and single-step adversarial perturbation, respectively. For the adversarial test, we use the PGD-20 and Auto Attack (Croce & Hein, 2020) to evaluate model robustness.
Datasets and Model Architectures We conducted extensive experiments on the benchmark datasets Cifar- 10/100 (Krizhevsky et al., 2009), SVHN (Netzer al., 201 and Tmag-em (Netzer et al.,

---

Published as a conference paper at ICLR 2024
Table The CIFAR-10/100 hywe000hpppprrg hyperparameter settings are divided by slashes. The 1st are general settings, and the 4th to 9th columns are DOM settings.
3rd columns
Learning rate (Lz. decay)
Training Epoch
Warm-up Epoch
Loss Threhood
AUGMIX Strength
AUGMIX lteration
RandAugment Strength
RandAugment Iteration
Method
Natural
0.1 (150, 225)
300
150
0.240.45
50%
3/2
10%
3/2
PGD-10
0.1 (100, 150)
200
100
1.5/4.0
50%
2
0%
2
RS-FGSM
0.0-0.2 (cyclical)
10050
5025
2,04.66
50%
5
10%
3
Table 2. Natural training test on CIFAR10/100. The results averaged over 3 random seeds and reported with the standard deviation.
CIFARI
CIFARIOO
Network
Method
Best (4)
Last (U)
Diff (L)
Best
(()
Dift (L)
Baseline
4,70 0.09
4.84 = U.D4
-0.14
21.32 a 0.03
21.61 a 0.03
-0.29
DOMRE
4.55 0.19
4.63 0.19
-0.08
21.35 + 0.20
21.44 + 0.06
-0.09
PrenctResNet-18
AUGMIX
0.18
21.79 0.32
+ +DOMDA
4.13 0.14
4.24 = 0.02
-0.11
21.67 + 0.06
21.79 + 0.30
-0.12
RandAngment
4,02 0.08
4.31 0,06
-0.29
21.13 a 0.05
21.61 a: 0.11
-0,48
+ DOMMDD
3.96 + 0.08
4.07 = 0.13
.0.11
21.11 + 0.09
21.49 + 0.06
+0.38
Raseline
0.11
3.86 0.19
-0.15
18.24 + 0.19
18.57 + 0.06
-0.33
DOMRE
3.63
3,75 0.11
-0.12
18.30 + 0.04
18.52 + 0.07
-0.22
AUGMIX
+.00
3.69 = 0.13
-0.26
18.23 + 0.18
18.43 0.21
-0.20
WideResNet-34
+DOMpA
3,42
3.58 0.03
-0.16
18.18 0.18
18.36 0.01
-0.18
RandAngment
3.20
0.08
17.61 0.10
-0.36
+ DOMDA
2.98+
3.20
-0.22
17.88 0.18
17.93 + 0.01
-0.05
2011). The settings and results for SVHN and Imny-Im are provided in Appendix and Ap- pendix D, respectively. We train the PreactResNet-18 18 (He et al., 2016), WideResNet-34 (Zagoruyko & Komodakis, 2016) and ViT-small (Dosovitskiy et al., 2020) architectures on these datasets by utilizing the SGD optimizer with momentum of 0.9 and weight decay of 5 x 10-4. The results of ViT-small can be found in Appendix E. Other hyperparameters setting, including learning rate schedule, training epochs E, warm-up epoch KC, loss threshold T, data augmentation strength / and data augmentation iteration are summarized in Table 1. We also evaluate our methods on the gradual learning rate schedule, as shown Appendix
4.2 PERFORMANCE EVALUATION
Natural Training Results. In Table 2, present evaluation of the proposed framework against competing baselines on 10/100 datasets. We report the test accuracy at both the highest (Best) and final (Last) checkpoint during training, well the generalization gap between them (Diff). Firstly, we can observe that DOMRE, which is trained on a strict subset of natural patterns, can consistently outperform baselines at the final checkpoint. Secondly, DOMDA can achieve superior performance at the both highest and final checkpoints. It's worth noting that DOMDA applies data augmentation limited epochs and training patterns. Finally and most importantly, both DOMRE and DOMDA can successfully reduce the model's generalization gap, which substantiates our perspective that over-memorization hinders model generalization, and preventing it can alleviate overfitting.
Adversarial Training Results. To further explore the over-memorization, we extend our framework both multi-step and single-step AT. Importantly, the detection of over-memorization adversarial patterns relies exclusively on the loss of the corresponding natural pattern. From Table 3, it's evident that both DOMRE and DOMDA are effective eliminating RO under PGD-20 attack. However, under Auto Attack, the DOMDA remains its superior robustness, whereas DOMRE is comparatively weaker. This difference Auto Attack could be attributed to DOMRE directly removing training patterns, potentially ignoring some useful information. Table 4 illustrates that both DOMRE and DOMpA are effective in mitigating CO. However, the proposed framework shows its limitation in preventing CO when using DOMDA with AUGMIX on CIFAR100. This result could stem from the weakness of the original data augmentation method, which remains inability to break over-memorization after the framework's iterative operation.
7

---

Published as a conference paper at ICLR 2024
Table 3. Multi-step adversarial training test accuracy on CIFARI0/100. The results are averaged over 3 random seeds and reported with the standard deviation.
Danaset
Method
Natural (1)
PGD-20 (1)
Auto Attack (1)
Natural (1)
PGD-20
Auto Attack (1)
Baseline
81.70 a 0,48
52.33 a 0.25
48.02 a: 0.49
83.59 0.15
45.16 1.20
42,70 a 1.16
DOMpp
AUGMIX
79.92 0.77
$2.76 0.07
47.91
81,07 0.39
47.71 1.50
44.71 1.06
CIFARIO
-DOMpA
80.57 0,98
53.54 0.15
47.98 0.14
$4.15 0.26
49.31 0.83
45.51 0.85
Rand&Aggmmnt
48.39 0.03
62.40 1.46
47.81
DOMpA
83.49 a 0.69
52.83 0.06
48.41 +: 0.28
83.74 a 0.57
50.39 0,91
46.02 0.69
Baseline
56.04 0.33
29.32 0.04
25.19
$7.09 0.32
21.92 0.53
19.81 0.49
DOMBE
52,70 0.71
29.45 0.33
20:41 0.56
52.67 + 0.96
25,14 0.39
17.59 0.35
AUGMIX
52.46 0.73
29.54 0.24
24.15 0.14
57.53 0.62
24.15 = 0.10
21.22 0.08
CIFAR1O0
DOMDA
0.02
57.70 0.02
0.36
RandAugment
55.12 1.33
28.62 0.04
23.80 0.21
$5.71 #: 1.62
23. #: 1.32
20.03 1.00
DOMDA
24.10 0.38
56.21
25.84
20.79
Table 4. Single-step adversarial training final checkpoint's test accuracy on CIFARI0/100. The results are averaged over 3 random seeds and reported with the standard deviation.
CIFARIO
CIFARI00
Method
Natural (1)
PGD-20 (f)
Auto Attack (1)
Natural (1)
PGD-20 (f)
Auto Attack (1)
Buseline
87.77 3.02
0.00 0.00
0.00 + 0.00
60.28 3.34
0.00 = 0.00
0.00 + 0.00
DOMRE
71.66 0.29
47.09 0,36
17.10 0.82
26.39 1.06
12.68 0.62
7.65 0.59
AUGMIX
0.00
0.00
DOMDA
84.31 0.59
45.15 # 0.06
41.16 + 0.11
63.03 0.19
0.00 = 0.00
0.00 + 0.00
RandAugment
84,63 0.83
0.00 0.00
0.00 + 0.00
59.39 0.62
0.00 = 0.00
0.00 0.00
DOMDA
84.52 0.28
50.10 a 1,53
42.53 a 1.41
55.09 a 1.73
27.44 a 0.12
21.38 a 0,78
Overall Results. In summary, the DOM framework can effectively mitigate different types of over- fitting by consistently preventing the shared behaviour over-memorization, which first-time employs unified perspective to understand and address overfitting across different training paradigms.
4.3 ABLATION STUDIES
In this section, we investigate the impacts of algorithmic components using PreactResNet-1e 18 on CIFAR1O. For the loss threshold and warm-up epoch selection, we employ DOMRE. while for data augmentation strength and iteration selection, use DOMDA with AUGMIX in the context of NT. When tuning a specific hyperparameter, we keep other hyperparameters fixed.
Loss Threshold Selection. To investigate the role of loss threshold, we present the variations in test error across three training paradigms. As depicted in Figure 5 (a: left), we can observe that employing small threshold might not effectively filter out over-memorization patterns, resulting in suboptimal generalization performance. On the other hand, adopting a larger threshold might lead the exclusion lot of training patterns, consequently resulting in the model underfitting. In light of this trade-off, we set the loss threshold as 0.2 for NT. Interestingly. this trade-off does not seem exist in the context of AT, where higher loss thresholds tend to result higher PGD robustness, as shown in Figure 5 (a: middle and right). Nevertheless, the above experiments indicate that this approach could also increase the vulnerability to Auto Attack. Hence, determining an appropriate loss threshold is critical for all training paradigms. We also evaluate methods unified adaptive loss threshold (Berthelot et al., 2021; Li et al., 2023) as shown Appendix G.
Warm-Up Epoch Selection. The observations from Figure 5 (b: left) indicate that short warm- up period might hinder the DNNs from learning essential information, leading to a decline in the performance. Conversely, longer warm-up period cannot promptly prevent the model from over- memorizing training patterns, which also results in compromised generalization performance. Based on this observation, simply align the warm-up epoch with the model's first learning rate decay.
8

---

Published as a conference paper at ICLR 2024
60
Roust Accuracy (%)
Roust Accuracy (%)
Test Error (%)
4.84
4.85
178
4.77
4.72
40
Threshold
Threshold
-0.0
-0.5
-1.0
30
0000
500
01
$1
OCO
$20
0P0
S#0
05'0
50
100
150
200
25
50
75
100
Threshold
Epoch
Epoch
(a) The role of loss threshold in natural, multi-step and single-step adversarial Praiiig((from left right).
95.37
95.76
95.76
Test Accuracy (%
Test Accuracy (%)
95.65
95.70
Test Accuracy (%)
95.67
95.17
95.18
95.10
95.55
95.51
95.00
95.48
95.48
95.45
95.38
95.35
Base
100
125
150
175
200
Base 0%
25% 50%
75%
100%
Base
3
5
line
Argumentation (h) The role warm-up epoch, data argumentation strength and itcration (from left to right).
Warm-up Epoch
line Data Argumentation Strength
Figure 5. Ablation Study
Data Augmentation Strength and Iteration Selection. We also examine the impact of data augmen- tation strengths and iterations, as shown Figure (b: middle and right). We can observe that, even when the augmentation strength is set to 0% or the number of iterations is limited to 1, our approach can still outperform the baseline (AUGMIX). Moreover, both insufficient (weak strengths or few iterations) and aggressive (strong strengths or excessive iterations) augmentations will lead to subpar performance. This is due insufficient augmentations limiting the pattern transformation to diverse styles, while aggressive augmentations could exacerbate classification difficulty and even distort the semantic information (Bai et al., 2022; Huang et al., 2023b). Therefore, we select the augmentation strength 50% and iteration as 3 to achieve the optimal performance. The computational overhead analysis can be found in the Appendix H.
CONCLUSION
Previous research has made significant progress in understanding and addressing natural, robust, and catastrophic overfitting, individually. However, the common understanding and solution for these overfitting have remained unexplored. To the best of our knowledge, our study first-time bridges this gap by providing unified perspective overfitting. Specifically, examine the memorization effect in deep neural networks, and identify shared behaviour termed over-memorization across various training paradigms. This behaviour is characterized by the model suddenly becoming high- confidence predictions and retaining a persistent memory in certain training patterns, subsequently resulting in a decline in generalization ability. Our findings also reveal that when the model over- memorizes an adversarial pattern, it tends to simultaneously exhibit high-confidence in predicting the corresponding natural pattern. Building on the above insights, we propose general framework named Distraction meeoiiamoo (DOM), designed to holistically mitigate different types of overfitting by proactively preventing over-memorization training patterns.
Limitations. This paper offers a shared comprehension and remedy for overfitting across various training paradigms. Nevertheless, a detailed theoretical analysis the underlying mechanisms among these overfitting types remains open question for future research. Besides, the effectiveness of the proposed DOMpA method is dependent on the quality of the original data augmentation technique, which could potentially limit its applicability in some scenarios.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
The authors would like thank Huaxi Huang, reviewers and area chair for their helpful and valuable comments. Bo Han was supported by the NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation Nos. 2022A1515011633 and 2024A151501 S1559199,, HKBU Faculty Niche Research Areas No. RC-FNRA-IG/22 -FNNA-IGG22-2MSSIO00 and HKBU CSD Departmental Incentive Scheme. Tongliang Liu is partially supported by the following Australian Research Council projects: F1220100318, DP220102121,
LP220100527, L
LPP20200999, and IC110100031.
REFERENCES
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems, 33:16048-166 2020.
Devansh Arpit, Stanistaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning. pp. 233-242.
PMLR, 2017.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. pp. 274-283. PMLR, 2018.
Jimmy Ba and Brendan Frey. Adaptive dropout for training deep neural networks. Advances in neural information processing systems, 26, 2013.
Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang. Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. Understanding and improving early stopping for learning with noisy labels. Advances in Neural Information Processing Systems, 34:24392-24403, 2021.
Yingbin Bai, Erkun Yang, Zhaoqing Wang, Yuxuan Du Bo Han, Cheng Deng, Dadong Wang, and Tongliang Liu. Rsa: Reducing semantic shift from aggressive augmentations for self-supervised learning. Advances in Neural Information Processing Systems, 35:21 1211212211, 2022.
David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, and Alexey Kurakin. Adamatch: A unified approach to semi-supervised learning and domain adaptation. In International Conference on Learning Representations, 2021.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. Advances in neural information processing systems, 32, 2019.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In International Conference Learning Representations, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Francesco Croce, Sven Gowal, Thomas Brunner, Evan Shelhamer, Matthias Hein, and Taylan Cemgil. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning, pp. 4421 4435. PMLR, 2022.
Ekin D Cubuk, Barret Zoph. Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation policies from data, arXiv preprint /tX.xxv:00000000000 2018
Ekin Cubuk, Barret Zoph, Jonathon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition workshops, pp. 702-703, 2020.
Terrance Vries and Graham W Taylor. Improved regularization convolutional neural networks
with cutout. arXiv preprint arXiv: aet.::00800.5555 2017.
10

---

Published as a conference paper at ICLR 2024
Tom Dietterich. Overfitting and undercomputing in machine learning. ACM computing surveys (CSUR), 27(3)3326 -27, 1995.
Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang. Zhijie Deng, Hang Su, and Jun Zhu. Exploring memorization in adversarial training. In International Conference on Learning Representations 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
Vitaly Feldman. Does learning require memorization? a short tale about long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 959,959. 2020.
Zeinab Golgooni, Mehrdad Saberi, Masih Eskandar, and Mohammad Hossein Rohban. Zerograd: Costless conscious remedies for catastrophic overfitting in the fgsm adversarial training. Intelligent Systems with Applications, 19:200258, 2023.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint 1oXiv:2010.03593, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference computer vision, pp. 630-645. Springer, 2016.
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2019.
Zhichao Huang, Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, and Jue Wang. Fast adversarial training with adaptive step size. IEEE Transactions on Image Processing, 2023a.
Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing aat-of-distribution examples via augmenting content and style. In ICLR, 2023b.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson Averaging weights leads to wider optima and better generalization. In 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, pp. 876-885. Association For Uncertainty in Artificial Intelligence (AUAI), 2018
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Muyang Li. Runze Wu, Haoyu Liu, Jun Yu, Xun Yang, Bo Han, and Tongliang Liu. Instant: Semi- supervised learning with dnstance thresholds. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Runqi Lin, Chaojian Yu, and Tongliang Liu. Eliminating catastrophic overfitting via abnormal adver- sarial examples regularization. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a.
Yexiong Lin, Yu Yao, Yuxuan Du, Jun Yu, Bo Han, Mingming Gong, and Tongliang Liu. Do we need to penalize variance of losses for learning with label noise? arXiv preprint 2xXiv:2201.12739 2022.
Yexiong Lin, Yu Yao, Xiaolong Shi, Mingming Gong, Xu Shen, Dong Xu, and Tongliang Liu. Cs-isolate: Extracting hard confident examples by content and style isolation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
11

---

Published as a conference paper at ICLR 2024
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generaliza- tion in deep learning. Advances neural information processing systems, 30, 2017.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sensitivity and generalization in neural networks: an empirical study. In International Conference on Learning Reprexentations, 2018.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference Machine Learning, pp. 8093-8104. PMLR, 2020.
Leslie N Smith. Cyclical learning rates for training neural networks. In 2017 IEEE winter conference on applications of computer vision (WACV), pp- 464-472. IEEE, 2017.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj. et al. Towards efficient and effective adversarial training. Advances in Neural Information Processing Systems, 14:11821-11833 2021.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine 2014.
learning research,
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun. and Rob Fergus. Regularization of neural networks using dropconnect. In International conference on machine learning, pp. 1058-1066. PMLR, 2013.
Eric Wong, Leslic Rice, and Zico Kolter. Fast better than free: Revisiting adversarial training. In International Conference Learning Representations, 2019.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 13:2958-2969, 2020.
Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Sample selection with uncertainty of losses for learning with noisy labels. In International Conference on Learning Representations, 2021.
Xiaobo Xia, Bo Han, Yibing Zhan, Jun Yu, Mingming Gong, Chen Gong, and Tongliang Liu. Com- bating noisy labels with sample selection by mining high-discrepancy examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1833 1843, 2023.
Chaojian Yu, Bo Han, Mingming Gong, Li Shen, Shiming Ge, Bo Du, and Tongliang Liu. Robust weight perturbation for adversarial training. arXiv preprint arXiv:2205. 1482, 2022a.
Chaojian Yu, Bo Han, Li Shen, Jun Yu, Chen Gong, Mingming Gong, and Tongliang Liu. Under- standing robust overfitting of adversarial training and beyond. International Conference on Machine Learning, pp. 25595-256101 PMLR, 2022b.
Suqin Yuan, Lei Feng, and Tongliang Liu. Late stopping: Avoiding confidently learning from mislabeled examples. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 16079-16088, 2023.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Procedings of the British Machine Vision Conference 2016. British Machine Vision Association, 2016.
Matthew D Zeiler and Rob Fergus. Stochastic pooling for regularization of deep convolutional neural networks: 1st international conference on learning representations, iclr 2013. In 1st International Conference Learning Representations, ICLR 2013, 2013.
12

---

Published as a conference paper at ICLR 2024
Chaoning Zhang, Kang Zhang, Axi Niu, Chenshuang Zhang, Jiu Feng, Chang D Yoo, and In So Kweon. Noise augmentation is all you need for fgsm fast adversarial training: Catastrophic overfitting and robust overfitting require different augmentation. arXiv e-prints, pp. arXiv-2202, 2022.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107-115. 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmenta- tion. Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 2020.
Dawei Zhou, Nannan Wang, Bo Han, and Tongliang Liu. Modeling adversarial noise for adversarial training. In International Conference on Machine Learning, pp. 27353-27366. PMLR, 2022.
13

---

Published as a conference paper at ICLR 2024
A DETAILED EXPERIMENT SETTINGS
In Section 3, we conducted all experiments on the CIFAR-10 10 dataset using PreactResNet-118 We analyzed the proportion of natural and adversarial patterns by examining the respective natural and adversarial training loss. In Section 3.1, we categorized between original and transformed high- confidence patterns using an auxiliary model, which was saved at the first learning rate decay (1 (150th epoch). In Section 3.2 Figure 4, we grouped adversarial patterns based on their corresponding natural training loss, employing a loss threshold of 1.5.
B TRADES RESULTS
Adversarial Pattern
80%
R0
Acquracy
80%
60%
60 Matweneed
60
40%
40%
40
Natual Tuaninu -TRADDS Tranning
20%
125.
120.25)
20%
[1.0 1.5%
[1.5.22.
Gnuzp
Group
Group
PCDD Trating
# L...L00 1
800 0.5)
Group (LA))
50
100
50
100
700
100
200
Epoch
Epoch
Epoch
Epoch
Figure 6. TRADES adversarial training. 1st Panel: The training and test accuracy of adversarial training. 2nd/3rd Panel:
Proportion of adversaria//natural patterns based on varying training loss ranges. 4th Panel: The overlap rate between natural and adversarial patterns grouped by training loss rankings.
We further explored this observation in the LADDESttrinee model, which encounters natural patterns during the training process. From Figure 6, we can observe that TRADES demonstrates a consistent memory tendency with PGD in the over-memorization samples. This tendency manifests as, when DNNs over memorize certain adversarial patterns, they tend to simultaneously exhibit high-confidence in predicting the corresponding natural patterns.
C SETTINGS AND RESULTS ON SVHN
SVHN Settings. In accordance with the settings of Rice et al. (2020); Wong et al. (2019), we adopt gradually increasing perturbation step size the initial 10 and epochs for multi-step and single-step 1/255. Other hyperparameters
AT, respectively. In the meantime, the PGD step size is set setting, including learning rate schedule, training epochs E, loss threshold T. warm-up epoch K,, data augmentation strength 3 and data augmentation times are summarized in Table 5.
Table 5. The SVHN hyperparameter settings. The 1st to 3rd columns general settings, and the 4th
to 9th columns are DOM settings.
Lr. (I. decay)
Training epoch
Warm-up spoch
Loss threshold
AUGMIX strength
AUGMIX times
RandAngmeet strength
RandAugment
Method
times
Natural
0.01 (150, 225)
300
150
0.02
PGD-10
0.01 (100, 150)
200
100
0.75
50%
25%
2
RS-FGSM
0.0-0.01 (syelical)
20
10
1.0
50%
10%
SVHN Results. To verify the pervasive applicability of our perspective and method, extend the DOM framework to the SVHN dataset. The results for NT, multi-step AT, and single-step AT are reported in Table 6, Table 7, and Table 8, respectively. From Table is clear that both DOMRE and DOMDA not only achieve superior performance at both the highest and final checkpoints but also succeed in reducing the generalization gap, thereby effectively mitigating NO.
14

---

Published as a conference paper at ICLR 2024
Table 6. Natural training test error on SVHN. The results are averaged over 3 random seeds and reported with the standard deviation.
PreactResNet-1 18
WideResNet-34
Method
Best (L)
Last (L)
Diff (U)
Best (J)
Last (J)
Diff (J)
Baseline
3.45
3,53 0.10
-0.08
3.36
3.61 0.07
-0.25
DOMRe
3.43 0.02
3.50 0.05
-0.07
2.75 0.01
2.87 0.05
-0.12
AUGMIX
3,44 = 0.02
3.52 0.05
-0.08
3.06 0.08
3.17 + 0.08
-0.11
DDOMDA
3.44 0.06
3.50 0.05
-0.06
3.10 0.02
3.15 0.04
-0.05
RandAugment
0.07 DOMDa
3.12 0.02
-0.12
2.65 0.01
2.84 0.09
-0.19
2.98 0.01
3.07 0.03
-0.09
2.60 0.08
2.77 0.10
-0.17
From Table 7, we can observe that the DOMRE can demonstrate improved robustness against PGD, while DOMDA shows better robustness against both PGD and Auto Attack at the final checkpoint, which confirms their effectiveness eliminating RO.
Table 7. Multi-step adversarial training test accuracy on SVHN. The results are averaged over random seeds and reported with the standard deviation.
Best
Last
Method
Natural (f)
PGD-20 (f)
Auto Attack (f)
Natural (f)
PGD-20 (1)
Auto Attack (f)
Raseline
DOMRR
91.40 = 0.51
54.31 0.62
41.74 4: 0.55
91.53 = 0.41
49.59 1,37
31,22 + 1.00
92.44
53.75
93.79 =
51.12
42.73
DOMDA
92.73 = 0.51
55.31 0.23
45,72 +: 1.07
92.05 + 0.89
53.64 # 0.42
43.14 + 0.65
Rand/nggeett
93.01 0.11
54,06 0.25
46.02 0.06
93.38 = 0.81
52.36 0.36
44.28 1.21
DOMDA
93.16 0.80
56.13 0.30
44.82 1.27
92.81 = 1,31
54.01 # 1.52
44.59 + 0.67
Table 8 indicates that both DOMRE and DOMDA can effectively mitigate CO in all test scenarios. Overall, the above results not only emphasize the extensiveness of over-memorization, but also highlight the effectiveness of the DOM across diverse datasets.
Table Single-step adversarial training final checkpoint's test accuracy on SVHN. The results are averaged over 3 random seeds and reported with the standard deviation.
Method
Natural (f)
PGD-20 (t)
Auto Attack (f)
Baseline
98.21 = 0.35
0.02 0.03
0.00 0.00
DOMRE
89.96 = 0.55
47.92 # 0.63
32.22 # 1.00
AUGMIX
98.09 + 0.23
0.07 + 0.03
0.00 + 0.01
+DOMDA
90.67 = 0.49
49.88 # 0.37
37.67 0.12
RandAugment
98.04 = 0.60
0.35 0.22
0.05 0.04
DOM
85.88 # 3.02
51.57 + 1.79
36.69 1.55
D SETTINGS AND RESULTS ON TINY-MMAGENET
We also verified the effectiveness of our method the larger-scale dataset Tiny-ImageNe (Netzer
et al., 1). We set the loss threshold T to 0.2, and other hyperparameters remain as the original settings.
15

---

Published as a conference paper at ICLR 2024
Table 9. Tiny-ImageNet: The natural training test error the best and last checkpoint using PreactResNet-18. The results are averaged over random seeds and reported with the standard deviation.
Method
Best ())
Last (4)
Diff' (4)
Baseline
35.03 0.03
35.24 0.02
-0.21
DOMRE
34.89 0.02
34.99 0.01
-0.10
AUGMIX
34.98 0.02
35.15 0.03
-0.17
+ +DOMDA
34.56 + 0.04
34.77 = 0.02
-0.21
RandAugment
33.46 + 0.05
33.89 0.06
-0.45
+ DDOMAA
33.44 0.04
33.59 0.02
-0.20
Table 9 illustrates the effectiveness of our method, DOMRE and DOM on the Tiny-ImageNet dataset. These results indicate that preventing memorizzaiinn improve model performance and reduce the generalization gap on large-scale datasets.
E SETTINGS AND RESULTS ON VIT
We have validated the effectiveness of our method within CNN-based architectures, demonstrating its ability to alleviate overfitting by preventing over-memorization. To further substantiate our perspec- tive, verify method on the Transformer-based architecture. Constrained by computational resources, we trained a ViT-small model (Dosovitskiy et al., 2020), initializing it with pre-trained weights from the Timm Python library. The training spanned 100 epochs, starting with an initial learning rate of 0.001 and divided by 10 at the 50th and 75th epochs. We set the batch size to 64 and the loss threshold T 0.1, maintaining other hyperparameters as the original settings.
Table 10. Vit: The natural training test error at the best and last checkpoint on CIFAR 10. The results averaged over random seeds and reported with the standard deviation.
Method
Best (1)
Last (L)
Diff (L)
Baseline
1.49 0.01
1.75 + 0.01
-0.26
+ DOOMRE
1.47 = 0.02
1.67 # 0.01
-0.20
AUGMIX
1.24 = 0.01
1.29 0.01
-0.05
+DOMDA
1.20 + 0.01
1.27
-0.07
Rand/Augment
1.21 0.01
1.27
-0.06
DOMDA
1.17 0.01
1.22 # 0.01
-0.05
Table 10 shows the effectiveness of our method on the Transformer-based architecture. By mitigating memorizaiiin, both DOMRE and DOM not only improve model performance at both the best and last checkpoints, but also contribute to alleviating overfitting.
F GRADUALLY LEARNING RATE RESULTS
To further assess method, conducted experiments using gradual learning rate schedule (Smith, 2017) in natural training. We set the cyclical lcarning rate schedule with 300 epochs, reaching the maximum learning rate of 0.2 at the midpoint of 150 epochs.
From Table 11, it apparent that although the cyclical learning rate reduces the model's 's generalization gap. it also leads to reduction in performance compared to the step learning rate. Nevertheless, method consistently showcases its effectiveness in improving model performance and completely eliminating the generalization gap by mitigating over-memorization.
16

---

Published as a conference paper at ICLR 2024
Table 11. Cyclical learning rate:
The natural training test error the best and last checkpoint on CIFAR 10 using PreactResNet-1D 18. The results are averaged over random seeds and reported with the standard deviation.
Method
Best (4)
Last (4)
Diff (4)
Baseline
4.80 0.03
4.89 + 0.03
-0.09
+DOMR
4.79 0.02
4.79 = 0.02
-0.00
AUGMIX
4.75 0.01
4.79 0.02
-0.02
+DOMDA
4.49 + 0.01
4.49
-0.00
RandAugment
4.41 + 0.03
4.42 + 0.01
-0.01
DOMDA
4.25 0.01
4.25 + 0.01
-0.00
G ADAPTIVE LOSS THRESHOLD
By utilizing the fixed loss threshold DOM, we have effectively verified and mitigated over- memorization, which negatively impacts DNNs' generalization ability. However, general framework, finding an optimal loss threshold for different paradigms and datasets can be cumbersome. To address this challenge, propose all experimental settings. Specifically, we utilize an adaptive loss threshold (Berthelot et al., 2021), whose value is dependent the loss of the model's current training batch. For all experiments, set this adaptive loss threshold T to 40%, maintaining other hyperparameters as the original settings.
general and unified loss threshold applicable across
Table 12. Adaptive loss threshold: The natural and PGD-20 test error for natural training (NT) and adversarial training (AT) using PreactResNet-11 18. The results are averaged over 3 random seeds and reported with the standard deviation.
Dataset
Paradigm
Method
Best (1)
Last (L)
Diff (L)
Baseline
4.70 = 0.09
4.84 + 0.04
-0.14
DOMRE
4.62 0.06
4.68 0.02
-0.06
CIFAR1O NT
AUGMIX
4.35 0.18
4.52 - 0.01
-0.17
+DOMDA
4.24
4.37 # 0.08
-0.13
Rand/Augment
4.02 = 0.08
4.31 = 0.06
-0.29
+ DOMDA
3,86 + 0.05
3.94 + 0.06
-0.08
Baseline
21.32 = 0.03
21.59 + 0.03
-0.27
CIFAR100 NT
DOMRE
21.20 + 0.07
21.43 0.04
-0.23
Baseline
47.67 + 0.25
54.84 + 1.20
-7.17
CIFAR10
Multi-step AT
DOMRE
46.57 = 0.64
52.83 + 0.28
-6.26
Baseline
57.83 # 1.24
100.00 0.00
-42.17
CIFAR10
Single-step AT
DOMRE
54.52 # 0.57
56.36 # 0.92
-1.84
Table 12 demonstrates the effectiveness of the adaptive loss threshold across different paradigms and datasets. This threshold can not only consistently identify over-memorization patterns and mitigate overfitting, but also be easily transferable without the need for hyperparameter tuning.
H COMPUTATIONAL OVERHEAD
We analyze the extra computational overhead incurred by the DOM framework. Notably, both DOMRE R and DOMPA implemented after the warm-up period (half of the training epoch).
Based on Table 13, we can observe that DOM R does not involve any additional computational overhead. Although DOMpA require iterative forward propagation, its overall training time does not
17

---

Published as a conference paper at ICLR 2024
Table 13. The training cost (epoch/second) on CIFARIO using PreactResNet-18 with single NVIDIA
RTX 4090 GPU.
Method
Before warm-up (4)
After warm-up ())
Overall ())
Baseline
6.28
6.26
6.27
+ DOMRE
6.28
6.28
6.28
+ AUGMIX
12.55
12.76
12.66
+DOM
6.28
28.45
17.37
+ RandAugment
8.29
8.27
8.28
DOMDA
6.24
12.75
9.50
significantly increase, because the data augmentation is only applied to a limited number of epochs and training samples. Additionally, the multi-step and single-step AT inherently have higher basic training time (generate adversarial perturbation), but the extra computational overhead introduced by the DOM framework is relatively consistent. As a result, our approach has relatively smaller impact on the overall training overhead in these scenarios.
18


Reviews:
[{"review_id": "d2eJfnRX9I", "reviewer": "Reviewer_2dX7", "summary": "This paper considers a unified perspective on various overfitting, including NO (natural overfitting), RO (robust overfitting), and CO (catastrophic overfitting). On top of this, the authors discover the \"over-memorization\" phenomenon that the overfitted model tends to exhibit high confidence in predicting certain training patterns and retaining a persistent memory for them. Unlike previous methods, this paper proposes a general framework called DOM (Distraction Over-Memorization) to alleviate the unified over-fitting issue. Experiments show that the proposed method outperforms other baselines.", "questions": "See above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The discovery of the behavior \"over-memorization\" unifies different types of overfittings, which is of great help when analyzing the cause of overfitting.\n2. The paper is generally well-written, and the motivation is stated clearly.\n3. The proposed DOM framework seems promising.", "weaknesses": "1. In the DOM framework, the loss threshold is set with a fixed value. However, with different datasets and loss functions, the optimal threshold could be different. Therefore, the given threshold may not be general on other occasions. The authors should further conduct ablation studies about this and discuss how to overcome this issue.\n2. The experiment settings are not precisely introduced in 3.1 and 3.2, making these conclusions challenging to reproduce. \n3. In section 3.2, the authors claim, “the AT-trained model never actually encounters natural patterns.” However, methods like TRADES do encounter natural patterns. What will happen in this case? Are the conclusions observed in this paper still applicable?\n4. Why are there many 0.00 in Table 4? The authors need to give more explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "daWwPiD9UZ", "reviewer": "Reviewer_3ew1", "summary": "The paper provides an empirical investigation into the generalization capabilities of deep neural networks (DNNs), focusing on understanding various facets of overfitting. The authors introduce the concept of over-memorization, a phenomenon where DNNs excessively retain specific training patterns, leading to diminished generalization. To mitigate this issue, the paper suggests techniques such as the removal of high-confidence natural patterns and the application of data augmentation. The effectiveness of these strategies is demonstrated through a series of experiments.\n\nThis paper makes a valuable contribution to the field by shedding light on the over-memorization behavior in DNNs and its implications for generalization. By addressing the highlighted areas for improvement, the authors have the potential to further enhance the significance and applicability of their work.", "questions": "1. Expand Empirical Analysis: To strengthen the paper's contributions, the authors should consider conducting additional empirical analyses across diverse network architectures and datasets.\n\n2. Deepen Analysis on Overfitting Types: A more nuanced exploration of the varying impacts on different types of overfitting would provide a richer understanding of the phenomena at play.\n\n3. Consider Larger-Scale and Diverse Architectures: Incorporating experiments with larger datasets and a variety of neural network architectures would ensure that the findings are more widely applicable and representative of the broader deep learning landscape.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clarity and Structure: The paper is commendable for its well-organized structure and clear exposition. The authors have provided a thorough background and review of related work, successfully setting the stage for their empirical analysis.\n\n2. Robust Experimental Design: The experimental setup is meticulously designed, encompassing various types of overfitting and delving into the over-memorization behavior of DNNs. This comprehensive approach enhances the validity of the findings.\n\n3. Novel Insight into Overfitting: The identification of over-memorization as a common thread linking different types of overfitting is an innovative contribution. This insight adds depth to our understanding of how overfitting impacts the generalization abilities of DNNs.", "weaknesses": "1. Limited Scope of Empirical Analysis: The paper's empirical analysis predominantly focuses on a specific network architecture and dataset. Expanding the analysis to include a wider array of cases or providing a theoretical framework to support the observed behaviors would bolster the generality and impact of the findings.\n\n2. Partial Improvement on Overfitting Types: According to the results presented in Tables 2-4, the proposed strategies seem to predominantly ameliorate Class Overfitting (CO), with only marginal improvements on Natural Overfitting (NO) and Random Overfitting (RO). A more detailed exploration of why these discrepancies occur would provide valuable insights.\n\n3. Need for Larger-Scale Evaluation: The experiments are confined to relatively simple datasets (CIFAR-10/100) and ResNet-based architectures. Extending the evaluation to encompass larger-scale datasets and alternative architectures, such as transformers, would enhance the representativeness of the results and the applicability of the findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "10Es8yxA2K", "reviewer": "Reviewer_mcnh", "summary": "This paper proposes a general framework for explicitly preventing over-memorization by either removing or augmenting the high-confidence natural patterns. It is based on the observation that the model suddenly exhibits high confidence in predicting certain training patterns, which subsequently hinders the DNNs’ generalization capabilities.", "questions": "Refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Strength:**\n\n-   This paper is overall well-structured and easy to follow.\n-   Extensive empirical evaluation with various training paradigms, baselines, datasets, and network architectures demonstrates its effectiveness. Results are reported with the standard deviation.\n- Significant performance improvements are demonstrated.", "weaknesses": "**Weakness**\n\n-   According to Figure 5, the proposed method may require careful hyper-parameter (i.e. loss threshold) selection, which could be a significant drawback.\n-   The proposed method might result in repeated gradient computation and extensive extra computation. It is also interesting to include a detailed analysis of the introduced extra computation.\n-   The terminology \"pattern\" might be confusing and could be further explained. Does it refer to specific samples in datasets?\n-   Lack of results on large-scale datasets. It will be convincing to have some on Tiny-ImageNet or ImageNet\n-   Lack of results on diverse network backbone architectures beyond ResNets.\n-   As discussed in the related works, there are various techniques for mitigating the overfitting issues. Comparisons with other techniques like dropout, ensemble, smoothing, etc. can be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pg0UUaYmgD", "reviewer": "Reviewer_uFwf", "summary": "This paper analyzes three types of overfitting (natural, robust, and catastrophic) observed during the training process of deep neural networks and introduces methodologies to mitigate these phenomena. The authors are particularly motivated by the observation that, during periods of learning decay of standard training, the training loss for certain datasets sharply decreases. They designate these specific datasets as \"transformed data\" to differentiate them from the rest. When this transformed data is excluded from training, a reduction in the generalization gap is observed. This trend is similarly noted in settings where both robust and catastrophic overfitting are evident. Drawing from these observations, it is inferred that the transformed data might be excessively memorized, leading to overfitting. To counteract this, the authors propose the \"distraction over memorization (DOM)\" methodology, which emphasizes data augmentation specifically for the transformed data. Experimental results suggest that models trained using this approach exhibit a superior generalization gap compared to those trained with data augmentation applied across the entire dataset.", "questions": "- When compared to the analysis performed in the previously cited study (MLCAT) mentioned under weaknesses, are there notable strengths in this paper that I might have missed, aside from the observation that similar phenomena manifest across standard, adversarial, and fast adversarial settings?\n- In the \"distraction over memorization\" methodology, is there a specific reason for applying data augmentation iteratively rather than in a straightforward manner?\n- Has the study investigated whether similar phenomena occur with learning rate scheduling methods that decrease at a more gradual pace, such as cosine, as opposed to the step learning decay?\n- Are there any experimental results comparing the proposed approach to traditional methodologies under the same settings?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper demonstrates that natural overfitting can be mitigated by removing data characterized by a rapid decrease in training loss, termed \"transformed data.\" Through this analysis, the authors highlight the occurrence of overfitting in standard settings due to such data and propose a method to distinguish data that has been excessively memorized. Furthermore, the properties of transformed data are not limited to natural overfitting; they exhibit similar trends in other types of overfitting, namely robust and catastrophic overfitting. The authors suggest a universal overfitting mitigation method by applying various data augmentation techniques to the transformed data. Experimental results are presented to validate the efficacy of this approach.", "weaknesses": "The motivation behind this paper, specifically the analysis of transformed data, has already been explored in a paper that introduced the MLCAT methodology [1]. The distinction is that the previous study limited its analysis to robust overfitting, whereas the current paper expands the analysis to three types of overfitting, demonstrating that these phenomena manifest commonly across all three. However, given that there isn't much difference in the learning algorithms or model structures between the standard, adversarial, and fast adversarial settings, one could easily anticipate that the characteristics of transformed data in the adversarial setting, as delineated in MLCAT [1], would manifest similarly in both the standard and fast adversarial settings. Therefore, the current analysis does not offer much novelty beyond the findings of the previous study. While the proposed methodology of applying data augmentation specifically to transformed data does have the advantage of being universally applicable to various types of overfitting, it only demonstrates an improved generalization gap in comparison to the baseline model. Given the inherent differences in training data for the standard, adversarial, and fast adversarial settings, one might question the necessity of a universally applicable overfitting mitigation method. To bolster this claim, the authors should compare the proposed method against methodologies in individual overfitting studies (natural, robust, catastrophic) and demonstrate that their approach offers competitive performance.\n\n[1] Chaojian Yu, Bo Han, Li Shen, Jun Yu, Chen Gong, Mingming Gong, and Tongliang Liu. Understanding robust overfitting of adversarial training and beyond. In International Conference on Machine Learning, pp. 25595–25610. PMLR, 2022b.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)