Bag of Tricks for FGSM Adversarial Training
Authors: Zichao Li
Li Liu
Zeyu Wang
Yuyin Zhou
Cihang Xie
Venue: NeurIPS 2022 Submitted (2022)
Keywords: adversarial training

Abstract: Adversarial training (AT) with samples generated by Fast Gradient Sign Method (FGSM), also known as FGSM-AT, is a computationally simple method to train robust networks. However, during its training procedure, an unstable mode of ``catastrophic overfitting'' has been identified in~\cite{Wong2020FastIB}, where the robust accuracy abruptly drops to zero within a single training step. Existing methods use gradient regularizers or random initialization tricks to attenuate this issue, whereas they either take high computational cost or lead to lower robust accuracy. In this work, we provide the first study which thoroughly examines a collection of tricks from three perspectives: Data Initialization, Network Structure, and Optimization, to overcome the catastrophic overfitting in FGSM-AT. Surprisingly, we find that simple tricks, i.e., masking partial pixels (even without randomness), setting a large convolution stride and smooth activation functions, or regularizing the weights of the first convolutional layer can effectively tackle the overfitting issue. Extensive results on a range of network architectures validate the effectiveness of each proposed tricks, and the combinations of tricks are also investigated. For example, trained with PreActResNet-18 on CIFAR-10, our method attains 51.3\% accuracy against PGD-10 attacker and 46.4\% accuracy against AutoAttack, demonstrating that pure FGSM-AT is capable of enabling robust learners. We will release our code to encourage future exploration on unleashing the potential of FGSM-AT.

Bag of Tricks for FGSM Adversarial Training
Anonymous Author(s)
Affiliation
Address
email
Abstract
Adversarial training (AT) with samples generated by Fast Gradient Sign Method (FGSM), also known as FGSM-AT, is computationally simple method to train robust networks. However, during its training procedure, an unstable mode of "catastrophic overfitting" has been identified in Wong et al. (2020), where the robust accuracy abruptly drops to zero within a single training step. Existing methods use gradient regularizers or random initialization tricks to attenuate this issue, whereas they either take high computational cost lead to lower robust accuracy. In this work, we provide the first study which thoroughly examines a collection of tricks from three perspectives: Data Initialization, Network Structure, and Optimization, to overcome the catastrophic overfitting in FGSM-AT. Surprisingly, we find that simple tricks, i.e., masking partial pixels (even without randomness), setting a large convolution stride and smooth activation functions, or regularizing the weights the first convolutional layer can effectively tackle the overfitting issue. Extensive results on a range of network architectures validate the effectiveness of each proposed tricks, and the combinations of tricks also investigated. For example, trained with PreActResNet-11 18 on CIFAR-10, method attains 51.3% accuracy against PGD-10 10 attacker and 46.4% accuracy against AutoAttack, demonstrating that pure FGSM-AT is capable of enabling robust learners. We will release our code to encourage future exploration on unleashing the potential of FGSM-AT.
5
6
11
12
13
14
17
18
19
Introduction
20
Convolution neural networks (CNNs), though achieving compelling performances various visual recognition tasks, vulnerable to adversarial perturbations Szegedy et al. [2013). To effectively defend against such malicious attacks, adversarial examples are utilized as training data for enhancing
23
24 model robustness, process known as adversarial training (AT). To generate adversarial examples,
25 one of the leading approaches is to perturb the data using the sign of the image gradients, namely the
26 Fast Gradient Sign Method (FGSM) Goodfellow et 2015).
The adversarial training with FGSM (FGSM-AT) is computationally efficient, and it lies the founda- tion for many followups Kurakin et al. 20166) Madry al. (2018) Zhang al. [2019] Nonetheless, interestingly, FGSM-AT is not widely used today because of the catastrophic overfitting: the model robustness will collapse after a few training epochs Wong et al. (2020) To mitigate the catastrophic overfitting and stabilize FGSM-AT, several methods have been proposed. For instance, WWng et al. 2020] pre-add uniformly random noises to images to generate adversarial examples, i.e., turn the FGSM attacker into the PGD-1 attacker. [ndriishchhenko and Flammarion (2020) propose GradAlign, which regularizes the AT via maximizing the gradient alignment of the perturbations. While these approaches successfully alleviate the catastrophic overfitting, some limitations... For
28
29
30
31
32
33
Submitted to 35th Conference Neural Information Processing Systems (NeurIPS 2021). Do distribute.

---

36
example, GradAlign requires an extra forward pass compared to the vanilla FGSM-AT, which sig- nificantly increases the computational cost; Fast-AT' in Wong et al. (020) shows a relatively lower robustness, and may still collapse training with larger networks.
37
38
39 In this paper, aim develop more effective and computationally efficient solutions for attenuating
40 this catastrophic overfitting. Specifically, revisit FGSM-AT and propose to stabilize its training from the following three perspectives:
41
42
i' Data Initialization. Following the idea of adding random perturbations Madry et al. 2018 Wong et al. 2020]. we propose to randomly mask a subset of the input pixels to stabilize FGSM-AT,
43
44
dubbed FGSM-Mask. Surprisingly, additional analysis suggests that the masking process does not necessarily need to be set as random during training-we find that applying a pre-defined masking pattern the training set also effectively stabilizes FGSM-AT. This observation also holds for adding perturbations the initialization in Wong al. 2020 challenging the general belief that randomness is the key factor for stabilizing AT.
45
46
49
Network Structure. We identify two architectural elements that affect FGSM-AT. Firstly, in addition boosting robustness shown in Xie et al. (2020), , we find a smoother activation function make FGSM-AT more stable. Secondly, we find vanilla FGSM-AT can effectively train ViTs without showing catastrophic overfitting. We conjecture this phenomenon may be related to how CNNs and ViTs extract features: i.e., CNNs extract features from overlapped image regions (where stride size < kernel size), while VIT extract features from non-overlapped image patches (where stride size = kernel size). By simply increasing the stride size of the first convolution layer in CNN, we find the resulted model can stably train with FGSM-AT.
51
52
58
54
55
56
57
Optimization. GradAlign Andriushchenko and Flammarion 2020 stabilizes the FGSM-AT by setting the norm of the gradients as a regularization term. To further reduce the computational cost, we propose ConvNorm, regularization term that simply constrains the weights of the first convolution layer. Different from GradAlign which introduces significant amount of extra computations, our ConvNorm can work as nearly computationally efficient as the vanilla FGSM-AT.
58
59
61
Our contributions. In summary, we discover bag of tricks that effectively alleviate the catastrophic overfitting in FGSM-AT from different perspectives. We extensively validate the effectiveness of our methods with a range of different network structures on the popular CIFAR-10 dataset. Based on our results, we can conclude that the pure FGSM-AT is capable of enabling robust learners.
63
64
65
Preliminaries
66
67
Given neural classifier ff with parameters 6, we denote x and y as input data and labels from the data generator D, respectively.. 8 represents the perturbations and the cross-entropy loss typically used for image classification tasks.
68
89
70
Adversarial Training: We can formulate the adversarial training as an optimization problem Madry et al. 2018 as:
71
min B(e,)) max L(Jo(x + +),,
SEA
72
Among different methods for generating adversarial examples, we chose two popular ones study:
FGSM: Goodfellow et al. 2015 first propose Fast Gradient Sign Method (FGSM) to generate the
74
perturbation ó follows:
Esign(V, VV=L(f((z),
(2)
75 PGD: Madry et al. 2018 propose a strong iterative version with random start based on FGSM,
78 name projected gradient descent (PGD) as:
Zt+1 ll Se (Ix + asign( 2 C(fo(z)) v)))
(3)

---

77
where the a denotes the step size of each iteration. PGD provides better choice for adversarial examples, but will also cost much more time than FGSM. In the following sections, we call adversarial training with FGSM as FGSM-AT and correspondingly. PGD-AT. where denotes the maximum size of perturbations.
78
79
80
81
Catastrophic Overfitting: [Wong et al. 2020] believe that non- zero initialization for perturbations the key to avoiding the overfitting issue and propose to add uniform random noise during each training iteration. The detailed procedure is illustrated in the following equations:
82
8 = Uni Uniform(-e,te)
+ a sign (V,LVVVV(....))))
(4)
max(min(6, -c)
Andriushchenko and Flammarion 2020] propose a regularization method GradAlign maximize 85 the gradient alignment between various sets as:
84
E (a,))> cos (ooo((((((()))))) V V-C(Jo(a +n)+ m),
(5)
86
where 17 denotes random noise.
Bag of Tricks
87
We aim to investigate simple yet effective solutions overcome the catastrophic overfitting in 88
89
FGSM-AT. To stabilize FGSM-AT and make the trained model more robust adversarial attacks,
propose strategies from three general perspectives: Data Initialization, Network Structure, and Optimization. In this section, the experiments are done CIFAR- 10 dataset Krizhevsky [2009] with PreAAtRRsNNe-- 18 He et al. (2016) under the for adversarial attack of maximal perturbation
92
93
of E 8/255 without using any additional data. Two kinds of adversarial attacks are designed to
94
evaluate the robustness of models at the end of training: 10-steps projected gradient descent attack
95
(OGD-10) Madoryyyyyayyyyyyyyoooo et al. [2018) and the standard version of AutoAttack (AA) Croce and Hein (2020b).
96
Specifically, for the PGD-10 attack, we apply untargeted mode using the ground-truth annotations with step size a 2/ 2/255. AutoAttack comprises AutoPGD-CE,, AutoPGDDTTaretter FAB Croce and Hein 22200a], and Square attack Andriushchenko al. [2020).
97
99
Default setting, We set the training framework and hyper-parameters following Pang et al. 2021 We apply SGD optimizer with a momentum of 0.9, weight decay of x 10 and an initial learning rate of 0.1. ReLU function (without applying label smoothing) used as the default activation function. For the CIFAR dataset, we apply random flip and random crop data augmentation methods. Following the framework settings in Pang et al. [202T]. all models trained for 110 epochs. The learning rate decays 1h5th and 110th epochs. Specially, we report the robustness results on the last checkpoint. It should be noted that the final result might not be the best during the training process. Our experiments are conducted with NVIDIA TITAN XP GPUs.
100
101
102
105
106
PreActResNet-18
WideResNet-344 34-10 10
Methods
Clean
PGD-10
AA
Clean
PGD-10
AA
F+FGSM
86.4%
46.7%
41.0%
89.4%
0%
0%
Baseline
FGSM+GradAAlig
81.2%
48.7%
44.0%
81.2%
48.7%
44.0%
PGD-10
82.6%
53.1%
48.3%
86.1%
56.5%
52.2%
FGSM-Mask
82.5%
50.0%
44.2%
79.9%
33.7%
29.7%
Data initialization
RESM-Mask-fiveee
80.7%
48.6%
43.1%
72.3%
24.3%
20.9%
FGSM-Smooth
74.8%
48.5%
43.1%
75.6%
48.6%
44.2%
Network Structure
FGSM-Str2
83.1%
48.7%
44.4%
85.0%
50.4%
46.7%
FGSM+GradNormm
82.4%
47.2%
42.7%
82.8%
50.7%
46.2%
Optimization
FeiM--eightNorm
81.7%
48.3%
42.8%
85.7%
48.8%
45.7%
Table 1: Robustness performances of various methods on PreActResNet--18 and WideResNee-34-10.
3

---

add musk
x FGSM-Mask
add random noise
after F-FGSM
Figure FGSM-Mask V.S. F+FGSM on an input image
107
3.1 Data Initialization
108
Wong et 2020 firstly identify the catastrophic overfitting faced in FGSM-AT and propose to resolve this issue by initializing images with uniform noise with size namely Fast FGSM-AT (F+FGSM). As shown in Equation 4 the method is also termed "random initialization" since it randomly adds uniform perturbations during different training iterations. This method has been shown the capability to prevent general catastrophic overfitting and defend the models from PGD) attacks.
112
113
114 FGSM-Mask. Inspired by the core idea of F+FGSM, in this paper, we propose to mask randomly a proportion of the input pixels to stabilize the training procedure of FGSM-AT, which we term as FGSM-Mask. Fig demonnsttts the comparison FGSM PGSM-Mask and F+FGSM when generating adversarial examples. In each iteration, FGSM-Mask zeros out some randomly chosen pixels of each
115
116
image with mask M according given mask ratio. Then the masked image M is fed the model generate adversarial examples via FGSM as:
8 = a sign V V.gM M))
(6)
120
Compared with the random initialization method in F+FGSM (Equation our method exhibits a much simpler form Our FGSM-Mask simply randomizes the mask instead of manipulating the original pixel values.
121
122
To demonstrate the effectiveness of FGSM-Mask, mask images with different ratios and present the robust accuracy in Table 2 and Figure (a). With mask ratio of 0%, our method is reduced to the vanilla FGSM-AT, and therefore it suffers from catastrophic overfitting, As the mask ratio increases, the models trained with FGSM-Mask become more stable. A small mask ratio like 10% or 20% can already attenuate the overfitting issue but the robust accuracy still drops to near- near-zero zero after decreasing the learning rate. With mask ratio higher than 30%, the catastrophic overfitting is entirely resolved: the robust accuracy stably remains at 50.0%, outperforming F+FGSM (46.7%) by
125
126
127
128
129
more than 3%.
Additionally, we observe that the randomness of mask- ing is not necessary for different train-
FGSM-Mask-Fiie
Randomized Mask Ratio
Robust Accuracy
Fixed
Robust Accuracy
Mask Ratio
0-20%
0%
0-20%
0%
134
ing iterations. Instead, simply using
30%
50.0%
30%
0%
135
a fixed masking pattern throughout the
40%
49.3%
40%
48. 18.6%
136
training process is enough to help stabi-
50%
49.0%
50%
48.6%
137
lize FGSM-AT. It is worth to be noted
Table 2: Robust accuracy V.S. mask ratio for FGSM-Mask and FaskeMask-Fixed.
138
that the AT with fixed masks is equiva-
lent preparing pre-defined masked
adversarial dataset which will be fixed in the entire training process. The model trained with such

---

0.5
0.5
Robust Accuracy(9)
0.4
Robust Accuracy(
0.4
0.3
0.2
0.2
0.1
03
0.0
20
100
Epochs
Epochs
(a) FGSM-Mask
(b) FGSM-Mask-iixd
Figure 2: Robust accuracy of FGSM with various mask ratios. (a) is with the random mask, and (b) with the fixed mask.
141
masked dataset achieves remarkably stable and decent robust accuracy without applying any addi- tional tricks, as shown in Table We call this method MREDM-MMsk-FFxxe Similar to FGSM-Mask, with relatively lower mask ratios (2 30%), the catastrophic overfitting cannot be fully resolved by FGSM-Mask-Fixed, and the trained model result in final robust accuracy of 0%. As shown in Figure 2 (b), when increasing the mask ratio 50%, the model trained with FGSM-Mask Maakk-Fixed reaches robust accuracy of 48.6%, outperforming F+FGSM by about 2%. To show how randomized mask ratios and the fixed mask ratios influence the final robustness performance, Table Daaeeppreeentt the robust accuracy with both FGSM-Mask and FGSM-Mask-Fixxe under various mask ratios.
142
143
147
148
149
The findings in our Data Initialization section challenge the traditional belief that the randomness of initialization in different training iterations plays crucial role in AT Chen et al. [2020] which inspires to revisit the data initialization strategy in F+FGSM. We further find that it not necessary to pursue the randomness of uniform noise during different training epochs. Instead, fixing the uniform noise of F+FGSM can also stabilize the FGSM-AT and finally reach a robust accuracy of 46.5% under PGD-10 adversarial attack, which is comparable to the vamilla F+FGSM.
150
152
153
154
3.2
Network Structure
Existing studies have demonstrated that well-designed network structure improve the model robustness. Xie al. [2020]. [Singla al. [2021]. Wu et al. (2020). When trained with FGSM-AT,
158
Vision Transformers (ViTs) Dosovitskiy et al. [2020] have shown better robustness compared with
159
CNNs Bai et al.) [2021] Paul and Chen 2022 Shao et al. 2021 Furthermore, Xie et al. (2020),
160
Singla et al. 2021). Gowal et al. 20200] effectively boost the model robustmess by replacing the
161
original ReLU activation function with smoother ones. However, these approaches only focus on
162
improving the model robustness in the general training process, but have overlooked the potential
value of network structure for addressing the catastrophic overfitting in FGSM-AT. In this section, investigate the role of network structure in FGSM-AT following the ideas of ViTs and smooth activation functions.
186
Larger stride for the first convolution layer. We first examine whether using ViTs can resolve the pverfitting issue. We implement vanilla FGSM-AT with the compact Vision Transformer (CVT) Has sani et al. [021]_ Transformer architecture designed for the dataset with smaller resolution. We observe that the robust accuracy under PGD attacks does not drop to zero during the whole training process, without applying any other tricks, neither random initialization nor regularization. The fact
189
170
171
that ViTs can successfully avoid the overfitting issue motivates us to rethink whether we can achieve the same goal simply by modifying the architecture CNNs. one big difference between ViTs and CNNs lies in how they process images the beginning of the network, we propose simply modify the first convolution layer of CNNs to approach the similar behaviour of ViTs. ViTs begin with patchify operation, which splits image into sequence of non-overlapping patches. Whereas
172
173
5

---

178
for CNNs, taking PreActResNet- 18 as an example, the first layer is a 3 x 3 convolutional layer with stride I, which results overlapping sliding windows when computing the convolution features. To mimic the behaviour of ViTs, we propose to enlarge the stride size CNNs to reduce the overlapped regions between adjacent sliding windows. By simply increasing the stride size from to or the catastrophic overfitting problem successfully addressed. As shown in Figure when the stride is zero. When the stride is set be 2 or the robust accuracy curve performs much more stable. Among different stride options in our study, we find that FGSM-AT with a stride as 2 achieves the highest robust accuracy. Therefore we adopt this setting in later experiments, namely FGSM-Str2.
177
178
179
180
set to be 1, the robust aceuracy quickly drops to
184
0.5
AMM
Robust Accuracy
0.4
17.12
0.-
0.1
grad
wright
200
Epochs
Epochs
Figure 3: Robust accuracy and clean accu- racy of Large Stride Size CNN. larger stride size builds the robustness success- fully.
Figure 4: Trend of // and the maximum value of weights. Both increase dramatically when the overfitting happens.
185
Smooth activation function. We also investigate the role the activation function in FGSM-
186
AT. We replace the original ReLU activation function with smoother ones and then explore their
187
effectiveness for addressing the overfitting problem. We select four smooth activation functions:
SiLU Ramachandran al. (2018). ELU Cluere et al. [2016]. SoftPlus Nair and Hinton (010),, and
189
GELU Hendrycks and Gimpel [0166). We display the curves of these activation functions and record
190
their robust accuracy during FGSM-AT in Figure (a). It can be observed that smooth activation functions can all mitigate or even fully prevent catastrophic overfitting.
131
192
We also find that the degree of smoothness affects the robustness. For instance, ELU is smoother than GELU and accordingly the robust accuracy of ELU is stabler than that of GELU. Following Xie al. [2020), choose SoftPlus to study the effect of function smoothness because the scaler in Parametric SoftPlus can control its smoothness as the following:
193
195
1 a
f(a, 2) log(1 + exp(ax))
196
Figure shows the curves of SoftPlus when the a is 2, 5, 10 and the according robust accuracy the robust accuracy becomes
197
curves. As a decreases, the activation function becomes smoother, stabler. Figure (b) validates that the smoothness of activation functions has positive correlation with the stability of FGSM-AT. Here choose SoftPlus with 2 as baseline shown in Table it performs the best among smooth activation functions, and we call this method FGSM-Smooth.
198
3.3
Optimization
202
Adding an extra regularization term has been shown capable to prevent the catastrophic overfit- ting in FGSM-AT but can usually result in extra computation overhead. One typical example is
203
204
GradAlign Andriushchenko and Flammarion 22000. which adds an additional objective to maximize
205
the gradient alignment inside the perturbation set. GradAlign is quite effective for stabilizing FGSM-
AT. However, it comes at the cost of extra computational burden due extra forward and
backward propagation to compute the gradient of an adversarial set V C.f(fo 7), y) (Equation (5).

---

51.3
SILU
0.8
SoftFuu
Robust Accuracy(ss
0.45
SoftPli
GELU
GELU
A(X)
0.40
0.2
0.35
-02
0.30
-2.6
-0.25
20
100
Epochs
0.5
JkFand
0.8
apha-10
Robust Accuracy("6)
SofiPlus(3)
mmagngy
0.6
0.2
0.1
0.0
-1.00
-0.75
-0.30
-0.25
0.00
0.25
0.50
0.75
Epochs
(b)
Figure 5: Curves of activation functions and their corresponding robust accuracy. (a) shows the comparisons between various activation functions. (b) shows the comparisons between Softplus with different
208 In this paper, to avoid the extra forward propagation GradAlign, we first introduce a novel regularization method which directly regularizes the L2 norm of gradients on input images, referred as GradNorm. Then further reduce the computation cost, we design another simple but efficient method by only regularizing the weights the first layer, referred to WeightNorm. Both GradNorm and WeightNorm successfully address the overfitting issue and achieve comparable robust accuracy with GradAlign, while WeightNorm significantly reduces the computational cost. For instance, WightNorm and GradAlign take 42 seconds and 56 seconds for each training epoch. WeightNorm 24% faster than GradAlgin. Next we illustrate the technical details of GradNorm and WeightNorm.
209
210
213
214
215
216
GradNorm. By taking a closer look at the L2 norm of gradients on input images, we observe that the || V becomes 100x larger after the th epoch as shown in Figure This observation aligns with the conclusion in Kim et al. [202I). which points
217
218
that the increasing gradient norm leads decision boundary distortion and highly curved loss surface during adversarial training. This distortion hence makes the adversarially trained model vulnerable to multi-step adversarial attacks (e.g., PGD attacks) and leads to catastrophic overfitting. This phenomenon inspires us to design new regularizer by directly constraining the gradient norm [B]] V xe [|a]:
222
Li 8), v) ~1|V211
where the hyper- ppper-prrameter 3 controls the weight of the regularizer. As shown in Table [ GradNorm successfully overcome the overfitting issue and achieves high robust accuracy of 47.2% against
224
225
PGD-10 attacks, which is comparable to the result of GradAlign (48.7%).
228
WeightNorm. Both GradAlign and GradNorm are highly effective in addressing the overfitting issue. However, as aforementioned, they both suffer from high computational cost due to the additional back- propagation requirement. We hereby aim to design a novel regularization method which addresses the overfitting issue without introducing an extra computational burden. propose WeightNorm, regularization term that directly exploits the intermediate features of vanilla FGSM-AT models. Since the goal of adversarial training to let the predictions of adversarial examples close to that of
227
228
229
230
clean samples much possible: fø + 8) + fe(z), design to optimize the training process
7

---

233
by constraining the prediction difference. For simplicity, we only examine initial features generated by the first convolution layer Jw- where the w denotes the weights of the first convolution layer. The fw (ar + 6) far (a:) can be represented as w(x + 5) wH, which is equal to wÓ. Therefore, therefore, pushing fi( + 6) + (1) is to minimize wd. We can either regularize the å (i.e., gradients of images) or the weights w.
234
235
236
237
238
Regularizing w is cheaper than constraining the image gradient (which is essentially 8) as only part of model parameters are regularized, which avoids the second-order back propagation. After observing the change of w, we find that the maximum value the weights also significantly increases
239
240
241
when the catastrophic overfitting occurs. As shown in Figure 4 larger weights suggest that the network overfits the training data. Therefore, we design regularizer aiming at constraining w. The intuition of this regularizer design is to both avoid large values in weights and also reduce the distance between clean features generated by clean samples and adversarial features generated by adversarial samples. We select L1 norm to define the regularizer as:
242
243
min AC (f), +))
where the controls the weight of the regularizer and ō the adversarial perturbation. The proposed 247 regularizer constrains and pushes the first-layer intermediate features of adversarial examples be
246
closer to that of clean samples. Experiments show that this regularizer could prevent the catastrophic overfitting and it does not require extra forward pass like GradAlign shown in Equation
3.4
Combination of Tricks
251
Each approach we propose can mitigate the catastrophic overfitting problem individually. To investi- gate the aggregated effect, we combine some of them and show results in Table Adding the mask to images and increasing the stride size at the same time do not improve the performance. WeightNorm
252
253
254
does not benefit other tricks. Smooth activation function can benefit masking image or large stride size, showing improvement in the robustness performances. After trying different combinations, find that combining large stride size and smooth activation functions have the best performance.
Methods
Performances
Mask
Large Stride Size
Smooth Activation Function
WeightNorm
Clean
PGD-10
AA
82.5%
49.4%
45.1%
81.1%
51.2%
46.1%
82.2%
51.3%
46.4%
81.3%
51.2%
46.1%
Table 3: Performances of FGSM-AT with combined tricks
257
4
Scalability to Large Networks
258
Compared with small networks, the larger networks are more likely to overfit the training data as the network parameters increase, and the mentioned tricks might not work. As displayed in Table T when the size of the network increases (from PreActResNett-18 results in 0% of robust accuracy under the adversarial attack. To comprehensively validate the effectiveness of the methods mentioned above, we conduct experiments on WideResNNe-33-110 with the same training recipe PreActResNet-15 18. eexhiii the robustness performances of different methods on these two networks, and the displayed results are taken at the final checkpoint. For the masking methods in Data Initialization, the mask ratio is set larger on WideResNet. Compared with Net, the effectiveness of masking methods declines, but they still exhibit higher robust performances than the vanilla F+FGSM. For the methods in Network Structure, both the FGSM-AT
259
260
WideResNer-34-11 10), F+FGSM
261
262
285
286
267
268
with larger stride size (FGSM-Str2) and with smooth activation functions (FGSM-Smootee perform stably on WedeResNet, showing comparable results with PreActRest. On both PrtActRestNet and WideRestNet, the FGSM-Str2 generally outperforms the other three tricks. Furthermore, the combination of tricks is also validated on WideRestNet. Following the optimal settings from Table 3
289

---

272 we combine the smooth activation function and large stride size in FGSM-AT. With the combined
273 tricks, the models respectively achieves a robust accuracy of 51.8% and 47.3% under PGD-10 attack
274 and AA, outperforming all other FGSM-AT methods.
275
5
Related Work
Adversarial training. Adversarial training has been regarded as of the most effective strategies to defend against the adversarial threats to machine learning systems. The idea of adversarial training origins in Goodfellow et al. [2015] who proposes to combine clean samples and adversarial examples to train the model. Madry et al.
278
279
2018) first demonstrate the optimization problem in
280
adversarial attack. Furthermore, advanced adversarial 2019 apply regularization term to achieve the balance between robustness and clean performance. Shafahi al. [2019] reduce the high cost of adversarial
adversarial training and proposes the PGD
281
training methods are proposed. [Zhang et al.
282
training by recycling the gradient information. Carmon et al. [2099] first augment CIFAR-10 with
500K unlabeled extra data from 80 Million Tiny Images dataset. Some works also summarise the
tricks of AT and the optimal settings for AT. Pang et al. [021] list the optimal hyperparameters for
286
PGD-AT on CIFAR-10 dataset. Gowal et al. [2000] introduce weight average (A)) to adversarial
287
training and find the optimal ratios of extra data to get the best adversarial robustness.
Catastrophic overfitting. Though an efficient method, FGSM-AT is not popular now because of its failure against severe attacks, like PGD adversarial attack. Wong et al. 2020) first find that the robust accuracy under PGD adversarial attack of FGSM-AT will drop to zero after several epochs, and this phenomena is named as catastrophic overfitting. Rice et al. [020) think that catastrophic overfitting is a special case only existed FGSM-AT and this overfitting phenomenon is due to weaker adversarial attacker. Kim al. [021] visualize the decision boundary during adversarial training and find the decision boundary distortion is closely related to the catastrophic overfitting.
289
280
291
292
293
They believe that the fixed distance from adversarial examples to clean images the key causing the distortion and propose to apply various step sizes for each image.
296
Data initialization. Data initialization has been common trick in adversarial training, where random noise is added to images before AT during each iteration. Madry et al. [018]] first add random start for PGD-AT. Tramer et al. 2018) first propose R+FGSM combining a Gaussian random initialization in a single-step AT. They add Gaussian random noise to images and do FGSM-AT with
299
300
301
a step size of a = c/2, which is not effective against PGD adversarial attack. WWon et al. 2020 believe that non-zero initialization for perturbations is the key to avoiding overfitting and propose adding uniform random noise to prevent overfitting.
302
303
304
Regularization. Wong et al. 2020 point out that early stopping is an effective method to get a robust model trained by FGSM-AT, but the robustness underperforms as the training epochs are inadequate. |Andriushchenko and Flammarion (2020) demonstrate that the catastrophic overfitting irrelevant to the sizes of networks. Instead, the local non-linearity was the true reason. To prevent overfitting, they propose regularization method called GradAlign, which maximizes the gradient alignment between various set stop the catastrophic overfitting.
305
306
307
Conclusion
310
311
This study proposes a range of tricks to address the catastrophic overfitting in FGSM-AT and comprehensively examine their effectiveness on networks with different scales. Our results show that the proposed tricks can be simple yet effective solutions to stabilize FGSM-AT at minimal computational cost. We hope this study could contribute to the achievement of a fully stabilized FGSM-AT in the future.
312
313

---

References
318
317
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast ad advessaiial training, ArXiv, abs/2007.02617, 2020.
318
319
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box box adversarial attack via random search. ArXiv, abs/ 1bss/1912.0004,, 2020.
320
Yutong Bai, Jieru Mei, Alan Loddon Yuille, and Cihang Xie. Are transformers more robust than cnns? In NeurlPS, 2021.
323
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. ArXiv, abs/1905.13336, 2019.
325
Chen, Yu Cheng, Zhe Gan, Quanquan Gu, and Jingjing Liu. Efficient robust training via backward smoothing. ArXiv, abs/2010.01278, 2020.
326
327
Arnek-Aréé Clevert Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv: Learning, 2016.
328
329
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive
330
boundary attack. ICMI,, 2020a.
331
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. ArXiv, abs/2003.01690, 2020b.
332
333
Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprins ar arXiv:2010.11929, 2020.
334
335
336
337 Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/14112.5722 2015.
338
329
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy A. Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. ArXiv, abs/2010.03593,
340
341
2020.
342
Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi. Escaping the big data paradigm with compact transformers. ArXiv, abs/2104.05704 2021.
343
Kaiming He, X. Zhang. Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ArXiv, abs/1603.00007, 2016.
345
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: Learning. 2016.
347
Hoki Kim, Woojin Lee, and Jaewook Lee. Understanding catastrophic overfitting in single-step adversarial training. In AAAI, 2021.
349
Alex Krizhevsky, Learning multiple layers of features from tiny images. 2009.
350
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale, arXiv preprint 1r1.v:661. 2016.
351
352
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.
353
Towards deep learning models resistant to adversarial attacks. ArXiv, abs/1 706.06083, 2018.
Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
355
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. ArXiv, abs/2010.00467, 2021.
10

---

358
Sayak Paul and Pin- Yu Chen. Vision transformers are robust learners. ArXiv, abs/2105.07581, 2021.
359
Prajit Ramachandran, Barret Zoph, and Quoe V. Le. Searching for activation functions. ArXiv, abs/1710.05941, 2018.
361
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. In ICMI,, 2020.
362
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurIPS, 2019.
384
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yu Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of vision transformers. ArXiv, abs/2103.11111111 2021.
367
Vasu Singla, Sahil Singla, David Jacobs, and Soheil Feizi. Low curvature activations reduce overlitting in adversarial training. ArXiv, abs/2102.07861, 2021.
368
389
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 13Xiv: 2013.
370
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, and Patrick Medaniel. Ensemble adversarial training: Attacks and defenses. ArXiv, Abs//775.0772x 2018.
373
Eric Wong. Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. ArXiv, abs/2001.03994, 2020.
374
375
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? 2020.
376
Cihang Xie, Mingxing Tan, Boqing Gong, Alan Loddon Yuille, and Quoc V. Le. Smooth adversarial training. ArXiv, 1b5320006.14536 2020.
378
379
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.

---

Checklist
381
382
1. For all authors...
383
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
384
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the paragraph in Sec. 6
386
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See the first paragraph in Sec. [ for potential impacts. Negative ones have not been identified
387
388
since we target at defense.
389
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
390
381
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] Our contribu- tions are mostly empirical.
393
394
(b) Did you include complete proof's of all theoretical results? [N/A] Our contributions are mostly empirical.
395
396
3. If you ran experiments...
397
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] All the code would be released with detailed instructions and annotations. The data are described in
398
399
Sec. 3
401
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Details are presented Sec. 3
402
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] Error bars are not reported and computation amount are reported.
404
405
406
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Our devices are described in Sec
408
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
409
(a) If your work uses existing assets, did you cite the creators? [Yes] We state the source of DNNs in experimental setup in Sec. 3
410
411
(b) Did you mention the license of the assets? [Yes] All used code assets are under MIT
412
License.
(c) Did you include any new assets either in the supplemental material as URL? [Yes] We would release our code as an asset.
415
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] Our empirical studies are based on public datasets.
416
417
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] Our used public datasets generally do not contain personally identifiable information or offensive content.
418
419
420
5. If you used crowdsourcing or conducted research with human subjects...
421
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] Our experiments involve no human subjects and crowdsourcing. (b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A] Our experiments involve no human subjects or crowdsourcing.
422
426
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] Our experiments involve no human subjects or crowdsourcing.
427
12


Reviews:
[{"review_id": "t33PO9bNX_7", "reviewer": "Reviewer_ASuB", "summary": "The paper focuses on the problem of catastrophic forgetting overfitting in Adversarial training using FGSM. It investigates the data initialization, network structure, and optimization of CIFAR-10. It shows that\n* applying masking pattern to the training set,\n* smooth activations and using ViTs, and\n* constraining first layer convolution weights in the convolution layers\n \nstabilizes FGSM adversarial training.\n", "questions": "Other than the questions in the previous section, I had the following clarifications and suggestions:\n\n- Did the authors also check the effect of batch-norm and its variants on FGSM-AT? It would also be interesting to see if the regularizers like mixup, cutmix, augmix etc. can further be used as tricks for FGSM-AT.\n\n- The intuition behind the attenuation is unclear to me. Can the authors comment on the reason for attenuation at some point for the masks? Further, did the authors experiment with less than 10% masking? \n\n- Which architecture was used for conducting architectures in Table 2 and Figure 2 a). Was the same architecture used for all experiments other Table 1? Did the authors check the observations for the other architecture and observe similar behavior?", "limitations": "While the paper mentions that limitations are highlighted in Section 6, the limitations were missing from this section. Therefore, I would suggest including a separate section highlighting the limitations and societal impact of the paper; this can also be included in the supplementary material, but I believe the authors can easily accommodate half a page in the current manuscript.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I am well-familiar with the literature and read the full paper in detail. Accordingly, now I’ll describe the strengths and weaknesses of the paper in the order of originality and significance, quality of the paper, and clarity. \n\n---\n\n### Originality and Significance\n\n#### Strengths\n\n* The paper’s motivation is sound; it investigates the catastrophic forgetting overfitting from different perspectives, including data initialization, network structure, and optimization. While many previous works have understood this issue, the paper showcases that these simple tricks can help mitigate this problem to a certain extent.\n\n* The tricks are novel for FGSM adversarial training, especially the results showing that a pre-defined masking pattern to the training, the effect of ViTs, and regularization terms are simple, easy to implement, and effective in isolation, as demonstrated by the results.\n\n* The paper does a good job of positioning the proposed work. Furthermore, it does a good job in related work (section 5) explaining the existing works. I appreciate the paper using the optimal hyper-parameters from Pang et al. 2021. for their experiments.\n\n#### Weaknesses\n\nI do not have any significant concerns regarding the originality and contributions of the paper to the community. One minor suggestion is to include a discussion with Pang et al. 2021 when discussing smooth activation functions in Section 3.2, as the advantage of smooth activations was shown in their paper. \n\n---\n### Quality\n\nThe conducted experiments are interesting and valuable to show the performance gains by the proposed tricks for FGSM-AT. However, they were not exhaustive for me to evaluate the proposed tricks conclusively. Therefore, I have the following suggestions and questions that would be helpful to gain more insights and should improve the paper:\n\n* The paper does not provide confidence intervals, making it hard to judge the improvement gains. For example, in Table 1, the weight norm seems to provide no benefit over the grad norm; it is also hard to see the differences between FGSM-Smooth and FGSM-Str without the confidence intervals. Similarly, in Table 3, given the minute differences between various combinations, it is challenging to judge the contributions of each of them individually.\n* The evaluation protocol was also unclear, is the masking done only during training or even during the evaluation? In my opinion, it should be done only during training to avoid obfuscated gradients.\n* Lastly, the effect of \\lambda from eq. 9 was not highlighted in the paper. Other than mentioning the time taken per epoch in line 221, it would be beneficial to mention the total training time for both the methods up to convergence. \n\n---\n### Clarity\n\nThe paper was easy to read; however, I have various suggestions regarding the paper’s writing, as elaborated below.\n\n* In Figure 5, I would suggest including Relu for a comparison. Further, I don’t think the left figures in Figure 5 are essential in the main paper. They can be moved to the appendix and the space can be utilized for important text like WideResNet results mentioned in Section 4.\n* I would suggest repositioning section 4 and explaining the architectures in the experimental settings, followed by comparing the architectures' observations in the following sections.\n* The images in Figure 1 can be positioned side-by-side horizontal instead of vertical.\n* According to the NeurIPS template, the caption should have been on top of the table.\n* The paper had an incorrect usage of \\citep and \\citet. For instance, lines 22, 26, 95, 98, etc., should have used \\citep, but the paper uses \\citet for all citations.\n\n#### Other writing suggestions:\n* Line 35: Extra space before the period\n* Line 87: Extra space after f \n* Line 248 - Refer to the experiments being talked about here.\n* A period in equation 2.\n* While the operators may seem obvious, for uninformed readers, it would be helpful to define $\\Pi$ , $x_t$, $t$ in section 2.\n* Missing comma after equation 5\n* The paper should also mention the number of GPUs used for ease of reproducibility in Line 108\n* The notation of FGSM-Str2 is not self-explanatory.\n* Therefore, therefore -> Therefore in line 238\n* Extra space after the period in line 277\n* I suggest using the conference proceedings references for the papers; in many cases, it refers to the ArXiv version.\n\n---\n\n### Reproducibility\nThe authors provide the code to replicate the results in the supplementary material.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "W9ywEFjb3y1", "reviewer": "Reviewer_N6sV", "summary": "This paper proposes various methods to mitigate the drastic overfitting (and resulting worse robust test accuracy) that can occur when adversarial training using the fast gradient sign method (FGSM). The proposed methods include masking input pixels, increasing the stride size of the first convolutional layer, smoother activation functions, and regularization. The authors show that each of these approaches successfully mitigates catastrophic overfitting on its own, and that the combination of multiple of these techniques results in slightly better robust performance than any technique alone (including those in previous works).", "questions": "Questions\n- Do you use a validation set or do you measure the robust accuracy using the test set during training? \n- Did you re-train other methods in Table 1 or use downloaded weights?\n- What is the main advantage of using the fixed mask, given it does not appear to outperform any instance of the random masking? \n\nSuggestions\n- It would be good to discuss the trade-off in clean performance amongst the different techniques. Additionally, it would be interesting to include clean performance in Table 2. \n- You could better clarify what catastrophic overfitting is in the introduction for the reader that is unfamiliar with the term/phenomenon.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- This paper improves upon the performance of FGSM adversarial training by incorporating new techniques to mitigate catastrophic overfitting.\n- The authors provide some light hypotheses behind why these techniques might work, and some empirical testing of these hypotheses. \n- The paper is clearly written for the most part.   \n \nWeaknesses \n- It is not made entirely clear which technique in the “bag of tricks” performs best and whether your combination of techniques is state-of-the-art (i.e. it would be good to state this clearly in the introduction, add bold font to your tables, and include the best of the results in Table 3 as a line in Table 1 as well).\n- The paper could use another round of edits to improve readability: you should use \\citep for parenthetical citations, and there are a few typos i.e. in lines 214-215. \n- The experimental evaluation is not as thorough as in previous work - a stronger/more standard PGD evaluation uses 50 steps and 10 restarts rather than just 10 steps, and training time is excluded in the experimental results. \n- While there are some attempts to explain why these techniques are successful at mitigating catastrophic overfitting, the paper could be strengthened by including only the most effective of the techniques, and providing a more thorough investigation into each technique.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NcV543cL0sH", "reviewer": "Reviewer_zdGm", "summary": "This paper investigates training tricks used for FGSM-AT, including data initialization, network structure and optimization. The combined tricks can effectively alleviate the catastrophic overfitting in FGSM-AT, making FGSM-AT a more practical way towards robustness.", "questions": "My main concern is that the proposed tricks largely hurt the clean accuracy.", "limitations": "No", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The catastrophic overfitting in FGSM-AT is an important research topic.\n- The investigated tricks like random masking / increasing stride size are interesting and underexplored in the literature.\n- WeightNorm seems an efficient substitute for GradNorm.\n\nWeaknesses:\n- My main concern is that the proposed tricks largely hurt the clean accuracy (e.g., Table 1 FGSM-Mask drops clean accuracy from 89.4% to 79.9% on WRN-34-10; FGSM-Smooth drops clean accuracy from 86.4% to 74.8% on PreActResNet-18).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "40TVLQkRoc5", "reviewer": "Reviewer_kmGb", "summary": "This work investigates several modifications that can avoid Catastrophic Overfitting in FGSM-AT which would make AT much more efficient compared to using multi-step methods such as PGD. In particular, they suggest 4 different \"tricks\": Masking some pixels of the image, using a larger stride (default 2), using smooth activation functions (already suggested in prior work) and using a weight normalization regularizer. They empirically observe that some of their tricks achieve better performance than FGSM+GradAlign for CIFAR10 and $\\epsilon=8/255$.", "questions": "1. Why do authors think that CVT does not suffer from CO? Have they tried larger perturbation radii? \n\n2. It would be helpful to add a column for the computational cost (either in seconds per epoch or as a relative cost) for the different tricks and compared methods since one of the strengths of the proposed methods is efficiency.", "limitations": "As previously discussed, authors should test other datasets and larger perturbation radii to claim that the presented methods prevent CO.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The topic of Catastrophic Overfitting and in particular efficient AT is relevant. \n2. Results shown are encouraging.\n3. Some results are surprising like the fact that a fixed mask works as well as randomly masking each image.\n\nWeaknesses:\n\n1. My main concern is that I do not think the experimental evidence is enough to support the claims. As reported in GradAlign work (https://arxiv.org/abs/2007.02617) some single-step AT methods (like F+FGSM) could prevent CO for moderate $\\epsilon$ (such as 8/255) but would suffer from CO for larger radii. Therefore, the tested settings in this work (only CIFAR-10 and $\\epsilon=8/255$) are too narrow to be reliable when stating that some particular method prevents CO. Additional perturbation radii should be compared and it would be sensible to include at least another dataset.\n\n2. There is a lack of insight in the proposed tricks. While empirical results are definitely interesting, they should be discussed more in order to understand why do they work.\n\n3. Some of the \"tricks\" are not very novel (e.g. smooth activation functions was already introduced in the context of adversarial training). Despite it is novel that they can mitigate CO in some settings, without more insight into why this would happen I consider the finding lacks impact.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject