Towards Understanding Fast Adversarial Training
Authors: Bai Li
Shiqi Wang
Suman Jana
Lawrence Carin
Venue:  (2021)
Keywords: fast adversarial training
adversarial examples

Abstract: Current neural-network-based classifiers are susceptible to adversarial examples. The most empirically successful approach to defending against such adversarial examples is adversarial training, which incorporates a strong self-attack during training to enhance its robustness. This approach, however, is computationally expensive and hence is hard to scale up. A recent work, called fast adversarial training, has shown that it is possible to markedly reduce computation time without sacrificing significant performance. This approach incorporates simple self-attacks, yet it can only run for a limited number of training epochs, resulting in sub-optimal performance. In this paper, we conduct experiments to understand the behavior of fast adversarial training and show the key to its success is the ability to recover from overfitting to weak attacks. We then extend our findings to improve fast adversarial training, demonstrating superior robust accuracy to strong adversarial training, with much-reduced training time.

Under review as a conference paper at ICLR 2021
TOWARDS UNDERSTANDING FAST ADVERSARIAL TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
Current eeural-network-bassee classifiers are susceptible to adversarial examples. The most empirically successful approach to defending against such adversarial examples adversarial training, which incorporates a strong self-attack during training to enhance its robustness. This approach, however. is computationally expensive and hence hard scale up. A recent work, called fast adversarial training, has shown that it is possible to markedly reduce computation time without sacrificing significant performance. This approach incorporates simple self- att-kktscks, yet it can only run for a limited number of training epochs, resulting in sub-optimal performance. this paper, we conduct experiments to understand the behavior fast adversarial training and show the key to its success is the ability to recover from overfitting to weak attacks. We then extend our findings to improve fast adversarial training, demonstrating superior robust accuracy to strong adversarial training, with much-reduced training time.
INTRODUCTION
Adversarial examples are carefully crafted versions of the original data that successfully mislead classifier (Szegedy et al. 2013), while realizing minimal change in appearance when viewed by most humans. Although deep neural networks have achieved impressive success on variety of challenging machine learning tasks, the existence of such adversarial examples has hindered the application of deep neural networks and drawn great attention in the deep-learning community.
Empirically, the most successful defense thus far based on Projected Gradient Descent (PGD) adversarial training (Goodfellow et al.) 2014 Madry et al. (017). augmenting the data of interest with strong adversarial examples, to help improve model robustness. Although effective, this approach is not efficient and may take multiple days to train moderately large model. On the other hand, one of the early versions adversarial training, based on a weaker Fast Gradient Signed Method (FGSM) attack, is much more efficient but suffers from "catastrophic overfitting," phenomenon where the robust accuracy with respect to strong attacks suddenly drops to almost zero during training (Tramer) al. 2017 Wong et al. 2019), and fails to provide robustness against strong attacks.
Fast adversarial training (Wong et al. 2019) is a simple modification to FGSM, that mitigates this issue. By initializing FGSM attacks with large randomized perturbations, can efficiently obtain robust models against strong attacks. Although the modification is simple, the underlying reason for its success remains unclear. Moreover, fast adversarial training only compatible with a cyclic learning rate schedule (Smith & Topin 2019), with a limited number of training epochs, resulting in sub-optimal robust accuracy compared to PGD adversarial training (Rice al.) 2020). When fast adversarial training runs for a large number of epochs, it still suffers from catastrophic overfitting, similar to vanilla FGSM adversarial training. Therefore, it remains an unfinished task to obtain the efffectiveness of PGD adversarial training and the efficiency of FGSM adversarial training simultaneously.
In this paper, we conduct experiments to show that the key to the success of fast adversarial training is not avoiding catastrophic overfitting, but being able to retain the robustness of the model when catastrophic overfitting occurs. We then utilize this understanding to propose simple fix to fast adversarial training, making possible the training of it for a large number of epochs, without sacrificing efficiency. We demonstrate that, as result, we yield improved performance.

---

Under review as a conference paper at ICLR 2021
We also revisit previously developed technique, FGSM adversarial training warmup (Wang itt al. (2119), and combine with our training strategy to further improve performance with small additional computational overhead. The resulting method outperforms the state-of- the-hhh-rttt approach, PGD adversarial training (Rice et al. 2020), while consuming much less training time.
Our contributions are summarized follows:
We conduct experiments to explain both the success and the failure of fast adversarial training for various cases.
We propose an alternative training strategy as a fix to fast adversarial training, which is equivalently efficient but allows training for a large number of epochs, and hence achieves better performance.
We propose to utilize the improved fast adversarial training as a warmup for PGD adversarial training, to outperform the ofte-of-theeart adversarial robustness, with reduced computation.
BACKGROUND AND RELATED WORK
The existence of adversarial examples in deep learning was initially reported in (Szegedy al. 2013). Since then, many approaches have been proposed to mitigate this issue and improve the ad adversarial robustness of models. A straightforward method is data augmentation, where adversarial examples are generated before the back- proppaaaaantio at each iteration and used for model updates. This approach referred to as adversarial training. It was first used with gradient- based single-step adversarial attack, also known as the Fast Gradient Sign Method (FGSM) (Goodfellow et al., (004)). Later, (Kurakin et al. (0016) found that models trained with FGSM tend to overfit and remain vulnerable to stronger attacks. They proposed multi-step version of FGSM, namely the Basic Iterative Method (BIM), seeking to address its weaknesses. Randomized initialization for FGSM then was introduced in (Tramer et al. 2017), leading to R+FGSM to increase the diversity of attacks and mitigate the overfitting issue. Finally, (Madry et al. 2017) combined randomized initialization with multi-step attacks to propose projected gradient descent (PGD) attacks, and showed its corresponding adversarial training is able to provide strong adversarial robustness (Athalye et al. 2018). As PGD adversarial training is effective, many works have tried improve upon it (Zhang al. 201 9b
Xie et al. 2019). However, a recent study (Rice et al. (020) conducted extensive experiments on ad adversarially trained models and demonstrated that the performance gain from almost all recently proposed algorithmic modifications to PGD adversarial training is no better than simple piecewise learning rate schedule and early stopping to prevent overfitting.
In addition to adversarial training, great number of adversarial defenses have been proposed. yet most remain vulnerable to stronger attacks (Goodfellow et al.] 2014 Moosavi-Dezzooll et a [2016] Papernot et al., 2016 Kurakin et al., 2016; Carlini & Wagner 2017 Brendel et al. 2017]] Athalye et 20.1). major drawback of many defensive models is that they are heuristic and vulnerable to
adaptive attacks that are specifically designed for breaking them (Carlini et al. 2019 Tramer et al.
2020). To address this concern, many works have focused on providing provable/certifie robustness of deep neural networks (Hein & Andriushchenko, 2017 Raghunathan al. 2018 Kolter & Wong
2017
Weng et al.) 2018 Zhang et al.]| Dvijotham et al., 20088] Wong et al. 2018]) Wang et al.,
2018 Lecuyer et al. 2018 Li et al.. 2019 Cohen al. 2019), yet their certifiable robustness cannot match the empirical robustness obtained by adversarial training.
Among all adversarial defenses that claim empirical adversarial robustness, PGD adversarial training has stood the test of time. The only major caveat to PGD adversarial training is its computational cost, due to the iterative attacks at each training step. Many recent works try to reduce the computational overhead of PGD adversarial training. (Shafahi et al. (019) proposes to update adversarial perturba- tions and model parameters simultaneously. By performing multiple updates on the same batch, it is possible to imitate PGD adversarial training with accelerated training speed. Redundant calculations are removed in (Zhang et al.. 2019a) during back-propagation for constructing adversarial examples, to reduce computational overhead. Recently, (Wong et al. 2019) shows surprising results that FGSM adversarial training can obtain strongly robust models if a large randomized initialization is used for FGSM attacks. However, they forced to use cyclic learning rate schedule (Micikevicius et al.) 2017) and a small number of epochs for the training. This issue limits its performance, especially when compared to ofate-of-the-art PGD adversarial training with early stopping (Rice et al. 2020
2

---

Under review as a conference paper at ICLR 2021
3 FAST ADVERSARIAL TRAINING
3.1 PRELIMINARIES
We consider the task of classification over samples (I, y) E (X,3)). Consider classifier fo y parameterized by 0, and a loss function L. For a natural example I E X, an adversarial example x' satisfies (xx, x²) < for small > O, and fx(x) & fo()'), where -)--)) some distance metric, i.e., x' is close to but yields a different classification result. The distance is often described in terms of an Ep metric, and we focus on the E metric in this paper.
Adversarial training is an approach for training robust model against adversarial attacks. It represents the objective of obtaining adversarial robustness in terms of a robust optimization problem, defined as
min B(z,a)=x'
max (C(fo(='),))
(1)
approximates the inner maximization by constructing adversarial examples based on natural examples, and then the model parameters 0 are updated via optimization method with respect to the adversarial examples, instead of the natural ones. One of the simplest choices of attack for adversarial training is the Fast Gradient Sign Method (FGSM) (Goodfellow al. 2014)
= x + esign V (2((((()))))
(2)
Before the introduction of fast adversarial training (Wong al. 2019), which will introduce later, it was commonly believed that FGSM adversarial training tails to provide strong robustness (Kurakin al. 2.16). During FGSM adversarial training, the robust accuracy of the model would suddenly drop to almost 0% after a certain point, when evaluated against PGD attacks. This phenomenon was referred as "catastrophic overfitting" in (Wong al. 2019). The cause of catastrophic overfitting was studied extensively in (Tramer et al., (017):: during training, since FGSM is a simple attack, the model learns to fool the FGSM attacks by inducing gradient
gradient (Athalye et al., (0188;; that is, the gradient is no longer useful direction for constructing adversarial examples.
The existence of catastrophic overfitting has prohibited the use of FGSM adversarial training.
To mitigate this issue, (Madry et al., 2017) introduced a multi-step variant of FGSM, namely Projected Gradient Descent (PGD), which takes multiple small steps with stepsize a to construct adversarial examples instead of one large step as in FGSM:
2/41 1|
(x4 + asign V ((F)
(3)
Extensive experimental results (aadry et al. 2017 Athalye et al. (018)) have shown that, unless the model particularly designed for creating obfuscated gradients Tramer et al., 2020), PGD attacks generally exempt from overfitting. Consequently, adversarial training with PGD leads to robust
models against strong attacks, although its computational cost is often an order of magnitude more expensive than standard training and FGSM adversarial training.
Recently, in contrast to conventional believe, (Wong et al. 2019) proposed fast adversarial training and suggested it is possible to construct strongly robust models via FGSM adversarial training. They showed it is important to initialize a FGSM attack with large randomized perturbations, to protect FGSM adversarial training from overfitting. Although randomly initialized FGSM (R+FGSM) has been used in previous works (Trmer et al.] (017). (Wong et al. (019) points out that the scale of the randomized initialization was restrictive and needs to be enlarged. As result, this simple modification enables R+FGSM adversarial training to obtain reasonable robustness against strong attacks.
3.2 SUB-OPTIMAL PERFORMANCE OF FAST ADVERSARIAL. TRAINING
2019 it is claimed that fast adversarial training has comparable performance as In (Wong et al.
PGD adversarial training, yet they only compared to the original results from (Madry et al. 2017). Recent work (Rice et al. 2020) has shown PGD adversarial training can be greatly improved with the standard piecewise learning rate schedule and early stopping.
In Figure we compare fast adversarial training (solid lines) and PGD adversarial training (dash lines) on a PreAct ResNet-18 (He et al., 2016a) model for classifying CIFAR- 10 (Krizhessy et al. 2009) under 10-step PGD attacks (e 8/255). For fast adversarial training, we cyclic learning
3

---

Under review as a conference paper at ICLR 2021
ClesnCCyr RobustiCykll...
(lan(PPa) RebustPdD)
0.8
Test Accuraty(9))
0.7
0.8
0.8
Test Accuracy
Test Accuracy(85)
0.6
0.6
0.5
0.4
0.4
0.4
0.2
30
50 Number of epochs
40
0.0
0.0
20
500
1000
1500
Training Epochs
Training Batches
Figure The epoch number where overfitting happens for FastAdv with the piecewise learning rate schedule, versus the best clean (orange) and ro- bust (green) accuracy before overfit- ting. The solid lines and dashed lines are the clean and robust accuracy for FastAdv with cyclic learning rates and PGD adversarial training with piece- wise learning rates.
Comparison of success mode (orange) and failure mode (green) of fast adversarial training for 30 epochs. In plot (a), show the test clean and robust accuracy for each epoch. In plot (b), we report the same quantities for every 10 batches for the last epochs. Both models encounter catastrophic overfitting, but only the model in the failure mode never recovers.
Figure 2:
rate schedule (Smith & Topin 2019). which linearly increases and decreases the learning rate. In particular, we linearly increase the learning rate from to 0.2 in the first 12 epochs and decrease to O for the last 18 epochs as recommended in (Wong et al. 2019). For PGD adversarial training, we use piecewise learning rate schedule which starts at 0.1 and decay by a factor of 0.1 at the 50th and the 75th epoch for a total of 100 epochs.
A clear gap in the robust accuracy is illustrated in Figure [ There are two main factors accounting for this performance gap. First, although model trained with the cyclic learning rate can converge in only few epochs, it often results in sub-optimal results in the adversarial training setting (Rice itt al.) 2020) compared to the piecewise learning rate schedule. The issue is that fast adversarial training is forced to cyclic learning rate schedule. If piecewise learning rate schedule is used for fast adversarial training for a large number of epochs, the model will still encounter catastrophic overfitting. We fast adversarial training with 25 different random seeds for 100 epochs, with the same piecewise learning rate schedule for PGD adversarial training, and terminated it when catastrophic overfitting happened. We add in Figure T the epoch number where the overfitting happens, versus the best clean and robust accuracy before overfitting.
The results show that none of the training progress exceeds even the 70th epochs without encountering catastrophic overfitting, For training progress terminated before the 50th epoch, where the learning rate drops, their performance inferior due insufficient training. On the other hand, although the rest of training progress also terminate early, they consistently outperformed fast adversarial training with the cyclic learning rate schedule. other words, if fast adversarial training can run for more epochs with the piecewise learning rate schedule, it has the potential to improve upon fast adversarial training with the cyclic learning rate schedule.
Another reason for the inferior performance of fast adversarial training is the inherent weakness of FGSM compared to PGD attacks. As PGD in general a better approximation to the solution for the inner maximization problem in ( it is expected produce more robust models. We seek to address this issue in Section
3.3 UNDERSTANDING FAST ADVERSARIAL TRAINING
Although Wong et al. 2019) has shown that initialization with large randomized perturbation results in effective FGSM adversarial training, the underlying mechanism for its effectiveness remains puzzle. Moreover, even with the recommended setting, catastrophic overfitting still happens on occasion. Plot (a) in Figure 2 shows both a success mode (orange) and a failure mode (green) when fast adversarial training with cyclic learning rate for 30 epochs. It seems that the model
4

---

Under review as a conference paper at ICLR 2021
in the success mode never encounters overfitting, while the model in the failure mode encounters catastrophic overfitting at around the 26th epoch. However, surprisingly. if we look closer at the training progress in plot (b), where (with batch size of 128) for the last 5 epochs, is observed that the model the success mode also encounters sudden drop in test robust accuracy, indicating catastrophic overfitting, but it recovers immediately.
report the test clean and robust accuracy for every 10 batches
This finding explains why fast adversarial training could run more epochs than vanilla FGSM adversarial training. It is not because it can completely avoid catastrophic overfitting, as previously believed; rather, it is because can recover from the catastrophic overfitting in a few batches. This has not been observed before because normally a model is only evaluated per epoch, while such "overfit-and-recover behavior happens within span of a few batches.
The observation in Figure 2 also suggests that, when catastrophic overfitting happens, although the model quickly transforms into non-robust one, it is fundamentally different from an ordinary non-robust model. In fact, the non-robust model due catastrophic overfitting can quickly retain its robustness once the corresponding attack find the correct direction for constructing attacks again. Thus, the root of the effectiveness of randomized initialization in fast adversarial training is its ability to help escaping from catastrophic overfitting. On the other hand, also explains why fast adversarial training still overfits after long training progress. Randomized initialization works with high probability, but not always. As the training progress continues, the model is more capable of overfitting, and fast adversarial training is less likely to find the correct direction for constructing FGSM attacks.
To verify our analysis, we conduct experiments to exclude the use of randomized initialization, that is to use vanilla FGSM adversarial training. but also run PGD adversarial training for a few batches when catastrophic overfitting happens. In particular, we monitor the PGD robust accuracy on validation set during training. Once there is sudden drop of the validation robust accuracy, which indicates the occurrence of overfitting, we run PGD adversarial training for few batches help the model recover from overfitting, as an alternative to R+FGSM. The same piecewise learning rate schedule as in Figure is used. We also run the vanilla fast adversarial training as a reference.
Clean
Robust Acc
(b)
Valid Accuracytss
0.9
Valid Accurecy
0.8
Accuraccy
0.8
0.6
0.6
0.6
0.4
0.4
0.2
0.0
0.0
0.0
5000
10000
15000
10000
20000
30000
40000
10000 20000 30000 40000 Training Batches
Training Batches
Training Batches
Figure 3: Validation clean and robust accuracy evaluated for every 20 batches. (a) Fast adversarial training. (b) FGSM with randomized initialization but using PGD to mitigate overfitting. (c) FGSM with randomized initialization and PGD to mitigate overfitting. All training uses the same piecewise learning rate schedule.
The results are illustrated in Figure 3 where we report the robust accuracy under 10-step PGD attacks with size e 8/255 for every 20 batches. Comparing plot (a) and (b), while the fast adversarial train- ing overfits after around 15,000 batches, FGSM adversarial training without randomized initialization obtains robust accuracy despite several "overfit-and-reoover behaviors. This confirms our hypothesis that the essential nature of successful FGSM adversarial training is not the randomized initialization, but the ability to recover from catastrophic overfitting.
4 A SIMPLE FIX TO FAST ADVERSARIAL TRAINING
The analysis and experimental results above suggest that (i) FGSM adversarial training is useful as long as it can recover from catastrophic overfitting, and (ii) fast adversarial training can only run for limited epochs because the randomized initialization is not reliable. Therefore, reliable way
5

---

Under review as a conference paper at ICLR 2021
to mitigate catastrophic overfitting is needed. To this end, propose simple fix to fast adversarial training. incorporating PGD adversarial training when catastrophic overfitting is observed.
Algorithm Improved fast adversarial training for T epochs, given some radius c, PGD steps, step size a, threshold e, frequency of detection 8, and a dataset of size M for a network fa.
for t = 1...7 T do
for = 1... .M do
if Ace > Accvalid + c then 5 = PGD Aft Jz. Vs) // Overfitring happens, run PGD adversarial training else
8 = R+FGSM Attack vi) // No Overfitting, use R+ R+FGSM adversarial training
end if
0 0 Vol(fe(z; 8), v)) // Update model weights with some optimizer, e.g. SGD
end for
if 1% == then
Let Accist = Accvalid- Update robust accuracy Accvalid under PGD attacks.
end if
end for
The proposed approach described Algorithm T The core idea simple and has been described briefly in the previous section: hold out validation set and monitor its robust accuracy for detecting overfitting. When there a drop on the validation robust accuracy beyond a threshold, which point catastrophic overfitting happens, we run 10-step PGD adversarial training for a few batches to help the model regain its robustness.
Note that although the training progress in plot (a) of Figure 3 overfits around the 15,000th batch, the frequency of catastrophic overfitting is much lower compared to the training progress in plot (b), where randomized initialization used. This also imply that although randomized initialization cannot prevent catastrophic overfitting, it effectively reduces its occurrences. To verify this conjecture, we perform the same experiment as in plot (b), but now with randomized initialization, and show the training progress in plot (c) of Figure 3 The occurrences of catastrophic overfitting is much fewer than in plot (b), confirming our conjecture. Therefore, we keep the large randomized initialization for resulting in R+FGSM adversarial training. The infrequent occurrences of catastrophic overfitting also ensures the additional PGD adversarial training adds little computational overhead to the training progress.
FGSM adversarial training in Algorithm !
Table 1: CIFAR-10 10 standard and robust accuracy on PreAct ResNet-18 18 for vanilla PGD adversarial training with early stopping, fast adversarial training (FastAdv), improved fast adversarial training (FastAdv+) and fast adversarial training as a warmup for PGD adversarial training (FastAdyW).
Method
Clean Accuracy
PGD(e 8/255)
Time/Epoch(min) Total Time(min)
PGD
83.43 + 0.25%
51.74 + 0.17%
1.66
166.45
FastAdy
83.41 + 0.13%
46.14 + 0.08%
0.31
12.35
FastAdv+
83.54 + 0.22%
48.43 + 0.14%
0.32
32.14
FastAdvW
83.18 + 0.18%
53.09 = 0.11%
0.41
40.73
Hyperparameters In Table I we report the final clean and robust accuracy of the improved fast adversarial training (FastAdv+). We use the same piecewise learning rate schedule and early stopping as used in PGD adversarial training. We detect overfitting every s = 20 batches with a randomly sampled validation batch, and PGD adversarial training runs for s overfitting happens. The threshold for detecting catastrophic overfitting is 0.1. The robust accuracy is evaluated against 50-step PGD attacks with 10 restarts for = 8/255. Note we use half-precision computations (Micikevicius et al. (017) as recommended in (Wong al. (019) for all training methods, for acceleration. All experiments are repeated for 5 times, and both the mean and the standard deviation are reported.
batches until the when
Efficiency Although FastAdv consumes less time, thus seems to be more efficient, it is worth noting that the computational time per epoch is almost the same for FastAdy and FastAdv+. FastAdy consumes less time merely because the training progress is forced to stop before its convergence. On

---

Under review as a conference paper at ICLR 2021
the other hand, our proposed FastAdv+ allows the training process converge and results in better performance.
5 FAST ADVERSARIAL TRAINING AS A WARMUP
We are able to improve the performance of fast adversarial training by allowing longer training progress. However, the associated robust accuracy is still noticeably worse than PGD adversarial training. This is expected, as PGD is inherently a stronger attack than FGSM. In this section, we adapt a previously studied technique (Wang et al., 2019), using FGSM adversarial training as a warmup for PGD adversarial training, to close the gap between fast adversarial training and PGD adversarial training.
It has been observed in (Wang et al., 2019) that using FGSM at the early stage of PGD adversarial training does not degrade its performance, and even provides slight improvement. The intuition behind this is that at the early stage of training, the model is vulnerable to adversarial attacks, and therefore there is no difference between using a weak attack and a strong attack. As the training proceeds, the model becomes more robust to weak attacks, and sometimes even overfits, where stronger attacks are more effective at increasing the robustness of the model.
However, due to the risk of catastrophic overfitting, only a few epochs of FGSM adversarial training were used in (Wang et al. 2019) as a warmup, and consequently it does not provide much improvement on the robust accuracy, nor does it save much training time. As FastAdy+ can run for many epochs as needed, it is possible to use it for most of the training epochs and PGD adversarial training for only few epochs at the end.
Starting Point of PGD Adversarial Training Since early stopping always happens few epochs after the learning rate decay, we starts PGD adversarial training a few epochs before the learning rate decay, to minimize the span of PGD adversarial training for the purpose of efficiency. In the experiments, we run the improved fast adversarial training for the first 70 epochs and change to PGD adversarial training. The early stopping happens at the 78th epoch, meaning we only run PGD adversarial training for no more than 10 epochs.
We report in Figure 4 the validated clean and robust accuracy during the whole training progress for FastAdv+ warmup, termed FastAdvW. While
FastAdr+ PSD
FazzAdVW)
FaszAdwV
Valid Accuracy(Ni)
using FastAdyW improves upon FastAdv+, still suffers from overfitting (however, not catastrophic overfitting) in the later stage of training. We assume this is due the fact that the FGSM attack with a large randomized initialization is already strong, in contrast to vanilla FGSM adversarial training used in (Wang et al. 2019). Following the intuition that only a weak attack needed in the early stage, we reduce the size of perturbation from 8/255 to 4/255 for the stage of fast adversarial training. As shown in Figure 4 this change of the attack size (FastAdvW 4-8) allows the model to reach higher robust accuracy. We also report the final test clean and robust accuracy, which is based on early stopping[ on the test set in Table This shows FastAdvW outperforms PGD adversarial training in robust accuracy and is comparable on clean accuracy, while consuming much less time.
0.35
0.30
0.25
20
60
100
Training Epochs
Figure 4: Robust accuracy on a validation set, evaluated for FastAdv+, PGD adversarial train- ing, FastAdvW W with a constant perturbation size c 8/255, and with = 4/255 and 8/255 in two stages.
6 ADDITIONAL EXPERIMENTS
In the above analyses, we only evaluated the proposed approach on CIFAR-10 using the PreAct ResNet- 18 architecture. In this section, we run more experiments on various data sets and model architectures to show the generality of our results.
For FastAdyW. W. we only consider stopping after PGD adversarial training starts.
7

---

Under review as a conference paper at ICLR 2021
We first show in Table 2 results for both CIFAR-10 and CIFAR- 100 the Wide-ResNe 34-10 (Zagoruyko & Komodakis 2016) as this model structure widely used in the adversarial training literature (Madry al. [017] Shafahi et al. 2019 Zhang et al. 2019b Rice et al., 2020). The same setting of hyperparameters in Section 4 is used, except the threshold for detecting "catastrophic overfitting" is reduced to c 0.05 for CTFAR-100
accommodate for its range of robust accuracy.
In addition, we also include in Table 2 results for "free" adversarial training (Shafahi et al. 2019).
This approach reduces the computational cost of PGD adversarial training utilizing the "minibatch replay" technique, which adds adversarial perturbations and updates the model simultaneously on the same minibatch for several iterations, imitate PGD adversarial training. As a result, this approach only needs to run for several epochs to converge. In this experiments, we follow the recommendation in (Shafahi et al. 2019) and replay each batch m times for a total 25 epochs.
Finally, we conduct experiments on Tiny ImageNet, with results also summarized in Table Although previous works (Wong et al. 2019 [Shafahi et al. 2019) conduct experiments on ImageNet, it still requires several GPUs to run. As we only run experiments on single RTX 2080ti, we considered Tiny ImageNet, which consists of 200 ImageNet classes at 64 x 64 resolution. The architecture we use is ResNet-50 (He et al. (016b) and the hyperparameters, such as learning rate and size of attacks, are kept the same as for CIFAR datasets.
Table CIFAR- 10 and CIFAR-100 standard and robust accuracy, and the corresponding training time on Net 34-10 for PGD adversarial training, free ad adversarial training (Free), FastAdv, FastAdv+ and FastAdwW. PGD attacks with 50 iterations and 10 restarts are used for evaluation.
Data
Method
Clean Accuracy
PGD(E 8/255)
Total Time(min)
PGD
85.27 + 0.31%
54.10 0.20%
998.27
Free (m=8)
85.87 + 0.37%
46.13 + 0.19%
358.53
CIFAR-10
FastAdv
85.21 + 0.22%
46.36 = 0.13%
82.32
FastAdv+
86.52 = 0.25%
51.01 + 0.18%
143.17
FastAdvW
85.91 + 0.36%
55.13 + 0.22%
237.87
PGD
61.92 + 0.20%
26.60 + 0.14%
1014.47
Free (m=8)
62.11 + 0.22%
25.37 + 0.00%
362.83
CIFAR-100
FastAdy
55.71 + 0.17%
27.50 + 0.13%
103.83
FastAdv+
2 60.57 0.25%
184.50
FastAdvW
1.21
28.88 - 0.17%
257.13
PGD
45.3410.00 0.30%
21.62 0.11%
2099.83
Free (m=8)
34.75 0.11%
14.30 + 0.07%
743.94
Tiny ImageNet
FastAdv
48.31 + 0.21%
19.96 + 0.08%
194.51
FastAdv+
48.02 = 0.19%
20.05 + 0.11%
330.65
FastAdvW
46.73 + 0.31%
21.82 = 0.19%
592.22
The results in Tableeaare consistent with what we have observed the PreAct ResNet- architecture for CIFAR-10. While FastAdv+ outperforms vanilla fast ad adversarial training as result of longer training progress, its robust accuracy no better than PGD adversarial training. However, when we use FastAdv+ as warmup, its clean accuracy becomes comparable to PGD adversarial training. while its robust accuracy constantly outperforms PGD adversarial training. In addition, FastAdvW only consumes 25% of the training time of PGD adversarial training. It is also worth noting that although free adversarial training uses a piecewise learning rate schedule as well, it only obtains its best performance at the end of the training progress, thus does not benefit from early stopping in both the performance and the efficiency.
6.1 SANITY CHECK
We now perform sanity check following the suggestions from (Athalye et al., 2018) ensure proposed approaches are truly robust. For all the sanity checks, we evaluate our approaches on both PreAct ResNet-18 18 and Wede-ResNet 34-10 for classifying CIFAR-10.
We first show the clean and robust accuracy under 10-step PGD attacks under varying sizes from 0 12/255 in Figure 5 It can be observed that the decreasing trend of robust accuracy of our proposed approaches are consistent with models trained via PGD-AT and FastAdv, where FastAdyW outperforms PGD-AT and FastAdv+ outperforms FastAdv consistently.
8

---

Under review as a conference paper at ICLR 2021
PGD-AT
FastAAvv
FastAdv+
FastAdvW
Robust Accuracy( (%)
(a)
Robust Accuracy(
(b)
80
80
60
60
40
40
5 Size of Attacks
10
10
Size of Attacks
Figure 5: Robust accuracy under 10-step PGD with step sizes from 0 to 12/255. The evaluations are performed (a) PreAct ResNet-18 and (b) Wide-ResNet 34-10 for classifying CIFAR-10. We also include results for PGD-AT, FastAdy for comparison.
In addition, we show robust accuracy under other types attacks that do not rely estimating gradients. AutoAttack (Croce & Hein 2020) ensembles four diverse attacks reliably evaluate robustmess: two improved PGD attacks, boundary based attack FAB (Croce & Hein 2019), and query- query-based black-box attack Square Attack (Andriushchenko et al. 2019). We also use the recently proposed state-of-the-art black-box attacks (Tashiro et al. (2200) to evaluate our models. This attack utilizes output diversified sampling to construct transferable black-box attacks and almost as powerful as white-box PGD attack. The results are summarized on Table
Table 3: CIFAR- 10 standard and robust accuracy under various types of attacks on PreAct ResNet-18 and Wide-ResNet 34-10 for PGD adversarial training, FastAdv, FastAdv+ and FastAdvW.
Architecture
Method
Clean Accuracy
PGD-10
AutoAttack
Transfer
PGD
83.21%
51.62%
46.75%
56.13%
FastAdv
83.55%
46.86%
41.63%
49.86%
R8sNet-18
FastAdy+
83.59%
48.29 48.29%
43.40%
52.06%
FastAdvW
83.11%
54.09%
47.61%
57. 57.13%
PGD
85.42%
54.70%
49.91%
59.94%
FastAdy
85.34%
48.55%
43.58%
51.98%
WideResNet
FastAdv+
86.43%
50.11%
44.83%
55.39%
FastAdvW
86.44%
55.50%
50.57%
62.16%
We observe that no attack significantly reduces the robust accuracy of our proposed approaches, and different approaches consistent. AutoAttack reduces approxi-
the relative performance mately 5% more robust accuracy than 10-step PGD, which is consistent with the observations in (Croce & Hein 2020
CONCLUSION
We have conducted experiments to show that the key the success of FGSM adversarial training is the ability to recover from "catastrophic overfitting". Fast adversarial training utilizes randomized initialization to achieve this goal but still suffers from catastrophic overfitting for large number of training epochs. We design new training strategy that mitigates this caveat and enables the commonly used piecewise learning rate schedule for fast adversarial training and, as a result, improves clean and robust accuracy. We also the improved fast adversarial training adversarial training, and find it is sufficient to use this warmup for a majority of the training epochs to time and further improve model robustness. As result, obtain superior performance the expensive, state-of-te of-the--rttttt--e PGD adversarial training with much-reduced training time. Our proposed approaches easy implemented, thus could be used baselines for empirical adversarial defense.
warmup for PGD

---

Under review as a conference paper at ICLR 2021
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. arXiv preprint 19X.:::9/2.00000 2019.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: 2018.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv: 2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Security and Privacy (SP), 2017 IEEE Symposium on, pp. 39-57. IEEE, 2017.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705. 2019.
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint
2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. arXiv preprint arXiv:1907.02044 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. arXiv preprint XXiv:2003.01690. 2020.
Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and Pushmeet Kohli. A dual approach to scalable verification of deep networks. arXiv preprint arXiv:1803.06567. 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016b.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems, pp- 2266-2276, 2017.
J Zico Kolter and Eric Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv: 1711v0171..008:: 2017.
Alex Krizhevsky et Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint (tf:::/611..02555 2016.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness adversarial examples with differential privacy. arXiv preprint arXiv: 1rXiv:1802.000 1802.0347,, 2018.
Bai Li. Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. In Advances in Neural Information Processing Systems, pp. 9459-9469, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1006:067060008, 2017.
10

---

Under review as a conference paper at ICLR 2021
Paulius Micikevicius, Sharan Narang. Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXhv:1710.037//44 2017.
Seyed-Mohsen Moosavi-Dezfool Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 2574-2582, 2016.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In Security and Privacy (SP), 2016 IEEE Symposium on, pp. 582-597. IEEE, 2016.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. arXiv preprint arXiv:1801.09:3 2018.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning, arXiv preprint arXiv:2002. 1569, 2020.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3353-3364, 2019.
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Malti- Operations Applications, volume 11006, pp. 1100612. International Society for Optics and Photonics, 2019.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv: 2013.
Yusuke Tashiro, Yang Song, and Stefano Ermon. Output diversified initialization for adversarial attacks. arXiv preprint arXiv:2003.06872 2020.
Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint TrX.::/705.07200 2017.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002.088 083477 2020.
Shiqi Wang, Yizheng Chen, Ahmed Abdou, and Suman Jana. Mixtrain: Scalable training of formally robust neural networks. arXiv preprint arXiv:1811.0200 0.0622,, 2018.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. International Conference on Machine Learning, pp.
2019.
Tsui- Wei Weng, Huan Zhang. Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit Dhillon, and Luca Daniel. Towards fast computation of certified robustness for relu networks. arXiv preprint arXiv: 880499969, 2018
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. arXiv preprint arXiv: 12514, 2018.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2019.
Cihang Xie, Yuxin Wu, Laurens der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 501-509, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.00 2016.
11

---

Under review as a conference paper at ICLR 2021
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227-238, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint frXiv:1901.08573 2019b.
Huan Zhang, Tsui- Wei Weng, Pin- Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Advances in Neural Information Processing Systems, pp. 4944-4953, 2018.
A APPENDIX
B
STRONG ATTACKS AFTER OVERFITTING
When FastAdv+ used for training model, even though the model can recover from catastrophic overfitting via PGD adversarial training, is possible that the model overfits PGD attacks and stays vulnerable to other attacks. Therefore, we extract the model right after its recovery from catastrophic overfitting and run several kinds of attacks, including 10-step PGD attacks, 50-step PGD attacks with 10 restarts, C&W attacks (Carlini & Wagner 2017) and fast adaptive boundary (FAB) attacks (Croce Hein 2019), on this model.
Table 4: attacks.
CIFAR-10 standard and robust accuracy PreAct ResNet-18 18 under various types of
Attacks
PGD-10
PGD-50
C&W
FAB
Robust Accuracy
40.22%
39. 39.41%
41.05%
38 38.68%
The result shows the model recovered from catastrophic overfitting is indeed robust. Note the robust accuracy is relatively low are not using the final model.
C ABLATION ANALYSIS ON ADJUSTED ATTACK SIZE
In Section 5 we show it is possible to improve the performance of FastAdvW via using smaller size of attacks for FGSM adversarial training. It is possible that the adjusted size attacks benefits not only our approach, but also PGD adversarial training. Therefore, the same setting (4/255 for the first epochs and 8/255 for the rest) for full PGD adversarial training and compare it to vanilla PGD adversarial training.
Table 5: CIFAR-10 10 standard and robust accuracy PreAct ResNet-18 for vanilla PGD adversarial training and PGD adversarial training with adjusted size of attacks (4/255 and 8 8/255).
Method
Standard Accuracy PGD(c 8/255)
PGD
83.43 + 0.25% 83.11 + 0.11%
51.74 0.17%
PGD(adjusted size)
52.14 + 0.28%
The results show that PGD adversarial training enjoys limited benefits from the adjusted size of attacks. This strategy is more compatible with our proposed FastAdvW.
12


Reviews:
[{"review_id": "mf_r8wO1T8b", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "################################## Summary ##################################\n\nThis paper shows that the main reason for the success of Fast Adversarial Training ([1], will be referred to as FBF in this review) is its ability to recover from catastrophic overfitting. Based on this observation, the authors propose to utilize PGD multi-step training for a few iterations (when catastrophic overfitting occurs), and resume single-step training after the model recovers. Further, the authors also propose to use this improved Fast Adversarial Training (FastAdv+) as a warmup for PGD Adversarial Training, and demonstrate improved performance over PGD-AT, at a significantly lower computational cost. \n\n[1] Wong et al. Fast is Better than Free: Revisiting Adversarial Training, ICLR 2020\n\n################################### Pros #####################################\n  \n  -  The authors present a very interesting finding, that FBF has catastrophic overfitting for a few intermediate iterations, and recovers very quickly from this. \n  -  Based on this observation, they propose to use PGD based training for a few intermediate iterations, which seems to prevent this overfitting effectively \n  -  The authors also propose the use of this training as warm up for PGD training, and demonstrate significantly improved results. \n  -  This is certainly a significant contribution of the paper since it achieves improved results at a much lower computational cost. \n\n################################### Cons #####################################\n\n  -  Since the proposed defenses involve single-step training, the absence of gradient masking needs to be justified using thorough validation as discussed by Carlini et al. [1], using gradient-free attacks such as Square attack/ SPSA, black-box transfer based attacks, attacks with multiple steps and multiple random restarts. Also, the sanity checks proposed by Athalye et al. [2] need to be demonstrated. \n  -  The paper discusses results only on PGD attack, which is not the current state-of-the-art. The proposed defenses (FastAdv+, FastAdvW) must be evaluated on stronger attacks such as AutoAttack [3] and MultiTargeted Attack [4]. \n  -  While the use of the modified FGSM attack would improve the efficiency of PGD training, it is not clear why it should lead to improved robustness.  \n  -  Could the authors clarify if all the other training/ hyperparameter settings (such as batch size, weight decay, optimizer, initial learning rate and schedule, number of epochs, initial random noise added for the attack, validation split, use of early stopping, batch norm in train/ eval mode during attack generation [5]) are similar for PGD-AT (reported in Table-4 of the Appendix) and FastAdvW (reported in Table-1)? \n  -  The learning rate schedule used for PGD-AT (Sec. 3.2) is different from that used by Rice et al. So, the results could be sub-optimal. Could the authors use similar settings as Rice et al. (SGD optimizer using a batch size of 128, a step-wise learning rate decay set initially at 0.1 and divided by 10 at epochs 100 and 150, total epochs 200 and weight decay 5e-4) for reporting the PGD baseline results? \n \n[1] Carlini et al., On evaluating Adversarial Robustness, https://arxiv.org/abs/1902.06705\n\n[2] Athalye et al., Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018\n\n[3] Croce et al., Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks, ICML 2020\n\n[4] Gowal et al., An Alternative Surrogate Loss for PGD-based Adversarial Testing, https://arxiv.org/pdf/1910.09338.pdf\n\n[5] Bag of Tricks for Adversarial Training, https://openreview.net/forum?id=Xb8xvrtB8Ce\n\n############################## Reasons for score #################################\n \nThe paper highlights a very interesting finding in FBF training, proposes a single-step defense, and also an improvement to speed-up PGD-AT. However, it does not show sufficient experimental results for reliable evaluation of the defenses and to ensure the absence of gradient masking. Hence I think the paper is marginally below the acceptance threshold. I will be happy to increase the score if the required experimental results are presented during the rebuttal. \n\n######################### Questions during rebuttal period ###########################\n\n  -  Could the authors provide the following results (for CIFAR-10) for both the proposed defenses (FastAdv+, FastAdvW). \n        *  Evaluation against AutoAttack [3] and MultiTargeted Attack [4]. (It would help to also report the corresponding results for the baselines - FBF, PGD-AT)\n        *  Plot of robust accuracy vs. attack distortion bound (epsilon for PGD-10 step attack) as discussed by Athalye et al. [2] \n        *  Accuracy on black box transfer based attacks (using normally trained model of the same architecture as a source)\n  -  Could the authors also report PGD-10 step accuracy for PGD-AT and FastAdvW for CIFAR-10 dataset on PreActResNet-18 and WRN-34-10? This would help with comparison of baselines against those reported in prior work.\n  -  In the proposed defense FastAdv+, could the authors clarify what is the fraction of validation split used for detecting catastrophic overfitting and what is the number of steps used for the PGD attack on the validation set? Is the validation time included in the time reported in Table-1? If not, could the authors report the total time including validation?\n  -  Could the authors clarify the size of the validation split used for early stopping (also in Fig.4) and the number of steps used for the PGD attack for early stopping? Is this consistent across all experiments? \n  -  (good to have) Loss surface plots for the proposed defenses to show the absence of gradient masking. (similar to those reported in [5])\n\n################## Additional Feedback (not part of decision assessment) ###################\n\n  -  The authors mention the following: “although the model quickly transforms into a non-robust one, it is fundamentally different from an ordinary non-robust model”. It would be insightful to study the properties of this intermediate model that suffers from catastrophic overfitting, to understand more about why it is able to recover so quickly. \n  -  The authors could visualize the loss surface of the FBF trained model during the catastrophic overfitting, and immediately after it recovers. This can lead to insightful findings on what is happening in the vicinity of the data samples in very few iterations. The loss surface can be plotted similar to Fig.1 in [5]. \n\n[5] Tramer et al., Ensemble Adversarial Training: Attacks and Defenses, ICLR 2018\n\n############################# Update after rebuttal ###############################\n\nThe authors response addresses my concerns and I would like to update my score to 7. The paper presents insightful findings about FBF, and proposes a simple and effective method for stabilizing single-step adversarial training. This is useful not only for FBF but also for other single-step defenses. Although the gain in robustness is marginal, stabilizing single-step training is useful. The authors also propose a computationally efficient method of achieving robustness similar to PGD training. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "kpU-COGQEuq", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes a method called: improved fast adversarial training (FastAdv+) which improves fast adversarial training by replacing randomized initialization with PGD adversarial training when there is overfitting issue during training.\n\nStrengths:\nThe idea is reasonable and easy to implement. There is a marginal improvement of accuracy under PGD attack ($\\epsilon=8/255$) of around 2%, comparing Fast adversarial training (FastAdv) with improved fast adversarial training (FastAdv+) on CIFAR10. \n\nWeakness:\n1. FastAdv+ itself does not perform better than PGD adversarial training. When use the FastAdv+ result as starting point then do PGD adversarial training, it outperforms PGD adversarial training. Therefore, FastAdvW (the combined method) is actually more computational expensive than PGD adversarial training. \n2. The advantage of Fast adversarial training is that it is less computational expensive than PGD adversarial training so it can scale up to large dataset like ImageNet. However, FastAdvW does not have this advantage anymore.\n3. The improvement of FastAdv+ over FastAdv on CIFAR100 and TinyImagenet is marginal.\n4. The methods are evaluated under one attack strength ($\\epsilon=8/255$ for CIFAR10). It is better to evaluate the methods under different attack strengths.\n\nClarity:\nThe paper is clearly written and easy to follow.\n\nReproducibility:\nDetails of the algorithm is provided but code is not.\n\nConclusion:\nThe method is novel but the contribution is not strong.", "extra_scores": {}, "extra_text": {}}, {"review_id": "SBHkrIl_LY0", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors claimed in this paper that as the most empirically successful approach to defending adversarial examples, PGD-based adversarial training, is computationally inefficient. Fast adversarial training could mitigate this issue by training a model using FGSM attacks initialized with large randomized perturbations, but the underlying reason for its success remains unclear and it may still suffer from catastrophic overfitting. The authors conducted a series of experiments to figure out the key to the success and properties of fast adversarial training. The experimental results showed that fast adversarial training cannot avoid catastrophic overfitting, but could be able to recover from catastrophic overfitting quickly. Based on all of the observations, the authors proposed a simple method to improve fast adversarial training by using PGD attack as training instead of R+FGSM attack (proposed in fast adversarial training) when overfitting happens, or using fast adversarial training as a warmup. The proposed methods could achieve slightly better performance than the current state-of-art approach while reducing the training time significantly.\n\n#####################################################################\n\nOverall, I vote for weak reject (marginally). I like the idea of exploring the properties of adversarial training, the experiments may also be inspiring. But my major concern is that the interpretation about the ‘catastrophic overfitting’ is not clear, and the interpretation about the effectiveness of R-FGSM and PGD against overfitting is also not clear. Hopefully, the authors can address my concern in the rebuttal period. \n\nPros:\n#####################################################################Pros: \n \n1.Attempting to interpret the successful reason for a previous work is interesting. And the exploratory experiments may be inspiring for other researchers.\n\n2. Overall, the paper was well written. All the motivations and conjectures are easy to follow and understand. \n\n3. This paper provides a lot of experiments to show the effectiveness of the proposed methods which appeared slightly better than the SOTA PGD-training while reducing training time significantly.\n#####################################################################\n\n\nCons: \n \n1. Although the authors attempted to explain the key to the success of fast adversarial training, it might be still not clear theoretically: \n\n(1)\tWhy R-FGSM and PGD could guide the model to recovery from ‘catastrophic overfitting’, but FGSM could not? Does it mean that stronger attacks could guide the model to recovery?\n\n(2) Why the ‘catastrophic overfitting’ happened a lot of times when using FGSM training, but R-FGSM and PGD could mitigate it? Does it mean that stronger attacks could mitigate it?\n\n2. As I understand, Figure 3(c) should be the result of proposed FastAdv+. From Figure 3(c), it can be observed that there are ‘catastrophic overfitting’ in FastAdv+, but this phenomenon could not be seen from Figure 4. Do you have any idea to explain it?\n\n3. As concerned in my 1.(1) and (2), if weaker attacks lead to more ‘catastrophic overfitting’ and could not guide the model to recovery, why the FastAdvW 4-8 using a weaker attack as a warmup could outperform FastAdv+ and FastAdvW 8-8.\n\n4. Though the proposed methods appear useful, they may be a bit straightforward and have a limited novelty (using PGD attacked samples when ‘catastrophic overfitting’ happens)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "L0ybp9vLHOL", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an improvement over fast adversarial training to improve the robustness of the model in an efficient manner. Overall it's a well written paper but it can be improved in certain ways as follows...\n\n- Fig. 3 lacks important information about what specific attack was used to compute the robust accuracy. was it PGD? If yes, what are the PGD parameters?\n- It's not clear if the improvements are due to few iterations of PGD or due to piecewise linear learning rate regime? An explicit experiments comparing the two learning rate regimes (cyclic and piecewise) would be good to confirm. \n- For section 5, when authors are using FGSM as a warm-up, they should refer/compare with https://arxiv.org/pdf/2002.04237.pdf where clean training is used as warm-up. Author mention that strength of the adversary doesn't matter in the initial phase of training, so should compare results with varying strength going to zero. \n- In most of the experiments, the PGD step size is missing. Moreover, it will be good to see the robustness of this technique against varying PGD step size. \n- I think authors use R+FGSM and fast adversarial training interchangeably which creates some confusion.. it will be nice to be consistent with the terminology. \n- ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper first investigates the behavior (e.g., catastrophic overfitting) of fast adversarial training (FastAdv) through experiments. It finds that the key to its success is the ability to recover from overfitting to weak attacks. Then, it presents a simple fix (FastAdv+) that incorporates PGD adversarial training when catastrophic overfitting is observed. The resulting method is shown to be able to train for a large number of epochs. It also presents a version (FastAdvW) that use the improved fast adversarial training as a warmup of PDG-adversarial training, similar as in previous work. Overall, the analysis is useful and the ideas are valid. The empirical results also show promise. However, the main weakness of such empirical analysis is that it may be sensitive to the settings (e.g., # of epochs, splitting of datasets, …). The authors’ rebuttal also reflected such potential concerns.
Decision: Reject