Fast is better than free: Revisiting adversarial training
Authors: Eric Wong
Leslie Rice
J. Zico Kolter
Venue:  (2020)
Keywords: adversarial examples
adversarial training
fast gradient sign method

Abstract: Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD).  In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice.  Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost.  Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy at epsilon=8/255 in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at epsilon=2/255 in 12 hours, in comparison to past work based on ``free'' adversarial training which took 10 and 50 hours to reach the same respective thresholds. 

Published as a conference paper at ICLR 2020
FAST IS BETTER THAN FREE:
REVISITING ADVERSARIAL TRAINING
Eric Wong"
Leslie Rice
Machine Learning Department Careeiie Mellon University Piusburgh, PA 15213, USA ericwonge cmu. edu
Computer Science Department
Carnegic Mellon University Pittsburgh, PA 15213, USA Lariceècs. cmu edu
Zico Kolter
Computer Science Department
Cameeii Mellon University and Bosch Center for Artífical Intelligence Pittsburgh, PA 15213, USA zkolterêcs, emu.edu
ABSTRACT
Adversarial training, method for learning robust deep networks, is typically sumed to be more expensive than traditional training due to the necessity of con- structing adversarial examples via first-order method like projected gradient de- cent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversar- ial training with the fast gradient sign method (FGSM), when combined with ran- dom initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further acceler- ated by using standard techniques for efficient training deep networks, allowing to learn a robust CIFAR 10 classifier with 45% robust accuracy to PGD attacks with = 8/255 in minutes, and robust ImageNet classifier with 43% robust accuracy
2/255 in 12 hours, in comparison to past work based on "free" ad- versarial training which took 10 and 50 hours to reach the same respective thresh- "catastrophic overfitting"
olds. Finally, identify failure mode referred which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https: / thub. com/ locus 1 ab fast adversarial
INTRODUCTION
Although deep network architectures continue to be successful in a wide range applications, the problem of learning robust deep networks remains active area of research. In particular, safety and security focused applications are concerned about robustness to adversarial examples, data points which have been adversarially perturbed to fool model (Szegedy al. 2013). The goal here is to learn a model which not only accurate on the data, but also accurate on adversarially perturbed versions of the data. To this end, number of defenses have been proposed the problem and improve the robustness of deep networks, with some of the most reliable being certified defenses and adversarial training. However, both these approaches come at non-trivial, additional computational cost, often increasing training time by an order of magnitude over standard training. This has slowed progress in researching robustness in deep networks, due the compu- tational difficulty in scaling to much larger networks and the inability to rapidly train models when experimenting with new ideas. In response this difficulty, there has been a recent surge in work
mitigate
"Equal contribution.

---

Published as a conference paper at ICLR 2020
that tries to to reduce the complexity of generating an adversarial example, which forms the bulk of the additional computation in adversarial training (Zang et al., 2019 Shafahi et al. 2019). While these works present reasonable improvements to the runtime of adversarial training, they still significantly slower than standard training, which has been greatly accelerated due to competitions for optimizing both the speed and cost of training (Coleman al. (017).
In this work, we argue that adversarial training, in fact, is not as hard as has been suggested by this past line of work. In particular, we revisit one of the the first proposed methods for adversarial training, using the Fast Gradient Sign Method (FGSM) to add adversarial examples to the training process (Goodfellow et al. 2014). Although this approach has long been dismissed as ineffective, we show that by simply introducing random initialization points, FGSM- baseed training is as effective as projected gradient descent based training while being an order of magnitude more efficient. Moreover, FGSM adversarial training (and to lesser extent, other adversarial training methods) can be drastically accelerated using standard techniques for efficient training of deep networks, including e.g. cyclic learning rates (Smith & Topin 2018), -reecisieeiiion training (Micikevicius) ct al. 2017). and other similar techniques. The method has extremely few free parameters to tune, and can be easily adapted to most training procedures. We further identify failure mode that we call "catastrophic overfitting", which may have caused previous attempts at FGSM adversarial training to fail against PGD-based attacks.
The end result is that, with these approaches, we are able to train (empirically) robust classifiers far faster than in previous work. Specifically, train foo robust CIFAR1O model to 45% accuracy at = 8/255 (the same level attained in previous work) in 6 minutes; previous papers reported times of 80 hours for PGD-based training (Madry et al., 2017) and 10 hours for the more recent "free" adversarial training method (Shafahi et al. 2019). Similarly, we train an fac robust ImageNet classifier to 43% top-1 accuracy E = 2/255 (again matching previous results) 12 hours of training (compared to 50 hours in the best reported previous work that we are aware of (Shatahi et al. 2019)). Both of these times roughly match the comparable time for quickly training a standard non-robust model to reasonable accuracy. We extensively evaluate these results against strong PGD- based attacks, and show that they obtain the same empirical performance as the slower, PGD-based training. Thus, we argue that despite the conventional wisdom, adversarially robust training is not actually more challenging than standard training of deep networks, and can be accomplished with the notoriously weak FGSM attack.
2 RELATED WORK
After the discovery of adversarial examples by Szegedy et al. (2013). Goodfellow et al. (2014) proposed the Fast Gradient Sign Method (FGSM) to generate adversarial examples with a single gradient step. This method was used to perturb the inputs to the model before performing back- propagation as an early form of adversarial training. This attack was enhanced by adding a ran- domization step, which was referred to R+FGSM (Tramer et al. 2017]. Later, the Basic Iterative Method improved upon FGSM by taking multiple, smaller FGSM steps, ultimately rendering both FGSM-based adversarial training ineffective (Kurakin et al., 2016). This iterative adversarial attack was further strengthened by adding multiple random restarts, and was also incorporated into the adversarial training procedure. These improvements form the basis of what is widely understood today as adversarial training against a projected gradient descent (PGD) adversary, and the result- ing method is recognized as an effective approach to learning robust networks (Madry et al. 2017).
Since then, the PGD attack and its corresponding adversarial training defense have been augmented with various techniques, such as optimization tricks like momentum to improve the adversary (Dong let al. 2018). combination with other heuristic defenses like matrix estimation (Yang et al. 2019) or logit pairing (Mosbach et al., 2018)) and generalization to multiple types of adversarial attacks Tramer & Bonch, 2019 Maini et al. 2019).
In addition to adversarial training, a number of other defenses against adversarial attacks have also been proposed. Adversarial defenses span a wide range of methods, such preprocessing tech- niques (Guo et al., 2017| [Bckkman et al.] 2018 Song et al. (017)), detection algorithms (Metzen et al., 2017 Feinman et al.] [201]] Carlini & Wagner 2017a), verification and provable defenses (Katz et al. 2017: et al. [211)] Wong & Kolter 2017: Raghhnathnn et al. 20188), and various theoretically motivated heuristies (Xiao et al. 2018
Croce al. 2018). While certified defenses
2

---

Published as a conference paper at ICLR 2020
have been scaled to reasonably sized networks (Wong et al. 2018: Mirman et al. 2018 Gowal et al. 2018 Cohen et al. 2019 Salman et al. 2019). the guarantees don't 't match the empirical robustness obtained through adversarial training.
With the proposal of many new defense mechanisms, of great concern in the community is the use of strong attacks for evaluating robustness: weak attacks can give a misleading sense of security, and
the history of adversarial examples is littered with adversarial defenses (Papernot et al.] 2016 Lu
2017 Kannan et al. 2018 Tao et al. (0118) which were ultimately defeated by stronger attacks (Carlini & Wagner 2016 2017b] Athalye et al. 2017 Engstrom et al. 2018: Carlini 2019). This
al.
highlights the difficulty evaluating adversarial robustness, as pointed out by other work which
began to defeat proposed defenses en masse (Uesato et al.) 2018 Athalye et al. (0118). Since then,
several best practices have been proposed to mitigate this problem (Carlini et al. 2019)
Despite the eventual defeat of other adversarial defenses, adversarial training with a PGD adversary remains empirically robust this day. However, running a strong PGD adversary within an inner loop of training is expensive, and some earlier work in this topic found that taking larger but fewer steps did not always significantly change the resulting robustness combat the increased computational overhead of the PGD defense, some recent work has looked at regressing the k-step PGD adversary to a variation of its single-step FGSM predecessor called "free" adversarial training. which can be computed with little overhead over standard training by using single backwards pass to simultaneously update both the model weights and also the input pertur- bation (Shafahi al.] (219)). Finally, when performing multi-step PGD adversary, it is possible to cut out redundant calculations during backpropagation when computing adversarial examples for additional speedup (Zhang et al. 2019).
network (Wang 2018 To
Although these improvements are certainly faster than the standard adversarial training procedure, they are not much faster than traditional training methods, and can still take hours days com- pute. On the other hand, top performing training methods from the DAWNBench competition (Coleman al., 2017) are able to train CIFARIO and ImageNet architectures to standard bench- mark metrics in mere minutes and hours respectively, using only modest amount of computational resources. Although some of the techniques can be quite problem specific for achieving bleeding- edge performance, more general techniques such as cyclic learning rates (Smith & Topin 2018 and half-precision computations (Micikevicius et al. 2017) have been quite successful in the top ranking submissions, and can also be useful for adversaria training.
ADVERSARIAL TRAINING OVERVIEW
Adversarial training is method for learning networks which are robust to adversarial attacks. Given a network fe parameterized by 0, a dataset (I,, y/), a loss function é and a threat model A, the learning problem is typically cast the following robust optimization problem.
min Σ max El (fo(x; +6), 3/)) SEA
A typical choice for threat model is to take A {5 : |10 S c} for some > 0. This is the foo threat model used by Madry et al. (2017) and is the setting we study in this paper. The procedure for adversarial training to some adversarial attack approximate the inner maximization over A, followed by some variation of gradient descent on the model parameters 0. For example, one of the earliest versions of adversarial training used the Fast Gradient Sign Method to approximate the inner maximization. This could be seen as a relatively inaccurate approximation of the inner maximization for loo perturbations, and has the following closed form (Goodfellow et al. 2014):
8" = c- sign (ll(((() v))
A better approximation of the inner maximization is to take multiple, smaller FGSM steps of size instead. When the iterate leaves the threat model, it is projected back to the set A (for for perturba- tions. This is equivalent to clipping 5 to the interval e)), Since this is only local approximation of a non-convex function, multiple random restarts within the threat model A typically improve the approximation of the inner maximization even further. A combination of all these techniques is
3

---

Published as a conference paper at ICLR 2020
Algorithm PGD adversarial training for T epochs, given some radius adversarial step size and PGD steps and dataset of size M for network Jo
for = 1... do for = ..... M do // Perform PGD adversarial attack
8 = 0// randomly initialized
for 1 1-.1.NN do
8 = 5 ô = max(min(0,e), -e)
sign( V Vsl(fo(z; -8) v:))
end for
0 = 0 Vol(e(( + 8). vx) // Update model weights with some optimizer, e.g. SGD
end for end for
Algorithm "Free" adversarial training for T epochs, given some radius e, N minibatch replays, and dataset of size M for network fo
0
// Iterate T/N times to account for minibatch replays and run for T total epochs for t = 1... T/N do
for = 1 1... M do
// Perform simultaneous FGSM adversarial allack and model weight updates T times for 1...N... do
// Compute gradients for perturbation and model weights simultaneously V. Ve = Ve(ff (z, + 8), y:)
8 = 8 c sign (V.)
8 = max(min(6, -c)
0 = 0 V // Update model weights with some optimizer, e.g. SGD
end for
end for
end for
known as the PGD adversary (Madry et al. 2017). and usage adversarial training is summa- rized in Algorithm D1
Note that the number of gradient computations here proportional to O(MN) single epoch, where M is the size of the dataset and N is the number of steps taken by the PGD adversary. This N times greater than standard training (which has ((M), ) gradient computations per epoch), and adversarial training is typically N times slower than standard training.
3.1 "FREE" ADVERSARIAL TRAINING
To get around this slowdown of factor N, Shafahi al. (2019) instead propose "free" adversarial training. This method takes FGSM steps with full step sizes followed by updating the model weights for N iterations on the same minibatch (also referred as "minibatch replays"). The algorithm is summarized in Algorithm 2. Note that perturbations are not reset between minibatches. To account for the additional computational cost of minibatch replay, the total number of epochs is reduced by factor of N to make the total cost equivalent to T epochs of standard training- Although "free" adversarial training is faster than the standard PGD adversarial training, it is not as fast we'd d like: SSafahi al, (2019) need to run over 200 epochs in over 10 hours to learn robust CIFAR 10 classifier and two days to learn a robust ImageNet classifier, whereas standard training be accomplished in minutes and hours for the same respective tasks.
FAST ADVERSARIAL TRAINING
To speed up adversarial training and move towards the state of the art in fast standard training methods, first highlight the main empirical contribution of the paper: that FGSM adversarial
4

---

Published as a conference paper at ICLR 2020
Algorithm 3 FGSM adversarial training for T epochs, given some radius N PGD steps, step size a, and dataset of size M for network Jo
for t 1. .... do for = ......... M do
// Perform FGSM udversarial attack
8 Uniform(-c,
8 = + a (gg((ve(ffll + +0, 8 max(min( -c)
0 = Vol(fo(2; + 8), y:) ) // Update model weights with some optimizer, e.g. SGD
end for
end for
Table I: Standard and robust performance of various adversarial training methods on CIFARIO for
8/255 and their corresponding training times
Method
Standard accuracy
PGD 8/255)
Time (min)
FGSM DAWNBench
+ zero init
85.18%
0.00%
12.37
early stopping
71.14%
38.86%
7.89
+ previous init
86.02%
42.37%
12.21
random init
85.32%
44.01%
12.33
+ a 10/255 step size
83.81% %
46.06%
12.17
a 16/255 step size
86.05%
0.00%
12.06
early stopping
70.93%
40.38%
8.81
"Free" (m 8) Shafahi al. 2019
85,96%
46.33%
785
+ DAWNBench
78.38%
46.18%
20,91
PGD-7 (Madry al. 2017 + DAWNBBench
87.30%
45.80%
4965.71
82.46%
50.69%
68.8
training combined with random initialization is just as effective defense as PGD-based training. Following this, we discuss several techniques from the DAWNBench competition (Coleman al.) 2017) that are applicable to all adversarial training methods, which reduce the total number of epochs needed for convergence with cyclic learning rates and further speed up computations with mixed- precision arithmetic.
4.1 REVISITING FGSM ADVERSARIAL TRAINING
Despite being quite similar to FGSM adversarial training, free adversarial training empirically robust against PGD attacks whereas FGSM adversarial training is not believed be robust. To analyze why, we identify a key difference between the methods: a property of free adversarial training that the perturbation from the previous iteration is used as the initial starting point for the next iteration. However, there is little reason to believe that adversarial perturbation for previous minibatch is reasonable starting point for the next minibatch. As result, we hypothesize that the main benefit comes from simply starting from non-zero initial perturbation.
In light of this difference, our approach is to use FGSM adversarial training with random initializa- tion for the perturbation, shown in Algorithm 3 We find that, in contrast to what was previously believed, this simple adjustment to FGSM adversarial training can used as an effective defense on par with PGD adversarial training. Crucially, we find that starting from non-zero initial pertur- bation the primary driver for success, regardless of the actual initialization. fact, both starting with the previous minibatch's perturbation initializing from uniformly random perturbation al-
10 restarts, which strictly weaker using different network architecture and an adversary with 20 steps adversary used in this paper.
As reported Shafahi
2019
As reported by Madry al. (2017) using different network architecture and adversary and adversary with 20 steps and restarts, which strictly weaker than the adversary used in this paper
5

---

Published as a conference paper at ICLR 2020
low FGSM adversarial training to succeed at being robust to full-strength PGD adversarial attacks. Note that randomized initialization for FGSM is not a new idea and was previously studied by Tramer et al. (2017). Crucially, Tramer et al. (2017) use different, more restricted random initial- ization and step size, which does not result in models robust to full-strength PGD adversaries. A more detailed comparison of their approach with ours is in Appendix A
To test the effect of initialization in FGSM adversarial training, we train several models to be robust radius e = 8/ /25 on CIFARIO, starting with the most "pure" form of FGSM, which takes steps of size a = e from a zero-initialized perturbation. The results, given in Table T are consistent with the literature. and show that the model trained with zero-initialization is not robust against a PGD adversary. However, surprisingly, simply using a random or previous minioouuhiiibtchhhhhh initialization instead of a zero initialization actually results in reasonable robustness levels (with random initialization performing slightly better) that are comparable to both free and PGD adversarial training methods. The adversarial accuracies in Table [ are calculated using a PGD adversary with 50 iterations, step size = 2/255, and 10 random restarts. Specific optimization parameters used for training these models can be found in Appendix B
FGSM step size Note that an FGSM step with size a = e from non-zero initialization is not guaranteed to lie on the boundary of the lo ball, and so this defense could potentially be seen as too weak. We find that increasing the step size by a factor of 1.25 to 10/ 10/255 further improved on par with the best reported result from free adversarial
the robustness of the model so that training. However, we also found that forcing the resulting perturbation to lie on the boundary with a step size of a = 2e resulted in catastrophic overfitting: it does not produce a model robust to adversarial attacks. These two failure modes (starting from -ro-initialized perturbation and generating perturbations at the boundary) may explain why previous attempts at FGSM adversarial training failed, as the model overfits restricted threat model, and is described in more detail in Section[3.4] A full curve showing the effect of a range of FGSM step sizes on the robust performance can be found in Appendix C
Computational complexity A second key difference between FGSM and free adversarial training is that the latter uses a single backwards pass to compute gradients for both the perturbation and the model weights while repeating the same minibatch m times in a row, called "minibatch replay". In comparison, the FGSM adversarial training does not need to repeat minibatches, but needs two backwards passes to compute gradicnts separately for the perturbation and the model weights. As a result, the computational complexity for an epoch of FGSM adversarial training is not truly free and is equivalent to two epochs of standard training.
4.2 DAWNBENCH IMPROVEMENTS
Although free adversarial training is of comparable cost per iteration to traditional standard training methods, it is not quite comparable in total cost to more recent advancements in fast methods for standard training. Notably, top submissions to the DAWNBench competition have shown that CI- FAR10 and ImageNet classifiers can be trained at significantly quicker times and at much lower cost than traditional training methods. Although some of the submissions can be quite unique in their approaches, we identify two generally applicable techniques which have a significant impact on the convergence rate and computational speed of standard training.
Cyclic learning rate Introduced by Smith (201)) for improving convergence and reducing the amount of tuning required when training networks, cyclic schedule for a learning rate can drasti- cally reduce the number of epochs required for training deep networks (Smith & Topin (0018). A simple cyclic learning rate schedules the learning rate linearly from zero, to a maximum learning rate, and back down to zero (examples can be found in Figure Using cyclic learning rate allows CIFAR10 10 architectures to converge to benchmark accuracies in tens of epochs instead of hundreds, and is crucial component of some of the top DAWNBench submissions.
Mlxeed-preeision arithmetic With newer GPU architectures coming with tensor cores specifically built for rapid half- half-precision calculations, using mixed prrecision arithmetic when training deep net- works also provide significant speedups for standard training (Micikevicius et al. 2017). This
6

---

Published as a conference paper at ICLR 2020
0.2
0.4
0.1
0.2
0.0
0.0
5
10
10
15
(a) CIFARIO
(b) ImageNet
Figure I: Cyclic learning rates used for FGSM adversarial training on CIFAR and ImageNet over epochs. The ImageNet cyclic schedule is decayed further by factor of 10 in the second and third phases.
Table 2: Robustness FGSM and PGD adversarial training on MNIST
Method
Standard accuracy
PGD (c = 0.1)
PGD (c 0.3)
Verified (c = 0.1)
PGD
99.20%
97.66%
89.90%
96.7%
FGSM
99.20%
97.53%
88.77%
96.8%
can drastically reduce the memory utilization, and when tensor cores are available, also reduce run- time. In some DAWNBench submissions, switching mixed- precision computations was key to achieving fast training while keeping costs low.
We adopt these techniques for use in adversarial training, which allows us drastically reduce the number of training epochs as well as the runtime on GPU infrastructure with tensor cores, while using modest amounts of computational resources. Notably, both of these improvements can be easily applied to existing implementations of adversarial training by adding a few lines of code with very little additional engineering effort, and are easily accessible by the general research community.
EXPERIMENTS
To demonstrate the effectiveness of FGSM adversarial training with fast training methods, we run number of experiments MNIST, CIFAR10 10, and ImageNet benchmarks. All CIFARIO experi- ments in this paper are run on single GeForce RTX 2080ti using the PreAct ResNet18 architecture, single machine with four GeForee RTX 2080tis using the
and all ImageNet experiments ResNet50 architecture (He et al. 2016). Repositories for reproducing all experiments and the cor- responding trained model weights available at htttps://gg http://tww.....coco.c./caaaa//// adversarial
com wtt.o////o.csssaaaaasseeeeeo
All experiments using FGSM adversarial training in this section carried out with random initial starting points and step size a 1.25c as described Section 4.1 All PGD adversaries used at evaluation
with 10 random restarts for 50 iterations (with the same hyperparameters those used by Shafahi et al. (2019) but further strengthened with random restarts). Speedup with mixed- precision was incorporated with the Apex amp package at the 01 optimization level for ImageNet experiments and 02 without loss scaling for CIFAR10 10 experiments
5.1 VERIFIED PERFORMANCE ON MNIST
Since the FGSM attack is known to be significantly weaker than the PGD attack, it is understand- able if the reader is still skeptical of the true robustness the models trained using this method. To demonstrate that FGSM adversarial training confers real robustness the model, in addition to evaluating against a PGD adversary, we leverage mixed-integer linear programming (MILP) meth- ods from formal verification to calculate the exact robustness of small, but verifiable models (Tjeng
Since CIFAR10 did not suffer from loss scaling problems, found using the 02 optimization level with- loss scaling for mixed-precision anthmetic slightly faster.

---

Published as a conference paper at ICLR 2020
80
80
40
5
15 25 35 45
15 25 35
45
8
32
56
80
(a) FGSM
(b) PGD
(c) "Free" (m 8)
Figure 2: Performance of models trained on CIFARIO at 8/255 with cyclic learning rates and half precision, given varying numbers of epochs across different adversarial training methods. Each point denotes the average model performance over 3 independent runs, where the axis denotes the number epochs N the model was trained for, and the axis denotes the resulting accuracy. The orange dots measure accuracy on natural images and the blue dots plot the empirical robust accuracy on adversarial images. The vertical dotted line indicates the minimum number of epochs needed to train model 45% robust accuracy.
Table 3: Time to train robust CIFARTO 10 classifier to 45% robust accuracy using various adversarial training methods with the DAWNBench techniques of cyclic learning rates and mixed- precci-ir arithmetic, showing significant speedups for all forms of adversarial training.
Method
Epochs
Seconds/epoch
Total time (minutes)
DAWNBench + PGD-7
10
104.94
17.49
DAWNBench + Free (m 8)
80
13.08
17.44
DAWNBench + FGSM
15
25.36
6.34
PGD-7 (Madry et 2017
205
1456.22
4965.71
Free (m 8) (Shafahi a. 2019
205
197.77
674.39
et al. 2017). We train two convolutional networks with 16 and 32 convolutional filters followed by fully connected layer of 100 units, the same architecture used by Tjeng et al. (2017). We use both PGD and FGSM adversarial training at = 0.3, where the PGD adversary for training has 40 itera- tions with step size 0.01 as done by Madry al. (2017). The exact verification results be seen in Table 2 where we find that FGSM adversarial training confers empirical and verified robustness which is nearly indistin to that of PGD adversarial training on MNIST
5.2
FAST CIFAR10
We begin our CIFARIO experiments by combining the DAWNBench improvements from Section cyclic learning rate that increases linearly from 0 to over the first N/2 epochs, then decreases linearly from À to 0 for the remaining epochs, where À is the maximum learning rate. For each method, individually tune to be as large as possible without causing the training loss to diverge, which is the recommended learning rate test from Smmihh Topin (2018).
4.2 with various forms of adversarial training. For N epochs,
To identify the minimum number of epochs needed for each adversarial training method, we repeat- edly run each method over range maximum epochs N, and then plot the final robustness each trained model in Figure 2 While all the adversarial training methods benefit greatly from the cyclic learning rate schedule, we find that both FGSM and PGD adversarial training require much fewer epochs than free adversarial training, and consequently reap the greatest speedups.
'Exact verification results at 0.3 for both the FGSM and PGD trained models are not possible since the size of the resulting MILP is too large to be solved in reasonable amount of time. The same issue also prevents from verifying networks trained datasets larger than MNIST, which have to rely on empirical tests for evaluating robustness.
8

---

Published as a conference paper at ICLR 2020
Table Imagenet classifiers trained with adversarial training methods e 2/255 and 4/255.
Method
Standard acc.
PGD+1 restart
PGD+10 restarts
Total time (hrs)
FGSM
2/255
60.90%
43.46%
43.43%
12. 14
Free (m 4)
2/255
64.37%
43.31%
43.28%
52.20
FGSM
4/255
55.45%
30.28%
30.18%
12.14 52.20
Free (m -
4/255
60.42%
31.22%
31.08%
Table 5:
Time to train robust ImageNet classifier using various fast adversarial training methods
Method
Precision
Epochs
Min/epoch
Total time (hrs)
FGSM (phase 1)
single
22.65
2.27
FGSM (phase 2)
single
65.97
6.60
FGSM (phase 3)
single
1 114.45
5.72
FGSM
single
15
14.59
Free (m 4)
single
92
34.04
52.20
FGSM (phase 1)
mixed
20.07
2.01
FGSM (phase 2)
mixed
53.39
5.34
FGSM (phase 3)
mixed
95.93
4.80
FGSM
mixed
15
12.14
Free (m = 4)
mixed
92
25.28
38.76
Using the minimum number of epochs needed for each training method reach a baseline of 45% robust accuracy, we report the total training time in Table 3 We find that while all adversarial train- ing methods benefit from the DAWNBench improvements, FGSM adversarial training is the fastest, capable of learning robust CIFAR10 classifier in 6 minutes using only 15 epochs. Interestingly, we also find that PGD and free adversarial training take comparable amounts of time, largely because free adversarial training does not benefit from the cyclic learning rate as much as PGD FGSM adversarial training.
5.3 FAST IMAGENET
Finally, we apply all of the same techniques (FGSM adversarial training, mixed-precision, and cyclic learning rate) on the ImageNet benchmark. In addition, the top submissions from the DAWNBench competition for ImageNet utilize two more improvements on top of this, the first of which is the removal of weight decay regularization from batch normalization layers. The second addition is to progressively resize images during training, starting with larger batches of smaller images in the beginning and moving on to smaller batches of larger images later. Specifically, training is divided into three phases, where phases I and 2 use images resized to 160 and 352 pixels respectively. and 2/255 and 4/255 and
phase 3 uses the entire image. We train models to be robust compare to free adversarial training in Table showing similar levels of robustness. In addition to using ten restarts, also report the PGD accuracy with one restart to reproduce the evaluation done by Shafahi et al. (2019).
RRntimms calculated hardware using the publicly available training code https://github. com/ wttp///allllaaaaaaaaaaaaaa 1_ Tengel
"Rnttimes calculated our hardware using the publicly available training code httsss://githh... com htmm/aahaaaatt/eeeeaa..

---

Published as a conference paper at ICLR 2020
With these techniques, we can train an ImageNet classifier using 15 epochs in 12 hours using FGSM adversarial training, taking a fraction of the cost of free adversarial training as shown in Table We compare to the best performing variation of free adversarial training which which uses minibatch replays over 92 epochs of training (scaled down accordingly to 23 passes over the data). Note that free adversarial training can also be enhanced with mixed-precision arithmetic, which reduces the runtime by 25%, but is still slower than FGSM-based training. Directly combining free adversarial training with the other fast techniques used in FGSM adversarial training for ImageNet results in reduced performance which we describe in Appendix F
5.4 CATASTROPHIC OVERFITTING
While FGSM adversarial training works in the context of this paper, many other researchers have tried and failed to have FGSM adversarial training work. In addition to using a zero initialization or too large of step size or numbers of epochs) for the training procedure can also make it more likely for FGSM adversarial training fail. However, all of these failure modes result in what we call "catastrophic overfitting", where the robust accuracy with respect to a PGD adversarial suddenly and drastically drops to 0% (on the training data). Due to the rapid deterioration of robust performance, these alternative versions of FGSM adversarial training can be salvaged to some degree with a simple carly-stopping scheme by measuring PGD accuracy on small minibatch of training data, and the recovered results for Catastrophic overfitting and the early-stopping
in Table other design decisions (like specific learning rate schedules
some of these failure modes are shown in Table [ scheme are discussed in more detail in Appendix
5.5 TAKEAWAYS FROM FGSM ADVERSARIAL TRAINING
While it may be surprising that FGSM adversarial training can result in robustness to full PGD ad- versarial attacks, this work highlights some empirical hypotheses and takeaways which we describe below.
1. Adversarial examples need to span the entire threat model. One of the reasons why FGSM and R+FGSM as done by Tramer et al.) (2017) may have failed is due to the restricted nature of the generated examples: the restricted (or lack of) initialization results in perturbations which perturb each dimension by either 0 or the, and so adversarial examples with feature perturbations in between are never seen. This is discussed further in Appendix D
2. Defenders don't need strong adversaries during training. This work suggests that rough approximations to the inner optimization problem are sufficient for adversarial training. This is in contrast to the usage of strong adversaries at evaluation time, where it is standard practice to use multiple restarts and a large number of PGD steps.
6 CONCLUSION
Our findings show that FGSM adversarial training, when used with random initialization, can in fact be just as effective as the more costly PGD adversarial training. While a single iteration of FGSM adversarial training double the cost of free adversarial training. it converges significantly faster, especially with cyclic learning rate schedule. As result, we are able to learn adversarially robust classifiers for CIFAR 10 minutes and for ImageNet in hours, even faster than free adversarial train- ing but with comparable levels of robustness. We believe that leveraging these significant reductions in time to train robust models will allow future work to iterate even faster, and accelerate research in learning models which are resistant to adversarial attacks. By demonstrating that extremely weak adversarial training is capable of learning robust models, this work also exposes a new potential di- rection in more rigorously explaining when approximate solutions to the inner optimization problem are sufficient for robust optimization, and when they fail.
We use the implementation of free adversarial training for ImageNet publicly available at https: 7/gi thub. ccm hamm//mayaraaaaaa jibi / FreeAdverss saa saria a allraaim ngj and reran it the our machines to account for any timing discrepancies due to differences in hardware
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. arXiv preprint arXiv:1707.0739) 2017.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv:1802. 1802.004200 2018.
Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: way to resist adversarial examples. 2018.
One hot
Nicholas Carlini. Is ami (attacks meet interpretability) robust to adversarial examples? arXiv preprint arXiv:1902.024: 2019.
Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples.
arXiv preprint arXiv:1607.000 2016.
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14. ACM, 2017a.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017b.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06700 06705, 2019.
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv: /902. 19,,,, 2019.
Cody Coleman, Deepak Narayanan, Daniel Kang, Tian Zhao, Jian Zhang, Luigi Nardi, Peter Bailis, Kunle Olukotun, Chris Ré, and Matei Zaharia. Dawnbench: An end-to-end deep learning bench- mark and competition. Training, 100(101):102, 2017.
Francesco Croce, Maksym Andriushchenko, and Matthias Hein. Provable robustness of relu net- works via maximization of linear regions. arXiv preprint arXiv:1810.07488. 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193 2018.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.1022 2018.
Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00411 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Ue- sato, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprini arXiv: 1810.1215, 2018.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv: arXiv:1803.065 1803.06373, 2018
11

---

Published as a conference paper at ICLR 2020
Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pp. 97-117. Springer. 2017.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprini arXiv: 1607.v/600.02555 00233,, 2016.
Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth. No need to worry about adversarial examples in object detection in autonomous vehicles. arXiv preprint arXiv: 1707.0350000 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:/706.06088. 2017.
Pratyush Maini, Eric Wong, and Zico Kolter. Adversarial robustness against the union of multiple perturbation models. arXiv preprint arXiv:1909.04400 2019.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. arXiv preprint arXiv: /702.04267 2017.
Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg. Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint urXiv:/710.000
2017.
Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for prov- ably robust neural networks. In International Conference on Machine Learning, pp. 3575-3583, 2018.
Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, and Dietrich Klakow. Logit pairing methods can fool gradient-based attacks. arXiv preprint a8X.v:/810.1 12042, 2018.
Nicolas Papernot, Patrick McDanicl, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense adversarial perturbations against deep neural networks. In 2016 IEEE Symposium Security and Privacy (SP), pp. 582-597. IEEE, 2016.
Aditi Raghunathan, Jacob Steinhardt, and Percy S Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Advances in Neural Information Processing Systems, pp. 10877-10887, 2018
Hadi Salman, Greg Yang. Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Sebastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. arXiv preprint arXiv:1906.0458. 2019.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! arXiv preprint arXiv:1904. /2843, 2019.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv: 1710. 10571, 2017.
Leslie N Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Confer- ence on Applications of Computer Vision (WACV), pp. 4422. IEEE, 2017.
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of residual networks using large learning rates. 2018.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv: 1ot0.11100.06 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312,6/99 2013.
12

---

Published as a conference paper at ICLR 2020
Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretability: Attribute-stecred detection of adversarial samples. In Advances in Neural Information Processing Systems, pp. 7717-7728, 2018.
Vincent Tjeng, Kai Xiao, and Russ Tedrake. Evaluating robustness of neural networks with mixed integer programming. arXiv preprint arXiv:1711.07356. 2017.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. arXiv preprint arXiv: /3000, 2019.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, lan Goodfellow, Dan Bonch, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv: 2017.
Jonathan Uesato, Brendan O' Donoghue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. arXiv preprint ar2iv:1802.05666, 2018.
Jianyu Wang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. arXiv preprint 181 2018.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.00851, 2017.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems, pp. 8400-8409, 2018.
Kai Y Xiao, Vincent Tjeng. Nur Muhammad Shafiullah, and Aleksander Madry. Training for faster adversarial robustness verification via inducing relu stability. arXiv preprint woXiv:/809.03000 2018.
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. arXiv preprint arXiv: arXiv:/905. 1971, 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Painless adversarial training using maximal principle. arXiv preprint arXiv:1905.000 2019.
13

---

Published as a conference paper at ICLR 2020
Table 6: Ablation study showing the performance of R+FGSM from Tramèr et (2017) and the various changes for the version of FGSM adversarial training done in this paper, over 10 random seeds.
Method
Step size
Initialization
Robust accuracy
R+FGSM (Tramèr al. 2017)
0.15
Hypercube(0.15)
34.58 + 36.06%
R+FGSM (+full step sıze)
0.30
Hypercube(0.155
26.53 + 32.48%
R+FGSM (+uniform init.)
0.15
Uniform(0.3)
72.92 + 10.40%
Uniform full (ours)
0.30
Uniform(0.3)
86.21 = 00.75%
Table Training parameters used for the DAWNBench experiments of Table
Parameter
FGSM
PGD
Free
Epochs
30
40
96
Max learning rate
0.2
0.2
0.04
A A DIRECT COMPARISON R+FGSM FROM TRAMÈR ET AL.. (2017)
While randomized version of FGSM adversarial training proposed by Tramèr et al. (2017), was not shown to be as effective as adversarial training against a PGD adversary. Here, we note the two main differences between our approach and that of Tramer al. (2017).
1. The random initialization used different. For a data point r, we initialize with the uniform distribution in the entire perturbation region with
Uniform(cc,e).
In comparison, Tramer et al. (2017) instead initialize on the surface hypercube with radius e/2 with
a' = x Normal(O, 1).
2. The step sizes used for the FGSM step different. We use full step size of c, whereas Tramèr et al. (2017) use a step size a = e/2.
To study the effect of these two differences, we run all combinations of either initialization with either step size on MNIST. The results are summarized in Table 6
We find that using a uniform initialization adds the greatest marginal improvement to the original R+FGSM attack, while using full step size doesn't seem to help on its own. Implementing both of these improvements results in the form of FGSM adversarial training presented in this paper. Additionally, note that R+FGSM as done by Tramer et al. (2017) has high variance in robust perfor- mance when done over multiple random seeds, whereas our version of FGSM adversarial training is significantly more consistent and has very low standard deviation over random seeds.
B TRAINING PARAMETERS FOR TABLE
For all methods, we use batch size of 128, and SGD optimizer with momentum 0.9 and weight decay * 10-4, We report the average results over random seeds. The remaining parameters for learning rate schedules and number of epochs for the DAWNBench experiments in Table 7 For runs using carly-stopping we use 5-step PGD adversary with I restart on training minibatch to detect overfitting the FGSM adversaries, as described in more detail in Section
14

---

Published as a conference paper at ICLR 2020
50
% PGD Accuracy
30
20
10
0
2
10
12
14
16
Step Size (x/255)
Figure 3: Robust test performance of FGSM adversarial training over different step sizes for c = 8/255.
6
PGD Test
100
FGSM Train
75
4
% Error
Loss
50
2
25
PGSM Tramn
0
Epochs
Epochs
Figure 4: Learning curves for FGSM adversarial training plotting the training loss and error rates incurred by an FGSM and PGD adversary when trained with zero-initialization FGSM at 8/255, depicting the catastrophic overfitting where PGD performance suddenly degrades while the model overfits to the FGSM performance.
C OPTIMAL STEP SIZE FOR FGSM ADVERSARIAL TRAINING
Here, test the effect of step size the performance of FGSM adversarial training. We plot the mean and standard error of the robust accuracy for models trained for 30 epochs over 3 random seeds in Figure and vary the step size from 1/555 to a 16/255.
We find that we get increasing robust performance as we increase the step size up to a = 10/255. Beyond this, we see no further benefit, find that the model prone overfitting the adversarial examples, since the large step size forces the model to overfit to the boundary of the perturbation region.
D CATASTROPHIC OVERFITTING AND THE EFFECT OF EARLY STOPPING
While the main experiments in this paper work is (with the cyclic learning rate and FGSM ad- versarial training with uniform random initialization), many of the variations of FGSM adversarial training which have been found span of couple epochs) appear to overfit to the FGSM adversarial examples. What previously a reasonably robust model will quickly transform into a non-robust model which suffers 0% robust accuracy (with respect ting, can be seen in Figure which plots the learning curves for standard, vanilla FGSM adversarial training from zero-initialle pero-iniiialization.
succeed fail similarly: the model will very rapidly (over the
PGD adversary). This phenomenon, which call catastrophic overfit-
Indeed, one of the reasons for this failure may lie in the lack of diversity in adversarial examples generated by these FGSM adversaries. For example, using zero initialization using the random
15

---

Published as a conference paper at ICLR 2020
0.5
Fraction of perturbations
Robust model
0.4
Catastrophically overfitted model
0.3
0.2
0.1
1)
2)
3)
4
5)
61
71
17. 8)
[0.
[1.
[2
13,
[4.
15
16
Absolute value of perturbations
Figure 5: Histogram of the resulting perturbations from PGD adversary for each feature for
successfully trained robust CIFARIO model and a catastrophically overfitted CIFAR1O model.
50
40
% PGD Accuracy
30
20
0
2
10
12
14
16
Step (x/255)
Figure 6: Robust test performance of FGSM adversarial training over different step sizes for c 8/255 with early stopping to avoid catastrophic overfitting.
initialization scheme from Tramer al. (2017) will result in adversarial examples whose features have been perturbed by 0,e). and so the network learns a decision boundary which is robust only these perturbation values. This be verified running PGD adversarial attack models which have catastrophically overfitted, where the perturbations tend to be more in between the origin and the boundary of the threat model (relative to non-overfitted model, which tends to have perturbations near the boundary), as seen in Figure 5
These failure modes, including the other failure modes discussed Section 5.4 tected by evaluating the PGD performance on small subset of the training data, as the catastrophic failure will result in 0% robust accuracy for PGD adversary on the training In practice, we find that this can be simple as single minibatch with 5-step PGD adversary, which can be quickly checked at the end of the epoch. If robust accuracy with respect this adversary suddenly drops, then we have catastrophic overfitting. Using PGD adversary training minibatch to detect catastrophic overfitting, we can early stop to avoid catastrophic overfitting and achieve a reasonable amount of robust performance.
be easily de-
For a concrete example, recall from Section that step sizes larger than 11/255 result in 0% robust accuracy, due to this catastrophic overfitting phenomenon. By using early stopping to catch the model at its peak performance before overfitting, FGSM adversarial training with larger step sizes actually achieve some degree of robust accuracy, shown in Figure
16

---

Published as a conference paper at ICLR 2020
Table Training parameters used for Figure
Parameter
FGSM
PGD
Free
Max learning rate
0.2
0.2
0.04
Table 9; ImageNet classifiers trained with free adversarial training methods at m = 3 minibatch replay when augmented with DAWNBench optimizations, against lo perturbations of radius e 4/ 4/255, where 30 epochs of free training is equivalent to 15 epochs of FGSM training
Method
Step size
Epochs
Standard acc.
PGD+1
PGD+10
Bree
4/255
15
49.87%
22.78%
22.18%
Free+DAWNBench
5/255
15
50.48%
22.88%
22.25%
Free-DAWNBench
4/255
30
49.87%
28.17%
27.08%
Free+DAWNBench
5/255
30
50.48%
28.73%
27.81%
Free (m 4)
4/255
92
60.42%
31.22%
31.08%
FGSM
5/255
15
55.45%
30.28%
30.18%
E TRAINING PARAMETERS FOR FIGURE 2
For all methods, decay 5 * 10-4, We report the average results over 3 random seeds. Maximum learning rates used for the eyclic learning rate schedule are shown in Table
batch size of 128, and SGD optimizer with momentum 0.9 and weight
COMBINING FREE ADVERSARIAL TRAINING WITH DAWNBENCH IMPROVEMENTS ON IMAGENET
While adding mixed- -eeeciiieciiionn is direct speedup to free adversarial training without hurting perfor- mance, using other optimization tricks such as the cyclic learning rate schedule, progressive resizing, and batch-norm regularization may affect the final performance of free adversarial training. Since ImageNet is too large to run comprehensive search over the various parameters as was done for CIFARIO in Table 3 we instead test the performance of free adversarial training when used drop-in replacement for FGSM adversarial training with all the same optimizations used for FGSM adversarial training. We use free adversarial training with m 3 minibatch-replay with 2 epochs for phase one, 2 epochs for phase two, and 1 epoch for phase three be equivalent to 15 epochs of standard training. PGI+N denotes the accuracy under PGD adversary with N restarts.
word of caution: this not to claim that free adversarial training is completely incompatible ImageNet. By giving free adversarial training may be possible recover the same or better performance. However, tuning the DAWNBench techniques be optimal for free adversarial training is merely to show what happens if we naively apply the same DAWNBench tricks used for FGSM adversarial training to free adversarial training. Since free adversarial training requires more epochs even when tuned with DAWNBench improvements for CIFAR 10, we suspect that the same behavior occurs here for ImageNet, and so 15 epochs is likely not enough to obtain top performance for free adversarial training. Since one epoch of FGSM adversarial training is equivalent to two epochs of free training, a fairer comparison is to give free adversarial training epochs instead of Even with double the epochs (and thus the same compute time as FGSM adversarial training), we find that it gets closer but doesn't quite recover the original performance of free adversarial training.
with the DAWNBench optimizations
epochs,
the objective of this paper, and this
17


Reviews:
[{"review_id": "Hyxi6-9rYS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper revisits Random+FGSM method to train robust models against strong PGD evasion attacks. Coupled together with tricks for accelerating natural training, such as cyclic learning rate, mixed precision, the robust models can be trained faster than previous methods. \n\n+The experimental results are impressive. The trained model is robust (at Madry’s PGD level), and the total training procedure is fast (6 min for CIFAR-10 and 12 hr for ImageNet).\n\n+ The method is simple, and I guess reproducible. \n\n+The paper shows surprising facts of a well-known method.\n\n+The paper is generally well-written and easy to follow.\n\nI do have some concerns of the work\n- The paper is empirical and the techniques are combinations of previous methods. Even for the surprising fact that Random+FGSM, it has been discussed in several previous papers, for example,  ICLR 2019 Defensive Quantization: When Efficiency Meets Robustness  https://openreview.net/forum?id=ryetZ20ctX. So the main contribution of the paper is limited to show RFGSM works well when combined with optimization tricks like cyclic learning rate. \n\n-In previous methods claiming random+FGSM can train robust model, their method seems to be slightly different from Alg 3 in page 4 of this paper. The alg in this paper seems to be identical to Madry’s implementation of R-FGSM, which is shown not robust to PGD attacks. See discussions in https://openreview.net/forum?id=rJzIBfZAb and https://openreview.net/forum?id=ryetZ20ctX. I would like the authors to clarify their method to resolve such conflicts and make it clear how R-FGSM can be as robust as PGD as in table 1. \n\n-The first two paragraphs of section 4.1 seem to be inaccurate. One important trick in the “adversarial training for free” paper is to replay each minibatch m times. It is hard to say how much nonzero initialization helps. According to “universal adversarial training” (https://arxiv.org/pdf/1811.11304.pdf). It may help, but cannot compete with Madry’s PGD training when defending against PGD attacks. \n\n-I am not sure if using a larger norm 1.25 * \\epsilon is a fair comparison. A baseline of PGD training bounded by 1.25 * \\epsilon would help. \n\n-Could the authors combine table 4 and 5 for easy comparison of robust accuracy and training time? Did the authors try the optimization tricks on ImageNet for the baseline free adversarial training method?\n\n\n\n================== after rebuttal =================\nI change my rating to weak accept. I tend to accept for the following reason\n(1) there seems to be no obvious flaw in the authors implementation. I quickly skimmed their code, and looks like a few researchers have tried their code and responded in public discussion. The surprising robustness of RFGSM, though the originality is questionable and the technical difference comparing to previous methods are subtle, seems to hold true. \n(2) The authors work hard to address the comments. \nI still have some concerns, mainly regarding the fairness of experimental comparison. \n(1) As pointed out in public discussion, the success of the proposed RFGSM relies on early stopping. I am wondering if early stopping also helps other methods since it turns out to be some sort of selection procedure.\n(2) The authors did not update time in table 1 for CIFAR-10 results, which I consider almost no extra efforts. I am wondering how much more time each method needs from 45% in figure 2 to higher robust accuracy in table 1.\n(3) I cannot understand why the proposed method is a particular good fit with cyclic LR and low precision tricks comparing to other methods. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HygOpABTFS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors claimed a classic adversarial training method, FGSM with random start, can indeed train a model that is robust to strong PGD attacks. Moreover, when it is combined with some fast  training methods, such as cyclic learning rate scheduling and mixed precision, the adversarial training time can be significantly decreased. The experiment verifies the authors' claim convincingly.\nOverall, the paper provides a novel finding that could significantly change the adversarial training strategy. The paper is clearly written and easy to follow. I recommend the acceptance.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "ByeiH0AoFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main claim of this paper is that a simple strategy of randomization plus fast gradient sign method (FGSM) adversarial training yields robust neural networks. This is somewhat surprising because previous works indicate that FGSM is not a powerful attack compared to iterative versions of it like projected gradient descent (PGD), and it has not been shown before that models trained on FGSM can defend against PGD attacks. Judging from the results in the paper alone, there are some issues with the experiment results that could be due to bugs or other unexplained experiment settings. \n\nThe most alarming part of the results is the catastrophic failure with larger step sizes 16/255 for CIFAR10 in Table 1. This is very strange because the method works well when using epsilon=10/255 to defend against an adversary with epsilon=8/255. \nThe authors explain this with overfitting, but this is not satisfactory. Suppose I want to use the method to defend against an adversary with power epsilon=14/255, then it is conceivable that I would use a slightly larger step size, say 16/255, as suggested by the authors.  The results in the table tells me that this method will fail completely, because it cannot defend against epsilon=8/255, let alone the target perturbation 14/255. The method is probably not failing completely, because it does have good accuracy on clean data and does learn something. So it cannot be due to the model not having enough capacity to learn against an epsilon=16/255 adversary. \n\nThe authors should check some potential issues with the experiments: \n1. Is there any label leakage in the FGSM training? \n2. The pseudo-code does not contain any projection onto the feasible set; the authors should check it. \n\nSince the claim of this paper is somewhat unexpected given previous works on defending against adversaries, the experiment results have to be very solid. With these issues with the experiments I don't believe the current paper is ready for publication yet. \n\nAfter rebuttal: \nThe authors' new experiments and response answers most of my concerns. \n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper provides a surprising result: that randomization and FGSM can produce robust models faster than previous methods given the right mix of cyclic learning rate, mixed precision, etc. This paper produced a fair bit of controversy among both the community and the reviewers to the point where there were suggestions of bugs, evaluation problems, and other issues leading to the results. In the end, the authors released the code (and made significant updates to the paper based on all the feedback). Multiple reviewers checked the code and were happy. There was an extensive author response, and all the reviewers indicated that their primary concerns were address, save concerns about the sensitivity of step-size and the impact of early stopping.

Overall, the paper is well written and clear. The proposed approach is simple and well explained. The result is certainly interesting, and this paper will continue to generate fruitful debate. There are still things to address to improve the paper, listed above. I strongly encourage the authors to continue to improve the work and make a more concerted effort to carefully discuss the impacts of early stopping.

Decision: Accept (Poster)