Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors claimed in this paper that as the most empirically successful approach to defending adversarial examples, PGD-based adversarial training, is computationally inefficient. Fast adversarial training could mitigate this issue by training a model using FGSM attacks initialized with large randomized perturbations, but the underlying reason for its success remains unclear and it may still suffer from catastrophic overfitting. The authors conducted a series of experiments to figure out the key to the success and properties of fast adversarial training. The experimental results showed that fast adversarial training cannot avoid catastrophic overfitting, but could be able to recover from catastrophic overfitting quickly. Based on all of the observations, the authors proposed a simple method to improve fast adversarial training by using PGD attack as training instead of R+FGSM attack (proposed in fast adversarial training) when overfitting happens, or using fast adversarial training as a warmup. The proposed methods could achieve slightly better performance than the current state-of-art approach while reducing the training time significantly.

#####################################################################

Overall, I vote for weak reject (marginally). I like the idea of exploring the properties of adversarial training, the experiments may also be inspiring. But my major concern is that the interpretation about the ‘catastrophic overfitting’ is not clear, and the interpretation about the effectiveness of R-FGSM and PGD against overfitting is also not clear. Hopefully, the authors can address my concern in the rebuttal period. 

Pros:
#####################################################################Pros: 
 
1.Attempting to interpret the successful reason for a previous work is interesting. And the exploratory experiments may be inspiring for other researchers.

2. Overall, the paper was well written. All the motivations and conjectures are easy to follow and understand. 

3. This paper provides a lot of experiments to show the effectiveness of the proposed methods which appeared slightly better than the SOTA PGD-training while reducing training time significantly.
#####################################################################


Cons: 
 
1. Although the authors attempted to explain the key to the success of fast adversarial training, it might be still not clear theoretically: 

(1)	Why R-FGSM and PGD could guide the model to recovery from ‘catastrophic overfitting’, but FGSM could not? Does it mean that stronger attacks could guide the model to recovery?

(2) Why the ‘catastrophic overfitting’ happened a lot of times when using FGSM training, but R-FGSM and PGD could mitigate it? Does it mean that stronger attacks could mitigate it?

2. As I understand, Figure 3(c) should be the result of proposed FastAdv+. From Figure 3(c), it can be observed that there are ‘catastrophic overfitting’ in FastAdv+, but this phenomenon could not be seen from Figure 4. Do you have any idea to explain it?

3. As concerned in my 1.(1) and (2), if weaker attacks lead to more ‘catastrophic overfitting’ and could not guide the model to recovery, why the FastAdvW 4-8 using a weaker attack as a warmup could outperform FastAdv+ and FastAdvW 8-8.

4. Though the proposed methods appear useful, they may be a bit straightforward and have a limited novelty (using PGD attacked samples when ‘catastrophic overfitting’ happens)

review_title: A useful paper needing more theoretical interpretations

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper proposes a method called: improved fast adversarial training (FastAdv+) which improves fast adversarial training by replacing randomized initialization with PGD adversarial training when there is overfitting issue during training.

Strengths:
The idea is reasonable and easy to implement. There is a marginal improvement of accuracy under PGD attack ($\epsilon=8/255$) of around 2%, comparing Fast adversarial training (FastAdv) with improved fast adversarial training (FastAdv+) on CIFAR10. 

Weakness:
1. FastAdv+ itself does not perform better than PGD adversarial training. When use the FastAdv+ result as starting point then do PGD adversarial training, it outperforms PGD adversarial training. Therefore, FastAdvW (the combined method) is actually more computational expensive than PGD adversarial training. 
2. The advantage of Fast adversarial training is that it is less computational expensive than PGD adversarial training so it can scale up to large dataset like ImageNet. However, FastAdvW does not have this advantage anymore.
3. The improvement of FastAdv+ over FastAdv on CIFAR100 and TinyImagenet is marginal.
4. The methods are evaluated under one attack strength ($\epsilon=8/255$ for CIFAR10). It is better to evaluate the methods under different attack strengths.

Clarity:
The paper is clearly written and easy to follow.

Reproducibility:
Details of the algorithm is provided but code is not.

Conclusion:
The method is novel but the contribution is not strong.
review_title: Improving PGD adversarial training with FGSM adversarial training as warmup (mitigate the overfitting issue with PGD)

final_score: rating: 7: Good paper, accept
review_content: ################################## Summary ##################################

This paper shows that the main reason for the success of Fast Adversarial Training ([1], will be referred to as FBF in this review) is its ability to recover from catastrophic overfitting. Based on this observation, the authors propose to utilize PGD multi-step training for a few iterations (when catastrophic overfitting occurs), and resume single-step training after the model recovers. Further, the authors also propose to use this improved Fast Adversarial Training (FastAdv+) as a warmup for PGD Adversarial Training, and demonstrate improved performance over PGD-AT, at a significantly lower computational cost. 

[1] Wong et al. Fast is Better than Free: Revisiting Adversarial Training, ICLR 2020

################################### Pros #####################################
  
  -  The authors present a very interesting finding, that FBF has catastrophic overfitting for a few intermediate iterations, and recovers very quickly from this. 
  -  Based on this observation, they propose to use PGD based training for a few intermediate iterations, which seems to prevent this overfitting effectively 
  -  The authors also propose the use of this training as warm up for PGD training, and demonstrate significantly improved results. 
  -  This is certainly a significant contribution of the paper since it achieves improved results at a much lower computational cost. 

################################### Cons #####################################

  -  Since the proposed defenses involve single-step training, the absence of gradient masking needs to be justified using thorough validation as discussed by Carlini et al. [1], using gradient-free attacks such as Square attack/ SPSA, black-box transfer based attacks, attacks with multiple steps and multiple random restarts. Also, the sanity checks proposed by Athalye et al. [2] need to be demonstrated. 
  -  The paper discusses results only on PGD attack, which is not the current state-of-the-art. The proposed defenses (FastAdv+, FastAdvW) must be evaluated on stronger attacks such as AutoAttack [3] and MultiTargeted Attack [4]. 
  -  While the use of the modified FGSM attack would improve the efficiency of PGD training, it is not clear why it should lead to improved robustness.  
  -  Could the authors clarify if all the other training/ hyperparameter settings (such as batch size, weight decay, optimizer, initial learning rate and schedule, number of epochs, initial random noise added for the attack, validation split, use of early stopping, batch norm in train/ eval mode during attack generation [5]) are similar for PGD-AT (reported in Table-4 of the Appendix) and FastAdvW (reported in Table-1)? 
  -  The learning rate schedule used for PGD-AT (Sec. 3.2) is different from that used by Rice et al. So, the results could be sub-optimal. Could the authors use similar settings as Rice et al. (SGD optimizer using a batch size of 128, a step-wise learning rate decay set initially at 0.1 and divided by 10 at epochs 100 and 150, total epochs 200 and weight decay 5e-4) for reporting the PGD baseline results? 
 
[1] Carlini et al., On evaluating Adversarial Robustness, https://arxiv.org/abs/1902.06705

[2] Athalye et al., Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018

[3] Croce et al., Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks, ICML 2020

[4] Gowal et al., An Alternative Surrogate Loss for PGD-based Adversarial Testing, https://arxiv.org/pdf/1910.09338.pdf

[5] Bag of Tricks for Adversarial Training, https://openreview.net/forum?id=Xb8xvrtB8Ce

############################## Reasons for score #################################
 
The paper highlights a very interesting finding in FBF training, proposes a single-step defense, and also an improvement to speed-up PGD-AT. However, it does not show sufficient experimental results for reliable evaluation of the defenses and to ensure the absence of gradient masking. Hence I think the paper is marginally below the acceptance threshold. I will be happy to increase the score if the required experimental results are presented during the rebuttal. 

######################### Questions during rebuttal period ###########################

  -  Could the authors provide the following results (for CIFAR-10) for both the proposed defenses (FastAdv+, FastAdvW). 
        *  Evaluation against AutoAttack [3] and MultiTargeted Attack [4]. (It would help to also report the corresponding results for the baselines - FBF, PGD-AT)
        *  Plot of robust accuracy vs. attack distortion bound (epsilon for PGD-10 step attack) as discussed by Athalye et al. [2] 
        *  Accuracy on black box transfer based attacks (using normally trained model of the same architecture as a source)
  -  Could the authors also report PGD-10 step accuracy for PGD-AT and FastAdvW for CIFAR-10 dataset on PreActResNet-18 and WRN-34-10? This would help with comparison of baselines against those reported in prior work.
  -  In the proposed defense FastAdv+, could the authors clarify what is the fraction of validation split used for detecting catastrophic overfitting and what is the number of steps used for the PGD attack on the validation set? Is the validation time included in the time reported in Table-1? If not, could the authors report the total time including validation?
  -  Could the authors clarify the size of the validation split used for early stopping (also in Fig.4) and the number of steps used for the PGD attack for early stopping? Is this consistent across all experiments? 
  -  (good to have) Loss surface plots for the proposed defenses to show the absence of gradient masking. (similar to those reported in [5])

################## Additional Feedback (not part of decision assessment) ###################

  -  The authors mention the following: “although the model quickly transforms into a non-robust one, it is fundamentally different from an ordinary non-robust model”. It would be insightful to study the properties of this intermediate model that suffers from catastrophic overfitting, to understand more about why it is able to recover so quickly. 
  -  The authors could visualize the loss surface of the FBF trained model during the catastrophic overfitting, and immediately after it recovers. This can lead to insightful findings on what is happening in the vicinity of the data samples in very few iterations. The loss surface can be plotted similar to Fig.1 in [5]. 

[5] Tramer et al., Ensemble Adversarial Training: Attacks and Defenses, ICLR 2018

############################# Update after rebuttal ###############################

The authors response addresses my concerns and I would like to update my score to 7. The paper presents insightful findings about FBF, and proposes a simple and effective method for stabilizing single-step adversarial training. This is useful not only for FBF but also for other single-step defenses. Although the gain in robustness is marginal, stabilizing single-step training is useful. The authors also propose a computationally efficient method of achieving robustness similar to PGD training. 

review_title: Interesting paper, needs more evaluation

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes an improvement over fast adversarial training to improve the robustness of the model in an efficient manner. Overall it's a well written paper but it can be improved in certain ways as follows...

- Fig. 3 lacks important information about what specific attack was used to compute the robust accuracy. was it PGD? If yes, what are the PGD parameters?
- It's not clear if the improvements are due to few iterations of PGD or due to piecewise linear learning rate regime? An explicit experiments comparing the two learning rate regimes (cyclic and piecewise) would be good to confirm. 
- For section 5, when authors are using FGSM as a warm-up, they should refer/compare with https://arxiv.org/pdf/2002.04237.pdf where clean training is used as warm-up. Author mention that strength of the adversary doesn't matter in the initial phase of training, so should compare results with varying strength going to zero. 
- In most of the experiments, the PGD step size is missing. Moreover, it will be good to see the robustness of this technique against varying PGD step size. 
- I think authors use R+FGSM and fast adversarial training interchangeably which creates some confusion.. it will be nice to be consistent with the terminology. 
- 
review_title: A good paper but needs improvements 

Meta-review: This paper first investigates the behavior (e.g., catastrophic overfitting) of fast adversarial training (FastAdv) through experiments. It finds that the key to its success is the ability to recover from overfitting to weak attacks. Then, it presents a simple fix (FastAdv+) that incorporates PGD adversarial training when catastrophic overfitting is observed. The resulting method is shown to be able to train for a large number of epochs. It also presents a version (FastAdvW) that use the improved fast adversarial training as a warmup of PDG-adversarial training, similar as in previous work. Overall, the analysis is useful and the ideas are valid. The empirical results also show promise. However, the main weakness of such empirical analysis is that it may be sensitive to the settings (e.g., # of epochs, splitting of datasets, …). The authors’ rebuttal also reflected such potential concerns.
Decision: Reject