Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper focus on the catastrophic overfitting issue in FGSM adversarial training. The work proposes to avoid clipping adversarial perturbations to mitigate the CO and they regard adding noise as a data augmentation method. This paper analyze the proposed tricks in detail and provide a huge of experimental results to prove them.  
strengths_and_weaknesses: Strengths:
- This work is easy to follow.  And authors use clear math formulas to show their methods. Their experimental results look good.
- They regard the random initialization  as a data augmentation trick which can help explain many problems.

Weakness:
- Although the authors provide some theoretical proofs and the loss landscapes,  the proof and figures are more about the influence brought by the noise augmentation. They dose not show why the larger noise magnitude can prevent the CO better. 

questions: - Do you provide the comparisons with other single-step methods against AutoAttack?
- Do you consider to compare with this single-step AT method[1]?

Citations:
[1] Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses
limitations: See above
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper finds that using a stronger noise around the clean images combined with not clipping can prevent catastrophic overftting and propose Noise-FGSM (N-FGSM) to improve model robustness. Evaluations on some datasets demonstrate that the proposed method outperform existing methods.
strengths_and_weaknesses: The novelty in my opinion is limited.  Using a larger noise combined with not clipping to conduct fast adversarial training is unfair to other  fast adversarial training methods. The authors should compare the proposed method with other fast adversarial training with  a larger noise combined with not clipping. Moreover, the training epoch is very small. Some research [1][2][3] have found that when the epoch of  fast adversarial training is set to larger, the fast adversarial training can also meet catastrophic overftting, such as 100, 200.

[1] Andriushchenko, M., Flammarion, N.: Understanding and improving fast adversarial training. In: Annual Conference on Neural Information Processing Systems (2020)

[2] Sriramanan, G., Addepalli, S., Baburaj, A., et al.: Guided adversarial attack for evaluating and enhancing adversarial defenses. Advances in Neural Information Processing Systems 33, 20297–20308 (2020)

[3] Sriramanan, G., Addepalli, S., Baburaj, A., et al.: Towards efficient and effective adversarial training. Advances in Neural Information Processing Systems 34 (2021)
questions: 1. supplementary material is same as the submitted manuscript

2. The main problem is that the authors have not compared their approach properly with some other methods that exist in the literature [1][2][3]

3. Lack of comparative experiments on large datasets, such as ImageNet. Previous works all conduct experiments on ImageNet

[1] Andriushchenko, M., Flammarion, N.: Understanding and improving fast adversarial training. In: Annual Conference on Neural Information Processing Systems (2020)

[2] Sriramanan, G., Addepalli, S., Baburaj, A., et al.: Guided adversarial attack for evaluating and enhancing adversarial defenses. Advances in Neural Information Processing Systems 33, 20297–20308 (2020)

[3] Sriramanan, G., Addepalli, S., Baburaj, A., et al.: Towards efficient and effective adversarial training. Advances in Neural Information Processing Systems 34 (2021)
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper revisits the generation process of adversarial perturbations in adversarial training with single-step FGSM attack. To solve the catastrophic overfitting problem, they propose to add random noises on benign images, and utilize FGSM to optimize adversarial perturbations based on these augment samples without restricting perturbations to lie inside ε-ball. This unrestricted method helps to balance the effect of noises and perturbations. Their experiments demonstrate that the proposed method achieves comparable robustness while being faster than previous methods.
strengths_and_weaknesses: Strengths
1. The presentation of this paper is good.
2. Fig 1 (left) provides a good illustration for Noise-FGSM. And the analysis between the noise magnitude and the attack strength is interesting.
3. This paper provides a different view to understanding single-step FGSM in adversarial training.

Weaknesses
1. The trade-off between noise magnitude and attack strength should be discussed in detail. Maybe authors can dynamically adjust these two to further prove their points.
2. The clipping ensures that generated adversarial examples are still lie inside the class space. The unrestricted method with a larger noise may undermine this assumption. In addition, the toy experiment (Fig 2 (right)) shows that the unrestricted method works well when the magnitude of noise is equal to 2ε or 4ε. It suggests that larger noise (4ε) may not further prevent CO. The statement of “Larger noise is also necessary to prevent CO” is not rigorous.
3. As shown in line 173-183, N-FGSM obtains a non-distorted loss surface similar to that obtained by GradAlign regularizer. It would be better if the authors could explain why noise augmentation can works as a regularizer?
4. RS-FGSM and GradAlign both performed experiments on ImageNet. Thus, the experiments on ImageNet should also be included in this paper.
5. How will the performance of this method change after improving ε of PGD?
6. Overall, this paper proposes an effective but simple method, and some points are not clear.
questions: 1. How will the performance of this method change after improving ε of PGD?
2. Why noise augmentation can works as a regularizer?

limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors have proposed a single-step adversarial training procedure, termed Noise-FGSM (N-FGSM), by considering added noise step as data augmentation rather than part of the attack. The resultant method centers adversarial perturbations with respect to noise-augmented samples (rather than wrt the clean one) and does not clip around the clean samples either, which allows much larger magnitude of the noise during the attack steps. Doing so can effectively avoid catastrophic overfitting (CO) in which a model that is adversarially trained with single-step FGSM suddenly becomes vulnerable to multi-step attacks. Also, the clean accuracy does not hurt. Experimentally, the proposed N-FGSM method reaches on-par or surpasses the SOTA method GradAlign while enjoying 3x speed-up.

strengths_and_weaknesses: Strengths:
Preventing the catastrophic overfitting (CO) in which a model suddenly becomes vulnerable to multi-step attacks despite remaining robust to single-step attacks is quite important in adversarial defense. The authors have proposed an adversarial training procedure where the noise is used as a form of data augmentation. The paper has good organization and presentation. Although the proposed method is not significantly different from prior work in terms of mathematical formulation, I do believe that it has its own merit and the authors have provided pretty thorough discussion and analysis in Section 4 and 5, helping readers to understand better the difference between the proposed method and prior work on single-step adversarial training that also incorporates noise. Experimentally, the proposed method performs competitively against prior work. Also, some important ablative studies are also conducted. 

Weaknesses:
I think overall this is a solid paper. At this reviewing stage, I do not spot apparent weaknesses. 

questions: I do not have questions at this point. 

limitations: Yes.
review_title: null

Meta-review: metareview: This paper enhances single-step adversarial training by adopting a much stronger noise for initialization. The initial concerns were mostly about missing ablations and misunderstandings/confusions, which were well addressed in the rebuttal. As a result, all reviewers unanimously agree to accept this submission.

In the final version, the authors should include all the clarifications and the additional empirical results provided in the rebuttal.
confidence: Certain
Decision: Accept