Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In this paper, the authors studied improving the efficiency of adversarial training. The authors first analyzed the difference of perturbation generated by FGSM-RS and PGD and proposed an efficient single-step adversarial training method I-PGD-AT by adopting I-PGD attack for training. Extensive empirical evaluations on CIFAR-10 and Tiny ImageNet demonstrate that the proposed I-PGD-AT can improve the robustness compared with the baselines and significantly delay catastrophic overfitting. 
main_review: 1 . I wonder if the approximation in (6) makes sense? In (6), the gradient in later iterations can go far outside the perturbation limit and thus make the update nowhere near the actual case where all PGD iterations’ gradients are strictly limited within the perturbation ball.

2 . The authors claimed that one of the problems of one-step AT is that the perturbations are not diverse enough while in Figure 1, they showed that PGD-k should have diverse discrete perturbations. However, in practice, PGD attacks are usually located on the boundary with common choice attack step size (also mentioned in [1]).   

[1] "Parsimonious black-box adversarial attacks via efficient combinatorial optimization." International Conference on Machine Learning. PMLR, 2019. 

Also, even if Figure 1 is correct or aligned with practice, to me, it still does not explain how adopting random initialization can mitigate such a phenomenon.

3 . In the experiments, I notice that the authors actually used a large attack step size such as epsilon or 1.25epsilon, so that the final projection point is no longer located at those discrete points as in Figure 1. Rather, it will again be located on the boundary of the norm ball.

4 . The robustness improvement is marginal compared with Fast-AT as shown in Table 1. Also, it seems that the proposed method is actually trading natural accuracy for robustness? 

5 . In table 2, PGD2-AT performance is even worse than Fast-AT which is clearly not reasonable. The authors might want to double-check their experimental results.

6 . From figure 3, it seems that both I-PGD-AT and Fast-AT are experiencing catastrophic overfitting problems, just that Fast-AT happened first. Since this phenomenon may occur with randomness, it is hard to decide whether I-PGD-AT really solves the catastrophic overfitting problem. The authors should at present the result for repeated runs and see whether this is consistent.

7 . The following two papers also work on fast adversarial training. The authors might also want to comment and compare with them:

[2] "Towards understanding fast adversarial training." arXiv preprint arXiv:2006.03089 (2020).
[3] "Efficient robust training via backward smoothing." arXiv preprint arXiv:2010.01278 (2020).




summary_of_the_review: The intuition of the proposed method is not really making sense and the empirical results are not quite convincing, therefore, I recommend reject for this paper
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper notices that the existing methods of adversarial training has the problem of catastrophic overfitting, and proposes an efficient single-step adversarial training method I-PGD-AT by adopting I-PGD attack for training, in which I-PGD imitates the perturbation of PGD based on the magnitude of gradient. Extensive empirical evaluations on CIFAR-10 and Tiny ImageNet demonstrate that I-PGD-AT can improve the robustness compared with the baselines and delay catastrophic overfitting.
main_review: ##########################################################################

Pros: 

1. The paper focuses on the catastrophic overfitting problem caused by the limited number of adversarial perturbation patterns generated by the single-step attack method during the adversarial training process, and proposes I-PGD-AT to alleviate the problem.

2. The paper pays attention to the both problems of high computations in PGD-based adversarial training and limited attack patterns in FGSM-based adversarial training, and gives a compromise method to absorb the advantages of both methods and discard the disadvantages at the same time.

3. This paper provides comprehensive experiments on CIFAR10 and Tiny ImageNet to demonstrate the effectiveness of I-PGD-AT. And further analysis on catastrophic overfitting problem is provided.

##########################################################################

Cons: 

1. The author only focuses on $\delta’$ and ignores $\rho$ in Fig. 1 to demonstrate that PGDk can generate more diverse search space and further find more diverse adversarial examples in the process of attack. However, since both FGSM-RS and PGDk utilizes the random start in the beginning of the attack, I think both of the methods can explores the whole search space in \Deta.

2. Compared to Fast-AT, I-PGD-AT only slightly improves the robustness of the model under attack (average ~2% in Tab. 1), but the classification accuracy on clean images has dropped obviously (the drop also reaches ~2% in Tab. 1).

3. The author has too few methods to compare in the experiments. Since a large number of methods in adversarial training haves been proposed in recent years, more methods should be compared to demonstrate the effectiveness of the I-PGD-AT.

#########################################################################

Some typos: 

(1) two lines above Sec. 3.4: \mathcal{I}=\{0: 1-p, 2: 1\} -> \mathcal{I}=\{0: 1-p, 2: p\}

(2) the same problem with (1) in Appendix A.1

(3) immediately after the problem of (2), “arbitra ry” should have no blank space




summary_of_the_review: Overall, I vote for weak reject. The paper proposes I-PGD-AT to imitate PGD virtually through single gradient calculation. My major concern is about the margin improvement and the limited compared methods in experiments (see cons below). Hopefully the author can address my concern in the rebuttal period. 

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

UPDATE

After reading all the response and comments from other reviewers, I will keep my scores. My main concern is the insufficient experiments.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Summary:

The paper introduces a new adversarial training method to improve the performance of Fast adversarial training (FAT), an adversarial training method that depends on only one gradient query. Adversarial training using PGD attack is not efficient, but FAT performs worse than PGD adversarial training. To improve the performance of FAT and maintain the efficiency, the authors study the characteristics of PGD perturbation and proposes a method to imitate PGD perturbation based on only one gradient query. 


main_review: ##########################################################################

Pros: 

1. The problem the paper is trying to solve is an important one. Adversarial robustness is of great importance and adversarial training has
been shown to be one of the most effective methods. However, the efficiency of multi-step adversarial training is a problem. 

 
2. The intuition behind the proposed method makes sense. The difference between PGD generated adversarial examples and FGSM generated ones comes from the perturbations. The proposed method introduces a heuristic way to imitate PGD adversarial perturbation based on one gradient query, which is efficient and novel. 

 
3. Experimental results on CIFAR10 and TinyImageNet show that I-PGD-AT outperforms several fast adversarial training methods against state-of-the-art attacks. Besides, it does increase the training time much compared to FAT. Studies on catastrophic overfitting are also done to investigate contributing factors to the issue.

 
##########################################################################

Cons: 

 
1. My main concern is that the method is not evaluated on ImageNet. In the paper of Fast adversarial training (Wong et al., 2020), the method was tested on ImageNet. It might be hard to address this problem as performing adversarial training (even the fast version) may take several days.

 
2. Overall, the paper is clearly written and well-organized, but some details are missing, making it hard to follow. (See detailed comments below.) But, this can addressed during rebuttal.
 
##########################################################################

Detailed Comments: 

(1) I like the fact that pseudo code of computing possible discrete values and their corresponding probabilities. However, the process of generating quantile set Q based on I is just one sentence. It would be better if some details of this process is provided in the appendix as well. (Maybe I'm slow in understanding this. I read the code to figure out the process.)

(2) Section 3.3, "We only need to consider the absolute value and the smaller |g|[i] is, the smaller value..." is a little bit confusing. Suggestion: "We only need to consider the absolute value. The smaller |g|[i] is, the smaller value..."

(3) What is "the sign of uniform perturbation"? It's not mentioned previously. Suddenly, the technique is used to deal with catastrophic overfitting. 

Please address and clarify the cons and detailed comments above.
summary_of_the_review: Reasons for score: 

 
Overall, I vote for weakly accepting. I like the idea of imitating the PGD perturbation to improve the performance of one-step adversarial training. Experiments show that the performance of the proposed method is better than fast adversarial training and has similar training
time. My major concern is that the proposed method is not evaluated on large image dataset, like ImageNet. Some methods perform well 
on small datasets like CIFAR10 but do not perform well on large datasets. Since Fast adversarial training can be deployed on ImageNet 
(experiments can be found in the paper), why not compare I-PGD-AT with FAT on ImageNet?

##########################################################################
Updates: Thanks for the authors' response. Part of my concerns are addressed by the author. The concern about lack experiments on ImageNet is hard to address during rebuttal period. Therefore, I'll keep my rating.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: **Few sentences summary**: the paper proposes a single-step adversarial training method called I-PGD-AT to improve over the Fast Adversarial Training method. Based on the fact that not all the perturbations lie on the boundary of the perturbation set when using multiple steps PGD, this method aims at replicating this behaviour while still using only one single adversarial training step. 

For the **proposed method**, it uses:
* Quantiles over the magnitudes of the gradient across the image.
* A model approximating how the perturbation space is "crowded" after $k$ PGD steps under the assumption that gradients sign changes between iterations occur with a fixed probability $p$.
* Quantiles over the gradient magnitudes and quantiles over the occupation of the perturbation space are matched such that the strongest gradients are assigned to the furthest position in the perturbation space.

For the **experiments**:
* Evaluation on CIFAR-10 and TinyImageNet against other single-step methods: Fast-AT, GradAlign and Kim et al.(2021).
* Evaluation on the extension of the proposed method to the PGD case.
* The paper evaluates the sign consistency of the gradients between two consecutive iterations as the proposed method relies on the assumption that sign changes between iterations occur with a fixed probability $p$.
* General investigation on catastrophic overfitting for single step methods.

main_review: **Strengths**:
* Great empirical results by improving over Fast-AT with the same runtime as well as delaying catastrophic overfitting.
* The paper is well written.
* The idea of using the quantiles of the gradient magnitudes to produce multiple step sizes  within a single step adversarial method is a clever idea.
* The code is attached to the supplementary material and looks usable for easy reproducibility.

**Weaknesses**:
1) Regarding equation (6) and "We approximately ignore the projection operator at each iteration as the step size $\alpha$ is expected to be small", the theoretical justification does not seem well-supported. First, according to the experiment section, $\alpha$ is not small enough compared to $\epsilon$. Indeed, if the initial random perturbation $\rho$ is not equal to 0, the perturbation can get "stuck" on the pertubation set boundary after a few steps. If $\rho$ is close to $\epsilon$, the perturbation boundary can even be reached after one step. Hence, we cannot ignore the projections. Second, if we cannot ignore the projections, we lose the grid structure (like in Figure 1) used in the rest of the analysis to determine the algorithm. I might be wrong for both points but it would require some clarifications.
2) Regarding the sign consistency experiment. Could the high value for the sign consistency be due to the perturbation being "stuck" on the perturbation set boundary? If yes, the fact that the projections which were ignored in order to come up with the algorithm would be the responsible for the high value of the sign consistency. That would go against "The proposed method I-PGD is based on the hypothesis that due to the neighborhood constraint by adversarial attacks, the probability $p$ that a specific element in the input obtains the same sign in two consecutive iterations would be high and stable".
3) The theoretical approach boils down to a simple result "{0 : 1 − p, 2 : p}" for  I-PGD2-AT and  "{1 : 1 − p, 3 : p}" for I-PGD3-AT. There might be other approaches leading to this simple result without making the assumptions discussed above which need clarifications. It could just be an heuristic approach such as: PGD leads to perturbations close to the boundary or closer to the random start. This last sentence translates into your result "{0 : 1 − p, 2 : p}" and would just require a sweep over the probability $p$. Such a sweep is done in Table 3 for I-PGD2-AT and it would be great to have the same for "{1 : 1 − p, 3 : p}" with I-PGD3-AT.
4) The ablation study on the probability $p$ in the appendix should have been in the main paper, more than the catastrophic overfitting discussion.
5) Minor. There are a few typos in the paper. Page 5 and in the appendix: {0 : 1 − p, 2 : 1} ->  {0 : 1 − p, 2 : p}. In Section 4.6, the subscripts of  "I-PGD2-AT" are lacking compared to Figure 5.
6) Minor. Why doing the catastrophic overfitting study in Figure 4 on Fast-AT rather than on the proposed method?
7) Minor. In Figure 5, you should add a curve "PGD10 Acc on PGD10-AT" to the plot for a more complete comparison.
summary_of_the_review: The paper proposes a method which could be very useful to the community as it provides a single-step adversarial training method with clearly improved performance compared to existing methods like Fast-AT with the same runtime. I am vouching for acceptance due to the great experimental results but currently I am not yet satisfied by the theoretical justification of the method as a few statements are not well-supported. I think that it can be clarified in the rebuttal as using the quantiles of the gradient magnitudes to produce multiple step sizes  within a single step adversarial method is a clever idea.
review_title: null

Meta-review: This paper aims to improve the efficiency of adversarial training. Specifically, by analyzing the differences between the adversarial perturbations generated by FGSM-RS and the adversarial perturbations generated by PGD, this paper proposes a new single-step attacker I-PGD (which imitates PGD by creating diverse adversarial perturbations) to accelerate adversarial training. Empirical results are provided on CIFAR-10 and Tiny ImageNet to support the effectiveness of the proposed method.

Overall, the reviewers think it is an interesting paper, but are severely concerned about some statements. During the discussion period, the authors actively clarify these points. However, the Reviewer bLbt is not fully convinced and believes 1) the approximation in Eq. (6) is incorrect and 2) the proposed method is loosely motivated by imitating the behavior of PGD. The authors fail to further follow up on this discussion. The Reviewer bLbt and the Reviewer Dz2K are also concerned that the proposed I-PGD-AT only yields margin improvements over Fast Adversarial Training. In addition, as suggested by the reviewer AAHj, given this work focuses on developing efficient adversarial training, it is important to include results on larger-scale datasets like ImageNet.

I encourage the authors to incorporate all the reviewers' comments and make a stronger submission next time.
Decision: Reject