I-PGD-AT: E FFICIENT A DVERSARIAL T RAINING VIA I MITATING I TERATIVE PGD A TTACK
Anonymous authors
Paper under double-blind review
A BSTRACT
To improve the efﬁciency of adversarial training, recent studies leverage Fast
Gradient Sign Method with Random Start (FGSM-RS) for adversarial training.
Unfortunately, such methods lead to relatively low robustness and catastrophic overﬁtting , which means that the robustness against iterative attacks ( e.g . Projected
Gradient Descent (PGD)) would suddenly drop to 0%.
Different approaches have been proposed to address this problem, however, recent studies show that catastrophic overﬁtting still remains.
In this paper, motivated by the fact that expensive iterative adversarial training methods achieve high robustness without catastrophic overﬁtting, we ask: Can we perform iterative adversarial training in an efﬁcient way?
To this end, we ﬁrst analyze the differences in perturbations generated by FGSM-RS and PGD and ﬁnd that PGD tends to craft diverse discrete values instead of ± 1 in FGSM-RS.
Based on this observation, we propose an efﬁcient single-step adversarial training method I-PGD-AT by leveraging I-PGD attack, in which I-PGD imitates PGD virtually.
Unlike FGSM that crafts the perturbations directly using the sign of gradient, I-PGD imitates the perturbation of PGD based on the magnitude of gradient.
Extensive empirical evaluations on CIFAR-10 and Tiny ImageNet demonstrate that our I-PGD-AT can improve the robustness compared with the baselines and signiﬁcantly delay catastrophic overﬁtting.
Moreover, we explore and discuss the factors that affect catastrophic overﬁtting.
Finally, to demonstrate the generality of I-PGD-AT, we integrate it into
PGD adversarial training and show that it can even further improve the robustness.
I NTRODUCTION
The breakthroughs in Deep Neural Networks (DNNs) have brought unprecedented boom to various ﬁelds, such as Computer Vision (Krizhevsky et al., 2012;
He et al., 2016) and Natural Language
Processing (Devlin et al., 2019).
However, recent studies have revealed that DNNs are vulnerable to adversarial examples , such that imperceptible perturbations with small magnitude added to the input are sufﬁcient to mislead the target model (Szegedy et al., 2014; Goodfellow et al., 2015; Xiao et al., 2018a;b; Bhattad et al., 2020).
Such perturbations are often transferable across different models (Liu et al., 2017; Dong et al., 2018; Wang & He, 2021) and can be generated without the knowledge of the target model under black-box setting (Brendel et al., 2018; Li et al., 2020b; 2021).
Several works have been proposed to mitigate the aforementioned problem and improve the robustness of DNNs, e.g .
adversarial training (Goodfellow et al., 2015; Madry et al., 2018), input transformations (Xie et al., 2018; Naseer et al., 2020), certiﬁed defenses (Raghunathan et al., 2018; Gowal et al., 2019) etc .
Among these methods, adversarial training is one of the most effective empirical defense methods (Athalye et al., 2018).
Adversarial training can be formulated as a minimax optimization problem (Madry et al., 2018), in which the inner maximization aims to search for an adversarial example that maximizes the classiﬁcation loss, while the outer minimization aims to train a robust classiﬁer against the worst-case adversarial perturbations.
Projected Gradient Descent Adversarial
Training (PGD-AT) (Madry et al., 2018) approximately solves the inner maximization problem by multiple steps of projected gradient descent and achieves empirical robustness (Carlini & Wagner, 2017; Athalye et al., 2018; Croce & Hein, 2020b).
However, PGD-AT is much more expensive than standard training, owing to the iterative optimization process of perturbation generation.
To reduce the computational burden, Wong et al.
(2020) propose Fast Adversarial Training (Fast-AT), which adopts Fast Gradient Sign Method with Random Start (FGSM-RS) for training and leads to robust models.
However, there are two limitations for Fast-AT: a) Fast-AT suffers from catastrophic overﬁtting , i.e . the robustness against iterative attack ( e.g . PGD) suddenly drops after a few training epochs.
b) There is still a gap between the robustness of Fast-AT and PGD-AT.
Fast-AT directly adopts early stopping to prevent catastrophic overﬁtting.
Several attempts (Andriushchenko & Flammarion, 2020; S. & Babu, 2020; Sriramanan et al., 2020; Kim et al., 2021) are made to avoid catastrophic overﬁtting, but these approaches either are computationally inefﬁcient or hurt the robustness.
In this work, motivated by the beneﬁts of PGD-AT, we propose I-PGD Adversarial Training (I-PGDAT), to improve the robustness of Fast-AT with the same computational cost and delay catastrophic overﬁtting.
In particular, we ﬁrst analyze the differences in perturbations generated by FGSM-RS and PGD and ﬁnd that the perturbation of PGD is more diverse in which the perturbations are different discrete values instead of ± 1 in FGSM-RS.
Based on this observation, we imitate the perturbation of PGD virtually via a single gradient calculation.
Speciﬁcally, we ﬁrst calculate all the candidate discrete values for PGD attack ( e.g . 0 and ± 2 for two-step PGD) and its corresponding probability.
Then we generate a new gradient vector by ﬁlling the cells with the candidate discrete value based on the corresponding absolute gradient while keeping the original sign information.
Unlike FGSM-RS which directly crafts adversarial perturbation using the sign of gradient, we leverage this generated vector with more diverse discrete values to generate adversarial perturbation for efﬁcient training.
We conduct extensive empirical evaluations on CIFAR-10 and Tiny ImageNet datasets, which demonstrate that I-PGD-AT can signiﬁcantly improve the robustness against various adversarial attacks.
We also show that I-PGD-AT can effectively stabilize the training and delay catastrophic overﬁtting compared with Fast-AT.
Furthermore, we empirically validate our hypothesis that under the same (cid:96) ∞ -norm perturbation constraint, training the model using perturbation with a smaller mean absolute value can delay catastrophic overﬁtting , which could provide new insights to address such issues.
Additionally, we extend our imitation strategy for multi-iteration, which achieves substantial improvements on the robustness compared with PGD-AT with different iterations, and delays the robust overﬁtting (Rice et al., 2020) in PGD-AT.
R ELATED W ORK
To mitigate the threat of adversarial examples, a large variety of defense method has been proposed, including adversarial training (Goodfellow et al., 2015; Madry et al., 2018), input transformations (Xie et al., 2018;
Guo et al., 2018;
Liu et al., 2019), denoising (Liao et al., 2018; Mustafa et al., 2019;
Naseer et al., 2020), certiﬁed defense (Raghunathan et al., 2018;
Cohen et al., 2019; Gowal et al., 2019;
Li et al., 2019).
Among them, PGD-AT (Madry et al., 2018), as the most popular adversarial training method, brings consistent improvements on robustness against various attacks and inspires several adversarial training methods (Zhang et al., 2019b;
Ding et al., 2020; Wang et al., 2020;
Song et al., 2020; Dong et al., 2020).
Furthermore, Rice et al.
(2020) identify robust overﬁtting in
PGD-AT and ﬁnd that early stopping or data augmentation (Tack et al., 2021) helps PGD-AT achieve comparable robustness with advanced adversarial training methods (Zhang et al., 2019b).
However, PGD-AT is computationally inefﬁcient due to the iterative optimization for adversarial examples.
To combat the computational overhead, recent works (Shafahi et al., 2019; Zhang et al., 2019a) utilize a single backpropagation for both training and PGD adversary generation to accelerate the training process.
Wong et al.
(2020) identify catastrophic overﬁtting in single-step adversarial training and propose Fast Adversarial Training (Fast-AT) for efﬁcient training.
To avoid catastrophic overﬁtting, Andriushchenko & Flammarion (2020) propose GradAlign, which maximizes the gradient alignment inside the perturbation set to strengthen the model’s local linearity.
However, it leads to 3 × slowdown due to the double backpropagation.
Kim et al.
(2021) identify decision boundary distortion after catastrophic overﬁtting, in which a smaller perturbation is sufﬁcient to fool the model, while the model is robust against larger perturbation used in Fast-AT.
They further propose to search appropriate step size for each input sample to avoid catastrophic overﬁtting, but it also hurts the robustness performance.
Improved fast AT (FastAdv+)
(Li et al., 2020a) incorporates
PGD-AT into Fast-AT when observing the catastrophic overﬁtting to stabilize the training.
Chen et al.
(2020) propose backward smoothing which adopts a back-propogation for initialization to implement single-step training with the objtective of TRADES (Zhang et al., 2019b).
In this work, we identify the differences in perturbations generated by PGD and FGSM-RS and propose a single-step adversarial training method I-PGD-AT to boost the robustness of models and delay catastrophic overﬁtting effectively.
M ETHODOLOGY
In this section, we ﬁrst provide preliminaries and revisit FGSM-RS and PGD to identify the differences in perturbations generated by them.
Then we provide a detailed description of the proposed I-PGD
Adversarial Training (I-PGD-AT) and a discussion on catastrophic overﬁtting.
3. P RELIMINARIES
Let J ( x, y, θ ) denote the loss function of deep neural net f with parameters θ on the input ( x, y ) drawn from data distribution D .
Adversarial training can be formulated as a minimax optimization problem (Madry et al., 2018):
[Equation]
(1) where ∆ is the perturbation set.
In this paper, we focus on (cid:96) ∞ attack to align with previous work, i.e . ∆ = { δ ∈ R d : (cid:107) δ (cid:107) ∞ < (cid:15) } , where (cid:15) is the maximum magnitude of the perturbation.
This minimax problem is usually solved by ﬁrstly crafting adversarial examples to solve the inner maximization problem and then optimizing the model parameters θ using the generated adversarial examples.
Fast Gradient Sign Method Adversarial Training (FGSM-AT) (Goodfellow et al., 2015) is the ﬁrst adversarial training method that solves the inner maximization problem using one-step update:
[Equation]
However, FGSM-AT exhibits limited robustness against iterative attacks ( e.g . iterative FGSM (Kurakin et al., 2017)).
Later, Madry et al.
(2018) propose Project Gradient Descent Adversarial Training (PGD-AT), which iteratively solves the inner maximization problem using PGD:
[Equation]
where δ t is the adversarial perturbation at the t -th iteration and δ 0 ∼ U ( − (cid:15), (cid:15) ) , Π ∆ ( · ) is the projection function and α < (cid:15) is the step size.
Although PGD-AT is empirically robust, compared with FGSM-AT, the inner loop of PGD to craft adversarial examples is very expensive.
Further, Wong et al.
(2020) identify that FGSM-AT suffers from the catastrophic overﬁtting phenomenon, and propose Fast Adversarial Training (Fast-AT) using FGSM adversarial example with random start (FGSM-RS), i.e ., ρ ∼ U ( − (cid:15), (cid:15) ) :
[Equation]
[Equation]
Fast-AT adopts early stopping to avoid catastrophic overﬁtting.
3. R EVISITING FGSM-RS AND PGD
Let
δ (cid:48) ∈
R d denote the perturbation obtained by gradient ascent and g i denote the gradient at i -th iteration calculated by the corresponding adversarial attack.
Suppose that the input x is normalized into [0 , 1] , for existing gradient-based adversarial attacks, the generated adversarial example x adv can be formulated as:
[Equation]
where ρ is the initialization of perturbation ( e.g . ρ = 0 for FGSM and ρ ∼ U ( − (cid:15), (cid:15) ) for FGSM-RS and PGD).
We could reformulate the ﬁnal perturbation as δ = Π ∆ ( ρ + δ (cid:48) ) .
For FGSM-RS, we could easily obtain δ (cid:48) FGSM − RS =
α · sign( ∇ x J ( x + ρ, y ; θ ))
=
α · sign( g 0 ) from Eq.
For PGD k where k is the number of iteration of PGD, we can approximately obtain that:
[Equation]
where g k = ∇ x J ( x + δ k , y, θ ) .
By ignoring the projection operator at each iteration as the step size α is expected to be small w.r.t. the input, and we can see that δ (cid:48)
PGDk (cid:39) α · (cid:80) k − 1 i =0 sign( g i ) for PGD k . xx FGSM-RS xx
PGD xx PGD xx
PGD
Since FGSM-RS and PGD adopt the same random initialization, we only focus on the difference of δ (cid:48) .
For simplicity, we treat the step size α as a scaled factor and do not consider it by normalizing the accumulated sign δ (cid:48) into [ − 1 , +1] .
Suppose the input sample x = ( x 1 , x 2 ) ∈
[0 , 1] 2 , the perturbation set for δ (cid:48) would be a square with side length 2 centered at the origin.
Note that the output of sign( · ) is either 1 or − 1 .
As shown in Fig.
1, for FGSM-RS, δ (cid:48)
FGSM − RS would be one of the four vertices of the square.
In contrast, taking PGD2 for example, there are four cases (cid:104) +1 , +1 (cid:105) , (cid:104) +1 , − 1 (cid:105) , (cid:104)− 1 , +1 (cid:105) , (cid:104)− 1 , − 1 (cid:105) of the sign for each pixel x i ∈ x , which lead to three different accumulated signs AS (2) =
{ +2 , 0 , − 2 } .
The general accumulated sign set for PGD k can be formulated as:
[Equation]
We show the search space for PGD2, PGD3 and PGD4 in Fig.
and we can see that the larger k is, the more diverse the search space for PGD k is.
From the decision boundary distortion perspective (Kim et al., 2021), it supports that PGD k can ﬁnd more diverse adversarial examples inside the perturbation set so that it does not lead to catastrophic overﬁtting.
Such difference of perturbation generated by
FGSM-RS and PGD motivates us to consider the following question:
Could we imitate the diverse discrete values of the perturbation generated by PGD k with a single gradient calculation to improve the robustness of single-step adversarial training and mitigate the catastrophic overﬁtting?
3. I-PGD A DVERSARIAL T RAINING
From Sec.
3.2, we ﬁnd that FGSM-RS directly reaches the boundary of perturbation set, which is one possible reason for catastrophic overﬁtting (Kim et al., 2021) in Fast-AT that performs adversarial training with perturbation generated by FSGM-RS, while PGD can generate diverse perturbation inside the boundary.
To some extent, adopting random initialization which leads to more diverse perturbation, can mitigate such phenomenon.
Apart from catastrophic overﬁtting, the robustness is also weaker for Fast-AT compared with PGD-AT.
Considering the properties of perturbation generated by PGD, we aim to explore how to improve the robustness of Fast-AT and mitigate catastrophic overﬁtting by imitating the perturbation of PGD.
It is noted that both FGSM-RS and PGD adopt the sign of the gradient without considering the relative magnitude of each element in the gradient vector g ∈ R d .
Due to the multiple iterations, PGD leads to diverse but still discrete candidate perturbation for δ (cid:48) .
In order to imitate the diverse values of PGD by single gradient calculation, we need to generate a new imitative gradient vector ˜ g ∈ R d for crafting adversarial perturbation by additionally utilizing the magnitude of gradient.
Generally speaking, to achieve this goal, there are two issues to address.
What discrete values does ˜ g contain?
To imitate a speciﬁc PGD k attack, we can effectively calculate the accumulated sign set from Eq.
For instance, if k = 2 , we have three values {− 2 , 0 , +2 } which can be assigned to each element.
For any k , the candidate values are symmetric regarding the origin.
Thus, we only need to consider the absolute values for simplicity, i.e . we only have { 0 , 2 } for PGD2, and the assigned values keep the same sign with the corresponding gradient dimensions.
Which value do we assign to the element ˜ g
[ i ] ?
After determining what values does ˜ g contain, we can assign the value to each element to obtain ˜ g by utilizing the relative magnitude of the gradient g .
We only need to consider the absolute value.
The smaller | g | [ i ] is, the smaller value is assigned to | ˜ g | [ i ] .
Speciﬁcally, we ﬁrst calculate the probability of each possible value for PGD k .
Let p denote the probability that a speciﬁc element obtains the same sign ( i.e . (cid:104)· · · , +1 , +1 · · · (cid:105) or (cid:104)· · · , − 1 , − 1 · · · (cid:105) ) in two consecutive iterations, and we assume that p > 1 / 2 due to the neighborhood constraint by adversarial attacks.
We can calculate the probability for each value by summing the probability for each possible sequence of sign.
For a given gradient g , we can calculate the quantile on | g | according to the accumulated probability P ( | δ (cid:48) | [ i ] ≤ v ) for each candidate value v and set all the elements in this quantile range with value v to obtain ˜ g for adversarial training.
We denote adopting ˜ g for attack as I-PGD attack and such adversarial training method as I-PGD Adversarial Training (I-PGD-AT).
Taking PGD2 for example, we could get that the imitative value is 0 and 2 with the probability
P ( | δ (cid:48)
| [ i ] = 0)
= 1 − p, P ( | δ (cid:48)
| [ i ] = 2) =
p , i.e . I = { 0 : 1 − p, 2 : 1 } .
We provide the detailed calculation for I =
{ v i : p i } in Appendix A.1 and overall algorithm of I-PGD-AT in Algorithm 1.
3. D ISCUSSION ON C ATASTROPHIC O VERFITTING
The major difference between FGSM-RS and I-PGD attack is that for each input sample, FGSM-RS generates the adversarial example only using the sign of gradient, while I-PGD adopts the imitative gradient vector ˜ g of PGD to craft adversarial example.
Since PGD-AT does not lead to catastrophic overﬁtting and our I-PGD-AT utilizes I-PGD for adversarial training, we wonder if such imitation could effectively avoid catastrophic overﬁtting.
Unfortunately, as discussed in Sec. 4.5, we ﬁnd that catastrophic overﬁtting still occurs in I-PGD-AT.
However, compared with Fast-AT, I-PGD-AT can stabilize the training process and delay catastrophic overﬁtting effectively.
We further explore the relationship between the magnitude of perturbation and catastrophic overﬁtting and validate a new hypothesis that under the same (cid:96) ∞ -norm constraint on the perturbation, the perturbation with a smaller mean absolute value can effectively delay catastrophic overﬁtting.
This hypothesis sheds new light on how to avoid catastrophic overﬁtting in single-step adversarial training effectively.
E XPERIMENTS 4. E XPERIMENTAL S ETTING
Datasets and Models.
We conduct empirical evaluations on CIFAR-10 (Krizhevsky et al., 2009) and
Tiny ImageNet 1 using PreAct ResNet-18 (He et al., 2016).
All the images are normalized into [0 , 1] .
Baselines.
To verify the effectiveness of I-PGD-AT, we imitate PGD2 and PGD3 dubbed I-PGD2-AT and I-PGD3-AT with single-step adversarial training, respectively.
We compare them with FreeAT (Shafahi et al., 2019) and four single-step adversarial training methods, i.e .
Fast-AT (Wong et al., 2020), GradAlign (Andriushchenko & Flammarion, 2020),
Kim et al . (Kim et al., 2021) and
[Table]
FastAdv+ (Li et al., 2020a).
We also extend I-PGD into PGD2-AT and PGD7-AT to imitate more powerful iterative adversarial training and compare them with the corresponding PGD-AT methods.
Training Details.
We use SGD optimizer with momentum of 0 .
9 and decay factor of 5 ×
10 − 4 , and adopt the perturbation budget (cid:15) = 8 / 255 for adversarial training.
For single-step adversarial training, we train the model with 30 epochs on CIFAR-10 and 40 epochs on Tiny ImageNet, using cyclic learning rate (Smith, 2017) with the maximum learning rate of 0 . 2 and increasing until the half of the training epochs.
For PGD k -AT, we train the model with 200 epochs using the maximum learning rate of 0 . 2 , which decays to 0 .
02 at 50 -th epoch.
The probability p is set to 2 / 3 and α = (cid:15) for I-PGD-AT.
Evaluations.
We evaluate the robustness using various attacks, i.e . FGSM (Goodfellow et al., 2015),
PGD (Madry et al., 2018), Momentum Iterative Method (MIM) (Dong et al., 2018), Carlini-Wagner (CW) (Carlini & Wagner, 2017) and AutoAttack (AA) (Croce & Hein, 2020b).
More details about the experimental setting are summarized in Appendix A.3.
4. E VALUATION ON I-PGD-AT
To validate the effectiveness of the proposed I-PGD-AT, we empirically evaluate the robustness of
I-PGD2-AT and I-PGD3-AT, which performs the single-step training to imitate PGD with different steps for perturbation optimization on CIFAR-10 and Tiny ImageNet.
Results on CIFAR-10.
As Fast-AT may lead to catastrophic overﬁtting after 24 -th epoch on CIFAR10 (Andriushchenko & Flammarion, 2020), we train Fast-AT multiple times and provide the best results.
As shown in Table 1a, Kim et al .
achieves the lowest robustness against various attacks but higher clean accuracy due to the smaller generated perturbation on average.
Fast-AT exhibits better robustness than Free-AT while GradAlign and FastAdv+ achieve higher robustness than Fast-AT and do not lead to catastrophic overﬁtting.
Our proposed I-PGD2-AT could stabilize the training process and achieve the highest robustness compared with the three baselines, without catastrophic overﬁtting during the training.
It is noted that I-PGD3-AT, which only adopts different imitative values and corresponding portions ( i.e . { 1 : 5 / 9 , 3 : 4 / 9 } for I-PGD3-AT and { 0 : 1 / 3 , 2 : 2 / 3 } for I-PGD2-AT), exhibits higher performance than I-PGD2-AT with the same computational cost.
It further shows the reasonability and effectiveness of the proposed imitation strategy.
As for the comparison of training time , although GradAlign improves the robustness, it leads to roughly 3 × slowdown compared with Fast-AT.
Free-AT takes smaller computational cost for each epoch but larger training epochs, leading to longer training time.
FastAdv+ takes much longer times due to the larger training epochs and using PGD10 to recover catastrophic overﬁtting.
Our proposed methods take about the same training time as Fast-AT since we only process the calculated gradient with constant time complexity, and achieves the highest robustness. [Table]
Results on Tiny ImageNet.
To further validate the effectiveness of I-PGD-AT, we perform similar experiments on Tiny ImageNet.
Due to the high computational cost of GradAlign, here we do not take it as baseline.
As shown in Table 1b, the results consistently demonstrate that our I-PGD-AT can improve the robustness over Fast-AT without extra computational cost.
We ﬁnd that the improvement of I-PGD3-AT over I-PGD2-AT is not so obvious as on CIFAR-10, which might due to the fact that the sign consistency on Tiny ImageNet is higher than that on CIFAR-10 as discussed in Sec. 4.4.
4. I NTEGRATING I-PGD INTO PGD-AT
Though the primary goal of I-PGD-AT is to boost Fast-AT, the imitation strategy is not limited to single-step adversarial training and can be integrated with PGD-AT during each step.
Speciﬁcally, at each iteration, we incorporate I-PGD2 into PGD2-AT and PGD7-AT, denoted as I-PGD2-AT PGD and I-PGD2-AT PGD7 , to imitate more powerful iterative adversarial training, namely PGD4-AT and
PGD14-AT.
As shown in Table 2, I-PGD2-AT PGD2 and I-PGD2-AT PGD7 consistently achieve better robustness than the corresponding baselines PGD2-AT and PGD7-AT, with almost the same training time.
Surprisingly, I-PGD2-AT PGD2 exhibits much better robustness than PGD4-AT.
I-PGD2-AT PGD achieves slightly lower robustness than PGD14-AT, which takes double iterations for optimizing the inner maximization problem of adversarial training and doubles the training time.
4. S IGN C ONSISTENCY FOR PGD
[Figure] The proposed method I-PGD is based on the hypothesis that due to the neighborhood constraint by adversarial attacks, the proba- bility p that a speciﬁc element in the input obtains the same sign in two consecutive iterations would be high and stable.
To validate this hypothesis, we deﬁne the sign consistency (SC) for i-th iteration of PGD as follows:
[Equation]
where g i denotes the gradient at i -th iteration and 1 ( · ) is the indicator function.
To some extent,
SC ( i ) also indicates the local linearity of the deep neural models.
If SC ( i ) = 1 , PGD attack would degenerate to FGSM-RS attack because the direction for perturbation update in Eq.
3 is constant at each iteration so that the iterations do not make any difference.
In contrast, the perturbation of PGD differs from that of FGSM-RS signiﬁcantly if SC ( i ) is small.
Besides, the stability of SC ( i ) makes it possible for I-PGD-AT to adopt a ﬁxed probability to imitate PGD-AT effectively.
We calculate SC ( i ) for PGD50 attack on Fast-AT at various training epochs over 3 random seeds for training.
As shown in Fig.
2, generally speaking, the sign consistency of PGD50 is high and stable at various iterations but training longer degrades the sign consistency and increases the standard deviation.
Moreover, on Tiny ImageNet, the sign consistency decreases much slower with smaller standard deviation than that on CIFAR-10, which might be the reason why I-PGD3-AT cannot signiﬁcantly enhance I-PGD2-AT and Fast-AT is more stable on Tiny ImageNet. [Figure] [Figure] [Figure]
4. F ACTORS R ELATED TO C ATASTROPHIC O VERFITTING
[Figure]
In this section, we investigate what factors would affect the catastrophic overﬁtting in single-step adversarial training.
First, we explore if our IPGD-AT can avoid catastrophic overﬁtting.
We train the model withepochs using two learning rate schedulers, i.e . single cyclic learning rate and the cyclic learning rate in the ﬁrst 30 epochs followed with piecewise learning rate, in which the learning rate is decayed by a factor of 0 .
1 per 10 epochs, dubbed CLR and CPLR respectively.
As shown in Fig.
3, with CLR, catastrophic overﬁtting occurs in both Fast-AT and I-PGD2-AT but I-PGD2-AT postpones it for 7 epochs compared with Fast-AT.
Besides, with CPLR, Fast-AT still leads to catastrophic overﬁtting while I-PGD2-AT stably maintains its high robustness after 30th epochs.
This indicates that our I-PGD-AT can effectively delay catastrophic overﬁtting and stablize the training process.
We further explore the factors that affect catastrophic overﬁtting on CIFAR-10.
Previous works (Andriushchenko & Flammarion, 2020; Kim et al., 2021) show that small (cid:96) ∞ -norm constraint (cid:15) for training prevents catastrophic overﬁtting.
We ﬁnd
that even with the same (cid:15) , the mean absolute value of the perturbation ¯ |
δ | generated by I-PGD2 is 6 . 5 , which is a little smaller than the one of FGSM-RS.
Furthermore, we ﬁnd that adopting the sign of uniform perturbation for initialization also prevents catastrophic overﬁtting, which leads to ¯ | δ | = 5 . 0 .
Thus, we speculate that under the same (cid:96) ∞ -norm constraint on the perturbation δ , smaller ¯ | δ | can effectively delay catastrophic overﬁtting .
To validate this hypothesis, we ﬁx (cid:15) = 8 / 255 , while vary the step size α in Eq.
4 from (cid:15)/ 2 to 2 · (cid:15) to adjust ¯ | δ | for Fast-AT with uniform perturbation as initialization, dubbed Fast-AT-U. To show the generality of such hypothesis, we further adopt the perturbation sampled from {− 1 , +1 } uniformly as initialization, denoted as Fast-AT-S. As shown in Fig.
4a, when we increase the value of α , ¯ | δ | increases in Fast-AT-U while ¯ | δ | ﬁrst decreases before α = 1 , then increases linearly in Fast-AT-S.
This is because when α = 1 , the random perturbation inconsistent with the sign of gradient will cancel each other out, resulting in the minimum perturbation.
As shown in Fig.
4b, before the catastrophic overﬁtting occurring, the larger α which means larger ¯ | δ | leads to better robustness for both initialization methods.
If we continue to increase the value of α , as depicted in Fig.
4c, the catastrophic overﬁtting occurs and the larger α leads to the earlier epoch for catastrophic overﬁtting.
It supports the hypothesis and shows the trade-off between high robustness and catastrophic overﬁtting.
4. A BLATION STUDY
To further gain insights on the performance improvement obtained by I-PGD-AT, we conduct ablation study for the probability p for I-PGD2-AT on CIFAR-10. As shown in Sec. A.1, the higher probability p indicates the higher portion that the value 2 would occupy in | ˜ g | at line 7-9 in Algorithm 1, leading
[Table]
to larger | ¯ δ | .
We vary p from 0 .
5 to 1 and summarize the results in Table 3.
As we can see, when we increase the value of p , the clean accuracy decreases a little but the robustness increases signiﬁcantly when p ≤ 0 .
However, when we continue to increase p , | ¯ δ | increases and the catastrophic overﬁtting happens.
The larger p indicates the larger | ¯ δ | and results in the earlier epoch for catastrophic overﬁtting.
This is also consistent with our experiments and analysis in Sec.
4.5 and further support our hypothesis that under the same (cid:96) -norm constraint on the perturbation δ , smaller | ¯ δ | can effectively delay the catastrophic overﬁtting .
4. R OBUST O VERFITTING IN I-PGD-AT
[Figure]
[Figure]
Rice et al.
(2020) identify robust overﬁtting in PGD-AT that the robust accuracy begins to decrease as training progresses after a few epochs.
Here we brieﬂy explore whether I-PGDAT is helpful to delay or avoid the robust overﬁtting.
As shown in Fig.
5, I-PGD2-AT consistently exhibits better robust accuracy than PGD2-AT and PGD4-AT throughout the training process.
As for the robust overﬁtting, I-PGD-AT maintains the peak performance between 50 -th and 75 -th epochs, while the robust accuracy of PGD2-AT and PGD4AT begins to decrease around 50 -th epoch.
Hence, it is clear that I-PGD-AT can delay (not avoid) the robust overﬁtting in PGD-AT with better ﬁnal robustness, and the sustainability for peak robustness might be helpful to choose a better epoch for early stopping.
4. C OMPARING I-PGD WITH PGD A TTACKS
Though we have observed that I-PGD-AT can signiﬁcantly improve the robustness of single-step and multi-step adversarial training, we wonder whether I-PGD attack indeed imitates the attack effectiveness of PGD.
To verify this, we incorporate the imitation strategy for PGD2 into PGD attack to evaluate the robustness of I-PGD2-AT on CIFAR-10.
We denote such attack as I-PGD k where k is the number of iteration.
As illustrated in Fig.
6, we ﬁnd that for various number of iteration k , I-PGD k consistently achieves similar attack performance as PGD k .
The imitation for PGD3 also exhibits the same trend and we do not show it in the ﬁgure for clarity.
The reasons should be two-fold: a) As shown in Fig.
6, increasing the number of iteration cannot effectively improve the attack performance for PGD k when k ≥ 20 .
b) Our imitation only considers the diverse value of perturbation without effectively predicting the sign of generated perturbation at each iteration, which might limit the attack performance.
We believe that it is possible to improve the attack performance through imitating powerful attacks and leave this as our future work.
C ONCLUSION
In this work, we identiﬁed the differences in perturbations crafted by FGSM-RS and PGD.
Based on this observation, we proposed I-PGD adversarial training (I-PGD-AT) by adopting I-PGD attack that imitates PGD virtually through single gradient calculation.
Experiments on CIFAR-10 and Tiny
ImageNet showed that I-PGD-AT could effectively improve the robustness and delay catastrophic overﬁtting and robust overﬁtting.
We believe that this work would inspire more precise imitation of
PGD, which can improve the attack effectiveness and further enhance the robustness of DNNs.
A A PPENDIX A. C ALCULATION FOR I = { v i : p i }
In this section, we ﬁrst provide the detailed calculation for I = { v i : p i } with PGD2 as an example and then summarize a general algorithm for PGD k .
Taking PGD2 for example, there would be four cases of the sign for each pixel x i ∈ x , namely (cid:104)
+1 , +1 (cid:105) , (cid:104) +1 , − 1 (cid:105) , (cid:104)− 1 + 1 (cid:105) and (cid:104)− 1 , − 1 (cid:105) .
Suppose the sign of the ﬁrst iteration is either +1 or − 1 with equal probability, we could calculate the probability for each sign sequence as following:
[Equation]
where p is the probability that a speciﬁc element x i obtains the same sign ( i.e .
[Equation]
· · + 1 , +1 · · · (cid:105) or (cid:104)· · · − 1 , − 1 · · · (cid:105) ) in two consecutive iterations.
The imitative value would be { 0 , 2 } and we could calculate the corresponding probability:
[Equation]
) = p Thus, we could obtain that I [0] = P ( v ≤ 0)
= 1 − p
and
I [2] = P ( v ≤ 2) = 1 − p + p = p , i.e . I = { 0 : 1 − p, 2 : 1 } .
For an arbitrary value of k , we ﬁrst traverse each possible sign sequence to calculate the corresponding probability.
Then we calculate the probability for each potential imitative value by accumulating the probability of related sign sequences.
To obtain the quantile for each possible value, we further accumulate the probability based on the imitative value in ascending order.
The general calculation for any k is summarized in Algorithm 2.
probability A N EXAMPLE OF CALCULATING THE QUANTILE To better illustrate the algorithm, we also provide a simple example of imitating PGD2 with the probability p = 2 / 3 as follows. First, we could obtain the probability P ( | δ (cid:48)
| [ i ] = 0) = 1 − p = 1 / 3 , P ( | δ (cid:48)
| [ i ] = 2) = p = 2 / 3 and calculate the imitative value and accumulated probability I = { 0 : 1 / 3 , 2 : 1 } .
Suppose we obtain the gradient matrix g =
[Equation]
.(cid:35)
at line 5 of Algorithm 1, we need to calculate the start and end value for the range with the portion from 0 to 1 / 3 and from 1 / 3 to 1 on the sorted array [0 . 1 , 0 . 1 , 0 . 2 , 0 . 9] (the absolute values of g ), respectively. Thus, we could get the quantile Q = { 0 . 1 , 0 . 2 , 0 . 9 } at Line 6 of Algorithm 1 (0 is the minimal value for the portion of 0, 0.2 is the smaller 3-quantile for the portion of 1/3 and 0.9 is the maximum value for the portion of 1.). After the for-loop at line 7-9 of Algorithm 1, we have ˜ g
= {+(cid:34) 
+}
22 22 0(cid:35) by calculating each element ˜ g [ i ] {+= (cid:26) 
 if 0 .+}
1 ≤ | g | [ i ] ≤ 0 .2 if 0 . 2 < | g | [ i ] ≤ 0 . 9 . Then we let ˜ g share the same sign of g at line 10 and obtain ˜ g
{+= (cid:34) 
+}
− 22 − 2   (cid:35) , which could be used to generate the perturbation for adversarial training at line 11-12 of Algorithm 1.
A. E XPERIMENTAL SETTING
In this section, we provide more details about the experimental setting, including datasets, hyperparameters for baselines and the attack settings.
Datasets.
We adopt two widely investigated benchmark datasets for the evaluation, namely CIFAR10 and Tiny ImageNet.
CIFAR-10 contains 10 classes with 5 , 000 training and 1 , 000 testing images of resolution 32 × 32 per class.
Tiny ImageNet contains 120 , 000 images of 200 classes downsized to 64 × 64 colored images, in which each class has 500 training images, 50 validation images andtesting images.
Hyper-parameters for baselines.
For Free-AT, we set α = (cid:15) , m = 8 and train the model forepochs.
We adopt α = 1 . 25 · (cid:15) for Fast-AT (Wong et al., 2020), set the regularization parameter for GradAlign to 0 .
2 (Andriushchenko & Flammarion, 2020) and utilize 3 checkpoints for Kim et al .
(Kim et al., 2021).
We follow the setting of FastAdv+ (Li et al., 2020a) which reports the best results among 200 epochs with piecewise learning rate and adopts PGD-10 to recover the catastrophic overﬁtting.
For PGD k -AT, the step size α is set to max(2 / 255 , (cid:15)/k ) .
For all the baselines, we evaluate the effectiveness of the models at the last training epoch (Wong et al., 2020; Andriushchenko & Flammarion, 2020; Kim et al., 2021).
Attack settings.
CW is implemented by adopting the margin-based loss function (Carlini & Wagner, 2017) and using PGD for optimization.
We use 20 and 50 steps for PGD and CW and 50 steps for
MIM with 10 random restarts.
The perturbation budget (cid:15) is set to 8 / 255 and step size α is set to 2 / 255 .
AutoAttack adopts the default setting (Croce & Hein, 2020b) with four adversarial attacks, namely untargeted APGD-CE, targeted APGD-DLR, targeted FAB (Croce & Hein, 2020a) and
Squared attack (Andriushchenko et al., 2020).