I-PGD-AT: Efficient Adversarial Training via Imitating Iterative PGD Attack 

1 INTRODUCTION . The breakthroughs in Deep Neural Networks ( DNNs ) have brought unprecedented boom to various fields , such as Computer Vision ( Krizhevsky et al. , 2012 ; He et al. , 2016 ) and Natural Language Processing ( Devlin et al. , 2019 ) . However , recent studies have revealed that DNNs are vulnerable to adversarial examples , such that imperceptible perturbations with small magnitude added to the input are sufficient to mislead the target model ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Xiao et al. , 2018a ; b ; Bhattad et al. , 2020 ) . Such perturbations are often transferable across different models ( Liu et al. , 2017 ; Dong et al. , 2018 ; Wang & He , 2021 ) and can be generated without the knowledge of the target model under black-box setting ( Brendel et al. , 2018 ; Li et al. , 2020b ; 2021 ) . Several works have been proposed to mitigate the aforementioned problem and improve the robustness of DNNs , e.g . adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) , input transformations ( Xie et al. , 2018 ; Naseer et al. , 2020 ) , certified defenses ( Raghunathan et al. , 2018 ; Gowal et al. , 2019 ) etc . Among these methods , adversarial training is one of the most effective empirical defense methods ( Athalye et al. , 2018 ) . Adversarial training can be formulated as a minimax optimization problem ( Madry et al. , 2018 ) , in which the inner maximization aims to search for an adversarial example that maximizes the classification loss , while the outer minimization aims to train a robust classifier against the worst-case adversarial perturbations . Projected Gradient Descent Adversarial Training ( PGD-AT ) ( Madry et al. , 2018 ) approximately solves the inner maximization problem by multiple steps of projected gradient descent and achieves empirical robustness ( Carlini & Wagner , 2017 ; Athalye et al. , 2018 ; Croce & Hein , 2020b ) . However , PGD-AT is much more expensive than standard training , owing to the iterative optimization process of perturbation generation . To reduce the computational burden , Wong et al . ( 2020 ) propose Fast Adversarial Training ( Fast-AT ) , which adopts Fast Gradient Sign Method with Random Start ( FGSM-RS ) for training and leads to robust models . However , there are two limitations for Fast-AT : a ) Fast-AT suffers from catastrophic overfitting , i.e . the robustness against iterative attack ( e.g . PGD ) suddenly drops after a few training epochs . b ) There is still a gap between the robustness of Fast-AT and PGD-AT . Fast-AT directly adopts early stopping to prevent catastrophic overfitting . Several attempts ( Andriushchenko & Flammarion , 2020 ; S. & Babu , 2020 ; Sriramanan et al. , 2020 ; Kim et al. , 2021 ) are made to avoid catastrophic overfitting , but these approaches either are computationally inefficient or hurt the robustness . In this work , motivated by the benefits of PGD-AT , we propose I-PGD Adversarial Training ( I-PGDAT ) , to improve the robustness of Fast-AT with the same computational cost and delay catastrophic overfitting . In particular , we first analyze the differences in perturbations generated by FGSM-RS and PGD and find that the perturbation of PGD is more diverse in which the perturbations are different discrete values instead of ±1 in FGSM-RS . Based on this observation , we imitate the perturbation of PGD virtually via a single gradient calculation . Specifically , we first calculate all the candidate discrete values for PGD attack ( e.g . 0 and ±2 for two-step PGD ) and its corresponding probability . Then we generate a new gradient vector by filling the cells with the candidate discrete value based on the corresponding absolute gradient while keeping the original sign information . Unlike FGSM-RS which directly crafts adversarial perturbation using the sign of gradient , we leverage this generated vector with more diverse discrete values to generate adversarial perturbation for efficient training . We conduct extensive empirical evaluations on CIFAR-10 and Tiny ImageNet datasets , which demonstrate that I-PGD-AT can significantly improve the robustness against various adversarial attacks . We also show that I-PGD-AT can effectively stabilize the training and delay catastrophic overfitting compared with Fast-AT . Furthermore , we empirically validate our hypothesis that under the same ` ∞-norm perturbation constraint , training the model using perturbation with a smaller mean absolute value can delay catastrophic overfitting , which could provide new insights to address such issues . Additionally , we extend our imitation strategy for multi-iteration , which achieves substantial improvements on the robustness compared with PGD-AT with different iterations , and delays the robust overfitting ( Rice et al. , 2020 ) in PGD-AT . 2 RELATED WORK . To mitigate the threat of adversarial examples , a large variety of defense method has been proposed , including adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) , input transformations ( Xie et al. , 2018 ; Guo et al. , 2018 ; Liu et al. , 2019 ) , denoising ( Liao et al. , 2018 ; Mustafa et al. , 2019 ; Naseer et al. , 2020 ) , certified defense ( Raghunathan et al. , 2018 ; Cohen et al. , 2019 ; Gowal et al. , 2019 ; Li et al. , 2019 ) . Among them , PGD-AT ( Madry et al. , 2018 ) , as the most popular adversarial training method , brings consistent improvements on robustness against various attacks and inspires several adversarial training methods ( Zhang et al. , 2019b ; Ding et al. , 2020 ; Wang et al. , 2020 ; Song et al. , 2020 ; Dong et al. , 2020 ) . Furthermore , Rice et al . ( 2020 ) identify robust overfitting in PGD-AT and find that early stopping or data augmentation ( Tack et al. , 2021 ) helps PGD-AT achieve comparable robustness with advanced adversarial training methods ( Zhang et al. , 2019b ) . However , PGD-AT is computationally inefficient due to the iterative optimization for adversarial examples . To combat the computational overhead , recent works ( Shafahi et al. , 2019 ; Zhang et al. , 2019a ) utilize a single backpropagation for both training and PGD adversary generation to accelerate the training process . Wong et al . ( 2020 ) identify catastrophic overfitting in single-step adversarial training and propose Fast Adversarial Training ( Fast-AT ) for efficient training . To avoid catastrophic overfitting , Andriushchenko & Flammarion ( 2020 ) propose GradAlign , which maximizes the gradient alignment inside the perturbation set to strengthen the model ’ s local linearity . However , it leads to 3× slowdown due to the double backpropagation . Kim et al . ( 2021 ) identify decision boundary distortion after catastrophic overfitting , in which a smaller perturbation is sufficient to fool the model , while the model is robust against larger perturbation used in Fast-AT . They further propose to search appropriate step size for each input sample to avoid catastrophic overfitting , but it also hurts the robustness performance . Improved fast AT ( FastAdv+ ) ( Li et al. , 2020a ) incorporates PGD-AT into Fast-AT when observing the catastrophic overfitting to stabilize the training . Chen et al . ( 2020 ) propose backward smoothing which adopts a back-propogation for initialization to implement single-step training with the objtective of TRADES ( Zhang et al. , 2019b ) . In this work , we identify the differences in perturbations generated by PGD and FGSM-RS and propose a single-step adversarial training method I-PGD-AT to boost the robustness of models and delay catastrophic overfitting effectively . 3 METHODOLOGY . In this section , we first provide preliminaries and revisit FGSM-RS and PGD to identify the differences in perturbations generated by them . Then we provide a detailed description of the proposed I-PGD Adversarial Training ( I-PGD-AT ) and a discussion on catastrophic overfitting . 3.1 PRELIMINARIES . Let J ( x , y , θ ) denote the loss function of deep neural net f with parameters θ on the input ( x , y ) drawn from data distribution D. Adversarial training can be formulated as a minimax optimization problem ( Madry et al. , 2018 ) : min θ E ( x , y ) ∼D [ max δ∈∆ J ( x+ δ , y , θ ) ] , ( 1 ) where ∆ is the perturbation set . In this paper , we focus on ` ∞ attack to align with previous work , i.e . ∆ = { δ ∈ Rd : ‖δ‖∞ < } , where is the maximum magnitude of the perturbation . This minimax problem is usually solved by firstly crafting adversarial examples to solve the inner maximization problem and then optimizing the model parameters θ using the generated adversarial examples . Fast Gradient Sign Method Adversarial Training ( FGSM-AT ) ( Goodfellow et al. , 2015 ) is the first adversarial training method that solves the inner maximization problem using one-step update : δ = · sign ( ∇xJ ( x , y , θ ) ) . ( 2 ) However , FGSM-AT exhibits limited robustness against iterative attacks ( e.g . iterative FGSM ( Kurakin et al. , 2017 ) ) . Later , Madry et al . ( 2018 ) propose Project Gradient Descent Adversarial Training ( PGD-AT ) , which iteratively solves the inner maximization problem using PGD : δt+1 = Π∆ ( δt + α · sign ( ∇xJ ( x+ δt , y , θ ) ) ) , ( 3 ) where δt is the adversarial perturbation at the t-th iteration and δ0 ∼ U ( − , ) , Π∆ ( · ) is the projection function and α < is the step size . Although PGD-AT is empirically robust , compared with FGSM-AT , the inner loop of PGD to craft adversarial examples is very expensive . Further , Wong et al . ( 2020 ) identify that FGSM-AT suffers from the catastrophic overfitting phenomenon , and propose Fast Adversarial Training ( Fast-AT ) using FGSM adversarial example with random start ( FGSM-RS ) , i.e. , ρ ∼ U ( − , ) : δ = Π∆ ( ρ+ α · sign ( ∇xJ ( x+ ρ , y , θ ) ) ) . ( 4 ) Fast-AT adopts early stopping to avoid catastrophic overfitting . 3.2 REVISITING FGSM-RS AND PGD . Let δ′ ∈ Rd denote the perturbation obtained by gradient ascent and gi denote the gradient at i-th iteration calculated by the corresponding adversarial attack . Suppose that the input x is normalized into [ 0 , 1 ] , for existing gradient-based adversarial attacks , the generated adversarial example xadv can be formulated as : xadv = Π [ 0,1 ] ( x+ Π∆ ( ρ+ δ ′ ) ) , ( 5 ) where ρ is the initialization of perturbation ( e.g . ρ = 0 for FGSM and ρ ∼ U ( − , ) for FGSM-RS and PGD ) . We could reformulate the final perturbation as δ = Π∆ ( ρ+ δ′ ) . For FGSM-RS , we could easily obtain δ′FGSM−RS = α · sign ( ∇xJ ( x + ρ , y ; θ ) ) = α · sign ( g0 ) from Eq . 4 . For PGDk where k is the number of iteration of PGD , we can approximately obtain that : δk = Π∆ ( ρ+ δ ′ ) = Π∆ ( δk−1 + α · sign ( gk−1 ) ) = Π∆ ( Π∆ ( δk−2 + α · sign ( gk−2 ) ) + α · sign ( gk−1 ) ) = Π∆ ( Π∆ ( · · ·Π∆ ( ρ+ α · sign ( g0 ) ) ) + α · sign ( gk−1 ) ) ' Π∆ ( ρ+ α · k−1∑ i=0 sign ( gi ) ) , ( 6 ) where gk = ∇xJ ( x+ δk , y , θ ) . By ignoring the projection operator at each iteration as the step size α is expected to be small w.r.t . the input , and we can see that δ′PGDk ' α · ∑k−1 i=0 sign ( gi ) for PGDk . Since FGSM-RS and PGD adopt the same random initialization , we only focus on the difference of δ′ . For simplicity , we treat the step size α as a scaled factor and do not consider it by normalizing the accumulated sign δ′ into [ −1 , +1 ] . Suppose the input sample x = ( x1 , x2 ) ∈ [ 0 , 1 ] 2 , the perturbation set for δ′ would be a square with side length 2 centered at the origin . Note that the output of sign ( · ) is either 1 or −1 . As shown in Fig . 1 , for FGSM-RS , δ′FGSM−RS would be one of the four vertices of the square . In contrast , taking PGD2 for example , there are four cases 〈+1 , +1〉 , 〈+1 , −1〉 , 〈−1 , +1〉 , 〈−1 , −1〉 of the sign for each pixel xi ∈ x , which lead to three different accumulated signs AS ( 2 ) = { +2 , 0 , −2 } . The general accumulated sign set for PGDk can be formulated as : AS ( k ) = { ± ( 2i+ k % 2 ) : 0 ≤ i ≤ bk/2c } . ( 7 ) We show the search space for PGD2 , PGD3 and PGD4 in Fig . 1 and we can see that the larger k is , the more diverse the search space for PGDk is . From the decision boundary distortion perspective ( Kim et al. , 2021 ) , it supports that PGDk can find more diverse adversarial examples inside the perturbation set so that it does not lead to catastrophic overfitting . Such difference of perturbation generated by FGSM-RS and PGD motivates us to consider the following question : Could we imitate the diverse discrete values of the perturbation generated by PGDk with a single gradient calculation to improve the robustness of single-step adversarial training and mitigate the catastrophic overfitting ?

Summary: **Few sentences summary**: the paper proposes a single-step adversarial training method called I-PGD-AT to improve over the Fast Adversarial Training method. Based on the fact that not all the perturbations lie on the boundary of the perturbation set when using multiple steps PGD, this method aims at replicating this behaviour while still using only one single adversarial training step.   For the **proposed method**, it uses: * Quantiles over the magnitudes of the gradient across the image. * A model approximating how the perturbation space is "crowded" after $k$ PGD steps under the assumption that gradients sign changes between iterations occur with a fixed probability $p$. * Quantiles over the gradient magnitudes and quantiles over the occupation of the perturbation space are matched such that the strongest gradients are assigned to the furthest position in the perturbation space.  For the **experiments**: * Evaluation on CIFAR-10 and TinyImageNet against other single-step methods: Fast-AT, GradAlign and Kim et al.(2021). * Evaluation on the extension of the proposed method to the PGD case. * The paper evaluates the sign consistency of the gradients between two consecutive iterations as the proposed method relies on the assumption that sign changes between iterations occur with a fixed probability $p$. * General investigation on catastrophic overfitting for single step methods.