Towards fast and effective single-step adversarial training

1 INTRODUCTION . Deep neural networks have achieved remarkable performance on a variety of tasks ( He et al. , 2015 ; Silver et al. , 2016 ; Devlin et al. , 2019 ) . However , it is well known that they are vulnerable to small worst-case perturbations around the input data – commonly referred to as adversarial examples ( Szegedy et al. , 2014 ) . The existence of such adversarial examples poses a security threat to deploying models in sensitive environments ( Biggio & Roli , 2018 ) . This has motivated a large body of work towards improving the adversarial robustness of neural networks ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ; Tramèr et al. , 2018 ) . The most popular family of solutions to obtain robust neural networks is based on the concept of adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) . In a nutshell , adversarial training can be posed as a min-max problem where instead of minimizing some loss over a dataset of clean samples , we augment the inputs with worst-case perturbations that are generated online during training . However , obtaining such perturbations is NP-hard ( Weng et al. , 2018 ) and hence , different approaches have been suggested to approximate them . They are commonly referred to as adversarial attacks . In their seminal work , Goodfellow et al . ( 2015 ) proposed the Fast Gradient Sign Method ( FGSM ) , that generates adversarial attacks by running one step of gradient ascent on the loss function . However , while FGSM-based adversarial training provides robustness against single-step FGSM adversaries , Madry et al . ( 2018 ) ; Tramèr et al . ( 2018 ) showed that these models were still vulnerable to multi-step attacks , namely those allowed to perform multiple gradient ascent steps instead of a single one . Notably , Madry et al . ( 2018 ) introduced the multi-step Projected Gradient Descent ( PGD ) attack . PGD-based attacks have now become the de facto standard for adversarial training ; yet , their cost increases linearly with the number of steps . As a result , several works have focused on reducing the cost of adversarial training by approximating the worst-case perturbations with single-step attacks ( Wong et al. , 2020 ; Shafahi et al. , 2019 ; Vivek & Babu , 2020 ) . In particular , Wong et al . ( 2020 ) studied FGSM adversarial training and discovered that it suffers from a characteristic failure mode , in which a model suddenly becomes vulnerable to multi-step attacks despite remaining robust to single-step attacks . This phenomenon is referred to as catastrophic overfitting . Moreover , they argued that adding a random perturbation prior to FGSM ( RS-FGSM ) seemed sufficient to prevent catastrophic overfitting and yield robust models . Recently , Andriushchenko & Flammarion ( 2020 ) observed that RS-FGSM still leads to catastrophic overfitting as we increase the perturbation radii . They suggested a regularizer ( GradAlign ) that , on the one hand avoids catastrophic overfitting in all the settings they considered , but on the other hand requires the computation of a double derivative – which significantly increases the computational cost compared to RS-FGSM . This has motivated other works that aim at achieving the same level of robustness with a lower computational overhead ( Golgooni et al. , 2021 ; Kim et al. , 2021 ) . In this paper , we revisit two key components that are common among previous works combining noise and FGSM ( Tramèr et al. , 2018 ; Wong et al. , 2020 ) : the role of noise , i.e . the random step , and the role of the clipping step . In Section 4.1 , we study how these two components affect model robustness ; our experiments suggest that adding noise with a large magnitude in the random step and removing the clipping step improves model robustness and prevents catastrophic overfitting , even against large perturbation radii . We combine these observations and propose a new method called Noise-FGSM ( N-FGSM ) , an illustration of which is presented in Figure 1 ( left ) . N-FGSM allows to match , or even surpass , the robust accuracy of the regularized FGSM introduced by Andriushchenko & Flammarion ( 2020 ) , while providing a 3× speed-up . To corroborate the effectiveness of our solution , we present an experimental survey of recently proposed single-step attacks and empirically demonstrate that N-FGSM trades-off robustness and computational cost better than other single-step approaches , evaluated over a large spectrum of perturbation radii ( see Figure 1 , middle and right panels ) , over several datasets ( CIFAR-10 , CIFAR100 , and SVHN ) and architectures ( PreActResNet18 and WideResNet28-10 ) . We will release our code reproducing all experiments . 2 RELATED WORK . Since the discovery of adversarial examples , many defense mechanisms have been proposed . Preprocessing techniques try to modify the input image to neutralize adversarial attacks ( Guo et al. , 2018 ; Buckman et al. , 2018 ; Song et al. , 2018 ) . Adversarial detection methods focus on detecting and rejecting adversarial attacks ( Carlini & Wagner , 2017 ; Ma et al. , 2018 ; Yang et al. , 2020 ; Tian et al. , 2021 ) . Certifiable defenses provide theoretical guarantees for the lower bound performance of networks subjected to worst-case adversarial attacks , however , they incur additional costs during inference and , empirically , they yield sub-optimal performance ( Cohen et al. , 2019 ; Wong & Kolter , 2018 ; Raghunathan et al. , 2018 ; Balunovic & Vechev , 2020 ) . Adversarial training methods are based on a special form of data augmentation designed to make the network robust to worst-case perturbations ( Zhang et al. , 2019 ; Athalye et al. , 2018 ; Kurakin et al. , 2017 ) . However , computing a worst-case perturbation is an NP-hard problem that needs to be solved at every iteration . To minimize the overhead of adversarial training , Goodfellow et al . ( 2015 ) proposed FGSM which requires one additional gradient step per iteration . Tramèr et al . ( 2018 ) first proposed performing a random step before taking the adversarial step ( R+FGSM ) , but they observed that neither method yields robust models against PGD attacks ( Madry et al. , 2018 ) . Since then , augmenting the training with PGD attacks has been one of the most popular approaches for robustness , but its cost increases linearly with the number of steps , which presents a severe practical limitation . To reduce the cost of PGD , Shafahi et al . ( 2019 ) proposed Free Adversarial Training ( Free-AT ) , that exploits a single back-propagation step to both update the network parameters and compute the attack . Wong et al . ( 2020 ) explored a variation of R+FGSM , namely RS-FGSM , and showed it can yield robust networks against multi-step attacks . Andriushchenko & Flammarion ( 2020 ) found that RS-FGSM only works for limited perturbation radii and introduced GradAlign – a regularizer to linearize the loss surface . However , optimizing GradAlign triplicates the computational cost . This motivated a new series of works that aim at matching the performance of GradAlign without the additional computational overhead ( Golgooni et al. , 2021 ; Kim et al. , 2021 ) . Other strategies that attempted to improve FGSM included introducing dropout in every layer ( Vivek & Babu , 2020 ) and perturbing intermediate feature maps together with the input ( Park & Lee , 2021 ) . Li et al . ( 2020 ) suggested combining RS-FGSM and PGD attacks during training , however , the proposed strategy requires a frequent monitoring of the PGD robust accuracy and , in the worst-case , is computationally equivalent to PGD training . Gilmer et al . ( 2019 ) ; Fawzi et al . ( 2018 ) suggested a strong link between robustness to adversarial attacks and to random noise . Motivated by this , we revisit the idea of combining noise and FGSM and propose N-FGSM . Our method is closely related to RS-FGSM , however , we find that using a larger amount of noise and removing the constraint that attacks must lie in the − l∞ ball is key to obtaining robust models . We note that Kang & Moosavi-Dezfooli ( 2021 ) concurrently studied RS-FGSM without clipping , however , as opposed to our work , they did not investigate and provide insights on the impact of noise , and the learned models were not robust against large perturbations . 3 PRELIMINARIES ON SINGLE-STEP ADVERSARIAL TRAINING . Given a classifier fθ : X → Y parameterized by θ and a perturbation set S , the classifier fθ is said to be robust at x ∈ X under S if the following holds for all δ ∈ S : fθ ( x + δ ) = fθ ( x ) . One of the most popular definitions for S is the − ` ∞ ball , i.e . S = { δ : ‖δ‖∞ ≤ } . This is known as the l∞ threat model and is the setting we adopt throughout this work . To train networks that are robust against ` ∞ threat models , adversarial training modifies the classical training procedure of minimizing a loss function over a dataset D = { ( xi , yi ) } i=1 : N of images xi ∈ X and labels yi ∈ Y . In particular , adversarial training instead minimizes the worst-case loss over the perturbation set S , i.e . trains on adversarially perturbed samples { ( xi+δi , yi ) } i=1 : N . When using the l∞ threat model , we can formalize adversarial training as solving the following problem : min θ N∑ i=1 max δ L ( fθ ( xi + δ ) , yi ) , subject to ‖δ‖∞ ≤ , ( 1 ) where L is typically the cross-entropy loss for image-classification models . Due to the difficulty of finding the exact inner maximizer , the most common procedure for adversarial training is to approximate the worst-case perturbation through several PGD steps ( Madry et al. , 2018 ) . While this has been shown to yield robust models , it comes at a cost of a linear increase in the computational overhead with the number of PGD steps . As a result , several works have focused on reducing the cost of adversarial training by approximating the inner maximization with single-step attacks . If we assume that the loss function is locally linear with respect to changes in the input , then the inner maximization of Equation ( 1 ) enjoys a closed form solution . Goodfellow et al . ( 2015 ) leveraged this result to propose the FGSM method , which takes one step in the direction of the sign of the gradient . Tramèr et al . ( 2018 ) proposed adding a random initialization prior to FGSM . However , both methods were later shown to be vulnerable against multi-step attacks , such as PGD ( Madry et al. , 2018 ) . Contrary to prior intuition , recent work from Wong et al . ( 2020 ) observed that combining a random step with FGSM can actually lead to a promising robustness performance . In particular , we note that most recent single-step methods approximate the worst-case perturbation that results from solving the inner maximization problem in Equation ( 1 ) with the following general form : δ = ψ ( η + α · sign ( ∇xiL ( fθ ( xi + η ) , yi ) ) ) , where η ∼ Ω ( 2 ) and Ω is the distribution from which we draw noise perturbations . For example , when ψ is the projection operator onto the ` ∞ ball and Ω is the uniform distribution in [ − , ] , this recovers RSFGSM with the following update : δRS-FGSM = Proj‖δ‖∞≤ ( η + α · sign ( ∇xiL ( f ( xi + η ) , yi ) ) ) , where η ∼ U [ − , ] d. ( 3 ) On the other hand , with a different noise setting where Ω = ( −α ) · sign ( N ( 0 , I ) ) and by choosing the step size α to be in [ 0 , ] we recover R+FGSM by Tramèr et al . ( 2018 ) that initially explored the idea of combining noise with FGSM but reported no improvement over adversarial training with FGSM . If we consider Ω to be deterministically 0 and ψ to be the identity map , we recover the FGSM . Finally , if we adjust the choice of the loss function L to include a gradient alignment regularizer , this recovers the GradAlign algorithm by Andriushchenko & Flammarion ( 2020 ) .

Summary: This paper aims to address a failure mode in the traditional single-step adversarial training known as catastrophic overfitting. They show that compared to the common practice of generating the adversarial perturbation, adopting larger random initialization and avoiding clipping the perturbation can effectively mitigate catastrophic overfitting. The effects of these two techniques are analyzed empirically in detail, followed by a comprehensive comparison to other methods.