Towards fast and effective single-step adversarial training

1 INTRODUCTION . Deep neural networks have achieved remarkable performance on a variety of tasks ( He et al. , 2015 ; Silver et al. , 2016 ; Devlin et al. , 2019 ) . However , it is well known that they are vulnerable to small worst-case perturbations around the input data – commonly referred to as adversarial examples ( Szegedy et al. , 2014 ) . The existence of such adversarial examples poses a security threat to deploying models in sensitive environments ( Biggio & Roli , 2018 ) . This has motivated a large body of work towards improving the adversarial robustness of neural networks ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ; Tramèr et al. , 2018 ) . The most popular family of solutions to obtain robust neural networks is based on the concept of adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) . In a nutshell , adversarial training can be posed as a min-max problem where instead of minimizing some loss over a dataset of clean samples , we augment the inputs with worst-case perturbations that are generated online during training . However , obtaining such perturbations is NP-hard ( Weng et al. , 2018 ) and hence , different approaches have been suggested to approximate them . They are commonly referred to as adversarial attacks . In their seminal work , Goodfellow et al . ( 2015 ) proposed the Fast Gradient Sign Method ( FGSM ) , that generates adversarial attacks by running one step of gradient ascent on the loss function . However , while FGSM-based adversarial training provides robustness against single-step FGSM adversaries , Madry et al . ( 2018 ) ; Tramèr et al . ( 2018 ) showed that these models were still vulnerable to multi-step attacks , namely those allowed to perform multiple gradient ascent steps instead of a single one . Notably , Madry et al . ( 2018 ) introduced the multi-step Projected Gradient Descent ( PGD ) attack . PGD-based attacks have now become the de facto standard for adversarial training ; yet , their cost increases linearly with the number of steps . As a result , several works have focused on reducing the cost of adversarial training by approximating the worst-case perturbations with single-step attacks ( Wong et al. , 2020 ; Shafahi et al. , 2019 ; Vivek & Babu , 2020 ) . In particular , Wong et al . ( 2020 ) studied FGSM adversarial training and discovered that it suffers from a characteristic failure mode , in which a model suddenly becomes vulnerable to multi-step attacks despite remaining robust to single-step attacks . This phenomenon is referred to as catastrophic overfitting . Moreover , they argued that adding a random perturbation prior to FGSM ( RS-FGSM ) seemed sufficient to prevent catastrophic overfitting and yield robust models . Recently , Andriushchenko & Flammarion ( 2020 ) observed that RS-FGSM still leads to catastrophic overfitting as we increase the perturbation radii . They suggested a regularizer ( GradAlign ) that , on the one hand avoids catastrophic overfitting in all the settings they considered , but on the other hand requires the computation of a double derivative – which significantly increases the computational cost compared to RS-FGSM . This has motivated other works that aim at achieving the same level of robustness with a lower computational overhead ( Golgooni et al. , 2021 ; Kim et al. , 2021 ) . In this paper , we revisit two key components that are common among previous works combining noise and FGSM ( Tramèr et al. , 2018 ; Wong et al. , 2020 ) : the role of noise , i.e . the random step , and the role of the clipping step . In Section 4.1 , we study how these two components affect model robustness ; our experiments suggest that adding noise with a large magnitude in the random step and removing the clipping step improves model robustness and prevents catastrophic overfitting , even against large perturbation radii . We combine these observations and propose a new method called Noise-FGSM ( N-FGSM ) , an illustration of which is presented in Figure 1 ( left ) . N-FGSM allows to match , or even surpass , the robust accuracy of the regularized FGSM introduced by Andriushchenko & Flammarion ( 2020 ) , while providing a 3× speed-up . To corroborate the effectiveness of our solution , we present an experimental survey of recently proposed single-step attacks and empirically demonstrate that N-FGSM trades-off robustness and computational cost better than other single-step approaches , evaluated over a large spectrum of perturbation radii ( see Figure 1 , middle and right panels ) , over several datasets ( CIFAR-10 , CIFAR100 , and SVHN ) and architectures ( PreActResNet18 and WideResNet28-10 ) . We will release our code reproducing all experiments . 2 RELATED WORK . Since the discovery of adversarial examples , many defense mechanisms have been proposed . Preprocessing techniques try to modify the input image to neutralize adversarial attacks ( Guo et al. , 2018 ; Buckman et al. , 2018 ; Song et al. , 2018 ) . Adversarial detection methods focus on detecting and rejecting adversarial attacks ( Carlini & Wagner , 2017 ; Ma et al. , 2018 ; Yang et al. , 2020 ; Tian et al. , 2021 ) . Certifiable defenses provide theoretical guarantees for the lower bound performance of networks subjected to worst-case adversarial attacks , however , they incur additional costs during inference and , empirically , they yield sub-optimal performance ( Cohen et al. , 2019 ; Wong & Kolter , 2018 ; Raghunathan et al. , 2018 ; Balunovic & Vechev , 2020 ) . Adversarial training methods are based on a special form of data augmentation designed to make the network robust to worst-case perturbations ( Zhang et al. , 2019 ; Athalye et al. , 2018 ; Kurakin et al. , 2017 ) . However , computing a worst-case perturbation is an NP-hard problem that needs to be solved at every iteration . To minimize the overhead of adversarial training , Goodfellow et al . ( 2015 ) proposed FGSM which requires one additional gradient step per iteration . Tramèr et al . ( 2018 ) first proposed performing a random step before taking the adversarial step ( R+FGSM ) , but they observed that neither method yields robust models against PGD attacks ( Madry et al. , 2018 ) . Since then , augmenting the training with PGD attacks has been one of the most popular approaches for robustness , but its cost increases linearly with the number of steps , which presents a severe practical limitation . To reduce the cost of PGD , Shafahi et al . ( 2019 ) proposed Free Adversarial Training ( Free-AT ) , that exploits a single back-propagation step to both update the network parameters and compute the attack . Wong et al . ( 2020 ) explored a variation of R+FGSM , namely RS-FGSM , and showed it can yield robust networks against multi-step attacks . Andriushchenko & Flammarion ( 2020 ) found that RS-FGSM only works for limited perturbation radii and introduced GradAlign – a regularizer to linearize the loss surface . However , optimizing GradAlign triplicates the computational cost . This motivated a new series of works that aim at matching the performance of GradAlign without the additional computational overhead ( Golgooni et al. , 2021 ; Kim et al. , 2021 ) . Other strategies that attempted to improve FGSM included introducing dropout in every layer ( Vivek & Babu , 2020 ) and perturbing intermediate feature maps together with the input ( Park & Lee , 2021 ) . Li et al . ( 2020 ) suggested combining RS-FGSM and PGD attacks during training , however , the proposed strategy requires a frequent monitoring of the PGD robust accuracy and , in the worst-case , is computationally equivalent to PGD training . Gilmer et al . ( 2019 ) ; Fawzi et al . ( 2018 ) suggested a strong link between robustness to adversarial attacks and to random noise . Motivated by this , we revisit the idea of combining noise and FGSM and propose N-FGSM . Our method is closely related to RS-FGSM , however , we find that using a larger amount of noise and removing the constraint that attacks must lie in the − l∞ ball is key to obtaining robust models . We note that Kang & Moosavi-Dezfooli ( 2021 ) concurrently studied RS-FGSM without clipping , however , as opposed to our work , they did not investigate and provide insights on the impact of noise , and the learned models were not robust against large perturbations . 3 PRELIMINARIES ON SINGLE-STEP ADVERSARIAL TRAINING . Given a classifier fθ : X → Y parameterized by θ and a perturbation set S , the classifier fθ is said to be robust at x ∈ X under S if the following holds for all δ ∈ S : fθ ( x + δ ) = fθ ( x ) . One of the most popular definitions for S is the − ` ∞ ball , i.e . S = { δ : ‖δ‖∞ ≤ } . This is known as the l∞ threat model and is the setting we adopt throughout this work . To train networks that are robust against ` ∞ threat models , adversarial training modifies the classical training procedure of minimizing a loss function over a dataset D = { ( xi , yi ) } i=1 : N of images xi ∈ X and labels yi ∈ Y . In particular , adversarial training instead minimizes the worst-case loss over the perturbation set S , i.e . trains on adversarially perturbed samples { ( xi+δi , yi ) } i=1 : N . When using the l∞ threat model , we can formalize adversarial training as solving the following problem : min θ N∑ i=1 max δ L ( fθ ( xi + δ ) , yi ) , subject to ‖δ‖∞ ≤ , ( 1 ) where L is typically the cross-entropy loss for image-classification models . Due to the difficulty of finding the exact inner maximizer , the most common procedure for adversarial training is to approximate the worst-case perturbation through several PGD steps ( Madry et al. , 2018 ) . While this has been shown to yield robust models , it comes at a cost of a linear increase in the computational overhead with the number of PGD steps . As a result , several works have focused on reducing the cost of adversarial training by approximating the inner maximization with single-step attacks . If we assume that the loss function is locally linear with respect to changes in the input , then the inner maximization of Equation ( 1 ) enjoys a closed form solution . Goodfellow et al . ( 2015 ) leveraged this result to propose the FGSM method , which takes one step in the direction of the sign of the gradient . Tramèr et al . ( 2018 ) proposed adding a random initialization prior to FGSM . However , both methods were later shown to be vulnerable against multi-step attacks , such as PGD ( Madry et al. , 2018 ) . Contrary to prior intuition , recent work from Wong et al . ( 2020 ) observed that combining a random step with FGSM can actually lead to a promising robustness performance . In particular , we note that most recent single-step methods approximate the worst-case perturbation that results from solving the inner maximization problem in Equation ( 1 ) with the following general form : δ = ψ ( η + α · sign ( ∇xiL ( fθ ( xi + η ) , yi ) ) ) , where η ∼ Ω ( 2 ) and Ω is the distribution from which we draw noise perturbations . For example , when ψ is the projection operator onto the ` ∞ ball and Ω is the uniform distribution in [ − , ] , this recovers RSFGSM with the following update : δRS-FGSM = Proj‖δ‖∞≤ ( η + α · sign ( ∇xiL ( f ( xi + η ) , yi ) ) ) , where η ∼ U [ − , ] d. ( 3 ) On the other hand , with a different noise setting where Ω = ( −α ) · sign ( N ( 0 , I ) ) and by choosing the step size α to be in [ 0 , ] we recover R+FGSM by Tramèr et al . ( 2018 ) that initially explored the idea of combining noise with FGSM but reported no improvement over adversarial training with FGSM . If we consider Ω to be deterministically 0 and ψ to be the identity map , we recover the FGSM . Finally , if we adjust the choice of the loss function L to include a gradient alignment regularizer , this recovers the GradAlign algorithm by Andriushchenko & Flammarion ( 2020 ) .

Summary: This paper methodically studied the catastrophic overfitting in fast adversarial training (Fast-AT), and revisited the role of noise and clipping operation in Fast-AT. Based on the empirical findings, this paper discovered that the absence of clipping as well as using stronger noise could help avoid catastrophic overfitting. The author further proposed Noise-FGSM, utilizing single-step FGSM and noise-augmented samples to generate adversarial examples for training. Empirical studies showed the superiority of N-FGSM both in terms of performance and speed.