Target Training: Tricking Adversarial Attacks to Fail

1 INTRODUCTION . Neural network classifiers are vulnerable to malicious adversarial samples that appear indistinguishable from original samples ( Szegedy et al. , 2013 ) , for example , an adversarial attack can make a traffic stop sign appear like a speed limit sign ( Eykholt et al. , 2018 ) to a classifier . An adversarial sample created using one classifier can also fool other classifiers ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) , even ones with different structure and parameters ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Papernot et al. , 2016b ; Tramèr et al. , 2017b ) . This transferability of adversarial attacks ( Papernot et al. , 2016b ) matters because it means that classifier access is not necessary for attacks . The increasing deployment of neural network classifiers in security and safety-critical domains such as traffic ( Eykholt et al. , 2018 ) , autonomous driving ( Amodei et al. , 2016 ) , healthcare ( Faust et al. , 2018 ) , and malware detection ( Cui et al. , 2018 ) makes countering adversarial attacks important . Gradient-based attacks use the classifier gradient to generate adversarial samples from nonadversarial samples . Gradient-based attacks minimize at the same time classifier adversarial loss and perturbation ( Szegedy et al. , 2013 ) , though attacks can relax this minimization to allow for bigger perturbations , for example in the Carlini & Wagner attack ( CW ) ( Carlini & Wagner , 2017c ) for κ > 0 , in the Projected Gradient Descent attack ( PGD ) ( Kurakin et al. , 2016 ; Madry et al. , 2017 ) , in FastGradientMethod ( FGSM ) ( Goodfellow et al. , 2014 ) . Other gradient-based adversarial attacks include DeepFool ( Moosavi-Dezfooli et al. , 2016 ) , Zeroth order optimization ( ZOO ) ( Chen et al. , 2017 ) , Universal Adversarial Perturbation ( UAP ) ( Moosavi-Dezfooli et al. , 2017 ) . Many recent proposed defenses have been broken ( Athalye et al. , 2018 ; Carlini & Wagner , 2016 ; 2017a ; b ; Tramer et al. , 2020 ) . They fall largely into these categories : ( 1 ) adversarial sample detection , ( 2 ) gradient masking and obfuscation , ( 3 ) ensemble , ( 4 ) customized loss . Detection defenses ( Meng & Chen , 2017 ; Ma et al. , 2018 ; Li et al. , 2019 ; Hu et al. , 2019 ) aim to detect , cor- rect or reject adversarial samples . Many detection defenses have been broken ( Carlini & Wagner , 2017b ; a ; Tramer et al. , 2020 ) . Gradient obfuscation is aimed at preventing gradient-based attacks from access to the gradient and can be achieved by shattering gradients ( Guo et al. , 2018 ; Verma & Swami , 2019 ; Sen et al. , 2020 ) , randomness ( Dhillon et al. , 2018 ; Li et al. , 2019 ) or vanishing or exploding gradients ( Papernot et al. , 2016a ; Song et al. , 2018 ; Samangouei et al. , 2018 ) . Many gradient obfuscation methods have also been successfully defeated ( Carlini & Wagner , 2016 ; Athalye et al. , 2018 ; Tramer et al. , 2020 ) . Ensemble defenses ( Tramèr et al. , 2017a ; Verma & Swami , 2019 ; Pang et al. , 2019 ; Sen et al. , 2020 ) have also been broken ( Carlini & Wagner , 2016 ; Tramer et al. , 2020 ) , unable to even outperform their best performing component . Customized attack losses defeat defenses ( Tramer et al. , 2020 ) with customized losses ( Pang et al. , 2020 ; Verma & Swami , 2019 ) but also , for example ensembles ( Sen et al. , 2020 ) . Even though it has not been defeated , Adversarial Training ( Kurakin et al. , 2016 ; Szegedy et al. , 2013 ; Madry et al. , 2017 ) assumes that the attack is known in advance and takes time to generate adversarial samples at every iteration . The inability of recent defenses to counter adversarial attacks calls for new kinds of defensive approaches . In this paper , we make the following major contributions : • We develop Target Training - a novel , white-box adversarial defense that converts untargeted gradient-based attacks into attacks targeted at designated target classes , from which correct classes are derived . Target Training is based on the minimization at the core of untargeted gradient-based adversarial attacks . • For all attacks that minimize perturbation , we eliminate the need to know the attack or to generate adversarial samples during training . • We show that Target Training withstands non-L∞ adversarial attacks without resorting to increased network capacity . With default accuracy of 84.3 % in CIFAR10 , Target Training achieves 86.6 % against the DeepFool attack , and 83.2 % against the CW-L2 ( κ=0 ) attack without using adversarial samples and against an adaptive attack aware of our defense . Against an adaptive CW-L2 ( κ=40 ) attack , we achieve 75.6 % while using adversarial samples . Our choice of low-capacity classifiers makes Target Training not withstand L∞ adaptive attacks , except for CW-L∞ ( κ=0 ) in MNIST . • We conclude that Adversarial Training might not be defending by populating sparse areas with samples , but by minimizing the same minimization that Target Training minimizes . 2 BACKGROUND AND RELATED WORK . Here , we present the state-of-the-art in adversarial attacks and defenses , as well as a summary . Notation A k-class neural network classifier that has θ parameters is denoted by a function f ( x ) that takes input x ∈ Rd and outputs y ∈ Rk , where d is the dimensionality and k is the number of classes . An adversarial sample is denoted by xadv . Classifier output is y , yi is the probability that the input belongs to class i. Norms are denoted as L0 , L2 and L∞ . 2.1 ADVERSARIAL ATTACKS . Szegedy et al . ( 2013 ) were first to formulate the generation of adversarial samples as a constrained minimization of the perturbation under an Lp norm . Because this formulation can be hard to solve , Szegedy et al . ( 2013 ) reformulated the problem as a gradient-based , two-term minimization of the weighted sum of perturbation , and classifier loss . For untargeted attacks , this minimization is : minimize c · ‖xadv − x‖22 + lossf ( xadv ) ( Minimization 1 ) subject to xadv ∈ [ 0 , 1 ] n where f is the classifier , lossf is classifier loss on adversarial input , and c a constant value evaluated in the optimization . Term ( 1 ) is a norm to ensure a small adversarial perturbation . Term ( 2 ) utilizes the classifier gradient to find adversarial samples that minimize classifier adversarial loss . Minimization 1 is the foundation for many gradient-based attacks , though many tweaks can and have been applied . Some attacks follow Minimization 1 implicitly ( Moosavi-Dezfooli et al. , 2016 ) , and others explicitly ( Carlini & Wagner , 2017c ) . The type of Lp norm in term ( 1 ) of the minimization also varies . For example the CW attack ( Carlini & Wagner , 2017c ) uses L0 , L2 and L∞ , whereas DeepFool ( Moosavi-Dezfooli et al. , 2016 ) uses the L2 norm . A special perturbation case is the Pixel attack by Su et al . ( 2019 ) which changes exactly one pixel . Some attacks even exclude term ( 1 ) from the Minimization 1 and introduce an external parameter to control perturbation . The FGSM attack by Goodfellow et al . ( 2014 ) , for example , uses an parameter , while the CW attack ( Carlini & Wagner , 2017c ) uses a κ confidence parameter . The Fast Gradient Sign Method by Goodfellow et al . ( 2014 ) is a simple , L∞-bounded attack that constructs adversarial samples by perturbing each input dimension in the direction of the gradient by a magnitude of : xadv = x+ · sign ( ∇xloss ( θ , x , y ) ) . The current strongest attack is CW ( Carlini & Wagner , 2017c ) . CW customizes Minimization 1 by passing c to the second term , and using it to tune the relative importance of the terms . With a further change of variable , CW obtains an unconstrained minimization problem that allows it to optimize directly through back-propagation . In addition , CW has a κ parameter for controlling the confidence of the adversarial samples . For κ > 0 and up to 100 , the CW attack allows for more perturbation in the adversarial samples it generates . The DeepFool attack by Moosavi-Dezfooli et al . ( 2016 ) follows the Minimization 1 implicitly . DeepFool ( Moosavi-Dezfooli et al. , 2016 ) looks at the smallest distance of a point from the classifier decision boundary as the minimum amount of perturbation needed to change its classification . DeepFool approximates the classifier with a linear one , estimates the distance from the linear boundary , and then takes steps in the direction of the closest boundary until an adversarial sample is found . Black-box attacks Black-box attacks assume no access to classifier gradients . Such attacks with access to output class probabilities are called score-based attacks , for example the ZOO attack ( Chen et al. , 2017 ) , a black-box variant of the CW attack ( Carlini & Wagner , 2017c ) . Attacks with access to only the final class label are decision-based attacks , for example the Boundary ( Brendel et al. , 2017 ) and the HopSkipJumpAttack ( Chen et al. , 2019 ) attacks . Multi-step attacks The PGD attack ( Kurakin et al. , 2016 ) is an iterative method with an α parameter that determines a step-size perturbation magnitude . PGD starts at a random point x0 , projects the perturbation on an Lp-ball B at each iteration : x ( j + 1 ) = ProjB ( x ( j ) + α · sign ( ∇xloss ( θ , x ( j ) , y ) ) . The BIM attack ( Kurakin et al. , 2016 ) applies FGSM ( Goodfellow et al. , 2014 ) iteratively with an α step . To find a universal perturbation , UAP ( Moosavi-Dezfooli et al. , 2017 ) iterates over the images and aggregates perturbations calculated as in DeepFool . 2.2 ADVERSARIAL DEFENSES . Adversarial Training . Adversarial Training ( Szegedy et al. , 2013 ; Kurakin et al. , 2016 ; Madry et al. , 2017 ) is one of the first and few , undefeated defenses . It defends by populating low probability , so-called blind spots ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ) with adversarial samples labelled correctly , redrawing boundaries . The drawback of Adversarial Training is that it needs to know the attack in advance , and it needs to generate adversarial samples during training . The Adversarial Training algorithm 2 in the Appendix is based on Kurakin et al . ( 2016 ) . Madry et al . ( 2017 ) formulate their defense as a robust optimization problem , and use adversarial samples to augment the training . Their solution however necessitates high-capacity classifiers - bigger models with more parameters . Detection defenses Such defenses detect adversarial samples implicitly or explicitly , then correct or reject them . So far , many detection defenses have been defeated . For example , ten diverse detection methods ( other network , PCA , statistical properties ) were defeated by attack loss customization ( Carlini & Wagner , 2017a ) ; Tramer et al . ( 2020 ) used attack customization against ( Hu et al. , 2019 ) ; attack transferability ( Carlini & Wagner , 2017b ) was used against MagNet by Meng & Chen ( 2017 ) ; deep feature adversaries ( Sabour et al. , 2016 ) against ( Roth et al. , 2019 ) . Gradient masking and obfuscation Many defenses that mask or obfuscate the classifier gradient have been defeated ( Carlini & Wagner , 2016 ; Athalye et al. , 2018 ) . Athalye et al . ( 2018 ) identify three types of gradient obfuscation : ( 1 ) Shattered gradients - incorrect gradients caused by nondifferentiable components or numerical instability , for example with multiple input transformations by Guo et al . ( 2018 ) . Athalye et al . ( 2018 ) counter such defenses with Backward Pass Differentiable Approximation . ( 2 ) Stochastic gradients in randomized defenses are overcome with Expectation Over Transformation ( Athalye et al. , 2017 ) by Athalye et al . Examples are Stochastic Activation Pruning ( Dhillon et al. , 2018 ) , which drops layer neurons based on a weighted distribution , and ( Xie et al. , 2018 ) which adds a randomized layer to the classifier input . ( 3 ) Vanishing or exploding gradients are used , for example , in Defensive Distillation ( DD ) ( Papernot et al. , 2016a ) which reduces the amplitude of gradients of the loss function . Other examples are PixelDefend ( Song et al. , 2018 ) and Defense-GAN ( Samangouei et al. , 2018 ) . Vanishing or exploding gradients are broken with parameters that avoid vanishing or exploding gradients ( Carlini & Wagner , 2016 ) . Complex defenses Defenses combining several approaches , for example ( Li et al. , 2019 ) which uses detection , randomization , multiple models and losses , can be defeated by focusing on the main defense components ( Tramer et al. , 2020 ) . ( Verma & Swami , 2019 ; Pang et al. , 2019 ; Sen et al. , 2020 ) are defeated ensemble defenses combined with numerical instability ( Verma & Swami , 2019 ) , regularization ( Pang et al. , 2019 ) , or mixed precision on weights and activations ( Sen et al. , 2020 ) .

Summary: This paper addresses the task of adversarial defense, particularly against untargeted attack. It starts from the observation that these attacks mostly minimize the perturbation and the classification loss, and proposes a new training strategy named Target Training. The method duplicate training examples with a special ground-truth label, to fool the adversarial attackers. Experiments are conducted on MNIST and CIFAR10 under several attacks.