Walking on the Edge: Fast, Low-Distortion Adversarial Examples

1 INTRODUCTION . Adversarial examples ( Szegedy et al. , 2013 ) are small , usually imperceptible perturbations of images or other data ( Carlini & Wagner , 2018 ) that can arbitrarily modify a classifier ’ s prediction . They have been extended to other tasks like object detection or semantic segmentation ( Xie et al. , 2017 ) , and image retrieval ( Li et al. , 2018 ; Tolias et al. , 2019 ) . They are typically generated in a white-box setting , where the attacker has full access to the classifier model and uses gradient signals through the model to optimize for the perturbation . They are becoming increasingly important because they reveal the sensitivity of neural networks to their input ( Simon-Gabriel et al. , 2018 ; Fawzi et al. , 2016 ; Amsaleg et al. , 2017 ) including trivial cases ( Azulay & Weiss , 2018 ; Engstrom et al. , 2017 ) and they easily transfer between different models ( Moosavi-Dezfooli et al. , 2016a ; Tramèr et al. , 2017b ) . Adversarial examples are typically evaluated by probability of success and distortion . In many cases , white-box attacks have probability of success near one , then only distortion matters , as a ( weak ) measure of imperceptibility and also of the ease with which adversarial samples can be detected . The speed of an attack is less frequently discussed . The fast single-step FGSM attack ( Goodfellow et al. , 2014 ) produces high-distortion examples where adversarial patterns can easily be recognized . At the other extreme , the Carlini & Wagner ( C & W ) attack ( Carlini & Wagner , 2017 ) , considered state of the art , is notoriously expensive . Decoupling direction and norm ( DDN ) ( Rony et al. , 2018 ) has recently shown impressive progress in distortion but mostly in speed . Speed becomes more important when considering adversarial training ( Goodfellow et al. , 2014 ) . This defense , where adversarial examples are used for training , was in fact introduced in the same work as FGSM . The latter remains the most common choice for generating those examples because of its speed . However , unless a powerful iterative attack is used ( Madry et al. , 2017 ) , adversarial training is easily broken ( Tramèr et al. , 2017a ) . In this work , we investigate in more depth the speed-distortion trade-off in the regime of probability of success near one . We observe that iterative attacks often oscillate across the classification boundary , taking long time to stabilize . We introduce a new attack that rather walks along the boundary . As a result , we improve the state of the art in distortion while keeping iterations at a minimum . Illustrating the attacks . To better understand how our attack works , we illustrate it qualitatively against a number of existing attacks in Fig . 1 . On this toy 2d classification problem , the class boundary and the path followed by the optimizer starting at input x can be easily visualized . PGD2 , an ` 2 version of I-FGSM ( Kurakin et al. , 2016 ) , a.k.a . PGD ( Madry et al. , 2017 ) , is controlled by a distortion target and eventually follows a path on a ball of radius centered at x ( cf . Fig . 1 ( a ) ) . Section 4 shows that varying is an effective yet expensive strategy . It can only be done for a limited set of values so that the optimal distortion target per image may only be found by luck . C & W ( Carlini & Wagner , 2017 ) depends on a parameter that controls the balance between distortion and classification loss . A low value may lead to failure . A higher value may indeed reach the optimal perturbation , but with oscillations across the class boundary ( cf . Fig . 1 ( b ) ) . Therefore , an expensive line search over is performed internally . DDN ( Rony et al. , 2018 ) ( cf . Fig . 1 ( c ) ) increases or decreases distortion on the fly depending on success and at the same time pointing towards the gradient direction . It arrives quickly near the optimal perturbation but still suffers from oscillations across the boundary . On the contrary , boundary projection ( BP ) , introduced in this work ( cf . Fig . 1 ( d ) ) , cares more about quickly reaching the boundary , not necessarily near the optimal solution , and then walks along the boundary , staying mostly in the adversarial ( red ) region . It therefore makes steady progress towards the solution rather than going back and forth . Our attack . Our key idea is that , once we reach the adversarial region near the boundary , the problem becomes optimization on a manifold ( Absil et al. , 2009 ) : in particular , minimization of the ` 2 distortion on a level set of the classification loss . When in the adversarial region , we project the distortion gradient on the tangent space of this manifold . We do this simply by targeting a particular reduction of the distortion while moving orthogonally to the gradient of the classification loss . Our benchmark . Quantization is another major issue in this literature . Most papers implicitly assume that the output of a white-box attack is a matrix where pixel values are real numbers in [ 0 ; 1 ] . Rony et al . ( 2018 ) is one of the rare works where the output is a quantized . We agree with this definition of the problem . Indeed , an adversarial image is above all an image . The goal of an attacker is to publish images deluding the classifier ( for instance on the web ) , and publishing implies compliance with pixels encoded in bytes . Contributions . We make the following contributions . To our knowledge , we are the first to 1 . Study optimization on the manifold of the classification boundary for an adversarial attack , providing an analysis under a number of constraints , such as staying on the tangent space of the manifold and reaching a distortion target . 2 . Investigate theoretically and experimentally the quantization impact on the perturbation . 3 . Achieve at the same speed as I-FGSM ( Kurakin et al. , 2016 ) ( 20 iterations ) and under the constraint of a quantization , less distortion than state-of-the-art attacks including DDN , which needs 100 iterations on ImageNet . 2 PROBLEM , BACKGROUND AND RELATED WORK . 2.1 PROBLEM FORMULATION . Preliminaries . Let X : = f0 ; ; : : : ; 1 ; 1gn with : = 1= ( L 1 ) denote the set of grayscale images of n pixels quantized to L levels , and let X̂ : = [ 0 ; 1 ] n denote the corresponding real-valued images . An image of more than one color channels is treated independently per channel ; in this case n stands for the product of pixels and channels . A classifier f : X̂ ! Rk maps an image x to a vector f ( x ) 2 Rc+ representing probabilities per class over c given classes . The parameters of the classifier are not shown here because they remain fixed in this work . The classifier prediction : X̂ ! [ c ] : = f1 ; : : : ; cg maps x to the class label having the maximum probability : ( x ) : = argmax k2 [ c ] f ( x ) k : ( 1 ) If a true label t 2 [ c ] is known , the prediction is correct if ( x ) = t. Problem . Let x 2 X be a given image with known true label t. An adversarial example y 2 X is an image such that the distortion kx yk is small and the probability f ( y ) t is also small . This problem takes two forms : 1 . Target distortion , minimal probability : min y2X f ( y ) t ( 2 ) subject to kx yk ; ( 3 ) where is a given distortion target . The performance is then measured by the probability of success Psuc : = P ( ( y ) 6= t ) as a function of . 2 . Target success , minimal distortion : min y2X kx yk ( 4 ) subject to ( y ) 6= t : ( 5 ) The performance is then measured by the expected distortion D : = E ( kx yk ) . This work focuses on the second form , but we present example attacks of both forms in section 2.2 . Untargeted attack . The constraint ( y ) 6= t in ( 5 ) is referred to as an untargeted attack , meaning that y is misclassified regardless of the actual prediction . As an alternative , a targeted attack requires that the prediction ( y ) = t0 is a target label t0 6= t. We focus on the former . Loss function . We focus on a white-box attack in this work . Such an attack is specific to f , which is public . In this setting , attacks typically rely on exploiting the gradient of some loss function , using variants of gradient descent . A classification loss is defined on the probability vector p = f ( y ) with respect to the true label t. For an untargeted attack , this is typically the negative of cross-entropy ` ( p ; t ) : = log pt : ( 6 ) We should warn that , while the cross-entropy is appropriate for bringing examples into the region of class t during classifier training , its negative ( 6 ) is in general not appropriate for pulling them out during an attack . This is because this function is mostly flat in the class region . A common solution is to normalize the gradient of ` ( Goodfellow et al. , 2014 ; Rony et al. , 2018 ) , assuming it is nonzero . We consider more options in this work . A targeted attack on the other hand may use log pt0 , which works fine because it brings examples into class t0 region . Distortion . This work focuses on the 2-norm k k as a measure of distortion . Alternatives like 1-norm and 1-norm are also common ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ) . It is known that none is appropriate for measuring the imperceptibility of adversarial attacks , while more sophisticated measures like structural similarity ( SSIM ) ( Wang et al. , 2004 ) are limited too ( Sharif et al. , 2018 ) . Measuring imperceptibility is arguably as difficult as classification itself . Integral constraint . The constraint y 2 X in ( 3 ) and ( 5 ) is typically relaxed to y 2 X̂ during optimization . Some works conclude the attack by loosely quantizing the optimal solution onto X , typically by truncation towards zero . To our knowledge , DDN ( Rony et al. , 2018 ) is the only work to do rounding instead , and at the end of each iteration . Quantization is becoming an important issue in adversarial examples because the distortions achieved in recent papers are so small that quantization impacts a lot the perturbations . Appendix A provides a more in-depth study of the impact of the quantization .

Summary: This paper introduces a parameterized approach to generate adversarial samples by balancing the speed-distortion trade-off. The method first tries to reach the boundary of classes in the classifier space, then walks on the classifier manifold to find adversarial samples that make the classifier to fail in prediction while minimizing the level of distortion in the sample. Having a limited number of iterations, the method reduces the fluctuations around the boundary and paves the classification manifold.