ADef: an Iterative Algorithm to Construct Adversarial Deformations

1 INTRODUCTION . In a first observation in Szegedy et al . ( 2013 ) it was found that deep neural networks exhibit unstable behavior to small perturbations in the input . For the task of image classification this means that two visually indistinguishable images may have very different outputs , resulting in one of them being misclassified even if the other one is correctly classified with high confidence . Since then , a lot of research has been done to investigate this issue through the construction of adversarial examples : given a correctly classified image x , we look for an image y which is visually indistinguishable from x but is misclassified by the network . Typically , the image y is constructed as y = x+ r , where r is an adversarial perturbation that is supposed to be small in a suitable sense ( normally , with respect to an ` p norm ) . Several algorithms have been developed to construct adversarial perturbations , see Goodfellow et al . ( 2014 ) ; Moosavi Dezfooli et al . ( 2016 ) ; Kurakin et al . ( 2017b ) ; Madry et al . ( 2018 ) ; Carlini & Wagner ( 2017b ) and the review paper Akhtar & Mian ( 2018 ) . Even though such pathological cases are very unlikely to occur in practice , their existence is relevant since malicious attackers may exploit this drawback to fool classifiers or other automatic systems . Further , adversarial perturbations may be constructed in a black-box setting ( i.e. , without knowing the architecture of the DNN but only its outputs ) ( Papernot et al. , 2017 ; Moosavi-Dezfooli et al. , 2017 ) and also in the physical world ( Kurakin et al. , 2017b ; Athalye & Sutskever , 2017 ; Brown et al. , 2017 ; Sharif et al. , 2016 ) . This has motivated the investigation of defenses , i.e. , how to make the network invulnerable to such attacks , see Kurakin et al . ( 2017a ) ; Carlini & Wagner ( 2017a ) ; Madry et al . ( 2018 ) ; Tramèr et al . ( 2018 ) ; Wong & Kolter ( 2018 ) ; Raghunathan et al . ( 2018 ) ; Athalye et al . ( 2018 ) ; Kannan et al . ( 2018 ) . In most cases , adversarial examples are artificially created and then used to retrain the network , which becomes more stable under these types of perturbations . Most of the work on the construction of adversarial examples and on the design of defense strategies has been conducted in the context of small perturbations r measured in the ` ∞ norm . However , this is not necessarily a good measure of image similarity : e.g. , for two translated images x and y , the norm of x−y is not small in general , even though x and y will look indistinguishable if the translation is small . Several papers have investigated the construction of adversarial perturbations not designed for norm proximity ( Rozsa et al. , 2016 ; Sharif et al. , 2016 ; Brown et al. , 2017 ; Engstrom et al. , 2017 ; Xiao et al. , 2018 ) . In this work , we build up on these ideas and investigate the construction of adversarial deformations . In other words , the misclassified image y is not constructed as an additive perturbation y = x + r , but as a deformation y = x ◦ ( id + τ ) , where τ is a vector field defining the transformation . In this case , the similarity is not measured through a norm of y−x , but instead through a norm of τ , which quantifies the deformation between y and x . We develop an efficient algorithm for the construction of adversarial deformations , which we call ADef . It is based on the main ideas of DeepFool ( Moosavi Dezfooli et al. , 2016 ) , and iteratively constructs the smallest deformation to misclassify the image . We test the procedure on MNIST ( LeCun ) ( with convolutional neural networks ) and on ImageNet ( Russakovsky et al. , 2015 ) ( with Inception-v3 ( Szegedy et al. , 2016 ) and ResNet-101 ( He et al. , 2016 ) ) . The results show that ADef can succesfully fool the classifiers in the vast majority of cases ( around 99 % ) by using very small and imperceptible deformations . We also test our adversarial attacks on adversarially trained networks for MNIST . Our implementation of the algorithm can be found at https : //gitlab.math . ethz.ch/tandrig/ADef . The results of this work have initially appeared in the master ’ s thesis Gauksson ( 2017 ) , to which we refer for additional details on the mathematical aspects of this construction . While writing this paper , we have come across Xiao et al . ( 2018 ) , in which a similar problem is considered and solved with a different algorithm . Whereas in Xiao et al . ( 2018 ) the authors use a second order solver to find a deforming vector field , we show how a first order method can be formulated efficiently and justify a smoothing operation , independent of the optimization step . We report , for the first time , success rates for adversarial attacks with deformations on ImageNet . The topic of deformations has also come up in Jaderberg et al . ( 2015 ) , in which the authors introduce a class of learnable modules that deform inputs in order to increase the performance of existing DNNs , and Fawzi & Frossard ( 2015 ) , in which the authors introduce a method to measure the invariance of classifiers to geometric transformations . 2 ADVERSARIAL DEFORMATIONS . 2.1 ADVERSARIAL PERTURBATIONS . Let K be a classifier of images consisting of P pixels into L ≥ 2 categories , i.e . a function from the space of images X = RcP , where c = 1 ( for grayscale images ) or c = 3 ( for color images ) , and into the set of labels L = { 1 , . . . , L } . Suppose x ∈ X is an image that is correctly classified by K and suppose y ∈ X is another image that is imperceptible from x and such that K ( y ) 6= K ( x ) , then y is said to be an adversarial example . The meaning of imperceptibility varies , but generally , proximity in ` p-norm ( with 1 ≤ p ≤ ∞ ) is considered to be a sufficient substitute . Thus , an adversarial perturbation for an image x ∈ X is a vector r ∈ X such that K ( x+ r ) 6= K ( x ) and ‖r‖p is small , where ‖r‖p = cP∑ j=1 |rj |p 1/p if 1 ≤ p < ∞ , and ‖r‖∞ = maxj=1 , ... , cP |rj | . ( 1 ) Given such a classifier K and an image x , an adversary may attempt to find an adversarial example y by minimizing ‖x− y‖p subject to K ( y ) 6= K ( x ) , or even subject to K ( y ) = k for some target label k 6= K ( x ) . Different methods for finding minimal adversarial perturbations have been proposed , most notably FGSM ( Goodfellow et al. , 2014 ) and PGD ( Madry et al. , 2018 ) for ` ∞ , and the DeepFool algorithm ( Moosavi Dezfooli et al. , 2016 ) for general ` p-norms . 2.2 DEFORMATIONS . Instead of constructing adversarial perturbations , we intend to fool the classifier by small deformations of correctly classified images . Our procedure is in the spirit of the DeepFool algorithm . Before we explain it , let us first clarify what we mean by a deformation of an image . The discussion is at first more intuitive if we model images as functions ξ : [ 0 , 1 ] 2 → Rc ( with c = 1 or c = 3 ) instead of discrete vectors x in RcP . In this setting , perturbing an image ξ : [ 0 , 1 ] 2 → Rc corresponds to adding to it another function ρ : [ 0 , 1 ] 2 → Rc with a small Lp-norm . While any transformation of an image ξ can be written as a perturbation ξ + ρ , we shall restrict ourselves to a particular class of transformations . A deformation with respect to a vector field τ : [ 0 , 1 ] 2 → R2 is a transformation of the form ξ 7→ ξτ , where for any image ξ : [ 0 , 1 ] 2 → Rc , the image ξτ : [ 0 , 1 ] 2 → Rc is defined by ξτ ( u ) = ξ ( u+ τ ( u ) ) for all u ∈ [ 0 , 1 ] 2 , extending ξ by zero outside of [ 0 , 1 ] 2 . Deformations capture many natural image transformations . For example , a translation of the image ξ by a vector v ∈ R2 is a deformation with respect to the constant vector field τ = v. If v is small , the images ξ and ξv may look similar , but the corresponding perturbation ρ = ξv − ξ may be arbitrarily large in the aforementioned Lp-norms . Figure 1 shows three minor deformations , all of which yield large L∞-norms . In the discrete setting , deformations are implemented as follows . We consider square images of W ×W pixels and define the space of images to be X = ( RW×W ) c . A discrete vector field is a function τ : { 1 , . . . , W } 2 → R2 . In what follows we will only consider the set T of vector fields that do not move points on the grid { 1 , . . . , W } 2 outside of [ 1 , W ] 2 . More precisely , T : = { τ : { 1 , . . . , W } 2 → R2 | τ ( s , t ) + ( s , t ) ∈ [ 1 , W ] 2 for all s , t ∈ { 1 , . . . , W } } . An image x ∈ X can be viewed as the collection of values of a function ξ : [ 0 , 1 ] 2 → Rc on a regular grid { 1/ ( W+1 ) , . . . , W/ ( W+1 ) } 2 ⊆ [ 0 , 1 ] 2 , i.e . xs , t = ξ ( s/ ( W+1 ) , t/ ( W+1 ) ) for s , t = 1 , . . . , W . Such a function ξ can be computed by interpolating from x . Thus , the deformation of an image x with respect to the discrete vector field τ can be defined as the discrete deformed image xτ in X by xτs , t = ξ ( ( s , t ) + τ ( s , t ) W + 1 ) , s , t ∈ { 1 , . . . , W } . ( 2 ) It is not straightforward to measure the size of a deformation such that it captures the visual difference between the original image x and its deformed counterpart xτ . We will use the size of the corresponding vector field , τ , in the norm defined by ‖τ‖T = maxs , t=1 , ... , W ‖τ ( s , t ) ‖2 ( 3 ) as a proxy . The ` p-norms defined in ( 1 ) , adapted to vector fields , can be used as well . ( We remark , however , that none of these norms define a distance between x and xτ , since two vector fields τ , σ ∈ T with ‖τ‖T 6= ‖σ‖T may produce the same deformed image xτ = xσ . )

Summary: The paper introduces an iterative method to generate deformed images for adversarial attack. The core idea is to perturb the correctly classified image by iteratively applying small deformations, which are estimated based on a first-order approximation step, until the image is misclassified. Experimental results on several benchmark datasets (MNIST, ImageNet) and commonly used deep nets (CNN, ResNet, Inception) are reported to show the power of adversarial deformations.