Towards Physical, Imperceptible Adversarial Attacks via Adversarial Programs

1 INTRODUCTION . Adversarial examples were originally defined as imperceptible perturbations added to correctly classified inputs which cause a deep neural network ( DNN ) to misclassify ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) . Since then , many works have shown how to generate imperceptible adversarial examples ( e.g. , Carlini & Wagner ( 2017 ) ; Moosavi-Dezfooli et al . ( 2016 ) ; Madry et al . ( 2018 ) ) . A main motivation of this line of works is to expose the level of vulnerability of DNN-based systems . However , these attacks often can not be executed in the physical world , for example against DNNs used by autonomous vehicles ( Lu et al. , 2017 ) . This gave rise to works that focus on physical attacks , executed by adding patches or stickers to objects or cameras ( Brown et al. , 2017 ; Eykholt et al. , 2018 ; Li et al. , 2019 ) or 3D-printed objects ( Athalye et al. , 2018 ; Sharif et al. , 2018 ) . While 3D-printing enables the attacker to generate imperceptible adversarial examples , it is not the case for patch/sticker attacks . These attacks incur highly visible perturbations . However , in some settings ( e.g. , traffic sign attacks ) , patch attacks are easier to execute than 3D-printing attacks . This raises the question : is it possible to generate adversarial examples with imperceptible patches ? We consider a new space of targeted attacks expressed as patch sequences . The patches we consider are of rectangular shape and uniformly perturb their associated pixels according to scale and shift factors . Given an input , a DNN , and a target label , the attacker ’ s goal is to compute a patch sequence resulting in a targeted adversarial example of minimum distance to the original input . That is , the attacker chooses the patches ’ position , dimensions , perturbation factors , and the number of patches such that the output of the patch sequence is an imperceptible adversarial example . Because the goal is to generate a sequence of patches , each is constrained by a structure , it is natural to view each patch as a program instruction and the overall problem as an instance of Syntax-guided Synthesis ( SyGuS ) ( Alur et al. , 2013 ) , where a high-level requirement ( e.g. , computing an adversarial example ) has to be implemented in a given programming language ( e.g. , the patch sequence ) . A common challenge of SyGuS instances is looking for a solution ( i.e. , a program ) in a discrete space . Unlike common SyGuS instances , our instructions ( i.e. , patches ) are defined also over continuous variables ( the perturbation factors ) . Thus , our setting is not easily amenable to standard SyGuS approaches , neither to standard numerical optimization approaches ( because of the discrete variables ) . We propose APSyn , an adversarial program synthesizer , to compute patch sequences . APSyn builds on enumerative program synthesis which enumerates programs from the shortest to the longest until obtaining a solution ( Udupa et al. , 2013 ) . This approach guarantees to return the shortest solution among all solutions , which is desirable when aiming at simplifying the physical execution of the attack . While enumerative synthesis is useful , it is not sufficient by its own , as there are infinitely many ways to instantiate a patch ( up to computer representation ) . To search for the optimal patch parameters , we leverage the loss proposed by Carlini & Wagner ( 2017 ) . To optimize this loss for patch sequences , we propose a gradient estimate for the discrete variables using zeroth order optimization . We further allow discrete variables to take real values to avoid skipping optimal solutions . Our evaluation results for MNIST , Fashion-MNIST and CIFAR-10 show that the success rate of APSyn is at least 96 % , when allowed up to five patches . The L2 distortion of these attacks is on average greater by up to 1.2 than the C & W ’ s L2 distortion , while the L0 distortion is smaller by at least 3x and up to 10x compared to the C & W ’ s attack . We further show that APSyn produces attacks which generalize better : at least 2.7x and up to 23x more than the C & W attack . We believe the higher generalization rate stems from the structure imposed on our attacks . For ImageNet , APSyn ’ s success rate is 92 % when allowed up to ten patches . Figure 1 shows several attacks that APSyn synthesized along with visualizations of the patch sequences . To conclude , our main contributions are : • A new class of adversarial attacks consisting of multi-patch sequences . • An effective algorithm to compute multi-patch attacks combining enumerative synthesis and numerical optimization . The latter relies on gradient estimation of the discrete factors . • An extensive evaluation of our approach on four datasets . Results show that our attack is competitive with the C & W attack , in terms of the success rate and L2 distortion , but has significantly lower L0 distortion and is thus easier to physically execute . 2 ADVERSARIAL PROGRAMS . In this section , we provide the terms and notation used throughout the paper and define our problem . We are given a classifier N : [ 0 , 1 ] d1×d2 → [ 0 , 1 ] C , mapping a two-dimensional image to a probability vector over C classes . We note that all our definitions extend to colored images by considering each color channel separately . We assume a white-box access to the classifier . We denote by class ( N ( x ) ) the classification of input x by N , i.e. , the index with the maximal probability . We are also given an input x with class c , which N classifies correctly , i.e. , class ( N ( x ) ) = c , and a target adversarial label t ̸= c. The goal of the attack is to compute an input x′ which is similar to x but is classified as t. Commonly , this is phrased as constrained optimization , where the goal is to compute a perturbation matrix M ∈ [ 0 , 1 ] d1×d2 such that the adversarial example x′ = x +M is classified as t ( class ( N ( x′ ) ) = t ) and the p-norm of the perturbation matrix ||M ||p is minimal . In this work , we aim at adding structure to adversarial attacks with the goal of obtaining attacks which are easier to physically execute , are more explainable , and generalize better . We draw inspiration from program synthesis where the goal is automatically generate a program over a given programming language satisfying a set of requirements ( Gulwani et al. , 2017 ) . In particular , we view our problem as an instance of Syntax-guided Synthesis ( SyGuS ) ( Alur et al. , 2013 ) , where the goal is to implement a high-level specification in a given programming language , typically constrained and small . Having a program has various advantages : it can be used on other inputs , the implementation can provide an alternative characterization of the user intent , or even provide a more efficient implementation than the user initially had . These applications are useful within our scope : our adversarial programs can be executed on other inputs in a negligible time and our programs expose the ( almost imperceptible ) shapes that cause the DNN to change its decision . Adversarial programs We introduce a programming language for adversarial multi-patch attacks . Our language consists of sequences of patch instructions from a family of patches I . Our patch family generalizes the perturbation matrices discussed before to scaling and shift perturbations over rectangular masks . A rectangular mask is a tuple consisting of a starting and ending position p = ( rs , cs , re , ce ) capturing a single rectangle in the input . A patch instruction ( or simply , a patch ) Iα , β , p ∈ I is associated with a mask p , scale factor α , and a shift factor β , and takes the form : Iα , β , p ( y ) = y → lambda v : ( α · v + β ) if idx ( v ) ∈ p where α , β ∈ R and p ∈ ( { 1 , . . . , d1 } × { 1 , . . . , d2 } ) 2 . Given y ∈ [ 0 , 1 ] d1×d2 , Iα , β , p ( y ) maps every value v in y to α · v + β , if its index is inside the mask . Values outside the mask remain as are . For value v with idx ( v ) = ( rv , cv ) and mask p = ( rs , cs , re , ce ) , we say v ’ s index is inside p if rs ≤ rv < re and cs ≤ cv < ce . Problem definition Given a network N , an input-output pair ( x , c ) , a target class t , and a program length k , our goal is to compute a program P over I with k or fewer patches such that N classifies P ( x ) as t and P ( x ) is as close as possible to x with respect to the L2 norm : min ||P ( x ) − x||2 s.t . P ( x ) = Ik′ ( . . . ( I1 ( x ) ) ) I1 , . . . , Ik′ ∈ I , k′ ≤ k class ( N ( P ( x ) ) ) = t Note that we allow patches to overlap , which may lead to perturbation compositions . Designing a solution If we set aside the constraint that P is a program over I , a common approach to solve the above constrained optimization is by relaxing the attack constraint . For example , Carlini & Wagner ( 2017 ) propose to relax it to a term which is then minimized along with the norm : L ( P ) = max ( max i ̸=t N ( P ( x ) ) i −N ( P ( x ) ) t , 0 ) + λ · ||P ( x ) − x||2 where λ is a hyper-parameter balancing the two optimization goals . If P ( x ) were differentiable , gradient descent could minimize this loss . If the size of I were relatively small , enumerative search would have been feasible . However , neither of these hold . We thus approach this search by combining the two ideas . First , we introduce a gradient estimation approach for the discrete variables , and then let Adam ( Kingma & Ba , 2015 ) use it to minimize L ( P ) . Second , to compute a shortest patch sequence and search more efficiently , we employ enumerative synthesis and gradually synthesize the adversarial program ’ s patches . We explain these steps in the next two sections . 3 GRADIENT ESTIMATION OF THE PATCHES ’ DISCRETE PARAMETERS . In this section , we explain our approach to search for an optimal patch attack consisting of exactly k patches from I . Unlike previous patch attacks , our approach jointly optimizes over all the patches ’ parameters – position , dimensions , and perturbation factors . Since we allow patches to overlap , this optimization may lead to complex patterns , despite the rectangular shape of the patches . Our approach builds on the C & W attack , which minimizes the loss defined in the previous section . Since our attacks involve discrete variables , to numerically optimize this loss , we define “ gradients ” for these variables . The idea is to estimate the partial derivatives of the discrete variables using zeroth order optimization . We note that we do not claim that the partial derivatives exist , but rather that estimating them can guide the optimizer how to change a discrete variable so that our loss decreases . Gradient estimation Given a program P , an input x , and the target label t , the partial derivatives of the real-valued variables ( the scale and shift parameters α , β ) are computed as usual . The partial derivative of a discrete variable w is estimated by adapting the standard definition of the derivative . As a first step , we estimate the derivative in each direction : L ( P ) +w ≜ ∂L ( P ) + ∂w = limh→0+ L ( P [ w → w + h ] ) − L ( P ) h ∼= L ( P [ w → w + 1 ] ) − L ( P ) . The last step is obtained by replacing h with the closest discrete value to 0 . The notation P [ w → w + h ] denotes the program identical to P except that w is increased by h. Similarly , L ( P ) −w ∼= L ( P ) − L ( P [ w → w − 1 ] ) . These “ derivatives ” simulate the role of the gradient in optimization . If L ( P ) +w and L ( P ) −w are negative , then increasing w will decrease the loss and thus we define the gradient as L ( P ) +w . If L ( P ) +w and L ( P ) −w are positive , then decreasing w will decrease the loss , and thus we define the gradient as L ( P ) −w . If L ( P ) +w is negative and L ( P ) −w is positive , we define the gradient based on their magnitude ; if L ( P ) +w is positive and L ( P ) −w is negative , the gradient is zero . Optimization Using this estimation , we can minimize our loss using Adam ( Kingma & Ba , 2015 ) and Hill Climbing . Each optimization step begins with an optimization of the discrete variables followed by a standard , joint , Adam step for the continuous variables . For the discrete variables , we perform Hill Climbing and Adam . That is , for each discrete variable , we separately estimate its gradient and update the variable according to Adam ’ s update steps . If the discrete variable is assigned an out-of-bounds value , its previous value is restored . We note that the optimizer has two optimization steps : ηc for the continuous variables and ηd for the discrete variables . In practice , it is best that ηd ≈ 100 ·ηc . To reduce execution times , the optimization terminates if within T iterations , the loss has decreased by less than p % , where T and p are hyper-parameters . At the end , our adapted Adam optimizer returns a program with the optimized continuous and discrete parameters . Real-value relaxation Although patches ’ position and dimensions are discrete variables , we observe that it is better to allow them to take real values during optimization . This relaxation enables the optimizer to use small step sizes , and thereby reach solutions which may have been missed if the variables would only take discrete values . With this relaxation , the discrete variables are rounded whenever P is invoked on an input ( as part of the optimization or when executing the attack ) .

Summary: This paper proposes APSyn to generate a sequence of imperceptible adversarial patches for a physical and imperceptible attack against image classification models. This attack is a combination of physical patch attacks and conventional imperceptible perturbation attacks. In APSyn, the authors formulate the attack problem as adversarial program synthesis and use gradient-based optimization to search for the optimal solution (a method for estimating the gradient for discrete values is also proposed). The experiments on small subsets of the MNIST, Fashion-MNIST, CIFAR-10, and ImageNet demonstrate that the proposed approach can achieve smaller L_0 distortion and larger L_2 distortion compared to the C&W attack.