Towards Physical, Imperceptible Adversarial Attacks via Adversarial Programs

1 INTRODUCTION . Adversarial examples were originally defined as imperceptible perturbations added to correctly classified inputs which cause a deep neural network ( DNN ) to misclassify ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) . Since then , many works have shown how to generate imperceptible adversarial examples ( e.g. , Carlini & Wagner ( 2017 ) ; Moosavi-Dezfooli et al . ( 2016 ) ; Madry et al . ( 2018 ) ) . A main motivation of this line of works is to expose the level of vulnerability of DNN-based systems . However , these attacks often can not be executed in the physical world , for example against DNNs used by autonomous vehicles ( Lu et al. , 2017 ) . This gave rise to works that focus on physical attacks , executed by adding patches or stickers to objects or cameras ( Brown et al. , 2017 ; Eykholt et al. , 2018 ; Li et al. , 2019 ) or 3D-printed objects ( Athalye et al. , 2018 ; Sharif et al. , 2018 ) . While 3D-printing enables the attacker to generate imperceptible adversarial examples , it is not the case for patch/sticker attacks . These attacks incur highly visible perturbations . However , in some settings ( e.g. , traffic sign attacks ) , patch attacks are easier to execute than 3D-printing attacks . This raises the question : is it possible to generate adversarial examples with imperceptible patches ? We consider a new space of targeted attacks expressed as patch sequences . The patches we consider are of rectangular shape and uniformly perturb their associated pixels according to scale and shift factors . Given an input , a DNN , and a target label , the attacker ’ s goal is to compute a patch sequence resulting in a targeted adversarial example of minimum distance to the original input . That is , the attacker chooses the patches ’ position , dimensions , perturbation factors , and the number of patches such that the output of the patch sequence is an imperceptible adversarial example . Because the goal is to generate a sequence of patches , each is constrained by a structure , it is natural to view each patch as a program instruction and the overall problem as an instance of Syntax-guided Synthesis ( SyGuS ) ( Alur et al. , 2013 ) , where a high-level requirement ( e.g. , computing an adversarial example ) has to be implemented in a given programming language ( e.g. , the patch sequence ) . A common challenge of SyGuS instances is looking for a solution ( i.e. , a program ) in a discrete space . Unlike common SyGuS instances , our instructions ( i.e. , patches ) are defined also over continuous variables ( the perturbation factors ) . Thus , our setting is not easily amenable to standard SyGuS approaches , neither to standard numerical optimization approaches ( because of the discrete variables ) . We propose APSyn , an adversarial program synthesizer , to compute patch sequences . APSyn builds on enumerative program synthesis which enumerates programs from the shortest to the longest until obtaining a solution ( Udupa et al. , 2013 ) . This approach guarantees to return the shortest solution among all solutions , which is desirable when aiming at simplifying the physical execution of the attack . While enumerative synthesis is useful , it is not sufficient by its own , as there are infinitely many ways to instantiate a patch ( up to computer representation ) . To search for the optimal patch parameters , we leverage the loss proposed by Carlini & Wagner ( 2017 ) . To optimize this loss for patch sequences , we propose a gradient estimate for the discrete variables using zeroth order optimization . We further allow discrete variables to take real values to avoid skipping optimal solutions . Our evaluation results for MNIST , Fashion-MNIST and CIFAR-10 show that the success rate of APSyn is at least 96 % , when allowed up to five patches . The L2 distortion of these attacks is on average greater by up to 1.2 than the C & W ’ s L2 distortion , while the L0 distortion is smaller by at least 3x and up to 10x compared to the C & W ’ s attack . We further show that APSyn produces attacks which generalize better : at least 2.7x and up to 23x more than the C & W attack . We believe the higher generalization rate stems from the structure imposed on our attacks . For ImageNet , APSyn ’ s success rate is 92 % when allowed up to ten patches . Figure 1 shows several attacks that APSyn synthesized along with visualizations of the patch sequences . To conclude , our main contributions are : • A new class of adversarial attacks consisting of multi-patch sequences . • An effective algorithm to compute multi-patch attacks combining enumerative synthesis and numerical optimization . The latter relies on gradient estimation of the discrete factors . • An extensive evaluation of our approach on four datasets . Results show that our attack is competitive with the C & W attack , in terms of the success rate and L2 distortion , but has significantly lower L0 distortion and is thus easier to physically execute . 2 ADVERSARIAL PROGRAMS . In this section , we provide the terms and notation used throughout the paper and define our problem . We are given a classifier N : [ 0 , 1 ] d1×d2 → [ 0 , 1 ] C , mapping a two-dimensional image to a probability vector over C classes . We note that all our definitions extend to colored images by considering each color channel separately . We assume a white-box access to the classifier . We denote by class ( N ( x ) ) the classification of input x by N , i.e. , the index with the maximal probability . We are also given an input x with class c , which N classifies correctly , i.e. , class ( N ( x ) ) = c , and a target adversarial label t ̸= c. The goal of the attack is to compute an input x′ which is similar to x but is classified as t. Commonly , this is phrased as constrained optimization , where the goal is to compute a perturbation matrix M ∈ [ 0 , 1 ] d1×d2 such that the adversarial example x′ = x +M is classified as t ( class ( N ( x′ ) ) = t ) and the p-norm of the perturbation matrix ||M ||p is minimal . In this work , we aim at adding structure to adversarial attacks with the goal of obtaining attacks which are easier to physically execute , are more explainable , and generalize better . We draw inspiration from program synthesis where the goal is automatically generate a program over a given programming language satisfying a set of requirements ( Gulwani et al. , 2017 ) . In particular , we view our problem as an instance of Syntax-guided Synthesis ( SyGuS ) ( Alur et al. , 2013 ) , where the goal is to implement a high-level specification in a given programming language , typically constrained and small . Having a program has various advantages : it can be used on other inputs , the implementation can provide an alternative characterization of the user intent , or even provide a more efficient implementation than the user initially had . These applications are useful within our scope : our adversarial programs can be executed on other inputs in a negligible time and our programs expose the ( almost imperceptible ) shapes that cause the DNN to change its decision . Adversarial programs We introduce a programming language for adversarial multi-patch attacks . Our language consists of sequences of patch instructions from a family of patches I . Our patch family generalizes the perturbation matrices discussed before to scaling and shift perturbations over rectangular masks . A rectangular mask is a tuple consisting of a starting and ending position p = ( rs , cs , re , ce ) capturing a single rectangle in the input . A patch instruction ( or simply , a patch ) Iα , β , p ∈ I is associated with a mask p , scale factor α , and a shift factor β , and takes the form : Iα , β , p ( y ) = y → lambda v : ( α · v + β ) if idx ( v ) ∈ p where α , β ∈ R and p ∈ ( { 1 , . . . , d1 } × { 1 , . . . , d2 } ) 2 . Given y ∈ [ 0 , 1 ] d1×d2 , Iα , β , p ( y ) maps every value v in y to α · v + β , if its index is inside the mask . Values outside the mask remain as are . For value v with idx ( v ) = ( rv , cv ) and mask p = ( rs , cs , re , ce ) , we say v ’ s index is inside p if rs ≤ rv < re and cs ≤ cv < ce . Problem definition Given a network N , an input-output pair ( x , c ) , a target class t , and a program length k , our goal is to compute a program P over I with k or fewer patches such that N classifies P ( x ) as t and P ( x ) is as close as possible to x with respect to the L2 norm : min ||P ( x ) − x||2 s.t . P ( x ) = Ik′ ( . . . ( I1 ( x ) ) ) I1 , . . . , Ik′ ∈ I , k′ ≤ k class ( N ( P ( x ) ) ) = t Note that we allow patches to overlap , which may lead to perturbation compositions . Designing a solution If we set aside the constraint that P is a program over I , a common approach to solve the above constrained optimization is by relaxing the attack constraint . For example , Carlini & Wagner ( 2017 ) propose to relax it to a term which is then minimized along with the norm : L ( P ) = max ( max i ̸=t N ( P ( x ) ) i −N ( P ( x ) ) t , 0 ) + λ · ||P ( x ) − x||2 where λ is a hyper-parameter balancing the two optimization goals . If P ( x ) were differentiable , gradient descent could minimize this loss . If the size of I were relatively small , enumerative search would have been feasible . However , neither of these hold . We thus approach this search by combining the two ideas . First , we introduce a gradient estimation approach for the discrete variables , and then let Adam ( Kingma & Ba , 2015 ) use it to minimize L ( P ) . Second , to compute a shortest patch sequence and search more efficiently , we employ enumerative synthesis and gradually synthesize the adversarial program ’ s patches . We explain these steps in the next two sections . 3 GRADIENT ESTIMATION OF THE PATCHES ’ DISCRETE PARAMETERS . In this section , we explain our approach to search for an optimal patch attack consisting of exactly k patches from I . Unlike previous patch attacks , our approach jointly optimizes over all the patches ’ parameters – position , dimensions , and perturbation factors . Since we allow patches to overlap , this optimization may lead to complex patterns , despite the rectangular shape of the patches . Our approach builds on the C & W attack , which minimizes the loss defined in the previous section . Since our attacks involve discrete variables , to numerically optimize this loss , we define “ gradients ” for these variables . The idea is to estimate the partial derivatives of the discrete variables using zeroth order optimization . We note that we do not claim that the partial derivatives exist , but rather that estimating them can guide the optimizer how to change a discrete variable so that our loss decreases . Gradient estimation Given a program P , an input x , and the target label t , the partial derivatives of the real-valued variables ( the scale and shift parameters α , β ) are computed as usual . The partial derivative of a discrete variable w is estimated by adapting the standard definition of the derivative . As a first step , we estimate the derivative in each direction : L ( P ) +w ≜ ∂L ( P ) + ∂w = limh→0+ L ( P [ w → w + h ] ) − L ( P ) h ∼= L ( P [ w → w + 1 ] ) − L ( P ) . The last step is obtained by replacing h with the closest discrete value to 0 . The notation P [ w → w + h ] denotes the program identical to P except that w is increased by h. Similarly , L ( P ) −w ∼= L ( P ) − L ( P [ w → w − 1 ] ) . These “ derivatives ” simulate the role of the gradient in optimization . If L ( P ) +w and L ( P ) −w are negative , then increasing w will decrease the loss and thus we define the gradient as L ( P ) +w . If L ( P ) +w and L ( P ) −w are positive , then decreasing w will decrease the loss , and thus we define the gradient as L ( P ) −w . If L ( P ) +w is negative and L ( P ) −w is positive , we define the gradient based on their magnitude ; if L ( P ) +w is positive and L ( P ) −w is negative , the gradient is zero . Optimization Using this estimation , we can minimize our loss using Adam ( Kingma & Ba , 2015 ) and Hill Climbing . Each optimization step begins with an optimization of the discrete variables followed by a standard , joint , Adam step for the continuous variables . For the discrete variables , we perform Hill Climbing and Adam . That is , for each discrete variable , we separately estimate its gradient and update the variable according to Adam ’ s update steps . If the discrete variable is assigned an out-of-bounds value , its previous value is restored . We note that the optimizer has two optimization steps : ηc for the continuous variables and ηd for the discrete variables . In practice , it is best that ηd ≈ 100 ·ηc . To reduce execution times , the optimization terminates if within T iterations , the loss has decreased by less than p % , where T and p are hyper-parameters . At the end , our adapted Adam optimizer returns a program with the optimized continuous and discrete parameters . Real-value relaxation Although patches ’ position and dimensions are discrete variables , we observe that it is better to allow them to take real values during optimization . This relaxation enables the optimizer to use small step sizes , and thereby reach solutions which may have been missed if the variables would only take discrete values . With this relaxation , the discrete variables are rounded whenever P is invoked on an input ( as part of the optimization or when executing the attack ) .

Summary: The paper proposed a new adversarial perturbation by stacking imperceptible adversarial patches. Each adversarial patch could be regarded as a disturbance rectangle mask and contains learnable coordinates, scaling, and shifting factors. By applying an adversarial patch into the input sample (e.g., image), the values of input inside the mask will be scaled and shifted. These adversarial patches are stacked and generated by solving an adversarial program problem with zero-order optimization and numerical optimization.