Structured Adversarial Attack:  Towards General Implementation and Better Interpretability

1 INTRODUCTION Deep learning achieves exceptional successes in domains such as image recognition ( He et al. , 2016 ; Geifman & ElYaniv , 2017 ) , natural language processing ( Hinton et al. , 2012 ; Harwath et al. , 2016 ) , medical diagnostics ( Chen et al. , 2016 ; Shi et al. , 2018 ) and advanced control ( Silver et al. , 2016 ; Fu et al. , 2017 ) . Recent studies ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Nguyen et al. , 2015 ; Kurakin et al. , 2016 ; Carlini & Wagner , 2017 ) show that DNNs are vulnerable to adversarial attacks implemented by generating adversarial examples , i.e. , adding well-designed perturbations to original legal inputs . Delicately crafted adversarial examples can mislead a DNN to recognize them as any target image label , while the perturbations appears unnoticeable to human eyes . Adversarial attacks against ∗Equal contribution DNNs not only exist in theoretical models but also pose potential security threats to the real world ( Kurakin et al. , 2016 ; Evtimov et al. , 2017 ; Papernot et al. , 2017 ) . Several explanations are proposed to illustrate why there exist adversarial examples to DNNs based on hypotheses such as model linearity and data manifold ( Goodfellow et al. , 2014 ; Gilmer et al. , 2018 ) . However , little is known to their origins , and convincing explanations remain to be explored . Besides achieving the goal of ( targeted ) mis-classification , an adversarial example should be as “ similar ” to the original legal input as possible to be stealthy . Currently , the similarity is measured by the ` p norm ( p = 0 , 1 , 2 , ∞ ) of the added perturbation ( Szegedy et al. , 2013 ; Carlini & Wagner , 2017 ; Chen et al. , 2017b ; a ) , i.e. , ` p norm is being minimized when generating adversarial example . However , measuring the similarity between the original image and its adversarial example by ` p norm is neither necessary nor sufficient ( Sharif et al. , 2018 ) . Besides , no single measure can be perfect for human perceptual similarity ( Carlini & Wagner , 2017 ) and such adversarial attacks may fail to capture key information hidden in the input such as spatial structure or distribution . Spurred by that , this work implements a new attack model i.e. , structured attack ( StrAttack ) that imposes group sparsity on adversarial perturbations by extracting structures from the inputs . As shown in Fig . 1 , we find that StrAttack identifies minimally sufficient regions that make attacks successful , but without incurring extra pixel-level perturbation power . The major contributions are summarized as below . • ( Structure-driven attack ) This work is the first attempt towards exploring group-wise sparse structures when implementing adversarial attacks , but without losing ` p distortion performance when compared to state-of-the-art attacking methods . • ( Generality ) We show that the proposed attack model covers many norm-ball based attacks such as C & W ( Carlini & Wagner , 2017 ) and EAD ( Chen et al. , 2017a ) . • ( Efficient implementation ) We develop an efficient algorithm to generate structured adversarial perturbations by leveraging the alternating direction method of multipliers ( ADMM ) . We show that ADMM splits the original complex problem into subproblems , each of which can be solved analytically . Besides , we show that ADMM can further be used to refine an arbitrary adversarial attack under the fixed sparse structure . • ( Interpretability ) The generated adversarial perturbations demonstrate clear correlations and interpretations between original and target images . With the aid of adversarial saliency map ( Papernot et al. , 2016b ) and class activation map ( Zhou et al. , 2016 ) , we show that the obtained group-sparse adversarial patterns better shed light on the mechanisms of adversarial perturbations to fool DNNs . Related work Many works studied norm-ball constrained adversarial attacks . For example , FGM ( Goodfellow et al. , 2014 ) and IFGSM ( Kurakin et al. , 2017 ) attack methods were proposed to maximize the classification error subject to ` ∞-norm based distortion constraints . Moreover , L-BFGS ( Szegedy et al. , 2013 ) and C & W ( Carlini & Wagner , 2017 ) attacks found an adversarial example by minimizing its ` 2-norm distortion . By contrast , JSMA ( Papernot et al. , 2016b ) and one-pixel ( Su et al. , 2017 ) attacks attempted to generate adversarial examples by perturbing the minimum number of pixels , namely , minimizing the ` 0 norm of adversarial perturbations . Different from the above norm-ball constrained attacks , some works ( Karmon et al. , 2018 ; Brown et al. , 2017 ) crafted adversarial examples by adding noise patches . However , the resulting adversarial perturbations are no longer imperceptible to humans . Here we argue that imperceptibility could be important since it helps us to understand how/why DNNs are vulnerable to adversarial attacks while perturbing natural examples just by indistinguished adversarial noise . In the aforementioned norm-ball constrained adversarial attacks , two extremely opposite principles have been applied : C & W attack ( or ` ∞ attacks ) seeks the minimum image-level distortion but allows to modify all pixels ; one-pixel attack only perturbs a few pixels but suffers a high pixel-level distortion . Both attacking principles might lead to a high noise visibility due to perturbing too many pixels or perturbing a few pixels too much . In this work , we wonder if there exists a more effective attack that can be as successful as existing attacks but achieves a tradeoff between the perturbation power and the number of perturbed pixels . We will show that the proposed StrAttack is able to identify sparse perturbed regions that make attacks successful , but without incurring extra pixel-level perturbations . It is also worth mentioning that one-pixel attack has much lower attack success rate on ImageNet than C & W attack and StrAttack . In addition to adversarial attacks , many defense works have been proposed . Examples include defensive distillation ( Papernot et al. , 2016c ) that distills the original DNN and introduces temperature into the softmax layer , random mask ( Anonymous , 2019 ) that modifies the DNN structures by randomly removing certain neurons before training , adversarial training through enlarging the training dataset with adversarial examples , and robust adversarial training ( Madry et al. , 2017 ; Sinha et al. , 2018 ) through the min-max optimization . It is commonly known that the robust adversarial training method ensures the strongest defense performance against adversarial attacks on MNIST and CIFAR-10 . In this work , we will evaluate the effectiveness of StrAttack to three defense methods , a ) defensive distillation ( Papernot et al. , 2016c ) , b ) adversarial training via data augmentation ( Tramèr et al. , 2018 ) and c ) robust adversarial training ( Madry et al. , 2017 ) . Although the adversarial attack and defense have attracted an increasing amount of attention , the visual explanation on adversarial perturbations is less explored since the distortion power is minimized and the resulting adversarial effects become imperceptible to humans . The work ( Dong et al. , 2017 ) attempted to understand how the internal representations of DNNs are affected by adversarial examples . However , only an ensemble-based attack was considered , which fails to distinguish the effectiveness of different norm-ball constrained adversarial attacks . Unlike ( Dong et al. , 2017 ) , we employ the interpretability tools , adversarial saliency map ( ASM ) ( Papernot et al. , 2016b ) and class activation map ( CAM ) ( Zhou et al. , 2016 ) to measure the effectiveness of different attacks in terms of their interpretability . Here ASM provides sensitivity analysis for pixel-level perturbation ’ s impact on label classification , and CAM localizes class-specific image discriminative regions ( Xiao et al. , 2018 ) . We will show that the sparse adversarial pattern obtained by StrAttack offers a great interpretability through ASM and CAM compared with other norm-ball constrained attacks . 2 STRUCTURED ATTACK : EXPLORE GROUP STRUCTURES FROM IMAGES . In the section , we introduce the concept of StrAttack , motivated by the question : ‘ what possible structures could adversarial perturbations have to fool DNNs ? ’ Our idea is to divide an image into sub-groups of pixels and then penalize the corresponding group-wise sparsity . The resulting sparse groups encode minimally sufficient adversarial effects on local structures of natural images . Let ∆ ∈ RW×H×C be an adversarial perturbation added to an original image X0 , where W × H gives the spatial region , and C is the depth , e.g. , C = 3 for RGB images . To characterize the local structures of ∆ , we introduce a sliding maskM with stride S and size r × r × C. When S = 1 , the mask moves one pixel at a time ; When S = 2 , the mask jumps 2 pixels at a time while sliding . By adjusting the stride S and the mask size r , different group splitting schemes can be obtained . If S < r , the resulting groups will contain overlapping pixels . By contrast , groups will become non-overlapped when S = r. A sliding maskM finally divides ∆ into a set of groups { ∆Gp , q } for p ∈ [ P ] and q ∈ [ Q ] , where P = ( W − r ) /S + 1 , Q = ( H − r ) /S + 1 , and [ n ] denotes the integer set { 1 , 2 , . . . , n } . Given the groups { ∆Gp , q } , the group sparsity can be characterized through the following sparsity-inducing function ( Yuan & Lin , 2006 ; Bach et al. , 2012 ; Liu et al. , 2015 ) , motivated by the problem of group Lasso ( Yuan & Lin , 2006 ) : g ( ∆ ) = ∑P p=1 ∑Q q=1 ‖∆Gp , q‖2 , ( 1 ) where ∆Gp , q denotes the set of pixels of ∆ indexed by Gp , q , and ‖ · ‖2 is the ` 2 norm . We refer readers to Fig . A1 for an illustrative example of our concepts on groups and group sparsity . 3 STRUCTURED ADVERSARIAL ATTACK WITH ADMM . In this section , we start by proposing a general framework to generate prediction-evasive adversarial examples , where the adversary relies only on gradients of the loss function with respect to inputs of DNNs . Our model takes into account both commonly-used adversarial distortion metrics and the proposed group-sparsity regularization that encodes spatial structures in attacks . We show that the process of generating structured adversarial examples leads to an optimization problem that is difficult to solve using the existing optimizers Adam ( for C & W attack ) and FISTA ( for EAD attack ) ( Carlini & Wagner , 2017 ; Chen et al. , 2017a ) . To circumvent this challenge , we develop an efficient optimization method via alternating direction method of multipliers ( ADMM ) . Given an original image x0 ∈ Rn , we aim to design the optimal adversarial perturbation δ ∈ Rn so that the adversarial example ( x0 + δ ) misleads DNNs trained on natural images . Throughout this paper , we use vector representations of the adversarial perturbation ∆ and the original image X0 without loss of generality . A well designed perturbation δ can be obtained by solving optimization problems of the following form , minimize δ f ( x0 + δ , t ) + γD ( δ ) + τg ( δ ) subject to ( x0 + δ ) ∈ [ 0 , 1 ] n , ‖δ‖∞ ≤ , ( 2 ) where f ( x , t ) denotes the loss function for crafting adversarial example given a target class t , D ( δ ) is a distortion function that controls the perceptual similarity between a natural image and a perturbed image , g ( δ ) = ∑P p=1 ∑Q q=1 ‖δGp , q‖2 is given by ( 1 ) , and ‖ · ‖p signifies the ` p norm . In problem ( 2 ) , the ‘ hard ’ constraints ensure the validness of created adversarial examples with - tolerant perturbed pixel values . And the non-negative regularization parameters γ and τ place our emphasis on the distortion of an adversarial example ( to an original image ) and group sparsity of adversarial perturbation . Tuning the regularization parameters will be discussed in Appendix F. Problem ( 2 ) gives a quite general formulation for design of adversarial examples . If we remove the group-sparsity regularizer g ( δ ) and the ` ∞ constraint , problem ( 2 ) becomes the same as the C & W attack ( Carlini & Wagner , 2017 ) . More specifically , if we further set the distortion function D ( δ ) to the form of ` 0 , ` 2 or ` ∞ norm , then we obtain C & W ` 0 , ` 2 or ` ∞ attack . If D ( δ ) is specified by the elastic-net regularizer , then problem ( 2 ) becomes the formulation of EAD attack ( Chen et al. , 2017a ) . In this paper , we specify the loss function of problem ( 2 ) as below , which yields the best known performance of adversaries ( Carlini & Wagner , 2017 ) , f ( x0 + δ , t ) = c ·max { max j 6=t Z ( x0 + δ ) j − Z ( x0 + δ ) t , −κ } , ( 3 ) where Z ( x ) j is the jth element of logits Z ( x ) , representing the output before the last softmax layer in DNNs , and κ is a confidence parameter that is usually set to zero if the attack transferability is not much cared . We choose D ( δ ) = ‖δ‖22 for a fair comparison with the C & W ` 2 adversarial attack . In this section , we assume that { Gp , q } are non-overlapping groups , i.e. , Gp , q ∩ Gp′ , q′ = ∅ for q 6= q′ or p 6= p′ . The overlapping case will be studied in the next section . The presence of multiple non-smooth regularizers and ‘ hard ’ constraints make the existing optimizers Adam and FISTA ( Carlini & Wagner , 2017 ; Chen et al. , 2017a ; Kingma & Ba , 2015 ; Beck & Teboulle , 2009 ) inefficient for solving problem ( 2 ) . First , the subgradient of the objective function of problem ( 2 ) is difficult to obtain especially when { Gp , q } are overlapping groups . Second , it is impossible to compute the proximal operations required for FISTA with respect to all non-smooth regularizers and ‘ hard ’ constraints . Different from the existing work , we show that ADMM , a firstorder operator splitting method , helps us to split the original complex problem ( 2 ) into a sequence of subproblems , each of which can be solved analytically . We reformulate problem ( 2 ) in a way that lends itself to the application of ADMM , minimize δ , z , w , y f ( z + x0 ) + γD ( δ ) + τ ∑PQ i=1 ‖yDi‖2 + h ( w ) subject to z = δ , z = y , z = w , ( 4 ) where z , y and w are newly introduced variables , for ease of notation let D ( q−1 ) P+p = Gp , q , and h ( w ) is an indicator function with respect to the constraints of problem ( 2 ) , h ( w ) = { 0 if ( x0 + w ) ∈ [ 0 , 1 ] n , ‖w‖∞ ≤ , ∞ otherwise . ( 5 ) ADMM is performed by minimizing the augmented Lagrangian of problem ( 4 ) , L ( z , δ , y , w , u , v , s ) = f ( z + x0 ) + γD ( δ ) + τ ∑PQ i=1 ‖yDi‖2 + h ( w ) + u T ( δ − z ) +vT ( y − z ) + sT ( w − z ) + ρ 2 ‖δ − z‖22 + ρ2‖y − z‖ 2 2 + ρ 2 ‖w − z‖22 , ( 6 ) where u , v and s are Lagrangian multipliers , and ρ > 0 is a given penalty parameter . ADMM splits all of optimization variables into two blocks and adopts the following iterative scheme , { δk+1 , wk+1 , yk+1 } = argmin δ , w , y L ( δ , zk , w , y , uk , vk , sk ) , ( 7 ) zk+1 = argmin z L ( δk+1 , z , wk+1 , yk+1 , uk , vk , sk ) , ( 8 ) u k+1 = uk + ρ ( δk+1 − zk+1 ) , vk+1 = vk + ρ ( yk+1 − zk+1 ) , sk+1 = sk + ρ ( wk+1 − zk+1 ) , ( 9 ) where k is the iteration index , steps ( 7 ) - ( 8 ) are used for updating primal variables , and the last step ( 9 ) is known as the dual update step . We emphasize that the crucial property of the proposed ADMM approach is that , as we demonstrate in Proposition 1 , the solution to problem ( 7 ) can be found in parallel and exactly . Proposition 1 When D ( δ ) = ‖δ‖22 , the solution to problem ( 7 ) is given by δk+1 = ρρ+2γa , ( 10 ) [ wk+1 ] i = { min { 1− [ x0 ] i , } bi > min { 1− [ x0 ] i , } max { − [ x0 ] i , − } bi < max { − [ x0 ] i , − } bi otherwise , for i ∈ [ n ] , ( 11 ) [ yk+1 ] Di = ( 1− τρ‖ [ c ] Di‖2 ) + [ c ] Di , i ∈ [ PQ ] , ( 12 ) where a : = zk − uk/ρ , b : = zk − sk/ρ , c : = zk − vk/ρ , ( x ) + = x if x ≥ 0 and 0 otherwise , [ x ] i denotes the ith element of x , and [ x ] Di denotes the sub-vector of x indexed by Di . Proof : See Appendix B . It is clear from Proposition 1 that introducing auxiliary variables does not increase the computational complexity of ADMM since ( 10 ) - ( 12 ) can be solved in parallel . Moreover , if another distortion metric ( different from D ( δ ) = ‖δ‖22 ) is used , then ADMM only changes at the δ-step ( 10 ) . We next focus on the z-minimization step ( 8 ) , which can be equivalently transformed into minimize z f ( x0 + z ) + ρ 2 ‖z− a′‖22 + ρ 2 ‖z− b′‖22 + ρ 2 ‖z− c′‖22 , ( 13 ) where a′ : = δk+1 + uk/ρ , b′ : = wk+1 + sk/ρ , and c′ : = yk+1 + vk/ρ . We recall that attacks studied in this paper belongs to ‘ first-order ’ adversaries ( Madry et al. , 2017 ) , which only have access to gradients of the loss function f . Spurred by that , we solve problem ( 13 ) via a linearization technique that is commonly used in stochastic/online ADMM ( Ouyang et al. , 2013 ; Suzuki , 2013 ; Liu et al. , 2018 ) or linearized ADMM ( Boyd et al. , 2011 ; Liu et al. , 2017 ) . Specifically , we replace the function f with its first-order Taylor expansion at the point zk by adding a Bregman divergence term ( ηk/2 ) ‖z− zk‖22 . As a result , problem ( 13 ) becomes minimize z ( ∇f ( zk + x0 ) ) T ( z− zk ) + ηk 2 ‖z− zk‖22 + ρ 2 ‖z− a′‖22 + ρ 2 ‖z− b′‖22 + ρ 2 ‖z− c′‖22 , ( 14 ) where 1/ηk > 0 is a given decaying parameter , e.g. , ηk = α √ k for some α > 0 , and the Bregman divergence term stabilizes the convergence of z-minimization step . It is clear that problem ( 14 ) yields a quadratic program with the closed-form solution zk+1 = ( 1/ ( ηk + 3ρ ) ) ( ηkz k + ρa + ρb + ρc−∇f ( zk + x0 ) ) . ( 15 ) In summary , the proposed ADMM algorithm alternatively updates ( 7 ) - ( 9 ) , which yield closed-form solutions given by ( 10 ) - ( 12 ) and ( 15 ) . The convergence of linearized ADMM for nonconvex optimization was recently proved by ( Liu et al. , 2017 ) , and thus provides theoretical validity of our approach . Compared to the existing solver for generation of adversarial examples ( Carlini & Wagner , 2017 ; Papernot et al. , 2016b ) , our algorithm offers two main benefits , efficiency and generality . That is , the computations for every update step are efficiently carried out , and our approach can be applicable to a wide class of attack formulations .

Summary: The paper proposes a method to find adversarial examples in which the changes are localized to small regions of the image. A group-sparsity objective is introduced for this purpose and it is combined with an l_p objective that was used in prior work to define proximity to the original example. ADMM is applied to maximize the defined objective. It is shown that adversarial examples in which all changes are concentrated in just few regions can be found with the proposed method.