Adversarial Attack Generation Empowered by Min-Max Optimization

1 Introduction . Training a machine learning model that is capable of assuring its worst-case performance against possible adversaries given a specified threat model is a fundamental and challenging problem , especially for deep neural networks ( DNNs ) [ 64 , 22 , 13 , 69 , 70 ] . A common practice to train an adversarially robust model is based on a specific form of min-max training , known as adversarial training ( AT ) [ 22 , 40 ] , where the minimization step learns model weights under the adversarial loss constructed at the maximization step in an alternative training fashion . In practice , AT has achieved the state-of-the-art defense performance against ` p-norm-ball input perturbations [ 3 ] . Although the min-max principle is widely used in AT and its variants [ 40 , 59 , 76 , 65 ] , few work has studied its power in attack generation . Thus , we ask : Beyond AT , can other types of min-max formulation and optimization techniques advance the research in adversarial attack generation ? In this paper , we give an affirmative answer corroborated by the substantial performance gain and the ability of self-learned risk interpretation using our proposed min-max framework on several tasks for adversarial attack . ⇤Equal contributions . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We demonstrate the utility of a general formulation for minimizing the maximal loss induced from a set of risk sources ( domains ) . Our considered min-max formulation is fundamentally different from AT , as our maximization step is taken over the probability simplex of the set of domains . Moreover , we show that many problem setups in adversarial attacks can in fact be reformulated under this general min-max framework , including attacking model ensembles [ 66 , 34 ] , devising universal perturbation to input samples [ 44 ] and data transformations [ 6 , 10 ] . However , current methods for solving these tasks often rely on simple heuristics ( e.g. , uniform averaging ) , resulting in significant performance drops when comparing to our proposed min-max optimization framework . Contributions ¨ With the aid of min-max optimization , we propose a unified alternating onestep projected gradient descent-ascent ( APGDA ) attack method , which can readily be specified to generate model ensemble attack , universal attack over multiple images , and robust attack over data transformations . ≠ In theory , we show that APGDA has an O ( 1/T ) convergence rate , where T is the number of iterations . In practice , we show that APGDA obtains 17.48 % , 35.21 % and 9.39 % improvement on average compared with conventional min-only PGD attack methods on CIFAR-10 . Æ More importantly , we demonstrate that by tracking the learnable weighting factors associated with multiple domains , our method can provide tools for self-adjusted importance assessment on the mixed learning tasks . Ø Finally , we adapt the idea of the domain weights into a defense setting [ 65 ] , where multiple ` p-norm perturbations are generated , and achieve superior performance as well as intepretability . 1.1 Related work . Recent studies have identified that DNNs are highly vulnerable to adversarial manipulations in various applications [ 64 , 12 , 27 , 33 , 26 , 14 , 77 , 20 , 15 , 31 ] , thus leading to an arms race between adversarial attacks [ 13 , 3 , 23 , 48 , 45 , 72 , 1 , 18 ] and defenses [ 40 , 59 , 76 , 65 , 42 , 71 , 74 , 68 , 53 , 16 ] . One intriguing property of adversarial examples is the transferability across multiple domains [ 36 , 67 , 47 , 62 ] , which indicates a more challenging yet promising research direction – devising universal adversarial perturbations over model ensembles [ 66 , 34 ] , input samples [ 44 , 43 , 56 ] and data transformations [ 3 , 6 , 10 ] . Besides , many recent works started to produce physical realizable perturbations that expose real world threats . The most popular approach [ 4 , 21 ] , as known as Expectation Over Transformation ( EOT ) , is to train the attack under different data transformation ( e.g. , different view angles and distances ) . However , current approaches suffer from a significant performance loss for resting on the uniform averaging strategy or heuristic weighting schemes [ 34 , 56 ] . We will compare these works with our min-max method in Sec . 4 . As a natural extension following min-max attack , we study the generalized AT under multiple perturbations [ 65 , 2 , 28 , 17 ] . Finally , our min-max framework is adapted and inspired by previous literature on robust optimization over multiple domains [ 50 , 51 , 38 , 37 ] . To our best knowledge , only few works leverage min-max principle for adversarial attack generation while the idea of producing the worst-case example across multiple domains is quite natural . Specifically , [ 7 ] considered the non-interactive blackbox adversary setting and proposed a framework that models the crafting of adversarial examples as a min-max game between a generator of attacks and a classifier . [ 57 ] introduced a min-max based adaptive attacker ’ s objective to craft perturbation so that it simultaneously evades detection and causes misclassification . Inspired by our work , the min-max formulation has also been extended to zero-order blackbox attacks [ 35 ] and physically realizable attacks [ 73 , Adversarial T-shirt ] . We hope our unified formulation can stimulate further research on applying min-max principle and interpretable domain weights in more attack generation tasks that involve in evading multiple systems . 2 Min-Max Across Domains . Consider K loss functions { Fi ( v ) } ( each of which is defined on a learning domain ) , the problem of robust learning over K domains can be formulated as [ 50 , 51 , 38 ] minimize v2V maximize w2P P K i=1 wiFi ( v ) , ( 1 ) where v and w are optimization variables , V is a constraint set , and P denotes the probability simplex P = { w |1Tw = 1 , wi 2 [ 0 , 1 ] , 8i } . Since the inner maximization problem in ( 1 ) is a linear function of w over the probabilistic simplex , problem ( 1 ) is thus equivalent to minimize v2V maximize i2 [ K ] Fi ( v ) , ( 2 ) where [ K ] denotes the integer set { 1 , 2 , . . . , K } . Benefit and Challenge from ( 1 ) . Compared to multi-task learning in a finite-sum formulation which minimizes K losses on average , problem ( 1 ) provides consistently robust worst-case performance across all domains . This can be explained from the epigraph form of ( 2 ) , minimize v2V , t t , subject to Fi ( v ) t , i 2 [ K ] , ( 3 ) where t is an epigraph variable [ 8 ] that provides the t-level robustness at each domain . In computation , the inner maximization problem of ( 1 ) always returns the one-hot value of w , namely , w = ei , where ei is the ith standard basis vector , and i = argmaxi { Fi ( v ) } . However , this one-hot coding reduces the generalizability to other domains and induces instability of the learning procedure in practice . Such an issue is often mitigated by introducing a strongly concave regularizer in the inner maximization step to strike a balance between the average and the worst-case performance [ 38 , 50 ] . Regularized Formulation . Following [ 50 ] , we penalize the distance between the worst-case loss and the average loss over K domains . This yields minimize v2V maximize w2P P K i=1 wiFi ( v ) 2 kw 1/Kk 2 2 , ( 4 ) where > 0 is a regularization parameter . As ! 0 , problem ( 4 ) is equivalent to ( 1 ) . By contrast , it becomes the finite-sum problem when ! 1 since w ! 1/K . In this sense , the trainable w provides an essential indicator on the importance level of each domain . The larger the weight is , the more important the domain is . We call w domain weights in this paper . 3 Min-Max Power in Attack Design . To the best of our knowledge , few work has studied the power of min-max in attack generation . In this section , we demonstrate how the unified min-max framework ( 4 ) fits into various attack settings . With the help of domain weights , our solution yields better empirical performance and explainability . Finally , we present the min-max algorithm with convergence analysis to craft robust perturbations against multiple domains . 3.1 A Unified Framework for Robust Adversarial Attacks . The general goal of adversarial attack is to craft an adversarial example x0 = x0 + 2 Rd to mislead the prediction of machine learning ( ML ) or deep learning ( DL ) systems , where x0 denotes the natural example with the true label t0 , and is known as adversarial perturbation , commonly subject to ` p-norm ( p 2 { 0 , 1 , 2,1 } ) constraint X : = { | k kp ✏ , x0 + 2 [ 0 , 1 ] d } for a given small number ✏ . Here the ` p norm enforces the similarity between x0 and x0 , and the input space of ML/DL systems is normalized to [ 0 , 1 ] d. Ensemble Attack over Multiple Models . Consider K ML/DL models { Mi } Ki=1 , the goal is to find robust adversarial examples that can fool all K models simultaneously . In this case , the notion of ‘ domain ’ in ( 4 ) is specified as ‘ model ’ , and the objective function Fi in ( 4 ) signifies the attack loss f ( ; x0 , y0 , Mi ) given the natural input ( x0 , y0 ) and the model Mi . Thus , problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wif ( ; x0 , y0 , Mi ) 2 kw 1/Kk 2 2 , ( 5 ) where w encodes the difficulty level of attacking each model . Universal Perturbation over Multiple Examples . Consider K natural examples { ( xi , yi ) } Ki=1 and a single model M , our goal is to find the universal perturbation so that all the corrupted K examples can fool M. In this case , the notion of ‘ domain ’ in ( 4 ) is specified as ‘ example ’ , and problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wif ( ; xi , yi , M ) 2 kw 1/Kk 2 2 , ( 6 ) where different from ( 5 ) , w encodes the difficulty level of attacking each example . Adversarial Attack over Data Transformations . Consider K categories of data transformation { pi } , e.g. , rotation , lightening , and translation , our goal is to find the adversarial attack that is robust to data transformations . Such an attack setting is commonly applied to generate physical adversarial examples [ 5 , 20 ] . Here the notion of ‘ domain ’ in ( 4 ) is specified as ‘ data transformer ’ , and problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wiEt⇠pi [ f ( t ( x0 + ) ; y0 , M ) ] 2 kw 1/Kk 2 2 , ( 7 ) where Et⇠pi [ f ( t ( x0+ ) ; y0 , M ) ] denotes the attack loss under the distribution of data transformation pi , and w encodes the difficulty level of attacking each type of transformed example x0 . We remark that if w = 1/K , then problem ( 7 ) reduces to the existing expectation of transformation ( EOT ) setup used for physical attack generation [ 5 ] . Benefits of Min-Max Attack Generation with Learnable Domain Weights w : We can interpret ( 5 ) - ( 7 ) as finding the robust adversarial attack against the worst-case environment that an adversary encounters , e.g. , multiple victim models , data samples , and input transformations . The proposed min-max design of adversarial attacks leads to two main benefits . First , compared to the heuristic weighting strategy ( e.g. , clipping thresholds on the importance of individual attack losses [ 56 ] ) , our proposal is free of supervised manual adjustment on domain weights . Even by carefully tuning the heuristic weighting strategy , we find that our approach with self-adjusted w consistently outperforms the clipping strategy in [ 56 ] ( see Table 2 ) . Second , the learned domain weights can be used to assess the model robustness when facing different types of adversary . We refer readers to Figure 1c and Figure 6 for more details .

Summary: This paper proposes a unified alternating projected gradient descent-ascent (APGDA) attack by considering the min-max operation for adversarial attack settings. The proposed method can be applied in generating model ensemble attacks, universal adversarial attacks, and robust attacks over data transformations. Extensive results show that the proposed method can achieve better attack performances for these three applications.