Adversarial Attack Generation Empowered by Min-Max Optimization

1 Introduction . Training a machine learning model that is capable of assuring its worst-case performance against possible adversaries given a specified threat model is a fundamental and challenging problem , especially for deep neural networks ( DNNs ) [ 64 , 22 , 13 , 69 , 70 ] . A common practice to train an adversarially robust model is based on a specific form of min-max training , known as adversarial training ( AT ) [ 22 , 40 ] , where the minimization step learns model weights under the adversarial loss constructed at the maximization step in an alternative training fashion . In practice , AT has achieved the state-of-the-art defense performance against ` p-norm-ball input perturbations [ 3 ] . Although the min-max principle is widely used in AT and its variants [ 40 , 59 , 76 , 65 ] , few work has studied its power in attack generation . Thus , we ask : Beyond AT , can other types of min-max formulation and optimization techniques advance the research in adversarial attack generation ? In this paper , we give an affirmative answer corroborated by the substantial performance gain and the ability of self-learned risk interpretation using our proposed min-max framework on several tasks for adversarial attack . ⇤Equal contributions . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We demonstrate the utility of a general formulation for minimizing the maximal loss induced from a set of risk sources ( domains ) . Our considered min-max formulation is fundamentally different from AT , as our maximization step is taken over the probability simplex of the set of domains . Moreover , we show that many problem setups in adversarial attacks can in fact be reformulated under this general min-max framework , including attacking model ensembles [ 66 , 34 ] , devising universal perturbation to input samples [ 44 ] and data transformations [ 6 , 10 ] . However , current methods for solving these tasks often rely on simple heuristics ( e.g. , uniform averaging ) , resulting in significant performance drops when comparing to our proposed min-max optimization framework . Contributions ¨ With the aid of min-max optimization , we propose a unified alternating onestep projected gradient descent-ascent ( APGDA ) attack method , which can readily be specified to generate model ensemble attack , universal attack over multiple images , and robust attack over data transformations . ≠ In theory , we show that APGDA has an O ( 1/T ) convergence rate , where T is the number of iterations . In practice , we show that APGDA obtains 17.48 % , 35.21 % and 9.39 % improvement on average compared with conventional min-only PGD attack methods on CIFAR-10 . Æ More importantly , we demonstrate that by tracking the learnable weighting factors associated with multiple domains , our method can provide tools for self-adjusted importance assessment on the mixed learning tasks . Ø Finally , we adapt the idea of the domain weights into a defense setting [ 65 ] , where multiple ` p-norm perturbations are generated , and achieve superior performance as well as intepretability . 1.1 Related work . Recent studies have identified that DNNs are highly vulnerable to adversarial manipulations in various applications [ 64 , 12 , 27 , 33 , 26 , 14 , 77 , 20 , 15 , 31 ] , thus leading to an arms race between adversarial attacks [ 13 , 3 , 23 , 48 , 45 , 72 , 1 , 18 ] and defenses [ 40 , 59 , 76 , 65 , 42 , 71 , 74 , 68 , 53 , 16 ] . One intriguing property of adversarial examples is the transferability across multiple domains [ 36 , 67 , 47 , 62 ] , which indicates a more challenging yet promising research direction – devising universal adversarial perturbations over model ensembles [ 66 , 34 ] , input samples [ 44 , 43 , 56 ] and data transformations [ 3 , 6 , 10 ] . Besides , many recent works started to produce physical realizable perturbations that expose real world threats . The most popular approach [ 4 , 21 ] , as known as Expectation Over Transformation ( EOT ) , is to train the attack under different data transformation ( e.g. , different view angles and distances ) . However , current approaches suffer from a significant performance loss for resting on the uniform averaging strategy or heuristic weighting schemes [ 34 , 56 ] . We will compare these works with our min-max method in Sec . 4 . As a natural extension following min-max attack , we study the generalized AT under multiple perturbations [ 65 , 2 , 28 , 17 ] . Finally , our min-max framework is adapted and inspired by previous literature on robust optimization over multiple domains [ 50 , 51 , 38 , 37 ] . To our best knowledge , only few works leverage min-max principle for adversarial attack generation while the idea of producing the worst-case example across multiple domains is quite natural . Specifically , [ 7 ] considered the non-interactive blackbox adversary setting and proposed a framework that models the crafting of adversarial examples as a min-max game between a generator of attacks and a classifier . [ 57 ] introduced a min-max based adaptive attacker ’ s objective to craft perturbation so that it simultaneously evades detection and causes misclassification . Inspired by our work , the min-max formulation has also been extended to zero-order blackbox attacks [ 35 ] and physically realizable attacks [ 73 , Adversarial T-shirt ] . We hope our unified formulation can stimulate further research on applying min-max principle and interpretable domain weights in more attack generation tasks that involve in evading multiple systems . 2 Min-Max Across Domains . Consider K loss functions { Fi ( v ) } ( each of which is defined on a learning domain ) , the problem of robust learning over K domains can be formulated as [ 50 , 51 , 38 ] minimize v2V maximize w2P P K i=1 wiFi ( v ) , ( 1 ) where v and w are optimization variables , V is a constraint set , and P denotes the probability simplex P = { w |1Tw = 1 , wi 2 [ 0 , 1 ] , 8i } . Since the inner maximization problem in ( 1 ) is a linear function of w over the probabilistic simplex , problem ( 1 ) is thus equivalent to minimize v2V maximize i2 [ K ] Fi ( v ) , ( 2 ) where [ K ] denotes the integer set { 1 , 2 , . . . , K } . Benefit and Challenge from ( 1 ) . Compared to multi-task learning in a finite-sum formulation which minimizes K losses on average , problem ( 1 ) provides consistently robust worst-case performance across all domains . This can be explained from the epigraph form of ( 2 ) , minimize v2V , t t , subject to Fi ( v ) t , i 2 [ K ] , ( 3 ) where t is an epigraph variable [ 8 ] that provides the t-level robustness at each domain . In computation , the inner maximization problem of ( 1 ) always returns the one-hot value of w , namely , w = ei , where ei is the ith standard basis vector , and i = argmaxi { Fi ( v ) } . However , this one-hot coding reduces the generalizability to other domains and induces instability of the learning procedure in practice . Such an issue is often mitigated by introducing a strongly concave regularizer in the inner maximization step to strike a balance between the average and the worst-case performance [ 38 , 50 ] . Regularized Formulation . Following [ 50 ] , we penalize the distance between the worst-case loss and the average loss over K domains . This yields minimize v2V maximize w2P P K i=1 wiFi ( v ) 2 kw 1/Kk 2 2 , ( 4 ) where > 0 is a regularization parameter . As ! 0 , problem ( 4 ) is equivalent to ( 1 ) . By contrast , it becomes the finite-sum problem when ! 1 since w ! 1/K . In this sense , the trainable w provides an essential indicator on the importance level of each domain . The larger the weight is , the more important the domain is . We call w domain weights in this paper . 3 Min-Max Power in Attack Design . To the best of our knowledge , few work has studied the power of min-max in attack generation . In this section , we demonstrate how the unified min-max framework ( 4 ) fits into various attack settings . With the help of domain weights , our solution yields better empirical performance and explainability . Finally , we present the min-max algorithm with convergence analysis to craft robust perturbations against multiple domains . 3.1 A Unified Framework for Robust Adversarial Attacks . The general goal of adversarial attack is to craft an adversarial example x0 = x0 + 2 Rd to mislead the prediction of machine learning ( ML ) or deep learning ( DL ) systems , where x0 denotes the natural example with the true label t0 , and is known as adversarial perturbation , commonly subject to ` p-norm ( p 2 { 0 , 1 , 2,1 } ) constraint X : = { | k kp ✏ , x0 + 2 [ 0 , 1 ] d } for a given small number ✏ . Here the ` p norm enforces the similarity between x0 and x0 , and the input space of ML/DL systems is normalized to [ 0 , 1 ] d. Ensemble Attack over Multiple Models . Consider K ML/DL models { Mi } Ki=1 , the goal is to find robust adversarial examples that can fool all K models simultaneously . In this case , the notion of ‘ domain ’ in ( 4 ) is specified as ‘ model ’ , and the objective function Fi in ( 4 ) signifies the attack loss f ( ; x0 , y0 , Mi ) given the natural input ( x0 , y0 ) and the model Mi . Thus , problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wif ( ; x0 , y0 , Mi ) 2 kw 1/Kk 2 2 , ( 5 ) where w encodes the difficulty level of attacking each model . Universal Perturbation over Multiple Examples . Consider K natural examples { ( xi , yi ) } Ki=1 and a single model M , our goal is to find the universal perturbation so that all the corrupted K examples can fool M. In this case , the notion of ‘ domain ’ in ( 4 ) is specified as ‘ example ’ , and problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wif ( ; xi , yi , M ) 2 kw 1/Kk 2 2 , ( 6 ) where different from ( 5 ) , w encodes the difficulty level of attacking each example . Adversarial Attack over Data Transformations . Consider K categories of data transformation { pi } , e.g. , rotation , lightening , and translation , our goal is to find the adversarial attack that is robust to data transformations . Such an attack setting is commonly applied to generate physical adversarial examples [ 5 , 20 ] . Here the notion of ‘ domain ’ in ( 4 ) is specified as ‘ data transformer ’ , and problem ( 4 ) becomes minimize 2X maximize w2P P K i=1 wiEt⇠pi [ f ( t ( x0 + ) ; y0 , M ) ] 2 kw 1/Kk 2 2 , ( 7 ) where Et⇠pi [ f ( t ( x0+ ) ; y0 , M ) ] denotes the attack loss under the distribution of data transformation pi , and w encodes the difficulty level of attacking each type of transformed example x0 . We remark that if w = 1/K , then problem ( 7 ) reduces to the existing expectation of transformation ( EOT ) setup used for physical attack generation [ 5 ] . Benefits of Min-Max Attack Generation with Learnable Domain Weights w : We can interpret ( 5 ) - ( 7 ) as finding the robust adversarial attack against the worst-case environment that an adversary encounters , e.g. , multiple victim models , data samples , and input transformations . The proposed min-max design of adversarial attacks leads to two main benefits . First , compared to the heuristic weighting strategy ( e.g. , clipping thresholds on the importance of individual attack losses [ 56 ] ) , our proposal is free of supervised manual adjustment on domain weights . Even by carefully tuning the heuristic weighting strategy , we find that our approach with self-adjusted w consistently outperforms the clipping strategy in [ 56 ] ( see Table 2 ) . Second , the learned domain weights can be used to assess the model robustness when facing different types of adversary . We refer readers to Figure 1c and Figure 6 for more details .

Summary: The paper provides a general min-max framework to formulate several popular problems in the literature including Ensemble attack over multiple models, Universal Perturbation over multiple examples, and adversarial attack over data transformations. For each case, a standard first-order approach converging to a stationary point of the problem is provided. The effectiveness of the min-max approach compared to the standard empirical risk minimization has been demonstrated by extensive experiments.