Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

1 INTRODUCTION . Deep neural networks are vulnerable to maliciously crafted adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) , which aim to induce erroneous model predictions by adding small perturbations to the normal inputs . Due to the threats , a multitude of defense strategies have been proposed to improve adversarial robustness ( Kurakin et al. , 2017b ; Madry et al. , 2018 ; Liao et al. , 2018 ; Wong & Kolter , 2018 ; Zhang et al. , 2019b ; Dong et al. , 2020a ; Pang et al. , 2020 ) . However , many defenses have later been shown to be ineffective due to incomplete or incorrect robustness evaluations ( Athalye et al. , 2018a ; Uesato et al. , 2018 ; Carlini et al. , 2019 ; Croce & Hein , 2020b ; Dong et al. , 2020b ; Tramer et al. , 2020 ) , making it particularly challenging to understand their effects and identify the actual progress of the field . Therefore , developing methods that can evaluate adversarial robustness accurately and reliably is of profound importance . Adversarial attacks are broadly adopted as an indispensable solution to evaluate adversarial robustness for different models . One of the most prominent attacks is the projected gradient descent ( PGD ) method ( Madry et al. , 2018 ) , which generates an adversarial example by performing iterative gradient updates to maximize a classification loss ( e.g. , cross-entropy loss ) w.r.t . the input . Although recent methods improve upon PGD by introducing different loss functions ( Carlini & Wagner , 2017 ; Gowal et al. , 2019 ; Croce & Hein , 2020b ; Sriramanan et al. , 2020 ) and adjusting the step size ( Croce & Hein , 2020b ) , most of them adopt hand-designed optimization algorithms , such as vanilla gradient descent , momentum ( Polyak , 1964 ) , and Adam ( Kingma & Ba , 2015 ) . However , it has been shown that these attacks can be sub-optimal ( Croce & Hein , 2020b ; Tramer et al. , 2020 ) , arousing an overestimation of adversarial robustness for some defenses . It is thus imperative to develop more effective optimization algorithms for improving adversarial attacks . Nevertheless , designing a generic optimization algorithm in a hand-crafted manner is non-trivial , considering the different defense models with varying network architectures , defense strategies , and training datasets . To overcome this issue and develop stronger adversarial attacks , in this paper , we propose a ModelAgnostic Meta-Attack ( MAMA ) approach to learn optimization algorithms in adversarial attacks automatically . In particular , we parameterize the optimizer with a recurrent neural network ( RNN ) to mimic the behavior of iterative attacks , which outputs an update direction at each time step during adversarial example generation . By learning to solve a class of optimization problems with different data samples and defense models , the RNN optimizer could benefit from long-term dependencies of iterative attacks and exploit common structures among the optimization problems . To improve and stabilize training , we then propose a prior-guided refinement strategy , which imposes a prior given by the PGD update rule on the outputs of the RNN optimizer , whose parameters are trained via the maximum a posteriori ( MAP ) estimation framework . Consequently , the RNN optimizer can learn to yield more effective update directions than hand-designed ones . Despite the effectiveness , the learned optimizer may not generalize well to attacking unseen defenses due to potential overfitting to the defenses used for training . Although training a different optimizer for each defense is possible , the training process can be time-consuming and inconvenient in practice when it is utilized to benchmark adversarial robustness . To endow the learned optimizer with a better generalization ability , we develop a model-agnostic training algorithm with a gradient-based metatrain and meta-test process to simulate the shift from seen to unseen defenses . Therefore , the learned optimizer can directly be deployed to attack unseen defenses without retraining or fine-tuning . Extensive experiments validate the effectiveness and generalization ability of the learned optimizers for attacking different defense models . We also demonstrate the flexibility of our method integrating with various attacks , including those with different loss functions ( Carlini & Wagner , 2017 ; Gowal et al. , 2019 ; Croce & Hein , 2020b ) and initialization ( Tashiro et al. , 2020 ) . Our method consistently improves the attacking performance over baselines , while introducing little extra computational cost . By incorporating our method with an orthogonal technique — output diversified initialization ( ODI ) ( Tashiro et al. , 2020 ) , we achieve lower robust test accuracy on all 12 defense models that we study than other state-of-the-art attacks , including the MultiTargeted ( MT ) attack ( Gowal et al. , 2019 ) and AutoAttack ( Croce & Hein , 2020b ) , leading to a more reliable evaluation of adversarial robustness . Moreover , MAMA provides a plug-and-play module in adversarial attacks , which may benefit future adversarial attacks if new improvements have been made in different aspects . 2 PRELIMINARIES AND RELATED WORK . 2.1 ADVERSARIAL ATTACKS . Let f ( x ) : x ∈ RD → z ∈ RK denote a classifier , which outputs the logits z : = [ z1 , ... , zK ] over K classes for an input image x . The prediction of f is Cf ( x ) = arg maxi=1 , ... , K zi . Given the true label y of x , adversarial attacks aim to generate an adversarial example x̂ that is misclassified by f ( i.e. , Cf ( x̂ ) 6= y ) , while the distance between the adversarial example x̂ and the natural one x measured by the ` ∞ norm is smaller than a threshold as ‖x̂ − x‖∞ ≤ . Although we introduce our approach based on the ` ∞ norm only , the extension to other ` p norms is straightforward . The adversarial example x̂ can be generated by solving a constrained optimization problem as x̂ = arg max x′ L ( f ( x′ ) , y ) , s.t . ‖x′ − x‖∞ ≤ , ( 1 ) where L is a loss function on top of the classifier f ( x ) . For example , L could be the cross-entropy loss as Lce ( f ( x ) , y ) = − log py where py = ezy/∑Ki=1 ezi denotes the predicted probability of class y , or the margin-based CW loss ( Carlini & Wagner , 2017 ) as Lcw ( f ( x ) , y ) = maxi6=y zi − zy . A lot of gradient-based methods ( Goodfellow et al. , 2015 ; Kurakin et al. , 2017a ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Dong et al. , 2018 ) have been proposed to solve this optimization problem . The projected gradient descent ( PGD ) method ( Madry et al. , 2018 ) performs iterative updates as x̂t+1 = ΠB ( x ) ( x̂t + αt · sign ( ∇xL ( f ( x̂t ) , y ) ) ) , ( 2 ) where B ( x ) = { x′ : ‖x′ − x‖∞ ≤ } denotes the ` ∞ ball centered at x with radius , Π ( · ) is the projection operation , and αt is the step size at the t-th attack iteration . PGD initializes x̂0 randomly by uniform sampling within B ( x ) and adopts a fixed step size α in every iteration . To develop stronger attacks for reliable robustness evaluation , recent improvements upon PGD include adopting different loss functions ( Gowal et al. , 2019 ; Croce & Hein , 2020b ; Sriramanan et al. , 2020 ) , adjusting the step size ( Croce & Hein , 2020b ) , and designing sampling strategies of initialization ( Tashiro et al. , 2020 ) . For the update rules , besides the vanilla gradient descent adopted in PGD , recent attacks ( Dong et al. , 2018 ; Gowal et al. , 2019 ) suggest to use the momentum ( Polyak , 1964 ) and Adam ( Kingma & Ba , 2015 ) optimizers . In contrast , we aim to enhance adversarial attacks by learning optimization algorithms in an automatic way . 2.2 ADVERSARIAL DEFENSES . To build robust models against adversarial attacks , numerous defense strategies have been proposed , but most defenses can be evaded by new adaptive attacks ( Athalye et al. , 2018a ; Tramer et al. , 2020 ) . Among the existing defenses , adversarial training ( AT ) is arguably the most effective defense technique , in which the network is trained on the adversarial examples generated by attacks . Based on the primary AT frameworks like PGD-AT ( Madry et al. , 2018 ) and TRADES ( Zhang et al. , 2019b ) , improvements have been made via ensemble learning ( Tramèr et al. , 2018 ; Pang et al. , 2019 ) , metric learning ( Mao et al. , 2019 ; Pang et al. , 2020 ) , semi-supervised learning ( Alayrac et al. , 2019 ; Carmon et al. , 2019 ; Zhai et al. , 2019 ) , and self-supervised learning ( Chen et al. , 2020a ; Hendrycks et al. , 2019 ; Kim et al. , 2020 ) . Due to the high computational cost of AT , other efforts are devoted to accelerating the training procedure ( Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ) . Recent works emphasize the training tricks ( e.g. , weight decay , batch size , etc . ) in AT ( Gowal et al. , 2020 ; Pang et al. , 2021 ) . However , it has been shown that the robust test accuracy of numerous adversarially trained models can be degraded significantly by using stronger attacks ( Croce & Hein , 2020b ; Tramer et al. , 2020 ) , indicating that the reliable evaluation of adversarial robustness remains an imperative yet challenging task . In this paper , we focus on evaluating adversarial robustness of AT models due to their superior robustness over other defense techniques . 2.3 META-LEARNING . Meta-learning ( learning to learn ) studies how to learn learning algorithms automatically from a meta perspective , and has shown promise in few-shot learning ( Finn et al. , 2017 ) and learning optimizers ( Andrychowicz et al. , 2016 ) . The latter is called learning to optimize ( L2O ) . The first L2O method leverages a coordinate-wise long short term memory ( LSTM ) network as the optimizer model , which is trained by gradient descent on a class of optimization problems ( Andrychowicz et al. , 2016 ) . The optimizer can also be trained by reinforcement learning ( Li & Malik , 2017 ) . Bello et al . ( 2017 ) propose to adopt an RNN controller to generate the update rules of optimization algorithms , which can be described as a domain specific language . L2O is then extended to efficiently optimize black-box functions ( Chen et al. , 2017 ) . Recent methods focus on improving the training stability and generalization of the learned optimizers by designing better optimizer models and training techniques ( Lv et al. , 2017 ; Wichrowska et al. , 2017 ; Chen et al. , 2020b ) . The general idea of meta-learning has been investigated in adversarial attacks . Most related methods learn to generate adversarial examples by using convolutional neural networks ( CNNs ) , which usually take clean images as inputs and return adversarial examples/perturbations ( Baluja & Fischer , 2017 ; Poursaeed et al. , 2018 ) . Differently , our method adopts an RNN model as the attack optimizer to mimic the behavior of iterative attacks . Empirical results validate the superiority of our approach compared to the CNN-based generators . Meta-learning has also been used in black-box attacks ( Du et al. , 2020 ) and AT defenses ( Xiong & Hsieh , 2020 ; Jiang et al. , 2021 ) . It is noteworthy that Xiong & Hsieh ( 2020 ) propose a similar approach to ours that uses an RNN optimizer to learn attacks for AT . Although the RNN optimizer is also adopted in our approach , we improve the attacking performance and generalization ability of the RNN optimizer with new techniques , as presented in Sec . 3 . More detailed comparisons between our work and Xiong & Hsieh ( 2020 ) are shown in Appendix A .

Summary: This paper proposes a model-agnostic meta-attack and achieves promising adversarial attack performance compared with state-of-the-art adversarial attack algorithms. The proposed algorithm overcomes many issues, such as the vanishing gradient problem, training instability problem, generalization to other defense models. The detailed experiments support the proposed method.