Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

1 INTRODUCTION . Deep neural networks are vulnerable to maliciously crafted adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) , which aim to induce erroneous model predictions by adding small perturbations to the normal inputs . Due to the threats , a multitude of defense strategies have been proposed to improve adversarial robustness ( Kurakin et al. , 2017b ; Madry et al. , 2018 ; Liao et al. , 2018 ; Wong & Kolter , 2018 ; Zhang et al. , 2019b ; Dong et al. , 2020a ; Pang et al. , 2020 ) . However , many defenses have later been shown to be ineffective due to incomplete or incorrect robustness evaluations ( Athalye et al. , 2018a ; Uesato et al. , 2018 ; Carlini et al. , 2019 ; Croce & Hein , 2020b ; Dong et al. , 2020b ; Tramer et al. , 2020 ) , making it particularly challenging to understand their effects and identify the actual progress of the field . Therefore , developing methods that can evaluate adversarial robustness accurately and reliably is of profound importance . Adversarial attacks are broadly adopted as an indispensable solution to evaluate adversarial robustness for different models . One of the most prominent attacks is the projected gradient descent ( PGD ) method ( Madry et al. , 2018 ) , which generates an adversarial example by performing iterative gradient updates to maximize a classification loss ( e.g. , cross-entropy loss ) w.r.t . the input . Although recent methods improve upon PGD by introducing different loss functions ( Carlini & Wagner , 2017 ; Gowal et al. , 2019 ; Croce & Hein , 2020b ; Sriramanan et al. , 2020 ) and adjusting the step size ( Croce & Hein , 2020b ) , most of them adopt hand-designed optimization algorithms , such as vanilla gradient descent , momentum ( Polyak , 1964 ) , and Adam ( Kingma & Ba , 2015 ) . However , it has been shown that these attacks can be sub-optimal ( Croce & Hein , 2020b ; Tramer et al. , 2020 ) , arousing an overestimation of adversarial robustness for some defenses . It is thus imperative to develop more effective optimization algorithms for improving adversarial attacks . Nevertheless , designing a generic optimization algorithm in a hand-crafted manner is non-trivial , considering the different defense models with varying network architectures , defense strategies , and training datasets . To overcome this issue and develop stronger adversarial attacks , in this paper , we propose a ModelAgnostic Meta-Attack ( MAMA ) approach to learn optimization algorithms in adversarial attacks automatically . In particular , we parameterize the optimizer with a recurrent neural network ( RNN ) to mimic the behavior of iterative attacks , which outputs an update direction at each time step during adversarial example generation . By learning to solve a class of optimization problems with different data samples and defense models , the RNN optimizer could benefit from long-term dependencies of iterative attacks and exploit common structures among the optimization problems . To improve and stabilize training , we then propose a prior-guided refinement strategy , which imposes a prior given by the PGD update rule on the outputs of the RNN optimizer , whose parameters are trained via the maximum a posteriori ( MAP ) estimation framework . Consequently , the RNN optimizer can learn to yield more effective update directions than hand-designed ones . Despite the effectiveness , the learned optimizer may not generalize well to attacking unseen defenses due to potential overfitting to the defenses used for training . Although training a different optimizer for each defense is possible , the training process can be time-consuming and inconvenient in practice when it is utilized to benchmark adversarial robustness . To endow the learned optimizer with a better generalization ability , we develop a model-agnostic training algorithm with a gradient-based metatrain and meta-test process to simulate the shift from seen to unseen defenses . Therefore , the learned optimizer can directly be deployed to attack unseen defenses without retraining or fine-tuning . Extensive experiments validate the effectiveness and generalization ability of the learned optimizers for attacking different defense models . We also demonstrate the flexibility of our method integrating with various attacks , including those with different loss functions ( Carlini & Wagner , 2017 ; Gowal et al. , 2019 ; Croce & Hein , 2020b ) and initialization ( Tashiro et al. , 2020 ) . Our method consistently improves the attacking performance over baselines , while introducing little extra computational cost . By incorporating our method with an orthogonal technique — output diversified initialization ( ODI ) ( Tashiro et al. , 2020 ) , we achieve lower robust test accuracy on all 12 defense models that we study than other state-of-the-art attacks , including the MultiTargeted ( MT ) attack ( Gowal et al. , 2019 ) and AutoAttack ( Croce & Hein , 2020b ) , leading to a more reliable evaluation of adversarial robustness . Moreover , MAMA provides a plug-and-play module in adversarial attacks , which may benefit future adversarial attacks if new improvements have been made in different aspects . 2 PRELIMINARIES AND RELATED WORK . 2.1 ADVERSARIAL ATTACKS . Let f ( x ) : x ∈ RD → z ∈ RK denote a classifier , which outputs the logits z : = [ z1 , ... , zK ] over K classes for an input image x . The prediction of f is Cf ( x ) = arg maxi=1 , ... , K zi . Given the true label y of x , adversarial attacks aim to generate an adversarial example x̂ that is misclassified by f ( i.e. , Cf ( x̂ ) 6= y ) , while the distance between the adversarial example x̂ and the natural one x measured by the ` ∞ norm is smaller than a threshold as ‖x̂ − x‖∞ ≤ . Although we introduce our approach based on the ` ∞ norm only , the extension to other ` p norms is straightforward . The adversarial example x̂ can be generated by solving a constrained optimization problem as x̂ = arg max x′ L ( f ( x′ ) , y ) , s.t . ‖x′ − x‖∞ ≤ , ( 1 ) where L is a loss function on top of the classifier f ( x ) . For example , L could be the cross-entropy loss as Lce ( f ( x ) , y ) = − log py where py = ezy/∑Ki=1 ezi denotes the predicted probability of class y , or the margin-based CW loss ( Carlini & Wagner , 2017 ) as Lcw ( f ( x ) , y ) = maxi6=y zi − zy . A lot of gradient-based methods ( Goodfellow et al. , 2015 ; Kurakin et al. , 2017a ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Dong et al. , 2018 ) have been proposed to solve this optimization problem . The projected gradient descent ( PGD ) method ( Madry et al. , 2018 ) performs iterative updates as x̂t+1 = ΠB ( x ) ( x̂t + αt · sign ( ∇xL ( f ( x̂t ) , y ) ) ) , ( 2 ) where B ( x ) = { x′ : ‖x′ − x‖∞ ≤ } denotes the ` ∞ ball centered at x with radius , Π ( · ) is the projection operation , and αt is the step size at the t-th attack iteration . PGD initializes x̂0 randomly by uniform sampling within B ( x ) and adopts a fixed step size α in every iteration . To develop stronger attacks for reliable robustness evaluation , recent improvements upon PGD include adopting different loss functions ( Gowal et al. , 2019 ; Croce & Hein , 2020b ; Sriramanan et al. , 2020 ) , adjusting the step size ( Croce & Hein , 2020b ) , and designing sampling strategies of initialization ( Tashiro et al. , 2020 ) . For the update rules , besides the vanilla gradient descent adopted in PGD , recent attacks ( Dong et al. , 2018 ; Gowal et al. , 2019 ) suggest to use the momentum ( Polyak , 1964 ) and Adam ( Kingma & Ba , 2015 ) optimizers . In contrast , we aim to enhance adversarial attacks by learning optimization algorithms in an automatic way . 2.2 ADVERSARIAL DEFENSES . To build robust models against adversarial attacks , numerous defense strategies have been proposed , but most defenses can be evaded by new adaptive attacks ( Athalye et al. , 2018a ; Tramer et al. , 2020 ) . Among the existing defenses , adversarial training ( AT ) is arguably the most effective defense technique , in which the network is trained on the adversarial examples generated by attacks . Based on the primary AT frameworks like PGD-AT ( Madry et al. , 2018 ) and TRADES ( Zhang et al. , 2019b ) , improvements have been made via ensemble learning ( Tramèr et al. , 2018 ; Pang et al. , 2019 ) , metric learning ( Mao et al. , 2019 ; Pang et al. , 2020 ) , semi-supervised learning ( Alayrac et al. , 2019 ; Carmon et al. , 2019 ; Zhai et al. , 2019 ) , and self-supervised learning ( Chen et al. , 2020a ; Hendrycks et al. , 2019 ; Kim et al. , 2020 ) . Due to the high computational cost of AT , other efforts are devoted to accelerating the training procedure ( Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ) . Recent works emphasize the training tricks ( e.g. , weight decay , batch size , etc . ) in AT ( Gowal et al. , 2020 ; Pang et al. , 2021 ) . However , it has been shown that the robust test accuracy of numerous adversarially trained models can be degraded significantly by using stronger attacks ( Croce & Hein , 2020b ; Tramer et al. , 2020 ) , indicating that the reliable evaluation of adversarial robustness remains an imperative yet challenging task . In this paper , we focus on evaluating adversarial robustness of AT models due to their superior robustness over other defense techniques . 2.3 META-LEARNING . Meta-learning ( learning to learn ) studies how to learn learning algorithms automatically from a meta perspective , and has shown promise in few-shot learning ( Finn et al. , 2017 ) and learning optimizers ( Andrychowicz et al. , 2016 ) . The latter is called learning to optimize ( L2O ) . The first L2O method leverages a coordinate-wise long short term memory ( LSTM ) network as the optimizer model , which is trained by gradient descent on a class of optimization problems ( Andrychowicz et al. , 2016 ) . The optimizer can also be trained by reinforcement learning ( Li & Malik , 2017 ) . Bello et al . ( 2017 ) propose to adopt an RNN controller to generate the update rules of optimization algorithms , which can be described as a domain specific language . L2O is then extended to efficiently optimize black-box functions ( Chen et al. , 2017 ) . Recent methods focus on improving the training stability and generalization of the learned optimizers by designing better optimizer models and training techniques ( Lv et al. , 2017 ; Wichrowska et al. , 2017 ; Chen et al. , 2020b ) . The general idea of meta-learning has been investigated in adversarial attacks . Most related methods learn to generate adversarial examples by using convolutional neural networks ( CNNs ) , which usually take clean images as inputs and return adversarial examples/perturbations ( Baluja & Fischer , 2017 ; Poursaeed et al. , 2018 ) . Differently , our method adopts an RNN model as the attack optimizer to mimic the behavior of iterative attacks . Empirical results validate the superiority of our approach compared to the CNN-based generators . Meta-learning has also been used in black-box attacks ( Du et al. , 2020 ) and AT defenses ( Xiong & Hsieh , 2020 ; Jiang et al. , 2021 ) . It is noteworthy that Xiong & Hsieh ( 2020 ) propose a similar approach to ours that uses an RNN optimizer to learn attacks for AT . Although the RNN optimizer is also adopted in our approach , we improve the attacking performance and generalization ability of the RNN optimizer with new techniques , as presented in Sec . 3 . More detailed comparisons between our work and Xiong & Hsieh ( 2020 ) are shown in Appendix A .

Summary: This paper proposes an automatic approach for attacking classifiers, by approximating a possible adaptive attack that can take place on a newly-published defense. The methodology (MAMA) applies meta machine learning techniques, by training it on different defenses to let it grasp what might be a good signal to follow when attacking an unknown classifier. The authors then test this strategy and a naive version of it (that is, the one that is only trained on the attacks, and not fine tuned on a test set) against other attacks proposed in literature. Results suggest a modest improvement against defenses hardened with adversarial training.