On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

1 INTRODUCTION . Meta-learning , which can offer fast generalization adaptation to unseen tasks ( Thrun & Pratt , 2012 ; Novak & Gowin , 1984 ) , has widely been studied from model- and metric-based methods ( Santoro et al. , 2016 ; Munkhdalai & Yu , 2017 ; Koch et al. , 2015 ; Snell et al. , 2017 ) to optimizationbased methods ( Ravi & Larochelle , 2016 ; Finn et al. , 2017 ; Nichol et al. , 2018 ) . In particular , model-agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) is one of the most intriguing bi-level optimization-based meta-learning methods designed for fast-adapted few-shot learning . That is , the learnt meta-model can rapidly be generalized to unforeseen tasks with only a small amount of data . It has successfully been applied to use cases such as object detection ( Wang et al. , 2020 ) , medical image analysis ( Maicas et al. , 2018 ) , and language modeling ( Huang et al. , 2018 ) . In addition to generalization-ability , recent works ( Yin et al. , 2018 ; Goldblum et al. , 2019 ; Xu et al. , 2020 ) investigated MAML from another fundamental perspective , adversarial robustness , given by the capabilities of a model defending against adversarially perturbed inputs ( known as adversarial ∗Corresponding : Ren Wang ( wangren348117609 @ gmail.com , renwang @ umich.edu ) . †Work is done at the MIT-IBM Watson AI Lab examples/attacks ) ( Goodfellow et al. , 2014 ; Xu et al. , 2019b ) . The challenge of lacking robustness of deep learning ( DL ) models has gained increasing interest and attention . And there exists a proactive arm race between adversarial attack and defense ; see overview in ( Carlini et al. , 2019 ; Hao-Chen et al. , 2020 ) . There have existed many defensive methods in the context of standard model training , e.g. , ( Madry et al. , 2017 ; Zhang et al. , 2019b ; Wong et al. , 2020 ; Carmon et al. , 2019 ; Stanforth et al. , 2019 ; Xu et al. , 2019a ) , however , few work studied robust MAML except ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) to the best of our knowledge . And tackling such a problem is more challenging than robustifying the standard model training , since MAML contains a bi-leveled learning procedure in which the meta-update step ( outer loop ) optimizes a task-agnostic initialization of model parameters while the fine-tuning step ( inner loop ) learns a task-specific model instantization updated from the common initialization . Thus , it remains elusive when ( namely , at which learning stage ) and how robust regularization should be promoted to strike a graceful balance between generalization/robustness and computation efficiency . Note that neither the standard MAML ( Finn et al. , 2017 ) nor the standard robust training ( Madry et al. , 2017 ; Zhang et al. , 2019b ) is as easy as normal training . Besides the algorithmic design in robust MAML , it is also important to draw in-depth explanation and analysis on why adversarial robustness can efficiently be gained in MAML . In this work , we aim to re-visit the problem of adversarial robustness in MAML ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) and make affirmative answers to the above questions on when , how and why . Contributions Compared to the existing works ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) , we make the following contributions : • Given the fact that MAML is formed as a bi-level learning procedure , we show and explain why regularizing adversarial robustness at the meta-update level is sufficient to offer fast and effective robustness adaptation on few-shot test tasks . • Given the fact that either MAML or robust training alone is computationally intensive , we propose a general but efficient robustness-regularized meta-learning framework , which allows the use of unlabeled data augmentation , fast ( one-step ) adversarial example generation during meta-updating , and partial model training during fine-tuning ( only fine-tuning the classifier ’ s head ) . • We for the first time show that the use of unlabeled data augmentation , particularly introducing an auxiliary contrastive learning task , can provide additional benefits on adversarial robustness of MAML in the low data regime , 2 % robust accuracy improvement and 9 % clean accuracy improvement over the state-of-the-art robust MAML method ( named as adversarial querying ) in ( Goldblum et al. , 2019 ) . Related work To train a standard model ( instead of a meta-model ) , the most effective robust training methods include adversarial training ( Madry et al. , 2017 ) , TRADES that places a theoreticallygrounded trade-off between accuracy and robustness ( Zhang et al. , 2019b ) , and their many variants such as fast adversarial training methods ( Shafahi et al. , 2019 ; Zhang et al. , 2019a ; Wong et al. , 2020 ; Andriushchenko & Flammarion , 2020 ) , semi-supervised robust training ( Carmon et al. , 2019 ; Stanforth et al. , 2019 ) , adversarial transfer learning and certifiably robust training ( Wong & Kolter , 2017 ; Dvijotham et al. , 2018 ) . Moreover , recent works ( Hendrycks et al. , 2019 ; Chen et al. , 2020a ; Shafahi et al. , 2020 ; Chan et al. , 2020 ; Utrera et al. , 2020 ; Salman et al. , 2020 ) studied the transferability of robustness in in the context of transfer learning and representation learning . However , the aforementioned standard robust training methods are not directly applicable to MAML in few-shot learning considering MAML ’ s bi-leveled optimization nature . A few recent works studied the problem of adversarial training in the context of MAML ( Goldblum et al. , 2019 ; Yin et al. , 2018 ) . Yin et al . ( 2018 ) considered the robust training in both fine-tuning and meta-update steps , which is unavoidably computationally expensive and difficult in optimization . The most relevant work to ours is ( Goldblum et al. , 2019 ) , which proposed adversarial querying ( AQ ) by integrating adversarial training with MAML . Similar to ours , AQ attempted to robustify meta-update only to gain sufficient robustness . However , it lacks explanation for the rationale behind that . We will show that AQ can also be regarded as a special case of our proposed robustnesspromoting MAML framework . Most important , we make a more in-depth study with novelties summarized in Contributions . Another line of research relevant to ours is efficient MAML , e.g. , ( Raghu et al. , 2019 ; Song et al. , 2019 ; Su et al. , 2019 ) , where the goal is to improve the computation efficiency and/or the generalization of MAML . In ( Song et al. , 2019 ) , gradient-free optimization was leveraged to alleviate the need of second-order derivative information during meta-update . In ( Raghu et al. , 2019 ) , MAML was simplified by removing the fine-tuning step over the representation block of a meta-model . It was shown that such a simplification is surprisingly effective without losing generalization-ability . In ( Su et al. , 2019 ) , a self-supervised representation learning task was augmented to the meta-updating objective and resulted in a meta-model with improved generalization . Although useful insights were gained from MAML in the aforementioned works , none of them took adversarial robustness into account . 2 PRELIMINARIES AND PROBLEM STATEMENT . In this section , we first review model-agnostic meta learning ( MAML ) ( Finn et al. , 2017 ) and adversarial training ( Madry et al. , 2017 ) , respectively . We then motivate the setup of robustness-promoting MAML and demonstrate its challenges in design when integrating MAML with robust regularization . MAML MAML attempts to learn an initialization of model parameters ( namely , a meta-model ) so that a new few-shot task can quickly and easily be tackled by fine-tuning this meta-model over a small amount of labeled data . The characteristic signature of MAML is its bi-level learning procedure , where the fine-tuning stage forms a task-specific inner loop while the meta-model is updated at the outer loop by minimizing the validation error of fine-tuned models over cumulative tasks . Formally , consider N few-shot learning tasks { Ti } Ni=1 , each of which has a fine-tuning data set Di and a validation set D′i , where Di is used in the fine-tuning stage and D′i is used in the meta-update stage . Here the superscript ( ′ ) is preserved to indicate operations/parameters at the meta-upate stage . MAML is then formulated as the following bi-level optimization problem ( Finn et al. , 2017 ) : minimize w 1 N ∑N i=1 ` ′ i ( w ′ i ; D′i ) subject to w′i = argminwi ` i ( wi ; Di , w ) , ∀i ∈ [ N ] ( 1 ) where w denotes the meta-model to be designed , w′i is the Ti-specific fine-tuned model , ` ′i ( w′i ; D′i ) represents the validation error using the fine-tuned model , ` i ( wi ; Di , w ) denotes the training error when fine-tuning the task-specific model parameters wi using the task-agnostic initialization w , and for ease of notation , [ K ] represents the integer set { 1 , 2 , . . . , K } . In ( 1 ) , the objective function and the constraint correspond to the meta-update stage and fine-tuning stage , respectively . The bilevel optimization problem is challenging because each constraint calls an inner optimization oracle , which is typically instantiated into a K-step gradient descent ( GD ) based solver : w ( k ) i = w ( k−1 ) i − α∇wi ` i ( w ( k−1 ) i ; Di , w ) , k ∈ [ K ] , with w ( 0 ) i = w. We note that even with the above simplified fine-tuning step , updating the meta-model w still requires the second-order derivatives of the objective function of ( 1 ) with respect to ( w.r.t . ) w. Adversarial training The min-max optimization based adversarial training ( AT ) is known as one of the most powerful defense methods to obtain a robust model against adversarial attacks ( Madry et al. , 2017 ) . We summarize AT and its variants through the following robustness-regularized optimization problem : minimize w λE ( x , y ) ∈D [ ` ( w ; x , y ) ] + E ( x , y ) ∈D [ maximize ‖δ‖∞≤ g ( w ; x+ δ , y ) ] ︸ ︷︷ ︸ R ( w ; D ) , ( 2 ) where ` ( w ; x , y ) denotes the prediction loss evaluated at the point x with label y , λ ≥ 0 is a regularization parameter , δ denotes the input perturbation variable within the ` ∞-norm ball of radius , g represents the robust loss evaluated at the model w at the perturbed example x + δ given the true label y , and for ease of notation , let R ( w ; D ) denote the robust regularization function for model w under the data set D. In the rest of the paper , we consider two specifications of R : ( a ) AT regularization ( Madry et al. , 2017 ) , where we set g = ` and λ = 0 ; ( b ) TRADES regularization ( Zhang et al. , 2019b ) , where we define g as the cross-entropy between the distribution of prediction probabilities at the perturbed example ( x+ δ ) and that at the original sample x. Robustness-promoting MAML Integrating MAML with AT is a natural solution to enhance adversarial robustness of a meta-model in few-shot learning . However , this seemingly simple scheme is in fact far from trivial , and there exist three critical roadblocks as elaborated below . First , it remains elusive at which stage ( fine-tuning or meta-update ) robustness can most effectively be gained for MAML . Based on ( 1 ) and ( 2 ) , we can cast this problem as a unified optimization problem that augments the MAML loss with the robust regularization under two degrees of freedom characterized by two hyper-parameters γout ≥ 0 and γin ≥ 0 : minimize w 1 N ∑N i=1 [ ` ′ i ( w ′ i ; D′i ) + γoutRi ( w′i ; D′i ) ] subject to w′i = argminwi [ ` i ( wi ; Di , w ) + γinRi ( wi ; Di ) ] , ∀i ∈ [ N ] . ( 3 ) HereRi denotes the task-specific robustness regularizer , and the choice of ( γin , γout ) determines the specific scenario of robustness-promoting MAML . Clearly , the direct application is to set γin > 0 and γout > 0 , that is , both fine-tuning and meta-update steps would be carried out using robust training , which calls additional loops to generate adversarial examples . Thus , this would make computation most intensive . Spurred by that , we ask : Is it possible to achieve a robust meta-model by incorporating robust regularization into only either meta-update or fine-tuning step ( corresponding to γin = 0 or γout = 0 ) ? Second , both MAML in ( 1 ) and AT in ( 2 ) are challenging bi-level optimization problems which need to call inner optimization routines for fine-tuning and attack generation , respectively . Thus , we ask whether or not the computationally-light alternatives of inner solvers , e.g. , partial fine-tuning ( Raghu et al. , 2019 ) and fast attack generation ( Wong et al. , 2020 ) , can promise adversarial robustness in few-shot learning . Third , it has been shown that adversarial robustness can benefit from semi-supervised learning by leveraging ( unlabeled ) data augmentation ( Carmon et al. , 2019 ; Stanforth et al. , 2019 ) . Spurred by that , we further ask : Is it possible to generalize robustness-promoting MAML to the setup of semi-supervised learning for improved accuracy-robustness tradeoff ?

Summary: It is an interesting paper empirically addressing adversarial robustness of model agnostic meta learning (MAML). The paper investigates where to incorporate robust regularization in MAML in order to improve adversarial robustness, and based on that *efficient* robust MAML methods are proposed. Interestingly, contrastive learning is incorporated and derive a more robust MAML model.