On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

1 INTRODUCTION . Meta-learning , which can offer fast generalization adaptation to unseen tasks ( Thrun & Pratt , 2012 ; Novak & Gowin , 1984 ) , has widely been studied from model- and metric-based methods ( Santoro et al. , 2016 ; Munkhdalai & Yu , 2017 ; Koch et al. , 2015 ; Snell et al. , 2017 ) to optimizationbased methods ( Ravi & Larochelle , 2016 ; Finn et al. , 2017 ; Nichol et al. , 2018 ) . In particular , model-agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) is one of the most intriguing bi-level optimization-based meta-learning methods designed for fast-adapted few-shot learning . That is , the learnt meta-model can rapidly be generalized to unforeseen tasks with only a small amount of data . It has successfully been applied to use cases such as object detection ( Wang et al. , 2020 ) , medical image analysis ( Maicas et al. , 2018 ) , and language modeling ( Huang et al. , 2018 ) . In addition to generalization-ability , recent works ( Yin et al. , 2018 ; Goldblum et al. , 2019 ; Xu et al. , 2020 ) investigated MAML from another fundamental perspective , adversarial robustness , given by the capabilities of a model defending against adversarially perturbed inputs ( known as adversarial ∗Corresponding : Ren Wang ( wangren348117609 @ gmail.com , renwang @ umich.edu ) . †Work is done at the MIT-IBM Watson AI Lab examples/attacks ) ( Goodfellow et al. , 2014 ; Xu et al. , 2019b ) . The challenge of lacking robustness of deep learning ( DL ) models has gained increasing interest and attention . And there exists a proactive arm race between adversarial attack and defense ; see overview in ( Carlini et al. , 2019 ; Hao-Chen et al. , 2020 ) . There have existed many defensive methods in the context of standard model training , e.g. , ( Madry et al. , 2017 ; Zhang et al. , 2019b ; Wong et al. , 2020 ; Carmon et al. , 2019 ; Stanforth et al. , 2019 ; Xu et al. , 2019a ) , however , few work studied robust MAML except ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) to the best of our knowledge . And tackling such a problem is more challenging than robustifying the standard model training , since MAML contains a bi-leveled learning procedure in which the meta-update step ( outer loop ) optimizes a task-agnostic initialization of model parameters while the fine-tuning step ( inner loop ) learns a task-specific model instantization updated from the common initialization . Thus , it remains elusive when ( namely , at which learning stage ) and how robust regularization should be promoted to strike a graceful balance between generalization/robustness and computation efficiency . Note that neither the standard MAML ( Finn et al. , 2017 ) nor the standard robust training ( Madry et al. , 2017 ; Zhang et al. , 2019b ) is as easy as normal training . Besides the algorithmic design in robust MAML , it is also important to draw in-depth explanation and analysis on why adversarial robustness can efficiently be gained in MAML . In this work , we aim to re-visit the problem of adversarial robustness in MAML ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) and make affirmative answers to the above questions on when , how and why . Contributions Compared to the existing works ( Yin et al. , 2018 ; Goldblum et al. , 2019 ) , we make the following contributions : • Given the fact that MAML is formed as a bi-level learning procedure , we show and explain why regularizing adversarial robustness at the meta-update level is sufficient to offer fast and effective robustness adaptation on few-shot test tasks . • Given the fact that either MAML or robust training alone is computationally intensive , we propose a general but efficient robustness-regularized meta-learning framework , which allows the use of unlabeled data augmentation , fast ( one-step ) adversarial example generation during meta-updating , and partial model training during fine-tuning ( only fine-tuning the classifier ’ s head ) . • We for the first time show that the use of unlabeled data augmentation , particularly introducing an auxiliary contrastive learning task , can provide additional benefits on adversarial robustness of MAML in the low data regime , 2 % robust accuracy improvement and 9 % clean accuracy improvement over the state-of-the-art robust MAML method ( named as adversarial querying ) in ( Goldblum et al. , 2019 ) . Related work To train a standard model ( instead of a meta-model ) , the most effective robust training methods include adversarial training ( Madry et al. , 2017 ) , TRADES that places a theoreticallygrounded trade-off between accuracy and robustness ( Zhang et al. , 2019b ) , and their many variants such as fast adversarial training methods ( Shafahi et al. , 2019 ; Zhang et al. , 2019a ; Wong et al. , 2020 ; Andriushchenko & Flammarion , 2020 ) , semi-supervised robust training ( Carmon et al. , 2019 ; Stanforth et al. , 2019 ) , adversarial transfer learning and certifiably robust training ( Wong & Kolter , 2017 ; Dvijotham et al. , 2018 ) . Moreover , recent works ( Hendrycks et al. , 2019 ; Chen et al. , 2020a ; Shafahi et al. , 2020 ; Chan et al. , 2020 ; Utrera et al. , 2020 ; Salman et al. , 2020 ) studied the transferability of robustness in in the context of transfer learning and representation learning . However , the aforementioned standard robust training methods are not directly applicable to MAML in few-shot learning considering MAML ’ s bi-leveled optimization nature . A few recent works studied the problem of adversarial training in the context of MAML ( Goldblum et al. , 2019 ; Yin et al. , 2018 ) . Yin et al . ( 2018 ) considered the robust training in both fine-tuning and meta-update steps , which is unavoidably computationally expensive and difficult in optimization . The most relevant work to ours is ( Goldblum et al. , 2019 ) , which proposed adversarial querying ( AQ ) by integrating adversarial training with MAML . Similar to ours , AQ attempted to robustify meta-update only to gain sufficient robustness . However , it lacks explanation for the rationale behind that . We will show that AQ can also be regarded as a special case of our proposed robustnesspromoting MAML framework . Most important , we make a more in-depth study with novelties summarized in Contributions . Another line of research relevant to ours is efficient MAML , e.g. , ( Raghu et al. , 2019 ; Song et al. , 2019 ; Su et al. , 2019 ) , where the goal is to improve the computation efficiency and/or the generalization of MAML . In ( Song et al. , 2019 ) , gradient-free optimization was leveraged to alleviate the need of second-order derivative information during meta-update . In ( Raghu et al. , 2019 ) , MAML was simplified by removing the fine-tuning step over the representation block of a meta-model . It was shown that such a simplification is surprisingly effective without losing generalization-ability . In ( Su et al. , 2019 ) , a self-supervised representation learning task was augmented to the meta-updating objective and resulted in a meta-model with improved generalization . Although useful insights were gained from MAML in the aforementioned works , none of them took adversarial robustness into account . 2 PRELIMINARIES AND PROBLEM STATEMENT . In this section , we first review model-agnostic meta learning ( MAML ) ( Finn et al. , 2017 ) and adversarial training ( Madry et al. , 2017 ) , respectively . We then motivate the setup of robustness-promoting MAML and demonstrate its challenges in design when integrating MAML with robust regularization . MAML MAML attempts to learn an initialization of model parameters ( namely , a meta-model ) so that a new few-shot task can quickly and easily be tackled by fine-tuning this meta-model over a small amount of labeled data . The characteristic signature of MAML is its bi-level learning procedure , where the fine-tuning stage forms a task-specific inner loop while the meta-model is updated at the outer loop by minimizing the validation error of fine-tuned models over cumulative tasks . Formally , consider N few-shot learning tasks { Ti } Ni=1 , each of which has a fine-tuning data set Di and a validation set D′i , where Di is used in the fine-tuning stage and D′i is used in the meta-update stage . Here the superscript ( ′ ) is preserved to indicate operations/parameters at the meta-upate stage . MAML is then formulated as the following bi-level optimization problem ( Finn et al. , 2017 ) : minimize w 1 N ∑N i=1 ` ′ i ( w ′ i ; D′i ) subject to w′i = argminwi ` i ( wi ; Di , w ) , ∀i ∈ [ N ] ( 1 ) where w denotes the meta-model to be designed , w′i is the Ti-specific fine-tuned model , ` ′i ( w′i ; D′i ) represents the validation error using the fine-tuned model , ` i ( wi ; Di , w ) denotes the training error when fine-tuning the task-specific model parameters wi using the task-agnostic initialization w , and for ease of notation , [ K ] represents the integer set { 1 , 2 , . . . , K } . In ( 1 ) , the objective function and the constraint correspond to the meta-update stage and fine-tuning stage , respectively . The bilevel optimization problem is challenging because each constraint calls an inner optimization oracle , which is typically instantiated into a K-step gradient descent ( GD ) based solver : w ( k ) i = w ( k−1 ) i − α∇wi ` i ( w ( k−1 ) i ; Di , w ) , k ∈ [ K ] , with w ( 0 ) i = w. We note that even with the above simplified fine-tuning step , updating the meta-model w still requires the second-order derivatives of the objective function of ( 1 ) with respect to ( w.r.t . ) w. Adversarial training The min-max optimization based adversarial training ( AT ) is known as one of the most powerful defense methods to obtain a robust model against adversarial attacks ( Madry et al. , 2017 ) . We summarize AT and its variants through the following robustness-regularized optimization problem : minimize w λE ( x , y ) ∈D [ ` ( w ; x , y ) ] + E ( x , y ) ∈D [ maximize ‖δ‖∞≤ g ( w ; x+ δ , y ) ] ︸ ︷︷ ︸ R ( w ; D ) , ( 2 ) where ` ( w ; x , y ) denotes the prediction loss evaluated at the point x with label y , λ ≥ 0 is a regularization parameter , δ denotes the input perturbation variable within the ` ∞-norm ball of radius , g represents the robust loss evaluated at the model w at the perturbed example x + δ given the true label y , and for ease of notation , let R ( w ; D ) denote the robust regularization function for model w under the data set D. In the rest of the paper , we consider two specifications of R : ( a ) AT regularization ( Madry et al. , 2017 ) , where we set g = ` and λ = 0 ; ( b ) TRADES regularization ( Zhang et al. , 2019b ) , where we define g as the cross-entropy between the distribution of prediction probabilities at the perturbed example ( x+ δ ) and that at the original sample x. Robustness-promoting MAML Integrating MAML with AT is a natural solution to enhance adversarial robustness of a meta-model in few-shot learning . However , this seemingly simple scheme is in fact far from trivial , and there exist three critical roadblocks as elaborated below . First , it remains elusive at which stage ( fine-tuning or meta-update ) robustness can most effectively be gained for MAML . Based on ( 1 ) and ( 2 ) , we can cast this problem as a unified optimization problem that augments the MAML loss with the robust regularization under two degrees of freedom characterized by two hyper-parameters γout ≥ 0 and γin ≥ 0 : minimize w 1 N ∑N i=1 [ ` ′ i ( w ′ i ; D′i ) + γoutRi ( w′i ; D′i ) ] subject to w′i = argminwi [ ` i ( wi ; Di , w ) + γinRi ( wi ; Di ) ] , ∀i ∈ [ N ] . ( 3 ) HereRi denotes the task-specific robustness regularizer , and the choice of ( γin , γout ) determines the specific scenario of robustness-promoting MAML . Clearly , the direct application is to set γin > 0 and γout > 0 , that is , both fine-tuning and meta-update steps would be carried out using robust training , which calls additional loops to generate adversarial examples . Thus , this would make computation most intensive . Spurred by that , we ask : Is it possible to achieve a robust meta-model by incorporating robust regularization into only either meta-update or fine-tuning step ( corresponding to γin = 0 or γout = 0 ) ? Second , both MAML in ( 1 ) and AT in ( 2 ) are challenging bi-level optimization problems which need to call inner optimization routines for fine-tuning and attack generation , respectively . Thus , we ask whether or not the computationally-light alternatives of inner solvers , e.g. , partial fine-tuning ( Raghu et al. , 2019 ) and fast attack generation ( Wong et al. , 2020 ) , can promise adversarial robustness in few-shot learning . Third , it has been shown that adversarial robustness can benefit from semi-supervised learning by leveraging ( unlabeled ) data augmentation ( Carmon et al. , 2019 ; Stanforth et al. , 2019 ) . Spurred by that , we further ask : Is it possible to generalize robustness-promoting MAML to the setup of semi-supervised learning for improved accuracy-robustness tradeoff ?

Summary: This paper explores a way to promote the adversarial robustness in Model-agnostic meta-learning (MAML). It conducts extensive experiments to show regularizing adversarial robustness at meta-update level is sufficient to offer fast and effective robustness adaptation on few-shot test tasks. However, it lacks the theoretical analysis for this conclusion. Also, the experiments only are conducted on few-shot image classification task on miniImageNet dataset. This makes this conclusion lack sufficient credibility.