Attacking Few-Shot Classifiers with Adversarial Support Sets

1 INTRODUCTION . Standard deep learning approaches suffer from poor sample efficiency ( Krizhevsky et al. , 2012 ) which is problematic in tasks where data collection is difficult or expensive . Recently , few-shot learners have been developed which address this shortcoming by supporting rapid adaptation to a new task using only a few labeled examples ( Finn et al. , 2017 ; Snell et al. , 2017 ) . This success has meant that few-shot learners are becoming increasingly attractive for real-life applications . They have been applied to user personalization in recommender systems ( Lee et al. , 2019 ) , matching potential users to businesses ( Li et al. , 2020 ) , personalized talking head models ( Zakharov et al. , 2019 ) , and on-device gaze estimation ( He et al. , 2019 ) . As few-shot learners improve , they are also being applied to increasingly sensitive applications where the repercussions of confidentlywrong predictions are severe . Examples include clinical risk assessment ( Sheryl Zhang et al. , 2019 ) , glaucoma diagnosis ( Kim et al. , 2017 ) , identification of diseases in skin lesions ( Mahajan et al. , 2020 ) , and tissue slide annotation in cancer immuno-therapy biomarker research ( Lahiani et al. , 2018 ) . As few-shot learners gain popularity , it is essential to understand how robust they are and whether there are potential avenues for their exploitation . It is well known that standard classifiers are vulnerable to inputs that have been purposefully modified in a minor way to cause incorrect predictions ( Biggio & Roli , 2017 ) . Such examples may be presented to a model either at test time , called evasion attacks ( Biggio et al. , 2017 ) or adversarial examples ( Szegedy et al. , 2014 ) , or at training time , which is referred to as poisoning ( Newsome et al. , 2006 ; Rubinstein et al. , 2009 ) . While previous work has considered adversarial attacks on few shot learners , data poisoning attacks have not been studied and are the focus of this paper . Data poisoning attacks are of particular relevance in the few-shot learning setting for two reasons . First , since the datasets are small , a handful of poisoned patterns might have a significant effect . Second , many applications of few-shot learning require labeled data from users to adapt the system to a new task , essentially providing a direct interface for outsiders to influence the model ’ s behaviour . If few-shot learning systems are not robust to poisoning of their training dataset , then this weakness could be exploited . An attacker performing a man-in-the-middle ( Conti et al. , 2016 ) data poisoning attack could cause a recommender system ’ s personalization to perform badly or suggest certain results to influence a user ’ s decision . Applied at scale to many users , an attacker could cause significant damage . Similarly , a doctor attempting to commit medical insurance fraud may submit images causing a benign skin condition to be incorrectly classified as a skin disease requiring expensive treatment ; or a malicious party may ruin a research study that uses automated annotation of samples by tampering imperceptibly with only a few images . If these attacks could be achieved with malicious patterns that can not be reliably distinguished from real training data , it would be difficult to defend against them . Before detailing the key contributions of the paper , it is necessary to briefly introduce the lexicon of few-shot learning . During training , few-shot learners are typically presented with many different tasks . The model must learn to perform well on each task , hopefully arriving at a point where it can adapt effectively to a new task at test time . At test time , the model is presented with an unseen task containing a few labeled examples , the support set , and a number of unlabeled examples to classify , called the query set . The paper makes the following contributions : 1 . We define a novel attack on few-shot classifiers , called an Adversarial Support Set Attack , which applies adversarial perturbations to the support set that are calculated to minimize model accuracy over a set of query points . To the best of the authors ’ knowledge , this is the first work considering the impact of poisoning attacks on trained few-shot classifiers . 2 . We demonstrate that few-shot classifiers are surprisingly vulnerable to Adversarial Support Set attacks . The adversarial support set attack is more effective than the baselines considered , and generalizes well , i.e . the compromised classifier is highly likely to be inaccurate on a randomly sampled query set from the task domain . 3 . We demonstrate the effectiveness of our approach against a variety of few-shot classifiers including MAML ( Finn et al. , 2017 ) , ProtoNets ( Snell et al. , 2017 ) , and CNAPS ( Requeima et al. , 2019a ) , on both small scale ( miniImageNet ( Vinyals et al. , 2016 ) ) and large scale ( META-DATASET ( Triantafillou et al. , 2020 ) ) few-shot classification benchmarks . 4 . We show that adversarial support sets transfer effectively to fine-tuning based few-shot classifiers when the few-shot classifier and the fine-tuner utilize similar feature extractors . The rest of the paper proceeds as follows : Section 2 provides background about the meta-learning models under consideration , relevant adversarial attack methods , and the threat model under consideration . Section 3 discusses how evasion and poisoning attacks may be generalized to few-shot learners . Section 4 presents the experimental results and Section 5 concludes the paper . Additional results and experimental details are in the Appendix . 2 BACKGROUND . In this section we lay the necessary groundwork for adversarial support set attacks . We focus on image classification . We denote input images x ∈ Rch×W×H where W is the image width , H the image height , ch the number of image channels and image labels y ∈ { 1 , . . . , C } where C is the number of image classes . We use bold x and y to denote a set of images and labels , respectively . 2.1 META-LEARNING . We consider the few-shot image classification scenario using a meta-learning approach . Rather than a single , large dataset D , we assume access to a dataset D = { τt } Kt=1 comprising a large number of training tasks τt , drawn i.i.d . from a distribution p ( τ ) . The data for a task consists of a support set DS = { ( xn , yn ) } Nn=1 comprising N elements , with the inputs xn and labels yn observed , and a query set DQ = { ( x∗m , y∗m ) } Mm=1 with M elements for which we wish to make predictions . We may use the shorthand DS = { x , y } and DQ = { x∗ , y∗ } for brevity . Here the inputs x∗ are observed and the labels y∗ are only observed during meta-training ( i.e . training of the meta-learning algorithm ) . Note that the query set examples are drawn from the same set of labels as the examples in the support set . At meta-test time , the classifier f is required to make predictions for query set inputs of unseen tasks . Often , the meta-test tasks will include classes that have not been seen during meta-training , and DS will contain only a few observations . An example task is shown in Fig . 1 . Episodic Training The majority of modern meta-learning methods employ episodic training ( Vinyals et al. , 2016 ) . During meta-training , a task τ is drawn from p ( τ ) and randomly split into a support set DS and query set DQ . The meta-learner g takes as input the support set DS and produces task-specific classifier parameters ψ = g ( DS ) which are used to adapt the classifier f to the current task . The classifier can now make task-specific predictions f ( x∗ , ψ = g ( DS ) ) for any test input x∗ ∈ DQ . Refer to the diagram labeled Clean in Fig . 2 . A loss function L ( f ( x∗ , ψ ) , y∗ ) then computes the loss between the predictions for the label f ( x∗ , ψ ) and the true label y∗ . Assuming that L , f , and g are differentiable , the meta-learning algorithm can then be trained with stochastic gradient descent by back-propagating the loss and updating the parameters of f and g. Common Few-shot Learning Algorithms There has been an explosion of meta-learning based few-shot learning algorithms proposed in recent years . For an in-depth review see Hospedales et al . ( 2020 ) . Here , we briefly describe methods that are relevant to our experiments . Arguably the most widely used is the gradient-based approach , the canonical example for modern systems being MAML ( Finn et al. , 2017 ) . With MAML , the task-specific parameters ψ are the parameters of the classifier f after applying one or more gradient steps taken on the support set DS . Another widely used class of meta-learners are amortized-inference or black box based approaches e.g , VERSA ( Gordon et al. , 2019 ) and CNAPS ( Requeima et al. , 2019a ) . In these methods , the task-specific parameters ψ are generated by one or more hyper-networks , g ( Ha et al. , 2016 ) . An important special case of this approach is Prototypical Networks ( ProtoNets ) ( Snell et al. , 2017 ) which is based on metric learning and employs a nearest neighbor classifier , and the task-specific parameters ψ are the mean feature vectors for each class c ∈ C of the support set DS . Finally , a very competitive , non-episodic approach to few-shot learning is simple fine-tuning ( Yosinski et al. , 2014 ; Tian et al. , 2020 ) . In this approach , a pretrained feature extractor is used in combination with a final layer linear classifier , whose weights serve as the task-specific parameters ψ that are learned by taking a number of gradient steps on the support set data . 2.2 ADVERSARIAL ATTACKS . Szegedy et al . ( 2014 ) have shown that it is possible to craft adversarial examples , inputs that are indistinguishable from legitimate examples to both humans and algorithms ( Carlini & Wagner , 2017 ) , yet are classified incorrectly by deep neural networks with high confidence ( Nguyen et al. , 2014 ) . Adversarial examples are typically generated by taking a clean , unperturbed input image and adding a small amount of carefully crafted , almost imperceptible noise ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ) . The perturbation δ is calculated by performing the following optimization argmaxδ L ( θ , x + δ , y ) , where L is the loss function e.g . the cross entropy , θ denotes the model parameters , x is the input being perturbed , and y is its label . The perturbation size is typically constrained by some norm such as ` ∞ or ` 2 . As stated above , the optimization produces an untargeted attack , in the sense that we do not require the generated image to be misclassified as any particular class , only to be classified incorrectly . An alternative formulation of the optimization problem , argminδ L ( θ , x + δ , ỹ ) , would produce a targeted attack , i.e . an image that is misclassified as belonging specifically to the class with label ỹ 6= y . In this paper , labels for targeted attacks are generated by shifting the clean class label by 1 mod C. Many algorithms can be used to generate adversarial examples , including Projected Gradient Descent ( Madry et al. , 2017 ) , the Carlini and Wagner ` 2 attack ( Carlini & Wagner , 2017 ) , or ElasticNet ( Chen et al. , 2017a ) . Throughout this paper , without loss of generality , we utilize Projected Gradient Descent ( PGD ) with an ` ∞ norm , because is it simple to implement , inexpensive to compute , and effective . We show that by careful application of PGD , we can successfully poison meta-learners at meta-test time .

Summary: This paper introduces Adversarial Support Set Attack, an attack that perturbs the support set samples under the few-shot paradigm. It makes use of a seed query set to find adversarial samples that maximize the loss on this set, with the hope that it will generalize to any query sample. Empirical results show that this attack is effective against a variety of few-shot algorithms and datasets. Additionally, adversarial samples constructed on traditional few-shot algorithms are empirically shown to transfer to the fine-tuning few-shot algorithm (when using similar feature extractors).