Attacking Few-Shot Classifiers with Adversarial Support Sets

1 INTRODUCTION . Standard deep learning approaches suffer from poor sample efficiency ( Krizhevsky et al. , 2012 ) which is problematic in tasks where data collection is difficult or expensive . Recently , few-shot learners have been developed which address this shortcoming by supporting rapid adaptation to a new task using only a few labeled examples ( Finn et al. , 2017 ; Snell et al. , 2017 ) . This success has meant that few-shot learners are becoming increasingly attractive for real-life applications . They have been applied to user personalization in recommender systems ( Lee et al. , 2019 ) , matching potential users to businesses ( Li et al. , 2020 ) , personalized talking head models ( Zakharov et al. , 2019 ) , and on-device gaze estimation ( He et al. , 2019 ) . As few-shot learners improve , they are also being applied to increasingly sensitive applications where the repercussions of confidentlywrong predictions are severe . Examples include clinical risk assessment ( Sheryl Zhang et al. , 2019 ) , glaucoma diagnosis ( Kim et al. , 2017 ) , identification of diseases in skin lesions ( Mahajan et al. , 2020 ) , and tissue slide annotation in cancer immuno-therapy biomarker research ( Lahiani et al. , 2018 ) . As few-shot learners gain popularity , it is essential to understand how robust they are and whether there are potential avenues for their exploitation . It is well known that standard classifiers are vulnerable to inputs that have been purposefully modified in a minor way to cause incorrect predictions ( Biggio & Roli , 2017 ) . Such examples may be presented to a model either at test time , called evasion attacks ( Biggio et al. , 2017 ) or adversarial examples ( Szegedy et al. , 2014 ) , or at training time , which is referred to as poisoning ( Newsome et al. , 2006 ; Rubinstein et al. , 2009 ) . While previous work has considered adversarial attacks on few shot learners , data poisoning attacks have not been studied and are the focus of this paper . Data poisoning attacks are of particular relevance in the few-shot learning setting for two reasons . First , since the datasets are small , a handful of poisoned patterns might have a significant effect . Second , many applications of few-shot learning require labeled data from users to adapt the system to a new task , essentially providing a direct interface for outsiders to influence the model ’ s behaviour . If few-shot learning systems are not robust to poisoning of their training dataset , then this weakness could be exploited . An attacker performing a man-in-the-middle ( Conti et al. , 2016 ) data poisoning attack could cause a recommender system ’ s personalization to perform badly or suggest certain results to influence a user ’ s decision . Applied at scale to many users , an attacker could cause significant damage . Similarly , a doctor attempting to commit medical insurance fraud may submit images causing a benign skin condition to be incorrectly classified as a skin disease requiring expensive treatment ; or a malicious party may ruin a research study that uses automated annotation of samples by tampering imperceptibly with only a few images . If these attacks could be achieved with malicious patterns that can not be reliably distinguished from real training data , it would be difficult to defend against them . Before detailing the key contributions of the paper , it is necessary to briefly introduce the lexicon of few-shot learning . During training , few-shot learners are typically presented with many different tasks . The model must learn to perform well on each task , hopefully arriving at a point where it can adapt effectively to a new task at test time . At test time , the model is presented with an unseen task containing a few labeled examples , the support set , and a number of unlabeled examples to classify , called the query set . The paper makes the following contributions : 1 . We define a novel attack on few-shot classifiers , called an Adversarial Support Set Attack , which applies adversarial perturbations to the support set that are calculated to minimize model accuracy over a set of query points . To the best of the authors ’ knowledge , this is the first work considering the impact of poisoning attacks on trained few-shot classifiers . 2 . We demonstrate that few-shot classifiers are surprisingly vulnerable to Adversarial Support Set attacks . The adversarial support set attack is more effective than the baselines considered , and generalizes well , i.e . the compromised classifier is highly likely to be inaccurate on a randomly sampled query set from the task domain . 3 . We demonstrate the effectiveness of our approach against a variety of few-shot classifiers including MAML ( Finn et al. , 2017 ) , ProtoNets ( Snell et al. , 2017 ) , and CNAPS ( Requeima et al. , 2019a ) , on both small scale ( miniImageNet ( Vinyals et al. , 2016 ) ) and large scale ( META-DATASET ( Triantafillou et al. , 2020 ) ) few-shot classification benchmarks . 4 . We show that adversarial support sets transfer effectively to fine-tuning based few-shot classifiers when the few-shot classifier and the fine-tuner utilize similar feature extractors . The rest of the paper proceeds as follows : Section 2 provides background about the meta-learning models under consideration , relevant adversarial attack methods , and the threat model under consideration . Section 3 discusses how evasion and poisoning attacks may be generalized to few-shot learners . Section 4 presents the experimental results and Section 5 concludes the paper . Additional results and experimental details are in the Appendix . 2 BACKGROUND . In this section we lay the necessary groundwork for adversarial support set attacks . We focus on image classification . We denote input images x ∈ Rch×W×H where W is the image width , H the image height , ch the number of image channels and image labels y ∈ { 1 , . . . , C } where C is the number of image classes . We use bold x and y to denote a set of images and labels , respectively . 2.1 META-LEARNING . We consider the few-shot image classification scenario using a meta-learning approach . Rather than a single , large dataset D , we assume access to a dataset D = { τt } Kt=1 comprising a large number of training tasks τt , drawn i.i.d . from a distribution p ( τ ) . The data for a task consists of a support set DS = { ( xn , yn ) } Nn=1 comprising N elements , with the inputs xn and labels yn observed , and a query set DQ = { ( x∗m , y∗m ) } Mm=1 with M elements for which we wish to make predictions . We may use the shorthand DS = { x , y } and DQ = { x∗ , y∗ } for brevity . Here the inputs x∗ are observed and the labels y∗ are only observed during meta-training ( i.e . training of the meta-learning algorithm ) . Note that the query set examples are drawn from the same set of labels as the examples in the support set . At meta-test time , the classifier f is required to make predictions for query set inputs of unseen tasks . Often , the meta-test tasks will include classes that have not been seen during meta-training , and DS will contain only a few observations . An example task is shown in Fig . 1 . Episodic Training The majority of modern meta-learning methods employ episodic training ( Vinyals et al. , 2016 ) . During meta-training , a task τ is drawn from p ( τ ) and randomly split into a support set DS and query set DQ . The meta-learner g takes as input the support set DS and produces task-specific classifier parameters ψ = g ( DS ) which are used to adapt the classifier f to the current task . The classifier can now make task-specific predictions f ( x∗ , ψ = g ( DS ) ) for any test input x∗ ∈ DQ . Refer to the diagram labeled Clean in Fig . 2 . A loss function L ( f ( x∗ , ψ ) , y∗ ) then computes the loss between the predictions for the label f ( x∗ , ψ ) and the true label y∗ . Assuming that L , f , and g are differentiable , the meta-learning algorithm can then be trained with stochastic gradient descent by back-propagating the loss and updating the parameters of f and g. Common Few-shot Learning Algorithms There has been an explosion of meta-learning based few-shot learning algorithms proposed in recent years . For an in-depth review see Hospedales et al . ( 2020 ) . Here , we briefly describe methods that are relevant to our experiments . Arguably the most widely used is the gradient-based approach , the canonical example for modern systems being MAML ( Finn et al. , 2017 ) . With MAML , the task-specific parameters ψ are the parameters of the classifier f after applying one or more gradient steps taken on the support set DS . Another widely used class of meta-learners are amortized-inference or black box based approaches e.g , VERSA ( Gordon et al. , 2019 ) and CNAPS ( Requeima et al. , 2019a ) . In these methods , the task-specific parameters ψ are generated by one or more hyper-networks , g ( Ha et al. , 2016 ) . An important special case of this approach is Prototypical Networks ( ProtoNets ) ( Snell et al. , 2017 ) which is based on metric learning and employs a nearest neighbor classifier , and the task-specific parameters ψ are the mean feature vectors for each class c ∈ C of the support set DS . Finally , a very competitive , non-episodic approach to few-shot learning is simple fine-tuning ( Yosinski et al. , 2014 ; Tian et al. , 2020 ) . In this approach , a pretrained feature extractor is used in combination with a final layer linear classifier , whose weights serve as the task-specific parameters ψ that are learned by taking a number of gradient steps on the support set data . 2.2 ADVERSARIAL ATTACKS . Szegedy et al . ( 2014 ) have shown that it is possible to craft adversarial examples , inputs that are indistinguishable from legitimate examples to both humans and algorithms ( Carlini & Wagner , 2017 ) , yet are classified incorrectly by deep neural networks with high confidence ( Nguyen et al. , 2014 ) . Adversarial examples are typically generated by taking a clean , unperturbed input image and adding a small amount of carefully crafted , almost imperceptible noise ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ) . The perturbation δ is calculated by performing the following optimization argmaxδ L ( θ , x + δ , y ) , where L is the loss function e.g . the cross entropy , θ denotes the model parameters , x is the input being perturbed , and y is its label . The perturbation size is typically constrained by some norm such as ` ∞ or ` 2 . As stated above , the optimization produces an untargeted attack , in the sense that we do not require the generated image to be misclassified as any particular class , only to be classified incorrectly . An alternative formulation of the optimization problem , argminδ L ( θ , x + δ , ỹ ) , would produce a targeted attack , i.e . an image that is misclassified as belonging specifically to the class with label ỹ 6= y . In this paper , labels for targeted attacks are generated by shifting the clean class label by 1 mod C. Many algorithms can be used to generate adversarial examples , including Projected Gradient Descent ( Madry et al. , 2017 ) , the Carlini and Wagner ` 2 attack ( Carlini & Wagner , 2017 ) , or ElasticNet ( Chen et al. , 2017a ) . Throughout this paper , without loss of generality , we utilize Projected Gradient Descent ( PGD ) with an ` ∞ norm , because is it simple to implement , inexpensive to compute , and effective . We show that by careful application of PGD , we can successfully poison meta-learners at meta-test time .

Summary: This work proposes adversarial attacks on few-shot learning systems. Evasion attacks are developed which can be viewed a simple applications of PGD/FGSM. The authors then apply evasion attacks to a support dataset, such that the n-shot classifier's loss is maximised on a query set. Additionally, poisoning attacks that cause the classifier to learn a shifted label set are introduced. Experiments on miniImageNet are performed with two meta-learning algorithms (MAML and ProtoNets) showing that these attacks lead to a large decrease in accuracy in comparison to baselines such as random noise attacks or universal adversarial perturbations (UAP) [1]. The authors then perform additional experiments such as measuring drops in accuracy for different numbers of n in n-shot learning and different query set sizes.