On sensitivity of meta-learning to support data

1 Introduction . Meta-learning , or learning to learn [ 29 ] , is the problem of training models that can adapt to new tasks quickly , using only a handful of examples . The problem is inspired by humans ’ ability to learn new skills or concepts at a rapid pace ( e.g . recognizing previously unknown objects after seeing only a single example [ 20 ] ) . Meta-learning has found applications in many domains , including safety-critical medical image analysis [ 23 ] , autonomous driving [ 35 ] , visual navigation [ 43 ] and legged robots control [ 40 ] . In this paper , we investigate the vulnerabilities of modern meta-learning algorithms in the context of few-shot image classification problem [ 25 ] , where a meta-learner needs to solve a classification task on classes unseen during training using a small number ( typically 1 or 5 ) of labeled samples from each of these classes to adapt . Specifically , we demonstrate that the performance of modern meta-learning algorithms on few-shot image recognition benchmarks varies drastically depending on the examples provided for adaptation , typically called support data , raising concerns about its safety in deployment . Despite the many empirical successes of artificial intelligence , its vulnerabilities are important to explore and mitigate in our pursuit of safe AI that is suitable for critical applications . Sensitivity to small ( possibly adversarial ) perturbations to the inputs [ 13 ] , backdoor attacks allowing malicious model manipulations [ 6 ] , algorithmic biases [ 2 ] and poor generalization outside of the training domain [ 18 ] are some of the prominent examples of AI safety failures . Some of these issues have also been studied in the context of meta-learning , e.g . adversarial robustness [ 45 , 11 , 44 ] and fairness [ 37 ] . Most of the aforementioned AI-safety dangers are associated with adversarial manipulations of the train or test data , or significant distribution shifts at test time . In meta-learning , prior works demonstrated that an adversary can create visually imperceptible changes of the test inputs [ 11 ] or the support data [ 44 , 30 ] causing meta-learning algorithms to fail on various few-shot image recognition benchmarks . In this work we demonstrate adversary-free and in-distribution failures specific to meta-learning . Sohn et al . [ 39 ] studied a similar problem in the context of semi-supervised learning . A distinct feature of meta-learning is the adaptation stage where the model is updated using a scarce amount of labeled examples from a new task . In practice , these examples need to be selected and presented to a human for labeling . Then a meta-learner adapts and can be used to classify the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . remaining images without a human . Currently , meta-learning algorithms choose support examples for labeling at random . Dhillon et al . [ 8 ] noted that the standard deviation of accuracies computed with random support examples may be large . We demonstrate that if the support data is not carefully curated , the performance of the meta-learner can be unreliable . In Figure 1 we present examples of unaltered support images ( i.e . they are not modified adversarially or in any other way ) representative of the corresponding task ( in-distribution ) that lead to vastly different performance of the meta-learner after adaptation . Support examples causing poor performance are not prevalent , however , they are also not data artifacts as there are multiple of them . The existence of such examples might be concerning when deploying meta-learning in practice , even when adversarial intervention is out of scope . Our main contributions are as follows : 1 . We present a simple algorithm for finding the best and the worst support examples and empirically demonstrate the sensitivity of popular meta-learning algorithms to support data . 2 . We demonstrate that a popular strategy for achieving robustness [ 22 ] adapted to our setting fails to solve the support data sensitivity problem . 3 . We explain the existence of the worst case examples from the margin perspective suggesting that robust meta-learning requires class margins more stringent than classical supervised learning . 2 Meta-learning approaches . Meta-learning approaches are typically categorized into model-based [ 34 , 26 , 33 ] , metric-based [ 17 , 41 , 38 ] and optimization-based methods [ 10 , 28 , 14 ] . We refer to Hospedales et al . [ 15 ] for a recent survey of the area . In this paper we mostly focus on the optimization-based method popularized by the Model Agnostic Meta Learning ( MAML ) [ 10 ] bi-level formulation ( metric-based prototypical networks [ 38 ] are also considered in the experiments ) . Let { Tn } ni=1 be a collection of n tasks Ti = { Ai , Di } each consisting of a support Ai and a query ( or validation ) Di datasets . MAML bi-level optimization problem is as follows [ 10 ] : min ✓ 1 n nX i=1 ` ( ✓i , ✓ ; Di ) , such that ✓i = argmin ✓i ` ( ✓i , ✓ ; Ai ) , i = 1 , . . . , n. ( 2.1 ) The first line of equation 2.1 is typically minimized using gradient-based optimization of ✓ and is called meta-update . The second line is the adaptation step and its implementation differs depending on whether the fine-tuned ✓i are treated as parameters of the end-to-end neural network or as the “ head ” parameters , i.e . the last linear classification layer . We refer to Goldblum et al . [ 12 ] for an empirical study of the differences between the two adaptation perspectives . MAML [ 10 ] and Reptile [ 28 ] are examples of algorithms that fine-tune all network parameters during the adaptation . Instead of solving the argmin of ✓i exactly they approximate it with a small number of gradient steps . For one gradient step approximation equation 2.1 can be written as min ✓ 1 n nX i=1 ` ( ✓ ↵ @ ` ( ✓ ; Ai ) @ ✓ ; Di ) ( 2.2 ) for some step size ↵ . MAML directly optimizes equation 2.2 differentiating through the inner gradient step , which requires expensive Hessian computations . Reptile introduces an approximation by-passing the Hessian computations and often performs better . Another family of meta-learning algorithms considers ✓ as the neural network feature extractor parameters shared across tasks and adapts only the linear classifier parametrized with ✓i , that takes the features extracted with ✓ as inputs . The advantage of this perspective is that the adaptation minimization problem is convex and , for many linear classifiers , can be solved fast and exactly . Let a ( ✓ , A ) denote a procedure that takes data in A , passes it through a feature extractor parametrized with ✓ , and returns ✓i , i.e . optimal parameters of a linear classifier using the obtained features to predict the corresponding labels . Then equation 2.1 can be written as min ✓ 1 n nX i=1 ` ( ✓ , a ( ✓ , Ai ) ; Di ) . ( 2.3 ) This approach requires a ( ✓ , A ) to be differentiable . R2D2 [ 4 ] casts classification as a multi-target ridge-regression problem and utilizes the corresponding closed-form solution as a ( ✓ , A ) . Lee et al . [ 21 ] propose MetaOptNet , where a ( ✓ , A ) is a differentiable quadratic programming solver [ 1 ] , and implement it with linear support vector machines ( SVM ) and ridge-regression . Prototypical networks [ 38 ] is a metric-based approach that can also be viewed from the perspective of equation 2.3 . Here a ( ✓ , A ) outputs class centroids in the neural-feature space and the predictions are based on the closest class centroids . Last-layer adaptation approaches , especially MetaOptNet , outperform full-network approaches on most benchmarks [ 21 , 12 ] . Both adaptation perspectives have a weakness underlying our study : their test performance is based on an optimization problem with large number of parameters and as little as 5 data points ( 1-shot 5-way setting ) . To understand the problem , consider the last-layer adaptation approach : even when the feature extractor produces linearly separable representations , the dimension is large ( for example , Lee et al . [ 21 ] utilize a ResNet-12 architecture with 2560-dimensional last layer features that we adopt in our experiments ) making the corresponding linear classifier extremely sensitive to the support data . In Section 3 we demonstrate the problem empirically and provide theoretical insights in Section 4 . 2.1 Meta-learning benchmarks . Before presenting our findings , we discuss the meta-learning benchmarks we consider . Meta-learning algorithms are often compared in a few-shot image recognition setting . Each task typically has five unique classes , i.e . 5-way , and one or five examples per class for adaptation , i.e . 1-shot or 5-shot . Classes that appear at test time are not seen during training . Few-shot learning datasets are typically the derivatives of the existing supervised learning benchmarks , e.g . CIFAR-100 [ 19 ] and ImageNet [ 7 ] . Few-shot problem is setup by disjoint partitioning of the available classes into train , validation and test . Tasks are obtained by sampling 5 distinct classes from the corresponding pool and assigning a subset of examples for adaptation and a different subset for meta-update during training or accuracy evaluation for reporting the performance . CIFAR-FS [ 4 ] is a dataset of 60000 32⇥32 RGB images from CIFAR-100 partitioned into 64 , 16 and 20 classes for training , validation and testing , respectively . FC-100 [ 31 ] is also a derivative of CIFAR-100 with a different partition aimed to reduce semantic overlap between 60 classes assigned for training , 20 for validation , and 20 for testing . MiniImageNet [ 41 ] is a subsampled , downsized version of ImageNet . It consists of 60000 84⇥84 RGB images from 100 classes split into 64 for training , 16 for validation , and 20 for testing . 3 Finding adaptation vulnerabilities . In this section we study the performance range of meta-learners trained with a variety of algorithms : MAML [ 10 ] , Meta-Curvature ( MC ) [ 32 ] , Prototypical networks [ 38 ] , R2D2 [ 4 ] , and MetaOptNet with Ridge and SVM heads [ 21 ] . At test time a meta-learner receives support data and its performance is recorded after it adapts . Typical evaluation protocols sample support data randomly and report average performance across tasks . While this is a useful measure for comparing algorithms , safetycritical applications demand understanding of potential performance ranges . We demonstrate that there are support examples the lead to vastly different test performances . To identify the worst and the best case support examples for a given task we perform an iterative greedy search . Let X = { xkm } k2 [ K ] m2 [ M ] be a set of potential support examples for a given task . K is the number of classes , M is the number of examples per class ( we assume same M for each class for simplicity ) , and [ M ] = { 0 , . . . , M 1 } . Let D be the evaluation set for this task . In a J-shot setting , let Z = { zkj } k2 [ K ] j2 [ J ] be a set of indices of the support examples ( z k j 6= z k j0 for any k , j 6= j 0 and zkj 2 [ M ] for any k , j ) . Denote R ( Z , X , D ) a function computing accuracy of a meta learner on the query set D after adapting on A = { xk zkj } k2 [ K ] j2 [ J ] . Finding the worst/best case accuracy amounts to finding indices Z minimizing/maximizing R ( Z , X , D ) . We solve this greedily by updating a single index zkj at a time , holding the rest of Z fixed , iterating over [ K ] and [ J ] multiple times . We summarize the procedure for finding the worst case accuracy in Algorithm 1 ( the best case accuracy is analogous ) .1 Due to the greedy nature , this algorithm finds a local optima , but it is sufficiently effective as we see in the following section . Algorithm 1 Finding the worst case support examples Input : trained meta-learner , potential support examples X to search over , query data D. repeat Initialize indices Z = { zkj } k2 [ K ] j2 [ J ] randomly . for j 2 [ J ] , k 2 [ K ] do zkj argminzkj R ( Z , X , D ) , z k j 2 [ M ] \ { z k j0 } j0 6=j end for until R ( Z , X , D ) stops decreasing Output : support examples indices Z = { zkj } k2 [ K ] j2 [ J ] Visualizing the worst case support search In Figure 2 we visualize a single iteration of Algorithm 1 on a 1-shot task from the CIFAR-FS dataset . This is a 5-way task with “ snail ” , “ red pepper ” , “ bed ” , “ plain ” , and “ telephone ” as classes . In round 1 , i.e . k = 0 , of iteration 1 we start with a random example per class and evaluate post-adaptation accuracy on the query data D for each potential support “ snail ” , i.e . { x0m } m2 [ M ] . Here the first line corresponds to “ snail ” indexed with m = 0 , i.e . x00 , and post-adaptation accuracy of 73.5 % . We select a “ snail ” support image corresponding to the worst accuracy of 53.8 % and proceed to round 2 , i.e . k = 1 , of iteration 1 . In round 2 we repeat the same procedure for the corresponding class “ red pepper ” ( again the first line corresponds to “ red pepper ” indexed with m = 0 , i.e . x10 ) , holding the support “ snail ” image selected previously and 1This is a simple approach , however we found it faster and more efficient then a more sophisticated continuous relaxation using weights of the potential support examples with sparsity constraints . support images for other classes fixed . The algorithm finds the worst case support “ red pepper ” image corresponding to the post-adaptation accuracy of 37 % . Then the algorithm proceeds analogously to rounds 3 , 4 , and 5 of iteration 1 , resulting in a combination of support examples corresponding to 2.9 % post-adaptation accuracy . This is already sufficiently low , however on some tasks and higher-shot settings it may be beneficial to run additional iterations . On iteration 2 , the algorithm will again go through all the classes starting from the support examples found on iteration 1 instead of random ones . In our experiments we always run Algorithm 1 for 3 iterations . In Appendix B we empirically study the convergence of the algorithm justifying this choice .

Summary: This paper is studying the performance of several meta-learning approaches depending on the support images used for meta-test tasks.  First, the authors propose a simple greedy algorithm that select for each class in a given task, the image giving the worst/ the best performance. With this algorithm, they show that popular meta-learning approaches have a very high variance in their performance and that the worst case scenario is often worse than a random guess. The authors argue that the worst tasks found by their algorithm are not artifacts in the distribution, and can realistically appear.  Then, they apply their algorithm to train the meta-learning methods in an adversarial way, by learning on the worst cases found by their algorithm. Even though the adversarial training improves the worst-case performance on the meta-training tasks, there are no improvements on the worst-case accuracy on meta-testing tasks. Finally, the authors argue that the worst-case performance are better when the task embeddings are sufficiently separated.