On sensitivity of meta-learning to support data

1 Introduction . Meta-learning , or learning to learn [ 29 ] , is the problem of training models that can adapt to new tasks quickly , using only a handful of examples . The problem is inspired by humans ’ ability to learn new skills or concepts at a rapid pace ( e.g . recognizing previously unknown objects after seeing only a single example [ 20 ] ) . Meta-learning has found applications in many domains , including safety-critical medical image analysis [ 23 ] , autonomous driving [ 35 ] , visual navigation [ 43 ] and legged robots control [ 40 ] . In this paper , we investigate the vulnerabilities of modern meta-learning algorithms in the context of few-shot image classification problem [ 25 ] , where a meta-learner needs to solve a classification task on classes unseen during training using a small number ( typically 1 or 5 ) of labeled samples from each of these classes to adapt . Specifically , we demonstrate that the performance of modern meta-learning algorithms on few-shot image recognition benchmarks varies drastically depending on the examples provided for adaptation , typically called support data , raising concerns about its safety in deployment . Despite the many empirical successes of artificial intelligence , its vulnerabilities are important to explore and mitigate in our pursuit of safe AI that is suitable for critical applications . Sensitivity to small ( possibly adversarial ) perturbations to the inputs [ 13 ] , backdoor attacks allowing malicious model manipulations [ 6 ] , algorithmic biases [ 2 ] and poor generalization outside of the training domain [ 18 ] are some of the prominent examples of AI safety failures . Some of these issues have also been studied in the context of meta-learning , e.g . adversarial robustness [ 45 , 11 , 44 ] and fairness [ 37 ] . Most of the aforementioned AI-safety dangers are associated with adversarial manipulations of the train or test data , or significant distribution shifts at test time . In meta-learning , prior works demonstrated that an adversary can create visually imperceptible changes of the test inputs [ 11 ] or the support data [ 44 , 30 ] causing meta-learning algorithms to fail on various few-shot image recognition benchmarks . In this work we demonstrate adversary-free and in-distribution failures specific to meta-learning . Sohn et al . [ 39 ] studied a similar problem in the context of semi-supervised learning . A distinct feature of meta-learning is the adaptation stage where the model is updated using a scarce amount of labeled examples from a new task . In practice , these examples need to be selected and presented to a human for labeling . Then a meta-learner adapts and can be used to classify the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . remaining images without a human . Currently , meta-learning algorithms choose support examples for labeling at random . Dhillon et al . [ 8 ] noted that the standard deviation of accuracies computed with random support examples may be large . We demonstrate that if the support data is not carefully curated , the performance of the meta-learner can be unreliable . In Figure 1 we present examples of unaltered support images ( i.e . they are not modified adversarially or in any other way ) representative of the corresponding task ( in-distribution ) that lead to vastly different performance of the meta-learner after adaptation . Support examples causing poor performance are not prevalent , however , they are also not data artifacts as there are multiple of them . The existence of such examples might be concerning when deploying meta-learning in practice , even when adversarial intervention is out of scope . Our main contributions are as follows : 1 . We present a simple algorithm for finding the best and the worst support examples and empirically demonstrate the sensitivity of popular meta-learning algorithms to support data . 2 . We demonstrate that a popular strategy for achieving robustness [ 22 ] adapted to our setting fails to solve the support data sensitivity problem . 3 . We explain the existence of the worst case examples from the margin perspective suggesting that robust meta-learning requires class margins more stringent than classical supervised learning . 2 Meta-learning approaches . Meta-learning approaches are typically categorized into model-based [ 34 , 26 , 33 ] , metric-based [ 17 , 41 , 38 ] and optimization-based methods [ 10 , 28 , 14 ] . We refer to Hospedales et al . [ 15 ] for a recent survey of the area . In this paper we mostly focus on the optimization-based method popularized by the Model Agnostic Meta Learning ( MAML ) [ 10 ] bi-level formulation ( metric-based prototypical networks [ 38 ] are also considered in the experiments ) . Let { Tn } ni=1 be a collection of n tasks Ti = { Ai , Di } each consisting of a support Ai and a query ( or validation ) Di datasets . MAML bi-level optimization problem is as follows [ 10 ] : min ✓ 1 n nX i=1 ` ( ✓i , ✓ ; Di ) , such that ✓i = argmin ✓i ` ( ✓i , ✓ ; Ai ) , i = 1 , . . . , n. ( 2.1 ) The first line of equation 2.1 is typically minimized using gradient-based optimization of ✓ and is called meta-update . The second line is the adaptation step and its implementation differs depending on whether the fine-tuned ✓i are treated as parameters of the end-to-end neural network or as the “ head ” parameters , i.e . the last linear classification layer . We refer to Goldblum et al . [ 12 ] for an empirical study of the differences between the two adaptation perspectives . MAML [ 10 ] and Reptile [ 28 ] are examples of algorithms that fine-tune all network parameters during the adaptation . Instead of solving the argmin of ✓i exactly they approximate it with a small number of gradient steps . For one gradient step approximation equation 2.1 can be written as min ✓ 1 n nX i=1 ` ( ✓ ↵ @ ` ( ✓ ; Ai ) @ ✓ ; Di ) ( 2.2 ) for some step size ↵ . MAML directly optimizes equation 2.2 differentiating through the inner gradient step , which requires expensive Hessian computations . Reptile introduces an approximation by-passing the Hessian computations and often performs better . Another family of meta-learning algorithms considers ✓ as the neural network feature extractor parameters shared across tasks and adapts only the linear classifier parametrized with ✓i , that takes the features extracted with ✓ as inputs . The advantage of this perspective is that the adaptation minimization problem is convex and , for many linear classifiers , can be solved fast and exactly . Let a ( ✓ , A ) denote a procedure that takes data in A , passes it through a feature extractor parametrized with ✓ , and returns ✓i , i.e . optimal parameters of a linear classifier using the obtained features to predict the corresponding labels . Then equation 2.1 can be written as min ✓ 1 n nX i=1 ` ( ✓ , a ( ✓ , Ai ) ; Di ) . ( 2.3 ) This approach requires a ( ✓ , A ) to be differentiable . R2D2 [ 4 ] casts classification as a multi-target ridge-regression problem and utilizes the corresponding closed-form solution as a ( ✓ , A ) . Lee et al . [ 21 ] propose MetaOptNet , where a ( ✓ , A ) is a differentiable quadratic programming solver [ 1 ] , and implement it with linear support vector machines ( SVM ) and ridge-regression . Prototypical networks [ 38 ] is a metric-based approach that can also be viewed from the perspective of equation 2.3 . Here a ( ✓ , A ) outputs class centroids in the neural-feature space and the predictions are based on the closest class centroids . Last-layer adaptation approaches , especially MetaOptNet , outperform full-network approaches on most benchmarks [ 21 , 12 ] . Both adaptation perspectives have a weakness underlying our study : their test performance is based on an optimization problem with large number of parameters and as little as 5 data points ( 1-shot 5-way setting ) . To understand the problem , consider the last-layer adaptation approach : even when the feature extractor produces linearly separable representations , the dimension is large ( for example , Lee et al . [ 21 ] utilize a ResNet-12 architecture with 2560-dimensional last layer features that we adopt in our experiments ) making the corresponding linear classifier extremely sensitive to the support data . In Section 3 we demonstrate the problem empirically and provide theoretical insights in Section 4 . 2.1 Meta-learning benchmarks . Before presenting our findings , we discuss the meta-learning benchmarks we consider . Meta-learning algorithms are often compared in a few-shot image recognition setting . Each task typically has five unique classes , i.e . 5-way , and one or five examples per class for adaptation , i.e . 1-shot or 5-shot . Classes that appear at test time are not seen during training . Few-shot learning datasets are typically the derivatives of the existing supervised learning benchmarks , e.g . CIFAR-100 [ 19 ] and ImageNet [ 7 ] . Few-shot problem is setup by disjoint partitioning of the available classes into train , validation and test . Tasks are obtained by sampling 5 distinct classes from the corresponding pool and assigning a subset of examples for adaptation and a different subset for meta-update during training or accuracy evaluation for reporting the performance . CIFAR-FS [ 4 ] is a dataset of 60000 32⇥32 RGB images from CIFAR-100 partitioned into 64 , 16 and 20 classes for training , validation and testing , respectively . FC-100 [ 31 ] is also a derivative of CIFAR-100 with a different partition aimed to reduce semantic overlap between 60 classes assigned for training , 20 for validation , and 20 for testing . MiniImageNet [ 41 ] is a subsampled , downsized version of ImageNet . It consists of 60000 84⇥84 RGB images from 100 classes split into 64 for training , 16 for validation , and 20 for testing . 3 Finding adaptation vulnerabilities . In this section we study the performance range of meta-learners trained with a variety of algorithms : MAML [ 10 ] , Meta-Curvature ( MC ) [ 32 ] , Prototypical networks [ 38 ] , R2D2 [ 4 ] , and MetaOptNet with Ridge and SVM heads [ 21 ] . At test time a meta-learner receives support data and its performance is recorded after it adapts . Typical evaluation protocols sample support data randomly and report average performance across tasks . While this is a useful measure for comparing algorithms , safetycritical applications demand understanding of potential performance ranges . We demonstrate that there are support examples the lead to vastly different test performances . To identify the worst and the best case support examples for a given task we perform an iterative greedy search . Let X = { xkm } k2 [ K ] m2 [ M ] be a set of potential support examples for a given task . K is the number of classes , M is the number of examples per class ( we assume same M for each class for simplicity ) , and [ M ] = { 0 , . . . , M 1 } . Let D be the evaluation set for this task . In a J-shot setting , let Z = { zkj } k2 [ K ] j2 [ J ] be a set of indices of the support examples ( z k j 6= z k j0 for any k , j 6= j 0 and zkj 2 [ M ] for any k , j ) . Denote R ( Z , X , D ) a function computing accuracy of a meta learner on the query set D after adapting on A = { xk zkj } k2 [ K ] j2 [ J ] . Finding the worst/best case accuracy amounts to finding indices Z minimizing/maximizing R ( Z , X , D ) . We solve this greedily by updating a single index zkj at a time , holding the rest of Z fixed , iterating over [ K ] and [ J ] multiple times . We summarize the procedure for finding the worst case accuracy in Algorithm 1 ( the best case accuracy is analogous ) .1 Due to the greedy nature , this algorithm finds a local optima , but it is sufficiently effective as we see in the following section . Algorithm 1 Finding the worst case support examples Input : trained meta-learner , potential support examples X to search over , query data D. repeat Initialize indices Z = { zkj } k2 [ K ] j2 [ J ] randomly . for j 2 [ J ] , k 2 [ K ] do zkj argminzkj R ( Z , X , D ) , z k j 2 [ M ] \ { z k j0 } j0 6=j end for until R ( Z , X , D ) stops decreasing Output : support examples indices Z = { zkj } k2 [ K ] j2 [ J ] Visualizing the worst case support search In Figure 2 we visualize a single iteration of Algorithm 1 on a 1-shot task from the CIFAR-FS dataset . This is a 5-way task with “ snail ” , “ red pepper ” , “ bed ” , “ plain ” , and “ telephone ” as classes . In round 1 , i.e . k = 0 , of iteration 1 we start with a random example per class and evaluate post-adaptation accuracy on the query data D for each potential support “ snail ” , i.e . { x0m } m2 [ M ] . Here the first line corresponds to “ snail ” indexed with m = 0 , i.e . x00 , and post-adaptation accuracy of 73.5 % . We select a “ snail ” support image corresponding to the worst accuracy of 53.8 % and proceed to round 2 , i.e . k = 1 , of iteration 1 . In round 2 we repeat the same procedure for the corresponding class “ red pepper ” ( again the first line corresponds to “ red pepper ” indexed with m = 0 , i.e . x10 ) , holding the support “ snail ” image selected previously and 1This is a simple approach , however we found it faster and more efficient then a more sophisticated continuous relaxation using weights of the potential support examples with sparsity constraints . support images for other classes fixed . The algorithm finds the worst case support “ red pepper ” image corresponding to the post-adaptation accuracy of 37 % . Then the algorithm proceeds analogously to rounds 3 , 4 , and 5 of iteration 1 , resulting in a combination of support examples corresponding to 2.9 % post-adaptation accuracy . This is already sufficiently low , however on some tasks and higher-shot settings it may be beneficial to run additional iterations . On iteration 2 , the algorithm will again go through all the classes starting from the support examples found on iteration 1 instead of random ones . In our experiments we always run Algorithm 1 for 3 iterations . In Appendix B we empirically study the convergence of the algorithm justifying this choice .

Summary: This paper empirically finds that popular meta-learning algorithms are very sensitive to the selections of the support images. The author shows that six meta-learning algorithms can achieve testing accuracy within a wide accuracy (worst case accuracy is extremely poor) given different support samples. They also find that this problem cannot be trivially solved by adversarial training.