How Important is the Train-Validation Split in Meta-Learning?

1 INTRODUCTION . Meta-learning , also known as “ learning to learn ” , has recently emerged as a powerful paradigm for learning to adapt to unseen tasks ( Schmidhuber , 1987 ) . The high-level methodology in metalearning is akin to how human beings learn new skills , which is typically done by relating to certain prior experience that makes the learning process easier . More concretely , meta-learning does not train one model for each individual task , but rather learns a “ prior ” model from multiple existing tasks so that it is able to quickly adapt to unseen new tasks . Meta-learning has been successfully applied to many real problems , including few-shot image classification ( Finn et al. , 2017 ; Snell et al. , 2017 ) , hyper-parameter optimization ( Franceschi et al. , 2018 ) , low-resource machine translation ( Gu et al. , 2018 ) and short event sequence modeling ( Xie et al. , 2019 ) . A common practice in meta-learning algorithms is to perform a sample splitting , where the data within each task is divided into a training split which the prior uses to adapt to a task-specific predictor , and a validation split on which we evaluate the performance of the task-specific predictor ( Nichol et al. , 2018 ; Rajeswaran et al. , 2019 ; Fallah et al. , 2020 ; Wang et al. , 2020a ) . For example , in a 5-way k-shot image classification task , standard meta-learning algorithms such as MAML ( Finn et al. , 2017 ) use 5k examples within each task as training data , and use additional examples ( e.g . k images , one for each class ) as validation data . This sample splitting is believed to be crucial as it matches the evaluation criterion at meta-test time , where we perform adaptation on training data from a new task but evaluate its performance on unseen data from the same task . Despite the aformentioned importance , performing the train-validation split has a potential drawback from the data efficiency perspective — Because of the split , neither the training nor the evaluation stage is able to use all the available per-task data . In the few-shot image classification example , each task has a total of 6k examples available , but the train-validation split forces us to use these data separately in the two stages . Meanwhile , performing the train-validation split is also not the only option in practice : there exist algorithms such as Reptile ( Nichol & Schulman , 2018 ) and Meta-MinibatchProx ( Zhou et al. , 2019 ) that can instead use all the per-task data for training the task-specific predictor and also perform well empirically on benchmark tasks . These algorithms modify the loss function in the outer loop so that the training loss no longer matches the meta-test loss , but may have the advantage in terms of data efficiency for the overall problem of learning the best prior . So far it is theoretically unclear how these two approaches ( with/without train-validation split ) compare with each other , which motivates us to ask the following Question : Is the train-validation split necessary and optimal in meta-learning ? In this paper , we perform a detailed theoretical study on the importance of the train-validation split . We consider the linear centroid meta-learning problem ( Denevi et al. , 2018b ) , where for each task we learn a linear predictor that is close to a common centroid in the inner loop , and find the best centroid in the outer loop ( see Section 2 for the detailed problem setup ) . This problem captures the essence of meta-learning with non-linear models ( such as neural networks ) in practice , yet is sufficiently simple that allows a precise theoretical characterization . We use a biased ridge solver as the inner loop with a ( tunable ) regularization parameter , and compare two outer-loop algorithms of either performing the train-validation split ( the train-val method ) or using all the per-task data for both training and evaluation ( the train-train method ) . Specifically , we compare the two methods when the number of tasks T is large , and examine if and how fast they converge to the ( properly defined ) best centroid at meta-test time . We summarize our contributions as follows : • On the linear centroid meta-learning problem , we show that the train-validation split is necessary in the general agnostic setting : As T → ∞ , the train-val method converges to the optimal centroid for test-time adaptation , whereas the train-train method does not without further assumptions on the tasks ( Section 3 ) . The convergence of the train-val method is expected since its ( population ) training loss is equivalent to the meta-test time loss , whereas the non-convergence of the train-train method is because these two losses are not equivalent in general . • Our main theoretical contribution is to show that the train-validation split is not necessary and even non-optimal , in the perhaps more interesting regime when there are structural assumptions on the tasks : When the data are generated from noiseless linear models , both the train-val and traintrain methods converge to the common best centroid , and the train-train method achieves a strictly better ( asymptotic ) estimation error and test loss than the train-val method ( Section 4 ) . This is in stark contrast with the agnostic case , and suggests that data efficiency may indeed be more important when the tasks have a nice structure . Our results build on tools from random matrix theory in the proportional regime , which may be of broader technical interest . • We perform meta-learning experiments on simulations and benchmark few-shot image classification tasks , showing that the train-train method consistently outperforms the train-val method ( Section 5 & Appendix D ) . This validates our theories and presents empirical evidence that samplesplitting may not be crucial ; methods that utilize the per-task data more efficiently may be preferred . 1.1 RELATED WORK . Meta-learning and representation learning theory Baxter ( 2000 ) provided the first theoretical analysis of meta-learning via covering numbers , and Maurer et al . ( 2016 ) improved the analysis via Gaussian complexity techniques . Another recent line of theoretical work analyzed gradient-based meta-learning methods ( Denevi et al. , 2018a ; Finn et al. , 2019 ; Khodak et al. , 2019 ; Ji et al. , 2020 ) and showed guarantees for convex losses by using tools from online convex optimization . Saunshi et al . ( 2020 ) proved the success of Reptile in a one-dimensional subspace setting . Wang et al . ( 2020b ) compared the performance of train-train and train-val methods for learning the learning rate . Denevi et al . ( 2018b ) proposed the linear centroid model studied in this paper , and provided generalization error bounds for train-val method ; the bounds proved also hold for train-train method , so are not sharp enough to compare the two algorithms . Wang et al . ( 2020a ) studied the convergence of gradient-based meta-learning by relating to the kernelized approximation . On the representation learning end , Du et al . ( 2020 ) ; Tripuraneni et al . ( 2020a ; b ) showed that ERM can successfully pool data across tasks to learn the representation . Yet the focus is on the accurate estimation of the common representation , not on the fast adaptation of the learned prior . Lastly , we remark that there are analyses for other representation learning schemes ( McNamara & Balcan , 2017 ; Galanti et al. , 2016 ; Alquier et al. , 2016 ) . Empirical understandings of meta-learning Raghu et al . ( 2020 ) investigated the representation learning perspective of meta-learning and showed that MAML with a full finetuning inner loop mostly learns the top-layer linear classifier and does not change the representation layers much . This result partly justifies the validity our linear centroid meta-learning problem in which the features ( representations ) are fixed and only a linear classifier is learned . Goldblum et al . ( 2020 ) investigated the difference of the neural representations learned by classical training ( supervised learning ) and meta-learning , and showed that the meta-learned representation is both better for downstream adaptation and makes classes more separated than the classically trained one . Although the classical training method in ( Goldblum et al. , 2020 ) does not perform a train-validation split , it is not exactly the same as the train-train method considered in this work as it effectively performs a supervised learning on all tasks combined and does not do a per-task adaptation . Multi-task learning Multi-task learning also exploits structures and similarities across multiple tasks . The earliest idea dates back to Caruana ( 1997 ) ; Thrun & Pratt ( 1998 ) ; Baxter ( 2000 ) , initially in connections to neural network models . They further motivated other approaches using kernel methods ( Evgeniou et al. , 2005 ; Argyriou et al. , 2007 ) and multivariate linear regression models with structured sparsity ( Liu et al. , 2009 ; 2015 ) . More recent advances on deep multi-task learning focus on learning shared intermediate representations across tasks Ruder ( 2017 ) . These multi-task learning approaches usually minimize the joint empirical risk over all tasks , and the models for different tasks are enforced to share a large amount of parameters . In contrast , meta-learning only requires the models to share the same “ prior ” , which is more flexible than multi-task learning . 2 PRELIMINARIES . In this paper , we consider the standard meta-learning setting , in which we observe data from T ≥ 1 supervised learning tasks , and the goal is to find a prior ( or “ initialization ” ) using the combined data , such that the ( T + 1 ) -th new task may be solved sample-efficiently using the prior . Linear centroid meta-learning We instantiate our study on the linear centroid meta-learning problem ( also known as learning to learn around a common mean , Denevi et al . ( 2018b ) ) , where we wish to learn a task-specific linear predictor wt ∈ Rd in the inner loop for each task t , and learn a “ centroid ” w0 in the outer loop that enables fast adaptation to wt within each task : Find the best centroid w0 ∈ Rd for adapting to a linear predictor wt on each task t. Formally , we assume that we observe training data from T ≥ 1 tasks , where for each task index t we sample a task pt ( a distribution over Rd × R ) from some distribution of tasks Π , and observe n examples ( Xt , yt ) ∈ Rn×d × Rn that are drawn i.i.d . from pt : pt ∼ Π , ( Xt , yt ) = { ( xt , i , yt , i ) } ni=1 where ( xt , i , yt , i ) iid∼ pt . ( 1 ) We do not make further assumptions on ( n , d ) ; in particular , we allow the underdetermined setting n ≤ d , in which there exists ( one or many ) interpolators w̃t that perfectly fit the data : Xtw̃t = yt . Inner loop : Ridge solver with biased regularization towards the centroid Our goal in the inner loop is to find a linear predictor wt that fits the data in task twhile being close to the given “ centroid ” w0 ∈ Rd . We instantiate this through ridge regression ( i.e . linear regression with L2 regularization ) where the regularization biases wt towards the centroid . Formally , for any w0 ∈ Rd and any dataset ( X , y ) , we consider the algorithm Aλ ( w0 ; X , y ) : = arg min w 1 n ‖Xw − y‖2 + λ ‖w −w0‖2 = w0 + ( X > X + nλId ) −1 X > ( y −Xw0 ) , where λ > 0 is the regularization strength ( typically a tunable hyper-parameter ) . As we regularize by ‖w −w0‖2 , this inner solver encourages the solution to be close to w0 , as we desire . Such a regularizer is widely used in practical meta-learning algorithms such as MetaOptNet ( Lee et al. , 2019 ) and Meta-MinibatchProx ( Zhou et al. , 2019 ) . In addition , as λ → 0 , this solver recovers gradient descent fine-tuning : we have A0 ( w0 ; X , y ) : = lim λ→0 Aλ ( w0 ; X , y ) = w0 + X† ( y −Xw0 ) = arg minXw=y ‖w −w0‖2 ( where X† ∈ Rd×n denotes the pseudo-inverse of X ) . This is the minimum-distance interpolator of ( X , y ) and also the solution found by gradient descent 1 on ‖Xw − y‖2 initialized at w0 . Therefore our ridge solver with λ > 0 can be seen as a generalized version of the gradient descent solver used in MAML ( Finn et al. , 2017 ) . Outer loop : Finding the best centroid In the outer loop , our goal is to find the best centroid w0 . The standard approach in meta-learning is to perform a train-validation split , that is , ( 1 ) execute the inner solver on a first split of the task-specific data , and ( 2 ) evaluate the loss on a second split , yielding a function of w0 that we can optimize . This two-stage procedure can be written as Compute wt ( w0 ) = Aλ ( w0 ; Xtraint , ytraint ) , and evaluate ∥∥yvalt −Xvalt wt ( w0 ) ∥∥2 . where ( Xtraint , y train t ) = { ( xt , i , yt , i ) } n1i=1 and ( Xvalt , yvalt ) = { ( xt , i , yt , i ) } n i=n1+1 are two disjoint splits of the per-task data ( Xt , yt ) of size ( n1 , n2 ) , with n1 + n2 = n. Written concisely , this is to consider the “ split loss ” ` tr-valt ( w0 ) : = 1 2n2 ∥∥yvalt −Xvalt Aλ ( w0 ; Xtraint , ytraint ) ∥∥2 . ( 2 ) In this paper , we will also consider an alternative version , where we do not perform the trainvalidation split , but instead use all the per-task data for both training and evaluation . Mathematically , this is to look at the “ non-split loss ” ` tr-trt ( w0 ) : = 1 2n ‖yt −XtAλ ( w0 ; Xt , yt ) ‖2 . ( 3 ) Our overall algorithm is to solve the empirical risk minimization ( ERM ) problem on the T observed tasks , using either one of the two losses above : L̂tr-valT ( w0 ) : = 1 T T∑ t=1 ` tr-valt ( w0 ) and L̂ tr-tr T ( w0 ) : = 1 T T∑ t=1 ` tr-trt ( w0 ) , ŵ { tr-val , tr-tr } 0 , T : = arg min w0 L̂ { tr-val , tr-tr } T ( w0 ) . ( 4 ) Let L { tr-val , tr-tr } ( w0 ) : = Ept∼Π , ( Xt , yt ) ∼pt [ ` { tr-val , tr-tr } t ( w0 ) ] be the population risks . ( Meta- ) Test time The meta-test time performance of any meta-learning algorithm is a joint function of the ( learned ) centroid w0 and the inner algorithm Alg . Upon receiving a new task pT+1 ∼ Π and training data ( XT+1 , yT+1 ) ∈ Rn×d × Rn , we run the inner loop Alg with prior w0 on the training data , and evaluate it on an ( unseen ) test example ( x′ , y′ ) ∼ pT+1 : Ltest ( w0 ; Alg ) : = EpT+1∼ΠE ( XT+1 , yT+1 ) , ( x′ , y′ ) iid∼pT+1 [ 1 2 ( x′ > Alg ( w0 ; XT+1 , yT+1 ) − y′ ) 2 ] . Additionally , for both train-val and train-train methods , we need to ensure that the inner loop used for meta-test is exactly the same as that used in meta-training . Therefore , the meta-test performance for the train-val and train-train methods above should be evaluated as Ltestλ , n1 ( ŵ tr-val 0 , T ) : = L test ( ŵtr-val0 , T ; Aλ , n1 ) , Ltestλ , n ( ŵtr-tr0 , T ) : = Ltest ( ŵtr-tr0 , T ; Aλ , n ) , where Aλ , m denotes the ridge solver with regularization strength λ > 0 on m ≤ n data points . Finally , we let w0 , ? ( λ ; n ) = arg min w0 Ltestλ , n ( w0 ) ( 5 ) denote the best centroid if the inner loop uses Aλ , n . The performance of the train-val algorithm ŵtr-val0 , T should be compared against w0 , ? ( λ , n1 ) , whereas the train-train algorithm ŵ tr-tr 0 , T should be compared against w0 , ? ( λ , n ) . 1with a small step-size , or gradient flow .

Summary: The authors verify the importance of train-validation split in meta-learning theoretically, which is commonly used in the meta-learning paradigms. By analyzing the linear centroid meta-learning problem, the authors show that the splitting method converges to the optimal prior as expected, whereas the non-splitting method does not in general, without structural assumptions on the data. The authors validate the theories empirically through both simulations and real meta-learning tasks.