How Important is the Train-Validation Split in Meta-Learning?

1 INTRODUCTION . Meta-learning , also known as “ learning to learn ” , has recently emerged as a powerful paradigm for learning to adapt to unseen tasks ( Schmidhuber , 1987 ) . The high-level methodology in metalearning is akin to how human beings learn new skills , which is typically done by relating to certain prior experience that makes the learning process easier . More concretely , meta-learning does not train one model for each individual task , but rather learns a “ prior ” model from multiple existing tasks so that it is able to quickly adapt to unseen new tasks . Meta-learning has been successfully applied to many real problems , including few-shot image classification ( Finn et al. , 2017 ; Snell et al. , 2017 ) , hyper-parameter optimization ( Franceschi et al. , 2018 ) , low-resource machine translation ( Gu et al. , 2018 ) and short event sequence modeling ( Xie et al. , 2019 ) . A common practice in meta-learning algorithms is to perform a sample splitting , where the data within each task is divided into a training split which the prior uses to adapt to a task-specific predictor , and a validation split on which we evaluate the performance of the task-specific predictor ( Nichol et al. , 2018 ; Rajeswaran et al. , 2019 ; Fallah et al. , 2020 ; Wang et al. , 2020a ) . For example , in a 5-way k-shot image classification task , standard meta-learning algorithms such as MAML ( Finn et al. , 2017 ) use 5k examples within each task as training data , and use additional examples ( e.g . k images , one for each class ) as validation data . This sample splitting is believed to be crucial as it matches the evaluation criterion at meta-test time , where we perform adaptation on training data from a new task but evaluate its performance on unseen data from the same task . Despite the aformentioned importance , performing the train-validation split has a potential drawback from the data efficiency perspective — Because of the split , neither the training nor the evaluation stage is able to use all the available per-task data . In the few-shot image classification example , each task has a total of 6k examples available , but the train-validation split forces us to use these data separately in the two stages . Meanwhile , performing the train-validation split is also not the only option in practice : there exist algorithms such as Reptile ( Nichol & Schulman , 2018 ) and Meta-MinibatchProx ( Zhou et al. , 2019 ) that can instead use all the per-task data for training the task-specific predictor and also perform well empirically on benchmark tasks . These algorithms modify the loss function in the outer loop so that the training loss no longer matches the meta-test loss , but may have the advantage in terms of data efficiency for the overall problem of learning the best prior . So far it is theoretically unclear how these two approaches ( with/without train-validation split ) compare with each other , which motivates us to ask the following Question : Is the train-validation split necessary and optimal in meta-learning ? In this paper , we perform a detailed theoretical study on the importance of the train-validation split . We consider the linear centroid meta-learning problem ( Denevi et al. , 2018b ) , where for each task we learn a linear predictor that is close to a common centroid in the inner loop , and find the best centroid in the outer loop ( see Section 2 for the detailed problem setup ) . This problem captures the essence of meta-learning with non-linear models ( such as neural networks ) in practice , yet is sufficiently simple that allows a precise theoretical characterization . We use a biased ridge solver as the inner loop with a ( tunable ) regularization parameter , and compare two outer-loop algorithms of either performing the train-validation split ( the train-val method ) or using all the per-task data for both training and evaluation ( the train-train method ) . Specifically , we compare the two methods when the number of tasks T is large , and examine if and how fast they converge to the ( properly defined ) best centroid at meta-test time . We summarize our contributions as follows : • On the linear centroid meta-learning problem , we show that the train-validation split is necessary in the general agnostic setting : As T → ∞ , the train-val method converges to the optimal centroid for test-time adaptation , whereas the train-train method does not without further assumptions on the tasks ( Section 3 ) . The convergence of the train-val method is expected since its ( population ) training loss is equivalent to the meta-test time loss , whereas the non-convergence of the train-train method is because these two losses are not equivalent in general . • Our main theoretical contribution is to show that the train-validation split is not necessary and even non-optimal , in the perhaps more interesting regime when there are structural assumptions on the tasks : When the data are generated from noiseless linear models , both the train-val and traintrain methods converge to the common best centroid , and the train-train method achieves a strictly better ( asymptotic ) estimation error and test loss than the train-val method ( Section 4 ) . This is in stark contrast with the agnostic case , and suggests that data efficiency may indeed be more important when the tasks have a nice structure . Our results build on tools from random matrix theory in the proportional regime , which may be of broader technical interest . • We perform meta-learning experiments on simulations and benchmark few-shot image classification tasks , showing that the train-train method consistently outperforms the train-val method ( Section 5 & Appendix D ) . This validates our theories and presents empirical evidence that samplesplitting may not be crucial ; methods that utilize the per-task data more efficiently may be preferred . 1.1 RELATED WORK . Meta-learning and representation learning theory Baxter ( 2000 ) provided the first theoretical analysis of meta-learning via covering numbers , and Maurer et al . ( 2016 ) improved the analysis via Gaussian complexity techniques . Another recent line of theoretical work analyzed gradient-based meta-learning methods ( Denevi et al. , 2018a ; Finn et al. , 2019 ; Khodak et al. , 2019 ; Ji et al. , 2020 ) and showed guarantees for convex losses by using tools from online convex optimization . Saunshi et al . ( 2020 ) proved the success of Reptile in a one-dimensional subspace setting . Wang et al . ( 2020b ) compared the performance of train-train and train-val methods for learning the learning rate . Denevi et al . ( 2018b ) proposed the linear centroid model studied in this paper , and provided generalization error bounds for train-val method ; the bounds proved also hold for train-train method , so are not sharp enough to compare the two algorithms . Wang et al . ( 2020a ) studied the convergence of gradient-based meta-learning by relating to the kernelized approximation . On the representation learning end , Du et al . ( 2020 ) ; Tripuraneni et al . ( 2020a ; b ) showed that ERM can successfully pool data across tasks to learn the representation . Yet the focus is on the accurate estimation of the common representation , not on the fast adaptation of the learned prior . Lastly , we remark that there are analyses for other representation learning schemes ( McNamara & Balcan , 2017 ; Galanti et al. , 2016 ; Alquier et al. , 2016 ) . Empirical understandings of meta-learning Raghu et al . ( 2020 ) investigated the representation learning perspective of meta-learning and showed that MAML with a full finetuning inner loop mostly learns the top-layer linear classifier and does not change the representation layers much . This result partly justifies the validity our linear centroid meta-learning problem in which the features ( representations ) are fixed and only a linear classifier is learned . Goldblum et al . ( 2020 ) investigated the difference of the neural representations learned by classical training ( supervised learning ) and meta-learning , and showed that the meta-learned representation is both better for downstream adaptation and makes classes more separated than the classically trained one . Although the classical training method in ( Goldblum et al. , 2020 ) does not perform a train-validation split , it is not exactly the same as the train-train method considered in this work as it effectively performs a supervised learning on all tasks combined and does not do a per-task adaptation . Multi-task learning Multi-task learning also exploits structures and similarities across multiple tasks . The earliest idea dates back to Caruana ( 1997 ) ; Thrun & Pratt ( 1998 ) ; Baxter ( 2000 ) , initially in connections to neural network models . They further motivated other approaches using kernel methods ( Evgeniou et al. , 2005 ; Argyriou et al. , 2007 ) and multivariate linear regression models with structured sparsity ( Liu et al. , 2009 ; 2015 ) . More recent advances on deep multi-task learning focus on learning shared intermediate representations across tasks Ruder ( 2017 ) . These multi-task learning approaches usually minimize the joint empirical risk over all tasks , and the models for different tasks are enforced to share a large amount of parameters . In contrast , meta-learning only requires the models to share the same “ prior ” , which is more flexible than multi-task learning . 2 PRELIMINARIES . In this paper , we consider the standard meta-learning setting , in which we observe data from T ≥ 1 supervised learning tasks , and the goal is to find a prior ( or “ initialization ” ) using the combined data , such that the ( T + 1 ) -th new task may be solved sample-efficiently using the prior . Linear centroid meta-learning We instantiate our study on the linear centroid meta-learning problem ( also known as learning to learn around a common mean , Denevi et al . ( 2018b ) ) , where we wish to learn a task-specific linear predictor wt ∈ Rd in the inner loop for each task t , and learn a “ centroid ” w0 in the outer loop that enables fast adaptation to wt within each task : Find the best centroid w0 ∈ Rd for adapting to a linear predictor wt on each task t. Formally , we assume that we observe training data from T ≥ 1 tasks , where for each task index t we sample a task pt ( a distribution over Rd × R ) from some distribution of tasks Π , and observe n examples ( Xt , yt ) ∈ Rn×d × Rn that are drawn i.i.d . from pt : pt ∼ Π , ( Xt , yt ) = { ( xt , i , yt , i ) } ni=1 where ( xt , i , yt , i ) iid∼ pt . ( 1 ) We do not make further assumptions on ( n , d ) ; in particular , we allow the underdetermined setting n ≤ d , in which there exists ( one or many ) interpolators w̃t that perfectly fit the data : Xtw̃t = yt . Inner loop : Ridge solver with biased regularization towards the centroid Our goal in the inner loop is to find a linear predictor wt that fits the data in task twhile being close to the given “ centroid ” w0 ∈ Rd . We instantiate this through ridge regression ( i.e . linear regression with L2 regularization ) where the regularization biases wt towards the centroid . Formally , for any w0 ∈ Rd and any dataset ( X , y ) , we consider the algorithm Aλ ( w0 ; X , y ) : = arg min w 1 n ‖Xw − y‖2 + λ ‖w −w0‖2 = w0 + ( X > X + nλId ) −1 X > ( y −Xw0 ) , where λ > 0 is the regularization strength ( typically a tunable hyper-parameter ) . As we regularize by ‖w −w0‖2 , this inner solver encourages the solution to be close to w0 , as we desire . Such a regularizer is widely used in practical meta-learning algorithms such as MetaOptNet ( Lee et al. , 2019 ) and Meta-MinibatchProx ( Zhou et al. , 2019 ) . In addition , as λ → 0 , this solver recovers gradient descent fine-tuning : we have A0 ( w0 ; X , y ) : = lim λ→0 Aλ ( w0 ; X , y ) = w0 + X† ( y −Xw0 ) = arg minXw=y ‖w −w0‖2 ( where X† ∈ Rd×n denotes the pseudo-inverse of X ) . This is the minimum-distance interpolator of ( X , y ) and also the solution found by gradient descent 1 on ‖Xw − y‖2 initialized at w0 . Therefore our ridge solver with λ > 0 can be seen as a generalized version of the gradient descent solver used in MAML ( Finn et al. , 2017 ) . Outer loop : Finding the best centroid In the outer loop , our goal is to find the best centroid w0 . The standard approach in meta-learning is to perform a train-validation split , that is , ( 1 ) execute the inner solver on a first split of the task-specific data , and ( 2 ) evaluate the loss on a second split , yielding a function of w0 that we can optimize . This two-stage procedure can be written as Compute wt ( w0 ) = Aλ ( w0 ; Xtraint , ytraint ) , and evaluate ∥∥yvalt −Xvalt wt ( w0 ) ∥∥2 . where ( Xtraint , y train t ) = { ( xt , i , yt , i ) } n1i=1 and ( Xvalt , yvalt ) = { ( xt , i , yt , i ) } n i=n1+1 are two disjoint splits of the per-task data ( Xt , yt ) of size ( n1 , n2 ) , with n1 + n2 = n. Written concisely , this is to consider the “ split loss ” ` tr-valt ( w0 ) : = 1 2n2 ∥∥yvalt −Xvalt Aλ ( w0 ; Xtraint , ytraint ) ∥∥2 . ( 2 ) In this paper , we will also consider an alternative version , where we do not perform the trainvalidation split , but instead use all the per-task data for both training and evaluation . Mathematically , this is to look at the “ non-split loss ” ` tr-trt ( w0 ) : = 1 2n ‖yt −XtAλ ( w0 ; Xt , yt ) ‖2 . ( 3 ) Our overall algorithm is to solve the empirical risk minimization ( ERM ) problem on the T observed tasks , using either one of the two losses above : L̂tr-valT ( w0 ) : = 1 T T∑ t=1 ` tr-valt ( w0 ) and L̂ tr-tr T ( w0 ) : = 1 T T∑ t=1 ` tr-trt ( w0 ) , ŵ { tr-val , tr-tr } 0 , T : = arg min w0 L̂ { tr-val , tr-tr } T ( w0 ) . ( 4 ) Let L { tr-val , tr-tr } ( w0 ) : = Ept∼Π , ( Xt , yt ) ∼pt [ ` { tr-val , tr-tr } t ( w0 ) ] be the population risks . ( Meta- ) Test time The meta-test time performance of any meta-learning algorithm is a joint function of the ( learned ) centroid w0 and the inner algorithm Alg . Upon receiving a new task pT+1 ∼ Π and training data ( XT+1 , yT+1 ) ∈ Rn×d × Rn , we run the inner loop Alg with prior w0 on the training data , and evaluate it on an ( unseen ) test example ( x′ , y′ ) ∼ pT+1 : Ltest ( w0 ; Alg ) : = EpT+1∼ΠE ( XT+1 , yT+1 ) , ( x′ , y′ ) iid∼pT+1 [ 1 2 ( x′ > Alg ( w0 ; XT+1 , yT+1 ) − y′ ) 2 ] . Additionally , for both train-val and train-train methods , we need to ensure that the inner loop used for meta-test is exactly the same as that used in meta-training . Therefore , the meta-test performance for the train-val and train-train methods above should be evaluated as Ltestλ , n1 ( ŵ tr-val 0 , T ) : = L test ( ŵtr-val0 , T ; Aλ , n1 ) , Ltestλ , n ( ŵtr-tr0 , T ) : = Ltest ( ŵtr-tr0 , T ; Aλ , n ) , where Aλ , m denotes the ridge solver with regularization strength λ > 0 on m ≤ n data points . Finally , we let w0 , ? ( λ ; n ) = arg min w0 Ltestλ , n ( w0 ) ( 5 ) denote the best centroid if the inner loop uses Aλ , n . The performance of the train-val algorithm ŵtr-val0 , T should be compared against w0 , ? ( λ , n1 ) , whereas the train-train algorithm ŵ tr-tr 0 , T should be compared against w0 , ? ( λ , n ) . 1with a small step-size , or gradient flow .

Summary: In meta-learning, a common practice is to do a train/validation split of the data within each that, so that optimization of meta-parameters is performed on validation, not training, losses. In this paper, the author argue that this split is important for correcting model misspecification, but if the model is correctly specified, not doing a split might actually lead to better learning rates. They provide theoretical justification under a simple linear model, and some experiments on synthetic and real data.