Meta-learning with negative learning rates

1 INTRODUCTION . Deep Learning models represent the state-of-the-art in several machine learning benchmarks ( LeCun et al . ( 2015 ) ) , and their performance does not seem to stop improving when adding more data and computing resources ( Rosenfeld et al . ( 2020 ) , Kaplan et al . ( 2020 ) ) . However , they require a large amount of data and compute to start with , which are often not available to practitioners . The approach of fine-tuning has proved very effective to address this limitation : pre-train a model on a source task , for which a large dataset is available , and use this model as the starting point for a quick additional training ( fine-tuning ) on the small dataset of the target task ( Pan & Yang ( 2010 ) , Donahue et al . ( 2014 ) , Yosinski et al . ( 2014 ) ) . This approach is popular because pre-trained models are often made available by institutions that have the resources to train them . In some circumstances , multiple source tasks are available , all of which have scarce data , as opposed to a single source task with abundant data . This case is addressed by meta-learning , in which a model gains experience over multiple source tasks and uses it to improve its learning of future target tasks . The idea of meta-learning is inspired by the ability of humans to generalize across tasks , without having to train on any single task for long time . A meta-learning problem is solved by a bi-level optimization procedure : an outer loop optimizes meta-parameters across tasks , while an inner loop optimizes parameters within each task ( Hospedales et al . ( 2020 ) ) . The idea of meta-learning has gained some popularity , but a few recent papers argue that a simple alternative to meta-learning is just good enough , in which the inner loop is removed entirely ( Chen et al . ( 2020a ) , Tian et al . ( 2020 ) , Dhillon et al . ( 2020 ) , Chen et al . ( 2020b ) , Raghu et al . ( 2020 ) ) . Other studies find the opposite ( Goldblum et al . ( 2020 ) , Collins et al . ( 2020 ) , Gao & Sener ( 2020 ) ) . It is hard to resolve the debate because there is little theory available to explain these findings . In this work , using random matrix theory and exact solutions of linear models , we derive an algebraic expression of the average test loss of MAML , a simple and successful meta-learning algorithm ( Finn et al . ( 2017 ) ) , as a function of its hyperparameters . In particular , we study its performance as a function of the inner loop learning rate during meta-training . Setting this learning rate to zero is equivalent to removing the inner loop , as advocated by recent work ( Chen et al . ( 2020a ) , Tian et al . ( 2020 ) , Dhillon et al . ( 2020 ) , Chen et al . ( 2020b ) , Raghu et al . ( 2020 ) ) . Surprisingly , we find that the optimal learning rate is negative , thus performance can be increased by reducing the learning rate below zero . In particular , we find the following : • In the problem of mixed linear regression , we prove that the optimal learning rate is always negative in overparameterized models . The same result holds in underparameterized models provided that the optimal learning rate is small in absolute value . We validate the theory by running extensive experiments . • We extend these results to the case of nonlinear regression and wide neural networks , in which the output can be approximated by a linear function of the parameters ( Jacot et al . ( 2018 ) , Lee et al . ( 2019 ) ) . While in this case we can not prove that the optimal learning rate is always negative , preliminary experiments suggest that the result holds in this case as well . 2 RELATED WORK . The field of meta-learning includes a broad range of problems and solutions , see Hospedales et al . ( 2020 ) for a recent review focusing on neural networks and deep learning . In this context , metalearning received increased attention in the past few years , several new benchmarks have been introduced , and a large number of algorithms and models have been proposed to solve them ( Vinyals et al . ( 2017 ) , Bertinetto et al . ( 2019 ) , Triantafillou et al . ( 2020 ) ) . Despite the surge in empirical work , theoretical work is still lagging behind . Similar to our work , a few other studies used random matrix theory and exact solutions to calculate the average test loss for the problem of linear regression ( Advani & Saxe ( 2017 ) , Hastie et al . ( 2019 ) , Nakkiran ( 2019 ) ) . To our knowledge , our study is the first to apply this technique to the problem of meta-learning with multiple tasks . Our results reduce to those of linear regression in the case of one single task . Furthermore , we are among the first to apply the framework of Neural Tangent Kernel ( Jacot et al . ( 2018 ) , Lee et al . ( 2019 ) ) to the problem of meta-learning ( a few papers appeared after our submission : Yang & Hu ( 2020 ) , Wang et al . ( 2020a ) , Zhou et al . ( 2021 ) ) . Similar to us , a few theoretical studies looked at the problem of mixed linear regression in the context of meta-learning . In Denevi et al . ( 2018 ) , Bai et al . ( 2021 ) , a meta-parameter is used to bias the taskspecific parameters through a regularization term . Kong et al . ( 2020 ) looks at whether many tasks with small data can compensate for a lack of tasks with big data . Tripuraneni et al . ( 2020 ) , Du et al . ( 2020 ) study the sample complexity of representation learning . However , none of these studies look into the effect of learning rate on performance , which is our main focus . In this work , we focus on MAML , a simple and successful meta-learning algorithm ( Finn et al . ( 2017 ) ) . A few theoretical studies have investigated MAML , looking at : universality of the optimization algorithm ( Finn & Levine ( 2018 ) ) , bayesian inference interpretation ( Grant et al . ( 2018 ) ) , proof of convergence ( Ji et al . ( 2020 ) ) , difference between convex and non-convex losses ( Saunshi et al . ( 2020 ) ) , global optimality ( Wang et al . ( 2020b ) ) , effect of the inner loop ( Collins et al . ( 2020 ) , Gao & Sener ( 2020 ) ) . Again , none of these studies look at the effect of the learning rate , the main subject of our work . The theoretical work of Khodak et al . ( 2019 ) connects the learning rate to task similarity , while the work of Li et al . ( 2017 ) meta-learns the learning rate . 3 META-LEARNING AND MAML . In this work , we follow the notation of Hospedales et al . ( 2020 ) and we use MAML ( Finn et al . ( 2017 ) ) as the meta-learning algorithm . We assume the existence of a distribution of tasks τ and , for each task , a loss function Lτ and a distribution of data pointsDτ = { xτ , yτ } with input xτ and label yτ . We assume that the loss function is the same for all tasks , Lτ = L , but each task is characterized by a different distribution of the data . The empirical meta-learning loss is evaluated on a sample of m tasks , and a sample of nv validation data points for each task : Lmeta ( ω ; Dt , Dv ) = 1 mnv m∑ i=1 nv∑ j=1 L ( θ ( ω ; D ( i ) t ) ; x v ( i ) j , y v ( i ) j ) ( 1 ) The training set D ( i ) t = { x t ( i ) j , y t ( i ) j } j=1 : nt and validation set D ( i ) v = { x v ( i ) j , y v ( i ) j } j=1 : nv are drawn independently from the same distribution in each task i . The function θ represents the adaptation of the meta-parameter ω , which is evaluated on the training set . Different meta-learning algorithms correspond to a different choice of θ , we describe below the choice of MAML ( Eq.3 ) , the subject of this study . During meta-training , the loss of Eq.1 is optimized with respect to the meta-parameter ω , usually by stochastic gradient descent , starting from an initial point ω0 . The optimum is denoted as ω ? ( Dt , Dv ) . This optimization is referred to as the outer loop , while computation of θ is referred to as the inner loop of meta-learning . During meta-testing , a new ( target ) task is given and θ adapts on a set Dr of nr target data points . The final performance of the model is computed on test data Ds of the target task . Therefore , the test loss is equal to Ltest = Lmeta ( ω ? ( Dt , Dv ) ; Dr , Ds ) ( 2 ) In MAML , the inner loop corresponds to a few steps of gradient descent , with a given learning rate αt . In this work we consider the simple case of a single gradient step : θ ( ω ; D ( i ) t ) = ω − αt nt nt∑ j=1 ∂L ∂θ ∣∣∣∣ ω ; x t ( i ) j , y t ( i ) j ( 3 ) If the learning rate αt is zero , then parameters are not adapted during meta-training and θ ( ω ) = ω . In that case , a single set of parameters in learned across all data and there is no inner loop . However , it is important to note that a distinct learning rate αr is used during meta-testing . A setting similar to this has been advocated in a few recent studies ( Chen et al . ( 2020a ) , Tian et al . ( 2020 ) , Dhillon et al . ( 2020 ) , Chen et al . ( 2020b ) , Raghu et al . ( 2020 ) ) . We show that , intuitively , the optimal learning rate at meta-testing ( adaptation ) time αr is always positive . Surprisingly , in the family of problems considered in this study , we find that the optimal learning rate during meta-training αt is instead negative . We note that the setting αt = 0 effectively does not use the nt training data points , therefore we could in principle add this data to the validation set , but we do not consider this option here since we are interested in a wide range of possible values of αt as opposed to the specific case αt = 0 .

Summary: This paper studies meta-learning in the mixed linear regression setting, focusing on the effect of the within-task step-size on performance. For over-parameterized, under-parameterized, and NTK regimes they derive expressions for test-time loss that suggest that negative or close-to-zero learning rates are optimal, and provide experiments that closely match these results. However, some aspects of the mathematical approach are unclear, and the work's impact is limited without an investigation of the consequences of the analysis.