Effective Meta-Regularization by Kernelized Proximal Regularization

1 Introduction . Humans , by leveraging prior knowledge and experience , can easily learn new tasks from a handful of examples . In contrast , deep networks are data-hungry , and a large number of training samples are required to avoid overfitting . To reduce the labor-intensive and time-consuming process of data labeling , meta-learning ( or learning to learn ) [ 3 , 37 ] aims to exact meta-knowledge from seen tasks to accelerate learning on unseen tasks . Recently , meta-learning has been receiving increasing attention due to its diverse successful applications in few-shot learning [ 41 , 12 , 39 , 35 ] , hyperparameter optimization [ 14 ] , neural architecture search [ 24 , 44 ] , and reinforcement learning [ 29 ] . Many meta-learning algorithms operate on two levels . A base learner learns task-specific models in the inner loop , and a meta-learner learns the meta-parameter in the outer loop . A popular class of algorithms is based on meta-initialization [ 12 , 25 , 11 , 38 ] , such as the well-known MAML [ 12 ] . It learns a model initialization such that a good model for an unseen task can be learned from limited samples by a few gradient updates . However , computing the meta-gradient requires back-propagating through the entire inner optimization path , which is infeasible for large models and/or there are many gradient steps . During testing , it is common for MAML ’ s base learner to perform many gradient steps to seek a more accurate solution [ 12 ] . However , for regression using a linear base learner and square loss , we will show that though the meta-learner can converge to the optimal meta-initialization , the base learner may overfit the training data at meta-testing . Another class of meta-learning algorithms is based on meta-regularization [ 28 , 43 , 7 , 8 , 9 ] , in which the base learner learns the task-specific model by minimizing the loss with a proximal regularizer ( a biased regularizer from the meta-parameter ) . Denevi et al . [ 7 ] uses a linear model with efficient closed-form solution for the base learner . However , extending to nonlinear base learners requires computing the meta-gradient using matrix inversion , which can be infeasible for deep networks [ 28 ] . To introduce nonlinearity to the base learner , a recent approach is to make use of the kernel trick . For example , R2D2 [ 4 ] and MetaOptNet [ 22 ] use deep kernels [ 42 ] in meta-learning for few-shot ∗Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . classification . Specifically , the deep network is learned in the meta-learner , while a base kernel is used in the base learner . Though they achieve state-of-the-art performance , their base learners use a Tikhonov regularizer rather than a learnable proximal regularizer as in meta-regularization methods . As learning a meta-regularization has been shown to be effective in linear models for regression [ 7 ] and classification [ 8 ] , in this paper we propose a kernel-based algorithm to meta-learn a proximal regularizer for a nonlinear base learner . After kernel extension , the learnable function in the proximal regularizer is a function in the reproducing kernel Hilbert space ( RKHS ) induced by the base kernel . The proposed algorithm is guaranteed to converge to a critical point of the meta-loss and its global convergence is also established . Experiments on various benchmark regression and classification datasets demonstrate the superiority of the proposed algorithm over the state-of-the-arts . Notations . Vectors are denoted by lowercase boldface , and matrices by uppercase boldface . For a vector x , ‖x‖ = √∑ i x 2 i and diag ( x ) constructs a diagonal matrix with x on the diagonal . ‖ · ‖H is the norm on the Hilbert spaceH . N ( 0 , σ2 ) is the univariate normal distribution with mean zero and variance σ2 . N ( m , Σ ) is the multivariate normal distribution with mean m and covariance matrix Σ . 2 Related Work . In meta-learning , a collection T of tasks are used to learn a meta-parameter θ and base learner ’ s parameters { w1 , . . . , w|T | } . Each task τ is sampled from a given distribution p ( τ ) , and has a support set Sτ = { ( xi , yi ) : i = 1 , . . . , ns } and a query set Qτ = { ( xi , yi ) : i = 1 , . . . , nq } , where x ∈ Rd are the features and y the labels . Let f ( · ; w ) be a model parameterized by w and L ( D ; w ) ≡ ∑ ( x , y ) ∈D ` ( f ( x ; w ) , y ) be the supervised loss on data set D , where ` ( · , · ) is a loss function that is assumed to be convex w.r.t . the first argument . In each meta-training iteration , a batch B of tasks are randomly sampled from T . The base learner takes a task τ from B and the meta-parameter θ to build the model f ( · ; wτ ) . After all tasks in the batch are processed by the base learner , the meta-learner minimizes the loss ∑ τ∈B L ( Qτ ; wτ ) w.r.t . θ , and the iteration repeats . During meta-testing , given an unseen task τ ′ ∼ p ( τ ) , a model f ( · ; wτ ′ ) is similarly learned from Sτ ′ and θ . Finally , its performance is evaluated on Qτ ′ . Popular meta-learning algorithms usually construct the task-specific model by : ( i ) metainitialization [ 12 , 25 , 11 , 38 ] , ( ii ) meta-regularization [ 7 , 8 , 9 , 28 , 43 ] , or ( iii ) metric learning [ 35 , 4 , 22 , 27 ] . A representative meta-learning algorithm based on meta-initialization is MAML [ 12 ] . Its base learner learns wτ by gradient descent from a learnable initialization . Computing the meta-gradient ∇θL ( Qτ ; wτ ) needs to back-propagate through the inner optimization path and involves second-order derivatives . This can be expensive for large models and/or when there are many gradient steps . For meta-learning algorithms based on meta-regularization , the base learner learns wτ by minimizing a proximal regularized loss L ( Sτ ; w ) + λ 2 ‖w − θ‖2 , ( 1 ) where λ > 0 is the regularization parameter . The meta-gradient can be computed directly from the learned wτ , without back-propagating through the inner optimization trajectory [ 28 ] . Though more efficient , this still takes O ( n3θ ) time [ 28 ] , where nθ is the number of parameters in θ . Metric learning methods have been widely studied in few-shot learning [ 39 , 35 , 4 , 22 , 27 ] . The meta-learner maps raw samples to an embedding space via a a deep network , then feeds the embeddings to the base learner to train a simple task model . Typical models include non-parametric prototype classifier ( ProtoNet [ 35 ] ) , linear models like ridge regression ( R2D2 [ 4 ] ) , SVM classifier ( MetaOptNet-SVM [ 22 ] ) , and softmax classifier ( ANIL [ 27 ] ) . In particular , the base learners in R2D2 [ 4 ] , MetaOptNet [ 22 ] and DKT [ 26 ] seek solutions in the dual space , which achieve state-ofthe-art performance . However , their base learners use a Tikhonov regularizer , which is not learnable as in meta-regularization approaches . 3 Meta-Regularization by Kernelized Proximal Regularization . 3.1 Meta-Initialization versus Meta-Regularization . In this section , we consider a simple regression setting with linear model and square loss . Each task τ is a linear regressor with parameter w∗τ ∈ Rd . We assume that each input x is sampled Algorithm 1 MAML [ 12 ] . Require : step size γ and ηt , batch size b ; 1 : for t = 1 , 2 , 3 , . . . do 2 : sample a batch Bt of tasks from p ( τ ) ; 3 : base learner : 4 : for τ ∈ Bt do 5 : w ( gd ) τ = ψt − γX > τ ( Xτψt − yτ ) ; 6 : gτ = 12 ∑ ( x , y ) ∈Qτ ∇ψt ( x > w ( gd ) τ − y ) 2 ; 7 : end for 8 : meta-learner : ψt+1 = ψt − ηtb ∑ τ∈Bt gτ ; 9 : end for Algorithm 2 CommonMean [ 7 ] . Require : hyperparameter λ , step size ηt , batch size b ; 1 : for t = 1 , 2 , 3 , . . . do 2 : sample a batch Bt of tasks from p ( τ ) ; 3 : base learner : 4 : for τ ∈ Bt do 5 : w ( prox ) τ = minw 12‖Xτw−yτ‖ 2+ λ 2 ‖w−θt‖2 ; 6 : gτ = 12 ∑ ( x , y ) ∈Qτ ∇θt ( x > w ( prox ) τ − y ) 2 ; 7 : end for 8 : meta-learner : θt+1 = θt − ηtb ∑ τ∈Bt gτ ; 9 : end for from N ( 0 , σ2xI ) , and the output y is obtained as x > w∗τ + ξ , where ξ ∼ N ( 0 , σ2ξ ) is the random noise . We compare two representative meta-learning algorithms : ( i ) MAML [ 12 ] , which is based on meta-initialization and performs one gradient descent step in the inner loop of the bilevel optimization problem ; and ( ii ) learning around a common mean ( denoted CommonMean ) [ 7 ] , which is based on meta-regularization . It learns the model parameters for task τ by minimizing the loss with a proximal regularizer around the meta-parameter θ : w ( prox ) τ =argmin w ∑ ( xi , yi ) ∈Sτ 1 2 ( w > xi−yi ) 2+ λ 2 ‖w−θ‖2 = ( λI+X > τ Xτ ) −1 ( λθ+X > τ yτ ) , ( 2 ) where Xτ = [ x > 1 ; . . . ; x > ns ] is the sample matrix from Sτ , and yτ = [ y1 ; . . . ; yns ] is the corresponding label vector . Algorithms 1 and 2 show MAML and CommonMean , respectively , for this problem . Recently , Balcan et al . [ 2 ] study the convex online meta-learning setting and show that both approaches achieve the same average task regret . Here , we consider the offline setting . First , the following Proposition shows that both MAML and CommonMean converge to the same meta-parameter . All proofs are in the appendix . Proposition 1 . Let ηt = 1/t . Assume that γ < 1/σ2x . Both ψt in MAML ( with one inner gradient step ) and θt in CommonMean converge to w̄ = Eτw∗τ . The following Proposition shows that w̄ in Proposition 1 is also the best ψ ( for MAML ) or θ ( for CommonMean ) with the smallest population risk for this meta-learning problem . Proposition 2 . Assume that γ < 1/σ2x . We have w̄ = argminθ EτESτEQτ ∑ ( x , y ) ∈Qτ ( x > w ( prox ) τ − y ) 2 =argminψ EτESτEQτ ∑ ( x , y ) ∈Qτ ( x > w ( gd ) τ −y ) 2 . During meta-testing , we sample a task τ ′ ∼ p ( τ ) with parameter w∗τ ′ . Let Xτ ′ be the sample matrix from Sτ ′ , and yτ ′ be the corresponding label vector . We assume that Xτ ′ is full rank and ns < d. To simplify notations , we drop the subscript τ ′ in the following . Let the singular value decomposition of X be UΣV > ( where Σ = diag ( [ ν1 , . . . , νns ] ) ) , and V ⊥ be V ’ s orthogonal complement . As only forward passes are needed , it is common for the base learner in MAML to perform multiple gradient steps [ 12 ] . With the convex loss and linear model here , the base learner can obtain a globally optimal solution w ( gd∞ ) directly ( which is equivalent to taking infinite gradient steps ) . As is common in few-shot learning , the number of support samples is much smaller than feature dimensionality . Hence , w ( gd∞ ) is not unique but depends on the learned initialization . Let w ( gd∞ ) be written as w ( gd∞ ) = Va ( gd∞ ) + V⊥b ( gd∞ ) . For gradient descent , its update direction X > ( Xw − y ) = VΣU > ( Xw − y ) is always in the span of V and so b ( gd∞ ) remains unchanged . Let w∗ and θ be written as w∗ = Va∗ + V⊥b∗ and θ = Va0 + V⊥b0 . Moreover , let ã = a0 − a∗ and b̃ = b0 − b∗ . Proposition 3 ( [ 17 ] ) . Assume that γ < min1≤j≤ns 1/ν2j . We have Eξ‖w ( gd∞ ) −w∗‖2 = ‖b ( gd∞ ) − b∗‖2 + ∑ns j=1 ( σξ νj ) 2 , where the expectation is over the label noise vector ξ . For CommonMean , using the Woodbury matrix identity , w ( prox ) = θ+X > ( λI+XX > ) −1 ( y−Xθ ) = V ( a0 + ΣU > ( λI + XX > ) −1 ( y − Xθ ) ) + V⊥b0 ≡ Va ( prox ) + V⊥b ( prox ) , where b ( prox ) = b0 . Assume that w ( gd∞ ) is initialized with θ . Since b ( gd∞ ) remains unchanged , w ( prox ) and w ( gd∞ ) only differ in the components lying in the column space of V. Proposition 4 . Eξ‖w ( prox ) −w∗‖2 = ‖b̃‖2 + ∑ns j=1 ( λãj λ+ν2j ) 2 + ∑ns j=1 ( σξ ( λ/νj ) +νj ) 2 , where the expectation is over the label noise vector ξ . As can be seen , when the labels are noise-free ( σ2ξ = 0 ) , w ( gd∞ ) performs better than w ( prox ) . However , when the labels are noisy , as ns < d , gradient descent always converges to zero training error and overfits the noisy labels . On the other hand , the estimation error of w ( prox ) equals to that of w ( gd∞ ) when λ = 0 . For λ > 0 , it trades off between fitting the noisy labels ( the last term in Proposition 4 ) and introducing an estimation bias of a∗ ( the second term in Proposition 4 ) .

Summary: The authors proposed a kernelized proximal regularization method (MetaProx) for meta learning. MetaProx combines deep kernel and proximal meta regularization, allowing learnable regularization by reformulating the proximal problem in dual space. Thorough theoretical analysis are provided as well as empirical studies on regression and classification task.