Effective Meta-Regularization by Kernelized Proximal Regularization

1 Introduction . Humans , by leveraging prior knowledge and experience , can easily learn new tasks from a handful of examples . In contrast , deep networks are data-hungry , and a large number of training samples are required to avoid overfitting . To reduce the labor-intensive and time-consuming process of data labeling , meta-learning ( or learning to learn ) [ 3 , 37 ] aims to exact meta-knowledge from seen tasks to accelerate learning on unseen tasks . Recently , meta-learning has been receiving increasing attention due to its diverse successful applications in few-shot learning [ 41 , 12 , 39 , 35 ] , hyperparameter optimization [ 14 ] , neural architecture search [ 24 , 44 ] , and reinforcement learning [ 29 ] . Many meta-learning algorithms operate on two levels . A base learner learns task-specific models in the inner loop , and a meta-learner learns the meta-parameter in the outer loop . A popular class of algorithms is based on meta-initialization [ 12 , 25 , 11 , 38 ] , such as the well-known MAML [ 12 ] . It learns a model initialization such that a good model for an unseen task can be learned from limited samples by a few gradient updates . However , computing the meta-gradient requires back-propagating through the entire inner optimization path , which is infeasible for large models and/or there are many gradient steps . During testing , it is common for MAML ’ s base learner to perform many gradient steps to seek a more accurate solution [ 12 ] . However , for regression using a linear base learner and square loss , we will show that though the meta-learner can converge to the optimal meta-initialization , the base learner may overfit the training data at meta-testing . Another class of meta-learning algorithms is based on meta-regularization [ 28 , 43 , 7 , 8 , 9 ] , in which the base learner learns the task-specific model by minimizing the loss with a proximal regularizer ( a biased regularizer from the meta-parameter ) . Denevi et al . [ 7 ] uses a linear model with efficient closed-form solution for the base learner . However , extending to nonlinear base learners requires computing the meta-gradient using matrix inversion , which can be infeasible for deep networks [ 28 ] . To introduce nonlinearity to the base learner , a recent approach is to make use of the kernel trick . For example , R2D2 [ 4 ] and MetaOptNet [ 22 ] use deep kernels [ 42 ] in meta-learning for few-shot ∗Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . classification . Specifically , the deep network is learned in the meta-learner , while a base kernel is used in the base learner . Though they achieve state-of-the-art performance , their base learners use a Tikhonov regularizer rather than a learnable proximal regularizer as in meta-regularization methods . As learning a meta-regularization has been shown to be effective in linear models for regression [ 7 ] and classification [ 8 ] , in this paper we propose a kernel-based algorithm to meta-learn a proximal regularizer for a nonlinear base learner . After kernel extension , the learnable function in the proximal regularizer is a function in the reproducing kernel Hilbert space ( RKHS ) induced by the base kernel . The proposed algorithm is guaranteed to converge to a critical point of the meta-loss and its global convergence is also established . Experiments on various benchmark regression and classification datasets demonstrate the superiority of the proposed algorithm over the state-of-the-arts . Notations . Vectors are denoted by lowercase boldface , and matrices by uppercase boldface . For a vector x , ‖x‖ = √∑ i x 2 i and diag ( x ) constructs a diagonal matrix with x on the diagonal . ‖ · ‖H is the norm on the Hilbert spaceH . N ( 0 , σ2 ) is the univariate normal distribution with mean zero and variance σ2 . N ( m , Σ ) is the multivariate normal distribution with mean m and covariance matrix Σ . 2 Related Work . In meta-learning , a collection T of tasks are used to learn a meta-parameter θ and base learner ’ s parameters { w1 , . . . , w|T | } . Each task τ is sampled from a given distribution p ( τ ) , and has a support set Sτ = { ( xi , yi ) : i = 1 , . . . , ns } and a query set Qτ = { ( xi , yi ) : i = 1 , . . . , nq } , where x ∈ Rd are the features and y the labels . Let f ( · ; w ) be a model parameterized by w and L ( D ; w ) ≡ ∑ ( x , y ) ∈D ` ( f ( x ; w ) , y ) be the supervised loss on data set D , where ` ( · , · ) is a loss function that is assumed to be convex w.r.t . the first argument . In each meta-training iteration , a batch B of tasks are randomly sampled from T . The base learner takes a task τ from B and the meta-parameter θ to build the model f ( · ; wτ ) . After all tasks in the batch are processed by the base learner , the meta-learner minimizes the loss ∑ τ∈B L ( Qτ ; wτ ) w.r.t . θ , and the iteration repeats . During meta-testing , given an unseen task τ ′ ∼ p ( τ ) , a model f ( · ; wτ ′ ) is similarly learned from Sτ ′ and θ . Finally , its performance is evaluated on Qτ ′ . Popular meta-learning algorithms usually construct the task-specific model by : ( i ) metainitialization [ 12 , 25 , 11 , 38 ] , ( ii ) meta-regularization [ 7 , 8 , 9 , 28 , 43 ] , or ( iii ) metric learning [ 35 , 4 , 22 , 27 ] . A representative meta-learning algorithm based on meta-initialization is MAML [ 12 ] . Its base learner learns wτ by gradient descent from a learnable initialization . Computing the meta-gradient ∇θL ( Qτ ; wτ ) needs to back-propagate through the inner optimization path and involves second-order derivatives . This can be expensive for large models and/or when there are many gradient steps . For meta-learning algorithms based on meta-regularization , the base learner learns wτ by minimizing a proximal regularized loss L ( Sτ ; w ) + λ 2 ‖w − θ‖2 , ( 1 ) where λ > 0 is the regularization parameter . The meta-gradient can be computed directly from the learned wτ , without back-propagating through the inner optimization trajectory [ 28 ] . Though more efficient , this still takes O ( n3θ ) time [ 28 ] , where nθ is the number of parameters in θ . Metric learning methods have been widely studied in few-shot learning [ 39 , 35 , 4 , 22 , 27 ] . The meta-learner maps raw samples to an embedding space via a a deep network , then feeds the embeddings to the base learner to train a simple task model . Typical models include non-parametric prototype classifier ( ProtoNet [ 35 ] ) , linear models like ridge regression ( R2D2 [ 4 ] ) , SVM classifier ( MetaOptNet-SVM [ 22 ] ) , and softmax classifier ( ANIL [ 27 ] ) . In particular , the base learners in R2D2 [ 4 ] , MetaOptNet [ 22 ] and DKT [ 26 ] seek solutions in the dual space , which achieve state-ofthe-art performance . However , their base learners use a Tikhonov regularizer , which is not learnable as in meta-regularization approaches . 3 Meta-Regularization by Kernelized Proximal Regularization . 3.1 Meta-Initialization versus Meta-Regularization . In this section , we consider a simple regression setting with linear model and square loss . Each task τ is a linear regressor with parameter w∗τ ∈ Rd . We assume that each input x is sampled Algorithm 1 MAML [ 12 ] . Require : step size γ and ηt , batch size b ; 1 : for t = 1 , 2 , 3 , . . . do 2 : sample a batch Bt of tasks from p ( τ ) ; 3 : base learner : 4 : for τ ∈ Bt do 5 : w ( gd ) τ = ψt − γX > τ ( Xτψt − yτ ) ; 6 : gτ = 12 ∑ ( x , y ) ∈Qτ ∇ψt ( x > w ( gd ) τ − y ) 2 ; 7 : end for 8 : meta-learner : ψt+1 = ψt − ηtb ∑ τ∈Bt gτ ; 9 : end for Algorithm 2 CommonMean [ 7 ] . Require : hyperparameter λ , step size ηt , batch size b ; 1 : for t = 1 , 2 , 3 , . . . do 2 : sample a batch Bt of tasks from p ( τ ) ; 3 : base learner : 4 : for τ ∈ Bt do 5 : w ( prox ) τ = minw 12‖Xτw−yτ‖ 2+ λ 2 ‖w−θt‖2 ; 6 : gτ = 12 ∑ ( x , y ) ∈Qτ ∇θt ( x > w ( prox ) τ − y ) 2 ; 7 : end for 8 : meta-learner : θt+1 = θt − ηtb ∑ τ∈Bt gτ ; 9 : end for from N ( 0 , σ2xI ) , and the output y is obtained as x > w∗τ + ξ , where ξ ∼ N ( 0 , σ2ξ ) is the random noise . We compare two representative meta-learning algorithms : ( i ) MAML [ 12 ] , which is based on meta-initialization and performs one gradient descent step in the inner loop of the bilevel optimization problem ; and ( ii ) learning around a common mean ( denoted CommonMean ) [ 7 ] , which is based on meta-regularization . It learns the model parameters for task τ by minimizing the loss with a proximal regularizer around the meta-parameter θ : w ( prox ) τ =argmin w ∑ ( xi , yi ) ∈Sτ 1 2 ( w > xi−yi ) 2+ λ 2 ‖w−θ‖2 = ( λI+X > τ Xτ ) −1 ( λθ+X > τ yτ ) , ( 2 ) where Xτ = [ x > 1 ; . . . ; x > ns ] is the sample matrix from Sτ , and yτ = [ y1 ; . . . ; yns ] is the corresponding label vector . Algorithms 1 and 2 show MAML and CommonMean , respectively , for this problem . Recently , Balcan et al . [ 2 ] study the convex online meta-learning setting and show that both approaches achieve the same average task regret . Here , we consider the offline setting . First , the following Proposition shows that both MAML and CommonMean converge to the same meta-parameter . All proofs are in the appendix . Proposition 1 . Let ηt = 1/t . Assume that γ < 1/σ2x . Both ψt in MAML ( with one inner gradient step ) and θt in CommonMean converge to w̄ = Eτw∗τ . The following Proposition shows that w̄ in Proposition 1 is also the best ψ ( for MAML ) or θ ( for CommonMean ) with the smallest population risk for this meta-learning problem . Proposition 2 . Assume that γ < 1/σ2x . We have w̄ = argminθ EτESτEQτ ∑ ( x , y ) ∈Qτ ( x > w ( prox ) τ − y ) 2 =argminψ EτESτEQτ ∑ ( x , y ) ∈Qτ ( x > w ( gd ) τ −y ) 2 . During meta-testing , we sample a task τ ′ ∼ p ( τ ) with parameter w∗τ ′ . Let Xτ ′ be the sample matrix from Sτ ′ , and yτ ′ be the corresponding label vector . We assume that Xτ ′ is full rank and ns < d. To simplify notations , we drop the subscript τ ′ in the following . Let the singular value decomposition of X be UΣV > ( where Σ = diag ( [ ν1 , . . . , νns ] ) ) , and V ⊥ be V ’ s orthogonal complement . As only forward passes are needed , it is common for the base learner in MAML to perform multiple gradient steps [ 12 ] . With the convex loss and linear model here , the base learner can obtain a globally optimal solution w ( gd∞ ) directly ( which is equivalent to taking infinite gradient steps ) . As is common in few-shot learning , the number of support samples is much smaller than feature dimensionality . Hence , w ( gd∞ ) is not unique but depends on the learned initialization . Let w ( gd∞ ) be written as w ( gd∞ ) = Va ( gd∞ ) + V⊥b ( gd∞ ) . For gradient descent , its update direction X > ( Xw − y ) = VΣU > ( Xw − y ) is always in the span of V and so b ( gd∞ ) remains unchanged . Let w∗ and θ be written as w∗ = Va∗ + V⊥b∗ and θ = Va0 + V⊥b0 . Moreover , let ã = a0 − a∗ and b̃ = b0 − b∗ . Proposition 3 ( [ 17 ] ) . Assume that γ < min1≤j≤ns 1/ν2j . We have Eξ‖w ( gd∞ ) −w∗‖2 = ‖b ( gd∞ ) − b∗‖2 + ∑ns j=1 ( σξ νj ) 2 , where the expectation is over the label noise vector ξ . For CommonMean , using the Woodbury matrix identity , w ( prox ) = θ+X > ( λI+XX > ) −1 ( y−Xθ ) = V ( a0 + ΣU > ( λI + XX > ) −1 ( y − Xθ ) ) + V⊥b0 ≡ Va ( prox ) + V⊥b ( prox ) , where b ( prox ) = b0 . Assume that w ( gd∞ ) is initialized with θ . Since b ( gd∞ ) remains unchanged , w ( prox ) and w ( gd∞ ) only differ in the components lying in the column space of V. Proposition 4 . Eξ‖w ( prox ) −w∗‖2 = ‖b̃‖2 + ∑ns j=1 ( λãj λ+ν2j ) 2 + ∑ns j=1 ( σξ ( λ/νj ) +νj ) 2 , where the expectation is over the label noise vector ξ . As can be seen , when the labels are noise-free ( σ2ξ = 0 ) , w ( gd∞ ) performs better than w ( prox ) . However , when the labels are noisy , as ns < d , gradient descent always converges to zero training error and overfits the noisy labels . On the other hand , the estimation error of w ( prox ) equals to that of w ( gd∞ ) when λ = 0 . For λ > 0 , it trades off between fitting the noisy labels ( the last term in Proposition 4 ) and introducing an estimation bias of a∗ ( the second term in Proposition 4 ) .

Summary: The authors proposed an effective meta-regularization algorithm (MetaProx) for meta-learning. A learnable proximal regularizer with deep kernel is introduced to learn the base learner. The computation complexity of meta-gradient is linear in the number of meta-parameters. Experiments on synthetic dataset and mini-ImageNet show that the proposed method outperforms the popular few-shot learning methods.