Towards Enabling Meta-Learning from Target Models

1 Introduction . Meta-learning means improving performance measures over a family of tasks by their training experience [ 22 ] . It has been researched in various fields such as image classification [ 11 , 16 ] and reinforcement learning [ 6 , 14 ] . By reusing transferable meta-knowledge extracted from previous tasks , we can learn new tasks with a higher efficiency or a shortage of data . A typical meta-learning algorithm can be decomposed into two iterative phases . In the first phase , we train a solver of a task on its training set with assistance of meta-model . In the second phase , we optimize the solver ’ s performance to update meta-model . One key factor in this procedure is the way to evaluate the solver because the evaluation result acts as the supervision signal for meta-model . Early meta-learning algorithms [ 19 , 23 ] directly use the solver ’ s training loss as its performance metric , and optimize this metric over a distribution of tasks . Obviously , inner-task over-fitting may happen during the training of task-specific solvers , resulting in an inaccurate supervision signal for the meta-model . This drawback is even more amplified in applications where the training set of each task is limited such as few-shot learning and noisy learning . Intuitively , assessment of solvers should be independent of their training sets . This principle draws forth two important meta-learning algorithms in 2016 [ 24 , 28 ] , which respectively export solver evaluation from the perspective of “ data ” and “ model ” . In this paper , we call these two methodologies S/Q protocol and S/T protocol . In S/Q protocol , S means support set and Q means query set . They ∗De-Chuan Zhan is the corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . contain non-overlapping instances sampled from a same distribution . By training the solver on S and evaluating it on Q , we are able to obtain an approximate generalization error of the solver and eventually provide the meta-model with a reliable supervision signal . Another choice is to compare the trained solver with an ideal target model T . Assuming that T works well on a task , we can minimize the discrepancy between the trained solver and T to pull the solver closer to T . Here T can be Bayesian optimal solution to a task or a model trained on a sufficiently informative dataset . Figure 1 gives an illustration of both S/Q protocol and S/T protocol . Although appeared in the same year , S/Q protocol is more widely accepted by meta-learning society [ 4 , 8 , 13 , 10 ] while the research about how to leverage target models remains immature . The main reason is the simplicity of S/Q and the computational hardness of S/T . However , S/T protocol has some unique advantages . Firstly , it does not depend on possibly biased and noisy query sets . Secondly , by viewing support sets and their corresponding target models as ( feature , label ) samples , meta-learning is reduced to supervised learning and we can transfer insights from supervised learning to improve meta-learning [ 2 ] . Thirdly , we can treat the target model as a teacher and incorporate a teacher-student framework like knowledge distillation [ 7 ] and curriculum learning [ 1 ] in metalearning . Thus , it is necessary and meaningful to study S/T protocol in meta-learning . This paper looks into S/T protocol and takes a step towards enabling meta-learning from target models . We mainly answer two questions : ( 1 ) If we already have access to target models , how to learn from them ? What are the benefits of learning from them ? ( 2 ) In a real-world application , how to obtain target models efficiently and make S/T protocol computationally tractable ? For the first question , we propose to match the task-specific solver to the target model in output space . Learning from target models brings us more robust solvers . For the second question , we focus on a typical application scenario of meta-learning , i.e. , few-shot learning . We construct target models by fine-tuning the globally pre-trained network on those hard tasks to maintain efficiency . 2 Related Work . Meta-Learning . Meta-learning aims at extracting task-level experience ( so-called meta-knowledge ) from seen tasks , while generalizing the learned meta-knowledge to unseen tasks efficiently . Researchers have studied several kinds of meta-knowledge like model initialization [ 3 , 25 ] , embedding network [ 21 , 12 , 9 , 20 , 4 ] , external memory [ 19 , 5 ] , optimization strategy [ 15 , 18 ] , and data augmentation strategy [ 13 ] . Despite their diversity in meta-knowledge , most existing models are trained under S/Q protocol , and rely on a randomly sampled and possibly biased query set . Actually , most algorithms are protocol-agnostic , and both S/Q protocol and S/T protocol can be applied to them . Thus , our work on S/T is general , and it has a wide application field . Learning from Target Models . The idea of learning from target models in meta-learning is first proposed by [ 28 ] . In [ 28 ] , the authors constructed a model regression network that explicitly regresses between small-sample classifiers and target models in parameter space . Here both solvers and target models are limited to low-dimensional linear classifier , making it feasible to regress between them . From our perspective , matching two models ’ parameters is not practical when the dimension of parameters is too high . Thus , we match two models in output space in this paper . Similarly , there are other papers focusing on meta-learning from target models [ 27 , 31 ] . The most similar work to us is [ 31 ] , which constructs target models with abundant instances and matches task-specific solvers and target models . However , they all assume that every single task has a target model , increasing both space and time complexity of S/T protocol . To summarize , we claim that one key point in putting S/T protocol into practice is reducing the requirement for target models . In this paper , we focus on those hard tasks , and find that by learning from a small ratio of informative target models , classic meta-learning algorithms can be improved . 3 Preliminary . Meta-learning extracts high-level knowledge by a meta-model from meta-training tasks sampled from a task distribution p ( τ ) and reuses the learned meta-model on new tasks belonging to the same distribution . Each task τ has a task-specific support set S = { ( xi , yi ) } |S|i=1 , and we can train on S a solver g : X→ Y parameterized by γg . Without loss of generality , a meta-model can be defined as f : S→ G parameterized by θf that receives a support set as input and outputs a solver . Here S is the space of support sets and G is the space of solvers . In other words , f encodes the training process of g on S under the supervision of meta-knowledge θf . Taking two well-known meta-learning algorithms , MAML [ 3 ] and ProtoNet [ 21 ] , as examples , we have the following concrete forms of f : • MAML meta-learns a model initialization θf and fine-tunes it on each S with one gradient descent step to obtain a task-specific solver g. It can be written as Equ ( 1 ) . η is step size and ` : Y× Y→ R+ is some loss function . f ( S ; θf ) = g · ; θf − η ∇γ ∑ ( xi , yi ) ∈S ` ( g ( xi ; γ ) , yi ) ∣∣∣∣∣∣ γ=θf ( 1 ) • ProtoNet meta-learns an embedding function φθf parameterized by θf and generates a lazy solver which classifies an instance to the category of its nearest class center . Here g is implicitly parameterized by both θf and embedded support instances . f ( S ; θf ) = g ( · ; θf , { φθf ( xi ) | ( xi , yi ) ∈ S } ) ( 2 ) S/Q Protocol . How to evaluate the solver g trained on S ? The answer to this question differs conventional S/Q protocol [ 24 ] from S/T protocol . In S/Q protocol , we sample another query set Q = { xj , yj } |Q|j=1 apart from S for each task . Instances in S and Q are i.i.d . distributed and have a same label set , and we evaluate g by its loss on Q . Since S and Q contain non-overlapping instances , loss on Q is a more reliable supervision signal . S/Q protocol can be formulated as Equ ( 3 ) . Here Dtr is the meta-training set and we can sample meta-training tasks τ tr from it . min f ∑ τ tr= ( S tr , Qtr ) ∈Dtr ∑ ( xj , yj ) ∈Qtr ` ( f ( S tr ) ( xj ) , yj ) ( 3 ) S/T Protocol . Any sampled query set Q can be biased and noisy , which may cause an inaccurate evaluation of the solver . An alternative is directly matching the task-specific solver g = f ( S ) and a target model T that works well on the corresponding task . By computing the distance from the solver to target model , we obtain a more robust training signal to update meta-model . By replacing the solver evaluation part in Equ ( 3 ) , we have the following S/T protocol Equ ( 4 ) . Here L : G×G→ R+ is some loss function to measure the discrepancy between g and target model T . min f ∑ τ tr= ( S tr , T tr ) ∈Dtr L ( f ( S tr ) , T tr ) ( 4 ) 4 Effect of Target Model . We have introduced some basic concepts in meta-learning , and formulate S/Q protocol and S/T protocol in Section 3 . In this section , we assume that target models are available , and study how to utilize them to assist meta-learning . Firstly , we propose a model matching framework based on output comparison . Secondly , we verify the effectiveness of our proposal in a synthetic experiment . Moreover , we try to decrease the ratio of tasks that have target models , and show that it is possible to reduce the resource consumption of S/T protocol . 4.1 Model Matching In S/T protocol , one key point is how to match the solver g and its target model T . In other words , we need to specify the concrete formulation of L ( g , T ) . Generally , methods to match g to T can be classified into two categories . Firstly , we can directly match two models ’ parameters or use another model to regress between two models ’ parameters [ 28 ] . For example , let γg and γT be the parameters of g and T , we can set L ( g , T ) = ∑ ( xi , yi ) ∈S tr ` ( g ( xi ) , yi ) + λ‖γg − γT ‖22 . Here λ is a balancing hyperparameter . This method may work well for low-dimensional parameters , but is not suitable for complex models like deep neural networks . A better alternative is to match two models in their output space , i.e. , L ( g , T ) =∑ ( xi , yi ) ∈S tr [ ( 1− λ ) ` ( g ( xi ) , yi ) + λD ( T ( xi ) , g ( xi ) ) ] . Here D ( · , · ) is a function that measures the discrepancy between T ( xi ) and g ( xi ) . If we instantiate D ( · , · ) as KL divergence KL ( ·||· ) for classification problem , the aforementioned loss function is equivalent to that of knowledge distillation . Figure 2 is an illustration of approaches to matching a solver to a target model .

Summary: This paper investigated the S/T meta-learning framework which is not widely used due to the huge computational cost. The proposed method estimates the hardness of tasks and selects the target models of a small ratio of hard tasks. Then, the meta-model is learned with the S/Q meta-loss and the knowledge distillation loss between the selected target models and solvers. Experiments show that the proposed method outperforms the two classic meta-learning methods with the cost of constructing target models.