Model agnostic meta-learning on trees

1 INTRODUCTION . Deep learning models require a large amount of data in order to perform well when trained from scratch . When data is scarce for a given task , we can transfer the knowledge gained in a source task to quickly learn a target task , if the two tasks are related . The field of Multi-task learning studies how to learn multiple tasks simultaneously , with a single model , by taking advantage of task relationships ( Ruder ( 2017 ) , Zhang & Yang ( 2018 ) ) . However , in Multi-task learning models , a set of tasks is fixed in advance , and they do not generalize to new tasks . The field of of Meta-learning is inspired by the ability of humans to learn how to quickly learn new tasks , by using the knowledge of previously learned ones . Meta-learning has seen a widespread use in multiple domains , especially in recent years and after the advent of Deep Learning ( Hospedales et al . ( 2020 ) ) . However , there is still a lack of methods for sharing information across tasks in meta-learning models , and the goal of our work is to fill this gap . In particular , a successful model for meta-learning , MAML ( Finn et al . ( 2017 ) ) , does not diversify task relationships according to their similarity , and it is unclear how to modify it for that purpose . In this work , we contribute the following : • We propose a novel modification of MAML to account for a hierarchy of tasks . The algorithm uses the tree structure of data during adaptation , by pooling gradients across tasks at each adaptation step , and subsequent steps follow down the tree ( see Figure 1a ) . • We introduce new benchmarks for testing a hierarchy of tasks in meta-learning on a variety of synthetic non-linear ( sinusoidal ) and multidimensional linear regression tasks . • We compare our algorithm to MAML and a baseline model , where we train on all tasks but without any meta-learning algorithm applied . We show that the algorithm has a better performance with respect to both of these models in the sinusoidal regression task and the newly introduced synthetic task because it exploits the hierarchical structure of the data . 2 RELATED WORK . The problem of quantifying and exploiting task relationships has a long history in Multi-task learning , and is usually approached by parameter sharing , see Ruder ( 2017 ) , Zhang & Yang ( 2018 ) for reviews . However , Multi-task Learning is fundamentally different from Meta-learning as it does not consider the problem of generalizing to new tasks ( Hospedales et al . ( 2020 ) ) . Recent work includes Zamir et al . ( 2018 ) , who studies a large number of computer vision tasks and quantifies the transfer between all pairs of tasks . Achille et al . ( 2019 ) proposes a novel measure of task representation , by assigning an importance score to each model parameter in each task . The score is based on the gradients of each task ’ s loss function with respect to each model parameter . This work suggests that gradients can be used as a measure of task similarity , and we use this insight in our proposed algorithm . In the context of Meta-learning , a few papers have been published on the problem of learning and using task relationships in the past months . The model of Yao et al . ( 2019 ) applies hierarchical clustering to task representations learned by an autoencoder , and uses those clusters to adapt the parameters to each task . The model of Liu et al . ( 2019 ) maps the classes of each task into the edges of a graph , it meta-learns relationships between classes and how to allocate new classes by using a graph neural network with attention . However , these algorithms are not model-agnostic , they have a fixed backbone and loss function , and are thus difficult to apply to new problems . Instead , we design our algorithm as a simple generalization of Model-agnostic meta-learning ( MAML , Finn et al . ( 2017 ) ) , and it can be applied to any loss function and backbone . A couple of studies looked into modifying MAML to account for task similarities . The work of Jerfel et al . ( 2019 ) finds a different initial condition for each cluster of tasks , and applies the algorithm to the problem of continual learning . The work of Katoch et al . ( 2020 ) defines parameter updates for a task by aggregating gradients from other tasks according to their similarity . However , in contrast with our algorithm , both of these models are not hierarchical , tasks are clustered on one level only and can not be represented by a tree structure . As far as we know , ours is the first model-agnostic algorithm for meta-learning that can be applied to a tree structure of tasks . 3 THE META-LEARNING PROBLEM . We follow the notation of Hospedales et al . ( 2020 ) . We assume the existence of a distribution over tasks τ and , for each task , a distribution over data points D and a loss function L. The loss function of the meta-learning problem , Lmeta , is defined as an average across both distributions of tasks and data points : Lmeta ( ω ) = E τ E D|τ Lτ ( θτ ( ω ) ; D ) ( 1 ) The goal of meta-learning is to minimize the loss function with respect to a vector of metaparameters ω . The vector of parameters θ is task-specific and depends on the meta-parameters ω . Different meta-learning algorithms correspond to a different choice of θτ ( ω ) . We describe below the choice of TreeMAML , the algorithm proposed in this study . During meta-training , the loss is evaluated on a sample of m tasks , and a sample of nv validation data points for each task Lmeta ( ω ) = 1 mnv m∑ i=1 nv∑ j=1 Lτi ( θτi ( ω ) ; Dij ) ( 2 ) For each task i , the parameters θτi are learned by a set of nt training data points , distinct from the validation data . During meta-testing , a new ( target ) task is given and the parameters θ are learned by a set of nr target data points . In this work , we also use a batch of training data points to adapt θ at test time . No training data is used to compute the final performance of the model , which is computed on separate test data of the target task . 3.1 MAML . MAML aims at finding the optimal initial condition ω from which a good parameter set can be found , separately for each task , after K gradient steps ( Finn et al . ( 2017 ) ) . For task i , we define the single gradient step with learning rate α as Ui ( ω ) = ω − α nt nt∑ j=1 ∇L ( ω ; Dij ) ( 3 ) Then , MAML with K gradient steps corresponds to K iterations of this step ( here we assume that the same batch of training data points is used at each step , because these are task specific ) θτi ( ω ) = Ui ( Ui ( ... Ui ( ω ) ) ) ( K times ) ( 4 ) This update is usually referred to as inner loop , and is performed separately for each task , while optimization of the loss 2 is referred to as outer loop . 3.2 TREEMAML . We propose to modify MAML in order to account for a hierarchical structure of tasks . The idea is illustrated in Figure 1 . At each gradient step k , we assume that tasks are aggregated into Ck clusters , and the parameters for each task are updated according to the average gradient across tasks within the corresponding cluster ( in Fig.1b , we use K = 3 steps and C1 = 2 , C2 = 4 , C3 = 8 ) . We denote by Tc the set of tasks in cluster c. Then , the gradient update for the parameters of each task belonging to cluster c is equal to Uc ( ω ) = ω − α nt |Tc| ∑ i∈Tc nt∑ j=1 ∇L ( ω ; D ( i ) j ) ( 5 ) Furthermore , we denote by cki the cluster to which task i belongs at step k. Then , TreeMAML with k gradient steps corresponds to K iterations of this step θτi ( ω ) = UcKi ( UcK−1i ( ... Uc1i ( ω ) ) ) ( 6 ) The intuition is the following : if each task has scarce data , gradient updates for single tasks are noisy , and adding up gradients across similar tasks increases the signal . Note that we recover MAML if Ck is equal to the total number of tasks m at all steps . On the other hand , if Ck = 1 then the inner loop would take a step with a gradient averaged across all tasks . Because at one specific step the weight updates are equal for all tasks within a cluster , it is possible to define the steps of the inner loop update per cluster c instead of per task θτi . Given a cluster c and its parent cluster pc in the tree , the update at step k is given by θc , k = θpc , k−1 − α nt |Tc| ∑ i∈Tc nt∑ j=1 ∇L ( θpc , k−1 ; Dij ) ( 7 ) where θck is the parameter value for cluster c at step k. In terms of the notation used in expression 6 , we have the equivalence θτi ( ω ) = θci , K , which depends on the initial condition ω . The full procedure is described in Algorithm 1 We consider two versions of the algorithm , depending on how we obtain the tree structure ( similar to Srivastava & Salakhutdinov ( 2013 ) ) : • Fixed tree . The tree is fixed by the knowledge of the tree structure of tasks , when this structure is available . In that case , the values of Ck are determined by such tree . • Learned tree . The tree is unknown a priori , and is learned using a hierarchical clustering algorithm . In that case , the values of Ck are determined at each step as a result of the clustering algorithm . In the latter case , we cluster tasks based on the gradients of each task loss , consistent with recent work ( Achille et al . ( 2019 ) ) . After each step k at cluster ci , the clustering algorithm takes as input the gradient vectors of the children tasks i gik = 1 nt nt∑ j=1 ∇L ( θci , k ; Dij ) ( 8 ) and these gradients are further allocated into clusters according to their similarity . The clustering algorithm is described in subsection 3.3 . Similar to MAML , adaptation to a new task is performed by computing θ ( i ) ( ω ) on a batch of data of the target task . In order to exploit task relationships , we first reconstruct the tree structure by using a batch of training data and then we introduce the new task . Algorithm 1 TreeMAML Require : distribution over tasks p ( τ ) ; distribution over data for each task p ( D|τ ) ; Require : number of inner steps K ; number of training tasks m ; learning rates α , β ; Require : number of clusters Ck for each step k ; loss function Lτ ( ω , D ) for each task randomly initialize ω while not done do sample batch of i = 1 : m tasks { τi } ∼ p ( τ ) for all tasks i = 1 : m initialize a single cluster ci = 1 initialize θ1,0 = ω for steps k = 1 : K do for tasks i = 1 : m do sample batch of j = 1 : nv data points { Dij } ∼ p ( D|τi ) evaluate gradient gik = 1nt ∑nt j=1∇Lτi ( θci , k−1 ; Dij ) end for regroup tasks into Ck clusters Tc = { i : ci = c } according to similarity of { gik } and parent clusters { pc } update θc , k = θpc , k−1 − α|Tc| ∑ i∈Tc gik for all clusters c = 1 : Ck end for update ω ← ω − β 1mnv ∑m i=1 ∑nv j=1∇ωLτi ( θci , K ( ω ) ; Dij ) end while

Summary: The submission proposes a meta-learning algorithm attuned to the hierarchical structure of a dataset of tasks. Hierarchy is enforced in a set of synthetically-generated regression tasks via the data-sampling procedure, which is modified from the task-sampling procedure of [1] to include an additional source of randomness corresponding to which of a set of cluster components task parameters are generated from. The authors propose to adapt the model-agnostic meta-learning algorithm (MAML) of [1] to reflect this hierarchical structure by either observing (Section 4.1, FixedTree MAML) or inferring (Section 4.2, LearnedTree MAML) an assignment of tasks to clusters at each step of the inner loop (task-specific adaptation phase) of MAML; if tasks belong to the same cluster, the correspond task-parameters receive the same update at that step (in particular, the update direction is averaged). It is assumed that there are increasingly many clusters at each step, so that task-specific parameter updates are increasingly granular.