Model agnostic meta-learning on trees
Authors: Jezabel Garcia
Federica Freddi
Jamie McGowan
Tim Nieradzik
Da-shan Shiu
Ye Tian
Alberto Bernacchia
Venue:  (2021)
Keywords: Meta-learning
hierarchical data
clustering

Abstract: In meta-learning, the knowledge learned from previous tasks is transferred to new ones, but this transfer only works if tasks are related, and sharing information between unrelated tasks might hurt performance. A fruitful approach is to share gradients across similar tasks during training, and recent work suggests that the gradients themselves can be used as a measure of task similarity.
We study the case in which datasets associated to different tasks have a hierarchical, tree structure. While a few methods have been proposed for hierarchical meta-learning in the past, we propose the first algorithm that is model-agnostic, a simple extension of MAML. As in MAML, our algorithm adapts the model to each task with a few gradient steps, but the adaptation follows the tree structure: in each step, gradients are pooled across task clusters, and subsequent steps follow down the tree. We test the algorithm on linear and non-linear regression on synthetic data, and show that the algorithm significantly improves over MAML. Interestingly, the algorithm performs best when it does not know in advance the tree structure of the data.

Under review as a conference paper at ICLR 2021
MODEL AGNOSTIC META- ON TREES
Anonymous authors
Paper under double-blind review
ABSTRACT
In meta-learning, the knowledge learned from previous tasks is transferred to new ones, but this transfer only works if tasks are related, and sharing information between unrelated tasks might hurt performance. A fruitful approach is to share gradients across similar tasks during training, and recent work suggests that the gradients themselves can be used as measure of task similarity. We study the case in which datasets associated to different tasks have a hicrarchical, tree structure. While few methods have been proposed for hierarchical meta-learning in the past, we propose the first algorithm that is model-agnostic, a simple extension MAML. As in MAML, our algorithm adapts the model each task with few gradient steps, but the adaptation follows the tree structure: in each step, gradients are pooled across task clusters, and subsequent steps follow down the tree. We test the algorithm on linear and non-liness regression on synthetic data, and show that the algorithm significantly improves over MAML.. Interestingly, the algorithm performs best when it does not know in advance the tree structure of the data.
INTRODUCTION
Deep learning models require a large amount of data in order to perform well when trained from scratch. When data scarce for given task, we can transfer the knowledge gained in a source task to quickly learn target task, if the two tasks are related. The field of Multi-task learning studies how to learn multiple tasks simultancously, with a single model, by taking advantage of task relationships (Ruder (2017), Zhang & Yang (2018)). However, in Multi-task learning models, set of tasks is fixed in advance, and they do not generalize to new tasks. The field of of Meta-learning is inspired by the ability of humans to learn how to quickly learn new tasks, by using the knowledge of previously learned ones.
Meta-leamiig has the advent of Deep Learning (Hospedales et al.] (2020)). However, there is still a lack of methods for sharing information across tasks meta-learning models, and the goal of our work is fill this gap. In particular, a successful model for meta- meta-learning, MAML (Finn et al. (2017)), does not diversify task relationships according their similarity, and is unclear how to modify it for that purpose.
widespread use in multiple domains, especially in recent years and after
In this work, we contribute the following:
We propose a novel modification of MAML to account for hierarchy of tasks. The algo- rithm uses the tree structure of data during adaptation, by pooling gradients across tasks at each adaptation step, and subsequent steps follow down the tree (see Figure la).
We introduce new benchmarks for testing hierarchy of tasks in meta-learning on a variety of synthetic non- non-linear (sinusoidal) and multidimensional linear regression tasks.
We compare our algorithm MAML and baseline model, where we train on all tasks but without any meta-learning algorithm applied. We show that the algorithm has better performance with respect to both of these models in the sinusoidal regression task and the newly introduced synthetic task because it exploits the hierarchical structure of the data.
2 RELATED WORK
The problem of quantifying and exploiting task relationships has long history in Multi-task learn- ing, and is usually approached by parameter sharing, see Ruder (2017). Zhang & Yang (2088) for

---

Under review as a conference paper at ICLR 2021
reviews. However, Multi-task Learning is fundamentally different from Meta-learning as it does not consider the problem of generalizing to new tasks (Hospedales et al. (2020)) Recent work includes Zamir et al. (2018), who studies large number of computer vision tasks and quantifies the transfer between all pairs of tasks. Achille et al. (2019) proposes a novel measure of task representation, by assigning an importance score to each model parameter in each task. The score is based on the gradients of each task's 's loss function with respect to each model parameter. This work suggests that gradients can be used as measure of task similarity, and we use this insight in our proposed algorithm.
In the context of Meta-learning, few papers have been published on the problem of learning and using task relationships in the past months. The model of Yao et al. (2019) applies hierarchical clustering to task representations learned by an autoencoder, and uses those clusters to adapt the parameters to each task. The model Li et al. (2019) maps the classes of each task into the edges of graph, meta-learns relationships between classes and how to allocate new classes by using a graph neural network with attention. However, these algorithms are not model-agnostic, they have a fixed backbone and loss function, and are thus difficult to apply to new problems. Instead, we simple generalization of Model-agnostic meta-learning (MAML, Finn ct al. (017)), and it can be applied to any loss function and backbone.
design our algorithm
A couple of studies looked into modifying MAML to account for task similarities. The work Jerfel et al. (2019) finds a different initial condition for each cluster of tasks, and applies the algorithm to the problem of continual learning. The work of Katoch al. (2020) defines parameter updates for a task by aggregating gradients from other tasks according to their similarity. However, in contrast with our algorithm, both of these models are not hierarchical, tasks are clustered on one level only and cannot be represented by tree structure. As far as we know, ours is the first model-agnostic algorithm for meta- leaa-nen that can be applied to a tree structure of tasks.
THE META-LEARNIN PROBLEM
We follow the notation of Hospedales al. (020)). We assume the existence of distribution over tasks 7 and, for each task, a distribution over data points D and a loss function C. The loss function of the meta-learning problem, fmmtt, is defined as an average across both distributions of tasks and data points:
Cmeta (w) = E J C, (w); Djr
The goal of meta-learnin is to minimize the loss function with respect to vector of meta- parameters w. The vector of parameters 0 is task-specific and depends on the meta-parameters w. Different meta-learning algorithms correspond to different choice of (w).. We describe below the choice of TreeMAML, the algorithm proposed in this study.
During meta-training, the loss is evaluated sample of m tasks, and sample of n. validation data points for each task
Ameta (w)
Lr, (0,, Dis)
(2)
For each task i, the parameters 0, are learned by set of n, training data points, distinct from the validation data. During meta-testing, new (target) task is given and the parameters O are learned by a set of n, target data points. this work, we also use a batch of training data points to adapt 0 at test time. No training data is used to compute the final performance of the model, which is computed on separate test data of the target task.
3.1 MAML
MAML aims at finding the optimal initial condition w from which good parameter set can be found, separately for each task, after K gradient steps (Finn et al. (2017)). For task i, we define the single gradient step with learning rate a as
Tbc V (((w;D())
U,(Ww w
(3)
n4e
2

---

Under review as a conference paper at ICLR 2021
Then, MAML with K gradient steps corresponds to K iterations of this step (here assume that the same batch of training data points used at each step, because these are task specific)
(w) = (((Ui(..U((W)))
(K times)
(4)
This update is usually referred to as inner loop, and performed separately for each task, while optimization of the loss 2 is referred to outer loop.
3.2 TREEMAML
We propose modify MAMI. in order account for hierarchical structure of tasks. The idea is illustrated in Figure
01
(a)
(b)
8
meta-learning Beaning/adaptation
V
La
V
0g
61
Illustration of the MAML(a), and TreeMAML(b) algorithms. Both algorithms are de- signed to quickly adapt to new tasks with small number of training samples. MAML achieves this by introducing gradient step in the direction of the single task. TreeMAML follows similar approach but exploiting the relationship between tasks by introducing a hierarchical aggregation of the gradients.
Figure 1:
At each gradient step k, we assume that tasks are aggregated into Cş clusters, and the parameters for each task updated according the average gradient across tasks within the corresponding cluster (in lbg we use K 3 steps and C₁ = 2, C2 = 4, C3 8). We denote by Te the set of tasks in cluster Then, the gradient update for the parameters of each task belonging cluster is equal to
Uw(W) = n4
VL(W: JE
(5)
T
Furthermore, we denote by et the cluster to which task i belongs at step h. Then, TreeMAML with gradient steps corresponds K iterations of this step
0.. (w) (.. (w))
The intuition is the following: if each task has scarce data, gradient updates for single tasks noisy, and adding gradients across similar tasks increases the signal. Note that we recover MAML if Cs is equal the total number of tasks m all steps. On the other hand, if C₁. 1 then the inner loop would take step with gradient averaged across all tasks.
Because at one specific step the weight updates are equal for all tasks within cluster, it is possible to define the steps of the inner loop update per cluster instead of per task 0,,- Given a cluster and its parent cluster P. the tree, the update at step & given by
X2
0.,k PP.
VL( k(10p..k-1:2. k-1; 1-;;
n4 Ta

---

Under review as a conference paper at ICLR 2021
where og is the parameter value for cluster at step k. In terms of the notation used in expression
6 we have the equivalence θ,, (w) Oc, which depends on the initial condition w. The full
procedure is described in Algorithm I
We consider two versions of the algorithm, depending on how we obtain the tree structure (similar to Srivastava & Salakhutdinov (2013)):
Fixed free. The tree is fixed by the knowledge of the tree structure of tasks, when this structure is available. In that case, the values of Cj are determined by such tree.
Learned tree. The tree is unknown a priori, and is learned using a hierarchical clustering algorithm. In that case, the values of Cs are determined at each step as result of the clustering algorithm.
In the latter case, we cluster tasks based on the gradients of each task loss, consistent with recent work (Achille et al. (2019)). After each step ht at cluster a, the clustering algorithm takes as input the gradient vectors of the children tasks i
Sih
The
and these gradients are further allocated into clusters according to their similarity. The clustering algorithm is described in subsection
Similar to MAML, adaptation the target task. In order to exploit task relationships, we first reconstruct the tree structure by using batch of training data and then we introduce the new task.
task is performed by computing 0 batch of data of
Algorithm TreeMAML
Require: distribution over tasks P(T); distribution over data for each task P(D]T);
Require: number of inner steps K; number of training tasks m; learning rates a,
Require: number of clusters C2 for each step k; loss function (w D) for each task randomly initialize w while not done do
sample batch o = 1: m tasks {T.) ~ p(T)
for all tasks = 1 initialize single cluster c; 1
initialize 01.0 = w
for steps k 1 K do
for tasks i = : m do sample batch of j 1: n. data points (Dij} ~ pl p(D(T))
evaluate gradient giA- ni Σ f VL,, k. Dij) end for
regroup tasks into Cg clusters Te (i: =
according to similarity of { and parent clusters {pe}
update 0c.k- = 0p [7.1 ΣiET, gik for all clusters = 1: Cs
end for update - w 3 mnn Em Σ Vwh. (ww) Dzy) end while
3.3 CLUSTERING ALGORITHM
In the learned tree case we employ hierarchical clustering algorithm to cluster the gradients of our model parameters. We specifically opt for an online clustering algorithm to maximise computational efficiency at test time and scalability. When new task is evaluated, we reuse the tree structure that was generated for a training batch and add the new task. This saves us from computing a new task hierarchy from scratch for every new task. Moreover, with offline hierarchical clustering all the data needs to be available to the clustering algorithm at the same time, which becomes a problem when dealing with larger batch sizes. Therefore online clustering favours scalability.
4

---

Under review as a conference paper at ICLR 2021
We follow the online top down (OTD) approach set out by Menon et al. (2019) and adapt this to approximate to non-binary tree structures. Our clustering algorithm is shown in Algorithm Specifically, we introduce two modifications to the original OTD algorithm:
Maximum Tree Depth Parameter D: This is equivalent to the number of inner steps to take in the TreeMAMI,, since the tree is representation of the inner loop where each layer in the tree represents single inner step.
Non-binary Tree Approximation: We introduce hyperparameter & which represents how far the similarity of a new task needs be to the average cluster similarity in order to be considered child of that same cluster. This is not an absolute value of distance, but it is a multiplicative factor the standard deviation of the intracluster similarities. Introducing this factor allows clusters at any level to have number of children greater than two.
Algorithm 2 Online top down (OTD) Non- Non-binary
Require: origin cluster node C with a given set of children = {#1, T2, .. task maximum depth allowed D; similarity metric, w()
Require:
Require:
standard deviation multiplicative hyperparameter {;
if | then
new task becomes new child A = {x}
else A then
add new task to set of children A + AU
else if w( J {a}} > (AA) then identify most similar child r. arg min, (((,,,, if reached maximum depth Cdephh +
then
add new task to set of children A + AU {z}
else
recursively perform OTD to create new node C" = OTD xT), add new node to set of children + (A U
end if
else (A U {x}) < (AA fØr then current node and new task become children to new cluster A [C,x)
else
add new task of children A (x} end if
EXPERIMENTS
4.1 FIXED TREEMAML
In this section report experiments where assumed knowledge about the structure of the un- derlying tasks and used this specifically to aggregate the gradients.
Experiment 1: Sinusoidal Regression Tasks We start with a modification of the regression problem used in the Finn et al. (2017) paper, the regression of sine wave, where the amplitude and phase of the sinusoid are varied between tasks. We introduce single modification to the experiment, in the original dataset the amplitude varies within [0.1, 5.0] and the phase varies within [0. л.. but in our experiments we have selected subsets these values to simulate structured data, as shown in Figure 2 To increase the difficulty of the task, we added different levels of noise to the datasets.
As in the original experiment, the data points for each task are sampled uniformly between [-5.0, 5.0]. During training and testing, the loss used is the mean-squared error. The model is neural network with hidden layers of size 40 with ReLU nonlincarities. We assume relationship between tasks as the one represented in Figure[(c) where the total depth of the tree is 2 and leaves in total. These leaves represent the finer level clusters and each task is fed to the TreeMAML algorithm with an assigned label corresponding one of these.
We evaluate performance by fine-tuning the model learned by MAML, TreeMAML and baseline on K 53,5, 100) data points. We also evaluate for the case where K = data points provided for
5

---

Under review as a conference paper at ICLR 2021
0
85
9p) 0(a
9go 9c0)
(a)
(b)
(c)
Figure 2: We show the tasks distribution over amplitude and phase of the original dataset used in Finn et al. (2017) (a) and in our modified dataset (b). The change tasks distribution was introduced to simulate structured data. (c) Shows the equivalent fixed tree structure that was used to represent the data.
training, but a single data point was provided at meta-testing, see point marked as star in Figure
Noise level 0.1
Noise level 0.5
MAML
MAML
TreeMAML Baseline
TreeMAML
Baseline
25
0.6
10
02
00
10'
Number
Figure 3: Results of sinusoidal regression task for Fixed TreeMAML, MAML and baseline. The plots show the loss (MSE) at K shot regression for different numbers meta-testing samples. (a) shows the results for noise level 0.1 and (b) shows results for noise level 0.5. Fixed TreeMAMI. always performs performs better than or as good as MAML and the baseline, especially in the case of small number of data points being available.
Experiment 2a: Multidimensional Linear Regression Task Here we consider multidimensional
linear regression problem y = Σen Pas; + where the tasks are randomly sampled from a set of defined clusters of multidimensional parameters P. 17 is randomly Even in this case, the parameter clusters are arranged hierarchically such that C₁ 2, C2 4. The data points for the tasks are sampled uniformly æ; U[-5.0, 5.0] for all training and testing tasks where. The models are then trained and tested on a set of tasks with K = 4, 8, 16, 32, 64 and 128 data points.
generated Gaussian noise.
4.2 LEARNED TREEMAML
In this section report experiments where assume prior knowledge of the underlying struc- ture of the data. Therefore the hierarchy of the data is learnt per-batch using the modified OTD algorithm described in section 3.3
Experiment 2b: Learned Tree with Multidimensionall Linear Regression Tasks For the multidi- mensional linear regression task, set the maximum depth of the clustering algorithm to 2 and use the cosine similarity metric. This is equivalent to 3 inner steps, beacuse there one last step that task-specifie; therefore for these experiments MAML is also set perform inner steps.

---

Under review as a conference paper at ICLR 2021
Noise level 0.1
Noise level 0.5
MAML TreeMAML boseline
0.08
MAMIL
0.2
TreeMAML
baseline
0.07
010
0.06
0.08
0.05
0.06
0.04
10a
10a
Number of data points
Number of data points
(a)
(b)
Results of the multidimensional (N=64) linear regression task for Fixed TreeMAML, MAML and baseline for varying number of task data points K = 4, 8, 16, 32, 64 and 128. Similarly to the sinusoidal regression task, TreeMAML performs better than MAML, especially when the number tasks data points K is low.
Figure 4:
Model
K=5
k=10
k=20
MAML
1.025 = 0.068
0.950 = 0.048
0.785 + 0.028
Baseline
1.293 + 0.074
1.055 0.047
1.139 + 0.061
Fixed TreeMAML (ours)
0.621 + 0.038
0.535 + 0.024
0.483 1 0.016
Learned TrecMAML (ours)
0.758 + 0.047
0.510 + 0.024
0.495 + 0.018
Loss (MSE) 195% confidence intervals on multidimensional linear regression task, aver- aged over 400 meta-testing tasks. The results are presented for varying numbers of K data points and noise level of 0.01
Table 1:
In Table I we show that TreeMAML outperforms the Baseline and MAML across all numbers of data points. Learned TreeMAML performs relatively better for larger number of data points: this is expected because, as the number of data points increases, the gradients used to cluster the tasks will be less affected by the noise and become more accurate, leading also to better clustering.
Experiment 3: Mixed Synthetic Regression Tasks Here we follow similar setup as described in Yao et al.] (2019), however we choose different parameters for our tasks in order to have better defined clusters of tasks. Again, we sample data points for the regression tasks as
U|-5.0, 5.0]. We define total 6 clusters as follow: (1) Linear Positive Slopes: y = a1 + bL4, ai+ U[1.0, 2.0 and bi+ U(0, 1.0]; (2) Quadratic Positive Slopes: y ag+ 2 + by+ Cq+i Ag+ U10.1,0.2 0.2,, bp+ ~ U[10, 2.0] and Cg+ ~ U2200 3.0]; (3) Linear- Negative Slopes: y = a x
a-- ~ (1-200-1.0)) and by
U[0 0): (4) Quadratic Negative Slopes: y ag ag_x² +
b + cg., ag- 0.2, 0.1], b - -100 and cy U|-3.0, -2.0] (5) Cubic: = ax box² + c. + de. ae ~ 0[-0.1, be ~ U[2.22.0]
~ U1-2.0, -1.0] and
de 3.0 (6) Sinusoidals: y = a, sin(2) + by a, ~ U/4.0, 5.0) and b, U/2.0, π). To each
of these we add noise variable sampled for each data point sampled from ~ U|-0.01,0.01 logically arranged this set of tasks in tree structure as the one shown in 5 we used this as reference for the Fixed TreeMAML experiments. We train all tasks with K=10 data points. The model used is a neural network with 2 hidden layers of size 40 with ReLU nonlinearities.
We
The results stated here are averaged over 600 test tasks (100 from each cluster). For these experi- ments we set the number of MAML inner steps to 4 and the depth of the Learnt TreeMAML to 3 in order to match the number of steps in the hierarchy in Figure [ The results for the models shown in Table 2 for a number of epochs E = {1, 5, 10). The results confirm that TreeMAML has clear advantage the MAMI. and baseline algorithms.
7

---

Under review as a conference paper at ICLR 2021
0
02
03
0%
8(g))
0:a
((4)
((o)
9(6)
Figure 5: This tree structure shows the task relationship that was logically designed and used for the Fixed TreeMAMI.
Model
epoch
5 epochs
10 epochs
MAML
4.135 = 1.148
2.340 = 0.632
1.804 + 0.290
Baseline
20.745 1.416
20.519 = 1.406
20.130 = 1.390
Fixed TreL (ours)
1.955 + 0.279
1.427 + 0.219
1.397 + 0.222
Learned TAML (ours)
1.637 + 0.255
1.338 + 0.225
1.366 + 0.240
Table 2: Performance (MSE loss) 195% confidence intervals on mixed regression task, averaged over 600 tasks. N-epoch results tasks with K=10 data points and 0.01 noise level.
DISCUSSION
We proposed a simple modification of MAML to address the problem of meta-learning hierarchical task distributions. The proposed algorithm is based the intuitive notion that learning tasks by gradient descent may benefit from gradient sharing, across similar tasks. Inspired by recent work Achille et al. (2019), we use the insight that similarity of tasks can be measured by the similarity of gradient themselves, thus reducing the problem of task transfer to gradient clustering.
We show that the new algorithm, which we term TreeMAML, performs better than MAML when the task structure is hierarchical. However, our tests are so far limited to synthetic data, and future work will have to validate our approach to more realistic settings. For example, some computer vision datasets have hierarchical structure (Deng et al. (2009)) and thus may represent a good test bed for our algorithm.
We presented very basic instance of our algorithm, thaht can be improved in several ways. For example, not all parameters need to be adapted, and recent work suggests that removing the inner loop for subset of parameters increases performance, especially when data is scarce (Zintgral et al. (019)] Raghu et al. (2020)). Another possible modification the algorithm to have different number of clusters for different subset of parameters. In general, given that algorithm is relatively simple modification of MAML, several tricks to improve training of the latter could be used for our algorithm well.
REFERENCES
Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Char- less Fowlkes, Stefano Soatto, and Pietro Perona. Task2Vec: Task Embedding for Meta- Learning. arXiv: 1902.03545 /cs, stat), February 2019. URL http:://axxi..org//////// arXiv: 1902.03545.
1902. 03545
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. CVPR, pp. 8, 2009.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adap- tation of Deep Networks. orXiv:1703 1703.03400 [cs], March 2017. URL attp: //tp///aavvvv//////////ea//awwiv www.iiioor org/ arXiv: 1703.03400.
03400
8

---

Under review as a conference paper at ICLR 2021
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey, Meta-Learning in Neural Networks: Survey. arXiv:2004.05433 [cs, stat], April 2020. URL http::/axxxv.orrt
abs/2004. 05439
arXiv: 2004.05439.
Ghassen Jerfel, Thomas L Griffiths, Erin Grant, and Katherine Heller. Reconciling meta-learning and continual learning with online mixtures of tasks. NIPS, pp. 12, 2019.
Sameeksha Katoch, Kowshik Thopalli, Jayaraman J. Thiagarajan, Pavan Turaga, and Andreas
Spanias. Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured
Meta Learning. arXiv: arXiv:1911. 19111.0010000 [cs). February 2020. URL http://arxiv. wtt.xxii.orggab////
1911.10600
arXiv: 1911.10600.
Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. Learning to Propagate for Graph Meta-Leamning arXiv:1909.0502 [es, stat), November 2019. URL http::/arxiv.
org/abs/1905 05024
arXiv: 1909.05024.
Aditya Krishna Menon, Anand Rajagopalan, Baris Sumengen, Gui Citovsky, Qin Cao, and Sanjiv
Kumar. Online hierarchical clustering approximations. arXiv preprint arXiv: 1909.09677, 2019.
Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML. arXiv:1909. 091577 [cs, stat], Febru- ary 2020. UKL//ttpe//aarxx..o/////////9 09157
arXiv: 1909.09157.
Sebastian Ruder. An Overview of Multi-Tas Learning in Deep Neural Networks. arXiv: 1706600069099 [cs, stat], June 2017. htRt.hhtt:////xii...t
abs 706. 05098
arXiv: 1706.05098.
Nitish Srivastava and Russ Salakhutdinov. Discriminative Transfer Learning with Tree- based Priors. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinherger (eds.), Advances in Neural Information Processing Systems 26, pp. 2094 2102. Curran Associates, Inc., 2013. URL. http::/ nips. cc/paper/ 5029 discriminative-trane transfer learning- weeteninggwitt--ae pdf
priors..
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically Structured Meta-learning.
1905:09905005300 [cs, stat), November 2019. URL http:://aaxlvv.rrg/bbbb
05301
arXiv: 1905.05301.
Amir Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling Task Transfer Learning. NrXXiv:/00..00 [cs], April 2018. URI.
httpp://axxiv.oot ort.abs//rgg////o.o 804 08328
arXiv: 1804.08328.
Yu Zhang and Qiang Yang. Survey on Multi-Tart Learning. 170.00170..8814 [cs], July 2018. arXiv: 1707.08114.
URL httth:////aaa
org/ aww./l07.... 08114
Luisa M. Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast [cs, stat], June 2019. URL htth:t
Context Adaptation via Meta-Learning. arXiv:
arXiv: 18 1810.03642.


Reviews:
[{"review_id": "v5U9qJMdjT", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The submission proposes a meta-learning algorithm attuned to the hierarchical structure of a dataset of tasks. Hierarchy is enforced in a set of synthetically-generated regression tasks via the data-sampling procedure, which is modified from the task-sampling procedure of [1] to include an additional source of randomness corresponding to which of a set of cluster components task parameters are generated from. The authors propose to adapt the model-agnostic meta-learning algorithm (MAML) of [1] to reflect this hierarchical structure by either observing (Section 4.1, FixedTree MAML) or inferring (Section 4.2, LearnedTree MAML) an assignment of tasks to clusters at each step of the inner loop (task-specific adaptation phase) of MAML; if tasks belong to the same cluster, the correspond task-parameters receive the same update at that step (in particular, the update direction is averaged). It is assumed that there are increasingly many clusters at each step, so that task-specific parameter updates are increasingly granular.\n\n##### Strengths:\n1) **Clarity**: The experimental setting and exactly how the data-generating process relates to the proposed algorithms are clearly described.\n2) **Significance**: Results on the hierarchically structured synthetic regression task datasets demonstrate that {Fixed|Learned}Tree MAML: is at least as good as MAML, and often outperforms MAML; that it learns more efficiently than a MAML in terms of the cumulative number of datapoints observed; and that both MAML and {Fixed|Learned}Tree MAML outperform a naive baseline.\n\n##### Weaknesses:\n1) **Significance**: Since the evidence provided in favor of the proposed algorithm is in the form of an empirical evaluation on a synthetically generated dataset, the present impact of the algorithm is limited. In particular, there is no evidence that (i) the algorithm works for larger and/or more complex datasets; and (ii) that natural datasets of interest to the community exhibit a hierarchical structure analogous to the synthetic datasets presented in the submission.\n2) **Novelty**: The algorithm modifies and combines previously introduced components: the MAML algorithm of [1]; the online top-down clustering algorithm of [2], and the task-similarity-as-gradient-similarity approach of [3].\n3) **Clarity**: Specific details surrounding the relationship between Algorithm  1 and Algorithm 2 are insufficiently discussed: \n  i) Algorithm 2 as it appears in the text is very similar to Algorithm 1 (The OTD algorithm) in [2] with the exception of the new hyperparameter $\\xi$, and introduces new symbols that do not appear elsewhere in the text. It is therefore not sufficiently adapted for clarity in the context of this work.\n  ii) Whether Algorithm 2 acts as a strict subroutine of Algorithm 2 is not stated. I believe it is not because the clustering decision for a new task relies on tree structures that are \"generated for a training batch,\" although what a \"training batch\" refers to is not clear. Similarly, how the \"online\"/\"offline\" distinction in the context of the clustering algorithm fits into the training/evaluating setup borrowed from [1] is not made clear.\n  iii) How exactly the task-similarity approach of [3] is employed in Algorithm 2 is not made clear. The only mention of the use of [3] is briefly around Eq. (8) before the main algorithm (Algorithm 1) is introduced, and Algorithm 2 only refers to a generic \"similarity metric\" (as in the original work, [1]).\n\n##### References\n\n[1] [Finn, Chelsea, Pieter Abbeel, and Sergey Levine. \"Model-agnostic meta-learning for fast adaptation of deep networks.\" arXiv preprint arXiv:1703.03400 (2017).](https://arxiv.org/abs/1703.03400)\n\n[2] [Menon, Aditya Krishna, Anand Rajagopalan, Baris Sumengen, Gui Citovsky, Qin Cao, and Sanjiv Kumar. \"Online Hierarchical Clustering Approximations.\" arXiv preprint arXiv:1909.09667 (2019).](https://arxiv.org/abs/1909.09667)\n\n[3] [Achille, Alessandro, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C. Fowlkes, Stefano Soatto, and Pietro Perona. \"Task2vec: Task embedding for meta-learning.\" In Proceedings of the IEEE International Conference on Computer Vision, pp. 6430-6439. 2019.](https://arxiv.org/abs/1902.03545)", "extra_scores": {}, "extra_text": {}}, {"review_id": "_gpSY4lXeks", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\nThe paper studies a simple modification of MAML to address the problem of meta-learning hierarchical\ntask distributions. It is based on the assumption that learning tasks by gradient descent may benefit from gradient sharing, across similar tasks. The authors convert the problem of task transfer to gradient clustering by considering the point that the similarity of tasks can be measured by the similarity of gradients.\n\n##########################################################################\n\nReasons for score: \n\n \nOverall, I vote for rejecting this paper. Unfortunately, I found the novelty of the paper very low. I believe that this paper is like an incremental study of exist work like MAML[1] and HSML[2].\n[1]https://arxiv.org/abs/1703.03400\n[2]http://proceedings.mlr.press/v97/yao19b/yao19b.pdf\n\n \n##########################################################################\nPros: \n- The paper is clear and cites the most relevant research studies and papers.\n- The experimental results presents the promising performance of the method.\n- It is a strength point of the paper that the authors address both foxed tree and  learned tree structures.\n\n##########################################################################\n\nCons: \n- The novelty of the paper is very low. The authors refer to [2] in the section 2 of the paper and mention that it is not task agnostic.\na) I believe HSML[2] is also built on MAML and not sure about this claim.\n- Lack of extensive experiments:\na) The paper does not include any experiment or discussion comparing the proposed method with HSML[2] or other exist method. I would suggest the authors to consider running more experiments.\nb) The time complexity of the method has not been discussed.\n \n##########################################################################\n\nQuestions during rebuttal period: \n\n \nPlease address and clarify the cons above \n\n \n#########################################################################\n\n\n\n\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "R31GImAdF2N", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "#### Summary\n- In the context of gradient-based meta-learning for few-shot learning, the authors propose TreeMAML, an algorithm that leverages the existence of a tree structure in a task distribution in order to pool inner-loop gradients between tasks. More specifically, the inner-loop corresponds to a level-by-level traversal from root to leaves, where at each step, task-leaves that are children to a common node in the current level average their gradients for this step's update. \n- The authors consider two cases: one in which the ground-truth tree structure is known (Fixed TreeMAML) and one in which it is unknown and must be discovered online (Learned TreeMAML). The authors extend a hierarchical clustering algorithm from prior work for this purpose. \n- The authors evaluate TreeMAML in three scenarios: sinusoid regression (Finn et al. 2017), linear regression, and mixed regression (Yao et al. 2019). TreeMAML compares favorably against MAML and a naive multi-task learning baseline. \n- Interestingly, Learned TreeMAML also consistently outperforms Fixed TreeMAML.\n\n\n#### Strengths\n- The authors investigate an important and under-considered problem in meta-learning: how to leverage structure within a task distribution. \n\n- The proposed TreeMAML algorithm is conceptually simple.\n\n#### Weaknesses\n- The motivation for the TreeMAML algorithm is very weak. Yes, averaging gradients across tasks might decrease variance, but presumably always increases bias. This crucial trade-off is not even mentioned.\n\n- The fact that Learned outperforms Fixed is troubling. It suggests that TreeMAML is not properly leveraging the ground-truth task hierarchy. Dissecting Learned to look at the tree structure it proposes would help diagnose this issue.\n\n- This is a purely empirical paper which only presents results in toy regression settings. More comprehensive empirical evaluation is needed, e.g. the image classification benchmark proposed in Yao et al. (2019), which is perfectly suitable for this paper (and indeed the authors took Experiment 3 from this work). \n\n- The comparison between TreeMAML and MAML might not be very fair. MAML is artificially constrained to use the same number of inner-loop steps as the depth of the task tree. Since MAML makes no assumptions about the task tree, this should be a tunable hyperparameter.\n\n#### Recommendation\n- I currently recommend a clear reject (3). Given the weaknesses outlined above, this submission does not meet the bar for acceptance.\n\n#### Questions\n- How is the Baseline model trained? Its MSE in Table 2 is uncommonly high.\n\n#### Minor suggestions\n- Please fix the numerous typos throughout the submission. Just in the first paragraph: inappropriate capitalization of multi-task and meta-learning; \"The field of of\".\n- K is used to denote the number of inner gradient steps in Alg. 1, but the number of datapoints per task in Sec. 4.1.\n- The num_shots=3 case in Fig. 3b directly contradicts the caption.", "extra_scores": {}, "extra_text": {}}, {"review_id": "9z_ERwfOt6", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "What is this paper about, what contributions does it make, what are the main strengths and weaknesses?\nThis paper proposed a learning algorithm of meta-learning: TreeMAML, to share information across tasks in meta-learning models. The paper compared the results of TreeMAML with MAML and the Baseline on SinusRegression Task and Linear Regression Task.\n\nThe main concerns are:\n\n1.\tSome statements in the introduction session are improper. \n\n1.1\t“there is still a lack of methods for sharing information across tasks in meta-learning models, and the goal of our work is to fill this gap.” See references:\na.\tInvenio: Discovering hidden relationships between tasks/domains using structured meta learning. 2019. \nb.\tHierarchical meta learning. 2019. \nc.\tHierarchically structured meta-learning. 2019. \nd.\tLearning to propagate for graph meta-learning. 2019. \n\n1.2\t\"However, these algorithms are not model-agnostic.\" Most of the related models are model-agnostic as long as replacing the task representation module. \n\n\n\n2.\tThe contribution of this work is supposed to be emphasized. The innovation of this paper seems insufficient. \n\n2.1\tYao et al. (2019). also aims to utilize the task relation and apply gradient-based adaptation methods. And the model structure presented in figure 1 is similar to the model framework in Yao’s paper.  \n\n2.2\tAs claimed in this paper, the hierarchical clustering algorithm, which is an essential part of the proposed model, is introduced with minor modification from Menon et al. (2019). \n\n3.\tAs claimed in this paper, the proposed model lacks scalability since it works well only in the tree-structured tasks. \n\n4.\tThe baselines exclude many relevant works such as the papers listed above.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a variant of MAML for meta-learning on tasks with a hierarchical tree structure. The proposed algorithm is evaluated on synthetic datasets, and it compares favorably to MAML. The reviewers identified several significant weaknesses, including: (1) the experimental evaluation is limited, and it only includes small synthetic datasets; (2) the proposed algorithm is incremental over MAML. The reviewers agreed that the paper cannot be accepted in its current form. I recommend reject.
Decision: Reject