Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: ### Paper summary
This paper addresses a problem of model-agnostic meta-learning (MAML) and most of its variations/extensions - learning only a single parameter initialization for the entire task distribution, which might not be effective when the task distribution is too diverse. Inspired by gradient boosting, which aims to train a new learner to predict the residuals of the previously predicted result for each step, this paper proposes a meta gradient boosting framework. Specifically, the proposed framework represents the meta-learned global initialization as a base learner, consisting of the first few weak learners, which is responsible for acquiring sharable transferable knowledge by learning across all tasks (or task modes). Then, many gradient-boosting modules learn to capture task-specific information to fit diverse distributions more effectively. This paper evaluates the proposed framework and the baselines, including MAML, Multimodal MAML, and LEO, on both the few-shot regression task and the few-shot image classification task. To create diverse task distributions, it combines different function families (linear, sinusoidal, etc.) for the regression and merges multiple datasets (Omniglot, miniImageNet, etc.) for the classification. The experiments show that the proposed meta-gradient boosting framework (with 5 gradient boosting modules) achieves better or competitive results compared to the baselines. Ablation studies justify some design choices of the proposed framework, including the architecture of the weak learner, the number of gradient-boosting modules, the updating strategy for the gradient boosting modules, the boosting rate, etc. I believe this work studies an important problem and proposes an interesting framework. Yet, I have a few concerns regarding experimental setup and results which prevent me from accepting this paper (see below for details).

### Paper strengths
**motivation**
The motivation for addressing the issue of model-agnostic meta-learning (MAML) - learning only a single parameter initialization for the entire task distribution, which might not be effective when the task distribution is too diverse, is convincing.

**novelty**
As far as I am concerned, the idea of utilizing the intuition of gradient boosting is novel and interesting. This paper presents a reasonable way to implement this idea.

**technical contribution & ablation study**
Ablation studies provide insights that help understand the proposed framework and justify many design choices. This includes the architecture of the weak learner, the number of gradient boosting modules, the updating strategy for the gradient boosting modules, the boosting rate, etc. 

**clarity**
The writing is very clear and the figures illustrate the ideas well. Also, the organization of the paper is easy to follow.

**experimental results**
- The description of the experimental setup is comprehensive. The presentation of the experimental results is clear. 
- The proposed meta-gradient boosting framework outperforms or at least performs competitively compared to the representative baselines.
- This paper studies a variety (i.e. different numbers of task modes) of settings, which provides great insights.

### Paper weaknesses
**baselines**
I believe this paper ignores many important baselines. As a result, the experimental conclusions are less convincing. I list some of the baselines and brief reasons why I believe they should be included as below:
- Hierarchically Structured Meta-Learning (HSML): is designed to perform soft clustering on tasks. It would be interesting to see if HSML can handle the task distributions considered in this paper. Also, it shows superior performance compared to MAML and a workshop version of Multimodal MAML. So, it is a stronger baseline to be included.
- Probalistic MAML / Bayesian MAML: both of these two methods consider , outperforming MAML. Intuitively, this probabilistic schema should deal with multimodal task distributions better since it inherently learns to handle uncertainty. It would be great to compare against at least one of them.
- Proto-MAML: presented in the meta-dataset paper (Triantafillou et al. in ICLR 2020), is designed to deal with multiple datasets combined and therefore should be able to perform well on the setup considered in this paper. Proto-MAML shows the strongest performance compared to MAML based methods and even outperforms some metric-based meta-learning methods. Showing the proposed framework can outperform or perform competitively compared to Proto-MAML would make this paper much stronger.
- Metric-based meta-learning methods: this paper does not include comparisons against any metric-based meta-learning methods such as matching networks, prototypical networks, relation networks, TADAM, etc. Yet, the state-of-the-art results on the few-shot image classification are mostly achieved by metric-based meta-learning methods. Therefore, I believe it would be essential to include representative metric-based meta-learning methods.

**MAML with a comparable number of parameters**
Since the proposed meta-gradient boosting framework has more parameters than the vanilla MAML, it is possible that the performance gain comes from the larger capacity. It would important to include a MAML baseline that has a comparable number of parameters to justify this. 

**RL experiments**
As far as I am concerned, the few-shot regression task is more a task for detailed analysis for research rather than a task with a wide range of applications, and the state-of-the-art results of the few-shot image classification task have been mostly achieved by metric-based meta-learning methods. Therefore, I am mainly interested in the model-agnostic meta-learning  line of work because of its potential in reinforcement learning where the ability to adapt to unseen scenarios is crucial. Yet, this paper does not include any experiments on RL without any reasons, which makes the paper less convincing to me.

**MGB-1 vs. MAML**
It seems that the proposed meta-gradient boosting model with one gradient-boosting module outperforms the vanilla MAML by a significant margin on the regression task yet only performs similarly to the vanilla MAML on the classification. Can the authors give some intuition about why this is the case?

**meta-dataset**
To evaluate if the proposed framework and the baselines can deal with diverse task distributions on the few-shot image classification task, this paper combines four different few-shot learning datasets to produce a 4-mode classification task. Yet, the meta-dataset has been proposed for this purpose. Also, the meta-dataset paper provides a comprehensive comparison of recent meta-learning methods. Therefore, I believe this paper should evaluate the proposed framework on the meta-dataset and see if it outperforms the baselines.

**stddev of the classification task**
It seems that the performance gap between the baselines and the proposed framework is insignificant on the image classification task. In this case, it would be important to also provide the standard deviation of each task.

=== After rebuttal ===

I am not satisfied with the response from the authors. I can only hardly recognize the effort made by the authors during the rebuttal - most of my points were only briefly discussed in the response without revising the paper. 

- The suggested baselines were merely briefly discussed but not added to the comparison.
- The results of the meta-dataset, which in my opinion is the most suitable dataset for the purpose, are still not included in the revised paper.
- The stddev of the classification task is not still not provided, making it hard to justify the performance gain.
- Why is RL left to future work?

I have read the reviews from other reviewers. With the little revision from the authors, I have decided to keep my original rating and would not recommend this paper to be accepted.
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposed a method to incorporate neural networks into gradient boosting for meta-learning with a limited number of samples in each task. 

The simulated experiments, where the generated tasks contain four continuous function, demonstrated that the proposed model outperformed MAML and its related variants, and on the classification task, it performed similarity to a previously proposed method MMAML.

In general, I am very interested in seeing neural networks being combined with boosting algorithms, but I do have quite a few questions.

1. The paper claimed that the learning and updating strategy proposed in the method ensured a weak base learner. I have to say that I am not satisfied by the claim. One can simply construct a linearly-separable dataset, which doesn't contribute to the overall tasks, and it only takes few updates for a two-layer neural network to converge, then the learner itself is not weak anymore. In addition, it adds noise to other tasks in the task set.

2. Normally, when a weak learner is learnt, a 1-D line search is conducted to find the optimal contribution of this particular learner. Although, shrinkage is usually applied in practice to anneal the contribution of new learners as a regularisation method to avoid overfitting. The paper seemed to have ignored the line-search part which can be done efficiently in 1D, whilst only the shrinkage was applied in the modelling, and I was wondering if there was a specific reason for that. 

3. In terms of the application of the proposed algorithm, I think it is rather limited. My understanding is that, although the proposed algorithm worked better than MAML and its variants on simulated data, the algorithm at the same time involves many hyperparameters, including (1) the design of the base feature extractor, (2) the design of gradient boost modules, (3) the boosting rates, and the annealing of them, (4) number of local update steps,  (5) number of global update steps, and many other hyperparams regarding to the training of neural networks. The first two hyperparameters or designs are critical in the success of the gradient boosting algorithm as weak learners are needed for learning so that the gradient is informative for the subsequent learners. Tuning those hyperparameters could be a time-consuming and also non-trivial task itself already, compared to the MAML algorithm itself, or generally multi-task learning approaches for meta learning, this algorithm doesn't seem to be outstanding. 

4. A fair comparison, IMO, could be to use the same feature extractor for producing vector representations of samples, and then directly apply gradient boosting with trees for meta learning. As mentioned in the paper, the authors also agreed that the feature extractors themselves have a huge impact on the final performance, therefore, it might be a good practice to check how well gradient boosting is able to handle meta-learning on top of extracted features.
review_title: An effective yet quite restricted approach

final_score: rating: 6: Marginally above acceptance threshold
review_content: This study is presented clearly, and the core idea is interesting. However, the presented novelty is limited to a globally (for all tasks) and locally (task-specific)  learning paradigm using a framework inspired by (Badirli et al., 2020). The authors have presented experimental results for both regression and classification setups, which are interesting.
In my opinion, the paper has relatively high quality and can be interesting for the ICLR community. 
One question regarding Figure 4 (a), you have mentioned that adding more weak learners causes difficulties in capturing multi-mode patterns, but from this figure, one can see that it is not completely true for the early epochs (before 400), comparing the two weak learners versus one weak learner case and it seems more like fluctuations, how would you explain it. 

review_title: This paper presents a gradient boosting framework for Meta-optimization, wherein an initialization condition can be generalized to different distributions related to different tasks. The main idea is learning a general base learner across all the distributions of different tasks, and later learning gradient boosted modules that are adjusted to each specific task. In this study, a deep neural net is used to build the ensemble of sub-networks.

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors propose a meta-gradient boosting framework that uses a base learner to learn shared information across tasks and gradient-boosted modules to capture task-specific information. The proposed approach is applied to various regression and classification tasks.

To me the motivation of using a gradient boosting framework remained unclear throughout the paper. It is stated in the last paragraph of the Introduction that ‘Recent research.. with achieving low training errors’. However, why this is important to the meta-learning problem remains unclear.


In Section 3, the model is not explained clearly:
-	What is a weak learner? In the introduction it is mentioned that ‘Gradient boosting aims to build a new learner towards…We call the learner for each step as weak learner.’ What is a step here?
-	‘..the first few learners are regarded as the base learner for learning the shared information across tasks..’ This was also not evident how it is designed to achieve this. Given that I have no prior on working with gradient boosting directly, I found the motivation and the method quite hard to follow.
-	
Section 3.1. Second sentence: ‘..where K is the number of adaptions’-what is adaptions?
What is the definition of a gradient boost?
How is $\theta_k$ defined in equation (1)?

Sec 3.2:
The first sentence- what does it mean?
It is mentioned here that a first-order MAML type approach is taken to learn the ‘base learner’. But how the two goal of using gradient boost modules to ensure task-specific  information is achieved is not clear.

Figure 2,3,  and 4: The axes and fonts are unreadable.

In Results, the authors state that ‘ the results show that incorporating task identities can significantly improve the performance of multi-mode learning’. It was not clear to me how this is true. First, how is task identity even abstracted or assigned here? What features of the approach are able to get this information is not clear.

The gain classification tasks also seems marginal compared to the other approaches. The key emphasis of the authors is on the importance of the approach on multi-mode distributions. It is not made clear why the approach would be suited to this setting. This is supported numerically only in the case of regression experiments.

review_title: Marginal contribution

Meta-review: The paper proposes a meta-gradient boosting framework to tackle the model-agnostic meta-learning problem. The idea is to use a base learn that learns shared information across tasks, and gradient boosted modules to capture task-specific modules. The experiments show that the proposed meta-gradient boosting framework (with 5 gradient boosting modules) achieves better or competitive results compared to the baselines. However, there were several issues that the author feedback did not addressed properly. For instance, R2 were not satisfied by discussing briefly the suggested baselines without adding the comparison, or R1 pointed out that the claim “the learning and updating strategy proposed in the method ensured a weak base learner” because clear separable datasets could convergence quickly and weak is not anymore applicable. Besides these two specific concerns, the reviewers expected a large revision of the paper due to several cons about the paper. All reviewers agreed a mayor revision is needed before acceptance. Therefore I recommend rejection.
Decision: Reject