Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a theoretical formulation for meta-learning that uses task similarity based on task gradients, which helps learning in the presence of outlier tasks. The inner loop parameter update is given by linear kernel regression, where the kernel function computes similarity between gradients of different tasks. While the paper includes experiments that outperform MAML and Meta-SGD on estimating randomized linear predictors, and randomized sinusoids with outlier data-points, these are not sufficient to establish the efficacy of the approach. 

Pros : 
1. This paper proposes a solution to the problem of meta-learning with dissimilar tasks, which is a central problem in meta-learning. The formulated approach is a generalization of MAML and Meta-SGD, as the update direction isn't necessarily the gradient direction, and there is an additional regulation term on meta-parameters in the outer loop. Tasks with similar gradients have a similar update in the meta-learning inner loop. 

2. The formulation involving linear kernel regression which enables including the task similarity in the kernel function seems novel and could provide the basis for subsequent work in the field for dealing with dissimilar tasks. The usage of similarity between task gradients to guide updates is similar in spirit to gradient projection techniques used in continual learning to avoid catastrophic forgetting. 

3. The included experiments do show much superior performance to MAML and Meta-SGD on the datasets considered, which included tests with outlier data points for sinusoid regression.

Cons :
1. While the experiments show some promise for the method, these on simplistic datasets involving synthetic datasets for estimating randomized linear and sinusoid predictors. Given that the paper discusses MAML and Meta-SGD in some detail for setting up the new method, experiments on the Omniglot and MiniImagenet datasets considered in both those papers would help to better evaluate the proposed approach. 

2. The paper has no ablations or analysis for particular parts of their method, such as removing the gradient from the kernel function or removing the regularization term from the outer loop. Thus even on the simplistic datasets considered, it is hard to judge which aspects of the method make it work better. 

I am willing to increase my score if the authors include experiments on the datasets mentioned above, and include additional analysis/ablations of their method. 

review_title: Interesting theoretical formulation, but insufficient experimental analysis 

final_score: rating: 4: Ok but not good enough - rejection
review_content: === Summary ===

The paper proposes a meta-learning method based on a notion of task similarity/dissimilarity. In particular, the paper motivates its proposed method TANML through a generalization of Meta-SGD wherein the learnable parameter-wise learning rate in the inner update of Meta-SGD is replaced by a quadratic pre-conditioner matrix.

The proposed method TANML closely resembles gradient-based meta-learners in the outer update but replaces the inner update by the matrix-vector product of kernel regression coefficient matrix and task similarity vector based on a kernel function. In that, the kernel function effectively quantifies the similarity of the loss gradients of the different tasks, evaluated at a learnable parameter initialization. Overall, the coefficient matrix can be understood as a look-up matrix in which each row holds the learned parameter vector for one meta-training task, the final adapted parameters are a linear combination of these parameter vectors in, weighted by the kernel between current task and the meta-train tasks.

In two simple simulated experiments, the paper demonstrates that TANML is able to outperform MAML and Meta-SGD when the meta-train tasks are set up in a pathological way (e.g. by combining two dissimilar clusters of tasks or by adding outlier tasks).

=== Reviewer’s main argument ===

Overall, the idea of incorporating a notion of task-similarity into the meta-learner and the particular proposal to use the kernel between the task loss gradients to quantify such similarity is sound and is a valuable contribution in itself. 

Unfortunately, the relationship between Generalized Meta-SGD to TANML is unclear. Usually the connection between linear regression (c.f. Eq. 1 in the paper) and kernel regression (Eq. 2) is established through the particular form of the kernel regression coefficients. However, since the coefficient matrix is (meta-)learned in the paper, it is unclear how TANML relates to Meta-SGD. In fact, TANML seems more like a learned linear combination of task parameters which does not resemble much commonalities with MAML. Overall, the connection to MAML seems a bit set-up/artificial. Discussing the particular relationship between MAML/Meta-SGD and TANML would improve the storyline of the paper. For instance, if TANML is a generalization of MAML, it would be good to state with which particular choices of $\theta_0$ and $\Psi$, we can recover MAML.

The related work section is quite minimalistic. For instance, discussing how TANML is different from e.g. multi-task nonparametric methods (e.g. [1-2]) that also use a kernel between tasks, would better clarify how TANML relates to previous work.

The numerical experiments are very simple / limited and designed in a pathological way. Thus, it is not surprising that MAML/Meta-SGD perform worse than TANML. How applicable the experimental results are in more realistic meta-learning setups is unclear. Despite the simplicity of the experiments, there is not enough information to properly reproduce the experiment. For instance, how are A and $\omega$ in experiment 2 sampled, how are the x in experiment 1 sampled and how many data points per task are used in experiment 1? The following would strengthen the experiment section:
- A real-world use case in which we expect to see a meta-training set with e.g. outliers similar to experiment 2
- Experiments with real world meta-learning datasets. For real-world & small-scale meta-learning environments for regression, see e.g. [3].
- An additional meta-learning setup without outliers / clusters of meta-learning tasks. This way one can assess how the proposed method compares to MAML/Meta-SGD in standard setting
- Adding missing details, e.g. to the appendix, which are necessary for reproducing the experiment.

=== Overall assessment ===

I vote for rejecting the paper. In the current state, the storyline from MAML to TANML provides little value to me as a reader. The proposed algorithm resembles a classical kernel-weighted linear combination of parameters and the pathological toy experiments provide little value for assessing the actual usefulness of TANML in realistic meta-learning scenarios. However, using the kernel between the task loss gradients as a similarity metrics of task is a nice idea and is a valuable contribution. I highly encourage the authors to further improve the paper. Overall, TANML has scientific merit - when introduced with a convincing storyline and properly supported by realistic experiments and relevant baseline comparisons, this would be a clear accept.

=== Minor remarks ===

- Section 2: Eq. 1: move the comma. It should be $[\theta_0^\top, \nabla_{\theta_0} \mathcal{L}$ …
- Section3: Either the $\Psi$ should be a $T \times D$ matrix, or there should be no transpose in Eq. 2
- Section 3 Eq 2: The kernel in the sum should probably be between i and i’, not between i and i.
- Section 4.1, 2nd paragraph: “... could be ascribed [to] its linear nature …”


[1] Bonilla, Edwin V., Kian M. Chai, and Christopher Williams. "Multi-task Gaussian process prediction." Advances in neural information processing systems. 2008.

[2] Micchelli, Charles A., and Massimiliano Pontil. "Kernels for Multi--task Learning." Advances in neural information processing systems. 2005.

[3] Rothfuss, Jonas, Vincent Fortuin, and Andreas Krause. "PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees." arXiv preprint arXiv:2002.05551 (2020).

review_title: Overall a sound algorithm, but framed/motivated in a strange way and hardly supported by relevant/realistic experiments

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper introduced a meta-learning framework in which a kernel describing similarity between the tasks is used to construct an RKHS which is used to perform kernel regression. The framework is instantiated in a form of an algorithm: TANML which can be viewed as an extension to a popular Meta-SGD algorithm. The experiments on two regression tasks are presented to analyse the efficacy of the proposed method.

1. I consider the method mathematically sound, ie. I don't see theoretical reasons which would make it obvious that it wouldn't work.
2. The combination of using task-similarity and kernels is not present in the literature known to me, so the work under review contains (some elements of) novelty.
3. However, both "explicitly employing task-similarity" (Achille et al. 2019) and "using kernel methods" (Vinyals et al. 2016) (separately) is well represented in past works.
4. In my opinion moving kernels from space of images/classes (like in Vinyals and other kernel methods cited in the paper) to the space of tasks doesn't, on its own, demonstrate the level of novelty that is required by accepted papers. Comparison to and commentary on the previous metric-based meta-learning papers present in the work under review is unsatisfactory.
5. The performed experiments are extremely toyish: the results are not sufficient to support the claims of the paper. This is further exacerbated by the fact that authors need to settle with optimization tricks to learn their TANML model.

I recommend against publication at ICLR. The novelty of the authors' work is limited and the experiments are not convincing. I appreciate that the goal of the work is not to beat SOTA, and rather to introduce and investigate a small change to MAML, but I believe that reducing the scope of the research in this way grants an expectation of extensive experimental evidence which this paper is lacking.

Suggestion:
To expand the research to higher-dimensional, few-shot classification tasks, one could hardcode the kernel based on the human understanding of the classes. This way, one could take a problem which Meta-SGD is known to be working with and see if an introduction of a kernel gives a measurable improvement.

Technical:
1. "Training for tasks individually will result in a predictor that overfits to $\mathcal{X}$, $\mathcal{Y}$, and generalizes poorly": It's unclear what "individually" means here (is MAML with batch size = 1 training for tasks individually?). It's also far from obvious, in particular in the context of Raghu et al. (Rapid Learning or Feature Reuse?). I encourage authors to avoid using such statements without referring to argumentation behind them.
2. I am also confused by the sentence "while there have been studies on deriving features and metrics for understanding the notion of similarity between data sources or datasets, they have not been used in the actual design of meta-algorithms": isn't the (cited by authors) Achille et al. (2019) one example of such work?
3. Along the whole paper, \citep is used, even when \citet is appropriate. See when to use each one [here](https://www.reddit.com/r/LaTeX/comments/5g9kn1/whats_the_difference_between_cite_citep_and_citep/).
4. TA*N*ML in Sec. 4.
5. .. realizations of tasks is reported **in** Table **2**. In Sec. 4.2.
review_title: Simple idea, results only in toy settings

final_score: rating: 3: Clear rejection
review_content: ---- Update ----

I thank the authors for clarifications. I trust that the suggestions of all reviewers, taken together, provide substantial avenues for improving the work. However, at this point I must keep my score and encourage the authors to continue the work with the valuable honest feedback provided here.

---- Original Review ----

Summary:

The paper aims to formalize “task similarity” in meta-learning settings by making use of nonparametric kernel regression techniques; such similarity information is then proposed as a means to alleviate some of the current issues with meta-learning algorithms such as MAML/Meta-SGD, namely reliance on large sets of similar meta-training tasks. Experiments focus on standard toy regression tasks with the added meta-training data scarcity.



Strong points: 
- Principled approach to a challenging and current problem of interest for the sub-field of meta-learning and beyond. However, formalization of meta-learning approaches in terms of the NTK is recent but not novel [see reference Wang 2020 in paper].
- Good work in progress, but the attempt to publish is premature.



Weak points:
- Very poor representation of relevant and conceptually similar recent work, see [1] for a comprehensive review, and specifically [2, 3, 4, 5, 6] for similar approaches.
- Inaccurate claims of novelty are made; they must be made more specific and put into context. For example, the claim that task descriptors have not been used in the design of meta-learning algorithms is false, see [5, 6]. That said, the current approach could be used to analyze such SOTA approaches and perhaps explain their performance.
- Meta-training data reuse across tasks at test time has been proposed previously, e.g. [7], so it is also not novel to this paper.
- Very weak experimental evidence. Please use some of the few-shot image classification datasets, or standard RL tasks available since MAML was published.
- Proposed method needs extensive approximations to scale up to more interesting problems.




Recommendation and Rationale:

I believe the paper should be rejected in current form, but I strongly encourage the authors to add more experimental data and submit to a workshop.



References:
[1] Meta-Learning in Neural Networks: A Survey
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, Amos Storkey. https://arxiv.org/pdf/2004.05439.pdf
[2] Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, Thomas Griffiths. https://arxiv.org/abs/1801.08930
[3] Bayesian Model-Agnostic Meta-Learning. Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, Sungjin Ahn. https://papers.nips.cc/paper/7963-bayesian-model-agnostic-meta-learning.pdf 
[4] Probabilistic Model-Agnostic Meta-Learning. Chelsea Finn, Kelvin Xu, Sergey Levine. http://papers.nips.cc/paper/8161-probabilistic-model-agnostic-meta-learning
[5] Meta-Learning with Latent Embedding Optimization. Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, Raia Hadsell. https://arxiv.org/abs/1807.05960
[6] Few-Shot Image Recognition by Predicting Parameters from Activations. Siyuan Qiao, Chenxi Liu, Wei Shen, Alan Yuille. https://arxiv.org/abs/1706.03466
[7] Meta-Q-Learning. Rasool Fakoor, Pratik Chaudhari, Stefano Soatto, Alexander J. Smola. https://arxiv.org/abs/1910.00125

review_title: Good work in progress but further work is needed, both theoretical and experimental

Meta-review: This paper addresses a method that incorporates the task-similarity (via task gradients) into the meta-learning. The inner loop update is done by kernel regression with the similarity between gradients of tasks considered, and the outer loop is the gradient update with a particular regularization. Without any doubt, it is a timely and important topic to develop a meta-learning method in the presence of outlier tasks. All reviewers criticized the experiments were done on only simple datasets without any ablation study. Authors revised their manuscript, to include more experiments, and tried to clarify the relation of their method to MetaSGD. Unfortunately, however, even after the author response, reviewers were not convinced that their concerns were resolved. In particular, it was claimed that the revised version still lacks comparisons to previous relevant work. 

Decision: Reject