Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Summary:
The current paper deals with meta-learning and essentially proposes a generalization of MAML (a popular gradient-based meta-learning algorithm) that mostly builds upon two main recent advances in meta-learning: 1) an architectural one (see e.g. T-Nets), which consists in optimizing the parameters of additional layers during the meta-learning outer loop (as opposed to only optimizing the initial conditions of the original parameters like in MAML), and 2) a theoretical one (see e.g. Meta-SGD, Meta-curvature), which is based on the geometrical observation that one set of parameters can precondition a second set of parameters that are consequently being optimized in a "warped" geometry, possibly speeding up learning.
The authors provide a great and thorough overview of the literature, in particular for gradient-based meta-learning methods, which helps putting all this in perspective.
The way they obtain the mentioned "warped" geometry in practice is by adding additional so-called warp-layers to an architecture that is being trained with meta-learning. Such warp-layer are generic deep learning modules (such as convolutions followed by BatchNorm, or LSTM layers), which are being trained in the outer-loop of the meta-learning optimization. In this sense, WarpGrad extend T-Nets, which only allowed for linear layers.
The second main innovation of WarpGrad is the proposal of a new meta-learning objective, which incorporates a meta-learning internal loop of only one step of (preconditioned) SGD, meaning that, as the authors notes, "in contrast to MAML-based approaches (Eq. 1), [...] avoids backpropagation through learning processes".
The authors test their algorithm on several meta-learning benchmarks, including few- and multi-shot learning tasks demonstrating very competitive performance when their algorithm is combined with MAML or Leap. They then deploy WarpGrad on a maze navigation reinforcemente learning task to demonstrate training of recurrent architectures, and on a continual learning toy dataset to show that their objective can be adapted to mitigate catastrophic forgetting. 

Decision:
This is a good paper which proposes an interesting generalization of previous gradient-based meta-learning methods like MAML and T-Net, with an impressive number of experiments. However, some of the statements regarding the advantages of WarpGrad over previous algorithms seem a little bit misleading, in particular in situations where WarpGrad needs to be combined with these same algorithms. For instance (and I might have completely misunderstood things here), it seems that when the WarpGrad objective is being combined with MAML (which requires backpropagation through multiple-step gradient descent trajectories), then also the resulting combined objective will necessarily need to backprop through the same multi-step trajectory, defeating the stated advantage of the WarpGrad algorithm (i.e. that its objective avoids backpropagating through the learning processes).
In general, even if one only considers the WarpGrad objective eq. (10), that comprises a meta-learning inner loop which consists of one step of (preconditioned) gradient descent. However, it seems like an arbitrary (and limiting) choice of the authors to only perform one step, as opposed to multiple ones. As a matter of fact, even very sophisticated second order gradient descent methods like natural gradient descent typically require more than one step to reach a local minimum. That is to say, that the main advantage showcased by the authors (the fact that the WarpGrad objective avoids backprop through a whole learning trajectory) seems like a limitation, rather than the result of a principled derivation.
It would be beneficial if the authors could clarify this points. In particular, whether combining WarpGrad with MAML does not indeed negate the stated advantages of WarpGrad over MAML, and whether there is a principled way of demonstrating that executing only one step in the inner loop of the WarpGrad objective is completely general (i.e., additional steps do not help the inner loop).

Minor:
- The authors use the wrong citation key when referring to the T-net paper: it should be Lee et al 2018, instead of Lee et al. 2017
- I believe that when the authors mention Fast and slow weights, they are being described in the opposite way: slow weights should be in charge of meta-learning information, while fast ones are in charge of task-specific information.
- Line 3 and 4 of Algorithm 1 and 2: shouldn't it say "mini-batch of tasks" (plural), instead of "mini-batch of task", since several tasks are being sampled? Otherwise, it might be erroneously interpreted as "mini-batch of (samples belonging to) task T".
- The comment that "learning to precondition gradients can be seen as a Markov Process of order 1" is never clearly elucidated or developed. It would help to develop this.
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: The authors propose warped gradient descent (WarpGrad) an optimisation framework for facilitating gradient-based meta-learning. WarpGrad interleaves within the learner meta-learned warp-layers that implicitly precondition the gradients of the task-specific parameters during backpropagation. In contrast to the linear projection layers employed in T-Nets, warp-layers are unrestricted in form and induce a full Jacobian preconditioning matrix. The warp layers are meta-learned in a trajectory-agnostic fashion, thus obviating the need to backpropagate through the gradient steps to compute the updates of their parameters. The framework is readily applicable to standard gradient-based meta-learners, and is shown to yield a significant boost in performance on both few-shot and multi-shot learning tasks, as well as to have promising applications to continual learning.

The paper is well-structured and well-motivated: the problem statement is clearly laid out from the outset, with appropriate context, and explanations supported well diagramatically. The idea, and perhaps more so the applications thereof, is seemingly novel and its explanation is given straightforwardly while avoiding getting bogged down in technical details. Clear comparisons and distinctions with previous work are drawn - for instance with the update rules for several gradient-based methods - MAML and its derivatives - being laid out in standard form (though it might also be nice to echo this with the WarpGrad update rule).

The experiments are logically ordered with the initial set covering the standard few-shot learning benchmarks with appropriate baselines (though the results for few-shot tieredImageNet are lacking in this respect), with most essential details given in the main text and full details, including those related to the datasets in question and hyperparameter selection, documented in Appendix H. Meta-learning does seem uniquely well-positioned for tackling the task of continual learning and it's heartening to see this being explored here with a degree of success - it would be interested to see how its performance compares with standard continual learning methods (such as EWC) on the same task. Particularly impressive is the depth into which the Appendices regarding the experiments, both elaborating on the details given in the main text as well as additional ablation studies.
 
Minor errors:

- Page 7: "a neural network that dynamically **adapt** the parameters..." - should be "adapts"
- Page 22: "where $I$ is the **identify** matrix" - should be "identity" 
- Page 27: "The task target function $g_\tau$ is **partition** into 5 sets of **sub-task**" - should be "partitioned" and "sub-tasks", respectively
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: This paper proposes a learning strategy to precondition gradients for meta-learning. I really enjoyed reading the paper though I admit that I couldn't fully grasp all the details yet (paper is dense). My comments below are mostly to improve the readability of the paper for readers like me (knowing a thing or two in optimization and meta-learning)



1- The authors emphasize on the method being trajectory-agnostic. Can you explain why this is very important? What methods are not trajectory-agnostic?

2 - Also in various places, the authors claim the method does not suffer from vanishing/exploding gradients and credit-assignment problem. This needs to be properly verified (and explained as I do not see the connections clearly)

3- Some claims are based on the Omniglot experiments (eg., the effect of the stop-gradient). It would be good if this can be done on Mini-imagenet instead.

4- I am not sure I understand the stop-gradient operator, can you be more explicit there?

5- I read the conversation regarding linear units on openreview and I disagree with your statement. A cascade of linear layers does not necessarily match one linear layer unless some constraints on the rank of layers are envisaged, a bottleneck in the middle ruin everything. 

review_title: Official Blind Review #1

Meta-review: A strong paper reporting improved approaches to meta-learning.
Decision: Accept (Talk)