Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Summary:
The authors present a PyTorch based framework for performing second-order
reverse mode autodiff for meta-learning.

First, the authors present a formalization of a general prototypical
meta-learning setting.
They then provide an algorithm that solves this problem via gradient based
optimization.
Finally, perhaps the main contribution is a specific PyTorch implementation
of said algorithm.

The type of meta-learning setting the authors consider is one where a gradient
based inner loop optimizer finds $\theta^\star$ by performing a finite number of steps.
The inner loop optimizer is parameterized through $\varphi$ that consists of
two parts $\varphi^\text{loss}$ and $\varphi^\text{opt}$.
The parameters $\varphi^\text{loss}$ are somehow part of the loss used for
training in the inner loop. Example: Regularization paramter.
The parameters $\varphi^\text{opt}$ do not occur in the loss but in the
optimizer step. Example: Learning rate.

Example of an inner loop step:
$\theta^{k+1} := \theta^k - \alpha (\nabla L(\theta^k) + \lambda \nabla R(\theta^k))$
where $\theta$ are the parameters of a neural network, $L$ is the training loss,
$R$ is the regularizer, $\alpha$ is the learning rate, $\lambda$ is the regularization parameter.
In this example we would have $\varphi = (\alpha \lambda)^T$.

The authos assume $\theta^K$, the output of the inner loop after $K$ steps to
be differentiable wrt $\varphi$.
Furthermore, the meta-learning loss is assumed to be differentiable wrt $\theta$
so that a gradient of the meta-learning loss wrt to $\varphi$ can be computed.
The authors also assume the meta-learning loss to be sufficient smooth in $\varphi$
such that a gradient based optimization can even be used for meta learning to
a local optimum.

The authors explicitly write down the reverse mode auto differentiation of
the inner loop and show how to, in that way, compute the gradient of the
meta-learning loss wrt to $\varphi$.

The reverse (adjoint) mode auto differentiation of the above example inner loop step
is the following step (iterated over in reverse down from $k = K$ to $k = 1$:
$\bar \theta^k := (I - \alpha (\nabla^2 L(\theta^k) + \lambda \nabla^2 R(\theta^k)))^T \bar \theta^{k+1}$
$\bar \alpha := \bar \alpha - (\nabla l(\theta^k) + \lambda \nabla R(\theta^k))^T \bar \theta^{k+1}$
$\bar \lambda := \bar \lambda - \alpha \nabla R(\theta^k)^T \bar \theta^{k+1}$
where $\bar \alpha$ accumulates the gradient of $\theta$ wrt $\alpha$ and
$\bar \lambda$ accumulates the gradient of $\theta$ wrt $\lambda$.

The authors give some implementation details specific to some frameworks necessary
for implementing such "gradient of an inner loop".

The authors present experiments where they show how to meta-learn learning rates
with their framework.
They also how their framework can be used to quickly implement a MAML type
meta-learning optimizer ablation study comparing various combinations of
architecture, optimizer and inner loop steps etc..

Recommendation:
I propose to reject the paper.
In my eyes the only contribution is the implementation of a meta-learner in
PyTorch based on well known methods.
The provided unifying formalization is theoretically inaccurate (see below)
and to me come across as merely a motivation for their framework
(but no value added compared to existing literature).
There is no new insight provided on the software engineering level either as
far as I can see.

Detailed comments:
- Page 1: ...provides tooling for analysing the provable requirements...
	seems like a complicated way of saying something that could be said simple

- Page 2: Without loss of generality, ...
	You are assuming a parametric model. Not sure what generality this phrase
	refers to.

- Page 2: A formalization as $\theta^\star = argmin(\theta, L(\theta, \varphi))$
	is inaccurate in the sense that it does not acknowledge the existing of
	multiple optima.
	I would recommend not to use the $argmin$ operator here, since $argmin$ for
	something like a neural network would for example either return a global
	optimum (which no optimizer used in practice finds, and is not ment here)
	or would take on a set value with multiple local minima for example.

	In the same context, the authors should mention the issues about uniqueness
	of optima (we are not even really finding optima when training neural networks),
	implicit functions / implicit differentiation

	In the context of using stochastic optimizers one should also at least
	mention something about the differentiability of outputs of such optimizers
	and how they potentially depend on randomness of mini-batches
	(what if different randomness is used with the same or a perturbed
	hyper parameter?)

- Page 3: You mention the potential statefulness of the optimizer.
	Why not explicitly carry it in the math notation?
	Probably things would get cluttered but saying it should be covered within
	$\varphi$ does not seem reasonable to me.

- Page 3: While this may seem like a fairly trivial formalization...
	Yes, but also nesting this in an outer loop is fairly trivial in the sense
	that it is a well known approach.

- Page 4: there exist continuous hyperparam...
	If they are not continuous then they should not even occur in this
	formalization so saying there exist... does not make much sense to me here

	$\alpha \subseteq \varphi^\text{opt}$ implies that $\varphi^\text{opt}$ is
	a set from notation although we are treating it as a vector everywhere else

- Page 4: All of section 2.4 seems somewhat trivial to me, but I guess that is
	highly subjective.

- Page 5: in the definition of stop operator perhaps use $:\Leftrightarrow$

- Page 5: Perhaps explicitly mention how your approach differs from a reverse
	mode differentiation of training or if it does not differ, say this.

- Page 14: When talking about _S_GD (instead of just GD) perhaps mention
	something about non-existence of mini-batch randomness / being deterministic

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This work presented a general formulation of a wide class of existing meta-learning approaches, and proved the requirements that must be satisfied for such approaches to be possible.

Half of the work is focused on describing the unnamedlib library, which extends PyTorch to enable the easy
and natural implementation of such meta-learning approaches.

The early sections are interesting, especially section 2, which gives some great insights to the existing inner loop pattern in meta-learning. However, from section 3, the paper has turned to examples and related works, where I was hoping the author would give more detailed analysis of the pattern. My concern is the authors have spent too much space on the unnamedlib library. So http://www.jmlr.org/mloss/ might be a more suitable place for publication.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The authors propose the general formulation of recent meta-learning methods and propose a good library to use.

Pros:
1. The general formulation of recent meta-learning methods is reasonable.
2. The proposed library is easy to use.

Cons:

The paper lacks technical novelty. I understand the goal of this paper is to build a library. However, the paper only describes a general formulation for recent meta-learning methods (e.g., MAML) and implement the formulation. It is better to clarify and some key engineering challenges and do the corresponding experiments.

In addition, in the experiment parts, the authors only compare the results with MAML++. It will be more convincing if the authors can analyze other popular meta-learning methods (e.g.. Prototypical network [1], meta-LSTM [2]). 

Another suggestion is that the authors can give some examples to connect current meta-learning models with the proposed general formulation. For example, the meaning of \phi_i^opt, \phi_i^loss in MAML, Prototype, Reptile, etc.

It is better to explain the meaning of different colors in Figure 3.

[1] Snell, Jake, Kevin Swersky, and Richard Zemel. "Prototypical networks for few-shot learning." Advances in Neural Information Processing Systems. 2017.
[2] Ravi, Sachin, and Hugo Larochelle. "Optimization as a model for few-shot learning." ICLR (2016).



Decision after rebuttal: I have read the authors' responses. Like review 2, I also think the "generalization" is overclaimed, it only provides a general formulation. Thus, I finally decide to keep my score.
review_title: Official Blind Review #3

Meta-review: The reviewers agree that the technical innovations presented in this paper are not great enough to justify acceptance.  The authors correctly point out to the reviewers that the ICLR CFP states that the topics of "implementation issues, parallelization, software platforms, hardware” are acceptable.  I would point out that most papers in these spaces describe *technical innovations* that enable improvements in "parallelization, software platforms, hardware" rather than implementations of these improvements.   However, it is certainly true that a software package is an acceptable (although less common) basis for a publication, provided is it sufficiently unique and impactful.  After pointing this out to the reviewers and collecting opinions, the reviewers do not feel the combined technical and software contributions of this paper are enough to justify acceptance. 
 
Decision: Reject