Title: No title found

Abstract: No abstract found

:



Figures and Tables:

Peer review (iclr2020):
This paper proposes a gradient-based meta-learning algorithm that extends MAML by applying a multiplicative correction to the dimensions of the gradients in the inner loop. For each task, a learned context vector is decoded to produce a low-rank (ie. bottlenecked) correction term for the inner loop gradients. It is an interesting idea and the results seem compelling. I think this could be quite a good paper, but there are a number of issues in its current state, and I have to lean towards rejection. I strongly encourage the authors to address my comments below by the end of rebuttal, as I believe the paper has potential. My main concern is that the writing in general is quite unclear in parts and there are a lot of typos and grammar errors throughout. For example, the first sentence in the main body, *Approaches, ....* doesn*t seem to make sense; *low-shot* should be replaced with *few shot*; in Section 3, *one can expect that a few iterations of the gradient descent, even with limited data, to lead to a well-adapted model*; and Section 4: *GECCO, while not entirely resolves...* I think a thorough proofread is needed to rectify these issues. Next, I think the paper could better discuss GECCO*s relationship to relevant work such as CAVIA (Zintgraf et al, 2019) and LEO (Rusu et al, 2019). These are cited in passing (eg. *CAVIA and LEO deliver cheaper solutions that generate the modulation and additional layers for the main networks ...*), but there are closer similarities that should be explicitly discussed. For example, LEO learns a low-dimensional latent space which is used to linearly generate the parameters of a classifier, while GECCO uses a low-rank approximation to modulate the gradient dimensions directly. Similarly, the context vectors in this work seem to be estimated in the same way as in CAVIA (starting at zero and then adapted in the inner loop), and the main difference seems to be in the way the context vectors are used: by modulating the gradient components multiplicatively rather than as an additive input bias to the network. Given the similarities to LEO (linearly modulating gradient dimensions versus linearly generating parameters), and CAVIA (using context vectors to modulate gradients versus acting as an additive bias), it is surprising that GECCO performs comfortably better on benchmarks. I think the differences should be more clearly and precisely communicated, and closer analysis is required to delineate the source of this improvement over CAVIA, LEO, and other approaches. Some other questions: - What architectures and hyperparameters were used in the miniImageNet benchmark? Are they equivalent for all baselines / is it a fair comparison? Specifying these details in the appendix would give readers the confidence that the empirical results are comparable. - In page 4, *In what follows..., we assume n = n1 × n2 ...*; I*m not sure what this refers to or what the point of the method is. - Is the Vec() function in Eqn 3 vectorization of the matrix? I don*t think this is clear; it could be explained in the text. The footnote is also confusing: is the softmax output *multiplied* element-wise o ver the columns of h? What shapes are the input arguments?