Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors introduce EvoGrad, a method for approximating second order gradients for use in Meta-Learning applications. The method approximates the inner loop of optimization with "evolutionary" update. The authors show their method is more computationally efficient than the T1-T2 method, yields good approximations to the second order gradients and is able to converge in the toy setting, and achieves better results than MetaWeightNet, a second order method, on a cross-domain few-shot classification task. 
main_review: The main idea of the paper is not that technically novel but the paper is well written and easy to understand  -- explanations are clear and straight-forward.. The evolutionary update consists of a weighted sum of randomly selected perturbations of the current parameter values and reminds me of the finite differences approximation, although not an approximation to the first order gradient in this case. Instead, the first order gradient is removed from the update procedure so that optimizing hyperparameters does not require second order gradients. The weights for the parameter sum are selected based on a softmax procedure using their individual losses giving more weight to better performing parameter sets. 

I found the computational arguments consisting of time requirements, memory requirements and scalability convincing. 

I think I am misunderstanding something about the plot in figure 2: the red curve represents the ground truth hypergradient but the curve does not match the closed form hypergradient function defined on line 192.

The experiments analyzed in sections 4.1 and 4.2 use bounded hypergradients. Figure 2 seems to indicate that as lambda gets smaller the error between the approximation and the true hypergradient gets larger but, for this example, it's not too much of an issue since lambda is bounded below. Could we see divergence if the approximation error gets too large in unbounded cases? 

I found the experiment section was good but there are still a few questions left unanswered in my opinion:
- The inner-loop is replaced only for the hyperparameter gradient. How good is the model that is fully trained using the "evolutionary update", ie. actually uses that inner-loop during training?

- How well does this model perform to the "first-order" approximations used by MAML and many other MAML-based models where higher order gradients are ignored via a stop-gradient procedure in the automatic differentiation software? Depending on which hyperparameter you are optimizing, this procedure would have different interpretations but would be easy to implement in all cases.

- How well does this compare to IFT methods? The authors mentioned that IFT methods assume that the inner-loop is run until convergence and that "this makes them unsuitable for the majority of practical applications above where training the inner loop to convergence for each hypergradient step is infeasible." However, without more explanation I don't see why this would be the case especially since EvoGrad uses an inner-loop approximation as well. Also, the authors state that in IFT methods, the "costs come from the associated overhead with approximation an inverse Hessian" but that this Hessian doesn't have to be stored in memory. However, what is required is the computation of Hessian-vector products which is a much cheaper computation.

-  In section 4.4, why does EvoGrad have better performance in terms of accuracy compared to MetaWeightNet? Isn't EvoGrad designed to be an approximation to the second order method used in MetaWeightNet? Is there something about the inductive bias in the approximation that would yield better results?

Due to these questions, I think this paper is marginally above the threshold of acceptance but I look forward to reading the opinions of the other reviewers and the response from the authors.
limitations_and_societal_impact: Yes, there is a good paragraph addressing limitations and societal impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper presents EvoGrad, a technique that replaces the inner loop of meta learning with an evolutionary technique to reduce the cost of meta learning - allowing meta learning to be applied to larger models. The paper demonstrates the EvoGrad can allow scaling up LFT from ResNet18 to ResNet34 and reduces the computation cost on MWN models.
main_review: I vote to accept the paper.

The paper is clear and concise, the idea is neat and novel, the details are presented clearly and the experiments evaluate the efficacy of the method.

Originality: The idea of doing away with second-order gradients using a simple evolutionary technique in the inner loop is novel and, to the best of my knowledge hasn't been tried before. Second-order gradients are indeed responsible for the bottle neck in computation costs and getting rid of this computation entirely while retaining the accuracy is a pretty neat result.

Quality: The main contribution of EvoGrad is retention of performance and reduction in computation costs. These claims are substantiated as shown in Figures 4 and 5 and Tables 4 and 5. Moreover, the authors were able to scale up LFT to ResNet34.

As the authors mention, the hyper gradient estimate could be much worse on a different suite of problems. An excellent version of this paper would have evaluated the technique on more meta learning tasks. Alternatively, the paper could evaluate EvodGrad on task that is unrelated to computer vision to demonstrate that the hyper gradient approximation generalizes across domains.

Significance: If the hyper gradient approximation generalizes well to other meta learning tasks, the community will benefit from EvoGrad in that they will be able to use meta learning on larger models - an important contribution.

A minor comment on presentation - Figure 3 is not very readable - at first glance it wasn't clear to me if the three pictures are even different.

Typo in line 70 
> or even a a synthetic training
limitations_and_societal_impact: As described in the paper - the accuracy of the hyper gradient approximation could depend on the domain EvoGrad is applied to -  in the paper, all experimental tasks were related to image based data.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper presents an efficient gradient-based meta-learning algorithm based on evolutionary techniques. The proposed algorithm doesn’t require second order gradient computation, and thus, is more efficient. The proposed approach was evaluated on two meta-learning tasks for few-shot feature-wise transformation and noisy label learning, on which time and memory efficiencies were demonstrated.
main_review:  # Originality

- The idea of using evolutionary methods to eliminate the need for higher-order gradients is novel.
- The related work is adequately cited, the proposed algorithm differs from the prior work in being more efficient in runtime and computation. 

# Quality

- The submission is technically sound and the claims are supported by empirical results, but could be limited (please see below).
- The approach lacks the theoretical analysis for its effectiveness. 
- The empirical analysis is limited to mostly computer vision applications.
- The proposed approach is appropriate for the studied applications, however, it’s not clear if the variance in the estimates for the meta-gradient would significantly affect the results for other applications.
- The authors are open about the limitations for their approach and discuss these challenges in the discussion section.

# Clarity

- The paper is clear, well organized, and easy to follow.

# Significance

- The work is significant as it aims at scaling meta-learning algorithms via using an evolutionary procedure to estimate meta-gradients without the need for computing higher order gradients. 
- However, this estimate could be noisy and suffer from high variance (see Figure 2), it’s not clear how to mitigate this issue without increasing the population size leading to strong hits for the energy requirements for the proposed approach.
limitations_and_societal_impact: - The estimates for the meta-gradients could be noisy and suffer from high variance, it’s not clear how to solve this issue without increasing the population size which comes at an increased energy consumption, thus, the proposed approach presents a tradeoff between computing higher order derivatives, and possibly increasing the required memory and energy requirement.
 - As the authors discuss in the discussion section, the proposed approach is greedy and global convergence could not be guaranteed.
- The paper lacks theoretical analysis to the proposed approach.
- The empirical results are mostly limited to the computer vision domain, it’s not clear if the approach scales to larger models from the language and speech learning domains.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes to replace gradient descent with evolutionary search in the inner loop of meta-learning. This makes the computation of the outer loop first-order optimization, which saves compute and memory compared to second-order optimization.
main_review: The writing is very clear, and the problem the paper is solving is also well-motivated: the cost of second-order optimization required in meta-learning makes it prohibitive for many real-world applications.

The idea is original, but the experimental results in the paper do not convince the reader about its significance. The authors mention that the so-called T1-T2 approach is often used as a first-order approximation to calculating the second-order meta gradients, and remark that EvoGrad is asymptotically more efficient. While it might be more efficient to take a single EvoGrad step compared to a gradient step, we might need a lot more EvoGrad steps compared to gradient steps (in fact, there are also classical theoretical results to this effect [1]). This means that in practice, it is likely that EvoGrad is not as efficient as T1 - T2.

The authors should show experimental comparisons with T1 - T2 to convince the reader that EvoGrad is more efficient. I would also recommend benchmarking on standard meta-learning datasets like OmniGlot or MiniImageNet to increase the significance of the results.

Edit: In my original review, I did not understand that the authors were targeting their work specifically for settings where first-order gradient approximations do not exist, e.g. hyperparameter optimization. This is quite different from experimental benchmarks on OmniGlot/MiniImageNet where MAML-like algorithms are commonly being evaluated on. At my request, the authors added in experiments that show that their evolutionary algorithm based approach significantly outperforms a gradient based approach like a linear hypernet. I'm changing my score to a 6 to reflect this.

[1]: Query Complexity of Derivative-Free Optimization. Kevin G. Jamieson, Robert Nowak, Ben Recht.
limitations_and_societal_impact: Please see the main review.
review_title: null

Meta-review: All the reviewers agree on acceptance. This is a clear decision. The authors should take into account the reviewers' feedback to improve the paper for the camera-ready submission.
Decision: Accept (Poster)