EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
Authors: Ondrej Bohdal
Yongxin Yang
Timothy Hospedales
Venue: NeurIPS 2021 Poster (2021)
Keywords: Meta-learning
Hyperparameter optimization
Evolution

Abstract: Gradient-based meta-learning and hyperparameter optimization have seen significant progress recently, enabling practical end-to-end training of neural networks together with many hyperparameters. Nevertheless, existing approaches are relatively expensive as they need to compute second-order derivatives and store a longer computational graph. This cost prevents scaling them to larger network architectures. We present EvoGrad, a new approach to meta-learning that draws upon evolutionary techniques to more efficiently compute hypergradients. EvoGrad estimates hypergradient with respect to hyperparameters without calculating second-order gradients, or storing a longer computational graph, leading to significant improvements in  efficiency. We evaluate EvoGrad on three substantial recent meta-learning applications, namely cross-domain few-shot learning with feature-wise transformations, noisy label learning with Meta-Weight-Net and low-resource cross-lingual learning with meta representation transformation. The results show that EvoGrad significantly improves efficiency and enables scaling meta-learning to bigger architectures such as from ResNet10 to ResNet34.

EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
Ondrej Bohdal', Yongxin Yang', Timothy Hospedales
School of Informatics, The University of Edinburgh 2 Samsung Al Research Centre, Cambridge
{ondrej bohdal, yongxin. biggxin.yag.. yang, d.o uk
Abstract
Gradient-based meta-learning and hyperparameter optimization have seen signifi- cant progress recently, enabling practical end-to-end training of neural networks together with many hyperparameters. Nevertheless, existing approaches are rel- atively expensive as they need to compute second-order derivatives and store a longer computational graph. This cost prevents scaling them to larger network architectures. We present EvoGrad, a new approach to meta-learning that draws upon evolutionary techniques more efficiently compute hypergradients. Evo- Grad estimates hypergradient with respect to hyperparameters without calculating second-order gradients, storing longer computational graph, leading signifi- cant improvements efficiency. We evaluate EvoGrad on three substantial recent meta-learning applications, namely cross-domain few-shot learning with feature- wise transformations, noisy label learning with Meta- eeta-Weight-Net and low-resource cross-lingual learning with meta representation transformation. The results show that EvoGrad significantly improves efficiency and enables scaling meta-learning bigger architectures such from ResNet10 ResNet34.
Introduction
Gradient-based meta-learning and hyperparameter optimization have been of long-standing interest in neural networks and machine learning 2121 4). Hyperparameters (aka meta-parameters) can take diverse forms, especially under the guise of meta-learning, where there has recently been an explosion of successful applications addressing diverse learning challenges [9]. For example to name just few:
training optimizer initial condition in support of few-shot learning 7 I 15 training instance-wise
weights for cleaning noisy datasets 31 26]: training loss functions in support of generalisation [14]
and learning speed; and training stochastic regularizers in support of cross-domain robustness 33
Most of these applications share the property that meta petta-prrseterss impact validation loss only indirectly through their effect model parameters, and so computing validation loss gradients with respect to meta- reta-parameters usually leads to the need to compute second-order derivatives, and store longer computational graphs for backpropagation. This eventually becomes bottleneck execution time, and more severely to scaling the size of the underlying models, given the practical limitation of GPU memory. There has been steady progress the development of diverse practical algorithms for computing validation loss with respect to meta- perameeessssssss 220 19 21 Nevertheless they mostly share some form of the aforementioned limitations. In particular, the majority of recent successful 17 30 essentially use some variant of the T₁ T2 algorithm Bey of validation loss w.r.t. hyperparameters. This approach computes DA
practical applications 31
33 14
2
5
20 to estimate the gradient
the gradient online at each step updating the base model and estimates it as 8ty. ss Biy 8t for training loss εx. As with many alternative estimators, this requires second-order derivatives, and extending the computational graph. Besides the additional computation cost, this limits the size the
Øx 20 300x
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

base model that can be used in given GPU, since the memory cost of meta meta-learning is now multiple times the size of vanilla backpropagation. This in turn prevents the application of meta-learning to problems where large state-of-the tfrte-rr- model architectures are required.
To address this issue, we draw inspiration from evolutionary optimization methods 27 to develop EvoGrad, a meta-gradient algorithm that requires no higher-order derivatives and as such is sig- nificantly faster and lighter than the standard approaches. In particular, we take the novel view of estimating meta-gradients via a putative inner-loop evolutionary update to the base model. As this requires no gradients itself, the meta- gettdgga can then be computed using first-order gradients alone, and without extending the computational graph leading to efficient hyperparameter updates. Meanwhile for efficient and accurate base model learning, the real inner-loop update can separately be carried out by conventional gradient descent.
Our EvoGrad is general meta-optimizer applicable to many meta-learning applications, among which we choose three to demonstrate its impact: the LFT model [3] observes that a properly tuned stochastic regularizer can significantly improve cross-domain few-shot learning performance. We show that by training those regularizer parameters with EvoGrad, rather than the standard second- order approach, we can obtain the same improvement in accuracy with significant reduction in time and memory cost. This allows us to scale LFT from the original ResNet to ResNet34 within 12GB GPU. Second, the Meta- Weight-Net (MWN) 31 model deals with label noise by meta-learning an auxiliary network that re-weights instance- wise losses to down-weight noisy instances and improve validation loss. We also show that EvoGrad can replicate MWN results with significant cost savings. Third, we demonstrate the benefits of EvoGrad on an application from NL.,, in addition to the ones from computer vision: low-resource cross- moos-llnggul learning using MetaXI. approach (5
To summarize, our main contributions are: (1) We introduce EvoGrad, a novel method for gradient- based meta-learning and hyperparameter optimization that is simple to implement and efficient in time and memory requirements. (2) We evaluate EvoGrad on a variety of illustrative and substantial meta-learning problems, where we demonstrate significant compute and memory benefits compared to standard second-orde approaches. (3) In particular, we illustrate that EvoGrad allows us to scale meta-learning to bigger models than was previously possible on a given GPU size, thus bringing meta-lcarning closer to the state-of-the-are frontier of real applications. We provide source code for EvoGrad at battps:://gtahhe com/ /ndre joodall
2 Related work
Gradient-base baseed meta- meta-leamming solves a bilevel optimization problem where validation loss is optimized with respect to the meta-knowledge by backpropagating through the update of the model on training data and with ketaeenno The meta-knowledge updates form outer loop, around an inner loop of base model updates. The inner loop can run for one (0), few 29 21, or many [19] steps within each outer-loop iteration. Meta-Knowllege can take many forms, for example, it can be an initialization of the model weights (7], feature- wise transformation layers [33], regularization to improve domain generalization [ or even synthetic training set (34 Most substantial practical
applications use a one or few-step inner loop for efficiency.
More recently, several methods [19 25 have utilized Implicit Function Theorem (IFT) to develop new gradient- based meta-learners. These methods use multiple inner-loop steps without the need to backpropagate through whole inner loop, which significantly improves memory efficiency over methods that need keep track of the whole inner loop training process. However, IFT methods assume the model has converged in the inner loop. This makes them unsuited for the majority of practical applications above where training the inner loop to convergence for each hypergradient step is infeasible. Furthermore, the hypergradient is still more costly compared to one-step Ti T2 method. The costs come from the associated overhead with approximating an inverse Hessian of the training data with respect to the model parameters. Note that the Hessian itself does not need to be stored due to the mechanics of reverse-mode mode differentiation 3. However, this does not eliminate the remaining calculations which still require higher-order gradients that result in backpropagation via longer graphs due to additional gradient nodes. For these reasons, we focus comparison on the more widely used Ti T2 strategy which is oriented single-step inner loops similar EvoGrad.
Theoretically it is also possible to use hypernetworks [8] to find good hyperparameters in first-order way. Hypernetworks take hyperparameters as inputs and generate model parameters. However, the

---

Dr
Meta-knowledge À
81
k inner-loop steps
0' wi10, Wz02
(
(((+k)
Wp W2 =/sof tmax (-1,, -2)
E2
0z
dly
Ey Dv
02
Figure 1: Graphical illustration of single EvoGrad update using model copies.
approach is not commonly used, likely due to the difficulty of generating well-performing model parameters. We provide experimental results support this hypothesis in the supplementary material.
Meta-learning can be categorized into several groups, depending on the type of meta keoleknnoeedge and also if the model is trained from scratch as part of the inner loop [9]. Offline meta- meta-learning approaches train model from scratch per each update of the meta meta-knowledge. while online meta- learning approaches train the model and meta mheaa-knowledge jointly. As result, offline meta- meta-learning is extremely expensive 6 37 when scaled beyond few-shot learning problems where only a few
iterations are sufficient for training [[]). Therefore most larger-scale problems [16 10
35
use online learning in practice, and this is where we focus our contribution.
The meta-knowledge to learn can take different forms. A particular dichotomy is between the special case where the meta-knowledge corresponds to the base model itself, the form of an initialization; and the more general cases where it does not. The former initialization meta-learning has been popularized by MAML 77. and is widely used in few-shot learning. This can solved relatively efficiently, for example using first-order approximation of MAML 7 Reptile
[4 minibatch
proximal update [36]. On the other hand, there are vastly more cases [9] where the meta weta--nnoleedee is different from the model itself, such LFT's stochastic regularizer improve cross- moos-domain generalization [33, MWN's instance-win wise loss weighting network for label noise robustness [3],
label generation network improve self-supervised generalization [17] Feature-Critte loss improve domain generalization [14] and many others. In this more general case, most applications rely on Ti Tz-like algorithm, the efficient approximations specific MAML do not apply. The ability to significantly improve the efficiency of gradient- -based meta-learning would have a large impact as methods like these would directly benefit from in runtime and energy consumption. More crucially, they could scale to bigger and more state-of-the neural network architectures.
3 Methods
3.1 Background: meta-learning bilevel optimization
We aim to solve bilevel optimization problem where our goal is to find hyperparameters that minimize the validation loss Ły of the model parametrized by and trained with loss by and A:
= arg min (A), where & (A) = &v (2, (A)) and 0" (A) arg min ( (A)
(I)
In order to meta-learn the value of using gradient-based methods, we need to calculate the hypergradient 8X
dey We can expand its calculation follows:
(2)
(e, 0- (A))
dlv 0 (A)) 00 (A)
(2)
0A
ax
00*(A)
ax
3

---

In meta-learning and hyperparameter optimization more broadly, the direct term (A is typically zero because the hyperparameter does not directly influence the value of the validation loss it influences it via the impact on the model weights 0. However, the model weights O are themselves trained using gradient optimization, which gives rise to higher-order derivatives. We propose a variation this step where the update of the model weights is inspired by evolutionary methods, allowing us to eliminate the need for higher-order derivatives. We consider the setting where the hypergradient of hyperparameter À is estimated online (20] together with updating the base model 6,
as this is the most widely used setting in substantial practical applications [31 33
30 16
3.2 The EvoGrad update
Given the current model parameters 0 RRM, hyperparameters À E RV, training loss by and validation loss εv, we aim to estimate Byy for efficient gradient-based hyperparameter learning. The key idea is solely for the purpose of hypergradient estimation to consider simple evolutionary rather than mradient-baseed inner-loop step on A.
2x
Evolutionary inner step First, sample random perturbations E IRM NN0,0 and apply
them to O. Sampling K perturbations, we can create a population of K variants {0x )= of the current model as 0g = + Eg. We can now compute the training losses [(x]k=] for each the K models, l fDr [0,, A) using the current minibatch Dy drawn from the training set. Given these
loss values, we can calculate the weights (sometimes called fitness) of the population of candidate models as
-22, soltaa((=f,_ee....--i (k)]/T)
W1, W/2 WK
where T a temperature parameter that rescales the losses to control the scale of weight variability,
Given the weights {wx 1 k= we complete the current step of evolutionary learning by updating the model parameters via the affine combination
0t
wi 0 + u202 + ................................ + w 0k-
Computing the hypergradient We now evaluate the updated model 0* for a minibatch from the validation set Dy and take gradient of the validation loss ev f(Dv (0") w.r.t. the hyperparameter:
dey Of(Dvl0")
(5)
ax
ax
One can easily verify that the computation in Eq. 5 does not involve second-order gradients as no first-order gradients were used in the inner loop. This is contrast to the typical approach [1 of applying gradient-based updates in the inner loop and differentiating through it (in either forward-mode or modee, or even applying the implicit function theorem (IFT) [19] all of which trigger higher-order gradients and an extended computation graph.
Algorithm flow
In practice we follow the flow of - 20 used by many substantive applications
33
31
2
16
35).
We take alternating steps 0 using the exact gradient Ø6r and on À using the
00
hypergradient
0yy 2X
which in EvoGrad is estimated as in Eq, 5
3.3
EvoGrad hypergradient as a random projection
To understand EvoGrad, observe that the hyper-gradient in Eq, expands as
dlv Olv 00
dlv dw
Olv E Ow de
(6)
ax 00 ax a0
Ox ae de ax
where = Eg......... [K] is the M x K matrix formed by stacking EN 's as columns, w is the K-dimensional (w [W1, W2,.... wx) vector of candidate model weights, and e [f1, f2,,,,...,44 [x] is the K-dimensional vector of candidate model losses.
Recall that E is random matrix, so the operation 06y can be understood as randomly projecting the M-dimensional validation loss' gradient to new low-dimensional space dimension K & M. Alternatively, we can interpret the update as factorising the parrameter-to-hyyprpraaaa &w of size M x K and K N. derivative (sized M x N) into two much smaller matrices & and OX Ox
ag

---

In terms of implementation, Béy is obtained by backpropagation and & is sampled on the fly. The äg term Ow Ow Dt is computed by the softmax-to- tt-logit derivative (K K) and the derivative of the K' candidate models training losses w.r.t. hyperparameters. It is noteworthy that the K elements of ax are completely independent, and can be computed in parallel where multiple GPUs are available.
ät Aλ
3.4 Comparison to other methods
We compare EvoGrad to the most related and widely-used alternative 7i - T2 20 in Table Ti T2 requires higher-order gradients and associated longer computational graphs due the need to backpropagate through gradient nodes. This leads to increased memory and time cost compared to vanilla backpropagation. In contrast, EvoGrad requires no higher-order gradients, large matrices, and no substantial expansion of the computational graph.
Table I: Comparison of hypergradient approximations of Ti -T2 and EvoGrad.
Method
Hypergradient approximation
T T2 22
aty
aby
&r
ax
0e
20ax
EvoGrad (ours)
My
00
The de
DE OX
afv DX
Myy. 00
x
E softmaxí( F))
8x
We analyse the asymptotic big-O time and memory requirements of EvoGrad vs Ti Tr in Table 2 The dominant cost in terms of both memory and time is the cost of backpropagation. Backpropagation is significantly more expensive than forward propagation because forward propagation does not need Note that even if EvoGrad keeps multiple copies of the model weights in memory, this cost is small compared to the cost of backpropagation. and the latter is done with only one of weights 0*, We remark that our main empirical results are obtained with only K = 2 models, so we can safely ignore this in our asymptotic analysis.
to store all intermediate variables in memory (25 g
In addition, elaborate how higher-order gradients contribute to increased memory and time costs. Results from computing the first-order gradients are added into the computation graph as new nodes in the graph that higher-order gradients, we backpropagate through this longer computational graph, which directly increases the memory and time costs. The current techniques (20 rely longer computational graphs, while EvoGrad significantly shortens the graph and reduces memory cost by avoiding this step. This consideration is not visible in the big-O analysis, but contributes to improved efficiency.
calculate the higher-order gradients. When calculating the
Table 2: Comparison of asymptotic memory and operation requirements of EvoGrad and Ti T2 meta-learning strategies. P is the number of model parameters, H is the number of hyperparameters. K < H is the number of model copies in EvoGrad. Note this is first-principles analysis, so the time requirements are different when using c.g. mode backpropagation that uses parallelization.
Method
Time requirements
Memory requirements
Ti Ta 20
O(PH)
O H))
EvoGrad (ours)
OKP+
O(P+M) H)
4 Experiments
We first consider two simple problems: 1) a 1-dimensional problem where we try to find the minimum of a function, and 2) meta-learning feature-transformer to find the rotation that correctly aligns images whose training and validation sets differ in rotation. This serves as proof-of-concee problem to show our method is capable of meta-learning suitable hyperparameters. We then consider three real problems where meta-learning has been used to solve different learning challenges. We show that EvoGrad makes significant impact in terms of reducing the memory and time costs (while keeping the accuracy improvements brought by meta-learning
3) Cross-domain few-shot classification via learned feature-wise transformation (33, 4) Meta- Weight-Net: learning an explicit mapping for sample weighting [31]. 5) MetaXL: meta representation transformation for low-resource cross-lingual learning 35). We provide a brief overview of each problem, together with evaluation and analysis. Further details and experimental settings are described in the supplementary material.
5

---

4.1 Illustration using a 1-dimensional problem
In this problem we minimize a validation loss function (v) = (x 0.5)² where parameter is optimized using SGID with training loss function fr(2) (xr 1)2 + >|<=|1/ that includes a meta-parameter X. A closed-form solution for the hypergradient is available and is equal to g(A) (A-1)/(A 1)², which allows to compare EvoGrad against the ground-truth gradient.
Our first analysis studies the estimated EvoGrad hypergradient for grid of À values between and 2. For each value of À we show the mean and standard deviation of the estimated 0fv//0 over 100 repetitions (with random choice of z)). We use temperature 7 = 0.5, c E between 2 and 100 models copies in the population. The results in Figure 2 show that EvoGrad estimates have a similar trend to the ground-trun toound-truth gradient, even if the EvoGrad estimates are noisy. The level of noise decreases with more models in the population, but the correct trend is visible even we use only 2 models.
_NO,, 1) and consider
Our second analysis studies the trajectories that parameters À follow if they are both optimized online using SGD with learning rate of 0. for 5 steps, starting from five different positions (circles). The hypergradients
either estimated using EvoGrad or directly using the ground-truth formula. Figure Ssgure]shows shows that the trajectories both variations are similar, and they become more similar as we more models in the population. In all cases the parameters converge towards the region where the validation loss is the lowest at x 0.5.
Population:
Population: 10
Population: 100
1,00
1.02
1.50
Hypargrad
0.25
0.25
0.25
-0.25
-0.25
-0.25
-0.0
-8.50
Ground truth
0.75
-0.55
EvoGrad
0.25
0.50
0.75
1.00
1.25
3.50
135
2.00
0.25
0.50
0.70
1.00
1.25
1.30
1.75
2.50
0.23
0.30
0.75
1.50
1.5
150
1.72
200
Figure 2: Comparison of the hypergradient /0A estimated by EvoGrad vs the ground-truth
Population:
Population: 10
Population:
1.5
1.5
1.5-
1.0
1.0
1.0
0.0
0.0
0.0
-2
Figure 3: Trajectories of parameters x; when following 0fr and 05v/0A using SGD for random starting positions. Comparison of trajectories using EvoGrad estimated (blue) ground-truth (red) hypergradient. The initial position is marked with a circle, and the final position after steps is marked with a cross. The shading is validation loss fv (z).
4.2 Rotation transformation
In this task we work with MNIST images 12 and assume that the validation and test sets are rotated by 30 compared to the conventionally oriented training images. Clearly, directly training a model and applying it will lead to low performance. We therefore assume meta-knowledge in the form of a hidden rotation. The rotation transformation applied to the training images before learning, and should itself be meta-learned by the validation loss obtained by the CNN trained on the rotated training set. Thus solving the meta-learning problem should result 30 rotation, and base CNN that generalises to the rotated validation set.

---

The problem is framed as online meta-learning where each update of the base model is followed by meta- peta-par update using EvoGrad. We use EvoGrad with 2 model copies, temperature and Ø- = 0.001 for ~ osign(NV( I)). Our LeNet [33 base model is trained for 5 epochs.
0.05
We repeat the experiments times and show a comparison of the results in Table 3 A baseline model achieves 98.40 + 0.07% accuracy if the test images are not rotated, but its accuracy drops to 81.79 + 0.64% the same images are rotated by 30". A model trained with EvoGrad and the rotation transformer is able to accurately classify rotated images, with a similar accuracy as the baseline model can classify unrotated images. This confirms we can successfully optimize hyperparameters with EvoGrad. The meta- leamnee rotation is also close to the true value.
Table 3: Rotation transformation learning. The goal is to accurately classify MNIST test images rotated by 30 degrees compared to the training set orientation. Test accuracies (%) of a baseline model, and one whose training set has been rotated by the EvoGrad's ' meta-learned rotation, and associated EvoGrad rotation estimate (*). Accuracy for rotation matched train/test sets is 48.40%
True Rotation
Baseline Acc.
EvoGrad Acc.
EvoGrad Rotation Est.
30
81.79 + 0.64 98.11 + 0.32
28.47 + 5,23
4.3
Cross-domain few-shot classification via learned feature-wise transformation
As the next task we consider cross-domain few-shot classification (CD-FSL), CD-FSL is considered an important and highly challenging problem at the forefront of computer vision. The state-of-the-ar approach learned feature-wiss transformation (LFT) (33) aims to meta-learn stochastic feature- wise transformation layers that regularize metric-based few-shot learners to improve their few-shot learning generalisation in cross-domain conditions. The method includes two key steps: 1) updating on pseudo-seen task and 2) updating the meta- perta-prsssseess
the model with the meta-parameel by evaluating the model on a pseudo-unseen task by backpropagating through the first step. As feature-wise transformation is not directly used for the pseudo-unseen unseen task, this leads to higher-order gradients. Note that the problem itself is memory-intensive because we work with larger images of size 224 x 224 within episodic learning tasks. As a result, a significantly more efficient meta-learning approach could allow us to scale from the ResNet10 model used in the paper to a larger model.
We experiment with the LFT-RelationNet 32 metric-based few-shot learner and consider the exact same experiment settings as (33] using the official PyTorch implementation associated with the paper. LFT introduces 3712 hyper- parameters to train for ResNet10, and 9344 for ResNet34. All our experiments are conducted on Titan GPUs with 12GB of memory using K for EvoGrad.
Table shows the baseline performance of vanilla unregularised ResNet (-), manually tuned FT layers (FT), FT layers meta-learned by second-order gradient (LFT) and by EvoGrad. The results show that EvoGrad matches the accuracy of the original LFT approach, leading to clear accuracy improvements over training with no feature-wise transformation training with fixed feature-wiss parameters selected manually. At the same time EvoGrad is significantly more efficient in terms of the memory and time costs as shown in Figure 4 The memory improvements from EvoGrad allow us to scale the base feature extractor to ResNet34 within the standard 12GB GPU. The original LFT with its Ti T2 style second-order algorithm cannot be extended in the available memory if we keep the same settings of the few-shot learning tasks. Thus, we are able to improve state-of- ofte-of-theearr accuracy on both 5-way 1 and 5-shot tasks. For ResNet34, we include baselines without any feature- reature-wise transformation and with manually chosen feature-wist wise transformation to confirm the benefit of meta-learning
4.4 Label noise with Meta- Meta-Weighte Weight-Net
We consider further highly practical real problem where online meta-learning has led to significant improvements learning from noisy labelled data. The Meta- Weight-Net framework trains an auxiliary neural network that performs instance-wise loss we-whtitting on the training set [31]. The base model is updated using the sum of weighted instance-wise losses for noisy data, while the Meta- Weita-eiggtt-eet itself is updated by evaluating the updated model on clean validation data and by backpropagating through the model update. We use the official implementation of the approach 31 and follow the same experimental settings, using K = 2 for EvoGrad.
7

---

Table 4: RelationNet test accuracies (%) and 95% confidence intervals across test tasks on various unseen datasets. LFT EvoGrad can scale ResNet34 on all tasks within 12GB GPU memory, while vanilla second-order LFT Tj T2 cannot. We also report the results of our own rerun of the LFT approach using the official code denoted as our run. EvoGrad can clearly match the accuracies obtained by the original approach that uses T T2-
Model
Approach
CUB
Cars
Places
Plantac
44.33 0.59
29.53
0.45
47.76
+ 0.63
33.76 0.52
FT
44.67
0.58
30.38 + 0.47
48.40
0.64
35.40 0.53
5 Dawa 1-shot
ResNet10
LFT Ti T2 LFT Ti T- (our run)
48.38 + 0.63
32.21
50.74
0.66
35.00 + 0.52
46.03 + 0.60
+
49.29
+ 0.65
36.34 + 0.59
LFT EvoGrad
50.70
36.00 0.56
4..6
0.46
0.66
35.03 0.54
ResNet34
FT
45.15 0.59
30.28 + 0.44
49.96
35.69 0.54
LFT EvoGrad
45.97 + 0.60
33.21 + 0.54
50.76
38.23 + 0.58
62.13 0.74
40.64 +
40.64 0.54
46.29 0.57 0.56
FT
63.64 0.77
42.24 0.57
0.54
47.81 + 0.51
Saw 5-shot
ResNet10
LFT 7i T3 LFT Ti LFT EvoGrad
64.99 0.54
43,44 0.59
6.53
50.39 0.52
T3 (our run)
65.94 0.56
43.88
0.57
51.43 0.55
64.63 + 0.56
42.64 0.58
66.54
52.92 0.57
1 6
40.50
6.59
50.20 1 0.55
ResNet34
FT
62.48 1 0.56
41.06 1 0.52
50.08 1 0.55
LFT EvoGrad
66.40 = 0.56
44.25 = 0.55
67.23 + 0.56
52.47 + 0.56
300
LFT EvoGrad
Memory usage (GB)
10
250
LFT T1 T2
Time per epoch
254.7
8.3
150
167.3
5.3
115.88
4.8
5-way
5-way
5-way
5-way
Figure 4: Cross- few-shot learning with LFT [3]: analysis of memory and time efficiency of EvoGrad vs standard second-order Ti T- approach. Mean and standard deviation reported across experiments with different test datasets. EvoGrad significantly more efficient in terms of both memory usage and time per epoch.
Our results in Table coonnfirm replicate the benefits of training with Meta- Meta-Weight-Nee, clearly surpassing the accuracy of the baseline when there is label noise. We also note that EvoGrad can improve the accuracy over the Ti Ta-based approach because the two approaches distinct and provide different estimates of the true hypergradient. Figure 5 shows that our method leads significant improvements in memory and time costs (over half of the memory saved and the runtime is improved by about a third).
Table Test accuracies (%) for Meta- Weight-Net label noise experiments with ResNet-32 means and standard deviations across 5 repetitions for the original second-order algorithm vs EvoGrad. EvoGrad is able to match or even exceed the accuracies obtained by the original MWN approach.
Dataset
Noise rate
Baseline
MWN 7i Ta
MWN Ti T2 (our run)
MWN EvoGrad
0%
92.89 + 0.32
92.04 + 0.15
91.10 + 0.19
92.02 + 0.31
CIFAR-10
20%
76.83 + 2.30
90.33 = 0.61
89.31 = 0.40
89.86 + 0.64
40%
70.77 1 2.31
87.54 1 0.23
85.90 - 0.45
87.74 1 0.54
0%
70.50 + 0.12
70.11 + 0.33
68.42 = 0.36
69.16 = 0.49
CIFAR- 100
20%
50.86 + 0.27
64.22 = 0.28
63.43 0.43
64.05 = 0.63
40%
43.01 + 1.16
58.64 + 0.47
56.54 0.90
57.44 + 1.25

---

2.00
203
MWN EvoGrad
Memory usage (GB)
Time per epoch (s)
MWN T1 T-
1:52
150
1.35
195
2.00
100
144.9
2.75-
75-
142.0
98.7
92.9
0.4
CIFAR-10
CIFAR-100
CIFAR-10
CIFAR-100
Figure 5: Analysis of memory and time cost MWN EvoGrad the original second-order MWN, showing significant efficiency improvements by EvoGrad. Mean and standard deviation is reported across 5 repetitions 40% label noise problem.
4.5 Low-resource cross-lingual learning with MetaXL
The previous two real applications of meta-learning considered computer vision problems. To highlight EvoGrad is general method that can make an impact in any domain, we also demonstrate its benefits on meta-learning application from NLP. More specifically, we use EvoGrad for MetaXI. [35, which meta-learns meta representation transformation better transfer from source languages to low-resource target languages.
We have selected the named entity recognition (NER) task with English source language (WikiAnn dataset (24)), which is one of the key experiments in the MetaXL paper (35). Table shows EvoGrad matches and in fact surpasses the average F1 score of MetaXL with the original
T2 meta- learning method. Figure shows EvoGrad significantly improves both memory and time consumption compared to MetaXL 7i Tz. Overall these results confirm EvoGrad is suitable for meta-learning in various domains, including both computer vision and NLP.
Table 6: Test Fl score in % for named entity recognition task. English source language. The first two rows are taken from the MetaXL paper, while our own runs are in the following rows. EvoGrad clearly matches and even surpasses the performance of Ti T2 baseline. Joint-training (JT) represents simple non-meta-learning baseline approach.
Method
qu
edo
ilo
xmf
mhr
gn
Average
JT
66.10
55.83
80.77
69.32
71.11
82.29
61.61
65.44
69,06
MetaXL Ti Tg
68.67
55.97
77.57
73.73
68.16
88.56
66.99
69.37
71.13
(our run)
59.75
49.19
79.43
68.85
68.42
89.94
61.90
69.44
68.37
MetaXL Ti T2 (our run)
65.29
56.33
76.50
67.24
71.17
89.41
66.67
64.11
69.59
MetaXL EvoGrad
71.00
57.02
85.99
70.40
65.45
88.12
66.97
70.91
71.98
Merooy usage (GB)
13.0
MetaXL EvoGrad
17.1
Time per epach
MetaXL T, TT
10.0
12.5
3.0
Figure 6: Analysis of memory and time cost of MetaXL EvoGrad the original second-order MetaXL, in the context of a simple foint-training (JT) baseline. EvoGrad consumes significantly less memory than Tj T3 and is faster. Mean and standard deviation is calculated over the different target languages.

---

4.6
Scalability analysis
We use the Meta- Seeta-Weigtt-ee benchmark study how the number of model parameters affects the memory usage and training time of EvoGrad, comparing it to the standard second-orde Ti T₃ approach. We vary model size by changing the number of filters in the original ResNet32 model, multiplying the filter number ..,..., xo. The smallest model had around 0.5M parameters and the largest one around 1 11M parameters.
The results in Figure 7 show our EvoGrad leads to significantly lower training time and memory usage, and that the margin over the standard second-order optimizer grows as the model becomes larger. Further, we have analysed the impact of modifying the number of hyperparameters from 300 up to 30,000. The impact on memory and time was negligible, and both remained roughly constant, which is caused by the main model being significantly larger. It is also because of the fact that reverse-mode differentiation costs scale with the number of model parameters rather than hyperparameters [22] recall that backpropagation is the main driver of memory and time costs [25]. Moreover, we have done experiments that varied the number of model copies in EvoGrad. The results showed the training time per epoch increased slightly, while the memory costs remained similar.
700
MWN EvoGrad
Memory usage (GB)
Time epoch (s)
600
MWN T2
500
3-
400
2-
300-
100
0.8
100
0.0
0.2
Number of model parameters
Number of model parameters
Figure 7: Memory and time scaling of MWN EvoGrad vs original second-order Meta- Weight-Net. Efficiency margins of EvoGrad are larger for larger models.
5 Discussion and limitations
Similar to many other gradient- based meta-learning methods, our method is greedy as considers only the current state of the model when updating the hyperparameters rather than the whole training process. However, this greediness allows the method to be used larger-scale settings where we train the hyperparameters and the model jointly. Further, our method approximates the hypergradient stochastically. While results were good for the suite of problems considered here using only = 2, the gradient estimates may be too noisy in other applications. This could lead poor outcomes which could be a problem in socially important applications. Alternatively, it may necessitate using a larger model population (Figure 2. While as observed in Section 3.3 the candidate models be trivially parallelized to scale population size, this still imposes a larger energy cost (28). Another limitation is that similarly to IFT-based estimators [19], EvoGrad is not suitable for optimizing learner hyperparameters such as learning rate. Currently we have used the simplest possible evolutionary update in the inner loop, and upgrading EvoGrad to state-of-the-are evolutionary strategy may lead to better gradient estimates and improve results further.
6 Conclusions
We have proposed a new efficient method for meta-learning that allows us scale gradient- based meta-learning to bigger models and problems. We have evaluated the method on a variety of problems, most notably meta-learning feature-wise transformation layers, training with noisy labels using Meta- Weight-Net, and meta-learning meta representation transformation for low-resource cross-lingual learning. In all cases we have shown significant time and memory efficiency improvements, while achieving similar or better performance compared to the existing meta-learning methods.
10

---

Acknowledgments and Disclosure of Funding
This work was supported in part by the EPSRC Centre for Doctoral Training in Data Science, funded by the UK Engineering and Physical Sciences Research Council (grant EP/L016427/1) and the University of Edinburgh.
References
[1] Antoniou, A., Edwards, H., and Storkey, A. (2019). How train your MAML. In ICLR.
[2] Balaji, Y., Sankaranarayanan, S., and Chellappa, (2018). MetaReg: towards domain general- ization using metu-regullarization In NeurlPS.
[3] Baydin, A. G., Pearlmutter, B. A., and Siskind, J. M. (2018). Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research, 18:1-43.
[4] Bengio, Y. (2000). Gradient-based optimization of hyperparameters. Neural Computation, 12(8):1889-199)
[5] Bohdal, O., Yang, Y., and Hospedales, T. (2020). Flexible dataset distillation: learn labels instead of images. In NeurlPS MetaLearn 2020 workshop.
[6] Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., and Le, Q. V. (2019). Autoaugment: learning augmentation policies from data. CVPR.
[7] Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In ICML.
[8] Griewank, A. (1993). Some bounds on the complexity of gradients, Jacobians, and Hessians. In Complexity in Numerical Optimization, pages 128-162.
[9] Hospedales, T. M., Antoniou, A., Micaelli, P., and Storkey, J. (2021). Meta-learning in neural networks: survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1P:1-1.
[10] Jaderberg, M., Czarnecki, W. M., Dunning. I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C.. Rabinowitz, N. C., Morcos, A. S., Ruderman, A., Sonnerat, N., Green, T., Deason, L.. Leibo, J. Z., Silver, D., Hassabis, D., Kavukcuoglu, K., and Graepel, T. (2019). Human-level performance in 3d multiplayer games with population-hanl reinforcement learning. Science, 364((444):8599-88)
[11] Larsen, J., Hansen, L. K., Svarer, C., and Ohisson, M. (1996). Design and regularization of neural networks: the optimal use of a validation set. In Neural Networks for Signal Processing Proceedings of the IEEE Workshop, pages 62-71. IEEE.
[12] LeCun, Y., Cortes, C., and Burges, C. (1998). MNIST handwritten digit database.
[13] LeCun, Y., Jackel, L. D.,, Boser, B., Denker, J. S., Graf, H. P., Guyon, I., Henderson, D., Howard, R. E., and Hubbard, W. (1989). Handwritten digit recognition: applications of neural network chips and automatic learning. IEEE Communications Magazine, 27(11):41-44
[14] Li, Y., Yang, Y., Zhou, W., and Hospedales, T. M. (2019). Feature-critic networks for heteroge- neous domain generalization. ICML.
[15] Li, Z., Zhou, F., Chen, F., and Li, H. (2017). Meta-SGD: learning to learn quickly for few-shot learning. In arXiv. arXiv.
[16] Liu, H., Simonyan, K., and Yang, Y. (2019a). DARTS: differentiable architecture search. In ICLR.
[17) Liu, S., Davison, A. J., and Johns, E. (2019b). Self-supervised generalisation with meta auxiliary learning. In NewrIPS.
[18] Lorraine, J. and Duvenaud, D. (2018). Stochastic hyperparameter optimization through hyper- networks. In arXiv.

---

[19] Lorraine, J., Vicol, P., and Duvenaud, D. (2020). Optimizing millions of hyperparameters by implicit differentiation. In AISTATS.
[20] Luketina, J., Berglund, M., Klaus Greff, A., and Raiko, (2016). Scalable gradient-based tuning of continuous regularization hyperparameters. In ICML.
[21] Maclaurin, D., Duvenaud, D., and Adams, R. P. (2015). Gradient-based hyperparameter optimization through reversible learning. In ICML.
[22] Micaelli, P. and Storkey, A. (2020). Non-greedy gradient-based hyperparameter optimization over long horizons. arXiv.
[23] Nichol, A., Achiam, J., and Schulman, (2018). On first-order meta-learning algorithms. In arXiv.
[24] Pan, X., Zhang, B., May, J., Nothman, J., Knight, K., and Ji, H. (2017). Cross-lingual name tagging and linking for 282 languages. ACL.
[25] Rajeswaran, A., Finn, C., Kakade, S., and Levine, S. (2019). Meta-learning with implicit gradients. In NeurIPS.
[26] Ren, M., Zeng, W., Yang, B., and Urtasun, R. (2018). Learning reweight examples for robust deep learning. In ICML, volume 10.
[27] Salimans, T., Ho, J.. Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution strategies as scalable alternative to reinforcement learning. In arXiv.
[28] Schwartz, R., Dodge, J., Smith, N. A., and Etzioni, O. (2019). Green AI. arXiv.
[29] Shaban, A., Cheng, C.-A., Hatch, N., and Boots, B. (2019). Truncated back-propagation for bilevel optimization. In AISTATS.
[30] Shan, S., Li, Y., and Oliva, B. (2020). Meta-Neighhorhoods In NeurlPS.
[3]] Shu, J., Xie, Q., Yi, L., Zhao, Q., Zhou, S., Xu, Z., and Meng. (2019). Meta- Sita-Weight-Ner learning an explicit mapping for sample weighting. In NeurIPS.
[32] Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., and Hospedales, T. M. (2018). Learning to compare: relation network for few-shot learning. CVPR.
[33] Tseng, H.-Y, Lee, H.-Y., Huang, J.-B., and Yang, M.-H. (2020). Cross-domain few-shot classification via learned feature-wis transformation. In ICLR.
[34] Wang, T,, Zhu, J.-Y., Torralba, A., and Efros, A. A. (2018). Dataset distillation. In arXiv.
[35] Xia, M., Zheng, G., Mukherjee, S., Shokouhi, M., Neubig, G., and Awadallah, A. H. (2021). MetaXL: meta representation transformation for low-resource cross-lingual learning. In NAACL.
[36] Zhou, P., Yuan, X.-T., Xu, H., Yan, S., and Feng. J. (2019). Efficient meta learning via minibatch proximal update. NeurIPS.
[37] Zoph, B. and Le, Q V. (2017). Neural architecture search with reinforcement learning. In ICLR.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] These are discussed as part of a
section for discussion and limitations.
(c) Did you discuss any potential negative societal impacts your work? [Yes] These are discussed as part of section for discussion and limitations.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] Code and the instructions part of the supplemental material.
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] A short summary is provided in the main paper with further details in the supplemental material.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] All assets are available for free, and they are the same as the cited papers use.
(c) Did you include any
assets either in the supplemental material URL? [Yes]
We provide code for EvoGrad in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you'le 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "dwPschLPmS", "reviewer": "Reviewer_xeM1", "summary": "This paper presents EvoGrad, a technique that replaces the inner loop of meta learning with an evolutionary technique to reduce the cost of meta learning - allowing meta learning to be applied to larger models. The paper demonstrates the EvoGrad can allow scaling up LFT from ResNet18 to ResNet34 and reduces the computation cost on MWN models.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I vote to accept the paper.\n\nThe paper is clear and concise, the idea is neat and novel, the details are presented clearly and the experiments evaluate the efficacy of the method.\n\nOriginality: The idea of doing away with second-order gradients using a simple evolutionary technique in the inner loop is novel and, to the best of my knowledge hasn't been tried before. Second-order gradients are indeed responsible for the bottle neck in computation costs and getting rid of this computation entirely while retaining the accuracy is a pretty neat result.\n\nQuality: The main contribution of EvoGrad is retention of performance and reduction in computation costs. These claims are substantiated as shown in Figures 4 and 5 and Tables 4 and 5. Moreover, the authors were able to scale up LFT to ResNet34.\n\nAs the authors mention, the hyper gradient estimate could be much worse on a different suite of problems. An excellent version of this paper would have evaluated the technique on more meta learning tasks. Alternatively, the paper could evaluate EvodGrad on task that is unrelated to computer vision to demonstrate that the hyper gradient approximation generalizes across domains.\n\nSignificance: If the hyper gradient approximation generalizes well to other meta learning tasks, the community will benefit from EvoGrad in that they will be able to use meta learning on larger models - an important contribution.\n\nA minor comment on presentation - Figure 3 is not very readable - at first glance it wasn't clear to me if the three pictures are even different.\n\nTypo in line 70 \n> or even a a synthetic training", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As described in the paper - the accuracy of the hyper gradient approximation could depend on the domain EvoGrad is applied to -  in the paper, all experimental tasks were related to image based data.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "XjKzJX3vB1y", "reviewer": "Reviewer_ULem", "summary": "The paper proposes to replace gradient descent with evolutionary search in the inner loop of meta-learning. This makes the computation of the outer loop first-order optimization, which saves compute and memory compared to second-order optimization.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The writing is very clear, and the problem the paper is solving is also well-motivated: the cost of second-order optimization required in meta-learning makes it prohibitive for many real-world applications.\n\nThe idea is original, but the experimental results in the paper do not convince the reader about its significance. The authors mention that the so-called T1-T2 approach is often used as a first-order approximation to calculating the second-order meta gradients, and remark that EvoGrad is asymptotically more efficient. While it might be more efficient to take a single EvoGrad step compared to a gradient step, we might need a lot more EvoGrad steps compared to gradient steps (in fact, there are also classical theoretical results to this effect [1]). This means that in practice, it is likely that EvoGrad is not as efficient as T1 - T2.\n\nThe authors should show experimental comparisons with T1 - T2 to convince the reader that EvoGrad is more efficient. I would also recommend benchmarking on standard meta-learning datasets like OmniGlot or MiniImageNet to increase the significance of the results.\n\nEdit: In my original review, I did not understand that the authors were targeting their work specifically for settings where first-order gradient approximations do not exist, e.g. hyperparameter optimization. This is quite different from experimental benchmarks on OmniGlot/MiniImageNet where MAML-like algorithms are commonly being evaluated on. At my request, the authors added in experiments that show that their evolutionary algorithm based approach significantly outperforms a gradient based approach like a linear hypernet. I'm changing my score to a 6 to reflect this.\n\n[1]: Query Complexity of Derivative-Free Optimization. Kevin G. Jamieson, Robert Nowak, Ben Recht.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please see the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "ARZI0x0OIXo", "reviewer": "Reviewer_7RkH", "summary": "The authors introduce EvoGrad, a method for approximating second order gradients for use in Meta-Learning applications. The method approximates the inner loop of optimization with \"evolutionary\" update. The authors show their method is more computationally efficient than the T1-T2 method, yields good approximations to the second order gradients and is able to converge in the toy setting, and achieves better results than MetaWeightNet, a second order method, on a cross-domain few-shot classification task. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main idea of the paper is not that technically novel but the paper is well written and easy to understand  -- explanations are clear and straight-forward.. The evolutionary update consists of a weighted sum of randomly selected perturbations of the current parameter values and reminds me of the finite differences approximation, although not an approximation to the first order gradient in this case. Instead, the first order gradient is removed from the update procedure so that optimizing hyperparameters does not require second order gradients. The weights for the parameter sum are selected based on a softmax procedure using their individual losses giving more weight to better performing parameter sets. \n\nI found the computational arguments consisting of time requirements, memory requirements and scalability convincing. \n\nI think I am misunderstanding something about the plot in figure 2: the red curve represents the ground truth hypergradient but the curve does not match the closed form hypergradient function defined on line 192.\n\nThe experiments analyzed in sections 4.1 and 4.2 use bounded hypergradients. Figure 2 seems to indicate that as lambda gets smaller the error between the approximation and the true hypergradient gets larger but, for this example, it's not too much of an issue since lambda is bounded below. Could we see divergence if the approximation error gets too large in unbounded cases? \n\nI found the experiment section was good but there are still a few questions left unanswered in my opinion:\n- The inner-loop is replaced only for the hyperparameter gradient. How good is the model that is fully trained using the \"evolutionary update\", ie. actually uses that inner-loop during training?\n\n- How well does this model perform to the \"first-order\" approximations used by MAML and many other MAML-based models where higher order gradients are ignored via a stop-gradient procedure in the automatic differentiation software? Depending on which hyperparameter you are optimizing, this procedure would have different interpretations but would be easy to implement in all cases.\n\n- How well does this compare to IFT methods? The authors mentioned that IFT methods assume that the inner-loop is run until convergence and that \"this makes them unsuitable for the majority of practical applications above where training the inner loop to convergence for each hypergradient step is infeasible.\" However, without more explanation I don't see why this would be the case especially since EvoGrad uses an inner-loop approximation as well. Also, the authors state that in IFT methods, the \"costs come from the associated overhead with approximation an inverse Hessian\" but that this Hessian doesn't have to be stored in memory. However, what is required is the computation of Hessian-vector products which is a much cheaper computation.\n\n-  In section 4.4, why does EvoGrad have better performance in terms of accuracy compared to MetaWeightNet? Isn't EvoGrad designed to be an approximation to the second order method used in MetaWeightNet? Is there something about the inductive bias in the approximation that would yield better results?\n\nDue to these questions, I think this paper is marginally above the threshold of acceptance but I look forward to reading the opinions of the other reviewers and the response from the authors.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, there is a good paragraph addressing limitations and societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "6GEG1S6mO4D", "reviewer": "Reviewer_p6R1", "summary": "The paper presents an efficient gradient-based meta-learning algorithm based on evolutionary techniques. The proposed algorithm doesn’t require second order gradient computation, and thus, is more efficient. The proposed approach was evaluated on two meta-learning tasks for few-shot feature-wise transformation and noisy label learning, on which time and memory efficiencies were demonstrated.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": " # Originality\n\n- The idea of using evolutionary methods to eliminate the need for higher-order gradients is novel.\n- The related work is adequately cited, the proposed algorithm differs from the prior work in being more efficient in runtime and computation. \n\n# Quality\n\n- The submission is technically sound and the claims are supported by empirical results, but could be limited (please see below).\n- The approach lacks the theoretical analysis for its effectiveness. \n- The empirical analysis is limited to mostly computer vision applications.\n- The proposed approach is appropriate for the studied applications, however, it’s not clear if the variance in the estimates for the meta-gradient would significantly affect the results for other applications.\n- The authors are open about the limitations for their approach and discuss these challenges in the discussion section.\n\n# Clarity\n\n- The paper is clear, well organized, and easy to follow.\n\n# Significance\n\n- The work is significant as it aims at scaling meta-learning algorithms via using an evolutionary procedure to estimate meta-gradients without the need for computing higher order gradients. \n- However, this estimate could be noisy and suffer from high variance (see Figure 2), it’s not clear how to mitigate this issue without increasing the population size leading to strong hits for the energy requirements for the proposed approach.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- The estimates for the meta-gradients could be noisy and suffer from high variance, it’s not clear how to solve this issue without increasing the population size which comes at an increased energy consumption, thus, the proposed approach presents a tradeoff between computing higher order derivatives, and possibly increasing the required memory and energy requirement.\n - As the authors discuss in the discussion section, the proposed approach is greedy and global convergence could not be guaranteed.\n- The paper lacks theoretical analysis to the proposed approach.\n- The empirical results are mostly limited to the computer vision domain, it’s not clear if the approach scales to larger models from the language and speech learning domains.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers agree on acceptance. This is a clear decision. The authors should take into account the reviewers' feedback to improve the paper for the camera-ready submission.
Decision: Accept (Poster)