Title: Meta-AdaM: An Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Few shot learning

Meta Learning
Abstract: We introduce Meta-AdaM, a meta-learned adaptive optimizer with momentum, designed for few-shot learning tasks that pose significant challenges to deep learning models due to the limited number of labeled examples. Meta-learning has been successfully employed to address these challenges by transferring meta-learned prior knowledge to new tasks. Most existing works focus on meta-learning an optimal model initialization or an adaptive learning rate learner for rapid convergence. However, these approaches either neglect to consider weight-update history for the adaptive learning rate learner or fail to effectively integrate momentum for fast convergence, as seen in many-shot learning settings. To tackle these limitations, we propose a meta-learned learning rate learner that utilizes weight-update history as input to predict more appropriate learning rates for rapid convergence. Furthermore, for the first time, our approach incorporates momentum into the optimization process of few-shot learning via a double look-ahead mechanism, enabling rapid convergence similar to many-shot settings. Extensive experimental results on benchmark datasets demonstrate the effectiveness of the proposed Meta-AdaM.

Meta-AdaM: A Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning

Abstract

We introduce Meta-AdaM, a meta-learned adaptive optimizer with momentum, designed for few-shot learning tasks that pose significant challenges to deep learning models due to the limited number of labeled examples. Meta-learning has been successfully employed to address these challenges by transferring meta-learned prior knowledge to new tasks. Most existing works focus on meta-learning an optimal model initialization or an adaptive learning rate learner for rapid convergence. However, these approaches either neglect to consider weight-update history for the adaptive learning rate learner or fail to effectively integrate momentum for fast convergence, as seen in many-shot learning settings. To tackle these limitations, we propose a meta-learned learning rate learner that utilizes weight-update history as input to predict more appropriate learning rates for rapid convergence. Furthermore, for the first time, our approach incorporates momentum into the optimization process of few-shot learning via a double look-ahead mechanism, enabling rapid convergence similar to many-shot settings. Extensive experimental results on benchmark datasets demonstrate the effectiveness of the proposed Meta-AdaM.

1 Introduction

Deep learning has demonstrated its capability in solving various challenging tasks in many-shotlearning settings, where each classification class contains sufficient training examples [12]. However, its performance is hindered on small datasets or under few-shot-learning (FSL) settings [37]. In FSL, unseen tasks with limited training examples pose significant challenges for optimizing deep learning models [45]. With limited training data, deep learning models may suffer from severe underfitting or overfitting problems [28]. Numerous meta-learning methods have been proposed to address this challenge by identifying and encoding shared knowledge among different data distributions within the model. With prior generalized knowledge, a meta-learning model can be rapidly transferred to a new task by fine-tuning it with limited unseen data [13].

Existing optimization-based meta-learning methods for FSL facilitate this transfer process by finetuning a meta-learned model on new few-shot tasks. There are two primary directions for using meta-learning in FSL tasks: learning an optimal initialization and generating high-quality gradient updates for rapid convergence. In the first direction, most methods [7, 8, 29, 24, 17, 18] employ an inner loop to fine-tune the meta-model on specific tasks and iteratively optimize the meta-model in an outer loop based on the adapted model from the inner loop. The resulting initial model weights enable the adapted model to converge more quickly to an optimal solution for a new task. In the second direction, some works [38, 2, 46] focus on generating high-quality and task-adaptive gradient updates, allowing the optimizer to rapidly converge to an appropriate solution.

Our work focuses on the second direction. Most existing works in this direction aim to use adaptive learning rates by considering weights and gradients [18, 30, 3]. However, it has been shown that the weight-update history is more important than the weights themselves for determining an appropriate learning rate. Additionally, existing works often fail to incorporate momentum in FSL, which has been shown to be effective in accelerating the optimization process in many-shot settings. This is due to the fact that momentum typically exhibits high fluctuations during the initial updates [19]. Another challenge for the meta-trained base model is the incorporation of prior knowledge. Certain classes exhibit features that align more closely with prior knowledge than others, leading to an imbalance in predictions generated by the model.

To address these challenges, we propose a meta-learned adaptive optimizer with momentum (MetaAdaM). To learn a more accurate adaptive learning rate, our approach meta-learns a learning rate learner based on gradient update history instead of weights. The learned learning rate learner, along with the meta-learned model initialization, is applied when adapting to new tasks. Moreover, we introduce a dynamic class weight scheme for classification tasks, aimed at effectively balancing the loss distribution among various classes, which can help the model generate unbiased results.

2 Related Works

In this section, we introduce related works to our proposed approach.

2.1 Few-Shot Learning and Meta Learning

A formal definition of machine learning can be described as follows: a computer program can learn some experience E from a set of classes that belongs to task T with performance metric P if P increases with E on T measured by P [22]. Few-shot-learning problems are a subset of machine learning problems in which E only contains a limited number of samples [45]. Under this setting, training deep neural networks can be challenging. The empirical risk R generated from the limited experience E can differ greatly from the expected empirical risk. Meta-learning enhances the performance P by learning shared features from other data. The metric-based [36] and model-based meta-learning methods [35, 23, 21] rely on extra features or models to improve the few-shot learning capabilities. Recently, optimization-based meta-learning methods have obtained more attention for their strong generalization ability. The optimization-based methods reduce the meta-Learning problem into a bi-level optimization problem. The inner loop optimizes the base model on a certain task, and the outer loop optimizes the base model across several tasks to adjust the initial weight for quick adaption. Without introducing new elements, such a structure has the potential to adapt better to unseen data. The most representative optimization-based method is the MAML method [7]. Subsequent MAML variants [8, 29, 24, 17, 18, 34, 9] focus on optimizing the optimization process.

2.2 Gradient-Descent Based Optimizer

The optimizer plays a crucial role in enhancing the efficiency of the training process in deep learning. Most popular optimizers are based on the gradient descent algorithm [1]. To further improve training efficiency and increase convergence speed, existing optimizers, such as AdaM [14], often utilize the first and second orders of momentum. The first order of momentum, also known as momentum, is the accumulated sum of historical gradients during the training process, which can be considered a more effective update direction compared to gradients. The second order of momentum is used to control the step size for each parameter by estimating the update velocity for each parameter. Moreover, based on AdaM, other optimizers [14, 6, 19] introduce additional elements to better estimate the update direction and step size. Learned optimizers [2, 1, 43] utilize a trainable model to estimate the update direction and step size, which can better adapt to new tasks.

2.2.1 Optimizer for Few-Shot Learning

Meta-learning has shown a strong ability to solve few-shot learning problems. So, researchers also use meta-learning methods to train optimizers for few-shot learning problems. [18] propose an optimizer that works as the stochastic gradient descent. [30, 3] propose to use linear models to carefully estimate the learning rate for each parameter. [25] consider estimating curvature instead of gradients. However, these works only consider the gradient change in one step and ignore the

<latexi sh

1_b

64="F

D2ypIW3JXRUw

Z

B

>A

C

n

c

V

S

N

O

d9

Hr0

Tm

L

K

o

M

/

Q

P

Y

+

v

q

5

7

G

<latexi sh

1_b

64="m wzg3kM

pCX

I+/ZGuOE

o

>A

H

c

VDLS

N

FJ

rPU

d

W

R

n

T

y

9

Q

v

0

j

q

58

B

Y

7

<latexi sh

1_b

64="NTQ9u

S2y/jPrYMnD

qX

+8

o

>A

B

c

V

L

FJ3U

v

gE

WR

p

OZG

0

d

kz

w

5

f

m

H

C

7

I

K

<latexi sh

1_b

64="KduSQkF0

o

VnqXCw

/r

E

>A

B8

c

DL

gM

3U

2v jN

G

RTZ

7

Om

Y

zJH

9

P

W

5

p

y

I

f

+

<latexi sh

1_b

64="kmATZ7

HWYGS

B

Q2vXL

>

/n

c

VD

gM

F

3

rPU

Kq

J

y

fu

z

w

5I

o

9

+j

C

0

E

N

p

R

O

8

d

<latexi sh

1_b

64="0c wRIFqjnfGOVH

L8M

rXz

>A

C

D

S

N

J3U

2v

Ey

5

Zd

o

mk

B

Q

g

/

Y

7

p

+

K

u

9

W

P

T

<latexi sh

1_b

64="IL9

FC

Q

m

w

DdT

>A

B3

c

V

SgN

E

y

r

UY+

2

HoM

v

R

0

j

k

n

X

G

/

P

p

K

7

u

5

W

O

8

z

<latexi sh

1_b

64="I5MRCLZvAJK

oN

k

P

Q

y

c

>

B/3

VD

S

F

r

U

d

G

q

2

0

wm

H

j

u

T

X

O

8

f

+

W

9

n

E

p

g

Y

<latexi sh

1_b

64="

OmqC

c5f

R/A

+LEVU9H

3

>

B

n

D

S

N

FJ

2v yM

g

W

o

Y

Q

ZT

kw

z

0I

X

j

P

d7

p

8Kr

G

u

<latexi sh

1_b

64="KduSQkF0

o

VnqXCw

/r

E

>A

B8

c

DL

gM

3U

2v jN

G

RTZ

7

Om

Y

zJH

9

P

W

5

p

y

I

f

+

<latexi sh

1_b

64="kmATZ7

HWYGS

B

Q2vXL

>

/n

c

VD

gM

F

3

rPU

Kq

J

y

fu

z

w

5I

o

9

+j

C

0

E

N

p

R

O

8

d

<latexi sh

1_b

64="0c wRIFqjnfGOVH

L8M

rXz

>A

C

D

S

N

J3U

2v

Ey

5

Zd

o

mk

B

Q

g

/

Y

7

p

+

K

u

9

W

P

T

<latexi sh

1_b

64="H

ONgy qkjRm

ZCdp

85B

w

>A

M

c

V

EPW

fJS

0F

X

o

Q+G

7

U

K

r

/

3Y

D

z

n

u

2

L

T

v

I

9

<latexi sh

1_b

64="I5MRCLZvAJK

oN

k

P

Q

y

c

>

B/3

VD

S

F

r

U

d

G

q

2

0

wm

H

j

u

T

X

O

8

f

+

W

9

n

E

p

g

Y

<latexi sh

1_b

64="KduSQkF0

o

VnqXCw

/r

E

>A

B8

c

DL

gM

3U

2v jN

G

RTZ

7

Om

Y

zJH

9

P

W

5

p

y

I

f

+

<latexi sh

1_b

64="G7uZy

A

083T

2W

gL

vXnU

Q

>

c

VD

S

N

FJ

q

M

E

Ro

Y

C

kw

z

I

j

P

d

p

m

59

Kr

H

O

+

f

<latexi sh

1_b

64="kmATZ7

HWYGS

B

Q2vXL

>

/n

c

VD

gM

F

3

rPU

Kq

J

y

fu

z

w

5I

o

9

+j

C

0

E

N

p

R

O

8

d

<latexi sh

1_b

64="0c wRIFqjnfGOVH

L8M

rXz

>A

C

D

S

N

J3U

2v

Ey

5

Zd

o

mk

B

Q

g

/

Y

7

p

+

K

u

9

W

P

T

H

H

H

θ

s

θ

′

s

θ

′

s

θ

A

θ

B

θ

C

ˆ

θ

ˆ

θ

ˆ

θ

θ

∗

θ

∗

θ

∗

A.Regular initilization with random optimization

C. Meta-learned Optimizer With Meta-learned Initialization

gradient change history in the fine-tuning process, which is not optimal. In this work, we propose an optimizer called Meta-AdaM, specially designed for the inner loop in meta-learning. We design an LSTM-based meta-learner to adaptively estimate the learning rate. Also, we make wiser use of momentum to dynamically determine the optimal updating direction. This can help the base model to converge to the optimum in fewer steps.

3 Meta-AdaM: A Meta-Learned Adaptive Optimizer with Momentum

We propose a Meta -learned Ada ptive optimizer with M omentum ( Meta-AdaM ) for FSL.

3.1 Motivations and Challenges

Few-shot learning (FSL) is a formidable problem in machine learning, where models are expected to generalize effectively to new tasks with only a handful of labeled examples. The limited amount of training data, however, poses a significant challenge for traditional optimizers, as they tend to overfit and yield subpar solutions [28]. As demonstrated in [45], the empirical risk minimizer or optimizer in FSL is unreliable due to its propensity to overfit a limited number of training examples, resulting in unsatisfactory solutions, as illustrated in Figure 1 (A). Consequently, optimization steps are often restricted to prevent overfitting, which makes it difficult for an optimizer to discover suitable solutions for FSL tasks. Generally, two primary strategies exist to tackle this issue. One involves identifying an optimal initialization for model parameters, as depicted in Figure 1 (B). The other focuses on generating high-quality gradient updates for rapid convergence to an appropriate solution, as displayed in Figure 1 (C). These strategies are orthogonal and can be combined. The objective of this work is to develop an optimizer capable of producing high-quality gradient updates, allowing the model to achieve a satisfactory optimum with a limited number of updates.

The most commonly used optimization algorithms for training deep learning models are variants of gradient descent, which utilize the following update formula:

In the update formula, θ t i represents the value of a trainable parameter θ i of the model after undergoing t updates. The variables η t i , g t i , and m t i refer to the learning rate, gradient, and first-order momentum for θ i at the t th update, respectively. Additionally, α t i and β t i are coefficients to balance the gradients and momentum, where momentum is an exponentially weighted moving average of the gradients.

In many-shot learning scenarios, where each learning task has abundant labeled data, several optimizers [41, 14, 32], such as Adam [14] and SGD with momentum [27], have been developed to speed up

the convergence speed by incorporating adaptive learning rates [5] and momentum. However, these optimization algorithms cannot be directly applied to FSL tasks as the variance of momentum in the initial optimization steps tends to be high [19], leading to unstable gradient updates with limited optimization steps and resulting in poor local optima.

Meta-learning has recently demonstrated a robust capacity to address FSL problems. Specifically, a meta-learner is trained on similar tasks and employed to learn a learner for any new task [38, 2, 46]. While several approaches [18, 30, 3] have been proposed to meta-learn adaptive learning rates in FSL by incorporating gradients and weights, the relationship between learning rates and weight-changing history, rather than just weight values, has been shown to be crucial [11, 15]. Furthermore, effectively utilizing momentum has been challenging due to its high variance in the initial updates [19]. In light of these challenges, we propose a novel approach, called Meta-AdaM, which meta-learns an adaptive optimizer with momentum. Our approach leverages the power of meta-learning to learn a model that predicts adaptive learning rates based on gradient update history, and effectively integrates momentum to speed up the convergence speed. By doing so, Meta-AdaM enables a learner to quickly adapt to new few-shot learning tasks with only a limited number of update steps.

3.2 Update History-Based Adaptive Learning Rates through Meta-Learning

In this section, we propose utilizing meta-learning to develop a learner that generates adaptive learning rates based on gradient update history. Existing methods, such as [30, 3], depend on a meta-trained multi-layer perceptron to predict learning rates using model parameter values and gradients. However, recent studies indicate that the relationship between learning rates and weight-changing history is more critical than between learning rates and weight values [11, 15]. As a result, it is essential to predict adaptive learning rates based on weight-changing history, specifically gradient g t i and momentum m t -1 i . To achieve this, we treat the trajectory of g t i and m t i as sequence data and employ a long short-term memory network (LSTM) [39]. The LSTM accepts momentum and new gradients as inputs and outputs an adaptive learning rate for each trainable parameter θ i . Formally, the update formula for θ i with the proposed meta-learner is defined as follows:

Algorithm 1 MetaMomentumInner

1: Input: θ : meta model, learning rate learner: LSTM, S T : data of a task.

2: Require: η : base step size, K : number of inner loops, f ( · ) : loss function.

3: Set θ 0 = θ , m 0 = 0

4: for all t = 1 to K do

5: g t = ∇ θ t -1 f ( S T , θ t -1 )

-

1

7: θ t g = θ t -1 -η g t

6: θ t m = θ t -1 -η m t

8: ∆ L t m = f ( S T , θ t m ) -f ( S T , θ t -1 )

9: ∆ L t g = f ( S T , θ t g ) -f ( S T , θ t -1 )

10: α t , β t = σ (∆ L t m , ∆ L t g )

11: η t = LSTM ( α t g t , β t m t -1 )

12: m t = α t g t + β t m t -1

13: θ t = θ t -1 -η t × m t

14: end for

15: return θ K

where η t i is the predicted learning rate for the trainable parameter θ i in the t th update. Compared to previous methods that primarily employ multi-layer perceptrons to predict learning rates based on model parameter values and gradients, our approach naturally considers weight-changing history, leading to more accurate learning rate predictions. For now, we set α t i = 1 and β t i = 0 temporarily to exclude momentum from the update. This is because the variance of momentum is exceptionally high in the initial update steps, rendering it unsuitable for direct application in FSL tasks. In the next section, we propose an effective method to incorporate momentum, accelerating convergence.

It is crucial to emphasize that our approach fundamentally differs from Meta-LSTM [30]. MetaLSTM primarily employs a multi-layer perceptron to predict learning rates and weight decay, taking current weights and gradients as input without considering the weight-update history.

3.3 Double Lookahead for Effective Momentum Integration

In this section, we propose to effectively integrate momentum in the FSL optimizer to accelerate the convergence speed by using double lookahead. To generate more effective gradients updates during the training process for few-shot-learning tasks, we compute the adaptive model updates by applying lookahead to both momentum and new gradients, leading to a more stable learning

trajectory and better generalization. Lookahead is an effective optimization strategy by looking ahead at a sequence of fast weights [47] for fast converge and a corresponding slow gradient to smooth oscillating. However, in few-shot-learning context, the fast weight will easily trigger the overfitting problem. And the limited update steps number make the slow gradient hard to smooth the oscillating.

We propose addressing this challenge by judiciously selecting the coefficients of the momentum and new gradients using one-step lookahead. Specifically, before performing the accumulation process in Eq. (2), we apply lookahead to both the momentum and the new gradients, resulting in two losses. In this context, the aggregation of weight updates should prioritize the update direction with lower loss. To achieve this, we employ a softmax operator to obtain the coefficients by normalizing the loss decreases. Formally, the proposed approach is defined as in Algorithm 1. In this algorithm, S T is a batch of support data, η is a fixed learning rate, σ ( · ) is a softmax operator, and f ( · ) is a loss function that computes loss for a batch data S T with a given model θ . We

first generate two lookahead models by updating the current model parameters θ t -1 using momentum m t -1 and new gradients g t separately in line (6) and line (7), which results in θ t -1 m and θ t g . Then f ( · ) is applied to both models with S T , yielding loss changes for both lookahead models: ∆ L t -1 m and ∆ L t g . In line (10), the softmax operator is used to normalize two loss changes so that the aggregation coefficients sum to 1. The resulting coefficients are used to predict adaptive learning rates in line (11), and aggregate the momentum and new gradients in line (12). Finally, line (13) updates the model parameters using accumulated momentum m t .

With dynamic coefficients generated through lookahead, if either the momentum or new gradients lead to an ineffective optimization direction, the small coefficient in line (12) can mitigate the negative impact and place greater emphasis on more effective optimization directions. As a result, the double-lookahead strategy can stabilize the momentum update and accelerate the convergence speed.

3.4 Dynamic Class Weighting Scheme for Classification Tasks

In FSL classification tasks employing meta-learning, the meta-trained base model is significantly influenced by prior knowledge. As a result, certain classes with features closer to prior knowledge become easier to classify, leading to an imbalance that compromises the model's performance. To address this issue, we introduce a dynamic class weighting schema, which emphasizes classes contributing more to model optimization. We estimate a class's contribution by observing the changes in its loss. If a class has been well-learned, the loss will be stable, and minor loss changes will be observed. In contrast, if there is a substantial loss drop for a particular class, it may contribute more to future updates. Based on the loss changes for each class, we use a softmax operator to obtain aggregation weights for every class. The accumulated loss is the weighted summation of all class losses. This process is defined as follows:

where f 2 ( · ) is a loss function that computes losses for each class based on support data S T , C is the number of classes in task T , and T is the temperature used in the softmax operator σ ( · ) . In the above schema, if the class k has a larger loss L t k , its corresponding aggregation weight w k will be larger, leading to a stronger optimization signal from class k . Thus, the class that needs to be more well-trained will obtain more attention, leading to more effective optimization.

3.5 Meta-AdaM

We present our proposed Meta-AdaM in Algorithm 2. Given a task distribution p ( T ) , our goal is to meta-learn an optimal model initialization θ and an adaptive learning learner θ l . In each iteration, we sample a batch of tasks from p ( T ) . For each sampled task, we fine-tune an adapted model θ ′ and update θ and θ l using Algorithm 1. During testing, we directly utilize θ and θ l for a new task.

4 Experiments

We conduct experiments to evaluate the proposed Meta-AdaM using three benchmark datasets.

4.1 Experimental Settings

This section describes datasets and backbone models used to evaluate our Meta-AdaM.

Datasets . We evaluate the proposed methods using three datasets: Mini-ImageNet [10], TieredImageNet [33], and Cifar100 [16] datasets. MiniImageNet dataset contains 100 classes with 600 samples per class. Each data sample is an 84 × 84 colored image. Following previous works [30], we split 100

classes into 64, 16, and 20 class groups for training, validation, and testing. The image size of Tiered-ImageNet is also 84 × 84. This dataset consists of 608 classes with 779,165 images. These 608 classes are further combined into 34 high-level classes. These high-level classes are divided into 20, 6, and 8 class groups for training, validation, and testing [31], respectively. Tiered-ImageNet considers the class similarity when divide the datatset to ensure that the distribution of train and test data is very different. Cifar100 dataset for few-shot learning contains 60,000 images with 100 classes. The image size is 32 × 32. The 100 classes are divided into 60, 20, and 20 groups for training, validation, and testing. These datasets are bench-marking datasets in the FSL domain.

Backbone models . We evaluate the proposed Meta-AdaM using two backbone models: Convnet4 and Resnet12 [12]. Convnet4 is a 4-layer Convolutional Neural Network with 32.9 thousand trainable

parameters, while Resnet12 is an 18-layer Convolutional Neural Network featuring 4 million trainable parameters. By employing these two models, we demonstrate that our proposed optimizer is effective on both small and large networks.

Hyperparameters . We show the hyperparamters in Table 2, which applies to each dataset and each backbone. To ensure a fair comparison, we follow the setting of [3] and report the test performance of the ensemble of top-5 performing models on the validation set.

4.2 Results Using A Small Network

We first evaluate our Meta-AdaM using the Convnet4 network on three datasets. The experiments are conducted under two settings: 5-way-1-shot and 5-way-5-shot, where a -wayb -shot means each task contains a classes and there are b training examples for each class. We compare our Meta-AdaM with Meta-LSTM [30], MAML [7], Meta-SGD [18], and MAML+ALFA [3] as main baseline. A summary of the comparison results can be found in Table 1. From the results, we can observe that our method achieves promising results on all three datasets under two settings. Under the 5-way1-shot setting, our Meta-adaM outperforms MAML and MAML+ALFA by margins of 3.30 % and 1.42 % on Mini-ImageNet, 4.87 % and 0.77 % on TieredImageNet, and 2.91 % and 1.34 % on Cifar100, respectively. Under the 5-way-5-shot setting, our method outperforms MAML and MAML+ALFA by margins of 7.59 % and 1.58 % on Mini-ImageNet, 5.18 % and 2.12 % on TieredImageNet, and 6.38 % and 2.93 % on Cifar100, respectively.

These promising results demonstrate that our Meta-AdaM can consistently yield more effective optimization results on various benchmark datasets. Notably, the performance gains are even larger in the 5-way-5-shot setting. Given that both the momentum and the new gradients estimated from a few training examples could be noisy with only one training example per class, it is difficult to produce effective gradient updates from them. The power of Meta-AdaM is better reflected by slightly increasing the number of training examples. We also compare our methods with other recent research works, including e3bm [20], Sparse-MAML [44], MAML+SiMT [42] and MeTAL [40]. We can observe that our methods outperform the other methods in most of the settings.

4.3 Results Using A Large Network

In the previous section, we evaluated our Meta-AdaM using a small network, Convnet4, as the backbone model. However, optimizing a large model is more challenging due to the risk of overfitting. In this section, we conduct experiments on a larger backbone model, Resnet12, to further demonstrate the effectiveness of our Meta-AdaM. The evaluation is conducted under 5-way-1-shot and 5-way-5-

shot settings using the Mini-ImageNet, Tiered-ImageNet, and Cifar100 datasets. The comparison results with MAML and MAML+ALFA are summarized in Table 3.

Under the 5-way-1-shot setting, we observe margins of 0.15%, 0.69%, and 0.24% improvements compared to previous state-of-the-art models on the Mini-ImageNet, TieredImageNet, and Cifar100 datasets, respectively. Under the 5-way-5-shot setting, our Meta-AdaM outperforms ALFA by margins of 2.76%, and 1.60% on TieredImageNet, and Cifar100 datasets respectively. On Mini-ImageNet, our method gives a equal performance compared to MAML+ALFA. Our method also outperforms other methods, including MAML+SiMT [42] and MeTAL [40]. The consistently promising results on the three benchmark datasets demonstrate that our Meta-AdaM is effective when used to optimize a larger backbone model.

4.4 Ablation Study

In this study, we introduce three novel components in our Meta-AdaM: momentum, double lookahead and an adaptive learning rate via a meta-learned learner. Also, we introduce a dynamic class-weighting scheme for loss functions specially designed for classification tasks. In this section, we evaluate the individual contributions of each component. To accomplish this, we conduct experiments in both 5-way-1-shot and 5-way-5-shot settings, utilizing Convnet4 as the backbone model on the Mini-ImageNet dataset.

4.4.1 Momentum, Double lookahead and Adaptive learning rate

Specifically, we test varying combinations of the first three components and assess the performance of the resulting optimizers. These comparative results can be summarized in Table 4. The results in Table 4 demonstrate that each component contributes to the overall performance. Specifically, heuristic aggregation of momentum does not significantly enhance the performance, with a 0.42% decrease observed in the 1-shot and a modest 0.16% increase in the 5-shot setting compared to the original MAML. However, incorporating momentum with a double look-ahead strategy leads to superior results over those from the momentum-only setting. This enhanced optimizer surpasses the original MAML by 0.6% in the 1-shot and 0.98% in the 5-shot settings.The adaptive learning rate also plays a crucial role. By solely applying this adaptive learning rate in the inner loop, the resultant optimizer outperforms the original MAML by 0.32% in the 1-shot and 4.87% in the 5-shot settings. When these three components are combined, the performance improvement is even more significant. The momentum with look-ahead results in a more stable and efficient training optimization process, thereby aiding the meta-learner in better estimating the adaptive learning rates. From these results, we see a performance boost of 2.94% in the 1-shot and 5.69% in the 5-shot settings. To further evaluate the effect of weight update history, we conduct additional experiments that substitute the LSTM network with a MLP network with same inputs. The results are summarized in Table 5. From

Losschangefortest taskonCifar100

LosschangefortesttaskonTiered-ImageNet

LosschangefortesttaskonMini-ImageNet

10

Meta-AdaM

10

Meta-AdaM

Meta-AdaM

ALFA

ALFA

ALFA

8

8

6

6

6

4

4

4

2

2

2

0

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

step

Step

Step

the result, LSTM outperform MLP network in 5 out of 6 experiment setting, which indicates that the weight update history is important.

4.4.2 dynamic class-weighting scheme

To demonstrate the effect of dynamic class, we conduct additional experiments on four configurations: ALFA, ALFA with Dynamic Class Weighting, Meta-AdaM excluding class weighting, and full Meta-AdaM and summarize in result in Table 6. The result shows that Meta-AdaM without Dynamic Class Weighting outperforms ALFA in 5 out of 6 evaluated settings. Noticeably, Dynamic Class Weighting provides a tangible enhancement when integrated with ALFA.

4.5 Visualization of Optimization Process

In this section, we visualize the optimization process of our Meta-AdaM and compare it with ALFA. We randomly select 900 testing tasks from three datasets: Mini-ImageNet, Tiered-ImageNet, and Cifar100. The experiment setting is 5-way-1-shot using Convnet4 as the backbone model. We use the checkpoints with the highest validation accuracy for Meta-AdaM and ALFA. We calculate the average loss change for all 900 tasks in each update step and plot the results in Figure 2. From the plot, our Meta-AdaM helps the model find a better optimum than ALFA.

Additionally, We conducted experiments on 300 5-way-1-shot test tasks on Cifar100 with Convnet4 as the backbone model. We include the change of inner learning rate η and α and β in the "Double look ahead strategy." in Figure 3. As illustrated in Figure 3 on the left, our optimizer tends to initially predict smaller learning rates due to the substantial conflict between momentum and gradient. A smaller learning rate will lead to a slow convergence rate in the first few steps, providing spaces for accumulating high-quality momentum. Therefore, from Figure 2, the loss from our method decreases slower than ALFA. Once the momentum quality is good after several steps, the predicted learning

0.175

lr

0.60

Alpha

rate

Beta

0.150

0.55

0.125

Value

0.50

0.100

0.45

P

0.075

0.40

0.050

1

2

3

4

5

2

3

4

5

Step

Step

rates become larger, leading to a better optimum than ALFA. This also highlights our contribution to using double-look-ahead to handle highly variable momentum.

Figure 3 on the right shows that the gradients give lower losses until the first three steps, so the gradient α coefficient is larger than for momentum β . However, after accumulating for a few steps, the accumulated momentum can generate a lower loss and a larger coefficient. Such results support the intuition that the momentum quality improves over the trajectory.

4.6 Efficiency Study of Meta-AdaM

In this section, we assess the efficiency of our MetaAdaM. In Meta-AdaM, we employ a double lookahead strategy to evaluate the momentum and new gradients, which may introduce additional computational costs. To evaluate these added costs, we compare the total training time of MAML, MAML+ALFA, and Meta-AdaM. We use the experimental setting of 5way-1-shot on the Mini-ImageNet dataset and record

the running time on an NVIDIA RTX A4000 GPU. The results are summarized in Table 7. The results indicate that our Meta-AdaM takes slightly longer running time than ALFA. However, considering the performance improvements brought by Meta-AdaM, the additional computational resources required by Meta-AdaM are justifiable.

4.7 Predicted Negative Learning Rate

During our experiments, the predicted learning rate in our adaptive learning rate component could occasionally be negative. This phenomenon has been described in [4] and observed in ALFA [3], which employs a two-layer network to predict learning rates. Interestingly, applying negative learning rates to trainable parameters in some update steps can lead to better optimization results. However, determining which parameters are suitable for a negative learning rate and when to apply the negative learning rate remains to be clarified. Further research exploring the role of negative learning rates in meta-learning may contribute to improving the optimization of meta-learning models.

5 Conclusion

In this work, we propose a meta-learning-specific optimizer, Meta-AdaM, which incorporates three novel optimization components that can better utilize update signals: momentum and new gradients. Specifically, we propose to meta-learn an adaptive learning rate learner using an LSTM network by looking back at the weight update history. We also introduce a double lookahead algorithm to dynamically accumulate momentum, leading to a more stable and effective momentum update and gradient update. Moreover, we propose a dynamic class weighting scheme to balance the losses across different classes. Experimental results demonstrate the effectiveness of our proposed Meta-AdaM.

References

S.-i. Amari. Backpropagation and stochastic gradient descent method. Neurocomputing , 5(4-5): 185-196, 1993.

M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. De Freitas. Learning to learn by gradient descent by gradient descent. Advances in neural information processing systems , 29, 2016.

S. Baik, M. Choi, J. Choi, H. Kim, and K. M. Lee. Meta-learning with adaptive hyperparameters. Advances in neural information processing systems , 33:20755-20765, 2020.

A. Bernacchia. Meta-learning with negative learning rates. arXiv preprint arXiv:2102.00940 , 2021.

M. Donini, L. Franceschi, O. Majumder, M. Pontil, and P. Frasconi. Marthe: scheduling the learning rate via online hypergradients. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence , pages 2119-2125, 2021.

T. Dozat. Incorporating nesterov momentum into adam. The Eleventh International Conference on Learning Representations , 2016.

C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR, 2017.

C. Finn, K. Xu, and S. Levine. Probabilistic model-agnostic meta-learning. Advances in neural information processing systems , 31, 2018.

S. Flennerhag, A. A. Rusu, R. Pascanu, F. Visin, H. Yin, and R. Hadsell. Meta-learning with warped gradient descent. arXiv preprint arXiv:1909.00025 , 2019.

S. Gidaris and N. Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 43674375, 2018.

I. Goodfellow, Y. Bengio, and A. Courville. Deep learning . MIT press, 2016.

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770778, 2016.

T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey. Meta-learning in neural networks: A survey, 2020.

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

M. J. Kochenderfer and T. A. Wheeler. Algorithms for optimization . Mit Press, 2019.

A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. Master's thesis, University of Tront , 2009.

K. Lee, S. Maji, A. Ravichandran, and S. Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10657-10665, 2019.

Z. Li, F. Zhou, F. Chen, and H. Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835 , 2017.

L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265 , 2019.

Y. Liu, B. Schiele, and Q. Sun. An ensemble of epoch-wise empirical bayes for few-shot learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16 , pages 404-421. Springer, 2020.

N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel. A simple neural attentive meta-learner. arXiv preprint arXiv:1707.03141 , 2017.

M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of machine learning . MIT press, 2018.

T. Munkhdalai and H. Yu. Meta networks. In International conference on machine learning , pages 2554-2563. PMLR, 2017.

A. Nichol, J. Achiam, and J. Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 , 2018.

E. Park and J. B. Oliva. Meta-curvature. Advances in Neural Information Processing Systems , 32, 2019.

A. Phaphuangwittayakul, Y. Guo, and F. Ying. Fast adaptive meta-learning for few-shot image generation. IEEE Transactions on Multimedia , 24:2205-2217, 2021.

B. T. Polyak. Some methods of speeding up the convergence of iteration methods. Ussr computational mathematics and mathematical physics , 4(5):1-17, 1964.

J. Rajendran, A. Irpan, and E. Jang. Meta-learning requires meta-augmentation. Advances in Neural Information Processing Systems , 33:5705-5715, 2020.

A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine. Meta-learning with implicit gradients. Advances in neural information processing systems , 32, 2019.

S. Ravi and H. Larochelle. Optimization as a model for few-shot learning. In International conference on learning representations , 2017.

M. Ren, E. Triantafillou, S. Ravi, J. Snell, K. Swersky, J. B. Tenenbaum, H. Larochelle, and R. S. Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 , 2018.

S. Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 , 2016.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115(3):211-252, 2015.

A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero, and R. Hadsell. Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960 , 2018.

A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning , pages 1842-1850. PMLR, 2016.

J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems , 30, 2017.

Y. Song, T. Wang, S. K. Mondal, and J. P. Sahoo. A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities. arXiv preprint arXiv:2205.06743 , 2022.

Q. Sun, Y. Liu, T.-S. Chua, and B. Schiele. Meta-transfer learning for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 403-412, 2019.

M. Sundermeyer, R. Schlüter, and H. Ney. Lstm neural networks for language modeling. In Thirteenth annual conference of the international speech communication association , 2012.

H. K. D. C. J. M. K. M. L. Sungyong Baik, Janghoon Choi. Meta-learning with task-adaptive loss function for few-shot learning. In International Conference on Computer Vision (ICCV) , 2021.

I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning. In International conference on machine learning , pages 11391147. PMLR, 2013.

J. Tack, J. Park, H. Lee, J. Lee, and J. Shin. Meta-learning with self-improving momentum target. In Advances in Neural Information Processing Systems , 2022.

P. Vicol, L. Metz, and J. Sohl-Dickstein. Unbiased gradient estimation in unrolled computation graphs with persistent evolution strategies. In International Conference on Machine Learning , pages 10553-10563. PMLR, 2021.

J. von Oswald, D. Zhao, S. Kobayashi, S. Schug, M. Caccia, N. Zucchet, and J. Sacramento. Learning where to learn: Gradient sparsity in meta and continual learning, 2021.

Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur) , 53(3):1-34, 2020.

O. Wichrowska, N. Maheswaranathan, M. W. Hoffman, S. G. Colmenarejo, M. Denil, N. Freitas, and J. Sohl-Dickstein. Learned optimizers that scale and generalize. In International conference on machine learning , pages 3751-3760. PMLR, 2017.

M. Zhang, J. Lucas, J. Ba, and G. E. Hinton. Lookahead optimizer: k steps forward, 1 step back. Advances in neural information processing systems , 32, 2019.

1

2

3

4

5

Step

0

1

2

3

4

5

6

7

Loss

Loss for each class--No dynamic class weight

class1

class2

class3

class4

class5

1

2

3

4

5

Step

0

0.5

1

1.5

2

2.5

3

3.5

Loss

Loss for each class--dynamic class weight

class1

class2

class3

class4

class5

7 Appendix

7.1 Result on regression task

To further prove the efficiency of our method, we conduct additional experiments on few-shot regression tasks and show the result in Table 8. We follow the experiment setting of the MetaSGD[18]. The regression task is to map the underlying function based on the input. The target function is y ( x ) = A sin( ωx ) , where amplitude A , frequency ω , and phase b all follow a uniform distribution with the interval [0.1, 5.0], [0.8, 1.2], and [0, π ]. We train three few-shot regression learners on 5-shot, 10-shot, and 20-shot tasks separately. Each learner will test on 100 5-shot, 10-shot, and 20-shot testing tasks. We report MSE errors with a 95 % confidence interval.

We compare our approach with MAML[7], Meta-SGD[18], Meta-SGD+ALFA[3], and Meta-SGD + META-AadM. The result has shown that our method outperforms the baselines, Meta-SGD and Meta-SGD+ALFA, in all settings. The promising results indicate that our proposed method is also effective on few-shot regression tasks.

7.2 Study of the Dynamic Class Weighting Scheme

In previous section, we present quantitative results to demonstrate the effectiveness of the dynamic class weighting scheme. In this section, we visualize the loss changes for each class when using this scheme. We compare the loss function using our scheme and the one without a class weighting scheme. Each optimization process consists of five update steps. The left figure in Figure 4 displays

the loss changes of each class without employing a class weighting scheme. Class 2 has a significantly lower loss than the other classes. For the last four steps, only the loss of class 2 is optimized, while the other four classes remain under-optimized. Such situations can lead to model collapse problems in meta-learning [26], which means the model tends to classify all testing examples into one or two classes. The right figure in Figure 4 illustrates the loss changes when using the dynamic class weighting scheme. Class 4 has a lower loss in the first step compared to the other classes. The optimizer focuses more on the other four classes by emphasizing those with larger losses. Ultimately, all class losses converge towards the optimization endpoint, effectively mitigating the risk of model collapse problems.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents Meta-AdaM, a meta-learned adaptive optimizer that also includes momentum. The parameter of the optimizer in the inner loop, i.e. inner learning rate and momentum, are predicted by a LSTM network, that takes into account the previous gradient and momentum steps. The optimization process also includes a double look-ahead strategy for the prediction of the coefficients for the gradient and the momentum. Lastly, the optimization also includes a dynamic class weighting scheme, using a softmax to assign weights to each class according to the loss changes for each class. \nThe optimizer is then combined with MAML and evaluated on three benchmarks datasets (mini-Imagenet, tiered-Imagenet and CIFAR100) with two different backbones (conv4 and Resnet12).\n\nSTRENGTHS:\n### Originality\nThe paper builds upon ALFA [1] by replacing the learned network that predicts the *inner learning rate* and *weight decay* parameters, by a LSTM network, that predicts *inner learning rate* and *momentum* parameters. The LSTM allows taking into account previous update steps. The dynamic class weighting scheme is also an interesting regularization approach to reduce overfitting to easy classes in the inner loop.\n### Significance\nThe proposed method shows strong results on the different benchmarks considered.  \n\n[1] : Baik, S., Choi, M., Choi, J., Kim, H., & Lee, K. M. (2023). Learning to learn task-adaptive hyperparameters for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence.\n\nWEAKNESSES:\n### Clarity\n\n- The paper is difficult to follow as there is a lot of text to describe the method, but it is missing a formal definition of the process. Thankfully, the two algorithms help to clarify the text, but the equations in the algorithms could be directly introduced and described in the text. There is also several concepts vaguely defined such as the loss function $f_2$ in Equation 5 and the temperature $T$ in Equation 6 that is never mentioned in the hyperparameters. \n- The discussion around the dynamic class weighting scheme seems unrelated to the original idea described throughout the paper. The way it is introduced in the optimization is also not really explained, neither in the text nor in the algorithms. \n- The ablation study mentions four different components, but it is not clear what the *momentum* component refers to.  \n### Quality\n- The gradient descent formula described in Equation 1 and 2 is not the commonly used one, described in [2] for instance. Here, the learning rate is impacting the update coming from the momentum. \n- In line 175, it is said that the coefficients $\\alpha$ and $\\beta$ should sum to 1, hence the use of a softmax function, but it is not clear to me why it should be the case, and it is never explained.\n### Significance\nThe introduction of the dynamic class weighting scheme, while being an interesting addition, makes the work specific to the problem of few-shot classification, even though this component does not seem necessary in the approach. From Table 4 in the ablation study, we can see that it has a strong impact on the performance, mainly in the 5-shot setting, and makes the method more effective than others in the mini-imagenet benchmark. Results without this weighting and in other tasks, such as few-shot regression, would make the paper and the evaluation of the approach more solid. \n\n[1] : Baik, S., Choi, M., Choi, J., Kim, H., & Lee, K. M. (2023). Learning to learn task-adaptive hyperparameters for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence.\n\n[2] : Sutskever, I., Martens, J., Dahl, G., & Hinton, G. (2013, May). On the importance of initialization and momentum in deep learning. In International conference on machine learning (pp. 1139-1147). PMLR.\n\nQUESTIONS:\n- From the paper and the equations, it seems that the optimization process does not include weight decay. This difference from the ALFA [1] paper is not really explained in the related work section, what is the motivation behind this change ?\n- Why $\\alpha$ and $\\beta$ need to sum to 1 ? Why the learning rate is included in the update coming from the momentum ?\n- How the dynamic class weighting scheme is included in the optimization process ? Does it impact both the loss of the LSTM and the base learner ?\n- What are the results without the class weighting ? And in different tasks than classification ?\n\nLIMITATIONS:\nThe authors did not include any discussion of the limitations of their work, nor of potential negative impacts. An important limitation is the limitation to few-shot classification tasks as mentioned above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce an adaptive optimizer for meta-learning that uses double lookahead to incorporate momentum in scenarios where ‘conventional’ approaches of using momentum usually fall short (few-shot inner-loop, due to high momentum/gradient fluctuations during initial steps). The use of an LSTM to predict the adapted learning rates allows for consideration of the weight-update history.  To further boost the obtained results, the authors additionally introduce dynamic weighting of the class-specific losses to aid a more balanced optimization / loss minimization.\n\nSTRENGTHS:\n* Well written paper, easy to read and follow, good introduction of method and required background. Placement within related work well accomplished, and approach well motivated.\n* Generally interesting idea and approach: Incorporating momentum with the double look-ahead and predicting the learning rate based on the history are to me the main contributions.\n* Results of the overall approach seem solid.\n* For the most part, interesting evaluations including run-time.\n\nWEAKNESSES:\nMajor: \n\n1. **Entanglement of ‘dynamic class weighting scheme’ and ‘optimiser approach’**: \n While an interesting contribution in itself, the “dynamic class weighting scheme” is highly entangled with the actual meta-adam approach throughout the reported results. While I do understand the pressure to ‘outperform’ other works, the reported results in its current state do in my opinion not allow for an insightful comparison of approaches & limits the reader’s insights in an unnecessary manner. I would highly encourage the authors to: \n   - Add the Meta-AdaM (w/o dyn-cl-w) to the tables\n   - Evaluate MAML or others together with their proposed dyn-cl-w. \n\n   This would allow the reader to better judge the influence of the individual contributions and their influence beyond this particular work!\n\n   Note: I am aware that Table 4 shows the result w/o dynamic class loss, but since the *paper’s main selling point and TITLE focuses on the optimiser*, this should be individually demonstrated and be part of an honest discussion.\n\n2. **Usefulness of the LSTM / history**: \n  How essential is the fact that the adaptive learning rate being predicted by an LSTM? How important is the history, really – which is one of the main claims of the paper! -> How does it compare to e.g. using an MLP with similar complexity and the same input (prev. momentum and current gradient)?\n\n\nMinor:\n\n3. **Dataset inconsistency** (Description & References):\n  The authors state 'Cifar100' as few-shot learning benchmark – however, there two inconsistencies that should be improved:\n   * The FSL community uses two different versions: Cifar-FS and FC100, that yield very different results due to their different complexities. The provided reference in the paper links neither of them, but the original non-FSL Cifar  -> clarification here is required.\n   * The description of the dataset (sec 4.1) states “*with 100 different labels of birds*”, which seems more like a description for the popular CUB dataset than Cifar -- I assume some mixup has happened here?\n\nQUESTIONS:\nThe three main points & questions are listed in the weaknesses section. However, I have several additional suggestions/questions I'd like comments from the authors on:\n\nSeveral experiments / ablation studies would imo strengthen this work: \n* What are the actual learning rates predicted within the inner loop? (Is there some ‘commonality’ within the same step across different test episodes?)\n* What is the ‘average’ mix of gradient and momentum contribution across inner-loop update steps (i.e. Algorithm 1, l.10)? Does this reflect  ‘intuitions’ of growing momentum along the trajectory, or maybe defy these? (visualization/comment would be interesting)\n* How big is the influence of the base learning rate in Algorithm 1? You are using it in lines 6-9 to predict the effect of taking a step of this size into the gradient or momentum direction – the step-size (and direction) of the actual step then taken however differs and is according to your comments ‘not limited’ (not even sign-wise) -> Is the lookahead still a ‘good and robust’ measure, or did you encounter difficulties? Does it change when choosing larger / smaller base-lrs, or is it rather robust (and why)? \n\nAdditional questions:\n* What is the size of the LSTM, and how much complexity does it add?\n* How is the temperature chosen for the dynamic class weighting, and how much does it matter & potentially differ across datasets?\n* I’m slightly confused about the message of Figure 2: While Meta-AdaM does converge to a better minimum/lower loss at the final step, ALFA seems to converge faster – It would be helpful for the authors to comment on this.  (In its current state, I am not convinced about this part of the paper in general, since we already see in the tables that Meta-Adam seems to converge to a better minimum, and the only fact the figure shows me is that it does so in a slower way than ALFA?) \n* Algorithm 2 ln 12/13 indicate SGD for outer loop – is this a simplification for conciseness, or did you choose SGD for the outer loop?\n\n\nSome remarks to further improve presentation:\n* Stating where the results for MAML are taken from would be helpful, since some of the datasets and architectures have not been evaluated in the original paper\n* Line 36/37: “[…] has been shown that weight-update history is more important than the weights themselves […]” -> Reference would be important! I am aware that some are given in line 131, but two entire books without any sub-indication are a rather large search space for the reader! -> more recent publication or chapter/sub-chapter of book would be appreciated\n* Sorting the references in ascending order (easier for reader to cross-check)\n* Typos/Layout: Fig. 1: space after “A”; mix of capitalized and non-capital words within sub-captions inconsistent\n\n**Note**: I do think the idea presented in this paper is interesting! Depending on the response from the authors, I am happy to reconsider my current rating.\n\n> UPDATE: raised rating to weak accept after rebuttal.\n\nLIMITATIONS:\nThe authors provide the runtime in comparison to related approaches, which adequately addresses the main limitation I see for this approach.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Meta-Adam, a meta-learned adaptive optimizer that employs momentum to rapidly adapt a meta-learned initial model to a few-shot task. The proposed framework consists of four components. First, an LSTM that predicts a learning rate for each weight inside the model using the weight update history in previous iterations. Furthermore, the optimizer employs momentum and additionally develops a look-ahead procedure for identifying environments where momentum hurts optimization and adjusting the parameters accordingly. Lastly, the loss function is weighted class-wise based on the class-specific loss during each gradient step to ensure that all class losses are optimized collectively. Experiments on mini-, tiered-ImageNet and Cifar100 benchmarks demonstrate the efficacy of the proposed method.\n\nSTRENGTHS:\n- The method proposed explores interesting ideas around better-optimizing learning rates and momentum in few-shot adaptive meta-learned models. The components of the method are well-motivated, derived to address specific problems, and empirically effective.\n- Experiments demonstrate improvements in performance across various benchmarks compared to MAML-based benchmarks.\n- Ablation studies provided demonstrate the empirical efficacy of each component of the proposed method and how much they contribute to the overall improvements in few-shot image classification accuracy.\n- The paper is written clearly with reasonable notation and is easy to follow.\n\nWEAKNESSES:\n- All baselines that are compared are MAML-based. Although it's understandable given the focus of the paper on gradient adaptive few-shot image classification models, the method would still be compared against other groups (for instance, metric-based [1, 2, 3, 4]) when deciding to use a few-shot learner in an applied setting. It's useful to provide a discussion of this and potentially include a comparison to other non-MAML-based baselines.\n- Figure 2 / Section 4.5 - it would be interesting to see what the variance bounds are for each optimization method as averages can be uninformative as to how much loss reduction can vary per task. Furthermore, ALFA seems to achieve much faster convergence although a worse optimum compared to Meta-AdaM. Why is that the case?\n- 4.6 Dynamic class loss discussion lacks details as to the task/benchmark used in Figure 3. Without said details, it's difficult to evaluate whether the trends observed are statistically significant to support claims on reducing loss across all classes.\n\n[1] Prototypical Networks for Few-shot Learning\n[2] Improved Few-Shot Visual Classification\n[3] Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes\n[4] Enhancing Few-Shot Image Classification with Unlabelled Examples\n\nQUESTIONS:\nSee weaknesses for questions. Happy to improve my rating once the authors have addressed the concerns above.\n\nLIMITATIONS:\nThere is no explicit discussion of limitations or potential negative societal impact of the work, although a brief discussion of some directions for future work, in particular with respect to negative learning rates, is provided. I strongly encourage the authors to provide a discussion of other potential limitations of the work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel meta-learned optimizer with momentum for few-shot learning. The proposed approach incorporates the parameter-update history by leveraging LSTMs to adaptively adjust the learning rate. To enhance convergence speed,  the momentum is integrated into the meta-optimizer by using a double look-ahead mechanism. Furthermore, to address imbalanced learning, a dynamic class weighing schema is introduced, encouraging optimal model optimization by assigning more weight to influential classes. The effectiveness of the proposed method is evaluated on multiple benchmarks, showcasing its performance in few-shot learning scenarios.\n\nSTRENGTHS:\n* The proposed meta-optimizer is novel, as it considers the history of weight updates and momenta when adapting the learning rate, which distinguishes it from previous works in the field. However, as my research expertise lies outside of the meta-optimizer domain, I cannot ascertain the paper's coverage of previous literature. Thus, I eagerly anticipate feedback from other reviewers with expertise in this topic.\n\n* The paper is written in a clear and concise manner, facilitating readers' comprehension of the ideas and technical contributions. Each sub-problem is precisely defined, including its limitations, followed by the proposal of techniques to address those issues. This clarity enhances the paper's accessibility and aids in conveying the research concepts effectively.\n\nWEAKNESSES:\n* The related work section may have missed some recent works or baseline methods on meta-optimizer for few-shot learning (FSL). All the existing works mentioned in Section 2.2.1 were published before 2020, which raises the question of whether there have been significant developments in this topic between 2021 and 2023. This could lead to confusion and a potential gap in the literature review.\n\n* The writing in the paper lacks conciseness. The author repeats high-level ideas and the limitations of previous works using very similar expressions multiple times throughout the entire paper. For instance, the description provided in Line 118, Line 129, and Line 139 are identical, which can be redundant and could benefit from consolidation.\n\n* The method section contains some content that should be included in the related works section. For example, in Line 145, the comparison between the proposed idea and a previous work should be presented in the related works section.\n\nQUESTIONS:\n* I would appreciate it if the authors and other reviewers could carefully verify the inclusion of all relevant previous works in this area within the paper. This thorough examination is crucial for evaluating the novelty of the paper, and it will significantly influence my final rating.\n\n* I would expect the author could improve the conciseness of the writing.\n\n* The dynamic loss contributes to the accuracy of the proposed method, but the paper does not mention such a technique in the introduction. I expect the author could provide an explanation.\n\nLIMITATIONS:\nThe paper includes the limitation in section 4.8 but fails to mention the negative social impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors aim to solve the few-shot learning problem by proposing a meta-learned learning rate learner. This learner utilizes weight-update history as input to predict more appropriate learning rates for rapid convergence. Furthermore, they incorporate momentum into the optimization process of few-shot learning via a double look-ahead mechanism, enabling rapid convergence similar to many-shot settings. Extensive experimental results are provided to show the effectiveness of the proposed method.\n\nSTRENGTHS:\n- This paper is well-written and easy to read. A lot of figures are included to explain the proposed method.\n\n&nbsp;\n\n- The proposed method is technically sound. It is reasonable to use meta-learning to adjust the learning rates. \n\n&nbsp;\n\n- Extensive experimental results are provided.\n\nWEAKNESSES:\n- The proposed idea of using a model to output the learning rates is not novel. A similar thing has been done in [a], which uses a meta-learned model to output a series of hyperparameters, including learning rates.\n\n&nbsp;\n\n- The performance of the proposed method is much lower compared to recent few-shot learning methods. For example, the 1-shot accuracy on miniImageNet is only 59.89% in Table 3. Existing methods, e.g., [b] and [c], achieve more than 68% accuracy in the same setting. It is not necessary that the proposed method beats all existing methods. However, it should not be much lower than other methods. At least, the performance should be comparable. Besides, it is necessary to indicate the proposed method doesn’t achieve the SOTA performance and compare the SOTA in the tables. \n\n&nbsp;\n\n- All important related works are missing, e.g., [a-d]. They should be compared and discussed. \n\n&nbsp;\n\nOverall, I think the quality of this paper is not satisfactory. The idea is a little bit incremental. The overall performance is not significant. Many important related works are missing. Therefore, I recommend rejection. The authors need to show the proposed method can be applied to recent popular baselines, e.g., [b, c, d]. I don’t think the current version is ready to be presented at NeurIPS.\n\n&nbsp;\n\n[a] An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning, ECCV 2020. \n\n[b] Partner-Assisted Learning for Few-Shot Image Classification, ICCV 2021.\n\n[c] DeepEMD: Differentiable Earth Mover’s Distance for Few-Shot Learning, TPAMI. \n\n[d] Rectifying the Shortcut Learning of Background for Few-Shot Learning, NeurIPS 2021.\n\nQUESTIONS:\nPlease address my concerns in the \"weaknesses\"\n\nLIMITATIONS:\nSee my concerns in the \"weaknesses\"", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-09"}