Meta-Learning with Self-Improving Momentum Target
Authors: Jihoon Tack
Jongjin Park
Hankook Lee
Jaeho Lee
Jinwoo Shin
Venue: NeurIPS 2022 Accept (2022)
Keywords: meta-learning
momentum network
knowledge distillation

Abstract: The idea of using a separately trained target model (or teacher) to improve the performance of the student model has been increasingly popular in various machine learning domains, and meta-learning is no exception; a recent discovery shows that utilizing task-wise target models can significantly boost the generalization performance. However, obtaining a target model for each task can be highly expensive, especially when the number of tasks for meta-learning is large. To tackle this issue, we propose a simple yet effective method, coined Self-improving Momentum Target (SiMT). SiMT generates the target model by adapting from the temporal ensemble of the meta-learner, i.e., the momentum network. This momentum network and its task-specific adaptations enjoy a favorable generalization performance, enabling self-improving of the meta-learner through knowledge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-training. Our experimental results demonstrate that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods under various applications, including few-shot regression, few-shot classification, and meta-reinforcement learning. Code is available at https://github.com/jihoontack/SiMT.


Meta-Learning with Self-Improving Momentum Target
Jihoon Tack', Jongjin Park', Hankook Lee', Jaehoo Lee", Jinwoo Shin Korea Advanced Institute of Science and Technology (KAIST)
2Pohang University of Science and Technology (POSTECH) jiiwoos)jkaaliioo ac.kr
(jihoontack, jongj in Lark, hankook. jaeho lee@postech. ac.
Abstract
The idea of using a separately trained target model (or teacher) improve the performance of the student model has been increasingly popular in various machine learning domains, and meta-learning is no exception; recent discovery shows that utilizing task-wise target models can significantly boost the generalization performance. However, obtaining a target model for each task can be highly expen- sive, especially when the number of tasks for meta-learning is large. To tackle this issue, we propose a simple yet effective method, coined Self-improving Momentum Target (SiMT). SiMT generates the target model by adapting from the tempo- ral ensemble of the meta-learner, i.c., the momentum network. This momentum network and its task-specific adaptations enjoy favorable generalization perfor- mance, enabling self-improving of the meta-learner through knowledge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-traie meta-training. Our experimental results demonstrate that SiMT brings significant performance gain when combined with wide range of meta-learning methods under various applications, including few-shot regres- sion, few-shot classification, and meta-reinforcemmen learning. Code available at https: //gi thub. com/ jhhootack//SMM..
Introduction
Meta-leaming [51] is the art of extracting and utilizing the knowledge from the distribution of tasks better solve relevant task. This problem is typically approached by training meta- meta-model that can transfer its knowledge to a task-specific solver, where the performance of the meta-model is evaluated on the basis of how well each solver performs on the corresponding task. To learn such meta- motel, one should be able to (a) train an appropriate solver for each task utilizing the knowledge transferred from the meta-model, and (b) accurately evaluate the performance of the solver. standard way to do this is the so-called S/Q (support/query) protocol [55, 34]: for (a), use a set of support set samples to train the solver; for (b), use another set of samples, called query set samples to evaluate the solver!.
Recently, however, an alternative paradigm-calle much attention [58, 62, 32]. The approach assumes that the meta-learner has an access task-specific target models, i.c., an expert model for each given task, and uses these models to evaluate task-specific solvers by measuring the discrepancy of the solvers from the target models. Intriguingly, has been observed that such knowledge distillation procedure [43, 21] helps to improve the meta-generee generalizztinnn performance [62], in a similar way that such teacher-student framework helps avoid overfitting under non-meta -earning contexts [30, 24].
S/T (support/target) protocol-has received
We give an overview of terminologies used in the paper to guide readers new to this field (see Appendix A).
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Meta-model
Task-spedfic solver
Perturbed solver
Dropout
Temporal ensemble
Distillation
Self-improvin process
S
Momentum network
Momentum target
An overview of the proposed Self-improving Momentum Target (SiMT): the momentum network efficiently generates the target model, and by distilling knowledge to the task-specific solver, it forms a improving process. S and denote the support and query datasets, respectively.
Figure I:
Despite such advantage, the S/T protocol difficult to be used in practice, as training target models for each task usually requires excessive computation, especially when the number of tasks is large. Prior works aim to alleviate this issue by generating target models in a compute-efficien manner. For instance, Lu et al. [32] consider the case where the learner has access to model pre-trained on a global data domain that covers most tasks (to be meta-trained upon), and propose to generate task-wise target models by simply fine-tuning the model for each task. However, the method still requires to compute for fine-tuning on a large number of tasks, and more importantly, hard to be used when there is effective pre-trained model available, e.g., globally pre- traindrne model is usually not available in reinforcement learning, as collecting "global" data is a nontrivial task [9].
In this paper, we ask whether can generate the task-specific target models by (somewhat ironically) using meta-learning. We draw inspiration from recent observations semi/self-supervise learning literature [50, 16, 5] that the temporal ensemble of model, i.e., the momentum network [27], can be an effective teacher of the original model. It turns out that a similar phenomenon happens in the meta-learning scenario: one can construct momentum network of the meta-model, whose task-specific adaptation is an effective target model from which the task-specific knowledge can be distilled to train the original meta-modes model.
Contribution. We establish a novel framework, coined Meta LeeannLn with Self-improving Momen- tum Target (SiMT), which brings the benefit of the S/T protocol to the S/Q-like scenario where task-specific target models are not available (but have access to query data). The overview of SiMT illustrated in Figure 1. In a nutshell, SiMT is comprised of two (iterative) steps:
Momentum target: We generate the target model by adapting from the momentum network, which shows better adaptation performance than the meta soota-mmoell itself. In this regard, generating the target model becomes highly efficient, e.g., one single forward required when obtaining the momentum target for ProtoNet [45].
Self-improving process: The meta-model enables to improve through the knowledge distillation from the momentum target, and this recursively improves the momentum network by the temporal ensemble. Furthermore, we find that perturbing parameters of the task-specific solver of the meta-model, e.g., dropout [47], further stabilizes this self-improving process by preventing fast convergence of the distillation loss during meta-training.
We verify the effectiveness of SiMT under various applications of meta-learning, including few- shot regression, few-shot classification, and meta- reifforeino learning (meta-RL). Overall, our experimental results show that incorporating the proposed method can consistently and significantly improve the baseline meta-learning methods [10, 31, 36, 45]. In particular, our method improves the few-shot classification accuracy of Conv4 [55] trained with MAML [10] on mini-ImageNet [55] from 47.33% - 51. .49% for 1-shot, and from 63.27% + 68.744 68.74% for 5-shot, respectively. Moreover, we show that our framework could even notably improve on the few-shot regression and meta-R RI. tasks, which supports that proposed method is indeed domain-agnostic.

---

2
Related work
Learning from target models. Learning from an expert model, i.e., the target model, has shown its effectiveness across various domains [30, 35, 65, 52]. As follow-up, recent papers demonstrate that meta-learning can also be the case [58, 62]. However, training independent task-specific target models is highly expensive due to the large space of task distribution in meta-learning. To this end, recent work suggests pre-training a global encoder on the whole meta-training set and finetune target models on each task [32]; however, they limited specific domains and still require some computations, e.g., they take more than 6.5 GPU hours to pre-train only 10% of target models while ours require GPU hours for the entire meta-learning process (ProtoNet [45] of ResNet- 12 [34]) the same GPU. Another recent relevant work is bootstrapped meta-learning [1 I]. which generates the target model from the meta-model by further updating the parameters of the task-specific solver for some number of steps with the query dataset. While the bootstrapped target models can be obtained efficiently, their approach is specialized in gradient-based meta-learning schemes, e.g., MAML [10]. In this paper, suggest an efficient and more generic way to generate the target model during the meta- veta-training.
Learning with momentum networks. The idea of temporal ensembling, i.c., the momentum network, has become an essential component of the recent semi/self-sus smmi/self-supervieee learning algorithms [3. 5]. For example, Mean Teacher [50] first showed that the momentum network improves the performance of semi-supervised image classification, and recent advanced approaches [2, 46] adopted this idea for achieving state-of- the-art performances. Also, in sulf--upe learning methods which enforce invariance to data augmentation, such momentum networks are widely utilized as a target network [19, 16] to prevent collapse by providing smoother changes in the representations. In meta-learning. a concurrent work [6] used stochastic weight averaging [23] (a similar approach to the momentum network) learn low-rank representation. In this paper, we empirically demonstrate that the momentum network shows better adaptation performance compare to the original meta-model, which motivates us utilize it for generating the target model in a compute-efficiet manner.
Problem setup and evaluation protocols
In this section, we formally describe the meta-learning setup under consideration, and SIQ and S/T protocols studied in prior works.
Let p(T) be distribution of tasks. The goal of meta-learning is to train a meta-model for parameterized by the meta-model parameter 6 which can transfer its knowledge to help to train a solver for new task. More formally, we consider some adaptation subroutine Adapt(--)) which uses both information transferred from 0 and the task-specific dataset (which we call support set) S to output a task-specific solver as ø² = Adapt S*). For example, the model-agnostic meta-learning algorithm (MAML: [10] uses the adaptation subroutine of taking a fixed number of SGD on S", starting from the initial parameter 0. this paper, we aim to give a general meta-learning framework that can be used in conjunction with any adaptation subroutine, instead of designing a method specialized for a specific one.
Problem setup:
Meta-learning,
The objective is to learn nice meta-model parameter from a set of tasks sampled from p(T) (or sometimes the task distribution itself), such that the expected loss of the musk-specific adaptations is small. i.c., ming Er>-pr) [eT (Adapt(( (0, ST)) )}], where (-(-) denotes the test loss on task r. To train such meta-model, we need mechanism evaluate and optimize (c.g.. via gradient descent). For this purpose, existing approaches take one of two approaches: the S/Q protocol or the SIT protocol.
S/Q protocol. The majority of the existing meta-learning frameworks (e.g., [55, 34)) splits the task-specific training data into two, and use them for different purposes. One is the support set S which is used perform the adaptation subroutine. Another is the query set Q" which is used for evaluating the performance of the adapted parameter and compute the gradient with respect to 0. In other words, given the task datasets (Si, Q1), E1),((3) (S2, 22),..., (SN, QN). the S/Q protocol solves
1 N
min
Σ C (Adaptt() 2*)). 0 N
(1)
where L(o, Q) denotes the empirical loss of a solver o on the dataset Q
Heer, while we assumed a static batch of tasks for notational simplicity, the expression is readily extendible to the case of stream of tasky drawn from p(r).
3

---

SIT protocol. Another line of work considers the scenario where the meta-learner additionally has an access to set of target models Prarget for each training task [58, 32]. In such case, one can use teacher-student framework to regularize the adapted solver to behave similarly (or have low prediction discrepancy, equivalently) to the target model. Here, a typical practice is to not split each task dataset and measure the discrepancy using the support dataset that is used for the adaptation [32]. In other words, given the task datasets S2, ......,,, SN and the corresponding target models T Ptarget: Otarget:..... Stargets the S/T protocol updates the meta-model by solving
min Σ
1
Cieacs (Adapt(O, ST:). ) Ptarger: STD)
where Creaca(ó, (0, Q Orarget, S) denotes a discrepancy measure between the adapted model ø and the target model Prarget. measured using the dataset S.
Meta-learning with self-improving momentum target
In this section, we develop compute-cfficien framework which bring the benefits of SIT protocol to the settings where we do not have access to target-specific tasks or general pretrained model, as in general S/Q-like setups. In a nutshell, our framework iteratively generates meta-target model which generalizes well when adapted to the target tasks, by constructing a momentum network [50] of the meta-model itself. The meta-model is then trained, using both the knowledge transferred from the momentum target and the knowledge freshly learned from the query sets. We first briefly describe our meta-model update protocol (Section 4.1), and then the core component, coined Delf-Improving Momentum Target (SiMT), which efficiently generates the target model for each task (Section 4.2).
4.1 Meta-model update with a SIQ-S/T hybrid loss
To update the meta-model, we use hybrid loss function of the S/Q protocol (1) and the S/T protocol (2). Formally, let (S1, Q1), (S,, Q.)..... (Sy: 2.v) be given task datasets with support-query split, and let Obargets Ptarget. ..... Pearget be task-specife target models generated by our target generation procedure (which be explained with more detail in Section 4.2). We train the meta- vetal as
min Σ 1=1
(1 ((daapt (1- )*) +
Lraach (AcietAAAApttt.... (0,5*) Drarger: 2T)) (3)
where X E [0, 1) is the weight hyperparameter. We note two things about Eq. 3. First, while we training using the target model, we also use S/Q loss term. This is because our method trains the meta-target model and the meta-model simultaneously from scratch, instead of requiring fully-trained target models. Second, unlike in the S/T protocol, we evaluate the discrepancy Lreach using the query set Q instead of the support set. to improve the generalization performance of the student model. In particular, the predictions of adapted models on query set samples are softer (i.c., having less confidence) than on support set samples, and such soft predictions are known to be beneficial on the generalization performance of the student model in the knowledge distillation literature [64, 49].
4.2 SiMT: Self-improving momentum target
We now describe the algorithm propose, SiMT (Algorithm 1), to generate the target model in a compute-efficient manner. In a nutshell, SiMT is comprised of two iterative steps: momentum target and self-improving process. To efficiently generate a target model, SiMT utilizes the temporal ensemble of the network, i.e., the momentum network, then distills the knowledge of the generated target model into the task-specific solver of the meta-model to form a self-improving process.
Momentum target. For the compute-efficient generation of target models, we utilize the momentum network Q of the meta-model. Specifically, after every meta-model training iteration, we compute the exponential moving average of the meta-model parameter 0 as
Quonent f- Quoment + (1 -0,
where v) E [0, 1) is the momentum coefficient. We find that Anomant can adapt better than the meta- model 0 itself and observe that the loss landscape has flatter minima (see Section 5.5), which can

---

Algorithm/ SiMT: Self-Im Improving Momentum Target
Require: Distribution over tasks p(r). adaptation subroutine Adapt(-), momentum coefficient
weight hyperparameter X, dropout probability p, task batch size N, learning rate B.
1: Initialize 0 using the standard initialization scheme.
2: Initialize the momentum network with the meta-model parameter, Psonent f- 0.
3: while not done do
4: Sample N tasks {T: from p(r)
5: for = 1 to N do
6:
Sample support set S and query set Q from Uomeent = Adapt(Pee (fnoment; S.
7:
> Generate momentum target. P Adapt a task-specific solver. > Perturb the solver.
8:
6-. Adapt(O )
9:
Th drop
= Dropout(() (p) .p).
10:
S (0) = (1 - A) L [(o @T) + A Creach Creaca(@arop drop o
)
>> Compute loss.
11:
end for
0 N Vo Ei Lioral (0).
Train the meta-model. Update the momentum network.
13:
Onanent & Onoxant (1 11)
14:
end while
be hint for understanding the generalization improvement [29, 12]. Based on this, we propose to generate the task-specific target model, i.e., the momentum target @nonent, by adapting from the momentum network Inonent. For a given support set S, we generate the target model for each task as
o nement = Adapt(0aomett
noment, S
Vi E N1.
(5)
We remark that generating momentum targets does not require an excessive amount of compute (see Section 5.5), e-g.. ProtoNet [45] requires single forward of support set, and MAML [10] requires few-gradient steps without second-ore gradient computation for the adaptation.
Self-improving process via knowledge distillation. After generating the momentum target, utilize its knowledge to improve the generalization performance of the mota-model. To this end, we choose the knowledge distillation scheme [21], which is simple yet effective across various domains, including meta-learning [32]. Here, our key concept is that the momentum target self- self-improves during the training due to the knowledge transfer. To be specific, the knowledge distillation from the momentum target improves the meta-model itself, which recursively improves the momentum through the temporal ensemble. Formally, for given query set Q, we distill the knowledge of the momentum target (mnomnnt to the task-specific solver of the meta-model d as
Lreach (0, Onomant, Q
1kD (S (x), So(x)).
(6)
(a,y)eQ
where lgo is the distillation loss and |. is the cardinality of the set. For regression tasks, we use the MSE loss, i.e., lgD(21, (21, 22) : ||2| - 2g 12. and for classification tasks, we use the KL divergence with temperature scaling [17], i.e., (xn 22) T (( || (=>//T)), where T is the temperature hyperparameter, is the softmax function and æ1, are logits of the classifier, respectively. We present the detailed distillation objective of reinforcement learning tasks in Appendix C. Also, note that optimizing the distillation loss only propagates gradients to the meta- model I, not to the momentum network
9 Puneent. i.c., known as the stop-gradien operator [5, 7].
Furthermore, we find that the distillation loss (6) sometimes converges too fast during the meta- training, which can stop the self-improving process. To prevent this, we suggest perturbing the parameter space of é. Intuitively, injecting noise to the parameter space of 6 forces an asymmetricity between the momentum target's 's prediction, hence, preventing fo and f farmum from reaching a similar prediction. To this end, we choose the standard dropout regularization [47] due to its simplicity and generality across architectures and also have shown its effectiveness under distillation research [60]: Derop == Dropout(() p) where p the probability of dropping activations. In the end, we use the perturbed task-specific solver odrop and the momentum target Coonent for our evaluation protocol (3).
5

---

Table I: Few-shot regression results ShapeNet and Pascal datasets. We report the angular error for ShapeNet, and MSE for Pascal. SiMT use the momentum network at meta-test time. Reported results averaged over three trials, subscripts denote the standard deviation, and bold denotes the best result of each group.
ShapeNet
Pascal
Method
10-shot
15-shot
10-shot
15-shot
MAMI [10]
29.655-10.600
22.286113.60
2.612:10.280
2.513:10.20
MAML [10] SiMT
18.913-12.655
16.100
1.462.10.230
1.229
ANIL [36]
39.915::0.65
38.202
6,600=10.600
6.517=+0.20
ANIL [36] SIMT
3..424
29.478::0.12
5.339 :0.321
5.007:
MetaSGD [31]
1..353+1.110
15.768+1.200
3.532:40.881
2.833-+0:216
MetaSGD [311+ SIMT
16.121
14.377:10.58
2.300
1.879:101344
Table 2: Few-shot in-domain adaptation accuracy (%) on 5-way mini- and liered-ImageNet. SiMT use the momentum network at meta-fest time. Reported results are averaged over three trials, subscripts denote the standard deviation, and bold denotes the best result of each group.
mini-ImageNet
tiered- ImtgeNet
Model
Method
1-shot
5-shot
1-shot
5-shot
MAML [10]
47.33:10455
63.27:0.14
50.19
66.05:10.19
MAML [10] SiMT
51.49:10.18
68.74110.2
52.51
69.58:10.11
ANIL|]36 ANIL [36] SiMT
47.71+:047
63. 63.1340.45
49.57=0.04
66.344:028
50.81
67.99
51.66.10.2
68.88
Conv4 [55]
MetaSGD [31]
50.66:+0.18
65.5540.54
52.48
71.06:+0.20
MetaSGD [31] + SiMT
51.700+0..0
69.1341.40
52.98=0.07
71.46:10.12
ProtoNet [45]
47.97++0.99
65.16400.7
51.90=0.55
71.51+0025
ProtoNet [45] + SiMT 51.25:+0.55
68.7110.35
53.25=0027
72.69:00.27
MAMI. [10] MAMI. [10] SiMT
52.66.10.60
68.69
57.32.10.59
73.78:10.27
56.28
10.26
59.72.1022
74.40
ANIL [36] ANIL [36] + SiMT
51.80.10.59
68.38:10.20
57.52.10.68
73.50:10.35
54.4411.27
69.98-10.66
58.1820031
75.59
ResNet-12 [34]
MctaSGD [31] MctaSGD [31] SiMT
54.95:10.1
40.65+0043
58.97
76.37:20.11
55.72-10.9
74.01
61.03=0005
78.04:9048
ProtoNet [45] ProtoNet [45] + SiMT
52.84:10.21
68.35
61.16:10.17
79.94
55.8441057
72.45+0032
62.01
81.82+0.12
5 Experiments
In this section, we experimentally validate the effectiveness of the proposed SiMT by measuring its performance on various meta-learning applications, including few-shot regression (Section 5.1), few-shot classification (Section 5.2), and meta-reinforcement learning (meta-RL; Section 5.3).
Common setup. By following the prior works, we chose the checkpoints and the hyperparameters on the meta-validation set for the few-shot learning tasks [33, 56). For RI., we chose it based on the best average return during the training [10]. We find that the hyperparameters, e.g., momentum coefficient 7) or the weight hyperparameter X, are not sensitive across datasets and architectures but can vary on the type of the meta-learning scheme tasks. We provide further details in Appendix D. Moreover, we report the adaptation performance of the momentum network for SiMT.
5.1 Few-shot regression
For regression tasks, we demonstrate our experiments on ShapeNet [13] and Pascal [63] datasets, where they aim to predict the object pose of gray-scale image relative to the canonical orientation. To this end, we use the following empirical loss train the meta-model: the angular loss for the ShapeNet (llss)elllos((((() cos(y))l2 (ssn(f((z)) sin(y) and the MSE loss for the

---

Table 3: Few-shot cross-domain adaptation accuracy (%) ResNet-12 trained under 5-way mini- and hered-ImageNee We consider CUB and Cars as cross-domain datasets. SiMT use the momentum network meta-test time. Reported results averaged over three trials, subscripts denote the standard deviation, and bold denotes the best result of each group.
mini-ImageNet
tiered ImageNe-
Problem
Method
CUB
Cars
CUB
Cars
MAML [10] MAML [10] SiMT
39.50=00:1
32.87=0.20
42.32:0069
36.62:+0.12
42.32:10.2
33.73=0.63
44.33:10.3
37.21
ANIL [36] ANIL [36] + SIMT
37.30
3..28=1.03
42.29:10.33
36.27:10.58
38.86=+0.98
3..34=0.95
44.53+1.21
36.92:10.56
1-shot
MetaSGD [31]
41.9810.18
34.52: 0.56
46.48.12.10
38,09 1.09911.2
MetaSGD [31] SIMT
43.50.1
33.92.10300
46.62.10..1
38.69 0.226
ProtoNet [45] ProtoNet [45] + SiMT
41.22.10.11
32.79
37.59 17.75.10.56
44.13....00
34.53 0.40
48.89.10.6
38.07.10.44
MAML [10]
56.17:0099
44.56=0.79
65.00++0.89
51.08+0.28
MAML [10] + SIMT
09.22:1:0339
46.59=0z
67.58:10.61
5..8
ANIL [36]
53.42:+0.97
41.65=0.67
62. 62.48:+0.85
50.50+1.18
ANIL [36] + SiMT
56.03:1:.40
45.88=088
66.30-10.99
0..60
5-shot
MetaSGD [31]
58.90:+ 1.30
47.44 27.44=11 1.55
70.38:10.27
56,28:+0.07
MetaSGD [31] SiMT
65.07 1.89
49.86 0.84
73.93:10.42
17.97
ProtoNet [45]
57.87++0.77
48.06 06.006=1 1.10
74.35+:0.93
57.23:+0.25
ProtoNet [45] + SiMT
63.85 0.776
51.67 0.29
75.97 1.009
59.01 1:0.0
y)l²), following the prior works [63, 13]. For the backhone meta-learning schemes we use gradient-based approaches, including MAML [10], ANIL [36], and MetaSGD [31]. For all methods, we train the convolutional neural network with 7 layers [63] and apply dropout regularization [47] before the max-pooling layer for SiMT. Table summarizes the results, showing that SiMT significantly improves the overall meta-leaming schemes in all tested cases.
Pascal (( [[)())) (llf:))))
5.2 Few-shot classification
For few-shot classification tasks, we use the cross-entropy loss for the empirical loss term L to train the meta-model f., i.e., Lowyee [c=(fo(z), y) where Ice is the cross-entropy loss. We train the meta-model on mini-ImageNNt [55] and tiered-ImageNet [38] datasets, following the prior works [32, 56). Here, we consider the following gradient-based and metric-based meta-learning approaches our backbone algorithm to show the wide usability of our method: MAML, ANIL, MetaSGD, and ProtoNet [45]. We train each method on Conv4 [55] and ResNet- 12 [34], and apply dropout before the max-pooling layer for SiMT. For the training details, mainly follow the setups from each backbone algorithm paper. See Appendix D. for more details.
In-domain adaptation. In this setup, we evaluate the adaptation performance on different classes of the same dataset used in veta-training. As shown in Table 2, incorporating SiMT into existing meta-learning methods consistently and significantly improves the in-domain adaptation performance. In particular, SiMT achieves higher accuracy gains on the mini-ImageNNt dataset, e.g., 5-shot performance improves from 63.27% + 68. 68%74% on Conv4. We find that this is due to the overfitting issue of backbone algorithms on the mini- ImageNet dataset, where SiMT is more robust to such issues. For instance, when training mini-ImageN ImageNet 5-shot classification Conv4, MAML starts to overfit after the first 40% of the training process, while SiMT does not overfit during the training.
Cross-domain adaptation. We also consider the cross-domain adaptation scenarios. Here, we adapt the meta-model on different datasets from the meta- meta-training: we use CUB [57] and Cars [26] datasets. Such tasks are known to be challenging, as there exists large distribution shift between training and testing domains [18]. Table shows the results. Somewhat interestingly, SiMT also improves the cross-domain adaptation performance of the base meta-learning methods across the considered datasets. These results indicate that SiMT successfully learns the ability to generalize unseen tasks even for the distributions that highly differ from the training.
7

---

Table 4: Comparison with bootstrapped targets (Bootstrap) [16] on few-shot in-domain adaptation tasks. We report the adaptation accuracy (%) of Conv4 trained under 5-way mini- and hered-Imagett SiMT use the momentum network at meta-test time. Reported results are averaged over three trials, subscripts denote the standard deviation, and bold indicates the best result of each group.
mini mini-ImageNet
tiered dered-ImageNee
Method
1-shot
5-shot
1-shot
5-shot
MAML [10]
47.33
63,27:4014
50. 10.19
66.05=0.19
MAML [10] Bootstrap [16]
48.68=0.33
68.45-+0.40
49.34::0.26
08.8440.37
MAML [10] + SiMT
51.49
68.74:10.12
52.51
69.58=2011
ANIL [36]
47.71
63.13:10.33
49.57:10.04
66. 06.24
ANIL [36] + Bootstrap [16]
47.74.10.44
65.16:10.04
48.85
66.09.0.07
ANIL [36] + SiMT
50.81
67.99+0.19
51.66+#0.26
68.88=0.08
5.3 Reinforcement learning
The goal of meta-RL is training an agent quickly adapt a policy to maximize the expected return for unseen tasks using only a limited number of sample trajectories. Since the expected return is usually not differentiable, we use policy gradient methods update the policy. Specifically, we use vanilla policy gradient [59], and trust-region policy optimization (TRPO; [39]) for the task-specific solver and motalmodel, respectively, following MAML [10]. The overall training objective of meta-RL. is in Appendix C, including the empirical loss L, and the knowledge distillation loss Lreach. We evaluate SiMT on continuous control tasks based on OpenAI Gym [4] environments. In these experiments, we choose MAMI. as our backbone algorithm, and train multi-layer perceptron policy network with two hidden layers of size 100 by following the prior setup [10]. We find that the distillation loss is already quite effective even without the dropout regularization, and applying it does not improve more. We conjecture that dropout on such a small network may not be effective as it designed to reduce the overfitting of large networks [47]. We provide more experimental details in Appendix D.I.
2D Navigation. We first evaluate SiMT on 2D Navigation task, where a point agent moves to different goal positions which are randomly chosen within a 2D unit square. Figure 2 shows the adaptation performance of learned models with up to three gradi- ent steps. These results demonstrate that SiMT could consistently improve the adap- tation performance of MAML. Also, SiMT makes faster performance improvements than vanilla MAML with additional gradient steps.
Acerag starre
Auannc MOUTE
4D0
MAM +SMT
14M SMIT
(a) 2D Navigation
(b) Half-cheetah Direction
Figure Meta-RL results for (a) 2D navigation and (b) Half-cheetah locomotion tasks. The solid line and shaded regions represent the truncated mean and stan- dard deviation, respectively, across five runs.
Locomotion. To further demonstrate the ef- fectiveness of our method, we also study high- dimensional, complex locomotion tasks based on the MuJoCo [53] simulator. We choose set of goal direction tasks with a planar cheetah ("Half-cheetah))) following previous works [10, 36]. the goal direction tasks, the reward is the magnitude of the velocity in either the forward or backward direction, randomly chosen for each task. Figure 2b shows that SiMT significantly improves the adaptation performance of MAML even with a single gradient step.
5.4 Comparison with other target models
In this section, we compare SiMT with other meta-learning schemes that utilize the target model, including bootstrapped [16] and task-wise pre-trained target models [32].
Bootstrapped target model. We compare SiMT with recent relevant work, bootstrapped meta- learning (Bootstrap) [16]. The key difference is on how to construct the target model Órarget: SiMT utilizes the momentum network while Bootstrap generates the target by further updating the parameters of the task-specifie solver. Namely, Bootstrap relies on gradient-base adaptation while SiMT does not. This advantage allows SiMT to incorporate various non-gradien con-gradient-bane based meta-learning approaches, c.g., ProtoNet [45], shown in Table 2. Furthermore, SiMT shows not only wider
8

---

Table 5: Comparison with pre-trained target models few-shot in-domain adaptation tasks. We report the adaptation accuracy (%) of ResNet-12 trained on 5-way mini- and tiered- Irree-ImaeeNet. SiMT use the learned momentum network at meta-test time. Reported results are averaged over three trials, subscripts denote the standard deviation, and bold indicates the best result of each group. indicates the values from the reference and percentage after the bar denotes the proportion of tasks with pre-trained target models for meta-training [32].
1-shot train cost
Muni-ImageNet
tiered-ImageNNet
Method
(GPU hours)
1-shot
5-shot
1-shot
5-shot
MAML [10]*
1.31
38,8450025
74.62+0.38
63.02:20.0
67.26=0032
MAML [10] + Lu et al. [32] 5%"
5,04
19.1333333
75.77=0229
64.52=00.0
68.39=0344
MAMI [10] + La al. [32] 10%*
8.32
60.06:10.35
76.34:10:2
65.23:005
70.02-0033
MAML [10] + SiMT
1.64
62.05:+0.99
78.77:00:5
63.91
77.43=0077
Table Ablation study on each component of SiMT. We report the few-shot in-domain adaptation accuracy (%) Conv4 trained with mini-ImageNet. Here, use the learned momentum network at meta-test time, except for the first experiment of the table. The reported results are averaged over three trials, subscripts denote the standard deviation, and bold denotes the best result.
Momentum
Distillation
Dropout
1-shot
5-shot
45.
63.27=0.14
48.98+0.32
66.12=0.21
/
49. 29.23
66.52=2011
/
49.25
65.25=0.15
<
51.49:1.188
68.74=0.12
applicability but also better performance than Bootstrap. shown in Table 4, SiMT consistently outperforms Bootstrap in few-show learning experiments, which implies that the momentum target is more effective than the bootstrapped target.
Pre-trained target model. We compare SiMT with Lu et al. [32] which utilizes the task-wise pre- trained target model for meta-learning. To this end, we train SiMT upon a ResNet-12 backbone pre-trained on the meta-training set, by following [32]. As shown in Table 5, SiMT consistently improves over MAML, and more intriguingly. SiMT performs even better than Lu et [32]. We conjecture this is because SiMT can fully utilize the target model for all tasks due to its efficiency, while Lu et al. [32] should partially sample the tasks with the target model due to the computational burden: note that when generating target models, SiMT only requires additional 0.3 GPU hours for all tasks while et al. [32] requires more than 3.7 GPU hours for 5% of tasks.
5.5 Ablation study
Throughout this section, unless otherwise specified. we perform the experiments in 5-shot in-domain adaptation on
with Conv4, where MAML is the backbone letanlea scheme.
Component analysis. We perform an analysis on each component of our method in both 1-shot and 5-shot classification (b) the distillation loss Lteach (6), and (c) the dropout regularization Dropout(-). by comparing the accuracies. The results in Table show each component is indeed important for the improvement. We find that a naive combination of the distillation loss and the momentum network does not show significant improvements. But, by additionally applying the dropout, the distillation loss becomes more effective and further improves the performance. Note that this improvement does not fully come from the dropout itself, as only using dropout slightly degrades the performance in some cases.
mmni--em namely, the use of (a) the momentum network 0
Computational efficiency. Our method may be seemingly compute- compute-inefficiee inefficient when incorporating meta-learning methods (due the momentum target generation); however, we show that it is not. Although SiMT increases the total training time of MAML by roughly 1.2 times, we have observed that it is 3 times faster to achieve the best performance of MAML: in Figure 3a, compare the accuracy under the same training wall-clock time with MAML.
Comparison of the momentum network and meta-model. To understand how the momentum network improves the performance of the meta-model, we compare the adaptation performance of the momentum network and the meta-mon metal during training SiMT. As shown in Figure 3b, we observe

---

70
Validation accuracy (%)
Validaton acouracy (%)
68
3xcfaster
66
64
62-
MAML
SIMT
Momentum network Bmoment
50
100
150
200
250
300
350
10
30
50
Training time (min)
Training Iterations (x10')
(a) Computation efficiency
(b) Network choice for the adaptation
Figure 3: Validation accuracy curves of 5-shot mini mini-ImageNet on Conv4: we compare the adaptation performance of (a) MAML and SiMT, under the training wall-clock time, and (b) the meta- model and the momentum network of SiMT, under the same number of training steps. The solid line and shaded regions represent the mean and standard deviation, respectively, across three
that the performance of the momentum network is consistently better than the meta-model, which implies that the proposed momentum target is nice target model in our self-improving mechanism.
Loss landscape analysis. We visualize the loss landscape of the momentum network 0 noment and the meta-model 0, to give insights into the general- ization improvement. To do this, train MAML with a momentum network (without distillation and dropout) and visualize the loss by perturbing each parameter space [29] (See Appendix D.2 for the detail of the visualization method). As shown in Figure 4, the momentum network forms a flatter loss landscape than the meta-model, where recent studies demonstrate that such a flat landscape is effective under various domains [12].
(a) Meta-model
Momentum network
Figure 4: Loss landscape visualization of Conv4 trained on 5-shot mini-ImageNet with MAML.
6 Discussion and conclusion
In this paper, propose simple yet effective method, SiMT, for improving meta-learning. Our key idea is to efficiently generate target models using a momentum network and utilize its knowledge to self-improve the meta-learner. Our experiments demonstrate that SiMT significantly improves the performance of meta-leaming methods on various applications.
Limitations and future work. While SiMT is a compute-efficien way to target models in meta-learning, it is still built on top of cxisting meta-model update techniques. Since existing meta- learning methods have limited scalability (to large-scale scenarios) [44], SiMT no exception. Hence, improving the scalability of meta-learning schemes is an intriguing future research direction, where believe incorporating SIMT into such scenarios is worthwhile.
Potential negative impacts. Meta-learning often requires large computation due to the numerous task adaptation during meta-training, therefore raising environmental concerns, e.g., carbon gener- ation [41]. As SiMT built upon the meta-learning methods, practitioners may need to consider some computation for successful training. To address this issue, sparse adaptation scheme [42] or lightweight methods for meta-learning [28] would be required for the applications.
Acknowledgements
We thank Younggyo Seo, Jaehyun Nam, and Minseon Kim for providing helpful feedbacks and suggestions in preparing earlier version of the manuscript. This work was supported by Institute of Information & communications Technology Planning Evaluation (IITP) grant funded by the Artificial Intelligence Graduate School Program Korea government (MSIT) Artificial Intelligence Graduate School Program (POSTECH), and (KAIST), No.2019-0-01906, No.2022-0-00711, Meta- Meta-learning applicable to real-world problems).
10

---

References
[1] M. R. Arnold, P. Mahajan, D. Datta, I. Bunner, and S. Zarkias. learn2leamm: library for Meta-Learning research. arXiv preprint arXiv:2008. 12284, 2020.
[2] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel. MixMatch: holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 2019.
[3] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel. ReMix- Match: Semi-supervised learning with distribution alignment and augmentation anchoring. In International Conference on Learning Representations, 2020.
[4] G. Brockman, V. Cheung, L.. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAl gym. arXiv preprint 1606.
2016.
[5] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerg- ing properties in supervied vision transformers. In IEEE International Conference on Computer Vision, 2021.
[6] K. Chen and C.-G. Lee. Meta-free few-shot learning via representation learning with weight averaging. In International Joint Conference on Neural Networks, 2022.
[7] X. Chen and K. He. Exploring simple Siamese representation learning. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[8] Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel. Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning, 2016.
[9] G. Dulac-Amold, D. Mankowitz, and T. Hester. Challenges of real-world reinforcement learning. In International Conference on Machine Learning, 2019.
[10] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference Machine Learning. 2017.
[11] S. Flennerhag, Y. Schroecker, T. Zahavy, H. van Hasselt, D. Silver, and S. Singh. Bootstrapped meta-learning. International Conference on Learning Representations, 2022.
[12] P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021.
[13] N. Gao, H. Ziesche, N. A. Vien, M. Volpp. and G. Neumann. What matters for meta-learning vision regression tasks? In IEEE Conference Computer Vision and Pattern Recognition, 2022.
[14] V. Garcia and J. Bruna. Few-shot learning with graph neural networks. International Conference Learning Representations, 2018.
[15] G. Ghiasi, T.-Y. Lin, and Q. V. Le. Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems, 2018
[16] J.-B. Grill, F. Strub, F. Altché, C. Tallec, Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al. Bootstrap your own latent - new ap- proach self-supervised learning. In Advances in Neural Information Processing Systems, 2020.
[17] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
[18] Y. Guo, N. C. Codella, L. Karlinsky, J. V. Codella, J. R. Smith, K. Saenko, T. Rosing, and R. Feris. A broader study of cross-domain few-shot learning. In European Conference on Computer Vision, 2020.
[19] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
11

---

[20] N. Hilliard, L. Phillips, S. Howland, A. Yankov, C. D. Corley, and N. O. Hodas. Few-shot learning with agtric- conditional embeddings. arXiv preprint arXiv:1802.0437,, 2018.
[21] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint drXiv:1503.02531, 2015.
[22] S. loffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference Machine Learning. 2015.
[23] Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson. Averaging weights leads to wider optima and better generalization. In Conference on Uncertainty in Artificial Intelligence, 2018.
[24] Y. Jang, H. Lee, S. J. Hwang, and J. Shin. Learning what and where to transfer. In International Conference on Machine Learning, 2019.
[25] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
[26] J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3D object representations for fine-graine ane-grained categorization. International IEEE Workshop on 3D Representation and Recognition (3dRR- 13), 2013.
[27] S. Laine and Aila. Temporal ensembling for semi-supervised learning. In International Conference on Learning Representations, 2017. 2017.
[28] Lee, J. Tack, N. Lee, and J. Shin. Meta-learning sparse implicit neural representations. In Advances in Neural Information Processing Systems, 2021.
[29] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein. Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems, 2018.
[30] Z. Li and D. Hoiem. Learning without forgetting. IEEE Transactions Pattern Analysis and Machine Intelligence, 2017.
[31] Z. Li, F. Zhou, F. Chen, and H. Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv: 1707:0707..9835,, 2017.
[32] S. Lu, H.-J Ye, L. Gan, and D.-C. Zhan. Towards enabling meta-learning from target models. In Advances in Neural Information Processing Systems, 2021.
[33] J. Oh, H. Yoo, C. Kim, and S.-Y. Yun. Boil: Towards representation change for few-shot learning. In International Conference on Learning Representations, 2021.
[34] B. Oreshkin, P. Rodríguez López, and A. Lacoste. Tadam: Task dependent adaptive metric for
improved few-shot learning. In Advances in Neural Information Processing Systems, 2018.
[35] W. Park, D. Kim, Y. Lu, and M. Cho. Relational knowledge distillation. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[36] A. Raghu, M. Raghu, S. Bengio, and O. Vinyals. Rapid learning or feature reuse? towards under- standing the effectiveness of maml. In International Conference Learning Representations, 2020.
[37] S. Ravi and H. Larochelle. Optimization model for few-shot learning. In International Conference on Learning Representations, 2017.
[38] M. Ren, E. Triantafillou, S. Ravi, J. Snell, K. Swersky, J. B. Tenenbaum, H. Larochelle, and R. S. Zemel. Meta-learning for semi-supervised few-shot classification. In International Conference on Learning Representations, 2018.
[39] Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz. Trust region policy optimization.
In International Conference on Machine Learning, 2015.
12

---

[40] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations, 2016.
[41] R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni. Green ai. arXiv preprint arXiv:1907.10597, 2019.
[42] J. R. Schwarz and Y. W. Teh. Meta-learning sparse compression networks. Transactions on Machine Learning Research, 2022.
[43] A. Sharif Razavian, H. Azizpour, J. Sullivan, and S. Carlsson. Cnn features off-the-shelf: an astounding baseline for recognition. In IEEE Conference Computer Vision and Pattern Recognition Workshops, 2014.
[44] J. Shin, H. B. Lee, B. Gong, and S. J. Hwang. Large-scale meta-learning with continual trajectory shifting. In International Conference on Machine Learning. 2021.
[45] J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, 2017.
[46] K. Sohn, D. Berthelot. N. Carlini, Z. Zhang, Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li. Fixmatch: Simplifying semi- sumi-supervised learning with consistency and confidence. In Advances Neural Information Processing Systems, 2020.
[47] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014.
[48] R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. MIT Press, 2018.
[49] J. Tang, R. Shivanna, Z. Zhao, D. Lin, A. Singh, E. H. Chi, and S. Jain. Understanding and improving knowledge distillation. arXiv preprint arXiv:2002.03532 2020.
[50] A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems, 2017.
[51] S. Thrun and L. Pratt. Learning Learn. Springer, 1998.
[52] Y. Tian, D. Krishnan, and P. Isola. Contrastive representation distillation. In International Conference on Learning Representations, 2020.
[53] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In JEEE/RSJ International Conference on Intelligent Robots and Systems, 2012.
[54] H.-Y. Tseng, H.-Y. Lee, J.-B. Huang, and M.-H. Yang. Cross-domain few-shot classification via learned feature-wise transformation. In International Conference on Learning Representations, 2020.
[55] O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for one shot learning. Advances in Neural Information Processing Systems, 2016.
[56] J. Von Oswald, D. Zhao, S. Kobayashi, S. Schug, M. Caccia, N. Zucchet, and J. Sacramento. Learning where learn: Gradient sparsity in meta and continual learning. In Advances in Neural Information Processing Systems, 2021.
[57] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Binds-200-2011 Dataset. Technical Report CNS-TR-2011 10000,,, California Institute of Technology, 2011.
(58) Y.-. Wang and M. Hebert. Learning to learn: Model regression networks for easy small sample learning. In European Conference on Computer Vision, 2016.
[59] R. J. Williams. Simple statistical follinnt algorithms for connectionist reinforcement learning, Machine learning, 1992.
13

---

[60] Q. Xie, M.-T. Luong, E. Hovy, and Q. Le. Self-training with noisy student improves imagenet classification. IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[61] H. Yao, L.-K. Huang, L. Zhang, Y. Wei, L. Tian, J. Zou, J. Huang, et al. Improving generalization in meta-learning via task augmentation. In International Conference on Machine Learning, 2021.
[62] H. Ye, L. Ming, D. Zhan, and W. Chao. Few-shot learning with a strong teacher. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
[63] M. Yin, G. Tucker, M. Zhou, Levine, and C. Finn. Meta-learning without memorization. In International Conference on Learning Representations, 2020.
[64] L. Yuan, F. E. Tay, G. Li, T. Wang, and J. Feng. Revisiting knowledge distillation via label smoothing regularization. IEEE Conference on Computer Vision and Puttern Recognition, 2020.
[65] S. Yun, J. Park, K. Lee, and J. Shin. Regularizing class wise predictions via self- -nflleeowedge distillation. IEEE Conference on Computer Vision and Pattern Recognition, 2020.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 6.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as URL)? [Yes] The code is available at https: /ttg//ggttu///t/ com/ www.otack//a/oooooo /iMT.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 5 and Appendix D.1 for the training details.
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] See Section 5.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix D.1 for details.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you're you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "rdNnH9f8LQQ", "reviewer": "Reviewer_mjLb", "summary": "The paper introduces a new method for few-shot learning that utilises a self-improving momentum target. The idea is to learn from a meta-model that is more quickly adapted than the standard meta-model - the momentum target is updated using exponential moving average rather than SGD. The approach can be combined with a wide variety of few-shot learners and consistently leads to strong improvements in performance.", "questions": "* Based on Figure 3b, it seems that even just using momentum network instead of the meta-model leads to strong performance. Is it the case that if we simply replace the outer loop SGD update with exponential moving average in MAML it would significantly improve its performance? Is it something which any other few-shot learners already do?\n* How easy is it to select the additional hyperparameters for SiMT?\n", "limitations": "Yes, the discussion of limitations and societal impacts is well-written.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The approach can be combined with various few-shot learning methods and consistently leads to strong improvements in performance.\n* The idea itself is intuitive and novel, even if the idea has some similarities to bootstrapped meta-learning and momentum networks\n* The paper provides excellent explanations and illustrations to show how the method works\n* The empirical evaluation is extensive and covers few-shot regression, few-shot classification (both in domain and across domains) and also reinforcement learning - with strong results across all\n* There is an ablation study evaluating the various components showing that all parts are important\n\nWeaknesses:\n* There are quite many additional components in the approach\n* The approach introduces several additional hyperparameters for which we need to select values (but perhaps selecting them is not too difficult)\n\nI’ve enjoyed reading the paper and the strengths significantly outweigh the weaknesses, so I recommend acceptance (the weaknesses are rather minor). The paper will likely be interesting and valuable for the ML community, especially for the researchers focused on meta-learning.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nozw_xmlIG", "reviewer": "Reviewer_x6gw", "summary": "This paper studies $\\mathcal{S}$-$\\mathcal{T}$ protocol in meta-learning, which means comparing a task-specific solver to a target model. The main contribution of this paper is proposing an efficient method to construct target models. The authors generate target models by adapting from the momentum network, which shows better adaptation performance than the meta-model itself. In the proposed method, the construction of target models and the training of meta-model are combined and iteratively performed. Experiment results demonstrate the effectiveness of the proposed method.", "questions": "Questions\n1. The target models are generated according to the momentum network. Could authors evaluate the quality of target models? Do these target models perform well on their tasks? \n2. Since the momentum network is updated during the meta-training process, early target models tend to be weak. Could authors give some explanations?\n3. In Equation (3), the authors evaluate the task-specific solver on $\\mathcal{Q}^{\\tau_{i}}$, which is different from [1]. Could authors show the advantages of doing so?\n4. In [1], the target models are constructed only for those hardest tasks (5% or 10%). It is interesting to see what will happen if only a part of meta-training tasks have target models. Do the number of target tasks have severe evident influences on the performance?\n5. The test accuracies in Table 2 are not satisfying. For example, with ResNet-12 backbone, we often achieve 65% (or higher) accuray on 5-way 1-shot miniImageNet using ProtoNet [2,3]. Is it because the authors do not pre-train the backbone on the meta-training split?\n\nReferences\n\n[1] Lu S, Ye H J, Gan L, et al. Towards Enabling Meta-Learning from Target Models[J]. Advances in Neural Information Processing Systems, 2021, 34: 8060-8071.\n\n[2] Ye H J, Hu H, Zhan D C, et al. Few-shot learning via embedding adaptation with set-to-set functions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 8808-8817.\n\n[3] Wang Y, Chao W L, Weinberger K Q, et al. Simpleshot: Revisiting nearest-neighbor classification for few-shot learning[J]. arXiv preprint arXiv:1911.04623, 2019.", "limitations": "The authors have included the limitations and future work in the paper, and some other limitations are listed in the *Strengths and Weaknesses* part.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. This paper studies an important and interesting problem in meta-learning, i.e., $\\mathcal{S}$-$\\mathcal{T}$ protocol, which may bypass the limitations of query sets.\n2. Experiments cover both few-shot learing and reinforcement learning, which are two main applications of meta-learning.\n3. This paper is well written and easy to understand.\n\nWeaknesses\n1. **Imperfect target models**. In Line 7 of Algorithm 1, we can see that the authors generate a target model according to current momentum network $\\phi_{\\mathrm{moment}}$. However, the momentum network is updated during the meta-training procedure, and it is weak in early epochs. The definition of target model is a strong model (expert) on a specific task, but the generated target models by the proposed algorithm are not optimal.\n2. **Insufficient interpretation**. Some technical details are not discussed adequately in the paper. For example, in Equation (3), the second terms stands for the loss of $\\mathcal{S}$-$\\mathcal{T}$ protocol. However, the authors evaluate the task-specific solver on the query set $\\mathcal{Q}^{\\tau_{i}}$ rather than the support set $\\mathcal{S}^{\\tau_{i}}$, which is different from [1]. In Line 146, the authors claim that the predictions of target models tend to generate soft predictions, which is not verified empirically or theoretically. Apart from this, the quality of generated target models are not evaluated. Since target models are experts on corresponding tasks, it is necessary to evaluate them.\n\nReferences\n\n[1] Lu S, Ye H J, Gan L, et al. Towards Enabling Meta-Learning from Target Models[J]. Advances in Neural Information Processing Systems, 2021, 34: 8060-8071.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XQoQUiLodSa", "reviewer": "Reviewer_DCr7", "summary": "This paper proposes a simple algorithm to use the benefits of the knowledge distillation (KD) in meta-learning. More specifically, since it is not possible to train an individual teacher for each meta-training tasks (which could be hundreds of thousands tasks in standard setup), this method obtains a momentum network using exponential moving average of the meta-model and then adapts this momentum network into each task using support set of that task. The adapted model is used as teacher to guide the main few-shot adaptation process.\n", "questions": "Please refer to the weaknesses part.\n", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n\nThis work introduces a very interesting idea to improve the generalization of the meta-learning algorithms. From KD point of view, it is similar to Born Again Neural Networks where a model becomes its own teacher. However, instead of training from scratch for several times, it uses momentum network to train the teacher (obtained by adaptation from this momentum network and called target network). Even though the algorithm is very simple, it works very well and provided results show improvement on baseline for different backbones and in different applications including classification, reinforcement learning and regression. \n\nThe writing is easy to follow and concepts are defined well, however, there are some concerns regarding this work.\n\nWeaknesses:\n\n\nProposed SiMT algorithm can be considered as a regularizer which prevents the baseline meta-learning algorithm from overfitting to meta-train classes. Since the results proposed in this paper are way behind state-of-the-art (SOTA) performance, I expect to see at least the comparison with other regularizer-like methods for few-shot learning like:\n- Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation, ICLR’20 \n\n\nIn addition, as the method can be considered in the line of works that uses KD for meta-learning, comparison with previous works needs to be included:\n- Towards enabling meta-learning from target models, NeurIPS’21\n\n\nFor few-shot classification results, as we cannot generally evaluate the whole possible set of meta-test tasks, the results usually presented as the mean and standard deviation over a batch of tasks. However, this paper just reports the std on three trials which is not accurate enough.\n\n\nFor RL results in Figure 2, why the std of the proposed model is much higher than the baseline? As SiMT uses momentum network, I expect to see less std in the proposed method.\n\n\nFor computational efficiency, the analysis is very limited and is presented just for a specific configuration.\n\n\nRegarding Table 4 (Ablation studies), what does it mean to use the momentum without distillation? Do you mean that the meta-learner is updated using momentum equation?\n\n\nIn Fig. 3(b) Configuration is not clear. If same config, why is this different from the 3(a)?\n\n\n\nThere are also some typos in the paper:\n\n- In line 118, it should be \\mathcal{L}(\\phi^{\\matchal{T}_i})\n\n- same problem with line 126, and equation (6)\n\n- in Fig 3(a) is should be 3X faster\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Kg4n0DHxy5", "reviewer": "Reviewer_siDw", "summary": "An algorithm based on knowledge distillation together with a momentum network working as experts on the unseen fields to solve meta-learning problems. The proposed algorithm shows compatibility with multiple existing meta-models and improvement. The general idea is simple and easy to implement.  ", "questions": "1. It seems like the main idea of the proposed method is to stitch a momentum network and knowledge distillation. What is the motivation for doing this? I think one thing could be good to show is that how the momentum network works on each task after the adaptation, since the knowledge is distilled from it.\n2. The adapted moving average parameter set, momentum, is adapted to the query set, to be an expert. Is it similar to say that the whole idea is to regularise the meta-model to its momentum model? \n3. Why the perturb (dropout) is important? Without dropout how does the model perform differently? ", "limitations": "The paper is presented in an unfriendly way for the readers who do not have an efficient background. In the beginning, especially in the introduction section, a lot of concepts, target model, original model and task-specific solvers, are introduced in the cause levels making the paper very confusing. Some components of the whole algorithm are explained very independently. It is not very clear why one relies on the other. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The experiment shows the good performance of the model on a variety of tasks. \n2. The model is analysed from different perspectives, loss landscape, and computational efficiency. \n\nWeakness: \n1. From my understanding, despite all the complicated presentation about all the algorithm components, the algorithm only forces the meta-model to get close to its momentum model which works as a regulariser. From this perspective, the novelty is quite limited. \n2. The proposed algorithm shows the improvement and compatibility on multiple MAML models but lacks comparison with other SOTA meta-models. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept