Meta-learning with an Adaptive Task Scheduler
Authors: Huaxiu Yao
Yu Wang
Ying Wei
Peilin Zhao
Mehrdad Mahdavi
Defu Lian
Chelsea Finn
Venue: NeurIPS 2021 Poster (2021)
Keywords: adaptive task scheduler
meta-learning
sampling

Abstract: To benefit the learning of a new task, meta-learning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning algorithms randomly sample meta-training tasks with a uniform probability, under the assumption that tasks are of equal importance. However, it is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. To prevent the meta-model from being corrupted by such detrimental tasks or dominated by tasks in the majority, in this paper, we propose an adaptive task scheduler (ATS) for the meta-training process. In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta-model to unseen tasks. We identify two meta-model-related factors as the input of the neural scheduler, which characterize the difficulty of a candidate task to the meta-model. Theoretically, we show that a scheduler taking the two factors into account improves the meta-training loss and also the optimization landscape. Under the setting of meta-learning with noise and limited budgets, ATS improves the performance on both miniImageNet and a real-world drug discovery benchmark by up to 13% and 18%, respectively, compared to state-of-the-art task schedulers.

Meta-learning with an Adaptive Task Scheduler
Huaxiu Yao'; Yu Wang", Ying Wei Peilin Zhao
Mehrdad Mahdavi", Defu Lian", Chelsea Finn'
Stanford University, FUniversity of Science and Technology, 3 Tencent AI Lab PPennsylvania State University, CCiyy University of Hong Kong (huaxiu,cbfinn] @es.stanford.eduu 2 Iwangyu,llandeffff @ustc. edu.en Wyingwei@cityu.edii "masonzhao @ tencent..om, mmzm61 psu.
Abstract
To benefit the learning of new task, meta-lcarning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning algorithms randomly sample meta-training tasks with uniform probability, under the assumption that tasks are of equal importance. However, it is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. To prevent the meta-model from being corrupted by such detrimental tasks or dominated by tasks in the majority, in this paper, we propose
adaptive task scheduler (ATS) for the meta-training process. In ATS, for the first time, we design a neural scheduler to decide which meta- training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta- model unseen tasks. We identify two peeta-model-relteee factors the input the neural scheduler, which characterize the difficulty a candidate task to the meta-model. Theoretically, show that scheduler taking the two factors into account improves the meta-training loss and also the optimization landscape. Under the setting of meta-learning with noise and limited budgets, ATS improves the performance on both minilmageNNet and a real-world drug discovery benchmark by up 13% and 18%, respectively, compared state-of-the-art task schedulers.
1 Introduction
Meta-learning has emerged in recent years as a popular paradigm to benefit the learning of a new task sample-efficient way. by meta-training a meta-model (e.g.. initializations for model parameters) from set of historical tasks (called meta-training tasks). To learn the meta-model during meta- training, the majority of existing meta-learning methods randomly sample meta-training tasks with uniform probability. The assumption behind such uniform sampling is that all tasks are equally important, which often not the case. First, some tasks could be noisy. For example, our experiments on drug discovery, all compounds (i.e., examples) in some target proteins (i.e., tasks) are even labeled with the same bio-activity value due to improper measurement. Second, the number of meta-training tasks is likely limited, that the distribution There are 2, 523 target proteins in the binding family among all 4, 276 proteins for meta- meta-trainin training. but only 55 of them belong to the ADME family. To fill the gap, meta-learning framework with a task scheduler that determines which tasks should be used for meta-training in the current iteration.
different clusters of tasks is uneven.
motivated to equip the
Recently, a few studies have started consider introducing a task scheduler into meta-learning, by adjusting the class selection strategy for construction of each few-shot classification task 17 19
H. Yao and Y. Wang contribute equally; correspondence to: Y. Wei
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

directly using a self-paced regularizer [3], or ranking the candidate tasks based on the amount of information associated with them [.]. While the early success of these methods is a testament the benefits of introducing a task scheduler, developing task scheduler that adapts to the progress of the meta-model remains an open challenge. Such a task scheduler could both take into account the complicated learning dynamics of meta-learning algorithm better than existing manually defined
schedulers, and explicitly optimize the generalization capacity to avoid meta-overfitting [2] 30].
To address these limitations, in this paper, we propose an Adaptive Task Scheduler (ATS) for meta- learning. Instead of fixing the scheduler throughout the meta-training process, we design a neural scheduler to predict the probability of each training task being sampled. Concretely, we adopt a bi-level optimization strategy to jointly optimize both the meta-model and the neural scheduler. The meta-model is optimized with the sampled meta- training tasks by the neural scheduler, while the neural scheduler is learned to improve the generalization ability of the meta-model on a set of validation tasks. The neural scheduler considers two meta-model model--ellaeed factors as its input: 1) the loss of the meta-model with respect to task, and 2) the similarity between gradients of the meta-model with respect to the support and query sets of task, which characterize task difficulty from the perspective of the outcome and the process of learning, respectively, On this account, the neural scheduler avoids the pathology of a poorly generalized meta-model that is corrupted by limited budget of tasks or detrimental tasks (e.g., noisy tasks).
The main contribution of this paper is an adaptive task scheduler that guides the selection of meta- training tasks for meta-learning framework. We identify two meta-modell-reatee factors as building blocks of the task scheduler, and theoretically reveal that the scheduler considering these two factors improves the meta-training loss as well as the optimization landscape. Under different settings (i.e.. meta-learning with noisy or a limited number of tasks), we empirically demonstrate the superiority of our proposed scheduler over state-of-the-an schedulers on both an image classification benchmark (up to 13% improvement) and real-world drug discovery dataset (up to 18% improvement). The proposed scheduler demonstrates great adaptability, tending to 1) sample non-noisy tasks with smaller losses if there are noisy tasks but 2) sample difficult tasks with large losses when the budget is limited.
2 Related Work
Meta-leaming has emerged as an effective paradigm for learning with small data, by leveraging the knowledge learned from previous tasks. Among the two dominant strands of meta-learning algorithms, prefer pradient-based 4 over metric-based 26 for their general applicability in both classification and regression problems. Much of the research up to now considers all tasks to be equally important, so that a task is randomly sampled in each iteration. Very recently, Jabri et al. [8] explored unsupervised task generation in meta reinforcement learning according to variations of a reward function, while in [T 20) a task is sampled from existing meta- veta-training tasks with the probability proportional to the amount of information it offers. Complementary to these methods specific to reinforcement learning. a difficulty-
meta-loss function [15] and a greedy class-pair based task sampling strategy (17) have been proposed to attack supervised meta-lcarning problems. Instead of using these manually defined and fixed sampling strategies, we pursue an automatic task scheduler that learns to predict the sampling probability for each task to directly minimize the generalization error.
There has been a large body of literature that is concerned with example sampling, dating back to importance sampling (12) and AdaBoost (5). Similar to AdaBoost where hard examples receive more attention, the strategy of hard example mining (25) accelerates and stabilizes the SGD optimization of deep neural networks. The difficulty of an example is calibrated by its loss [16), the magnitude of its
gradient (31), or the uncertainty (2). On the contrary, self-paced learning [13] presents the examples in the increasing order of their difficulty, so that deep neural networks do not memorize those noisy examples and generalize poorly M. The order is implemented by a soft weighting scheme, where easy examples with smaller losses have larger weights in the beginning. More self- self-paced learning variants [9 28 are dedicated to designing the scheme to appropriately model the relationship between the loss and the weight of an example. Until very recently, Jiang et al. [10] and Ren et al. [24] proposed to learn the scheme automatically from a clean dataset and by maximizing the performance on a hold-out validation dataset, respectively. Nevertheless, task scheduling poses more challenges than example sampling that these methods are proposed for.
2

---

Sampled tasks
Obtain Temporal meta-model 8
Training far
Reward
Meta- model fg
Update 0g
Candidate Tasks
Neural Scheduler +
Temporal Meta- model B
Task Pool
Val Tasks
Sample sampling probability
Update d
Obtain meta-model related factors
Optimize neural scheduler
Optimize meta-model
Illustration of ATS: (1) ATS calculates the meta-modes veta-model-related) factors for each candidate task (i.e., grey arrow). (2) ATS leverages the neural scheduler to sample tasks from the candidate tasks and use them learn the temporal meta-model ds. which in turn used to optimize the neural scheduler according to the feedback from validation tasks (i.e., blue arrow). (3) The updated neural scheduler is used to resample the tasks and update the meta-model (i.c., orange arrow).
Figure 1:
Preliminaries and Problem Definition
Assume that have task distribution p(T). Each task is associated with data set D., which is further split into a support set Dj and a query set Dj Gradient-based meta-learning [4) which we focus in this work, learns well-generalized parameter Og (a.k.a., meta-model) of a base predictive learner f from N meta-training tasks {Tr) The base learner initialized from Og adapts to the i-th task by taking k gradient descent steps with respect to its support set (inner-loop optimization), i.e., 0: = 0o Va ;)). To evaluate and improve the initialization, we measure the performance of 02 on the query DV and the corresponding loss to optimize the initialization as (out-loop optimization):
N
0ft+2) 0
> L(D);
where a and B denote the learning rates for task adaptation and initialization update, respectively. After training K time steps, we learn the well-generalizee model parameter initializations 0j During the meta-testing time, Of can be adapted each new task T7 by performing few gradient steps on the corresponding support set, i.e., 0, 0g a a. (oD) 0).
In practice, without loading all N tasks into memory, a batch of B tasks {T for training the k-th meta-training iteration. Most of existing meta-learning algorithms the uniform sampling strategy, except for a few recent attempts towards manually defined task schedulers (e.g.. (7). Either the uniform sampling or manually defined task schedulers may be sub-optimal and at a high risk of overfitting.
| are sampled
4 Adaptive Task Scheduler
To address these limitations, we aim to design an adaptive task scheduler (ATS) in meta-learning to decide which tasks to next. Specifically, illustrated in Figure we design neural scheduler to predict the probability of each candidate task being sampled according to the real-time feedback of meta-modellbased factors at each meta-training iteration. Based on the probabilities, we sample B tasks to optimize the meta-model and the neural scheduler in an alternating way. In the following subsections, detail task scheduling strategy and bi-level optimization process.
4.1 Adaptive Task Scheduling Strategy
The goal for this subsection is to discuss how to select the most informative tasks via ATS. We define the scheduler as y with parameter & and formulate the sampling probability w. of each candidate task T; at training iteration as
wi (k)
g(Ti, // ()))
(
3

---

where w. (8) is conditioned on task Tr and the meta-model 0a4) at the current meta- veta-training iteration. To quantify the information covered in T, and o we here propose two representative factors: 1) the loss C(D;; ) on the query set, where 0! is updated by performing few- www-gradient steps starting from o 2) the gradient similarity between the support and target sets with respect to the current meta-model ( i.e., (v ( (() o ((D;; Here, use inner product as an exemplary similarity measurement, other metrics like cosine similarity can also be applied in practice. They are associated with the learning outcome and learning process of the task Ti, respectively. Specifically, the gradient similarity signifies the generalization gap from the support to the query set. large query loss may represent true hard task the gradient similarity is large; task with noises in its query set, however, could lead to large query loss but small gradient similarity. Considering these two factors simultaneously, we reformulate Eqn. (2) as:
(K) W
(C(D) ))
(v C(D;: og ((D)) :
(3)
After obtaining the sampling probability weight wº directly use it to sample B tasks from the candidate task pool for the current meta-training iteration, where larger value of with) represents higher probability. The out-loop optimization is revised as:
B E 0ft)
0(++1) o
4.2 Bi-level Optimization with Gradient Approximation
In this subsection, we aim to discuss how to jointly learn the parameters of neural scheduler 6 and the meta-model Bu during the meta-training process. Instead of directly using the meta peta-training tasks to optimize both meta-model and the neural scheduler, ATS aims to optimize the loss on a disparate validation set with N, tasks, i.e., (T.))=m, where the performance on the validation set could be regarded as the reward or fitness. Specifically, ATS searches the optimal parameter o' of the neural scheduler by minimizing the average loss *.. E (To 0 (Ø)) over the validation tasks, where the optimal parameter 6% is obtained by optimizing the meta-training loss R Σ (er(Te; 80, 6) over the sampled tasks. Formally, the bi-level optimization process is formulated as:
No
1 Σ Coae (To; (T))), where (b()) min B (T= min No
1
(5)
It is computational expensive to optimize the inner loop in Eqn. 5 directly. Inspired by the differentiate hyperparameter optimization [8,, we propose a strategy to approximate (o) by performing one gradient step starting from the current meta-model g0 as:
B
()) so o((+t) = of .V 0
Σ
Lar (Tr 000)) )
B
(6)
1=1
s.t. 0 = 00 L(''
where the task-specific parameter g is adapted with few gradient steps starting from a As such, for each meta- veca-training iteration, the bi-level optimization process in Eqn. (5) is revised as:
No. E
min ((E) No
Lual (Toi Coai(To;
s.t., 0
g(++1) (k)) (0)))
(k)
2V 000
1 Σ La (T: ,00))
B
We then focus on optimizing the neural scheduler in the outer loop of Eqn. 7 In ATS, tasks are sampled from the candidate task pool according to the sampling probabilities wi (k) It is intractable to directly optimizing the validation loss Leet, since the sampling process is nin-differentiable. We thereby use policy gradient method to optimize ((k), where REINFORCE 29 is adopted. We also

---

Algorithm Meta-training Process with ATS
Require: learning rates a, 8, task distribution p(7), batch size B, candidate task pool size Nvoot
1: Initialize the meta-model 6g and the neural scheduler ø
2: for each training iteration k do
3: Randomly select Noool tasks and construct the candidate task pool
4: for each task Ti in the candidate task pool do
5: Compute two factors L(D;; ( and vv L
( V ogg L(D): by using the
support set Di and the query set D 6: Calculate the sampling probability w(t) by Eqn. end for
7:
8:
Sample B tasks from the candidate task pool via the sampling probabilities w
9:
Calculate the training loss and obtain temporal meta-model via I -step gradient descent
Sample N. validation tasks
Calculate the accuracy (reward) R using the temporal meta-model (
12:
Update the neural scheduler by Eqn. (8) and get g(E+1)
13:
Sample another B tasks (T: Ji=1 by using the updated task scheduler g(++1)
Update the meta-model as: 06 ()++1) (g)*)) =0² 3V ( Ei= (T: (((+1))
14:
15:
end for
equip a baseline function for REINFORCE to reduce the gradient variance. Regarding the accuracy of each sampled validation task T, the reward R(a), define the optimization process
(k+1)
(k)
VV aek) log ((W()) (R)
R
b).
(8)
where W(A) = (w) Mes and the baseline à defined as the moving average of validation accuracies. After updating the parameter of neural scheduler o, use it to resample B tasks from the candidate task pool and update the meta-model from 8/4) 0(++1) via Eqn. The whole optimization
algorithm of ATS is illustrated in Alg. I
5 Theoretical Analysis
In this section, we would extend the theoretical analysis [6] to our problem of task scheduling, to theoretically study how the neural scheduler g improves the meta-training loss well as the optimization landscape. Without loss of generality, here we consider a weighted version of ATS with hard sampling, where the meta-model On is updated by solving the meta-training loss weighted by the task sampling probability w; g(T, 0o; Φ) over all candidate tasks in the task pool, i.e.,
Nroot
0g arg min Op
w. w,D((D; 0, =0 (oL((D):))
(9)
Denote the meta-training loss without and with the task scheduler as L(fo) Npuot Ei=1 L(D?:0.)
Noool
and L. (Bo) = EmT w:L(D); 0c), respectively. Then we have the following result:
Novot
Proposition Suppose that w= [W1, [ denotes random variable for sampling prob- abilities, Ce, [C(D; fo),. L(D Npont Bu)) denotes the random vuriable for the loss using the meta-model, and V 9u I(Vo, C(D;; 9o), Vo, L(D;; (:;;00 fo)) denotes the random variable for the inner product between gradients of the support and query sets with respect the mote- Then the following equation connecting the meta-learning losses with and without the task scheduler holds:
(V ...)...., C(Ds, (N fo): Vo, C(D2 ppoat fo)))
(6)) = L(6)) + Cov(Ls,, w) aCov V 8o, w).
(10)
From Proposition] 1 we conclude that the task scheduler improves the meta-training loss, as long as the sampling probability is negatively correlated with the loss but positively correlated with the gradient similarity between the support and the query set. Specifically, if the loss L(D), 0,) is large
5

---

as result of a quite challenging noisy task T; the sampling probability w; is expected to be small. Moreover, a large value of w, is anticipated. when a large inner product between the gradients of the support and the query set with respect to the meta-model (Vo, (D;; 0o), Vo, (DD 0o)) signifies that the generalization gap from the support set Dj to the query set DD is small.
Consistent with [6], the optimal meta-model is assumed to also minimize the covariance Cov(Ce, w) and maximize the covariance Cov( (eo. w), i.c., 0 arg argmin [()b) arg min (Cov(Cag w) (Ve, aCov( Ve, w)J. Under this assumption, the task scheduler does not change the global minimum, i.e.
6 arg min ((6)) = arg min L (9b). while modifies the optimization landscape as the following.
Proposition 2. With the sampling probability defined
6 -1
(11)
the following hold:
(00) L (06) 2 ((6)) L(06), voo: Cov our (Gog -o Vor a oogh (00) - (06) ((0)) L(06).
v0u: Cov V Gor
(Fog -oVog) 20
Proposition 2 sheds light on how the optimization landscape is improved by an ideal task sched- uler: 1) for those parameters 8g that far from the optimal meta-model fj (i.e., Cov(Ca, a <Voo, e (Gog (Vogg) 2 0), the gradients towards the direction of 0j become overall steeper for speed-up; 2) for those parameters Oo that are within the variance of the optimum (i.e., Cov(Co, - a Veg -(Cog "Voo))
-Var(Co; oo;))) )), the minima tends be flat with bet- ter generalization ability [4]. Though the optimal meta-model 0f remains unknown and the ideal task scheduler with the sampling probabilities in (TT) is inaccessible, we learn neural scheduler to dynamically accommodate the changes of both the loss Le, and the gradient similarity V Ve.. Detailed proof's of Proposition andd 2 are provided in Appendix A.
Experiments
In this section, we empirically demonstrate the effectiveness of the proposed ATS through compre- hensive experiments on both regression and classification problems. Specifically, two challenging settings are studied: meta-learning with noise and limited budgets.
Dataset Description and Model Structure. We conduct comprehensive experiments on two datasets. First, we use minilmagenet as the classification dataset, where we apply the conventional N-way, K- shot setting to create tasks 4 For minilmagenet, we use the standard model with four convolutional blocks, where each block contains 32 neurons. We report accuracy with 95% confidence interval over all meta-testing tasks. The second dataset aims to predict the activity of drug compounds [21), where each task an assay covers drug compounds for one target protein. There are 4,276 assays in total, and we split 4, 1,100/ / 76 100 tasks for meta- meta-training / validation/ testing, respectively. We use two fully connected layers in the drug activity prediction as the base model, where each layer contains 500 neurons. For each assay, the performance is measured by the square of Pearson coefficient (N?) between the predicted values and the actual values. Follow (2), report the mean and medium I²² as well as the number of assays with R² > 0.3, all of which are considered as reliable metrics in the pharmacology domain. We provide more detailed descriptions of datasets in Appendix B.
In terms of the neural scheduler o. we separately encode the loss on the query set and the gradient similarity by two bi-directional LSTM network. The percentage of training iterations are also fed into the neural scheduler to indicate the progress of meta-training. Finally, all encoded information are concatenated and feed into two-layer MLP for predicting the sampling probability (More detail descriptions are provided in Appendix B.2).
Baselines. We compare the proposed ATS with the following two categories of baselines. The first category contains easy-to-implement example sampling methods that can be adapted for task scheduling, including focal loss (oss) [16] and self-paced learning loss (SPI.) 13] The second category covers ofate-of-thh-arr task schedulers for meta-learning that are non-adaptive, which includes DAML [15), GCP 17 and PAML []]. Note that GCP is class- -lass-driven task scheduler

---

Table Overall performance on meta-learning with noise. For minilmagenet, we report the average accuracy with 95% confidence interval. For drug activity prediction, the performance is evaluated by mean R², medium R², and the number of assays with R² >
minilmagenet-noisy
Drug-noisy medium
Model
5-way I-shot
5-way 5-shot
>0.
Uniform
41.67 = 0.80%
55.80 + 0.71%
0.202
0.113
21
SPL
42.13 0.79%
56.19 = 0.70%
0.211
0.138
24
FocalLoss
41.91 = 0.78%
53.58 + 0.75%
0.205
0.106
23
GCP
41.86 + 0.75%
54.63 + 0.72%
N/A
N/A
N/A
PAML
41.49 + 0.74%
52.45 + 0.69%
0.204 0.197
0.120
24
DAML
41.26 + 0.73%
55.46 + 0.70%
0.113
24
ATS (Ours)
44,21 + 0.76%
59.50 + 0.71 %
0.233
0.152
31
* means the result significant according Student's T-test level 0.01 compared to SPL.
and thus it can not be applied to regression problems such as drug activity prediction here. We also slightly revise the loss of DAML that it can be applied to both classification and regression problems. For all baselines and ATS, we use the same base model and adopt ANIL [22] as the backbone meta-learning algorithm.
6.1 Meta-learning with Noise
Experimental Setup. We first apply ATS on meta-learning with noisy tasks, where each noisy task is constructed by only adding noises on the labels of the support set. Therefore, each noisy task contains noisy support set and clean query set. In this way, adapting the peta-model on the support set causes inaccurate task-specific parameters and leads to negative impacts on the meta-training process. Specifically, for minilmagenet, we apply the symmetry flipping on the labels of the support set [27]. The default ratio of noisy tasks is set as 0.6. For the drug activity prediction, we sample the label noise from a Gaussian distribution y * (0, 1), where the noise sealar 1) used to control the noise level. Note that, empirically we find that the effect of adding noise on the drug activity prediction is not as great as adding noise on the minilmagenet, and thus we add noise to all assays and use the scalar 1 to control the noise ratio. By default, we set the noise scalar as 4 during the meta-training process. Besides, since ATS uses the clean validation task, for fair comparison, all other baselines also fine-tuned on the validation tasks. Detailed experimental setups and hyperparameters are provided in Appendix C.
Results. Table I reports the overall results of ATS and other baselines. Our key observations are: (1) The performance of adaative task schedulers (i.e., GCP, PAML,, DAML) similar to the uniform sampling, indicating that manually designing the task scheduler may not explain the complex dynamics of meta-learning, being sub-optimal. (2) ATS outperforms traditional example sampling methods and non-
160
Noisy
120
Frequency
adaptive task schedulers, demonstrating its effectiivness on improving the robustness of meta-learning algorithms by adaptively adjusting the scheduler policy based on the real- time feedback of meta- meta-model--ne medel-related factors. The findings are further strengthened by the distribution comparison of sampling weights between clean and noisy tasks in Figure 2 where ATS pushes most noisy tasks to small weights (i.e., less contributions in meta-training).
40
0.0
0.1
0.2
0.3
0.4
0.5
Sampling Weights
Figure 2: Distribution comparison of sampling weights between clean and noisy tasks.
Ablation Study. We further conduct an ablation study under the noisy task setting and investigate five ablation models detailed follows.
Rank by Sim/Loss: the first ablation model, we heuristically determine and select tasks by ranking tasks according to a simple combination of the loss and the gradient similarity, i.c., Sim/Loss. We assume that tasks with higher gradient similarity but smaller losses should be prioritized.
Random de Random é, we remove both retae factors, where the model parameter o of the neural scheduler is randomly initialized at each meta- training iteration.
7

---

Table 2: Ablation Study under the meta-learning with noise setting.
minilmagenntt-oisy
Drug-noisy medium
Ablation Model
5-way I-shot
5-way 5-shot
mean
>0.3
Random &
41.95 + 0.80%
56.07 = 0.71%
0.204
0.100
22
Rank by Sim/Loss
42.84 + 0.76%
$7.90 0.68%
0.181
0.109
22
LosLos
42,45 + 0.80%
56.65 0.75%
0.212
0.122
6+-Sim
42.28 + 0.82%
56.71 + 0.72%
0.214
0.122
29
Reweighting
42.19 + 0.80%
56.48 + 0.72%
0.217
0.118
28
ATS (d+Losss+iim)
44.21 + 0.76%
59.50 +: 0.71 %
0.233
0.152
31
means the result is significant according to Student's T-test level 0.01 compared Weighting
Table 3: Performance w.r.t. Noise Ratio. Under the minilmagenet 1-shot setting (Image), the noise ratio is controlled by the proportion of noisy tasks. In drug activity prediction, the noise ratio is determined by the value of noise scaler 17. BNS represents the best non-adaptive scheduler.
Noise Ratio
0.2
0.4
0.6
0.8
Uniform BNS
43.46 + 0.82%
42.92 0.78%
41.67
0.80%
36.53 + 0.73%
Image
44.04 1 0.81%
43.36 + 0.75%
42.13 + 0.79%
38.21 + 0.75%
ATS (Ours)
45.55 1: 0.80%
44.50 1 0.86%
44.21
0.76%
42.18 0.73%
Noise Scaler
1/=2
r/44
8/=
Uniform BNS
0.222
0.139
26
0.202
0.113
21
0.196
0.131
22
0.194
0.100
21
Drug
0.229
0.136
31
0.211
0.138
24
0.208
0.116
24
0.200
0.101
24
ATS (Ours)
0.235
0.160
33
0.233 0.152
31
0.221
0.136
28
0.219 0.133
28
means all results are significant according to Student's T-test at level 0.01 compared to BNS
Loss or 6+Sim:: The third (é+Loss) and forth (ó+Sim) ablation models remove the gradient similarity between the support and query sets, as well as the loss of the query set, respectively.
Reweighting: Instead of selecting tasks from the candidate pool, in the last ablation model, we direct reweigh all tasks in meta batch, where the weights are learned via the neural scheduler.
We list the results of all ablation models in Table 2 where ATS (o++ is also reported for comparison. The results indicate that (1) simply selecting tasks according to the ratio Sim/Loss significantly underperforms ATS since the contribution of each metric is hard to manually define. Besides, the contribution of each metric evolves as training proceeds, which has been ignored such simple combination but modeled in the neural scheduler with the percentage of training iterations as input; (2) the superiority of @+Looss and @++Smm over Random şb shows the effectiveness of both the query loss and the gradient similarity; (3) the performance gap between Reweight+Loss+Sin and ATS is potentially caused by the number of effective tasks, where more candidate tasks are considered by ATS; (4) including all meta--oodl--elateed factors (i.e., ATS) achieves the best performance, coinciding with our theoretical findings in Section 5
Effect of Noise Ratio. We analyze the performance of ATS with respect to the noise ratio and show the results of minilmagenet and drug activity prediction in Table The performances the uniform sampling and the best non-adaptive scheduler (BNS) are also reported for comparison. We summarize the key findings: (1) ATS consistently outperforms the uniform sampling and the best non-adaptive task scheduler, indicating its effectiveness of adaptively sampling tasks to guide the meta-training process; (2) With the increase of the noise ratio, ATS achieves more significant improvements. In particular, the results of ATS in minilmagenet almost stable even with very large noise ratio, suggesting that involving an adaptive task scheduler does improve the robustness of the model.
Analysis of the Meta-model- Factors. To analyze our motivations about designing the meta- model-related factors, we randomly select 1000 meta-training tasks and visualize the correlation between the sampling weight w and each factor in Figures Ba 3d In these figures, we rank the sampling weight w² and normalize the ranking to [0, 1], where a larger normalized ranking value is associated with a larger sampling weight. Then, we split these tasks into 20 bins according to the rank of sampling weights. For tasks within each bin, we show the mean and standard error of their query losses and gradient similarities. According to these figures, we find that tasks with larger losses are
8

---

(md Sim betwen 27 an0 DD
betwors DPt mi
0.5-
0.025
Low Etoo.
0.2
Loe ctp%.
0.5
0,000
0.1
0.3
0.4
0,015
0.2
5,000
0.3
0.000
0.5
0.35
0.25
0.5
0.75
1.0
6.35
0.25
Normalioed Rank Weigho
Nommalzedd
Nortel lood Rank Weight
Nomallzed Rack Weights
(a) minilmagenet
(b) minilmagenet
(c) Drug
(d) Drug
Figure 3: Correlation between sampling weight w, and (a)&(c) query set loss L(D) 8.); (b)&(d); gradient similarity between Di and Dj under the meta-learning with noise setting. The larger normalized rank of weights correspond to larger sampling weights.
Table Overall performance on meta-learning with limited budgets. For minilmagenet, control the number of meta-training classes. For drug activity prediction, all meta-training tasks are used for meta-training under this setting.
minilmagenet-Lininn
Drug-Fui Full medium
Model
5-way 1-shot
5-way 5-shot
>0.3
Uniform
33.61 + 0.66%
45.97 + 0.65%
0.233
0.140
33
SPL
34.28 + 0.65%
46.05 + 0.69%
0.232
0.135
29
FocalLoss
33.11 = 0.65%
46.12 + 0.70%
0.229
0.140
28
GCP
34.69 + 0.67%
46.86 + 0.68%
N/A
N/A
N/A
PAML
33.64 + 0.62%
45.01 + 0.69%
0.238
0.144
32
DAML
34.83 + 0.69%
46.66 + 0.67%
0.227
0.141
28
ATS (Ours)
35.15
0.67% %
47.76 + 0.68 %
0.252
0.179
36
means the result is significant according to Student's T-test level 0.01 compared PAML
associated with smaller sampling weights, verifying our assumption that noisy tasks (large query loss) tend to have smaller sampling weight (Figures 3a 30). Our motivation is further strengthened by the fact that tasks with more similar support and query sets have larger sampling weights (Figures 3b 3d).
6.2 Meta-learning with Limited Budgets
Experimental Setup. We further analyze the effectiveness of ATS under the meta-learning setting with limited budgets. Follow [4) In few-shot classification problem, each training episode is few-shot task by subsampling classes as well as data points and two episodes that share the same classes are considered be the same task. Thus, we treat the budgets in meta-learning number of meta-traie meta-training tasks. In minilmagenet, the original number of meta-training classes is 64, corresponding more than 7 million 5-way combinations. Thus, we control the budgets by reducing the number of meta-training classes to 16, resulting in 4,368 combinations, For drug activity prediction, since it only has 4. 4.100 tasks, do not reduce the number tasks and use the full dataset for meta-training. We provide more discussions about the setup in Appendix D.
the
We report the performance Aligning with the meta-learning with noise setting, ATS consistently outperforms other baselines with limited budgets. Besides, compared with the results in minilmagenet, ATS achieves more significant improvement in the drug activity prediction problem under this setting. This is what we expected - as we have discussed in Section the drug dataset contains noisy and imbalanced tasks.
Results.
minilmagenet and drug activity prediction in Table 4
The effectiveness of ATS is further strengthened by the ablation study under the limited budgets setting, where report the results in Table 5 Similar to the findings in the noisy task setting, the ablation study further verifies the contributions of the proposed two meta-mode meta-moodll-rlateee factors on the meta-tret veta-training process.
Analysis of the Meta-model-relatee Factors. Under the limited budgets setting, we analyze the 4a
correlation between the sampling weight and the reeaeee factors in Figure 4a
4d
From
these figures, we can see that the gradient similarity indeed reflects the task difficulty (Figure 4b 4rd
where larger similarities correspond to more useful tasks (i.e., larger weights). Interestingly, the

---

Table 5: Performance (aceuracy + 95% confidence interval) of different ablated versions of AT' under the setting of meta-learning with limited tasks.
minilmagenet-Limice
Drug-Full medium
Ablation Model
5-way 1-shot
5-way 5-shot
mean
>0.3
Random &
33.97 0.63%
46.37 + 0.70%
0.238
0.159
35
Rank by Sim/Loss
33,42 = 0.64%
46.38 + 0.70%
0.187
0.099
24
@+Looss
34.08 0.66%
46.48 + 0.67%
0.241
0.171
36
@+-ii
34.46 0.65%
47.34 0.70%
0.246
0.161
34
Reweighting
35.03 0.65%
46.70 + 0.65%
0.248
0.158
ATS (d+Loss+Sim)
35.15 0.67%
47.76
0.68 %
0.252
0.179
Stn betwenn DS Dt!
0.6
Low c(0)-8))
2.0
0.50
Loo cipfte
0.5
0.30
1.8
0.57
0.3
9.4
0.20
1.6
0.56
0.15
9.1
Sun
0.85
0.10
0.25
0.75
1.0
0.25
0.25
0.75
1.0
Narmalized Rank Waights
Nomalized
Weights
Nomallzz Rank Weights
Nomallzzd Ratk of Weigtt
(a) minilmagenet
(b) minilmagenet
(c) Drug
(d) Drug
Figure 4: Correlation between weight wy and (a)&(c) query loss; (b)&(d); gradient similarity, Larger normalized rank of weights correspond to larger sampling weights.
correlation between the query loss and the sampling weight under the limited budgets setting is opposite to the correlation under the noisy setting. is expected since tasks with larger query losses are associated with "more difficult" tasks under this setting, which may contain more valuable information that further benefits the meta-training process.
Effect of the Budgets. In addition, we analyze the effect of budgets by changing the number of meta-training tasks in minilmagenet. The results of uniform sampling, GCP (best non-adaptive task scheduler), ATS under 1-shot scenario are illustrated in Table 6 We observe that our model achieves the best performance in all scenarios. In addition, compared with uniform sampling, ATS achieves more significant improvements with less budgets, indicating its effectiveness on improving the meta-training efficiency.
Table 6: Performance w.r.t budgets (the number of meta-training classes). Accuracy + 95% confidence interval is reported.
Budgets
16
32
48
64
Uniform
33.61 0.66%
40.48
0.75%
44.07 + 0.80%
45.73 +: 0.79%
GCP
34.69 + 0.67%
41.27 + 0.74%
44.30 + 0.79%
45.35 + 0.81 %
ATS (Ours)
35.15 + 0.67%
41.68 + 0.78%
44.89
0.79%
46.27 + 0.80 %
7
Conclusion and Discussion
This paper proposes a new adaptive task sampling strategy (ATS) to improve the meta-training process. Specifically, we design neural scheduler with two meta-model-related factors. At each meta-training iteration, the neural scheduler predicts the probability of each meta-training task being sampled according to the received factors from each candidate task. The meta- meta-model and the neural scheduler are optimized a bi-level optimization framework. Our experiments demonstrate the effectiveness of the proposed ATS under the settings of meta-learning with noise and limited budgets.
One limitation in this paper is that we only consider how to adaptively schedule tasks during the meta- deta-training process. In the future, it would be meaningful to investigate how to incorporate the task scheduler with the sample scheduler within each task. Another limitation is that using ATS is more computationally expensive than random sampling since we alternatively learn the neural scheduler and the meta-model. would be interesting to explore how to reduce the computational cost, e.g., compressing the neural scheduler.
10

---

Acknowledgement
This work was supported in part by JPMorgan Chase & Co. Any views or opinions expressed herein solely those of the authors listed, and may differ from the views and opinions expressed by JPMorgan Chase & Co. or its affiliates. This material is not product of the Research Department of J.P. Morgan Securities LLC. This material should not be construed an individual recommendation for any particular client and is not intended as a recommendation of particular securities, financial instruments strategies for a particular client. This material does constitute solicitation offer in any jurisdiction. This work was also supported in part by the start-up grant from City University of Hong Kong (9610512). Besides, Y. Wei would like to acknowledge the support from the Tencent AI Lab Rhino-Bire Gift Fund. D. Lian was supported by grants from the National Natural Science Foundation of China # 62022077.
References
[1] Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In ICML, pages 233-242, PMLR, 2017.
[2] Haw-Shiuan Chang, Erik Learned- Miller, and Andrew McCallum. Active bias: training more accurate neural networks by emphasizing high variance samples. In NeurlPS, pages 1003-1013, 2017.
[3] Dong Chen, Lingfei Wu, Siliang Tang, Fangli Xu, Juncheng Li, Chang Zong, Chilie Tan, and Yueting Zhuang. Robust meta-learning with noise via eigen-reptile, 2021.
[4] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In ICML, pages 1 1122-11355 2017.
[5] Yoav Freund and Robert E Schapire. A decision-theoretie generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):1 19 139, 1997.
[6] Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep networks. In ICML, pages 2535-2544. PMLR, 2019.
[7] Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. In UAI, pages 876-885, 2018.
[8] Allan Jabri, Kyle Hsu, Ben Eysenbach, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Unsupervised curricula for visual meta-reiiforrement learning. In NeurIPS, 2019.
[9] Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander Hauptmann. Self-paced curriculum learning. In AAAI, volume 29, 2015.
[10] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, pages 2304 2313. PMLR, 2018
[11] Jean Kaddour, Steindór Sæemundsson, and Mare Peter Deisenroth. Probabilistic active meta- learning. 2020.
[12] Herman Kahn and Andy W Marshall. Methods of reducing sample size in monte carlo compu- tations. Journal of the Operations Reseurch Society of America, 1(5):263-278, 1953.
[13] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In NewrlPS, volume 1, page 2, 2010.
[14] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In NewrIPS, pages 6391-6401, 2018.
11

---

[15] Xiaomeng Li, Lequan Yu. Yueming Jin, Chi-Wing Fu, Lei Xing, and Pheng-An Ann Heng. Difficult-awaree meta-learning for rare disease diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 357-366. Springer, 2020.
[16] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.
[17] Chenghao Liu, Zhihao Wang, Doyen Sahoo, Yuan Fang, Kun Zhang, and Steven CH Hoi. Adaptive task sampling for meta-learning. arXiv preprint arXiv:2007.08735, 2020.
[18] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv: 180.005500 2018.
[19] Su Lu, Han-Jia Ye, and De-Chuan Zhan. Support-target protocol for meta-learning. arXiv preprint arXiv:2/04.03336, 2021.
[20] R Luna Gutierrez and M Leonetti. information-theoreti task selection for meta-reinforreement learning. In NeurIPS. Leeds, 2020.
[21] Eric J Martin, Valery R Polyakov, Xiang-Wa Zhu, Li Tian, Prasenjit Mukherjee, and Xin Liu. All-assay-maxe pqsar: Activity predictions as accurate as four-concentration ic50s for 8558 novartis assays. Journal of chemical information and modeling. 59(10)44450 4459, 2019.
[22] Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv: 1909.09157, 2019.
[23] Janarthanan Rajendran, Alex Irpan, and Eric Jang. Meta-learning requires meta-augmeetatioon In International Conference on Learning Representations, 2020.
[24] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. ICML, pages 4334 4343. PMLR, 2018.
[25] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region- based object detectors with online hard example mining. In CVPR, pages 761-769, 2016.
[26] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurlPS, pages 4080-4090, 2017.
[27] Brendan Van Rooyen, Aditya Krishna Menon, and Robert C Williamson. Learning with symmetric label noise: The importance of being unhinged. arXiv preprint 1.XXiv:505.076:: 2015.
[28] Yixin Wang, Alp Kucukelbir, and David M Blei. Robust probabilistic modeling with bayesian data reweighting. In ICML, pages 3646-3655 PMLR, 2017.
[29] Ronald J Williams. Simple statistical folliig algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-25) 1992.
[30] Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn. Meta- learning without memorization. In International Conference on Learning Representations, 2020.
[31] Peilin Zhao and Tong Zhang. Stochastic optimization with importance sampling for regularized loss minimization. In ICML, pages 1 1-9. PMLR, 2015.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 7
(c) Did you discuss any potential negative societal impacts of your work? [No] This work proposes general adaptive sampling framework for meta-learning, which does not have extra negative societal impacts beyond meta-learning.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section and Appendix A.
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 5 and Appendix
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] see the URL in supplemental material
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes] See the subsection "Experiments Setup" in Section and Section
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See all the tables in Section 6
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix C. and D.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [Yes] These datasets are public datasets and we have cited the related reference.
(c) Did you include any new assets either in the supplemental material or URL? [No] We will open-source the code once the paper is accepted.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "lV44hVTgPC", "reviewer": "Reviewer_nPYJ", "summary": "This paper proposed Adaptive Task Scheduler (ATS)\u0010 for meta-learning, which trains a neural network to adaptively select tasks. A bi-level optimization strategy is used to optimize the the meta-model and neural scheduler. Experiments are conducted on miniImagenet  and drug compounds, which beats the baseline methods. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is a good attempt on the adaptive task scheduler. The proposed algorithm totally make sense to me. In general, the paper is well-written and easy to follow. The experimental results are nice. \n\n\nTypos:\nLine 140: computationally\nEquation (6): the definition of $\\theta$ should be $\\theta_{i}^{(k)}$\nEquation (7): should be $\\theta_{i}^{(k)}$ in the bottom equation.\nEquation (9): should be $\\theta_{i}=\\theta_{0}-\\alpha \\nabla_{\\theta} \\mathcal{L}\\left(\\mathcal{D}_{i}^{s} ; \\theta_0\\right)$", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think the story brought by Proposition 1 is questionable. Indeed, certain sampling probability $w$ could make the meta-training loss lower. However, meta-training loss is not our ultimate goal. Our goal is the performance on the test set. In some extreme cases where some tasks available in the training phase have very low noise, such that one would give them high sampling probability to reduce the meta-training loss. However, they might be unrelated to some tasks in the test phase. So I am not sure how meaningful is Proposition 1.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "WuSA8DX1rns", "reviewer": "Reviewer_jUKV", "summary": "The author proposed a dynamic sampling approach to form meta learning tasks, with the sampling distribution determined by 1) loss of the tasks, 2) the similarity between gradient of support and query data. Experimental results on the setting of noisy meta learning and limited budgets demonstrate the effectiveness of the proposed approach.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1)   I agree on the author’s intuition which is to sample out tasks that  are informative and not outliers. The author used loss to determine on the difficulty and used the alignment of gradient between support and query data to determine outlier. However, this alignment of gradient can’t infer the task is not an outlier, i.e., when the tasks are from more than one domain, task of another domain can align perfectly by itself but still of little value to the current domain. The algorithm design needs improvement to cover such cases.\n\n2)   The author used LSTM stacks to process on the two factors. Given meta learning models are usually light weight by itself, it would be beneficial to know on the computational cost from this added component and how it impacts the training efficiency.\n\n\n3)    Better to add a little description on motivation of adding label noise in the data. i.e. will this make it more close to real world scenarios? Or it is for a fair comparison with previous works and less bias prone? etc.\n\n4)  The paper works on noisy meta learning setting and demonstrate the effectiveness of the proposed approach. I would like to know what is the performance on the normal meta learning setting without noise. \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discussed limitations.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "EmQGCkbXllS", "reviewer": "Reviewer_n8ad", "summary": "In this paper, the authors propose an adaptive task scheduler to pick the set of tasks that a meta-learner should solve at a given iteration which will improve its generalizability/transferability to downstream tasks rather than picking a set of tasks at random. Instead of choosing a metric/heuristic based approach, the authors propose to use a second network that learns-to-learn the right set of tasks to sample and the overall training progresses via an interleaved (bi-level) optimization procedure. Authors provide empirical evidence based on two datasets that their proposed scheduler ATS performs better than similar existing approaches. The paper also contains some theoretical justifications on why ATS performs better than random sampling.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In this paper, the authors propose an adaptive task scheduler to pick the set of tasks that a meta-learner should solve at a given iteration (rather than picking a set of tasks at random) with an aim to improve its generalizability/transferability for downstream tasks. The authors propose to use a learning based approach by using a second network to automatically determine the weightage of each task from the task pool and then picking the top K tasks from that pool. For the meta-learning part, the authors use a standard gradient based learner (ANIL) and for the scheduling-learner part, they use an LSTM based network and use REINFORCE to train the scheduler network. The authors perform experiments on one classification and one regression task and show that their proposed task scheduler performs better than existing ones when it comes to the final test performance. Authors also provide some theoretical justifications as to why their method provides better generalization compared to random selection.\n\nOverall, the work is good and the paper was easy to follow - the narrative and the notations were all clear. The idea also has enough novelty, especially using another network to select the \"hardest\" tasks (based on support-query difference and query loss) and then using REINFORCE to train that in a DARTS-style (NAS) bi-level setup.\n\nI have a few concerns that I would like authors to address:\n*  The authors mention using two LSTMs for the task selector network. Why was LSTM chosen? What is the temporal piece of information that you wanted to capture?  \n*  The authors perform some ablation studies but there is one that I think is missing - what if, instead of using another network to learn the right task to choose, you just heuristically determine the best tasks by performing some combination of the two metrics selected -- the  query loss and the gradient similarity between the support and query? For example, what happens if you just pick the top K tasks by taking a simple average of these two metrics instead of having another network that takes these as inputs and then learns to pick the top K tasks? Will there be a significant performance difference between these two?\n*  To me, it looks like there will be non-trivial computational overhead in adopting this approach vs random sampling. I'd like to see some training time comparison between the methods which will be more informative for a user to decide if the extra computation gain is worth it.\n* For MiniImageNet, the authors mention that number of tasks is $4368$ when $16$ classes are chosen because $16  \\choose 5$ is $4368$. However, while picking $5$-way tasks, you also need to pick $5$-shots i.e. choosing $5$ images per class from $600$ images. Therefore, number of distinct tasks is way more than $4368$. I would like the authors to clarify why they mention $4368$ as the total number of tasks in this case. \n* Finally, pardon my naivety but I do not follow the derivation of proposition 1 (given in appendix). How did you go from line 1 to line 2 in the proof (starting at line 6 in the appendix)? Also, in the manuscript, it should be mentioned that the proof is given in the appendix. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There are no potential negative societal impact of their work as far as I understand.", "review_text": "", "extra_scores": {"ethical_concerns": "There are no ethical concerns of their work as far as I understand.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5-6  hours"}, "extra_text": {}}, {"review_id": "16xU_hwWHW6", "reviewer": "Reviewer_VBvz", "summary": "Meta-learning is a popular strategy to learn a well-generalized model from different meta-training tasks. Classical meta-learning assumes equal importance for all the meta-training tasks. Recent work has proposed sampling of the tasks based on various factors such as difficulty, amount of information, etc. The paper presents an extension to these ideas by proposing an adaptive task scheduler. In contrast to the prior approaches where the task distribution is hard-crafted and fixed, the authors propose a learnable task scheduler that outputs the sampling probability score of each task in a task pool to directly minimize the generalization error. The inputs to the task scheduler - the query loss and the similarity between the gradients on the query and support set are theoretically grounded and well-motivated. The meta-model and the tasks scheduler are learned using a bi-level optimization; in particular, REINFORCE is used to update the task scheduler. The authors conduct experiments on the mini-imagenet (classification) and drug discovery (regression) datasets to validate the effectiveness of the task scheduler. The results indicate the robustness of the task scheduler on both the datasets under noisy conditions and limited task budget.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Meta-learning is among the popular approaches for learning generalizable models. An aspect of this learning paradigm that could be further optimized was task selection. While the recent approaches along this direction use a fixed or hand-crafted method to select the tasks, this paper proposes a novel method to learn the distribution automatically. This is a valuable contribution. I liked the theory motivating the two factors input to the automatic task scheduler. This is the strongest contribution of the paper. I am sure that this will spawn new focussed research on improving the task scheduler for meta-learning. Having said that, there are a couple of issues with the methodology as discussed below.\n\nRandomly selecting $N^{pool}$ of tasks is understandable due to computational constraints. However, I do not see the need for a second round of task selection from this candidate pool based on the estimated weights. Why cannot the meta-model be trained using all the $N^{pool}$ tasks? According to Section C.1, the meta-batch size is 2 and $N^{pool}=10$. Is there any issue with $N^{pool}=4$ and the meta-batch containing all the tasks in the $N^{pool}$? I ask this as the theory seems to be more aligned with weighted task loss rather than sampled task loss.\n\nExperiments on the noisy mini-imagenet and drug discovery datasets validate the effectiveness of the ATS. The ablation studies are also adequate. However, there are two issues with the experimental setup, as elaborated below.\n\n- The authors restrict the evaluation to only the noisy version of the mini-imagenet. While ATS might work best on noisy data, an optimal ATS module should work on any generic meta-learning task (even ones without noise). The bare minimum would be to validate ATS on the original mini-imagenet, tiered-imagenet, and CIFAR-FS benchmarks widely adopted in the literature. Table 3 suggests that ATS does not yield significant improvements with reduced noise levels. Thus, it is natural to gauge the performance on a ‘noiseless’ dataset. No noise in the dataset does not mean the tasks are easily learnable. The suggested benchmark datasets are extremely challenging and are perhaps hard learning tasks. This will test the performance of ATS on truly hard tasks. I agree that ATS might perform better on highly noisy datasets (noise level of 60% and above), but such settings as illustrated in the paper are quite unrealistic.\n\n- ATS is compared only against other sampling strategies. In my opinion, this is an unfair comparison. Liu et al. eccv 2020 [16] - GCP; compares the performance of various meta-learning approaches on the standard benchmark datasets using different backbones. I believe it is necessary to compare the performance of the ATS with some of these approaches. This will address a fundamental question - is task selection critical for meta-learning? The paper implicitly assumes it and does not present compelling evidence. It would be worthwhile to adopt and compare the sampling strategies on meta-learning approaches that allow for it, instead of restricting only to the ANIL framework. \n\nIn my humble opinion, addressing the following comments will further strengthen the contribution.\n\n1. The results on all the datasets require additional significance tests. This is especially true with the drugs dataset where the difference in the mean $\\mathcal{R}^2$ is in the second decimal place.\n2. Line 268 - Are the weights for the 1000 meta-training tasks estimated at the same time by the ATS? If so, at what stage in the training process is these weights estimated?\n3. The authors mention in the supplementary material that warm-start initialization refers to the meta-model that is pre-trained using the original datasets (without noise). I think this is unfair, as the meta-model has already seen the clean meta-training tasks. The proposed training process is only trying to maintain the performance. This also negates the primary assumption of the paper - training with noisy and hard tasks. I believe a fair comparison warrants training from scratch on the noisy datasets directly.\n4. Line 231 mentions a set of clean validation tasks for ATS. What is the role of this validation set? This can be further added to Algorithm 1.\n5. Would you please explain the pre-training process of a neural scheduler? Is the meta-model $\\theta_0$ used during pre-training randomly initialized or pre-trained? If $\\theta_0$ is pre-trained, is it pre-trained along the lines of supervised learning on meta-train set or episodic regimen? Does pre-training of neural scheduler $\\phi$ follow episodic regimen? Does pre-training of $\\phi$ and $\\theta_0$ both occur on clean data? Is pre-trained $\\theta_0$ used as such (warm start), or $\\theta_0$ is reinitialized during the iterative optimization phase?\n\nThe paper is well structured and written. The ideas are easy to understand and well-articulated.\n\nCorrections to the theory in the supplementary material----\n\n- In the proof of proposition 1, we are adding and subtracting two terms $\\mathcal{L(D_i^q;\\theta_i})$ and $\\sum_i^{N^{pool}}w_i\\mathcal{L}(D_i^q;\\theta_i)$. The second term in the first step of the proof should be $-\\sum_{i=1}^{N^{pool}}w_i(\\mathcal{L}(D_i^q;\\theta_i)-\\mathcal{L}(D_i^q;\\theta_i))$\n\n- Also, the second order term in equation 3 of the supplementary material is incorrectly stated (Though it does not impact the rest of the analysis). It should be $\\alpha^2 \\nabla_{\\theta_0}\\mathcal{L}(D_i^s;\\theta_0)^T\\nabla_{\\theta_0}^2\\mathcal{L}(D_i^q;\\theta_0)\\nabla_{\\theta_0}\\mathcal{L}(D_i^s;\\theta_0)$\n\nTypos\n\nLine 141 - differentiate - differentiable\n\nLine 203 - follow - following\n\nPost response************\nI thank the authors for providing a detailed response to my queries.\n\nI am satisfied with the authors' response to most of the questions. I still have a few lingering questions/suggestions that will further add clarity to the results\n\nQ2 and Q3 I am not convinced that the results in Table 5 represent the full budget setting. The accuracies for MAML in the 5.1 and 5.5 settings stand at 48.7% and 63.1% respectively as reported in the literature. However, the table reports only 45.7% accuracy. Furthermore, in the response to Q3, the authors state MAML performance as 47.09% and 63.1% for the two settings. So, why are the accuracies for the classical MAML different at different places within the paper?\n\nQ4 - I am assuming that the term after $\\pm$ is the standard deviation. Significance tests do not mean only reporting the standard deviation, but checking the p-value for this the results are significant. At the moment, no such test has been performed.\n\nOverall, I am inclined to raise my rating of the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The idea of the tasks scheduler for meta-learning is interesting. The solution is technically sound and intuitive. The theoretical analysis is also interesting, although it requires more discussions on when the assumption can hold. The experiments are quite exhaustive. The discussion on the computational cost is non-trivial, and more detailed analysis is needed
Decision: Accept (Poster)