Meta-Learning under Task Shift
Authors: Lei Sun
Yusuke Tanaka
Tomoharu Iwata
Venue: Accepted by TMLR ()

Abstract: A common assumption in meta-learning is that meta-training and meta-test tasks are drawn from the same distribution. However, this assumption is often not fulfilled. Under such task shift, standard meta-learning algorithms do not work as desired since their unbiasedness is no longer maintained. In this paper, we propose a new meta-learning method called Importance Weighted Meta-Learning (IWML), which preserves unbiasedness even under task shift. Our approach uses both labeled meta-training datasets and unlabeled datasets in tasks obtained from the meta-test task distribution to assign weights to each meta-training task. These weights are determined by the ratio of meta-test and meta-training task densities. Our method enables the model to focus more on the meta-training tasks that closely align with meta-test tasks during the meta-training process. We meta-learn neural network-based models by minimizing the expected weighted meta-training error, which is an unbiased estimator of the expected error over meta-test tasks. The task density ratio is estimated using kernel density estimation, where the distance between tasks is measured by the maximum mean discrepancy. Our empirical evaluation of few-shot classification datasets demonstrates a significant improvement of IWML over existing approaches.

Published in Transactions on Machine Learning Research (09/2024)
Meta-Learning under Task Shift
Lei Sun
sun.lei.ss5 sn..iss...... Www...aiis...........
Nara Institute of Science and Technology
Yusuke Tanaka
ysk. tanaka @ytt. com
NTT Communication Science Laboratories
Tomoharu Iwata
tomohara twnla Ond.. com
NTT Communication Science Laboratories
Reviewed on OpenReview: https: openreview. net/ forum? id- OnV75W90FM
Abstract
A common assumption in meta-learning is that meta-training and meta-test tasks are drawn from the same distribution. However, this assumption is often not fulfilled. Under such task shift, standard meta-learning algorithms do not work as desired since their unbiasedness is no longer maintained. In this paper, we propose a new meta-learning method called Importance Weighted Meta-Learning (IWML), which preserves unbiasedness even under task shift. Our approach uses both labeled meta-training datasets and unlabeled datasets in tasks obtained from the meta-test task distribution to assign weights cach meta- training task. These weights are determined by the ratio of meta-test and meta-training task densities. Our method enables the model to focus more the meta-training tasks that closely align with meta-test tasks during the meta-training process. We meta-learn neural network- based models by minimizing the expected weighted meta-training error, which is an unbiased estimator of the expected error over mela-test lasks. The task density ratio is estimated using kernel density estimation, where the distance between tasks is measured by the maximum mean discrepancy. Our empirical evaluation of few-shot classification datascts demonstrates a significant improvement of IWML over existing approaches.
1 Introduction
The ability to learn quickly from a limited number of examples a major distinction between humans and artificial intelligence (AI). Humans can acquire new tasks rapidly, leveraging their past learning experiences. On the other hand, most modern AI methodologics their utility to that particular task alone. Therefore, when faced with new tasks, it is required to newly gather huge amount of data for training. However, collecting sufficient training data is often expensive and time-consuming, presenting a considerable challenge in real-world applications. Meta-learning, a widely used approach adopted to address this challenge, employs the transferable knowledge gained from meta-training tasks to boost. the learning efficacy for meta-test tasks (Vilalta & Drissi, 2002; Nichol al., 2018; Vanschoren, 2018; Santoro et al., 2016; Khodak al., 2019). When formulating meta-learning approach, it generally assumed that the meta-training and mela-test lasks are drawn from the same distribution (Yoon et al., 2018; Li et al,, 2017; Rajeswaran et al., 2019; Yin al., 2019; Zintgraf et al., 2019). This assumption, however, is frequently unfulfiledd. This mismatch hampers our ability to learn transferable knowledge that could be beneficial for meta-test tasks, thereby leading to erroneous predictions. We refer to this phenomenon, where meta-training and meta-test tasks follow different distributions, as 'task shift::
trained specifically for single task, which restricts
Standard meta-learning algorithms (Finn al., 2017; Snell al., 2017; Garnelo al., 2018) optimize model parameters by minimizing the expected meta-training error. However, these methods fall short under task shifts, the expected meta-training error does not provide unbiased estimate of the expected meta-test

---

Published in Transactions on Machine Learning Research (09/2024)
error. To address task shifts, a method called Weighted Meta-Learning was proposed by Cai et al. (2020). This approach aims to find weights for each meta-training task to minimize the distance between the meta- training tasks and single meta-test task. However, it assumes that there is only one meta-test task, requires labeled data from tasks that appear in the meta-test phase for weight calculation, and is limited to regression scenarios. These limitations hinder its application in real-world settings. Additionally, lask augmentation strategies (Rajendran et al., 2020; Yao et al., 2021a; Wu et al., 2022) aim to generate tasks that are not provided during the meta-training phase to improve performance on mela-test tasks. Yet, in the case of task shifts, the newly generated tasks follow the meta-training task distribution and fail to enhance performance tasks from the meta-test task distribution.
In this paper, we introduce a new meta-learning approach called Importance Weighted Meta-Learning (TWML), designed to reduce misestimation caused by task shift. Our method assigns weights to each meta- training task, reflecting the importance of a meta-training task for learning meta-test tasks. This weight is determined by the ratio of meta-test and meta-training task densitics. Our method enables the model to focus more on the mela-training tasks that closely align with meta-lest. tasks during the meta-training process. In the proposed method, meta-learn neural network-based models by minimizing the expected weighted meta-training error, which is an unbiased estimator of the expected error over meta-test tasks. We estimate these densities using labeled meta-tras veta-training datasets and unlabeled datasets in tasks obtained from the meta-test task distribution. The task density ratio is estimated using Kernel Density Estimation (KDE), where the distance between tasks is measured by the Maximum Mean Discrepancy (MMD). Figure illustrates the meta- vetta-taainin procedure of IWML.
The proposed method is applicable in situations where we have datasets from multiple tasks that are related to the meta-training tasks, but they follow different distributions. For example, consider image classification problem in different scenarios where the meta-training datasets consist of many tasks composed of images taken by ordinary cameras, and the meta-test datasets consist of many tasks composed of images taken under a microscope. Even though the images come from different distributions, they are related.
The following are the main contributions of this paper:
We propose a new meta-learning method that provides an unbiased estimator of the expected error over meta-test tasks, even under a task shift.
We propose task density estimation method by leveraging KDE in conjunction with the MMD distance.
To the best of our knowledge, our work represents the first effort to tackle the issue of task shift meta-learning by leveraging unlabeled datasets from tasks derived from the meta-test task distribu- tion. Furthermore, reasonable weights, even when the tasks utilized for weight computation are different from those in the meta-test phase.
experimental results demonstrate that method can accurately estimate
We empirically demonstrate that our proposed method outperforms existing meta-learning algo- rithms, the weighted meta-learning method, and state-of-the-art task augmentation strategics under a task shift.
2 Related works
2.1 Covariate shift
The covariate shift problem is common issue in conventional supervised learning scenarios (Shimodaira, 2000). This problem arises when supervised learning algorithms fail to provide an unbiased estimator be- cause the feature vectors used for training and test come from different distributions, while the conditional distributions of the output given the feature vectors remain unchanged.
The covariate shift problem related the task shift problem in that both involve differences in distributions between the training and test phases. However, the covariate shift problem occurs within single task, where
2

---

Published in Transactions on Machine Learning Research (09/2024)
the distribution of feature vectors shifts between the training and test phases. In contrast, the task shift problem in meta-learning refers to a shift in the distribution of tasks themselves between the meta-training and meta-test phases. Various methods have been developed to address the covariate shift problem by estimating weights and applying them to the training feature vectors (Sugiyama et al., 2007; Bickel et al., 2009; Sugiyama et al., 2008). However; these methods are inapplicable to lask shift. Although these mot and the proposed method both use weights to calculate an unbiased error, they are only capable of computing weights for each sample within single lask. In cases where it is necessary to compute weights for each task, such in task shift situations, these methods are inapplicable.
2.2 Unsupervised domain adaptation
Although the methods developed to address Unsupervised Domain Adaptation (UDA) and the proposed method both make use of unlabeled test datasets to improve performance, these UDA techniques are typically designed for scenarios involving only two tasks: training and test. As such, they are not adaptable to meta- learning frameworks, which involve multiple meta-training and meta-test tasks (Finn et al., 2017; Yao et al., 2019; Snell et al., 2017).
UDA aims to transfer knowledge from a labeled source domain to an unlabeled larget domain. The dominant approaches for UDA focus on acquiring domain-invariant features and can be categorized into two principal strategies. The first strategy aims to explicitly minimize domain discrepancies by utilizing measures such Maximum Mcan Discrepancy (MMD) (Long et al., 2015; 2017; Borgwardt et al., 2006) to evaluate domain similarities. Meanwhile, other studies introduce metrics based on second-order or higher-order statistical differences (Sun et al., 2016; Sun & Saenko, 2016). An alternative approach employs adversarial training to develop domain-invariant features (Saito et al., 2018; Sankaranarayanan et al., 2018), a method pioneered by the work on Anvees--d Neural Networks (DANN) (Ganin & Lempitsky, 2015; Ganin et al., 2016). This technique involves training a domain discriminator to distinguish between source and target domains while concurrently training a feature extractor to deceive the discriminator, leading to the alignment of features.
2.3 Meta-learning
A number of frameworks have been proposed for meta-learning (Hospedales et al., 2021), including gradient- based (Finn et al., 2017; Li et al., 2017; Yoon et al., 2018; Finn et al., 2018), metric-based (Snell et al., 2017; Koch et al., 2015; Vinyals et al., 2016), and black box-based (Garnelo et al., 2018; Mishra et al., 2017; Hochreiter et al., 2001) methods. Among them, Model-agnostic Meta-Learning (Finn el al., 2017), Prototypical Networks Snell et al. (2017), and Conditional Neural Processes (Garnelo et al., 2018) serve as reprosentative methods for each respective framework. These existing meta-lcaruing methods assume that meta-training and meta-test tasks adhere to the same distribution, and they optimize model parameters by minimizing the expected meta-training error. In Section 3.2, we explain that in cases of task shift, due to inconsistencies between expected meta-training and meta-test errors, optimizing model parameters by minimizing the expected meta-training error does not guarantee a reduction in meta-test error.
Yao et al. (2021b) have conducted study on meta-learning and proposed a method called Adaptive Task Scheduler (ATS). Although both ATS and our method assign weights to each meta-training task, there are several differences between the two methods. The main difference between lies in the problems they aim to solve. ATS introduces weights to optimize the generalization capacity of the model and to avoid meta-overfitting, whereas our proposed method addresses meta-learning under task shift. These are two distinct problems,, and because of the differences in the problems they solve, the methods for calculating weights are also different. ATS calculates weights based on the loss on the query set and the gradient similarity between the support set and the query set. In contrast, our method calculates weights based on the ratio of meta-test and meta- training task densities. While ATS is effective in optimizing the generalization ion capacity of the model and avoiding overfitting, it is not designed to address task shift issues.
proposed method and ATS
3

---

Published in Transactions on Machine Learning Research (09/2024)
2.4 Weighted meta-learning
Cai et al. (2020) have conducted study on meta-learning under lask shift and proposed a method called Weighted Meta-Learning (WML). In WML, it is assumed that there is only one meta-test task, and weights are assigned to each meta-training task to minimize the distance between the single meta-test task and multiple meta-training tasks. While we also introduce a method that assigns weights to meta-training tasks, our approach differs from WML in several aspects. (1) WML assumes there only one meta-test task, whereas our proposed method can consider multiple meta-test tasks by using the meta-test distribution. (2) WML requires recalculating the weights when applied to new meta-test task, whereas our proposed method can be applied to unseen mela-lest. tasks that follow the meta-test task distribution without the need for weight recalculation. (3) WML nccossitates labeled datasets from tasks involved in the meta-test phase for weight computation. In contrast, our proposed method allows for the use of unlabeled datasets from tasks that are drawn from the meta-test distribution but do not appear in the meta-test phase to compute weights. (4) WML restricted to regression scenarios and constrained to the use of square and hinge loss as its loss functions. Conversely, our proposed method is not only applicable in regression scenarios but also extends to multi-classsficationn scenarios, offering flexibility with limitations the choice of the loss function, parameter learning methods, and models.
2.5 Task augmentation strategies
The objective of task augmentation strategies in meta-learning is to generate tasks that not provided during the meta-training phase, thereby enhancing the performance of unknown tasks that are presented during the meta-test phase. To accomplish this objective, several existing methods have been developed. For instance, Meta-Aug (Rajendran et al., 2020) enhances a task by adding random noise to the labels of both support and query sets, while Meta-Mix (Yao et al., 2021a) blends support and query examples within a lask. Furthermore, Adversarial Task Up-sampling (ATU) (Wu et al., 2022) aims to train a network that up-samples tasks adversarially, generating tasks that align with the true task distribution.
These task augmentation methods can improve performance for unseen tasks by generating additional tasks in scenarios where mela-training and mela-test lasks share the same lask distribution. However, when the meta-training and meta-test tasks follow different task distributions, the newly generated tasks, derived from the meta-training task distribution, fail to improve the performance on tasks from the meta-test task distribution.
3
Preliminary
3.1 Problem setting
In standard meta-learning, the model is trained and evaluated on episodes of few-shot learning tasks. During the meta-training phase, let us consider we have a collection of meta-training tasks {T,TR) ] drawn from mela-training lask distribution p (T). Each task is associated with labeled meta-training dataset DJR [x]] yin () ) where x E RRD the D-dimensional feature vector of the n-th instance in the I-th meta-
TR)) NTR 1 training task, Btn is the corresponding label, 7 TR is the number of veeattraining tasks, and N,TR is the number of instances.
In the meta-test phase, are given meta-test task T,TE drawn from meta-test task distribution y TE(T). associated with labeled dataset S = {(x,, Da) )N which is called support set. Here, the number of instances Ns is small. The goal is to predict label for each feature vector x, which is called a query, in the meta-test task.
3.2 Meta-learning
In standard learnig,, the expected error over meta-training tasks (T.RR) It= is calculated by:
TR = ETH
(1)
4

---

Published in Transactions on Machine Learning Research (09/2024)
where E denotes the expectation, support set and query set are denoted S {xn, Un J and Q (xn: Y/ W which contains pair of feature vectors and labels randomly generated from associated datasets of each task [DTR)TTT 1==
(Q25:0) No
E Elv, X.VER
(2)
is the loss on the query set given the support set, Ne is the number of paired data in the query set, 0 is parameters of model f, and fxx, S; is the output of the model for imput x, adapted using support set S. The loss function represented by Ly. ff (x,,S;0 S))) measures the discrepancy between true label at imput feature vector x and its estimate f(x, S; 0). Similarly, the expected error over target tasks is calculated from associated support set S and query set Q:
RTE -TTTT-TE(T) (pTE(T) (xs.)))
(3)
In the situation where p (T) (T). the expected meta-training error R matches the expected er- ror over meta-test tasks TE Therefore, optimizing the model parameters by minimizing R ensures a corresponding reduction in ATE However, when p (T) * (T), referred to as task shift, expected meta- training error TR and expected error over meta-test tasks R do not match: RTR + RTE, Consequently, general meta-learning methods are designed to minimize RTR by optimizing model parameters However,
in cases of task shift, this approach does not ensure that HT will be minimized as well.
4 Proposed method
4.1 Problem setting
In this paper, we consider a scenario where there is a task shift, meaning that the meta-training and meta- test tasks follow different probability distributions, but the supports of meta-training and meta-test. task distributions are overlapped.
During the meta-training phase, consider we have a set of meta-training tasks ) each drawn from p (T). Additionally, we also have TE meta-test tasks obtained from p (T)))) where each task is associated with unlabeled dataset T { (x]]E) {(x]
NT ] Here, x the D-dimensional feature vector of the n-th instance in the t-th meta-test task, and T,TE is the number of instances. It is important to note that these meta-test tasks, associated with unlabeled data, may differ from those in the meta-test phase. The settings for the meta-test phase are consistent with standard learnng..
4.2 Importance Weighted Meta-Learning
To address task shift, we propose Importance Weighted Meta- Mearning (IWML), a method that estimates model parameters 9 by minimizing the following expectod weighted meta-training error RIWML This error is an unbiased estimator of the expected error over the meta-test task distribution:
RIWLL Ey, En TR() w.B.s...)--rp((((1..00)0)
(4)
where
w, = p (T.TR) TR (T.R).
(5)
indicates the importance of T.TR, reflects how crucial this task is for solving meta-test tasks. T (T, indicates the meta-test task density of meta-training task T, while p (TR(T,TR) represents its meta-training task density.
5

---

Published in Transactions on Machine Learning Research (09/2024)
Meta-training phase
Meta-training task distribution
Meta-training task
Data task
(T)
Sampling
Labeled data D
R
Sampling
Adaptation
Query set
Support set
Backpropagation
Meta Meta-training task T
Importance estimation
Importance estimation
Labeled data TR
Evaluation
Adaptation
Assign weight
Meta-test task distribution p (T)
Input/Output
Mela-test task
Model
Importance of the task
Wz
Unlabeled data
Assign weight
W
Weighted error
Meta losa-tesftttt tesf'task task
TE
Backpropagation
S Support set
Unlabeled data
Q Query
Meta-test phase
Small labeled data in target task
Input
Adaptation
Trained
model
Output
Figure 1: Illustration of the IWML algorithm. During the meta-training phase, our model is trained by minimizing the weighted meta-tres eeta-training error across various meta- testa-traniig tasks. In the meta-test phase, the trained model predicts label y for individual feature vector x using a support set.
Under the task shift, expected weighted meta-training error IMML and expected mela-lest error TT match:
(T. (T.
RMWML
E ((s.)))-7-((()))) p (R(TT de
(T / (1s.0) (T, dt
(T
E [1s.0)-077((((1):000)
RE
(6)
given that expected weighted meta-training error RIWML and expected meta-test error RT match, optimiz- ing the parameter by minimizing IIWML can enhance performance on tasks following the meta-test task distribution.
4.3 Importance Estimation
In the proposed method, we use the ratio of the densities of meta-test and meta-training tasks at feature vector distribution of task T to approximate the density ratio of TE (Tt) and y (R.))
p TE(T))
p (T((((x))
(7)
TR (T.) (x)))
where (x) represents the feature vector distribution of task Te. In many meta-learning studies (Yao et al., 2019; Wu al., 2022), task Te is represented by corresponding labeled datasct D, {(x Mtn) N= In our setting, we only have unlabeled datascts in meta-test tasks obtained from the meta-test task distribution in the meta-training phase, which does not allow to represent the task by its labeled dataset. Therefore,

---

Published in Transactions on Machine Learning Research (09/2024)
assume that marginal feature vector distribution ((x) carries information about joint distribution q(x, y). This assumption ion allows us to only use set of unlabeled data to gain insight into the task. This supposition aligns with the cluster assumption found in semi-supervised learning (Ouali et al., 2020; Van Engelen & Hoos, 2020). This assumption posits that feature vectors, which are close to each other in the feature space, likely share the same label. Based on this assumption, we can leverage g(x) to gain useful insights into g(x.)) y). In Section 5, we validated this assumption through experiments.
We employ the Kernel Density Estimation (KDE) methodology to estimate the density of the feature vector distribution. According to the KDE, the densities of meta-test and meta-training calculated by:
T
dx(((x),, 11 (x))) )
TE (9(())) = ZTE TT
Σ exp
(8)
211
P=1
and
T
-d(q, (x). qp TT (xx)
p (g) = Z T
exp
(9)
2h²
where d: (x) represents the feature vector distribution of t'-th meta- training task, q. (x) represents the feature vector distribution of f'-th meta-test task, 122 is a smoothing scalar parameter, commonly referred to as the bandwidth, and d(q (x). (T)) represents the distance between (xxx) and q. (x). Normalization factors denoted TE and Z TR, are employed to ensure that the sum of the estimated densities equals one. They are calculated by:
T
TEE
[d(g,(x), ge (xx )
Z TE
TTE
exp
2h²
(10)
t=1
p
and
T
TR
((((((())
1r (x))
TR = Σ Σ exp
(11)
TTR
2h12
1=1
t=11
where (x) (xx) for t .,..., TR (x) TE 77- (x) for t TR ..... T, total number of meta-training and meta-test tasks represented by T TE + TR.
To estimate the distance between two distributions, we use Maximum Mean Discrepancy (MMD) (Borgwardt et al., 2006; Gretton et al., 2012). MMD is a distance measure of two probability distributions or two sets of points. Given two sets of feature vector Us = {xtm Nh ) drawn from (x),, U.E (x drawn from qe (x), empirical estimation of the MMD between these distributions is calculated by:
N7
d(g((x), MMD 2.,
(12)
where,
N
N
N
TE
2
Σ
Σ
k(Xn, x.))
k(Xn, x(n)
N(N - 1) n=1 nzt''
NNTE
(13)
TE NT Σ
Σ
M(XTE TE TT)
(NTE
where 1: a kernel. Considering that U and UTE are sets of high-dimensional feature vectors, such as a collection of images, we can use the deep kernel (Wilson al., 2016) for the calculating MMD. Utilizing deep kernels allows for the acquisition of representations particularly tailored to the given data, which
7

---

Published in Transactions on Machine Learning Research (09/2024)
potentially enhance the performance of KDE. Therefore, employ function 96 for feature extraction in deep Gaussian kernel calculation, where function 9e IR_ - R with parameters & serves as an encoder
for feature extraction, Here, M is the dimension of the feature vector after extraction. The deep Gaussian kernel used in Eq. 13 is given by
lig (xns X,x) exp (-190((x)) (x go((xx))1 )1²)
(14)
20
where o² is a smoothing scalar parameter, commonly referred to as the bandwidth. Similar Eq. 12, the distance between qe (x) and (x((x) used in Eq. 9 is calculated by:
d(<:(x), (x)) (x))
(MD?
(15)
where U = [x
TRI I drawn from de (x).
Through the importance estimation process, the weight w, of meta-training lask T, is calculated by Eq. 5. The meta-test and meta-tres cesta-trannig densities meta-metaanne testa-trainig task in Eq. are estimated using Eq. 8 and Eq. 9, based on feature vectors in [ and (4TE1T=1 shown in Algorithm 1.
M1=1 The importance estimation procedures of IWML are
4.4 Meta-training procedure
We estimate model parameters 0 and encoder parameters ó in two steps.
Step 1: We estimate parameters & of encoder y for kernel k in Eq. 14. In our experiments, we use prototypical networks (Snell et al., 2017) to train parameter ø meta-training tasks (T,TR)ZT) by minimizing the expected (unweighted) meta-training loss in Eq. meta-training datasets (D.R)TTT
Step 2: We estimate model parameter by minimizing expected weighted meta-training loss RIWML in Eq. 4 on meta-training datasets (D] Ji=1 The weight of 6-th meta-training task w, in Eq. 4 is calculated by Algorithm using t-th meta-training dataset and unlabeled meta-test datasets ( J1=1
In Step 2, prototypical networks (Snell et al., 2017) and model-agnostic meta-learning (Finn al., 2017) are used to train parameters 8 in our experiments. The parameters can be trained with various meta-learning methods, such as Meta-SGD (Li et al., 2017), and neural processes (Garnelo et al., 2018). Similarly to 0, parameters o of encoder g also can be trained with various methods. The meta-training procedures of IWML are shown in Algorithm 2.
5
Experimental Evaluation
5.1 Datasets
We evaluated our proposed method using three datasets: minilmageNeet, Omniglot, and tieredImageNet. The miniImageNet dataset, first introduced by Vinyals (Vinyals al., 2016), subset of the larger ILSVRC-12 dataset (Ruseakovsky et al., 2015). It consists of 60,000 color images, each 84x84 pixels in resolution, divided into 100 classes. Each class contains 600 examples.
Omniglot dataset (Lake et al., 2011) includes 1,623 unique, hand-drawn characters from 50 different al- phabets. Each character is associated with 20 images, each 28x28 pixels in resolution, all of which were produced by unique human subjects.
The ticredImageNet dataset, initially proposed by (Ren et al., 2018). comprises 779,165 color images 608 classes, grouped into 34 higher-level nodes within the ImageNet human-curated hierarchy. This set of nodes is further divided into 20, 6, and disjoint sets for training, validation, and test, respectively. Ren
8

---

Published in Transactions on Machine Learning Research (09/2024)
Algorithm Importance estimation procedure of IWML
Input: Meta-training datasets {DI unlabeled meta-test datasets {4, IT encoder Output: Weight w, for t-th meta-training task
1: for E T do
2: Calculate distance between the f-th meta- eeta-training task and f'tth meta-test task by Eq. 13 using the feature vectors DJR and U
end for
for j T do
Calculate distance between the t-th meta-training task and t'-th meta-training task by Eq. 13 using the feature vectors in D. and DJR
6: end for
7: Calculate normalization factors Z and Z TR by Eq. 10 and Eq. Estimate p (T,T)) and (T, by Eq. 8 and Eq. 9 Calculate weight w, = p (T, (T,R)
dleta-training procedure of IWML: RandomSample(S,.)) generates set of N elements chosen uniformly at random from set S without replacement.
Algorithm 2
Input: Meta-training datasets {D!RR Ji=1, unlabeled meta-test datasets {u ||= number support in- stances Ns, number of query instances NQ. batch size B. Output: Model parameters 0
1: randomly initialize 0, initialize loss t-
2: while not done do Seleet task indices for a mini-batch, M RandomSamplo(f1,, TTR], B) for E M do
Generate support set, S Random/Samlle(D! Ns)
Generate query set, Q < RandomSamplee D S, NQ)
Calculate weight W, by Algorithm 1 using TR and [UTTETT
7:
Calculate weighted loss by Eq. and Wt J J + w, LQ and its gradients
end for
10:
Update model parameters 0 using loss J/B and its gradient
11: end while
al. (2018) contend that splitting closer to the root of the ImageNet hierarchy leads to a more challenging but more realistic scenario, where test classes are less similar to training classes.
For both minilmageNet. and Omniglot, we simulated task shift scenario by splitting all classes within each dataset into two clusters based on MMD distance. Specifically, first, we randomly selected two classes. Next, for each class, calculated its MMD distance to these two selected classes using Eq. 13, clustering each class to the one it was closer to. To ensure an adequate number of classes in each cluster, we re-split the minilmageNet datasets if a cluster contained fewer than 15 classes. For Omniglot, we mandated that a cluster must have no fewer than 50 classes. Let denote the two clusters A and B. The meta-ttainingg-valialll datasets consisted
classes within cluster A and randomly selected 10% of the classes within cluster B. datasets, also
The meta-training datasets comprised 70% of the classes from the mecta-raining-validaat selected without replacement. The meta-validation datasets consisted of the remaining 30% of the classes from the meta-training- validation traliiattiinnnnnlnlaattiionn datasets that were not selected for meta-training, The meta-test datasets composed of 45% of the classes randomly selected from cluster B not used for seta--raining--aliide The unlabeled datascts for importance estimation was made up of the remaining 45% of the classes from or meta-test. To ensure overlap between the mela-training
cluster not used for satta--rriinn-valdddill and meta-test task distributions, the www.ttrraiiiniiiiiiee as well as 10% of the classes from cluster B. For ticredImageNet, bocause the test classes are less similar the training classes, we followed the original paper's (Ren et al., 2018) method split the training, validation, and test datasets. The unlabeled datasets used for importance estimation consisted of 40 classes
datasets included all classes from cluster A,
9

---

Published in Transactions on Machine Learning Research (09/2024)
randomly selected from the test dataset without replacement. The remaining classes in the test dataset, which were not used for importance estimation, formed the new test dataset. The meta-training datasets comprised all the classes from the training dataset, the meta-validation datasets included all the classes from the validation dataset, and the meta-test datasots contained all the classes from the new test dataset. Since the classes in the meta-lest and importance estimation datasets did not overlap, the datasets used for computing importance weights during the meta-training phase were entirely separate from those used for the meta-test.
5.2 Comparative Methods
We compared the proposed method with the following methods: (1) Model-agnostic Meta-Learning (MAML) (Fimn et al., 2017), which is a traseent-based meta-learning algorithm. (2) Prototypical Networks (PN) (Snell et al., 2017), which is a metric-based algorithm. (3) Meta-Mix (Yao et al., 2021a) and Adversarial Task Up-sampling (ATU) (Wu et al., 2022), which are state-of-the-art lask augmentation ion strategies for meta- learning, as we introduced in Section2.5. and (4) Weighted Meta-Learning (WML) (Cai al., 2020), which is a method aimed at addressing task shift, as we introduced in Section2.4. While MAML, PN, Meta-Mix, and ATU leverage labeled meta-training datasets during their meta-training phase, WML and our method utilize both labeled meta-training datasets and unlabeled datasets in tasks obtained from the meta-test task distribution. To mimic real-world situations more closely, We use unlabeled datasets from tasks that differ from the meta-test task.
5.3 Experimental Settings
Our network architecture, closely mirroring the one used in the referenced study (Vinyals et al., 2016), comprises four convolutional blocks. Each block consists of 3x3 convolution with 128 filters, subsequent batch normalization layer (Ioffe & Szegedy, 2015), ReLU nonlinearity, and finally 2x2 max-pooling layer. When applied to 28x 28 Omniglot images and 84x84 minilmageNet images, this architecture produces a 64- dimensional and 1600-dimensional output space, respectively. MAML, Meta-Mix, and ATU all necessitate a classifier at the final layer. In alignment with the baseline classifier utilized in the study (Vinyals et al., 2016), we feed the output of the last layer into the soft-max function. The loss function is defined as the cross-entropy error between the predicted and the actual label. For PN, we employ the Euclidean distance to compute the distance metric. We carry out a single gradient update with a fixed step size a 0.05 in the inner loop, while Adam (Kingma & Ba, 2014) serves the outer loop optimizer with step size 8 = 10-4. Our model and WML were trained using PN, Meta-Mix, and ATU were trained using MAML. Due to the original WML being limited to regression scenarios and constrained to the use of squared and hinge loss as its loss functions, in our experiments, to adapt WML for classification scenarios, we use Eq. 12 to calculate the distance between tasks.
During the meta-training phase, we trained the proposed IWML and the comparative methods using 250,000 mela-training tasks with meta-batch size of 50. Since both WML and the proposed IWML require unla- beled data to compute weights, used 30 importance estimation tasks, which consist only of unlabeled data, for weight estimation. However, the other comparative methods did not use any additional unlabeled data during the meta-training phase. We employed 30 validation tasks for early stopping. In the meta-test phase, we computed the classification accuracy based on an average of over 30 meta-test tasks. Meta-training, vali- dation, meta-test, and importance estimation tasks were randomly generated from their respective datasets, each task was constructed using 5-way setting with 1-shot, 2-shot, or 3-shot configurations for minilma- geNet and maaeed]m,geNNt, and 20-way with the same shot variations for the Omniglot datasets. We applied the same settings as used in training the PN each datasct for training encoder Iø- For smoothing scalar parameters and h used in Eq. 14 and Eq. 9, we designate both as task-specific smoothing scalar parame- ters. For a,, its value for each task is determined by calculating the median of (1l))(x,) - (xxx)) [ Similarly, for h, it is determined using the median of [d(g(x), q (x))))) Additionally, h in Eq. 8, Eq. 10, and Eq. 11 was determined in the similar calculation with Eq. 9.
10

---

Published in Transactions on Machine Learning Research (09/2024)
Method
1-shot
2-shot
3-shot
MAML
30.4+1.8
32.142.55
33.7+3.88
PN
31.0+3.0
34.122.7
36.523.4
Meta-Mix
29.8+2.9
34.8++2.8
34.4+3.7
ATU
28.84:3.6
35.0+:2.3
34.8=5.4
WML
31.7+:2.4
35.942.9
37.823.6
Ours
34.413.1
38.0+3.7
43.613.0
Table 1: Averaged accuracies (%) and standard errors are presented for 5-way classification on mela-lest tasks using minilmageNet. Values in bold typeface are not statistically different from the best-performing method at the 5% level by a paired t-test.
Method
1-shot
2-shot
3-shot
Method
1-shot
2-shot
3-shot
MAML
72.
82.6+0.7
85.6=100
MAML
31.3+3.6
32.9#3.1
35.
PN
68.9+0..
83.7.:0.4
87.4
PN
32.9
37.14:10
44.1+1144
Meta-Mix
71.6
82.4+1.1
86.3+13
Meta-Mix
32.243.8
34.0:2.2
37.7#4.0
ATU
72.71:1.3
83.910.6
85.2=0.9
ATU
32.813.1
36.7:5..5
37.814.1
WML
69.3
81.9+0.9
88.4=0.7
WML
33.6+1.2
38.7#1.8
45.6+2.0
Ours
92.9+-0.7
96.4+004
97.1+0.77
Ours
38.3+2..3
44.6-2.1
51.8+3..1
Table 2: Averaged accuracies (%) and standard errors are presented for 20-way classification on meta-test tasks using Omniglot. Values in bold typeface are not statistically different from the best-performing method at the 5% level by a paired I-test.
Table 3: Averaged accuracics (%) and standard er-
rors are presented for 5-way classification on meta- test tasks using tieredImageNett. Values in bold typeface are not statistically different from the best- cest-performin method at the 5% level by paired I-fest.
5.4 Results
We show the performance of the three datasets Tables 1, 2, and 3. On three datasets, the proposed method consistently outperforms the baseline methods. WML outperforms MAML. and PN but is inferior the proposed method. One reason for this is that WML faces difficulty computing more accurate weights for each meta-training task, especially when using tasks that align with the meta-test. task distribution but will not appear in the meta-test phase. In contrast, of each meta-training task within the distributions of both meta-test and meta-training tasks. This enables our method to compute accurate weights, even when utilizing tasks that conform to the meta-test task distribution but are absent from the meta-test phase. The performance of Meta-Mix and ATU does not improve over MAML. One reason is that, in the case of task shift, due to the different distributions between meta-training and meta-test tasks, they fail to generate tasks similar the meta-test tasks using the given meta-training lasks. However, the proposed method enables the model to leverage additional unlabeled data from the meta-test task distribution, allowing it to focus more meta-training tasks that closely align with the meta-test tasks during the meta-training process, resulting in better performance. Table 4 shows the training time for PN, WML, and the proposed method. Although our method takes longer than PN, it is much shorter than that of WMT. This is because WML assumes that there is only one meta-test task during the meta-test phase. When there are multiple meta- test tasks during the meta-test phase, WML needs to train spocific model paramoters for each meta-test task.
method calculates weights by estimating the density
PN
WML
Ours 0.3
0.2
6.7
Table 4: Training computational time in hours under the 1-shot 5-way setting on the minilmageNet.
For the ablation study, we evaluated the performance without using MMD or the deep kernel in Eq. 8 and Eq. 9. In the experiments without MMD, the distance between feature vector distributions was calculated
11

---

Published in Transactions on Machine Learning Research (09/2024)
follows:
N NT (xg)(x,) (x(xxx))) TE 211.
(x), de (x))
Σ
(16)
NN
For experiments without the deep kernel, the Gaussian kernel used in Eq.13 is given by:
k(xn, X.)) exp (=1xx, x
(17)
2
The results Table indicate a performance decline for the proposed method when MMD not used. This is because MMD can more accurately reflect the distribution differences between two sets of data compared to the mcan of point-to-point distances. Similarly, performance also decreases when the deep kernel is not used. This is because deep kernels can obtain feature representations specifically suited to the given data through encoder, potentially enhancing the performance of KDE compared to general kernel functions.
Ours 34.4 3.1
Without MMD Without deep kernel 32.9
32.5+288
Table 5: Ablation study: The averaged accuracies (%) and standard errors on the meta-test tasks under the 1-shot 5-way setting on the minilmageNet dataset were obtained using the proposed method with different MMD and deep kernel choices of Eq. 8 and Eq. 9.
For the encoder Gø, we evaluated the performance when trained using only labeled meta-training data and when trained using joint labeled meta-training data and unlabeled data obtained from the meta-test task distribution. The results presented in Table 6, where 'ours" indicates the performance using only labeled data, and 'ours with autocncoder' indicates the performance using both labeled and unlabeled data. In the experiment using both labeled and unlabeled data, we considered Go as part of an auloencoder (Rumelhart et al., 1986). The training process was divided into two stages: (1) When training with labeled meta- training data, a PN was added and trained together with the autocncoder. The autocncoder was trained by minimizing both the error of the PN and the reconstruction error. (2) When training with unlabeled data, the autoencoder was trained by minimizing the reconstruction error alone.
The results in Table 6 indicate that training the encoder de using joint labeled meta-training data and unlabeled data obtained from the meta-test task distribution achieves better performance compared to training with only labeled meta-training data. This is because when the encoder trained solely with labeled meta-training data, it struggles to derive suitable representations for data from the meta-test task distribution due to task shift, leading to decreased performance. meta-training data and unlabeled data from the meta-test task distribution allows the encoder more effectively obtain suitable representations for data from both task distributions, thereby enhancing overall performance.
contrast, training 9g with both labeled
Ours
Ours with autoencoder
34.4+ 3.1
35.6+t 2.5
Table 6: The averaged accuracics (%) and standard errors on the meta-test tasks under the 1-shot 5-way setting on the minilmageNet dataset where encoder Jø part of autoencoder are trained using joint labeled veta-traiinng data and unlabeled data obtained from the meta-test task distribution.
5.5 Visualization of the task shift
We visualize the task shift in the minilmageNel dataset under a 5-way 1-shot setup. Specifically, we randomly selected 50 mmeta-training tasks from the meta-trainin tetta-trainig datasets and 50 meta-test tasks from the meta-test datascts. We used t-SNE (Van der Maaten & Hinton, 2008) with the MMD distance for visualization. The results shown Figure 2 reveal that although there is a significant difference between the distribution of meta-training tasks and meta-test. tasks, portion of the meta-training tasks are closer to the meta-test
12

---

Published in Transactions on Machine Learning Research (09/2024)
Higher
(a) Meta-training tasks with weights: 1.324
Wiighei basining
(b) Meta-training tasks with weights: 0.816
(c) Meta-test task
Figure 2: Visualizing the task shift scenario in mini- ImageNet: Each black point represents a meta-test task, while points other colors indicate meta- training tasks. Tasks with higher assigned weights are depicted in richer shades of red.
Figure Examples of images in meta-training tasks with weights 1.324/0.814 are represented by (a) and (b), while those in meta-test task represented by (c). These tasks generated from minilma- geNet datasets using 1-shot, 5-way setting.
tasks. The proposed method can assign higher weights to these meta-training tasks that are closer to the meta-test tasks.
Figure 3 shows examples of images in meta-training tasks with weights 1.324/0.816 and those in meta- test task. As depicted in Figure 3(c), the data in meta-test tasks consists of images related to animals and insects. The proposed method assigned high weight to a task depicted in Figure 3(a) that contains more animal and insect images. In contrast, tasks with fewer animal images are assigned lower weights as shown in Figure 3(b). This indicates that our method enables us to find meta-training tasks that closely resemble the meta-test tasks. By assigning high weights these tasks, we can ensure that the distribution of meta-training lasks aligns closely possible with that of the mela-lest lasks.
6 Conclusion
We proposed meta-learning method that improves performance under task shift, where the task distribu- tions diller between the meta-training and meta-lest. phases. Tn the proposed method, weights are assigned to tasks using the density ratio between these distributions. Experiments demonstrate that our method outperforms existing approaches. However, the proposed method has its limitations. Specifically, (1) Our evaluation focuses on the minilmageNet, Omniglot, and tieredImageNet datasets, which, although widely used, do not capture the full range of challenges posed by more complex benchmarks such as Meta-Dataset (Triantafillou et al., 2019), VTAB (Zhai et al., 1910), and WILDS (Sagawa al., 2021). Future work will aim to address this by applying the method to these benchmarks and assessing its robustness in more challenging scenarios. (2) When the accessible unlabeled dataset and the meta-test dataset follow different distributions, the performance of our proposed method will decrease. (3) when the density of mota-training tasks used as the denominator in the density ratio nears zero, there is a risk that the calculation of weights becomes unfeasible. Therefore, it is preferable to estimate weights directly without estimating densities. For example, methods proposed to directly estimate weights to address covariate shift (Sugiyama al., 2008; Bickel al., 2009). In future research, we plan extend the proposed method estimate weights directly without needing to estimate the density of tasks.
13

---

Published in Transactions on Machine Learning Research (09/2024)
Broader Impact Statement
The proposed method uses unlabcled datasets derived from tasks that adhere to the meta-test task distribu- tion to compute weights for each meta-training task. Consequently, there is a potential risk that users might use biased datasets to calculate these weights without careful consideration, potentially leading to biased predictions. We encourage research into developing methods for automatically detecting biased datasets.
References
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning under covariate shift. Journal of Machine Learning Research, 10(9). 2009.
Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformalics, 22 (14):e49 e57, 2006.
Diana Cai, Rishit Sheth, Lester Mackey, and Nicolo Fusi. Weighted meta-learning. arXiv preprint arXiv:2003.09. 09446, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pp- 1126-1135. PMLR, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. Advances in Neural Information Processing Systems, 31, 2018.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Interna- tional Conference Machine Learning, pp. 1180 1189. PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversariall training of neural networks. Journal of Machine Learning Research, 17(59):1 35, 2016.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference Machine Learning, pp. 7704-1718 PMLR, 2018.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-777) 2012.
Sepp Hochreiter, A Steven Younger, and Peter R Conwell. Learning to learn using gradient descent. In Artificial Neural NetworksICANN 2001: International Conference Vienna, Austria, August 21-25, 2001 Proceedings 11, pp. 87 94. Springer, 2001.
Timothy Hospedales, Autreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE Transactions on Pottern Analysis and Machine Intelligence, 44(9))5149-5169) 2021.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International Conference on Machine Learning, pp. 448 456. pmlr, 2015.
Mikhail Khodak, Maria-Floo FariiaFlorina F Balcan, and Amect S Talwalkar. Adaptive gradient-based meta-learning methods. Advances in Neural Information Processing Systems, 32, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar Xiv preprint arXiv:1412. 6980, 2014.
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2. Lille, 2015.
Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the Ansuual Meeting of the Cognitive Science Sociely, volume 33, 2011.
14

---

Published in Transactions on Machine Learning Research (09/2024)
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint Xrx: (ff::1777009888 2017.
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International Conference Machine Learning, pp- 97-105. PMT.R, 2015.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael T Jordan. Deep transfer learning with joint adapta- tion networks. In International Conference on Machine Learning, pp. 2208 2217. PMLR, 2017.
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner. arXiv preprint vrxx::1777.0811 2017.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.00 029999, 2018.
Yassine Ouali, Cóline Hudelot, and Myriam Tami. An overview of deep semi-supervised learning. arXiv preprinl ar Xix:2006. 05278, 2020.
Janarthanan Rajendran, Alexander Irpan, and Eric Jang. Meta-learning requires mcta-augmentation. Ad- 2020.
vances in Neural Information Processing Systems,
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in Neural Information Processing Systems, 32, 2019.
Mengye Ren, Eleni Triantafllou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard $ Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint DrXiv:1803.00670 2018.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning internal ropresentations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. meclelland. vol. 1986. Biometrika, 71 (599-6007);6, 1986.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211-252. 2015.
Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklun, et al. Extending the wilds benchmark for unsupervised adaptation. arXiv preprint arXiv:21 12. 05099,, 2021.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3723-3732, 2018.
Swami Sankaranarayanan, Yogesh Balaji, Carlos Castillo, and Rama Chellappa. Generate to adapt: Align- ing domains using generative adversarial networks. In Proceedings of the IEEE Conference on Compuler 2018.
Vision and Pattern Recognition, pp.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In International Conference on Machine Learning, pp. 1842- 1850. PMLR, 2016.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90(2): 2000.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning, Advances in Newral Information Processing Systems, 30, 2017.
Masashi Sugiyama, Matthias Krauledat, and Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
15

---

Published in Transactions on Machine Learning Research (09/2024)
Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul Von Bünau, and Motoaki Kawan- abe. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60:699 746, 2008.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Eihon-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Purt III 14, pp. 443 450. Springer, 2016.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In Proceedings of the AAAT Conference Arlificial Intelligence, volume 30, 2016.
Eleni Triantafilluu, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, et al. Meta-dataset: A dataset of datasets for learning to learn from few examples. arXiv preprint arxiv:1933. 030000 2019.
Laurens Van der Maaten and Geoffrey Hinton. Research, 9(11), 2008.
Visualizing data using t-sne. Journal of Machine Learning
Jesper E Van Engelen and Holger H Hoos. A survey semi-supervised learning. Machine Learning, 109 (2):373 440, 2020.
Joaquin Vanschoren. Meta-learning: survey, arXiv preprint urXiv:1810.0333 2018
Ricardo Vilalta and Youssef Drissi. A perspective view and survey of meta-learning. Artificial Intelligence Review, 18:77 95, 2002.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for shot learning. Advances in Neural Information Processing Systems, 29, 2016.
Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In Artificial Intelligence and Statistics, pp. 370-378. PMLR, 2016.
Yichen Wu, Long- Kai Huang, and Ying Wei. Adversarial task up-sampling for leaanin,, Advances in Newral Information Processing Systems, 1-31102-31115 2022.
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning. In International Conference on Machine Learning, pp. 7045-7054. PMLR, 2019.
Huaxiu Yao, Long-Kai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou, Junzhou Huang, et al. Im- proving generalization in meta-learning via task augmentation. In International Conference on Machine Learning, pp. 11887-11897. PMLR, 2021a.
Huaxiu Yao, Yu Wang, Ying Wei, Peilin Zhao, Mehrdad Mahdavi, Defu Lian, and Chelsea Finn. Meta- learning with an adaptive task scheduler. Advances in Neural Information Processing Systems, 34:7497- 7509, 2021b.
Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn. Meta-learning without memorization. arXiv preprint ar Xiv:1911 038200, 2019.
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta-learning. Advances in Neural Information Processing Systems, 31, 2018.
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. A large-scale study of represen- tation learning with the visual task adaptation benchmark. arxiv 2019. arXiv preprint arXiv:1910.0/:: 046777 1910.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adap- tation via meta-learning, In International Conference Machine Learning, pp. 7693-7702. PMLR, 2019.
16


Reviews:
[{"review_id": "M2uAumXAUq", "reviewer": "Reviewer_ye1d", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths ##\n1. The paper tackles an important problem that is a missing piece of the existing literature\n2. The paper is well motivated and written with good use of notation that makes it relatively easy to understand\n3. At a high level, the method present is simple (since it makes use of existing pieces) though I do not doubt that fitting everything together was not easy\n\n## Weaknesses ## \n1. Some missing ablations. \n\n      a) No ablating MMD^2 or Deep Kernel as choices for use in KDE\n\n      b) $\\phi$ is trained on only the meta-train set. Would have been interesting to also see it train on the joint meta-train/unlabelled meta-test data. Especially if it is the case that meta-train and meta-test have different supports.\n\n2.  Need for more robust definition of task shift. The paper current defines task shift as $p^{TE}\\left(T_{t} \\right) \\neq p^{TR}\\left(T_{t} \\right)$ but there are many ways in which this could manifest -- like non-overlapping input/output supports, covariate shift in the input spaces. **Please make clear what types of shift your method is targetted at -- or if it is supposed to subsume all types of shifts, then show evidence of this**\n\n3. Not enough information about how baselines were implemented. Specifically for the WML method -- do you rerun meta-training  for each individual meta-test task ? My intuition was that WML would perform best (since the weights are learned per meta-test task) but would be too computationally intensive because it's being run for each individual meta-test task. Is your method outperforming WML partially because you aggregate the unlabelled meta-test data and so there is some transfer going on? Can you provide more details about the WML setup ?\n\n\n## Questions  ##\n1. Why just do 1-step GD on the inner loop ? What happens if you do > 1 step in the meta-learning inner loop ?\n2. I'm not sure how to phrase this question but it feels a bit circular, if the Dataset you created for experimentation uses the MMD distance to \"simulate task shift\" when it's the same distance you use in your method construction. Even if the distinction is more nuanced, I think a more \"realistic\" task shift could be some combination of dataset from the meta-dataset [1] paper.\n\n\n[1] https://arxiv.org/abs/1903.03096", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a method to perform meta-learning under task shift. Specifically, when the meta-test distribution differs from the meta-training distribution, there is currently no algorithm that ensures that the model learned during meta-train actually performs well during meta-test. \nThis paper presents an importance weighting based approach to fix this.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see a list of the questions posed above to be addressed as well as the weaknesses section.", "broader_impact_concerns": "The paper does not have a broader impact statement.\nPlease make sure to provide this", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nypUgTiQAs", "reviewer": "Reviewer_PArP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- This paper is well-structured and easy to comprehend.\n- The motivation to deal with the task shifts problem within meta-learning is clear， and the proposed reweighting methods align with intuitive understanding. \n- The inclusion of visualizations (Fig.2 and Fig.3) to showcase the learned weights for meta-training tasks is appreciated, as it validates the effectiveness of the proposed reweighting methods.\n\n**Weaknesses**\n- The construction of the task shift dataset also relies on Equation (13). How does the model perform in other task shift settings? I am skeptical about the generalization ability of the method in other task shift scenarios because there is a strong relationship between the task construction and the method itself.\n- This paper proposes assigning importance weights to different tasks within a meta-training task, which is closely related to [1]. I believe the authors need to draw a clear comparison between these two closely related methods.\n- I suggest that the author should offer specific weights within Fig.3 to facilitate a more effective comparison, rather than merely illustrating high and low weights. \n\n[1] Meta-learning with an adaptive task scheduler, NeurIPS 2021.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper delves into the issue of task shifting within meta-learning, specifically addressing the discrepancy between the distributions of meta-training and meta-testing datasets. It proposes a solution to assign importance weights to various meta-training tasks, utilizing kernel density estimation (KDE) and Maximum Mean Discrepancy (MMD) distance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Requested Changes that are critical to securing my recommendation for acceptance:**\n- The Weakness 1.  Addressing this issue is crucial in my view, as it will substantiate the effectiveness of the proposed method. If this concern is satisfactorily resolved, I am inclined to recommend acceptance of the work.\n\n**Requested Changes that can strengthen the work**\n- The Weakness 2 & 3.", "broader_impact_concerns": "Upon reviewing the work, I did not find any significant ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "F3C0hSiKYP", "reviewer": "Reviewer_yGSH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The presentation of this paper is easy to follow and well-organized.\n2. The motivation behind the proposed weighted meta-loss is clear and reasonable.\n3. The method for estimating importance weights is logically sound.\n4. The experimental results show significant improvements over previous methods.\n\n## Weaknesses\n1. It is unclear whether the Kernel Density Estimation is an existing technique or a new one proposed in this paper.\n2. The paper focuses on estimating weights for each meta-training task, but it does not address the scenario of having only a single meta-training task.\n3. The paper assumes access to an unlabeled meta-test dataset. However, it does not consider the situation where there is a significant shift between the accessible unlabeled dataset and the meta-test dataset.\n4. The experimental settings regarding the datasets and task descriptions are not clearly detailed. For example, it is unclear which parts are meta-training tasks and which are meta-test tasks. More importantly, detailed information about the unlabeled dataset used is lacking.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the meta-learning method in the presence of distribution shifts. Weights for meta-training tasks are estimated using Kernel Density Estimation, and an importance-weighted meta-learning loss is utilized. Experiments on several datasets demonstrate that the proposed weighted meta-loss outperforms previous related methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify what the new technique proposed in this paper is.\n2. Provide more analysis on using unlabeled datasets in the presence of distribution shifts.\n3. Present detailed information with clear task descriptions for the experiments.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission adapts the Weighted Meta-Learning approach of [Cai et al. (2020)](https://arxiv.org/abs/2003.09465) to a setting in which meta-test-time tasks to be labelled are drawn from a shifted distribution relative to meta-training time. The approach requires unlabeled data at meta-training time in order to compute importance weights to re-weight task-wise contributions to the meta-training objective. Results demonstrate an improvement on three few-shot image classification benchmarks: miniImageNet, Omniglot, and tieredImageNet.

The reviewers praised the paper's clear presentation, with ye1d noting the method's simplicity and yGSH highlighting improvements over previous methods. However, they raised concerns about generalizability, with PArP questioning performance in other task shift settings and ye1d requesting clarification on the types of shifts targeted, as well as methodology issues like missing ablations (ye1d). The reviewers suggested addressing these concerns, with PArP emphasizing the need to substantiate the method's effectiveness in various task shift scenarios, ye1d requesting more details on baseline implementations (especially WML), and yGSH asking for clearer experimental setup descriptions. These concerns were sufficiently addressed during the revision process by adjusting claims and including ablation results.

After reviewing the submission in detail, the Action Editor (AE) requests the following minor revisions:

1. The data budget for the proposed IWML method at training time from the comparison methods is not made sufficiently clear. For example, the submission states that, for the given experimental settings, "...30 tasks associated with the unlabeled dataset [were used] to calculate importance." The availability of additional, even unlabeled data, changes the interpretation of the empirical results, i.e., as being the product of algorithm (IWML) *plus* additional data (in the form of unlabelled test tasks). As such, **the authors should make clearer the data made available to each method alongside results included in the empirical evaluation.** 

2. The **authors should detail the train-test split enforced for the additional unlabelled data made available at meta-training time,** which is not currently made clear in the paper. Are the tasks/examples used to compute performance (meta-testing accuracy) entirely disjoint from the tasks/examples used to compute importance weights during meta-training, as would be appropriate to prevent fitting to the evaluation set? If not, the authors should rerun the evaluation with this split enforced to determine if the proposed method maintains the reported performance improvement.

3. The **authors should correct the revised claim that "...employing MMD within KDE is also a new technique introduced in this work,"** a claim that was introduced during the revision process. This claim is incorrect. See [Gretton et al. (2012)](https://jmlr.csail.mit.edu/papers/volume13/gretton12a/gretton12a.pdf) for an example of a prior use.

34. Though the AE agrees that three is an audience of individuals interested in algorithms for task shift in few-shot learning who may find interest in this work, this audience is limited by the fact that the evaluation is constructed from simple datasets (variants of the miniImageNet task of [Vinyals et al., 2016](https://arxiv.org/abs/1606.04080)). These datasets are of reduced interest in the transfer learning and few-shot learning literature relative to more complex few-shot benchmarks such as Meta-Dataset ([Triantafillou et al. 2020](https://arxiv.org/abs/1903.03096)) and VTAB ([Zhai et al., 2020](https://arxiv.org/abs/1910.04867)), and task shift benchmarks such as the WILDS dataset, especially v.2.0 with unlabelled data ([Sagawa et al., 2021](https://arxiv.org/abs/2112.05090)). To address this, **the authors should note the lack of demonstrated generalizability to more recent relevant benchmarks in the paper, e.g., in the conclusion or limitations section**.

Subject to these revisions, the AE can recommend acceptance to TMLR.
Decision: Accept (poster)