Meta-Learning without Memorization
Authors: Mingzhang Yin
George Tucker
Mingyuan Zhou
Sergey Levine
Chelsea Finn
Venue:  (2020)
Keywords: meta-learning
memorization
regularization
overfitting
mutually-exclusive

Abstract: The ability to learn new concepts with small amounts of data is a critical aspect of intelligence that has proven challenging for deep learning methods. Meta-learning has emerged as a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks. However, most meta-learning algorithms implicitly require that the meta-training tasks be mutually-exclusive, such that no single model can solve all of the tasks at once. For example, when creating tasks for few-shot image classification, prior work uses a per-task random assignment of image classes to N-way classification labels. If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes.  This requirement means that the user must take great care in designing the tasks, for example by shuffling labels or removing task identifying information from the inputs. In some domains, this makes meta-learning entirely inapplicable. In this paper, we address this challenge by designing a meta-regularization objective using information theory that places precedence on data-driven adaptation. This causes the meta-learner to decide what must be learned from the task training data and what should be inferred from the task testing input. By doing so, our algorithm can successfully use data from non-mutually-exclusive tasks to efficiently adapt to novel tasks. We demonstrate its applicability to both contextual and gradient-based meta-learning algorithms, and apply it in practical settings where applying standard meta-learning has been difficult. Our approach substantially outperforms standard meta-learning algorithms in these settings. 

Published as a conference paper at ICLR 2020
META- WITHOUT MEMORIZATION
Mingzhang Yin 12 George Tucker", Mingyuan , Levine Chelsea Finn
mzyinεutexas edu, egoogooole e com, mi mingyuan. www.ua...ouuueeeeeeeo Shho@mccombs, utexas edu eeeeveeseees.... berke edu, cbf iofinneee cs stanford.edd
UT Austin, GGogle Research, Brain team, UC Berkeley, *Stanford
ABSTRACT
The ability to learn new concepts with small amounts of data is a critical aspect of intelligence that has proven challenging for deep learning methods. Meta-learning has emerged as promising technique for leveraging data from previous tasks to enable efficient learning of new tasks. However, most meta-learning algorithms implicitly require that the meta-training tasks be mutually-exclussiv,, such that no single model can solve all of the tasks at once. For example, when creating tasks for few-shot image classification, prior work uses per-task random assignment of image classes to N-way classífication labels. If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes. must take great care in designing the tasks,
This requirement means that the for example by shuffling labels or removing task identifying information from the inputs. In some domains, this makes meta-learning entirely inapplicable. In this paper, we address this challenge by designing a meta-rcgularrzztion objective using information theory that places precedence on data-driven adaptation. This causes the meta-leamer to decide what must be learned from the task training data and what should be inferred from the task testing input. By doing so, our algorithm can successfully use data from ror-mulually-exxllsiee tasks to efficiently adapt novel tasks. We demonstrate its applicability to both contextual and gradient- based meta-learning algorithms, and apply it in practical settings where applying standard meta- peta-lea has been difficult. Our approach substantially outperforms standard meta-learning algorithms in these settings[
INTRODUCTION
The ability learn new concepts and skills with small amounts of data is critical aspect of intelli- gence that many machine learning systems lack. Meta-learning (Schmidhuber 1987) has emerged as a promising approach for enabling systems quickly learn new tasks by building upon experi- ence from previous related tasks (Thrun & Prati 2012) Koch et al. 2015 Santoro et al., 2016 Ravi & Larochelle| 2016 Finn et al. 2017). Meta-learning accomplishes this by explicitly optimizing for few-shot ieneralization across a set of meta-training tasks. The meta-learner is trained such that, after being presented with small task training set, can accurately make predictions on test datapoints for that meta-training task.
While these methods have shown promising results, current methods require careful design of the meta-training tasks to prevent subtle form of lask overfitling, distinct from standard overfitting in supervised learning. If the task can be accurately inferred from the test input alone, then the task training data can be ignored while still achieving low meta-training loss. In effect, the model will collapse to one that makes zero-shot decisions. This presents an opportunity for overfitting where the meta-learner generalizes data from novel tasks. We call this form of overfitting the memorization problem in meta-learning because the meta-learner memorizes function that solves all of the meta-training tasks, rather than learning to adapt.
meta-training tasks, but fails to adapt when presented with training
Existing meta-learning algorithms implicitly resolve this problem by carefully designing the meta- training tasks such that no single model can solve all tasks zero-shot; we call tasks constructed in this
Implementation and examples available here:
research
google e- weeerrrearcctteeee/attteteeeee/e 1ea rni ng www.tttooooooootttttttt....

---

Published as a conference paper at ICLR 2020
way mutually-exclusive For example, for N-way classification, each task consists of examples from N randomly sampled classes. The N classes are labeled from to N, and critically, for each task, randomize the assignment of classes to labels (1, N (visualized in Appendix Figure 3) This ensures that the task-specific class-to-label assignment cannot be inferred from a test input alone. However, the mutually-exclusivy tasks requirement places substantial burden the user to cleverly design the meta-training setup (c.g., by shuffling labels or omitting goal information). While shuffling labels provides a reasonable mechanism to force tasks to be mutually-exclusin with standard few-shot image classification datasets such as MinilmageNeet (Ravi & Larochelle, 2016). this solution cannot be applied all domains where we would like to utilize meta-learning. For example, consider meta-learning pose predictor that can adapt to different objects: even if N different objects are used for meta-training, a powerful model can simply learn to ignore the training set for each task, and directly learn to predict the pose of each of the N objects. However, such model would not be able to adapt to new objects at meta-test time.
The primary contributions of this work are: 1) to identify and formalize the memorization problem in meta-learning, and 2) to propose a meta-regularizer (MR) using information theory as a gen- eral approach for mitigating this problem without placing restrictions the task distribution. We formally differentiate the meta-learning memorization problem from overfitting problem in conven- tional supervised learning, and empirically show that naive applications of standard regularization techniques do not solve the memorization problem in meta-learning. 'he key insight of our meta- regularization approach that the model acquired when memorizing tasks is more complex than the model that results from task-specific adaptation because the memorization model is a single model that simultancously performs well on all tasks. It needs to contain all information in its weights needed to do well on test points without looking at training points. Therefore we would expect the information content of the weights of memorization model to be larger, and hence the model should be more complex. As a result, we propose an objective that regularizes the information complexity of the meta-learned function class (motivated by Alemi et al. (2016); Achille & Soatto (2118). Furthermore, we show that neta-regularizztion in MAML can be rigorously motivated by task
a PAC-Bayes bound on generalization. In a series of experiments on non-mutually--ex/oe distributions entailing both few-shot regression and classification, we find that memorization poses a significant challenge for both gradient-based (Finn et al.) 2017) and contextual (Garnelo et al.) 2018a) meta-learning methods, resulting in near random performance on test tasks in some cases. Our meta-regularizztion approach enables both of these methods to achieve efficient adaptation and generalization, leading to substantial performance gains across the board on con--utually-exlussiii tasks.
2 PRELIMINARIES
We focus on the standard supervised meta-learning problem (see, e.g., Finn et al. (2017). Briefly, we assume tasks T; are sampled from task distribution p(T). During meta-training, for each task, we observe a set of training data D; = (æt, y,) and a set of test data D; - (æ;, y²) with x; = (Ia1.-... ,..) ), 9; = (Vı1,..., .., Wlr) sampled from p(æ; y TT), and similarly for DJ We denote the entire meta-training set as M (Dr, DJ J. The goal of meta-training is to learn model for a new task T by leveraging what is learned during meta-training and a small amount of training data for the new task D. We use 0 to denote the meta- patammeerssssss learned during meta-training and use to denote the task-specific parameters that are computed based on the task training data.
M, we consider meta-learning algorithms that maximize conditional likelihood q(0" y "0|=", , 8, D),, which is com- posed of three distributions: M)) that summarizes meta-training data into a distribution on meta- parameters, ( (D, 0) that summarizes the per-task training set into distribution on task-specific parameters, and q(0" Jx=", ,6,0) that the predictive distribution. These distributions are learned to minimize
Following Grant et al. (2018); Gordon al. (2018), given a meta-training
Σ, EgcoLMo)((HDDD.)) 1 N
I E(e R
(w- wT));; log 7(5" y v0|x", de 0)]
For example, in MAML FFin et al. 2017). 0 and á are the weights of predictor network, (00l is a delta function learned over the meta- mnta-traimnn data, q()/D, 8) is a delta function centered at a point defined by gradient optimization. and ó parameterizes the predictor network q(""|2", .)) (Grant jet al. 2018). In particular, to determine the task-specific parameters &, the task training data D and 0 are used in the predictor model é 0 # EEEDED Vo log g(y|zr 0).
2

---

Published as a conference paper at ICLR 2020
Another family of meta-learning algorithms are contextual methods (Santoro al. 2016), such as conditional neural processes (CNP) (Garnelo et al., 2018b a). CNP instead defines q 8) as a mapping from D to a summary statistic @ (parametenized by 8). In particular, e Og o hg (D) output of an aggregator (o))) applied to features ho (D) extracted from the task training data. Then O parameterizes a predictor network that takes and x= as input and produces a predictive distribution 4(0 |a",, o, 0).
)) is the
In the following sections, we describe a common pittall for a variety of meta-learning algorithms, including MAML and CNP, and general meta-regularizztion approach to prevent this pitfall.
THE MEMORIZATION PROBLEM IN META-LEARNING
The ideal meta-learning algorithm will learn in such way that generalizes to novel tasks. However, we find that unless tasks are carefully designed, current meta-learning algorithms can overfit to the tasks and end up ignoring the task training data (i.e.. either q( 0) does not depend on D or q(0" |z o, 0) does not depend on o, as shown in Figure 1). which can lead to poor generalization. This memorization phenomenon is best understood through examples.
Consider a 3D object pose prediction problem (illustrated Figure D. where each object has a fixed canonical pose. The (z, y) pairs for the task are 2D grey-scale images of the rotated object (x) and the rotation angle relative to the fixed canonical pose for that object (y). In the most extreme case, for an unseen object, the task is impossible without using D because the canonical pose for the unseen object is unknown. The number of objects in the meta-training dataset is small, so is straightforward for a single network to memorize the canonical pose for each training object and to infer the object from the input image (i.e., task overfitting), thus achieving a low training error without using D. However, by construction, this solution will necessarily have poor generalization to test tasks with unseen objects.
As another example, imagine an automated medical prescription system that suggests medication prescriptions to doctors based on patient symptoms and the patient's previous record of prescrip- tion responses (i.e., medical history) for adaptation. In the meta-learning framework, each patient represents a separate task. Here, the symptoms and prescriptions have close relationship, so we cannot assign random prescriptions to symptoms, in contrast to the classification tasks where we can randomly shuffle the labels to create mutually-exclusiieness. For this non-mutuallly-exllsivee task distribution, a standard meta-learning system can memorize the patients' identity information in the training. leading it to ignore the medical history and only utilize the symptoms combined with the memorized information. As a result, it may issue highly accurate prescriptions on the mela-training set, but fail to adapt to new patients effectively. While such system would achieve a baseline level of accuracy for new patients, it would be no better than a standard supervised learning method applied to the pooled data.
We formally define (complete) memorization as: Definition 1 (Complete Meta-Learning Memorization). Complete memorization in meta-learning is when the learned model ignores the task training data such that 1 ( D( 1)) 0) = 0 (i.e.,
q("" |x* 0, D) = q(ğ" | = EDI [q(8* (x"0"==',11 0,, 0, D')).
Memorization describes an issue with overfitting the meta-training tasks, but it does not preclude the network from generalizing unseen (x, y) pairs on the tasks similar to the training tasks. Memorization becomes an undesired problem for generalization to new tasks when [ D[æ") >> [(""; D x² 0) (i.e., the task training data is necessary to achieve good performance, even with exact inference under the data generating distribution, to make accurate predictions).
A model with the memorization problem may generalize new datapoints in training tasks but cannot generalize to novel tasks, which distinguishes it from typical overfitting in supervised learn- ing. In practice, we find that MAML. and CNP frequently converge to this memorization solution (Table 2). For MAML, memorization can occur when a particular setting of 0 that does not adapt to the task training data can achieve comparable meta-training error to a solution that adapts For example, if setting of 0 can solve all of the meta-training tasks (i.e., for all (x, y) in D and D* the predictive error is close to zero), the optimization may converge to a stationary point of the MAML objective where minimal adaptation occurs based on the task training set (i.e., @ no 0). For novel task where it is necessary to use the task training data, MAML can in principle still leverage the task training data because the adaptation step is based gradient descent. However, in practice, the
3

---

Published as a conference paper at ICLR 2020
poor initialization of 0 can affect the model's ability to generalize from small mount of data. For CNP, memorization can occur when the predictive distribution network q(8" |a"., Ø, 8) can achieve low training error without using the task training summary statistics 6. On novel task, the network not trained to use Φ, so is unable to use the information extracted from the task training set to effectively generalize.
In some problem domains, the memorization problem can be avoided by carefully constructing the tasks. For example, for N-way classification, each task consists of examples from N randomly sampled classes. If the classes are assigned to a random permutation of N for each task, this ensures that the task-specific class-to-label assignment cannot be inferred from the test inputs alone. As result, a model that ignores the task training data cannot achieve low training error, preventing convergence to the memorization problem. We refer to tasks constructed in this way as mutually- exclusive. However, the mutually-exclusive tasks requirement places substantial burden on the user cleverly design the meta- deeannta setup (e.g.. by shuffling labels or omitting goal information) and cannot be applied to all domains where would like to utilize meta-learning.
Task training data 100f
Task test data
WW
112"
M
P(T)
Meta-training
Φj
D
Tj
2:40"
Meta-testing
Figure I: Left: An example of non mutually-exclusive exclusive pose prediction tasks, which may lead to memoriza- tion problem. The training tasks are non mutually-exxlun accurately without using task training data (left) in the training tasks, by memorizing the canonical orientation of the meta-training objects. For new object and canonical orientation (bottom). the task cannot be solved without using task training data (bottom left) infer the canonical orientation. Right: Graphical model for meta-learning. Observed variables shaded. Without either of the dashed arrows, Y is conditionally independent of given @ and X", which refer complete memorization (Definition I
because the test data label (right) can be inferred
META REGULARIZATION USING INFORMATION THEORY
At a high level, the sources of information in the predictive distribution (("" 0, D) come from the input, the meta-past paramearaes,rss, and the data. The memorization problem occurs when the model encodes task information in the predictive network that is readily available from the task training set (i.e., it memorizes the task information for each meta-training task). We could resolve this problem by encouraging the model to minimize the training error and to rely on the task training dataset as much as possible for the prediction of y (i.c., to maximize 1 (g"; x=" 0)). Explicitly maximizing [Ø;;; D w=" 0) requires intractable marginalization over task training sets compute q |x 0). Instead, we can implicitly encourage by restricting the information flow from other sources (x" and 8) to i" To achieve both low error and low mutual information between ÿ' and (x* 0),, the model must use task training data to make predictions, hence increasing the mutual information I(""; D xx" 0). leading reduced memorization. In this section, describe tractable ways to achieve this.
4.1 META REGULARIZATION ON ACTIVATIONS
Given 0, the statistical dependency between x² and 1² is controlled by the direct path from x" to 8" and the indirect path through D (see Figure D.. where the latter is desirable be- cause it leverages the task training data. We control the information flow between x" and "" by introducing an intermediate stochastic bottleneck variable z" such that q(ő" x=",, o. f) al00 1=" 0))(2")))))))))))))))) 10"" 0) dz" (Alemi et al. 2016) shown in Figure Now, would like
4

---

Published as a conference paper at ICLR 2020
to maximize ; to prevent memorization. We can bound this mutual information by
[0""
\0" [<=::00000 [x=; ; + ========0"""""
21(2";0" ()))))
=I(>";0" By(a (p(a))a(=* (0)
log ((2-10)
21(2";0" E log 0(=1<=" ,0 I (x= :0000000
(2)
r(z*)
where r(z*) ) is a variational approximation to the marginal, the first incquality follows from the statistical dependencies in our model (see Figure A and Appendix [A.2 for the proof). By simultane- ously minimizing E (g<=(((="la) ()([r(=))) ))] and maximizing the mutual information [(x"; (0,
we can implicitly encourage the model use the task training data D.
For non-m exxluusie problems, the true label y" is dependent on x². If the model has the
memorization problem and I(x"; (0) = O, then q(ū" |0," 0, D) = q( x-" 0) 7(5"'(00 (0), which means the model predictions do not depend on x* or 'D. Hence, in practical problems, the predictions generated from the model will have low accuracy.
This suggests minimizing the training loss in Eq. (1) can increase T(ū": D|æ" 8) I(x"; (0). Replacing the maximization of I (x"; ; (0) in Eq. (2) with minimizing the training loss results in the following regularized training objective
N Σ, (e
Σ JED
loga(0" = v² xx"' 0,0) 0) DDKx(((=") (000))((=)))))
K
where log q(""'="" 0) is estimated by log (0"","", 6,0) 0) with 2" -(2*|=",0) ,,, 9 modulates the regularizer and r(æ") can be set as NN, We refer to this regularizer as rectareeularizztioo (MR) on the activations.
As we demonstrate Section 6 we find that this regularizer performs well, but in some cases can fail to prevent the memorization problem. Our hypothesis is that in these cases, the network can sidestep the information constraint by storing the prediction of y in part of =", which incurs a small penalty in Eq. 3 and small lower bound in Eq. (2).
4.2 META REGULARIZATION ON WEIGHTS
Alternatively, we can penalize the task information stored in the meta-parameters 0. Here, we pro- vide an informal argument and provide the complete argument Appendix A..3 Analogous to the supervised setting (Achille & Soatto 2018), given meta-training dataset M, we consider as random variable where the randomness can be introduced by training stochasticity. We model the stochasticity over 0 with a Gaussian distribution (6 B, 0z) with learned mean and vari- ance parameters per dimension (Blundell et al.] 2015 Achille & Soatto (018)). By penalizing [Vi.N, PI.v; 0 xx) we limit the information about the training tasks stored in the meta- parameters 0 and thus require the network to use the task training data to make accurate predictions. We can tractably upper bound it by
I(vi.N DI:N: O) E log q(0)
((IM)
< E [DKL ((())))]))())))
(4)
where r(f) is variational approximation to the marginal, which set to (00 O, In practice, we apply meta-regularizztioo to the meta-paramett 0 that are not used to adapt to the task training data and denote the other parameters as In this way, we control the complexity of the network that predict the test labels without using task training data, but do not limit the complexity of the network that processes the task training data. Our final meta-regularized objective can be written as
N Σ, E (0000..0e))(0)1)..) -
Σ (a" (=*.0")ED;
log = "0,=",0,0,00 + 9DKL(q(0: 0n 0 (()))
(5)
5

---

Published as a conference paper at ICLR 2020
For MAML, apply meta-regularizztion to the parameters uninvolved in the task adaptation. For CNP, we apply meta-regularization to the encoder parameters. The detailed algorithms are shown in Algorithm and
and in in the appendix.
4.3 DOES META REGULARIZATION LEAD TO BETTER GENERALIZATION?
Now that we have derived meta regularization approaches for mitigating the memorization problem, theoretically analyze whether meta regularization leads to better generalization via a PAC-Bayes bound. In particular, we study meta regularization (MR) on the weights (W) of MAML, i.c. MR- MAML (W),
study.
Meta regularization on the weights MAML uses a Gaussian distribution (0; 0p. 0₂) to model the stochasticity in the weights. Given task and task training data, the expected error is given by
er(0 8 D) 10=N (.000,00))))000())) )o o 0,D), wy ())(a.l))))) [C(x", ylll=""y".o"" .y"
(6)
where the prediction loss C(x", .v" v,,"v"", o:) is bounded Then, we would like to minimize the error on novel tasks
er(0,.. 0,) = : T) [er(O,,, 6r D.
We only have a finite sample of training tasks, so computing er er(Q) is intractable, but we can form an empirical estimate:
ér(O,, to Di,Di D,. Di.
Σ
1oga(0"= = ""1r""
(8)
K
(e" (e"i)"))D:
(9.
where for exposition have assumed |D; K are the same for all tasks. We would like to
relate er(6,, , 8/) and ér(,,, 0, D1, Di Di..... D,., D",), ) but the challenge is that fn and 6g are derived from the meta-training tasks D,, Dj Di...... ...., D... D; There
sources of generalization error: (i)
error due to the finite number of observed tasks and (ii) error due to the finite number of examples observed per task. Closely following the arguments in (Amit Meir 2018), we apply standard PAC-Bayes bound to each of these and combine the results with a union bound, resulting in the following Theorem.
Theorem 1. Let P(O) be an arbitrary prior distribution over O that does not depend on the meta- training data. Then for any å E (0, 1I. with probability at least - 8, the following inequality holds uniformly for all choices of p, and 6..
0r) 5 Σ
er(O,,, 0g, D:)+
1=1
(V 2((K 1/2KK-1 -1 +
1 2(n 2(n
DKL(W((:0,, 0,) + log n(K + 1)
where n is the number of meta-training tasks and is the number of per-task validation datapoints.
We defer the proof the Appendix A.4 The key difference from the result in (Amit & Meir 2018 that we leverage the fact that the task training data is split into training and validation.
In practice, set P(0) r(0) O, If with high probability, our test error will also be low. As shown in the Appendix A..4] by a first order
achieve a low value for the bound, then
Taylor expansion of the the second term of the RHS in Eq. 9 and setting the coefficient of the KL
term as B 1/+
we recover the MR-MAML(W) objective (Eq.(5) 3 trades-
2 1/
off between the tightness of the generalization bound and the probability that it holds true. The result of this bound suggests that the proposed meta-regularization on weights does indeed improve generalization on the meta-test set.
In practice, C(x", y² (.) ))(r))))) MSE bounded target space or classification accuracy. We optimize the negative ligliikll as bound the 0-1 loss.

---

Published as a conference paper at ICLR 2020
5 RELATED WORK
Previous works have developed approaches for mitigating various forms of overfitting in meta- learning. These approaches aim to improve generalization in several ways: by reducing the number of parameters that are adapted in MAML (Zintgraf et al. 2019), by compressing the task embed- ding (Lee et al. 2019), through data augmentation from GAN (Zhang et al.) 2018), by using an auxiliary objective on task gradients (Guiroy et al.) 2019). and via an entropy regularization ob- jective (Jamal & Qi 2019). These methods all focus on the setting with mutually-excluse
task distributions. We instead recognize and formalize the memorization problem, particular form of overfitting that manifests itself with mutually--exllsiiii pled solution. Unlike prior methods, our approach is applicable to both contextual and gradient- based meta-learning methods. We additionally validate that prior regularization approaches, namely TAML (Jamal& & Qi 2019), are not effective for addressing this problem setting.
tasks, and offer a general and princi-
Our derivation uses a Bayesian interpretation of meta-learning (Tenenbaum) 1999, Fei-Fei et al.) 2003 Edwards & Storkey 2016 Grant et al.) [2018] Gordon et al., [0188] Finn et al. 2018 Kim
ct al.
2018; Harrison et al., 2018). Some Bayesian meta-learning approaches place a distributional
loss on the inferred task variables to constrain them to a prior distribution (Garnelo et al.) 2018b
Gordon et al., 2018: Rakelly et al., 2019). which amounts to an information bottleneck on the latent task variables. Similarly Zintgrat et al. (2019): Lee et al. (2019); Guiroy al. (2019) aim to produce SimilarlyZintgrrat simpler or more compressed task adaptation processes. Our approach does the opposite, penalizing information from the inputs and parameters, to encourage the task-specific variables contain greater information driven by the per-task data.
We use PAC-Bayes theory study the generalization error of meta-learning and meta-regularizzttonn Pentina & Lampert (2014) extends the single task PAC-Bayes bound (McAllester 1999) to the multi- task setting, which quantifies the gap between empirical error on training tasks and the expected error on new tasks. More recent research shows that, with tightened generalization bounds as the training objective, the algorithms can reduce the test error for mutually-exclusin Amit & Meir (2188). Our analysis is different from these prior works in that we only include pre- update meta parameters in the generalization bound rather than both pre-update and post-update parameters. In the derivation, we also explicitly consider the splitting of data into the task training set and task validation set, which is aligned with the practical setting.
tasks (Galanti et al. 2016
The memorization problem differs from overfitting in conventional supervised learning in several aspects. First, memorization occurs at the task level rather than datapoint level and the model mem- orizes functions rather than labels. In particular, within training task, the model can generalize to new datapoints, but it fails to generalize to new tasks. Second, the source of information for achiev- ing generalization is different. For meta-learning the information is from both the meta-training data and new task training data but in standard supervised setting the information is only from training data. Finally, the aim of regularization different. In the conventional supervised setting, regular- ization methods such as weight decay (Krogh & Hertz 1992). dropout (Srivastava et al., 2014), the information bottleneck (Tishby et al.) 22000; Tishby & Zaslavsky (015)). and Bayes-by- backkppp Blundell et al. 2015) are used to balance the network complexity and the information in the data. The aim of meta-regularization is different. It governs the model complexity to avoid one complex model solving all tasks, while allowing the model's s dependency on the task data to be complex. We further empirically validate this difference, finding that standard regularization techniques do not solve the memorization problem.
EXPERIMENTS
In the experimental evaluation, we aim to answer the following questions: (1) How prevalent the memorization problem across different algorithms and domains? (2) How does the memorization problem affect the performance of algorithms on non-mutually-exelusivee task distributions? (3) Is our regulaaaaaaii approach effective for mitigating the problem and is it compatible with multiple types of meta-learning algorithms? (4) Is the problem of memorization empirically distinct from that of the standard overfitting problem?
To answer these questions, we propose several meta- leea-learning problems involving non- non-mutually- exclusive task distributions, including two problems that adapted from prior benchmarks with mutually-exclusii task distributions. We consider model-agnostic meta-learning (MAML) and con- ditional neural processes (CNP) as representative meta-learning algorithms. We study both variants
7

---

Published as a conference paper at ICLR 2020
of our method in combination with MAML and CNP. When comparing with meta-learning algo- rithms with and without meta-roguuanization, we use the same neural network architecture, while other hyperparameters are tuned via cross-validation per-problem.
6.1 SINUSOID REGRESSION
First, we consider a toy sinusoid regression problem that is non-mutually-excluiivee The data for each task is created in the following way: the amplitdee A of the sinusoid is uniformly sampled from a set of 20 squualy--pacedd points {0.1, ......... :4; u is sampled uniformly from [-5, 5] and is sampled from NV (A sin( 0.12). We provide both and the amplitude A (as a one-hot vector) as input, i.e. x = (u, A). At the test time, we expand the range of the tasks by randomly sampling the data-generating amplitude A uniformly from [0.1, 4) and use random one-hot vector for the input to the network. The meta-training tasks are proper subset of the meta-test tasks.
Without the additional amplitude input, both MAML and CNP can easily solve the task and general- ize to the meta-test tasks. However, once we add the additional amplitude input which indicates the task identity, we find that both MA and CNP converge the complete memorization solution and fail to generalize well to test data (Table I and Appendix Figures 7 and 8). Both meta-regularized MAML and CNP (MR-MAML) and (MR-CNP) instead converge solution that adapts to the data, and as a result, greatly outperform the unregularized methods.
Table I: Test MSE for the non-mutually exxcluusee sinusoid regression problem. We compare MAMI. and CNP against reguularedd MAMI. (MR-MAMI, and meta-regularized CNP (MR-CNP) where regularization is either on the activations (A) or the weights (W). We report the mean over 5 trials and the standard deviation in parentheses.
MR-MAML (A) MR-MAML (W)
MR-CNP (A) (ours)
MR-CNP (W) (ours)
Methods
MAML
CNP
(ours)
(ours)
5 shot
0.46 (0.04)
0.17 (0.03)
0.16 (0.04)
0.91 (0.10)
0.10 (0.01)
0.11 (0.02)
10 shot
0.13 (0.01)
0.07 (0.02)
0.06 (0.01)
0.92 (0.05)
0.09 (0.01)
0.09 (0.01)
6.2 POSE PREDICTION
To illustrate the memorization problem on more realistic task, we create multi-task regression dataset based on the Pascal 3D data (Xiang et al. (014) (See Appendix/A.5.. A..3 a complete descrip- tion). We randomly select 50 objects for meta- cesta-taiinigg and the other 15 objects for meta-testing. For each object, we use MuJoCo (Todorov et al. (012) to render images with random orientations of the instance on a table, visualized in Figure I For the meta-learning algorithm, the observation (a)) is the 128 x 128 gray-scale image and the label (v) is the orientation relative to a fixed canonical pose. Because the number of objects in the veta-training dataset small, it straightforward for a single network to memorize the canonical pose for each training object and to infer the orientation from the input image, thus achieving a low meta-training error without using D. However, this solution performs poorly at the test time because it has not seen the novel objects and their canonical poses.
Optimization modes and hyperparameter sensitivity. We choose the learning rate from {0.0001, 0.0005, 0.001} for each method, 8 from {10- 10 5, .... .I} for reulaaegulaiizationn and report the results with the best hyperparameters (as measured on the vela--vaiiation set) for each method. In this domain, we find that the convergence point of the metalleaming algorithm is determined by both the optimization landscape of the objective and the training dynamics, which vary due to stochastic gradients and the random initialization. In particular, we observe that there two modes of the objective, one that corresponds to complete memorization and one that corresponds to successful adaptation to the task data, As illustrated in the Appendix, we find that models that converge to a memorization solution have lower training error than solutions which use the task training data, indicating a clear need for metaaregularization. When the meta-regularizztion is on the activations, the solution that the algorithms converge to depends on the learning rate, while MR on the weights consistently converges to the adaptation solution (See Appendix Figure for the sensitivity analy- sis). This suggests that MR on the activations is not always successful at preventing memorization. Our hypothesis is that there exists a solution in which the bottlenecked activations encode only the prediction y", and discard other information. Such a solution achieve both low training MSE and low regularization loss without using task training data, particularly if the predicted label con- tains small number of bits (i.e., because the activations will have low information complexity).
8

---

Published as a conference paper at ICLR 2020
However, note that this solution does not achieve low regularization error when applying MR to the weights because the function needed to produce the predicted label does not have low information the weights does not suffer from this pathology and
complexity. As result, netaareguuariiztioon robust to different learning rates. Therefore, we will use regularization on weights as the proposed methodology in the following experiments and algorithms in Appendix [..]
Quantitative results. We compare MAML and CNP with their meta-regularizze versions (Table 2
We additionally include fine-tuning as baseline, which trains single network on all the instances jointly, and then fine-tunes on the task training data. Meta-learning with meta-regularization (on weights) outperforms all competing methods by a large margin. We show test error as function of the meta-reeularizztion coefficient / in Appendix Figure 2 The curve reflects the trade-off when changing the amount of information contained in the weights. This indicates that / gives a knob that allows us to tune the degree to which the model uses the data to adapt versus relying on the prior.
MR-CNP
MR-MAML
10
6.0
9
5.5
8
5.0
7
4.5
MSE
MSE
4.0
5
3.5
3.0
2.5
2.0
0
le-5
le-4 Beta
1e-3
1e-2
0
1e-4
1e-3
1e-2 Beta
1e-1
1.0
Figure 2: The performance of MAML and CNP with mela-regularizztion on the weights, function of the regularization strength 3. We observe provides us knob with which we can control the degree to which the algorithm adapts versus memorizes. When B is small, we observe memorization, leading to large test error; when 8 is too large, the network does not store enough information in the weights to perform the task. Crucially, in the middle of these extremes, meta-regularization is effective in inducing adaptation, leading good generalization. The plot shows the and standard deviation across meta-training runs.
Table 2: Meta-test MSE for the pose prediction problem. We compare MR 11-MALLL (ours) with conventional
MAML and fine-tuning (FT). We report the average over 5 trials and standard deviation in parentheses.
MR-MAML (W)
MR-CNF (W)
Method
MAML
CNP
FT
FT Weight Decay
(ours)
(ours)
MSE
5.39 (1.31)
2.26 (0.09)
8.48 (0.12)
2.89 (0.18)
7.33 (0.35)
6.16 (0.12)
Comparison to standard regularization. We compare meta-regularization with standard regu- larization techniques, weight decay (Krogh & Hertz 1992) and Bayes-by-Be by--akkr-pBaappp-a--e--- (Blundell et al. 2015). in Table We observe that simply applying standard regularization to all the weights, as in conventional supervised learning, does not solve the memorization problem, which validates that the memorization problem differs from the standard overfitting problem.
Table 3: Meta-testing MSE for the pose prediction problem. We compare MR-CNP (ours) with conventional CNP, CNP with weight decay, and CNP with Bayesohy (BbB) regularization all the weights. We report the average over trials standard deviation parentheses.
Methods
CNP
CNP Weight Decay
CNP+ BbB
MR-CNP (W) (ours)
MSE
8.48 (0.12)
6.86 (0.27)
7.73 (0.82)
2.89 (0.18)
6.3 OMNIGLOT AND MINIIMGGENET CLASSIFICATION
Next, we study memorization in the few-shot classification problem by adapting the few-shot Om- niglot (Lake al. 2011) and Minilmagenet (Ravi & Larochelle, 2016 Vinyals al. 2016) bench-

---

Published as a conference paper at ICLR 2020
marks the non non-mutually-exclle cation problem, each class is (randomly) assigned a fixed classification label from to N. For each task, we randomly select corresponding class for each classification label and K task training data points and K task test data points from that c This ensures that each class takes only one clas- sification lahel across tasks and different tasks are non-mutually--xclusivee (See Appendix A.5.2 for details).
setting. In the ann-mutually-excllsstt N-way K-shot classifi-
We evaluate MAML, TAML (Jamal & Qi 2019), MR-MAML (ours), fine-tuning, and nearest classification tasks (TTblee4). We find that MR-MAML
neighbor baseline on con-mmtuulll--exeee significantly outperforms previous methods on all of these tasks. To better understand the problem, for the MAML variants, we calculate the pre-update accuracy (before adaptation on the task training data) on the meta-training data in Appendix Table The high pre-update meta-training accuracy and low meta-test accuracy are evidence of the memorization problem for MAML and TAML., indicating that it is learning a model that ignores the task data. In contrast, MR-MAML successfully controls the pre-update accuracy to be near chance and encourages the learner to use the task training data to achieve low meta-training error, resulting in good performance at meta-test time.
Finally, we verify that meta-regulariization does not degrade performance on the standard mutually- exclusive task. We evaluate performance as a function of regularization strength on the standard 20-way 1-shot Omniglot task (Appendix Figure [0), and we find that small values of / lead to slight improvements over MAMI.. This indicates that meta-regularization substantially improves performance in the non-mutualll--exlusive setting without degrading performance in other settings.
Table 4: Meta-test accuracy on
(NME) classification. The fine-tuning and nearest- 20-way S-shot
neighbor baseline results for Minilmagenet are from (Ravi & Larochelle 2016).
NME Onsngglot
20-way I-shot
NME Minilmagenet
5-way 1-shot 5-way 5-shot
MAML
7.8 (0.2)%
50.7 (22.9)%
Fine-tuning
28.9 (0.5))%
49.8 (0.8))%
TAMI. JJaal & Qi 2019
9.6 (2.3)%
67.9 (2.3)%
Ncarest-nnighhor
41.1 (0.7)%
(0.7) %
MR-MAML (W) (ours)
83.3 (0.8)%
94.1 (0.1)%
MAML
26.3 (0.7)%
41.6 (2.6)%
TAML Jamal & Qi 2019
26.1 (0.6)%
44.2 (1.7)%
MR-MAML (W) (ours)
43.6 (0.6)%
53.8 (0.9)%
7 CONCLUSION AND DISCUSSION
Meta-learning has achieved remarkable success in few-shot learning problems. However, we iden- tify pitfall of current algorithms: the need to create task distributions that are mutually exclusive. This requirement restricts the domains that meta-learning can be applied to. We formalize the fail- tasks
mode, i.e. the memorization problem, that results from training non-mutually--xcheeee and distinguish it as a function-level overfitting problem compared to the the standard label-level overfitting in supervised learning.
We illustrate the memorization problem with different meta-learning algorithms on a number of domains. To address the problem, we propose algorithm-agnostic meta-regularrization (MR) ap- proach that leverages an information-thcorrtie perspective of the problem. The key idea is that by placing soft restriction on the information flow from pattmearaeeters in prediction of test set la- bels, we can encourage the meta-learner to use task training data during meta-training. We achieve this by successfully controlling the complexity of model prior to the task adaptation.
The memorization issue is quite broad and is likely to occur in wide range of real-world appli- cations, for example. personalized speech recognition systems, learning robots that can adapt to different environments (Nagabandi et al. 2018), and learning goal-conditioned manipulation skills using trial-and-error data. Further, this challenge may also be prevalent other conditional predic- tion problems, beyond meta-learning, an interesting direction for future study. By both recognizing the challenge of memorization and developing general and lightweight approach for solving it, we believe that this work represents an important step towards making meta-learning algorithms applicable to and effective on any problem domain.
We assume that the number of classes in the meta-training set is larger than N.
10

---

Published as a conference paper at ICLR 2020
ACKNOWLEDGEMENT
The authors would like to thank Alexander A. Alemi, Kevin Murphy, Luke Metz, Abhishek Kumar and the anonymous reviewers for helpful discussions and feedback. M. Yin and M. Zhou acknowl- edge the support of the U.S. National Science Foundation under Grant IIS- 2SS-812260 1812699.
REFERENCES
Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep rep- resentations. The Journal of Machine Learning Research, 1944:1144--188, 2018.
Alexander A Alemi, lan Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.004100 2016.
Ron Amit and Ron Meir. Meta-learning by adjusting priors based on extended pac-bayes theory. In International Conference Machine Learning, pp. 205-214, 2018.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint arXivv1505.05424, 2015.
Thomas M Cover and Joy A Thomas. Elements of information theory, John Wiley & Sons, 2012.
Harrison Edwards and Amos Storkey, Towards neural statistician. 1006.2160..02185. 2016.
arXiv preprint
Li Fei-Fei et al. A bayesian approach to unsupervised one-shot learning of object categories. In Proceedings Ninth IEEE International Conference on Computer Vision, pp. 1134-1141. IEEE, 2003.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leariing for fast adaptation of deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 1126-1135. JMLR. org, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. Ad- vances in Neural Information Processing Systems, pp. 9516-9527, 2018.
Tomer Galanti, Lior Wolf, and Tamir Hazan. theoretical framework for deep transfer learning. Information and Inference: A Journal of the IMA, 5(2): 2016.
Marta Garnelo, Dan Rosenbaum, Chris Maddison, Tiago Ramalho, David Saxton, Murray Shana- han, Yee Whye Teh, Danilo J Rezende, and SM Eslami. Conditional neural processes. arXiv preprint wwXiv:1807.016133 2018a.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint, 2018b.
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner. Meta- learning probabilistic inference for prediction. arXiv preprint
2018.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-leaming as hierarchical bayes. arXiv preprint 1811v:1801.08930, 2018.
Simon Guiroy, Vikas Verma, and Christopher Pal. Towards understanding generalization in gradient- based meta-learning. arXiv preprint arXiv:1907.0720 2019.
James Harrison, Apoorva Sharma, and Marco Pavone. Meta-learning priors for efficient online bayesian regression. arXiv preprint
2018.
Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta- meta-learing for few-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11719- 11727, 2019.
Taesup Kim, Jaesik Yoon, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta-learning. arXiv preprint arXiv: /006.600..000 036603836, 2018
11

---

Published as a conference paper at ICLR 2020
Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, 2015.
Anders Krogh and John A Hertz. A simple weight decay improve generalization. In Advances in neural information processing systems, pp. 950-957, 1992.
Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science society, volume 33, 2011.
Yoonho Lee, Wonjae Kim, and Seungjin Choi. Discrete infomax codes for meta-learning. arXiv preprint arXiv:1905.11655 2019.
David A McAllester. 1999.
Pac-bayesian model averaging. In COLT, volume 99, pp. 164-170. Citeseer,
Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta- reinforcement learning. arXiv preprint arXiv:1803. 2018.
Anastasia Pentina and Christoph Lampert. pac-bayesian bound for lifelong learning. In Interna- fional Conference on Machine Learning, pp. 991-99919999991 2014.
Kate Rakelly, Aurick Zhou, Deirdre Quillen, Chelsea Finn, and Sergey Levine. Efficient off-policy learningg metaa-reinforcement learning via probabilistic context variables. arXiv preprint 19:3v00003000255 2019.
Sachin Ravi and Hugo Larochelle. Optimization as model for few-shot learning, In ICLR 2016, 2016.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with ing. pp. 1842 1850, 2016.
neural networks. In International conference on machine learn-
Jurgen Schmidhuber. Evolutionary principles in self-referential learning. On learning how to learn: The meta-meta.... hook.) Diploma thesis, Institut f. Informatik, Tech. Univ. Munich, 1:2, 1987.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 191)):9929-19888 2014.
Joshua Brett Tenenbaum. A Bayesian framework for concept learning. PhD thesis, Massachusetts Institute of Technology, 1999.
Sebastian Thrun and Lorien Pratt. Learning to learn. Springer Science & Business Media, 2012.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pp. 1-5. IEEE, 2015.
Naftali Tishby, Fernando Pereira, and William Bialek. The information bottleneck method. arXiv preprint physsics/004405,, 2000.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026-5033. IEEE, 2012.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pp. 3630-3638, 2016.
Yu Xiang, Roozbeh Mottaghi, and Silvio Savarese. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE Winter Conference on Applications of Computer Vision (WACV), 2014.
12

---

Published as a conference paper at ICLR 2020
Ruixiang Zhang, Tong Che, Zoubin Ghahramani, Yoshua Bengio, and Yangqiu Song. Metagan: An adversarial approach to few-shot learning. In Advances in Neural Information Processing Systems, pp.
2018.
Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast con- text adaptation via meta-lcarning. In Thitt International Conference on Machine Learning (ICML 20/9), 2019.
A APPENDIX
A.1 ALGORITHM
We present the detailed algorithm for meta-regularization on weights with conditional neural pro- cesses (CNP) in A and with model-agnostic meta-learning (MAML) in Algorithm 2 For CNP, we add the regularization on the weights of encoder and leave other weights unrestricted. For MAML, we similarly regularize the weights 0 from input to an intermediate hidden layer and leave the weights for adaptation unregularized. In this way, restrict the complexity of the pre-adaptation model not the post-adaptation model.
Algorithm 1: Meta- Regttarzeg CNP
input Task distribution p(T); Encoder weights distribution ((0;)) = (N(0; 7) with Gaussian parameters т (0,.: 0z); Prior distribution r(0) and Lagrangian multiplier 3; 0 that parameterizes feature extractor hg(+) and decoder (t(.). Stepsize a.
output: Network parameter 0.
Initialize r, randomly;
while not converged do
Sample mini-batch of { from p(T):
Sample 0 q0;;]] r) with reparameterization;
for all Ti E { do
Sample D; (z;, y:). DJ (a) yf) from Te
Encode observation 2; (m:)), 2 (at))
Compute task context Φ = allg(z(, y.), with aggregator a(-);
Update + + av Φ Σ
T. log q [Tp (2; o:));
Update + T aV, Σ T log ( (T o:)) [Dxx(((::+)1((00)
Algorithm 2: Meta- Regulaaege MAML
input Task distribution p(T); Weights distribution g(0;7) N((0;)) with Gaussian parameters
T= =-00,, 0z); Prior distribution r(0) and Lagrangian multiplier 3; Stepsize a, a'.
output: Network parameter 0.
Initialize r, randomly;
while not converged do
Sample mini-batch of {T,} from p(T); Sample 0 q(0; with reparameterizatioo; for all T E {T.) do
Sample D, (æ;, v.). DJ (m) y() from T
Encode observation 2; = go (m.)) 2 = go (x:)
Compute task specific parameter o, + a o
Update θ - 0 a En. log q 0:):
Update < r + av, log q 27 dx)) 3D (p((:(()))l
13

---

Published as a conference paper at ICLR 2020
Algorithm 3: Meta-Regularized Methods in Meta-testing
input : Meta-testing task with training data (a, y) and testing input x", optimized
parameters
output: Prediction 1j*
for k from K do
Sample 0g ~ q(0;
Encode observation 2 gea (z), 2 go, (xx):
Compute task specific parameter O% = alhg(= y)) for MR-CNP and
ØE + a' V'V, log (3)) #k: 0) for MR-MAML;
Predict U (0" 2 Øk
Return prediction i" E
A.2 META REGULARIZATION ON ACTIVATIONS
We show that 7 7xx;; ,= S T(O:: D|=", (0=)))). 0). By Figure By the chain rule of mutual information we have
we have that 1,
0.
Di=", = ;0² I[i"" (","0=" =*) =I(0"; (
-xx";"" 00,)" 10=,,
2 21(": =0,
(10)
A.3 META REGULARIZATION ON WEIGHTS
Similar to (Achille & Soatto 2018), we use & to denote the unknown parameters of the true data generating distribution. This defines a joint distribution M, 0) p()))((M())(())MM))) Fur- thermore, we have predictive distribution q( |=", D,0) = EœjoDD 10(0"==" Φ, 0)].
The meta-training loss in Eq. is an upper bound for the cross entropy H Hp.g(Vi.. <. DI,N 0). Using an information decomposition of cross entropy (Achille Soatto 2018). have
p-a (Vi.N Iii., DI.N. 0) H( DI,N, + L DI.N. 0) + (p(vi./( DINN 0)l PIN, 0))] I(Du,N; -Vvi.N DLN;; ())
(11)
Here the only negative term is the TyyiN: the meta-parameters contain about the meta-training data beyond what be inferred from the data generating parameters (i.c., memorization). Without proper regularization, the cross entropy loss can be minimized by maximizing this term. We can control its value by upper bounding
..N.) {), which quantifies the information that
TyiiN PI.N;ONai.,. 8) E log {)
(01loi.N. <)
((LM)
IE
log
(00/=i.Ns E)
EE DKL (9(0) ()).
I E [DLL (q(0) (M)l(())lr()
where the second equality follows because and g are conditionally independent given M. This gives the regularization in Section 4.2
A.4 PROOF OF THE PAC-BAYES GENERALIZATION BOUND
First, we prove a more general result and then specialize it. The goal of the meta-learner to extract information about the meta-tran training tasks and the test task training data to examples from the novel task. This information will be terms of a distribution Q over possible models. When learning a task, the meta-learner uses the training task data D and model
prior for test
14

---

Published as a conference paper at ICLR 2020
parameterized by 0 (sampled from Q(0)) and outputs distribution ((()D, models. Our goal is to learn Q such that it performs well on novel tasks.
To formalize this, define
wt where c(o(x*),y") is bounded loss in [0, 1]. Then, we would like minimize the error on novel lasks
er, = - [L(o(="),
(12)
er()) = min
(er
(13)
Q
Because we only have a finite training set, computing er(Q) is intractable, but we can form an empirical estimate:
ér(Q, Di; Dn, Dr)
Σ [(o()");"") """
(14)
n
K
(r(Q.D.)
where for exposition we assume K I is the same for all i. We would like to relate er(Q) and ér(Q, D1, Di, ..., D,, ). ). but the challenge is that Q may depend on D1, Di, Dn, D; due the learning algorithm. There of observed tasks and (ii) error due to the finite number of examples observed per task. Closely following the arguments in (Amit & Meir 2018), apply a standard PAC-Bayes bound to each of these and combine the results with union bound.
two sources of generalization error: (i) error due to the finite number
Let Q()) be distribution over parameters and let P(0) be a prior distribution. Then for any ō (0, 1), with probability least 5, the following inequality holds uniformly for all distributions Q
Theorem.
cr(()= S Σ D.) + (V2(X-11) + | 20 / DKZ(QIIP)+ log
n(K + +1
8
(15)
Proof. To start, we state classical PAC-Bayes bound and use it to derive generalization bounds on task and datapoint level generalization, respectively.
Theorem 2. Let X be sample space (i.e. a space of possible datapoints). Let P(X) be dis- tribution over X (i.e. data distribution). Let è be a hypothesis space. Given "loss function" 10,X) : ④ X X' + [0. and a collection of M i.i.d. random variables sampled from P(X), X1,..., XM. let n be a prior distribution over hypotheses in @ that does not depend on the samples but may depend on the data distribution P(X). Then, for any ó E (0, 11, the following bound holds uniformly for all posterior distributions p over θ
pl (Exx.P(X))0-00) [(00 <
Σ
M
M E. Bt ol [100 + $1 2(M - 1) (DKa(ell log 2 vp)
m=1
-1-6.
(16)
Meta-level generalization First, we bound the task-level generalization, that is relate er()) to 1 E Be (.) x (") ( [()((*)))))))))())))))))())))[) v.""), then Theorem says that for any dn (0,
n En cr(Q D., Tt). Letting the samples be X; = (D,, Tt), and 1(0, X,) =
2(n (DKL(QIP)) log " vQ
))) %0
P er(()) <
er er(Q, Tr) +
2 1 - -dp. (17)
where P prior over 6.
Within task generalization Next, we relate er(Q . Ta) to ér(Q. D,, Dj) via the PAC-Bayes bound, For fixed task i, task training data D,, a prior π ( that only depends the training
15

---

Published as a conference paper at ICLR 2020
data, and any o, (0, 11, we have that
(L (( [C(d)(x))).)) )."") sllo
Σ
((0i(x");)))
K
(a* "e"")")
K 8 vp) 2 85.
2(K - 1) (DLLollT) + log
Now, we choose T:) to be P(0) 8, Dj) and restrict p to be of the form f ( D, dd0 for any Q. While, π and p may be complicated distributions (especially, if they defined implicitly), we know that with this choice of and DKI(PlIT) DxI(QIIP) (Cover Thomas 2012), hence, we have
K (DRu(Q)P) + log
p er ( (
vQ
2
6:
2(K
8
(18)
Overall bound on meta-learner generalization Combining Eq. bound, we have
17 and
18
using the union
( Σ
(()) + V 2(K - 1) DxI(Q1))) log
K
n 1=1
8,
* 2(n -1) DKI(Q1 log " [. 21 Σ
6i + 50
(19)
50
Choosing 60 K+1 and K6 n(K+1j
8
then we have:
(1 (12KK-1)
p(er(Q) <
V DKI(Q]1P)+
"(K
vQ
2(n 2(n
8
21 6.
(20)
Because n is generally large, by Taylor expansion of the complexity term have
(V 2(K -1) (V2(K-1) + +1 2( 22n ((I)))/DDAOANN (DKLQ1IP) + log
n(K + +1)
8
1
1 2(K (V2LA*
+ V 2(n (DKLQIIP)+ 2
1
n(K
n(K+1) +1), o(1)
2 2olog n(K + 1)/8
8
Re-defining the coefficient of KL term 3 and omitting the constant and higher order term, recover the meta-regularizzatoo bound (5 when Q(0) = =N A,, 0z).
A.5
EXPERIMENTAL DETAILS
A.5.1
POSE PREDICTION
We create multi-task regression dataset based on the Pascal 3D data (Xiang et al.) 2014). The dataset consists of 10 classes of 3D object such as "aeroplane", "sofa", "TV monitor". etc. Each class has multiple different objects and there are 65 objects in total. We randomly select 50 objects for meta-training and the other 15 objects for meta-testing. For each object, we use MuJoCo (Todorov) et al. 2012) to render 100 images with random orientations of the instance Figure For the meta-learning algorithm. the observation (z: is the 128 128 gray-scale image and the label (y) is the orientation re-scaled be within [0, 10). For each task, randomly sample
table, visualized in
16

---

Published as a conference paper at ICLR 2020
30 (a, y) pairs for object and evenly split them between task training and task test data. We use meta batch-size of 10 tasks per iteration.
For MR-CNP, use convolutional encoder with fully connected bottom layer map the input image to a 20-dimensional latent representation & and z for task training input a and test input x respectively. The (2, y) concatenated and mapped by the feature extractor and aggregator which are fully connected networks to the 200 dimensional task summary statistics ø. The decoder is a fully connected network that maps (Ø, =*) to the prediction 0"
For MR-MAML, we use a convolutional encoder to map the input image to a 14 x 14 dimensional latent representation and =². The pairs (z, 3) are used in the task adaptation step to get task spe- cific parameter é via gradient descent. Then 2" is mapped to the prediction j' with convolutional predictor parameterized by Φ. The network is trained using gradient steps with learning rate 0.01 in the inner loop for adaptation and evaluated using 20 gradient steps at the test-time.
A.5.2 NON-MUTUALLY-EXCUUT
CLASSIFICATION
The Omniglot dataset consists of 20 instances of 1623 characters from 50 different alphabets. We randomly choose 1200 characters for meta-training and use the remaining for testing. The meta- training characters are partitioned into 60 disjoint sets for 20-way classification. The Minilmagenet dataset contains 100 classes of images including 64 training classes, 12 validation classes, and 24 test classes. We randomly partition the 64 meta-training classes into 13 disjoint sets for 5-way classification with one label having one less class of images than the others.
For MR-MAML we use convolutional encoder similar to the pose prediction problem. The dimen- sion of 2 and 2" is 14 x 14 for Omniglot and 20 20 for Minilmagenet. We use convolutional decoder for both datasets. Following (Finn et al. 2017). we use a meta batch-size of 16 for 20- way Omniglot classification and meta batch-size of 4 for 5-way Minilmagenet classification. The meta- learning rate chosen from (0.001, 0.005] and the 3 for meta-regularizee methods are chosen from {10-7, 7, 10 ,....,.0"" 10-33. The optimal hyperparameters chosen for each method separately via cross-validation.
A.6 ADDITIONAL ILLUSTRATION AND GRAPHICAL MODEL
We show a standard few-shot classification setup meta-learnigg illustrate mutually-excluse task distribution and a graphical model for the regularization on the activations.
Task training data
Task test data
Meta-training
Shuffle Isbela
Meta-testing
Figure 3: An example of mutually-exxluse task distributions. In each task of muuuullly-xxlluiing few-shot classification, different classes are randomly assigned to the N-way classification labels. The same class, such as the dog and butterfly in this illustration, can be assigned different labels across tasks which makes it impossible for one model to solve all tasks simultaneously.
17

---

Published as a conference paper at ICLR 2020
M
P(T)
Φ
D
T
jk
Figure 4: Graphical model of the regularization on activations. Observed variables are shaded and Z is bottleneck variable. The complete memorization corresponds the graph without the dashed arrows.
A.7 ADDITIONAL RESULTS
As shown in Figures 5 and when meta-learning algorithms converge to the memorization so- lution, the test tasks must be similar to the train tasks in order to achieve low test error. For CNP, although the task training set contains sufficient information to infer the correct amplitude, this in- formation is ignored and the regression curve at test-time is determined by the one-hot vector, As a result, CNP can only generalize to points from the curves it has seen in the training (Figure 7 first row). On the other hand, MAML does use the task training data (Figure 8 and Table T.. however, its performance is much worse than in the mutually-exclaa eexluusve task. MR-MAML and MR-CNP avoid converging to a memorization solution and achieve excellent test performance on sinusoid task.
5-shots
10-shots
1.0
1.0
MAML
0.8
MR- -AML(AA)
0.8
(R-MMAM((A) MR-MAML(()
MR- MMML(WW)
0.6
0.6
MSE
MSE
0.4
0.4
0.2
0.2
0.0
0.0
1
100 200
1
100 Gradient Steps
5
200
Gradient Steps
Figure 5: Test MSE on the mutually-non-exclusiee sinusoid problem as function of the number of gradient steps used in the inner loop of MAML and MR-MAMI. For each trial, we calculate the mean MSE over 100 randomly generated meta-testing tasks. We report the mean and standard deviation over random trials.
(a) CNP
(b) MR-CNP (W)
(c) MAML
(d) MR-MAML (W)
lecoal
11
Figure 6: Visualization of the optimized weight matrix W that connected to the inputs in the sinusoid regression example. The input x = (u, A) where u ~ Unif( (,if(-5,, 5). A is 20 dimensional one-hot vector and the intermediate layer is 100 dimensional, hence x E R221 and W E R 2000000 For both CNP and MAMI., the meta-regularizztioon restricts the part of weights that is connected to A close to 0. Therefore it avoids storing the amplitude information in weights and forces the amplitude to be inferred from the task training data D, hence preventing the memorization problem.
18

---

Published as a conference paper at ICLR 2020
(a) CNP
Trie Fiemmio Pezcitioon Cantcal Tonts
0-
(b) MR-CNP (A)
True Funcson
Presitton
(c) MR-CNP (W)
The
Controt Foints
Figure 7: Meta-test results on the non-mon con-mutually-xxllune sinusoid regression problem with CNP. For each row, the amplitudes the true curves (orange) are randomly sampled uniformly from [0.1,4] 4]. For illustrative purposes, we fix the one-hot vector component of the input. (a): The vanilla CNP cannot adapt to new task training data at test-time and the shape of prediction curve (blue) is determined by the one-hot amplitude not the task training data. (b) (c): Adding meta regularization both activation and weights enables the CNP use the task training data at meta training and causes the model generalize well at test-time.
19

---

Published as a conference paper at ICLR 2020
(a) MAML
Altor
After Sespo
(b) MR-MAML (A)
Tearnin Parks
Trus Punction
Aft
Mfer 100 Shept
1
(c) MR-MRRMMM (W)
After Staps
Afbe Stapa
Figure 8: Meta-test results on the non-mutuully--xclusie sinusoid regression problem with MAML. For each row, the true amplitudes of the true curves (orange) are randomly sampled uniformly from [0.1, 4), For illustrative purposes, adapts slowly and has large generalization error test-time (b) (c): Adding meta-regularization on both activation and weights recovers efficient adaptation.
fix the one-hot vector component of the input. (a): Due memorization, MAML
MR-CNP (A)
MR-CNP (A)
MR-CNP (W)
MR-CNP (W)
06
Lossttrain)
63
Learring rate
Leamiro rate
Learring FOO2
JeATTING rate
Figure Sensitivity of activation regularization and weight regularization with respect to the learning rate on the pose prediction problem. For activation regularization, lower training loss corresponds to higher test MSE which indicates that the memorization solution is not solved. For weights regularization, lower training loss corresponds to lower MSE which indicates proper training converge the adaptation solution.
20

---

Published as a conference paper at ICLR 2020
In Table 5 report the pre-update aceuracy for the con-mutually-xxllssitt classification exper- iment in Section 6.3 The pre-update accuracy is obtained by the initial parameters rather than the task adapted parameters ø. At the meta-training time, for both MAML and MR-MAML the post-update accuracy obtained by using @ gets close to 1. High pre-update accuracy reflects the memorization problem. For example, in 20-way 1-shot Omniglot example, the pre-update accuracy for MAML is 99. 2% at the training time, which means only 0.8% improvement in accuracy is due adaptation, so the task training data is ignored for MR-MAML is 5%, which means 95% improvement in accuracy during training is due to the adaptation. This explains why in Table the test accuracy of MR-MAML is much higher than that of MAML at the test-time, since the task training data is used to achieve fast adaptation.
large extent. The pre-update training accuracy
Table 5: Meta-training pre-update accuracy on non-mutually excllusive classification. MR-MAML controls the meta-training pre-update accuracy close to random guess and achieves low training after adaptation.
NME Omniglor
20-way 1-shot
20-way 5-shot
NME Minilmagenet
5 5-way I-shot 5-way 5-shot
MAML
99.2 (0.2)%
45.1 (38.91%
MAML
99.4 (0.1)%
21.0(1.))%
TAML
68.9(43.1)%%
6.7 (1.8)%
TAML
99.4 (0.1)%
20.8(0.4)%
MR-MAML (ours)
(0)%
5.0 (0)%
HR-MAML (ours)
20.0(0)%
20.2(0.1)%
Mutually-exclusive Omniglot 20-way 1-shot
0.8
Accuracy
0.6
0.4
0.2
MAML 1e-07 1e-06 1e-05 5e-05 0.0001 0.001
Beta
The test accuracy of MAML with meta-regularization on the weights a function of the regular- ization strength 8 the mutually- exclusive 20-way 1-shot Omniglot problem. The plot shows the mean and standard deviation across meta-train vectatrainnin runs. When 3 small, MR RR-M slightly outperforms MAML, indicating that meta- regularauaii does not degrade performance on mutually-exelusin tasks. The accuracy numbers are not directly comparable to previous work (e.g. (Finn et al. 2017) because we do not use data augmentation.
Figure 10:
21


Reviews:
[{"review_id": "rygYKn-hYr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "################################################################################\nSummary:\n\nThis paper illustrates, identifies, and formally defines a memorization problem in meta-learning -- the model can simply memorize meta-training tasks and ignore meta-training train sets. The paper proposes to optimize the mutual information between testing predictions and the training data (given input and meta model), and upper bound it by imposing a information bottleneck between output and input+model. Unlike related work, this paper specifically is able to generalize to meta-test even when the meta-train dataset is not made confusing enough (i.e. even when model can learn well from test data in meta-train alone), making it applicable to use cases where it is hard to make the dataset confusing.\n\n################################################################################\nDecision\n\nI vote for accepting this paper, since as far as I know this paper gives a novel insight to the overfitting problem in meta learning, and has formulated the problem formally with theoretical insight, and given a working solution with strong experiment results and clean comparative studies. Despite somewhat narrow experiments and sometimes confusing writing, the paper should provide new insight to meta-learning.\n\n################################################################################\nPros:\n! DISCLAIMER: I am not an expert in this field, so take my novelty judgements with a grain of salt.\n+ Novel view into meta-learning's overfitting problem\n(1) Large models can simply memorize which input data corresponds to which task, and memorize the meta-training tasks, without being able to generalize into meta-test tasks.\n(2) Formulates this into a low mutual information between meta-train training data and predictions.\n+ Easy to implement and quite widely applicable as a regularization loss addon to multiple existing meta-learning methods\n+ Impact-wise, the paper takes meta-learning further from memorization, making methods more capable of operating on less-carefully designed, more natural datasets (rather than permutation of datasets)\n+ Experiments are clean with ablation studies and hyperparameter sensitivity tests, and method performs well across real and toy datasets\n+ Motivation part is easy to read\n\n################################################################################\nCons:\n- I'm still skeptical of the novelty of the paper, since the conclusions are, in hindsight, very straightforward.\n- Sometimes sentences are very confusing to readers.\n(1) The term \"mutually-exclusive\" is confusing because the view-point example the paper gives seems to be mutually exclusive (each task has its own kind of data, hence \"exclusive\"). It is unclear whether the task data is exclusive, or the task function is exclusive, and not straightforward to see its relationship with memorization. Consider renaming it to e.g. \"mutually-confusing\" or \"mutually-contradictory mapping\".\n(2) Can you please rename \"information-theoretic meta-regularizer\" to \"meta-regularizer using information theory\"? It is hard to read for non-native speakers.\n(3) Paragraph under definition 1 is confusing and has redundancies.\n(4) Section 4.1, not very clear how the logic goes from the decomposition to adding the upper bound to the loss, and how the other term comes in.\n- For the motivation, it is better to give examples of use cases when it is impossible to make meta-train \"mutually-exclusive\". I'm sure even in the patient example you can shuffle classes or input dimensions.\n- An experiment comparing to others in mutually-exclusive datasets would be nice to have, in order to judge how much this compensates a badly-designed meta-learning dataset.\n\n################################################################################\nImprovements:\n- Clarify each point in the \"Cons\" section.\n- Please also clarify if all methods in all experiments are hyperparameter-tuned separately, i.e. that the experiments are not favoring the MR-* model in any way. (the paper only clarifies it in one of the experiments)\n- For future work, does recent developments in mutual information modeling (e.g. MINE https://arxiv.org/abs/1801.04062) help this method in any way? e.g. try increasing mutual information between some representation of the meta-train training data and some feature vectors before the prediction?\n- First parenthesis in Section 4.1 has a misplaced space\n\n\n\n################################################################################\nPost rebuttal\n################################################################################\nIt seems that reviewers agree that the contributions are novel (regardless of whether each reviewer thinks it is trivial). So that addresses my main concern. I think the contributions are novel enough since it gives theoretical guidance as well. Other concerns are mostly addressed by the rebuttal. I will keep my rating.\n\nAlthough I do urge the authors to reconsider the name choice \"mutual-exclusive tasks\" since it is not very informative and quite confusing to readers.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkgyWzXhFH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary:\n\nIn this paper, the authors propose a new method to alleviate the effect of meta over-fitting. The designed method is based on the information-theoretic meta-regularization objective. Experiments demonstrate the effectiveness of the proposed model.\n\nStrong Points:\n\n+ The authors aim to alleviate the effect of meta over-fitting. In this paper, they mainly focus on alleviating the effect of brute-force memorization in the meta-training process. The problem is important in the meta-learning field.\n\n+ The motivation for this paper is clear. The authors try to maximize the mutual information between x*, \\theta and \\bar{y}^*, D. \n\n+ Experiments on both sinusoid regression, pose regression and image classification show that MR-MAML outperforms MAML and MR-CNP outperforms CNP. \n\nWeak Points:\n\n- My first concern is about the novelty of the proposed model. The framework and the derivations are straightforward. I think the problem is very important, however, the technical contribution may not enough to be accepted. It is better for the authors to clarify their contributions.\n\n- It will be more helpful if the authors can describe the algorithm of the meta-testing process in Appendix A.1. In the meta-testing process, do we need to sample \\theta from q(\\theta|\\tau)? If so, is the accuracy calculated by the averaged value of tasks with sampled weight?\n\n- I am a little curious about the results in Table 5. The results of MAML and TAML is quite high. It would be better if the authors explain more.\n\nAfter rebuttal\nI think the authors' response and the revised paper address most of my concerns. I raise my score to 6.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Bylzova-cH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper analyses a pitfall of current meta-learning algorithms, where the task can be inferred from the meta-training data alone, leaving the task-training data unused. Such a meta-learner would generalise well on the meta-training tasks, but will fail to generalise on new tasks at test time. This kind of overfitting is formalised as the memorization problem. This problem is implicitly resolved in current meta-learning algorithms by constructing mutually-exclusive meta-training tasks, which is not easy to construct in all scenarios. The paper introduces an information-theoretic meta-regularizer which forces information extraction from the task data (D) by restricting information flow from meta-parameters (\\theta) and input (x^*). Experimental evaluation with one gradient based and one contextual meta-learning method, on non-mutually-exclusive tasks bring out the mettle of the proposed regulariser. \n\n+ves:\n+ The characterization of the memorization problem and the proposed regularizer are novel contributions. \n+ The paper motivates the problem well, before proposing the methodology.\n+ The paper is well-organised and the experimental setting is designed in a thoughtful manner.\n+ The results are promising.\n\nConcerns:\n- The key hypothesis that the proposed meta-regularization method is based on - is that a model with memorization tends to be more complex. What is the basis for such an assumption? This is an important one for the work at the outset.\n\n- Would the proposed regularizer help if mutually-exclusive meta-training tasks are available, as it forces the model to extract maximum information from task training data (D)? The paper does not comment on this, and this would have been useful to know.\n\n- How much is the training overhead (in terms of time) incurred while adding the regularizer to the baseline methods (MAML and CNP)? The paper does not talk about this additional complexity.\n\n- Evidently, the most important results in the Experiments section are the ones in Sec 6.3. However, the results do not clearly distinguish whether the meta-regularization was performed on the activations or weights here (earlier subsections do talk about this). This makes it difficult to make a conclusive inference on what aspect of the methodology actually helped here.\n\n- There have been recent efforts that have attempted addressing overfitting in meta-learning. The paper mentions these efforts in Sec 5, and states that these have been used for existing settings where tasks are mutually exclusive. It would have been useful to include at least one of these methods in the experiments to see how the proposed regularization differs from them in practice.\n\nMinor issues:\n- The abstract says: “This causes the meta-learner to decide what should be learned from data and what must be inferred from the input.” - what is the difference between data and input?\n- There are some minor typos in the work, which would benefit from a proofread. E.g: Sec 6.3 “neigbhor” -> “neighbor”\n\n===== POST-REBUTTAL COMMENTS =========\nI thank the authors for the response, the clarifications and the updated manuscript. I am happy to upgrade my rating to Accept. \n\nThe concerns regarding the ‘higher complexity of memorized models’ has been addressed convincingly in the narrative, and the visualization of weights for models with and without using the proposed regularizer makes the argument cogent. \n\nMentioning the number of  gradient steps used to obtain the results on mutually-exclusive meta-training tasks (Figure 9) in the narrative would help. Was the same number of steps used for MAML and MR-MAML experiments? The optimal \\beta value would be very important if we want to use MR-MAML in situations where mutual-exclusivity of tasks is not known a-priori (as alluded to in the rebuttal). \n\nI would encourage the authors to include training overhead (in terms of time) in the paper, even if it is minimal, as it would clear concerns of the reader. I would also highly encourage the authors to release the code as this would help easy reproducibility of the results. \n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces the concept of overfitting in meta learning and proposes some solutions to address this problem. Overall, this is a good paper. It would be good if the authors could relate this work to meta learning approaches, which are based on hierarchical (Bayesian) modeling for learning a task embedding.

[1] Hausman et al. (ICLR 2018): Learning an Embedding Space for Transferable Robot Skills 
https://openreview.net/pdf?id=rk07ZXZRb
[2] Saemundsson et al. (UAI 2018): Meta Reinforcement Learning with Latent Variable Gaussian Processes
http://auai.org/uai2018/proceedings/papers/235.pdf

Decision: Accept (Spotlight)