Variational Continual Bayesian Meta-Learning
Authors: Qiang Zhang
Jinyuan Fang
Zaiqiao Meng
Shangsong Liang
Emine Yilmaz
Venue: NeurIPS 2021 Poster (2021)
Keywords: meta-learning
non-stationary task distribution
dynamic Gaussian mixture model
structural variational inference

Abstract: Conventional meta-learning considers a set of tasks from a stationary distribution. In contrast, this paper focuses on a more complex online setting, where tasks arrive sequentially and follow a non-stationary distribution. Accordingly, we propose a Variational Continual Bayesian Meta-Learning (VC-BML) algorithm. VC-BML maintains a Dynamic Gaussian Mixture Model for meta-parameters, with the number of component distributions determined by a Chinese Restaurant Process. Dynamic mixtures at the meta-parameter level increase the capability to adapt to diverse tasks due to a larger parameter space, alleviating the negative knowledge transfer problem. To infer posteriors of model parameters, compared to the previously used point estimation method, we develop a more robust posterior approximation method -- structured variational inference for the sake of avoiding forgetting knowledge. Experiments on tasks from non-stationary distributions show that VC-BML is superior in transferring knowledge among diverse tasks and alleviating catastrophic forgetting in an online setting.



Variational Continual Bayesian Meta-Learning
Qiang Zhang' 1223+f, Jinyuan Fang", Zaiqiao Meng".0, Shangsong Liang*.0, Emine Yilmaz?
Hangzhou Innovation Center, Zhejiang University, China
College of Computer Science and Technology. Zhejiang University, China
3 AZFT Knowledge Engine Lab, China; Sun Yat-sen University, China
University of Glasgow, United Kingdom
Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates
University College London, United Kingdom
fangiy6@ coaeemmmalllm.m.m zaiqiao. wttpiiiammmenggmmlll..ooo @gmail.comm
[1iaagshhangoong@maalll com; emine amine..ilmazaaei yilmazuucl. bock
Abstract
Conventional meta-learning considers set of tasks from a stationary distribution. In contrast, this paper focuses on more complex online setting, where tasks arrive sequentially and follow a non-stationary distribution. Accordingly, we propose a Variational Continual Bayesian Meta- Meta-Learning (VC-BML) algorithm. VC-BML maintains a Dynamic Gaussian Mixture Model for peta-paraametes, with the number of component distributions determined by a Chinese Restaurant Process. Dynamic mixtures at the meta-pant peta-perrmeter level increase the capability to adapt diverse and dissimilar tasks due to larger parameter space, alleviating the negative knowledge transfer problem. To infer the posteriors model parameters, compared the previously used point estimation method, develop more robust posterior approximation method structured variational inference for the sake of avoiding forgetting knowledge. Experiments tasks from non-stationary distributions show that VC-BMI. is superior in transferring knowledge among diverse tasks and alleviating catastrophic forgetting in an online setting.
1 Introduction
Meta-learning inductively transfers knowledge among analogous low-resource tasks, i.c., tasks with scarce labeled data such few-shot image classification, to enhance model generalization and data efficiency 2 Conventional meta-learning assumes that these tasks follow a stationary distribution. However, this assumption may be unrealistic in online setting, where the task distribution is normally non-stationary, i.e., subsequent tasks are disparate and heterogeneous B 4 In reality, the newly arriving tasks may differ from previous
contradict to each other [5), e.g., the task of online landmark prediction can encounter images from unnamed classes. This leads to two issues: (1) single of model parameters exhibit decreased performance disparate tasks, called the negative knowledge transfer issue (6;; and (2) for fast adaptation to those disparate tasks, a model often has dramatically change its parameters and thus often forgets previously learnt knowledge, known as the catastrophic forgetting issue 77] Due to these issues, it is inefficient to use conventional meta-learning in online setting. This motivates
extend the conventional meta-learning to deal with the streaming low-resource tasks that follow non-stationary distribution, such that we can dynamically update the model effectively transfer knowledge while avoid forgetting.
'The work was done while at University College London Equal Contributions 1 Corresponniig Author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Several attempts have been made to study meta-learning for low-resource tasks from a non- stationary distribution. One branch is online meta-learning E through the perspective of nggeet-miniiization, where the goal is to minimize the accumulative loss of the best fixed model in hindsight. These algorithms require accumulating subsequent data for training, which may not be realistic as the size of datasets often prohibits frequent batch updating. Another branch is continual learning, which avoids revisiting previous data and aims to overcome the catastrophic forgetting issue. Relevant works fell in this branch are either based on single set of meta-parameters mixture of task-specific parameters. Typically, a single meta-parameter distribution [6] has a limited parameter space, which inhibits the capability to handle the boundlessly diverse tasks and incurs the negative knowledge transfer issue, leading to suboptimal performance. A mixture model has a larger parameter space and Jerfel et al. [ build mixture of task-specific parameter distributions, but the meta-parameters still follow delta distribution. Also, they use point estimation to infer parameters, which is prone suffer from the catastrophic forgetting issue in online setting.
To fill the research gap, this paper proposes Variational Continual Bayesian Meta-Learning (VC- BML) algorithm that tackles the issues of negative knowledge transfer and catastrophic forgetting for streaming low-resource tasks. As a fully Bayesian algorithm, VC-BML assumes meta-parameters and task-specific parameters follow their respective distributions. We set out to make theoretical and empirical contributions as follows. (1) We propose meta paraaeertsttrss to follow a mixture of dynamically updated distributions, each component of which is associated with a cluster of similar tasks. By assuming meta-paraame parameters follow Gaussian distributions, we model the whole mixture with Dynamic Gaussian Mixture Model (DGMM). Compared to single set of meta- parameters or a mixture of task-specific parameters, dynamic mixtures of meta-parameter distributions provide one more level of flexibility in larger parameter space and thus increase the capability alleviate the negative knowledge transfer issue. (2) Unlike the previous work that applies point estimation during the inference, which is prone to forgetting knowledge, we approximate the posterior distribu- tions of interest by deriving a structured variational inference method, Given that, we can sample from parameter distributions and quantify the model uncertainty. (3) Finally, extensive experiments show our VC-BML algorithm outperforms seven state-of state-of-the-arr art baselines on non-stationary task distributions from four benehmark datasets. It has empirically shown that the Bayesian formulation of our algorithm can alleviate negative transfer among dissimilar tasks and prevent dramatic parameter changes to overcome the catastrophic forgetting issue.
Literature Review
Online Meta- Mitta-Learning, Meta-learning extracts transferable knowledge from a set of meta-training
datasets to efficiently tackle low-resource tasks 22, such as few-shot image classification [10 and
robot control [11] Various approaches, including model-based (or black box) (12), metric-based
(or non-poranae con-parametric) [13, optimization-ban based [14] and their Bayesian counterparts [6] 17 18].
have been proposed. However, these algorithms assume that tasks are from a stationary distribution, which unrealistic in dynamic learning scenarios. To handle sequentially arriving tasks from non-stationary distribution, online meta-learning algorithms have been under studied. Two settings for online meta-learning are identified [5]: the online-within within-onlline setting where tasks and examples sequentially
within tasks arrive sequentially, and the online-withhe wwthin-thhiiaaieeee batch setting where tasks arrive weeeeeeeeeueiie but examples within tasks are in batch. Most of the concurrent works belong to the latter setting. From the viewpoint of eggret-mniiizzttoon Follow- Follow-The-Mer The-Meta-Leadee algorithm is proposed in k and generalized from convex to non-convex cases in 9 In [8) the meta-learner disentangled as meta-hierarchical graph consisting of multiple knowledge blocks in deterministic way. These algorithms make assumptions on regret functions and require accumulating subsequent datasets, which puts high demand for computational memory.
Continual Learning. Continual learning another paradigm for sequentially arriving tasks. It avoids revisit previous data and aims to overcome catastrophic forgetting 77. Techniques such as elastic weight consolidation 19. variational continual learning (20), online Laplace approximation [1] and
brain-inspired replay [22] have been developed. Although tasks arrive sequentially, most continual learning works primarily focus on supervised learning with large-scaled annotations, which is opposed to our low-resource tasks. Continual-meta learning [23] and meta-continual learning (24] bridge the gap between meta-learning and continual learning. The former aims for quickly recover performance previous tasks, which is a different research goal from this paper. The latter uses

---

Latent space of meta- meta-parant parameters
Task-specifie parameters
Task datasets
Task datasets
Df
D. (2j,2))
D2
:
:
D. (DE,D)
Df
$
Dr
D+1 (Di.1.
Φ DDe1
D
Figure 1: The framework of VC-BML.. It infers (dashed line) the variational distributions of meta- paramearre (Gaussian-shaped distributions) based on tasks. It then ranks candidate mata-parameter distributions in the latent space based on the posterior and select the one with highest probability. Subsequently, task task-specific parameters are derived from the meta-pat parammeessssrss and the support set, which efficient for making predictions (solid line).
meta-learning as a way to overcome catastrophic forgetting, but ignores the low-resource task setting. Moreover, they have been extended to online fast adaptation and knowledge accumulation 25
Some continual learning works combine meta-learning to deal with low-resource tasks. For example, Jerfel et al. [4] propose to cluster task-specifie parameter distributions and allow the meta-learner to select over mixture of these distributions. Nonetheless, the meta-parameters still follow delta distributions and the latent variables are inferred by point estimation, which found prone to suffer from the catastrophic forgetting issue. Yap et al. [6] make steps in Bayesian approximation to parameter distributions, but they only assumes single meta-parameter distribution, which is not capable enough to deal with boundlessly diverse low-resource tasks in streaming scenario. We overcome these weaknesses by representing neta-parameters with dynamically updated mixture model and inferring latent variables via structural variational inference.
3 Background
3.1 Problem Formulation
Given sequentially arriving tasks that follow a distribution p(r), at each time step f, a machine learning model receives task with dataset D, sampled from the data distribution p( (D.)Te). Similar the
conventional meta-learning setting [14, the dataset D, is split into support set D {(x,, ):)) N,
for training and a query set DI {(x,, vx)) Nº for validation. Two fundamental elements build up our research problem. First of all, p(T) can be non-stationary, i.e., subsequent tasks from this distribution can be disparate and heterogeneous 3 This heterogeneity leads negative knowledge transfer when tasks are dissimilar, and often requires substantial parameter changes for fast adaption, causing the issue of catastrophic forgetting (7). Second, D, contains few shots (i.e., small N₂¹) that are not enough for large-scale supervised training, which motivates us to update model parameters with meta- learning.. Without revisiting previous data, we aim to dynamically update the model parameters to efficiently transfer knowledge to new tasks and avoid the issue of catastrophic forgetting.
3.2 Preliminary
Meta-Learning, Meta-learning aims to discover the shared common patterns among a set of learning tasks drawn from distribution p(7), such that the model is able accomplish new learning tasks with limited number of training examples and trials. To this end, the learning procedure of meta- learning split into two steps. The first step learn the meta meta- parameters 0* from the meta-training dataset Dmeta- 0" arg max, log p( (eDDmmtttraa train 0) 0* can be viewed as the common knowledge among tasks from p(T). Using 0* as prior, the second step is to obtain the task-specific model parameters o that are able generalize well on the meta-test set Dmett test after few trials: Φ² arg max & log p( (''Dnttttte test (b, f*). The task-specific parameters &" can be seen as transferred knowledge from the common among collection of tasks.

---

Variational Continual Learning. Continual learning aims to overcome the catastrophic knowledge forgetting issue when dealing with sequentially occurring tasks from a non-stationary distribution. Variational continual learning (VCI.) (20) is a Bayesian approach to continual learning that offers an efficient way to infuse past knowledge via posteriors. With streaming datasets Di.t. VCL. factorizes the posterior of model parameters as:
p (0, Dir)) x p(O,) II p (Dr (0)) x (D, ( Dia-1)- #=1
(1)
The datasets are assumed be independent given the model parameters 9. As we can see from Equation [] the parameter posterior on the (1 1)-th dataset is consecutively taken as its prior on the (4) dataset. This motivates recursive way of posterior approximation: starting from the first posterior (0) Di) ~ ((01), subsequent approximations can be recursively accessed by multiplying the (t - 1)-th posterior approximation with the t-th dataset likelihood, and renormalizing. More details are in Section A of Appendix.
4 Variational Continual Bayesian Meta-Learning
As the formulated research problem premises few shots per sequentially arriving task, VC-BML sets out learn the meta- parameters O, in an online fashion, from which the task-specific parameters Φ, are derived. Similar to variational continual learning, the posterior distribution of 0, in VC-BML given in recursive way:
p (0, DI:)) (D, pc) Dia-1) (f p (D; (D(: 0)) 0b) p (0, Dia-1).
In the online scenario, we may encounter tasks from non-stationary task distribution, where some tasks are dissimilar or even contradicting each other. This leads problem that disparate tasks require more significant degree of adaptation. Albeit we compute a distribution of the meta- parameters, may not be able sufficiently adapt a diversity of tasks. Consequently it will be inefficient if the task-specific parametens Φ; are adapted from the simple Gaussian distribution of mnta-parameters o,.
4.1 Mixture of Meta-Paramete Distributions
To improve task adaption, we propose to maintain mixture of dynamically updated meta- parameter distributions, each of which is associated with a cluster of similar tasks. A schematic illustration is shown in Figure]] T In particular, Gaussian Mixture Model (GMM). Besides, our task requires that the parameters (i.c., the means and variances) of GMM can be dynamically updated designed to DGMM as opposed to a commonly-used single static Gaussian distribution.
consider the mixture of meta- peeta-arrmeter distributions
tasks emerges. Hence, p(0, | P1:e) is
Let be the latent categorical variable indicating the assignment of task to a cluster, which is equivalent to the selection of component in a mixture distribution. According to the definition of 2, we rewrite the conditional probability:
p(D, 0.,) = p(D, 02, (1)2)))(2))dd = / [ p(D, 6.)((d, 0, )d
p( p(z))dz..
(3)
component distribution 0 is selected from the mixture distribution 0, Upon this, the distribu- tion of task-specific parameters & can be derived. Compared to single distribution, the mixture of meta-parameter distributions able to increase the capacity of meta-learning to deal with more diverse tasks. However, it is non-trivial to determine the number of components in the mixture distribution to deal with the potentially boundlessly diverse tasks in the online setting.
Chinese Restaurant Process (CRP). To adaptively create clusters as new tasks require, we employ
the Chinese Restaurant Process to flexibly determine the number of task clusters (26 27 28 (i.e., the
number of mixture components in DGMM). This allows to create new meta- peeaa-parameter distributions for disparate tasks and recall existing distributions when new task is similar to current task clusters. Specifically, we formulate the task cluster prior CRP such that new cluster is created during

---

NL.
T
(a)
Figure 2: Variational Continual Bayesian Meta-Learning with (a) single distribution of the meta-parameters: (b) a potentially infinite number of meta--parmmeter distributions. trial controlled by the parameter a. Hence the probability of a task belonging the cluster is can be given by:
if the cluster &t exists,
((x =)
(4)
if k is new cluster,
where is the expected number of tasks in the cluster k. and > 0 is concentration parameter that controls the instantiation of a new cluster: larger a will produce more clusters (and fewer tasks per cluster). Since we assume each task arrives at timestamp, - indicates the total number of tasks except the current one. Hereby we use (R)((a) to denote Equation 4 and CRP-DGMM denote DGMM whose number of components is determined by CRP, which can potentially be infinite.
Generative process of CRP-DGMM. Mixture models need stick biickbbre representation of CRP to obtain component weights. With an infinite construction of stick-breaking representation, the probability vector π [Ti T2. ..... where each element is non-negative and the elements sum is defined as:
8x a Beta(L,
for k 1, 1...., 00,
(5)
k-1
T 3x
II (1 Bi)
for 1..... 0c,
(6)
which is equivalent to the weights implied by the CRP(a). Based on this, the generative process of CRP-DGMM is as follows:
2t π Categorical (TT)
0: 0: ~ N(kt 21))
(8)
Φ, 0az =k p(Φ. (i:=))
(9)
D. Φ. ~ p(De Φ.),
(10)
where μu is the mean vector and Σ is the semi-positin covariance matrix that parameterizes the k-th component distribution in DGMM. Figure (a) and (b) show the generative process of VC-BML with single and mixed distributions for meta-parameters respectively. For the sake of computational efficiency, a finite stick-breaking construction of CRP is used. It simply places upper bound on the total number of mixture components K DGMM, which, if chosen reasonably, allows us to enjoy the benefits of CRP at cheaper computational cost. The finite version of the generative process of CRP-DGMM is available in Section B of Appendix.
4.2 Structured Variational Inference for CRP-DGMM
The exact posterior of all latent variables, i.e., p( 02, =t, / D2) is intractable. Due to the large dimension of latent variables, Monte Carlo sampling (e.g., Gibbs sampling) can be computationally slow [29]: the expectation maximization used in [4] only provides point estimations of latent vari- ables. To allow efficient uncertainty quantification, we derive structured variational inference to approximate the exact posterior. At each timestep f, we use only the dataset DD at hand to infer latent variables including 0, and Φ,- We do revisit previous datasets D1:1-1 1 for such inference. The exact posterior can be approximated by using the following structured variational posterior:
K-1
q',, 0r, 3 D.) = q(0, , Dr: (0) DD De::b)g(ee))))
II
((3s 7w),
(11)
5

---

where Yk. 7, 2b, and X are parameters of four variational distributions, each of which is for 3k.- 24.
e and Φ respectively. The Evidence Lower Bound (ELBO) of observations at time can be:
LvC-BML (Ae, v Te 7)
(( (eee) DesAe) logp(D, Φ.)
KL (( , (PA::A))lp(0010013)
KL Pippe)) |Daa-ii]
- -By(mm
KL KL (1ll((2111)
[LL
(12)
where KL[-II- is the Kullback-Leible divergence. The ELBO of our likelihood function is repre- 2),, 77 7), and the variational distributions are optimized by maximizing
sented as this ELBO, We present the updates of variational parameters here; the detailed derivations are in Section C of Appendix. Pseudo codes for VC-BML are in Section D of Appendix.
Variational distribution of [. By only considering the terms related 3g in the ELBO, we analytically derive its optimal variational distribution: g(3x; 7x) = Beta( 7h,1, 7M.2), which is also a Beta distribution with parameters: 7 1 7,,, and 7x.2 + Σ -r= "h.
Variational distribution of 24. Similarly, the variational distribution of variable 2 can be derived as: ((2:: n.)) = Categorical & m.). which is Categorical distribution with parameters
log tr xx (1:r)) T
KL [q(O, Di; bb)ll( Dit--))]
KL [[(0 0 -k Dr; :: D;:Ai)ll(0000
(13)
100
(D
where s.t. Σk= 71, = (. = -I ((3gg 7k) is the variational distribution of B.
Variational distribution of 0,. The meta- peea--arrmeetr distribution is assumed contain mixture of component distributions. We assume the variational distribution of the k-th component to be:
(0: Das tp.) N (m- A,),.
(14)
which is a Gaussian distribution with parameters ", {m, } m is the mean vector and AB is the semi semi-positive covariance matrix of the k-tth component at the timestamp f. Without loss of generality, we assume the prior of 0 is given by the variational posterior of 0 i.e., μt Σ m ELBO with stochastic back propagation (30
= A The variational distribution of 0 can be recursively updated by maximizing the
Variational distribution of i For q( 1 Dr; & denotes the weights of deep neural network and X, denotes the variational parameters (i.e., means and standard deviations). Due to the high dimension of db. computationally difficult learn variational parameters A Hence, resort to amortized variational inference (AVI) [17], where a global initialization 0.* is used to produce X, from DE (De is split to a support set D for adapting tasc-fpec parameters di and query set D for task evaluation) by conducting several steps of gradient descents:
qΦ: 0t D;; ( SGD, , ())
(15)
where J is the gradient descent steps and is the learning rate. The procedure details of stochastic gradient descent, i.e., (0,* DU e), are presented in Section C of Appendix.
Deightiigh KL-divergence. Maximizing the ELBO will recover true posteriors the approx- imating family is sufficiently rich. However, the simple families used in practice typically lead to poor test-set performance. Some "Annealing" techniques, such as the Cyclical Annealing (31) and data augmented down-weighting [32 33), can alleviate this issue by introducing factor on the -ivdiieeeee term. In this paper, following (32 33), we introduce an additional hyperparameter
v(0 < 1) to deight the KL-divergence term of O,.

---

Table 1: Mean meta-test accuracies (%) at each meta meta-training stage.
Algorithms
Omniglot
CIFAR-FS
minilmagenet
VGG-Flowers
TOE
98.43 + 0.48
80.55 1.39
67.55 + 1.15
64.57 + 1.71
TFS
98.22 + 0.50
82.39 + 1.30
71.63 + 1.60
65.50 + 1.73
FTML
99.11 + 0.16
82.33 + 1.31
71.97 + 1.41
65.46 + 1.63
OSML
99.30 + 0.29
83.91 + 1.19
75.13 = 1.36
70.89 + 1.44
DPMM
99.33 0.28
83.52 1.21
73.96 1 1.37
70.09 1.40
OSAKA
99.01 0.29
82.87 1.24
74.34 1 1.48
67.18 1.64
BOMVI
96.25 0.66
83.05 1.25
74.98
1 1.43
76.56 1 1.46
VC-BML
98.62 0.38
86.95 1.15
78.29
1.34
79.37 1.43
4.3
Discussion
There is recent work [4] that uses mixture model in meta-learning. However, our proposed VC-BML differs from this work in three aspects that consequently lead to performance improvement. First, the meta-parameters in our model are represented by mixture of Gaussians compared to delta distributions [4]. Compared to distrbbbttiii,innss Gaussians generally enable a larger capacity in tackling related but moderately different tasks in cluster, and capture some uncertainty that celta-distributions with zero variance cannot do. Second, we derive a structural variational inference method approximate the posterior while Jerfel al. [4] only provide point estimates based maximum a posteriori (MAP). In streaming setting, point estimates on the past cannot imply doing well future data as underlying generation functions can be ambiguous even given some prior information. Third, despite in principle the posterior can be regularized by the prior in [4, it neither explicitly explains how to achieve this goal nor adopts this regularization method in their model (see Algorithm in 4). In contrast, the KL-divergence in the VI framework in our model can be naturally considered as the regularization term. By tuning the weight of this regularization term, can reduce overfitting to the incoming data, leading to improved performance (and we have proved this in the Experiments section).
Experiments
We verify the effectiveness of VC-BML via experimental comparison and analysis. In particular, we answer three research questions: (RQ1) Can VC-BML. outperform baselines on heterogeneous datasets with non-stationns stationary task distributions? (RQ2) How does the number of mixture components K affect results? (RQ3) What the contribution of each component in our VC-BML?
To evaluate VC-BML on non-stationary task distributions, seven state-of-the-art meta-learning and continual learning algorithms are chosen baselines: (1) Train- -n-errtthnnn (TOE), naive baseline that, at each time step I, the meta-parameters are re-initialized and meta-trained on all available data D1:; (2) Saan--Fro---rateee (TFS), another intuitive approach that, at each time step t, the meta-pant pattamerrsssesss are re- re-initialized and meta-trained on the current data D,; (3) FTML (3) which utilizes the Follow the Leader algorithm 34 to minimize the regret of meta-learner: (4) DPMM 4. which uses Dirichlet process mixture to model the latent task distribution; (5) OSML ( which constructs meta-knowledge pathways between knowledge blocks to quickly adapt new tasks; (6) OSAKA 23), which aims at online adaption to new tasks and fast remembering; (7) BOMVI [ which uses a Bayesian approach to meta- and continual learning with variational inference.
Similar to previous works 4 25 we conduct experiments on four datasets: Omniglot [35]. CIFAR- FS 13 minilmagenet
37 and VGG-Flowers (38) Different datasets correspond to disparate
tasks. and hence tasks chronologically sampled from these four datasets follow a non-stationary distribution. Consistently, we consider the conventional 5-way 5-shot image classification task, treat 5 classes randomly sampled from a dataset as task and generate a sequence of tasks by sampling the classes. We sequentially meta-train models on tasks sampled from the meta-training sets of these four datasets. That is, we firstly meta-train models on tasks sampled from the meta-training set of Omniglot, then proceed the next dataset. Performance evaluated on meta-test set. We tune the hyperparameters based on the validation sets. More details can be found in Section E of Appendix.
7

---

Omniglot
CIFAR-FS
minilmagenet
VGG-Flowers
Omnigla
CIFAR-FS
70
Meta test Accuracy
06-
VGG Flowors minilmagrnet
VE-RMI
64-
TDE
TPS
FTML
0SML
DYMM
-3 30-
O6AKA
Meta training Iteration
Figure 3: The evolution of meta-test accuracies (%) when algorithms are sequentially meta-trained on different datasets. Each row represents the meta-test accuracies for dataset over cumulative number of training iterations.
5.1 RQI: Performance on Non-Stationary Task Distribution
We report the mean meta-test accuracies previously Icarned datasets (i.e., D1.)) at each meta- training stage in Table I VC-BML achieves the best performance when sequentially meta-trained on different datasets CIFAR-FS, minilmagenet and VGG-Flowery which indicates that VC-BML can better handle tasks from non-stationary distribution. We suspect the reason why VC-BML does not obtain the best performance Omniglot is the over regularization caused by the KL-divergence in the ELBO. We verify this hypothesis in Section F.4 of Appendix. To obtain a more vivid presentation of knowledge transfer, we visualize the evolution of meta-test accuracies all timestamps in Figure 3 Although FTML, OSML, DPMM and OSAKA achieve good performance on tasks from stationary distribution (i.e., the same dataset), their performance degrades when the task distribution shifts to new datasets. In contrast, the meta-test accuracies of VC-BMI. on different datasets at different meta-training stages are stable and consistent. This shows VC-BML alleviates the problem of negative knowledge transfer.
To obtain deeper insights of catastrophic forgetting, we report in Table 2 the meta-test accuracies each dataset at the beginning (Omniglot) and ending (VGG-Flowers) of meta-training stage. The full results at each meta-training stage be found in Section F.1 of Appendix. It's shown that parametric algorithms (FTML, OSML. DPMM and OSAKA) suffer from a severer forgetting problem than Bayesian ones (BOMVI and VC-BML). For example, on Omniglot dataset, the accuracies of parametric algorithms decrease from 99.01 %-99.33%% to 87.5 37.84% -9.. 14%, while the accuracies of Bayesian algorithms barely change (from 96 06.25%%-8..6 62. to 96.60%-97.77 97.47%) Similar results be found on CIFAR-FS and minilmagenet datasets (see Section F.1 of Appendix for details). This can be explained that the parametric algorithms allow dramatically change parameters to quickly adapt to new tasks and thus forget previous knowledge, while the KL term in Bayesian algorithms can act as regularizer prevent such forgetting. Moreover, when only meta-trained on Omniglot, despite the performance of Bayesian algorithms on Omniglot is not the best, they always outperforms parametric counterparts on unseen datasets (i.e., CIFAR-FS. minilmagenet and VGG-Flowerss This is because parametric algorithms are tailored for specific task distributions while Bayesian algorithms allow for stronger generalization on tasks from unknown distributions. Therefore, Bayesian algorithms are reckoned to be more suitable for the online scenario with non-stationary task distributions. The reason why our VC-BMI. outperforms BOMVI is that BOMVI suffers from negative knowledge transfer when handling different task distributions, as it only maintains single veta-parameter distribution.
To show the findings are irrelevant the veta-training orders, we conduct further experiments by meta-training algorithms with different dataset orders. Results are shown in Section F.2 of Appendix.
5.2
RQ2: Number of Mixture Components
For the sake of computational efficiency, we use finite stick-breaking construction of CRP, where we place an upper bound on the number of mixture components, i.e., K. To study the impact of K, we conduct experiments by varying K and report their mean meta-test accuracies previously learned
8

---

Table 2: Meta-tes test accuracies (%) on each dataset at the beginning (Omniglot) and ending VVG--llower of meta-training stages. "Average" represents the meta-test accuracies on four datasets. (The best performance per dataset per meta-training stage is marked with boldface and the second best is marked with underline.)
Meta-training Stage
Algorithms
Omniglot
CIFAR-FS
miniImagenet
VGG-Flowers
Average
TOE
98.43 + 0.48
42,83 + 1.77
33.09 = 1.44
64.19 + 2.11
59.64 + 1.45
1:
191
33.49 1
2.05
59.56
1.46
FTML.
99.11 1: 0.16
42.48
1: 1.88
34,69
1.22
57.32 4: 2.06
58.40
1: 1.33
Omniglot
OSML
09,30
41.20
1.82
32.52
1.33
55.39 + 2.19
57.10
1.1
DPMM
99.33 1. 0.28
42.31
1: 1.62
34,40
1.61
57.58 12.16
58.45 1: 1.42
OSAKA
99.01 +: 0.29
43.24 + 1.79
35.44
1.35
60.13 2 2.08
59.46
+ 1.38
BOMVI
96.25 + 0.66
22
1.94
37.83
1.45
66.24
63.06
1.51
VC-RMI.
98.62 +: 0.38
50.86 +: 1,94
38.11 1.51
68.63 + 2,08
64.06 :: 1.48
TOE
93.12 1 1.18
50.07
2.20
41.97 1.74
73.13 1.71
64.57 1: 1.71
TFS
87.84 + 1.89
51.99 +: 1.89
41.53 = 1.74
80.62
1.41
65.50 + 1.73
FTML
87.01 + 1.35
52.29 1,90
41.43 1.7
81.10
1.58
65 45.46-2
1.63
VGG-Flowers
OSML
93.14 + 0.90
59.74 + 1.85
48.44 1.60
82.25 + 1.43
70.89 + 1.44
DPMM
92.71 + 0.76
58.76 - 58.76
46.95 1.54
81,92 - 1.48
70.09 - 1.40
OSAKA
90.05 + 1.13
53.37 2.01
44,93 1.68
80.37 1.73
67.18 - 1.64
BOMVI
96.60
+ 0.55
70.16 4: 1.84
$7.94 1.65
81.54 1.81
76.56 + 1.46
VC-BML
97.47 0.73
75.20 + 1.80
61.43
1.49
83.39
1.73
79.37
1..4
Table 3: Mean mesa accuracies (%) for VC-BML with different K at each meta mataa-taiinnng stage.
Omniglot
CIFAR-FS
minilmagenet
VGG-Flowers
K=1
98.32 + 0.57
81.04 = 1.48
71.76 1.67
73.09 1.54
K=2
98.59 = 0.49
85,28 = 1.15
76.48 + 1.28
78.23 + 1.42
K=4
98.35 + 0.52
85.91 = 1.15
78.07 + 1.40
79.08 + 1.52
K=6
98.62 = 0.38
86.24 = 1.09
78.29 + 1.01
79.37 = 1.44
datasets, i.e., D1.,, different meta-training stages. The results in Table show that VC-BML has the worst performance when K = 1, which suggests that maintaining a single meta- veeta-parameter distribution insufficient to diverse tasks. The performance of VC-BML improves as K gets larger, which indicates the benefit of instantiating more meta meta--arameter distributions. Note that the improvement of VC-BMI. insignificant when K (the actual number of task distributions). More results are available in Section F.3 of Appendix.
To verify that VC-BML is able to identify different task distributions, we visualize the posterior of (the latent task cluster assignment) after meta-training VC-BML with K = 6 on the four datasets. Specifically, we randomly sample 100 tasks from the meta-test set of each dataset and calculate the posteriors, which is presented in Figure 4 The results show that VC-BML can recognize different task distributions with separate mixture components. For example, component corresponds to Omniglot while the component 4 corresponds to VGG-Flowers. This can help explain the superiority of VC-BML when modeling stationot task distributions, as it can store knowledge about different task distributions in different mixture components, and recall these knowledge when similar tasks emerge. Moreover, the low posteriors of component and suggest that VC-BML only instantiates new meta- petta-prrmeter distributions when disparate tasks are discovered. After training DPMM on the four experimental datasets, there are actually task clusters, which is the same as the effective number of clusters in our model (although we set the upper bound of cluster number to be 6).
5.3 RQ3: Ablation Study
We conduct ablation experiments to analyze the reasons for the performance improvement, which have been discussed in Section 4.3 To examine the effect of each component, we introduce three variants of our VC-BML: -C-BMa where the variance of GMM is fixed to be low value (10-5) and hence GMM is converted to a mixture of delta distribution: VC-BML-MAP, where the structured VI for inference is replaced with MAP; and VC-BML-NOR, where the weight of KL-divergence not tuned and set to be 1. The experimental results are shown in Table 4 where the evaluation results are obtained in the same way as those in Table I over a mixture of delta distributions by comparing VC-BML-Delta and VC-BML. We find that the performance of DPMM and VC-BML-Delta is quite similar, but lower than that of VC-BML. on three datasets (CIFAR-FS, minilmageeet and VGG-Flowers). This comparison quantifies the contribution of GMM over a mixture of delta distributions. We then examine the advantage of structured VI
We first examine the contribution of GMM

---

Omniglot
CIFAR-FS
minilmagenet
VGG-Flowers
Posterior
0.5
+
0.4
0.3
0.2
0.1
3
2
5
Index of Mixture Components
Figure 4: Each panel (column) represents the posterior of different mixture components dataset. A higher posterior represents higher probability of assigning a task to specific task cluster.
Table 4: Model ablation study on all evaluation datasets.
Model
Omniglot
CIFAR-FS
minilmagenet
VGG-Flowers
DPMM
99.33 + 0.28
83.52 + 1.21
73.96 + 1.37
70.09 + 1.40
VC- -C-BML-Delta
99.48 + 0.16
84.43 + 1.05
75.45 = 1.33
71.27 + 1.52
VC-BML-MAP
99.50 + 0.16
84.98 #1.11
77.46 + 1.38
76.27 #1.51
VC-BML-NNOR 94.81 +0.51
80.76 0.99
70.38 + 1.46
66.88 + 1.32
VC-BMI.
98.62 0.38
86.95
1.15
78.29 + 1.34
79.37 + 1.43
over MAP by comparing VC-BML-MAP and VC-BML. From the table, we find that VC-BML outperforms VC-BML-MAP on three out of four datasets. The performance difference shows the quantified contribution of VI in our model. In addition, our model worse than VC-BML-Delta and VC-BML-MAP on the Omniglot dataset. This is, perhaps, due the influence of KL-divergence and its weight in the VC-BML. These comparisons motivate us to properly weigh the KL--iiergence in the VC-BMI. model, which consequently produced better performance on Omniglot as we showed in Section F.4 of the Appendix. Besides, in Table 2 of the Appendix, we show the optimal weights of KLLdivergence that leads to the best performance of weighting KL-divergence, we compare the performance of VC-BML-NOB and VC-BML. We notice that VC-BML-NOR performs much worse than VC-BML. Thus, if the KL-divergence is not properly scaled, the model is actually learning nothing but forcing the posterior to collapse to the prior, leading to the severe mode collapse problem.
all datasets. To further examine the effects
6 Limitations
This paper maintains a mixture of Gaussian distributions for peea-ppraaetees to deal with diverse tasks in streaming scenario. One limitation comes from the added space complexity. As conventional algorithms maintain one replica of meta satttaaarreeersss (either themselves or their distribution parameters such means and variances) for all tasks, we need to allocate more space to store K meta- parameter replicas for those disparate tasks.
7 Conclusion
This paper deals with low-resoures streaming tasks that follow non-stationary distributions. To avoid knowledge forgetting and negative transfer, we propose VC-BML, a Variational Continual Bayesian Meta-Learning algorithm, where the meta-parameters follow mixture of dynamic Gaussians op- posed to the commonly-used single static Gaussian. In particular, we employ the Chinese Restaurant Process adaptively determine the number of mixture components. To approximate the intractable posterior of interest, we develop a structural variational inference method. Extensive experiments show that our algorithm outperforms state-of-the-art baselines when adapting to diverse tasks and alleviating knowledge forgetting in an online setting with non-stationary task distributions.
As VC-BML can be widely applied to various tasks, it could exert potential negative social impacts. Although, to the best of our knowledge, we do not see a direct path to negative applications, employed for malicious purposes, it can facilitate the development of socially- ttreaatening models.
10

---

Acknowledgments
This research was partially supported by the National Natural Science Foundation of China (Grant No. 61906219) and the EPSRC Fellowship entitled "Task Based Information Retrieval," grant reference number EP/P024289/1. We acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU.
References
[1] Jonathan Baxter. A model of inductive bias learning. Journal of artificial intelligence research, 12:149-19K
2000.
[2] Bernhard Pfahringer, Hilan Bensusan, and Christophe G Giraud-Carrier. Meta-learning by landmarking various learning algorithms. In International Conference Machine Learning, pages 743-75 443-750, 2000.
[3] Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In Interna- tional Conference Machine Learning, pages 1920-1930 1930, 2019.
[4] Ghassen Jerfel, Erin Grant, Tom Griffiths, and Katherine A Heller. Reconciling meta-learning and continual learning with online mixtures of tasks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alché-Buc E. Fox, and R. Garnett, Aditors,Avaanee Advances in Neural Information Processing Systems, volume 32, 2019. meta-
[5] Giulia Denevi, Dimitris Stamos, Carlo Ciliberto, and Massimiliano Pontil.
learning. Advances Neural Information Processing Systems, volume 32, pages 1-11, 2019.
[6] Pau Ching Yap. Hippolyt Ritter, and David Barber. Bayesian online meta-learning. arXiv preprint arXiv:2005.001466 2021.
[7] Sang- Woo Lee, Jin Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung Tak Zhang. Overcoming catas- trophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems, volume 30, pages 4652-4662, 2017.
Sang-Woo
Huaxiu Yao, Yingbo Zhou, Mehrdad Mahdavi, Zhenhui Li, Richard Socher, and Caiming Xiong. Online structured meta-learming In International Conference on Learning Reprexentations, 2020.
Zhenxun Zhuang, Yunlong Wang, Kezi Yu, and Songtao La. No-regret non-convex online meta-lcaming. In IEFE International Conference on Acoustics, Speech and Signal Processing, pages 3942-3946, 2020.
[10] Guneet S Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. A baseline for few-shot image classification. arXiv preprint arxiv: (e0::1999.02220 990900229, 2019.
[11] Anusha Nagabandi,, Ignasi Clavera, Simin Liu, Ronald Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta-reinforeemen learning. In International Conference on Learning Representations, 2019.
[12] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with aeeeory-augmeetee neural networks. In International conference on machine learning. pages 1842-1850. 2016.
[13] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pages
2016.
[14] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1 1126-111 126-1133 135, 2017.
[15] Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner. Meta-learning probabilistic inference for prediction. arXiv preprini (005::::000009920 2018
[16] Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-bart -base meta- meea-leerning as hicrarchical bayes. In International Conference on Learning Representations, 2018.
[17] Sachin Ravi and Alex Beatson. Amortized bayesian meta-learning. In International Conference on Learning Representations, 2018
[18] Jaesik Yoon, Taesup Kim, Ousmane Dia. Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta- learning In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 7353,, 2018.

---

[19] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Gtabska-Barwinge et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. 114(13)3351-33526 2017.
[20] Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E. Turner. Variational continual learning. In International Conference on Learning Representations, 2017.
[21] Hippolyt Ritter, Alcksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, 2018.
[22] Gido van de Ven, Hava Siegelmann, and Andreas Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature communications, 11(1):1-14, 2020.
[23] He, Jakub Sygnowski, Alexandre Galashov, Andrei A Rusu, Yee Whye Teh, and Razvan Pascanu. Task agnostic continual learning via meta learning. In Lifelong Learning Workshop at ICML, 2019.
[24] Khurram Javed and Martha White. Meta-learning representations for continual learning. arXiv preprint arXiv: 12588, 2019.
[25] Massimo Caccia, Pau Rodriguez, Oleksiy Ostapenko. Fabrice Normandin, Min Lin, Lucas Page-Caccial Issam Hadj Laradji, Irina Rish, Alexandre Lacoste, David Vázquez, et al. Online fast adaptation and knowledge accumulation (osaka): new approach to continual learning. Advances in Neural Information Processing Systems, 33, 2020.
[26] Shangsong Liang, Emine Yilmaz, and Evangelos Kanoulas. Collaboratively tracking interests for user clustering in streams of short texts, IEEE Transactions on Knowledge and Data Engineering, 31(2)2257- 272, 2018.
[27] Shangsong Liang. Zhaochun Ren, Emine Yilmaz, and Evangelos Kanoulas. Collaborative user clustering for short text streams. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.
[28] Shangsong Liang, Emine Yilmaz, and Evangelos Kanoulas. Dynamic clustering streaming short documents. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 995 1004, 2016.
[29] Carl Edward Rasmussen et al. The infinite gaussian mixture model. In Advances in neural information processing systems, volume 12, pages 554-560, 1999.
[30] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014.
[31] Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cyclical annealing schedule: A simple approach to mitigating kl vanishing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 240-250, 2019.
[32] Kazuki Osawa, Siddharth Swaroop. Anirudh Jain, Runa Eschenhagen, Richard Turner, Rio Yokota, and Mohammad Emtiyaz Khan. Practical deep learning with bayesian principles. In Advances in Neural Information Processing Systems, 2019.
[33] Florian Wenzel, Kevin Roth, Bastiaan S Veeling. Jakub Swiatkowski. Linh Tran, Stephan Mandt, Jasper Snoek. Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really? In International Conference on Machine Learning, 2020.
[34] Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences, 71(3):29 1-307, 2005.
[35] Brenden Lake, Ruslan Salakhutdinev, Jason Gross, and Joshua Tenenhaum. One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science sociery, volume 33, 2011. 1.
[36] Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-leaming with differentiable closed-fone form solvers. In International Conference on Learning Representations, 2018.
[37] Sachin Ravi and Hugo Larochelle. Optimization Conference on Learning Representations. 2016.
model for few-shot learning. In International
[38] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over large number of classes. In Indian Conference on Computer Vision, Graphics & Image Processing, pages 722-729, 2008.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu-
tions and scope? [Yes] See Abstract and Section
(b) Did you describe the limitations your work? [Yes] Section 8.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] The main theoretical results the variational posteriors of interest, and we state assumptions when deriving these
results in Appendix.
(b) Did you include complete proofs of theoretical results? [Yes] Detailed derivations of varia- tional posterior distributions are provided in Appendix.
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as URL)? [Yes] Codes and datasct to reproduce the experiments
included in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters. how they were chosen)? [Yes] Experimental training details provided in Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] We report performance variance after running experiments for multiple times with different seeds in Section
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] The amount and the type computing resource used in our experiments are descrihed in Appendix.
4. If you using existing assets (e.g., code, data, models) curating/relcasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Citations for the existing assets (code, data and baseline models) are provided in Appendix.
(b) Did you mention the license of the assets? [Yes] We mentioned relevant licenses when we introduce respective codes and data.
(c) Did you include any new assets either in the supplemental material or as URL? [No] We did not include assets in this work.
(d) Did you discuss whether and how consent was obtained from people whose data you're 're us- ing/curating? [No] These data publicly available and have heen frequently used for research purposes.
(e) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [Yes] Detailed description for data our experiments are provided in Appendix.
5. If you used crowdsoureing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "xwbT8YId_h_", "reviewer": "Reviewer_U7b8", "summary": "This paper attempts to solve the problem of meta-learning when data arrive sequentially. Unlike previous approaches, the authors propose a method to maintain a dynamic Gaussian mixture model over the meta-parameters. The number of clusters is picked using a Chinese restaurant process. Various experiments have been conducted to show the accuracy and superiority of the model.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper attempts to solve the problem of meta-learning when data arrive sequentially. Unlike previous approaches, the authors propose a method to maintain a dynamic Gaussian mixture model over the meta-parameters. The number of clusters is picked using a Chinese restaurant process. Various experiments have been conducted to show the accuracy and superiority of the model.\n\nIt is great to see a truly Bayesian treatment for meta-learning. Maintaining a multimodal latent distribution for meta-parameters is a neat idea. I did not check all the derivations but the paper follows the standard structured variational inference derivations and looks legit. Though using Chinese restaurant process for selecting the clusters is ideal, I am wondering if it is overkill. Possibly, we could have made the number of clusters a parameter of the mixture and learn it through variational inference itself. In that case, the mixing coefficients could follow a Dirichlet distribution the parameters of each Gaussian would be Gaussian-Wishart. \n\nI would like the authors to discuss more about the practical aspects of the algorithm. What is the dimensionality of latent spaces?, how do we avoid mode collapse?, what is the computational time?, etc. If our initial distribution is not broad and diverse enough, I am skeptical if the posterior distribution can indeed pick the necessary modes of the new distribution of incoming data. As another limitation, if the parameters of the new underlying dataset are truly multimodal [1], the variational approximation would not be able to capture the variational distribution as it would pick only one mode by construction. \n\nMinor: \n* Line 58 - “It’s” - avoid contractions\n* Line 101 - Use parentheses as {(x,y)} to show that what repeats is the input-output pair.\n\n[1] Itkina et al., \"Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders,\" NeurIPS'20\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Adequate ", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "FfIGXMQf_b_", "reviewer": "Reviewer_AgJH", "summary": "This article presents a fully variational method to address the continual learning problem: training a neural network sequentially on different tasks, without forgetting preceding tasks.\n\nThe theoretical framework, namely variational inference with hierarchical random variables, is classical, which makes the general idea easy to understand.\n\nThe experiments are sound, concurrent methods have been replicated and tested (including naive baselines), and the main potential flaws or limitations have been discussed and addressed.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Detailed summary\n\nThe method is based on a hierarchical structure of random variables managing the final weights of the neural network to train. In order to deal with an unknown number of tasks, the weights $\\phi$ of the NN depend on a categorical random variable $z$, whose range of values increases during training. In a nutshell, the range of $z$ is supposed to increase when the number and the variety of tasks increases.\n\nThe main theoretical difficulty is deriving the update rules for all variational parameters.\n\n## Experiments\n\nSetup: train sequentially the model on 4 given tasks (image classification), in a given order.\n\nThe authors compare their method to 7 other ones, including 2 naive baselines, which is sufficient to evaluate the method. They compare the results on each (seen) dataset during the sequential training on each task. The difficulty consists in keeping a good performance on a dataset when the model is being trained on another dataset. The proposed methods dominates all concurrent methods, except when learning the first task (first phase in Fig. 3).\n\nAppendix F.2 is very important to check the fairness of the choice of the order of tasks in Section 5: the authors rerun the same experiments with the same tasks, *in inverse order*. The proposed method dominates all concurrent methods for each task, at any moment of the training. So we can reasonably be confident in the results presented in Fig. 3.\n\nThere is no comparison of training time between methods.\n\nSection 5.2 confirms the intuition behind the range of the variable $z$: $z$ takes as many values as the number of tasks to perform. However, to be rigorous, Figure 4 should be replicated with 3 and 2 tasks, in order to check that the number of values $z$ can take are respectively 3 and 2.\n\n## Questions\n\nWhy are results with OSAKA on Omniglot better in this paper than in the original one?\n\nIs data augmentation used? It may be the cause of the impact of the KL mentioned in Section 5.3.\n\nWhat about RNNs? For instance, text prediction (which is a simple task).\n\n## Clarity\n\nThere is no issue of clarity if the reader is already familiar with hierarchical variational inference. Otherwise, the article may be hard to understand. For instance, the name of Section 4.2 could be changed to mention the update of variational parameters, which is the object of the section.\n\n## Significance\n\nThe proposed method is an expected improvement in a field of medium importance (continual learning).\n\n## Rebuttal\n\nI am satisfied with the authors' answers. I maintain my rating, with a higher confidence.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As mentioned in the main review, potential limitations have been addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "EgPycxgp9Jf", "reviewer": "Reviewer_h2Cd", "summary": "**Update after author response:** I have raised my score from 4 to 6, even leaning towards 7 (but I would need to see the revised manuscript to confidently give a 7, which is unfortunately not possible, because a fair amount of my criticism is aimed at various passages throughout the paper and the overall claims and style of presenting them).\n\nThe paper addresses task agnostic continual (meta-) learning from a Bayesian perspective. The setting in continual learning is to learn a sequence of (potentially quite different) tasks, image classification tasks in the paper - the main difficulty is to learn novel tasks while avoiding catastrophic forgetting of previously seen tasks and negative interference (i.e. decreased learning or suboptimal solutions) from previously learned solutions. As discussed in the existing literature, a principled approach to do this is to perform Bayesian inference over the parameter-posterior given the current experience. This posterior can act as a prior for new experience - the standard sequential Bayesian posterior update. In practice, this update is intractable and the paper resorts to approximate inference: in essence a pior distribution is learned per cluster of related tasks. These task-group priors allow to rapidly adapt to each task within the cluster (which is performed in the paper in a MAML like fashion, i.e. via gradient update steps starting from an initial parameter-draw from the task-group pior). The main idea is thus to use a mixture distribution (mixture over Gaussians in the paper) to represent a multi-modal approximate posterior over task-parameters, which acts as a “good” prior for new observations, and thus allows for knowledge transfer between tasks of the same cluster. Since the number of clusters is not known in advance, and, more importantly, needs to grow with an increasing number of tasks observed the model employs a non-parametric posterior approximation (a Dirichlet process that models the number of Gaussian mixture components). The method is compared, and performs well, against a number of previously proposed methods.\n\n**Contributions:**\n\n1) Extension of the model used by a previous method [4] by using a mixture of Gaussians for the approximate posterior - each mixture component is supposed to model one cluster of tasks (in the previous paper each mixture component is a delta-distribution, leading to a Dirichlet Process Mixture Model DPMM). The number of mixture components is inferred from the data via a Dirichlet process (leading to a Dirichlet Process Gaussian Mixture Model DPGMM) - in contrast to the paper’s claims this has been done in precisely the same fashion in [4] (see Section 5 in that paper, which also uses the CRP construction). Significance: the improvement in the model is sensible but fairly incremental - it would have been interesting to allow for multi-modality on this level as well. Results are in favor of the improvement, but do not convincingly demonstrate that the observed improvements are solely caused by this modification (see comments below).\n\n2) In contrast to [4] (which derives a MAP inference scheme), a variational inference scheme is derived. The innermost step of that scheme (MAML-type optimization using a few gradient-descent steps) relies on meta-learned amortized variational inference, variational inference of \\theta relies on ELBO-type optimization via SGD and the reparametrization trick. Significance: the VI scheme is interesting and shown to scale to neural networks. Given the model complexity this is not trivial - however this complexity introduces many hyper-parameters and application of the model to new data might not be straightforward (the VI can easily break in subtle ways that are hard to diagnose and debug). Since the paper does not report in detail how most hyperparameters were found, and how sensitive the results are w.r.t. their choice becomes even more difficult.\n\n3) Experiments on a sequence of four image classification tasks with comparison against two baselines and several related models from the literature. Results suggest that the model outperforms all other models in most cases. Significance: results look indeed promising, but at the current stage it is hard to attribute the observed gains in performance to the various improvements made. Important control experiments and ablations are missing to make sure the observed gains are actually due to the hypothetical claims that went into the model and inference improvements (more on this, see Improvements below).", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality, Quality, Clarity, Correctness:**\n\nWhile the use of a Dirichlet-process-Gaussian-mixture-model is certainly not novel, and fairly straightforward from a Bayesian viewpoint on continual learning, I have not seen this particular instantiation of the model in the continual learning literature before (though I might have some gaps, hence my lower confidence score). Getting VI to work reliably for such a model (with high-dimensional neural network weight-matrices as the parameters at the bottom of the hierarchical model) is no easy feat, and results in the paper seem to suggest that this works successfully. Unfortunately some of the claims in the paper are overstated and/or not sufficiently backed up by experimental evaluation - perhaps most importantly the differences to the model in [4] are more incremental than the paper claims (see criticism below for details). The clarity of the writing could be improved (see comments), and while there is no part that is clearly wrong in the paper the correctness of the interpretation of the results is unclear since important ablations and control experiments are missing (to show that the observed performance gains are due to the reasons claimed).\n\n**Verdict:**\n\nI think the main ingredients for a good paper are there - the central idea is a small, but sensible improvement in the model and the VI scheme looks interesting. Results shown in the paper are promising, but now extra work needs to be put in to make sure that the observed results come from the main innovation and to disentangle which innovation contributes to the performance gains to which degree. If done thoroughly, I think the paper has the potential for a fairly significant impact in the continual learning community. At its current stage I personally think that the paper does not have sufficient rigor and experimental evidence to back up all the claims made. I therefore suggest a major revision, and want to strongly encourage the authors to “go the extra mile” and turn this into the strongest paper possible, rather than rushing with publishing preliminary findings. I am happy to reconsider my verdict of course in light of the other reviews and the authors’ response.\n\nAfter re-reading my review I want to point out that I’m not an author of [4] (or collaborator of the authors) - my aim is not to “defend” [4], but the differences need to made clear (both formally and experimentally).\n\n**Pros:**\n\n * Clearly articulated, focussed problem setting. Main model improvement is sensible.\n * Derivation of a fairly involved VI scheme.\n * Promising results\n\n**Cons:**\n\n * Overstated (or wrong) claims w.r.t. previous work - most importantly [4] in the paper uses a Dirichlet process to automatically infer (and increase) the number of mixture components.\n * Unclear whether the improved results (w.r.t. [4]) come from:\n   * the Gaussian mixture model\n   * the VI scheme as opposed to MAP inference in 4\n   * the annealing of the regularizer\n   * potentially more/better hyper-parameter tuning for the main method but not for the competitors\n * Some parts of the writing could be more rigorous (being precise about what exactly is more or less ‘Bayesian’, clearly describing what meta-parameters refers to, the params of the whole DPGMM?, the params of the Gaussian mixture?, the distr. over params induced by a single mixture component?, …)\n\n**Criticism w.r.t. specific claims**\n\n*Claimed advantages of the method*\n\n * L 221:  “the meta-parameters in VC-BML are represented by a dynamically updated mixture model compared to a single static distribution in [4].” - this claim is wrong, Section 5 in [4] uses the same CRP construction to dynamically adjust the number of mixture components.\n\n * L 229: “ Third, the KL-divergence in our objective function acts as a regularization term, the weight of which can be tuned to reduce over-fitting” - this is also true in principle for MAP estimates (which implies computing a posterior) as in [4], the posterior can be regularized via the prior.\n\n*Other claims*\n\n * L41: “they use point estimation to infer parameters”  - be more precise, which parameters?\n\n * Line 87 to 91: “propose to cluster task-specific parameter distributions and allow the meta-learner to select over a mixture of these distributions. Nonetheless, the meta-parameters still follow a single stationary distribution and the latent variables are inferred by point estimation, which is found prone to suffer from the catastrophic forgetting issue” - what do you mean with “follow a single stationary distribution”? As far as I can tell this refers to the mixture governed by the Dirichlet process, which is not stationary. If used like this, the word “single” also applies to the current paper (a single DPGMM distribution).\n\n * L94-95: “ We overcome these weaknesses by representing meta-parameters with a dynamically updated mixture model and inferring latent variables via structural variational inference.” Again, the first half or this sentence is what [4] also does, and it is unclear (from the arguments in the text) why “inferring latent variables via structural VI” (as opposed to MAP estimation?) helps overcome the weaknesses. Be more precise here.\n\n * Sec. 4 title - not sure what’s particularly Bayesian or non-Bayesian about the proposed model. Both, equation 1 and 2 approximate the Bayesian posterior over parameters \\theta. 2 uses a hierarchical model to do so (but the parameters \\phi could in principle be absorbed into a more complex parameter \\theta). There isn’t really anything more or less Bayesian about it - it’s just in one notation (Eq. 2) the hierarchical model is made explicit, whereas in Eq. 1 it could either be a hierarchical model or a flat model (the latter is of course more commonly seen in the literature). I agree that the model proposed is able to capture some uncertainty that cannot be captured by e.g. [4]. I personally would find it more fitting to call the model DPGMM, Dirichlet prior **Gaussian** mixture model.\n\n * Line 139-140: “Consequently it will be inefficient if the task-specific parameters \\phi_t are adapted from the same distribution of meta-parameters \\theta_t.” This is strictly speaking not true (unless the notation is interpreted in a particular way) - the distribution of meta-parameters does not necessarily mean that there is only a single mode for \\theta_t - the notation parameterizing the meta-distribution with \\theta_t includes the case where \\theta_t is a Gaussian mixture model or some other potentially multimodal distribution.\n\n * Line 144: “As meta-parameters follow Gaussian distributions” - why is this the case? Is this the modeling assumption? In general the meta-distribution should be multi-modal (which is exactly what is achieved with the mixture model), a Gaussian is clearly a wrong / overly simplifying modeling choice.\n\n * Line 228: “For most dissimilar tasks, point estimates can be dramatically changed, which is prone to forgetting previous knowledge”. This is to some degree also true for unimodal (Gaussian) distributions used in the paper (the only way they can cope with dissimilar tasks is by increasing their variance dramatically). The main mechanism to overcome this here (and in [4]) is via the mixture model.\n\n\n**Improvements:**\n\n1) Be more precise and very clear about differences and claimed novel advantages (see criticism above).\n\n2) The question is where exactly the observed performance differences, particularly w.r.t. DPMM [4] come from. The following ablations are needed to determine this (the emphasis on comparing against [4] is because it is a very similar model with a different inference procedure)\n\n2a) The proposed model reduces to DPMM if the variance of the Gaussians is manually fixed to very low values. Keeping everything else fixed, does the model perform better/worse or the same compared to DPMM? (this highlights the performance difference due to the different VI scheme).\n\n2b) Another source of the improvements could be the annealing of one of the regularizer terms. Show a control experiment without annealing, and ideally attempt a similar annealing for the competitor methods (I am aware the latter can be tricky). It’s important to know how much of the performance gain is due to annealing.\n\n2c) It is currently unclear how hyper-parameters for the method were found (including the parameter for the priors which act as regularizers). The performance gains might come from the fact more time went (perhaps accidentally while developing the method) into hyper-parameter tuning. Ideally show a clear process for hyper-parameter tuning and apply the same process to the competitor methods to make results more comparable.\n\n3) Ablation: show how sensitive the results are to the various hyper-parameters. Do they need to be selected very carefully, or is the VI scheme quite robust\n\n4) What was the (effective) number of clusters used by DPMM - is it comparable to DPGMM, and was there any attempt to tune the concentration hyper-parameter of DPMM?\n\n5) Just to be sure (and since the writing might suggest otherwise) - the DPMM implementation that is being compared against is using the Dirichlet process (as in Section 5 of [4]) and not the static model (as in the first part of [4])? How was the DPMM implementation obtained?\n\n7) Meta-comment: the benefits of the method (compared to point-wise meta-parameters as in [4]) should really become apparent when multiple different, but related tasks belong to the same cluster. This might or might not happen implicitly in the experiments shown (implicit subtasks within each dataset might exist). It would be interesting to see this clearly in a controlled experiment with synthetic data.\n\n\n\n\n\n\n**Comments:**\n\n * Line 206: “Variational inference of \\phi” - a point-wise global initialization is used in conjunction with (a few steps of) gradient descent. This sounds very similar to what [4] does via MAML. So what’s the difference between this and [4]? The appendix points out this similarity to MAML, what’s important to distinguish from [4] is whether the sample is drawn from the Gaussian or simply its mode/mean is selected for the global initialization (which would correspond exactly to [4]). The pseudocode in the appendix confirms that the value is sampled. Since this is one of the main differences this needs to be pointed out more clearly in the main text.\n * Compare (discuss, not necessarily via experiments) against “Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning” (Kessler, UAI 2021). What are advantages and disadvantages of the construction proposed in this paper? (I am not an author btw.)\n * Define in the intro: what is a ‘low resource’ task?\n * Error-bars in Table 2 often overlap between the first and second best solution. Please point this out in the text.\n * Another pass to check grammar and spelling\n * Line 185: “ Specifically, we use handy Dt to recursively infer latent variables” I don’t understand this sentence.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Some limitations of the method are discussed. What is missing is how to set the hyperparameters (annealing schedule, prior-parameters, etc.) in practice and how brittle the VI scheme is. Additionally current experiments show results for a small number of tasks, I’d expect difficulties with inference for larger numbers of tasks, but it’s unclear whether that problem becomes apparent for slightly larger numbers of tasks or whether the method is reliable for dozens of tasks.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4.5"}, "extra_text": {}}, {"review_id": "C80v8t5q6aX", "reviewer": "Reviewer_THpm", "summary": "The authors tackle the problem of continual meta-learning, where the tasks are streamed. This settings comprises the additional challenge of catastrophic forgetting.\nThey introduce a training method VC-BML, which updates a GMM over the meta-parameters dynamically. The idea is to cluster the posterior meta parameters over the modes of the GMM. Additionally, they leverage structured variational inference in order to approximate the posteriors.\nThe experiments show an improvement in terms of mean meta test accuracy over the stream of tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Originality\n1- **Are the tasks or methods new?**\n\nThe method is new overall.\n\n2- **Is the work a novel combination of well-known techniques?** \n\nThe work builds up on techniques from variational inference and meta learning, overall it's not a simple combination of well-known techniques.\n\n3- **Is it clear how this work differs from previous contributions?**\n\nThe difference from previous contributions is discussed thoroughly.\n \n4- **Is related work adequately cited ?**\n\nThe related works are adequately cited, to my knowledge.\n\n# Quality\n1- **Is the submission technically sound ?** \n\nThe submission is technically correct.\n\n2- **Are claims well supported (e.g., by theoretical analysis or experimental results) ?**  \n\n- The theoretical analysis is correct and supports the paper's motivation.\n- The experiments are clearly split and support the claims of the paper.\n\n3- **Are the methods used appropriate ?** \n\n- The methods used are tailored to the problem (DGMM, variational inference ....)\n\n4- **Is this a complete piece of work or work in progress ?**  \n\nIt is a complete piece of work.\n\n5-  **Are the authors careful and honest about evaluating both the strengths and weaknesses of their work ?** \n\nThe authors also evaluate the limits of their work in terms of :\n- The importance of the KL down-weighting parameter\n- How to select the maximum number of mixtures\n\n# Clarity: \n\n1- **Is the submission clearly written?** \n\nThe submission is easy to follow and clearly written. Also, the theoretical analysis is clearly outlined.\n\n2- **Is it well organized? (If not, please make constructive suggestions for improving its clarity.)** \n\nThe paper is well organized and easy to follow.\n\n3- **Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)**\n\n- The source code is provided.\n- The hyperparameters are also presented in the appendix.\n\n# Significance: \n1- **Are the results important?** \n\nBeyond the improvement of the state of the art, the results are important in the way they validate the DGMM for meta learning as a reliable proof of concept.\n\n2- **Are others (researchers or practitioners) likely to use the ideas or build on them?** \n\nIt is likely that practitioners may apply the method to real world systems, given the improvement in comparison with previous methods.\n\n3- **Does the submission address a difficult task in a better way than previous work?** \n\nThe results in Table 2 show that the proposed method outperforms previous works on most settings presented. \n\n4- **Does it advance the state of the art in a demonstrable way?** \n\nIt advances the state of the art, though the improvement is around 1 to 2 percents in terms of meta-test accuracy. \n\n# Minor comments\n- There is a typo in the page 2 of the Appendix, line 29 \"is infer\" -> \"is to infer\" :)  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "15"}, "extra_text": {}}, {"review_id": "9LnyRIz9GK", "reviewer": "Reviewer_SRM6", "summary": "In this paper, the authors propose Variational Continual Bayesian Meta-Learning (VC-BML) to handle various tasks which arrives sequentially. In particular, the authors consider the tasks follow a non-stationary distribution and assume that the previous dataset cannot be stored in memory. To deal with the non-stationarity and low-resource, the authors propose to perform sequential Bayesian inference where the meta-parameters are estimated by Dynamic Gaussian Mixture Model (DGMM) whose number of components are determined by Chinese Restaurant Process.\nIn the experiments, the authors demonstrate VC-BML successfully learns the new task without suffering the negative knowledge transfer issue while avoiding the catastrophic forgetting and maintaining the ability to solve the previous tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well organized and the idea is easy to follow.\nThe method is technically sound; a meta-learning framework is combined with DGMM and Chinese Restaurant Process which are well established method to deal with the unknown number of components.\nThe drawback of the method is that the method has to keep the meta-parameter for each task, which becomes difficult when the number of tasks increase.\nThe experimental results are convincing as the proposed method is compared with the baselines and existing methods including DPMM and BOMVI. It shows the benefit to combine the Chinese Restaurant Process.\n\nI have a few minor comments.\n1. D_t^S appears in Figure 1 and Equation (15) without any explicit explanation. Please add explanation on D_t^S.\n2. EBLO should be ELBO at Lines 205, 261, and 311.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I agree with the limitations raised by authors, and feel they are adequate.", "review_text": "", "extra_scores": {"ethical_concerns": "No issue", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new, fully-Bayesian method for online meta-learning: VC-BML. A mixture models over the meta-parameters is updated dynamically. This is used as an informative prior for the upcoming tasks.

The Reviewers highlighted the novelty of the method with respect to previous works. The paper is theoretically sound, and VC-BML improves over known meta-learning algorithms in practice. Therefore, I will recommend to accept the paper.

Please take into account all the comments of the Reviewers for the camera-ready version of the paper. Especially, Reviewer h2Cd pointed out a few problems in the writing of the paper and in the comparison with Jerfel et al. 2019. The Reviewer updated his/her evaluation of the paper following your detailed reply and your promise to revise the paper accordingly.
Decision: Accept (Poster)