Continuous Meta-Learning without Tasks
Authors: James Harrison
Apoorva Sharma
Chelsea Finn
Marco Pavone
Venue:  (2020)
Keywords: Meta-learning
Continual learning
changepoint detection
Bayesian learning

Abstract: Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task. In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task. We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme. The framework allows both training and testing directly on time series data without segmenting it into discrete tasks. We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.

Under review as a conference paper at ICLR 2020
CONTINUOUS META-LEARNING WITHOUT TASKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta- neta-learning literature thus far has focused on the lask segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task. In this work, we enable the application of generic meta-learning algorithms to settings where this task seg- mentation is unavailable, such as continual online learning with time-varying task. We present meta-learning via online changepoint analysis (MOCA), an ap- proach which augments meta-learning algorithm with differentiable Bayesian changepoint detection scheme. The framework allows both training and testing directly on time series data without segmenting it into discrete tasks. We demon- strate the utility of this approach nonlinear mega-regression benchmark well as two meta-image--llasifcatooo benchmarks.
INTRODUCTION
Meta-learning methods have recently shown promise as an effective strategy for enabling efficient few-shot learning in complex domains from image classification to nonlinear regression (Finn et al., 2017; Snell et al., 2017). These methods leverage an offline meta-training phase, in which they use data from a distribution of tasks to optimize learning performance on new tasks. These algo- rithms have focused on settings with task segmenlation, where the learning agent knows when tasks change. At meta-train time, these algorithms assume access a meta-dataset of datasets from indi- vidual tasks, and at meta-test time, the learner evaluated on a single task. However, there are many applications where task segmentation is unavailable, which have thus far been under-addressedd in the meta-learning literature. For example, consider a robot which must learn adapt to a chang- ing environment. The robot may switch from one environment to another during the course of deployment, and these task switches may not be directly observed. Furthermore, using an existing time series from interaction to craft a meta-dataset may require a difficult or expensive process of detecting switches in task.
In this work, we aim to enable meta-learning in task-unsegmented settings, operating directly on time series in which the latent task undergoes discrete, unobserved switches, rather than requiring a pre-segmented meta-dataset. Equivalently, this problem can be viewed from the perspective of continual learning, in that apply the meta-learning approach the standard online learning problem statement wherein an agent must sequentially make predictions and learn with a potentially varying latent data generating process. To accomplish this, we integrate a Bayesian changepoint estimation scheme with existing meta-lcarning approaches, allowing the algorithm to reason about whether or not the task has changed in a time series. Thus, we enable standard meta-learning algorithm, which is designed for the task segmented setting, to be both trained and tested directly on time series data without the need for task segmentation.
Contributions. The primary contribution of this work is an algorithmic framework for task unseg- mented meta-lcarning which we refer to as meta-learning via online changepoint analysis (MOCA). MOCA wraps arbitrary meta-learning algorithms in differentiable changepoint estimation algo- rithm, enabling application of meta-learning algorithms directly to problems the continuous learn- ing setting. By backpropagating through the changepoint estimation framework, MOCA learns both rapidly adaptive underlying predictive model (in the form of the meta-learning model), as well as an effective changepoint detection algorithm. MOCA a generic framework which can be paired with many existing meta- meea-learming algorithms. We demonstrate the performance of MOCA on both regression and classification settings with unobserved task switches.

---

Under review as a conference paper at ICLR 2020
2 PROBLEM STATEMENT
Our goal is to apply meta-learning tools to the problem of task-unsegmented continual learning, in which an agent is presented sequentially with input zt asked to make a (probabilistic) prediction thus ideally improve its predictions by
p(U+ 1 and is then given the true label y:, and learning from the labeled examples. Following the terminology of meta-learning, we assume that these data are drawn from a distribution according to some latent task Tí, p(a yt Tr) p(æz | Ty) 24, Tz)- We will write y Tr shorthand for æ; y p(.; y TT)) We assume distribution over tasks, which we write pTT), and that the initial task TI ~ p(7) ). At each timestep.
the task is ramsldd from p(T) with some probability À (which we refer to as the hazard rate), or remains the same.
Our goal is to optimize a learning agent to perform well in this setting. Let pe (gr Diat y1a-11 by the agent's 's prediction for yt given input æ and the past labeled examples. We will evaluate the learner's performance through a negative log likelihood loss, and our objective follows:
min e
E
Σ
log Po
11 11:1-1)
T--1 Te,
w.p. 1 1-
subject to Tt. Yt
Te
Te
w.p.
Ti ~ p(T), Te.nww P(T)
We assume that we have access to representative time series generated in the same manner from the same distribution of tasks, and use this time series to optimize 0 in an offline, meta-training phase. Critically, however, in stark contrast to standard meta-learning approaches, we do not as- sume access to task segmentation, i.e. that this offline data is pre-grouped by latent parameter T. Moreover, highlight that we consider the case of individual data points provided sequentially, in contrast to the common "k-shot, n- n-wayy" problem setting prevalent in few-shot learning (especially classification). Our setting may easily be extended to the setting in which multiple data points are observed simultaneously.
PRELIMINARIES
3.1 META-LEARNING
We begin by presenting unified perspective on meta-learning in the task-segmented setting, which allows straightforward presentation of the algorithms used in this work as well as generalization to the dusk-unsegmented case. The core idea of meta-learning is to directly optimize the few-shot learning performance of machine learning model over a distribution of learning tasks, rather than just a single task, with the goal of this learning performance generalizing to other tasks from this distribution.
meta-learning method consists of two phases: meta-training and online adaptation. Let 0 be the parameters of this model learned via meta-training. During online adaptation, the model uses context data D, (æ1a. 31.t) from within one task compute statistics
= (D.)
(2)
where ʃ is function parameterized by O. For example, in MAML (Finn al., 2017), the statisties are the neural network weights after gradient updates computed using D,. In neural processes (Gar- nelo et al., 2018). the statistics are the aggregated context parameters computed via encoding and aggregating the context data. For recurrent network-based meta-learning algorithms, these statisties correspond to the hidden state of the network. For a simple nearest neighbors model, 1 may sim- ply be the context data. The model then performs predictions by using these statistics to define conditional distribution on y given new inputs x,
x. D, ~ Po x. n.).
Adopting a Bayesian perspective, we refer to Po x. n:) as the posterior predictive distribution.
The performance of this model on this task can be evaluated by considering how well this posterior predictive distribution matches the true task data distribution,
L(D,, D(p(y) x. Te) p x. fo())))
2

---

Under review as a conference paper at ICLR 2020
where D is measure of the dissimilarity of the two distributions, e.g. the KL divergence, for which this objective becomes standard negative log likelihood minimization.
Meta-learning optimizes the parameters 0 such that the model performs well across a distribution of tasks,
min TT)((( [ED, [C(D,, 0)]l
Across most meta-lcarning algorithms, including all of those referenced above, both the update rule and the prediction function are chosen to he differentiable operations, such that the parameters can be optimized via stochastic gradient descent. Given dataset pre-segmented into groups of data from individual tasks, standard veta-learning algorithms operate via first sampling group for which T is fixed, treating part of that group as the context data Dj and sampling from the remainder to obtain test points (x, y) from the same task. While this strategy models that are capable of few-shot learning in complex domains, relies on task segmentation which in many settings, especially continual learning, is not easily available.
be very effective and produce expressive
3.2 BAYESIAN ONLINE CHANGEPOINT DETECTION
To enable meta-learning without task segmentation, we extend prior work in changepoint detec- tion. Specifically, we build on Bayesian online changepoint detection (Adams & MacKay, 2007), an approach for detecting changepoints (i.e. task switches) originally presented in a streaming un- conditional density estimation context, which we review here.
BOCPD operates by maintaining belief distribution over run lengths, i.e. how many of the past data points y/, correspond to the current task. At time f, run length of r2 = T indicates that the task has switched 7 timesteps ago, i.e. D yt-ritt are all drawn from shared task T. A belief that rs implies that there has been task switch, and that the current datapoint y, was drawn from new task T' ~ p(T). We denote this belief distribution at time as br(r.) = p(r: 91a-1))
Given Tt. we know the past T data points all correspond to the same task, and thus the density p(y, 91.c-1.7t) corresponds the posterior predictive density after conditioning on the past rt data points. We can reason about the overall posterior predictive by marginalizing over the run length according to (r.))
p(y (1:111) = Σ p(u (1)-1,,
re=0
where p(y: (1:t-1,7t) is referred to as the underlying predictive model (UPM). BOCPD recur- sively computes posterior predictive densities for each value f E {0, ......,,t t 1}, and then evalu- ates new datapoints yt+ under these posterior predictive densities to update the belief distribution b(rz). In this work, we extend this approach of Adams & MacKay (2007) beyond Bayesian uncon- ditional density estimation to apply general meta-learning models operating the conditional density estimation setting, and derive these update rules in more detail for our context.
META-LEARNING VIA ONLINE CHANGEPOINT ANALYSIS
MOCA uses Bayesian changepoint detection enable the application of meta-leaming algorithms to settings without task segmentation, both at train and test time. Specifically, we extend BOCPD to derive recursive Bayesian filtering algorithm for run length in the conditional and joint density estimation setting, and leverage a base meta-learning algorithm with parameters O to provide an underlying predictive model when conditioned on a run length. In the following subsections, we first derive MOCA's Bayesian filtering updates, and then outline how the full framework can be used both train and evaluate meta-learning models on time series without task segmentation.
4.1 BAYESIAN RUN-LENGTH FILTERING
As in BOCPD, MOCA maintains a belief over possible run lengths vi. Throughout this paper, we use b, to refer to the updated belief before observing data at that timestep, (Ix, yt). Note that by is discrete distribution with support over Ti E (0,..., 1).
At time I, the agent first observes the input 3, then makes a prediction p(y: F 914-1), and subsequently observes y:. Generally, the latent task can influence both the marginal distribution of the input, p(æ; #1 (1:t-1) 1) as well as the conditional distribution p(y. #1 91a-1). Thus,
3

---

Under review as a conference paper at ICLR 2020
the agent update its belief over run lengths once after observing the input 2t, and again after observing the label Vt. We will use by(r, (x)) = p(r: F1:t. 11;t-1) to represent the updated belief over run length after observing only Tt. and be æt, yt)) p(r, a 1a Mixz) to represent the fully updated belief over ra after observing yt- Finally, will propagate this forward in time according to our assumptions on task dynamics to compute be+ (Te+1), 1), which is used in the subsequent timestep.
To derive the Bayesian update rules, we start by noting that the updated posterior is proportional to the joint density,
br (x)) - -p(r: Dia, V1st-1 1) = Z p(re, Dc 01a-1, 11:--11)
- p(xt T1:-11 11a1-1,11--11 11:- Te)p 11a-1.911-11
- po(opo 7=- (r.))
where the normalization constant Z can be computed by summing over the finite support of 1 Importantly, this update requires (a (r=1,, the base meta-leaming algorithm's 's posterior predictive density over the inputs. Within classification, this density is available for generative models, and thus a generative approach is favorable to discriminative approach within MOCA. In regression, it is uncommon to estimate the distribution of the independent variable. We take the same approach in this work and assume that æ, is independent of the task for regression problems, in which case (r( z+) = (rz).
Next, upon observing yr. we can similarly factor the belief over run lengths for the next timestep,
(r Ft. yt) = z po (ye æt, 77- (reb æz).
(4)
Again, the normalization constant can be computed via a sum over the support of r..
Finally, we must propagate this belief forward in time to obtain b (1241):
b141(re+1) = p(re+1 Li.t yi:) = Σ p(re42,72 T1:s, 91:e)
Te
= =Σ p(re41 re, 3ia, Fia V1a) (rr Di. B1:e) Σ p(re+) I re)bb(ra xc, yr).
where we have exploited the assumption that the changes task, and hence the evolution of run length rr, happen independently of the data generation process. The conditional run-length distribu- tion p(r++1 rt) is defined by our model of task evolution.
Recall that we assume that the task switches with fixed probability X, the hazard rate. Thus, for all rt, p(r++1 = 0 rz) = A, implying
(r+1 0) Abr(rr xt, y/)) X.
(5)
Conditioned on the task remaining the same, re+1 k >> 0 and rs 1- Thus, p(r++1 = k rı) = (1 (r, = k - 1} implying
b (r+41 = k) = (1 - A)be(Te = k - Tz, yt).
(6)
Equations (5) and (6) together define b over its support r E (0.
4.2 META LEARNING WITHOUT TASK SEGMENTATION
By taking a Bayesian filtering approach to changepoint detection, we avoid hard assignments of changepoints and instead perform a soft selection over run lengths. In this way, MOCA is able to backpropagate through the changepoint detection and directly optimize the underlying predictive model, which may be any meta-learning model that admits a probabilistic interpretation.
MOCA processes time series sequentially. We initialize by (i( 0) = 1. and initialize the posterior statistics for no ir 0) as specified by the parameters 0 of the meta learning algorithm. Then, at timestep f, we first observe inputs D2 and update our belief over run length accordingly, computing br(r, (x)) according to (3). Next, we marginalize over this belief to make a probabilistic prediction
for the label y:.
P Liat V1o-1)
Σ be(re #:)po (g: Ze, 77-17111 (re]) P==0
We then observe the true label yı and incur the corresponding negative log likelihood loss. We can then use this observation to update both the belief over run length, computing br(rz a., yc)
4

---

end for Under review as a conference paper at ICLR 2020
Algorithm Meta-Learning via Online Changepoint Analysis: Training
Require: Training data 1in, Mioms number of training iterations N. initial model parameters 0
1: for to N do
2: Sample training batch #1:2; 91.r from the full timeseries.
3: Initialize belief over run length by (ri 0)
4: Initialize posterior statistics 7u(r according
5:
for tort=l T do
6:
Observe
7:
Compute (r' (x) according to (3)
8:
Predict (D Witt, 91a-1) according to (7)
9:
Observe y/1
Incur NLL loss 4
log (3 3i:t: 11:--1)
11:
Compute updated posteriors 77 [T)] for all vi according to (8)
12:
Compute be (r: | 2t, y1) according to (4)
13:
Compute updated belief over run length b,+1 according to (6) and (5)
14:
end for
15:
Compute V Ei
k++ & and perform gradient descent update to 0
16:
according to (4), as well as update the posterior statistics for all the run lengths using the labeled example. A recursive update rule for allows these parameters to be computed efficiently using the past values of r)
mc [r] = (ax, yr 7--1r- 1)) A 1p........
(8)
While MOCA could be used with an algorithm which didn't 't admit such a recursive update rule, this would require storing data online and running the non-recursive posterior computation (2) on D for every rt. which involves & operations using datasets of sizes from 0 to i, and thus can be an ( operation. In contrast, the recursive updates involve t operations involving just the latest datapoint, yielding (t) complexity. Finally, we propagate the belief over run length forward time according to (5) and (6) to obtain (T(4(11) to be ready to process the next data point.
Since all these operations switches Fins Wl.n, we can run this procedure, sum the NLL losses incurred each step, and use backpropagation within standard deep learning framework to optimize the parameters of the base learning algorithm 0. Algorithm outlines this training procedure. In practice, sample shorter time-series of length from the training data to ease computational requirements during training; we discuss implications of this in the appendix. If available, user can input various levels of knowledge task segmentation by manually updating b(re) any time; further details this task semi-segmented use case are provided in the appendix.
differentiable, given training time series in which there are task
MAKING YOUR MOCA: MODEL INSTANTIATIONS
Thus far, have presented MOCA abstract level, highlighting the fact that it can be used with any meta-lcarning model that admits the probabilistic interpretation as an underlying predictive model. However, there are several practical considerations the choice of meta-learning algorithm which can influence the computational efficiency and overall performance of MOCA. For the experi- ments in this paper, we leverage two meta-learning algorithms which offer clean Bayesian learning interpretation, relatively low-dimensional posterior statistics, recursive updates for these statistics, and computationally efficient likelihood evaluation under the posterior predictive. For regression experiments, we use ALPaCA (Harrison al., 2018); for classification experiments, we use a novel algorithm based on similar Bayesian updates which we refer to as PCOC, for probabilistic clustering for online classification. For completeness, offer high level overview of these algorithms and show how they fit into the MOCA framework in the following subsections.
5.1 ALPACA: BAYESIAN
FOR REGRESSION
ALPaCA (Harrison et al., 2018) is a meta-learning approach for which the base learning model is Bayesian linear regression in learned feature space a A V(KT) ((. w), E.)) where Φ(x, w) a feed-forward neural network with weights w mapping inputs æ to a -iddiemnsion feature space. ALPaCC maintains matrix-norma distribution over K, and thus, assuming Gaussian likelihood,
5

---

Under review as a conference paper at ICLR 2020
results in matrix- normall posterior distribution over K. This posterior inference may be performed exactly, and computed recursively. The matrix-normal distribution on the last layer results in Gaussian posterior predictive density.
We fix the prior K ~ (K, A. In this matrix-normal prior, Ko R x*y is the prior mean and Ao is nh x ns precision matrix (inverse of the covariance). Given this prior and data model, the posterior may be recursively computed as follows. First, we define Q: A, 4 K.. Then, the one step posterior update is
A = A,- (A, (A,"d((a0 ( (A, (
Qet1 = 10+10" +Q+
(9)
+ T (* ( (((x+1)
and the posterior predictive distribution is
(0 :1:++1:3:: = =V((A ((xx+1),, o² (a++1)A (((
(10)
In summary, ALPaCA a meta learning model for which the posterior statistics are 7) 1Q, and the recursive update rule h( y, is given by (9). The parameters that are meta-learned are the prior statistics, the feature network weights, and the noise covariance: =
Ko, Ao, w, }. Note that, as is typical in regression, ALPaCA only models the conditional den- sity pyy x). implicitly assuming that p(ar) independent of the underlying task.
5.2 PCOC: BAYESTAN META- LEAA-LG FOR CLASSIFICATION
In the classification setting, one can obtain a similar Bayesian meta-learning algorithm by perform- ing Gaussian discriminant analysis in a learned feature space. This is a novel approach to meta- learning for classification which term probabilistic clustering for online classification (PCOC, pronounced "peacock"). We present concise description of this algorithm here but defer to the appendix for more detailed discussion.
In PCOC we process labeled input/class pairs (Ie, Mr) by encoding the input through embedding network 2, Φ5 ax;; w) and performing Bayesian density estimation for every class. Specifically, we assume a Categorical-Gause Gaussian generative model in this embedding space, and impose the con- jugate Dirichlet prior over the class probabilities and Gaussian prior over the mean for each class, y1 Cat(pI,....PP..
Pi..... Pny Dir(ar)),
Ze Ve ~ ~N E.,.,)
Ege ~ (N A..
y.,0/-
Given labeled context data (at,, Wt), the algorithm updates its belief over the Gaussian mean for the corresponding class, as well as its belief over the probability of each class. As with ALPaCA, these posterior computations can performed through closed form recursive updates. Defining qi. = Mi,, we have
An = A 1 Er.y. where 1, denotes one-hot vector with a one at index i. Terms not related to class Ye are left unchanged in this recursive update. Given this set of posterior parameters n): {ox,, Qu:Ja, AIdt). the posterior predictive density in the embedding space can be computed as
Ot 04-1 + In
Lu.. 4y + Er,,, ((xx)
p(y) Ciyy./
p(z,y) p(u) A Mo.t ArJ Ex,y)
where [2; μ, Σ) denotes the Gaussian pdf with mean μ and covariance E evaluated at z. Applying Bayes rule, the posterior predictive on Vt+11 given x is
p(z ((x.), y == =j p(z
plu++1 =j #1.+11 v1:))
Ei=p(=== = (((a)) =)
This generative modeling approach also allows computing p(z++1 ne) by simply marginalizing out y from the joint density of p(z, v).
p(z++1 m.) Σ (1 Ee,y)
As this only depends on the input ar, length belief upon seeing a, and before predicting It-
use this likelihood within MOCA to update the run
In summary, PCOC performs Bayesian Gaussian discriminant analysis for online classification, and meta-learns the parameters 0 = (oxo, 91.1.0, A1.1,, w. Σx,1:)) for efficient few-shot online classifi- cation. practice, we assume that all the covariances are diagonal to limit memory footprint of the posterior parameters. PCOC can be thought of a Bayesian analogue of prototypical networks (Snell et al., 2017). Further details regarding PCOC can be found in the appendix.

---

Under review as a conference paper at ICLR 2020
6 RELATED WORK
Online Learning, Continuous Learning, and Concept Drift Adaptation. A substantial liter- ature exists on online, continual and lifelong learning (Hazan, 2016; Chen & Liu, 2016). While these terms often used interchangeably and inconsistently, they all roughly correspond to the problem of learning within a streaming series of tasks, wherein it is desirable to re-use information from previous tasks while avoiding negative transfer French (1999): Thrun & Pratt (2012). Typi- cally, continual learning assumes access task segmentation information, whereas online learning does not (Aljundi et al., 2019). Regularization approaches (Kirkpatrick et al., 2017: Hazan, 2016; Li & Hoiem, 2017) have been shown to be an effective method for avoiding forgetting in continual learning. By augmenting the loss function for new task with a penalty for deviation from the pa- rameters learned for previous tasks, the regularizing effects of a prior are mimicked; in contrast we explicitly learn prior over task weights that is meta-trained to be rapidly adaptive. Thus, MOCA capable of avoiding substantial negative transfer by detecting task change, and rapidly adapting to new tasks. Aljundi et al. (2019) loosen the assumption of task segmentation in continual learning and operate in a similar setting to that addressed herein, but their work still focuses on learning single set of parameters that perform well on all tasks; in contrast, we operate in the meta-learning setting, aiming to learn parameters that accelerate online adaptation within a task.
Meta- Meta-Learning for Continuous and Online Learning. While continual learning techniques have mitigated forgetting in changing problem settings, large learning models have been slow to adapt to new tasks, due in part to the propensity of neural network models to overfit to small amounts of data. In response to this, there has been substantial interest in applying ideas from meta-learning to continual learning to enable rapid adaptation to new tasks. Indeed, some modern meta-learning models such as MAML (Finn et al., 2017) may be interpreted as regularization methods (Grant et al., 2018), wherein the regularization term is explicitly learned for fast adaptation. In the streaming data setting, several works (Nagabandi et al., 2019a; He et al., 2019) use a sliding window approach, wherein a small amount of recent data is used for conditioning. By not explicitly detecting task change and choosing the window length in response, these models risk suffering from negative transfer. Indeed, MOCA may be interpreted as an adaptive sliding window model, that actively infers the optimal window length. Nagabandi et al. (2019b) and Jerfel et al. (2019) aim to detect task changes via combining mean estimation of the dependent variable with MAML models. However, these models are both less expressive than MOCA (which maintains a full Bayesian posterior) and are not capable of task-unsegmented training. Instead, these models require pre-training with meta-dataset that is segmented by task, limiting their applicability relative to MOCA.
Empirical Bayes for Changepoint Models. The Bayesian online changepoint framework of Adams & MacKay (2007) (which we leverage in this paper) and the similar, simultaneous work of Fearnhead & Liu (2007) have generated a substantial body of follow-on work since their publi- cation. Due to the simplicity of these algorithms- in particular, the ability to compute closed-form posteriors as opposed to being forced turn to approximate methods such MCMC- many prac- tical modifications and extensions have been developed. Of particular relevance are two works that investigate empirical Bayes for the underlying predictive model, which is a similar problem to that addressed herein. In particular, Paquet (2007) develop allows closed-form max likelihood estimation of the prior for simple distributions via EM. Turner et al. (2009) derive general-purre -uupose gradients for hyperparameter optimization within the BOCPD neural network meta-learning models
algorithm that
model. This approach is similar to our work, although and rely on automatic differentiation for gradient computation.
EXPERIMENTAL RESULTS
We investigate the performance of MOCA in three problem settings: one in regression and two in classification. Our primary goal is to characterize the impact on performance of using MOCA to move from the standard task-segmented meta-lcarning setting to the task- tusk-unsegmented case. To this end, we investigate the performance of MOCA versus "oracle" model that uses the same base meta-learning algorithm, but has access to exact task segmentation at train and test time. We additionally compare against baseline sliding window models of various window lengths, which again the same meta-learning algorithm, but always condition the last data points. These baselines are a competitive approach to learning in time-varying data streams (Gama et al., 2014) and have been used effectively for meta-learming in time-varying settings (see e.g. Nagabandi et al. (2019a)). Finally, we compare to "train on everything" model, which only learns prior and does
7

---

Under review as a conference paper at ICLR 2020
Run Length 50
100
-5
b)
50
b)
100
-5
Time
0
XXXX
150
-
250
X
X
-5
300
-2
0
10-5
10-1
Figure 1: The performance of MOCA the sinusoid regression problem. Right: versus time. The intensity of each point in the plot corresponds the belief in run length at the associated time. The red lines show the true changepoints. Left: Visualizations of the posterior predictive density corresponding the blue dotted lines the figure the right. The red line denotes the current function (task), and red points denote samples from that function. Green points denote data from previous tasks, where more faint points are older. a) visualization of the postcrior case which MOCA did not successfully detect the changepoint. this case, is because the pre- and post- change function (corresponding to figure and b) are highly similar. e) instance a multimodal posterior. d) The changepoint initially missed due likely under the previous posterior. e) After an unlikely data point, the model increases uncertainty as changepoint detected.
The belief over length
arbitrary time. b) The visualization of the posterior for
data generated from post-change function being highly
not adapt online, corresponding standard supervised learning approach. Many problems that are currently addressed with standard supervised learning in fact have underlying temporal structure that is ignored, and thus this baseline model is valuable point of comparison.
In addition, we investigate in isolation the effects of task-segmentation information when provided train-tine time-time and at test-time. To characterize the impact of test time segmentation, we train an oracle model and at test time, remove task segmentation and replace it with MOCA's 's run length estimation. We then compare this to the oracle model tested with segmentation, ability of test-time segmentation. Similarly, to characterize the impact train time segmentation, we provide model trained using MOCA with task segmentation at test time and compare this to the same MOCA model when tested without segmentation. Finally, we investigate the performance of MOCA under partial task segmentation. Due to space constraints, we defer this to the appendix. For sinusoid experiments, confidence intervals at 95% for three different models (trained with dif- ferent random seeds). For Rainbow MNIST and minilmageNet, confidence intervals are 95% for five different models.
the only difference is avail-
7.1 SINUSOID REGRESSION
To characterize MOCA in the regression setting, we investigate the performance on a switching sinusoid problem adapted from (Finn et al., 2017), in which a task change corresponds to a re- sampled sinusoid phase and amplitude. Qualitative results as well visualization the belief over run length at each time. Qualitatively, MOCA capable of accurate and calibrated posterior inference with only a handful of data points, and is capable of identifying task change extremely rapidly. Typically, it identifies task change in one timestep, if
visualized for the sinusoid in Fig. 1,
8

---

Under review as a conference paper at ICLR 2020
67.5
Negetivve Likellhood
2.0
94
65.0
Test Accuracy (96)
60.0
90
57.5
55.0
0.5
86
0.0
10"2
10
10"
10
10a2
10
Hazard
Hazard
Hazard
MOCA (Ours)
Dracle
Window (n=3)
Window (n- 10)
Window (n=50)
TOE
Figure Performance MOCA versus baselines in sinusoid regression (left: lower is better), Rainbow MNIST (center: higher better), minilmageNet (right: higher better). versus hazard rate. Note that for both problems, MOCA always outperforms the baselines and the performance degrades only slightly from the performance the oracle. contrast, sliding window methods result severely degraded performance.
the generated data does not happen to have high likelihood under the previous task as in Fig. 1d. Performance of MOCA versus baselines is presented in Fig. for all problem domains. For sinusoid (left), MOCA achieves performance close to the oracle model and substantially outperforms the sliding window approaches for all hazard rates.
Fig. 3 shows the performance of MOCA when augmented with task segmentation let), compared to unsegmented (blue), as well as the oracle model without test segmentation (teal) compared to with test segmentation (grey). We find that as the hazard rate increases, both the value of segmentation
time (vio-
training and value of segmentation at test time increases steadily. Because our regression version MOCA is not performing density estimation for the independent variable, is not able to detect changepoint before incurring the loss associated with an incorrect prediction. Thus, for high changepoints, considerable loss is incurred, increasing the value of task segmentation. Interestingly and counter-intuitively, the model trained with MOCA outperforms the model trained with oracle supervision, when both are given oracle supervision at test time. The MOCA training results in a small "curriculum" effect due to the non-zero weight on placed on the prior for every training iteration; in comparison, for the oracle model, a missed prediction with a highly concen- trated yet incorrect posterior occasionally results in very large loss than may destabilize training. When the oracle model is trained with a small belief weight on the prior (even down to e.g. 10-1%) the performance matches the MOCA model. This suggests that MOCA may be beneficial in training by acting form of curriculum.
7.2 RAINBOW MNIST
In the classification setting, we apply MOCA to the Rainbow MNIST dataset of Finn et (2019). In this dataset, MNIST digits have been perturbed via color transformation, rotation, and scaling, and each task corresponds to unique combination of these transformations. MOCA approaches oracle performance for most hazard rates, likely due in part the fact that task change can usually be detected via a change in digit color. Seven colors were used, and thus with probability 6/7, MOCA has very strong indicator of task change.
In Fig. 3, the relative effect of the MOCA train and test is visible. For high hazard rates, as expected, MOCA at test time performs slightly worse than the oracle model. The majority of performance degradation is thus due to MOCA training. Performance degradation due MOCA training is largest for this experiment, compared to the sinusoid and minilmageNet. Because the changing digit color results in a relatively clear indicator of changepoints, and MOCA performs belief update based on both the image and the label, MOCA performs comparably to the oracle model at test time.
7.3 MINIIMAGENET
Finally, we investigate the performance of MOCA on the minilmageNNt benchmark task (Vinyals et al., 2016). This dataset consists of 100 ImageNet categories (Deng et al., 2009), each with 600 RGB images of resolution 84 84. In our continual learning setting, a semantic label that is consistent between tasks. Specifically, we split the minilmageNet dataset in to five approximately balanced high level classes, which we refer to as super-classes, as five-way classification is standard for minilmageNet (Vinyals al., 2016; Snell al., 2017). For example,
associate each class with

---

Under review as a conference paper at ICLR 2020
0.35
0.30
94
66
Test Accuracy (99)
0.25
Accurscy
64
Negelive Log
0.20
92
0.15
0.10
60
0.05
90
58
0.00
10
10
10
10
10-2
10-2
Hazard
Hazerd
Hazard
MOCA (Ours)
MOCA Train/Oracle
Oracle TrainMMCA
Oracle
Figure 3: Performance change from augmenting model trained with MOCA with task supervision at test time (violet) and from using changepoint estimation at test time for a model trained with task-supervision (teal), for sinusoid (left), Rainbow MNIST (center), and minilmageNNt (right).
one super-class is dog breeds, while another food, kitchen and clothing items; details are provided in the appendix. Then, a new task corresponds to sampling new class within each super-class, and the problem is to classify re-use between classes, and corresponds to a continual learning scenario which each super-class experiences distributional shift. Note that this is somewhat different from the typical task in few-shot learning, where classes have priori semantic meaning.
image belonging
specific super-class. This enables knowledge
Fig. shows that MOCA outperforms baselines for all hazard rates. Fig. shows that, in contrast to the Rainbow MNIST experiment, there is a large and constant (with respect to hazard rate) per- formance decrease moving from oracle MOCA test time. Interestingly, one would expect the performance decrease with respect to hazard rate to attributable primarily to lack task segmen- tation at test time- in fact, it appears that the trend is primarily consequence of MOCA training. This also holds for the Rainbow MNIST experiments. This is likely consequence of the limited amount of data, as the trend is not apparent for the sinusoid experiment.
8 DISCUSSION AND CONCLUSIONS
Future Work. While MOCA addresses continual learning problem setting, we have not formu- lated MOCA online learning algorithm. Specifically, MOCA meta-trains series, and keeps the parameters 0 fixed online, whereas an online learning algorithm would not have this train/test distinction, and would consider updating 0 continuously (Hazan, 2016). However, in order do this with MOCA, we would need to keep running buffer of all data observed far and to use as training data to update 0, which may be expensive in real-world domains where large volumes of data (e.g. high definition video from a large collection of cameras on an autonomous vehicle). Extending MOCA toward either strictly online training replay buffer (Mnih et al., 2013; Vitter, 1985). is a promising direction of future work. Indeed, may be possible to use MOCA's 's changepoint analysis to inform which data to save.
offline time-
scheme maintain efficient
Beyond the continual learning extension, data efficiency may be improved by re-using information from previous tasks or modeling task evolution dynamics. Previous work (Nagabandi et al., 2019b; Jerfel et al., 2019; Knoblauch & Damoulas, 2018) has addressed the case in which tasks reoccur in both meta-learning and the BOCPD framework, and thus knowledge (in the form of posterior estimate) may be re-used, In this work. we address the case in which tasks are sampled i.i.d. from a (typically continuous) distribution, and thus knowledge re-use is often impractical or adds marginal value. Broadly, moving beyond the assumption of i.i.d. tasks task having associated dynamics (Al-Shedivat et al., 2018) represents promising future direction.
Conclusions. MOCA enables the application of existing meta-learning algorithms to problems without task segmentation, such as the problem setting of continual learning. We find that by lever- aging Bayesian perspective meta-learning algorithms and augmenting these algorithms with Bayesian changepoint detection scheme to automatically detect task switches within time-series. we can achieve similar predictive performance when compared to the standard task mask-segmented meta- learning setting, without the often prohibitive requirement of supervised task segmentation.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Ryan Prescott Adams and David JC arXiv:07 2007.
MacKay.
Bayesian
online
changepoint detection.
Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, llya Sutskever, Igor Mordatch, and Pieter Abbeel. Continuous adaptation via meta-learning in nonstationary and competitive environments. Inter- national Conference on Learning Representations (ICLR), 2018.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Kelsey R Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. International Conference on Machine Learning (ICML), 2019.
Wei-Yo Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. International Conference on Learning Representations (ICLR), 2019.
Zhiyuan Chen and Bing Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2016.
J. Deng, W. Dong, R. Socher, Li. K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Bradley Efron and Carl Morris. Stein's estimation rule and its competitorsan empirical Bayes ap- proach. Journal of the American Statistical Association, 1973.
Paul Fearnhead and Zhen Liu. On-line inference for multiple changepoint problems. Journal of the Royal Statistical Sociery: Series B (Statistical Methodology). 2007.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. International Conference Machine Learning (ICML), 2017.
Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. Inter- national Conference on Machine Learning (ICML), 2019.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 1999.
João Gama, Indre Žliobaite, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM computing surveys (CSUR), 2014.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S.M. Ali Es- lami, and Yee Whye Teh. Neural processes. International Conference Machine Learning (ICML), 2018.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-learning hierarchical Bayes. International Conference on Learning Representa- tions (ICLR), 2018
James Harrison, Apoorva Sharma, and Marco Pavone. Meta-learning priors for efficient online Bayesian regression. Workshop on the Algorithmic Foundations of Robotics (WAFR), 2018
Elad Hazan. Introduction to online convex optimization. Foundations and Trends in Optimization, 2016.
Xu He, Jakub Sygnowski, Alexandre Galashov, Andrei A Rusu, Yee Whye Teh, and Razvan Pas- canu. Task agnostic continual learning via meta learning. arXiv preprint www.iv:906.05220 2019.
Khurram Javed and Martha White. Meta-learning representations for continual learning. Neural Information Processing Systems (NeurIPS), 2019.
Ghassen Jerfel, Erin Grant, Thomas I. Griffiths, and Katherine Heller. Online gradient-based mix- tures for transfer modulation in meta-learning Neural Information Processing Systems (NeurIPS), 2019.
11

---

Under review as a conference paper at ICLR 2020
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2015.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A et al. Overcom-
Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 2017.
Jeremias Knoblauch and Theodoros Damoulas. Spatio-temporal bayesian on-line changepoint de- tection with model selection. International Conference on Machine Learning (ICML), 2018.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wier- stra, and Martin Riedmiller. Playing atari with deep reinforcement learning. Neural Information Processing Systems (NeurIPS), 2013.
Kevin P Murphy. Machine Learning: Probabilistic Perspective. MIT Press, 2012.
Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta- reinforcement learning. International Conference on Learning Representations (ICLR), 2019a.
Anusha Nagabandi, Chelsea Finn, and Sergey Levine Deep online learning via meta-learning: Continual adaptation for model-based RL. arXiv:1812.07671, 2019b.
Uirich Paquet. Empirical Bayesian change point detection. Graphical Models, 2007.
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classifica- tion. International Conference on Learning Representations (ICLR), 2018.
Y Saatci, R Turner, and CE Rasmussen. Gaussian process change point models. International Conference Machine Learning (ICML), 2010.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Neural Information Processing Systems (NeurIPS), 2017.
Charles Stein. Inadmissibility of the usual estimator for the mean of multivariate normal distribu- tion. Third Berkeley symposium on Mathematical statistics and Probability, 1956.
Sebastian Thrun and Lorien Pratt. Learning to learn. Springer, 2012.
Ryan Turner, Yunus Saatci, and Carl Edward Rasmussen. Adaptive sequential Bayesian change point detection. NeurIPS Workshop on Nonparametric Bayes, 2009.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Neural Information Processing Systems (NeurIPS), 2016.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions Mathematical Software (TOMS), 1985.
Robert C Wilson, Matthew R Nassar, and Joshua I Gold. Bayesian online learning of the hazard rate in change-point problems. Neural Computation, 2010.
12

---

Under review as a conference paper at ICLR 2020
Negative Log Likellhood
0.0775
Figure 4: Performance versus the training horizon (7) for the sinusoid with hazard 0.01. The lowest hazard was used to increase the effects of the short training horizon. A minor decrease in performance is visible for very small training horizons (around 20), but flat- tens off around 100 and above. It is expected that these diminishing marginal returns will occur for all systems and hazard rates.
0.0750
0.0725
0.0700
0.0675
0.0650
50
100
150
200
Train Episode Length
A BATCH TRAINING MOCA
In practice, we sample batches of length from the full training time series, and train on these com- ponents. While this artificially increases the observed hazard rate (as result of the initial belief over run length being 0 with probability 1), substantially reduces the computational burden training. Because MOCA maintains a posterior for each possible run length, computational requirements grow linearly with T' Iterating over the whole training time series without any hypothesis pruning can be prohibitively expensive. While a variety of different pruning methods within BOCPD have been proposed (Wilson et al., 2010; Saatci et al., 2010), we require a pruning method which does not break model differentiability. Note that at test-time, we no longer require differentiability and so previously developed pruning methods may be applied.
Empirically, we observe diminishing marginal returns when training on longer sequences. Fig. shows the performance of MOCA for varying training sequence lengths (?). In all experiments presented in the body of the paper, we use T 100. As discussed, small T values artificially inflate the observed hazard rate, so we expect to see performance improve with larger T values. Fig. shows that this effect results in diminishing marginal returns, with little performance improvement beyond T 100. Longer training sequences lead to increased computation per iteration (as MOCA is linear in the runlength), as well as an increased memory burden (especially during training, when the computation graph must be retained by automatic differentiation frameworks). Thus, we believe is best to train on the shortest possible sequences, and propose T 1/A (where À is the hazard rate) as rough rule of thumb.
B PROBABILISTIC CLUSTERING FOR ONLINE CLASSIFICATION
In this section we present in more detail probabilistic clustering for online classification (PCOC, pronounced "peacock"), a framework for Bayesian meta-learning for classification. PCOC extends embedding-basee meta-leamnn algorithms (e.g. Snell et al. (2017); Vinyals et al. (2016); Allen al. (2019); Ren et al. (2018)) enable expressive posterior distributions (which are useful for use within the MOCA framework). However, PCOC is a valuable meta- meta-learning algorithm outside of the MOCA framework, with many features of note for downstream applications.
PCOC maps each data point, It, through an embedding function' Φ : RT= + R We choose a neural network with weights for the embedding function. We assume generative model within the embedding space. We will assume that each class sampled from task dependant categorical distribution with class probabilities Pi..... Pry. We will assume that for each class j. z = of ()) (for with уı = j) follows Gaussian distribution with mean Σ, and variance E.j. This assumption is standard generative modelling assumption corresponding to a Gaussian mixture model (Murphy, 2012). As such, our classification strategy will be based on Gaussian discriminant analysis (GDA, also referred to as quadratic discriminant analysis).
Gauusiia Given this Categorical Gaussian generative model, we will fix a conjugate Dirichlet prior on class probabilities and Gaussian prior on the class conditional mean:
PI...., Pn, Dir(as)
3, NW A7,0) Vj
(11)
We will refer the mapped-to space the feature space the embedding space.
13

---

Under review as a conference paper at ICLR 2020
Algorithm 2 Probabilistic Clustering for Online Classification
Require: Meta-dataset D
1: Randomly initialize weights of Φb Dirichlet priors, prior mean and variance of each class
2: while not converged do
3: for all D E do
4: Split D into conditioning data { M1:k f and evaluation data {#++1:7 M+++1:1) }
5: Compute jjjk: A, and posterior Dirichlet concentration parameters for each class j
6: Evaluate probability of evaluation data under posterior densities (Eq. 16)
7: end for
8: Update network weights and prior terms based on maximum likelihood of evaluation data
9:
end while
This choice of conjugate prior and generative model means that the posterior distributions on and 2j remain Dirichlet and Gaussian respectively, and that the parameters of this posterior can be computed analytically.
The posterior parameters time after observing ky samples of class are
A = A5,0 + +ky Dy
(12)
j4jt AJJ (ky Pj. + A301350)
(13)
02 = ky.
(14)
where Φ5. is the sample mean of the embedded points corresponding to class j. These can also be computed recursively, as outlined in the main paper in equation (11).
Given these posteriors, the posterior predictive distribution for class j is Gaussian:
pl ( 1/++1 = =j, 01:1; y1::) = (Z Mj.t, A + Eeg)
(15)
Given this within-class posterior predictive, we can now consider the posterior predictive over classes. Note that, by Bayes rule,
p(1++1 yot1 = j. Vnd) æ1a. (16)
p(y++1 =j j 1c+11 Wit M1r))
P( yr41 ai Vl.e (1)))(()411111))))))))))))11))1
2 Mi:))
Because we have finite number of classes, computing the partition function in the denominator is tractable. The posterior Dirichlet probabilities take the form
Qj,
p j Fi y1+)) = p(y++1 = y1:)) Σ, ai,
(17)
and we choose to meta-learn the Dirichlet prior on In the general meta-classification setting, each label has task-dependent probability (the classes are not necessarily balanced, as is typically as- sumed in e.g. shot and 5 shot benchmarks). As such, online estimation of the posterior allows us to infer the class probabilities within one task, and the Dirichlet priors allow us to meta-learn a be- lief over label probabilities between tasks. In addition to learning priors on imbalanced classes, this approach allows our model to encode confidence in the class probabilities. For example, for small aj.0's s, the model will be highly sensitive to the empirical class counts within one task, whereas for large aj,o" s, the empirical counts within one task will have relatively small effect.
B.1 PCOC FOR EPISODIC
The standard episodic meta-classification benchmarks are typically of the form of k--hhot (corre- sponding to the number of context data points observed for each class), and n-way (corresponding to the number of classes) (Snell al., 2017; Finn et al., 2017; Vinyals et al., 2016). This setting based on association between the context data and the test data; the labels do not have priori semantic value. For example, a mnetacclasifiiattin problem is identical if two class labels are ex- changed. This property results in simplifications for the PCOC model. Maintaining prior each class individually is no longer logical, as two classes with different priors could have their la- bels switched with no change in the problem. Therefore, in this setting, we maintain a shared prior for the mean of all classes.
14

---

Under review as a conference paper at ICLR 2020
This generic prior over data is useful for modified problem statements. Consider setting in which we do not priori know the number of classes. this case, which data being provided sequen- tially, we wish to report if the data provided at time / corresponds to a previously unobserved class. Replacing the Dirichlet prior with Chinese restaurant process (as in e.g. Nagabandi et al. (2019b); Allen et al. (2019)) would enable a few-shot meta-classification model with a potentially expandable number of classes. Moreover, a better calibrated confidence in outputs is available, which is useful for downstream tasks.
B.2 PCOC FOR STREAMING SEET-CLASSSFICATIINN
We will now discuss modifications to the PCOC framework for the streaming setting. We will discuss two cases:
1. The set of labels is known a priori (and labels have semantic value-ia i.e. reporting class j has specific meaning beyond indicating that a data point belongs to the same class as other data points of class j).
2. The set of labels is not known in advance, and thus our streaming meta-classification algo- rithm must be able to predict when class is previously unseen.
In this paper consider the first case. In this setting, we may directly apply PCOC as described above. Importantly, this setting allows "zero-shot" classification, which is critical in the MOCA framework, as we have distinct, semantically meaningful priors for each class mean. In the second case, the set of labels would necessarily need be expanded over time, for which non-parametric model may be used as described above. There are several versions of this problem statement, which is more similar to a "lifclong learning" setting, and we defer them to future work.
B.3 DISCUSSION
PCOC extends line of work on meta-classification based on prototypical networks (Snell et al., 2017). This framework maps the context data to an embedding space, after which it computes the centroid for each class. For a new data point, it models the probability of belonging to each class as the softmax of the distances between the embedded point and the class centroids, for some distance metric. For Euclidean distances (which the authors focus on), this corresponds to performing fre- quentist estimation of class means, under the assumption that the variance matrix for each class is the identity m Indeed, this corresponds to the cheapest-to -heapest-to-evallate simplification of PCOC. Ren al. (2018) propose adding class-dependent length scale (which is a scalar), which corresponds to meta- veta-learnnng a frequentist estimate of the variance for each class. Moreover, it corresponds to assuming variance that takes the form of a scaled identity matrix. Indeed, assuming diagonality of the covariance matrix results in substantial performance improvement the matrix inverse may be performed clement-wise. This reduces the numerical complexity of this operation in the (fre- quently high-dimensional) embedding space from cubic to linear. However, in our implementation of MOCA, we assume diagonal covariances throughtout, resulting in comparable computational complexity to the different flavors of prototypical networks. If one were to use dense covariances, the computational performance decreases substantially (due to the necessity of matrix inversions), especially in high dimensional embedding spaces.
In contrast to this previous work, PCOC has several desirable features. First, both Snell et al. (2017) and Ren et al. (2018) 8) make the implicit assumption that the classes are balanced, whereas we per- form online estimation of class probabilities via Dirichlet posterior inference. Beyond this, our approach is explicitly Bayesian, and we maintain priors over the parameters that we estimate online. This is critical for utilization in the MOCA framework. Existence of these priors allows "zero-shot" learning-it it enables a model to classify incoming data to certain class, even if no data belonging to that class has been observed within the current task. Finally, because the posteriors concentrate (the predictive variance decreases more data is observed). we may better estimate when a change in the task has occurred. We also note that maximum likelihood estimation of Gaussian means is dom- inated by the James-Stein estimator (Stein, 1956), which shrinks the least squares estimator toward some prior. Moreover, the James-Stehl estimator paired with empirical Bayesian estimation of the woor-which the basis for Bayesian meta- meta-learning approaches such as ALPaCA and PCOC been shown to be a very effective estimator in this problem setting (Efron & Morris, 1973).
"Snell al. (2017) discuss this correspondence, they outline how the choice of metric corresponds different assumptions the distributions in the embedding space.
15

---

Under review as a conference paper at ICLR 2020
0.0
0.249
Test Supervision Rate
Figure S: Test negative log likelihood of MOCA the sinusoid problem with par- tial task segmentation. The partial segmenta- tion during training results in negligible per- formance increase, while partial supervision test time uniformly improves performance. Note that each column corresponds one trained model,
0.05
0.246
0.1
0.243
0.2
0.240
thus the randomly vary- ing performance across train supervision rates may be explained by simply results minor differences in individual models.
0.5
0.237
1.0
0.0
0.05
0.1
0.2
0.5
1.0
Train Supervision Rate
C MOCA WITH PARTIAL TASK SEGMENTATION
Since MOCA explicitly reasons about belief run-lengths, it operate anywhere in the spectrum of the lask-unsegmented case as presented so far, to the fully task-segmented setting of standard meta-learning. At every time step the user can override the belief (rr) to provide degree of supervision. At known changepoints, for example, the user override (r:)) to have all its mass on It = If the task is known not to change at the given time, the user can set the hazard probability to 0 when updating the belief for the next timestep. If user applies both of these overrides, it amounts effectively sidestepping the Bayesian reasoning over changepoints and revealing this information to the meta-learning algorithm. If the user only applies the former, the user effectively indicates to the algorithm when known changepoints occur, but the algorithm free propagate this belief forward time according to the update rules, and detect further changepoints that were not known to the user. Finally, the Bayesian framework allows a supervisor to provide their belief over a changepoint, which may not have probability mass entirely at r, = 0. Thus, MOCA flexibly incorporates any type of task supervision available system designer.
Fig. shows the performance partial task segmentation at both train and test for the sinusoid problem, for the hazard rate 0.2. This problem was chosen as the results were highly repeatable and thus the trend is more readily observed. Here, we label changepoint with some probability, which we refer to as the supervision rate. We do not provide supervision for any con-changepoint timesteps, and thus a supervision rate of corresponds to labeling every changepoint but is not equivalent to the oracle. Specifically, the model may still have false positive changepoints, but is incapable of false negatives. This figure shows that the performance monotonically improves with increasing train supervision rate, but is largely invariant under varying train supervision. This per- formance improvement agrees with Fig. 3, which shows that for the sinusoid problem, performance is improved by full online segmentation. Indeed, these results show that training with MOCA results in models with comparable test performance to those with supervised changepoints, and thus there little marginal value task segmentation during training.
D COMPUTATIONAL PERFORMANCE
Fig. 6 shows the computational performance at test time on the sinusoid problem. Note that the right hand side of the curve shows linear trend that is expected from the growing run length belief vector. However, even for 25000 iterations, the execution time is approximately 7ms for one iteration. These experiments were performed on Nvidia Titan Xp GPU. Interestingly, on the left hand side of the curve, the time per iteration effectively constant until the number of iterations approaches approximately 4500. Based on our code profiling, we hypothesize that this is an artifact of overhead in matrix multiplication computations done the GPU.
16

---

Under review as a conference paper at ICLR 2020
Figure 6: Time per iteration versus iteration number at test time. Note that the right hand side of the curve shows the expected linear complexity expected MOCA. Note that for these experiments, no hypothesis pruning was performed, and thus at test time performance could constant time as
Time per nerrttio (ms)
6-
5-
opposed to linear. This figure shows 95% confidence inter-
vals for 10 trials, but the repeatability of the computation time
is consistent enough that they are not visible.
5000
10000
15000 20000
25000
Test Iteration
E EXPERIMENTAL DETAILS
E SINUSOID
To test the performance of the MOCA framework combined with ALPaCA for the regression setting, investigate switching sinusoid regression problem. The standard sinusoid regression problem, in which randomly sampled phase and amplitude constitute a task, is a standard benchmark in meta- learning (Finn et al., 2017). Moreover, switching sinusoid problem is popular benchmark in continuous learning (He et al., 2019; Javed & White, 2019). Each task consists of a randomly sampled phase in the range [0, π and amplitude in [0.1,5], 5]. This task was investigated for varying hazard rates. For the experiments in this paper, samples from the sinusoid had additive zero-mean Gaussian noise of variance 0.05.
E.2 RAINBOW MNIST
The Rainbow MNIST dataset (introduced in Finn et al. (2019)) contains 56 different color/scalerrotation transformations of the MNIST dataset, where one transformation constitutes task. We split this dataset into train of 49 transformations and test set of 7. For hyperpa- rameter optimization, we split the train set into training set of 42 transformations and a validation of 7. However, because the dataset represents fairly small amount of tasks (relative to the sinusoid problem, which has infinite), after hyperparameters were we trained on all 49 tasks. We found this notably improved performance. Note that the same approach was used Snell et al. (2017).
E.3 MINIIMAGENET
We use the minilmageNet dataset of Vinyals et al. (2016), a standard benchmark in few-shot learn- ing. However, the standard few-shot learning problem does not require data points to be assigned to a certain class label. Instead, given context data, the goal is to associated the test data with the correct context data. We argue that this problem setting is implausible for the continual learning setting: while observing data stream, you are also inferring the set of possible labels. Moreover, after a task change, there is no context data to associate new point with. Therefore we instead assume known set of classes. We group the 100 classes of minilmageNet in to five super- super-classes, and perform five-way classification given these. These super-classes vary in intra-class diversity of sub-classes: for example, one of the super-class is entirely composed of sub-classes that are breeds of dogs, while another corresponds to buildings, furniture, and household objects. Thus, the strength of the prior information for each super-class varies. Moreover, the intra-class similarities are quite weak, and thus generalization from the train set to the test set is difficult and few-shot learning is still necessary and beneficial. The super-classes are detailed in table ??.
The super-classes
roughly balanced in terms of number of classes contained. Each task corre- spond to sampling a class from within each super-class, which was fixed for the duration of that task. Each super-class was sampled with equal probability.
E.4 BASELINES
Three baselines used, described below:
17

---

Under review as a conference paper at ICLR 2020
Class
Description
Train/Val/Test
Synsets
Train
n01532829, n01558993, n01704323, n
Non-dog animals
1
n01770081, n01843383, n01910747, m020047777 n02074367.
n02165456, n02457408. n02606052, m04275548
Validation
n01855672, n02138441, n02174001
Test
n019301 n01981276, n02129165, n02219486,
n02443484
Train
n02089867, n02091831, n02101006, n02105505,
Dogs, foxes, wolves
2
102108089, n02108551, n02108915, n021 11277
n02113712, n02120079
Validation
n02091244, n02114548
Test
n02099601, n02110063, n02110341, 402116733
Vehicles, musical instruments, nature/outdoors
Train
n02687172. n02966193, n03017168, n03838899,
3
n03854065, n04251 144, n04389033, n04509417,
n04515003, n04612504, n09246464, n13054560
Validation
n02950826, 102998192,, n03417042, n03584254,
n03773504, n09256479
Test
m03272010, n04146614
Food, kitchen equipment, clothing
Train
m02747177. n02795169, n02823428, n03047690,
m03062245, n03207743, n03337140, n03400231,
m03476684, n03527444, n03676483, n04596742,
n075841 10, n07697537, m07747607, n13133613
Validation
m03770439, n03980874
Test
n03146219, nO3775546, m04522168, n07613480
Building. furniture, household items
Train
n03220513, n03347037, n03888605, n03908618,
5
n03924679, n03998194, n04067472, n04243546.
n04258138, n04296562, n04435653, n04443257,
n04604644, n06794110
Validation
n02971356, n03075370, no3535780
Test
n02871525, n03127925, n03544143, n04149813,
n04418357
Table 1: Our super-class groupings for minilmageNet experiments.
This baseline consists ignoring task variation and treating the training timeseries one dataset. Note that many datasets contain latent temporal infor- mation that ignored, and this approach is effectively common practice.
Train on Everything:
Oracle: In this baseline, the same ALPaCA and PCOC models were used as MOCA, but with exact knowledge of the task switch times. Note that within a regret setting, one typically compares the best achievable performance. The oracle actually outperforms the best achieveable performance in this problem setting, as it takes at least one data point (and the associated prediction, on which loss is incurred) to become aware of the task variation. Sliding Window: The sliding window approach commonly used within problems that exhibit time variation, both within meta-learning (Nagabandi et al., 2019a) and continual learning (He et al., 2019; Gama et al., 2014). In this approach, the last data points are used for conditioning, under the expectation that the most recent data is the most predictive of the observations in the near future. Typically, some form validation is used to choose the window length, As MOCA is performing form of adaptive windowing, it should ideally outperform any fixed window length. We compare three window lengths (n 5, 10, 50), each of which are well-suited to part of the range of hazard rates that we consider.
E.5 TRAINING DETAILS
Sinusoid. A standard feedforward network consisting of two hidden layers of 128 units used with ReLU nonlinearities. These layers were followed by 32 units layer and another tanh nonlin- carity. Finally, the output layer (for which we learn prior) was of size 32 x 1. The same architecture was used for all baselines. This is the same architecture for sinusoid regression as used Harri- son et al. (2018) (with the exception of using ReLU nonlinearities instead of all tanh nonlinearities). The following parameters were used for training:
Optimizer: Adam (Kingma & Ba, 2015)
18

---

Under review as a conference paper at ICLR 2020
Learning rate: 0.02
Batch size: 50
Batch length: 100
Train iterations: 7500
Batch length here corresponds to the number of timesteps in each training batch. Note that longer batch lengths are necessary to achieve good performance on low hazard rates, as short batch lengths result of the assumption that each batch begins with new
artificially increase the hazard rate task. The learning rate was decayed every 1000 training iterations.
We allowed the noise variance to be learned by the model. This, counter-intuitively, resulted a substantial performance improvement over fixed (accurate) noise variance. This is due to cur- riculum effect, where the model early one increases the noise variance and learns roughly accurate features, followed by slowly decreasing the noise variance to the correct value.
Rainbow MNIST. In experiments, used the same architecture al. (2017); Vinyals et al. (2016). It often unclear in recent work on few-shot learning whether performance improvements are due to improvements in the meta-learning scheme or the network ar- chitecture used (although these things are not easily disentangled). As such, the architecture use in this experiment provides fair comparison consists of four blocks of 64 3 X 3 convolution filters, followed by a batchnorm, ReLU nonlinearity and 2 x max pool. On the last conv black, removed the batchnorm and the nonlinearity. For the 28 x 28 Rainbow MNIST dataset, this encoder leads a 64 dimensional embedding space. For the "train on everything" baseline, we used the same architecture followed by a fully connected layer and softmax. This architecture is standard for image classification and has comparable number of parameters to our model.
used in Snell
previous few-shot learning work. This architecture
We used a diagonal covariance factorization within PCOC, substantially reducing the number of terms in the covariance matrix for each class and improving the performance of the model (due the necessary inversion of the posterior predictive covariance). We learned a prior mean and variance for each class, as well as noise covariance for each class (again, diagonal). We also fixed the Dirichlet priors be large, effectively imbuing the model with the knowledge that the classes were balanced. The following parameters were used for training:
Optimizer: Adam
Learning rate: 0.02
Batch size: 10
Batch length: 100
Train iterations: 5000
The learning rate was decayed every 1500 training iterations.
minilmageNet. Finally, for minilmageNet, we used six convolution blocks, each as previous de- scribed. This resulted in 64 dimensional embedding space. We initially attempted to use the same four-conv backbone as for Rainbow MNIST, but the resulting 1600 dimensional embedding space had unreasonable memory requirements for batches lengths of 100. Again, for the "train on everything" baseline, used the same architectures with one fully connected layer followed by softmax. The following parameters were used for training:
Optimizer: Adam
Learning rate: 0.002
Batch size: 10
Batch length: 100
Train iterations: 3000
The learning rate was decayed every 1000 training iterations. We used the validation set to monitor performance, and as in Chen et al. (2019), we used the highest validation accuracy iteration for test. We also performed data augmentation as in Chen et al. (2019) by adding random reflections and color jitter to the training data.
E.6 TEST DETAILS.
For all problems, a test horizon of 400 was used. Again, the longest possible test horizon was used to avoid artificial distortion of the test hazard rate. Both both problems, a batch of 200 evaluations was performed, and all confidence intervals correspond 95%.
19


Reviews:
[{"review_id": "HyxY4QkRYr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper pushes meta-learning towards task-unsegmented settings. Different from the traditional offline meta-learning phase with explicit task segmentation, MOCA adopts a Bayesian changepoint estimation scheme for task change detection. The setting is novel and deserves research in-depth, and the idea is easy to understand. The proposed method can learn the meta-learning model and changepoint detection model simultaneously. Besides, the MOCA framework is not designed specifically for one algorithm and can be easily combined with other meta-learning models.\n\nHowever, I got some questions about this paper:\nQ1: Is the ‘Hazard’ in the experiment the same to $\\lambda$ in eq.1? I think notations should be consistent if they are the same.\nQ2: Can other changepoint models be compared in the experiment? I found many in the related work.\nQ3: The running times in Figure 2 should be reported to demonstrate the efficiency of MOCA, since the method is proposing online streams, efficiency should be promised for quickly processing.\nQ4: What is ‘T’ in Algorithm 1? Are experiment results sensitive to it? Experiments about this should be conducted and reported.\n\nLastly, I think [1] should be cited as related work about continual learning for proposing task-free continual learning, which is very similar to the setting in this paper.\n[1] Rahaf Aljundi, Klaas Kelchtermans, Tinne Tuytelaars. Task-Free Continual Learning. CVPR 2019", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "ByeuiSECKB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a solution for using meta-learning methods without the need of determining the task segmentation a priori. Authors identify that the task segmented setting is very similar to the problem of change-point detection (CPD) and particularly, they connect the generative model of the meta-learning approach to the Bayesian recursive method of Adams and MacKay 2007. The presentation of the meta-learning problem is well done and I understand its importance within the difficulties for modelling new unobserved tasks. The notation and description of the generative model included in the problem statement section is clearly understandable for any reader, this is a very positive point. They demonstrate a deep comprehension of the BOCPD model of Adams, and its extension for the meta-learning approach is original to me. Lastly, the presentation of the MOCA meta-learning algorithm is useful for reproducibility and I see it completely applicable to other scenarios. There is a noticeable effort of describing the solution for both regression and classification problems and the empirical results conclude a positive performance of MOCA.\n\nOverall, I consider that the paper is well-written with thorough explanations. Of course, there are some details that could be improved (I will comment this later). If done, I would be willing to increase my score. The contribution of the paper is significant to meta-learning models and I think it will help to spread the Adams’ model to other type of problems.\n\nThere is an important contribution in the paper that I have to mention and it is also relevant for the future application of the Adams model. If one reads the original BOCPD paper, in particular Eq (1), where the predictive posterior p(x_t+1|x_{1:t}) is defined from a marginalisation over the run length values, it is noticeable that this equation is not used in the final recursion of the CPD method. This is because the posterior p(r_t|x_{1:t}) is sufficient for determining if there is a CP on the t time-step or not. So the predictive posterior is never used in practice (in the original paper). When I first read Adams 2007, this detail was clear to me. Surprisingly, I find that the authors have find a practical use of this equation (Eq. (7)) and it is in the main core of the meta-learning algorithm, this is fantastic.\n\nThe details I think should be improved are:\n\n- The conditional run-length prior p(r_t|r_{t-1}) barely appears and without any detailed description is not obvious for non-familiar readers with the Bayesian CPD approach.\n- The first time I read the manuscript, the use of z_t in the BOCPD presentation made me feel a bit lost. Why not use x or y? At least specify that is a toy variable for the explanation. Later on, authors change again to the x,y notation.\n- Equations 3 and 4 are too similar, this looks a bit repetitive. Why not reusing one of them or say the change from one term to another? \n- If reading the literature of the BOCPD and posterior extensions, the likelihood model of the detector is often referred as the underlying predictive model (UPM). Using a similar term would help for orienting familiar readers into the solution.\n- As authors should have noted on their experiments, as one makes the run-length higher, the number of parameters \\eta[r_t] increases. A clear state on how this is solved would help.\n- In the PCOC subsection, I find the definition of y \\sim q(y) a bit weird. Using Cat(_) or Multinomial likelihood notation would be a bit better.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1eczZEItS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the meta-learning in the task un-segmented setting and apply bayesian online change point detection with meta-learning. The task un-segmented is claimed to exist in real applications and the paper explains the idea in a clear way. \n\nMy major concerns and questions are the following:\n\n1) In Eq(4), it requires a computation of normalization constant that needs to sum over support of r_t. The support gets larger and larger with the length of sequence increases over time. Does this method scale well with long sequence?\n\n2) The part I feel most confused are the experiments.  The paper tries to solve a problem in the task un-segmented setting, but why there is no such setting in experiments that the meta-training set cannot be segmented to different tasks?\n\n3) The Figure 1 is hard to read. What do the red and green points in the left panel mean? The hazard rate is not defined before the experiment which makes Figure 3 also hard to understand. \n\n4) What is the meta-learning algorithm used in the experiements, MAML, NP or RNN-based methods?  It claims MOCA can be used with any meta-learning algorithms and how does it show in experiments?\n\n5) In Rainbow MNIST, why in high hazard rates, all models perform comparable to “train on everything”? Does it mean meta-learning does not work in this configuration and why? Does “train on everything” includes fine-tuning on the meta-test training set?\n\n6) In Mini-IMAGENET, what does it mean by \"we associate each class with a semantic label that is consistent between tasks”? Why it needs to form “super-class”? \n\nI think the proposed method can be useful in the task un-segmented setting. But before the above-mention questions are solved and the experimental section gets more clear, I will give a conservative rating. \n\n\n######################\nPost-rebuttal review:\nThanks for the authors' feedback and it resolves some confusion parts in the paper. Though the author claims the reason of experiments setting, I believe it is necessary to have one experiment where task-segmentation is impossible and compare with standard sequential learning methods in that setting. Otherwise it remains a question whether the proposed method works only when the problem basically has task-segmentation.  And if the problem has task-segmentation, why not using traditional meta-learning methods, as shown as oracle methods with better performance in paper? \nSince the major contribution of the paper is providing a meta-learning method to work in the problems where task-segmentation is unavailable, not having an experiment in this setting (withholding segmentation information does not exactly fall in this setting because it adds a condition that the task-segmentation information is originally accessible) is a major reason for my current evaluation.  I recognize the careful design of the MOCA and agree with some positive points raised by other reviewers. I would not be bothered if the paper is accepted while I tend to maintain the current rating because of the above-mentioned concern.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper the authors view meta-learning under a general, less studied viewpoint, which does not make the typical assumption that task segmentation is provided. In this context, change-point analysis is used as a tool to complement meta-learning in this expanded domain. 
 
The expansion of meta-learning in this more general and often more practical context is significant and the paper is generally well written. However, considering this particular (non)segmentation setting is not an entirely novel idea; for example the reviewers have already pointed out [1] (which the authors agreed to discuss), but also [2] is another relevant work. The authors are highly encouraged to incorporate results, or at least a discussion, with respect to at least [2]. It seems likely that inferring boundaries could be more powerful, but it is important to better motivate this for a final paper. 
 
Moreover, the paper could be strengthened by significantly expanding the discussion about practical usefulness of the approach. R3 provides a suggestion towards this direction, that is, to explore the performance in a situation where task segmentation is truly unavailable. 
 
[1] Rahaf et el. "Task-Free Continual Learning". 
[2] Riemer et al. "Learning to learn without forgetting by maximizing transfer and minimizing interference".
 

Decision: Reject