user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continuous Meta-Learning without Tasks

Paper content:
Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks.
However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task.
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme.
The framework allows both training and testing directly on time series data without segmenting it into discrete tasks.
We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.
Meta-learning methods have recently shown promise as an effective strategy for enabling efficient few-shot learning in complex domains from image classification to nonlinear regression (Finn et al., 2017; Snell et al., 2017) .
These methods leverage an offline meta-training phase, in which they use data from a distribution of tasks to optimize learning performance on new tasks.
These algorithms have focused on settings with task segmentation, where the learning agent knows when tasks change.
At meta-train time, these algorithms assume access to a meta-dataset of datasets from individual tasks, and at meta-test time, the learner is evaluated on a single task.
However, there are many applications where task segmentation is unavailable, which have thus far been under-addressed in the meta-learning literature.
For example, consider a robot which must learn to adapt to a changing environment.
The robot may switch from one environment to another during the course of deployment, and these task switches may not be directly observed.
Furthermore, using an existing time series from interaction to craft a meta-dataset may require a difficult or expensive process of detecting switches in task.
In this work, we aim to enable meta-learning in task-unsegmented settings, operating directly on time series in which the latent task undergoes discrete, unobserved switches, rather than requiring a pre-segmented meta-dataset.
Equivalently, this problem can be viewed from the perspective of continual learning, in that we apply the meta-learning approach to the standard online learning problem statement wherein an agent must sequentially make predictions and learn with a potentially varying latent data generating process.
To accomplish this, we integrate a Bayesian changepoint estimation scheme with existing meta-learning approaches, allowing the algorithm to reason about whether or not the task has changed in a time series.
Thus, we enable a standard meta-learning algorithm, which is designed for the task segmented setting, to be both trained and tested directly on time series data without the need for task segmentation.
Contributions.
The primary contribution of this work is an algorithmic framework for task unsegmented meta-learning which we refer to as meta-learning via online changepoint analysis (MOCA).
MOCA wraps arbitrary meta-learning algorithms in a differentiable changepoint estimation algorithm, enabling application of meta-learning algorithms directly to problems in the continuous learning setting.
By backpropagating through the changepoint estimation framework, MOCA learns both a rapidly adaptive underlying predictive model (in the form of the meta-learning model), as well as an effective changepoint detection algorithm.
MOCA is a generic framework which can be paired with many existing meta-learning algorithms.
We demonstrate the performance of MOCA on both regression and classification settings with unobserved task switches.
Future Work.
While MOCA addresses a continual learning problem setting, we have not formulated MOCA as an online learning algorithm.
Specifically, MOCA meta-trains on an offline timeseries, and keeps the parameters θ fixed online, whereas an online learning algorithm would not have this train/test distinction, and would consider updating θ continuously (Hazan, 2016) .
However, in order to do this with MOCA, we would need to keep a running buffer of all data observed so far and to use as training data to update θ, which may be expensive in real-world domains where large volumes of data (e.g. high definition video from a large collection of cameras on an autonomous vehicle).
Extending MOCA toward either strictly online training or a scheme to maintain an efficient replay buffer (Mnih et al., 2013; Vitter, 1985) , is a promising direction of future work.
Indeed, it may be possible to use MOCA's changepoint analysis to inform which data to save.
Beyond the continual learning extension, data efficiency may be improved by re-using information from previous tasks or modeling task evolution dynamics.
Previous work (Nagabandi et al., 2019b; Jerfel et al., 2019; Knoblauch & Damoulas, 2018 ) has addressed the case in which tasks reoccur in both meta-learning and the BOCPD framework, and thus knowledge (in the form of a posterior estimate) may be re-used.
In this work, we address the case in which tasks are sampled i.i.d. from a (typically continuous) distribution, and thus knowledge re-use is often impractical or adds marginal value.
Broadly, moving beyond the assumption of i.i.d. tasks to task having associated dynamics (Al-Shedivat et al., 2018) represents a promising future direction.
Conclusions.
MOCA enables the application of existing meta-learning algorithms to problems without task segmentation, such as the problem setting of continual learning.
We find that by leveraging a Bayesian perspective on meta-learning algorithms and augmenting these algorithms with a Bayesian changepoint detection scheme to automatically detect task switches within time-series, we can achieve similar predictive performance when compared to the standard task-segmented metalearning setting, without the often prohibitive requirement of supervised task segmentation.
for the sinusoid with hazard 0.01.
The lowest hazard was used to increase the effects of the short training horizon.
A minor decrease in performance is visible for very small training horizons (around 20), but flattens off around 100 and above.
It is expected that these diminishing marginal returns will occur for all systems and hazard rates.
PCOC extends a line of work on meta-classification based on prototypical networks (Snell et al., 2017) .
This framework maps the context data to an embedding space, after which it computes the centroid for each class.
For a new data point, it models the probability of belonging to each class as the softmax of the distances between the embedded point and the class centroids, for some distance metric.
For Euclidean distances (which the authors focus on), this corresponds to performing frequentist estimation of class means, under the assumption that the variance matrix for each class is the identity matrix 2 .
Indeed, this corresponds to the cheapest-to-evaluate simplification of PCOC.
Ren et al. (2018) propose adding a class-dependent length scale (which is a scalar), which corresponds to meta-learning a frequentist estimate of the variance for each class.
Moreover, it corresponds to assuming a variance that takes the form of a scaled identity matrix.
Indeed, assuming diagonality of the covariance matrix results in substantial performance improvement as the matrix inverse may be performed element-wise.
This reduces the numerical complexity of this operation in the (frequently high-dimensional) embedding space from cubic to linear.
However, in our implementation of MOCA, we assume diagonal covariances throughtout, resulting in comparable computational complexity to the different flavors of prototypical networks.
If one were to use dense covariances, the computational performance decreases substantially (due to the necessity of matrix inversions), especially in high dimensional embedding spaces.
In contrast to this previous work, PCOC has several desirable features.
First, both Snell et al. (2017) and Ren et al. (2018) make the implicit assumption that the classes are balanced, whereas we perform online estimation of class probabilities via Dirichlet posterior inference.
Beyond this, our approach is explicitly Bayesian, and we maintain priors over the parameters that we estimate online.
This is critical for utilization in the MOCA framework.
Existence of these priors allows "zero-shot" learning-it enables a model to classify incoming data to a certain class, even if no data belonging to that class has been observed within the current task.
Finally, because the posteriors concentrate (the predictive variance decreases as more data is observed), we may better estimate when a change in the task has occurred.
We also note that maximum likelihood estimation of Gaussian means is dominated by the James-Stein estimator (Stein, 1956) , which shrinks the least squares estimator toward some prior.
Moreover, the James-Stein estimator paired with empirical Bayesian estimation of the prior-which is the basis for Bayesian meta-learning approaches such as ALPaCA and PCOC-has been shown to be a very effective estimator in this problem setting (Efron & Morris, 1973 ).
Note that each column corresponds to one trained model, and thus the randomly varying performance across train supervision rates may be explained by simply results of minor differences in individual models.

assistant: text: This paper pushes meta-learning towards task-unsegmented settings, where the MOCA framework adopts a Bayesian changepoint estimation scheme for task change detection.