# Continuous Adaptation with Online Meta-Learning for Non-Stationary Target Regression Tasks
## 1 Introduction  
Adapting to a non-stationary environment is important for applying many machine learning algorithms to the real world [1]. Most environments exhibit some sort of variations over time, e.g., physical mechanisms may wear out and change their characteristics, a task related to personal assistance (email classification [2], sentiment classification [3], etc.) also changes over time due to variations in the person's status, and any task in a competitive environment also varies over time as its competitors learn and improve their behaviours [4].  
In this work, we consider a task performed on a stream of data in such a non-stationary environment. This requires the model to learn as it receives data to perform better for the new incoming data. We assume that there is some prior knowledge (or data) about the task. The knowledge or data do not need to be accurate and precise since our model exploits them as long as they give an indication about the task. The data may be synthetically generated with a simulator, which is unlikely to match the real target. Having prior knowledge (or data) is not a mandatory requirement for using our method--we can incorporate prior knowledge whenever it exists.  
We tackle the task by learning the model from the data in a sliding window. As it turns out, this method does not perform well for complex target models, requiring much more data than the data available in the window. We resolve this issue by introducing Bayesian meta-learning [5, 6], which can also be seen as hierarchical Bayes [7] or empirical Bayes [8]. The empirical Bayes approach learns the model parameter priors from all available data. Because it learns the prior distributions rather than a point estimate, Bayesian meta-learning is robust against overfitting and can introduce uncertainties over the priors, which helps optimally combine the priors and training data to compute the posterior distribution of the parameters in the prediction stage.  
Our approach consists of three stages of learning. The first stage is offline meta-learning that learns the prior distribution with available data before the deployment. Thesecond stage is online meta-learning that improves the prior to better fit the actual target. The third stage is Bayesian regression with the sliding window, which learns a model to compute a predictive posterior distribution with the time-windowed data. We propose a computationally efficient method for the online learning parts (the second and third stages), which is important for deploying the algorithm in practice.  
The meta-learning approach is common in the few-shot learning domain [9, 10], and previous work exists. However, they consider discrete switching between tasks, and often offline learning algorithms. In contrast, our target task is a single task within a gradually changing environment. Our approach provides offline and online meta-learning to learn the priors from the data available before the task and after the deployment. There are a few works in the few-shot learning domain with online meta-learning. We discuss them in Section 4.  
Our contributions are summarised as follows:  
* We propose a method to adapt to a non-stationary target (concept drift) consisting of two components: Offline/Online Bayesian meta-learning (empirical Bayes) and Bayesian regression with sliding window.
* We show that these components work complementarily and achieve good performance for both stationary and non-stationary tasks.
* We decompose the algorithm with a computationally efficient recursive form.  
This paper is organised as follows. Section 2 introduces some background information for Bayesian linear regression and meta-learning. In Section 3, we introduce our method--concept-drift online recursive adaptive learning (CORAL). In Section 4, we discuss connections between our approach and the other related works. Section 5 presents our evaluation tasks and the results. Finally, Section 6 provides a conclusion of our work.