# Continuous Adaptation with Online Meta-Learning for Non-Stationary Target Regression Tasks
## 5 Evaluation
### Toy Problem  
First, we test CORAL on a simple non-stationary task, which clearly shows how algorithms that do not consider the non-stationarity struggle and how our approach helps in such a scenario. The task is predicting \(y\in\mathbb{R}\) from \(x\in\mathbb{R}\), and the model is simply \(y=ax+b\), where \(a\) and \(b\) are coefficients and both in \(\mathbb{R}\). \(x\) is sampled from a Gaussian distribution with mean zero and variance 100. At each time step, the algorithm receives the current \(x_{t}\) and generates the prediction \(\hat{y}_{t}\). It also receives the previous time step's target value \(y_{t-1}\) with some observation noise \(e_{t}\) to update the prediction model. The observation noise is normally distributed with mean 0 and variance 2. We test the model while slowly varying the coefficients \(a\) and \(b\) as follows:  
\[ \begin{split} a_{t}&=\sin(2\pi t/200)+t/2000-1.0\\ b_{t}&=\cos(2\pi t/200)+t/2000+2.0,\end{split}  \tag{13}\]  
where \(t\) is the time step starting from zero. Hence, both coefficients oscillate with a sample period of 200 time steps and also slowly increase their baselines. For the prediction model, we use \([x_{t},1.0]\) as the basis function (features) to reproduce the target model. We test three models: (1) using neither the sliding window nor the online meta-learning, (2) with sliding window (length 5) and without online meta-learning, (3) with sliding window (length 5) and online meta-learning. We set the prediction stage period \(m\) equal to the sliding window size, which is five. In this evaluation, we do not use offline meta-learning because the target model is very simple (only two parameters to estimate). The priors are initially set to a standard normal distribution. The posteriors of the coefficients (\(a\) and \(b\)) for the three models are shown in Figure 3 as well as their true values. The results without the sliding window (a) show that both \(a\) and \(b\) converge towards an average over the past true values. Furthermore, their variances are monotonically reduced to zero as it receives data. These behaviours are not favourable for the non-stationary target case, as they converge to wrong values, and they also become overconfident (underestimate its variances). The results of the model with a sliding window and without meta-learning (b) show that the posterior for \(a\) converges to the right value and variances. However, it struggles to achieve a good posterior for \(b\), especially for later time steps. This is because the true \(b\) values at the later time steps are more apart from their prior distribution; hence, it fails to move the posterior to the true value with the limited number of data in the sliding window. The results (c) from the model with the sliding window and online meta-learning mitigate the problem in the parameter \(b\) posterior. This is because the model with online meta-learning updates its prior distributions to keep up with the long-term variation. These priors are shown in Figure 4, where (a) shows the priors of the parameters \(a\) and \(b\) with neither meta-learning nor the sliding window, and (b) shows the priors without meta-learning but with a sliding window. They both do not use online meta-learning; hence, the priors stay in their initial setting (normal distribution.) Additionally, (b) shows the priors with meta-learning. We can see that they adapt to the mean level drift (slow drift). It does not adapt to the oscillations (fast drift), but their variances correctly cover the range of oscillations. Figure 5 shows the mean squared error (MSE) with a 200 time step moving average for the three prediction models. It clearly shows that the model without the sliding window fails in this non-stationary task. With the sliding window, the prediction accuracy improves dramatically; however, the result without online meta-learning (orange line) slowly worsens as the true parameter value deviates from its prior distribution.