# Continuous Adaptation with Online Meta-Learning for Non-Stationary Target Regression Tasks
## 3 The CORAL Algorithm
### Online Meta-Learning  
We now extend our approach to incorporate online meta-learning--incrementally learning the prior parameters \(\Lambda_{0},\overline{K}_{0}\) while receiving a data stream. Online meta-learning is advantageous when no data from the actual target models is available before deployment. In such cases, offline meta-learning can only learn weak priors. In contrast, online meta-learning can improve the prior based on data obtained in the actual deployed environment. Having the better prior may not be so significant in the stationary target case because the predictive posterior can use all observed data to improve its accuracy. However, in our non-stationary scenario, it is essential to have a good prior as the predictor can only rely on a limited number of observation samples. We propose a computationally efficient online meta-learning method that works well with the sliding window target adaptation method described in the previous section. Computational efficiency becomes essential for online learning settings as it requires working in real-time and often working on a device with limited computational power. The online meta-learning could use the same algorithm as the offline meta-learning described in Section 2.2. However, a straightforward implementation of offline meta-learning is computationally expensive. This is because the recursive methods (Equations (4) and (7)) assume that the prior parameters (\(\Lambda_{0}\) and \(\overline{K}_{0}\)) are fixed. Observe that the prior parameters are used to initialise the matrix \(\Lambda_{\tau}^{-1}\) and \(Q_{\tau}\), and they are updated with new data. If the prior parameters change, the recursive update of \(\Lambda_{\tau}^{-1}\) (Equation (5) or Equation (8)) requires a \(n_{\phi}\times n_{\phi}\) matrix inversion, which is computationally expensive. Our method consists of two stages, as shown in Figure 1. The first stage is the preparation stage, which computes \(\Lambda_{\tau}^{-1}\) and \(Q_{\tau}\) from the updated prior. By exploiting the fact that they depend only on the prior and the past \(n\) time steps' data (the data in the sliding window), they can be computed as follows. First, \(\Lambda_{\tau-1}\) and \(Q_{\tau-1}\) are initialised with the updated prior parameters, \(\Lambda_{0}^{-1}\) and \(\Lambda_{0}\overline{K}_{0}\), respectively, then the following update for \(n\) time steps is recursively applied, which are similar to Equations (4) and (5). \[\Lambda_{\tau}^{-1} =\Lambda_{\tau-1}^{-1}-\frac{(\Lambda_{\tau-1}^{-1}\phi(x_{\tau} ))(\Lambda_{\tau-1}^{-1}\phi(x_{\tau}))^{T}}{1+\phi(x_{\tau})^{T}\Lambda_{\tau -1}^{-1}\phi(x_{\tau})} \tag{9}\] \[Q_{\tau} =Q_{\tau-1}+\phi(x_{\tau})y_{\tau}^{T}.\]  
The second stage is the prediction stage, which updates \(\Lambda_{\tau}^{-1}\) and \(Q_{\tau}\), produces the predictive posterior and accumulates the prior updates. The \(\Lambda_{\tau}^{-1}\) and \(Q_{\tau}\) are updated with the recursive update equations with the sliding window, as shown in Equations (7) and (8). The update equations are as follows:  
\[\Lambda_{\tau}^{\prime-1} =\Lambda_{\tau-1}^{-1}-\frac{(\Lambda_{\tau-1}^{-1}\phi(x_{\tau} ))(\Lambda_{\tau-1}^{-1}\phi(x_{\tau}))^{T}}{1+\phi(x_{\tau})^{T}\Lambda_{ \tau-1}^{-1}\phi(x_{\tau})} \tag{10}\] \[\Lambda_{\tau}^{-1} =\Lambda_{\tau}^{\prime-1}-\frac{(\Lambda_{\tau}^{\prime-1}\phi( x_{\tau-n}))(\Lambda_{\tau-1}^{\prime-1}\phi(x_{\tau-n}))^{T}}{-1+\phi(x_{\tau-n})^{T} \Lambda_{\tau-n}^{\prime-1}\phi(x_{\tau-n})}\] \[Q_{\tau} =Q_{\tau-1}+\phi(x_{\tau})y_{\tau}^{T}-\phi(x_{\tau-n})y_{\tau-n} ^{T}.\]  
The predictive posterior computation becomes:  
\[\overline{K}_{\tau} =\Lambda_{\tau}^{-1}Q_{\tau} \tag{11}\] \[\Sigma_{\tau+1} =(1+\phi^{T}(x_{\tau+1})\Lambda_{\tau}^{-1}\phi(x_{\tau+1}))\Sigma _{e}\] \[p(y_{\tau+1}|\phi(x_{\tau+1}),\Phi,Y) =\mathcal{N}(\overline{K}_{\tau}^{T}\phi(x_{\tau+1}),\Sigma_{ \tau+1}).\]  
Finally, we compute the prior updates (online meta-learning) by applying SGD with the loss function shown in the offline meta-learning Equation (6).