# Optimal Prediction of Individual Vessel Trajectories Based on Sparse Gaussian Processes
## 3 Preliminaries
### Sparse Gaussian Process Regression  
Sparse Gaussian process regression (SGPR) improves the scalability of full Gaussian process regression (FGPR) [7], enabling the application of Gaussian processes (GPs) to big data. In this study, SGPR is applied using SoD, SoR, and FIC among the global approximation methods for the \(n\times n\) kernel matrix \(K_{nn}=K(\mathbf{X},\mathbf{X})\). Table 1 shows the time complexities of FGPR and SGPR. As shown in the table above, FGPR has the highest time complexity and storage costs. As the size of the training data increases, the time complexity and storage cost increase rapidly. Therefore, FGPR is usually applicable only to training datasets of approximately 1000 to 3000 data points [8]. SoD is the simplest SGPR method that selects a subset with size \(m\) of training data and uses a subset instead of the entire set of training data. Using this method, the full kernel matrix \(K_{mn}\) obtains a smaller kernel matrix \(K_{mm}\) which increases the time complexity and storage cost. In addition, the SoR models and FIC were applied as a sparse approximation method using inducing points \(m\) to the full kernel matrix, which causes considerable storage and high time complexity. SoR models apply a rank (at most) \(m\) kernel instead of the kernel function \(k\big{(}x_{i},x_{j}\big{)}\) of the FGPR method, as follows:  
\[k_{SoR}\big{(}x_{i},x_{j}\big{)}=k(x_{i},X_{m})K_{mm}^{-1}k\big{(}X_{m},\ x_{j} \big{)}. \tag{11}\]  
FIC approximates the kernel matrix as \(k_{SoR}\big{(}x_{i},x_{j}\big{)}\) plus diagonal matrix as follows:  
\[k_{FIC}\big{(}x_{i},x_{j}\big{)}=k_{SoR}\big{(}x_{i},x_{j}\big{)}+\delta_{ij} \big{[}k\big{(}x_{i},x_{j}\big{)}-k_{SoR}\big{(}x_{i},x_{j}\big{)}\big{]}, \tag{12}\]  
where \(\delta_{ij}\) is the Kronecker delta. If \(i\
eq j\), the FIC approximation is equal to the SoR approximation. If \(i=j\), kernel function \(k\big{(}x_{i},x_{j}\big{)}\) of FGPR is used. To use these approximation methods, it is necessary to specify the selection of the subset data or induce points from the entire dataset. Selection methods include random selection, clustering-based selection [33; 34], and forward selection based on information-theoretic principles [35]. Random selection, which is simpler and more scalable than other methods, was used in this study [9]. The outline for sparse Gaussian process regression algorithm is presented as follows (Algorithm 2). ```
Input: Dataset \((X,Y)=\{(x_{i},\ y_{i})\}_{i=1}^{N}\), inducing points \(Z=\Big{\{}z_{j}\Big{\}}_{j=1}^{M}\), prior distribution \(p(f|Z)\) over the latent function values \(f\), likelihood function \(p(y|f)\), kernel function \(k(\theta)\) with hyperparameters \(\theta\), approximation method \(M_{a}\), maximum iterations \(T\), convergence threshold \(\varepsilon\). Output: Posterior distribution \(p(f|X,Y)\), predicted values \(\overline{y}\) for new input \(x_{*}\), optimized kernel parameters \(\theta\). 1. Initialize\(Z_{0}\), \(\theta_{0}\), \(T\), and \(\varepsilon\). 2. for each iteration \(t=0\) to \(T\)
3. for each data point \((x_{i},\ y_{i})\)
4. Compute the sparse kernel matrices \(K_{X,Z}K_{Z,Z}^{-1}K_{Z,X}\). 5. Compute the approximate posterior distribution \(q(f|Z)\) using \(M_{a}\). 6. Compute the likelihood of the observation \(p(y_{i}|f_{i})\). 7. Update \(Z_{t}\) and \(q(f|X,\ Y)\). 8. Maximize the marginal likelihood with respect to \(\theta_{t}\), updating \(\theta_{t+1}\). 9. endfor
10. if\(|\theta_{t+1}-\theta_{t}|<\varepsilon\), terminate and return \(\theta_{t+1}\). 11. else\(t\gets t+1\)
12. endfor
13. Return\(p(f|X,Y)\), \(\overline{y}=\mathbb{E}[f(x_{*},\ Z)]\), and \(\theta_{T}\). ```  
**Algorithm 2**Sparse Gaussian Process Regression (SGPR)