## 4 Development of Propulsion Power Prediction Model  
### Support Vector Regression (SVR)  
As a machine learning model to predict the propulsion power of the vessel, SVR is applied. SVR, which was introduced by [PERSON] et al. [27], is an extension of support vector machine (SVM) that considers a regression problem as well. Like SVM, SVR tries to determine the hyperplane that maximizes the margin while ensuring that the error is tolerated. Suppose that a training data \(\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),\ldots,(\mathbf{x}_{n},y_{n})\}\) is given, SVR assumes the relationship between \(\mathbf{x}\) and \(y\) represented by the following:  
\[f(\mathbf{x})\ =\ \mathbf{w}^{T}\varphi(\mathbf{x})+b \tag{7}\]  
Figure 2: Data collection system in Voyage Data Recorder (VDR). where \(\varphi(\mathbf{x})\) denotes a kernel function that transforms the data into a higher dimensional space to make it possible to perform the linear separation, w is weight vector associated with vector \(\mathbf{x}\) and \(b\) is coefficient. To find good estimator, SVR tries to find linear function that is as flat as possible. This can be formulated as a convex optimization problem and formulated as follows:  
\[J(\mathbf{w})\ =\ \frac{1}{2}\mathbf{w}^{T}\mathbf{w} \tag{8}\]  
subject to \(\forall n:\left|y_{n}-f(x_{n})\right|\leq\varepsilon\). As shown in the above equation, SVR tries to minimize the norm of w while ensuring all residuals having a value less than \(\varepsilon\). However, functions \(f(\mathbf{x})\) may not be exist for satisfying the above condition. To deal with such an infeasible constraint, we slightly modify the problem as follows:  
\[J(\mathbf{w})\ =\ \frac{1}{2}\|\mathbf{w}\|^{2}+C\sum_{i\ =\ 1}^{N}\ ( \xi+\xi^{*}) \tag{9}\]  
subject to:  
\[ \begin{array}{c}\forall i:y_{i}-J(\mathbf{w})\leq\varepsilon+\xi_{i}\\ \forall i:J(\mathbf{w})-y_{i}\leq\varepsilon+\xi_{i}^{*}\\ \forall i:\xi_{i}\geq 0\\ \forall i:\xi_{i}^{*}\geq 0.\end{array} \]  
The slack variables \(\xi_{i}\) and \(\xi_{i}^{*}\) serves as soft margin that allows the regression error of each data point \(i\) to exist up value \(\xi_{i}\) and \(\xi_{i}^{*}\). The constraint \(C\) controls the overfitting of the model by imposing a penalty on observations that lie outside the margin \(\varepsilon\). The solution of Equation (7) can be optimized by solving a dual problem that is computed as follows:  
\[L(a)\ =\ \frac{1}{2}\sum_{i\ =\ 1}^{n}\sum_{j\ =\ 1}^{n}\ (a_{i}-a_{i}^{*})(a_{j}-a_{j}^{*})K(x_{ i},x_{j})\ +\ \varepsilon\sum_{i\ =\ 1}^{n}\ (a_{i}+a_{i}^{*})-\sum_{i\ =\ 1}^{n}\ y_{i}(a_{i}+a_{i}^{*}) \tag{10}\]  
subject to:  
\[ \begin{array}{c}\sum\limits_{i\ =\ 1}^{n}\ (a_{i}-a_{i}^{*})\ =\ 0\\ \forall n:0\leq a_{i}\leq C\\ \forall n:0\leq a_{i}\leq C\end{array} \]  
The function \(K(x_{i},x_{j})\ =\ \zeta\varphi(x_{i}),\varphi(x_{j})>\) is called as kernel function where \(\varphi(x_{i})\) is a transformation that maps \(x_{i}\) to a high-dimensional space. If we assume a linear model, linear kernel function \(K(x_{i},x_{j})\ =\ x_{i}^{T}x_{j}\) is utilized. If the non-linear relationship is assumed, radial basis function (RBF) kernel \(K(x_{i},x_{j})\ =\ \exp(-\|x_{i}-x_{j}\|^{2})\) or sigmoid kernel \(K(x_{i},x_{j})\ =\ \tanh(\gamma(x_{i}^{T}x_{j})+\theta)\) are widely adopted. Now the function that is used to predict new value \(x_{n}\) is equal to Equation (9):  
\[f(x)\ =\ \sum_{i\ =\ 1}^{n}\ (a_{n}-a_{n}^{*})K(x_{n},x)+b \tag{11}\]  
The predictive performance of SVR is significantly affected by the hyper-parameters. If we assume RBF as the kernel function, the performance of SVR is affected by hyper-parameters \(C\), \(\varepsilon\) and \(\sigma\). Parameters \(\varepsilon\) and \(C\) affect the way of penalizing the training error. The parameter \(\varepsilon\) controls the amount of error allowed in the model. If the residual is larger than \(\varepsilon\), then the training error is penalized by \(C\).