## 2 Experimental Test and Machine Learning Technique
### Machine Learning Technique  
#### 2.3.1 Single Model: Support Vector Regression  
Support vector regression (SVR), proposed by [PERSON] et al. [33], is a kind of supervised learning technique This method has a generalization ability even with restricted number of dataset since it is based on the principle of structural risk minimum rather than empirical risk minimization [34]. Here, generalization ability means the state that is not overfitting for specific and biased data.  
SVR is not sensitive to margin error (\(\epsilon\)) and learns to include as many datasets as possible in a limited \(\epsilon\)-tube (Figure 7). Even if the training set is newly added, the errors are considered as zero within the allowed \(\epsilon\). However, if the training sets are outside the \(\epsilon\)-tube, a non-zero value of the slack variable (\(\zeta\)) occurs, which indicates the degree of margin violation. Based on the above principle, SVR can be composed of the optimization problem (Equation (3)) that has a global optimum point;  
\[\underset{\mathbf{w},\ b,\ \zeta}{\text{minimize}}\frac{1}{2}\mathbf{w}^{T} \mathbf{w}+C\sum_{i=1}^{m}\bigl{(}\zeta_{i}^{+}+\zeta_{i}^{-}\bigr{)} \tag{3}\]  
Equation (3) must be subject to the following conditions;  
\[y_{i}-\left(\mathbf{w}^{T}\cdot\phi(\mathbf{x}_{i})\right)\,+\,b\right)\,\leq \,\epsilon+\zeta_{i}^{+} \tag{4}\]  
\[y_{i}-\left(\mathbf{w}^{T}\cdot\phi(\mathbf{x}_{i})\right.\,+\,b\right)\,\leq \,\epsilon+\zeta_{i}^{+} \tag{5}\]  
\[\zeta_{i}^{+},\ \zeta_{i}^{-}\geq 0 \tag{6}\]  
where, \(\mathbf{x}\) and \(y\) are an input vector and actual output. \(\mathbf{w}\) is a weight vector of input vector. \(C>0\) is a regularization parameter indicating a penalty for the sample error exceeding the margin. A larger \(C\) value means a smaller slack variable is allowed, whereas a smaller \(C\) value means a larger slack variable is allowed. \(\phi\) is a function that maps the input space of the training set into a high-dimensional feature space. To solve the Equation (3), Lagrangian dual formulation, which includes a dot product between two \(\phi\) functions, should be calculated. Fortunately, the dot product between \(\phi\) functions can be easily calculated as a kernel function (\(K\)) as shown in Equation (7);  
\[K\bigl{(}\mathbf{x}_{i},\mathbf{x}_{j}\bigr{)}\ =\ \phi(\mathbf{x}_{i})^{T}\phi( \mathbf{x}_{j}) \tag{7}\]  
Figure 7: Concept of support vector regression. It learns to include as many datasets as possible in a limited \(\epsilon\)-tube and a constraint of the slack variable. Even if the training set is newly added, the errors are regarded as zero within the allowed \(\epsilon\). However, if the training set is outside the \(\epsilon\)-tube, a non-zero value of the slack variable (\(\zeta\)) occurs, which indicates the degree of margin violation.  
\(K\) plays the role of expanding the input space of the training sets into a high-dimensional space, which makes it possible to express a nonlinearity problem. Since the related theory is out of the scope of this study, it is recommended to refer to the literature [33]. There are several types of \(K\) established by researchers, and the functions used in this study are summarized in Table 3.