## 2 Background
### SVM Algorithm  
The main idea of SVMs (support vector machines) is to minimize the structural risk and VC dimensionality theory in statistical learning [17; 18]. The SVM algorithm can be used to solve small sample problems and nonlinear regression models, which can overcome the problems of \"dimensionality catastrophe\" and \"ovelearning\" to a large extent [19; 20].  
For example, taking two types of data classification and letting the sample set be \(\{(x_{1},y_{1}),(x_{2},y_{2}),\cdots,(x_{i},y_{i})\}\), the optimal interface linear equation for the sample space can be expressed as follows:  
\[\omega^{T}\cdot x+b=0 \tag{1}\]  
where \(\omega\) denotes the normal vector and \(b\) denotes the displacement term [21; 22].  
\(\omega^{T}\cdot x\): This is the inner product (dot product) operation, which multiplies the weight vector \(\omega\) and the eigenvector \(x\) and adds their corresponding elements. The result of this part of the computation represents a linear combination between the hyperplane and the eigenvector \(x\). The result of this part of the computation represents a linear combination between the hyperplane and the eigenvector \(x\).  
\(=0\): This is the equational part of the equation. The regression hyperplane is defined as the hyperplane that satisfies this equation. In SVR, the goal of this equation is to make the hyperplane fit the training data points as closely as possible, and data points within the \(\varepsilon\)-tube are considered to fall on the hyperplane, with \(\varepsilon\) being a user-defined parameter.  
The goal of SVR is to minimize the error term and find a regression hyperplane such that the sum of its residuals with respect to the data points is as small as possible. This is why the SVR model uses this linear equation to describe the regression hyperplane, but by adjusting the weight vector \(\omega\) and the bias term \(b\) it can fit nonlinear data relationships.  
In summary, \(\omega^{T}\cdot x+b=0\) is the equation used in the SVR model to describe the regression hyperplane, where the weight vector \(\omega\) and the bias term \(b\) are determined by training the model to fit the training data and perform the regression.  
To ensure that the hyperplane (\(\omega\), \(b\)) can classify the training sample normally, the following should be satisfied:  
\[\begin{cases}\omega^{T}\cdot x_{i}+b\geq+1,y_{i}=+1\\ \omega^{T}\cdot x_{i}+b\leq-1,y_{i}=-1\end{cases} \tag{2}\]  
The exact derivation of the algorithm is described in Appendix A.  
Essentially, the SVR in the SVM is used to establish the margin of error, and predictions that fall within the margin of error are considered to be correct, which is different from the principle of the SVM model (maximum spacing between sample points closest to the hyperplane). The purpose of the SVR is to minimize the distance between the farthest sample point to the hyperplane.  
Support vectors can be categorized into two types, depicted as the first kind and second kind in Figure 1. The first kind of support vectors are data points that lie within the \(\varepsilon\)-tube, which is a user-defined width parameter used to determine the threshold of prediction error allowed by the model. The second kind of support vectors are data points that lie outside the \(\varepsilon\)-tube, i.e., those that are more than \(\varepsilon\) away from the regression hyperplane. It is important to help regularize the model, avoid overfitting, and maintain the complexity of the model.  
In summary, the first kind of support vectors and the second kind of support vectors represent different data points in SVR, and they are important for both model fitting and generalization performance. The first type of support vectors are used to construct regression hyperplanes, while the second type of support vectors help regularize the model and control the complexity of the model.