## Data Classification
### Brief Introductions to Selected Classifiers
#### 2.2.2 Support Vector Machines  
[PERSON] first proposed SVM for pattern recognition and classification [54]. By the inductive structural risk minimization principle of statistical learning theory [55,56], the SVM method is developed in such a way as to form one or a group of high or infinite dimensional hyperplanes [57,58].  
The fundamental objective of SVM is to find a hyperplane to separate the points of different classes in an \(n\)-dimensional space. The distance between the hyperplane and training data point is called the functional margin, which is used to indicate the confidence of classification results. As a maximum margin classifier, the hyperplane obtained in SVM with the highest functional margin to the closest training data point provides the best data separation. The nearest training data points are called Support Vectors.  
For a linearly separable problem as shown in Figure 3, the decision function is:  
\[f(\mathbf{x}) =\left(\sum\limits_{i=1}^{n}\alpha_{i}y_{i}\mathbf{x}_{i}\right)^{T} \mathbf{x}+b \tag{1}\] \[=\sum\limits_{i=1}^{n}\alpha_{i}y_{i}(\mathbf{x}_{i},\mathbf{x})+b\]  
where \(\mathbf{x}\) is an n-dimensional vector representing the sample to be classified, \(\mathbf{x}_{i}\) is the feature vector of training sample \(i\), \(y_{i}=1\) or \(-1\) if \(\mathbf{x}_{i}\) belongs to class 1 or 2, \(\alpha_{i}\) is Lagrange multiplier, and \(\langle_{r}\rangle\) denotes the inner product operation. The belonging state of \(\mathbf{x}\) can be obtained by the sign (positive or negative) of \(f(\mathbf{x})\) that calculated by function (1).  
Figure 3: An example linearly separable problem and the hyperplane that separates the data points.  
Figure 2: Demonstration of the \(K\)-nearest neighbors (KNN) method.  
Although practical classification problems are within finite dimensions, the actual sample data discriminations are not linear and involve more complex separation approaches in that finite space. As these samples are not linearly separable, the data dimension is mapped into a higher dimensional space for possible separation, as shown in Figure 4.  
Then, the decision function can be rewritten as:  
\[f(\mathbf{x})=\sum_{i=1}^{n}\alpha_{i}y_{i}(\phi(\mathbf{x}_{i}),\phi(\mathbf{x}))+b, \tag{2}\]  
where \(\phi\) indicates the mapping from the low original dimension to the higher dimension.  
In the new space with higher dimension, the hyperplanes are represented as the point sets with constant dot product in the presence of a vector. The hyperplanes themselves are vectors with linear properties. As the dot product computation in larger dimensional space could be complex, the kernel function is defined to address this issue [59; 60].  
A kernel function satisfies:  
\[K(\mathbf{x},\mathbf{z})=\langle\phi(\mathbf{x}),\phi(\mathbf{z})\rangle, \tag{3}\]  
so the inner product \(\langle\phi(\mathbf{x}_{i}),\phi(\mathbf{x})\rangle\) in a high dimension can be easily calculated by \(K(\mathbf{x}_{i},\mathbf{x})\) in a low dimension. Commonly used kernel functions include the linear kernel, polynomial kernel, radial basis function (RBF) kernel, sigmoid kernel, etc.  
Therefore, the accuracy of SVM is related to the definition of kernel parameters and the functional margin. Small data samples can sometimes be divided into an accurate separation with proper Support Vectors, but more data points may require more complex mappings.