## 2 Methodology
### Support Vector Machine (SVM)  
A support vector machine is a popular machine-learning classifier that is used in mathematical and engineering problems, such as speech recognition, image processing, text classification, and natural language processing. The simplest form is a binary SVM classifier: One sorting instances into two classes. Let us suppose there is a sample set \(S\) consisting of \((x_{i},y_{i})\), where \(i=1,2,3,\ldots N\)\(x_{i}\in\mathbb{R}^{n}\). \(N\) represents the length of the feature vectors \(x_{i}\) and \(y_{i}\). Here, every \(x_{i}\) is an \(n\)-dimensional feature vector that belongs to one of the two classes, \(C_{1}\) and \(C_{2}\), through \(y_{i}\), which defines the labels. Let us assume that the sample space \(S\) is highly linearly separable. There are several approaches to linearly separate instances into two classes, for example, the use of a multilayer perceptron. Solutions to such problems are infinite sets depending on certain key factors, like a learning algorithm and its parameters, initial weights \((w)\), and biases \((b)\). Here, the focus of interest is to find the hyperplane that has the optimal separating margins between the two classes. The optimal margins provide a wide boundary between the classes, thereby reducing the misclassification rate. If \(S\) is highly linearly separable, then there exists a hyperplane \(w^{T}x+b=0\) and a positive \(k>0\) that uniformly separate \(C_{1}\) and \(C_{2}\) (\(w^{T}x\) is the dot product of vector \(w\) and vector \(x\)). Therefore, any uniformly separating hyperplane \(\alpha\) can be represented by the following property:  
\[\left\{ \begin{array}{l}w^{T}x+b\geq 1,x\in C_{1}\\ w^{T}x+b\leq 1,x\in C_{2}\end{array} \right. \tag{1}\]  
A binary SVM classifier solves the following optimization problem:  
\[ \begin{array}{l}c\text{Minimize (with respect to}\alpha):\\ \sum\alpha_{i}-\frac{1}{2}\sum\sum\alpha_{i}\alpha_{i}y_{i}y_{i}y_{i}x_{i}^{T} x_{j},\\ \\ \text{csubject to :}\\ \sum_{i=1}^{N}\alpha_{i}y_{i}=0\text{ and }\alpha_{i}\geq 0,\ i=1,2,3, \ldots,N,\end{array}  \tag{2}\]  
where \(\alpha_{i}\) are the Lagrange multipliers. Figure 2 presents the mechanism of how a hyperplane in the SVM separates two classes (\(C_{1}\) and \(C_{2}\)) that are linearly separable. Interestingly, the SVM can be changed from a linear to a nonlinear classifier by introducing a kernel function into the objective function. According to [PERSON]'s theorem [22], if the data are not linearly separable into an input space \(I\), then it can be mapped to another feature space \(F\) where the data are linearly separable. In this case, the main objective is to find a map function \(\phi:I\to F\). The dot product in the new objective function is transformed into \(x_{i}^{T}x_{j}\rightarrow\Phi^{T}(x_{i})\Phi(x_{j})\). If the condition \(x_{i}^{T}x_{j}\rightarrow\Phi^{T}(x_{i})\Phi(x_{j})\) is met by a kernel function \(K\), then the optimization problem will entirely depend on the \(K\) function. Therefore, in the new feature space, we need to find \(F\) linear margins. Thus, the optimization conditions can be generalized as follows:  
\[ \begin{array}{l}c\text{Minimize (with respect to}\alpha):\\ \sum_{i=1}^{N}\alpha_{i}-\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}\alpha_{i} \alpha_{j}y_{i}y_{j}K\big{(}x_{i},x_{j}\big{)},\\ \\ \text{csubject to :}\\ \sum_{i=1}^{N}\alpha_{i}y_{i}=0\text{ and }0\leq\alpha_{i}\geq C,\ i=1,2,3, \ldots,N,\end{array}  \tag{4}\]  
Here, the kernel function plays an important role. There are several kernel functions available for developing a nonlinear SVM. The most commonly used kernel functions are a dot product kernel,polynomial kernel, radial basis (Gaussian) kernel, and sigmoid kernel. In this work, we chose a Gaussian kernel function for training and testing the SVM. Figure 3 shows a hyperplane drawn between two nonlinear separable classes by using a Gaussian kernel function. Figure 3: SVM demo for separating two nonlinearly separable classes via a hyperplane by using a Gaussian kernel function.