# Hybrid Oversampling and Undersampling Method (HOUM) via Safe-Level SMOTE and Support Vector Machine
## 3 Preliminaries
### Classification Techniques Used in the Present Study
#### 3.1.3 Support Vector Machine (SVM)  
An SVM [25] is a classification algorithm designed to identify the best decision boundary or hyperplane that separates two data classes. The objective is to maximize the margin between the classes, facilitating easier separation. SVMs can be categorized into two groups based on the nature of data separability: linear and non-linear.  
In this case, a linear hyperplane can be used to separate the data into two classes. The decision boundary for the linearly separable SVM is shown in Figure 1. The following symbols are included in the formulation of a linear SVM.  
\(\bigcirc\) **w**: weight vector in the hyperplane.  
\(\bigcirc\) **x**: input feature vector.  
\(\bigcirc\) **b**: bias scalar, representing the constant term in the hyperplane equation.  
\(\bigcirc\) **\(\|\textbf{w}\|\)**: Euclidean norm of the weight vector **w**.  
\(\bullet\)  Equation (2) represents the decision boundary for the linearly separable SVM:  
\[\textbf{{w}}\textbf{{x}}+b=0 \tag{2}\]  
In a linear SVM, **w** is perpendicular to the separating hyperplane, and b helps in positioning this hyperplane in the feature space.  
\(\bullet\)  The goal of the SVM is to maximize the margin, which can be represented in Equation (3)  
\[margin=\frac{2}{\|\textbf{{w}}\|} \tag{3}\]  
\(\bullet\)  The decision function (\(f(\textbf{{x}}_{i})\)) in Equation (4) assigns class labels based on which side of the hyperplane a data point lies:  
\[f(\textbf{{x}}_{i})=\left\{\begin{array}{ll}1,&if\textbf{{w}}\textbf{{x}}_{ i}+b\geq 1\\ -1,&if\textbf{{w}}\textbf{{x}}_{i}+b\leq-1\end{array}\right. \tag{4}\]  
In summary, a linear SVM aims to find the optimal hyperplane (\(\textbf{{w}}\textbf{{x}}+b=0\)) that separates classes by maximizing the margin between them. This optimization is achieved  
Figure 1: Hyperplane and margins for an SVM for samples with two classes.  
by adjusting the parameters \(w\) and \(b\) while considering the constraints represented by the class assignment function, as described in Equation (4).  
Non-Linearly Separable SVM (Radial)  
In most real-world problems, data cannot be linearly separated by a single hyperplane. To solve this problem, the data are mapped to a higher dimensional space, and then a hyperplane is defined there [34]. The following symbols are included in the formulations of a radial SVM.  
\(f(x)\): the classification function for input x.  
\(y_{i}\): class labels.  
\(a_{i}\): Lagrange multipliers obtained during SVM optimization.  
\(\varphi\left(x_{i}\right)\varphi\left(x_{j}\right)\): the dot product in a higher-dimensional space.  
\(K\left(X_{i},X_{j}\right)\): the kernel function \(K\) that computes the inner product without explicitly mapping data into higher dimensions.  
In Equation (5), the mapping solution is given by the SVM formula. In a non-linearly separable SVM, the quantities \(\varphi\left(x_{i}\right)\varphi\left(x_{j}\right)\) which need to be calculated are scalar products with vital properties. This is called the kernel function (K). When the kernel function is employed, the SVM is formulated as shown in Equation (6). Radial basis and sigmoid kernels are frequently used in studies. These kernels are shown by Equations (7) and (8), respectively [35]. \(\sigma\) in Equation (7) is a parameter that determines the spread of the Gaussian used. \(\delta\) in Equation (8) pertains to the sigmoid kernel function. This parameter affects the shape of the sigmoid function used for the kernel.  
\[f(x)=\text{sgn}\left(\sum_{i=1}^{i}y_{i}a_{i}\varphi(x_{i})\varphi\left(x_{j} \right)+b\right) \tag{5}\]  
\[f(x)=\text{sgn}\left(\sum_{i=1}^{i}y_{i}a_{i}K\left(x_{i},x_{j}\right)+b\right) \tag{6}\]  
\[\text{Radial basis}:K\left(X_{i},X_{j}\right)=exp\left(-\frac{1}{2\sigma^{2} }\parallel X_{i}-X_{j}\parallel^{2}\right) \tag{7}\]  
\[\text{Sigmoid kernel}:K\left(X_{i},X_{j}\right)=\tanh\left(kX_{i}X_{j}-\delta\right) \tag{8}\]  
In this study, both linear and radial basis kernels were tested with the SVM.