## 2 Materials and Methods
### Support Vector Machine  
Support Vector Machine theory is based on statistics which have a fundamental principle of estimating the optimal linear hyperplane in a feature space that maximally separates the two-mark groups or classes. SVM modeling geometrically finds an optimal hyperplane with the maximal interval to separate two groups or classes. The procedure for solving such a constraint problem is as follows [24; 25; 26; 27]:  
\[minimize_{w,b}\ \frac{1}{2}\ \|w\|^{2} \tag{4}\]  
Subject to:  
\(y_{i}\Big{(}w^{T}x_{i}+b\Big{)}\geq 1,\ i=1,2,3,\ldots,\ n,\ x=\) the feature vectors or the input pattern, \(w\) = the direction of the optimal hyperplane, \(b\) = bias  
To make rooms for errors, the optimization problem currently becomes:  
\[min_{w,b}\ \frac{1}{2}\ \|w\|^{2}+C\sum_{i=1}^{n}\varepsilon_{i} \tag{5}\]  
Subject to:  
\[y_{i}\Big{(}w^{T}x_{i}+b\Big{)}\geq 1-\varepsilon_{i},\ i=1,2,3,\ldots,\ n\]  
\[\varepsilon_{i}\geq 0,\ i=1,2,3,\ldots,\ n\]  
The Lagrange multiplier method assists us to obtain the two formulas, that is expressed in terms of the variable \(\alpha_{i}\):  
\[maximize_{a}\ \sum_{i=1}^{n}\alpha_{i}-\frac{1}{2}\sum_{i=1}^{n}\sum_{j=1}^{n }\alpha_{i}x_{j}y_{i}y_{j}x_{i}^{T}x_{j} \tag{6}\]  
Subject to:  
\[\sum_{i=1}^{n}\alpha_{i}y_{i}=1=0,\ 0<\alpha<C\]  
For all \(i=1,\ 2,\ 3\ldots,\ n\)  
The linear classifier based on a linear discriminant function is then given as:  
\[f(x)=\sum_{i=1}^{n}\alpha_{i}x_{i}^{T}x+b \tag{7}\]  
A non-linear classifier sometimes assists in providing better accuracy in many applications. A fragile way of making a non-linear classifier out of linear classifier is by mapping out data from the input space X to a feature space F based on a non-linear function \(\varnothing:X\to F\). Using kernel function, in the space F, the optimization assumes the following:  
\[maximize_{a}\ \sum_{i=1}^{n}\alpha_{i}-\frac{1}{2}\sum_{i=1}^{n}\sum_{j=1}^{n} \alpha_{i}\alpha_{j}y_{i}y_{j}k\big{(}x_{i},x_{j}\big{)} \tag{8}\]  
Subject to:  
\[\sum_{i=1}^{n}y_{i}\alpha_{i}=0,\ 0<\alpha<C\]  
For all \(i=1\), \(2\), \(3\)..., n  
The classifier \(K(x,\ x^{\prime})=\ (x.\ x^{\prime})^{d}\) is given as a SVM-Polynomial, where, if d is large, the kernel still requires n multiplications to compute. So, given  
\[K(x,\ x^{\prime})=\ (x.\ x^{\prime})^{2}=\ \big{(}(x_{1}\,\ x_{2}\ ). \big{(}x_{1}^{\prime}\,x_{2}^{\prime}\big{)}\big{)}^{2} \tag{9}\]  
The RBF classifier is given as  
\[K(x,\ x^{\prime})=\ \exp(-\gamma\big{\|}x-x^{\prime}\big{\|}^{2} \tag{10}\]  
In this current research paper, we implemented (7), (9) and (10) as the SVM kernels which represent Linear, Polynomial and Radial Basis Function (RBF) respectively. The cost function (C) presents the measure of how wrong a model is in terms of its ability to estimate the relationship between x and y as seen from (11). The cost parameter was optimized by grid search in the training dataset. Given a Hypothesis \(\
eg\) H0 (x) = \(\beta\)0 + \(\beta\)1x, then the cost function (C) is represented as:  
\[C=\frac{1}{2N}\sum_{i=1}^{N}\big{(}H_{0}(x)-y^{(i)}\big{)} \tag{11}\]  
where \(N\) is the number of observations.  
In developing the SVM classifiers, the values of the cost function used were (0.1, 1, 10, 100).