## 3 Methodology
### Classification  
A SVM is a supervised learning algorithm, used for classification and regression, originally described by [PERSON] and [PERSON] [PERSON] and [PERSON] (2006). An SVM classifies data points of different class by searching for the best hyperplane separating them. Starting with a training dataset, \(X\), comprising \(I\) training samples, \(X=x_{1}\), \(x_{2}\),..., \(x_{i}\), and target labels, \(y=y_{1}\), \(y_{2}\),..., \(y_{i}\), where \(y_{i}\)\(\xi\) {\(-1\), \(+1\)}, \(i\) = [1,2,..., \(I\)]. The equation of the linear decision hyperplane, \(f(x)\), can be defined:  
\[f(x)=\left(w^{T}\cdot x\right)+bias \tag{5}\]  
where \(w\) is the weight vector or direction of the hyperplane, and bias is the position in the space. To find the best hyperplane of the binary classification of TB and normal, candidate decision surfaces were normalized so that the value of the decision hyperplane \(f(x)\) for the support vectors is \((w^{T}\cdot x)\) + \(b\) = \(+1\) for TB class and \((w^{T}\cdot x)+b=-1\) for normal class. The best hyperplane is the one with the largest margin between the two classes. The maximum margin between two classes is equivalent to minimizing \(\|\)\(w\)\(\|^{2}\). Therefore, the best separating hyperplane is defined:  
\[\text{Minimize}:\|\) w \(\|^{2}\] (6) subject to : \[y_{i}\left[\left(w^{T}\cdot x_{i}\right)+bias\right]\geq 1,\ i=1,2, \ldots,I \tag{7}\]  
In a real application, the training data set is not usually linearly separable because some data points may fall inside or behind the margin, or wrong side of the decision hyperplane. Let denote \(\xi\), \(\xi\) {\(\xi_{1}\), \(\xi_{1}\),..., \(\xi\);i} as a vector of the error points for \(I\) training samples. The decision hyperplane for not linearly separable data is:  
\[\text{Minimize}:\|\) w \(\|^{2}+C\sum_{i=1}^{1}\xi_{i}\] (8) subject to : \[y_{i}\left[\left(w^{T}\cdot x_{i}\right)+bias\right]\geq 1-\xi_{i},\ \ i=1,2, \ldots,I \tag{9}\]  
where \(\xi_{i=0}\), \(0<\xi_{i}<1\), and \(\xi_{i}>1\) are error points, \(C\) is a penalty parameter used for minimizing the errors falling inside or on the other side of the margin.  
When the data is not linearly separable, SVM maps the feature space to a higher dimension using the mapping function or 'kernel trick' \(\varphi\): \(K\left(x_{j},x_{k}\right)=\)\(<\varphi\left(x_{j}\right)\), \(\varphi\left(x_{k}\right)>\). Two types of mapping functions, i.e., local (Gaussian radial basis function) and global (linear or polynomial) functions are commonly used for SVM and shown in Equations (10) to (13):\[K\big{(}x_{j},x_{k}\big{)}=x_{j}{}^{T}\cdot x_{k} \tag{10}\]  
Gaussian radial basis  
\[K\big{(}x_{j},x_{k}\big{)}=\exp\left(-\gamma\parallel x_{j}-x_{k}\parallel^{2} \right)\text{, }\gamma>0\text{, }\gamma=\frac{1}{2\sigma^{2}} \tag{11}\]  
Quadratic polynomial  
\[K\big{(}x_{j},x_{k}\big{)}=\left(x_{j}{}^{T}\cdot x_{k}+1\right)^{2} \tag{12}\]  
Cubic polynomial  
\[K\big{(}x_{j},x_{k}\big{)}=\left(x_{j}{}^{T}\cdot x_{k}+1\right)^{3} \tag{13}\]  
Using the mapping functions, a non-linear decision surface is defined mathematically:  
\[\text{f}(x)=\sum_{i=0}^{S^{\
u}}\alpha_{i}y_{i}\;\Phi\big{(}x_{j}\big{)}\Phi(x _{k})+\text{bias} \tag{14}\]  
where \(S^{\
u}\) denotes the number of support vectors, \(\alpha_{i}\) and \(y_{i}\), represent the Lagrange multipliers and target labels associated with \(S^{\
u}\), respectively. \(x_{j}\). and \(x_{k}\). represent the observations \(j\) and \(k\) in the training set \(X\)[53].  
The performance of an SVM relies heavily on the choice of its parameters. Optimal values of the SVM parameters were searched using the Bayesian algorithm [54], as shown in Figure 8. We first defined the initial parameter search space and used the Bayesian method to iteratively search for the optimal values until the maximum criteria reached or the validation accuracy unchanged.  
Figure 8: Workflow for support vector machine (SVM) parameter tuning using a Bayesian algorithm.