## 2 Background
### State-of-the-Art Algorithms
Support Vector Machines (SVM)Conventional ML classifiers are strongly affected by the high dimensionality of the features observation vectors, and they tend to overfit to the data in the presence of noise, or to perform poorly with few training samples. In the last few years, the use of SVM [28; 29] for ML practical applications has received wide attention. SVM are supervised learning models with associated learning algorithms that analyze data used for classification (SVC) and regression (SVR) analysis. SVM constructs a hyperplane or set of hyperplanes in a high or infinitely dimensional space. Intuitively, good separation is achieved by the hyperplane that has the largest distance to the nearest training data-point of any class, since in general the larger the margin, the better the generalization error of the classifier [30,31]. In contrast to previous ML algorithms, SVM maps the input vector to a higher-dimensional space. SVM can solve linear and non-linear problems, and works well for many practical problems. For this work we have used a linear kernel. We want to find the maximum-margin hyper plane that divides the group of points \(\mathbf{x}_{k}\), for which \(\mathbf{y}_{k}=1\) from the group of points \(\mathbf{x}_{m}\), for which \(\mathbf{y}_{m}=-1\). Given a nonlinear mapping \(\phi(\cdot)\), the SVM method solves:  
\[\min_{\mathbf{w},\mathbf{b},\beta_{l},\rho}\left\{\frac{1}{2}\|\mathbf{w}\|^{2 }+C\sum_{i}\beta_{l}\right\} \tag{5}\]  
In consideration of  
\[\mathbf{y}_{l}(\langle\phi(\mathbf{x}_{l}),\mathbf{w}\rangle+b\geq 1-\beta_{l}), \ \forall l=1\ldots L \tag{6}\]  
where \(\mathbf{w}\) and \(b\) define a linear classifier in the feature space and \(\beta\) are positive slack variables enabling one to deal with permitted errors. The appropriate choice of nonlinear mapping \(\phi\) guarantees that the transformed samples are more likely to be linearly separable in the feature space. Equation (5) is solved by using its dual problem counterpart, yielding \(\mathbf{w}=\sum_{l=1}^{L}y_{l}\alpha_{l}\phi(\mathbf{x}_{l})\), and the decision function for any test vector \(\mathbf{x}_{*}\) is finally given by  
\[f(\mathbf{x}_{*})=sgn\left(\sum_{l=1}^{L}y_{l}\alpha_{l}K(\mathbf{x}_{l}, \mathbf{x}_{*})+b\right) \tag{7}\]  
where \(\alpha_{l}\) are Lagrange multipliers corresponding to constraints in Equation (5), those training samples \(\mathbf{x}_{l}\) with \(\alpha_{l}\
eq 0\) being vectors; the bias term \(b\) is calculated by using the unbounded Langrange multipliers; and \(K\) represents the Mercer Kernels used to handle the nonlinear algorithm implementations. Gradient Boosting (GB) is an ML technique for regression and classification problems that produces a prediction model in the form of an ensemble of weak prediction models. It builds the model in a stage-wise fashion, and it generalizes the models by allowing optimization of an arbitrary differentiable loss function [32, 33, 34]. The goal is to find an approximation \(\hat{F}(\mathbf{x})\) to a function \(F(\mathbf{x})\) that minimizes the expected value of some specified loss function \(L(\mathbf{y},\hat{F}(\mathbf{x}))\); that is,  
\[\hat{F}=\operatorname*{arg\,min}_{F}\mathbb{E}_{\mathbf{x},\mathbf{y}}[L( \mathbf{y},F(\mathbf{X}))] \tag{8}\]  
The GB method assumes a real-valued \(\mathbf{y}\) and seeks an approximated \(\hat{F}(\mathbf{X})\) in the form of a weighted sum of functions \(h_{m}(\mathbf{x})\) called base (or weak) learners, in such a way that  
\[\hat{F}(\mathbf{x})=\sum_{m=1}^{M}\gamma_{i}h_{i}(\mathbf{x})+c \tag{9}\]  
where \(M\) is the number of weak models being used.