## 2 Problem Definition and Proposed Models
### Support Vector Machines  
SVM is a supervised machine learning model widely used for solving regression and classification problems. It was introduced by [PERSON] et al. at AT&T and Bell Laboratories [18]. SVM maps the training data into higher dimensional space with the help of a nonlinear mapping function also called the kernel function and then applies linear regression to separate data in high-dimensional space [19]. Data Separation is achieved by finding optimum hyperplanes, also called support vectors, that have maximum margins from the separated classes, as shown in Figure 4.  
The version of SVM used to solve the estimation problem is called SVR, which is an excellent choice and an efficient method as demonstrated by other related research [19]. Equations (2)-(12) show the principle workings of SVR from a mathematical perspective.  
While modelling, the objective of SVM is to find a linear decision function as shown in Equation (2):  
\[f(x)\leq\omega,\phi_{i}(x)>+b. \tag{2}\]  
Figure 4: SVM optimal hyperplane.  
Here, \(\omega\) represents the weight vector and \(b\) is a constant that has to be estimated from the considered dataset, whereas \(\phi\) is a kernel function or nonlinear mapping function. For the regression problem, the following risk function needs to be minimized:  
\[R(C)=\frac{C}{n}\sum_{i=1}^{N}L_{\varepsilon}(f(x_{i}),y_{i})+\frac{1}{2}\parallel \omega\parallel^{2}, \tag{3}\]  
where \(L_{\varepsilon}(f(x_{i}),y_{i})\) is the \(\varepsilon-intensive\ loss\ function\), which is given by the following equation:  
\[L_{\varepsilon}(f(x_{i}),y_{i})=\left\{\begin{array}{cl}|f(x)-y|-\varepsilon& |f(x)|\geq\varepsilon\\ 0&otherwise\end{array}\right. \tag{4}\]  
For the attainment of acceptable degree of error, slack variables \(\xi_{i}\) and \(\xi_{i}^{*}\) are used, which makes it a constrained minimum optimization problem, as shown in Equation (5):  
\[Min.R\left(\omega,\xi_{i}^{2}\right)=\frac{1}{2}\parallel\omega\parallel^{2}+ C\sum_{i=1}^{N}(\xi_{i}+\xi_{i}^{*}) \tag{5}\]  
subject to:  
\[\left\{\begin{array}{cl}y_{i}-<\omega,x_{i}>-b&\leq\varepsilon+\xi_{i}\\ <\omega,x_{i}>+b-y_{i}&\leq\varepsilon+\xi_{i}^{*}\\ \xi_{i},\xi_{i}^{*}&\geq 0\end{array}\right. \tag{6}\]  
where \(C\) is a regularized constant, which is greater than zero, and provides equilibrium between training error and flatness of model. \(C\) is also a penalty for the estimation of error (i.e., greater than \(\varepsilon\)). \(\xi_{i}\) and \(\xi_{i}^{*}\) are slack variables that represent the distance of actual values to corresponding boundary values of \(\varepsilon\). SVM tries to minimize \(\xi_{i}\), \(\xi_{i}^{*}\), and \(\omega^{2}\). A quadratic programming problem can be achieved by converting the above, optimization with constraint to Lagrangian multipliers as follows:  
\[f(x)=\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})K(x_{i},x)+b \tag{7}\]  
where \((\alpha_{i}\) and \(\alpha_{i}^{*})\) are Lagrange multipliers. Equation (7) is subject to following conditions:  
\[\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})=0 \tag{8}\]  
where \(0\leq\alpha_{i}\leq C,\ i=1,\ldots,n\) and \(0\leq\alpha_{i}^{*}\leq C,\ i=1,\ldots,n\). Here, \(K\) represents the kernel function, and its value can be obtained from the inner product of \(x_{i}\) and \(x_{j}\) in feature spaces \(\phi(x_{i})\) and \(\phi(x_{j})\), respectively, which satisfies [PERSON]'s rule. Therefore:  
\[K\big{(}x_{i},x_{j}\big{)}=\phi(x_{i}).\phi(x_{j}) \tag{9}\]  
In general, SVM has an advantage over MLR and ANN in high-dimensional space, especially when then the size of training data set is small because the support vectors rely only on a small subset of training data, hence giving a computational advantage to SVM over other classical techniques.