## 3 Machine Learning (ML) Algorithms
### Support Vector Machine (SVM)  
The support vector machine (SVM) is a supervised ML algorithm that predicts a model significantly with less computational power [36]. It can be used for both regression and classification tasks [37]. SVR is derived from the SVM and is a prediction method that utilizes the principle of ML to optimize prediction precision while preventing overfitting the data automatically. Rather than the conventional empirical risk minimization (ERM) principle, the SVM is trained using the systemic risk minimization (SRM) principle [38]. Only the errors associated with the training dataset are minimized by means of ERM, but SRM simultaneously minimizes the empirical risk and model complexity. In other words, a tradeoff between the consistency of the approximation of the model and the sophistication of the approximation function is implied by the SRM principle. SVR thus prefers smooth models that do not overfit the training data--a requirement for successful unseen (testing) data generalization capabilities. The principles of SVR are similar to the SVM for classification, but there are a few differences between them. It requires a set of inputs and corresponding outputs to predict the model. The SVM finds a hyperplane in some features which separates the data points into two classes [39]. Many hyperplanes could be chosen to separate the data points. This choice can be made by observing the maximum margin distance between the data points of both classes. Samples on the margin are called support vectors. SVR finds a function  
Figure 5: RF algorithm with n decision trees. \(f(x)\) that has the most deviated margin of tolerance \((\varepsilon)\) from the actual target value \(y\) for all of the training data. The linear function is defined as Equation (3):  
\[f(x)\ =\ (w,\ x)+\ b \tag{3}\]  
where \(x\) is used to estimate the scaler vector of \(y\) by means of the n-dimensional weighting coefficient \(w\), and the constant coefficient is \(b\). The margin of tolerance can be minimized and calculated as:  
\[minimize\ \frac{1}{2}\ \|w\|^{2}\ +\ C\sum_{i=1}^{n}|\varepsilon_{i}| \tag{4}\]  
It is assumed that the function \(f\) estimates all of the pairs \((x_{i},y_{i})\) with precision through the use of Equation (4). If the data cannot be placed into the margin, the slack variables \(\xi_{i}\) can be used to solve the problem via Equation (5):  
\[|y_{i}\ -\ w_{i}x_{i}|\ \leq\ \varepsilon\ +|\xi_{i}| \tag{5}\]  
Figure 6 graphically depicts the linear SVR. Generally, four types of kernels are used in the SVR models, namely linear, polynomial, radial basis function (RBF), and sigmoid. The mathematical representation of each kernel is given in Equation (6) [40]:  
\[K\big{(}X_{i},\ X_{j}\big{)}=\ \left\{\begin{array}{ll}X_{i}X_{j}&\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \  
where \(Y\) is the predicted value, \(\theta_{0}\) is the bias term, \(\theta_{1}\),..., \(\theta_{n}\) are the model parameters, and \(x_{1}\), \(x_{2}\),..., \(x_{n}\) are the feature values. The aforementioned hypothesis can also be represented by Equations (8) and (9):  
\[Y\ =\ \theta^{T}x \tag{8}\]  
\[f_{i}(\theta)\ =\ \theta^{T}x+\in_{i} \tag{9}\]  
where \(f_{i}(\theta)\) is the hypothesis function and \(\in_{i}\) is noise. An intercept term is added by appending a column of 1 s to the features. Regularization is often required in order to prevent overfitting by penalizing models with extreme parameter values. The LR model supports \(L_{1}\) and \(L_{2}\) regularization, which are added to the loss function [42]. The \(L_{1}\) norm can be determined from the sum of the absolute values of the vector, and the \(L_{2}\) norm can be obtained as the square root of the sum of the squared vector values that are used to minimize the error. One of the main mysteries revealed by deep learning methodology is the concept of benign overfitting: deep neural networks tend to predict well, even with a perfect match for noisy training results.