## 3 Materials and Methods
### Support Vector Machine
The kernel function, defined by a dot product in the Hilbert space, is expressed in Equation (16):  
\[\langle x_{1}\cdot x_{2}\rangle\gets K(x_{1},x_{2})=\langle\Phi(x_{1}) \cdot\Phi(x_{2})\rangle \tag{16}\]  
In Equation (16), the kernel is equivalent to mapping the data into a feature space V, allowing the SVM algorithm to compute and solve problems where data is not linearly separable. Therefore, the quadratic programming problem required to find the optimal hyperplane is convex only if the kernel function satisfies the Mercer conditions; thus, the kernel must satisfy \(K:S\times S\rightarrow\mathbb{R}\):  
\[\int_{S}\int_{S}g(x)k(x,x^{\prime})g(x^{\prime})dxdx^{\prime}\geq 0 \tag{17}\]  
for each square integral function \(g(x)\). If \(k\) satisfies Equation (17), then the matrix M, where:  
\[m_{ij}=k(x_{i},x_{j}),\forall x_{1},\dots,x_{n}\in S\ \ and\ \ \forall n\in \mathbb{N}\ \ is \tag{18}\]  
(1) Symmetric, (\(M=M^{\text{T}}\)) and  
(2) Positive Semi-Definite Matrix (PSD)  
A matrix is positive semi-definite if \(uMu^{T}\geq 0\) for each of the real vectors \(u\in\mathbb{R}^{n}\), in other words, all eigenvalues are non-negative. Choosing the appropriate kernel becomes vital and non-trivial when considering kernel effectiveness for non-linear data. This means it also becomes one more parameter to be considered as a convex optimization problem within the general conditions. An optimal kernel function can be chosen from a fixed set of kernels statistically rigorously using cross-validation. In general, there are some kernel functions, as shown in Equations (19)-(22). (1) Linear Kernel:  
\[K(x_{i},x_{k})=x_{i}^{T}x_{j} \tag{19}\]  
(2) Polynomial Kernel:  
\[K(x_{i},x_{k})=(\gamma x_{i}^{T}x_{j}+r)^{d},\ \ \ \gamma>0 \tag{20}\](3) Radial Basis Function (RBF) kernel:  
\[K(x_{i},x_{k})=exp(-\gamma\parallel x_{i}-x_{j}\parallel^{2}),\quad\gamma>0 \tag{21}\]  
(4) Sigmoid Kernel:  
\[K(x_{i},x_{k})=tanh(\gamma x_{i}^{T}x_{j}+r) \tag{22}\]  
Hyperparameters  
Adjusting the hyperparameters is crucial in creating an accurate mathematical prediction model. If they are not adjusted correctly, the model can produce sub-optimal results, leading to more errors. There are two ways to select hyperparameters: using default values provided by the software or manually configuring them. Additionally, data-dependent hyperparameter optimization strategies can be employed. These strategies are second-degree optimization procedures that minimize the expected error of the model by searching for candidate configurations of hyperparameters. Grid or random search is an example of such strategies, where a list of candidate parameters is specified. Bayesian optimization is another iterative strategy that's more complex [54]. Bayesian optimization was used to obtain the best set of hyperparameters of the model; its use was able to minimize the non-convex function in a way that some other method does not allow. The random search and grid search methods were initially used for the data analysis; nevertheless, for the forecast, Bayesian optimization was used to adjust the best candidate due to its superiority compared to the methods above [55]. As in other types of optimization, the Bayesian method seeks to find the minimum of a function \(f(x)\) on some bounded set \(X\); taken from a subset of \(R\to D\). Unlike other methods, Bayesian optimization builds a probabilistic model for \(f(x)\). It uses it to determine where to evaluate the function at \(X\) later while accounting for uncertainty. As a Bayesian method, it utilizes all available information to find the minima of non-convex functions with relatively few evaluations. However, this comes at the cost of additional calculations to determine the next point to prove [55].