## 3 Methods
### Support Vector Regression  
The Support Vector Regression is inspired by classifications problems, i.e., the determination of the best line (or hyperplane, in multiple dimensions) demarcating observations. Subsequently, a regression based on these principles was formulated and widely used in data analysis, especially for facing non-linear problems. For details, refer for example to [36]. In its linear form, Support Vector Regression is a constrained optimization problem, whose objective function contains the potentially conflicting targets of minimum norm of the coefficients vector (which means flatness) and residuals between the measurement and model estimate being lower than a threshold \(\epsilon\) for each observation. In practice, if \(\mathbf{x}\) is the matrix of observation and \(y\) is the target, for a linear model the function to minimize by selecting appropriate Support Vectors \(\alpha\) and \(\alpha^{*}\) is given in Equation (2):  
\[L(\alpha)=\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}(\alpha_{i}-\alpha_{i}^{*}) \left(\alpha_{j}-\alpha_{j}^{*}\right)\mathbf{x}_{i}^{\prime}\mathbf{x}_{j}+\epsilon \sum_{i=1}^{N}(\alpha_{i}+\alpha_{i}^{*})+\sum_{i=1}^{N}y_{i}(\alpha_{i}^{*}- \alpha_{i}), \tag{2}\]  
with the constraints (Equation (3))  
\[ \begin{split}\sum_{n=1}^{N}(\alpha_{n}-\alpha_{n}^{*})& =0\\ 0\leq\alpha_{n}&\leq C\\ 0\leq\alpha_{n}^{*}&\leq C.\end{split}  \tag{3}\]  
\(C\) is called the box constraint and it is a hyperparameter of the model, quantifying the cost of misclassified points: the higher the box constraint is, the higher the cost of misclassification is, and this leads to a better separation of the data. Referring to classifications problems, it can happen that the observed data cannot be easily classified as they are, but they become classifiable if transformed to a feature space, where for feature space it has to be intended as a higher dimensional space where the linear separation is possible. This idea is at the basis of the non-linear Support Vector Regression, which is obtained by replacing the scalar products between the observation matrix of Equation (2) with a non-linear Kernel function (Equation (4)):  
\[G(\mathbf{x}_{1},\mathbf{x}_{2})=\langle\varphi(\mathbf{x}_{1})\varphi(\mathbf{x}_{2})\rangle, \tag{4}\]  
where \(\varphi\) is a transformation mapping the \(\mathbf{x}\) observations into the feature space. A Gaussian Kernel selection is given in Equation (5), where \(\kappa\) is the kernel scale:  
\[G\big{(}\mathbf{x}_{i},\mathbf{x}_{j}\big{)}=e^{-\kappa\left\|\mathbf{x}_{i}-\mathbf{x}_{j} \right\|^{2}}. \tag{5}\]  
Once the model has been trained, it can be used for predicting the output, given the input \(\mathbf{x}\), using Equation (6):  
\[f(\mathbf{x})=\sum_{n=1}^{N}(\alpha_{n}-\alpha_{n}^{*})G(\mathbf{x}_{n},\mathbf{x}). \tag{6}\]  
If the data are non-standardized, it is necessary to add into Equation (6) a constant shift \(b\) which has the same meaning as the intercept in a linear model. The critical points regarding the application of Support Vector Regression are the optimization of the hyperparameters and the input variables' selection. As regards the former, the typical approach is fixing a certain number of model runs and, for each run, one hyperparameter at a time is varied randomly in a selected range. The data are divided in certain proportions (training and test) and the best configuration is selected by targeting a loss function (an error metric) to minimize for the test data set. For this study, the Root Mean Square Error was selected, which is given in Equation (7):  
\[RMSE=\sqrt{\frac{\sum(y-f(\mathbf{x})-ME)^{2}}{M}}, \tag{7}\]  
where \(M\) is the number of samples in the test data set and \(ME\) is the mean error, defined in Equation (8):  
\[ME=\frac{1}{M}\sum y-f(\mathbf{x}). \tag{8}\]  
The input variables' selection is dictated by the kind of problems to which the regression is applied.