## 3 Results
### Supervised Machine Learning Models
#### 3.5.4 Comparative Analysis  
The comparative analysis clearly summarizes the results from the previous sections. The procedure of the final comparison of models was as follows: The implemented models were optimized using cross-validation. In the case of linear regression models, the opti \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**Train Sets**} & \multicolumn{2}{c}{**Test Sets**} \\ \hline
**Nodesize** & **Avg MSE** & **Avg MAPE [in \%]** & **Avg MSE** & **Avg MAPE [in \%]** \\ \hline
1 & 0.08695614 & 11.50494 & 0.08595961 & 11.49746 \\
2 & 0.08749527 & 11.51510 & 0.08215867 & 11.51712 \\
3 & 0.09158489 & 11.60859 & 0.08529704 & 11.53510 \\
4 & 0.08874726 & 11.55299 & 0.08671523 & 11.42037 \\
5 & 0.08875384 & 11.42312 & 0.08212290 & 11.12689 \\
6 & 0.08836823 & 11.46728 & 0.08269044 & 11.40530 \\
7 & 0.08814417 & 11.55690 & 0.08692935 & 11.66649 \\
8 & 0.08624920 & 11.17409 & 0.08113720 & 11.24279 \\
9 & 0.08702532 & 11.33934 & 0.08559722 & 11.44480 \\
10 & 0.08676777 & 11.62171 & 0.08683290 & 11.64031 \\
11 & 0.08797640 & 11.50625 & 0.09115274 & 11.82005 \\
12 & 0.09239382 & 11.78369 & 0.08494978 & 11.38928 \\
13 & 0.09173662 & 11.71659 & 0.08616982 & 11.32459 \\
14 & 0.09357773 & 11.86118 & 0.09020726 & 12.14479 \\
15 & 0.09108029 & 11.75776 & 0.08853183 & 11.65980 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Random forest RF1—accuracy error metrics, 5-fold cross-validation. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**Train Sets**} & \multicolumn{2}{c}{**Test Sets**} \\ \hline
**Nodesize** & **MSE** & **MAPE [in \%]** & **MSE** & **MAPE [in \%]** \\ \hline
1 & 0.07778958 & 11.03145 & 0.07763078 & 11.62394 \\
2 & 0.07671013 & 11.03411 & 0.08017573 & 11.57939 \\
3 & 0.07418463 & 10.80416 & 0.08000238 & 11.93554 \\
4 & 0.07515196 & 10.96224 & 0.08058369 & 11.84307 \\
5 & 0.07596767 & 10.86967 & 0.07839428 & 11.69903 \\
6 & 0.07547489 & 11.28770 & 0.08239564 & 11.91910 \\
7 & 0.07736840 & 10.96301 & 0.07882811 & 11.65105 \\
8 & 0.08007958 & 11.17961 & 0.07992665 & 11.88414 \\
9 & 0.07885461 & 11.18626 & 0.07884320 & 11.77858 \\
10 & 0.08145756 & 11.22768 & 0.08252397 & 12.04561 \\
11 & 0.08104617 & 11.25166 & 0.08065937 & 11.87481 \\
12 & 0.08211288 & 11.36819 & 0.08366853 & 12.01931 \\
13 & 0.08280258 & 11.31227 & 0.08165990 & 11.84619 \\
14 & 0.08603487 & 11.54173 & 0.08693637 & 12.34537 \\
15 & 0.08243315 & 11.42266 & 0.08893866 & 12.50105 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: RF2 random forest—accuracy error metrics, 5-fold cross-validation. mization concerned the appropriate selection of independent variables. For the decision tree and random forest models, the optimization concerned the selection of the minbucket and nodesize parameters. Table 10 summarizes the results of the cross-validation. In decision trees and random forest models, only the models with the optimized value of hyperparameter are stated. The models RF1, RF2 and backward linear regression model LIN4 achieved the best generalization accuracy on the cross-validation set. In other words, these models achieved, on average, the lowest error rate when testing on multiple validation sets composed of new data (i.e., out-of-bag validation). The worst performers were vif models, which used fewer variables. Since cross-validation was used to find the optimal value of the hyperparameter, especially in the random forest and decision tree models, the final training of the model (60 observations) and the final testing on the test set, which included 19 observations, was performed. This was to show the true predictive power of the implemented models on our specific test set. Table 11 shows the results on the final test set. RDT2 tree model was more accurate than the RDT1 model and its percentage error in the final testing was 9%. A good sign is that in the final test set, none of the models achieved an error higher than 10%. Linear regression model implemented through backward regression was most accurate. Other models of linear regression were also very accurate.