## 4 Results and Discussion
### Sensitivity Analysis of the Meta-Ensembles  
#### 4.2.1 Comparing between Cross-Validation Techniques  
Table 5 and Figure 9 compare the performance of the hybrid meta-heuristic ensemble models across 3 cross validation methods. At first glance, it is observed that the method \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & \(\mathbf{R^{2}}\) & **RMSE** & **MAE** \\ \cline{2-4}  & & **\%** & **\%** \\ \hline REG & 0.67 & 0.73 & 0.54 \\ ANN & 0.59 & 0.81 & 0.61 \\ BLR & 0.19 & 1.14 & 1.02 \\ BDT & 0.82 & 0.53 & 0.44 \\ RDF & 0.86 & 0.48 & 0.38 \\ SE & **0.93** & **0.34** & **0.29** \\ VE & **0.93** & **0.33** & **0.26** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: ML regression performance metrics. relying on the TVS seems to generally produce the highest values of R\({}^{2}\) (with corresponding lower RMSE and MAE scores) from Table 5. This phenomenon is also demonstrated by the less degree of deviation of the predicted curve from the ideal line in Figure 9. On the other hand, the iterative method of training, testing, and validation for which kFCV and MCCV depend, tend to produce slightly lowers values of R\({}^{2}\) compared to the TVS method. As mentioned previously, both kFCV and MCCV are used to improve the prediction of ML regression analyses while also serving as fine-tuning mechanisms to the TVS technique. Hence, the modelling with training and testing carried out using TVS when compared to kFCV and MCCV techniques seems to now indicate slight overfitting. Nonetheless, from Table 5 it is observed that the MCCV method does generally provide slightly much better accuracy than its kFCV counterpart. When also comparing between models, the prediction offered by the meta-ensembles using MCCV as a technique of cross validation, can be said to be more accurate than those of the stand-alone and tree ensembles which rely on the TVS method (Table 4). Although, when considered in terms of their Bias-Variance trade-offs, the MCCV is mostly deemed as having greater biases than the kFCV but with the former seeming to provide slightly more confidence in ML predictions given that it is more repeatable than the later due to its capacity to provide results with lower degrees of variance. Besides being characteristically prone to giving results with high variance, one other reason for KFCV's poor performance is attributable to the number of its partitioning being limited by the number of folds used. However, a much closer examination of Table 5 indicates that ML testing and validation relying on a smaller number of folds (for both MCCV and kFCV) does marginally produce more accuracy. It is important to bear in mind that depending on the situation, or the regression problem faced with, both MCCV and kFCV have been reported in the past as being able to provide better predictions than TVS. Nevertheless, within the context of this study, it is seen that the reverse seems to be the case. Again, judging from the method of cross validation used, model combination through voting does outperforms that done by stacking as Figure 9 indicates. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & & & **R\({}^{2}\)** & **RMSE** & **MAE** \\ \cline{2-6}  & **CV Method** & **Set** & **No. of Folds** & & **\%** & **\%** \\ \hline \multirow{4}{*}{SE} & TVS & 20 & - & **0.93** & **0.34** & **0.29** \\  & & 10 & - & 0.67 & 0.66 & 0.62 \\  & KFCV & - & 5 & **0.88** & **0.44** & **0.33** \\  & & - & 10 & 0.86 & 0.44 & 0.33 \\  & MCCV & 10 & 5 & **0.89** & **0.40** & **0.31** \\  & & 20 & 5 & 0.88 & 0.46 & 0.35 \\  & & 10 & 10 & 0.88 & 0.43 & 0.34 \\  & & 20 & 10 & 0.88 & 0.46 & 0.36 \\  & TVS & 10 & - & 0.91 & 0.34 & 0.29 \\  & & 20 & - & **0.93** & **0.33** & **0.26** \\ VE & KFCV & - & 5 & **0.88** & **0.43** & **0.32** \\  & & - & 10 & 0.86 & 0.43 & 0.32 \\  & MCCV & 10 & 5 & **0.90** & **0.40** & **0.32** \\  & & 20 & 5 & 0.88 & 0.45 & 0.35 \\  & & 10 & 10 & 0.87 & 0.44 & 0.35 \\  & & 20 & 10 & 0.88 & 0.46 & 0.36 \\ \hline \hline \end{tabular}
\end{table}