## 4 Results
### Random Forest Results  
Table 6 summarizes three RF models, RF_1, RF_2, and RF_3, by varying the _n_estimators_, _max_depth_, and _ccp_alphas_ parameters. That is, for each model, one parameter is varied while the other two are fixed. The accuracy results against testing and LOOCV metrics of each RF model are plotted twice, once using all the twelve datasets and another using only the last five datasets. These results are plotted in Figure 6, Figure 7 and Figure 8, for each model, respectively.  
Figure 5: Average testing and LOOCV accuracies for the Logistic Regression models. (**a**) Across all the twelve datasets; (**b**) Across the last five datasets.  
Figure 6a,b report the performance of model RF_1. Both figures show that the graphs peak at \(n\_estimors=6\) and 12, suggesting that these two parameter values result in the model having the highest accuracy, while the other two parameters are fixed.  
Looking at Figure 7a,b, which plot the performance of model RF_2, we can see the disparate performance of the same model against the two evaluation metrics, testing and LOOCV. LOOCV results in much better prediction accuracy than testing, where the model performance peaks around \(max\_depth=4\) for LOOCV.  
Evaluating the performance of model RF_3 in Figure 8a,b, the accuracy is highest at \(ccp\_alphas=0.25\) for the LOOCV metric.  
Based on Figures 6-8, we can conclude that the best model parameters for the RF model would be \(n\_estimors=12\), \(max\_depth=4\), and \(ccp\_alphas=0.25\).  
All the experiments conducted thus far used the datasets whose dimensionality was reduced (from 428 dimensions) as per the feature selection process presented in Section 3.2. To maintain a more objective evaluation approach, we employ Principal Component Analysis (PCA) for dimensionality reduction [60; 61]. PCA transforms a high-dimensional dataset by projecting data points onto a new subspace defined by newly created dimensions, i.e., principal components. The idea is to utilize the original dimensions that contribute the most to explaining the data. The latter is conducted through the variance. That is, we perform PCA on our curated dataset (see Section 3.1) where the principal components explain 90% and 95% of the variance, respectively. This leads to the creation of two datasets; the first dataset contains 16 principal components, and the second dataset contains 18 principal components, respectively.  
Figure 6: Average testing and LOOCV accuracies for the \(n\_estimator\) parameter. (**a**) Across all the twelve datasets; (**b**) Across the last five datasets.  
Figure 7: Average testing and LOOCV accuracies for the \(max\_depth\) parameter. (**a**) Across all the twelve datasets; (**b**) Across the last five datasets.  
In Figure 9, PCA was conducted such that 90% of the data variance is maintained. This resulted in a dataset that contains 16 dimensions. We observe from both Figure 9a,b that the prediction model exhibits overfitting when trained and tested using a 50%-50% split. As such, it would not be fair to accept or conclude the high accuracy of 90.91% reported in Figure 9b.  
Similarly, Figure 10 reports the prediction results from using PCA where 95% of the data variance is explained by 18 principal components (dimensions). Figure 10a depicts  
Figure 8: Average testing and LOOCV accuracies for the _ccp_alphas_ parameter. (**a**) Across all the twelve datasets; (**b**) Across the last five datasets.  
Figure 9: Model accuracies after PCA with 16 principal components explaining 90% of the variance. (**a**) \(k\)NN; (**b**) Logistic Regression.  
fluctuation of accuracy across the different \(k\)NN models (see Table 4), but the model stabilizes under the configuration of \"kNN_M9\" and \"kNN_M10\". On the other hand, Figure 9(b) shows a robust and stable performance across most of the Logistic Regression models (see Table 5) and far fewer overfitting issues. Consequently, under \"LR_M11\", the highest reported accuracy can be considered.