## 4 Results
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Manually Selected Feature Set**} & \multicolumn{3}{c}{**Automatically Selected Feature Set**} \\ \cline{2-6}
**Algorithm** & \multicolumn{2}{c}{**Accuracy (\%)**} & \multicolumn{2}{c}{**SD**} & \multicolumn{2}{c}{**Accuracy (\%)**} & \multicolumn{2}{c}{**SD**} \\ \cline{2-6}  & **Training** & **Testing** & \multicolumn{2}{c}{**Training**} & \multicolumn{2}{c}{**Testing**} \\ \cline{2-6}  & \multicolumn{6}{c}{**Baseline Models**} \\ \hline  & \multicolumn{6}{c}{**SMOTE**} \\ \hline Logistic Regression & 97.78 & 97.78 & 3.14 & 97.92 & 98 & 4 \\ \hline Decision Tree & 100 & 100 & 0 & 100 & 100 & 0 \\ \hline Random Forest & 100 & 100 & 0 & 100 & 100 & 0 \\ \hline Support Vector Classifier & 97.78 & 97.78 & 3.14 & 100 & 97.778 & 4.44 \\ \hline Gaussian Naive Bayes & 100 & 100 & 0 & 100 & 100 & 0 \\ \hline Multi-layer Perceptron & 97.78 & 97.78 & 3.14 & 97.92 & 98 & 4 \\ \hline k-Nearest neighbors & 85.14 & 85.14 & 12.8 & 85.28 & 91.56 & 7.62 \\ \hline Custom Ensemble & 100 & 100 & 0 & 100 & 100 & 0 \\ \hline \multicolumn{6}{c}{**PCA**} \\ \hline Logistic Regression & 70 & 70 & 14.14 & 76.67 & 73.33 & 13.33 \\ \hline Decision Tree & 76.67 & 76.67 & 12.47 & 80 & 73.33 & 8.16 \\ \hline Random Forest & 80 & 80 & 8.16 & 86.67 & 90 & 13.33 \\ \hline Support Vector Classifier & 80 & 80 & 0 & 86.67 & 83.33 & 0 \\ \hline Gaussian Naive Bayes & 70 & 70 & 8.16 & 96.67 & 96.67 & 6.67 \\ \hline Multi-layer Perceptron & 73.33 & 73.33 & 9.43 & 86.67 & 86.67 & 19.44 \\ \hline k-Nearest neighbors & 76.67 & 76.67 & 4.71 & 83.33 & 76.67 & 22.61 \\ \hline Custom Ensemble & 73.33 & 73.33 & 17 & 86.67 & 86.67 & 12.47 \\ \hline \multicolumn{6}{c}{**SMOTE + PCA**} \\ \hline Logistic Regression & 95.56 & 95.56 & 6.29 & 95.83 & 95.78 & 5.18 \\ \hline Decision Tree & 85.14 & 85.14 & 2.77 & 93.61 & 89.78 & 10.96 \\ \hline Random Forest & 93.47 & 93.47 & 5.45 & 97.78 & 97.78 & 4.44 \\ \hline Support Vector Classifier & 95.56 & 95.56 & 6.29 & 95.83 & 98 & 4 \\ \hline Gaussian Naive Bayes & 93.47 & 93.47 & 5.45 & 95.69 & 95.78 & 5.18 \\ \hline Multi-layer Perceptron & 95.56 & 95.56 & 6.29 & 95.83 & 95.78 & 5.18 \\ \hline k-Nearest neighbors & 95.56 & 95.56 & 6.29 & 91.67 & 91.33 & 8.27 \\ \hline Custom Ensemble & 95.56 & 95.56 & 6.29 & 97.92 & 93.78 & 5.1 \\ \hline \multicolumn{6}{c}{**SMOTE + PCA + HPO**} \\ \hline Logistic Regression & 95.56 & 93.33 & 8.89 & 95.83 & 95.78 & 5.18 \\ \hline Decision Tree & 85.14 & 85.11 & 12.48 & 93.61 & 89.78 & 10.96 \\ \hline Random Forest & 91.39 & 89.11 & 7.04 & 97.78 & 97.78 & 4.44 \\ \hline Support Vector Classifier & 95.56 & 95.56 & 8.89 & 100 & 100 & 0 \\ \hline Gaussian Naive Bayes & 93.47 & 93.56 & 8.79 & 95.69 & 95.78 & 5.18 \\ \hline Multi-layer Perceptron & 91.39 & 89.11 & 7.04 & 100 & 100 & 0 \\ \hline k-Nearest neighbors & 95.56 & 95.56 & 8.89 & 95.83 & 95.78 & 5.18 \\ \hline Custom Ensemble & 95.56 & 95.56 & 8.89 & 90 & 86.67 & 12.47 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: _Cont._In Table 9, the accuracy of each model is provided both for the training and the testing dataset. In the latter case the cross-validation accuracy is shown. At this point, by inspecting the accuracy during training and testing it is possible to recognize which algorithms tend to create models that overfit or underfit. Therefore, first a set of baseline models are trained and tested, then SMOTE and PCA are applied separately, followed by the application of combined SMOTE and PCA on the same dataset and finally a set of models are created by combining SMOTE, PCA and hyperparameter optimization. By inspecting the results, it is observed that most of the baseline trained models for the manually selected features tend to either overfit or underfit, contrary to the dataset composed of the automatically selected features.