## 3 Results and Discussion
### Performance of the Three Hybrid Classifiers  
According to the results of the simple classification algorithms, KNN seems to be among the ones that present the poorest classification performance. This gave motivated us to investigate the range of classification performance of the KNN algorithm. For that purpose, 100,000 independent experiments of KNN classification were performed for various values of the training set size \(S\). The obtained results for \(k=3\) neighbors are presented in Figures 5 and 6. Figure 5 presents the average values and the error bars of the classification accuracy with respect to the training set size \(S\) that was obtained for 100,000 independent experiments. In Figure 6, the corresponding maximum, minimum, and average classification accuracy values that were obtained by these 100,000 independent experiments are plotted vs. \(S\). Figure 5 shows a strong dependence of the classification accuracy on the size \(S\) of the training set: the larger the training set, the higher the classification accuracy. In addition, as can be seen in \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & & \multicolumn{8}{c}{**Test Set Size**} \\ \cline{2-13}  & & \multicolumn{3}{c}{**0.3**} & \multicolumn{3}{c}{**0.4**} & \multicolumn{3}{c}{**0.5**} & \multicolumn{3}{c}{**0.9**} \\ \hline
**Classifiers** & **Acc** & **Pre** & **Rec** & **Acc** & **Pre** & **Rec** & **Acc** & **Pre** & **Rec** & **Acc** & **Pre** & **Rec** \\ \hline
1 & Decision Tree & 0.974 & 0.962 & 0.941 & 0.972 & 0.952 & 0.945 & 0.971 & 0.944 & 0.946 & 0.952 & 0.914 & 0.905 \\ \hline
2 & Random Forest & 0.979 & 0.957 & 0.962 & 0.979 & 0.961 & 0.959 & 0.978 & 0.955 & 0.962 & 0.958 & 0.942 & 0.895 \\ \hline
3 & KNN & 0.896 & 0.863 & 0.717 & 0.888 & 0.863 & 0.691 & 0.885 & 0.847 & 0.677 & 0.828 & 0.827 & 0.437 \\ \hline
4 & Logistic Regression & 0.925 & 0.895 & 0.815 & 0.921 & 0.883 & 0.812 & 0.926 & 0.884 & 0.824 & 0.890 & 0.883 & 0.670 \\ \hline
5 & SVM & 0.956 & 0.936 & 0.897 & 0.951 & 0.924 & 0.891 & 0.949 & 0.906 & 0.898 & 0.920 & 0.897 & 0.787 \\ \hline
6 & Naive Bayes & 0.942 & 0.958 & 0.815 & 0.937 & 0.952 & 0.808 & 0.937 & 0.943 & 0.805 & 0.939 & 0.950 & 0.812 \\ \hline
7 & Neural Network & 0.979 & 0.966 & 0.952 & 0.922 & 0.932 & 0.762 & 0.906 & 0.955 & 0.693 & 0.888 & 0.917 & 0.652 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of the seven simple classifiers with respect to accuracy, precision, and recall vs. test set size. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Test Set Size vs. Training Set Size**} \\ \cline{2-5}  & & **0.3 vs. 0.7** & **0.4 vs. 0.6** & **0.5 vs. 0.5** \\ \hline
**Classifiers** & **Overfit Factor** & **Overfit Factor** & **Overfit Factor** \\ \hline
1 & Decision Tree & 0.025 & 0.026 & 0.027 \\ \hline
2 & Random Forest & 0.019 & 0.019 & 0.019 \\ \hline
3 & KNN & 0.040 & 0.051 & 0.057 \\ \hline
4 & Logistic Regression & \(-0.001\) & 0.003 & 0.004 \\ \hline
5 & SVM & \(-0.006\) & 0.003 & 0.009 \\ \hline
6 & Naive Bayes & \(-0.001\) & 0.005 & 0.008 \\ \hline
7 & Neural Network & \(-0.007\) & 0.050 & 0.011 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Overfitting factor of the seven simple classifiers vs. test set size. Figure 6, despite its poor performance, the KNN classifier managed to achieve accuracy equal to 1 for training set size \(S=90\%\). A more detailed analysis of the results revealed that an accuracy equal to 1 resulted in 98 of 100,000 experiments, that is, for approximately 0.1% of the experiments. These findings led to the inquiry into the minimum size of the training set that could achieve an accuracy equal to 1 on the test set, which was realized with the use of Genetic Algorithms and the implementation of the three hybrid algorithms GAKNN, GAKMEANS, and GACLUST that were presented in Section 2.3. Representative results of the three hybrid classifiers developed in the present study are demonstrated in this section.