## 4 Experimental Results and Discussion
### Results
Through this process, the model's performance is assessed across different subsets of the data, thereby reducing the influence of any specific data split and enhancing the reliability of the results. In this study, a k value of 13 was chosen to enhance the effectiveness of the ML models' performance. The selection of the optimal k value depends on factors such as the size of the dataset and the complexity of the problem. By performing cross-validation with various k values, the accuracy values can be observed and considered before separating the data into training and testing sets. Figure 8 illustrates the process of cross-validation performed for various k values, displaying the accuracy values obtained for both the training and test  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline
**Models** & \multicolumn{3}{c}{**Precision (\%)**} & \multicolumn{3}{c}{**Recall (\%)**} & \multicolumn{3}{c}{**F1-Score (\%)**} & \multicolumn{3}{c}{**Accuracy (\%)**} & \multicolumn{3}{c}{**AUC (\%)**} \\ \hline  & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** \\ \hline LR & 82 & 80 & 85 & 86 & 82 & 86 & 84 & 81 & 85 & 81 & 81 & 86 & 59 & 60 & 59 \\ \hline KNN & 73 & 73 & 80 & 77 & 77 & 82 & 75 & 75 & 81 & 77 & 77 & 82 & 66 & 71 & 76 \\ \hline SVM  
data. This depiction provides insights into the relationship between the k value and model performance, enabling the selection of an appropriate k value that yields optimal results. Formula (8) represents the calculation of accuracy values for cross-validation with N k values. The dataset was divided into equal parts based on the obtained k value, and the accuracy scores for each partition were recorded. \[Accuracy=\frac{TP+TN}{TP+FP+TN+FN} \tag{4}\]1  
After conducting cross-validation, the models were subjected to the Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) methods. However, the specific details of these methods, such as their configurations or parameters, are not provided in the available information. Table 6 showcases the accuracy values of the models created with the dependent variable \"Ex\" after undergoing cross-validation. In the study, k-fold cross-validation was performed on the \"Ex\" dependent variable. Among the machine learning (ML) and deep learning (DL) models, the highest average results were obtained when utilizing the Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) methods. The confusion matrices corresponding to these models, which produced the best outcomes with the LDA method, are depicted in \[Accuracy=\frac{TP+TN}{TP+FP+TN+FN} \tag{4}\]2 A confusion matrix, also referred to as an error matrix, visually represents the performance of the models by displaying the various outcomes of their predictions. Among the ML models, LR, SVM, NB, and KNN demonstrated similar results and generally performed  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline
**Models** & \multicolumn{3}{c}{**Precision (\%)**} & \multicolumn{3}{c}{**Recall (\%)**} & \multicolumn{3}{c}{**Fl-Score (\%)**} & \multicolumn{3}{c}{**Accuracy (\%)**} & \multicolumn{3}{c}{**AUC (\%)**} \\ \hline  & N-M & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** & **N-M** & **PCA** & **LDA** \\ \hline LR & 85 & 82 & 98 & 96 & 96 & 98 & 96 & 96 & 99 & 92 & 87 & 99 & 66 & 65 & 99 \\ \hline KNN & 87 & 88 & 98 & 96 & 96 & 96 & 96 & 96 & 96 & 89 & 86 & 99  
well. RF and DT showed lower performance compared to the other models. Among the DL models, LSTM did not make any misclassifications. The performance of RNN was significantly lower than LSTM. LSTM was the most successful model among all, while RNN was the weakest model. Let us examine the provided example of the confusion matrix for the model created with the Decision Tree (DT) algorithm:  
* Total Test Data: 20
* True Positive (TP): The model correctly predicted 14 instances as \"Ex\".