## 4 Experimental Results
### Performance Analysis of the Proposed Model and Deep Learning Models  
In Table 8, we compare the performance of the proposed XmCNN model and the comparative DL models. The comparative DL model was a DNN model and three CNNs models. As mentioned in Section 3.4, the DNN model was composed of a fully connected layer, and the three CNN\({}_{\text{S}}\) models implemented convolutional neural networks using only a single filter size of 3, 4, and 5. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **FPR** & **FNR** & **Recall** & **Precision** & **Accuracy** & **F1-Score** \\ \hline RF & 0.0632 & 0.7506 & 0.5931 & 0.6629 & 0.8030 & 0.6073 \\ SVM & 0.4658 & 0.0154 & 0.7594 & 0.6656 & 0.6218 & 0.5990 \\ GBM & 0.1497 & 0.6369 & 0.6067 & 0.6081 & 0.7554 & 0.6074 \\ XGBoost & 0.1352 & 0.4710 & 0.6968 & 0.6847 & 0.7993 & 0.6903 \\ LightGBM & 0.0821 & 0.3264 & 0.6635 & 0.7541 & 0.8384 & 0.6903 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance evaluation of machine learning models. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **FPR** & **FNR** & **Recall** & **Precision** & **Accuracy** & **F1-Score** \\ \hline DNN & 0.1143 & 0.1870 & 0.8493 & 0.7918 & 0.8715 & 0.8143 \\ CNN\({}_{\text{S}(3)}\) & 0.0898 & 0.1941 & 0.8581 & 0.8177 & 0.8899 & 0.8352 \\ CNN\({}_{\text{S}(4)}\) & 0.0887 & 0.1902 & 0.8605 & 0.8201 & 0.8915 & 0.8376 \\ CNN\({}_{\text{S}(5)}\) & 0.0887 & 0.1889 & 0.8612 & 0.8204 & 0.8918 & 0.8387 \\ _XmCNN(proposed model)_ & 0.0756 & 0.1916 & 0.8664 & 0.8366 & 0.9018 & 0.8502 \\ \hline \hline \end{tabular}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **FPR** & **FNR** & **Recall** & **Precision** & **Accuracy** & **F1-Score** \\ \hline RF & 0.0632 & 0.7506 & 0.5931 & 0.6629 & 0.8030 & 0.6073 \\ SVM & 0.4658 & 0.0154 & 0.7594 & 0.6656 & 0.6218 & 0.5990 \\ GBM & 0.1497 & 0.6369 & 0.6067 & 0.6081 & 0.7554 & 0.6074 \\ XGBoost & 0.1352 & 0.4710 & 0.6968 & 0.6847 & 0.7993 & 0.6903 \\ LightGBM & 0.0821 & 0.3264 & 0.6635 & 0.7541 & 0.8384 & 0.6903 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Performance evaluation of deep learning models. As shown in Table 8, the proposed XmCNN model outperformed the DNN model and the three CNN\({}_{\text{S}}\) models, which are comparative DL models. First, among the CNN\({}_{\text{S}}\) models, the F1-score of the CNN\({}_{\text{S}(5)}\) model was the highest at 0.8387, but the overall performance of the CNN\({}_{\text{S}}\) models was similar. The accuracy of CNN\({}_{\text{S}(5)}\) increased by 2.33% compared to that of the DNN model, and the F1-score improved by 2.92%. This means that when a convolutional layer was added, the performance was improved compared to a DNN composed of only a fully connected layer. In addition, the accuracy of the proposed XmCNN model increased by 1.12%, and the F1-score increased by 1.44% compared to the CNN\({}_{\text{S}(5)}\) model using only a single filter size of 5. Therefore, it implies that multiple filters are effective in improving model performance. The hyperparameters of the proposed model were optimized and selected based on the performance of the validation data. We used the Adam optimizer [34] with an initial learning rate of 0.001 when training the DL models. We set up the learning rate decay scheduling to decrease the learning rate according to the change in validation loss. We halved the learning rate if the validation loss did not improve for 30 epochs. In addition, all DL models were trained with 500 epochs and mini-batch sizes of 64. We utilized TensorFlow 2.4 and Keras on a single NVIDIA GeForce RTX 3080 to perform the experiments. Additionally, the results of comparison between the performance of the DL model and the ML model are provided in Figure 5. ML models generally exhibited poor performance, owing to the large number of variables. However, DL models have significantly higher accuracy and F1-score compared to comparative ML models. Among them, the proposed XmCNN model outperformed all the comparative ML and DL models.