# Detecting and Analyzing Botnet Nodes via Advanced Graph Representation Learning Tools +
## 7 Experimental Assessment and Analysis
### Experimental Results
The results of this classification are shown in Figure 14.  
Figure 11: Classification performance evaluation of ML classifiers. Figure 10: Evaluation score of state-of-the-art methods for botnet detection in early and late stages. On the other hand, we extend our analysis using _deep learning algorithms_ to assess if DL models outperform those using ML algorithms. It should be noted that some experiments do not provide all metrics, which results in some missing scores. Figure 15 shows a comparative analysis of various ML classifiers based on four performance metrics: accuracy, recall, F1-score, and precision. The classifiers compared include our implementation of ML algorithms (i.e., Our-KNN, Our-DT, and Our-RF), KNN [63], DT [64], and RF [65], along with _Cross CNN_LSTM_ (SNC) [66] and our SIR-GN. The results indicate that SNC and SIR-GN outperform all traditional ML classifiers, achieving accuracy, recall, F1-score, and precision values close to 100%. Among the traditional ML models, RF achieves the highest performance, followed by DT and KNN. Additionally, the customized models (i.e., Our-KNN, Our-DT, and Our-RF) demonstrate improvements over their standard counterparts, suggesting modifications that enhance their classification capabilities. Figure 14: Four-class classification performance of SIR-GN. Figure 12: Binary classification performance of SIR-GN. Figure 13: Three-class classification performance of SIR-GN. On the other hand, Figure 16 provides a detailed comparison of DL models by evaluating their performance using the same four metrics: accuracy, recall, F1-score, and precision. The models compared include SNC and SIR-GN, which were also present in Figure 15, and additional _deep learning-based architectures_ (DG-CNN [62] and CNN [55]). The results indicate that SIR-GN achieves the highest performance across all four metrics, closely followed by SNC. As highlighted in Figures 15 and 16, DL models significantly outperform traditional ML classifiers, which highlights their superior ability to capture complex patterns in the data. The minimal performance gap among accuracy, recall, precision, and F1-score showcases that these DL models maintain strong _generalization_ capabilities while minimizing _false positives_ and _false negatives_. Finally, to conclude our experimental campaign, we conduct a comprehensive comparative analysis between our proposed inferential SIR-GN model and ABD-GN model [49]. The purpose of this comparison is to evaluate the strengths and limitations of both models in addressing the specific challenges of the botnet detection context. Figure 17 compares the node representations generated by inferential SIR-GN and ABD-GN, alongside the results obtained from the _neural network classifier_ for different network topologies (see Section 5). Despite the fact that the model was trained to represent nodes using _randomly generated graphs_ and specifically trained on 50 graphs from dataset, SIR-GN shows performance closely comparable to ABD-GN across all datasets. In contrast, the ABD-GN neural network model was trained on 80% (768 samples) of the dataset, which highlights the robust performance of inferential SIR-GN even with fewer training samples. Figure 17 shows the outstanding performance of the ABD-GN model for a single topology. The outcomes, however, range from excellent to very poor when these same trained models are tested against an invisible topology, as seen in Table 2.  
Figure 12.0 Classification performance of different DL models and SIR-GN. Figure 12.1 Classification performance of different ML models and SIR-GN. When evaluated on all other synthetic datasets, inferential SIR-GN and ABD-GN perform equally when trained on the _Kadem_ topology. Similarly to training on _Leet_ and _Chord_ data, the F1-score for ABD-GN falls below 10%, with no variance in performance on the other synthetic datasets.