# Detecting and Analyzing Botnet Nodes via Advanced Graph Representation Learning Tools +
## 7 Experimental Assessment and Analysis
### Experimental Results  
In this section, we present the experimental results obtained from our extensive campaign. Specifically, our experimental evaluation is structured into two main phases. In the first phase, we benchmark the performance of state-of-the-art approaches in the context of botnet detection. This provides a solid comparative foundation for assessing the capabilities of existing approaches. Following this, we conduct a thorough evaluation of our proposed SIR-GN method, applying it to both _binary_ and _multi-class_ classification tasks, by analyzing their effectiveness based on multiple evaluation metrics, including _accuracy_, _recall_, _F1-score_, and _precision_. We compare its performance against a selection of ML and DL algorithms to highlight its robustness and accuracy in detecting botnet nodes. In the second phase, we extend our analysis by directly comparing SIR-GN with ABD-GN, which aims to assess its effectiveness in handling both synthetic and real-life data. Although early detection of botnets is critical for preventing damage, research in this area remains limited. We begin our experimental evaluation by benchmarking several state-of-the-art approaches for botnet detection at both _early_ and _late_ stages. Specifically, we evaluate the following methods: (_i_) DRL-LR-NB [53]; (_ii_) PCA-Naive [54]; (_iii_) LSTM-RNN [23]; (_iv_) CNN [55]; (_v_) DT-KNN [56]; (_vi_) TRW [57]; (_vii_) C4.5-DT [58]; (_vi_) OCSVN-GWO [59]; (_ix_) KNN-DT-RF [60]; (_x_) RF-MLPN-LSTM [61]; and (_xi_) DG-CNN [62]. Figure 10 presents the evaluation scores of these methods in detecting botnets. Figure 9: Number of instances across different malware classes. Moreover, in order to perform our analysis over datasets, we have implemented three basic ML algorithms, _K-Nearest Neighbors_ (KNN) [63], _Decision Tree_ (DT) [64], and _Random Forest_ (RF) [65], due to the following reasons: (_i_) KNN is a _simple_, _efficient_, and _easy-to-apply_ algorithm used for classification and regression based on similarity scores such as _Euclidean Distance_. (_ii_) The DT algorithm supports decisions and potential outcomes with a hierarchical tree structure. It uses directed acyclic graphs, starting from a root node that divides into branches until it reaches leaf nodes, using the _entropy coefficient_. (_iii_) RF is a supervised algorithm used for classification and regression. It consists of many decision trees and predicts the final result based on the majority of predictions from all the trees. Figure 11 shows preliminary classification performance in terms of _accuracy_, _recall_, _F1-score_, and _precision_ for all three ML algorithms. As shown in Figure 11, the RF algorithm outperforms the KNN and DT algorithms in terms of all evaluation metrics, including _accuracy_, _precision_, _recall_, and _F1-score_. This superior performance highlights RF ability to effectively classify instances with higher reliability and robustness compared to the other models, making it a more suitable choice. After performing the preliminary analysis using classic ML algorithms, our experiments include both _binary_ and _multi-class_ classifications using our proposed SIR-GN method. Specifically, in the binary classification task, network traffic is categorized as either _malicious_ or _legitimate_. The performance evaluation results for this classification are presented in Figure 12.  
Additionally, we conduct two _multi-class classification tasks_. The first task involves classifying data into three distinct categories, _communication_, _Dissemination_, and _legitimate_, as illustrated in Figure 13. This classification helps in distinguishing between various types of activities based on their nature and intent. Whereas the second task involves a more _granular_ classification with four distinct categories, aimed at identifying and differentiating among the _Mirai_, _Bashlite_, and _Torii_ botnets, along with a fourth class to capture other unidentified threats.