# Hierarchical Classification of Botnet Using Lightweight CNN
## 4 Experimental Results  
In this investigation, we conducted two primary experiments: the initial involved training and fine-tuning the hierarchical model, while the second replicated the same procedures for the non-hierarchical model. These experiments aimed to evaluate the performance of the proposed HCNN classification and to assess whether the proposed model outperforms the existing state-of-the-art methods. The optimal hyper-parameters value has been obtained using grid-search, as indicated in Table 2.  
Figures 6 and 7 provide a visual representation of the learning curve and confusion matrix associated with the HCNN model. The depicted curve exhibits a notable smoothness, indicating a balanced learning process. Notably, there is an absence of both overfitting and underfitting, crucial aspects in assessing the model's generalization capabilities. This observation is particularly significant, as it suggests that the HCNN model effectively navigates the trade-off between complexity and simplicity, showcasing its ability to learn intricate patterns within the dataset. An absence of overfitting implies that the model does not excessively tailor itself to the training data, avoiding memorization of specific instances at the expense of broader applicability. On the other hand, the lack of underfitting signifies that the model is not oversimplified to the extent of failing to capture essential patterns within the data. The fact that the HCNN model achieves this equilibrium despite its lightweight design underscores its efficiency and competence in learning and generalizing from the given dataset. The outcome of the proposed approach presented in Figure 8 pertains to the classification performance of the models at the initial hierarchy level, encompassing two categories: \"attack\" and \"normal\". Examining the results outlined in Figure 9, it becomes evident that there is no notable performance improvement at the second level attributable to the hierarchical organization of the model, with both models showcasing comparable scores. Considering these observations, it can be inferred that the hierarchical structure's impact is not markedly significant at the first level. Further exploring the hierarchical structure of data during model development offers a primary advantage, particularly in enhancing the performance of the higher levels within the hierarchy. This advantage stems from the fact that the features learned at the initial levels of the hierarchy contribute significantly to distinguishing between classes that may be inherently challenging to separate. This is underscored by the substantial performance improvement observed at the third level hierarchy, as evidenced in Figure 10, where the \(F_{1}\) score for the HCNN stands at 98%, surpassing the non-hierarchical model's score of 93%. The results in Table 3 reveals that the model's performance remains robust despite the imbalanced dataset, as evidenced by the MCC and k values. Our approach involved utilizing the dataset in its natural form, showcasing the model's effectiveness under such conditions. Both evaluation metrics consistently yielded scores above 0.99 at each hierarchical level, affirming the proposed model's robustness and excellent performance, even in the face of an unbalanced dataset. The confusion matrix shown in Figure 7 illustrates the classification performance at each level, revealing minimal misclassifications with the model accurately identifying the majority of instances. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Hyper-Parameters** & **Possible Values** & **Optimal Values** \\ \hline Activation Function & ReLu, Leaky ReLu, tanh & tanh \\ Batch size & 8, 16, 32, 64, 128 & 32 \\ Initial learning rate & 0.01, 0.001, 0.0001 & 0.001 \\ Optimizer & SGD, RMSprop, Adam & Adam \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The potential configurations of hyperparameters and their corresponding optimal settings.