# Hierarchical Classification of Botnet Using Lightweight CNN
## 3 Materials and Method
### Proposed Approach
For the hierarchical network, we first produce an independent representation for the first level, with \(R_{1}\), given in Equation (2), where \(W_{1}\) is the weight for the first layer. The representation for the second level, denoted as \(R_{2}\), is formulated by performing a matrix multiplication between \(W_{2}\) and the concatenation of \(R_{0}\) with \(R_{1}\), as presented in Equation (3). \[R_{1}=W_{1}\times R_{0}=W_{1}\times GAP(F(X,\theta))=H(X,\theta_{1}) \tag{2}\]  
Figure 4: Developing distinct models for each hierarchical level results in a structure lacking hierarchy. (**a**) Represents the first-level classification with two classes, (**b**) represents the second-level classification with five classes, and (**c**) represents the third-level classification with 11 classes. Figure 5: Proposed hierarchical classification of botnet. \[R_{2}=W_{2}\times(R_{0}\oplus R_{1})=G(X,\theta_{2}) \tag{3}\]  
Finally, the representation for the third layer, denoted as \(R_{3}\), is expressed in Equation (4), where \(W_{3}\) denotes the weight matrix operating on the concatenation of vectors \(R_{0}\), \(R_{1}\), and \(R_{2}\). \[R_{3}=W_{3}\times(R_{0}\oplus R_{1}\oplus R_{2})=T(X,\theta_{3}) \tag{4}\]  
The loss of the network, Equation (5), is calculated by summing the loss of each level, \(loss_{l}\), where the losses are calculated using binary cross entropy given in Equation (6) for the first level, and categorical cross entropy as presented in Equation (7) for level 2 and 3. \[loss=\sum_{l=1}^{3}loss_{l} \tag{5}\]  
where the variable \"\(l\)\" denotes the hierarchical level within the model, and specifically, there are three hierarchical levels in consideration. \[loss_{1}=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}\cdot\log(p_{i})+(1-y_{i})\cdot\log( 1-p_{i})] \tag{6}\]  
where \(N\) is the total number of samples, \(y_{i}\) is the true label of the \(i^{th}\) sample (either 0 or 1), and \(p_{i}\) is the predicted probability of the \(i^{th}\) sample belonging to class 1. \[loss_{2/3}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{C}y_{ij}\cdot\log(p_{ij}) \tag{7}\]  
where \(N\) is the total number of samples, \(C\) is the number of classes, \(y_{ij}\) is a ground truth label for the \(i^{th}\) sample and belongs to class \(j\), and \(p_{ij}\) is the predicted probability that the \(i^{th}\) sample belongs to class \(j\). To guarantee the acquisition of accurate category structures, it is essential to introduce a supplementary loss function known as the dependency loss. The dependency loss, \(loss_{l}\), is calculated by Equation (8) as discussed in [24]. This loss function is formulated by assessing whether the model's predictions conform to the anticipated hierarchical relationships among classes in the classification task. When the predicted classes of consecutive layers deviate from the parent-child relationship within the hierarchy, this loss penalizes the model, intending to promote the acquisition of precise category structures. The definition of the loss function hinges on evaluating whether the model's predictions align with the expected hierarchical relationships among classes in the classification task. \[dloss_{l}=-(loss_{l-1})^{D_{l}l_{i-1}}(loss_{l})^{D_{l}l_{l}} \tag{8}\]  
where \(D_{l}\), and \(I_{l}\) determine if the model output conflicts with the hierarchical structure. \[D_{l}= \begin{cases}1&\text{if }p_{l}\gg p_{l-1}\\ 0&\text{otherwise}\end{cases}  \tag{9}\]  
\[I_{l}= \begin{cases}1&\text{if }p_{l}\
eq y_{l}\\ 0&\text{otherwise}\end{cases}  \tag{10}\]  
where \(p_{l}\) and \(y_{l}\) denote the prediction and actual label at level \(l\), respectively. If the category prediction at level \(l\) aligns with the prediction at level \(l-1\), D is set to 0; otherwise, it is set to 1. Similarly, if the prediction at level \(l\) aligns with the actual value at the same level, I is set to 0; otherwise, it is set to 1.