# Lightweight Model for Botnet Attack Detection in Software Defined Network-Orchestrated IoT
## 4 Experimental Results and Discussion  
In this section, the results of the 3 \(\times\) 3, and 5 \(\times\) 5 CNN models, the fused 3 \(\times\) 3-5 \(\times\) 5 CNN models, and other classical machine learning models will be presented and discussed one after the other. The result discussed in this section is obtained with batch size, epoch, and initial learning rate of 32, 150, and 0.001, respectively. Categorical cross-entropy loss is the function used to determine classification loss with the Adam optimizer. The 3 \(\times\) 3 model achieved 97.8% accuracy, 98% precision, 98% recall, and 0.98 F1 score as presented in Table 3. Its running time, memory requirement, and the number of parameters are 0.113 sec, 235 KB, and 47,622 respectively. Removing the final BN will increase the accuracy to 99.2%. The precision, recall, and F1 score increase to 99%. The other effects of removing the BN layer are a decrease in running time, the number of parameters, and memory requirement. In the same manner, the performance for 5 \(\times\) 5, and 3 \(\times\) 3-5 \(\times\) 5 architectures presented in Table 4 and Table 5 respectively, improves upon removing the final BN layers. The 3 \(\times\) 3-5 \(\times\) 5 architecture has identical performance in terms of F1 score, recall, accuracy, and precision. However, its running time and memory requirement increase significantly, although the loss decreases. The comparison of the proposed architecture is found in Figures 3 and 4. In comparison to the one with batch normalization, the performance of 3 \(\times\) 3 without BN is better. With F1 scores of 0.93 and 0.94 for the OS fingerprinting, and port scanning attack and 0.99 for both classes in the network without BN, there has been a significant increase in the detection of these attacks while removing the BN. For the network architecture of 5 \(\times\) 5, and 3 \(\times\) 3-5 \(\times\) 5, the detection of OS fingerprinting and port scanning attacks follows the patterns we saw in the 3 \(\times\) 3 architecture. Removal of the BN has also increased the detection of these attacks, with F1 scores of 0.97 for attack detection in the network with BN and 0.99 for both in the network without BN. The 5 \(\times\) 5 architecture has the lowest test loss of all of the suggested architectures with a loss of 0.008, and the 3 \(\times\) 3 architecture comes in second with a loss of 0.01 points. The 3 \(\times\) 3-5 \(\times\) 5 architecture's loss, however, has the highest value at 0.011. Figure 4: Comparison of the three proposed lightweight models in terms of loss. Figure 3: Comparison of the three proposed lightweight models in terms of accuracy. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Attack Type/Label**} & \multicolumn{2}{c}{**With Batch Normalization before GAP**} & \multicolumn{2}{c}{**Without Batch Normalization before GAP**} \\ \cline{2-7}  & **Precision (\%)** & **Recall (\%)** & **F1 Score** & **Precision (\%)** & **Recall (\%)** & **F1 Score** \\ \hline DDoS & 100 & 100 & 1.00 & 99 & 100 & 0.99 \\ \hline DoS & 100 & 98 & 0.99 & 100 & 99 & 0.99 \\ \hline Fuzzing & 100 & 100 & 1.00 & 100 & 100 & 1.00 \\ \hline Normal & 100 & 100 & 1.00 & 100 & 100 & 1.00 \\ \hline OS Fingerprinting & 100 & 88 & 0.93 & 100 & 98 & 0.99 \\ \hline Port Scanning & 89 & 100 & 0.94 & 98 & 100 & 0.99 \\ \hline
**Average** & **98** & **98** & **0.98** & **99** & **99** & **0.99** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The 3 \(\times\) 3 with and withoutbatch normalization before the global average pooling. We found that the performance of our proposed model has improved in terms of computational resource requirements and classification accuracy when compared to the performance of the current state-of-the-art models. While SVM and XG-Boost perform similarly to our method in terms of accuracy, the proposed CNN models perform significantly better in terms of running time and memory requirements.