# Intelligent Detection of IoT Botnets Using Machine Learning and Deep Learning
## 4 Experimental Evaluation
### Multiclass Classification
The model also mis-detected the ACK and scan of Mirai. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Botnet**} & \multirow{2}{*}{**Attack**} & \multirow{2}{*}{**Device**} & \multirow{2}{*}{**Benign**} & \multicolumn{4}{c}{**Bashlite**} & \multicolumn{4}{c}{**Mirai**} \\ \cline{4-11}  & & & & **COMBO** & & **Junk** & **Scan** & **TCP** & **UDP\({}^{\,1}\)** & **ACK** & **Scan** & **SYN** & **UDP \({}^{\,1}\)** & **Plain UDP \({}^{\,2}\)** \\ \hline \multirow{8}{*}{Mirai} & \multirow{4}{*}{**ACK**} & **Doorbell** & 0 & 0 & 0 & 1 & 0 & 0 & 2 & 0 & 0 & **29,856** & 895 \\  & & **Baby Monitor** & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & **25,858** & 1727 \\  & & **Security Camera** & 0 & 0 & 0 & 0 & 0 & 4 & 4 & 0 & 0 & **17,441** & 0 \\  & & **Webcam** & 5 & 0 & 0 & 2 & 0 & 0 & 628 & 0 & 0 & **30,584** & 1286 \\ \cline{2-11}  & \multirow{4}{*}{**SYN**} & **Doorbell** & 0 & 25,415 & 0 & 4 & 0 & 1 & 0 & **11,198** & 170 & 0 & 0 \\  & & **Baby Monitor** & 1 & **23,909** & 0 & 5 & 0 & 0 & 0 & **10,823** & 787 & 0 & 0 \\  & & **Security Camera** & 0 & **9066** & 23 & 14 & 0 & 4 & 0 & **8155** & 1391 & 0 & 0 \\  & & **Webcam** & 0 & 20,428 & 0 & 20 & 0 & 0 & 0 & **8413** & 7646 & 0 & 0 \\ \cline{2-11}  & \multirow{4}{*}{**UDP\({}^{\,1}\)**} & **Doorbell** & 0 & 0 & 0 & 0 & 0 & 4 & 2 & 0 & 0 & 70,420 & **904** \\  & & **Baby Monitor** & 0 & 0 & 0 & 0 & 3 & 0 & 0 & 0 & 64,477 & 518 \\  & & **Security Camera** & 0 & 0 & 0 & 2 & 0 & 2 & 0 & 0 & 0 & 47,510 & **0** \\  & & **Webcam** & 0 & 0 & 5 & 0 & 3 & 630 & 0 & 0 & 45,624 & **1011** \\ \cline{2-11}  & \multirow{4}{*}{**Plain UDP\({}^{\,2}\)**} & **Doorbell** & 2 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & **5344** & 19,001 \\ \cline{2-11}  & & **Baby Monitor** & 2 & 0 & 0 & 4 & 0 & 0 & 0 & 0 & 0 & **4354** & 19,898 \\ \cline{2-11}  & & **Security Camera** & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & **8216** & 7834 \\ \cline{2-11}  & & **Webcam** & 1 & 0 & 0 & 5 & 0 & 0 & 0 & 0 & **4250** & 20,827 \\ \hline \hline \end{tabular} The red bolded numbers are large numbers of misclassified samples; Gray shaded numbers indicate samples that are correctly classified; \({}^{\,1}\) UDP: UDP flooding; \({}^{\,2}\) Plain UDP: Less of an option of UDP flooding for higher packets per second.  
\end{table} 
Table 14: Average F1-score of the DL models. For RNN, the F1-score of each specific attack is shown in Table 15. The model correctly detected the benign and UDP of Mirai. For Bashlite attacks, the model mis-detected COMBO as benign (security camera). It mis-detected Junk as a scan of Bashlite (doorbell), the UDP of Mirai (baby monitor), benign (security camera), and a COMBO of Bashlite (webcam). It mis-detected scan as benign (baby monitor, security camera), TCP as the UDP of Mirai (doorbell), the UDP of Bashlite (baby monitor), benign (security camera), and the UDP of Bashlite (webcam). It also mis-detected UDP as the UDP of Mirai (doorbell) and benign (security camera). For Mirai attacks, the model mis-detected ACK as a COMBO of Bashlite (webcam) and scan as the Ack of Mirai (doorbell) and the scan of Mirai (baby monitor). It mis-detected SYN as the scan of Mirai (doorbell), the COMBO of Bashlite (baby monitor), and benign (security camera, webcam). It also mis-detected Plain UDP as the UDP of Mirai (doorbell, security camera) and benign (webcam). The F1-score of each specific attack for LSTM is shown in Table 16. The model correctly detected benign and COMBO Bashlite attacks. For Bashlite attacks, the model mis-detected junk as a COMBO of Bashlite (doorbell), the SYN of Mirai (baby monitor), and a COMBO of Bashlite (security camera). It mis-detected scan as the SYN of Mirai (baby monitor) and benign and TCP as the ACK of Mirai (doorbell, webcam), the UDP of Bashlite (baby monitor), and benign (security camera). It also mis-detected UDP as the Ack of Mirai (doorbell), benign (security camera), and the SYN of Mirai (webcam).