# An LSTM-Based Deep Learning Approach for Classifying Malicious Traffic at the Packet Level
## 4 Evaluation Results
From these two tables, we can observe that our method can perform the classification with nearly 100% accuracy and precision in major cases of the security attacks, which outperforms compared to the prior works, e.g., [6; 19; 20]. However, since the characteristics of traffic types and attacks  
One might question the aforementioned results of our proposed DL-based framework. That is, intuitively, the proposed framework can classify malicious packets because it obtains the features of malicious packets from the training dataset. Thus, we perform a validation test in which the Mirai-RGU dataset is used for training and packets from Mirai-CCU dataset are used for validation. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Accuracy** & **Precision** & **Recall** & **F-Score** & **FPR** \\  & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline ISCX-IDS-2012 & 99.99 & 99.98 & 99.99 & 99.99 & \(7.46\times 10^{-7}\) \\ USTC-TFC-2016 & 99.99 & 100 & 99.99 & 99.99 & \(1.1\times 10^{-7}\) \\ Mirai-RGU & 100 & 100 & 100 & 100 & 0 \\ Mirai-CCU & 99.46 & 99.63 & 99.38 & 99.51 & 0.026 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Performance evaluation results of the proposed solution on the four datasets with 10-fold validation. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Benign**} & \multicolumn{3}{c}{**Malware**} \\ \hline
**Type** & **All** & **Train/Test** & **Validate** & **Type** & **All** & **Train/Test** & **Validate** \\  & **All** & **(avg/60 s)** & **Type** & **All** & **Train/Test** & **(avg/60 s)** \\ \hline Facetime & 6000 & 6000 & 6000 & Tinba & 22,000 & 22,000 & 729 \\ Skype & 12,000 & 12,000 & 12,000 & Zeus & 93,141 & 93,141 & 105 \\ BitTorrent & 15,000 & 15,000 & 15,000 & Shifu & 500,000 & 100,000 & 3 \\ Gmail & 25,000 & 25,000 & 25,000 & Neris & 499,218 & 100,000 & 896 \\ Outlook & 15,000 & 15,000 & 15,000 & Cridex & 461,548 & 100,000 & 34 \\ World & & & & & & \\ Of & 140,000 & 100,000 & 140,000 & Nsis-ay & 352,266 & 100,000 & 5617 \\ Warcraft & & & & & & \\ MySQL & 200,000 & 100,000 & 200,000 & Geodo & 250,000 & 100,000 & 12 \\ FTP & 360,000 & 100,000 & 360,000 & Miuref & 88,560 & 88,560 & 7 \\ SMB & 925,453 & 200,000 & 925,453 & Virut & 440,625 & 100,000 & 858 \\ Weibo & 1,210,060 & 100,000 & 1,210,060 & Total & 2,707,358 & 803,701 & - \\ Total & 2,908,513 & 1,058,000 & - & & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Train/Test/Validate traffic proportion in the USTC-TFC-2016 dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**File** & **Nof of Pcap** & **All (Benign, Attack)** & **Train/Test** & **Validate(avg,/60 s)** \\ \hline Capture\_2 & 3 (N) & (129,178, 0) & 100,000 & 14,100 \\ Capture\_3 & 1 (M) & (54,61, 393,325) & 100,000 & 9795 \\ Capture\_4 & 6 (N, M) & (268,461,518,105) & 100,000 & 10,010 \\ Capture\_5 & 1 (N, M) & (67,239,519,376) & 100,000 & 9780 \\ Capture\_6 & 1 (N, M) & (66,989,519,609) & 100,000 & 9781 \\ Capture\_7 & 1 (N, M) & (67,061, 519,400) & 100,000 & 9783 \\ Capture\_8 & 1 (N, M, S) & (66,801, 981,651) & 100,000 & 9996 \\ Capture\_9 & 12 (N, M, G Our experimental results show that, even though the training and validation stages use different datasets, the proposed framework is still able to detect malicious packets with very high accuracy. The performance results are shown in Table 11.