# An LSTM-Based Deep Learning Approach for Classifying Malicious Traffic at the Packet Level
## 4 Evaluation Results  
Similar to most existing deep learning research, our proposed classification model has been implemented using TensorFlow/Keras (version 2.2.4, Google, Mountain View, CA, USA). All of our evaluations have performed on the GPU-enabled TensorFlow which is running on a 64-bit Ubuntu 18.04.2 LTS server with an Intel(R) Xeon(R) E5-2650 2.2 GHz processor, 32 GB RAM, and an NVIDIA RTX Tesla K80. To perform our evaluations, we have used the ISCX2012, USTC-TFC2016, Mirai-RGU and Mirai-CCU datasets. During the training and testing stages, we try to include hundreds to thousands of thousands of packets while balancing the benign and malicious traffic. At the validation stage, we run the trained model on the packets which is extracted randomly per 60 s from the selected datasets, i.e., packets in the consecutive 60 s in the original dataset are extracted while maintaining their temporal order. At this stage, the original (real) traffic is presented to the trained model without any manipulation or balancing of two types of traffic. Since our approach aims to gain a significant reduction in processing time, we have many interests in classifying the incoming packet whether it is malicious or not, instead of considering the attack type in detail. In practice, if a malicious packet is detected by the proposed system, it can raise an alarm to the network administrator and direct the packet for some offline computational intensive traffic classification systems. Therefore, in the following section, we define several measurements to evaluate the performance of the proposed solution in the term of a binary classifier. All of these metrics are all derived from the four values found in the confusion matrix in Table 4. The traffic proportion of training/testing/validation configuration of four datasets is listed in Tables 5-8.  
where:  
* True Positive (TP)--Attack packet that is correctly classified as an attack. * False Positive (FP)--Benign packet that is incorrectly classified as an attack. * True Negative (TN)--Benign packet that is correctly classified as normal. * False Negative (FN)--Attack packet that is incorrectly classified as normal. The accuracy (Equation (1)) measures the proportion of the total number of correct classifications:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN}. \tag{1}\]  
The precision implies the number of correct classifications that is influenced by the incorrect classification (Equation (2)):  
\[Precision=\frac{TP}{TP+FP}. \tag{2}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline  & & **Predicted Classs** & \\ \cline{2-4}  & & **Malicious** & **Benign** \\ \hline \multirow{2}{*}{Ground truth} & Malicious & True Positive (TP) & False Negative (FN) \\  & Benign & False Position (FP) & True Negative (TN) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix. The recall (Equation (3)) measures the number of correct classifications that is considered with the number of missed entries:  
\[Recall=\frac{TP}{TP+FN}. \tag{3}\]  
The false alarm or fair (Equation (4)) measures the proportion of benign packets incorrectly classified as malicious:  
\[Falsealarmrate(FAR)=\frac{FP}{FP+TN}. \tag{4}\]  
The \(F_{1}\)-score (Equation (5)) measures the harmonic mean of precision and recall, which serves as a derived effectiveness measurement:  
\[F_{1}-Score=2\times\frac{Precision*Recall}{Precision+Recall}. \tag{5}\]  
In our experiments, based on the flow statistics in the datasets and number of labeled packets in data pre-processing, we can calculate the number of packets (including benign packets) for training and number of real attack packets in testing, as summarized in Tables 5-8 below. Due to a large amount of input data, we set the mini-batch size value at 100 and the data training with less than 200 epochs. The performance results of our proposal for each kind of malicious traffic with 10-fold testing and 10-fold validation are shown in Tables 9 and 10, respectively.