## 6 Model Tuning
### Performance Metrics  
An unambiguous and thorough way to present the prediction results of a deep learning mode is to use a confusion matrix. Table 5 shows the confusion matrix produced by the test process of the multi-layer LSTM model with the optimal hyper-parameters and a flow-based dataset. For example, the model correctly predicts 681 cases of RDP, 571 cases of Skype, 670 cases of SSH, 694 cases of BitTorrent, 642 cases of HTTP-Facebook, 637 cases of HTTP-Wikipedia, 656 cases of HTTP-Google, and 598 cases of HTTP-Yahoo. It also misclassified 250 cases (all cases outside the diagonal positions in Table 5) out of the total number of all predicted cases. It also shows that the F1-scores of each application label retain at the high level with strong robustness, which confirms that the multi-layer LSTM model can effectively and stably distinguish network application. However, the accuracy can be misleading when there is an imbalance on the number of application labels. A model can predict the label of the majority application for all predictions and achieve a high classification accuracy, and the model is not useful in the problem domain. For every application label, therefore, we convert the multi-class confusion matrix into the binary confusion matrix. Table 6 is an example of the binary confusion matrix for the RDP application label, which is derived from Table 5. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline  & & & \multicolumn{6}{c}{**Predictive**} \\ \cline{3-11}  & \(n\) = 5400 & \multicolumn{2}{c}{**Positive**} & \multicolumn{2}{c}{**Negative**} \\ \hline \multirow{2}{*}{**Actual**} & Positive & \multirow{2}{*}{681 (True Positive: TP)} & \multirow{2}{*}{1 (False Negative: FN)} \\ \cline{3-11}  & & & & & & & & & & \\ \cline{3-11}  & & Negative & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The binary confusion matrix for the RDP application label in the multi-layer LSTM model test. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Application Label**} & \multicolumn{6}{c}{**Predicted**} \\ \cline{3-11}  & **RDP** & **Skype** & **SSH** & **BitTorrent** & \begin{tabular}{c} **HTTP-** \\ **Facebook** \\ \end{tabular} & \begin{tabular}{c} **HTTP-** \\ **Wikipedia** \\ \end{tabular} & \
\begin{tabular}{c} **HTTP-** \\ **Google** \\ \end{tabular} &
\begin{tabular}{c} **HTTP-** \\ **Yahoo** \\ \end{tabular} & **Sum** & **F1-score** \\ \hline \multirow{5}{*}{**HTTP-Yahoo**} & RDP & 681 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 682 & 1.00 \\  & Skype & 0 & 571 & 0 & 11 & 0 & 0 & 0 & 0 & 582 & 1.00 \\  & SSH & 2 & 0 & 670 & 0 & 0 & 0 & 0 & 0 & 672 & 1.00 \\  & BitTorrent & 0 & 2 & 0 & 694 & 0 & 1 & 0 & 0 & 697 & 1.00 \\  & HTTP-Facebook & 0 & 1 & 3 & 0 & 642 & 25 & 1 & 2 & 674 & 0.97 \\  & HTTP-Wikipedia & 0 & 0 & 0 & 0 & 45 & 637 & 1 & 26 & 709 & 0.97 \\  & HTTP-Google & 0 & 0 & 0 & 0 & 26 & 6 & 656 & 0 & 688 & 0.99 \\  & HTTP-Yahoo & 0 & 0 & 0 & 0 & 40 & 57 & 0 & 598 & 695 & 0.98 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The multi-class confusion matrix by the multi-layer LSTM model test with the dataset of 60 sequential packets per flow and 1024 pixels of payload size (The bold line is used to convert this multi-class confusion matrix into the binary confusion matrix of the RDP application label shown in Table 6). In the binary confusion matrix described by Table 6, we observe that, out of the total of 5400 RDP prediction cases, the multi-layer LSTM model predicts 683 (= 681 + 2) cases as RDP, and predict 4717 (= 1 + 4716) cases at the others. In actual, 682 (= 681 + 1) test datasets are RDP, while 4718 (= 2 + 4716) test datasets are the others. The true positive (TP) indicates the cases in which the actual label is positive (RDP) and the model prediction is also positive correctly. The false negative (FN) indicates the cases in which the actual label is positive, but the model prediction is negative incorrectly.