# Unknown Traffic Recognition Based on Multi-Feature Fusion and Incremental Learning
## 4 Evaluation
### Comparison Experiments  
To demonstrate the effectiveness and robustness of the proposed method, experiments were conducted under the five scenarios detailed in Table 5.  
* Scenario A: All unknown traffic in the test dataset has a completely different type of application service than the known traffic in the training dataset. * Scenario B: The unknown traffic in the test dataset has similar application service types as the known traffic in the training dataset. * Scenario C: All unknown attacks in the test dataset have completely different attack mechanisms than the known attacks in the training dataset. * Scenario D: The unknown attacks in the test dataset have similar attack patterns to the known attacks in the training dataset. * Scenario E: All unknown normal and malicious traffic in the test dataset has a completely different type of application service than the known traffic in the training dataset. Figures 7 and 8 show the normalized confusion matrix of the proposed MI-UTR model for the classification results of known and unknown traffic for each of the five scenarios. The vertical coordinate of the confusion matrix is the true label of the traffic, and the horizontal coordinate is the predicted label from the classifier, where the darker the color in the matrix, the higher the probability of correct classification. We calculated the accuracy, \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & \multicolumn{2}{c}{**Setting**} \\ \hline \multicolumn{3}{c}{Scenario A} & Scenario B \\ \hline \multirow{4}{*}{SCX-VPN-Tor} & \multirow{4}{*}{Known Classes} & \multirow{4}{*}{Email, Streaming, File Transfer, P2P} & \begin{tabular}{c} vpn\_sptify\_audio, vpn\_vimeo\_video, vpn\_aim\_chat, \\ vpn\_ftps\_filterrasfer, tor\_mail\_jmap, P2P\_multiplespeed, ic\_chat, \\ tor\_skip\_chat, \\ tor\_filterrasfer\_skipe, tor\_voir\_facebook, tor\_voir\_hangouts \\ \end{tabular} \\ \cline{2-3}  & \multirow{4}{*}{Unnown Classes} & \multirow{4}{*}{Chat, VoIP} & \begin{tabular}{c} vpn\_skype\_audio, vpn\_hangouts\_audio, facebook\_video, \\ vpn\_facebook\_chat, \\ tor\_p2P\_vuze, tor\_facebook\_chat, \\ tor\_video\_youtube, tor\_filterransfer\_ftp, tor\_filterransfer\_ftp, sol \\ \end{tabular} \\ \hline \multirow{4}{*}{NSL-KDD} & \multirow{4}{*}{Known Classes} & Scenario C & Scenario D \\ \cline{2-3}  & & \begin{tabular}{c} back, land, pad, portsweep, satan, \\ rootkit, ftp\_write, guses\_password, multihop, warezclient \\ \end{tabular} \\ \cline{2-3}  & \multirow{2}{*}{Unnown Classes} & \multirow{2}{*}{Probe, U2R} & \begin{tabular}{c} neptune, smurf, ipsweep, nmap, \\ loadmodule, inmap, warezmaster \\ \end{tabular} \\ \hline \multirow{4}{*}{SelfDataset} & \multirow{2}{*}{Known Classes} & \begin{tabular}{c} Altium, Vivado, AutoCAD, \\ Mathematica, exploitonu \\ \end{tabular} \\ \cline{2-3}  & \
\begin{tabular}{c} Unknown Classes \\ \end{tabular} &
\begin{tabular}{c} Cadence, ssh\_bruteforce, miner \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Known traffic and unknown traffic settings for different scenarios. misclassification rate, and \(kappa\) coefficient for each scenario based on the confusion matrix, and these values are indicated below the matrix. In addition to our proposed MI-UTR model, the other methods used for comparison were as follows:  
* CNN-GR [49] employs a standard CNN architecture consisting of three convolutional layers and one fully connected layer to identify unknown traffic through the gradient of the first backpropagation. * Open-CNN [30] model adds an OpenMax layer to the standard CNN model with two convolutional layers and two fully connected layers. The OpenMax layer improves upon the SoftMax layer by adding an extra class \"unknown\" to the activation vectors of SoftMax to detect the unknown attack. * HMCD [50] uses the WGAN-GP generation algorithm to enhance the data and a hybrid neural network based on CNN and LSTM to extract the hierarchical spatial-temporal features of the traffic, which can effectively detect unknown HTTP-based malicious communication traffic.