# AE-DTI: An Efficient Darknet Traffic Identification Method Based on Autoencoder Improvement
## 4 Experiment and Analysis
### AE-DTI Effectiveness Experiment
#### 4.4.3 AE-DTI Generalization Experiment  
To verify the generalization of AE-DTI, this paper conducts comparative experiments on the CSE-CIC-IDS2018 dataset and the CIC-Darknet2020 dataset. Various dimensionality reduction methods (PCA, LLE, ISOMAP, autoencoder, and AE-FS) were used to reduce the dimensionality of the two datasets to 49 and 64 dimensions, and then model training and classification experiments were performed, respectively. The experimental results are shown in Tables 6 and 7.  
According to the results in Table 6, when the dataset is downscaled to 49 dimensions, for in the CSE-CIC-IDS2018 dataset, the model trained by the dataset downscaled by AE-FS outperforms the other methods, with Acc, F1 value, and Recall improving by 1.5%, 0.8%, and 1.5% compared with PCA downscaling. It is slightly lower than the model trained on the dataset via ISOMAP dimensionality reduction in Pre, with a difference of about 0.1%. For the CIC-Darknet2020 dataset, the model trained on the dataset downscaled by AE-FS outperforms the other methods in Acc, Pre, F1 value, and Recall. At the same time, it is worth noting that for unbalanced datasets such as CIC-Darknet2020, AE-IDT still shows good classification results.  
According to the results in Table 7, when the dataset is downscaled to 64 dimensions, for the CSE-CIC-IDS2018 dataset, the models trained on the dataset downscaled by AE-FS outperform the other methods, with the Acc, F1 value, Pre, and Recall improving by 3.9%,  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Algorithm** & **Acc** & **F1** & **Pre** & **Recall** \\ \hline \multirow{6}{*}{CSE-CIC-IDS2018} & AE-FS & 96.6\% & 96.3\% & 96.1\% & 96.6\% \\  & AUTO & 95.8\% & 95.8\% & 95.8\% & 95.7\% \\  & ISOMAP & 95.9\% & 96.1\% & 96.2\% & 95.9\% \\  & LLE & 96.1\% & 96.1\% & 96.1\% & 96.1\% \\  & PCA & 95.1\% & 95.3\% & 95.5\% & 95.1\% \\ \hline \multirow{6}{*}{CIC-Darknet2020} & AE-FS & 87.1\% & 86.9\% & 86.8\% & 87.1\% \\  & AUTO & 85.8\% & 85.3\% & 85\% & 85.8\% \\ \cline{1-1}  & ISOMAP & 84.0\% & 84.2\% & 84.4\% & 84\% \\ \cline{1-1}  & LLE & 85.1\% & 84.4\% & 83.6\% & 85.1\% \\ \cline{1-1}  & PCA & 85.7\% & 85.3\% & 84.9\% & 85.7\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Experimental classification results with a feature number of 49.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Algorithm** & **Acc** & **F1** & **Pre** & **Recall** \\ \hline \multirow{6}{*}{CSE-CIC-IDS2018} & AE-FS & 98.8\% & 98.8\% & 98.9\% & 98.8\% \\  & AUTO & 98.3\% & 98.4\% & 98.4\% & 98.3\% \\  & ISOMAP & 95.8\% & 96\% & 96\% & 95.8\% \\  & LLE & 96.8\% & 96.8\% & 96.9\% & 96.8\% \\  & PCA & 94.9\% & 95.2\% & 95.4\% & 95\% \\ \hline \multirow{6}{*}{CIC-Darknet2020} & AE-FS & 89.5\% & 89.6\% & 89.6\% & 89.5\% \\  & AUTO & 85.8\% & 85.3\% & 85\% & 85.8\% \\ \cline{1-1}  & ISOMAP & 83.8\% & 84.2\% & 84.5\% & 83.8\% \\ \cline{1-1}  & LLE & 84.8\% & 84\% & 83.3\% & 84.8\% \\ \cline{1-1}  & PCA & 85.4\% & 84.9\% & 84.5\% & 85.4\% \\ \hline \hline \end{tabular}
\end{table}
Table 7: Experimental classification results with a feature number of 64.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Accuracy** & **Pre** & **Recall** \\ \hline [PERSON] et al. [21] & - & 84.3\% & 83.8\% \\ [PERSON] et al. [29] & 91\% & 91\% & 91\% \\ [PERSON] et al. [30] & - & 95.5\% & - \\ AE-DTI (25 features) & 98.8\% & 98.6\% & 98.7\% \\ AE-DTI (20 features) & 98.6\% & 98.5\% & 98.7\% \\ AE-DTI (18 features) & 98.5\% & 98.5\% & 98.5\% \\ AE-DTI (16 features) & 98.3\% & 98.3\% & 98.4\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance of different dark network traffic identification methods.  
3.6%, 3.5%, and 3.8% compared with PCA downscaling. For the CIC-Darknet2020 dataset, the models trained on the dataset with AE-FS dimensionality reduction also outperformed the other methods in terms of Acc, F1 value, Pre, and Recall. At the same time, it is worth noting that for unbalanced datasets such as CIC-Darknet2020, AE-IDT still shows good classification results.