# TransECA-Net: A Transformer-Based Model for Encrypted Traffic Classification
## 4 Experimental Result
### Comparative Experiment  
To further validate the performance of the TransECA-Net model, this paper compares it with several existing network traffic classification methods on the ISCX dataset. [PERSON] et al. [20] proposed an end-to-end network traffic classification approach based on 1D-CNN. [PERSON] et al. [21] introduced a CNN + SAE-based method, where the use of SAE improved the performance of traffic classification tasks. [PERSON] [23] proposed a network traffic classification method combining LSTM and CNN, integrating LSTM into the CNN architecture and adding the SENet attention mechanism, which significantly enhanced the classification performance for VPN traffic. [PERSON] et al. [24] proposed a temporal fusion encoder using graph neural networks (TFE-GNN) model for fine-grained encrypted traffic classification, which demonstrates superior performance in classification accuracy through graph-structured learning of packet interaction patterns. Under the same conditions of using the ISCX international public dataset, the TransECA Net model was compared with these existing literature classification methods, and the comparison results are shown in Table 6. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**VPN**} & \multicolumn{3}{c}{**Non-VPN**} \\ \cline{2-7}  & **Accuracy** & **Precision** & **Recall** & **Accuracy** & **Precision** & **Recall** \\ \hline
1D-CNN [20] & 94.48\% & 94.48\% & 93.57\% & 83.94\% & 83.89\% & 84.00\% \\ SAE + 1D-CNN [21] & 93.00\% & 97.80\% & 96.30\% & 87.62\% & 86.70\% & 88.80\% \\ CNN + LSTM [23] & 93.69\% & 94.57\% & 94.33\% & 90.91\% & 91.69\% & 92.00\% \\ TFE-GNN [24] & 95.91\% & 95.26\% & 95.36\% & 90.40\% & 93.16\% & 91.90\% \\ OUR & 98.90\% & 98.20\% & 97.93\% & 98.39\% & 97.43\% & 98.02\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The comparison between TransECA-Net and other methods. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Precision** & **Recall** & **Number of Parameters** & **FLOPs** \\ \hline Baseline CNN & 93.89\% & 93.37\% & 1.82 M & 0.36 G \\ CNN + ECANet & 96.88\% & 96.21\% & 1.90 M & 0.39 G \\ Transformer-only & 95.30\% & 94.07\% & 2.65 M & 1.24 G \\ TransECA-Net (full model) & 98.64\% & 98.13\% & 6.89 M & 13.45 G \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of experimental results of component ablation. As shown in Table 5, the 1D-CNN and its improved version, SAE + 1D-CNN, demonstrate excellent performance in VPN traffic classification, with accuracy, precision, and recall all exceeding 90%. However, on the non-VPN dataset, both methods experience a significant drop in performance, with the 1D-CNN achieving an accuracy of only 83.94% and SAE + 1D-CNN reaching just 87.62%. Although the CNN-LSTM hybrid model reported in the literature performs exceptionally well in both VPN and non-VPN traffic classification (with VPN accuracy exceeding 97% and non-VPN accuracy above 95%), in our local experimental setup, the optimal classification accuracy for this model was only 93.69% for VPN traffic and 90.91% for non-VPN traffic. By contrast, the TFE-GNN method, which integrates byte-level traffic graphs with graph neural networks, demonstrates significant advantages in fine-grained feature extraction, complex structure modeling, and multi-dimensional feature fusion, achieving a VPN traffic classification accuracy of 95.91%. Notably, the first four methods generally exhibit inadequate performance in non-VPN traffic classification, with average metric declines ranging from 5% to 8%. The proposed TransECA-Net model excels in both traffic classification tasks, with average accuracy, precision, and recall exceeding 98%. The experimental results indicate that this model achieves significant performance improvements in encrypted traffic classification, showcasing clear advantages over existing methods.