# TransECA-Net: A Transformer-Based Model for Encrypted Traffic Classification
## 4 Experimental Result
### Comparative Experiment
To facilitate a more comprehensive comparison of the temporal efficiency associated with various models during the training process, this study meticulously assesses both the training duration and convergence period for each model. The training duration is defined as the time necessary for the model to be trained on the training dataset, and it is influenced by several factors, including the number of iterations, model complexity, and the specific hardware utilized in the experiments. Conversely, convergence time pertains to the duration required for the model to attain a state of relatively stable performance, which is likewise affected by variables such as model complexity. As illustrated in Table 7, the time expenditure of diverse models operating on the ISCX dataset under consistent hardware conditions indicates that the proposed methodology significantly mitigates convergence time. Specifically, it necessitates only 51.16% of the duration required by the 1D-CNN, 53.78% of that for SAE + 1D-CNN, 62.56% of that for the CNN + LSTM model, and 46.87% of that for the TFE-GNN model. Consequently, it can be inferred that the TransECA-Net architecture accomplishes the classification task with reduced temporal expenditure during the training phase, thereby achieving enhanced classification efficiency in comparison to other methodologies. To comprehensively evaluate the performance of TransECA-Net, we conducted comparative experiments with baseline models including 1D-CNN, SAE + 1D-CNN, CNN-LSTM, and TFE-GNN on encrypted traffic datasets. To ensure the rigor of the performance comparison, each model underwent 30 independent repeated experiments, with the training and test sets partitioned at an 80%:20% ratio using random seeds for each trial. Paired t-tests were performed between TransECA-Net and all baseline models on the same test set: \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Training Time (s)** & **Convergence Time (s)** \\ \hline
1D-CNN [20] & 2104.89 & 720.54 \\ SAE + 1D-CNN [21] & 2215.78 & 685.36 \\ CNN + LSTM [23] & 1967.76 & 589.17 \\ TFE-GNN [24] & 1768.21 & 780.05 \\ OUR & 1001.20 & 368.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of operational efficiency of different models. Paired comparisons of accuracy rates between TransECA-Net and each baseline model on the identical test set were performed, with \(p\)-values calculated through statistical analysis. Null hypothesis (H\({}_{0}\)): No significant difference in performance between the two models (\(\mu\)TransECA-Net = \(\mu\)Baseline). Alternative hypothesis (H\({}_{1}\)): TransECA-Net performs significantly better (\(\mu\)TransECA-Net > \(\mu\)Baseline). Significance level: \(\alpha\) = 0.05 (one-tailed test). Compute [PERSON]'s d to quantify the magnitude of differences:  
\[d=\frac{\mu TransECAnet-\muBaseline}{\sigmapool}\]  
where  
\[\sigmapool=\sqrt{\frac{(n_{1}-1)\sigma_{1}^{2}+(n_{2}-1)\sigma_{2}^{2}}{n_ {1}+n_{2}-2}}\]  
The results demonstrate statistically significant improvements (\(p<0.001\), single-tailed) for TransECA-Net over 1D-CNN ([PERSON]'s d = 2.1), SAE + 1D-CNN (d = 1.8), CNN-LSTM (d = 1.6), and TFE-GNN (d = 1.2). The large effect sizes (d > 0.8) further confirm that the performance gaps are not only statistically significant but also practically meaningful. In deep learning tasks, it is generally desirable for the loss value to gradually decrease during training, as the loss is a metric that measures the difference between the model's predictions and the true labels. As the model learns better feature representations and adjusts its parameters, the loss should steadily decline, indicating that the model's predictions on the training data are improving. Figures 5-8 respectively show the loss curves during training for the three aforementioned models and the TransECA-Net model used in this study. The \(x\)-axis represents each training batch, while the \(y\)-axis indicates the loss value.