# Research on the Simulation Method of HTTP Traffic Based on GAN
## 5 Evaluation
### Evaluation Results
#### 5.4.2 Evaluation Metrics Based on Machine Learning  
To evaluate the quality of our generated data, we employed common neural network classifiers, namely MLP, SVM, Random Forest, Decision Tree, and Multinomial Logistic Regression. We assessed the performance of our data using evaluation metrics such as accuracy (Acc), AUC, and F1-Score.  
For MLP, we set the number of neurons to 256, while for Random Forest and Decision Tree, the depth was set to 56. Our objective was to measure the probability of our generated data being classified as fake data from the perspective of neural network classification. Figure 12 illustrates the accuracy values (Acc) of different models under the evaluation of the selected five neural network classifiers.  
In Table 6, the left column represents five specific methods based on neural network classification for evaluating the generated data, and the numerical values in the table represent the accuracy (Acc) of different models corresponding to their classification results. From Table 4, it can be observed that SEGAN outperformed other methods in terms of the performance measures across the five neural networks. Additionally, it was evident that the CGAN(Trans-EV) model, which incorporated the evolutionary algorithm (EV), generally exhibited lower successful identification probabilities compared to CGAN(Trans) that did not use the EV algorithm. This indicated the effectiveness of the EV algorithm proposed in this paper for network traffic data generation. Furthermore, the Acc values of AE-GAN, except for the dt column where it surpassed OCT-GAN, were consistently lower than those of OCT-GAN. This suggested that AE-GAN performed better than OCT-GAN in synthesizing network datasets.  
Figure 12: The number of nodes and packets is displayed.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**METHOD**} & \multicolumn{5}{c}{**MODEL**} \\ \cline{2-6}  & **SE-GAN** & **CGAN(Trans)** & **CGAN(Trans-EV)** & **AE-GAN** & **OCT-GAN** \\ \hline Corr & 0.713 & 0.431 & 0.508 & 0.611 & 0.591 \\ WD & 0.013 & 0.041 & 0.034 & 0.027 & 0.021 \\ JSD & 0.074 & 0.115 & 0.098 & 0.087 & 0.091 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Statistical similarity.  
In Table 7, we present the different scores of various models on the AUC metric, where our proposed SEGAN model outperformed other models such as AE-GAN and OCT-GAN. Since the AUC value measured the reasonableness of a model's ranking of a series of samples, and because we used the same GMM network to process different data, lower data indicators indicate better model performance. Thus, it was evident that the data quality generated by the AE-GAN model was higher than that of OCT-GAN. Analyzing the underlying reasons, OCT-GAN adopted a traditional one-hot encoding plus noise input form and applied specific pattern normalization to numerical features, resulting in sparse input and excessive sensitivity to data changes. However, both AE-GAN and SEGAN employed an autoencoder to map and compress input data, thereby exhibiting stronger robustness.  
As shown in Table 8, we presented the F1_score values corresponding to different models. By comparing the scores of SE-GAN and CGAN(Trans-EV), it was evident that SE-GAN achieved a significantly lower F1_score than the latter. This suggested that using the Swin Transformer block as the basic module in the generator led to better accuracy in generating different classes of network data compared to the traditional transformer structure. From the four tables presented above, it was observed that our SEGAN model consistently outperformed the state-of-the-art methods, AE-GAN and OCT-GAN. Additionally, in terms of the time required for model training to achieve similar performance, SEGAN required the shortest time, highlighting the effectiveness of our proposed approach.