# A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning
## 4 Experiment
### Experiment 2: Ablation Experiment  
In the first experiment, we tried to evaluate the data augmentation module of DeepPTSD. We compared the proposed payload data augmentation mechanism (abbreviated as **AugByKeyword**) with the following three variants of payload data augmentation mechanisms. 1. **NoAug**: It corresponded to the variant that no payload data augmentation was performed. 2. **RandomAug**: It corresponded to the variant that the noises were added to all the tokens without retraining the keywords. Note that the special characters (e.g., %, @, #) were retained. 3. **AugByDic**: It corresponded to the variant that the keywords needed to be retrained were predefined in a dictionary instead of being extracted from the dataset. Here, strategy A was used for evaluation and TextCNN was used as the classifier. The experiment results are shown in Table 3. First, RandomAug had the worst performance and it performed even worse than NoAug. It showed that adding randomized noises without considering the keywords would not increase the diversity of the training samples but even damaged the semantic integrity of the payloads. It verified the effectiveness of the keyword avoidance perturbation mechanism. Second, AugByDic and AugByKeyword outperformed NoAug. It demonstrated the necessity of the payload data augmentation module. Third, AugByKeyword had only a sight advantage over AugByDic. However, AugByDic required manual efforts for defining the dictionary, and thus AugByKeyword had stronger scalability. In the second experiment, we tried to evaluate the transfer learning module and the semi-supervised learning module. Given the five datasets \(D^{\text{PU}},D^{\text{TSL}},D^{\text{TSU}},GD^{\text{TSL}}\), and \(GD^{\text{TSU}}\) (specified in Algorithm 1), we compared the performance of the following three variants of DeepPTSD. 1. **DeepPTSD_None**: It referred to the standard deep learning model. Specifically, it trained the target detection model on \(D^{\text{TSL}}\) based on TextCNN (in Section 3.2). 2. **DeepPTSD_T**: It referred to the transfer learning model with augmented labeled data. Specifically, it trained the initial detection model on \(D^{\text{PU}}\), and fine-tuned it on \(D^{\text{TSL}}\cup GD^{\text{TSL}}\) to get the target detection model based on the transfer learning structure (in Section 3.4). Here, the first two layers of the initial detection model were frozen and we fine-tuned it to minimize the loss function in Equation (3). 3. **DeepPTSD_S**: It referred to the semi-supervised learning model with augmented labeled and unlabeled data. Specifically, it trained the target detection model on \(D^{\text{TSL}}\cup GD^{\text{TSL}}\cup D^{\text{TSU}}\cup GD^{\text{TSU}}\) based on the UDA framework (in Section 3.4). Here, strategy A was used for evaluating DeepPTSD_None, while strategy B was used for evaluating DeepPTSD, DeepPTSD_T, and DeepPTSD_S. All the methods were tested on \(D^{\text{TSU}}\). The experiment results are shown in Table 4 to control sequence. First, DeepPTSD_T outperformed DeepPTSD_None and DeepPTSD_S outperformed AugByKeyword and DeepPTSD_None. It demonstrated that both the transfer learning module and the semi-supervised learning module contributed to the detection model. As a result, DeepPTSD had the best performance. Second, AugByKeyword and DeepPTSD_A outperformed DeepPTSD_T. It indicated that the data augmentation and semi-supervised learning modules were more effective for the detection model than \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **NoAug** & **RandomAug** & **AugByDic** & **AugByKeyword** \\ \hline Precision & 0.8667 & 0.8020 & 0.9012 & 0.9107 \\ Recall & 0.8722 & 0.7944 & 0.8847 & 0.8913 \\ F1-Measure & 0.8694 & 0.7982 & 0.8929 & 0.9009 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The evaluation of the payload data augmentation module. the transfer learning module. This result might be affected by the experiment dataset.