# A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning
## 4 Experiment
### Experiment 3: Comparison Experiment  
To evaluate the competitive performance of DeepPTSD, we compared it with the following five baselines.  
1. **SVM_C**: It corresponded to the character-level feature based method. Specifically, it first segmented each HTTP traffic sample into characters, and built a feature vector to reflect the character distribution based on VSM (vector space model). Then, it trained a SVM (support vector machine) classifier based on the character feature vectors.
2. **SVM_W**: It corresponded to the word-level feature based method [18]. Specifically, it first segmented each HTTP traffic sample into words based on the approach proposed in Section 3.2, and built a feature vector to reflect the word distribution based on VSM. Then, it trained a SVM classifier based on the word feature vectors.
3. **CNN**: It trained the detection model based on the TextCNN model [14]. Here, a HTTP traffic sample was segmented based on the approach proposed in Section 3.2.
4. **RNN**: It trained the detection model based on a BiLSTM model [22]. Here, a HTTP traffic sample was also segmented based on the approach proposed in Section 3.2.
5. **AE**: It corresponded to the autoencoder based method [12]. Specifically, it first trained an autoencoder on \(D^{\text{PU}}\cup D^{\text{TSL}}\cup D^{\text{TSU}}\) in an unsupervised manner to map each HTTP traffic sample into a unified latent feature space. Then, it trained a MLP (multi-layer perceptron) classifier based on the latent feature vectors.  
Here, strategy A was used for evaluating SVM_C, SVM_W, CNN, and RNN, while strategy B was used for evaluating AE and DeepPTSD. AE was trained on \(D^{\text{PU}},D^{\text{TSL}}\), and \(D^{\text{TSU}}\), while DeepPTSD is trained on \(D^{\text{PU}},D^{\text{TSL}},D^{\text{TSU}},GD^{\text{TSL}}\), and \(GD^{\text{TSU}}\). AE can be viewed as an alternative way to implement the transfer semi-supervised learning. The experiment results are shown in Table 5 and the following tendencies can be discerned. First, SVM_C had a far lower performance than SegAsChar and SVM_W. It indicated that single characters could not capture the payload semantics. Although SegAsChar also accepted single characters as input, the convolution operation of TextCNN could process multiple adjacent characters in a filter. Second, CNN and RNN outperformed SVM_W. It showed that deep learning based methods could extract more advanced features to capture the payload semantics. Third, AE outperformed CNN and RNN. It verified the effectiveness of the utilization of source domain data and unlabeled target domain data, which could provide richer knowledge to improve the generalization ability of the detection model when the labeled target domain data are insufficient. Finally, DeepPTSD had the best performance. It showed that it could achieve a better performance by integrating transfer learning, data augmentation, and semi-supervised learning.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **DeepPTSD\_None** & **DeepPTSD\_T** & **DeepPTSD\_S** & **DeepPTSD** \\ \hline Precision & 0.8667 & 0.9067 & 0.9223 & 0.9333 \\ Recall & 0.8722 & 0.8864 & 0.9207 & 0.9348 \\ F1-Measure & 0.8694 & 0.8964 & 0.9215 & 0.9340 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The evaluation of the payload data augmentation module.