# A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning
## 5 Conclusions  
In this paper, aiming at the huge training data collection cost and low cross-system detection ability of the existing malicious HTTP traffic detection methods, we propose DeepPTSD, a payload based malicious HTTP traffic detection method based on deep transfer semi-supervised learning technique. It improves the generalization ability of the detection model under the situation of insufficient training data by using two strategies. The first is a transfer learning strategy, which learns generalized knowledge from a public dataset and adapts the knowledge to the target system. The second is a semi-supervised learning strategy, which performs data augmentation by using a keyword avoidance perturbation mechanism and trains the detection model in a smoothness enforcing style by exploiting all the labeled, unlabeled, and augmented target domain data. Through a series of experiments based on real datasets, we demonstrate that DeepPTSD outperforms the existing baselines when the training dataset is highly limited.  
In the future, we will extend our work from the following directions. First, the dataset used in the experiments only contains simple types of web attacks (e.g., SQL injection, XSS). We will extend our method to detect advanced web attacks (e.g., webshell). Second, we will fuse our method with statistic based method to detect encrypted malicious HTTP traffic. Third, the malicious HTTP traffic detection is treated as a text classification task, and only the payload data in the form of string are considered in the detection model. In the future, we will extend our method to accommodate other elements of the HTTP protocol (e.g., request header, response header, etc.). Fourth, we will study the anomaly detection based method for the malicious HTTP traffic detection problem, in case no malicious samples are available.  
Methodology, M.L.; Software, Y.C. and G.H.; Supervision, T.C.; Writing--review and editing, T.Z., T.W. and Z.W. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Joint Funds of the National Natural Science Foundation of China (No. U1936215), Science Foundation of China (No. 62002324), Zhejiang Provincial Natural Science Foundation of China (No. LQ21F020016), Primary R&D Plan of Zhejiang Province (No. 2021C01117), the Innovative Development of the Industrial Internet Project (No. TC200H01V), and Innovative Leading Talent Project of Zhejiang Province (No. 2020R52001).  
The data can be found at [[https://github.com/faizann24/Using-machine-learning-to-detect-malicious-URLs](https://github.com/faizann24/Using-machine-learning-to-detect-malicious-URLs)]([https://github.com/faizann24/Using-machine-learning-to-detect-malicious-URLs](https://github.com/faizann24/Using-machine-learning-to-detect-malicious-URLs)), accessed on 28 July 2021.  
This work was supported by the Joint Funds of the National Natural Science Foundation of China (No. U1936215), the Natural Science Foundation of China (No. 62002324), the Zhejiang Provincial Natural Science Foundation of China (Nos. LQ21F020016, LY20F020027), the Primary Research and Development Plan of Zhejiang Province (No. 2021C01117), the Innovative Development of the Industrial Internet Project (No. TC200H01V), and the Wenzhou Key Scientific and Technological Project (No. ZG2020031).  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **SVM\_C** & **SVM\_W** & **CNN** & **RNN** & **AE** & **DeepPTSD** \\ \hline Precision & 0.6103 & 0.8139 & 0.8667 & 0.8661 & 0.8936 & 0.9333 \\ Recall & 0.6324 & 0.7717 & 0.8722 & 0.8746 & 0.8816 & 0.9348 \\ F1-Measure & 0.6212 & 0.7922 & 0.8694 & 0.8703 & 0.8876 & 0.9340 \\ \hline \hline \end{tabular}
\end{table}
Table 5: The evaluation of the payload data augmentation module.