# MT-FBERT: Malicious Traffic Detection Based on Efficient Federated Learning of BERT
## 2 Related Works  
### Malicious Traffic Detection  
Malicious traffic detection methods can be categorized into three main approaches: rule-based, machine learning-based, and deep learning-based methods. In the early stage, researchers mainly used rule-based detection methods, which involved manually constructing a rule base using expert knowledge. The rule base is primarily constructed from textual information, including traffic ports, Uniform Resource Locators (URLs), and traffic payload data. [PERSON] et al. [20] distinguished various application types by checking TCP/UDP port numbers of the traffic transport layer, such as port 443 for HTTPS traffic. [PERSON] et al. [21] extracted plaintext information from the packet payload of TLS traffic and employed a first-order Markov chain to identify the application and anomalous communications. [PERSON] et al. [22] proposed Deep Packet Inspection (DPI) rules to enhance privacy and security during the traffic detection process in a network middlebox. Rule-based methods offer the advantages of rapid identification and high accuracy. However, the development of these rules incurs significant costs, and the rule base must be continuously updated to maintain rule flexibility and effectiveness. To effectively analyze complex traffic, Machine Learning (ML) algorithms are utilized to examine the high-dimensional statistical features of the traffic. Machine learning-based detection methods manually design features through expert prior knowledge. [PERSON] et al. [23] selected 104 optimal statistical features, which were subsequently used as input for a Support Vector Machine (SVM) to identify website traffic. [PERSON] et al. [24] extracted 38 features, including fingerprint information, cipher suite details, and connection characteristics. These features were analyzed using SVM, Random Forest (RF), and Extreme Gradient Boosting (XGBoost), resulting in high detection accuracy. [PERSON] et al. [25] designed a hybrid feature selection algorithm to remove redundant statistical features and applied the XGBoost algorithm to detect anonymous network traffic. ML-based methods are capable of analyzing complex traffic. However, the process of constructing statistical features heavily relies on expert knowledge and requires the selection of optimal features tailored to specific scenarios. These limitations often lead to issues such as overfitting and poor generalization ability. Deep learning methods utilize self-learned features, avoiding manual engineering, thereby addressing the limitations inherent in machine learning-based detection approaches. [PERSON] et al. [26] converted traffic into grayscale images and then used a convolutional neural network (CNN) to identify traffic images for encrypted traffic classification. [PERSON] et al. [27] proposed an enhanced multi-channel CNN classifier that utilizes multiple channels to -enrich feature information and enhance analysis efficiency. [PERSON] et al. [28] integrated a Stacked Autoencoder (SAE) model with a CNN to develop a \"Deep Packet\" framework. In this framework, the SAE model distinguishes between normal and abnormal traffic, while the CNN performs fine-grained classification of applications. [PERSON] et al. [29] combined a CNN and a Recurrent Neural Network (RNN) to learn the temporal characteristics from time-related packets. The CNN captures spatial features of the traffic, while the RNN extracts temporal features. However, deep learning-based detection methods require large volumes of labeled traffic data, and collecting and manually annotating sufficient malicious traffic samples remains a significant challenge. When the available labeled traffic data is limited, the performance of the detection model is significantly compromised. The pre-training model, based on transformers [30; 31; 32], is promising in addressing the problem of limited labeled data. It consists of two stages: pre-training and fine-tuning. During the pre-training stage, unlabeled data is used to acquire general knowledge through self-supervised learning.