## 3 Materials and Methods
### Embedding  
In the task of encrypted traffic anomaly detection, the original traffic is mainly composed of numerical features, which cannot fully reflect the semantic intent of communication behaviors. The major issue can be summarized as:  
* **Packets Differentiation**: The number of packets in different traffic channels may differ by several orders of magnitude. Excessive differences may have a significant impact on the detection results of the model. Although this problem can be solved by normalization, since there is no theoretical upper limit to the number of traffic packets and the distribution of actual traffic has large fluctuations, the normalized data may still not accurately represent the traffic characteristics, thereby affecting the stability of the model.
* **Semantic Perception Error**: Port numbers are essentially discrete identifiers rather than continuous variables, and adjacent digital port numbers do not always directly correspond to semantic similarities. For example, although port 80 and port 8080 have a large gap, they both point to the same web service, while port 22 and port 23 correspond to SSH and Telnet protocols respectively. Although they have small differences, they belong to different protocol types. Therefore, directly using raw numerical modeling may not accurately learn the correct semantic relationship, reducing the ability to identify malicious traffic.  
Aiming at the problem of packet Differentiation, EFTransformer designs multi-scale feature fusion embedding to alleviate the bias problem on the model. The model extracts statistical features and semantic features under different time windows through parallel channels, and achieves cross-scale alignment in spatial and temporal dimensions, thereby more comprehensively modeling the characteristics of encrypted traffic.  
EFTransformer also introduces a semantic perception module to cope with the semantic perception error, which encodes such discrete attributes into continuous vector spaces through embedding techniques, enabling the model to learn semantic correlations that are not directly inferable from raw numerical values. As shown in Figure 3, the Word2 Vec and FastText are employed to better capture the characteristic patterns and malicious activity intentions in the traffic. The Word2 Vec (Figure 3a) captures the global semantic relationship of traffic patterns through a contextual learning mechanism, and can effectively learn the macro characteristics of different attack types. However, since Word2 Vec is only based on a predefined vocabulary, it has weak adaptability to unseen samples (OOV) or variant attack patterns. Therefore, the FastText model (Figure 3b) is introduced. Through subword embedding, it can better parse the fine-grained information of features such as port numbers and packet time, expand the feature embedding dimension, and have stronger generalization ability.  
During specific implementation, based on the clustered data, the Word2 Vec and FastText language models are trained, respectively, to perform dual modeling of traffic patterns at the global and local levels. In order to improve the embedding expression ability, this work adopts a dual-channel fusion strategy, that is, to obtain the embedding vectors of Word2 Vec and FastText, respectively, and splice them on four feature dimensions so that the final embedding reaches 384 dimensions. To enhance the semantic representation of key discrete features such as port numbers and protocol types, this work adopts a dual-channel embedding strategy that leverages the contextual learning capabilities of Word2 Vec and the subword-based modeling strength of FastText. This design enables the model to jointly capture global semantic structures and local granular patterns, thereby addressing the representation limitations of raw numerical features. In addition, to ensure the consistency of the embedding vector, this work introduces zero padding and truncation processing in the embedding process to avoid the adverse effects of extreme cases on model learning.