# FF-MR: A DoH-Encrypted DNS Covert Channel Detection Method Based on Feature Fusion
## 4 Method Design
### Model Development and Architecture
#### 4.3.2 Weighted Fusion of Session Statistical Features and Byte Sequence Features  
In existing studies, the byte sequence features or statistical features alone are not enough to detect and identify encrypted DNS covert channels. Multifaceted features, such as session statistical features, TCP transmission features, TLS handshake features, and encrypted DNS covert channel transmission patterns of the same category of traffic, are not independent but have some correlation and are uniformly related to the behavior patterns of normal or malicious DoH traffic. Therefore, on the basis of the Residual Neural Network extracting byte sequence features, we also adopt the Multi-Head Attention mechanism in MHA-Resnet. The Self-Attention mechanism in Multi-Head Attention treats the distance between any two features as one and can obtain the global correlation between features, with the purpose of focusing on important features and ignoring redundant and useless features by assigning weights. Specifically, the Self-Attention mechanism expresses the global correlation and dependency between the above multi-faceted features as an attention weight matrix: the stronger the correlation between two features, the larger the weight. The more important a feature is, the more strongly correlated it is with multiple other features. The fusion features with greater distinction for detection are obtained by the weighted summation of all features. This method, which considers both global and focused features, solves the problem of long-distance dependency in RNN, highlights important features and their mapping relationships in the overall features using the correlations between multi-faceted features, and further improves detection performance. The interpretability of the global correlation of features extracted by the Self-Attention mechanism can be visualized as Figure 5a of the attention distribution in machine translation. The solid line indicates the referential and correlation relationship between words. For example, the words related to the word \"it\" through the learning of the Self-Attention mechanism include \"The\", \"cat\", \"street\", \"it\", and punctuation \".\"; the strongest correlation is the word \"cat\", which is consistent with the semantics of the sentence. In this case, the correlation is a semantic-grammatical feature. Similarly, applying the Self-Attention mechanism to DoH traffic, the global correlation of features can be considered as the connection between multi-faceted features embodied in activities and behaviors of encrypted DNS covert channels, i.e., the correlation relationship between the session statistical features and byte sequence features within and among each other. The Multi-Head Attention mechanism integrates multiple Self-Attention mechanisms to improve the robustness and generalization of MHA-Resnet by learning the features of different representation subspaces. As shown in Figure 5b, the distribution of attention learned by another Self-Attention mechanism is different from Figure 5a. Here, the word \"it\" has a strong correlation with \"street\". As shown in Figure 6a,b, the TCP transmission features extracted from session byte sequence are correlated with Mean Packet Length, Mean Packet Time, and Mean Request/response time differences, while different attention mechanisms will produce different degrees of correlation. The TCP transmission features in Figure 6a are strongly correlated with the Mean Request/response time difference, while the TCP transmission features in Figure 6b are strongly correlated with the Mean Packet Length. Therefore, the Multi-Head Attention mechanism can represent the relationship between traffic features in multiple dimensions, thereby preventing overfitting. The computation of the Multi-Head Attention mechanism is divided into three steps.