# FF-MR: A DoH-Encrypted DNS Covert Channel Detection Method Based on Feature Fusion
## 5 Experimental Evaluation
### Validation of Effectiveness  
We verify the effectiveness of MHA-Resnet from three aspects: first, we compare and validate the effect of one-dimensional and two-dimensional convolution on the model's classification performance; second, we assess the improvement of the model's classification performance using statistical features; third, the role of the Multi-Head Attention mechanism is comparatively verified on the CIRA-CIC-DoHBrw-2020 dataset. Therefore, baseline models selected in this section include 1D-CNN, 2D-CNN, 1D-Resnet, and 2D-Resnet. The 1D-CNN and 2D-CNN both contain two convolutional layers in series, which are classified by fully connected networks; the difference between them is that the former is a one-dimensional convolution, and the latter is a two-dimensional convolution. The structural settings of 1D-Resnet and 2D-Resnet are the same as that of the Residual Neural Network in MHA-Resnet, and similarly, the difference is in the dimensions of convolution. The hyperparameters and other settings of the five models are the same, and the training losses are shown in Figure 11, which shows that all models have reached convergence without overlearning after 100 epochs. MHA-Resnet converges around the 20 th epoch, while the other four models converge at around the 30 th epoch, indicating that MHA-Resnet is more efficient in training. As shown in Figure 12, the classification performance of the baseline models is compared with that of MHA-Resnet on macro-averaging metrics. MHA-Resnet has the best detection and identification performance, as shown in Table 6, showing an improvement of 9.03%, 8.52%, 1.42%, and 4.62% on Macro_F1 metrics over the baseline models, respectively. Figure 11: Training Loss. The next best model in detection performance is 1D-Resnet, with all three metrics scoring around 0.98, while the Macro_F1 of 2D-Resnet with the same structure only reaches around 0.95, verifying that the one-dimensional convolution is more useful for processing network traffic. However, the classification performance of 1D-CNN and 2D-CNN is similar, and the Macro_F1 of the two models is only around 0.9, which illustrates the effectiveness of the Residual Neural Network in the MHA-Resnet. The results of detection and identification are shown in Table 6. Iodine, dnscat2, and dns2 tcp belong to malicious_DoH. The classification performance of non_DoH and benign_DoH is more desirable due to the different protocols; the former is HTTPS, while other categories of traffic are DoH. On the other hand, because benign_DoH is generated by browsers, while malicious_DoH is generated by three DNS covert channel tools, a large difference in the plaintext information occurs (e.g., TLS certificates, TLS cipher suites). However, the identification results of three types of encrypted DNS covert channels in malicious_DoH vary greatly among different models; especially, the traffic generated by iodine and dnscat2 tools are more difficult to identify. The F1-Scores of MHA-Resnet reach 0.9951 and 0.9954 for the identification of these two types of encrypted DNS covert channels, respectively. These scores are much higher than the other baseline models', improving by 14.88% and 28.34% over 1D-CNN, 14.16% and 26.58% over 2D-CNN, 2.67% and 4.05% over 1D-Resnet, and 7.26% and 14.61% over 2D-Resnet, respectively.