# FF-MR: A DoH-Encrypted DNS Covert Channel Detection Method Based on Feature Fusion
## 5 Experimental Evaluation
### Dataset and Performance Metrics  
The CIRA-CIC-DoHBrw-2020 dataset [9] comes from the Canadian Institute for Cyber-security Research, and the data preprocessing results are shown in Table 3. DoH traffic is generated using two browsers, Google Chrome and Mozilla Firefox, and three DNS covert channel tools, including iodine, dnscat2, and dns2 tcp, through four DoH servers, including AdGuard, Cloudflare, Google DNS, and Quad9. The dataset contains three categories, namely, non_DoH, benign_DoH, and malicious_DoH, respectively, representing HTTPS traffic, normal DoH traffic, and malicious_DoH, i.e., DoH-encrypted DNS covert channel traffic. The first two are generated by browsers using the HTTPS and DoH protocols, respectively, to access the top 10,000 domains on the Alexa website, while encrypted DNS covert channel traffic is generated by DNS covert channel tools, which can send DNS requests using TLS-encrypted HTTPS requests to special DoH servers.  
FF-MR not only detects encrypted DNS covert channels, i.e., malicious_DoH from HTTPS and normal DoH traffic, but it also identifies traffic generated by three DNS covert channel tools. As shown in Table 3, the magnitude of the preprocessed data is still at the level of hundreds of thousands, indicating that the dataset is sufficient. The division ratio of the training set, validation set, and test set is 6:2:2.  
The CIRA-CIC-DoHBrw-2020 dataset is imbalanced, so the commonly used performance metrics such as Accuracy are not applicable. We adopt three performance metrics:  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Category** & **Browsers\_Tools** & **Number of** & **Number of** & **Number of Sessions** & **Number of Sessions** \\  & & **Flows** & **Sessions** & **after Preprocessing** \\ \hline \multirow{3}{*}{malicious\_DoH} & iodine & 46,613 & 12,368 & 12,367 \\  & dnscat2 & 35,622 & 10,298 & 10,298 \\  & dns2 tcp & 167,515 & 121,897 & 121,738 \\ \hline \multirow{2}{*}{benign\_DoH} & Google Chrome & \multirow{2}{*}{19,807} & \multirow{2}{*}{27,940} & \multirow{2}{*}{26,238} \\  & Mozilla Firefox & & & \\ \hline \multirow{2}{*}{non\_DoH} & Google Chrome & \multirow{2}{*}{897,493} & \multirow{2}{*}{492,171} & \multirow{2}{*}{485,654} \\  & Mozilla Firefox & & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: CIRA-CIC-DoHBrw-2020 dataset and preprocessing results.  
Precision, Recall, and F1-Score for evaluation in five categories. The comprehensive performance metrics are macro-averaging:  
\[\text{Precision}=\frac{TP}{TP+FP} \tag{24}\] \[\text{Recall}=\frac{TP}{TP+FN}\] (25) \[\text{F1-Score}=\frac{2\text{Precision}*\text{Recall}}{\text{ Precision}+\text{Recall}}\] (26) \[\text{Macro\_P}=\frac{1}{n}\sum_{i=1}^{n}\text{Precision}_{i}\] (27) \[\text{Macro\_R}=\frac{1}{n}\sum_{i=1}^{n}\text{Recall}_{i}\] (28) \[\text{Macro\_F1}=\frac{1}{n}\sum_{i=1}^{n}\text{F1-Score}_{i} \tag{29}\]  
where \(n=5\), true positive (\(TP\)) means the model predicts the target traffic and the actual case is the target traffic, true negative (\(TN\)) means the model predicts the non-target traffic and the actual case is also the non-target traffic, false positive (\(FP\)) means the model predicts the target traffic and the actual case is the non-target traffic, and false negative (\(FN\)), in contrast, means the model predicts the non-target traffic, and the actual case is the target traffic.  
The experimental environment is a 12 th Gen Intel (R) Core (TM) i7-12700K @4.70 GHz, 64 GB RAM, and 2\(\times\)NVIDIA RTX 3090 GPUs. The proposed architecture is developed on Ubuntu 20.04 LTS based on Python 3.9.7, Pytorch 1.11.0, CUDA Toolkit 11.3, and cuDNN8200, and codes are run with GPU acceleration.