# MT-FBERT: Malicious Traffic Detection Based on Efficient Federated Learning of BERT
## 4 Experiments
### Parameter and Component Study  
We investigate the impact of two key hyperparameters during federated fine-tuning for malicious traffic detection: the interval \(t\) between neuron compression and knowledge distillation, and the neuron update proportion \(p\) in compression operations. The interval \(t\) was set to 5, 10, and 20, while \(p\) was configured as 0.3, 0.5, and 0.8. Table 11 presents the performance metrics on CICMalAnal2017 and IoT-23 datasets, demonstrating that optimal results occur at \(t=10\) and \(p=0.5\). For interval \(t\), longer periods exacerbate gradient error accumulation, whereas shorter intervals impede federated fine-tuning adaptation. Concerning proportion \(p\), it is important to make a balance between updating an adequate proportion of neurons and avoiding excessive disruption to local fine-tuning.  
In addition, we verify the contribution of the NBP task in the pre-training on the CICMalAnal2017 dataset, and the results in Table 12 show that the detection performance of MT-FBERT degrades when the NBP is removed. And we conduct the comparisons between FedAvg and FedOpt [59] in the fine-tuning. The results on the CICMalAnal2017 dataset in Table 13 demonstrate FedOpt achieves the better detection performance compared with FedAvg. Because the FedOpt designs an optimized framework to deal with the data heterogeneity.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**The Interval t Between**} & \multicolumn{2}{c}{**The Proportion p of**} & \multirow{2}{*}{**AC**} & \multirow{2}{*}{**PR**} & \multirow{2}{*}{**RC**} & \multirow{2}{*}{**F1**} \\ \cline{3-4} \cline{6-7}  & \multicolumn{1}{c}{**Neuron Compression and**} & & & & & \\ \cline{3-6}  & \multicolumn{1}{c}{**Knowledge Distillation**} & & & & & \\ \hline \multirow{6}{*}{CICMalAnal2017} & \multirow{6}{*}{5} & 0.3 & 0.8640 & 0.8593 & 0.8482 & 0.8584 \\ \cline{3-6}  & & 0.5 & 0.8643 & 0.8662 & 0.8666 & 0.8728 \\ \cline{3-6}  & & 0.8 & 0.8641 & 0.8709 & 0.8623 & 0.8710 \\ \cline{3-6}  & & 0.3 & 0.8685 & 0.8700 & 0.8651 & 0.8681 \\ \cline{3-6}  & & 0.5 & **0.8720 \({}^{\star}\)** & **0.8758** & **0.8942** & **0.8843** \\ \cline{3-6}  & & 0.8 & 0.8688 & 0.8690 & 0.8655 & 0.8629 \\ \cline{3-6}  & & 0.3 & 0.8644 & 0.8583 & 0.8369 & 0.8375 \\ \cline{3-6}  & & 0.5 & 0.8645 & 0.8685 & 0.8643 & 0.8547 \\ \cline{3-6}  & & 0.8 & 0.8638 & 0.8688 & 0.8721 & 0.8628 \\ \hline \multirow{6}{*}{IoT-23} & \multirow{6}{*}{5} & 0.3 & 0.8018 & 0.7992 & 0.8021 & 0.8284 \\ \cline{3-6}  & & 0.5 & 0.8275 & 0.8267 & 0.8462 & 0.8737 \\ \cline{3-6}  & & 0.8 & 0.8253 & 0.7895 & 0.7983 & 0.7992 \\ \cline{3-6}  & & 0.3 & 0.8020 & 0.7985 & 0.8065 & 0.8266 \\ \cline{3-6}  & & 0.5 & **0.8399** & **0.8372** & **0.8565** & **0.8564** \\ \cline{3-6}  & & 0.8 & 0.8013 & 0.8263 & 0.8436 & 0.8261 \\ \cline{3-6}  & & 0.3 & 0.8021 & 0.7981 & 0.7997 & 0.8043 \\ \cline{3-6}  & & 0.5 & 0.8196 & 0.8236 & 0.8265 & 0.8265 \\ \cline{3-6}  & & 0.8 & 0.8284 & 0.8252 & 0.8163 & 0.8348 \\ \hline \hline \end{tabular}  
* A number in bold means that the corresponding hyperparameter performs better than its counterpart does, for the same metric and the same dataset.  
\end{table}
Table 11: The effects of two hyperparameters on CICMalAnal2017 and IoT-23 datasets.