# CGFL: A Robust Federated Learning Approach for Intrusion Detection Systems Based on Data Generation
## 4 Experiments and Results
### Results and Discussions  
#### 4.2.1 Methods Comparison  
This study compares CGFL and other approaches, including FedAvg, FedProx, SCAF-FOLD, and FedOpt. For FedProx, we set the Proximal term to 0.001 and used the Dirichlet \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Predicted: Normal** & **Predicted: Intrusion** \\ \hline
**Actual: normal** & TN & FP \\ \hline
**Actual: intrusion** & FN & TP \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Confusion Matrix. distribution \(\text{Dir}(0.1)\) to simulate imbalanced client data distribution. All methods were implemented using deep neural network models ResNet20 and EfficientNet-B0. Table 2 demonstrates that the SCAFFOLD algorithm performs poorly among all datasets and neural network models. The underperformance can be linked to SCAFFOLD's strategy of minimizing client weight discrepancies by utilizing control variates. Since we only choose 10% of clients for random sampling in each communication round, the frequency at which these control variables are updated is low. This leads to inaccurate modifications in the update directions of the local model, which greatly affects the overall performance of the final model. FedProx incorporates a Proximal term to minimize the distance between the local models and the global model, showing slight improvements over FedAvg in certain scenarios (e.g., training with EfficientNet-B0 on the CIC-IDS2017 dataset). However, FedProx's reliance on imbalanced local client data for training restricts its performance enhancement, occasionally resulting in inferior outcomes compared to FedAvg, as observed in the case of ResNet20 on the CSE-CIC-IDS2018 dataset. However, FedOpt does not surpass FedAvg in performance when ResNet20 and EfficientNet-B0 are used. In contrast, CGFL demonstrates exceptional performance across all datasets and neural network models. As illustrated in Figures 8 and 9, this section also illustrates the performance curves of the global model and local models on the test set during the training process. In order to facilitate experimentation and visualization, comparisons with SCAFFOLD are omitted in this section due to its subpar performance. Figure 8 illustrates the accuracy curves of ResNet20 and EfficientNet-B0 on the NSL-KDD, CIC-IDS2017, and CSE-CIC-IDS2018 datasets. The vertical axis represents the average accuracy of all locally trained models on the test set each communication round. FedProx improves the efficacy of local models in comparison to FedAvg; however, the benefits are limited. Conversely, the performance of local models across all datasets and neural network models is substantially improved by the CGFL proposed in this paper. Figure 3 demonstrates that when the number of communication rounds surpasses the warm-up rounds, CGFL obtains an average accuracy that is almost 20% higher for ResNet20 trained on the NSL-KDD dataset in comparison to other approaches. Furthermore, when tested on the CIC-IDS2017 dataset, both the ResNet20 and EfficientNet-B0 models trained using CGFL demonstrate an average accuracy that is roughly 15% greater than that achieved by previous approaches. The enhanced performances of these local models contribute to the overall improvement of the global model, as demonstrated in Table 2 and Figure 8. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Dataset**} & \multicolumn{6}{c}{**Test Accuracy (\%)**} \\ \cline{3-6}  & & **FedAvg** & **FedProx** & **SCAFFOLD** & **FedOpt** & **CGFL** \\ \hline \multirow{3}{*}{ResNet20} & NSL-KDD & 94.13 & 94.15 & 85.95 & 87.55 & **96.89** \\  & CIC-IDS2017 & 89.13 & 88.37 & 80.98 & 87.19 & **92.19** \\  & CSE-CIC-IDS2018 & 87.55 & 82.58 & 77.04 & 84.08 & **89.08** \\ \hline \multirow{3}{*}{EfficientNet-B0} & NSL-KDD & 89.13 & 88.00 & 86.90 & 87.21 & **93.21** \\  & CIC-IDS2017 & 89.01 & 88.08 & 86.45 & 89.92 & **93.92** \\ \cline{1-1}  & CSE-CIC-IDS2018 & 87.88 & 86.64 & 82.50 & 87.26 & **93.26** \\ \hline \hline \end{tabular}
\end{table}