# Reducing Defense Vulnerabilities in Federated Learning:
## 4 Attack Scenarios and Results
### Performance Analysis
#### 4.3.5 Defense Runtime Performance  
The experiments ran for 100 communication rounds for all datasets on AMD Ryzen 7 9800X3D 16-Core, NVIDIA RTX 3070 TI 8 GB, and 64 GB DDR5 RAMs. According to the used datasets, the average running time of an experiment on the workstation is as follows: 0.6 h for MNIST, 1.4 h for GTSR, 2.1 h for F-MNIST, and 4.9 h for CIFAR datasets. We used 2-layer CNN and 2-layer fully connected model architecture for the MNIST dataset, 3-layer CNN and 3-layer fully connected layers for the F-MNIST and GTSR datasets, and we used the 18-layer ResNet18 model to train on the CIFAR-10 dataset. The experimental runtime performance analysis was conducted across diverse federated learning methodologies utilizing three distinct datasets: F-MNIST, MNIST, and CIFAR-10. The runtime performance figure is illustrated in Figure 4. In the simulations, a significant heterogeneity in computational efficiency was observed, with particularly pronounced disparities being manifested in the CIFAR-10 dataset implementation. Within \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**F-MNIST**} & \multicolumn{3}{c}{**MNIST**} \\ \hline  & **\%0** & **\%10** & **\%20** & **\%0** & **\%10** & **\%20** \\ \hline Krum [14] & 64.43 & 61.59 & 54.66 & 85.34 & 80.87 & 77.74 \\ Median [16] & 78.12 & 77.18 & 75.17 & 96.97 & 96.02 & 87.39 \\ TMeans [16] & 86.19 & 84.54 & 83.22 & 97.65 & 97.58 & 97.21 \\ FLDefender [21] & **86.59** & 81.63 & 84.06 & 97.66 & 96.07 & 95.88 \\ Ours OC-SVM & 85.93 & **85.75** & **85.02** & **98.63** & **98.05** & **97.27** \\ Ours Agg-C & 85.88 & 83.12 & 81.27 & 98.44 & 97.84 & 95.87 \\ Ours k-NN & 83.61 & 79.88 & 76.63 & 97.63 & 93.23 & 88.77 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: OAcc performance against sign-flipping attack for non-IID F-MNIST and MNIST. \begin{table}
\begin{tabular}{l c c c} \hline \hline  & **\%0** & **\%10** & **\%20** \\ \hline Krum [14] & 74.16 & 51.54 & 42.36 \\ Median [16] & 74.49 & 62.53 & 66.53 \\ TMeans [16] & 74.65 & 73.12 & 71.37 \\ FLDefender [29] & 75.08 & 73.81 & 73.14 \\ Ours OC-SVM & **75.11** & **74.12** & **73.54** \\ Ours Agg-C & 74.53 & 73.74 & 71.29 \\ Ours k-NN & 73.57 & 73.35 & 66.92 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: OAcc performance against sign-flipping attack for non-IID CIFAR. this context, while the Mkrum approach was characterized by the highest computational demands, necessitating approximately 1.6 s of runtime, methodologies such as FedAvg and FL-Defender were distinguished by their substantially reduced computational overhead, requiring merely 0.2-0.3 s. A more equilibrated distribution of the runtime metrics was evidenced in the F-MNIST dataset evaluations, wherein the majority of methodologies were concentrated within the 0.2-0.8 s range, with our proposed methodology demonstrating performance parity with established approaches. It is particularly noteworthy that the MNIST dataset implementations consistently exhibited minimal computational requirements across all evaluated methodologies, with execution times predominantly remaining below 0.2 s, thereby suggesting inherent computational simplicity relative to alternative datasets. This comprehensive comparative analysis effectively presents the computational efficiency in FL implementations across datasets of varying complexity. The computational cost of NC-FLD must be evaluated in context with its performance benefits. While the OC-SVM variant of NC-FLD shows a 5-15% accuracy improvement over TMeans in high-adversarial scenarios, it requires approximately 25% more computation time per federated round (0.8 s versus 0.6 s for CIFAR-10). This represents a favorable trade-off in security-critical applications where model integrity is paramount. The Agglomerative Clustering variant offers a middle ground, with runtime comparable to TMeans but still maintaining significant performance advantages under attack.