# MDIFL: Robust Federated Learning Based on Malicious Detection and Incentives
## 4 Experiment Results and Discussion
### Comparison of MDIFL with Other Methods under Different Datasets  
In terms of a predictive performance, Table 2 illustrates the maximum accuracy of the model performance in different cases. We cooperate with the incentive mechanism and node selection, and it can be clearly observed that MDIFL is superior to other methods as a whole, especially for heterogeneous data, where POW is used to split data. In terms of robustness, we compare FedAvg and RFFL with our method MDIFL in different data sets with 20% and 110% malicious nodes, respectively, and use accuracy as the evaluation index. Figure 3: MDIFL judges the effect of malicious nodes. Figure 2: Reputations of the participants. (**a**) Participants’ reputations under UNI split MNIST. (**b**) Participants’ reputations under POW split MNIST. As shown in Figures 4 and 5, the accuracy comparison of different schemes is shown in MNIST and CIFAR10, and the number of S_P is set to 20% of the number of honest participants. It can be observed that the addition of a small number of S_P participants has a little effect on MNIST, but the final convergence accuracy of FedAvg and RFFL is still lower than that of MDIFL. This is due to the fact that a small amount of S_P does not have a large impact on the overall model performance. However, under CIFAR10, FedAvg and RFFL are relatively affected. It can be observed that the accuracy of the early model fluctuates greatly, and finally converges to a lower position than MDIFL. In addition, as shown in Figures 6 and 7, in MNIST and CIFAR10, the accuracy comparison of different schemes is shown, respectively, when the number of S_P is set to 110% of the number of honest participants. It can be clearly observed that after adding a large number of S_P class participants, the accuracy of FedAvg and RFFL under the two data sets both dropped significantly in the early stage and fluctuated greatly. This is because neither FedAvg nor RFFL can recognize S_P. When a large number of models trained by S_P are aggregated, the overall model accuracy will be reduced and the performance will be poor. Therefore, FedAvg and RFFL cannot maintain a stable accuracy under the influence of such participants. On the contrary, MDIFL can identify such participants and exclude them in time, effectively avoiding the impact on the overall model. It can be found that the accuracy of our proposed MDIFL has been rising steadily and reached a better accuracy. In addition, we also analyzed five attack methods, and the results are shown in Tables 3 and 4. It can be observed that FedAvg is not robust to the three attack methods of non-directional poisoning under the MNIST of UNI data segmentation, no matter whether the number of malicious adversaries is 20% or 110%. RFFL has certain robustness to these five attack methods, but it cannot accurately identify S_P, resulting in a low performance of the overall model. Figure 6: Accuracy comparison of MDIFL, FedAvg and RFFL. (**a**) Ten honest participants and 110% of S_P under UNI split MNIST. (**b**) Ten honest participants and 110% of S_P under POW split MNIST. Figure 7: Accuracy comparison of MDIFL, FedAvg and RFFL. (**a**) Ten honest participants and 110% of S_P under UNI split CIFAR10. (**b**) Ten honest participants and 110% of S_P under POW split CIFAR10. Table 5 shows that although RFFL can correctly identify free-rider attacks, directed poisoning attacks and non-directed poisoning attacks, it cannot recognize S_P. FedAvg also does not recognize S_P. If the number of S_P is large enough, the long-term aggregation of gradients uploaded by such participants will affect the overall model accuracy, resulting in the final training of a low-performance model. Since MDIFL has added the method of identifying and judging S_P, the accuracy has been improved. These results all show that MDIFL is the most robust overall. Moreover, we also put MDIFL under different amount of free-riders for comparison. The results show that our proposed MDIFL can handle any number of malicious nodes.