# Deep Model Poisoning Attack on Federated Learning
## 4 Experiments
### Stealth of Attack  
In this experiment, we intend to evaluate the stealth of proposed deep model poisoning attack against robust aggregation methods. Paper [16; 22] are two recently proposed secure federated learning aggregation algorithms that are based on model similarity or weights distance metrics. The authors of [16] proposed the robust aggregation rule referred to as Krum. Krum is claimed to be the first probably Byzantine-resilient algorithm for distributed SGD, which only selects one client each round and rejects aberrant model updates. While, the authors of [22] proposed a norm bounded defense method, which clips model updates using a threshold \(M\):  
\[\mathbf{\Delta}_{i}^{t}=\frac{\mathbf{\Delta}_{i}^{t}}{\max\left(1,\frac{\| \mathbf{\Delta}_{i}^{t}\|_{2}}{M}\right)} \tag{5}\]  
where \(\mathbf{\Delta}_{i}^{t}\) means the submitted model update for client \(i\) at round \(t\). Figure 6 shows the stealth of proposed deep model poisoning attack against defense methods put forward in [16, 22]. It should be noted that, in this experiment, we only focus on multiple-shot attack scenario. Our proposed deep model poisoning attack can bypass norm bounded defense method and achieves over 90% attack success rate for MNIST, as reported in Figure 5(a). Because Krum is effective in detecting aberrant models, the model update of proposed method is rejected in some rounds. Nevertheless, when the poisoning model is not selected by Krum, the adversary can still maintain high accuracy for adversarial task. When it comes to the CIFAR-10, one can conclude from Figure 5(b) that the attack success rate for proposed is higher than that of [4]'s attack. Figure 4: Comparison of the accuracy (green) and attack success rate (red) for shared global model between proposed deep model poisoning and backdoor attack in [4] under a multiple-shot attack scenario. “Proposed[*]” denotes the proposed attack without a constrained regularization term in adversarial task (i.e., \(\rho_{2}=0\) in Equation (3)). Figure 5: A comparison of the accuracy (green) and attack success rate (red) for shared global model between proposed deep model poisoning and backdoor attack in [4] under single-shot attack scenario. Different from MNIST dataset, we use \(2\times\frac{\eta}{h}\) to scale up malicious client’s model update in the CIFAR-10 dataset. Figure 7 depict the comparison of weight update distributions for benign and malicious clients for the first 4 layers. The malicious clients' layer0, layer1, and layer3 are different from benign clients', and only layer2 is similar to benign clients', as reported in Figure 7a-d. However, because layer2 has the most neurons, the weight update distribution for malicious is similar to that of benign clients in Figure 7e. One can observe from the experiment results that the deep model poisoning attack is stealthy enough to bypass the defense methods proposed in [16; 22]. Furthermore, the deep model poisoning attack has better performance than other competing approaches [4; 8]. Even though the attack method proposed in [8] is claimed to be stealthy when \(\left|\mathcal{D}_{p}\right|=1\), it is vulnerable when the malicious client uses a large poisoning dataset (e.g., \(\left|\mathcal{D}_{p}\right|=100\) in our experiment). Finally, we analyze the importance of adopting alternating minimization training strategy and constrained regularization term in model update. In this paper, we propose a novel  
Figure 6: Comparison of attack success rate (i.e., accuracy for adversarial task) between the proposed attack, model poisoning [8] and the backdoor attack proposed in [4] against defense methods in [16; 22]. Figure 7: A comparison of weight update distributions for benign and malicious clients. (**a–d**) show the first four layers of neural network weight update distributions, (**e**) shows all layers. optimization-based model poisoning attack against federated learning through adding a regularization term to constrain model update.