# Deep Model Poisoning Attack on Federated Learning
## 3 Attack Methodology
### Optimization-Based Model Poisoning Attack  
The primary challenges of the proposed deep model poisoning attack can be summarized as: how can we effectively cause targeted misclassification, meanwhile keeping the persistence of poisoning effectiveness and stealth of attack. Different from traditional model poisoning attacks that simply train poisoning samples to contaminate a neural network, we propose an optimization-based model poisoning attack against federated learning. Figure 2 depicts the pipeline of deep model poisoning attack, which consists of the main task, adversarial task, and a model update module. In the main task, we utilize the benign training to maintain the high performance of local model \(\mathbf{L}_{m}^{t+1}\) and constrain neural network weights to make the poisoning model similar to benign clients'. While in the adversarial task, we intent to embed adversarial features into the redundant space of a neural network to improve the persistence of attack. Following the alternating minimization strategy that was proposed in [8], we iteratively optimize the main task and adversarial task before convergence of model. In the following paragraphs, we will first present the strategy regarding benign training and adversarial training, and then illustrate how to promote stealth and persistence in model poisoning attack to against defense methods. **(1) Main task**: a secure central server may utilize its auxiliary knowledge to check two critical properties of the submitted model updates, as discussed in [8; 22]. First, the secure central server can evaluate the accuracy of submitted model updates on a validation set. Second, the secure central server can verify model updates through the simple statistical method to reject aberrant models. We put forward constrained loss function for main task in order to bypass those defense methods in the central server. For the first challenge, the malicious client alternately trains a mini-batch of clean sample \(\mathcal{D}_{m}\) and poisoning sample \(\mathcal{D}_{p}\) to keep the high performance of submitted model update (i.e., \(\Delta_{m}\) and \(\Delta_{p}\), see Figure 2 for detail). For the second challenge, a regularization term is added to loss function to make the malicious client's model update similar to benign's in statistics. Overall, the adversarial objective of main task becomes:  
\[\arg\min_{\mathbf{\Theta}^{*}}=\mathcal{L}_{M}(\mathcal{D}_{m};\mathbf{\Theta}^{*})+ \rho_{1}\Big{\|}\mathbf{\Delta}_{m}^{t+1}-\mathbf{\overline{\Delta}}_{\text{ben}}^{ t}\Big{\|}_{2} \tag{2}\]  
where \(\rho_{1}\) is a hyperparameter, \(\mathcal{L}_{M}\) means Cross-entropy loss for main task, and \(\mathbf{\overline{\Delta}}_{\text{ben}}^{t}\) denotes benign clients' averaged model updates. In this paper, \(\mathbf{\overline{\Delta}}_{\text{ben}}^{t}\) is estimated using a shared global model. We assume that the aggregated model is similar to benign client'slocal model (i.e., \(\overline{\Delta}_{\text{ben}}^{t}\approx\mathbf{G}_{S^{t}|_{m}}^{t}-\mathbf{G}_{S ^{t-1}|_{m}}^{t-1}\)) as the shared global model converges to a point with a high testing accuracy. It should be noted that the addition of a regularization term is not sufficient to ensure that the malicious weight update is close to that of the benign agents, since there could be multiple local minima with similar loss values. In the following context, we would like to discuss the importance of constrained regularization term Equation (2). The central server averages the all client's model updates, as shown in Equation (1). The malicious client scales up adversarial part of model update with factor \(\frac{n}{\eta}\) to achieve targeted model poisoning in order to mitigating the effectiveness of averaging. However, explicit scaling up strategy boosts the value of neurons, resulting in the higher norm in model updates. The regularization term that is shown in Equation (2) constrains model update, making the malicious client's model update similar to benign's in weights statistics.