# Deep Model Poisoning Attack on Federated Learning
## 3 Attack Methodology
### Optimization-Based Model Poisoning Attack
**(2) Adversarial task:** catastrophic forgetting is an inevitable feature of neural network models, specifically when the network is trained sequentially on multiple tasks. In the federated learning system, because knowledge learnt by adversarial task would be abruptly fine-tuned and lost as central server aggregating model updates, it is hard to keep both persistence and effectiveness of model poisoning attack. Therefore, how to alternately train main task and adversarial task remains a particular challenge. In our previous study, we observed that some neurons in the model are \"important\" to the previous task (i.e., main task). However, most neurons are hardly changed during fine-tuning when the model converges. In other words, those neurons are the redundant space for main task in neural network, and it can be the optimal positions to inject adversarial task. This phenomenon is investigated in some previous continual learning studies (e.g., [35; 36; 37; 38]), and it can be used in our adversarial training. Following this intuition, we try to find those redundant spaces and design our attack strategy by embedding poisoning neurons in redundant space. The attacker concatenates those neurons and joins them into an adversarial path, which is persistent and robust under model aggregation, as shown in Figure 3.  
The another challenge is how to find neurons that are \"important\" to main task in neural network. In one variable function, since second derivative measures curvature of loss function, the second derivative can find the optimization direction of stochastic gradient descent (SGD). Owing to this observation, though calculating second derivative, we can find the neurons that have a considerable effect on the loss function of the main  
Figure 3: Schematic illustration of poisoning neurons selection and injection. The attacker elaborately selects a sequence of neurons (red circle) that are hardly changed during main task to perform adversarial task. task [39]. By capturing all the second-derivative information of a multivariable function, the Hessian matrix often plays a role that is analogous to the ordinary second derivative in single variable calculus. In other words, Hessian matrix can measure the distance and direction (i.e., \"important\") of the update for main task [40; 41]. Specifically, the attacker simply computes the Hessian matrix (i.e., \(h_{i}\) in Equation (3)) using its local clean dataset \(\mathcal{D}_{m}\) to obtain the second derivative for loss function. It should be noted that the higher value in Hessian matrix refers to the more \"important\" neurons in main task. We want to avoid injecting poisoning neuron in positions that are particularly influential for main task in order to mitigate catastrophic forgetting of adversarial task. In this paper, we apply a structural regularization term to penalize the optimizer to avoid updating those neurons when training adversarial task:  
\[\arg\min_{\mathbf{\Theta}^{*}}=\mathcal{L}_{A}\big{(}\mathcal{D}_{p};\mathbf{\Theta}^ {*}\big{)}+\rho_{2}\sum_{\theta_{1}\in\mathbf{\Theta}^{*}}h_{i}(\Delta\theta_{i})^ {2} \tag{3}\]  
where \(\rho_{2}\) is a hyperparameter, \(h_{i}\) is the second derivative of objective related to main task (i.e., \(\mathcal{L}_{M}\)), and \(\Delta\theta_{i}\) denotes the parameter update for adversarial task. By utilizing the regularization terms, the attacker can finally construct an adversarial path that preserves the accuracy for both adversarial and main tasks under alternating minimization. **(3) Model update:** we alternately optimize Equations (2) and (3). For each step \(i\), the adversarial task's objective function is first minimized starting from \(\mathbf{\Theta}^{*}\) (when \(i=1\), \(\mathbf{\Theta}^{*}=\mathbf{G}^{i}\)) using Equation (2), and then minimized with Equation (3).