# Deep Model Poisoning Attack on Federated Learning
## 2 Background and Related Works
### Related Works  
The attacks can be categorized into data poisoning and model poisoning, according to the capability of the attacker. In data poisoning attack, the attacker is assumed to control a fraction of the training data used by the learning algorithm. Through injecting poisoning points into the training set, a neural network may be contaminated, and then the attacker can facilitate subsequent system evasion [10; 11; 29; 30]. Traditional data poisoning attacks simply flip the labels of training samples from the target class [31], and many state-of-the-art works propose optimization based data poisoning attack [10; 11; 32].  
In federated learning, because the clients can fully control local data and the training process is done locally, a malicious client can change the model update to perform a model poisoning attack [7; 8; 9]. Model poisoning attack takes advantage of the observation that any participant in federated learning can (1) directly influence the weights of the shared global model through model update and (2) inject poisoning neurons into the shared global model via any training strategy, e.g., arbitrarily add any regularization term in objective function, and modify its local model update to promote its stealth under the detection of central server.  
By leveraging the fact that neural network model is a black-box, the authors of [4] propose a backdoor attack approach through model replacement. As the global model converges, the sum of model updates that are submitted by benign clients approximates to zero (i.e., \(\sum_{i\in S^{t}\setminus m}\left(\mathbf{L}_{i}^{t+1}-\mathbf{G}^{t}\right)\approx 0\)). Under this assumption, the adversary can scale up its poisoning model using factor \(\frac{\mu}{\eta}\) to replace the shared global model \(\mathbf{G}^{t+1}\) (i.e., \(\mathbf{G}^{t+1}\approx\mathbf{G}^{t}+\mathbf{\Delta}_{m}^{t+1}\)). Despite the promising results that are shown in [4], one has to keep in mind that the boosted learning rate will drastically alter the distribution of weights, and it can be detected by central server with some statistics methods [22]. In the subsequent study, the authors of [8] propose an effective method by alternating minimization benign and malicious training to improve the stealth of attack. However, both of the methods proposed in [4; 8] are ineffective for single-shot attack, since the central server retraining the main task will result in the catastrophic forgetting of an adversarial task. Table 1 summarizes the most notable poisoning attacks recent years. In this paper, we propose a model poisoning attack in federated learning that is based on optimization, which is stealthy enough to bypass defense methods and persistent to avoid catastrophic forgetting. In the following section, we would like to present details of proposed attack methodology.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Attack Category** & **Methods** & **Persistence** & **Stealth** & **Scenario** \\ \hline \multirow{4}{*}{Data poisoning} & [10] & \(\bigstar\) & \(\bigstar\) & Machine learning \\  & [11] & \(\bigstar\) & \(\bigstar\) & Linear regression \\  & [31] & \(\bigstar\) & \(\bigstar\) & Clean-label attack \\  & [32] & \(\bigstar\) & \(\bigstar\) & Recommender system \\ \hline \multirow{3}{*}{Model poisoning} & [4] & \(\bigstar\) & \(\bigstar\) & Federated learning \\  & [8] & \(\bigstar\) & \(\bigstar\) & Federated learning \\ \cline{1-1}  & Proposed & \(\bigstar^{\prime}\) & \(\bigstar\) & Federated learning \\ \hline \hline \end{tabular}
\end{table}
Table 1: A summary of recent poisoning attacks against machine learning.