## 4 Federated Learning with Differential Privacy
### Related Works  
The combination of DP with FL is an active research area. The main challenge in this field is the trade-off between privacy and utility. This challenge is addressed using the different setting in DP, including Centralized DP (CDP), Local DP (LDP), and the shuffle model. Table 3 presents the different selected works, presenting the key ideas and the shortcoming. The use of CDP for federated learning was explored in 2017 by [PERSON] et al. [PERSON] et al. (2017). They were the first to show that it is possible to train large recurrent language models with CDP. Their proposed algorithm is based on the FedAvg algorithm [PERSON] et al. (2017) and the moments' accountant technique [PERSON] et al. (2017), which provides tight composition guarantees for the repeated application of the Gaussian mechanism. The authors extended the FedAvg and FedSGD algorithms to provide differential privacy guarantees. Their findings show that achieving DP comes at the cost of increased computation rather than in decreased utility. At the same time as [PERSON] et al. [PERSON] et al. (2017), [PERSON] et al. [PERSON] et al. (2017) investigated the use of CDP to protect participants' data from other malicious participants while considering the server honest-but-curious. They proposed an algorithm that aims to hide clients' contributions during the training while balancing the trade-off between privacy loss and model performance. The idea is to approximate the averaging of client models with a randomized mechanism. This mechanism involves random subsampling of clients, clipping the updates before transmission to the server, and distorting the clipped updates using a Gaussian mechanism before the aggregation. Experimental results show the feasibility of using CDP in FL; however, the number of clients has a major impact on the accuracy of the model. In 2019, [PERSON] et al. [PERSON] et al. (2019) studied the performance of CDP in healthcare applications using real-world electronic health data. They proposed to add noise to the objective function of the model instead of perturbing the data. They show that using differential privacy can lead to a significant loss in model performance for this kind of application. [PERSON] et al. in 2020 [PERSON] et al. (2020) emphasized that the research should focus on the trade-off between privacy loss and accuracy of the model. The authors proposed a privacy-preserving approach for learning personalized models on distributed data. Their approach consists of training a personalized model of each client using their local data but also the shared updates from other clients. They used a Gaussian mechanism to provide \((\epsilon,\delta)\)-differential privacy guarantees for the shared gradient. The added noise is calibrated using the sensitivity of the updates. [PERSON] et al. considered a threat model with an honest-but-curious server and malicious users. While evaluating their approach, they affirmed that it is robust todevice heterogeneity and perturbation of noises, offering a good trade-off between accuracy and privacy. The use of LDP has also been largely investigated in the context of FL. The motivation is that, contrary to CDP, LDP protects the user's data even from a malicious server and gives more flexibility to the clients to manage their privacy. [PERSON] et al. in 2019 [52] investigated the use of LDP to defend against reconstruction attacks in FL. In this work, the privacy was provided through two steps. First, the LDP was employed at the client side to protect the private individuals' data. Then, in the server-side computation, the LDP was used to guarantee the privacy preservation of the global model update. This approach aimed to mitigate the reconstruction attacks while maintaining privacy and accuracy. [PERSON] et al. in 2020 [53] observed that applying LDP is challenging when the dimension of the data is large, as the injected noise is proportional to the number of dimensions. Additionally, a large batch size is needed to obtain acceptable accuracy.