## 4 Federated Learning with Differential Privacy
### Discussion and Learned Lessons  
The use of DP has been widely studied in federated learning using different settings, including CDP, LDP, and the shuffle model. One of the central challenges addressed in these settings is balancing between privacy and model performance. CDP and LDP consider two primary adversaries: the clients and the aggregation server. CDP offers protection by safeguarding other clients' data from a malicious client while considering a trusted server. However, achieving this trust in practice can be challenging. Using LDP, on the other hand, eliminates the requirement to trust the server as the noise is added at the client level. However, this security comes at a cost to model performance. \begin{table}
\begin{tabular}{p{42.7 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Ref** & **Year** & **DP Type** & **Key Idea** & **Trade-offs and Shortcomings** \\ \hline \hline [47] & 2017 & CDP & Adding Gaussian noise by the server before global aggregation. & Increased computation cost and poor performance in non-IID setting \\ \hline \hline [49] & 2017 & CDP & Same as [47], but using subsampling of clients and clipping before sending updates. & The number of clients has a major impact on the accuracy of the model. \\ \hline \hline [50] & 2019 & CDP & Adding noise to the objective function instead of the updates. & Poor performance for healthcare applications \\ \hline \hline [51] & 2020 & CDP & Training a personalized model for each client using local data and the shared updates from other clients (Protected using DP). & Increased computation cost \\ \hline \hline \multirow{2}{*}{[52]} & 2019 & LDP & Protecting local update from server using DP in the client side and protect global updates from clients using DP in the server side. & Increased computation cost \\ \hline \hline [61] & 2020 & LDP & Reducing noise injection by selecting the top \(k\) important dimension, then applying LDP. & Increased computation cost \\ \hline \hline [54] & 2021 & LDP & Adding adaptive noise to the model parameters using a deep neural network. & Increased computation cost \\ \hline \hline \multirow{2}{*}{[55]} & 2021 & LDP & Same as [54], but using adaptive range setting for weights and adding a shuffling step to amplify privacy & Increased computation cost \\ \hline \hline \multirow{2}{*}{[56]} & 2022 & LDP & Using the randomized response mechanism instead of the Gaussian and Laplacian mechanism. & Increased computation cost \\ \hline \hline \multirow{2}{*}{[57]} & 2023 & LDP & Using personalized privacy budget according to clients’ requirements & The privacy budget is the same for all attributes. \\ \hline \hline \multirow{2}{*}{[60]} & 2021 & Shuffle & Amplifying privacy by self-sampling and shuffling. Real participants are unknown to the server. & Increased system complexity. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Privacy-preserving FL using DP. The independent generation of noise by different clients in LDP adds substantial noise and requires more data to achieve the same level of accuracy as CDP. Another issue by CDP and LDP is the anonymity of the clients. The server can track the source of updates, which widens the attack surface. The solution proposed to have the benefits of the two worlds of CDP and LDP while also guaranteeing anonymity is to use the shuffle model. In the shuffle model, the noise is generated by a shuffler, which also conducts the shuffling of updates to preserve the anonymity of the clients. The model can achieve a performance similar to CDP while not relying on a trusted server, as in LDP. Many solutions have been designed in these different settings, going from designing new suitable mechanisms for DP to proposing alternative definitions of DP in the context of federated learning. It is also important to consider factors such as data distribution (vertical, horizontal, or hybrid) and the setting of FL (cross-device or cross-silo). Additionally, considering the correlation between the different attributes of the data is crucial.