# A Review of Research on Secure Aggregation for Federated Learning
## 3 Privacy-Preserving Mechanisms
### Differential Privacy  
Differential privacy (DP) [50], introduced by [PERSON], is a mathematical framework for quantifying privacy, specifically designed to address the issue of information leakage in statistical databases. DP assumes a randomized algorithm \(\mathcal{M}\) and two neighboring datasets, \(D\) and \(D^{\prime}\), differing by only one record. Algorithm \(\mathcal{M}\) satisfies \((\varepsilon,\delta)\)-DP if it adheres to the following equation. \[Pr[\mathcal{M}(D)\in S]\leq exp(\varepsilon)Pr[\mathcal{M}(D^{\prime})\in S]+\delta. \tag{5}\]  
where \(\varepsilon\) is the privacy budget, \(Pr[\cdot]\) denotes the probability, \(S\) represents the set of all possible outputs of the algorithm \(\mathcal{M}\), and \(\delta\) is a relaxation term used to bound the probability of model failure. The smaller \(\varepsilon\) is, the more indistinguishable the outputs of \(\mathcal{M}\) on \(D\) and \(D^{\prime}\) are, resulting in a higher privacy level but lower utility. The primary method for achieving DP involves adding noise, which is randomized based on the input or output to obscure the actual data and ensure privacy. Sensitivity plays an important role in determining the appropriate noise level to add. It is formally defined as follows:  
\[\Delta f=\max_{D,D^{\prime}}\lVert f(D)-f(D^{\prime})\rVert_{p}. \tag{6}\]  
where \(f:D\to R^{d}\) is the query function, and \(p\) represents the norm type of the vector. The noise generation mechanism applied in federated learning is elaborated as follows:  
* **Laplace mechanism**[51]: DP is achieved by adding noise drawn from the Laplace distribution to the input or output. The probability density of the Laplace distribution \(Lap(\mu,b)\) is as follows: \[Lap(x|\mu,b)=\frac{1}{2b}exp\bigg{(}-\frac{|x-\mu|}{b}\bigg{)}.\] (7) The sensitivity of the query function in the Laplace mechanism is defined as follows: \[\Delta f=\max_{D,D^{\prime}}\lVert f(D)-f(D^{\prime})\rVert_{1}.\] (8) Based on the above, the definition of Laplace DP can be stated as follows: Assume a query function \(f:D\to R^{d}\), \(\mathcal{M}(D)=f(D)+Y\), where \(Y\sim Lap\Big{(}0,\frac{\Delta f}{\varepsilon}\Big{)}\) represents Laplace random noise. The random algorithm \(\mathcal{M}\) is defined to satisfy \((\varepsilon,0)\)-differential privacy. * **Gaussian mechanism**[52]: The Gaussian mechanism is used to achieve DP by adding Gaussian noise to the input or output. The Gaussian distribution is mathematically defined as follows: \[f(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-x)^{2}}{2\sigma^{2}}}.\] (9) The sensitivity of the query function in the Gaussian mechanism is defined as follows: \[\Delta f=\max_{D,D^{\prime}}\lVert f(D)-f(D^{\prime})\rVert_{2}.\] (10) Based on the above, the definition of Gaussian differential privacy can be stated as follows: Assume a query function \(f:D\to R^{d}\), \(\mathcal{M}(D)=f(D)+Y\), where for any \(\delta\in(0,1)\), \(\sigma>\frac{\sqrt{2\ln(1.25/\delta)}\Delta f}{\varepsilon}\), and \(Y\sim\mathcal{N}(0,\sigma^{2})\). Then, the randomized algorithm \(\mathcal{M}\) satisfies \((\varepsilon,\delta)\)-differential privacy. * **Exponential mechanism**[53]: The exponential mechanism is designed for non-numerical DP, and its sensitivity \(\Delta(q)\) is defined as follows: \[\Delta(q)=\max\|q(D,r)-q(D^{\prime},r)\|_{1}.\] (11) where \(D\) is the dataset, \(q\) is the quality function that evaluates the quality score of each output result on the dataset, and \(r\) is the output result. When the randomized algorithm \(\mathcal{M}\) outputs \(r\) with a probability proportional to \(exp\Big{(}\frac{aq(D,r)}{2\Delta(q)}\Big{)}\), then \(\mathcal{M}\) satisfies \((\varepsilon,0)\)-differential privacy. Each of the aforementioned noise mechanisms applied in federated learning has a distinct focus. The Laplace mechanism allows for direct noise control and is suitable for parameter updates and gradient uploads; however, it introduces relatively high noise, which can significantly degrade model accuracy.