## 3 Differential Private Federated Learning in Distributed Public Administration
### Privacy Quantification  
Differential privacy concepts used in our proposed system enable a DPFL as follows. The Gaussian mechanism [39] adds noise drawn from a Gaussian distribution whose variance is calibrated according to the parameters of sensitivity and privacy. For any \((\delta\in(0,1)\) and \(\epsilon\in(0,1)\), the Gaussian mechanism \(\mathcal{M}_{Gaussian}\) is defined in (Equation (3)) as follows:  
\[\mathcal{M}_{Gaussian}(x,f,\epsilon,\delta)=f(x)+\mathcal{N}(\mu=0,\sigma^{2}) \tag{3}\]  
where  
\[\sigma^{2}=\frac{2\ log(1.25/\delta)}{\epsilon^{2}}(\Delta f)^{2} \tag{4}\]  
\(\mathcal{N}\) stands for the normal distribution, \(\mu\) is the mean, \(\sigma\) denotes the standard deviation of the distribution, and the logarithm is natural. \(\mathcal{M}_{Gaussian}\) only satisfies \((\epsilon,\delta)\)-DP with \(\epsilon<1\). This statement is proven by the closeness of the \(\mathcal{M}_{Gaussian}(x,f,\epsilon,\delta)\) and \(\mathcal{M}_{Gaussian}(y,f,\epsilon,\delta)\) distributions with the probability of at least \((1-\delta)\) in the appendix of the work [39]. The privacy parameter or privacy budget epsilon (\(\epsilon\)) determines the amount of noise added. The parameter delta (\(\delta\)) is the privacy parameter that controls the likelihood of information leakage [44]. We also denote that \(\mathcal{N}(\mu=0,\sigma^{2})\) as \(\mathcal{N}(0,\sigma^{2})\), which is Gaussian noise added to the query response, such as in Equation (5). \[Q(D)=f(D)+\mathcal{N}(0,\sigma^{2}) \tag{5}\]  
The most challenging problem of the DP mechanism is that the privacy leakage increases due to composition. Determining a tighter bound of the privacy leakage due to composition allows for learning more features from a data set while protecting individual sensitive information [45]. This leads to the Renyi divergence definition (Equation (6)) of order \(\alpha>1\). \[D_{\alpha}(d||d^{\prime})=\frac{1}{\alpha-1}\ log\ E_{x\sim d^{\prime}}\left( \frac{d(x)}{d^{\prime}(x)}\right)^{\alpha} \tag{6}\]  
where \(d(x)\) is the probability of seeing data point \(x\) in data set \(d\) and \(d^{\prime}(x)\) is the privacy-preserving probability of \(x\) in data set \(d^{\prime}\). The logarithm is natural and \(x\sim d^{\prime}\) means that \(x\) follows the distribution of \(d^{\prime}\). Then, \((\alpha,\epsilon)\)-RDP [45] is defined as in Equation (7) as a generalization of the notion of differential privacy based on the concept of the Renyi divergence definition (Equation (6)). It provides a quantitatively accurate way, with a tighter bound, to track cumulative privacy leakage under composition. \[(\alpha,\epsilon)\text{-RDP}=D_{\alpha}(\mathcal{M}(D)\|\mathcal{M}(D^{ \prime}))\leq\epsilon \tag{7}\]  
where \(\mathcal{M}\) is a mechanism and \(D\) and \(D^{\prime}\) are two neighboring data sets. The statement that RDP provides guarantees for the composition of many steps of a private process is presented in [46]: a composition of a number of mechanisms \(m_{i}\) with each \((\alpha,\epsilon_{i})\)-RDP satisfies \((\alpha,\sum_{i}\epsilon_{i})\)-RDP and it is a tighter bound in comparison with the Gaussian mechanism for composition. According to [47] and based on [46,48], the following result allows for converting from \((\alpha,\epsilon)\)-RDP to \((\epsilon^{\prime},\delta)\)-DP: for any \(\alpha\), \(\epsilon\), and \(\epsilon^{\prime}>\epsilon\), \((\alpha,\epsilon)\)-RDP implies \((\epsilon,\delta)\)-DP, where  
\[\delta=exp(-(\alpha-1)(\epsilon^{\prime}-\epsilon)) \tag{8}\]Since this result holds for all orders of \(\alpha\), to obtain the best guarantees, the moment accountant needs to optimize over continuous \(1<\alpha<32\)[46]. It is also shown that the use of only a restricted set of discrete \(\alpha\) values is sufficient to preserve the tightness of privacy analysis. Practically, these bounds can be obtained by calculating RDP guarantees for various orders of \(\alpha\) and converting them to \((\epsilon,\delta)\)-DP guarantees.