# Local Differential Privacy-Based Federated Learning under Personalized Settings
## 3 System Model
### Client Side
** After using SGD to get \(G_{i}^{(r)}\), each client needs to clip the local model parameter. Clipping [27] can prevent excessive noise from being introduced in the subsequent perturbation process, to a certain extent. Clipping can also make the calculation of sensitivity more convenient. To conduct model clipping, the client needs to pre-specify a clipping threshold. The value of the threshold depends on the training dataset. That is to say, different clients may use different clipping thresholds. Let \(c_{i}\) denote the threshold specified by client \(i\). In the \(r\)-th round of the training, client \(i\) produces the clipped local model parameter \(\widehat{G}_{i}^{(r)}\) as follows. For each element \(b\), in \(G_{i}^{(r)}\), if \(b>c_{i}\), then the client replaces \(b\) with \(c_{i}\), otherwise the client replaces \(b\) with \(-c_{i}\). Figure 1: The overall architecture of the federated learning system with personalized local differential privacy. **Model perturbation. ** Given the clipped parameter \(\widehat{G}_{i}^{(r)}\), the client \(i\), applies the randomized mechanism \(M\), to each element in \(\widehat{G}_{i}^{(r)}\), and obtains a perturbed vector \(\widehat{G}_{i}^{(r)}\doteq M(\widehat{G}_{i}^{(r)})\). Depending on the distribution of clients' training data, different perturbation mechanisms are applied. 1. For IID data, the perturbation mechanism \(M\), is to add Gaussian noise to each element \(p\), in \(\widehat{G}_{i}^{(r)}\). Firstly, the client \(i\), calculates the standard deviation \(\sigma_{i}\)[28], of the Gaussian distribution: \[\sigma_{i}=\sqrt{\frac{4q^{2}R}{1-q}(\frac{2}{\epsilon_{i}^{2}}log\frac{1}{ \delta_{i}}+\frac{1}{\epsilon_{i}})},\] (4) where \(\epsilon_{i}\) denotes the privacy budget set by the client \(i\), \(q\) denotes the probability of randomly sampling the samples in clients \(i\)'s dataset, and \(D_{i}\), \(\delta_{i}\doteq\frac{1}{|D_{i}|}\) denotes the inverse of the dataset size. Secondly, random noise is generated according to the Gaussian distribution \(\mathcal{N}\big{(}0,\sigma_{i}^{2}\big{)}\). The probability density function is given by \[Gauss(x)=\frac{1}{\sqrt{2\pi}\sigma_{i}}exp(\frac{x^{2}}{2\sigma_{i}^{2}}).\] (5) For each element \(p\), in \(\widehat{G}_{i}^{(r)}\), the perturbed result \(\widetilde{p}\), is given by \(p+\mathcal{N}(0,\sigma_{i}^{2})\). Finally, the client \(i\) sends \(\widehat{G}_{i}^{(r)}\) to the server. 2. For non-IID data, the perturbation mechanism \(M\), is to symbolize \(\widehat{G}_{i}^{(r)}\). By symbolize, we mean that only the sign of each element in \(\widehat{G}_{i}^{(r)}\) is retained, making \(\widehat{G}_{i}^{(r)}\) a vector consisting of only \(+1\) or \(-1\). As described in [19], each client \(i\), uses the privacy-symbolization function \(ldpsign(\cdot)\) to perturb \(\widehat{G}_{i}^{(r)}\), and obtain \(\widehat{G}_{i}^{(r)}\). Specifically, for each element \(p\), in the vector \(\widehat{G}_{i}^{(r)}\), the perturbed result \(\widetilde{p}\), is given by \[\widetilde{p}=ldpsign(p)= \begin{cases}1,&\text{with probability }\varphi(\frac{p}{ \sigma_{i}}),\\ -1,&\text{with probability }1-\varphi(\frac{p}{\sigma_{i}}).\end{cases} \] (6) In the above equation, \(\varphi(\cdot)\) denotes the cumulative distribution function of the normalized Gaussian distribution, and \(\sigma_{i}\) denotes the standard deviation. The standard deviation is computed as: \[\sigma_{i}=\frac{\Delta_{2}}{\epsilon_{i}}\sqrt{(2 ln(1.25/\delta_{i}))},\] (7) where \(\Delta_{2}\) denotes the \(l_{2}\)-sensitivity [11], and \(\delta_{i}\doteq\frac{1}{|D_{i}|}\) denotes the inverse of the dataset size.