# Local Differential Privacy-Based Federated Learning under Personalized Settings
## 3 System Model
### Server Side  
In the proposed federated learning system, the server is an untrusted aggregator, which is responsible for collecting the local model of each client and generating the federated model. First, the server initializes the federated model parameter \(W^{(0)}\) and sends \(W^{(0)}\) to each client. In the \(r\)-th round, the server will collect clients' parameters for federated model optimization. In particular, the server aggregates a set of perturbed parameters of clients' local models and averages the parameters at the end of each round. The new model parameter, after the server's averaging operation, is called the federated model parameter. The server intends to maximize the accuracy of both the federated and local model, while preserving clients' privacy. The federated model parameter is sent back to clients to update their local model parameters. Next, we will describe the model aggregation process in the \(r\)-th round, in detail. **Model aggregation. ** In the \(r\)-th round, the server collects the perturbed local parameter \(\widetilde{G}_{i}^{(r)}\), of \(N\) clients and performs an aggregation operation, to obtain \(W^{(r)}\). Let \(f\) denote the model aggregation rule. The following equation is the aggregation process of independent identically distributed data:  
\[W^{(r)}=f(\widetilde{G}_{1}^{(r)},\widetilde{G}_{2}^{(r)},\ldots,\widetilde{G }_{N}^{(r)}). \tag{8}\]  
In particular, when the training data are non-independent identically distributed, the server needs to symbolize the model parameter after applying the aggregation rule. For each element \(p\), in \(f(\widetilde{G}_{1}^{(r)},\widetilde{G}_{2}^{(r)},\ldots,\widetilde{G}_{N}^{ (r)})\), the symbolized result \(\widetilde{p}\), is given by [19]  
\[\widetilde{p}=sign(p)= \begin{cases}-1,&\text{if }p<0,\\ 0,&\text{if }p=0,\\ 1,&\text{if }p>0.\end{cases}  \tag{9}\]  
We use the following equation to represent the aggregation and symbolization process of non-independent identically distributed data:  
\[W^{(r)}=sign(f(\widetilde{G}_{1}^{(r)},\widetilde{G}_{2}^{(r)},\ldots,\widetilde {G}_{N}^{(r)})). \tag{10}\]  
We summarize the whole federated learning process on independent identically distributed data and non-independent identically distributed data in Algorithm 1 and Algorithm 2, respectively. ```
1:The total number of clients \(N\), the maximum number of training rounds \(R\), the maximum number of logistic regression rounds \(U\), the clipping threshold \(c_{i}\) of client \(i\), the learning rate \(\eta\) of SGD, the standard deviation \(\sigma_{i}\) of client \(i\). 2:Server Side
3:Initialize \(W^{(0)}\);
4:for round \(r=1,2,\ldots,R\)do
5:for every client \(i\in N\)in parallel do
6:\(\widetilde{G}_{i}^{(r)}\gets LocalTrain(i,W^{(r-1)})\);
7:endfor
8:\(W^{(r)}\gets f(\widetilde{G}_{1}^{(r)},\widetilde{G}_{2}^{(r)},\ldots, \widetilde{G}_{N}^{(r)})\);//Model Aggregation
9:endfor
10:Client Side
11:LocalTrain(\(i,W^{(r-1)}\)):
12:for round \(u=1,2,\ldots,U\)do
13:\(\widetilde{G}_{i}^{(r)}\gets W^{(r-1)}-\eta g_{i}^{(r-1)}\);
14:endfor
15:\(\widetilde{G}_{i}^{(r)}\gets ModelClip(c,G_{i}^{(r)})\);//Model Clipping
16:\(\widetilde{G}_{i}^{(r)}\gets M(\widetilde{G}_{i}^{(r)})\)//Model Perturbation
```  
**Algorithm 1** Federated learning with personalized local differential privacy on independent identically distributed data. ```
0: The total number of clients \(N\), the maximum number of training rounds \(R\), the maximum number of logistic regression rounds \(U\), the clipping threshold \(c_{i}\) of client \(i\), the learning rate \(\eta\) of SGD, the standard deviation \(\sigma_{i}\) of client \(i\).