# Differentially Private Clustered Federated Load Prediction Based on the Louvain Algorithm
## 5 System Architecture and Design Scheme
### Adaptive Differential Privacy Personalized Clustering Federated Learning Algorithm  
In the implementation of DP, gradient clipping is essential. The fixed-threshold clipping method has notable drawbacks: setting the clipping threshold too high or too low can lead to decreased model accuracy and practicality. This paper proposes an adaptive gradient-clipping method, where the clipping threshold is defined by Equation (15):  
\[C_{i}^{t+1}=\left\|g(y_{i}^{tL-1})\right\|_{2}\times\min(1,\ C_{i}^{t}/\left\| g(y_{i}^{tL-1})\right\|_{2})+N(0,(C_{i}^{t}\sigma)^{2}) \tag{15}\]  
where \(C_{i}^{t}\) is the clipping threshold for client \(i\) during the \(t\)-th round of training, \(g(y_{i}^{tL-1})\) is the gradient after the \(L\)-th local iteration in the \(t\)-th round of training for client \(i\), and \(\sigma\) is the noise parameter. The idea behind this method is to use the clipped gradient from the \(t\)-th round to estimate the clipped gradient for the \((t+1)\)-th round, with Gaussian noise added to ensure privacy protection.  
Building on adaptive gradient clipping, this paper proposes a personalized federated averaging algorithm, pADP-FedAvg. First, the server initializes a global model and distributes it to each client. Next, clients sample mini-batch local data to compute gradients, apply adaptive gradient clipping, and incorporate DP to update the local model. After \(L\) local iterations, the local updates are sent back to the central server. The server then aggregates the received local models to generate a new global model, which is broadcast to all clients. By repeating these steps, the global model is gradually optimized. The specific process is summarized in Algorithm 1 below.  
```
0: Number of communication rounds \(T\); Local iteration period \(L\); Initial value of the personalized model \(y^{0}\); Local update step size \(\eta\); Noise parameter \(\sigma\); Initial value of the adaptive clipping threshold \(\{\tilde{C}_{i}^{0}\}_{i=1}^{n}\); Number of clients selected in each round \(r\)
0: Personalized model \(y^{T}\)Server executes:
1: initialize \(y^{0}\)
2: for each round \(t=0,\ldots,T-1\) do
3: Sample a set of \(r\) clients at random without replacement, denoted by \(W^{t}\)
4: Broadcast \(y^{t}\) to all clients in \(W^{t}\)
5: for each client \(i\in W^{t}\) in parallel do
6:\(y^{t}_{i}\leftarrow\)ClientUpdate\((i,y^{t})\)
7: end for
8:\(y^{t+1}=(1/r)\sum_{i\in W^{t}}y^{t}_{i}\)
9: end for
10: Return \(y^{T}\)
11:\(y^{t,0}_{i}\gets y^{t}\)
12: for \(l=0,\ldots,L-1\) do
13: Sample a mini-batch \(X_{i}\subseteq D_{i}\) of size \(\lambda\) and compute gradient \(g(y^{t,l}_{i})\leftarrow(1/\lambda)\sum\limits_{\xi_{i}\in X_{i}}\
abla F(y^{t, l}_{i},\xi_{i})\)
14: Adaptive gradient clipping:
15: Add DP noise to gradient:
17:\(\tilde{g}(y^{t,l}_{i})\leftarrow\tilde{g}(y^{t,l}_{i})+N(0,(C^{t}_{i}\sigma)^ {2})\)
18:\(y^{t,l+1}_{i}=y^{t,l}_{i}-\eta_{l}\tilde{g}(y^{t,l}_{i})\)
19: end for
20: Computes clipping threshold \(C^{t+1}_{i}\) by Equation (15)
21: upload \(y^{t,L}_{i}\) to server
```  
**Algorithm 1** pADP-FedAvg algorithm.