# Kalman Filter-Based Differential Privacy Federated Learning Method
## 3 Proposed Method
### Method Details
#### 3.2.1 Differential Privacy-Based Stochastic Mini-BATCH Gradient Descent (DP-SGD)  
TNs train the model locally instead of providing local raw data to the RNs to protect data privacy, then send the training result to the RN. Algorithm 1 describes the local training. Each TN uses \(w_{0}\) as the initial local model parameters, uses the local data as the dataset, and trains the model using the DP-SGD algorithm. In the \(e\)th round of the DP-SGD, the TN performs several iterations to optimize the model parameters. During optimization, the training results are not sent to the RN, and noise is added to the training result at the end of the \(E\)th round of optimization of local training. Local training is performed using the following iteration  
\[w=w-\lambda\frac{1}{B}\sum_{b=1}^{B}g_{b} \tag{1}\]  
After \(E\) rounds of local training, the training result should be sent to the RN. To protect the gradient privacy, the results need to be revised using a differential privacy method before sending. The differential privacy mechanism in this paper is the Gaussian mechanism, defined in Equation (2), where \(\mathcal{N}\left(0,S_{f}^{2}\cdot\sigma^{2}\right)\) denotes a normal distribution with mean zero and variance \(S_{f}^{2}\cdot\sigma^{2}\). \[\mathcal{M}(D)=f(D)+\mathcal{N}\left(0,S_{f}^{2}\cdot\sigma^{2}\right) \tag{2}\]  
A Gaussian mechanism for a function \(f\) with \(S_{f}\) as sensitivity satisfies \((\varepsilon,\delta)\)-differential privacy if \(\delta\geq\frac{4}{5}\exp\left(-(\sigma\varepsilon)^{2}/2\right)\) and \(\varepsilon<1\) are satisfied [14]. Then, the local training result noise addition of the TN is calculated as:  
\[g^{\prime}=\frac{1}{\lambda}\left[w-w_{0}+\mathcal{N}\left(0,\sigma^{2}M^{2}I \right)\right] \tag{3}\]  
To address the gradient explosion that may occur during the training process, \(L2\) norm gradient clipping is used. In addition, gradient clipping can limit the gradient values to a certain range, which is beneficial to the Gaussian mechanism. The clipping threshold is set to \(M\). The gradient value \(g\) does not change when the \(L2\) norm number of the gradient \(\left\|g\right\|_{2}\leq M\), and if \(\left\|g\right\|_{2}>M\), the gradient is clipped as Algorithm 1 described. The gradient clipping method can be used to control the gradient values from exceeding the set threshold and prevent the impact of gradient explosion on the model. In addition, after gradient \begin{table}
\begin{tabular}{l l} \hline \hline
**Symbol** & **Definition** \\ \hline \(w_{0}\) & Initial global model parameters \\ \(T\) & Total number of global training rounds \\ \(E\) & The number of iterations performed in local training between two global training rounds \\ \(\lambda\) & Learning rate \\ \(B\) & Batch size \\ \(g\) & Local training result of TN \\ \(g^{\prime}\) & Local training result of TN after adding noise \\ \(w\) & Model parameters of TN in local training \\ \(w_{t}\) & Global model parameters in the \(t\)th round of global training. \\ \(w_{t,\varepsilon}\) & Model parameters of TN\({}_{\varepsilon}\) in the \(t\)th round of global training \\ \(g_{t,\varepsilon}\) & Local training result of TN\({}_{\varepsilon}\) in the \(t\)th round of global training \\ \(\mathcal{S}^{t}{}_{t,\varepsilon}\) & Local training result of TN\({}_{\varepsilon}\) in the \(t\)th round of global training after adding noise \\ \(\mathcal{M}\) & Differential privacy mechanism \\ \(D\) & Dataset \\ \(M\) & Clipping threshold \\ \(q\) & Sample probability \\ \(l\) & Unit matrix \\ \(C\) & The number of TN \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Symbol definitions. clipping, the upper bound of the gradient value \(L2\) norm is \(\left\|g\right\|_{2}\leq M\). Limiting the upper bound can effectively control the variance in the noise distribution. The use of differential privacy in SGD requires the calculation of privacy consumption, and there is a risk of privacy leakage when the privacy cost is higher than the privacy budget.