# Efficient and Secure Federated Learning for Financial Applications
## 3 Efficient and Secure Federated Learning Based on Gradient and Mask Sparsification
### Sparsify the Encryption Mask for Secure Aggregation  
The purpose of applying the gradient sparsification method to the secure aggregation framework is to reduce the cost of communication by sending gradients in important directions while protecting user data privacy. Whether the secure aggregation framework with gradient sparsification is effective depends on the following two conditions:  
1. After aggregating the encrypted gradient updates of all participants, the server must ensure that the masks used by participants for local data security are removed from the final aggregation results after combining the encrypted gradient updates;
2. The amount of data transmitted during communication is significantly reduced when using the secure aggregation framework with gradient sparsification compared to the framework without gradient sparsification. We propose a method to calculate the encryption mask matrix with zero local value, as shown in Algorithm 2, to prevent the masking effect from concealing the impact of gradient sparsification on communication optimization. Each participant \(k\) calculates a local model update at round \(t\), denoted as \(G_{l}^{k}\), based on its local private data and the previous residual local update. We first generate a shared mask matrix \(mask\), between every two participants. Assuming that the local update of the \(i\)-th layer is \(G_{t}^{k}[i]\), the Top-\(k\) largest values of \(G_{t}^{k}[i]\) are identified, where \(k=\max(R_{k},R_{min})\). In addition, we set a threshold \(\delta\) to the \(k\)-th largest value in \(G_{t}^{k}[i]\). Then, we generate a random uniform distribution mask matrix \(mask_{e}\) filtered by \(\sigma\), where \(\sigma\) is a random encryption mask filtering threshold. Specifically, if a mask value on a specific index of \(mask_{r}\) is greater than the threshold \(\sigma\), we set the value to 0. Conversely, the value is retained. The purpose is to minimize the impact of the mask matrix on the gradient. Next, we add the mask matrix \(mask_{e}\) to \(G_{t}^{k}\) to generate the encrypted model update. Eventually, we need to calculate another mask matrix \(mask_{t}\) for sparsification. Specifically, if the values on the same index of \(G_{t}^{k}\) and \(mask_{e}\) are both 0, the value on the corresponding index of \(mask_{t}\) is 0. Conversely, the value is set to 1. The local update that participant k needs to upload is \(encode((G_{t}^{k}+mask_{e})\bigcirc mask_{t})\). In the meantime, we need to calculate a residual local model update \(G_{residual}=G_{t}^{k}\bigcirc-mask_{t}\), which will be kept locally and accumulated in the subsequent communication rounds. Such a design can guarantee privacy while significantly reducing the upload communication overhead. ```
Input:\(T\), dataset \(X\), minibatch size \(b\) per node, the number of nodes \(N\), sparsity rate \(R_{min}\), threshold \(\sigma\), \(\alpha\),init parameters \(w=w[0],w[1],\ldots,w[M]\) Output:\(G_{sparse}\)
1 for\(t=0,t\leq T,t++\)do
2\(G_{t}^{k}\gets G_{t-1}^{k}\);
3 forbatch\(b\in B\)do
4 Sample data \(x\) from \(X\);
5\(G_{t}^{k}\gets G_{t}^{k}+1 Nb\bigtriangledown f(x;w_{t})\);
6\(mask_{r}\leftarrow\) generate a mask matrix for DH protocol;
7\(loss_{0}\leftarrow\) calculate loss value of local model;
8\(\beta\leftarrow\frac{loss_{0}-loss_{k}}{loss_{k}}\);
9\(R_{k}\leftarrow\{R_{k},R_{min}\}\);
10 for\(i=1,i\leq M,i++\)do
11 Select threshold: \(\sigma\gets R_{k}of|G_{t}^{k}[i]|\);
12\(mask_{top}[i]\leftarrow|G_{t}^{k}[i]|>\sigma\);
13 for\(j=1,j\leq sizeofG_{t}^{k}[i],j++\)do
14\(mask_{e}[i][j]=\left\{ \begin{array}{ll}0&otherwise\\ mask_{r}[i][j]&mask_{r}[i][j]<\sigma\\ mask_{t}[i][j]=\left\{\begin{array}{ll}1&otherwise\\ 0&mask_{top}[i][j]=0\wedge mask_{e}[i][j]=0\end{array}\right.\end{array}