# A Review of Research on Secure Aggregation for Federated Learning
## 4 Aggregation Techniques for Federated Learning
### Fundamental Aggregation Algorithms  
Traditional aggregation algorithms have been proposed to address communication efficiency and privacy preservation issues in federated learning. These traditional aggregation algorithms do not optimize the aggregation mechanism itself; instead, they enhance the efficiency and accuracy of model training by employing mathematical techniques and tuning various parameters. These algorithms are usually integrated into the federated learning framework as foundational algorithms. The commonly used traditional aggregation algorithms are as follows:  
* **FedAvg**: The FedAvg [60] algorithm is a classic federated learning aggregation algorithm, where a subset of clients is randomly selected for aggregation in each training round. During aggregation, client parameters are weighted and averaged, with weights determined by the client's data volume relative to the total data volume. FedAvg has efficient communication, supports non-independent and non-identically distributed (non-IID) data, and achieves high accuracy. However, when the degree of non-IID data distribution is too high, model convergence slows down. Furthermore, security considerations are lacking, failing to ensure participant trustworthiness. FedAvg updates the global model using the following equation: \[\omega_{glob}^{t+1}\leftarrow\sum_{k}^{S_{t}}\frac{n_{k}}{n}\omega_{k}^{t+1}.\] (12)where \(\omega_{glob}^{t+1}\) is the aggregated global model, \(S_{t}\) represents the set of selected clients, \(\omega_{k}^{t+1}\) denotes the locally updated model for client \(k\) after local training, and \(\frac{\eta_{k}}{n}\) is the weight factor. * **FedProx**: FedProx [61] generalizes and reparameterizes FedAvg to address local optimization challenges in stochastic gradient descent (SGD)-based algorithms. It introduces a correction term to the client-side loss function, improving model performance and enhancing convergence speed. The correction term is defined as \(\frac{\mu}{2}\|\omega_{k}^{t}-\omega_{glob}^{t}\|_{2}\), the L2 norm of the difference between the local model and the global model, where \(\mu\) is the penalty constant for the regularization term, designed to penalize clients with large deviations from the global model, thereby constraining the behavior of clients participating in training. * **Scaffold**: Scaffold [62] is an update process that corrects both the global and local models by calculating the differences between the server-side and client-side control variables. This method effectively addresses the client drift problem caused by data heterogeneity in FedAvg. The model update in Scaffold consists of the following four steps: 1. The client-side local model is updated as follows: \[y_{i}\gets y_{i}-\eta_{l}(g_{i}(y_{i})-c_{i}+c).\] (13) 2. The client-side control variable is updated as follows: \[c_{i}^{+}\gets c_{i}-c+\frac{1}{K\eta_{l}}(x-y_{i}).\] (14) 3. The server global model is updated as follows: \[x\gets x+\eta_{g}\Delta x.\] (15) 4. The server control variable is updated as follows: \[c\gets c+\frac{1}{N}\Sigma_{i\in S}(c_{i}^{+}-c).\] (16) where \(\eta\) represents the learning rate, \(K\) denotes the number of local update steps, \(g_{i}(x)\) is the gradient computation function, \(S\) stands for the selected client set, and \(N\) is the total number of clients. * **FedBN**: FedBN [63] trains a local model on each client and incorporates a batch normalization (BN) layer for normalization. FedBN is designed to address the heterogeneity of federated learning data, particularly the feature drift issue. The FedBN normalization layer algorithm is as follows: \[\omega_{t+1,k}^{(l)}\leftarrow\frac{1}{P}\Sigma_{k=1}^{p}\omega_{t+1,k}^{(l)}.\] (17) where \(t\) is the training period, \(k\) is the client serial number, \(l\) is the number of neural network layers, and \(P\) is the set of clients. * **FedDF**: FedDF [64] applies the concept of distillation to model fusion. Each client trains a local model, which is subsequently fused through distillation.