# A Federated Incremental Learning Algorithm Based on Dual Attention Mechanism Mechanism
## 3 Federated Incremental Learning Algorithm
### Federated Incremental Learning Framework  
In addition to data privacy protection in FL, dealing with the dynamic changes in training tasks is also essential to research content. For example, in recommender systems, user data will be updated dynamically. The traditional training model is in the face of new tasks and data. Retraining all the old and new data will cost much training. Therefore, whether the user data can be dynamically updated has become the key to measuring the pros and cons of an algorithm. We need a federated learning algorithm that can cope with the dynamic changes in training tasks and keep the data confidential. Because incremental learning can continuously learn new concepts in the data stream [35; 36; 37], we consider introducing incremental learning in federated learning to help each client first train data when there are only a few classes initially, and then add classes can be gradually added for learning. Still, incremental learning has the problem of historical forgetting. Therefore, we consider adding a new training strategy iCaRL strategy to incremental learning to classify according to the nearest mean rule of samples, use preferential sample selection based on herd behavior, and perform representation learning for knowledge extraction and prototype rehearsal, without retraining Increases in dynamic processing resources. Adding incremental learning to traditional federated learning can reduce the cost of data storage and model retraining when adding classes and reduce the risk of data leakage due to model gradient updates.  
In addition, in traditional federated learning, the initial parameters of the global model are all randomly generated numbers, which will affect the convergence speed of the model. To speed up the convergence speed of the model, this paper considers adding a pre-training module. Generally, the pre-training module uses \(n\%\) of all data as pre-training data. However, this method can easily expand the influence of clients with large sample sizes on the global model, especially the influence of non-important features [38; 39; 40; 41; 42; 43; 44; 45; 46; 47] of large samples in incremental learning. We plan to extract the same number of samples from each client as pre-training data to ensure the balance of pre-training samples and reduce the impact of large-sample clients on the global model. The federated incremental learning framework is shown in Figure 2.  
The client first uses a fixed number of samples of class 1 for training and aggregation to obtain pre-trained global model parameters, which can speed up the convergence of the model and reduce the impact of non-important features in large-sample client data on the global model. The global model parameters are distributed to the client, and the client uses the remaining samples of class 1 for training. After the training, the client model parameters of the \(t\)th communication are sent to the server, and the server sends the fused model parameters to the client. Then the client uses the class 2 samples and some old data to form a training set for training, uses the feature extractor to extract feature vectors from the old and new data, and calculates the respective average feature vectors. The predicted value is brought into the loss function of the combination of distillation and classification loss for optimization. The client model parameters at the \((t+1)\)th communication are obtained and sent to the server, so storage until the incremental learning of all classes is completed.