# A Federated Incremental Learning Algorithm Based on Dual Attention Mechanism Mechanism
###### Abstract  
Federated incremental learning best suits the changing needs of common Federal Learning (FL) tasks. In this area, the large sample client dramatically influences the final model training results, and the unbalanced features of the client are challenging to capture. In this paper, a federated incremental learning framework is designed; firstly, part of the data is preprocessed to obtain the initial global model. Secondly, to help the global model to get the importance of the features of the whole sample of each client, and enhance the performance of the global model to capture the critical information of the feature, channel attention neural network model is designed on the client side, and a federated aggregation algorithm based on the feature attention mechanism is designed on the server side. Experiments on standard datasets CIFAR10 and CIFAR100 show that the proposed algorithm accuracy has good performance on the premise of realizing incremental learning. federated learning; dual attention mechanism; incremental learning +
Footnote †: journal: Medical Imaging  
0000-0001-9991-9999-2000-00001-9999-20001-9999-20001-9999-20001-9999-20001-9999-20001-9999-20001-9999-20001-99999-20001-9999-20001-99999-20001-9999-20001-99999-20001-99999-20001-999  
based on a federation mechanism in 2019 [11]; it can allow knowledge sharing without compromising user privacy. [PERSON] and others proposed a federated reinforcement learning framework in 2020 [12]; this framework adds Gaussian interpolation to shared information to protect data security and privacy. [PERSON] et al. proposed a federated semi-supervised learning method for network traffic classification in 2021 [13]; it can effectively protect data privacy and does not require a large amount of labeled data. Traditional FL can only be trained in a batch setting, classification information for all samples is known beforehand, model's classify ability is also fixed. In the face of new tasks and data, it has to re-train totally. Among the many directions of FL at present, research on incremental tasks is rare [14, 15]. [PERSON] [16] pointed out that traditional data processing technologies have outdated models, weakened generalization capabilities, and do not consider the security of multi-source data. They proposed an online federated incremental learning algorithm for blockchain in 2021. During incremental learning, there will be a problem of unbalanced client samples. Clients with a large sample size have greater weight and significantly impact the final model training results. Given the above discussion, to dynamically handle the increase of resources without retraining, reduce the impact of large sample clients on the final model training results, and the problem of feature imbalance when model aggregation is elusive. We propose a federated incremental learning algorithm based on a dual attention mechanism. This algorithm can dynamically handle the increase in resources without retraining while mining the characteristics of the overall client-side samples; it can enhance the capture performance of the model training server for the key information of client-side features. The contributions of this paper are as follows:  
(1) We design a federated incremental learning framework. First, the framework randomly sampling the same number of samples from each client, to ensure the balance of pre-training samples, and trains with the federated averaging model to obtain the preliminary period global model on the server. Then the iCaRL strategy [17] is applied to the traditional FL framework; this strategy can classify samples according to the nearest mean rule, use preferential sample selection based on herd behavior, perform representation learning for knowledge distillation, and dynamically handle resource increases without retraining. Therefore, the federated incremental learning framework can take the dynamic changes in training tasks and keep the data confidential. (2) The dual attention mechanism is added to the federated incremental learning framework.