# A Federated Incremental Learning Algorithm Based on Dual Attention Mechanism Mechanism
## 3 Federated Incremental Learning Algorithm
### Dual Attention Mechanism Module
As shown in the figure, the input image size is 32 \(\times\) 32, that is, M = 32, the number of channels C is 3, and it is input to the convolution layer 1. The convolution layer contains 16 convolution kernels, and the convolution kernel sizes 1 \(\times\) 1, that is, kernel = 1, and no pixels are filled around the input image matrix, that is, P = 0. We can think of the convolution kernel as a sliding window, which slides forward with a set step size, set the step size to 1, that is, bu = 1. According to the output calculation Formula (6) of the convolutional layer, the size of the output image can be calculated as 32 \(\times\) 32, that is, Ne = 32, and the number of channels C is 16. \[Ne=\frac{M-kernel+2P}{bu}+1 \tag{6}\]  
After that, we perform global average pooling on it, perform feature compression on the output image obtained through convolutional layer one along the spatial dimension, and turn each two-dimensional feature channel into an actual number, which has a global effect to some extent. The receptive field and t output dimension match the input's feature channels put, and the number of channels C is 16. After global average pooling, the feature dimension C = 16 is reduced to the dimension C = 8 through the fully connected layer 1, and then activated by ReLU. Then the dimension C = 8 is raised back to the original dimension 16 through the fully connected layer 2, which is used here. The two fully connected layers can better fit the complex correlation between channels with more nonlinearities, and significantly reduce the number of parameters and computation. After the fully connected layer 2, a sigmoid activation function is used to obtain a normalized weight between 0 and 1, and finally, a Scale operation is used to weight the normalized channel attention weight to the features of each channel; it can help focus on the overall feature importance of the sample and reduce the impact of noise on the final model training results. The Scale operation refers to the channel-by-channel weighting of the previous features through multiplication to complete the re-calibration of the original feature map in the channel dimension. After that, we continue using convolutional layer 2 and convolutional layer 3 for feature extraction on the noise-reduced feature map. And adding add activation function ReLU and MaxPool between convolutional layer 2 and convolutional layer 3. The activation function can extract helpful feature information and make the model more discriminative, and the pooling layer can avoid overfitting by down-sampling. Finally, because two or more fully connected layers can solve the nonlinear problem satisfactorily, we input the features of the convolutional layer 3 into the fully connected layer 3 and the fully connected layer 4 to obtain a 512-dimensional vector, then go through the full connection layer. Connection layer 5 receives a c-dimensional vector (c is the number of classes in the current dataset), then inputs it into the softmax classifier for classification, and outputs the predicted value y. As shown in Figure 3, the second dual attention mechanism module considers that incremental learning, when learning at the same time as incremental classification learning, it the problem of unbalanced features and difficulty in capturing important featurefeaturesd a feature-space attention mechanism to enhance the grasping performance of key information. Feature attention mechanism based global aggregation is shown in Figure 6.  
As shown in the figure above, to enhance the capture performance of key information and solve the problem that the respective features are unbalanced and the importance is difficult to capture in the incremental learning with the incremental learning of classification and learning at the same time, this paper introduces the feature attention mechanism for the model aggregation of the client, we improve model performance by capturing the importance of neural network layers in multiple local models.