# Personalized Federated Learning with Progressive Local Training Strategy and Lightweight Classifier
## 3 Methodology
### Progressive Local Training Strategy  
Current personalized federated learning frameworks decouple models into personalized headers and shared feature extractors. At the beginning of each round, they directly initialize the local feature extractor using the global feature extractor and concatenate it with the personalized head (which always resides locally) to complete the initialization of the local model. Essentially, they ignore the continuity of local training and cause the local knowledge to be forgotten directly. For a certain local client, if it does not participate in the global training for many rounds, the continuity of its local training will be severely damaged, leading to the direct forgetting of local knowledge. Additionally, in cases where there is high data heterogeneity among clients, this heterogeneity is transferred to the local models through the global model. As a result, clients may receive a model that significantly differs from their own data, which is detrimental to local training. To address this issue, we propose a Progressive Local Training Strategy which preserves the local model from the client's last participation in global training. This preserved model is then used to recall knowledge when initializing the global model in the next participating round. In prior personalized federated learning frameworks, the stage where the global feature extractor is concatenated with the local classifier was commonly referred to as the model initialization stage. This stage is crucial as it sets the foundation for the subsequent training process by combining the global and local components of the model. However, we propose to introduce a new stage prior to this, called the local knowledge recall stage. This additional stage aims to leverage the knowledge gained from previous training rounds, ensuring that the model retains valuable local information before integrating the global feature extractor. By incorporating the local knowledge recall stage, we aim to enhance the model's ability to utilize previously learned local information, thereby improving its overall performance and stability. The approach helps to mitigate the issue of discontinuity in local training and ensures a smoother transition to the model initialization stage. We refer to this overall strategy as a Progressive Local Training Strategy. This strategy not only improves the alignment between local and global models but also enhances the model's ability to generalize across diverse data distributions. In this stage, we denote the local feature extractor from the most recent round of participation in the federated process as \(\theta_{i}^{l}\), where 'l' refers to 'local'. The global feature extractor obtained in the current round is \(\theta_{g}\), and the output of the data after passing through the feature extractor \(\theta\) is \(z_{j}=f(\theta,x_{j})\), where \(f\) represents the mapping function from the data to the features. Next, for \(x_{j}^{i}\in D_{i}\) belonging to client i, we utilize \(z_{j}^{i}=f(\theta_{i}^{l},x_{j}^{i})\) as the soft label for \(x_{j}^{i}\). Correspondingly, the output of \(x_{j}^{i}\) on \(\theta_{g}\) is \(z_{j}^{g}\).