pMixFed: Mixing up model coefficients for Efficient Personalized Federated Learning

Authors: Yasaman Saadati, Mohammad Rostami, M. Hadi Amini
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Personalized federated learning, Distributed computing

Abstract:
Federated Learning  enables decentralized collaborative learning of machine learning models which presents challenges such as data privacy  and client drift for heterogeneous data. Traditional FL methods offer strong generalization but lack personalized solutions for non-IID data. Personalized federated learning (PFL) addresses   data heterogeneity by tackling these issues through balancing generalization and personalization level. It, however, still faces challenges such as optimal model partitioning and catastrophic forgetting that reduce quality and accuracy of both local and global models. To address these challenges,  we propose ``pMixFed'', a dynamic, layer-wise PFL approach integrating mixup between shared global and personalized local models. We develop adaptive partitioning between shared and personalized layers of the model,  gradual transition of personalization to allow seamless adaptation of local clients, improved generalization across clients, and mitigation of catastrophic forgetting. We provide theoretical analysis of pMixFed. Further, we conduct extensive experiments to demonstrate   its superior performance compared with the existing PFL methods. Empirical results hows faster training, increased robustness, and improved handling of  heterogeneity when using pMixFed as compared with the state-of-the-art PFL models.

Decision: desk reject

Reviews and discussion:

Metareview: