# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 1 Introduction  
Multi-tasking (MTL) is a powerful technique for learning several related tasks simultaneously [1, 2]. MTL improves the overall system performance, the training speed, and data efficiency, by leveraging the synergy among multiple related tasks. Each task in MTL setting has a single common encoder network to map the raw data into a lower dimensional shared representation space, in addition to a unique client-specific header network to infer task related prediction values from the shared representation. MTL is particularly suitable for distributed learning settings where no single entity has all the data and labels for all of the multiple different tasks. Federated learning (FL) [3] is a distributed learning paradigm in which many clients train a shared model with the assistance of a central unit called parameter server (PS) by keeping their data private and decentralized. Statistical heterogeneity becomes a major challenge for FL as the size of the distributed setting, namely the number of clients or the number of tasks in a FL setting, increases. This is caused by different task complexities and non-i.i.d. data distribution among clients. Statistical heterogeneity due to non-i.i.d. data distribution degrades the system performance [4, 5]. Further, synergy between the tasks may not always be positive, referred to as a negative transference, again degrading the performance [6]. Personalized federated learning (PFL) is introduced to deal with statistical heterogeneity, where the centralized server and clients learn a shared representation together, while each client trains its own client-specific header network further, referred to as personalization. PFL has the capability of learning user-specific models better while also capturing the distilled common knowledge from other clients [7; 8; 9]. As a result, PFL reduces the statistical heterogeneity among clients. Several PFL approaches have been proposed, where different local models are used to fit user-specific data, but also capture the common knowledge distilled from data of other devices for this purpose [7; 8; 9; 10; 11; 12; 13; 14; 15]. [PERSON] et al. [16] provides a unified model framework to prove multiple kinds of PFL methods. In this paper, we consider MTL in a FL setting, enhanced with personalization, namely, personalized federated multi-task learning (PF-MTL). PFL works that are closely related to our work are federated representation learning _FedRep_[7] and federated learning with personalization layers _FedPer_[8]. These works use a shared encoder with a unique task-specific header to enhance personalization in a FL setup. _FedRep_ and _FedPer_ aggregate the common encoder parameters by equal weighting. Our goal in this paper is to perform weighted gradient aggregation at the PS dynamically according to gradient updates coming from clients to overcome statistical heterogeneity. Several approaches involve setting the weights of tasks manually at the beginning of training [17; 18]. Task weights can also be set in an adaptive manner for each iteration. _GradNorm_ is a dynamic weighting approach in MTL that normalizes gradient norms by scaling the task loss functions to regulate learning speeds and fairness across tasks [19]. _GradNorm_ is proposed for a centralized learning setting. In our work, we introduce _FedGradNorm_ framework [20], where dynamic weighting is incorporated in PFL setting by considering some aspects of [7; 8; 19] together. We provide a theoretical convergence proof for _FedGradNorm_, while _FedPer_[8] and _GradNorm_[19] do not provide a convergence proof, and _FedRep_[7] provides a convergence proof only for the linear learning model setting. We investigate how the _FedGradNorm_ framework is affected by the characteristics of the wireless fading communication channel between clients and the PS. There are different channel conditions since clients are geographically spread out.