# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 2 System Model and Problem Formulation
### Hierarchical Federated Learning (HFL) for Wireless Fading Channels  
The characteristics of the communication channel should be considered in PF-MTL, since clients can be distributed in a large geographic area in an FL framework [41; 42]. The PS can be far away from the clients, and the communication between the PS and the clients can be noisy and subject to channel effects. Then, PF-MTL can be constructed in hierarchical setting by creating clusters of clients around IS to communicate with the PS instead of the direct communication of clients with the PS. Further, the communication can be performed over a shared wireless channel, where the transmission power and the bandwidth are constrained. Thus, we employ over-the-air aggregation (OTA) to address these issues [21; 22]. The generic HFL problem shown in Figure 2, with \(C\) clusters each containing an IS and \(N\) clients, can be formulated as  
\[\min_{\omega}\left\{F(\omega)\triangleq\frac{1}{CN}\sum_{l=1}^{C}\sum_{i=1}^ {N}p^{(l,i)}F^{(l,i)}(\omega)\right\} \tag{9}\]  
where \(p^{(l,i)}\) is the loss weight for client \(i\) in cluster \(l\) such that \(\sum_{i=1}^{N}p^{(l,i)}=N\), \(l\in[C]\), and \(F^{(l,i)}(\cdot)\) is the local loss function for client \(i\) in cluster \(l\). We consider a PFL setting of \(N\) clients within each cluster, in which client \(i\) of cluster \(l\) has its own local dataset \(D_{l,j}=\{(\mathbf{x}_{j}^{(l,i)},y_{j}^{(l,j)})\}_{j=1}^{n_{l,j}}\) where \(n_{l,i}\) is the size of the local dataset. Within cluster \(l\), \(T_{l,j}\) denotes the task of client \(i\), \(i\in[N]\), and \(l\in[C]\). \(T_{l,j}\) is assigned from the task set \(\mathcal{T}=\{T_{1},T_{2},\ldots,T_{N}\}\) such that \(T_{l,j}\
eq T_{l,j}\), for \(i\
eq i^{\prime}\) and for any \(l\in[C]\). Real-life scenarios might involve the same or very similar tasks for clients in a cluster. We assume that tasks are different due to the lack of prior information about it. We assume that clients in a cluster have error-free and high-speed connections to corresponding IS over local area networks (LANs). In addition, the PS and ISs share a bandwidth-limited wireless fading MAC. Using the wireless fading MAC, each IS sends the corresponding local gradient aggregations within its cluster to the PS. The broadcast from the PS to the ISs is considered error-free. As in the case of the simple PF-MTL setting illustrated in Figure 1, the system model in Figure 2 is composed of a global representation network \(q_{\omega}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d^{\prime}}\), which is a function parameterized by \(\omega\in\mathcal{W}\), that maps data points into a lower space of size \(d^{\prime}\). The same global representation network is shared by all clients in each cluster, which is synchronized through global aggregation. A client-specific head \(q_{h^{(l,i)}}:\mathbb{R}^{d^{\prime}}\rightarrow\mathcal{Y}\) is a function parameterized by \(h^{(l,i)}\in\mathcal{H}\) for all clients \(i\in[N]\) of every cluster \(l\in[C]\), mapping a low-dimensional representation space to a label space \(\mathcal{Y}\). The local model for client \(i\) of cluster \(l\) is the composition of the client's global representation model \(q_{\omega}\) and personalized model \(q_{h^{(l,i)}}\), shown as \(q_{l,i}(\cdot)=(q_{h^{(l,i)}}\circ q_{\omega})(\cdot)\). In addition, the local loss for the \(i\)th client of cluster \(l\) is shown as \(F^{(l,i)}(h^{(l,i)},\omega)=F^{(l,i)}(q_{l,i}(\cdot))=F^{(l,i)}((q_{h^{(l,i)} }\circ q_{\omega})(\cdot))\). Using alternating minimization, the PS and the clients learn the global representation \(\omega\) together, while only client \(i\) learns the the client-specific head \(h^{(l,i)}\) in cluster \(l\), \(i\in[N]\) and \(l\in[C]\). Specifically, \(\tau_{h}\) local updates are performed by client \(i\) in cluster \(l\) to optimize \(h^{(l,i)}\) when global network parameters at client \(i\) of cluster \(l\), i.e., \(\omega^{(l,i)}\), are frozen.