# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 3 Algorithm Description
### FedGradNorm Description
for Each client \(i\in[N]\)do  Initialize global shared network parameters for local updates by \(\omega_{k,0}^{(i)}\leftarrow\omega_{k}\) for\(j=1,\dots,\tau_{k}\)do \(h_{k,j}^{(i)}=\text{Update}(F^{(i)}(h_{k,j-1}^{(i)},\omega_{k,0}^{(i)}),h_{k,j-1}^ {(i)})\) \(F_{k}^{(i)}=0\) for\(j=1,\dots,\tau_{\omega}\)do \(\omega_{k,j}^{(i)}\leftarrow\omega_{k,j-1}^{(i)}-\beta g_{k,j}^{(i)}\) \(F_{k}^{(i)}=F^{(i)}(h_{k,\tau_{k}}^{(i)},\omega_{k,j}^{(i)})\) \(F_{k}^{(i)}\leftarrow\frac{1}{\omega_{k}}F_{k}^{(i)}\)  Client \(i\) sends \(g_{k}^{(i)}=\frac{1}{\omega_{k}}\sum_{j=1}^{\tau_{\omega}}g_{k,j}^{(i)}\), and \(F_{k}^{(i)}=\frac{F_{k}^{(i)}}{F_{0}^{(i)}}\) to the parameter server  After collecting \(g_{k}^{(i)}\), and \(F_{k}^{(i)}\) for active clients \(i\in[N]\), the parameter server performs the following operations in the order: * Constructs \(F_{\text{grad}}\left(k;\{p_{k}^{(i)}\}_{i=1}^{N}\right)\) using \(\{g_{k}^{(i)}\}_{i=1}^{N}\) and \(\{\tilde{F}_{k}^{(i)}\}_{i=1}^{N}\) as given in Equation (12). * Updates \(p_{k}^{(i)}\gets p_{k-1}^{(i)}-\alpha\
abla_{p^{(i)}}F_{\text{grad}}\), \(\forall i\in[N]\). * Aggregates the gradient for the global shared network by \(g_{k}=\frac{1}{N}\sum_{i=1}^{N}p_{k}^{(i)}g_{k}^{(i)}\). * Updates the global shared network parameters with the aggregated gradient by \(\omega_{k+1}=\omega_{k}-\beta g_{k}\). * Broadcasts \(\omega_{k+1}\) to clients for the next global iteration. ```
**Algorithm 2** Training with _FedGradNorm_

Each client within a cluster sends its gradient for the global model \(q_{\omega}\) to its corresponding IS via the LAN, where the channels between each client and the IS are assumed to be error-free inside a cluster. The corresponding IS performs a modified version of _FedGradNorm_ based on the client's gradients by taking the power allocation scheme into account. Specifically, the IS of cluster \(l\) computes the loss weight \(p_{k}^{(l,i)}\) for each client \(i\in[N]\) in cluster \(l\) via _FedGradNorm_ algorithm to eventually obtain the local weighted aggregation \(\sum_{l=1}^{N}p_{k}^{(l,i)}g_{k}^{(l,i)}\) at iteration \(k\), where \(g_{k}^{(l,i)}\) is the local gradient update of client \(i\) in cluster \(l\) for iteration \(k\). The power allocation vector \(\beta_{k}^{(l,i)}\) constructed by the IS of cluster \(l\) for each client \(i\) in the cluster is designed as

\[\beta_{k}^{(l,i)}(j)= \begin{cases}\frac{p_{k}^{(l)}}{H_{k}^{(l)}},&\text{if }|H_{k}^{(l)}(j)|^{2}\geq H_{k}^{\text{th}}\\ 0,&\text{otherwise}\end{cases}  \tag{14}\]

where \(\beta_{k}^{(l,i)}(j)\) is the \(j\)th entry of the power allocation vector \(\beta_{k}^{(l,i)}\in\mathbb{R}^{|\omega|}\), and \(H_{k}^{(l)}(j)\) is the \(j\)th entry of the channel gain vector \(H_{k}^{(l)}\in\mathbb{R}^{|\omega|}\), which represents the fading effect of the wireless channel between the IS and the PS of cluster \(l\). \(H_{k}^{(l)}(j)\) is assumed to be i.i.d. according to \(\mathcal{N}(0,\sigma_{l}^{2})\). The threshold \(H_{k}^{\text{th}}\) is set to satisfy the total average transmit power constraint given as follows,

\[\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}[\|x_{k}^{(l)}\|^{2}]\leq P \tag{15}\]where \(x_{k}^{(I)}=\sum_{i=1}^{N}x_{k}^{(I,i)}\) and \(x_{k}^{(I,i)}=\beta_{k}^{(I,i)}\circ g_{k}^{(I,i)}\), \(i\in[N]\), \(l\in[C]\), \(\circ\) represents the element-wise multiplication. The expectation is taken over the randomness of the channel gains. From the power allocation scheme in (14), each cluster transmits only the scaled entries of its weighted gradient for which the channel conditions are sufficiently good.