# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 3 Algorithm Description
### FedGradNorm Description
Consequently, \(F_{\text{grad}}\) is modified according to the power allocation scheme to have power efficient system design as follows,

\[F_{\text{grad}}^{(I)}\Big{(}k;\{p_{k}^{(L)}\}_{i=1}^{N}\Big{)} =\sum_{i=1}^{N}F_{\text{grad}}^{(I,i)}\Big{(}k;p_{k}^{(I,i)}\Big{)} \tag{16}\] \[=\sum_{i=1}^{N}\left\|p_{k}^{(I,i)}\right\|M_{k}^{(I)}\circ\
abla_ {\omega_{k}^{(I,i)}}F_{k}^{(I,i)}\right\|-\bar{G}_{\omega}^{(I)}(k)\times[r_{k} ^{(I,i)}]^{\gamma}\right\| \tag{17}\]

where \(M^{(I)}\in\{0,1\}^{|\omega|}\) is a mask matrix designed for the sparsification of cluster \(l\) as follows:

\[M_{k}^{(I)}(j)= \begin{cases}1,&\text{if }|H_{k}^{(I)}(j)|^{2}\geq H_{k}^{ \text{th}}\\ 0,&\text{otherwise}\end{cases}  \tag{18}\]

Here, \(\bar{\omega}_{k}^{(I,i)}\) is the last layer of the shared network at client \(i\) of cluster \(l\) at iteration \(k\). \(\bar{G}_{\omega}^{(I)}(k)\) is the average sparsified gradient norm across all clients (tasks) in cluster \(l\) at iteration \(k\). \(r_{k}^{(I,i)}=\frac{F_{k}^{(I,i)}}{E_{j-\text{task}}[F_{k}^{(I,i)}]}\) is the relative inverse training rate of task \(i\) in cluster \(l\) at iteration \(k\), and \(\gamma\) represents the strength of the restoring force, as defined in _FedGradNorm_ previously. Gradient sparsification used during the calculation of \(F_{\text{grad}}\) acts as an implicit constraint on \(F_{\text{grad}}\) minimization problem by considering the channel conditions. Consequently, it ensures that the learning speed of tasks is invariant to the dynamic channel conditions with an appropriate selection process of loss weights. In other words, the implicit constraint of the channel condition preserves the fairness of the learning speed among the clients, as shown in the experimental results. The second stage of the algorithm involves the process of global aggregation over the wireless fading MAC. The PS obtains a noisy estimate of the aggregated gradient over the wireless fading channel while updating the model parameters. Due to the additive nature of the wireless MAC, the summation of the signals transmitted by clusters arrives at the PS. The \(j\)th entry of the received signal at iteration \(k\), \(y_{k}\in\mathbb{R}^{|\omega|}\) is

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}H_{k}^{(I)}(j)x_{k}^{(I)}(j)+z_{k}(j) \tag{19}\]

where \(z_{k}(j)\) is the \(j\)th entry of the Gaussian noise vector \(z_{k}\) and is i.i.d. according to \(\mathcal{N}(0,1)\). \(\mathcal{M}_{k}(j)=\{c\in[C]:|H_{k}^{(I)}(j)|^{2}>H_{k}^{\text{th}}\}\) represents the set of clusters contributing to the \(j\)th entry of the received signal at the \(k\)th iteration. \(\mathcal{M}_{k}(j)\) is known by the PS, for \(j\in[|\omega|]\) since the PS has the perfect channel state information (CSI). By considering (14) and the definition of \(x_{k}^{(I)}\) in terms of the power allocation vector, we have

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]

where \(g_{k}^{(I,i)}(j)\) is the \(j\)th entry of \(g_{k}^{(I,i)}\). The noisy aggregated gradient estimate is

\[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]Then, the estimated gradient vector is used to update the model parameters as \(\omega_{k+1}=\omega_{k}-\beta\hat{g}_{k}\). The overall algorithm is shown in Algorithm 3. ```
1:Initialize \(\omega_{0}\), \(\{p_{0}^{(1,i)}\}_{l=1,l=1}^{C,N}\), \(\{h_{0}^{(1,i)}\}_{l=1,l=1}^{C,N}\)
2:for\(k\)=0 to \(K\)do
3: The PS broadcasts the current global shared network parameters \(\omega_{k}\) to the ISs. 4:for Each cluster \(l\in[C]\)do
5:\(\omega_{k}^{(l)}\leftarrow\omega_{k}\). 6: The IS \(l\) broadcasts \(\omega_{k}^{(l)}\) to clients within cluster.