# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 3 Algorithm Description
### FedGradNorm Description
## Appendix A Proof of Lemma 2

Since \(p_{k}\) depends on \(x_{k}\), the following holds by the chain rule,

\[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]

Then,

\[\left\|\frac{\partial F(x_{k},p_{k}^{D})}{\partial x_{k}}-\frac{ \partial F(x_{k},p^{*}(x_{k}))}{\partial x_{k}}\right\|= \left\|\
abla_{x}F(x_{k},p_{k}^{D})+\frac{\partial p_{k}^{D}}{ \partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{D})-\
abla_{x}F(x_{k},p^{*}(x_{k}))\right.\] \[\left.-\frac{\partial p^{*}(x_{k})}{\partial x_{k}}\
abla_{p}F(x_ {k},p^{*}(x_{k}))+\frac{\partial p^{*}(x_{k})}{\partial x_{k}}\
abla_{p}F(x_ {k},p_{k}^{D})\right.\] \[\left.-\frac{\partial p^{*}(x_{k})}{\partial x_{k}}\
abla_{p}F(x _{k},p_{k}^{D})\right\|\] (10) \[\leq \left\|\
abla_{x}F(x_{k},p_{k}^{D})-\
abla_{x}F(x_{k},p^{*}(x_{k }))\right\|\] \[+\left\|\frac{\partial p_{k}^{D}}{\partial x_{k}}-\frac{\partial p ^{*}(x_{k})}{\partial x_{k}}\right\|\left\|\
abla_{p}F(x_{k},p_{k}^{D})\right\|\] \[+\left\|\frac{\partial p^{*}(x_{k})}{\partial x_{k}}\right\|\left\| \
abla_{p}F(x_{k},p_{k}^{D})-\
abla_{p}F(x_{k},p^{*}(x_{k}))\right\|\] (11)

where the last inequality follows from the triangle inequality. The first term of (A) is bounded by Assumption 2 as follows,

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]0 (12)

Similarly, the third term of (A) is bounded by Assumption 1

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]1 (13)

Furthermore, the second term of (A) is bounded as

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]2 (14)

since we have \(\left\|\
abla_{p}F(x,p)\right\|\leq M\) by Assumptions 4 and 5. Then, (A) is upper bounded as

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]3 \[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]4 (15)

To upper bound (A) further, we bound \(\left\|\frac{\partial p_{k}^{D}}{\partial x_{k}}-\frac{\partial p^{*}
+++

By the gradient descent update of \(p\), i.e., \(p_{k}^{t}=p_{k}^{t-1}-\alpha\
abla_{p}g(x_{k},p_{k}^{t-1})\) for \(t=1,\ldots,D\), and by the chain rule,

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]5

Additionally, based on the optimality of \(p^{*}(x_{k})\), we have \(\
abla_{p}g(x_{k},p^{*}(x_{k}))=0\). Then, by taking the partial derivative with respect to \(x_{k}\),

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]6

By combining (10) and (11),

\[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]7 \[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]8 \[y_{k}(j)=\sum_{l\in\mathcal{M}_{k}(j)}\sum_{i=1}^{N}p_{k}^{(I,i)}g_{k}^{(I,i)} (j)+z_{k}(j) \tag{20}\]9 \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]0 \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]1 \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]2

[PERSON], by (11) and Assumptions 1, 2,

\[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]3

By (12), (13) and Assumption 2, we have

\[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]4 \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]5

Furthermore, based on \(\mu\)-strong convexity of \(g(x,\cdot)\) with respect to \(p\), we have \(\
abla_{p}^{2}g(x,\cdot)\geq\mu\) for any \(x\in\mathcal{H}^{N}\times\mathcal{W}\).