# Personalized Federated Multi-Task Learning over Wireless Fading Channels
## 3 Algorithm Description
### FedGradNorm Description
Then, (14) can be simplified as

\[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]6

In addition, the following equation is obtained from \(\mu\)-strong convexity of \(g(x,\cdot)\) with respect to \(p\) as well,

\[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]7

Inserting (16) into (15) and telescoping results as \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]8 \[\hat{g}_{k}(j)=\frac{y_{k}(j)}{|\mathcal{M}_{k}(j)|N},\quad j\in[|\omega|] \tag{21}\]9 \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]0 (102) \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]1

where the last inequality comes from \(\frac{\partial p_{k}^{0}}{\partial x_{k}}=0\) and (100). Then, the proof is completed by using (103) in (103) in addition to upper bounding \(\left\|\frac{\partial p^{*}(x_{k})}{\partial x_{k}}\right\|\) and \(\left\|p_{k}^{D}-p^{*}(x_{k})\right\|\) in (103) with (100) and (101), respectively. Proof of Lemma 3.: Based on \(L_{F}\)-smoothness of \(F(\cdot)\),

\[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]2

By the gradient descent update of \(x\)

\[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]3

By inserting (105) into (104), the following holds,

\[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]4 \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]5 \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]6 \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]7 (107) \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]8 \[\frac{\partial F(x_{k},p_{k}^{i})}{\partial x_{k}}=\
abla_{x}F(x_{k},p_{k}^{i})+ \frac{\partial p_{k}^{i}}{\partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{i})\]9

where the last inequality comes from

\[\left\|\frac{\partial F(x_{k},p_{k}^{D})}{\partial x_{k}}-\frac{ \partial F(x_{k},p^{*}(x_{k}))}{\partial x_{k}}\right\|= \left\|\
abla_{x}F(x_{k},p_{k}^{D})+\frac{\partial p_{k}^{D}}{ \partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{D})-\
abla_{x}F(x_{k},p^{*}(x_{k}))\right.\]0

by substituting \(x=\
abla F(x_{k},p_{k})\) and \(y=\hat{\
abla}F(x_{k},p_{k})\). Proof of Theorem 1.: By Lemma 3, we have

\[\left\|\frac{\partial F(x_{k},p_{k}^{D})}{\partial x_{k}}-\frac{ \partial F(x_{k},p^{*}(x_{k}))}{\partial x_{k}}\right\|= \left\|\
abla_{x}F(x_{k},p_{k}^{D})+\frac{\partial p_{k}^{D}}{ \partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{D})-\
abla_{x}F(x_{k},p^{*}(x_{k}))\right.\]1 \[\left\|\frac{\partial F(x_{k},p_{k}^{D})}{\partial x_{k}}-\frac{ \partial F(x_{k},p^{*}(x_{k}))}{\partial x_{k}}\right\|= \left\|\
abla_{x}F(x_{k},p_{k}^{D})+\frac{\partial p_{k}^{D}}{ \partial x_{k}}\
abla_{p}F(x_{k},p_{k}^{D})-\
abla_{x}F(x_{k},p^{*}(x_{k}))\right.\]2To upper bound \(A_{2}\), we use Lemma 2 and the fact that \((a+b+c)^{2}\leq 3a^{2}+3b^{2}+3c^{2},\forall a,b,c\in\mathbb{R}\), while also assuming \(\|p_{k}^{0}-p_{k}^{*}(x_{k})\|^{2}\leq\Delta\).