## 5 Personalized FL with Multi-Level Privacy-Preserving IoT Knowledge Sharing
### Personalized FL of IoT Users with Non-IID Data  
The data of clients requiring high privacy protection and general client data do not meet independently and, in some cases, are identically distributed. For example, in the smart home scenario, the privacy protection of health data recorded by wearable smart devices of family members has high requirements. The distribution of these health data and the indoor environment data (like temperature, humidity, etc.) recorded by other smart devices is highly likely to differ. Therefore, the PFL in IoT users with non-IID data also needs to be considered.  
In the case of non-IID, different clients adapt different models. We implement personalization through a federated multi-task learning framework, Ditto [42]. There are two \"tasks\" in the system that need to be considered: the global objective \(G\) and the local objective \(F^{k}(\theta)\). To better adapt to the heterogeneity of data between clients, personalized and client-specific models \(F^{k}(\theta)\) need to be emphasized. A regularization term is added to relate the two tasks by bringing each local personalized model close to the optimal global model. The \(k\)th client has the bi-level optimization target:  
\[\begin{array}{ll}\min\limits_{\theta}&h_{k}(v^{k};\bar{\theta}):=F^{k}(v^{k} )+\frac{\lambda}{2}\left\|v^{k}-\bar{\theta}\right\|\\ s.t.&\bar{\theta}\in\operatorname*{arg\,min}\limits_{\theta}G(F^{1}(\theta), \ldots,F^{K}(\theta))\end{array}, \tag{13}\]  
where the hyperparameter \(\lambda\) leads to a trade-off between local and global models. The smaller \(\lambda\) encourages training to approach a model to local data. When \(\lambda\) is set to 0, Ditto is reduced to local models. As \(\lambda\) grows larger, the impact caused by data from other clients continues to increase. When \((\lambda\rightarrow+\infty)\), it recovers the global model objective \(G\).  
There are three major constraints, namely fairness, robustness, and privacy in FL. The general approach makes it difficult to satisfy all three constraints simultaneously. The heterogeneity of the data distribution is the fundamental reason that limiting fairness and robustness cannot rise simultaneously. PFL helps improve fairness and robustness by learning the different models of each client to adapt to heterogeneity. Privacy can be guaranteed through grading the security requirements and control of the knowledge flow by the ACE algorithm. Therefore, our multi-level privacy-preserving PFL scheme has a good performance in terms of fairness, robustness, and privacy.  
Algorithm 4 depicts the local training of PFL in IoT users with non-IID data. It should be noted that the local personalized model that was trained is an objective function including the regularization term, while the update parameters uploaded to the server are calculated without regularization terms. The aggregation process in cloud server is the same as in the IID case, referring to Algorithm 3 for details.  
```
1:Local data \(D^{k}\), batch size \(B\), number of local epochs \(E\), initialized model parameters \(\theta_{s,t}\), client-specific parameter \(v^{k}\), interpolation control parameters \(\lambda\), and client encryption key \(ek\).
2:Ciphertext C.
3:\(\theta_{s,t}^{k}\leftarrow\theta_{s,t}\)
4:for local epoch \(e=1,2,\ldots,E\)do
5:for each batch\(\{(x_{i}^{k},y_{i}^{k})\}_{i=1}^{B}\) sampled from \(D^{k}\)do
6:\(\mathcal{L}(\theta_{s,t}^{k},D^{k})=\frac{1}{B}\sum_{i=1}^{B}(\theta_{s,t}^{k },x_{i}^{k},y_{i}^{k})\)
7:\(\theta_{s,t}^{k}\leftarrow\theta_{s,t}^{k}-\eta\
abla C(\theta_{s,t}^{k},D^{k})\)
8:\(v^{k}=v^{k}-\eta(\
abla F^{k}(v^{k})+\lambda(v^{k}-\theta_{s,t}^{k}))\)
9:endfor
10:endfor
11:\(m\leftarrow\theta_{s,t}^{k}\)
12:\(C\leftarrow\) ACE\({}^{n}.Enc(ek,m)\)
13:return C
```  
**Algorithm 4** Client procedure for personalized FL.