Year: 2021

Abstract:
In the conventional federated learning (FL), the local models of multiple clients are trained independently by their privacy data, and the center server generates the shared global model by aggregating local models. However, the global model often fails to adapt to each client due to statistical and systems heterogeneities, such as non-IID data and inconsistencies in clients’ hardware and bandwidth. To address these problems, we propose the Subclass Personalized FL (SPFL) algorithm for non-IID data in synchronous FL and the Personalized Leap Gradient Approximation (PLGA) algorithm for the asynchronous FL. In SPFL, the server uses the Softmax Normalized Gradient Similarity (SNGS) to weight the relationship between clients, and sends the personalized global model to each client. In PLGA, the server also applies the SNGS to weight the relationship between client and itself, and uses the first-order Taylor expansion of gradient to approximate the model of the delayed clients. To the best of our knowledge, this is one of the few studies investigating explicitly on personalization in asynchronous FL. The stage strategy of ResNet is further applied to improve the performance of FL. The experimental results show that (1) in synchronous FL, the SPFL algorithm used on non-IID data outperforms the vanilla FedAvg, Per-FedAvg, FedUpdate, and pFedMe algorithms, improving the accuracy by 1.81∼ 18.46% on four datasets (CIFAR10, CIFAR100, MNIST, EMNIST), while still maintaining the state of the art performance on IID data; (2) in asynchronous FL, compared with the vanilla FedAvg, Per-FedAvg, and FedAsync algorithms, the PLGA algorithm improves the accuracy by 0.23∼12.63% on the same four datasets of non-IID data.

heading: 1 INTRODUCTION
text: With the popularity of smartphones, personal computers, and other devices, the data stored on them has increased dramatically. These data are related to each other but exist independently on different devices. Also, their owners are often not willing to share their private data, which prompted Federated Learning (FL) (McMahan et al., 2017). Conventional FL is a distributed machine learning framework in which multiple clients train a global model together under a central server’s coordination without sharing their local data. It’s common process is as follows: (1) At each global iteration, the central server broadcasts its current global model to the client; (2) With the help of the global model, the client updates its local model which trained on its local data, and sends it back to the central server; (3) The central server aggregates the local models to get a new global model. The FL process will repeat these three steps until convergence. Under the premise of privacy protection, it can obtain a shared global model with higher generality. Nonetheless, the wide application of FL still is facing the following two major challenges:
• Statistical heterogeneity. As the user preference of each device may be different, the data distribution on each device may also be inconsistent. Consequently, the data as a whole on all the clients may be unbalanced and non-IID. This may cause undesirable performance of the FL participants.
• Systems heterogeneity. Due to the differences of users’ hardware and network bandwidth, conventional FL models are prone to generate stragglers whose models are easy to be discarded after server’s time-out.
Firstly, to address the statistical heterogeneity, the personalization of FL is widely studied. The majority of the available personalized FL (PFL) methods focus on one shared global model’s parameters updating by client’s fine-tune. FedAvg (McMahan et al., 2017) is treated by (Jiang et al., 2019a) as a client’s meta-learning-like process fine-tuning on one shared global model with higher accuracy. FedMeta (Chen et al., 2018), also a personalized FL based on meta-learning, generates a local and personalized model for each client using a common global model. Different from the aforementioned methods, we propose a Subclass Personalized Federated Learning (SPFL) algorithm, which utilizes the Softmax Normalized Gradient Similarity (SNGS) to generate global models with each specific to one client for its parameter fine-tune.
Secondly, to tackle the systems heterogeneity, the server of an asynchronous FL algorithm named FedAsync (Xie et al., 2019), can immediately perform aggregation after receiving a local model, reducing the waiting time for stragglers. In TWAFL (Chen et al., 2019), a time-weighted aggregation strategy is used by server to handle stragglers according to the staleness of the received model parameters. Although these asynchronous FL algorithms can overcome some difficulties encountered in synchronous aggregation, such as waiting for the response from client, they are still inefficient for directly aggregating the received local model on the server, without taking the iteration gap of stragglers’ local models at global communication round into consideration. Therefore, we propose the Leap Gradient Approximation (LGA) algorithm using SNGS to predict the local model of the stragglers in each global communication round. These local models estimated by server are also aggregated with non-stragglers’ local models on the server to attained one shared global model. Further, to consider the real scenarios where system heterogeneity and statistical heterogeneity often co-exist, personalization is employed in our asynchronous FL. Before each individual personalized model for client is dispatched from server, it is computed by using the shared global model and the SNGS value between server and that client. We refer this personalized asynchronous FL approach as Personalized Leap Gradient Approximation (PLGA) algorithm.
Contribution The main contributions of this paper towards personalized heterogeneous FL can be summarized as follows: (1) We propose the SPFL algorithm for both IID and non-IID data in personalized synchronous FL, where the server uses the SNGS to weight the relationship between clients, and delivers the personalized global model to each client; (2) We propose the PLGA algorithm for the personalized asynchronous FL, in which the server also applies the SNGS to weight the relationship between client and itself, and uses the first-order Taylor expansion of gradient to approximate the model of the delayed clients; (3) The stage strategy of ResNet is applied to improve the performance of both SPFL and PLGA algorithms.

heading: 2 RELATED WORK
text: In general, the personalized process in FL can be achieved by a variety of ways. The work (Hard et al., 2018) shows that by adding user-related features into the local inference process to get a personalized model may be better than the global one. The completely decentralized algorithms for each client by smoothing some similar tasks or model parameters between similar data distributions are studied in (Bellet et al., 2018; Vanhaesebrouck et al., 2017). Algorithms are also proposed to achieve personalization by jointly learning the model on a similar graph structure (Lalitha et al., 2019; He et al., 2021). FedFOMO (Zhang et al., 2020) is another approach to PFL, where each client only federates with other relevant clients for model aggregation. The pFedHN (Shamsian et al., 2021) algorithm is proposed using hypernetwork to generate different personalized model for each client.
Furthermore, PFL can also be studied by its relationship with other machine learning paradigms. The close relationship between FL and meta-learning is analyzed by (Khodak et al., 2019; Jiang et al., 2019b). Following the classic meta-learning algorithm named MAML (Finn et al., 2017), Reptile algorithm (Nichol et al., 2018) is proposed , which has a close relationship with the Federated Averaging algorithm. When each individualized model learning is treated as a task, such a scenario can be viewed as a multi-task learning(Zhang & Yang, 2017), in which each task learns a taskrelated model. MOCHA (Smith et al., 2017) is designed by regarding FL as a multi-task, which solves communication efficiency and fault tolerance. To provide a good global model, there are PFL applying transfer learning and domain transfer (Mansour et al., 2009; Cortes & Mohri, 2014; BenDavid et al., 2010). Model mixing is another strategy to approach PFL (Deng et al., 2020; Hanzely
& Richtárik, 2020; Liang et al., 2020). Alternatively, Other PFL methods apply clustering (Mansour et al., 2020; Ghosh et al., 2020; Duan et al., 2020). Finally, pFedMe (Dinh et al., 2020) applies Moreau Envelopes as clients’ regularized loss functions to achieve personalization.
There are just few studies on asynchronous FL. TWAFL (Chen et al., 2019) aggregates a temporally weighted local models on the server. FedAsync (Xie et al., 2019) introduced an algorithm to balance the parameters of the updated model and the last global model. Meanwhile, the distillation technique has also been applied in (Li & Wang, 2019; Bistritz et al., 2020).

heading: 3 METHOD
text: Our method generates a personalized model for each client during the server aggregation process. Therefore, the initialization model of each client is personalized. At the same time, we also consider the contribution of different neural network layers to the personalized model. We will introduce the application of this personalized algorithm to synchronous FL and asynchronous FL in the following subsections.
Algorithm 1 Subclass Personalized Federated Learning (SPFL) Algorithm Input: initialized global model parameters w0; initialized SNGS matrix S̃s ∈ RN×N ; initialized
personalized local model of each client’s {w0i } = w0; update frequency Γ and the number S of SNGS matrix; client learning rate β; server learning rate α; local epoch E; local batch size B of each client
Output: Model Parameters {wi} Def MainLoop:
for t = 0, 1, . . . do perform the following steps in parallel for each client Ci ∈ C
if t%Γ == 0 then wtgi = 1 N ∑N i w t i
{gti} = ClientUpdate (wtgi,Ci,β)
S̃s = UpdateS̃Matrix ({gti}) {gti} = ClientUpdate (wtgi,Ci,β) {wt+1si } = ServerUpdate ({wtgi}, {gti}, S̃s, α)
Def ClientUpdate(wtgi,Ci,β): wti = w t gi
for e = 0; e < E; e++ do for batch size b in B do
wti ← wti − 2β∇Li(f(wti , xb), yb) return {wtgi − wti}
Def UpdateS̃Matrix ({gi}): for s ∈ S do
perform (3) and (6) to return S̃s(i, j)
Def ServerUpdate ({wtsi},{gti},S̃s,α): for s ∈ S do
perform (7) to get wt+1si
return {wt+1gi }
Definition of personalized FL algorithm In FL, there is a set of N clients C = {Ci}, i = 1, . . . , N . Define the dataset on all clients as D = {Di}, i = 1, . . . , N , with Di as the corresponding dataset of Ci. The sample size of each datasets Di is |Di|. (xj , yj) is a sample of Di, with xj ∈ Rd as the corresponding input feature and yj ∈ Rt as the corresponding input label. The prediction model of
client Ci defined as ŷj = f(wi, xj), where wi ∈ Rz is the model parameter. The dataset on each client Ci is assumed to obey a distribution of Pi. And the loss function on each client Ci is Li(wi). The optimization goal for each client is:
argmin wi
Li(wi) = EDi∼Pi [Li(f(wi, xj), yj)] ≈ 1 |Di| |Di|∑ j Li(f(wi, xj), yj). (1)

heading: 3.1 SUBCLASS PERSONALIZED FL (SPFL) ALGORITHM
text: We compared the performance of each client’s independent training (i.e., the model in each client performs SGD only on local data available, and model averaging is not performed ) and conventional global FL algorithms, such as FedAvg. The results in Appendix B. show that FL usually outperforms each client’s independent training on IID data, demonstrating the global model trained with multiple client data improves the model generalizability ability of each client. On the other hand, for some clients, independent training has better performance on non-IID data. Our experiment in Appendix B illustrates that the global FL model cannot adapt well to the heterogeneous data. To tackle this problem, we propose a personalized FL algorithm that improved the performance of conventional global FL algorithms on non-IID data. At the same time, it still maintains a high performance on IID data.
Firstly, we introduce a similarity matrix S ∈ RN×N to model the relationship between different clients, with S(i, j) represents the similarity between Ci and Cj . If the data distributions Pi and Pj are similar, then the value of S(i, j) is relatively high, and vice versa.
For any two clients, Ci and Cj , supposing they receive the global model parameters wt at the same time t. After the local update of both clients using wt, their local model parameters are wti and w t j , respectively. Then the SGD updates of the corresponding gradients of wti and w t j are:
gti = w t − wti , gtj = w t − wtj .
(2)
The correlation between Ci and Cj based on the global model wt is measured by the cosine similarity of the gradient updates gti and g t j , as calculated in (3):
S(i, j) = gti T gtj
‖gti‖‖gtj‖ , (3)
where ‖ · ‖ is the vector L2 normal. And the aggregation strategy on the server for Ci is as follows:
wt+1i = w t i − α N∑ j |Dj | |D| S(i, j)g t j , (4)
where α is the learning rate of the server. As the gradient itself is extremely directionality sensitive, when S(i, j) is negative, multiplying it with the original gradient gtj will reverse the gradient direction. Therefore, softmax function is applied to make S(i, j) nonnegative and normalized as S̃(i, j) . S̃(i, j), the softmax version of similarity relationship between Ci and Cj , is defined as:
S̃(i, j) = eS(i,j)∑N j e S(i,j) . (5)
The experimental results in Appendix B show that different layers influence personalization differently in the convolutional neural network (CNN). Inspired by ResNet, we use stage strategy that contains a set of network layers to measure the similarity between clients. Therefore, the similarity matrix S̃ is calculated as S̃s based on a stage as in (6), where its subscript s indicates a stage:
S̃s(i, j) = eSs(i,j)∑N j e Ss(i,j) . (6)
Finally, the aggregation strategy on the server for Ci base on stage is defined as:
wt+1si = w t si − α N∑ j |Dj | |D| S̃s(i, j)g t j , (7)
where α is the learning rate of the server, gtsj and w t si are the updated gradient and model parameters of the stage s on the client i at iteration round t, respectively.
As the model changes with iteration round t, so does the gradient of the model. Consequently, the similarity matrix S̃ obtained based on the gradient will also change. Therefore, the similarity matrix is updated regularly during the training process. During each update, the model parameters of all clients are aggregated with equal weight to get wt of (2), and the similarity matrix is calculated based on wt. In the view of each aggregation, it is still similar to the federated average algorithm. However, each client obtains a personalized model based on its similarity matrix in the training rounds in each aggregation. In this process, each client is individually regarded as a subclass, and the algorithm is called Subclass Personalized Federated Algorithm (SPFL). The algorithm is shown in Alg. 1.

heading: 3.2 PERSONALIZED LEAP GRADIENT APPROXIMATION (PLGA) ALGORITHM
text: This section introduces an asynchronous updating strategy that addresses the system heterogeneity problem mentioned in Section . The server can perform model parameters aggregation immediately after receiving few local models from clients within a small time window, overcoming the inefficiency of the conventional synchronous updating strategy, which either updates the model parameters after all the clients uploads their local model or discards the straggers.
Algorithm 2 Personalized Leap Gradient Approximation (PLGA) Algorithm Input: Initialized global model parameters w0; local step E; local batch size B of each client; client learning rate β; server iteration round T ; client number N Output: Model parameters {wtpi} and {wt} Def MainLoop ({wti}, wtk):
for t = 0; t < T ; t++ do ClientUpdate(wt,Ck,β) The server receives multiple local models {wti}N ′
i=0 at the same time or within a small time window for the straggers wtk, do (8), (13), (11), (14) in sequence; the server aggregates all clinets’ model:wt+1 = 1N ′ ∑N ′ i=0 w t i
Def ClientUpdate (wt,Ck,β): wt−1k = w t
for e = 0; e < E; e++ do for b ∈ B do
wtk ← wt−1k − β∇Lk(f(wt−1k , xb), yb)
return wtk
Definition of asynchronous FL Assuming that the FL process consists of T global iterations that the number of the server aggregation. During each global iteration s, after receiving a global model parameters wts , 0 ≤ s ≤ T , from the server, the client Ci uses the local data Di for a local training of E epochs. After finishing the local training, each client uploads its local model wtsi to the server for aggregation. Then the next global iteration is performed. On the other hand, as shown in Fig. 1 for the conventional asynchronous FL process, the model parameters wt0k and w tτ i uploaded by different clients Ck and Ci, respectively, have equal importance in the server aggregation round τ + 1, ignoring the fact that model wtτi performs several more SGD (local updates) than model w t0 k that is called straggers. Intuitively, as the server iteration round increase,the gap between server and stragglers get bigger, the adverse impact on server aggregation increases.
Leap Gradient Approximation (LGA) Here, we let the server aggregation round s= 0 to simplify introduction. First, the client Ck receives the global model wt0 delivered by the server at the global iteration round 0 for initialization, and the wt0k is the local model of client Ck that is trained by local data Dk. wt1 is the global model obtained right after the server performing an aggregation at iteration round 1. So the gradient similarity S(k) of the server and the client Ck can be calculated as follows:
S(k) = (wt1 − wt0)T (wt0k − wt0 )
‖wt1 − wt0‖ ∥∥wt0k − wt0 ∥∥ . (8)
We also apply the softmax function to normalize S(k).
S̃(k) = eS(k)
e1 + eS(k) . (9)
As shown in as in Fig. 1, wt0k is directly used to compute the global model w tτ+1 in conventional asynchronous FL. To avoid this, assuming that the training loss function is L1-Lipschitz, inspired by DC-ASGD (Zheng et al., 2017), to alleviate the update gap between stragglers and the server, we use the approximation of the first-order Taylor expansion of the gradient to estimate the model of straggler Ck in the current server iteration round τ by (10):
wtτk = w t0 + S̃(k) ( wt0k − wt0 ) (wt0k − wt0 ) (wtτ − wt1)+ (wt0k − wt0 ) , (10)
where wtτk is the local model of straggler client Ck at the server aggregation round τ , and is the element-wise product. The last two terms on the RHS of (10) is the prediction of the changes of wt0k , which we called as gradient leap; especially, Diag (( wt0k − wt0 ) (wt0k − wt0 ))
is the approximation of the gradient of the gradient of wt0k . The first term on the RHS is the initial value of wt0k . Furthermore, we also proposes another view on calculating w tτ k as in (11)
wtτk = w tτ + S̃(k) ( wt0k − wt0 ) (wt0k − wt0 ) (wtτ − wt1)+ (wt0k − wt0 ) , (11)
where wtτ is the global model at the server aggregation round τ . The only difference between (10) and (11) is their first terms on their RHS. In (11), it says that we want to predict wtτk based on current global model wtτ , yet still use the prediction of the changes of wt0k in the previous FL iteration round.
Personalized Leap Gradient Approximation (PLGA) The personalized model for each client Ck in the server aggregation round τ is as follows:
wtτkp = w t0 +
( 1− S̃(k) ) ( wtτk − wt0 ) + S̃(k) ( wtτ − wt0) . (12)
where S̃ and wtτk are derived from (9) and (10) or (11), respectively. Similar to (6), we can also apply stage strategy to S̃(i), and obtain:
S̃s(k) = eSs(k)
e+ eSs(k) , (13)
and wtτskp = w t0 s + ( 1− S̃s(k) ) ( wtτsk − wt0s ) + S̃s(k) ( wtτs − wt0s ) . (14)
The personalized leap gradient approximation (PLGA) algorithm is introduced in Alg. 2.

heading: 4 EXPERIMENTS
text: In this section, we first evaluate the performances of SPFL compared with FedAvg(McMahan et al., 2017), Per-FedAvg(Fallah et al., 2020), FedUpdate(Jiang et al., 2019a) and pFedMe(Dinh et al., 2020) when the data distributions are IID and non-IID, we also explore the effect of hyperparameters k,N,E, T . We then compare PLGA with FedAvg (both synchronous and asynchronous settings), Per-FedAvg(Fallah et al., 2020), FedAsync(Xie et al., 2019) when the data distributions are non-IID. The source code is available at https://github.com/MondayCat/PLGA
Experimental Settings For both of SPFL and PLGA experiments, we use four datasets, namely, CIFAR10 (Krizhevsky et al., 2009), CIFAR100 (Krizhevsky et al., 2009), MNIST (Lecun et al., 1998), EMNIST (Cohen et al., 2017) datasets. These datasets are suitable for testing different realworld scenarios in the proposed FL framework, such as non-IID, balanceless datasets. We run the experiments with 10 clients and a server, where each client has access to 6,60,6,32 classes of the above datasets, respectively. The detailed introduction and construction of the datasets can be found in Appendix B. All datasets are split randomly with 80% and 20% for training and testing, respectively. All experiments were conducted using PyTorch (Paszke et al., 2019) version 1.1.0. All the baselines have the same parameters and network structures as our model. We use 10 seeds to get a average result for each method.

heading: 4.1 SPFL EVALUATION
text: In the synchronous FL, we compare the proposed SPFL with the FedAvg, FedUpdate, Per-FedAvg , respectively. Although the FedAvg algorithm is not a personalized learning algorithm, its experimental results can still be helpful as a referances. FedUpdate and Per-FedAvg are FL algorithms dedicated to personalization based on the meta-learning. And pFedMe is using Moreau Envelopes to get a PFL model. The common parameters are fixed as B = 128, α = β = 0.01, Γ = 10, E = 1, S = 2. We provide an extensive ablation study on design parameters’ choices in Appendix C. We also test the personalized learning based on weight fusion, and the aggregation strategy changes from wi = wi + α 1 M ∑ j S̃sgj to wi = 1 M ∑ j S̃swj . The weight-based personalized SPFL algorithm is defined as SPFL-w. Noting that twice local updates, which mimic MAML, are used in SPFL, to explore the influence of the number of local updates on the experiment, we also add two models using once local update on the clients as ablation studies: SPFL-w (1-step) and SPFL (1-step).
As shown in Table 1, the accuracies on the four datasets are quite different, and decreasing in the order as MNIST, EMNIST, CIFAR10, CIFAR100. It also reflects the difficulty of each dataset. On a whole, the handwritten number recognition is relatively simpler than the image classification. And the two datasets within the same category are also different, such as MNIST and EMNIST, or CIFAR10 and CIFAR100. For all methods evaluated, the accuracy on IID dataset is always higher than that of non-IID. Therefore, the evaluation results on the four datasets are relatively objective and generalized, and not overfitting to one dataset.
SPFL performs much better on four non-IID datasets, compared to FedAvg, FedUpdate and PerFedAvg. For IID datasets, Per-FedAvg has the best results except on EMNIST dataset. Nonetheless, the performance gap between SPFL and the best methods is still acceptable on IID datasets. The reason that SPFL performs a little bit worse on IID datasets is due to the delay of the gradient similarity calculation. Considering the actual application scenarios of personalization, in which the possibility of IID data on the client is relatively small, our algorithm can bring more benefit in the real environment.
Compared to FedAvg and independent training of each client in Appendix B, SPFL outperforms both no matter on IID datasets or non-IID datasets. To differentiate the influence of gradient and weight aggregation strategy, we observed that SPFL-w is slightly better in the case of IID data, but SPFL is much better than SPFL-w on non-IID data. Nonetheless, SPFL-w outperforms the FedAvg and the independent training model both on IID and non-IID datasets.
The expermental results also show that the twice local updates dominate its counterpart, which indicate that MAML alike updates is helpful in personalization than once local update.

heading: 4.2 PLGA EVALUATION
text: In the asynchronous FL, the proposed PLPA is compared with the FedAvg (in the synchronous and asynchronous conditions), FedAsync and Per-FedAvg. Although the FedAvg and Per-FedAvg are synchronous federated algorithms, their experimental results can still be helpful as references here. FedAvg(Sync) means that discard the straggers when server aggregate, while FedAvg(Async) means the straggers participate directly in server aggregation. FedAsync is an FL algorithm dedicated to asynchronous FL. The hyperparameters are fixed as B=64, β=0.001, E=20, T =800. Based on ablation experiments, we find that these parameters have best performance. We use N =10 clients to evaluate the performance of the PLGA algorithm. To make the simulation of the asynchronous environment simple and controllable, we fix the set of straggler of each asynchronous FL experiment as S , and the rest clients in set C \S participate in each server iteration round synchronously. For stragglers in S , we let them missing server aggregation round periodically with different fixed missing period as 1, 2, . . . , |S|, where the values correspond to the number of server iteration rounds. Suppose that the fixed missing period of the straggler in S is 2, the straggler will participate one server iteration and miss two server iterations, and then participate one server iteration and miss two server iterations again, and so on. Here, we use non-IID dataset (see Appendix B) for the evaluation as it is the most common data case in asynchronous FL. The experimental results in Table 2, where the total number of the stragglers is |S|= 5, show that the clients’ performances of PLGA on the four non-IID datasets have a notable advantage over the other baseline algorithms. Note that when |S|=0, it is equivalent to synchronous FL. We also notice that PLGA is better than LPA as a result of personalization, and that wtτk computed by (11) is better than by (10) for PFGA.

heading: 5 CONCLUSION AND FUTURE WORK
text: This paper studied the personalized heterogeneous FL with gradient similarity. We tackled the statistical and system heterogeneity in FL in two scenarios, namely synchronous FL and asynchronous FL with SPFL and PLGA algorithms, respectively. In both SPFL and PLGA algorithms, SNGS is proposed to capture the similarity between the participants of FL, ether among clients or between sever and client. Stage concept is also studied in the computing of SNGS. The experiment results demonstrate that the proposed algorithms have excellent performances on non-IID data, while still maintaining the state of the art performance on IID data. We are going to further work on the theoretical proof of the convergence of the proposed algorithms, and also the sampling strategies in the evaluation of SNGS between clients to alleviate the communication and computation overheads.

heading: A APPENDIX

heading: B PREPARATION OF IID AND NON-IID DATASETS
text: Here we choose four public benchmark datasets MNIST, CIFAR10, CIFAR100, and EMNIST, as the original data source to construct the IID and Non-IID datasets. A brief introduction to the four datasets is shown in table 3. It should be noted that all non-IID situations in this paper are feature distribution skewed, that is, while the distribution P (X) of feature X on each client may be different, the distribution of conditional probability P (Y |X) of label Y is the same. Specifically, the sampling steps designed in this paper are as follows: Assuming that the original dataset is D, the total number of classes in the dataset is K, and the dataset of class k is Dk. Supposing the total number of clients participating in personalized FL is N , for client c, the actual number of classses it has is kc, 2 ≤ kc ≤ K. The steps of preparing IID and non-IID datasets are as follows: 1. For any class k, its dataset Dk is randomly and equally partitioned into N parts, where N is the total number of clients. Dataset Dkn denotes the n-th partition portion of D
k. For the generation of IID dataset goto step 2, otherwise goto step 3.
2. For IID dataset, determine the number of classes each client as kc. Sample kc classes from the K classes, and denote the sampled class sets as Ksample, |Ksample| = kc. For each i, i ∈ Ksample, assign each data partition Din, n = 1, . . . , N , to each client without overlapping. Goto step 4.
3. For non-IID dataset, Pi(X), the distribution of feature X , on each client i may not be the same. Here the difference from step 2 is that in non-IID sampling, each client will independently sample a number of kc classes. The rest operations are the same as step 2.
4. Dataset on each client is partitioned randomly into training and testing sets. And two sets are IID for each client.
When the number of clients is N = 4, the total number of classes is K = 3, the total number of classes for each client is kc = 2, the generation of IID and non-IID is shown in Fig. 3 (Note for non-IID dataset generataion, C2 and C4 are IID by chance).

heading: C ANALYSIS OF PERSONALIZED ALGORITHM
text: We chose the CNN model architecture in the following table 4. In the experiment, the accuracy rate
is used as the evaluation index on each client, as shown in the formula (15), where TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative.
acc = TP + TN
TP + TN + FP + FN . (15)
After confirming the experimental parameters settings, we will conduct comparative experiments with the parameters settings in Table 5. The method used in the experiment is the federated average learning algorithm (FedAvg). Table 6 is the corresponding experimental result.
The experimental results show that the results of the FL algorithm are often better than the independent training method when the data is IID. The result of the FL will be worse than that of the independent training when the data is non-IID. This experimental conclusion shows that the federated average learning algorithm (FedAvg), which executes the average aggregation step of the model parameters , is more suitable for the case where the datasets on the client are IID. It has the opposite effect when the data is non-IID. To further explore the influence of the federated averaging algorithm on the model parameters under non-IID, we calculated the average value of each client’s upload gradient similarity in different distribution scenarios. The results are shown in Fig. 4 and Fig. 5. Based on the above experimental results, we can find that: (1) In independent and synchronized steps, the mean of the gradient similarity is close to 1. In a non-IID scenario, the mean of the
gradient similarity on the client differs. (2) In the non-IID, compared with the deeper layer features, the mean of the SNGS will be higher for the shallow layer features.

heading: D ANALYSIS OF HYPERPARAMETERS ON PERSONALIZED EXPERIMENTS
text: We explored the impact of update frequency of the hyperparameters similarity matrix Γ, the number of classes k, the number of clients N , and the different local learning rounds E on the experimental results, which are shown in Table. 7.

heading: E FINE-TUNE EXPERIMENTAL RESULTS
text: When the personalized FL algorithm is completed, the server delivers the final personalized model to each client. Like the meta-learning algorithm, after each client receives a proprietary personalized model, it can still make fine-tuning on itself datasets based on the model to achieve better results. The primary considerations for fine-tuning are the learning rate (β), the training batch size (B), and the number of steps the fine-tuning takes (step).
Table. 8 shows the model based on SPFL training, according to consideration factors, the results of multiple sets of different experimental parameters . First, we compared the accuracy of the model after fine-tuning with the previous model. It can be found that fine-tuning can indeed improve the performance of the model. However, compared with the improvement of SPFL in the federated average algorithm, the benefits of fine-tuning are not very good. It may because that the model after learning has original personalized attributes.

Review summaries:
This paper proposed a personalized federated learning algorithm which takes into account the similarity of gradient of different users to update the model. More formally, the authors define $\tilde{S}(i,j)$ as a measure of similarity between the gradients of two user $i$ and $j$, and then update the model of user $i$by weighting the gradient of user $j$ by $\tilde{S}(i,j)$. The authors study their method in various numerical settings. 

This paper proposes two methods for personalized federated learning, one synchronous and one asynchronous. The general approach taken in both cases is to adapt the weights when averaging information from different clients, so that clients with more similar gradients are given more weight in the update for each client. The two approaches are called SPFL (synchronous) and PLGA (asynchronous). The approaches are illustrated on small image classification tasks.