Question: The global iteration is considered a discrete-time Markov chain. However, the transition matrix Qt is not a static matrix. How does the Markov chain handle this dynamic transition matrix? Has these dynamics been analysed in any part of the paper?
Evidence: We design an effective alternating optimization framework to minimize the overall objective in equation 2. Specifically, the upper-level problem can be decomposed into N separate sub-problems with fixed generalized models and to be solved on local devices in parallel. Next, we can further settle the inner-level problem to derive the generalized models with fixed personalized models. Since the solution to the sub-problems of the upper-level objective has been well-explored in recent PFL methods, we hereby mainly focus on the inner-level problem. We alternately update the generalized models Ω K and the cluster indicator C K to obtain the optimal generalized models. We view the personalized models, i.e., Θ I = [θ i , ..., θ N ], as private data, and distributionally update the generalized models Ω K on clients with fixed cluster indicator C K . During each server round, the server conducts k-Means clustering on uploaded local parameters Ω t I,R to cluster each client into K disjoint groups, and the clustering results C K are rearranged to the matrix form as
and empty set, respectively), the element (P t ) i,j is defined as 1 |Cj | , or set 0 otherwise. In this way, the elements of every column in P t amount to 1, i.e. N i=1 (P t ) i,j = 1, ∀j, t. When considering the relationship between the consecutive P t , we can formulate the iterate as P t+1 = P t Q t , where Q t ∈ R K×K is a square matrix. We can find that to maintain the above property of P t (∀t), the matrix Q t must satisfies that:
It is noticed that the clustering is based on the latest model parameters Ω t+1 I that depends on Ω t I , and the latest gradient updates given by clients. Hence, P t+1 is determined by and only by P t and Algorithm 1 CGPFL: Personalized Federated Learning with Clustered Generalization
Server sends Ω t K to clients according to P t . 3:
for local device i = 1 to N in parallel do 4:
Initialization:
Local update for the sub-problem of G(ΘI , ΩK ): 6:
for r = 0 to R − 1 do 7:
for s = 0 to S − 1 do 8:
Update the personalized model:
end for 10:
Local update:
end for 12:
end for 13:
Clients send back ω t i,R and server conducts (k-means++) clustering on models Ω t I,R to obtain P t+1 . 14:
Global aggregation:
). 15: end for 16: return The personalized models Θ T I .
Q t . Then we can consider this global iteration as a discrete-time Markov chain and Q t corresponds the transition probability matrix.
During each local round, the clients need to first utilize local datasets to solve the regularized optimization objective, i.e., the upper-level objective in equation 2 with fixed ω t i,r
to obtain a δ-approximate solution θi (ω t i,r ). Then, each client is required to calculate the gradients ∇ ωi G( θi (ω t i,r ), ω t i,r ) with fixed θi (ω t i,r ) and update the model using
) , where β is the learning rate and
), ω t i,r . To reduce the communication overhead, our CGPFL allows the clients to process several iterations before uploading the latest model parameters to the server. The details of CGPFL is given in algorithm 1, from which we can summarize the parameters update process as:
where P t+1 = P t Q t and J t P t = I K (J t ∈ R K×N and I K is an identity matrix), ∀t.
Then we can consider this global iteration as a discrete-time Markov chain and Qt corresponds the transition probability matrix
Answer: Sorry that we don't analyze this dynamic transition matrix in this paper. We just notice that the global iteration is a discrete-time Markov chain, but we don't solve the optimization problem from the perspective of Markov chain. We believe that this fact may lead to some other approaches to the personalized federated learning framework which can be a future direction.