Paper: Enhancing Personal Decentralized Federated Learning through Model Decoupling (ICLR-2024)

Reviewer comment (Theory):
The reviewer is quite doubt about the final results as shown in Theorem 1 and Theorem 2. I’ve checked the theoretical proof in the appendix and do not find the exact expressions for the final convergence results, but only the $\mathcal{O}$ expression. The first questionable part is that the right-hand side of Eqs. (3)-(4) will goes to 0 as the number of rounds $T$ goes to infinity, while in reality, this is not true for non-i.i.d scenarios. There will exists some constant terms related with heterogeneity that are irrelevant to $T$. Please explain this.

Author response:
## W1: The doubts about Theorem 1 and Theorem 2.
- __Only the $\mathcal{O}$ expression.__ $\mathcal{O}$ is the final results after assuming the learning rates $\eta_u$ and $\eta_v$. Similar results can be found in [1-3]. Due to the limited space, the initial convergence analysis has been presented in Formula (19) and Formula (33). 
- __The right-hand side of Eqs. (3)-(4) will goes to 0 as the number of rounds $T$ goes to infinity.__ 
    - The gradient that goes to 0 is one of the optimality conditions in non-convex optimization.  As an optimization problem, we consider minimizing problem: $\min _{x \in \mathbb{R}^d} f(x)$ and find the global minima $x^*$, that satisfies $f(x) \geq f(x^*)$. In non-convex optimization, the minima can be defined by  $\nabla f\left(x^{\star}\right)=0$.  So we prove the convergence from the gradients going to 0, which satisfies $\| \nabla f(x^T)\| \leq \epsilon$. More details about non-convex optimization can be referred to [4,5].
    - We assume that $\eta_u= \mathcal{O}(1/L_uK_u\sqrt{T})$ and $\eta_v= \mathcal{O}(1/L_vK_v\sqrt{T})$, which means that the learning rate $\eta_u$ and $\eta_v$ tend to 0 as the number of rounds $T$ goes to infinity, and the gradients eventually tend to 0 correspondently. We deduce that the reviewer wants to say that the proposed methods will converge to a variance under the constant learning rate. Actually, our methods converge to a variance domain with the sublinear order convergence speed.
    -  Moreover, in a non-iid scenario, each client will achieve convergence and their gradient will approximate to 0 after sufficient large communication rounds. However, this does not mean that all clients have the same solution. For PFL, every client will own its unique solution $(u^*,v_i^*)$, the $v_i^*$ of which is different from each other.

Reviewer comment (Theory):
The second part that may not be true in the theoretical results is that the convergence speed is monotonically related with the spectral gap $\lambda$. If this is true, it solves the challenging topology design problem of decentralized federated learning, since a fully-connected topology is the optimal topology according to the theoretical results in this paper. There is no discussion about this point in current manuscript and this leads to a doubtful result.

Author response:
## W2: The convergence speed is monotonically related to the spectral gap.
- __The convergence speed is monotonically related to the spectral gap.__ The convergence speed is related to the statistical heterogeneity $\delta$, the smoothness $L_u$, $L_v$, $L_{vu}$ of loss functions, the local epochs $K_u$ and the communication topology (1 − $\lambda$) in the theoretical results. Due to the limited space, we only present the final results in Formula (3) and Formula (4). Please check the details in Formula (20-21) and Formula (34-36). 
- __Whether Fully-connected topology is the optimal topology according to the theoretical results.__ 
    - Theoretically, Fully-connected topology is the optimal topology among the comparison topologies and the comparison of the spectral gap of each topology is as follows ($m$ refers to the partition number of clients):
        Graph Topology | Spectral Gap $1-\lambda$ 
        --- | :--:|
        Fully-connected|1
        Disconnected|0
        Ring| $\approx 16\pi^2/3m^2$ 
        Grid| $\mathcal{O}(1/(mlog_2(m)))$
        Exponential|$2/(1 + log_2(m))$
    
        It is clear to see that the convergence bounds of different topologies are ranked as follows: Fully-connected > Exponential > Grid > Ring. __But it contains more communication cost compared with other topologies.__ So it is a trade-off between the convergence and communication cost in the real world. We have added the comparison to the revision.
    - Empirically, we conduct experiments on different topologies (i.e. Ring, Grid, Exponential and Fully-connected）in Fig 3, which suggests the personalized performance of different topologies is ranked as follows: Fully-connected > Exponential > Grid > Ring.

Reviewer comment (Evaluation):
Why is the convergence results not related with the number of workers? This is also a weird part.

Author response:
## W3: The convergence results are not related to the number of workers.
Refer to the table above, the effects of the participation workers $m$ have been involved in the spectral gap $(1-\lambda)$ of communication topologies. We have added it to the revision.

Reviewer comment (Theory):
Why Theorem 1 is related with the cross Lipschitz constant $L_{vu}$, and Theorem 2 is related with $L_{vu}$? How about $L_{uv}$?

Author response:
## W4: Theorem 1 and Theorem 2 are all related to the cross Lipschitz constant $L_{vu}$.
- First, based on the Lipschitz Smooth assumption, we define $\nabla_u F_i(u_i,v_i)$ is $L_u$--Lipschitz with respect to $u_i$ and $L_{uv}$--Lipschitz with respect to $v_i$ in Assumption 1, which means $\|\nabla_u F_i(u_i,V)-\nabla_u F_i(u_j,V)\| \leq L_u \|u_i - u_j\|$ and $\|\nabla_u F_i(u,v_i)-\nabla_u F_i(u,v_j) \| \leq L_{uv} \|v_i-v_j\|$.
    The same for $\nabla_v F_i(u_i,v_i)$ is $L_v$--Lipschitz with respect to $v_i$ and $L_{vu}$--Lipschitz with respect to $u_i$. More details about Lipschitz Smooth assumption can be found in [6].

 - Then, we conduct  
$\frac{1}{m}\sum_{i=1}^{m} E \big< \nabla_v F_i(\bar{u}^{t}, v_{i}^{t}) - \nabla_{v} F_{i} \left( u_{i}^{t}, v_{i}^{t} \right), v_{i}^{t+1} - v_{i}^{t} \big> \leq \frac{1}{2m} \sum_{i=1}^{m} \||\nabla_{v} F_{i} \left(\bar{u}^{t}, v_{i}^{t}\right)  - \nabla_{v} F_{i}\left(u_{i}^{t}, v_{i}^{t}\right)\||^2 +  \frac{1}{2m}\sum_{i=1}^m E \||v_{i}^{t+1} - v_{i}^{t}\||^2  \leq \frac{L_{vu}^2}{2m}\sum_{i=1}^m E \||\bar{u}^t-u_{i}^{t}\||^2 + \frac{1}{2m}\sum_{i=1}^m E \||v_{i}^{t+1} - v_{i}^{t}\||^2.$  
in the second line in Formula (15).
        
 - The theoretical analysis progress is consistence with the algorithm, which first updates the personalized parts $v_i$ and then the shared parts $u_i$. So Theorem 2 and Theorem 1 are associated with $L_{vu}$ not $L_{uv}$.

Reviewer comment (Experiments):
The results in Fig.3 are questionable according to the second comment. The reviewer is not sure if a fully-connected topology is the best.

Author response:
## W5: Is a fully-connected topology is the best?
Fully-connected topology achieves the best performance in the comparisons under different topologies. But it needs more communication costs for each client. So it is a trade-off of the test performance and communication cost to choose the topology.

Reviewer comment (Experiments):
In terms of experimental baselines, it is recommended that the authors include the most recent decentralized federated learning method ([2]) for a comprehensive comparison. This will enhance the paper's completeness and relevance in the context of the current state of the field.

Author response:
## W2: Add the most recent decentralized federated learning method for a comprehensive comparison.
Thanks for your suggestion. We provide a comparison with DFedSAM in the following and we have added this comparison in the revision. 

| **Dataset**   |    | **Cifar-10** |  |   |    | **Cifar-100** | |   |    
|:-------------:|:-------|:------------:|:-----:|:-----:|:-------:|:-------------:|:-----|:------|  
| **Algorithm** | Dir-0.1 | Dir-0.3      | Pat-2 | Pat-5 | Dir-0.1 | Dir-0.3       | Pat-5 | Pat-10 |        
| **DFedSAM**|84.96| 77.36|90.14|83.05|58.21|47.80|74.25|67.34|
| **DFedAvgM**  | 87.39   | 82.60        | 90.72 | 84.69 | 59.76   | 54.98         | 76.70 | 71.08  | 
| **DFedMDC**| 88.85   | 86.50        | 91.26 | 86.85 | 66.26   | 57.66         | 78.78 | 72.19  |         
| **DFedSMDC**  | 91.08 | 87.67|92.20|88.34|67.03|58.73|80.82|74.50|

From the comparison between DFedAvgM and DFedSAM we can see that adding the gradient perturbation to the full model will increase the model consistency but hurt the personalized performance. On the contrary, adding the perturbation to the shared representation parts in DFedSMDC outperforms DFedMDC clearly, which shows that both improving the shared representation parts consistency and keeping the private classifiers locally are both the key points to facilitate the personalized performance in decentralized PFL.

Reviewer comment (Theory):
Regarding the convergence analysis, it would be valuable to incorporate a discussion that compares the proposed method's convergence rate with the state-of-the-art (SOTA) approaches.

Author response:
## W3: compares the proposed method's convergence rate with the state-of-the-art (SOTA) approaches.
Compared with the SOTA bounds $\mathcal{O}\Big(\frac{1}{\sqrt{T}}+\frac{\sigma_l^2 + K \sigma_g^2}{K \sqrt{T}}+\frac{\sigma_l^2 + K\sigma_g^2+ KB^2}{K(1-\lambda)^2T^{3/2}}\Big)$  ( is the upper bound of the gradient) of existing work DFedAvg and $\mathcal{O}\Big(\frac{1}{\sqrt{KT}}+\frac{K(\sigma_g^2+\sigma_l^2)}{T}+\frac{\sigma_g^2+\sigma_l^2}{K^{1/2}(1-\lambda)^2T^{3/2}}\Big)$ of DFedSAM in decentralized works, our algorithms reflect the impact of the L-smoothness and the gradient variance of the shared model $u$ and personalized model $v$ on convergence rate. On the other hand, compared with the SOTA bound of FedAlt in personalized works, our algorithms reflect the impact of the communication topology $\lambda$ (the value of  that increases when connectivity is more sparse, which has been studied by existing work [1].) 

Therefore, the main contribution of our convergence analysis is the first to analyze the impact of model decoupling and the decentralized networks on the convergence rate instead of delivering the SOTA bound compared with others. Meanwhile, this theoretical analysis can be mutually verified with experimental results.

[1] Topology-aware Generalization of Decentralized SGD. ICML2023.

Reviewer comment (Theory):
Why introduce the SAM? It is unclear about the advantage of introducing this optimizer. Can you elaborate on it intuitively and theoretically?

Author response:
## W2: Why introduce the SAM? 
- Intuitively, SAM adds proper perturbation in the gradient direction to make the shared part become more robust and flatter, which will benefit the model average between clients.
- Theoretically, we give the effect on bound after adding the operation of SAM in Remark 2, where assuming $\rho = \mathcal{O}(\frac{1}{\sqrt{T}})$. Due to the limited space, we have  presented the bound before assuming $\rho = \mathcal{O}(\frac{1}{\sqrt{T}})$ in Formula (34) in Appendix C.2.2 on Page 26 as follows:
    - before using $\rho = \mathcal{O}(\frac{1}{\sqrt{T}})$:
    $\begin{aligned} &\frac{1}{T} \sum_{i=1}^T\left(\frac{1}{L_u} \mathbb{E}\left[\Delta_{\bar{u}}^t\right]+\frac{1}{L_v} \mathbb{E}\left[\Delta_v^t\right]\right)  \leq \mathcal{O}\left(\frac{F\left(\bar{u}^1, V^1\right)-F^*}{\sqrt{T}}+\frac{\sigma_v^2\left(L_v+1\right)}{L_v^2 \sqrt{T}}+\frac{L_u^2 \rho^2+\sigma_u^2+\delta^2}{L_u K_u \sqrt{T}}\right.  \left.+\frac{L_{v u}^2}{(1-\lambda)^2 \sqrt{T}}\left(\frac{\rho^2}{K_u}+\frac{\sigma_u^2+\delta^2}{L_u^2}\right)+\frac{L_u}{(1-\lambda)^2 T}\left(\frac{\rho^2}{K_u}+\frac{\sigma_u^2+\delta^2}{L_u^2}\right)\right) .\end{aligned}$
    - after using $\rho = \mathcal{O}(\frac{1}{\sqrt{T}})$:
    $\begin{aligned} &\frac{1}{T} \sum_{i=1}^T\left(\frac{1}{L_u} \mathbb{E}\left[\Delta_{\bar{u}}^t\right]+\frac{1}{L_v} \mathbb{E}\left[\Delta_v^t\right]\right)  \leq \mathcal{O}\left(\frac{F\left(\bar{u}^1, V^1\right)-F^*}{\sqrt{T}}
    +\frac{\sigma_v^2\left(L_v+1\right)}{L_v^2 \sqrt{T}}
    +\frac{\sigma_u^2+\delta^2}{L_u K_u \sqrt{T}}\right. \left.+\frac{L_{v u}^2(\sigma_u^2+\delta^2)}{L_u^2(1-\lambda)^2 \sqrt{T}}
    +\frac{\sigma_u^2+\delta^2}{L_u(1-\lambda)^2 T}+ \frac{L_u}{K_u\sqrt{T^3}}+\frac{L_{vu}^2}{K_u(1-\lambda)^2\sqrt{T^3}}+\frac{L_u}{K_u(1-\lambda)^2T^2}\right) .\end{aligned}$
- Empirically, the comparison between DFedSMDC and DFedMDC in different heterogeneity and different topologies can significantly demonstrate the effectiveness in enhancing the consistency of the shared models in decentralized PFL.

Reviewer comment (Theory):
In the theorem, why is it \( V^{t+1} \), instead of \( V^{t} \), and what does it mean?

Author response:
## W3: $V^{t}$ not $V^{t+1}$ in the theorem.
Thanks for your suggestion and we have polished it in the revision paper.

Reviewer comment (Evaluation):
Regarding "The test performance will get a great margin with the participation of clients decreasing": What will happen when the client number is less than 10, even 1? Does it mean no collaboration is the best?

Author response:
## W5:  "The test performance will get a great margin with the participation of clients decreasing.’’
We have updated more experiments when the client number is 10 and 5 in Fig 5 in the revision paper. The test performance still improves with the participation of clients decreasing. And we clarify our opinion theoretically and empirically as follows:
- Theoretically, the convergence bound is related to the topology, which is associated with the participation of clients. The relationship between the topology and the participation clients $m$ is as below:
    Graph Topology | Spectral Gap 1-$\lambda$ 
    --- | :--:|
    Fully-connected|1
    Disconnected|0
    Ring| $\approx 16\pi^2/3m^2$ 
    Grid| $\mathcal{O}(1/(mlog_2(m)))$
    Exponential Graph|$2/(1 + log_2(m))$

    When the participation clients $m$ decrease, the spectral gap increases and the convergence bound is tighter.
- Empirically, with the participation of clients decreasing, the number of local data increases. More data will help to achieve a better performance. 
- When the client number is 1, the problem will become a classical ML problem based on a centralized data assumption, not an FL problem based on a decentralized data setting. We aim to collaboratively train the personalized models without transmitting data.

[1]On bridging generic and personalized federated learning for image classification. ICLR2022.

[2]Fedbabu: Towards enhanced representation for federated image classification. ICLR2022.

[3]Personalized federated learning with feature alignment and classifier collaboration. ICLR2023.

[4]FedProto: federated prototype learning across heterogeneous clients AAAI2022.

Reviewer comment (Theory):
Could you give more explanation on Theorem 2? What is the difference/advantage compared with Theorem 1 as you introduce SAM into shared parameters?

Author response:
## Q1: What is the difference/advantage compared with Theorem 1 as you introduce SAM into shared parameters?
>Could you give more explanation on Theorem 2? What is the difference/advantage compared with Theorem 1 as you introduce SAM into shared parameters?
- The complete final bound in Theorem 1 is $\frac{1}{T}\sum_{i=1}^T \bigl(\frac{1}{L_u} E \bigl[\Delta_{\bar{u}}^t \bigr] + \frac{1}{L_v} E [\Delta_{v}^t \bigr] \bigr)  \leq \mathcal{O}\Big(\frac{F(\bar{u}^1, V^1) - F^*}{\sqrt{T}} + \frac{\sigma_v^2(L_v+1)}{L_v^2\sqrt{T}} + \frac{(L_{vu}^2+1)(\sigma_u^2+\delta^2)}{L_u^2 (1-\lambda)^2\sqrt{T}}+\frac{\sigma_u^2+\delta^2}{K_u L_u \sqrt{T}}+  \frac{\sigma_u^2+\delta^2}{K_u L_u(1-\lambda)^2 T} \Big)$ in Formula (20). And complete final bound in Theorem 2 is $\frac{1}{T}\sum_{i=1}^T \bigl(\frac{1}{L_u} E \bigl[\Delta_{\bar{u}}^t \bigr] + \frac{1}{L_v} E [\Delta_{v}^t \bigr] \bigr) 
        \leq \mathcal{O}\Big(\frac{F(\bar{u}^1, V^1) - F^*}{\sqrt{T}}  + \frac{\sigma_v^2(L_v+1)}{L_v^2\sqrt{T}} +   \frac{L^2_{vu}}{(1-\lambda)^2 \sqrt{T}}\Big( \frac{\rho^2}{K_u} + \frac{\sigma_u^2+\delta^2}{L_u^2} \Big) + \frac{L_u^2 \rho^2 + \sigma_u^2+\delta^2}{L_u K_u \sqrt{T}} + \frac{L_u}{(1\!-\!\lambda)^2 T}\Big(\frac{\rho^2}{K_u} + \frac{\sigma_u^2+\delta^2}{L_u^2}\Big) \Big)$ in Formular (34). SAM introduce the gradient perturbation $\rho$ in Theorem 2. 
- From the comparison, the dominant terms are the same in Theorem 1 and Theorem 2, which means SAM doesn't improve the convergence bound in theoretical analysis. However, it makes the shared models more robust and flatter, which will benefit the average process for each client.

Reviewer comment (Experiments):
Can you provide baseline results with more hyperparameter settings?

Author response:
## Q2: Provide baseline results with more hyperparameter settings.
Please check the reply in M1.