SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Assumption 6 For  $\forall w_{1},w_{2}\in \mathbb{R}^{d}$ , the non-convex local function  $f_{i}$  satisfies  $L$ -smooth if:
$$
\left\| \nabla f _ {i} \left(w _ {1}\right) - \nabla f _ {i} \left(w _ {2}\right) \right\| \leq L \| w _ {1} - w _ {2} \|, \tag {19}
$$
where  $L$  is a universal constant.
Assumption 7 For  $\forall w\in \mathbb{R}^d$  , the stochastic gradient is bounded by its expectation and variance as:
$$
\mathbb {E} \left[ g _ {i, k} ^ {t} \right] = \nabla f _ {i} \left(w _ {i, k} ^ {t}\right),
$$
$$
\mathbb {E} \| g _ {i, k} ^ {t} - \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} \leq \sigma_ {l} ^ {2}, \tag {20}
$$
where  $\sigma_l > 0$  is a universal constant.
Assumption 8 For  $\forall w\in \mathbb{R}^d$  , the heterogeneous similarity is bounded on the gradient norm as:
$$
\frac {1}{C} \sum_ {i \in \mathcal {C}} \| \nabla f _ {i} (w) \| ^ {2} \leq G ^ {2} + B ^ {2} \| \nabla f (w) \| ^ {2}, \tag {21}
$$
where  $G \geq 0$  and  $B \geq 1$  are two universal constants.
Assumption 9 For  $\forall w_{1},w_{2}\in \mathbb{R}^{d}$  , the global function  $f$  satisfies  $L_{G}$  -Lipschitz if:
$$
\left\| f \left(w _ {1}\right) - f \left(w _ {2}\right) \right\| \leq L _ {G} \| w _ {1} - w _ {2} \|, \tag {22}
$$
where  $L_{G}$  is a universal constant.
Assumption 10 For  $\forall w\in \mathbb{R}^d$  , let  $w^{\star}\in \arg \min_{w}f(w)$  , the global function satisfies PL-condition if:
$$
2 \mu (f (w) - f (w ^ {\star})) \leq \| \nabla f (w) \| ^ {2}, \tag {23}
$$
where  $\mu$  is a universal positive constant.
Discussion. Assumption 6~8 are three general assumptions to analyze the non-convex objective in FL, which is widely used in the previous works [15, 18, 19, 30, 36, 44, 46, 48]. Assumption 9 is used to bound the uniform stability for the non-convex objective, which is used in [11, 51]. Different from the analysis in the margin-based generalization bound [27, 29, 31, 38] that focus on understanding how the designed objective affects the final generalization performance, our work focuses on understanding how the generalization performance changes in the training process. We consider the entire training process and adopt uniform stability to measure the global generality in FL and theoretically study the importance of consistency to FL. For the general non-convex objective, one often uses the gradient norm  $\mathbb{E}\| \nabla f(w)\|^2$  instead of the loss difference  $\mathbb{E}[f(w^{\star}) - f(w)]$  to measure the training error. To construct and analyze the excess risk to further understand how the consistency affects the FL paradigm, we follow [51] to use Assumption 10 to bound the loss distance. Through this, we can establish a theoretical framework to jointly analyze the trade-off on the optimization and generalization in the FL paradigm.
A.2 Proofs for the Optimization Error
In this part, we prove the training error for our proposed method. We assume the objective function  $f(w) = \frac{1}{C}\sum_{i\in C}f_i(w)$  is  $L$ -smooth w.r.t  $w$ . Then we could upper bound the training error in the FL. Some useful notations in the proof are introduced in the Table 4.
Then we introduce some important lemmas used in the proof.
Table 4: Some abbreviations of the used terms in the proof of bounded training error.
Notation	Formulation	Description
wi,k	-	parameters at k-th iteration in round t on client i
wt	-	global parameters in round t
V1t	1/C ∑i∈C ∑k=0K-1E||wi,k-t-wt||2	averaged norm of the local updates in round t
V2t	E||w+t+1-wt||2	norm of the global updates in round t
Δt	1/C ∑i∈C E||wi,K-t-1-wt||2	inconsistency/divergence term in round t
D	f(w0)-f(w*)	bias between the initialization state and optimal
A.2.1 Important Lemmas
Lemma 2 (Bounded local updates) We first bound the local training updates in the local training. Under the Assumptions stated, the averaged norm of the local updates of total  $C$  clients could be bounded as:
$$
V _ {1} ^ {t} \leq 4 K \beta^ {2} \Delta^ {t} + 3 K ^ {2} \eta^ {2} \left(\sigma_ {l} ^ {2} + 4 K G ^ {2}\right) + 1 2 K ^ {3} \eta^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2}. \tag {24}
$$
Proof  $V_{1}$  measures the norm of the local offset during the local training stage. It could be bounded by two major steps. Firstly, we bound the separated term on the single client  $i$  at iteration  $k$  as:
$$
\begin{array}{l} \mathbb {E} _ {t} \| w ^ {t} - u _ {i, k} ^ {t} \| ^ {2} \\ = \mathbb {E} _ {t} \left\| w ^ {t} - w _ {i, k - 1} ^ {t} + \eta \left(g _ {i, k - 1} ^ {t} - \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right) + \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right) - \nabla f _ {i} \left(w ^ {t}\right) + \nabla f _ {i} \left(w ^ {t}\right)\right) \right\| ^ {2} \\ \leq \left(1 + \frac {1}{2 K - 1}\right) \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} + \eta \left(g _ {i, k - 1} ^ {t} - \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right)\right) \| ^ {2} \\ + 2 K \eta^ {2} \mathbb {E} _ {t} \| \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right) - \nabla f _ {i} \left(w ^ {t}\right) + \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} \\ \leq \left(1 + \frac {1}{2 K - 1}\right) \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} \| ^ {2} + \eta^ {2} \mathbb {E} _ {t} \| g _ {i, k - 1} ^ {t} - \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right) \| ^ {2} \\ + 4 K \eta^ {2} \mathbb {E} _ {t} \| \nabla f _ {i} \left(w _ {i, k - 1} ^ {t}\right) - \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} + 4 K \eta^ {2} \| \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} \\ \leq \left(1 + \frac {1}{2 K - 1} + 4 \eta^ {2} K L ^ {2}\right) \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} \| ^ {2} + \eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} \| \nabla f _ {i} (w ^ {t}) \| ^ {2} \\ \leq \left(1 + \frac {1}{K - 1}\right) \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} \| ^ {2} + \eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} \| \nabla f _ {i} (w ^ {t}) \| ^ {2}, \\ \end{array}
$$
where the learning rate is required  $\eta \leq \frac{\sqrt{2}}{4(K - 1)L}$  for  $K\geq 2$
Computing the average of the separated term on client  $i$ , we have:
$$
\begin{array}{l} \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k} ^ {t} \| ^ {2} \\ \leq \left(1 + \frac {1}{K - 1}\right) \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} \| ^ {2} + \eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} \frac {1}{C} \sum_ {i \in \mathcal {C}} \| \nabla f _ {i} (w ^ {t}) \| ^ {2} \\ \leq \left(1 + \frac {1}{K - 1}\right) \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k - 1} ^ {t} \| ^ {2} + \eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} G ^ {2} + 4 K \eta^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2}. \\ \end{array}
$$
Unrolling the aggregated term on iteration  $k \leq K$ . When local interval  $K \geq 2$ ,  $\left(1 + \frac{1}{K - 1}\right)^k \leq \left(1 + \frac{1}{K - 1}\right)^K \leq 4$ . Then we have:
$$
\begin{array}{l} \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} _ {t} \left\| w ^ {t} - w _ {i, k} ^ {t} \right\| ^ {2} \\ \leq \sum_ {\tau = 0} ^ {k - 1} \left(1 + \frac {1}{K - 1}\right) ^ {\tau} \left(\eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} G ^ {2} + 4 K \eta^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2}\right) \\ + \left(1 + \frac {1}{K - 1}\right) ^ {k} \frac {1}{C} \sum_ {i \in \mathcal {C}} \| w ^ {t} - w _ {i, 0} ^ {t} \| ^ {2} \\ \leq 3 (K - 1) \left(\eta^ {2} \sigma_ {l} ^ {2} + 4 K \eta^ {2} G ^ {2} + 4 K \eta^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2}\right) + 4 \beta^ {2} \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} _ {t} \| w ^ {t} - w _ {i, K} ^ {t - 1} \| ^ {2} \\ \end{array}
$$
$$
\leq 3 K \eta^ {2} \left(\sigma_ {l} ^ {2} + 4 K G ^ {2}\right) + 1 2 K ^ {2} \eta^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2} + 4 \beta^ {2} \Delta^ {t}.
$$
Summing the iteration on  $k = 0,1,\dots ,K - 1$
$$
\frac {1}{C} \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} _ {t} \| w ^ {t} - w _ {i, k} ^ {t} \| ^ {2} \leq 4 K \beta^ {2} \Delta^ {t} + 3 K ^ {2} \eta^ {2} \sigma_ {l} ^ {2} + 1 2 K ^ {3} \eta^ {2} G ^ {2} + 1 2 K ^ {3} \eta^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2}.
$$
Lemma 3 (Bounded global updates) The norm of the global update could be bounded by uniformly sampling. Under assumptions stated above, let  $\eta \leq \frac{1}{KL}$ , the norm of the global update of selected  $N$  clients could be bounded as:
$$
\begin{array}{l} V _ {2} ^ {t} \leq \frac {1 5 \beta^ {2}}{N} \Delta^ {t} + \frac {1 0 \eta^ {2} K}{N} \sigma_ {l} ^ {2} + \frac {3 9 \eta^ {2} K ^ {2}}{N} G ^ {2} \\ + \frac {3 9 \eta^ {2} K ^ {2} B ^ {2}}{N} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} + \frac {\eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}. \tag {25} \\ \end{array}
$$
Proof  $V_{2}$  measures the variance of the global offset after each communication round. We define an indicator function  $\mathbb{I}_{\text{event}} = 1$  if the event happens. Then, to bound it, we firstly split the expectation term:
$$
\begin{array}{l} \mathbb {E} \| w ^ {t + 1} - w ^ {t} \| ^ {2} \\ = \mathbb {E} \| \frac {1}{N} \sum_ {i \in \mathcal {N}} w _ {i, K} ^ {t} - w ^ {t} \| ^ {2} \\ = \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {N}} (w _ {i, K} ^ {t} - w ^ {t}) \| ^ {2} \\ = \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {C}} (w _ {i, K} ^ {t} - w ^ {t}) \mathbb {I} _ {i \in \mathcal {N}} \| ^ {2} \\ = \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} \left[ \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} + \beta \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \right] \| ^ {2} \\ = \frac {\eta^ {2}}{N C} \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| g _ {i, k} ^ {t} - \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} \left[ \sum_ {k = 0} ^ {K - 1} \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \beta \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \right] \| ^ {2} \\ \leq \frac {\eta^ {2} K \sigma_ {l} ^ {2}}{N} + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} \left[ \sum_ {k = 0} ^ {K - 1} \eta \nabla f _ {i} (w _ {i, k} ^ {t}) + \beta (w ^ {t} - w _ {i, K} ^ {t - 1}) \right] \| ^ {2}. \\ \end{array}
$$
To bound the second term, we can adopt the following equation. For the vector  $x_{i} \in \mathbb{R}^{d}$ , we have:
$$
\begin{array}{l} \mathbb {E} \left\| \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} x _ {i} \right\| ^ {2} = \mathbb {E} \left\langle \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} x _ {i}, \sum_ {j \in \mathcal {C}} \mathbb {I} _ {j \in \mathcal {N}} x _ {j} \right\rangle \\ = \sum_ {(i \neq j) \in \mathcal {C}} \mathbb {E} \langle \mathbb {I} _ {i \in \mathcal {N}} x _ {i}, \mathbb {I} _ {j \in \mathcal {N}} x _ {j} \rangle + \sum_ {(i = j) \in \mathcal {C}} \mathbb {E} \langle \mathbb {I} _ {i \in \mathcal {N}} x _ {i}, \mathbb {I} _ {j \in \mathcal {N}} x _ {j} \rangle \\ = \sum_ {(i \neq j) \in \mathcal {C}} \mathbb {E} \left\langle \mathbb {I} _ {i \in \mathcal {N}} x _ {i}, \mathbb {I} _ {j \in \mathcal {N}} x _ {j} \right\rangle + \sum_ {(i = j) \in \mathcal {C}} \mathbb {E} \left\langle \mathbb {I} _ {i \in \mathcal {N}} x _ {i}, \mathbb {I} _ {j \in \mathcal {N}} x _ {j} \right\rangle \\ = \frac {N (N - 1)}{C (C - 1)} \sum_ {(i \neq j) \in \mathcal {C}} \mathbb {E} \langle x _ {i}, x _ {j} \rangle + \frac {N}{C} \sum_ {(i = j) \in \mathcal {C}} \mathbb {E} \langle x _ {i}, x _ {j} \rangle \\ = \frac {N (N - 1)}{C (C - 1)} \sum_ {i, j \in \mathcal {C}} \mathbb {E} \langle x _ {i}, x _ {j} \rangle + \frac {N (C - N)}{C (C - 1)} \sum_ {(i = j) \in \mathcal {C}} \mathbb {E} \langle x _ {i}, x _ {j} \rangle \\ = \frac {N (N - 1)}{C (C - 1)} \mathbb {E} \| \sum_ {i \in \mathcal {C}} x _ {i} \| ^ {2} + \frac {N (C - N)}{C (C - 1)} \sum_ {i \in \mathcal {C}} \mathbb {E} \| x _ {i} \| ^ {2}. \\ \end{array}
$$
We firstly bound the first term in the above equation. Taking  $x_{i} = \sum_{k=0}^{K-1} \eta \nabla f_{i}(w_{i,k}^{t}) + \beta (w^{t} - w_{i,K}^{t-1})$  into  $\mathbb{E} \| \sum_{i \in \mathcal{C}} x_{i} \|^{2}$ , we have:
$$
\mathbb {E} \| \sum_ {i \in \mathcal {C}} \left[ \sum_ {k = 0} ^ {K - 1} \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \beta \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \right] \| ^ {2} = \eta^ {2} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}.
$$
Then we bound the second term in above equation. Taking  $x_{i} = \sum_{k=0}^{K-1} \eta \nabla f_{i}(w_{i,k}^{t}) + \beta (w^{t} - w_{i,K}^{t-1})$  into  $\sum_{i \in \mathcal{C}} \mathbb{E}\|x_{i}\|^{2}$ , we have:
$$
\begin{array}{l} \sum_ {i \in \mathcal {C}} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \beta \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \| ^ {2} \\ = \sum_ {i \in \mathcal {C}} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \left[ \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \frac {\beta}{K} \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \right] \| ^ {2} \\ \leq K \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \frac {\beta}{K} \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \| ^ {2} \\ = K \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) - \eta \nabla f _ {i} \left(w ^ {t}\right) + \eta \nabla f _ {i} \left(w ^ {t}\right) + \frac {\beta}{K} \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \| ^ {2} \\ \leq 3 \eta^ {2} K L ^ {2} \underbrace {\sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| w _ {i , k} ^ {t} - w ^ {t} \| ^ {2}} _ {C V _ {1} ^ {t}} + 3 \eta^ {2} K ^ {2} \sum_ {i \in \mathcal {C}} \mathbb {E} \| \nabla f _ {i} (w ^ {t}) \| ^ {2} + 3 \beta^ {2} \underbrace {\sum_ {i \in \mathcal {C}} \mathbb {E} \| (w ^ {t} - w _ {i , K} ^ {t - 1}) \| ^ {2}} _ {C \Delta^ {t}} \\ \leq 3 C \eta^ {2} K L ^ {2} V _ {1} ^ {t} + 3 C \beta^ {2} \Delta^ {t} + 3 C \eta^ {2} K ^ {2} G ^ {2} + 3 C \eta^ {2} K ^ {2} B ^ {2} \| \nabla f (w ^ {t}) \| ^ {2}. \\ \end{array}
$$
We bound all the components in  $V_2^t$  term. Let  $1 \leq N < C$ , to generate the final bound, summarizing the inequalities all above and adopting the bounded  $V_1^t$  in Lemma 2, then we have:
$$
\begin{array}{l} V _ {2} ^ {t} \leq \frac {\eta^ {2} K \sigma_ {l} ^ {2}}{N} + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \mathbb {I} _ {i \in \mathcal {N}} \left[ \sum_ {k = 0} ^ {K - 1} \eta \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) + \beta \left(w ^ {t} - w _ {i, K} ^ {t - 1}\right) \right] \| ^ {2} \\ \leq \frac {\eta^ {2} K \sigma_ {l} ^ {2}}{N} + \frac {3 (C - N)}{N (C - 1)} \left(\eta^ {2} K L ^ {2} V _ {1} ^ {t} + \beta^ {2} \Delta^ {t} + \eta^ {2} K ^ {2} G ^ {2} + \eta^ {2} K ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2}\right) \\ + \frac {(N - 1)}{C N (C - 1)} \eta^ {2} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} \\ \leq \frac {\eta^ {2} K \sigma_ {l} ^ {2}}{N} + \frac {3}{N} \left(\beta^ {2} \Delta^ {t} + \eta^ {2} K ^ {2} G ^ {2} + \eta^ {2} K ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2}\right) + \frac {\eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} \\ + \frac {3}{N} \left(4 \eta^ {2} K ^ {2} L ^ {2} \beta^ {2} \Delta^ {t} + 3 K ^ {3} \eta^ {4} L ^ {2} \left(\sigma_ {l} ^ {2} + 4 K G ^ {2}\right) + 1 2 K ^ {4} \eta^ {4} L ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2}\right) \\ = \frac {3 \beta^ {2}}{N} \left(1 + 4 \eta^ {2} K ^ {2} L ^ {2}\right) \Delta^ {t} + \frac {\eta^ {2} K}{N} \left(1 + 9 K ^ {2} \eta^ {2} L ^ {2}\right) \sigma_ {l} ^ {2} + \frac {3 \eta^ {2} K ^ {2}}{N} \left(1 + 1 2 \eta^ {2} K ^ {2} L ^ {2}\right) G ^ {2} \\ + \frac {3 \eta^ {2} K ^ {2} B ^ {2}}{N} \left(1 + 1 2 \eta^ {2} K ^ {2} L ^ {2}\right) \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} + \frac {\eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}. \\ \end{array}
$$
To minimize the coefficients of each term, we can select a constant order for the term  $\eta^2 K^2 L^2$ . For convenience, we directly select the  $\eta^2 K^2 L^2 \leq 1$  which requires the learning rate  $\eta \leq \frac{1}{KL}$ . This completes the proof.
Lemma 4 (Bounded divergence term) The divergence term  $\Delta^t$  could be upper bounded by the local update rules. According to the relaxed initialization in our method, under assumptions stated above, let the learning rate satisfy  $\eta \leq \frac{1}{KL}$  and the relaxed coefficient satisfy  $\beta \leq \frac{\sqrt{2}}{12}$ , the divergence term  $\Delta^t$  could be bounded as the recursion of:
$$
\begin{array}{l} \Delta^ {t} \leq \frac {\Delta^ {t} - \Delta^ {t + 1}}{1 - 7 2 \beta^ {2}} + \frac {5 1 \eta^ {2} K}{1 - 7 2 \beta^ {2}} \sigma_ {l} ^ {2} + \frac {1 9 5 \eta^ {2} K ^ {2}}{1 - 7 2 \beta^ {2}} G ^ {2} + \frac {1 9 5 \eta^ {2} K ^ {2} B ^ {2}}{1 - 7 2 \beta^ {2}} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} \\ + \frac {3 \eta^ {2}}{C N (1 - 7 2 \beta^ {2})} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}. \tag {26} \\ \end{array}
$$
Proof The divergence term measures the inconsistency level in the FL framework. According to the local updates, we have the following recursive formula:
$$
\underbrace{w^{t + 1} - w_{i,K}^{t}}_{\substack{\text{local bias in round $t$}}} = \beta \underbrace{(w_{i,K}^{t - 1} - w^{t})}_{\substack{\text{local bias in round $t$}}} + (w^{t + 1} - w^{t}) + \sum_{k = 0}^{K - 1}\eta g_{i,k}^{t}.
$$
By taking the squared norm and expectation on both sides, we have:
$$
\begin{array}{l} \mathbb {E} \| w ^ {t + 1} - w _ {i, K} ^ {t} \| ^ {2} = \mathbb {E} \| \beta (w _ {i, K} ^ {t - 1} - w ^ {t}) + w ^ {t + 1} - w ^ {t} + \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} \| ^ {2} \\ \leq 3 \beta^ {2} \mathbb {E} \| w _ {i, K} ^ {t - 1} - w ^ {t} \| ^ {2} + 3 \underbrace {\mathbb {E} \| w ^ {t + 1} - w ^ {t} \| ^ {2}} _ {V _ {2} ^ {t}} + 3 \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} \| ^ {2}. \\ \end{array}
$$
The second term in the above inequality is  $V_{2}$  we have bounded in lemma 3. Then we bound the stochastic gradients term. We have:
$$
\begin{array}{l} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} \| ^ {2} = \eta^ {2} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} g _ {i, k} ^ {t} \| ^ {2} \\ = \eta^ {2} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \left(g _ {i, k} ^ {t} - \nabla f _ {i} \left(w _ {i, k} ^ {t}\right)\right) \| ^ {2} + \eta^ {2} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} \\ \leq \eta^ {2} K \sigma_ {l} ^ {2} + \eta^ {2} K \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) - \nabla f _ {i} \left(w ^ {t}\right) + \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} \\ \leq \eta^ {2} K \sigma_ {l} ^ {2} + 2 \eta^ {2} K \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) - \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} + 2 \eta^ {2} K \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| \nabla f _ {i} \left(w ^ {t}\right) \| ^ {2} \\ \leq \eta^ {2} K \sigma_ {l} ^ {2} + 2 \eta^ {2} K L ^ {2} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| w _ {i, k} ^ {t} - w ^ {t} \| ^ {2} + 2 \eta^ {2} K ^ {2} \mathbb {E} \| \nabla f _ {i} (w ^ {t}) \| ^ {2}. \\ \end{array}
$$
Taking the average on client  $i$ , we have:
$$
\begin{array}{l} \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} \| ^ {2} \leq \eta^ {2} K \sigma_ {l} ^ {2} + \frac {2 \eta^ {2} K L ^ {2}}{C} \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \mathbb {E} \| w _ {i, k} ^ {t} - w ^ {t} \| ^ {2} + \frac {2 \eta^ {2} K ^ {2}}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} \| \nabla f _ {i} (w ^ {t}) \| ^ {2} \\ \leq \eta^ {2} K \sigma_ {l} ^ {2} + 2 \eta^ {2} K L ^ {2} V _ {1} ^ {t} + 2 \eta^ {2} K ^ {2} G ^ {2} + 2 \eta^ {2} K ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2}. \\ \end{array}
$$
Recalling the condition of  $\eta \leq \frac{1}{KL}$  and combining this and the squared norm inequality, we have:
$$
\begin{array}{l} \Delta^ {t + 1} = \frac {1}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} \| w ^ {t + 1} - w _ {i, K} ^ {t} \| ^ {2} \\ \leq 3 \beta^ {2} \Delta^ {t} + 3 V _ {2} ^ {t} + \frac {3}{C} \sum_ {i \in \mathcal {C}} \mathbb {E} \| \sum_ {k = 0} ^ {K - 1} \eta g _ {i, k} ^ {t} \| ^ {2} \\ \leq 3 \beta^ {2} \left(1 + \frac {1 5}{N} + 8 \eta^ {2} K ^ {2} L ^ {2}\right) \Delta^ {t} + 6 \eta^ {2} K ^ {2} B ^ {2} \left(1 + \frac {3 9}{2 N} + 1 2 \eta^ {2} K ^ {2} L ^ {2}\right) \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} \\ + 3 \eta^ {2} K \left(1 + \frac {1 0}{N} + 6 \eta^ {2} K ^ {2} L ^ {2}\right) \sigma_ {l} ^ {2} + 6 \eta^ {2} K ^ {2} \left(1 + \frac {3 9}{2 N} + 1 2 \eta^ {2} K ^ {2} L ^ {2}\right) G ^ {2} \\ + \frac {3 \eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2} \\ \leq 7 2 \beta^ {2} \Delta^ {t} + 5 1 \eta^ {2} K \sigma_ {l} ^ {2} + 1 9 5 \eta^ {2} K ^ {2} G ^ {2} + 1 9 5 \eta^ {2} K ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} \\ + \frac {3 \eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}. \\ \end{array}
$$
Let  $72\beta^2 < 1$  where  $\beta \leq \frac{\sqrt{2}}{12}$ , thus we add  $(1 - 72\beta^{2})\Delta^{t}$  on both sides and get the recursive formulation:
$$
\begin{array}{l} (1 - 7 2 \beta^ {2}) \Delta^ {t} \leq (\Delta^ {t} - \Delta^ {t + 1}) + 5 1 \eta^ {2} K \sigma_ {l} ^ {2} + 1 9 5 \eta^ {2} K ^ {2} G ^ {2} + 1 9 5 \eta^ {2} K ^ {2} B ^ {2} \mathbb {E} \| \nabla f (w ^ {t}) \| ^ {2} \\ + \frac {3 \eta^ {2}}{C N} \mathbb {E} \| \sum_ {i \in \mathcal {C}} \sum_ {k = 0} ^ {K - 1} \nabla f _ {i} \left(w _ {i, k} ^ {t}\right) \| ^ {2}. \\ \end{array}
$$
Then we multiply the  $\frac{1}{1 - 72\beta^2}$  on both sides, which completes the proof.
A.2.2 Expanding the Smoothness Inequality for the Non-convex Objective
For the non-convex and  $L$ -smooth function  $f$ , we firstly expand the smoothness inequality at round  $t$  as: