SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
T _ {2} \leq 8 \left(1 + \frac {2}{\beta}\right) \beta^ {2} G _ {f} ^ {2} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2}.
$$
Putting pieces together will conclude the proof.
The following lemma establishes the difference between the exact gradients computed on  $F$  and the gradients we actually used in Algorithm 1.
Lemma 4 (Gradient difference). For Algorithm 1, under the assumptions of Theorem 2, the following statement holds true:
$$
\begin{array}{l} \mathbb {E} \left. \right.\left\| \mathbf {g} _ {\mathbf {w}} ^ {t} - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \leq 4 G _ {g} ^ {2} \frac {\sigma^ {2}}{B} + 8 G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2} \\ \mathbb {E} \left. \right.\left\| \mathbf {g} _ {\boldsymbol {\alpha}} ^ {t} - \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \leq \sum_ {j = 1} ^ {N} G _ {g} ^ {2} \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2} \\ \end{array}
$$
Proof. By the definition of  $\mathbf{g}_{\mathbf{w}}^{t}$ , we have
$$
\begin{array}{l} \mathbb {E} \left\| \mathbf {g} _ {\mathbf {w}} ^ {t} - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ = \left\| \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \nabla g (z _ {j} ^ {t + 1}) (\nabla f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}; \xi_ {i} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t}; \xi_ {j} ^ {t})) - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ \leq 2 \mathbb {E} \left\| \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \nabla g \left(z _ {j} ^ {t + 1}\right) \left(\nabla f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t}; \xi_ {i} ^ {t}\right) - \nabla f _ {j} \left(\mathbf {w} ^ {t}; \xi_ {j} ^ {t}\right) - \left(\nabla f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t}\right) - \nabla f _ {j} \left(\mathbf {w} ^ {t}\right)\right)\right) \right\| ^ {2} \\ + 2 \mathbb {E} \left\| \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \left(\nabla g \left(z _ {j} ^ {t + 1}\right) - \nabla g \left(f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t}\right) - \nabla f _ {j} \left(\mathbf {w} ^ {t}\right)\right) \left(\nabla f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t}\right) - \nabla f _ {j} \left(\mathbf {w} ^ {t}\right)\right) \right. \right\| ^ {2} \\ \leq 4 G _ {g} ^ {2} \frac {\sigma^ {2}}{B} + 8 G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \mathbb {E} \left\| z _ {j} ^ {t + 1} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t})\right) \right\| ^ {2}. \\ \end{array}
$$
where at the last step we use the bounded variance assumption. Similarly by the definition of  $\mathbf{g}_{\alpha}^{t}$  we have:
$$
\begin{array}{l} \mathbb {E} \left. \left\| \mathbf {g} _ {\boldsymbol {\alpha}} ^ {t} - \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} = \mathbb {E} \left\| [ g (z _ {1} ^ {t + 1}), \dots , g (z _ {N} ^ {t + 1}) ] - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \right. \\ \leq \sum_ {j = 1} ^ {N} G _ {g} ^ {2} \mathbb {E} \left| \left| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - f _ {j} (\mathbf {w} ^ {t})) \right| \right| ^ {2}. \\ \end{array}
$$
Lemma 5 (Connection between stationary measure and iterates). For Algorithm 1, if we define
$$
G _ {w} (\boldsymbol {\alpha}, \mathbf {w}) = \frac {1}{\gamma} \left(\mathbf {w} - \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} + \gamma \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha}, \mathbf {w})\right)\right), G _ {\alpha} (\boldsymbol {\alpha}, \mathbf {w}) = \frac {1}{\eta} \left(\boldsymbol {\alpha} - \mathcal {P} _ {\Lambda} \left(\boldsymbol {\alpha} - \eta \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha}, \mathbf {w})\right)\right),
$$
then the following statements hold:
$$
\mathbb {E} \left\| G _ {\alpha} (\pmb {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \leq \frac {2}{\eta^ {2}} \mathbb {E} \left\| \pmb {\alpha} ^ {t} - \pmb {\alpha} ^ {t + 1} \right\| ^ {2} + 2 G _ {g} ^ {2} \sum_ {j = 1} ^ {N} \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2},
$$
$$
\mathbb {E} \left\| G _ {w} (\pmb {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \leq \frac {2}{\gamma^ {2}} \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\| ^ {2} + 4 G _ {g} ^ {2} \frac {\sigma^ {2}}{B} + 8 G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2}.
$$
Proof. We begin with proving the first statement. According to updating rule we have
$$
\begin{array}{l} \mathbb {E} \left\| G _ {\boldsymbol {\alpha}} (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ = \mathbb {E} \left\| \frac {1}{\eta} \left(\boldsymbol {\alpha} ^ {t} - \mathcal {P} _ {\Lambda} \left(\boldsymbol {\alpha} ^ {t} - \eta \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t})\right)\right) \right\| ^ {2} \\ \leq \frac {2}{\eta^ {2}} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t} - \mathcal {P} _ {\Lambda} \left(\boldsymbol {\alpha} ^ {t} - \eta \mathbf {g} _ {\boldsymbol {\alpha}} ^ {t}\right) \right\| ^ {2} + \frac {2}{\eta^ {2}} \mathbb {E} \left\| \mathcal {P} _ {\Lambda} \left(\boldsymbol {\alpha} ^ {t} - \eta \mathbf {g} _ {\boldsymbol {\alpha}} ^ {t}\right) - \mathcal {P} _ {\Lambda} \left(\boldsymbol {\alpha} ^ {t} - \eta \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t})\right) \right\| ^ {2} \\ \leq \frac {2}{\eta^ {2}} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t + 1} \right\| ^ {2} + 2 \mathbb {E} \left\| \mathbf {g} _ {\boldsymbol {\alpha}} ^ {t} - \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ \leq \frac {2}{\eta^ {2}} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t + 1} \right\| ^ {2} + 2 G _ {g} ^ {2} \sum_ {j = 1} ^ {N} \mathbb {E} \left\| z _ {j} ^ {t + 1} - \left(f _ {\hat {\mathcal {T}}} (\mathbf {w} ^ {t}) - f _ {j} (\mathbf {w} ^ {t})\right) \right\| ^ {2} \\ \end{array}
$$
where at last step we apply Lemma 4. Similarly, for the second statement we have:
$$
\begin{array}{l} \mathbb {E} \left\| G _ {\mathbf {w}} (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ = \mathbb {E} \left\| \frac {1}{\gamma} \left(\mathbf {w} ^ {t} - \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} ^ {t} + \eta \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t})\right)\right) \right\| ^ {2} \\ \leq \frac {2}{\gamma^ {2}} \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} ^ {t} + \gamma \mathbf {g} _ {\mathbf {w}} ^ {t}\right) \right\| ^ {2} + \frac {2}{\gamma^ {2}} \mathbb {E} \left\| \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} ^ {t} + \gamma \mathbf {g} _ {\mathbf {w}} ^ {t}\right) - \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} ^ {t} + \gamma \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t})\right) \right\| ^ {2} \\ \leq \frac {2}{\gamma^ {2}} \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\| + 2 \mathbb {E} \left\| \mathbf {g} _ {\mathbf {w}} ^ {t} - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \\ \leq \frac {2}{\gamma^ {2}} \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\| ^ {2} + 4 G _ {g} ^ {2} \frac {\sigma^ {2}}{B} + 8 G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2}. \\ \end{array}
$$
The following lemma connects  $F(\pmb{\alpha}^{t+1}, \mathbf{w}^{t+1})$  and  $F(\pmb{\alpha}^{t+1}, \mathbf{w}^t)$ .
Lemma 6 (Dual Variable One Iteration Analysis). Let  $L \coloneqq \max \left\{4G_f^2 L_g + 2G_gL_f, 2C\right\}$ . For Algorithm 1, under the assumptions of Theorem 2, the following statement holds true:
$$
\begin{array}{l} \mathbb {E} \left[ F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t + 1}\right) - F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}\right) \right] \\ \geq \left(\frac {1}{2 \gamma} - \frac {L}{2} - \frac {L ^ {2} \eta}{2}\right) \mathbb {E} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} - \frac {1}{2 \eta} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} - \left(2 \gamma G _ {g} ^ {2} + 8 \beta^ {2} L _ {g} ^ {2} G _ {f} ^ {2} \gamma\right) \frac {\sigma^ {2}}{B} \\ - 4 L _ {g} ^ {2} G _ {f} ^ {2} \gamma \sum_ {j = 1} ^ {N} \alpha^ {t} (j) (1 - \beta) ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t - 1}) - f _ {j} (\mathbf {w} ^ {t - 1})\right) \right\| ^ {2} \\ - 1 6 (1 - \beta) ^ {2} L _ {g} ^ {2} G _ {f} ^ {4} \gamma \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2}. \\ \end{array}
$$
Proof. By the optimality of projection and updating rule of  $\mathbf{w}$ , we have:
$$
\left\langle \gamma \mathbf {g} _ {\mathbf {w}} ^ {t} + \left(\mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1}\right), \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\rangle \leq 0
$$
Re-arranging terms yields:
$$
\left\langle \mathbf {g} _ {\mathbf {w}} ^ {t}, \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\rangle \leq - \frac {1}{\gamma} \| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \| ^ {2}
$$
Adding and subtracting  $\nabla_{\mathbf{w}}F(\pmb{\alpha}^t,\mathbf{w}^t)$  gives:
$$
\left\langle \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\rangle + \left\langle \mathbf {g} _ {\mathbf {w}} ^ {t} - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1} \right\rangle \leq - \frac {1}{\gamma} \| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \| ^ {2}
$$
Applying Cauchy-Schwartz inequality yields:
$$
\frac {1}{\gamma} \| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \| ^ {2} \leq \left\langle \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle + \frac {1}{2 \gamma} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} + \frac {1}{2} \gamma \left\| \mathbf {g} _ {\mathbf {w}} ^ {t} - \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2}
$$
Now, we take expectation over the randomness of  $\zeta_T^t$ ,  $\zeta_j^t$ , and apply Lemma 4 to get:
$$
\begin{array}{l} \frac {1}{2 \gamma} \mathbb {E} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} \leq \mathbb {E} \left\langle \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle + 2 \gamma G _ {g} ^ {2} \frac {\sigma^ {2}}{B} \\ + 4 \gamma G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \mathbb {E} \left\| z _ {j} ^ {t + 1} - (f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t})) \right\| ^ {2}. \\ \end{array}
$$
Plugging in Lemma 2 yields:
$$
\begin{array}{l} \frac {1}{2 \gamma} \mathbb {E} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} \leq \mathbb {E} \left\langle \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle + 2 \gamma G _ {g} ^ {2} \frac {\sigma^ {2}}{B} \\ + 4 \gamma G _ {f} ^ {2} L _ {g} ^ {2} \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \left((1 - \beta) ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\gamma}} \left(\mathbf {w} ^ {t - 1}\right) - f _ {j} \left(\mathbf {w} ^ {t - 1}\right)\right) \right\| ^ {2} \right. \\ \left. + 4 (1 - \beta) ^ {2} G _ {f} ^ {2} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2} + 2 \beta^ {2} \frac {\sigma^ {2}}{B}\right). \\ \end{array}
$$
On the other hand, from Proposition 2 on the smoothness of  $F$  with parameter  $L$  we have:
$$
\begin{array}{l} F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t + 1}) - F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}) \\ \geq \left\langle \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle - \frac {L}{2} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} \\ = \left\langle \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle - \frac {L}{2} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} + \left\langle \nabla F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}) - \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle \\ \geq \left\langle \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle - \frac {L}{2} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} \\ - \frac {1}{L \sqrt {\eta}} \left\| \nabla F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}) - \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| L \sqrt {\eta} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| \\ \geq \left\langle \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle - \frac {L}{2} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} - \frac {1}{2 \eta} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} - \frac {L ^ {2} \eta}{2} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} \\ \geq \left\langle \nabla F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\rangle - \left(\frac {L}{2} + \frac {L ^ {2} \eta}{2}\right) \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} - \frac {1}{2 \eta} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2}. \\ \end{array}
$$
Putting pieces together will conclude the proof:
$$
\begin{array}{l} \mathbb {E} \left[ F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t + 1}\right) - F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}\right) \right] \\ \geq \left(\frac {1}{2 \gamma} - \frac {L}{2} - \frac {L ^ {2} \eta}{2}\right) \mathbb {E} \| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \| ^ {2} - \frac {1}{2 \eta} \mathbb {E} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} - \left(2 \gamma G _ {g} ^ {2} + 8 \beta^ {2} L _ {g} ^ {2} G _ {f} ^ {2} \gamma\right) \frac {\sigma^ {2}}{B} \\ - 4 L _ {g} ^ {2} G _ {f} ^ {2} \gamma \sum_ {j = 1} ^ {N} \alpha^ {t} (j) (1 - \beta) ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t - 1}) - f _ {j} (\mathbf {w} ^ {t - 1})\right) \right\| ^ {2} \\ - 1 6 (1 - \beta) ^ {2} L _ {g} ^ {2} G _ {f} ^ {4} \gamma \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2}. \\ \end{array}
$$
The following lemma connects  $F(\pmb{\alpha}^{t+1}, \mathbf{w}^{t+1})$  and  $F(\pmb{\alpha}^t, \mathbf{w}^t)$ .
Lemma 7 (One Iteration Descent Lemma). For Algorithm 1, under the assumptions of Theorem 2, the following statement holds true:
$$
\begin{array}{l} \mathbb {E} \left[ F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t + 1}\right) - F \left(\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}\right) \right] \\ \geq \left(\frac {1}{2 \gamma} - \frac {L}{2} - \frac {L ^ {2} \eta}{2}\right) \mathbb {E} \| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \| ^ {2} - \left(1 6 (1 - \beta) ^ {2} L _ {g} ^ {2} G _ {f} ^ {4} \gamma + \frac {L ^ {2} \eta}{2}\right) \mathbb {E} \| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \| ^ {2} \\ + \left(\frac {\mu}{2} - \frac {3}{2 \eta} - \frac {1}{4 \eta (1 - \beta) ^ {2}}\right) \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} + \left(\mu - \frac {1}{2 \eta} - \frac {\eta L ^ {2}}{2}\right) \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \right\| ^ {2} \\ - \sum_ {j = 1} ^ {N} (1 - \beta) ^ {2} \left(4 \alpha^ {t} (j) L _ {g} ^ {2} G _ {f} ^ {2} \gamma + \eta G _ {g} ^ {2}\right) \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t - 1}) - f _ {j} (\mathbf {w} ^ {t - 1})\right) \right\| ^ {2} \\ - \left(2 \gamma G _ {g} ^ {2} + 8 \beta^ {2} L _ {g} ^ {2} G _ {f} ^ {2} \gamma\right) \frac {\sigma^ {2}}{B}. \\ \end{array}
$$
Proof. According to property of projection we have:
$$
\left\langle \mathbf {g} _ {\alpha} ^ {t} + \frac {1}{\eta} \left(\boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t}\right), \boldsymbol {\alpha} - \boldsymbol {\alpha} ^ {t + 1} \right\rangle \geq 0 \tag {5}
$$
$$
\left\langle \mathbf {g} _ {\alpha} ^ {t} + \frac {1}{\eta} \left(\boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t}\right), \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t + 1} \right\rangle \geq 0 \tag {6}
$$
$$
\left\langle \mathbf {g} _ {\alpha} ^ {t - 1} + \frac {1}{\eta} \left(\boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1}\right), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \geq 0 \tag {7}
$$
Since  $F(\cdot ,\mathbf{w})$  is strongly convex for any  $\mathbf{w}\in \mathcal{W}$  we have:
$$
\begin{array}{l} F (\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}) - F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) \geq \left\langle \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle + \frac {\mu}{2} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} \\ \geq \left\langle \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) - \mathbf {g} _ {\alpha} ^ {t - 1}, \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle + \frac {\mu}{2} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} \\ - \frac {1}{\eta} \left\langle \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1}, \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \\ \geq \left\langle \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle + \frac {\mu}{2} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} \\ + \left\langle \mathbf {g} _ {\alpha} ^ {t - 1} - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \\ - \frac {1}{\eta} \left\langle \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1}, \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \\ \end{array}
$$
where at second inequality we use the fact of (7). For the first dot product term, we observe that:
$$
\begin{array}{l} \left\langle \nabla_ {\alpha} F \left(\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}\right) - \nabla_ {\alpha} F \left(\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}\right), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle (8) \\ = \left\langle \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}) - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t - 1}), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \\ + \left\langle \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t - 1}) - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}), \Delta \right\rangle \\ + \left\langle \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t - 1}) - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}), \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \right\rangle \\ \geq - \frac {L ^ {2} \eta}{2} \| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \| ^ {2} - \frac {1}{2 \eta} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} - \frac {\eta L ^ {2}}{2} \| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \| ^ {2} - \frac {1}{2 \eta} \| \Delta \| ^ {2} + \mu \| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \| ^ {2} (9) \\ \end{array}
$$
where  $\Delta = (\pmb{\alpha}^{t + 1} - \pmb{\alpha}^t) - (\pmb{\alpha}^t -\pmb{\alpha}^{t - 1})$
For the second dot product, we have:
$$
\begin{array}{l} \mathbb {E} \left\langle \mathbf {g} _ {\alpha} ^ {t - 1} - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}), \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle \\ \geq - \eta (1 - \beta) ^ {2} \mathbb {E} \left\| \mathbf {g} _ {\alpha} ^ {t - 1} - \nabla_ {\alpha} F (\boldsymbol {\alpha} ^ {t - 1}, \mathbf {w} ^ {t - 1}) \right\| ^ {2} - \frac {1}{4 \eta (1 - \beta) ^ {2}} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} \\ \geq - \eta (1 - \beta) ^ {2} \sum_ {j = 1} ^ {N} G _ {g} ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t - 1}) - f _ {j} (\mathbf {w} ^ {t - 1})\right) \right\| ^ {2} - \frac {1}{4 \eta (1 - \beta) ^ {2}} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2}. \\ \end{array}
$$
where we plug in Lemma 4 at last step.
For the third dot product, we use the following identity:
$$
\frac {1}{\eta} \left\langle \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1}, \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\rangle = \frac {1}{\eta} \left(\frac {1}{2} \| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \| ^ {2} + \frac {1}{2} \| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \| ^ {2} - \frac {1}{2} \| \Delta \| ^ {2}\right) \tag {10}
$$
$$
\begin{array}{l} \mathbb {E} \left[ F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}\right) - F \left(\boldsymbol {\alpha} ^ {t}, \mathbf {w} ^ {t}\right) \right] \geq - \frac {L ^ {2} \eta}{2} \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2} + \left(\frac {\mu}{2} - \frac {1}{\eta} - \frac {1}{4 \eta (1 - \beta) ^ {2}}\right) \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} \\ + \left(\mu - \frac {1}{2 \eta} - \frac {\eta L ^ {2}}{2}\right) \mathbb {E} \| \boldsymbol {\alpha} ^ {t} - \boldsymbol {\alpha} ^ {t - 1} \| ^ {2} \\ - \eta (1 - \beta) ^ {2} \sum_ {j = 1} ^ {N} G _ {g} ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t - 1}\right) - f _ {j} \left(\mathbf {w} ^ {t - 1}\right)\right) \right\| ^ {2} \tag {11} \\ \end{array}
$$
Recall Lemma 6 gives the following lower bound of  $\mathbb{E}[F(\pmb{\alpha}^{t + 1},\mathbf{w}^{t + 1}) - F(\pmb{\alpha}^{t + 1},\mathbf{w}^t)]$ :
$$
\begin{array}{l} \mathbb {E} \left[ F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t + 1}\right) - F \left(\boldsymbol {\alpha} ^ {t + 1}, \mathbf {w} ^ {t}\right) \right] (12) \\ \geq \left(\frac {1}{2 \gamma} - \frac {L}{2} - \frac {L ^ {2} \eta}{2}\right) \mathbb {E} \left\| \mathbf {w} ^ {t + 1} - \mathbf {w} ^ {t} \right\| ^ {2} - \frac {1}{2 \eta} \mathbb {E} \left\| \boldsymbol {\alpha} ^ {t + 1} - \boldsymbol {\alpha} ^ {t} \right\| ^ {2} - \left(2 \gamma G _ {g} ^ {2} + 8 \beta^ {2} L _ {g} ^ {2} G _ {f} ^ {2} \gamma\right) \frac {\sigma^ {2}}{B} \\ - 4 L _ {g} ^ {2} G _ {f} ^ {2} \gamma \sum_ {j = 1} ^ {N} \alpha^ {t} (j) (1 - \beta) ^ {2} \mathbb {E} \left\| z _ {j} ^ {t} - \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t - 1}) - f _ {j} (\mathbf {w} ^ {t - 1})\right) \right\| ^ {2} \\ - 1 6 (1 - \beta) ^ {2} L _ {g} ^ {2} G _ {f} ^ {4} \gamma \mathbb {E} \left\| \mathbf {w} ^ {t} - \mathbf {w} ^ {t - 1} \right\| ^ {2}. (13) \\ \end{array}
$$