SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left\| \boldsymbol {v} ^ {r + 1} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {r}) \right\| ^ {2} \\ \leq (1 - \frac {1}{2} \mu \eta N K) \mathbb {E} \| \boldsymbol {v} ^ {r} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {r}) \| ^ {2} + \eta^ {2} N ^ {2} K ^ {2} e ^ {4} \delta^ {2} - \frac {1}{2} \eta N K \underbrace {\mathbb {E} [ \Phi (\hat {\boldsymbol {\alpha}} , \boldsymbol {v} ^ {r}) - \Phi (\hat {\boldsymbol {\alpha}} , \boldsymbol {v} ^ {*} (\hat {\boldsymbol {\alpha}})) ]} _ {\geq 0} \\ + \left(\frac {1}{2 \mu \eta N K} + 2\right) \left(2 4 4 e ^ {6} \eta^ {4} N ^ {4} K ^ {4} L ^ {4} + 2 5 6 \eta^ {2} N ^ {3} K ^ {2} \left(\frac {e}{4 R - e}\right) ^ {2}\right) G ^ {2} N \log (1 / p) \\ \leq \left(1 - \frac {1}{2} \mu \eta N K\right) \mathbb {E} \left\| \boldsymbol {v} ^ {r} - \boldsymbol {v} ^ {*} (\hat {\alpha}) \right\| ^ {2} + \eta^ {2} N ^ {2} K ^ {2} e ^ {4} \delta^ {2} \\ + \left(\frac {1}{2 \mu \eta N K} + 2\right) \left(2 4 4 e ^ {6} \eta^ {4} N ^ {4} K ^ {4} L ^ {4} + 2 5 6 \eta^ {2} N ^ {3} K ^ {2} \left(\frac {e}{4 R - e}\right) ^ {2}\right) G ^ {2} N \log (1 / p). \\ \end{array}
$$
Putting above inequality back to (18) yields:
$$
\begin{array}{l} \| \boldsymbol {v} ^ {r + 1} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {r + 1}) \| ^ {2} \leq \left(1 - \frac {1}{4 a}\right) \| \boldsymbol {v} ^ {r} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {r}) \| ^ {2} + 2 \eta^ {2} N ^ {2} K ^ {2} e ^ {4} \delta^ {2} + (1 + 4 a - 2) \kappa_ {\Phi} ^ {2} \| \boldsymbol {\alpha} ^ {r + 1} - \boldsymbol {\alpha} ^ {r} \| ^ {2} \\ + 2 \left(\frac {1}{2 \mu \eta N K} + 2\right) \left(2 4 4 e ^ {6} \eta^ {4} N ^ {4} K ^ {4} L ^ {4} + 2 5 6 \eta^ {2} N ^ {3} K ^ {2} \left(\frac {e}{4 R - e}\right) ^ {2}\right) G ^ {2} N \log (1 / p) \\ \leq \left(1 - \frac {1}{4 a}\right) \| \boldsymbol {v} ^ {r} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {r}) \| ^ {2} + 2 \eta^ {2} N ^ {2} K ^ {2} e ^ {4} \delta^ {2} \\ + 2 \left(\frac {1}{2 \mu \eta N K} + 2\right) \left(2 4 4 e ^ {6} \eta^ {4} N ^ {4} K ^ {4} L ^ {4} + 2 5 6 \eta^ {2} N ^ {3} K ^ {2} \left(\frac {e}{4 R - e}\right) ^ {2}\right) G ^ {2} N \log (1 / p) \\ + O \left(\frac {\kappa_ {\Phi} ^ {2}}{\mu \eta N K} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \kappa_ {g} ^ {2} L ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \left(\gamma^ {2} L (1 - \mu \gamma) ^ {r} D G + \frac {\gamma^ {3} \kappa \delta^ {2}}{M} + \frac {\gamma^ {2} \delta^ {2}}{M}\right)\right)\right) \\ \end{array}
$$
where at second inequality we plug in Lemma 14. Unrolling the recursion from  $r = R$  to 0, and plugging in  $\eta = \frac{4\log(NKR^3)}{\mu NKR}$  yields:
$$
\begin{array}{l} \| \boldsymbol {v} ^ {R} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {R}) \| ^ {2} \\ \leq \left(1 - \frac {1}{4} \mu \eta N K\right) ^ {R} \| \boldsymbol {v} ^ {0} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {0}) \| ^ {2} + \frac {1}{\mu} \eta N K e ^ {4} \delta^ {2} \\ + 8 \frac {1}{\mu} \left(\frac {1}{2 \mu \eta N K} + 2\right) \left(2 4 4 e ^ {6} \eta^ {3} N ^ {3} K ^ {3} L ^ {4} + 2 5 6 \eta N ^ {2} K \left(\frac {e}{4 R - e}\right) ^ {2}\right) G ^ {2} N \log (1 / p) \\ + O \left(\frac {\kappa_ {\Phi} ^ {2}}{\mu \eta N K} \sum_ {r = 0} ^ {R} \left(1 - \frac {1}{4 a}\right) ^ {R - r} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \kappa_ {g} ^ {2} L ^ {2} \bar {\zeta_ {i}} (\boldsymbol {w} ^ {*}) \left(\gamma^ {2} L (1 - \mu \gamma) ^ {r} D G + \frac {\gamma^ {3} \kappa \delta^ {2}}{M} + \frac {\gamma^ {2} \delta^ {2}}{M}\right)\right)\right) \\ \leq O \left(\frac {\| \boldsymbol {v} ^ {0} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {0}) \| ^ {2}}{N K R ^ {3}}\right) + \tilde {O} \left(\left(\frac {\kappa^ {4}}{R ^ {2}} + \frac {N}{\mu^ {2} R ^ {2}}\right) G ^ {2} N \log (1 / p) + \frac {\delta^ {2}}{\mu R}\right) \\ \left. \right. + \tilde {O} \left(R \kappa_ {\Phi} ^ {2} \sum_ {r = 0} ^ {R} \left(1 - \frac {\log (N K R ^ {3})}{R}\right) ^ {R - r} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \kappa_ {g} ^ {2} L ^ {2} \bar {\zeta_ {i}} (\boldsymbol {w} ^ {*}) \left(\gamma^ {2} L (1 - \mu \gamma) ^ {r} D G + \frac {\gamma^ {3} \kappa \delta^ {2}}{M} + \frac {\gamma^ {2} \delta^ {2}}{M}\right)\right)\right) \\ \end{array}
$$
Plugging in  $\gamma = \frac{\log(NKR^3)}{\mu R}$  yields:
$$
\begin{array}{l} \| \boldsymbol {v} ^ {R} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {R}) \| ^ {2} \leq O \left(\frac {\| \boldsymbol {v} ^ {0} - \boldsymbol {v} ^ {*} (\boldsymbol {\alpha} ^ {0}) \| ^ {2}}{N K R ^ {3}}\right) + \tilde {O} \left(\left(\frac {\kappa^ {4}}{R ^ {2}} + \frac {N}{\mu^ {2} R ^ {2}}\right) G ^ {2} N \log (1 / p) + \frac {\delta^ {2}}{\mu R}\right) \\ + \tilde {O} \left(\kappa_ {\Phi} ^ {2} R \left(\gamma^ {2} L ^ {2} \sum_ {r = 0} ^ {R} \left(1 - \frac {\log (N K R ^ {3})}{R}\right) ^ {R} \kappa_ {g} ^ {2} L ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) D G \right. \right. \\ \left. + \sum_ {r = 0} ^ {R} \left(1 - \frac {\log (N K R ^ {3})}{R}\right) ^ {R - r} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \frac {\kappa_ {g} ^ {2} L ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \gamma^ {2} \delta^ {2}}{M}\right)\right)\left. \right) \\ \leq O \left(\frac {D ^ {2}}{N K R ^ {3}}\right) + \tilde {O} \left(\left(\frac {\kappa^ {4}}{R ^ {2}} + \frac {N}{\mu^ {2} R ^ {2}}\right) G ^ {2} N \log (1 / p) + \frac {\delta^ {2}}{\mu R}\right) \\ + \tilde {O} \left(\frac {\kappa_ {\Phi} ^ {2} \kappa^ {2} \kappa_ {g} ^ {2} L ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) D G}{R} + \kappa_ {\Phi} ^ {2} R ^ {2} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \frac {\kappa_ {g} ^ {2} \kappa^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \delta^ {2}}{\mu^ {2} M R ^ {2}}\right)\right) \\ \end{array}
$$
Since  $\hat{\pmb{v}}_i = \pmb{v}^R$  and  $\hat{\alpha}_i = \alpha^R$ , we have the convergence of  $\| \hat{\pmb{v}}_i - \pmb{v}^* (\hat{\alpha}_i)\|^2$ . Plugging this convergence rate together with the convergence of  $\| \hat{\alpha}_i - \alpha^*\|^2$  from Lemma 15:
$$
\begin{array}{l} \left\| \hat {\boldsymbol {\alpha}} _ {i} - \boldsymbol {\alpha} ^ {*} \right\| ^ {2} \leq O \left(2 \left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\boldsymbol {\alpha}}} + O \left(\kappa_ {g} ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) L ^ {2} \frac {2}{\mu} (1 - \mu \gamma) ^ {R} + 2 \gamma \frac {\delta^ {2}}{\mu^ {2} M}\right)\right) \\ \leq \tilde {O} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + O \left(\kappa_ {g} ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) L ^ {2} \frac {2}{\mu} \frac {1}{R} + \frac {\delta^ {2}}{\mu^ {3} R M}\right)\right) \\ \end{array}
$$
together with applying Lemma 2 leads to:
$$
\begin{array}{l} \Phi \left(\boldsymbol {\alpha} _ {i} ^ {*}, \hat {\boldsymbol {v}} _ {i}\right) - \Phi \left(\boldsymbol {\alpha} _ {i} ^ {*}, \boldsymbol {v} _ {i} ^ {*}\right) \leq 2 L \left\| \hat {\boldsymbol {v}} _ {i} - \boldsymbol {v} ^ {*} (\hat {\boldsymbol {\alpha}} _ {i}) \right\| ^ {2} + \left(2 \kappa_ {\Phi} ^ {2} L + \frac {4 N G ^ {2}}{L}\right) \left\| \hat {\boldsymbol {\alpha}} _ {i} - \boldsymbol {\alpha} _ {i} ^ {*} \right\| ^ {2} \\ \leq O \left(\frac {L D ^ {2}}{N K R ^ {3}}\right) + \tilde {O} \left(\left(\frac {\kappa^ {4} L}{R ^ {2}} + \frac {N L}{\mu^ {2} R ^ {2}}\right) G ^ {2} N \log (1 / p) + \frac {L \delta^ {2}}{\mu R}\right) \\ + \tilde {O} \left(\frac {\kappa_ {\Phi} ^ {2} \kappa^ {2} \kappa_ {g} ^ {2} L ^ {3} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) D G}{R} + \kappa_ {\Phi} ^ {2} L R ^ {2} \left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \frac {L \kappa_ {\Phi} ^ {2} \kappa_ {g} ^ {2} \kappa^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \delta^ {2}}{\mu^ {2} M}\right) \\ + \left(2 \kappa_ {\Phi} ^ {2} L + \frac {4 N G ^ {2}}{L}\right) \tilde {O} \left(\left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \frac {\kappa_ {g} ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \kappa L}{R} + \frac {\delta^ {2}}{\mu^ {3} R M}\right) \\ \leq O \left(\frac {L D ^ {2}}{N K R ^ {3}}\right) + \tilde {O} \left(\left(\frac {\kappa^ {4} L}{R ^ {2}} + \frac {N L}{\mu^ {2} R ^ {2}}\right) G ^ {2} N \log (1 / p) + \frac {L \delta^ {2}}{\mu R}\right) \\ + \tilde {O} \left(\frac {\kappa_ {\Phi} ^ {2} \kappa^ {2} \kappa_ {g} ^ {2} L ^ {3} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) D G}{R} + \kappa_ {\Phi} ^ {2} L R ^ {2} \left(1 - \frac {1}{\kappa_ {g}}\right) ^ {T _ {\alpha}} + \frac {L ^ {2} \kappa^ {2} \kappa_ {g} ^ {2} \kappa_ {\Phi} ^ {2} \bar {\zeta} _ {i} (\boldsymbol {w} ^ {*}) \delta^ {2}}{\mu^ {2} M}\right). \\ \end{array}
$$