SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hence, we showed that the sequence  $\theta_{k}$  is ergodic. Similarly, we can show that the sequence  $\hat{\theta}_k$  is ergodic.
Finally, by ergodic theorem and Fatou's lemma, we have
$$
\mathbb {E} \left\| \theta_ {\infty} - \hat {\theta} _ {\infty} \right\| ^ {p} = \mathbb {E} \left[ \lim  _ {k \rightarrow \infty} \frac {1}{k} \sum_ {i = 1} ^ {k} \left\| \theta_ {i - 1} - \hat {\theta} _ {i - 1} \right\| ^ {p} \right] \leq \lim  _ {k \rightarrow \infty} \sup  _ {k \rightarrow \infty} \frac {1}{k} \sum_ {i = 1} ^ {k} \mathbb {E} \left\| \theta_ {i - 1} - \hat {\theta} _ {i - 1} \right\| ^ {p}. \tag {G.43}
$$
We can then apply (G.31) from the proof of Theorem G.1 to obtain:
$$
\mathcal {W} _ {p} ^ {p} \left(\nu_ {\infty}, \hat {\nu} _ {\infty}\right) \leq \frac {C _ {2}}{b n \mu} + \frac {C _ {3}}{n}, \tag {G.44}
$$
$$
C _ {2} := \frac {4 D ^ {2} K _ {2} ^ {2} \eta}{\mu} \left(2 ^ {p + 2} \left(\frac {8 \eta}{\mu} D ^ {2} K _ {2} ^ {2} \left(2 ^ {p + 1} (E / \mu) ^ {\frac {p}{p - 1}} + 5\right)\right) + 2 ^ {p + 2} (E / \mu) ^ {\frac {p}{p - 1}} + 1 0\right), \tag {G.45}
$$
$$
\begin{array}{l} C _ {3} := \frac {3 2 D ^ {3} K _ {2} ^ {3} \eta}{\mu^ {2}} (1 + K _ {1} \eta) \cdot 1 0 \cdot 2 ^ {p - 1} \left(2 ^ {p + 1} (E / \mu) ^ {\frac {p}{p - 1}} + 5\right) \\ + \frac {4 D K _ {2}}{\mu} \left(1 + K _ {1} \eta\right) \left(1 0 \cdot 2 ^ {p - 1} (E / \mu) ^ {\frac {p}{p - 1}} + 5\right). \tag {G.46} \\ \end{array}
$$