SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbf {W} _ {1} (\mathcal {L} (X), \mathcal {L} (Y)) \geq \inf  _ {\gamma \in \Gamma (\mathcal {L} (\| X \|), \mathcal {L} (\| Y \|))} \int_ {\mathbb {R} _ {+} \times \mathbb {R} _ {+}} | x - y | \mathrm {d} \gamma (x, y) (22) \\ = \mathbf {W} _ {1} (\mathcal {L} (\| X \|), \mathcal {L} (\| Y \|)). (23) \\ \end{array}
$$
This section contains the proofs of our theoretical findings.
C.1 Proof of Theorem 4
Proof. First, note that the unique stationary distributions of  $(X_{k})_{k\geq 0}$  and  $(X_{k}^{(n)})_{k\geq 0}$  are denoted respectively by  $\pi$  and  $\pi^{(n)}$ . Denote by  $E_{n}$  the event on which  $(X_{k}^{(n)})_{k\geq 0}$  has a stationary distribution  $\pi^{(n)}$  such that  $\mathbb{P}(E_n)\geq 1 - \epsilon_n$  (where  $\epsilon_{n}\in [0,1]$  is the probability that  $(X_{k}^{(n)})_{k\geq 0}$  is not ergodic, in the Wasserstein sense). Given  $E_{n}$ , let  $(\mathbf{A}_k,\mathbf{A}_k^{(n)})_{k\geq 1}$  and  $(\mathbf{b}_k,\mathbf{b}_k^{(n)})_{k\geq 1}$  be the sequences of i.i.d. optimal couplings for  $\mu_{\mathbf{A}}$  and  $\mu_{\mathbf{A}}^{(n)}$ ,  $\mu_{\mathbf{b}}$  and  $\mu_{\mathbf{b}}^{(n)}$  respectively. Therefore, by construction, for any  $k\in \mathbb{N}$ ,  $\mathbb{E}[\| \mathbf{A}_k - \mathbf{A}_k^{(n)}\| ] = \mathbf{W}_1(\mu_{\mathbf{A}},\mu_{\mathbf{A}}^{(n)})$ , and  $\mathbb{E}[\| \mathbf{b}_k - \mathbf{b}_k^{(n)}\| ] = \mathbf{W}_1(\mu_{\mathbf{b}},\mu_{\mathbf{b}}^{(n)})$ . Based on these two sequences, we consider the processes  $(X_{k})_{k\geq 0}$ ,  $(X_{k}^{(n)})_{k\geq 0}$ ,  $(Y_{k})_{k\geq 0}$  defined by the recursions:
1.  $X_{k + 1} = (\mathrm{I}_d - \eta \mathbf{A}_{k + 1})X_k + \eta \mathbf{b}_{k + 1}$ , where  $X_0\sim \pi$
2.  $X_{k + 1}^{(n)} = (\mathrm{I}_d - \eta \mathbf{A}_{k + 1}^{(n)})X_k^{(n)} + \eta \mathbf{b}_{k + 1}^{(n)}$ , where  $X_0^{(n)}\sim \pi^{(n)}$
3.  $Y_{k + 1} = (\mathrm{I}_d - \eta \mathbf{A}_{k + 1})Y_k + \eta \mathbf{b}_{k + 1}$ , where  $Y_0 = X_0^{(n)}$ .
Note that  $(X_{k})_{k\geq 0}$  corresponds to the online SGD recursion (7), and  $(X_{k}^{(n)})_{k\geq 0}$  to the offline SGD recursion (9). In addition, since these two Markov chains are started at stationarity, we have:
$$
\mathbf {W} _ {1} (\pi , \pi^ {(n)}) = \mathbf {W} _ {1} (\mathcal {L} (X _ {0}), \mathcal {L} (X _ {0} ^ {(n)})) = \mathbf {W} _ {1} (\mathcal {L} (X _ {k}), \mathcal {L} (X _ {k} ^ {(n)})), \forall k.
$$
With these definitions at hand and this observation, using triangle inequality, we can obtain:
$$
\begin{array}{l} \mathbf {W} _ {1} (\pi , \pi^ {(n)}) = \mathbf {W} _ {1} (\mathcal {L} (X _ {k}), \mathcal {L} (X _ {k} ^ {(n)})) \\ \leq \mathbf {W} _ {1} (\mathcal {L} (X _ {k}), \mathcal {L} (Y _ {k})) + \mathbf {W} _ {1} (\mathcal {L} (Y _ {k}), \mathcal {L} (X _ {k} ^ {(n)})). \\ \end{array}
$$
Now, by [GSZ21, Theorem 8], we have that  $Y_{k}$  is geometrically ergodic with respect to its stationary distribution  $\pi$ , i.e., there exist constants  $c_{\rho} > 0$ ,  $\rho \in (0,1)$  such that the following inequality holds:
$$
\mathbf {W} _ {1} \left(\mathcal {L} \left(Y _ {k}\right), \pi\right) \leq c _ {\rho} \mathbf {W} _ {1} \left(\mathcal {L} \left(Y _ {0}\right), \pi\right) \mathrm {e} ^ {- \rho k}, \text {f o r a n y} k \geq 0. \tag {24}
$$
Therefore, we have:
$$
\mathbf {W} _ {1} (\pi , \pi^ {(n)}) \leq c _ {\rho} \mathrm {e} ^ {- \rho k} \mathbf {W} _ {1} (\pi , \mathcal {L} (Y _ {0})) + \mathbf {W} _ {1} (\mathcal {L} (Y _ {k}), \mathcal {L} (X _ {k} ^ {(n)})),
$$
as  $\mathbf{W}_1(\mathcal{L}(X_k),\mathcal{L}(Y_k)) = \mathbf{W}_1(\pi ,\mathcal{L}(Y_k))\leq c_\rho \mathrm{e}^{-\rho k}\mathbf{W}_1(\pi ,\mathcal{L}(Y_0))$ . Rearranging the above terms, we get  $\mathbf{W}_1(\pi ,\pi^{(n)})(1 - c_\rho \mathrm{e}^{-\rho k})\leq \mathbf{W}_1(\mathcal{L}(Y_k),\mathcal{L}(X_k^{(n)}))$  implying:
$$
\mathbf {W} _ {1} \left(\pi , \pi^ {(n)}\right) \leq \left(1 - c _ {\rho} \mathrm {e} ^ {- \rho k}\right) ^ {- 1} \mathbf {W} _ {1} \left(\mathscr {L} \left(Y _ {k}\right), \mathscr {L} \left(X _ {k} ^ {(n)}\right)\right). \tag {25}
$$
To bound the right-hand side of (25), we consider the following difference by using the recursion definitions:
$$
\begin{array}{l} Y _ {k + 1} - X _ {k + 1} ^ {(n)} = Y _ {k} - \eta \mathbf {A} _ {k + 1} Y _ {k} + \eta \mathbf {b} _ {k + 1} - X _ {k} ^ {(n)} + \eta \mathbf {A} _ {k + 1} ^ {(n)} X _ {k} ^ {(n)} - \eta \mathbf {b} _ {k + 1} ^ {(n)} (26) \\ = \left(\mathrm {I} _ {d} - \eta \mathbf {A} _ {k + 1}\right) \left(Y _ {k} - X _ {k} ^ {(n)}\right) - \eta \left(\mathbf {A} _ {k + 1} - \mathbf {A} _ {k + 1} ^ {(n)}\right) X _ {k} ^ {(n)} + \eta \left(\mathbf {b} _ {k + 1} - \mathbf {b} _ {k + 1} ^ {(n)}\right). (27) \\ \end{array}
$$
Writing out the recursion, we can obtain:
$$
\begin{array}{l} Y _ {k + 1} - X _ {k + 1} ^ {(n)} = \left(Y _ {0} - X _ {0} ^ {(n)}\right) \prod_ {i = 0} ^ {k} \left(\mathrm {I} _ {d} - \eta \mathbf {A} _ {i + 1}\right) \\ - \eta \sum_ {i = 0} ^ {k} \left[ X _ {i} ^ {(n)} (\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}) \prod_ {j = i + 2} ^ {k + 1} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}) \right] \\ + \eta \sum_ {i = 0} ^ {k} \left[ (\mathbf {b} _ {i + 1} - \mathbf {b} _ {i + 1} ^ {(n)}) \prod_ {j = i + 2} ^ {k + 1} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}) \right], \\ \end{array}
$$
where for any sequence  $(a_{i})_{i\geq 0}$ , we let  $\prod_{i = j}^{k}a_{i} = 1$  when  $j > k$ . Now, taking the norm of both sides, using the triangle inequality, and taking expectations given  $E_{n}$ , we obtain:
$$
\begin{array}{l} \mathbb {E} [ \| Y _ {k + 1} - X _ {k + 1} ^ {(n)} \| ] \leq \mathbb {E} [ \| (Y _ {0} - X _ {0} ^ {(n)}) \prod_ {i = 0} ^ {k} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {i + 1}) \| ] \\ + \eta \mathbb {E} \left[ \sum_ {i = 0} ^ {k} \| X _ {i} ^ {(n)} \left(\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}\right) \prod_ {j = i + 2} ^ {k + 1} \left(\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}\right) \| \right] \tag {28} \\ + \eta \mathbb {E} [ \sum_ {i = 0} ^ {k} \| (\mathbf {b} _ {i + 1} - \mathbf {b} _ {i + 1} ^ {(n)}) \prod_ {j = i + 2} ^ {k + 1} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}) \| ]. \\ \end{array}
$$
We shall now analyze each of the three summands separately. For the first term in (28), as we start with  $Y_0 = X_0^{(n)}$ , it equals zero. For the second term in (28), we have:
$$
\begin{array}{l} \eta \mathbb {E} [ \sum_ {i = 0} ^ {k} \| X _ {i} ^ {(n)} (\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}) \prod_ {j = i + 2} ^ {k + 1} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}) \| ] \\ \stackrel {\mathrm {(a)}} {=} \eta \sum_ {i = 0} ^ {k} \mathbb {E} \left[ \left\| X _ {i} ^ {(n)} \left(\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}\right) \prod_ {j = i + 2} ^ {k + 1} \left(\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}\right) \right\| \right] \\ \stackrel {(b)} {\leq} \eta \sum_ {i = 0} ^ {k} \mathbb {E} [ \| X _ {i} ^ {(n)} (\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}) \| ] \mathbb {E} ^ {k - i} [ \| \mathbf {I} _ {d} - \eta \mathbf {A} _ {1} \| ] \\ \stackrel {\mathrm {(c)}} {\leq} \eta \sum_ {i = 0} ^ {k} (\mathbb {E} [ \| X _ {i} ^ {(n)} \| ^ {q} ]) ^ {\frac {1}{q}} (\mathbb {E} [ \| \mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)} \| ^ {p} ]) ^ {\frac {1}{p}} \mathbb {E} ^ {k - i} [ \| \mathbf {I} _ {d} - \eta \mathbf {A} _ {1} \| ], \\ \end{array}
$$
where (a) follows from the linearity of expectation, (b) follows as  $\mathbf{A}_1,\ldots ,\mathbf{A}_{i - 1}$  are i.i.d. and independent from  $\mathbf{A}_i,\dots ,\mathbf{A}_{k + 1}$ , and (c) is obtained by applying the Hölder's inequality, where  $p$  and  $q$  are such that  $1 / p + 1 / q = 1$ , with  $p,q\in (1, + \infty)$ ,  $q < \alpha$  and  $p > d / 2$  (used later in the proof), where  $\alpha$  is the tail index.
As the chain  $(X_{k}^{(n)})_{k\geq 0}$  starts from its stationary distribution  $\pi^{(n)}$ , we have that  $(\mathbb{E}[\| X_i^{(n)}\|^q])^{1 / q} = (\mathbb{E}[\| X_0^{(n)}\|^q])^{1 / q},\forall i$ . Using the fact that  $(\mathbf{A}_i - \mathbf{A}_i^{(n)})_{i = 1}^{k + 1}$  are i.i.d. random variables (the randomness arises from the empirical measure  $\mu_{\mathbf{A}}^{(n)}$ ),  $\mathbb{E}[\| \mathrm{I}_d - \eta \mathbf{A}_1\|^{\alpha}] = 1$  and  $\alpha >1$ , we have that:
$$
\delta_ {A} := \mathbb {E} [ \| \mathrm {I} _ {d} - \eta \mathbf {A} _ {1} \| ] <   1. \tag {29}
$$
Therefore, we can obtain the following bound:
$$
\begin{array}{l} \eta \mathbb {E} \left[ \sum_ {i = 0} ^ {k} \| X _ {i} ^ {(n)} \left(\mathbf {A} _ {i + 1} - \mathbf {A} _ {i + 1} ^ {(n)}\right) \prod_ {j = i + 2} ^ {k + 1} \left(\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}\right) \| \right] \\ \leq \eta (\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]) ^ {\frac {1}{q}} (\mathbb {E} [ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} ]) ^ {\frac {1}{p}} \sum_ {i = 0} ^ {k} \mathbb {E} ^ {k - i} [ \| \mathbf {I} _ {d} - \eta \mathbf {A} _ {1} \| ] \\ \leq \eta (\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]) ^ {\frac {1}{q}} (\mathbb {E} [ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} ]) ^ {\frac {1}{p}} \frac {1}{1 - \delta_ {A}}, \\ \end{array}
$$
where the last equality follows from (29) and the power series sum formula. Using the same arguments, we can bound the third term in (28):
$$
\eta \mathbb {E} [ \sum_ {i = 0} ^ {k} \| (\mathbf {b} _ {i + 1} - \mathbf {b} _ {i + 1} ^ {(n)}) \prod_ {j = i + 2} ^ {k + 1} (\mathrm {I} _ {d} - \eta \mathbf {A} _ {j}) \| ] \leq \eta \mathbb {E} [ \| \mathbf {b} _ {1} - \mathbf {b} _ {1} ^ {(n)} \| ] \frac {1}{1 - \delta_ {A}}.
$$
Combining the above, we can obtain that:
$$
\begin{array}{l} \mathbb {E} [ \| Y _ {k + 1} - X _ {k + 1} ^ {(n)} \| ] \leq \eta (\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]) ^ {\frac {1}{q}} \left(\mathbb {E} [ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} ]\right) ^ {\frac {1}{p}} \frac {1}{1 - \delta_ {A}} + \eta \mathbb {E} [ \| \mathbf {b} _ {1} - \mathbf {b} _ {1} ^ {(n)} \| ] \frac {1}{1 - \delta_ {A}} \\ = \frac {\eta}{1 - \delta_ {A}} \left(\left(\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]\right) ^ {\frac {1}{q}} \left(\mathbb {E} [ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} ]\right) ^ {\frac {1}{p}} + \mathbb {E} [ \| \mathbf {b} _ {1} - \mathbf {b} _ {1} ^ {(n)} \| ]\right) \\ \leq \frac {\eta}{1 - \delta_ {A}} \left(\left(\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]\right) ^ {\frac {1}{q}} \left(\mathbb {E} [ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} ]\right) ^ {\frac {1}{p}} + \left(\mathbb {E} [ \| \mathbf {b} _ {1} - \mathbf {b} _ {1} ^ {(n)} \| ^ {p} ]\right) ^ {\frac {1}{p}}\right) \\ = \frac {\eta}{1 - \delta_ {A}} \left(\left(\mathbb {E} \left[ \| X _ {0} ^ {(n)} \| ^ {q} \right]\right) ^ {\frac {1}{q}} \mathbf {W} _ {p} \left(\mu_ {\mathbf {A}}, \mu_ {\mathbf {A}} ^ {(n)}\right) + \mathbf {W} _ {p} \left(\mu_ {\mathbf {b}}, \mu_ {\mathbf {b}} ^ {(n)}\right)\right) (30) \\ \leq \frac {\eta}{1 - \delta_ {A}} \left(\left(\left(\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]\right) ^ {\frac {1}{q}} + 1\right) \mathbf {W} _ {p} \left(\mu_ {\mathbf {A}, \mathbf {b}}, \mu_ {\mathbf {A}, \mathbf {b}} ^ {(n)}\right)\right), (31) \\ \end{array}
$$
where the penultimate inequality follows from Jensen's inequality: it implies that  $\mathbf{W}_p(\mu, \nu) \leq \mathbf{W}_q(\mu, \nu)$ , for  $p \leq q$  (see, e.g., [Vil09]). The last inequality follows as  $\mathbf{W}_p(\mu_{\mathbf{A}}, \mu_{\mathbf{A}}^{(n)}) \leq \mathbf{W}_p(\mu_{\mathbf{A}, \mathbf{b}}, \mu_{\mathbf{A}, \mathbf{b}}^{(n)})$ , and  $\mathbf{W}_p(\mu_{\mathbf{b}}, \mu_{\mathbf{b}}^{(n)}) \leq \mathbf{W}_p(\mu_{\mathbf{A}, \mathbf{b}}, \mu_{\mathbf{A}, \mathbf{b}}^{(n)})$ . Together with (25), we have:
$$
\mathbf {W} _ {1} (\pi , \pi^ {(n)}) \leq ((\mathbb {E} [ \| X _ {0} ^ {(n)} \| ^ {q} ]) ^ {\frac {1}{q}} + 1) (1 - c _ {\rho} \mathrm {e} ^ {- \rho k}) ^ {- 1} \frac {\eta}{1 - \delta_ {A}} \mathbf {W} _ {p} (\mu_ {\mathbf {A}, \mathbf {b}}, \mu_ {\mathbf {A}, \mathbf {b}} ^ {(n)}).
$$
By taking the limit as  $k\to \infty$  , we can conclude the proof of Theorem 4.
Proof. Using results from Theorem 4, we now proceed to bound the Wasserstein- $p$  distance between the probability laws of  $\mathbf{A}_1$  and  $\mathbf{A}_1^{(n)}$ . Reminding ourselves that each element in the i.i.d. sequence  $(a_k)_{k \geq 1}$  follows a  $\mathrm{N}(0, \sigma^2 \mathrm{I}_d)$ , we denote this measure with  $\mu_a$ . Correspondingly, we denote its empirical measure with  $\mu_a^{(n)} = n^{-1} \sum_{j=1}^n \delta_{a_j}$ . Given  $E_n$ , we let  $(a_k, a_k^{(n)})_{k \geq 1}$  be the sequence of i.i.d. optimal couplings for  $\mu_a$  and  $\mu_a^{(n)}$ , so that by construction, for any  $k \in \mathbb{N}$ ,  $(\mathbb{E}[\|a_k - a_k^{(n)}\|^{2p}])^{1/2p} = \mathbf{W}_{2p}(\mu_a, \mu_a^{(n)})$ . Now:
$$
\begin{array}{l} \mathbf {W} _ {p} (\boldsymbol {\mu_ {A}}, \boldsymbol {\mu_ {A} ^ {(n)}}) \leq (\mathbb {E} [ \| \frac {1}{b} \sum_ {i \in \Omega_ {1}} a _ {i} a _ {i} ^ {\top} - \frac {1}{b} \sum_ {j \in \Omega_ {1} ^ {(n)}} a _ {j} a _ {j} ^ {\top} \| ^ {p} ]) ^ {1 / p} \\ \stackrel {(a)} {\leq} \frac {1}{b} b (\mathbb {E} [ \| (a _ {1} a _ {1} ^ {\top} - a _ {1} ^ {(n)} a _ {1} ^ {(n) \top}) \| ^ {p} ]) ^ {1 / p} \\ = \left(\mathbb {E} [ \| a _ {1} a _ {1} ^ {\top} - a _ {1} a _ {1} ^ {(n) \top} + a _ {1} a _ {1} ^ {(n) \top} - a _ {1} ^ {(n)} a _ {1} ^ {(n) \top} \| ^ {p} ]\right) ^ {1 / p} \\ \stackrel {(b)} {\leq} \left(\mathbb {E} \left[ \| a _ {1} a _ {1} ^ {\top} - a _ {1} a _ {1} ^ {(n) \top} \| ^ {p} \right]\right) ^ {1 / p} + \left(\mathbb {E} \left[ \| a _ {1} a _ {1} ^ {(n) \top} - a _ {1} ^ {(n)} a _ {1} ^ {(n) \top} \| ^ {p} \right]\right) ^ {1 / p} \\ = (\mathbb {E} [ \| a _ {1} \| ^ {p} \| a _ {1} - a _ {1} ^ {(n)} \| ^ {p} ]) ^ {1 / p} + (\mathbb {E} [ \| a _ {1} ^ {(n)} \| ^ {p} \| a _ {1} - a _ {1} ^ {(n)} \| ^ {p} ]) ^ {1 / p} \\ \stackrel {\mathrm {(c)}} {\leq} \left(\left(\mathbb {E} [ \| a _ {1} \| ^ {2 p} ]\right) ^ {1 / 2 p} + \left(\mathbb {E} [ \| a _ {1} ^ {(n)} \| ^ {2 p} ]\right) ^ {1 / 2 p}\right) \left(\mathbb {E} [ \| a _ {1} - a _ {1} ^ {(n)} \| ^ {2 p} ]\right) ^ {1 / 2 p} \\ \end{array}
$$
where (a) follows from Minkowski's inequality and the fact that  $(a_{k})_{k\geq 1}$  are i.i.d. random variables, as well as  $(a_k^{(n)})_{k\geq 1}$ . Inequality (b) follows due to Minkowski's inequality, and (c) follows using the generalized Hölder's inequality with  $1 / 2p + 1 / 2p = 1 / p$ .
Now, we can proceed to utilize Lemma 2 and (30). In order to apply Lemma 2 to the difference of measures between  $\mathbf{A}_1$  and  $\mathbf{A}_1^{(n)}$  (i.e., between  $a_1$  and  $a_1^{(n)})$ , as  $\mathbb{E}[\| X_0^{(n)}\| ^q ]^{1 / q} < \infty$  for all  $q < \alpha$ , we require  $q < \alpha$  and  $1 / p + 1 / q = 1$  (due to Hölder's inequality).
First, we select  $p$  large and  $q$  small enough for both  $q < \alpha$  and  $p > d / 2$ . Now, in order to satisfy the requirements of Lemma 2, we set  $\epsilon^* := \sqrt{\frac{1}{nC_1} \log \frac{4c_1}{\zeta}}$ . This choice allows us to obtain that, with probability greater than  $1 - \zeta / 2$ ,  $(\mathbb{E}[||a_1 - a_1^{(n)}||^{2p}])^{1/2p} = \mathbf{W}_{2p}(\mu_a, \mu_a^{(n)}) < \epsilon^*$ . Therefore, we have that, with probability greater than  $1 - \zeta / 2$ :
$$
(\mathbb {E} [ \| a _ {1} - a _ {1} ^ {(n)} \| ^ {2 p} ]) ^ {1 / 2 p} \leq \sqrt {\frac {1}{n C _ {1}} \log \frac {4 c _ {1}}{\zeta}} = \mathcal {O} (n ^ {- 1 / 2}),
$$
where the positive constants  $C_1$  and  $c_{1}$  depend only on  $p,d,\mu_{a_1}$  . Therefore, with probability greater than  $1 - \zeta /2$  ..
$$
\begin{array}{l} \left(\mathbb {E} \left[ \| \mathbf {A} _ {1} - \mathbf {A} _ {1} ^ {(n)} \| ^ {p} \right]\right) ^ {1 / p} \leq \left(\left(\mathbb {E} \left[ \| a _ {1} \| ^ {2 p} \right]\right) ^ {1 / 2 p} + \left(\mathbb {E} \left[ \| a _ {1} ^ {(n)} \| ^ {2 p} \right]\right) ^ {1 / 2 p}\right) \sqrt {\frac {1}{n C _ {1}} \log \frac {4 c _ {1}}{\zeta}} \\ = C _ {2} n ^ {- 1 / 2} \sqrt {\log \frac {c _ {3}}{\zeta}}, \tag {32} \\ \end{array}
$$
with  $C_2 = \sqrt{\frac{1}{C_1}}\left(\left(\mathbb{E}\left[\|a_1\|^{2p}\right]\right)^{1/2p} + \left(\mathbb{E}\left[\|a_1^{(n)}\|^{2p}\right]\right)^{1/2p}\right)$  and  $c_3 = 4c_1$ .
In order to bound  $(\mathbb{E}[\| \mathbf{b}_k - \mathbf{b}_k^{(n)}\|^p])^{1/p}$ , we can apply Lemma 2 again to obtain constants  $C_4$  and  $c_5$  such that, with probability greater than  $1 - \zeta/2$ :
$$
\left(\mathbb {E} \left[ \| \mathbf {b} _ {k} - \mathbf {b} _ {k} ^ {(n)} \| ^ {p} \right]\right) ^ {1 / p} \leq C _ {4} n ^ {- 1 / 2} \sqrt {\log \frac {c _ {5}}{\zeta}} = \mathcal {O} \left(n ^ {- 1 / 2}\right). \tag {33}
$$
Using (32), (33) with Theorem 4 (see (25) and (30)), and denoting  $C_6 = C_2(\mathbb{E}[\| X_0^{(n)}\|^q])^{1/q} + C_4$  and  $c_7 = \max \{c_3, c_5\}$  we can obtain, with probability greater than  $1 - \zeta$ :
$$
\mathbf {W} _ {1} \left(\pi , \pi^ {(n)}\right) \leq \frac {\eta}{1 - \delta} \left(C _ {6} \sqrt {\log \frac {c _ {7}}{\zeta}}\right) n ^ {- 1 / 2}. \tag {34}
$$
Therefore, we obtain that, with probability greater than  $1 - \zeta$ ,  $\mathbf{W}_1(\pi, \pi^{(n)}) = \mathcal{O}(n^{-1/2})$ .
Due to [GSZ21] we have  $\lim_{t\to \infty}t^{\alpha}\mathbb{P}\left(\| X_{\infty}\| >t\right)\in (0,\infty)$ . Now, let  $\epsilon >0$ . Then, there exists  $t_0$  s.t. for all  $t\geq t_0$ ,  $|t^{\alpha}\mathbb{P}(\| X_{\infty}\| >t) - c|\leq \epsilon$ , for some  $c\in (0,\infty)$ . From (34), we know that with probability greater than  $1 - \zeta$ , there exist constants  $\tilde{c}_1,\tilde{c}_2$  such that  $\mathbf{W}_1(\pi ,\pi^{(n)})\leq \tilde{c}_1\sqrt{\log\frac{\tilde{c}_2}{\zeta}} n^{-1 / 2}$ .
Now, using Lemma 3, we can obtain that, with probability greater than  $1 - \epsilon_{n} - \zeta$ :
$$
\begin{array}{l} \tilde {c _ {1}} \sqrt {\log \frac {\tilde {c _ {2}}}{\zeta}} \frac {1}{n ^ {1 / 2}} \geq \mathbf {W} _ {1} (\pi , \pi^ {(n)}) \\ \geq \int_ {0} ^ {\infty} | \mathbb {P} (\| X _ {\infty} \| > t) - \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t) | d t \\ \geq \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} | \mathbb {P} (\| X _ {\infty} \| > t) - \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t) | d t, \\ \end{array}
$$
where we have used that  $\lim_{t\to \infty}\mathbb{P}(\| X_{\infty}\| >t) = 0$  and  $\lim_{t\to \infty}\mathbb{P}(\| X_{\infty}^{(n)}\| >t) = 0$ , in order to select  $t^\prime$  and  $t''$  large enough for the last inequality to hold. Now, we have that
$$
\begin{array}{l} \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} | \mathbb {P} (\| X _ {\infty} \| > t) - \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t) | \mathrm {d} t \geq \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} \frac {c - \epsilon}{t ^ {\alpha}} \mathrm {d} t - \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t) \mathrm {d} t \\ \geq (t ^ {\prime \prime} - t ^ {\prime}) \frac {(c - \epsilon)}{(t ^ {\prime \prime}) ^ {\alpha}} - (t ^ {\prime \prime} - t ^ {\prime}) \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t ^ {\prime}). \\ \end{array}
$$
Therefore, by choosing  $t'' = 2t'$ , we can obtain:
$$
\mathbb {P} \left(\left\| X _ {\infty} ^ {(n)} \right\| > t ^ {\prime}\right) \geq \frac {c - \epsilon}{2 ^ {\alpha} \left(t ^ {\prime}\right) ^ {\alpha}} - \frac {\tilde {c} _ {1}}{t ^ {\prime} n ^ {1 / 2}} \sqrt {\log \frac {\tilde {c} _ {2}}{\zeta}}. \tag {35}
$$
Similarly, for any  $\epsilon > 0$  and  $t > t_0$ , we have  $t^{\alpha} \mathbb{P}(\|X_{\infty}\| > t) \leq \epsilon + c$ , and using similar arguments we obtain:
$$
\begin{array}{l} \tilde {c _ {1}} \sqrt {\log \frac {\tilde {c _ {2}}}{\zeta}} \frac {1}{n ^ {1 / 2}} \geq \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t) \mathrm {d} t - \int_ {t ^ {\prime}} ^ {t ^ {\prime \prime}} \frac {c + \epsilon}{t ^ {\alpha}} \mathrm {d} t \\ \geq (t ^ {\prime \prime} - t ^ {\prime}) \mathbb {P} (\| X _ {\infty} ^ {(n)} \| > t ^ {\prime \prime}) - (t ^ {\prime \prime} - t ^ {\prime}) \frac {c + \epsilon}{(t ^ {\prime}) ^ {\alpha}}. \\ \end{array}
$$
Finally, choosing as before  $t'' = 2t'$ , we can obtain:
$$
\mathbb {P} \left(\| X _ {\infty} ^ {(n)} \| > 2 t ^ {\prime}\right) \leq \frac {c + \epsilon}{\left(t ^ {\prime}\right) ^ {\alpha}} + \frac {\tilde {c} _ {1}}{t ^ {\prime} n ^ {1 / 2}} \sqrt {\log \frac {\tilde {c} _ {2}}{\zeta}}. \tag {36}
$$
Substituting for  $\bar{t} = 2t'$ , we can obtain:
$$
\mathbb {P} \left(\| X _ {\infty} ^ {(n)} \| > \bar {t}\right) \leq \frac {2 ^ {\alpha} (c + \epsilon)}{\bar {t} ^ {\alpha}} + \frac {2 \tilde {c} _ {1}}{\bar {t} n ^ {1 / 2}} \sqrt {\log \frac {\tilde {c} _ {2}}{\zeta}}. \tag {37}
$$
Combining (35) and (37), we obtain that with probability greater than  $1 - \zeta - \epsilon_{n}$ , for any  $\epsilon > 0$  and  $t > t_0$ :
$$
\mathbb {P} \left(\left\| X _ {\infty} ^ {(n)} \right\| > t\right) \geq \frac {1}{2 ^ {\alpha}} \frac {c - \epsilon}{t ^ {\alpha}} - \frac {\tilde {c} _ {1}}{t n ^ {1 / 2}} \sqrt {\log \frac {\tilde {c} _ {2}}{\zeta}}, \text {a n d} \tag {38}
$$
$$
\mathbb {P} \left(\left\| X _ {\infty} ^ {(n)} \right\| > t\right) \leq 2 ^ {\alpha} \frac {c + \epsilon}{t ^ {\alpha}} + \frac {2 \tilde {c} _ {1}}{t n ^ {1 / 2}} \sqrt {\log \frac {\tilde {c} _ {2}}{\zeta}}. \tag {39}
$$
This concludes the proof of Theorem 3.
Proof of results in Section 3.2
Let us now consider a setting where  $f(\cdot, z)$  is not necessarily a quadratic function. As before, we assume the data comes from an unknown data distribution  $\mu_z$  on  $\mathbb{R}^{d+1}$ . Furthermore, with  $(\hat{Z}_k^{(n)})_{k \geq 1}$ , we denote the i.i.d. random variables associated with the empirical measure  $\mu_z^{(n)} = n^{-1} \sum_{j=1}^n \delta_{\hat{Z}_j^{(n)}}$ , as defined in (15). We consider the case  $b = 1$  for simplicity.
C.3 Proof of Theorem 6
Proof. First, we utilize the tail index limits from [HM21] in the strongly convex setting from Theorem 2. We use  $\alpha$  and  $\beta$  in order to construct the proof and provide the same argument as in Section 3.1.
Denote by  $E_{n}$  the event on which  $(X_{k}^{(n)})_{k\geq 0}$  has a stationary distribution  $\pi^{(n)}$  such that  $\mathbb{P}(E_n)\geq 1 - \epsilon_n$ . Given  $E_{n}$ , let  $(Z_{k},\hat{Z}_{k}^{(n)})_{k\geq 1}$  be the sequences of i.i.d. optimal couplings for  $\mu_z$  and  $\mu_z^{(n)}$ . Therefore, by construction, for any  $k\in \mathbb{N}$ ,  $\mathbb{E}[\| Z_k - \hat{Z}_k^{(n)}\| ] = \mathbf{W}_1(\mu_z,\mu_z^{(n)})$ . Based on this sequence, we consider the processes  $(X_{k})_{k\geq 0}$ ,  $(X_{k}^{(n)})_{k\geq 0}$ ,  $(Y_{k})_{k\geq 0}$  defined by the recursions:
1.  $X_{k + 1} = X_k - \eta \nabla f(X_k,Z_{k + 1})$ , where  $X_0\sim \pi$
2.  $X_{k + 1}^{(n)} = X_k^{(n)} - \eta \nabla f(X_k^{(n)},\hat{Z}_{k + 1}^{(n)})$  , where  $X_0^{(n)}\sim \pi^{(n)}$
3.  $Y_{k + 1} = Y_k - \eta \nabla f(Y_k,Z_{k + 1})$  where  $Y_0 = X_0^{(n)}$ ,
with the unique stationary distributions of  $(X_{k})_{k\geq 0}$  and  $(X_{k}^{(n)})_{k\geq 0}$  being denoted by  $\pi$  and  $\pi^{(n)}$  respectively. Note again that  $(X_{k})_{k\geq 0}$  corresponds to the online SGD recursion (2), and  $(X_{k}^{(n)})_{k\geq 0}$  to the offline SGD recursion (4), both with  $b = 1$ . Under our assumptions, by the definition of geometric ergodicity, there exist constants  $c_{\rho} > 0$  and  $\rho \in (0,1)$  such that the following inequality holds:
$$
\mathbf {W} _ {1} \left(\mathcal {L} \left(Y _ {k}\right), \pi\right) \leq c _ {\rho} \mathbf {W} _ {1} \left(\mathcal {L} \left(Y _ {0}\right), \pi\right) \mathrm {e} ^ {- \rho k}, \text {f o r a n y} k \geq 0. \tag {40}
$$
Analogously to (25), we can conclude:
$$
\mathbf {W} _ {1} \left(\pi , \pi^ {(n)}\right) \leq \left(1 - c _ {\rho} \mathrm {e} ^ {- \rho k}\right) ^ {- 1} \mathbf {W} _ {1} \left(\mathscr {L} \left(Y _ {k}\right), \mathscr {L} \left(X _ {k} ^ {(n)}\right)\right). \tag {41}
$$
The proof then utilizes Taylor's expansion with the remainder part in integral form (see, e.g., [SCW20]) in the following way:
$$
\nabla f \left(Y _ {k}, z\right) - \nabla f \left(X _ {k} ^ {(n)}, z\right) = \mathbf {H} _ {k, z} ^ {(n)} \left(Y _ {k} - X _ {k} ^ {(n)}\right) \tag {42}
$$
with  $\mathbf{H}_{k,z}^{(n)}\coloneqq \int_0^1\nabla^2 f(Y_k - u(Y_k - X_k^{(n)}),z)\mathrm{d}u$  . Therefore: