Unbounded	General MC	Acceleration	Oracle complexity (Smooth and non-convex)	Oracle complexity (Smooth and strongly convex)
Method	Domain
i.i.d.	SGD 83170132	✓	X	N/A	X	O(L(f(x0)-f(x*) [1/ε+σ2/ε])	O(L/μ log||x0-x*||2+σ2/μxε)
ASGD 97113(1)	✓	✓	N/A	✓	O(L(f(x0)-f(x*) [1+ε2/ε2+σ2/ε])	O((1+δ2)/√L μ log||x0-x*||2+σ2/μxε)
Markovian	EMD 23(2)	X	X	✓	X	O(τG2D2/ε4)	X
MC SGD 20(3)	✓	X	X	X	O(h(G,L) [π/ε2]1/(1-q))	X
MC SGD 19(4)	✓	✓	✓	X	O(τL2(1+x*||2+||x0-x*||2))	O(eτ(L/μ)2 [h(L/μ) log||x0-x*||2+τL2(1+x*||2) / μ2ε]
ASGD 20(5)	X	X	X	✓	O(1/ε4 [B2+G0(L2τ2+1)])	O(√L/μ ||x0-x*||2+τ2(G2+μGD+μLD2/μ2ε)
MAG 21(6)	✓	X	✓	X	O(τ(G+L+L+B)2G2/ε4)	X
MC SGD 23(Sec. 5.1)(7)	✓	X	X	X	O(τ(L(f(x0)-f(x*))+σ2)/ε2+τ(L(f(x0)-f(x*))+σ2)σ2/ε4)	O(πL/μ log (f(x0)-f(x*))/μ+σ2/(μL)+σ2/μxε)
MC SGD 23(Sec. 5.2)(8)	✓	X	X	X	O(L/μ log||x0-x*||2+Lτ2/ε+σ2/μxε)	O(τ[1+δ2)/√L μ log||x0-x*||2+σ2/μxε)
RASGD (ours)	✓	✓	✓	✓	O(τL(f(x0)-f(x*)) [1+δ2/ε2+σ2/ε])	O(τ[1+δ2)/√L μ log||x0-x*||2+σ2/μxε)
notation:  $\mu$  and  $L$  are as in A1 and A2.  $G = \sup_{x,z} \| \nabla F(x,z) \|$ . Note that  $G \geq L$  and  $G^2 \geq \sigma^2$  under A4. We also set  $B = \sup_x |f(x)|$ ;  $x^0$  - starting point,  $x^*$  - solution,  $\mathcal{D}$  - optimisation domain;  $D = \sup_{x \in \mathcal{D}} \| x - x^* \|$ ,  $\sigma$  and  $\delta$ -stochasticity parameters (see A4);  $\sigma_*$  - stochasticity parameter in  $x^*$ ;  $\tau$  - mixing time of the chain (see A3);  $\varepsilon$  - accuracy of the solution, measured as  $\mathbb{E}[||\nabla f(x)||^2] \lesssim \varepsilon^2$  for non-convex problems and  $\mathbb{E}[||x - x^*||^2] \lesssim \varepsilon$  for the strongly convex ones. Functions  $h(L / \mu)$  and  $h(G,L)$  stands for an implicit dependence of the respective parameters.
(1) gives results with stepsize as a parameter, we choose it the close way as in our Corollary  $\underline{L}$ . (2) covers more general noise setting, then just Markovian. (3) for general state-space Markov noise the analysis of 90 requires reversibility. Parameter  $q\in (1 / 2;1)$  refers to the step size  $\sim 1 / k^q$ . (4) The fluctuation terms in [19] Theorem 1.3] contain hidden dependence on the initial error and  $\| x^{*}\|$  in the fluctuation terms, making the result comparison complicated. They also contain hidden factors, which are exponential in  $C = \tau /\log 4$  in the notations of our paper. Moreover, the analysis of 19 requires that  $F(x,z)$  is Lipschitz w.r.t.  $x$  for any  $z\in \mathbb{Z}$ . (5) considers Markovian noise with finite state space and a specifically decreasing step size. Moreover, in the proof of [20] Theorem 3] (equations (64) - (66)) the authors lost the factor  $C^2$ , with  $C = \tau /\log 4$ . The result in the table accounts for this lost factor. (6) considers the adaptive tuning of batch size, which is oblivious to  $\tau$ . (7) considers Markov noise with finite state space and additionally assumes that all stochastic realization  $F(\cdot ,Z)$  are  $L$  smooth. (8) considers Markovian noise with finite state space,  $\sigma_{*}$  bounds noise only in  $x^{*}$ , but additionally assumes that all stochastic realization  $F(\cdot ,Z)$  are  $L$  smooth and  $\mu$  strongly convex.