SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Mohammad Reza Karimi*
mkarimi@inf.ethz.ch
Ya-Ping Hsieh*
yaping.hsieh@inf.ethz.ch
krausea@ethz.ch
Non-convex sampling is a key challenge in machine learning, central to non-convex optimization in deep learning as well as to approximate probabilistic inference. Despite its significance, theoretically there remain some important challenges: Existing guarantees suffer from the drawback of lacking guarantees for the last-iterates, and little is known beyond the elementary schemes of stochastic gradient Langevin dynamics. To address these issues, we develop a novel framework that lifts the above issues by harnessing several tools from the theory of dynamical systems. Our key result is that, for a large class of state-of-the-art sampling schemes, their last-iterate convergence in Wasserstein distances can be reduced to the study of their continuous-time counterparts, which is much better understood. Coupled with standard assumptions of MCMC sampling, our theory immediately yields the last-iterate Wasserstein convergence of many advanced sampling schemes such as mirror Langevin, proximal, randomized mid-point, and Runge-Kutta methods.
Many modern learning tasks involve sampling from a high-dimensional density  $\pi \propto e^{-f}$ , where  $f$  is a non-convex potential representing, for instance, the loss function of a deep neural network. To this end, an approach that has found wide success is to discretize the continuous-time Langevin diffusion
$$
\mathrm {d} L _ {t} = - \nabla f (L _ {t}) \mathrm {d} t + \sqrt {2} \mathrm {d} B _ {t} \tag {LD}
$$
where  $B_{t}$  is a Brownian motion [57]. The idea behind this approach is that, since  $\pi$  is the stationary distribution of (LD), one can expect a similar behavior for discretizations of (LD). Such a framework has inspired numerous sampling schemes with per-iteration costs as cheap as stochastic gradient descent, which are particularly suitable for large-scale approximate probabilistic inference and Bayesian learning [2, 54, 57]. Moreover, several works have noticed that these Langevin-based schemes provide deep insights about minimizing  $f$  using stochastic oracles [22, 48], which serves as an important step toward explaining the empirical success of training deep neural networks.
The convergence of Langevin-based non-convex sampling has therefore attracted significant interest from both practitioners and theoreticians, whose intense study has led to a plethora of new guarantees; see related work for details. Despite such impressive progress, several challenges remain for the fully non-convex setup:
- The convergence is typically given on the averaged iterates instead of the more natural last iterates [4, 54]. This is especially problematic from the perspective of understanding the minimization of  $f$ , as in practice, the last iterates of an optimization algorithm play the most pivotal role for downstream tasks.
- An additional notable drawback of the current theory is its predominant focus on the basic Euler-Maruyama discretization of (LD) (see, e.g., [4, 20, 54]). As a result, the convergence analysis of more advanced sampling schemes remains largely unexplored in the fully non-convex regime [1, 24, 25, 34, 53, 61].
$\S$  Contributions and Approaches. To overcome the aforementioned challenges, our main contribution, from a high level, can be succinctly summarized as:
Under mild assumptions, we prove that the iterates of a broad range of Langevin-based sampling schemes converge to the continuous-time (LD) in Wasserstein distance.  $(\star)$
Combining  $(\star)$  with classical results on Langevin diffusion [45] immediately yields the last-iterate convergence in Wasserstein distances for a wide spectrum of sampling schemes, thus resolving all the challenges mentioned above. To illustrate this point, we state a simple version of our main result.
Theorem (Informal). Suppose we discretize (LD) as
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} (\nabla f (x _ {k}) + \text {n o i s e} + \text {b i a s}) + \sqrt {2 \gamma_ {k + 1}} \xi_ {k + 1}
$$
with step-sizes  $\{\gamma_k\}_{k\in \mathbb{N}}$  and i.i.d. standard Gaussians  $\{\xi_k\}_{k\in \mathbb{N}}$ . Then, under an easy-to-verify condition on the bias (see (5) in Assumption 3),  $\{x_{k}\}_{k\in \mathbb{N}}$  converges in Wasserstein distance to  $\pi$ . In addition, these conditions are satisfied by many advanced sampling schemes.
This result is achieved via a new dynamical perspective to study Langevin-based sampling. More specifically,
1. We introduce the Picard process, which is the sampling analogue of Picard's method of successive approximations for solving ODEs [17]. Contrary to most existing analyses, the Picard process allows us to completely bypass the use of relative entropy, which is the culprit for the appearance of averaged iterates [20].
2. Using the Picard process, we will prove that the iterates of various Langevin-based schemes generate a so-called Wasserstein asymptotic pseudotrajectory (WAPT) for the continuous-time (LD). The main motivation for considering WAPT is to connect Langevin-based schemes to the dynamical system theory of Benaim and Hirsch [7], which works for metric spaces and is last-iterate by design, and therefore particularly suitable for our purpose.
3. Finally, under standard stability assumptions in the literature [39, 51], we show how a tandem of our WAPT result and dynamical system theory yields the desirable convergence of various existing schemes, as well as motivates more efficient algorithms that enjoy the same rigorous guarantees.
§ Related work. There is a vast literature on structured non-convex sampling, where one imposes extra assumptions on the target density. Under these conditions, one can derive non-asymptotic rates for Langevin-based schemes [13, 15, 33, 35, 36, 41, 48, 56, 60, 63]. Our work is orthogonal to these works as we study generic non-convex sampling, an NP-hard problem whose convergence is asymptotic at best.
Most relevant to our paper are the works [4, 8, 20, 29, 54], which study the asymptotic convergence of Langevin-based schemes under minimal regularity assumptions on  $f$ . Compared to their results, our guarantees either improve upon existing ones or are incomparable; see Section 5.4 for a more detailed comparison.
2 The Langevin-Robbins-Monro Template
We consider the following general template for sampling algorithms: Starting from an initial point, the iterates  $\{x_{k}\}_{k\in \mathbb{N}}$  follow the recursion
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} \left\{v \left(x _ {k}\right) + Z _ {k + 1} \right\} + \sqrt {2 \gamma_ {k + 1}} \sigma \left(x _ {k}\right) \xi_ {k + 1}, \tag {LRM}
$$
where  $\gamma_{k}$ 's are step sizes,  $v$  is a vector field,  $Z_{k}$ 's are (random or deterministic) perturbations,  $\sigma$  is the state-dependent diffusion matrix, and  $\xi_{k}$ 's are i.i.d. standard Gaussian random variables. In the sequel, we will further decompose the perturbation as  $Z_{k} = U_{k} + b_{k}$ , where  $U_{k}$  is the (zero-mean)
noise and  $b_{k}$  is the bias. We call this recursion the Langevin-Robbins-Monro (LRM) template, as it is reminiscent of the Robbins-Monro template for stochastic approximation [50].
The generality of the LRM template allows us to capture many existing algorithms and suggests ways to design new ones. For illustration purposes, we showcase instances of (LRM) with the following examples. Other examples (SGLD and proximal) are provided in Appendix A. In the first three examples, the vector field  $v$  in (LRM) is  $-\nabla f$  and  $\sigma \equiv 1$ .
Example 1. The Randomized Mid-Point Method [24, 53] is an alternative discretization scheme to Euler-Maruyama and has been proposed for both overdamped and underdamped Langevin diffusion. For the overdamped case, its iterates are
$$
x _ {k + 1 / 2} = x _ {k} - \gamma_ {k + 1} \alpha_ {k + 1} \widetilde {\nabla} f (x _ {k}) + \sqrt {2 \gamma_ {k + 1} \alpha_ {k + 1}} \xi_ {k + 1} ^ {\prime}, \tag {RMM}
$$
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} \bar {\nabla} f \left(x _ {k + 1 / 2}\right) + \sqrt {2 \gamma_ {k + 1}} \xi_ {k + 1},
$$
where  $\{\alpha_k\}$  are i.i.d. and uniformly distributed in  $[0,1]$ ,  $\xi_k,\xi_k'$  are standard Gaussian random variables with cross-variance  $\sqrt{\alpha_k} I$ , and  $\widetilde{\nabla} f$  is a noisy evaluation of  $\nabla f$ . To cast (RMM) in the LRM template, we set  $U_{k + 1}\coloneqq \widetilde{\nabla} f(x_{k + 1 / 2}) - \nabla f(x_{k + 1 / 2})$  and  $b_{k + 1}\coloneqq \nabla f(x_{k + 1 / 2}) - \nabla f(x_k)$ .
Example 2. Inspecting the update rule of (RMM), we see that it requires two gradient oracle calls at each iteration. Inspired by the optimistic gradient methods in optimization and online learning [16, 47, 49], we propose to "recycle" the past gradients:
$$
x _ {k + 1 / 2} = x _ {k} - \gamma_ {k + 1} \alpha_ {k + 1} \widetilde {\nabla} f \left(x _ {k - 1 / 2}\right) + \sqrt {2 \gamma_ {k + 1} \alpha_ {k + 1}} \xi_ {k + 1} ^ {\prime}, \tag {ORMM}
$$
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} \widetilde {\nabla} f \left(x _ {k + 1 / 2}\right) + \sqrt {2 \gamma_ {k + 1}} \xi_ {k + 1},
$$
where  $\{\alpha_k\}, \xi_k, \xi_k'$ , and  $\widetilde{\nabla} f$  are the same as in (RMM). This is again an LRM scheme with  $U_{k+1} \coloneqq \widetilde{\nabla} f(x_{k+1/2}) - \nabla f(x_{k+1/2})$  and  $b_{k+1} \coloneqq \nabla f(x_{k+1/2}) - \nabla f(x_k)$ .
Notice that (ORMM) requires one gradient oracle, thereby reducing the per-iteration cost of (RMM) by 2. To our knowledge, the scheme (ORMM) is new.
Example 3. In addition to the simple (stochastic) Euler-Maruyama discretization in (SGLD), there exists a class of more sophisticated discretization methods of (LD) known as higher-order integrators. The Stochastic Runge-Kutta method [34] is an example of an order 1.5 integrator, with iterates
$$
h _ {1} = x _ {k} + \sqrt {2 \gamma_ {k + 1}} \left(c _ {1} \xi_ {k + 1} + c _ {2} \xi_ {k + 1} ^ {\prime}\right)
$$
$$
h _ {2} = x _ {k} - \gamma_ {k + 1} \widetilde {\nabla} f (x _ {k}) + \sqrt {2 \gamma_ {k + 1}} \left(c _ {3} \xi_ {k + 1} + c _ {2} \xi_ {k + 1} ^ {\prime}\right),
$$
$$
x _ {k + 1} = x _ {k} - \frac {\gamma_ {k + 1}}{2} \left(\widetilde {\nabla} f \left(h _ {1}\right) + \widetilde {\nabla} f \left(h _ {2}\right)\right) + \sqrt {2 \gamma_ {k + 1}} \xi_ {k + 1},
$$
where  $\xi_{k + 1}$  and  $\xi_{k + 1}^{\prime}$  are independent standard Gaussian random variables, and  $c_{1},c_{2},c_{3}$  are suitably chosen integrator constants. This algorithm is an LRM scheme with  $U_{k + 1}\coloneqq \frac{1}{2} (\widetilde{\nabla} f(h_1) - \nabla f(h_1)) + \frac{1}{2} (\widetilde{\nabla} f(h_2)) - \nabla f(h_2))$  and  $b_{k + 1}\coloneqq \frac{1}{2} (\nabla f(h_1) + \nabla f(h_2)) - \nabla f(x_k)$ .
Example 4. The Mirror Langevin algorithm [1, 25, 61], which is the sampling analogue of the celebrated mirror descent scheme in optimization [5, 43], uses a strongly convex function  $\phi$  to adapt to a favorable local geometry. In the dual space (i.e., the image of  $\nabla \phi$ ), its iterates follow
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} \nabla f \left(\nabla \phi^ {*} \left(x _ {k}\right)\right) + \sqrt {2 \gamma_ {k + 1}} \left(\nabla^ {2} \phi^ {*} \left(x _ {k}\right) ^ {- 1}\right) ^ {1 / 2} \xi_ {k + 1}, \tag {ML}
$$
where  $\phi^{*}$  is the Fenchel dual of  $\phi$  [52]. In our framework, (ML) fits into (LRM) by taking  $v = -\nabla f \circ \nabla \phi^{*}$  and  $\sigma = (\nabla^2 \phi^{*})^{-1/2}$ . Additionally, one can also consider a stochastic version of (ML) with noisy evaluations of  $\nabla f$ .
3 Technique Overview: A Dynamical System Perspective
The goal of our paper is to provide last-iterate guarantees for the general LRM schemes introduced in Section 2. There are two equivalent, commonly considered, ways of characterizing the dynamics of the iterates of an LRM scheme. The first one is to view the iterates  $\{x_{k}\}_{k\in \mathbb{N}}$  as a random trajectory in  $\mathbb{R}^d$ , which is perhaps the most natural way of describing a sampling algorithm. The second way is to view the distributions  $\{\rho_k\}_{k\in \mathbb{N}}$  of  $\{x_{k}\}_{k\in \mathbb{N}}$  as a deterministic trajectory in the Wasserstein space.
With these two characterizations in mind, in this section, we will devise a new framework based on the dynamical system theory and present its high-level ideas.
To understand our novelty, it is important to contrast our framework to the existing Wasserstein viewpoint towards Langevin-based sampling algorithms. Following the seminal work of Otto [44], one can view a sampling algorithm as the discretization of a class of well-studied dynamical systems—gradient flows. This viewpoint suggests using Lyapunov arguments, which has become the predominant approach in much prior work.
Despite its appealing nature, in the rest of this section, we will argue that Lyapunov analysis of gradient flows is in fact not suited for studying generic non-convex sampling. In particular, we will show how our new framework is motivated to overcome the several important limitations of gradient flow analysis. Finally, we give a high-level overview of the techniques used in our paper.
§ Langevin Diffusion as Gradient Flows. We denote by  $\rho_t$  the probability density of  $L_t$  in (LD), and consider the continuous curve  $t \mapsto \rho_t$  in the Wasserstein space  $\mathbb{W}_2$ . In their seminal works, Jordan et al. [27] and Otto [44] discover that this curve is the (exact) gradient flow of the relative entropy functional; that is, defining the functional  $F: \rho \mapsto D_{\mathrm{KL}}(\rho \| e^{-f})$ , one has  $\partial_t \rho_t = -\operatorname{grad} F(\rho_t)$ , where "grad" is the gradient in the Wasserstein sense. This gradient flow viewpoint of (LD) thus provides a clear link between sampling in  $\mathbb{R}^d$  and optimization in  $\mathbb{W}_2$ . Indeed, this suggests that the relative entropy is a natural choice for the Lyapunov function of the discrete-time sampling algorithm, which is a prominent approach for analyzing sampling algorithms in recent years [4, 21, 58].
Although the gradient flow viewpoint has led to a sequence of breakthroughs, it has a few important shortcomings:
(a) The usual Lyapunov-type analysis for sampling algorithms focuses on bounding the change in relative entropy across iterations. This is extremely challenging when one considers more advanced sampling algorithms, as one has to understand the effect of the additive bias and noise of the algorithm on the change of relative entropy. Crucially, this makes the Lyapunov analysis applicable only to the simple Euler-Maruyama discretization of (LD),<sup>2</sup> i.e.,  $x_{k + 1} = x_k - \gamma_{k + 1}\nabla f(x_k) + \sqrt{2\gamma_{k + 1}}\xi_{k + 1}$ , and fails to capture more advanced and biased sampling schemes such as Examples 1-4. Even for the simple (SGLD), the presence of stochastic gradients significantly complicates the Lyapunov analysis and requires extra assumptions such as convexity [21] or uniform spectral gap [48].
(b) This gradient flow-based analysis often requires an extra averaging step to decrease the relative entropy (see, e.g., [4]). This is the main reason why many existing works provide guarantees only on the averaged iterates  $(\bar{\rho}_k \coloneqq \frac{1}{k} \sum_{i=1}^k \rho_i)$  instead of the last ones  $(\rho_k)$ .
In this paper, we overcome these limitations by introducing a new perspective, whose two ingredients are as follows.
$\S$  Wasserstein Asymptotic Pseudotrajectories. A notion that will play a pivotal role in our analysis is the Wasserstein asymptotic pseudotrajectory (WAPT), which is a measure of "asymptotic closeness" in the Wasserstein sense, originally defined by Benaim and Hirsch [7] for metric spaces:
Definition 1 (Wasserstein asymptotic pseudotrajectory). We say the stochastic process  $(X_{t})_{t\geq 0}$  is a Wasserstein asymptotic pseudotrajectory (WAPT) of the SDE
$$
\mathrm {d} \Phi_ {t} = v (\Phi_ {t}) \mathrm {d} t + \sigma (\Phi_ {t}) \mathrm {d} B _ {t} \tag {SDE}
$$
if, for all  $T > 0$
$$
\lim  _ {t \rightarrow \infty} \sup  _ {0 \leq s \leq T} W _ {2} \left(X _ {t + s}, \Phi_ {s} ^ {(t)}\right) = 0. \tag {1}
$$
Here,  $\Phi_s^{(t)}$  is the solution of the SDE at time  $s$  initialized at  $X_{t}$ , and  $W_{2}$  is the 2-Wasserstein distance.
Despite the seemingly convoluted definition, WAPT can be intuitively understood as follows: Let  $\{x_{k}\}_{k\in \mathbb{N}}$  be the iterates of a sampling scheme. Then, (1) simply posits that for sufficiently large
2While the Lyapunov-type analysis has been applied to elementary (i.e., unbiased) discretization schemes for other SDEs, such as the under-damped (i.e., kinetic) Langevin dynamics [19], our primary focus in this paper remains centered on the over-damped Langevin diffusion and similar SDEs.
Figure 1: High-level overview of the two components of the dynamical perspective.
$m$ , one cannot distinguish between the "tail" iterates  $\{x_{k}\}_{k\geq m}$  versus the SDE solution starting at  $x_{m}$ , up to arbitrarily small error measured in terms of the Wasserstein distance. Since we are only interested in the asymptotic behavior of  $x_{k}$ , these controls on the tail iterates will suffice to conclude the last-iterate convergence. $^3$
Importantly, from the perspective of WAPT, the Langevin diffusion (LD) (or more generally,  $\Phi_s^{(t)}$ ) is simply viewed as a generic dynamical system and not as a gradient flow. In particular, relative entropy will play no role throughout our analysis, thereby resolving issue (b).
§ Langevin-Robbins-Monro Schemes. We have seen that the LRM template in Section 2 is capable of capturing a broad range of existing and new algorithms in a unified way. To resolve the remaining issue (a), we will further rely on the LRM template: for proving that (LRM) generates a WAPT of the corresponding SDE, we show that the key condition (1) in WAPT can be reduced to checking an easy-to-verify bound on the perturbation terms  $Z_{k}$ .
To achieve this, the most important step in our proof, which distinguishes our analysis from all existing works in non-convex sampling, is the construction of the so-called Picard process, the natural generalization of the Picard's successive approximation method [17] from ordinary differential equations to stochastic differential equations. In the stochastic approximation literature, similar techniques have been successfully applied to study optimization and games in various settings such as on Riemannian or primal-dual spaces [26, 28, 37]. The application to sampling has also been previously explored by Bubeck et al. [11], Chau et al. [12] in different contexts. What distinguishes our work from the existing literature is the advantage of generalizing the Picard process to encompass a vastly wider class of algorithms, specifically the LRM schemes. Moreover, the integration of the Picard process with the theory of WAPT plays a pivotal role in our analysis, and both of these aspects present original contributions.
$\S$  Framework overview. To conclude, for proving last-iterate convergence, we proceed as follows:
1. For a given LRM scheme  $\{x_{k}\}_{k\in \mathbb{N}}$ , we first construct a continuous-time trajectory  $(X_{t})_{t\geq 0}$  via interpolating the iterates (see (3)).
2. We prove that  $(X_{t})$  constitutes a WAPT of the SDE (see Theorem 1). This step relies heavily on the construction of the aforementioned Picard process.
3. By invoking the dynamical system theory of Benaim and Hirsch [7], the convergence of LRM schemes reduces to simply checking the stability condition (Theorem 2). In the Wasserstein space, this condition translates into boundedness of the second moments of the iterates  $\{x_{k}\}$ , for which there is a plethora of approaches; we present two such methods in Section 5.
Fig. 1 depicts a high-level overview of the ingredients needed in our framework, and their corresponding theorems.
4 The Dynamics of Langevin-Robbins-Monro Schemes
In this section, we view (LRM) as a noisy and biased discretization of (LD). To make this analogy precise, let  $(B_{t})_{t\geq 0}$  be a Brownian motion defined on a filtered probability space with filtration
Definition 1 is phrased in terms of a continuous-time stochastic process  $(X_{t})_{t\geq 0}$ . The discrete iterates  $\{x_{k}\}_{k\in \mathbb{N}}$  can be converted to a continuous-time process through a suitable interpolation; see (3).
$(\mathcal{F}_t)_{t \geq 0}$ , and define  $\tau_k = \sum_{n=1}^k \gamma_n$  to be the effective time that has elapsed at the iteration  $k$ . Using the Brownian motion, we can rewrite (LRM) as
$$
x _ {k + 1} = x _ {k} - \gamma_ {k + 1} \left\{v \left(x _ {k}\right) + Z _ {k + 1} \right\} + \sigma \left(x _ {k}\right) \left(B _ {\tau_ {k + 1}} - B _ {\tau_ {k}}\right), \tag {2}
$$
assuming that the filtration satisfies  $Z_{k}\in \mathcal{F}_{\tau_{k}}$  4 The (continuous-time) interpolation  $(X_{t})_{t\geq 0}$  of  $\{x_{k}\}_{k\in \mathbb{N}}$  is then defined as the adapted process
$$
X _ {t} = x _ {k} - (t - \tau_ {k}) \left\{v \left(x _ {k}\right) + \mathbb {E} \left[ Z _ {k + 1} \mid \mathcal {F} _ {t} \right] \right\} + \sigma \left(x _ {k}\right) \left(B _ {t} - B _ {\tau_ {k}}\right), \quad \text {f o r} t \in \left[ \tau_ {k}, \tau_ {k + 1} \right]. \tag {3}
$$
In addition, for a fixed  $t$ , consider the Brownian motion  $(B_s^{(t)})_{s\geq 0}$  where  $B_{s}^{(t)}\coloneqq B_{t + s} - B_{t}$ , and define the Langevin flow  $(\Phi_s^{(t)})_{s\geq 0}$  as the (strong) solution of (SDE) initialized at  $X_{t}$ . It is important to note that  $\Phi^{(t)}$  and  $X$  are synchronously coupled by sharing the same Brownian motion.
4.1 Technical Assumptions and Requirements
We now introduce the basic technical assumptions and discuss their generality.
Assumption 1. The vector field  $v$  is  $L$ -Lipschitz, and satisfies  $\langle x, v(x) \rangle \leq C_v (1 + \|x\|)$  for some  $C_v > 0$ . Moreover,  $\sigma$  is  $L$ -Lipschitz and is bounded in Hilbert-Schmidt norm.
Lipschitzness of  $v$  is a standard assumption and is also required to ensure the existence of a unique strong solution of (SDE). The second assumption on the vector field is exceedingly weak and when  $v = -\nabla f$ , is satisfied even for distributions without moments. The assumptions on diffusion coefficient  $\sigma$  are already satisfied when  $\sigma \equiv 1$ , and we show that it holds for practical schemes such as Example 4.
Assumption 2. The Robbins-Monro summability conditions hold:  $\sum_{k=1}^{\infty} \gamma_k = \infty$  and  $\sum_{k=1}^{\infty} \gamma_k^2 < \infty$ . Moreover, for some constant  $P$  to be defined in (20), we have
$$
\gamma_ {k + 1} / \gamma_ {k} + P \gamma_ {k} \gamma_ {k + 1} <   1 - \gamma_ {k}, \quad \forall k. \tag {4}
$$
The Robbins-Monro step size conditions are standard in the non-convex sampling literature [4, 20, 29, 30]. For (4), it can be verified that condition is satisfied even for slowly-decreasing step sizes such as  $\gamma_{k} \propto (\sqrt{k}\log k)^{-1}$ , which hence is not restrictive.
Assumption 3. The noises  $\{U_k\}_{k \in \mathbb{N}}$  form a martingale difference sequence, i.e.,  $\mathbb{E}[U_{k+1} | U_k] = 0$  and have uniformly bounded second moments. In addition, the bias terms satisfy
$$
\mathbb {E} \left[ \| b _ {k + 1} \| ^ {2} \mid \mathscr {F} _ {\tau_ {k}} \right] = \mathcal {O} \left(\gamma_ {k + 1} ^ {2} \| v (x _ {k}) \| ^ {2} + \gamma_ {k + 1}\right). \tag {5}
$$
A martingale difference sequence is more general than an i.i.d. sequence, allowing the noise to be state-dependent. The bias condition (5) simply states that the bias shall not overpower the signal  $v(x_{k})$ , and, as we show later, is satisfied by all our examples.
4.2 From Discrete to Continuous: LRM Schemes and WAPTs
We are now in a position to state our main theorems. Our first result below establishes a precise link between the discrete-time (LRM) and the continuous-time (SDE).
Theorem 1. Under Assumptions 1-3, the interpolation (3) of an LRM scheme is a Wasserstein asymptotic pseudotrajectory of (SDE).
§ Sketch of the Proof for Theorem 1. The proof of this theorem is heavily based on the notion of the Picard process and iterate moment bounds. The complete proof can be found in Appendix C.