SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Nic Fishman
Leo Klarner
Emile Mathieu
Michael Hutchinson
ENS Ulm
{njwfish,leojklarner}@gmail.com
Denoising diffusion models have recently emerged as the predominant paradigm for generative modelling on image domains. In addition, their extension to Riemannian manifolds has facilitated a range of applications across the natural sciences. While many of these problems stand to benefit from the ability to specify arbitrary, domain-informed constraints, this setting is not covered by the existing (Riemannian) diffusion model methodology. Recent work has attempted to address this issue by constructing novel noisng processes based on the reflected Brownian motion and logarithmic barrier methods. However, the associated samplers are either computationally burdensome or only apply to convex subsets of Euclidean space. In this paper, we introduce an alternative, simple noisng scheme based on Metropolis sampling that affords substantial gains in computational efficiency and empirical performance compared to the earlier samplers. Of independent interest, we prove that this new process corresponds to a valid discretisation of the reflected Brownian motion. We demonstrate the scalability and flexibility of our approach on a range of problem settings with convex and non-convex constraints, including applications from geospatial modelling, robotics and protein design.
In recent years, denoising diffusion models (Sohl-Dickstein et al., 2015; Song et al., 2019; Song et al., 2021; Ho et al., 2020) have emerged as a powerful paradigm for generative modelling, achieving state-of-the-art performance across a range of domains. They work by progressively adding noise to data following a Stochastic Differential Equation (SDE)—the forward noising process—until it is close to the invariant distribution of the SDE. The generative model is then given by an approximation of the associated time-reversed denoising process, which is also an SDE whose drift depends on the gradient of the logarithmic densities of the forward process, referred to as the Stein score. Building on the success of diffusion models for image generation tasks, De Bortoli et al. (2022) and Huang et al. (2022) have recently extended this framework to a wide range of Riemannian manifolds, enabling the specification of inherent structural properties of the modelled domain. This has broadened the applicability of diffusion models to problems in the natural and engineering sciences, including the conformational modelling of small molecules (Jing et al., 2022; Corso et al., 2022), proteins (Tripe et al., 2022; Watson et al., 2022; Yim et al., 2023) and robotic platforms (Urain et al., 2022).
However, in many data-scarce or safety-critical settings, researchers may want to restrict the modelled domain even further by specifying problem-informed constraints to make maximal use of limited experimental data or prevent unwanted behaviour (Morris, 2002; Han et al., 2006; Thiele et al., 2013; Lukens et al., 2020). As illustrated in Figure 1, such domain-informed constraints can be naturally represented as a Riemannian manifold with boundary. Training diffusion models on such constrained manifolds is thus an important problem that requires principled noising processes—and corresponding discretisations—that stay within the constrained set.
(a)  $\mathbb{R}^3$
(b)  $\mathcal{S}^2\subset \mathbb{R}^3$
Figure 1: When operating in data-scarce settings, it may often be beneficial to specify as much prior knowledge of the modelled domain as possible. Consider a distribution over a subset  $\mathcal{M}$  of the unit sphere  $S^2\subset \mathbb{R}^3$ . While a Euclidean diffusion model can approximate the distribution in  $\mathbb{R}^3$  (a), directly modelling it on  $S^2$  can make learning significantly easier (b). Restricting the problem space even further by only constructing a diffusion model on  $\mathcal{M}$  can lead to even better performance (c).
(c)  $\mathcal{M}\subset \mathcal{S}^2\subset \mathbb{R}^3$
Recent work by Fishman et al. (2023) has attempted to derive such processes and extend the applicability of diffusion models to inequality-constrained manifolds by investigating the generative modelling applications of classic sampling schemes based on log-barrier methods (Kannan et al., 2009; Lee et al., 2017; Noble et al., 2022; Kook et al., 2022; Gatmiry et al., 2022; Lee et al., 2018) and the reflected Brownian motion (Williams, 1987; Petit, 1997; Shkolnikov et al., 2013). While empirically promising, the proposed algorithms can be computationally and numerically burdensome, and require bespoke implementations for different manifolds and constraints. Concurrently, Lou et al. (2023) have investigated the use of reflected diffusion models for image applications. They focus on the high-dimensional hypercube, as this setting admits a theoretically grounded treatment of the static thresholding method which is widely used in image models such as Sahara et al. (2022). More recently, Liu et al. (2023) have investigated the use of log-barrier-based mirror maps to transform a constrained domain into an unconstrained dual space for applications in image watermarking. While both methods exhibit robust scaling properties and impressive results, they only consider convex subsets of Euclidean space and do not extend to more general manifolds.
Here, we propose a new method for generative modelling on constrained manifolds based on a Metropolis-based discretisation of the reflected Brownian motion. The Metropolised process' chief advantage is that it is lightweight: the only additional requirement over those outlined in De Bortoli et al. (2022) that is needed to implement a constrained diffusion model is an efficient binary function that indicates whether any given point is within the constrained set. This Metropolised approximation of the reflected Brownian motion is substantially easier to implement, faster to compute and more numerically stable than the previously considered sampling schemes. Our core theoretical contribution is to show that this new discretisation converges to the reflected SDE by using the invariance principle for SDEs with boundary (Stroock et al., 1971). To the best of our knowledge, this is the first time that such a process has been investigated. We demonstrate that our method attains improved empirical results on diverse manifolds with convex and non-convex constraints by applying it to a range of problems from geospatial modelling, robotics and protein design.
Riemannian manifolds. A Riemannian manifold is defined as a tuple  $(\mathcal{M},\mathfrak{g})$  with  $\mathcal{M}$  a smooth manifold and  $\mathfrak{g}$  a metric defining an inner product on tangent spaces. In this work, we will use the exponential map  $\exp_x:\mathrm{T}_x\mathcal{M}\to \mathcal{M}$ , as well as the extension of the gradient  $\nabla$ , divergence div and Laplace  $\Delta$  operators to  $\mathcal{M}$ . All of these quantities can be defined in local coordinates in terms of the metric. The extension of the Laplace operator to  $\mathcal{M}$  is called the Laplace-Beltrami operator, also denoted  $\Delta$  when there is no ambiguity. Using  $\Delta$ , we can define a Brownian motion on  $\mathcal{M}$ , denoted  $(\mathbf{B}_t)_{t\geq 0}$  and with density w.r.t. the volume form of  $\mathcal{M}$  denoted  $p_t$  for any  $t > 0$ . We refer to Appendix B for a more detailed exposition, to Lee (2013) for a thorough treatment of Riemannian manifolds and to Hsu (2002) for details on stochastic analysis on manifolds. In the following, we consider a constrained manifold  $\mathcal{M}$  defined by
$$
\mathcal {M} = \{x \in \mathcal {N}: f _ {i} (x) <   0, i \in \mathcal {I} \}, \tag {1}
$$
Figure 2: Visual comparison of a discretisation of the unconstrained Brownian motion (blue) and two discretisations of the reflected Brownian motion: one based on a reflection scheme (green) and the other based on our Metropolis sampler (red). The Metropolised trajectory is very close to that of the reflected one while being significantly easier to implement and cheaper to compute.
where  $(\mathcal{N},\mathfrak{g})$  is a Riemannian manifold,  $\mathcal{I}$  is an arbitrary finite indexing family and for any  $i\in \mathcal{I}$ ,  $f_{i}\in \mathrm{C}(\mathcal{N},\mathbb{R})$ . Since  $\mathcal{I}$  is finite and  $f_{i}$  continuous for any  $i\in \mathcal{I}$ ,  $\mathcal{M}$  is an open set of  $\mathcal{N}$  and inherits its metric  $\mathfrak{g}$ . This captures simple Euclidean polytopes and complex constrained spaces like Figure 1.
Denoising diffusion models. Denoising diffusion models (Song et al., 2019; Ho et al., 2020; Song et al., 2021) work as follows: let  $(\mathbf{X}_t)_{t\in [0,T]}$  be a noisig process that corrupts the original data distribution  $p_0$ . We assume that  $(\mathbf{X}_t)_{t\geq 0}$  converges to  $\mathrm{N}(0,\sigma^2\mathrm{Id})$ , with  $\sigma >0$ . Several such processes exist, but in practice we consider the Ornstein-Uhlenbeck (OU) process, also referred to as VP-SDE, which is defined by the following Stochastic Differential Equation (SDE)
$$
\mathrm {d} \mathbf {X} _ {t} = - \frac {1}{2} \mathbf {X} _ {t} \mathrm {d} t + \sigma \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} \sim p _ {0}. \tag {2}
$$
Under conditions on  $p_0$ , for any  $T > 0$ ,  $(\mathbf{Y}_t)_{t \in [0,T]} = (\mathbf{X}_{T - t})_{t \in [0,T]}$  is also the (weak) solution to a SDE (Anderson, 1982; Haussmann et al., 1986; Cattiaux et al., 2021)
$$
\mathrm {d} \mathbf {Y} _ {t} = \left\{\frac {1}{2} \mathbf {Y} _ {t} + \sigma^ {2} \nabla \log p _ {T - t} (\mathbf {Y} _ {t}) \right\} \mathrm {d} t + \sigma \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {Y} _ {0} \sim p _ {T}, \tag {3}
$$
where  $p_t$  denotes the density of  $\mathbf{X}_t$ . In practice,  $\nabla \log p_t$  is approximated with a score network  $(t,x) \mapsto s_\theta(t,x)$  trained by minimising either a denoising score matching (dsm) loss or an implicit score matching (ism) loss (Vincent, 2011)
$$
\ell (\theta) = \mathbb {E} _ {t \sim \mathcal {U} ([ 0, T ]), \left(\mathbf {X} _ {0}, \mathbf {X} _ {t}\right)} \left[ \lambda_ {t} \left(\frac {1}{2} \| s _ {\theta} (t, \mathbf {X} _ {t}) \| ^ {2} + \operatorname {d i v} \left(s _ {\theta}\right) (t, \mathbf {X} _ {t})\right) \right], \tag {4}
$$
where  $\lambda_t > 0$ . For a flexible score network, the global minimiser  $\theta^{\star} = \operatorname{argmin}_{\theta} \mathcal{L}(\theta)$  satisfies  $s_{\theta^{\star}}(t,\cdot) = \nabla \log p_t$ . De Bortoli et al. (2022) and Huang et al. (2022) have extended denoising diffusion models to the Riemannian setting. The time-reversal formula (3) remains the same, replacing the Euclidean gradient with its Riemannian equivalent. The ism loss can still be computed in that setting. However, the samplers used in the Riemannian setting differ from the classical Euler-Maruyama discretisation used in the Euclidean framework. De Bortoli et al. (2022) use Geodesic Random Walks (Jorgensen, 1975), which ensure that the samples remain on the manifold at every step. In this paper, we propose a sampler with similar properties in the case of constrained manifolds.
Reflected SDE. We conclude this section by recalling the framework for studying reflected SDEs, which is introduced via the notion of the Skorokhod problem. For simplicity, we focus on Euclidean space  $\mathbb{R}^d$  here, but note that reflected processes can be defined on arbitrary smooth manifolds  $\mathcal{N}$ . In the case of the Brownian motion, a solution to the Skorokhod problem is a process of the form  $(\bar{\mathbf{B}}_t,\mathbf{k}_t)_{t\geq 0}$ . Locally,  $(\bar{\mathbf{B}}_t)_{t\geq 0}$  can be seen as a regular Brownian motion  $(\mathbf{B}_t)_{t\geq 0}$  while  $(\mathbf{k}_t)_{t\geq 0}$  forces  $(\bar{\mathbf{B}}_t)_{t\geq 0}$  to remain in  $\mathcal{M}$ . Under mild additional regularity conditions on  $\mathcal{M}$  and  $(\bar{\mathbf{B}}_t,\mathbf{k}_t)_{t\geq 0}$ , see Skorokhod (1961),  $(\bar{\mathbf{B}}_t,\mathbf{k}_t)_{t\geq 0}$  is a solution to the Skorokhod problem if for any  $t\geq 0$
$$
\bar {\mathbf {B}} _ {t} = \bar {\mathbf {B}} _ {0} + \mathbf {B} _ {t} - \mathbf {k} _ {t} \in \mathcal {M}, \tag {5}
$$
$\left|\mathbf{k}\right|_t = \int_0^t\mathbf{1}_{\bar{\mathbf{B}}_s\in \partial \mathcal{M}}\mathrm{d}|\mathbf{k}|_s$  and  $\mathbf{k}_t = \int_0^t\mathbf{n}(\bar{\mathbf{B}}_s)\mathrm{d}|\mathbf{k}|_s$ , where  $(|\mathbf{k}|_{t})_{t\geq 0}$  is the total variation of  $(\mathbf{k}_t)_{t\geq 0}^1$ . Let us provide some intuition on this definition. When  $(\bar{\mathbf{B}}_t)_{t\geq 0}$  hits the boundary  $\partial \mathcal{M}$ ,  $-\mathbf{k}_t$  pushes the process back into  $\mathcal{M}$  along the inward normal  $-\mathbf{n}(\bar{\mathbf{B}}_t)$ , according to  $\mathbf{k}_t = \int_0^t\mathbf{n}(\bar{\mathbf{B}}_s)\mathrm{d}|\mathbf{k}|_s$ .
<sup>1</sup>In this case  $(\mathbf{k}_t)_{t\geq 0}$  is not regular enough, but if it were in the class  $C^1$ , its total variation would be given by  $\int_0^t |\partial_t\mathbf{k}_t|\mathrm{d}s$  in the one-dimensional case.
(a) Reflected Brownian motion
(b) Metropolised approximation of Brownian motion
Figure 3: Evolution of the density of the reflected Brownian motion and its Metropolis sampling-based approximation on the unit interval starting from a delta mass.
The condition  $|\mathbf{k}|_t = \int_0^t\mathbf{1}_{\bar{\mathbf{B}}_s\in \partial \mathcal{M}}\mathrm{d}|\mathbf{k}|_s$  is more technical and can be seen as imposing that  $\mathbf{k}_t$  remains constant so long as  $(\bar{\mathbf{B}}_t)_{t\geq 0}$  does not hit  $\partial \mathcal{M}$ . We refer to Fishman et al. (2023) and Lou et al. (2023) for a more thorough introduction of these concepts in the context of diffusion models.
3 Diffusion models for constrained manifolds via Metropolis sampling
In Section 3.1, we highlight the practical limitations of existing constrained diffusion models and propose an alternative Metropolis sampling-based approach. In Section 3.2, we outline our proof that this process corresponds to a valid discretisation of the reflected Brownian motion, justifying its use in diffusion models. An overview of the samplers we cover in this section is presented in Table 1.
3.1 Practical limitations of existing constrained diffusion models
Barrier methods. In the barrier approach, a constrained manifold is transformed into an unconstrained space via a barrier metric. This metric is defined by  $\nabla^2\phi (x)$  with  $\phi (x) = \sum_{i\in \mathcal{I}}\phi_i(d(x,f_i))$  where  $d(x,f_{i})$  is the minimum distance from the point  $x$  to the set defined by  $f_{i}(x) = 0$  and  $\phi_{i}$  is a monotonically decreasing function such that  $\lim_{z\to 0}\phi_i(z) = \infty$ . Under additional regularity assumptions,  $\phi$  is called a barrier function (see Nesterov et al. (1994)). This definition ensures that the barrier function induces a well-defined exponential map on the manifold, making the Riemannian diffusion model frameworks of De Bortoli et al. (2022) and Huang et al. (2022) applicable. In the log-barrier method of Fishman et al. (2023), evaluating  $\phi$  requires computing  $d(x,\partial \mathcal{M})$  (and its derivatives), which can be prohibitively expensive. Furthermore, since the exponential map under the induced manifold is difficult to compute, it is approximated by projecting the exponential map on the original manifold back onto the constraint set, incurring an additional bias. Liu et al. (2023) propose a more tractable method by constructing a mirror map that transforms a constrained domain into an unconstrained dual space, in which one can train a standard Euclidean diffusion model. However, this approach is only applicable to convex subsets of  $\mathbb{R}^d$  and does not extend to arbitrary Riemannian manifolds. More generally, warping the geometry of the modelled domain can adversely impact the interpolative performance of log-barrier-based diffusion models, as the space between data points expands rapidly when approaching the boundary.
Reflected stochastic processes. Fishman et al. (2023) and Lou et al. (2023) introduce diffusion models based on the reflected Brownian motion (RBM). In Fishman et al. (2023), the reflected SDE is discretised by (i) considering a classical step of the Euler-Maruyama discretization (or the Geodesic Random Walk in the Riemannian setting) and (ii) reflecting this step according to the boundary defined by  $\partial \mathcal{M}$ . To compute the reflection, one must check whether the step crosses the boundary. If it does, the point of intersection needs to be calculated in order to reflect the ray and continue the step in the reflected direction. This can require an arbitrarily large number of reflections depending on the step size, the geodesic on the manifold, and the geometry of the bounded region within the manifold. We refer to Appendix C for the pseudocode of the reflection step and additional comments. An alternative approach to discretising a reflected SDE is to replace the reflection with a projection (Slominski, 1994). However, the projection requires the most expensive part of the reflection algorithm: computing the intersection of the geodesic with the boundary. Lou et al. (2023) propose a more tractable approach that exploits the product structure of the unit hypercube to afford simulation-free sampling but does not extend to arbitrary Riemannian manifolds. Additionally, specifying convex constraints in their framework requires a bijection onto the hypercube, distorting the modelled geometry and incurring the same issues as outlined above.
Table 1: Comparison of the advantages and disadvantages of the different constrained (Riemannian) diffusion models covered in Section 3.1.
Both required for fast DSM loss
DIFFUSION MODEL	TRACTABLE
CONDITIONAL SCORE	SIMULATION-FREE
FORWARD SAMPLING	MODELLING
DOMAIN	PRESERVES
METRIC OF M
Reflected Diffusions
LOU ET AL. (2023)	✓	✓	convex⊂Rd	X
FISHMAN ET AL. (2023)	X	O(d2)	Any M	✓
METROPOLIS (OURS)	X	O(d)	Any M	✓
Barrier Diffusions
FISHMAN ET AL. (2023)	X	X	convex⊂any M	X
LIU ET AL. (2023)	✓	✓	convex⊂Rd	X
Metropolis approximation. Existing approaches to constrained (Riemannian) diffusion models either only apply to convex subsets of  $\mathbb{R}^d$  or require manifold- and constraint-specific implementations that become computationally intractable as the complexity of the modelled geometry increases. This limits their practicality even for relatively simple manifolds with well-defined exponential maps and linear inequality constraints such as for example polytopes.
In the following, we introduce a method that aims to solve both of these problems. The sampler we propose is similar to a classical Euler-Maruyama discretisation of the Brownian motion, except that, whenever a step would carry the Brownian motion out of the constrained region, we reject it (see Algorithm 1). This is a Metropolised version of the usual discretisation and is trivial to implement compared to the existing barrier, reflection and projection methods. Hence, this method enables the principled extension of diffusion models to arbitrarily constrained manifolds at virtually no added implementational complexity or computational expense.
Algorithm 1 Metropolis approx. of RBM
Require:  $p\in \mathcal{M}$ $\{f_i\}_{i\in \mathcal{I}}$  Sample  $\pmb {v}\sim \mathrm{N}(0,\mathrm{Id})\in \mathrm{T}_p\mathcal{M}$ $p^\prime \gets \exp_p(\pmb {v})$  if  $f_{i}(p^{\prime}) <   0\forall i$  then  $p\gets p^{\prime}$  end if return  $p$
3.2 Relating the Metropolis sampler to the reflected Brownian motion
In this section, we prove that the proposed Metropolis sampling-based process (Algorithm 1) corresponds to a valid discretisation of the reflected process, justifying its use in diffusion models. Here we focus on a concise presentation of the core concepts and the main results. A full proof can be found in Appendix D. For simplicity, we restrict ourselves to the Euclidean setting. All of our results require particular assumptions on  $\mathcal{M}$ , which we discuss at the end of this section. We begin with a definition of the Metropolis approximation of RBM.
Definition 1. For any  $\gamma >0$  and  $k\in \mathbb{N}$ , let  $X_0^\gamma \in \mathcal{M}$  and  $X_{k + 1}^{\gamma} = X_{k}^{\gamma} + \sqrt{\gamma} Z_{k}^{\gamma}$  if  $X_{k}^{\gamma} + \sqrt{\gamma} Z_{k}^{\gamma}\in \mathcal{M}$  and  $X_{k}^{\gamma}$  otherwise. The sequence  $(X_k^\gamma)_{k\in \mathbb{N}}$  is called the Metropolis approximation of RBM.
For any  $\gamma > 0$ , we consider  $(\mathbf{X}_t^\gamma)_{t \geq 0}$ , the linear interpolation of  $(X_k^\gamma)_{k \in \mathbb{N}}$  such that for any  $k \in \mathbb{N}$ ,  $\mathbf{X}_{k\gamma}^\gamma = X_k^\gamma$ . The following result is the main theoretical contribution of our paper.
Theorem 2. Under assumptions on  $\mathcal{M}$ , for any  $T \geq 0$ ,  $(\mathbf{X}_t^\gamma)_{t \in [0, T]}$  weakly converges to the RBM  $(\bar{\mathbf{B}}_t)_{t \in [0, T]}$  as  $\gamma \to 0$ .
The rest of the section is devoted to a high level presentation of the proof of Theorem 2. It is theoretically impractical to work directly with the Metropolis approximation of RBM. Instead, we introduce an auxiliary process, show this converges to the RBM, and finally prove that the convergence of the auxiliary process implies the convergence of our Metropolis discretisation.
Definition 3. For any  $\gamma >0$  and  $k\in \mathbb{N}$ , let  $\hat{X}_0^\gamma = x\in \mathcal{M}$  and  $\hat{X}_{k + 1}^{\gamma} = \hat{X}_{k}^{\gamma} + \sqrt{\gamma} Z_{k}^{\gamma}$  with  $Z_{k}^{\gamma}$  a Gaussian random variable conditioned on  $\hat{X}_k^\gamma +\sqrt{\gamma} Z_k^\gamma \in \mathcal{M}$ . The sequence  $(\hat{X}_k^\gamma)_{k\in \mathbb{N}}$  is called the Rejection approximation of RBM.
We call this process Rejection approximation of RBM since in practice,  $Z_{k}^{\gamma}$  is sampled using rejection sampling, see Algorithm 2. For any  $\gamma >0$  we also consider  $(\hat{\mathbf{X}}_t^\gamma)_{t\geq 0}$ , the linear interpolation of  $(\hat{X}_k^\gamma)_{k\in \mathbb{N}}$  such that for any  $k\in \mathbb{N}$ ,  $\hat{\mathbf{X}}_{k\gamma}^{\gamma} = \hat{X}_{k}^{\gamma}$ . In Appendix D, we prove the following result.
Theorem 4. Under assumptions on  $\mathcal{M}$ , for any  $T \geq 0$ ,  $(\hat{\mathbf{X}}_t^\gamma)_{t \in [0,T]}$  weakly converges to the Reflected Brownian Motion  $(\bar{\mathbf{B}}_t)_{t \in [0,T]}$  as  $\gamma \to 0$ .
Algorithm 2 Rejection approx. of RBM
Require:  $p\in \mathcal{M}$ $\{f_i\}_{i\in I}$
Sample  $\pmb{v} \sim \mathrm{N}(0, \mathrm{Id}) \in \mathrm{T}_p\mathcal{M}$
$p^{\prime}\gets \exp_{p}(\boldsymbol {v})$
while  $f_{i}(p^{\prime})\geq 0$  for any  $i$  do
Sample  $\pmb{v} \sim \mathrm{Id}(0,1) \in \mathrm{T}_p\mathcal{M}$
return  $p^{\prime}$
Proof. Here we give some elements of the proof. Details and full derivations are postponed to Appendix D. Our approach is based on the invariance principle of Stroock et al. (1971). More precisely, we show that we can compute an equivalent 'drift' and 'diffusion matrix' for the discretised process and that, as  $\gamma \to 0$ , the drift converges to zero and the diffusion matrix converges to Id. In the Euclidean setting, this result, accompanied by mild regularity and growth assumptions, ensures that the discretization weakly converges to the original SDE. However, the case with boundary is much more complicated, primarily because the approximate drift might explode near the boundary, thus we need to verify exactly how the drift behaves as  $\gamma \to 0$  and as the process approaches the boundary. We show that the normalised drift converges to the inward normal near the boundary. This ensures that (a) in the interior of  $\mathcal{M}$  the drift converges to zero, i.e. locally in the interior of  $\mathcal{M}$  the Brownian motion and the Reflected Brownian Motion coincide, (b) on the boundary, the drift pushes the samples inside the manifold according to the inward normal, mimicking  $(\mathbf{k}_t)_{t\geq 0}$  in (5). Finally, with results from Stroock et al. (1971) and Kang et al. (2017), we show the convergence to the RBM.