SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Parallel transport. We say that a vector field  $X$  is parallel to a curve  $\gamma : (0,1) \to \mathcal{M}$  if  $\nabla_{\gamma'}X = 0$ , where  $\gamma': (0,1) \to \mathrm{T}_{\gamma(t)}\mathcal{M}$ . For two points on the manifold  $p, q \in \mathcal{M}$  that are connected by a curve  $\gamma$ , and an initial vector  $X_0 \in \mathrm{T}_p\mathcal{M}$ , there is a unique vector field  $X$  that is parallel to  $\gamma$  such that  $X(p) = X_0$ . This induces a map between the tangent spaces at  $p$  and  $q$ $\tau_\gamma : \mathrm{T}_p\mathcal{M} \to \mathrm{T}_q\mathcal{M}$ , which is referred to as the parallel transport of tangent vectors between  $p$  and  $q$  and satisfies the condition that for  $\boldsymbol{v}, \boldsymbol{u} \in \mathrm{T}_p\mathcal{M}$ $\mathfrak{g}(p)(\boldsymbol{v},\boldsymbol{u}) = \mathfrak{g}(q)(\tau_\gamma(\boldsymbol{v}),\tau_\gamma(\boldsymbol{u}))$ .
Reflection. For an element  $\pmb{v} \in T_p\mathcal{M}$  in the tangent space of the manifold at point  $p$  and a constraint characterised by its unit normal vector  $\pmb{n} \in T_p\mathcal{M}$ , the reflection of  $\pmb{v}$  in the tangent space is given by  $\pmb{v}' = \pmb{v} - 2\mathfrak{g}(\pmb{v}, \pmb{n})\pmb{n}$ .
C Full Reflected Discretisation
Here, we reproduce the central algorithm for the full discretisation of the reflected Brownian motion (Algorithm 4) derived for Euclidean models in (Lou et al., 2023) and for Reimannian models in (Fishman et al., 2023). Its central component is the Reflected Step Algorithm (Algorithm 3), which gives a generic computation for the reflection in any manifold. Due to the need to balance speed and numerical instability issues around the boundary, an efficient practical implementation of the reflected step is highly non-trivial, even for simple polytopes in Euclidean space. More complex geometries and boundaries make this problem significantly worse: a constraint on the trace of SPD matrices under the log-Cholesky metric of (Lin, 2019) requires solving complex non-convex optimisation problems for each sample at each discretised sampling step in both the forward and reverse process. This motivates our work in this paper.
These problems motivated the development of our Metropolis approximation, which significantly simplifies the random walk. Instead of requiring the intersection, parallel transport and reflection, we simply need to be able to evaluate the constraint functions  $f_{i}$ . We highlight this simplicity in Algorithm 5.
Algorithm 3 Reflected Step Algorithm. The algorithm operates by repeatedly taking geodesic steps until one of the constraints is violated, or until the step is fully taken. Upon hitting the boundary, we parallel-transport the tangent vector to the boundary and then reflect it against it. We then start a new geodesic from this point in the new direction. The arg intersect $_t$  function computes the distance one must travel along a geodesic in direction  $s$  until constraint  $f_i$  is violated. For a discussion of paralleltransport,  $\exp_{\mathfrak{g}}$  and reflect see appendix B.
Require:  $x\in \mathcal{M},\pmb {v}\in \mathrm{T}_x\mathcal{M},\{f_i\}_{i\in \mathcal{I}}$ $\ell \leftarrow \| \boldsymbol {v}\|_{\mathfrak{g}}$ $s\gets v / \| v\|_{\mathfrak{g}}$    
while  $\ell \geq 0$  do  
 $d_{i} = \operatorname {arg intersect}_{\mathfrak{g}}(x,z;f_{i})$ $i\gets \arg \min_{i}d_{i}$ $\alpha \gets \min (d_i,\ell)$ $x^{\prime}\gets \exp_{\mathfrak{g}}(x,\alpha s)$ $s\gets \mathrm{paralleltransport}_{\mathfrak{g}}(x,s,x^{\prime})$ $s\gets \mathrm{reflect}(s,f_i)$ $\ell \leftarrow \ell -\alpha$ $x\gets x^{\prime}$   
end while  
return  $x$
Algorithm 4 Reflected Random Walk. Discretisation of the SDE  $\mathrm{d}\mathbf{X}_t = b(t,\mathbf{X}_t)\mathrm{d}t + \mathrm{d}\mathbf{B}_t - \mathrm{d}\mathbf{k}_t$
Require:  $T, N, X_0^\gamma, \{f_i\}_{i \in \mathcal{I}}$ $\gamma = T / N$   
for  $k \in \{0, \dots, N-1\}$  do  
[ Z_{k+1} \sim \mathrm{N}(0, \mathrm{Id}) ]  
[ X_{k+1}^\gamma = \text{ReflectedStep}[X_k^\gamma, \sqrt{\gamma} Z_{k+1}, \{f_i\}_{i \in \mathcal{I}}] ]  
end for  
return  $\{X_k^\gamma\}_{k=0}^N$
Algorithm 5 Metropolis Random Walk. Discretisation of the SDE  $\mathrm{d}\mathbf{X}_t = b(t,\mathbf{X}_t)\mathrm{d}t + \mathrm{d}\mathbf{B}_t - \mathrm{d}\mathbf{k}_t$
Require:  $T,N,X_0^\gamma ,\{f_i\}_{i\in \mathcal{I}}$ $\gamma = T / N$    
for  $k\in \{0,\dots ,N - 1\}$  do  
 $Z_{k + 1}\sim \mathrm{N}(0,\mathrm{Id})$ $X^{\prime}\gets \exp_{\mathfrak{g}}(X_k^\gamma ,\sqrt{\gamma} Z_{k + 1})$   
if  $\max_{i\in \mathcal{I}}f_i(X')\leq 0$  then  
 $X_{k + 1}^{\gamma} = X^{\prime}$   
else  
 $X_{k + 1}^{\gamma} = X_{k}^{\gamma}$   
end if  
end for  
return  $\{X_k^\gamma \}_{k = 0}^N$
D Convergence to the reflected process
In this note, we assume that  $\mathcal{M} = \{x\in \mathbb{R}^d:\Phi (x) > 0\}$  is compact, with  $\Phi \in \mathrm{C}^2 (\mathbb{R}^d,\mathbb{R})$ . We have that  $\partial \mathcal{M} = \{x\in \mathbb{R}^d:\Phi (x) = 0\}$ . In addition, we assume that for any  $x\in \partial \mathcal{M}$ ,  $\| \nabla \Phi (x)\| = 1$  and that  $\Phi$  is concave. The closure of  $\mathcal{M}$  is denoted  $\overline{\mathcal{M}}$ . The assumption that  $\Phi$  is concave is only used in Theorem 7-(d) and can be dropped. We consider it for simplicity.
Let  $(\hat{X}_k^\gamma)_{k\in \mathbb{N}}$  given for any  $\gamma >0$  and  $k\in \mathbb{N}$  by  $\hat{X}_0^\gamma = x\in \overline{\mathcal{M}}$  and for  $\hat{X}_{k + 1}^{\gamma} = \hat{X}_{k}^{\gamma} + \sqrt{\gamma} Z_{k}^{\gamma}$  with  $Z_{k}^{\gamma}$  a Gaussian random variable conditioned on  $\hat{X}_k^\gamma +\sqrt{\gamma} Z_k^\gamma \in \overline{\mathcal{M}}$  . In practice,  $Z_{k}^{\gamma}$  is sampled using rejection sampling. We define  $\hat{\mathbf{X}}^{\gamma}:\mathbb{R}_{+}\to \overline{\mathcal{M}}$  given for any  $k\in \mathbb{N}$  by  $\hat{\mathbf{X}}_{k\gamma}^{\gamma} = \hat{X}_{k}^{\gamma}$  and for any  $t\in [k\gamma ,(k + 1)\gamma),\hat{\mathbf{X}}_t^\gamma = \hat{X}_k^\gamma$  Note that  $(\mathbf{X}_t)_{t\in [0,T]}$  is a D([0,T],M) valued random variable, where  $\mathrm{D}([0,T],\overline{\mathcal{M}})$  is the space of right-continuous with left-limit processes which take values in  $\overline{\mathcal{M}}$  .We denote  $\hat{\mathbb{P}}^{\gamma}$  the distribution of  $(\hat{\mathbf{X}}_t^\gamma)_{t\in [0,T]}$  on  $\mathrm{D}([0,T],\overline{\mathcal{M}})$
Our goal is to show the following theorem.
Theorem 6. For any  $T \geq 0$ ,  $(\tilde{\mathbf{X}}_t^\gamma)_{t \in [0, T]}$  weakly converges to  $(\mathbf{X}_t)_{t \in [0, T]}$  such that for any  $t \in [0, T]$
$$
\mathbf {X} _ {t} = x + \mathbf {B} _ {t} - \mathbf {k} _ {t}, \quad | \mathbf {k} | _ {t} = \int_ {0} ^ {t} \mathbb {1} _ {\mathbf {X} _ {s} \in \partial \mathcal {M}} \mathrm {d} | \mathbf {k} | _ {s}, \quad \mathbf {k} _ {t} = \int_ {0} ^ {t} \mathbf {n} (\mathbf {X} _ {s}) \mathrm {d} | \mathbf {k} | _ {s}. \tag {6}
$$
Proof. In order to prove the result, we prove that the distribution of the Markov chain seen as an element of  $\mathrm{D}([0,T],\overline{\mathcal{M}})$  converges to a solution of the Skorokhod problem (6). In particular, we first show that the limiting distribution satisfies a submartingale problem following (Stroock et al., 1971, Theorem 6.3). The transition from a solution of a submartingale problem to a weak solution of the Skorokhod problem is given by (Kang et al., 2017, Theorem 1, Proposition 2.12) and (Ramanan, 2006, Corollary 2.10). In order to apply (Stroock et al., 1971, Theorem 6.3), we define an intermediate drift and diffusion matrix, see (55) and (51). To prove the theorem one needs to control the drift and diffusion matrix inside  $\mathcal{M}$  and show that it converges to 0 and Id respectively. The technical part of the proof comes from the control of the drift coefficient near the boundary. In particular, we show that if the intermediate drift is large then we are close to the boundary and the intermediate drift is pointing inward. To investigate the local properties of the drift near the boundary we rely on the notion of tubular neighborhood, see (Lee et al., 2012, Theorem 6.24).
Some key properties of the tubular neighborhood are stated in Appendix D.1. We then establish a few technical lemmas about the tail probability of some distributions in Appendix D.2. Controls on the diffusion matrix and lower bounds on the probability of belonging in  $\mathcal{M}$  are given in Appendix D.3. Properties of large drift terms are given in Appendix D.4. The convergence of the drift and diffusion matrix on compact sets is given in Appendix D.5. The convergence of the boundary terms is investigated in Appendix D.6. Finally, we conclude the proof in Appendix D.7.
D.1 Properties of the tubular neighborhood
Using the results of (Lee et al., 2012), we establish the existence of an open set of  $\overline{\mathcal{M}}$  (for the induced topology of  $\mathbb{R}^d$ ) satisfying several important properties.
Theorem 7. There exist  $\mathsf{U} \subset \overline{\mathcal{M}}$  open and  $C \geq 1, \bar{r} > 0$  such that for any  $\gamma \in (0, \bar{\gamma})$  with  $\bar{\gamma} = 1$  the following properties hold:
(a) For any  $x\in \mathsf{U}$ , there exist a unique  $\bar{x}\in \partial \mathcal{M}$  and  $\bar{\alpha} >0$  such that  $x = \bar{x} +\bar{\alpha}\nabla \Phi (\bar{x})$
(b) For any  $\bar{\alpha} \in [0, \bar{r}]$  and  $\bar{x} \in \partial \mathcal{M}$  such that  $\bar{x} + \bar{\alpha} \nabla \Phi(\bar{x}) \in \overline{\mathcal{M}}$ , let  $x = \bar{x} + \bar{\alpha} \nabla \Phi(\bar{x})$  and  $\mathsf{C}(x, \gamma)$  such that  $x + \sqrt{\gamma} z \in \mathsf{C}(x, \gamma)$  if
$$
- \bar {\alpha} \gamma^ {- 1 / 2} \leq \alpha <   \bar {r} \gamma^ {- 1 / 2}, \quad \| v \| ^ {2} \leq \left(\alpha \gamma^ {1 / 2} + \bar {\alpha}\right) / (C \gamma), \tag {7}
$$
$$
w i t h z = \alpha \nabla \Phi (\bar {x}) + v, w i t h v \perp \nabla \Phi (\bar {x}). T h e n \mathrm {C} (x, \gamma) \subset \overline {{\mathcal {M}}}.
$$
(c)  $\mathsf{V} = \{\bar{x} +\alpha \nabla \Phi (\bar{x}):\bar{x}\in \partial \mathcal{M},\alpha \in [0,\bar{r})\}$  is open in  $\overline{\mathcal{M}}$
(d) For any  $x \in \mathsf{U}$ ,  $x + \sqrt{\gamma} z \in \overline{\mathcal{M}} \cap \mathsf{C}(x, \gamma)^{\mathrm{c}}$  then  $\alpha \geq \bar{r}\gamma^{-1/2}$  or  $\| v\|^2 \geq (\alpha\gamma^{1/2} + \bar{\alpha})/(C\gamma)$  and  $\alpha\gamma^{1/2} + \bar{\alpha} \geq 0$ , with  $z = \alpha\nabla\Phi(\bar{x}) + v$ , with  $\bar{x}$  given in (a) and  $v \perp \nabla\Phi(\bar{x})$ .
(e) There exists  $R > 0$  such that  $\{x\in \overline{\mathcal{M}}:d(x,\partial \mathcal{M})\leq 2R\} \subset \vee$
Proof. Let  $\gamma \in (0, \bar{\gamma})$  with  $\bar{\gamma} = 1$ . First, note that for any  $\bar{x} \in \partial \mathcal{M}$ , the normal space is given by  $\{\alpha \nabla \Phi(\bar{x}) : \alpha \in \mathbb{R}\}$ . Using this result and (Lee et al., 2012, Theorem 6.24) there exists  $\tilde{r}_0 > 0$  such that  $\mathsf{U}_0 = \{\bar{x} + \alpha \nabla \Phi(\bar{x}) : \bar{x} \in \partial \mathcal{M}, \alpha \in (-\tilde{r}_0, \tilde{r}_0)\} \subset \mathbb{R}^d$  is open<sup>2</sup>. We have that for any  $\alpha \in [-r_0, 0)$  and  $\bar{x} \in \partial \mathcal{M}$
$$
\begin{array}{l} \Phi (\bar {x} + \alpha \nabla \Phi (\bar {x})) = \Phi (\bar {x}) + \alpha \| \nabla \Phi (\bar {x}) \| ^ {2} + \int_ {0} ^ {1} \nabla^ {2} \Phi (\bar {x} + t \alpha \nabla \Phi (\bar {x})) (\alpha \nabla \Phi (\bar {x})) ^ {\otimes 2} d t (8) \\ \leq \alpha + \tilde {C} _ {0} \alpha^ {2} <   0, (9) \\ \end{array}
$$
with  $r_0 = \min (\tilde{r}_0,1 / (2\tilde{C}_0))$ , where we have used that  $\Phi (\bar{x}) = 0$ ,  $\| \nabla \Phi (\bar{x})\| = 1$  and defined  $\tilde{C}_0 = \sup \{\| \nabla^2\Phi (\bar{x} +\alpha \nabla \Phi (\bar{x}))\| : \bar{x}\in \partial \mathcal{M}, \alpha \in [-\tilde{r}_0,\tilde{r}_0]\}$ . Reciprocally, we have for any  $\alpha \in [0,r_0)$  and  $\bar{x}\in \partial \mathcal{M}$
$$
\Phi (\bar {x} + \alpha \nabla \Phi (\bar {x})) = \Phi (\bar {x}) + \alpha \| \nabla \Phi (\bar {x}) \| ^ {2} + \int_ {0} ^ {1} \nabla^ {2} \Phi (\bar {x} + t \alpha \nabla \Phi (\bar {x})) (\alpha \nabla \Phi (\bar {x})) ^ {\otimes 2} d t \geq \alpha - C _ {0} \alpha^ {2}, \tag {10}
$$
where we have used that  $\Phi (\bar{x}) = 0$ ,  $\| \nabla \Phi (\bar{x})\| = 1$  and defined  $C_0 = \sup \{\| \nabla^2\Phi (\bar{x} +\alpha \nabla \Phi (\bar{x}))\| : \bar{x}\in \partial \mathcal{M},\alpha \in [-r_0,r_0]\}$ . Let  $r_1 = \min (r_0,1 / (2C_0))$ . Then,  $\mathsf{U}_1 = \{\bar{x} +\alpha \nabla \Phi (\bar{x}): \bar{x}\in \partial \mathcal{M},\alpha \in (-r_1,r_1)\} \subset \mathbb{R}^d$  is open and
$$
\cup_ {1} \cap \overline {{\mathcal {M}}} = \{\bar {x} + \alpha \nabla \Phi (\bar {x}): \bar {x} \in \partial \mathcal {M}, \alpha \in [ 0, r _ {1}) \}. \tag {11}
$$
In what follows, we define  $\mathsf{U} = \mathsf{U}_1 \cap \overline{\mathcal{M}}$ . Note that  $\mathsf{U}$  is open for the induced topology and that  $\partial \mathcal{M} \subset \mathsf{U}$ . In particular,  $\partial \mathcal{M}$  is compact,  $\mathsf{U}^{\mathrm{c}}$  is closed and  $\partial \mathcal{M} \cap \mathsf{U}^{\mathrm{c}} = \varnothing$ . Hence, there exists  $r > 0$  such that  $d(\partial \mathcal{M}, \mathsf{U}^{\mathrm{c}}) \geq 4r$ . Without loss of generality we can assume that  $r \leq 1/2$ . We also have  $\{x \in \overline{\mathcal{M}} : d(x, \partial \mathcal{M}) \leq 2r\} \subset \mathsf{U}$ . The proof of (a) follows from the definition of  $\mathsf{U}_0$ . In the rest of the proof, we define
$$
C ^ {1 / 2} = 2 \max  (1, \sup  \{\| \nabla^ {2} \Phi (\bar {x} + u) \|: \bar {x} \in \partial \mathcal {M}, \| u \| ^ {2} \leq r (r + 1) \}), \quad \bar {r} = \min  (1 / (2 C ^ {1 / 2}), r / 2). \tag {12}
$$
Let us prove (b). Consider  $x + \sqrt{\gamma} z \in \mathsf{C}(x, \gamma)$  with  $\mathsf{C}(x, \gamma)$  given by (7) and  $x = \bar{x} + \bar{\alpha}\nabla\Phi(\bar{x})$  and  $z = \alpha\nabla\Phi(\bar{x}) + v$  with  $v \perp \nabla\Phi(\bar{x})$ . In particular, we recall that we have
$$
- \bar {\alpha} \gamma^ {- 1 / 2} \leq \alpha <   \bar {r} \gamma^ {- 1 / 2}, \quad \| v \| ^ {2} \leq \left(\alpha \gamma^ {1 / 2} + \bar {\alpha}\right) / (C \gamma). \tag {13}
$$
$$
\bar {\alpha} + \sqrt {\gamma} \alpha \leq 2 \bar {r}, \quad \gamma \| v \| ^ {2} \leq 2 \bar {r} / C. \tag {14}
$$
First, using that  $C\geq 1$ $\| \nabla \Phi (\bar{x})\| = 1$  (14) and (12), we have
$$
\left\| x + \sqrt {\gamma} z - \bar {x} \right\| ^ {2} = (\bar {\alpha} + \sqrt {\gamma} \alpha) ^ {2} + \gamma \| v \| ^ {2} \leq r ^ {2} + r / C \leq r (r + 1). \tag {15}
$$
$$
\begin{array}{l} \Phi (x + \sqrt {\gamma} z) = \Phi (\bar {x}) + \bar {\alpha} + \sqrt {\gamma} \alpha + \int_ {0} ^ {1} \nabla^ {2} \Phi (\bar {x} + t (x + \sqrt {\gamma} z - \bar {x})) (x + \sqrt {\gamma} z - \bar {x}) ^ {\otimes 2} \mathrm {d} t (16) \\ \geq \bar {\alpha} + \sqrt {\gamma} \alpha - (C ^ {1 / 2} / 2) ((\bar {\alpha} + \sqrt {\gamma} \alpha) ^ {2} + \gamma \| v \| ^ {2}), (17) \\ \end{array}
$$
where we recall that
$$
C ^ {1 / 2} = 2 \max  (1, \sup  \{\| \nabla^ {2} \Phi (\bar {x} + u) \|: \bar {x} \in \partial \mathcal {M}, \| u \| ^ {2} \leq r (r + 1) \}), \quad \bar {r} = \min  (1 / (2 C ^ {1 / 2}), r / 2). \tag {18}
$$
First, using that  $r \leq 1/2$  and (14), we have  $\bar{\alpha} + \sqrt{\gamma}\alpha \leq 2r \leq 1$ . Since,  $\|v\|^2 \leq (\bar{\alpha} + \sqrt{\gamma}\alpha) / (C\gamma)$  and we have that  $\|v\|^2 < 1 / (C\gamma)$ . Let  $P(X) = X - (C^{1/2}/2)X^2 - (C^{1/2}/2)\gamma \|v\|^2$ . We have that  $P(x) \geq 0$  if and only if  $x \in [x_{\min}, x_{\max}]$  with
$$
x _ {\min } = \left(1 - \left(1 - C \gamma \| v \| ^ {2}\right) ^ {1 / 2}\right) / C ^ {1 / 2}, \quad x _ {\max } = \left(1 + \left(1 - C \gamma \| v \| ^ {2}\right) ^ {1 / 2}\right) / C ^ {1 / 2}. \tag {19}
$$
Using that for any  $t\in (0,1)$ $(1 - t)^{1 / 2}\geq 1 - t$  we have that
$$
x _ {\min } \leq \gamma C \| v \| ^ {2} / 2, \quad x _ {\max } \geq 1 / C ^ {1 / 2}. \tag {20}
$$
2This is the tubular neighborhood theorem which is key to the rest of the proof.
Since  $\| v \|^2 \leq (\sqrt{\gamma} \alpha + \bar{\alpha}) / (\gamma C)$ , we have that  $\bar{\alpha} + \sqrt{\gamma} \alpha \geq x_{\min}$ . In addition, using that  $\bar{\alpha} + \sqrt{\gamma} \alpha \leq 2\bar{r} \leq 1 / C^{1/2} \leq x_{\max}$ , we get that  $P(\bar{\alpha} + \sqrt{\gamma} \alpha) \geq 0$  and therefore  $x + \sqrt{\gamma} z \in \overline{\mathcal{M}}$  since  $\Phi(x + \sqrt{\gamma} z) \geq 0$ . This concludes the proof of (b). Note that the condition  $\alpha \geq -\gamma^{-1/2} \bar{\alpha}$  is implied by the condition  $\| v \|^2 \leq (\sqrt{\gamma} \alpha + \bar{\alpha}) / (\gamma C)$ . Using that  $V \subset \{x \in \overline{\mathcal{M}} : d(x, \partial \mathcal{M}) \leq 2r\} \subset U, (c)$  is a direct consequence of (Lee et al., 2012, Theorem 6.24)]. Next, we prove (d). Let  $x + \sqrt{\gamma} z \in \overline{\mathcal{M}} \cap C(x, \gamma)^c$ . If  $\alpha < -\bar{\alpha} \gamma^{-1/2}$  then since  $\Phi$  is concave, we have
$$
\Phi (x + \sqrt {\gamma} z) = \Phi (\bar {x}) + \bar {\alpha} + \sqrt {\gamma} \alpha + \int_ {0} ^ {1} \nabla^ {2} \Phi (\bar {x} + t (x + \sqrt {\gamma} z - \bar {x})) (x + \sqrt {\gamma} z - \bar {x}) ^ {\otimes 2} \mathrm {d} t <   0, \tag {21}
$$
where we have used that  $\Phi (\bar{x}) = 0$ . This is absurd, hence either  $\alpha \geq \bar{r}\gamma^{-1 / 2}$  or  $\| v\| ^2\geq (\alpha \gamma^{1 / 2} + \bar{\alpha}) / (C\gamma)$  and  $\alpha \gamma^{1 / 2} + \bar{\alpha}\geq 0$ , which concludes the proof. The proof of (e) is similar to the proof that  $\{x\in \overline{\mathcal{M}}:d(x,\partial \mathcal{M})\leq 2r\} \subset \mathsf{U}$ .
The main message of Theorem 7 is that using Theorem 7-(d), if we move in the direction of  $\nabla \Phi (\bar{x})$  (the inward normal) with magnitude  $\alpha$  then we are allowed to move in the orthonormal direction with magnitude  $\alpha^{1 / 2}$ . In the next paragraph, we discuss this fact in details and shows it is necessary for the rest of our study.
The necessity of Theorem 7-(b). At first sight one can wonder if the statement of Theorem 7-(b) could be simplify. In particular, it would be simpler to replace this statement with: for any  $\bar{\alpha} \in [0,\bar{r}]$  and  $\bar{x} \in \partial \mathcal{M}$  such that  $\bar{x} + \bar{\alpha}\nabla\Phi(\bar{x}) \in \overline{\mathcal{M}}$ , let  $x = \bar{x} + \bar{\alpha}\nabla\Phi(\bar{x})$  and  $\mathsf{C}(x,\gamma)$  such that  $x + \sqrt{\gamma}z \in \mathsf{C}(x,\gamma)$  if
$$
- \bar {\alpha} \gamma^ {- 1 / 2} \leq \alpha <   \bar {r} \gamma^ {- 1 / 2}, \quad \| v \| ^ {2} \leq \left(\alpha \gamma^ {1 / 2} + \bar {\alpha}\right) ^ {2} / (C \gamma), \tag {22}
$$
with  $z = \alpha \nabla \Phi(\bar{x}) + v$ , with  $v \perp \nabla \Phi(\bar{x})$ . Then  $\mathsf{C}(x, \gamma) \subset \overline{\mathcal{M}}$ . Note that  $\|v\|^2 \leq (\alpha \gamma^{1/2} + \bar{\alpha}) / (C\gamma)$  is replaced by  $\|v\|^2 \leq (\alpha \gamma^{1/2} + \bar{\alpha})^2 / (C\gamma)$ , see Figure 8 for an illustration. However, in that case Theorem 7-(d) becomes: in addition, if  $x + \sqrt{\gamma} z \in \overline{\mathcal{M}} \cap \mathsf{C}(x, \gamma)^{\mathrm{c}}$  then  $\alpha \geq r\gamma^{-1/2}$  or  $\|v\|^2 \geq (\alpha \gamma^{1/2} + \bar{\alpha})^2 / (C\gamma)$  and  $\alpha \gamma^{1/2} + \bar{\alpha} \geq 0$ .
In what follows, when controlling the properties of large drift, see the proof of Proposition 18 and the proof of Proposition 21, we need to control quantities of the form  $\mathbb{P}(x + \sqrt{\gamma} Z \in \mathsf{C}(x,\gamma)^{\mathrm{c}} \cap \overline{\mathcal{M}}) / \sqrt{\gamma^3}$ . Using the original Theorem 7-(d) it is possible to show that this quantity is bounded. However, if one uses the updated version of Theorem 7-(d) then one needs to show that there exists  $M \geq 0$  and  $\bar{\gamma} > 0$  such that for any  $\gamma \in (0,\bar{\gamma})$  (here we have assumed that  $\bar{\alpha} = 0$ , i.e.  $x \in \partial \mathcal{M}$  for simplicity)
$$
\int_ {0} ^ {r / \gamma^ {- 1 / 2}} \int_ {\nabla \Phi (\bar {x}) ^ {\perp}} \mathbb {1} _ {\| v \| ^ {2} \geq \alpha^ {2}} \varphi (v) \varphi (\alpha) \mathrm {d} v \mathrm {d} \alpha \leq M \sqrt {\gamma}, \tag {23}
$$
which is absurd.
D.2 Technical lemmas
We start with a few technical lemmas which will allow us to control some Gaussian probabilities outside of a compact set. We denote  $\Psi : \mathbb{R}_+ \times \mathbb{N} \to [0,1]$  such that for any  $k \in \mathbb{N}$ ,  $\Psi(\cdot, k)$  is the tail probability of a  $\chi$ -squared random variable with parameter  $k$ , i.e. for any  $k \in \mathbb{N}$  and  $t \geq 0$  we have
$$
\Psi (t, k) = \mathbb {P} (\| Z \| ^ {2} \geq t), \tag {24}
$$
with  $Z$  a Gaussian random variable in  $\mathbb{R}^k$  with zero mean and identity covariance matrix. We will make extensive use of the following lemma which is a direct consequence of (Laurent et al., 2000, Section 4, Lemma 1).
Lemma 8. For any  $k\in \mathbb{N}$  and  $t\in \mathbb{R}_+$  with  $t\geq 5k$ $\Psi (t,k)\leq \exp [-t / 5]$
Proof. Let  $k \in \mathbb{N}$ . First, note that for any  $x \geq k$ , we have that  $k + 2(kx)^{1/2} + 2x \leq 5x$ . Combining this result and (Laurent et al., 2000, Section 4, Lemma 1, Equation (4.3)), we have that for any  $x \geq k$
$$
\mathbb {P} \left(\| X \| ^ {2} \geq 5 x\right) \leq \exp [ - x ], \tag {25}
$$
with  $X$  a  $\mathbb{R}^k$ -valued Gaussian random variable with zero mean and identity covariance matrix. This concludes the proof upon letting  $t = 5x$ .
3The division by  $\sqrt{\gamma}$  comes from the definition of the intermediate drift (55).
Figure 8: The grey shaded area represents  $\overline{\mathcal{M}}$  while the blue shaded area represents  $\mathsf{C}(x,\gamma)$  for an arbitrary value of  $\gamma$  and  $x = \bar{x}\in \partial \mathcal{M}$ .
Let  $\varphi : \mathbb{R}^p \to \mathbb{R}_+$  given for any  $u \in \mathbb{R}$  by  $\varphi(u) = (2\pi)^{-p/2} \exp[-\|u\|^2/2]^4$ , i.e. the density of a real Gaussian random variable with zero mean and unit variance. While Lemma 9 appears technical, it will be central to provide quantitative upper bounds on the rejection probability, see Lemma 12 for instance.
Lemma 9. For any  $k\in \mathbb{N}$ $\alpha_0 > 0$ $\beta_0\in (0,1]$  and  $\delta >0$  we have
$$
\psi (\delta) = \sup  \left\{\int_ {0} ^ {+ \infty} \Psi \left(\alpha_ {0} t / \delta , k\right) ^ {\beta_ {0}} \varphi \left(t - t _ {0} / \delta\right) d t: t _ {0} \geq 0 \right\} \leq C _ {0} \delta , \tag {26}
$$
$$
w i t h C _ {0} = 5 (2 \pi) ^ {- 1 / 2} (k + 1) / (\alpha_ {0} \beta_ {0}).
$$
Proof. Let  $k \in \mathbb{N}$ ,  $\alpha_0 > 0$ ,  $\beta_0 \in (0,1]$  and  $\delta > 0$ . Let  $t_\delta = 5k\delta / \alpha_0$ . Note that if  $t \geq t_\delta$  then,  $\alpha_0 t / \delta \geq 5k$ . In addition, we have
$$
\begin{array}{l} \int_ {0} ^ {+ \infty} \Psi \left(\alpha_ {0} t / \delta , k\right) ^ {\beta_ {0}} \varphi \left(t - t _ {0} / \delta\right) \mathrm {d} t \leq (2 \pi) ^ {- 1 / 2} \int_ {0} ^ {+ \infty} \Psi \left(\alpha_ {0} t / \delta , k\right) ^ {\beta_ {0}} \mathrm {d} t (27) \\ \leq (2 \pi) ^ {- 1 / 2} \int_ {0} ^ {t _ {\delta}} \Psi \left(\alpha_ {0} t / \delta , k\right) ^ {\beta_ {0}} \mathrm {d} t + (2 \pi) ^ {- 1 / 2} \int_ {t _ {\delta}} ^ {+ \infty} \Psi \left(\alpha_ {0} t / \delta , k\right) ^ {\beta_ {0}} \mathrm {d} t. (28) \\ \end{array}
$$
Using that for any  $w > 0$ ,  $\int_0^{+\infty} \exp[-wt] \, \mathrm{d}t \leq (1/w)$ , that for any  $u \geq 0$ ,  $\Psi(u,k) \leq 1$  and that if  $u \geq 5k$ ,  $\Psi(u,k) \leq \exp[-u/5]$ , we get for any  $t_0 \geq 0$
$$
\int_ {0} ^ {+ \infty} \Psi \left(\alpha_ {0} t / \delta , k\right) \varphi (t - t _ {0} / \delta) \leq (2 \pi) ^ {- 1 / 2} \left[ 5 k \delta / \alpha_ {0} + 5 \delta / \left(\alpha_ {0} \beta_ {0}\right) \right] \leq \left(5 \left(2 \pi\right) ^ {- 1 / 2} (k + 1) / \left(\alpha_ {0} \beta_ {0}\right)\right) \delta , \tag {29}
$$
Finally, we have the following lemma, which is similar to Lemma 8 but will be used to control quantities related to the norm.
Lemma 10. For any  $k\in \mathbb{N}$ $\alpha_0 > 0$ $\beta_0\in (0,1]$  and  $\delta >0$  we have