SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Proposition 2. We recall that maps  $\underline{\mathbf{G}}_h$ ,  $\underline{\mathbf{g}}_h$  and  $\overline{\mathbf{G}}_h$  are respectively defined in (29), (30) and (31). Assume A1, A2. Let  $z^{(0)} \in \mathrm{T}^{\star}\mathsf{M}$ . We define  $\tilde{z}^{(0)} = s(z^{(0)})$ . Let  $h \in (0, h^{\star}(z^{(0)}))$ . By using Lemma 24 on  $\tilde{z}^{(0)}$ , we obtain the existence of  $z_h^{(1/2)} \in \mathrm{T}^{\star}\mathsf{M}$  with the following properties:
(a)  $z_{h}^{(1 / 2)}\in \underline{\mathbf{G}}_{h}(\tilde{z}^{(0)})$  ,i.e.,  $\underline{\mathrm{g}}_h(\tilde{z}^{(0)},z_h^{(1 / 2)}) = 0,$
(b)  $\operatorname{Jac}_{z'}[\underline{\mathbf{g}}_h](\tilde{z}^{(0)},z_h^{(1 / 2)})$  and  $\operatorname{Jac}[\overline{\mathbf{G}}_h](z_h^{(1 / 2)})$  are invertible,
(c)  $\operatorname{Jac}[\mathrm{G}_{h,x^{(0)}}](x_h^{(1/2)})$  is invertible.
We then define  $z_h^{(1)} = \overline{\mathrm{G}}_h(z_h^{(1/2)})$ . In particular,  $z_h^{(1)} \in \mathrm{G}_h(\tilde{z}^{(0)})$ , i.e.,  $z_h^{(1)} \in \mathrm{F}_h(z^{(0)})$  and  $\operatorname{Jac}[\mathrm{G}_{h,x^{(0)}}](x_h^{(1)}) = \operatorname{Jac}[\mathrm{G}_{h,x^{(0)}}](x_h^{(1/2)})$  is invertible. By combining the properties of  $z_h^{(1/2)}$  and with Lemma 25, we also obtain the existence of a neighbourhood  $\mathsf{U}' \subset \mathbb{T}^{\star}\mathsf{M}$  of  $\tilde{z}^{(0)}$  and a  $\mathbb{C}^1$ -diffeomorphism  $\xi_h: \mathsf{U}' \to \xi_h(\mathsf{U}') \subset \mathbb{T}^{\star}\mathsf{M}$  such that (i)  $\xi_h(\tilde{z}^{(0)}) = z_h^{(1)}$ , (ii) for any  $z \in \mathsf{U}'$ ,  $\xi_h(z) \in \mathrm{G}_h(z)$  and (iii)  $|\det \operatorname{Jac} \xi_h| \equiv 1$ . Therefore, we can define the subset  $\mathsf{U}$  and the map  $\gamma_h$  of Proposition 2 by
(a)  $\mathsf{U} = s(\mathsf{U}^{\prime})$  , neighbourhood of  $z^{(0)}$  in  $\mathrm{T}^{\star}\mathsf{M}$
(b)  $\gamma_h = \xi_h \circ s$ , such that (i)  $\gamma_h(z^{(0)}) = z_h^{(1)}$ , (ii) for any  $z \in \mathsf{U}$ ,  $\gamma_h(z) \in \mathrm{F}_h(z)$  and (iii)  $\det \operatorname{Jac}[\gamma_h] \equiv 1$ .
We now prove that  $\mathsf{U}$  can be reduced to a smaller subset such that  $\gamma_h(z)$  is the only element of  $\mathrm{F}_h(z)$  in  $\gamma_h(\mathsf{U})$  for any  $z \in \mathsf{U}$ . Motivated by Lemma 22, we first define, for any  $(x,x') \in \mathsf{M} \times \mathsf{M}$ ,  $\mathrm{F}_{h,x}(x')$  as the only element  $p \in \mathrm{T}_x^\star \mathsf{M}$  such that  $(x',p') \in \mathrm{F}_h(x,p)$  for any  $p' \in \mathrm{T}_{x'}^\star \mathsf{M}$ . It is clear that  $\mathrm{F}_{h,x}(x') = -\mathrm{G}_{h,x}(x')$ , where  $\mathrm{G}_{h,x}$  is defined in (32). We also define  $\mathrm{Z}_h: \mathsf{M} \times \mathsf{M} \to \mathrm{T}^\star \mathsf{M}$  by
$$
\mathrm {Z} _ {h} (x, x ^ {\prime}) = \left(x, \mathrm {F} _ {h, x} (x ^ {\prime})\right) = \left(x, - \mathrm {G} _ {h, x} (x ^ {\prime})\right),
$$
which is continuously differentiable since  $\mathfrak{g} \in \mathrm{C}^2(\mathsf{M}, \mathbb{R}^{d \times d})$ . Besides this, we obtain by Lemma 22 that  $\operatorname{Jac}[\mathrm{G}_{h,x^{(0)}}](x^{(0)}, x_h^{(1)})$  is invertible, and then  $\operatorname{Jac}[\mathrm{Z}_h](x_h^{(1)})$  is invertible. Therefore, by applying the inverse function theorem on  $Z_h$  at  $(x^{(0)}, x_h^{(1)})$ , it comes that  $Z_h$  is a  $\mathrm{C}^1$ -diffeomorphism in a neighbourhood of  $(x^{(0)}, x_h^{(1)})$ . We are now going to prove the result by contradiction. Assume for now that there is no neighbourhood  $U$  of  $z^{(0)}$  such that  $\gamma_h(z)$  is the only element of  $\mathrm{F}_h(z)$  in  $\gamma_h(U)$  for any  $z \in U$ . Then, we can find a sequence  $(z_i)_{i \in \mathbb{N}}$  which converges to  $z^{(0)}$  such that for any  $i \in \mathbb{N}$ , there exist two different elements  $z_{i,1} \in \mathrm{F}_h(z_i)$  and  $z_{i,2} \in \mathrm{F}_h(z_i)$ . Therefore, for any  $i \in \mathbb{N}$ ,  $x_{i,1} \neq x_{i,2}$  and by Lemma 26, we have
$$
\mathrm {F} _ {h, x _ {i}} \left(x _ {i, 1}\right) = \mathrm {F} _ {h, x _ {i}} \left(x _ {i, 2}\right) = p _ {i},
$$
$$
Z _ {h} \left(x _ {i}, x _ {i, 1}\right) = Z _ {h} \left(x _ {i}, x _ {i, 2}\right) = z _ {i}. \tag {51}
$$
Moreover, by continuity of  $\gamma_h$ , the sequences  $(z_{i,1})_{i\in \mathbb{N}}$  and  $(z_{i,2})_{i\in \mathbb{N}}$  converge to  $z_h^{(1)}$ , and therefore,  $(x_{i,1})_{i\in \mathbb{N}}$  and  $(x_{i,2})_{i\in \mathbb{N}}$  also converge to  $x^{(1)}$ . Combined with (51), this result of convergence is in contradiction with the fact that  $Z_h$  is a diffeomorphism in a neighbourhood of  $(x^{(0)}, x_h^{(1)})$ . Therefore, we can reduce  $U$  to a smaller subset such that  $\gamma_h(z)$  is the only element of  $F_h(z)$  in  $\gamma_h(U)$  for any  $z \in U$ , which concludes the proof of Proposition 2.
H Modification of n-BHMC algorithm with step-size conditioning
In the rest of the paper, for any  $z^{(0)}\in \mathrm{T}^{\star}\mathsf{M}$ , we will denote by  $h_\star (z^{(0)})$  the value of  $h_\star$  given by A3.
Beyond n-BHMC. A crucial part of the proof of reversibility in BHMC, as much for c-BHMC as for n-BHMC, relies on local symplectic properties of the integrator of the Hamiltonian dynamics. Although Algorithm 1 can be implemented without any practical limitation, it is hard to state such properties for its numerical integrator  $\Phi_h$  under A1, A2 and A3, given any value of  $h$ . Indeed, we know from A3 that  $\Phi_h$  is a local involution around  $z^{(0)} \in \mathrm{T}^{\star}\mathsf{M}$  when  $h < h_{\star}(z^{(0)})$ ; however, we cannot ensure this result when  $h > h_{\star}(z^{(0)})$ . To circumvent this issue, we propose to study Theoretical n-BHMC (Tn-BHMC), presented in Algorithm 3. In this modified version of n-BHMC, we actually enforce a condition on  $h$  to be small enough. We now get into the details of this new algorithm and assume A3 for the rest of this section.
Theoretical motivations. Let  $h > 0$ . We recall the definition of the set  $\mathsf{A}_h$  introduced in Section 4.2
$$
\mathsf {A} _ {h} = \left\{z \in \mathrm {T} ^ {\star} \mathsf {M}: h <   \min  _ {\tilde {z} \in \mathsf {B} _ {\| \cdot \| _ {z}} (z, 1)} h _ {\star} (\tilde {z}) \right\}.
$$
It is clear that  $\mathsf{A}_h\subset \mathrm{dom}_{\Phi_h}$ : indeed, if  $z^{(0)}\in \mathsf{A}_h$ , we have in particular that  $h < h_{\star}(z^{(0)})$ , and therefore  $z^{(0)}\in \mathrm{dom}_{\Phi_h}$  by A3. Let  $z^{(0)}\in \mathsf{A}_h$ . By A3, we know that  $\Phi_h$  is an involution on a neighbourhood of  $z^{(0)}$ ; in particular, it comes that  $(\Phi_h\circ \Phi_h)(z^{(0)}) = z^{(0)}$ . Hence, the condition (b) of the "involution checking step" in Algorithm 1 is de facto satisfied. This naturally leads to replace the condition " $z\in \mathrm{dom}_{\Phi_h}$ " by the more restrictive condition " $z\in \mathsf{A}_h$ ".
Algorithm 3: Tn-BHMC with Momentum Refreshment
Input:  $(x_0,p_0)\in \mathrm{T}^{\star}\mathsf{M},\beta \in (0,1],N\in \mathbb{N},h > 0,\eta >0,\Phi_h$  with domain  $\mathrm{dom}_{\Phi_h},\mathsf{A}_h$  Output:  $(X_{n},P_{n})_{n\in [N]}$    
1 for  $n = 1,\ldots ,N$  do   
2 Step 1:  $G_{n}\sim \mathrm{N}_{x}(0,\mathrm{I}_{d}),\tilde{P}_{n}\gets \sqrt{1 - \beta} P_{n - 1} + \sqrt{\beta} G_{n}$    
3 Step 2: solving a discretized version of ODE (3)   
4  $X_{n}^{\prime},P_{n}^{\prime}\leftarrow X_{n - 1},\tilde{P}_{n};\qquad X_{n}^{(0)},P_{n}^{(0)}\leftarrow (s\circ S_{h / 2})(X_{n},\tilde{P}_{n})$    
5 if  $Z_{n}^{(0)} = (X_{n}^{(0)},P_{n}^{(0)})\in \mathsf{A}_{h}$  then   
6  $Z_{n}^{(1)} = \Phi_{h}(Z_{n}^{(0)})$  if  $Z_{n}^{(1)}\in A_{h}$  then   
8  $|X_n',P_n'\gets (s\circ S_{h / 2})(Z_n^{(1)})|$  end   
10 end   
11 Step 3:  $A_{n}\gets \min (1,\exp [-H(X_{n}',P_{n}') + H(X_{n - 1},\tilde{P}_{n})])$    
12  $U_{n}\sim U[0,1]$    
13 if  $U_{n}\leq A_{n}$  then  $\bar{X}_n,\bar{P}_n\gets X_n',P_n'$    
14 else  $\bar{X}_n,\bar{P}_n\gets X_{n - 1},\tilde{P}_n$    
15 Step 4:  $X_{n},\hat{P}_{n}\gets s(\bar{X}_{n},\bar{P}_{n})$    
16 Step 5:  $G_{n}^{\prime}\sim \mathrm{N}_{x}(0,\mathrm{I}_{d}),P_{n}\gets \sqrt{1 - \beta}\hat{P}_{n} + \sqrt{\beta} G_{n}^{\prime}$    
17 end
Implementation of Tn-BHMC. In Algorithm 3, we highlight in yellow the modifications of Tn-BHMC in contrast to n-BHMC. Namely, we replace
(a) “ $Z_{n}^{(0)} \in \mathrm{dom}_{\Phi_{h}}$ ” (Line 5 in Algorithm 1) by “ $Z_{n}^{(0)} \in \mathsf{A}_{h}$ ” (Line 5 in Algorithm 3).
(b) “ $Z_{n}^{(1)} \in \mathrm{dom}_{\Phi_{h}}$ ” (Line 8 in Algorithm 1) by “ $Z_{n}^{(1)} \in \mathsf{A}_{h}$ ” (Line 7 in Algorithm 3).
Note that there is no need to maintain the "involution checking step" of Algorithm 1, since it is automatically verified once  $Z_{n}^{(0)} \in \mathsf{A}_{h}$ . On the whole, these new conditions are more restrictive than the conditions of Algorithm 1 since  $\mathsf{A}_h \subset \mathrm{dom}_{\Phi_h}$ ; moreover, they can be thought as conditions directly applied on the step-size  $h$ . The specific choice of  $\mathsf{A}_h$ , instead of another subset of  $\mathrm{dom}_{\Phi_h}$ , is actually sufficient to derive the proof of reversibility of Tn-BHMC (see Section 4.2).
I Proofs of Section 4.2
I.1 Expression and properties of  $r^\star$
Given a Riemannian manifold  $(\mathsf{M},\mathfrak{g})$  , we define  $r^{\star}(x)$  for any  $x\in \mathsf{M}$  by
$$
r ^ {\star} (x) = \min  \left(\| \mathfrak {g} (x) \| _ {2} ^ {- 1 / 2}, \| \mathfrak {g} (x) ^ {- 1} \| _ {2} ^ {- 1 / 2}\right). \tag {52}
$$
Note that  $r^{\star}$  is used in A3 and that  $r^{\star}(x) = 1 / C_{x}$ , where  $C_x$  is defined in Lemma 12. We prove below that  $r^{\star}$  has a smooth behaviour on  $\mathsf{M}$  under our main assumptions.
Lemma 27. Assume A1, A2. Also assume that  $x \in \mathsf{M} \mapsto \| \mathfrak{g}^{-1}(x) \|_2$  is bounded from above. As defined in (52),  $r^\star : \mathsf{M} \to (0, +\infty)$  satisfies the following properties:
(a)  $r^{\star}(x)\to 0$  as  $x\rightarrow \partial M$
(b) There exists  $\mathsf{L} > 0$  such that  $r^{\star}$  is L-Lipschitz-continuous on  $\mathsf{M}$  with respect to  $\| \cdot \| _2$ .
(c) There exists  $\mathbb{M} > 0$  such that  $r^{\star}(x)\leq \mathbb{M}$  for any  $x\in \mathbb{M}$
Proof. Assume A1, A2. Also assume that  $x \in \mathsf{M} \mapsto \| \mathfrak{g}^{-1}(x) \|_2$  is bounded from above. We first define  $r_1 : x \in \mathsf{M} \mapsto \| \mathfrak{g}(x) \|_2^{-1/2}$  and  $r_2 : x \in \mathsf{M} \mapsto \| \mathfrak{g}(x)^{-1} \|_2^{-1/2}$ , such that  $r^\star = \min(r_1, r_2)$ . Since  $\mathfrak{g} \in \mathrm{C}^2(\mathsf{M}, \mathbb{R}^{d \times d})$ , it is clear that  $r_1$  and  $r_2$  are continuously differentiable on  $\mathsf{M}$ . We have:
(a)  $r_1(x)\to 0$  as  $x\to \partial \mathsf{M}$  by Lemma 8, and
(b)  $r_2(x)\neq 0$  as  $x\to \partial \mathsf{M}$  , since  $1 / r_2^2:x\mapsto \| \mathfrak{g}(x)^{-1}\| _2$  is bounded on M.
Combining the fact that  $r_2(x) > 0$  for any  $x \in \mathsf{M}$  and item (a), we obtain item (a) of Lemma 27. We denote by  $d$  the distance induced by  $\|\cdot\|_2$  and now define, for any  $\varepsilon > 0$ ,
(i)  $\mu_{\varepsilon} = \inf_{\{y\in \mathsf{M}:d(y,\partial \mathsf{M})\leq \epsilon \}}r_2(y).$
(ii)  $\mathsf{M}_{\varepsilon} = \mathrm{Int}(\{x\in \mathsf{M}:d(x,\partial \mathsf{M})\leq \varepsilon ,r_1(x)\leq \mu_{\varepsilon}\}),$  open set in M.
(iii)  $\mathsf{M}_{-\varepsilon} = \mathsf{M}\backslash \mathsf{M}_{\varepsilon}$ , closed and bounded (and thus, compact) set in  $\mathsf{M}$ .
Using items (a) and (b), we can ensure the existence of some  $\varepsilon \in (0, \mathrm{diam}(\mathsf{M}))$  such that (i)  $\mu_{\varepsilon} > 0$  and (ii)  $\mathsf{M}_{\varepsilon}$  and  $\mathsf{M}_{-\varepsilon}$  are not empty. We consider such  $\varepsilon$  for the rest of the proof.
Smoothness of  $r_2$ . We define  $\delta = d(\mathsf{M}^{\mathrm{c}},\mathsf{M}_{-\varepsilon})$  and  $\mathsf{M}_{\delta} = \mathsf{M}^{\mathrm{c}} + \bar{\mathsf{B}} (0,\delta /4)$ . Note that
(i)  $\delta > 0$  since  $\mathsf{M}_{-\varepsilon} \subset \mathsf{M}$ ,
(ii)  $\mathsf{M}_{\delta}$  is closed since the ball  $\bar{\mathsf{B}} (0,\delta /4)$  is compact, and
(iii)  $\mathsf{M}_{\delta}\cap \mathsf{M}_{-\varepsilon} = \emptyset$
According to the smooth version Urysohn's lemma applied to  $\mathsf{M}_{\delta}$  and  $\mathsf{M}_{-\varepsilon}$ , there exists  $\chi \in \mathrm{C}^1 (\mathbb{R}^d,[0,1])$  such that  $\chi (\mathsf{M}_{-\varepsilon}) = 1$  and  $\chi (\mathsf{M}_{\delta}) = 0$ . We then define  $\tilde{r}_2:\mathbb{R}^d\to (0, + \infty)$  by  $\tilde{r}_2 = \chi r_2 + (1 - \chi)\mu_{\varepsilon}$ . In particular, (i) there exists  $\mathsf{L}_2 > 0$  such that  $\tilde{r}_2$  is  $\mathbf{L}_2$ -Lipschitz-continuous on  $\bar{\mathsf{M}}$  with respect to  $\| \cdot \| _2$ , since  $\tilde{r}_2\in \mathrm{C}^1 (\mathbb{R}^d,[0,1])$ , and (ii) for any  $x\in \mathsf{M}_{\varepsilon}$ ,  $\tilde{r}_2(x) > \mu_{\varepsilon}$ .
Smoothness of  $r_1$ . We now prove that  $r_1$  is 1-Lipschitz continuous on  $\mathsf{M}$  with respect to  $\| \cdot \|_2$ . Let  $x \in \mathsf{M}$ . Note that  $r_1(x) = (\| \mathfrak{g}(x) \|_2^2)^{-1/4}$  and we thus have
$$
\nabla r _ {1} (x) = (- 1 / 4) \| \mathfrak {g} (x) \| _ {2} ^ {- 5 / 2} h (x): D \mathfrak {g} (x),
$$
where  $h(x) = \partial_{\mathfrak{g}(x)}\| \mathfrak{g}(x)\| _2^2 = 2\| \mathfrak{g}(x)\| _2u(x)u(x)^\top$ ,  $u(x)$  being a normal eigenvector of  $\mathfrak{g}(x)$  corresponding to the eigenvalue  $\| \mathfrak{g}(x)\| _2$ . Hence,
$$
\begin{array}{l} \| \nabla r _ {1} (x) \| _ {2} \leq (1 / 2) \| \mathfrak {g} (x) \| _ {2} ^ {- 3 / 2} \| u (x) u (x) ^ {\top}: D \mathfrak {g} (x) \| _ {2} \\ \leq (1 / 2) \| \mathfrak {g} (x) \| _ {2} ^ {- 3 / 2} \| u (x) u (x) ^ {\top} \| _ {2} \| D \mathfrak {g} (x) \| _ {2} \\ \leq (1 / 2) \| \mathfrak {g} (x) \| _ {2} ^ {- 3 / 2} \times 2 \| \mathfrak {g} (x) \| _ {2} ^ {3 / 2} \leq 1 \tag {Definition1-(c)} \\ \end{array}
$$
which proves the result on  $r_1$ .
Smoothness of  $r$ . Let  $x \in \mathsf{M}$ . We can face two cases: either,  $x \in \mathsf{M}_{-\varepsilon}$ , then  $r_2(x) = \tilde{r}_2(x)$ ; or,  $x \in \mathsf{M}_{\varepsilon}$ , then  $\tilde{r}_2(x) > \mu_{\varepsilon} \geq r_1(x)$  and  $r_2(x) \geq r_1(x)$  by definition of  $\mathsf{M}_{\varepsilon}$ . Thus, we have for any  $x \in \mathsf{M}$ ,  $r^{\star}(x) = \min(r_1(x), \tilde{r}_2(x))$  where  $r_1$  and  $\tilde{r}_2$  are respectively 1 and  $\mathsf{L}_2$  Lipschitz-continuous on  $\mathsf{M}$  with respect to  $\| \cdot \|_2$ . By observing that  $2\min(r_1, \tilde{r}_2) = r_1 + \tilde{r}_2 - |r_1 - \tilde{r}_2|$ , we set  $\mathsf{L} = 1 + \mathsf{L}_2$  and thus obtain item (b) of Lemma 27. Finally, item (c) of Lemma 27 directly derives from item (b), since  $\mathsf{M}$  is bounded.
Note that the extra-assumption on  $\mathfrak{g}^{-1}$  used in Lemma 27 is not directly ensured by self-concordance but can be proved when  $\mathsf{M}$  is a polytope, as shown below.
Lemma 28. Consider a polytope  $\mathsf{M}$  defined by  $m$  constraints with  $m > d$  such that  $\mathsf{M} = \{x : Ax < b\}$ , where  $\mathsf{A} \in \mathbb{R}^{m \times d}$  is a full-rank matrix and  $b \in \mathbb{R}^m$ . We endow  $\mathsf{M}$  with the Riemannian metric  $\mathfrak{g}(x) = D^2\phi(x)$  where  $\phi : \mathsf{M} \to \mathbb{R}$  is the logarithmic barrier given for any  $x \in \mathsf{M}$  by  $\phi(x) = -\sum_{i=1}^{m} \ln (b_i - A_i^\top x)$ . In particular,  $\mathsf{M}$  and  $\mathfrak{g}$  verify A1 and A2. Then, the function  $r : \mathsf{M} \to (0, +\infty)$  defined by  $r(x) = \| \mathfrak{g}^{-1}(x) \|_2$ , for any  $x \in \mathsf{M}$ , is bounded from above.
Proof. Consider such manifold  $\mathsf{M}$  and metric  $\mathfrak{g}$ . We aim to show that the smallest eigenvalue of  $\mathfrak{g}(x)$  is bounded from below for any  $x \in \mathsf{M}$  by a constant  $c > 0$ , which does not depend on  $x$ , i.e., for any  $h \in \mathbb{R}^d$  and any  $x \in \mathsf{M}$ ,  $\mathfrak{g}(x)[h,h] \geq c\| h\|_2^2$ .
Since  $\mathrm{A}$  is full-ranked,  $\mathrm{A}^{\top}\mathrm{A}$  is positive-definite. In particular, for any  $h\in \mathbb{R}^{d}$ ,  $(\mathrm{A}^{\top}\mathrm{A})[h,h]\geq \lambda_{\min}(\mathrm{A}^{\top}\mathrm{A})\| h\|_{2}^{2}$ , where  $\lambda_{\min}(\mathrm{A}^{\top}\mathrm{A}) > 0$  is the smallest eigenvalue of  $\mathrm{A}^{\top}\mathrm{A}$ . We recall that we have for any  $x\in \mathsf{M}$ ,  $\mathfrak{g}(x) = \mathrm{A}^{\top}S(x)^{-2}\mathrm{A}$ , where  $S(x) = \operatorname {Diag}(b_i - \mathrm{A}_i^\top x)_{i\in [m]}$ . Let  $i\in [m]$ . The function  $r_i:x\in \mathsf{M}\mapsto S(x)_{i,i}^{-2}$  has the following properties: (i)  $r_i$  is continuous on  $\mathsf{M}$ , (ii)  $r_i(x) > 0$  for any  $x\in \mathsf{M}$  and (iii)  $r(x)\to +\infty$  as  $x\to \partial \mathsf{M}$ . Thus, there exists  $c_{i} > 0$  such that for any  $x\in \mathsf{M}$ ,  $r_i(x)\geq c_i$ . We define  $\tilde{c} = \min_{i\in [m]}c_i$  and we have for any  $x\in \mathsf{M}$ ,  $S(x)^{-2}\succeq \tilde{c}\mathrm{I}_{d}$ . We now define  $c = \tilde{c}\lambda_{\min}(\mathrm{A}^{\top}\mathrm{A})$  and we have for any  $x\in \mathsf{M}$
$$
\mathfrak {g} (x) [ h, h ] \geq \tilde {c} \cdot (\mathrm {A} ^ {\top} \mathrm {A}) [ h, h ] \geq c \| h \| _ {2} ^ {2}.
$$
In particular,  $\mathfrak{g}(x)^{-1} \preceq (1/c)\mathrm{I}_d$ , i.e.,  $\|\mathfrak{g}(x)^{-1}\|_2 \leq (1/c)$ , which concludes the proof.
I.2 Markov kernels of Algorithm 1
Based on the model of  $\bar{\mathsf{E}}_h$  defined in (24), we define the set  $\bar{\mathsf{E}}_h^\Phi \subset \mathrm{T}^\star \mathsf{M}$ , which will ensure that the maps derived from implicit integrators of n-BHMC are properly expressed
$$
\bar {\mathrm {E}} _ {h} ^ {\Phi} = \left(s \circ \mathrm {S} _ {h / 2}\right) ^ {- 1} \left(\operatorname {d o m} _ {\Phi_ {h}} \cap \Phi_ {h} ^ {- 1} \left(\operatorname {d o m} _ {\Phi_ {h}}\right)\right) = \left(s \circ \mathrm {S} _ {h / 2}\right) \left(\operatorname {d o m} _ {\Phi_ {h}} \cap \Phi_ {h} ^ {- 1} \left(\operatorname {d o m} _ {\Phi_ {h}}\right)\right). \tag {53}
$$
For any  $(z,z^{\prime})\in \bar{\mathsf{E}}_{h}^{\Phi}\times \mathrm{T}^{\star}\mathsf{M}$ , we define the acceptance probability  $\bar{a} (z^{\prime}|z)$  to move from  $z$  to  $z^{\prime}$  by
$$
\bar {a} (z ^ {\prime} | z) = a (z ^ {\prime} \mid z) \mathbb {1} _ {(s \circ \mathrm {S} _ {h / 2}) (z) = (\Phi_ {h} \circ \Phi_ {h} \circ s \circ \mathrm {S} _ {h / 2}) (z)} = a (z ^ {\prime} \mid z) \mathbb {1} _ {z = (\mathrm {R} _ {h} ^ {\Phi} \circ \mathrm {R} _ {h} ^ {\Phi}) (z)},
$$
where  $a(z' \mid z)$  is the acceptance probability defined in (9). We denote by  $\mathrm{Q}_n: \mathrm{T}^{\star}\mathsf{M} \times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M}) \to [0,1]$ , the transition kernel of the (homogeneous) Markov chain  $(x_n,p_n)_{n\in [N]}$  generated by Algorithm 1. We also denote by:
(a)  $\mathrm{Q}_0: \mathrm{T}^* \mathsf{M} \times \mathcal{B}(\mathrm{T}^* \mathsf{M}) \to [0,1]$ , the transition kernel referring to Step 1 (also Step 5) in Algorithm 1, defined in (25).
(b)  $\mathrm{Q}_{n,1}:\mathrm{T}^{\star}\mathsf{M}\times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M})\to [0,1]$ , the transition kernel referring to Step 2-3-4 in Algorithm 1.
We provide below details on Markov kernels  $\mathbf{Q}_n$  and  $\mathbf{Q}_{n,1}$ .
Kernel  $\mathbf{Q}_{n,1}$ . This kernel is deterministic and corresponds to the numerical integration of the Hamiltonian up until time  $h$ . For any  $(z,z') \in \mathrm{T}^{\star}\mathsf{M} \times \mathrm{T}^{\star}\mathsf{M}$ , we have
$$
\mathrm {Q} _ {n, 1} \left(z, \mathrm {d} z ^ {\prime}\right) = \mathbb {1} _ {\bar {\mathrm {E}} _ {h} ^ {\Phi}} (z) \left(s _ {\#} \mathrm {Q} _ {n, 2}\right) \left(z, \mathrm {d} z ^ {\prime}\right) + \mathbb {1} _ {\left(\bar {\mathrm {E}} _ {h} ^ {\Phi}\right) ^ {\mathrm {c}}} (z) \delta_ {s (z)} \left(\mathrm {d} z ^ {\prime}\right), \tag {54}
$$
$$
\mathrm {Q} _ {n, 2} (z, \mathrm {d} z ^ {\prime}) = \bar {a} \left(\mathrm {R} _ {h} ^ {\Phi} (z) \mid z\right) \delta_ {\mathrm {R} _ {h} ^ {\Phi} (z)} (\mathrm {d} z ^ {\prime}) + \left[ 1 - \bar {a} \left(\mathrm {R} _ {h} ^ {\Phi} (z) \mid z\right) \right] \delta_ {z} (\mathrm {d} z ^ {\prime}). \tag {55}
$$
Kernel  $\mathbf{Q}_n$ . This kernel corresponds to one step of Algorithm 1 (i.e., comprising Steps 1 to 5). For any  $(z, z') \in \mathrm{T}^* \mathsf{M} \times \mathrm{T}^* \mathsf{M}$ , we have
$$
\mathrm {Q} _ {n} (z, \mathrm {d} z ^ {\prime}) = \int_ {\mathrm {T} ^ {\star} \mathsf {M} \times \mathrm {T} ^ {\star} \mathsf {M}} \mathrm {Q} _ {0} (z, \mathrm {d} z _ {1}) \mathrm {Q} _ {n, 1} (z _ {1}, \mathrm {d} z _ {2}) \mathrm {Q} _ {0} (z _ {2}, \mathrm {d} z ^ {\prime}).
$$
I.3 Proof of reversibility in Algorithm 3
Let  $h > 0$ . Using notation from A3, we recall definition of the set  $\mathsf{A}_h$  introduced in Section 4.2
$$
\mathsf {A} _ {h} = \left\{z \in \mathrm {T} ^ {\star} \mathsf {M}: h <   \min  _ {\tilde {z} \in \mathsf {B} _ {\| \cdot \| _ {z}} (z, 1)} h _ {\star} (\tilde {z}) \right\} \subset \operatorname {d o m} _ {\Phi_ {h}}. \tag {56}
$$
We also recall that  $\bar{\pi}$ , as defined in (2), admits a density with respect to the product Lebesgue measure given for any  $z = (x,p) \in \mathbb{T}^*\mathsf{M}$  by
Since Algorithm 3 can be thought as a restrictive version of Algorithm 1, the Markov kernels from Tn-BHMC are similar to the kernels from n-BHMC, defined in Appendix I.2. In particular, the transition kernel corresponding to the Gaussian momentum update (Step 1 and 5 in Algorithm 3, Step 1 and 5 in Algorithm 1) is the same and is reversible (up to momentum reversal) with respect to  $\bar{\pi}$  (see Lemma 17). Namely, we replace
(a) the set  $\bar{\mathsf{E}}_h^\Phi$  , defined in (53), by  $\tilde{\mathsf{E}}_h^{\Phi}$
(b) the kernels  $\mathrm{Q}_{n,1}$  and  $\mathrm{Q}_{n,2}$ , respectively defined in (54) and (55), by  $\mathrm{Q}_1$  and  $\mathrm{Q}_2$ ,
$$
\tilde {\mathsf {E}} _ {h} ^ {\Phi} = \left(s \circ \mathrm {S} _ {h / 2}\right) ^ {- 1} \left(\mathsf {A} _ {h} \cap \Phi_ {h} ^ {- 1} (\mathsf {A} _ {h})\right) = \left(s \circ \mathrm {S} _ {h / 2}\right) \left(\mathsf {A} _ {h} \cap \Phi_ {h} ^ {- 1} (\mathsf {A} _ {h})\right), \tag {57}
$$
$$
\mathrm {Q} _ {1} (z, \mathrm {d} z ^ {\prime}) = \mathbb {1} _ {\tilde {\mathrm {E}} _ {h} ^ {\Phi}} (z) \left(s _ {\#} \mathrm {Q} _ {2}\right) (z, \mathrm {d} z ^ {\prime}) + \mathbb {1} _ {\left(\tilde {\mathrm {E}} _ {h} ^ {\Phi}\right) ^ {\mathrm {c}}} (z) \delta_ {s (z)} (\mathrm {d} z ^ {\prime}), \tag {58}
$$
$$
\mathrm {Q} _ {2} (z, \mathrm {d} z ^ {\prime}) = a \left(\mathrm {R} _ {h} ^ {\Phi} (z) \mid z\right) \delta_ {\mathrm {R} _ {h} ^ {\Phi}} \left(\mathrm {d} z ^ {\prime}\right) + \left[ 1 - a \left(\mathrm {R} _ {h} ^ {\Phi} (z) \mid z\right) \right] \delta_ {z} \left(\mathrm {d} z ^ {\prime}\right). \tag {59}
$$
We denote by  $\mathrm{Q}:\mathrm{T}^{\star}\mathsf{M}\times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M})\to [0,1]$ , the transition kernel of the (homogeneous) Markov chain  $(x_{n},p_{n})_{n\in [N]}$  generated by Algorithm 3. For any  $(z,z^{\prime})\in \mathrm{T}^{\star}\mathsf{M}\times \mathrm{T}^{\star}\mathsf{M}$ , we have
$$
\mathrm {Q} (z, \mathrm {d} z ^ {\prime}) = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} \mathrm {Q} _ {0} (z, \mathrm {d} z _ {1}) \mathrm {Q} _ {1} (z _ {1}, \mathrm {d} z _ {2}) \mathrm {Q} _ {0} (z _ {2}, \mathrm {d} z ^ {\prime}). \tag {60}
$$
We first turn to the reversibility up to momentum reversal of  $\mathrm{Q}_1$  with respect to  $\bar{\pi}$ . We start with the following lemma which is key to establish this result.
Lemma 29. Assume A1, A2, A3. Then, for any compact set  $\mathsf{G} \subset \mathbb{T}^{\star}\mathsf{M}$ , for any  $g \in \mathrm{C}(\mathrm{T}^{\star}\mathsf{M} \times \mathrm{T}^{\star}\mathsf{M},\mathbb{R})$  such that  $\operatorname{supp}(g) \subset \mathsf{G} \times \mathsf{G}$ , we have
$$
\int_ {\mathsf {G} _ {h}} g (z, \Phi_ {h} (z)) \mathrm {d} z = \int_ {\mathsf {G} _ {h}} g (\Phi_ {h} (z), z) \mathrm {d} z,
$$