SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this paper, we introduced a novel version of RMHMC, Barrier HMC (BHMC), which addresses the problem of sampling from a distribution  $\pi$  over a constrained convex subset  $M\subset \mathbb{R}^d$  equipped with a self-concordant barrier  $\phi$ . Our contribution highlights that the use of well known implicit schemes for ODE integration combined with these space constraints leads to asymptotic bias when it comes to their numerical implementation. We propose to solve it in a straightforward manner via our algorithm, n-BHMC, which relies on an additional "involution checking step". Under reasonable assumptions, our theory shows that this critical step removes the asymptotic bias. This result is supported by numerical experiments where we highlight this lack of bias compared to state-of-the-art methods. In future work, we would like to investigate the "coupled" behaviour of the hyperparameters  $h$  and  $\eta$  in practice, the study of irreductibility of n-BHMC and the influence of  $\eta$  on the bias. Moreover, we are interested in the application of n-BHMC for efficient polytope volume computation. More generally, we are convinced that our study is a first step for future work on designing implicit integration schemes for unbiased constrained sampling methods.
AD acknowledges support from the Lagrange Mathematics and Computing Research Center. AD and MN would like to thank the Isaac Newton Institute for Mathematical Sciences for support and hospitality during the programme The mathematical and statistical foundation of future data-driven engineering when work on this paper was undertaken. MN acknowledges funding from the grant SCAI (ANR-19-CHIA-0002).
Betancourt, M. A general metric for Riemannian manifold Hamiltonian Monte Carlo. In International Conference on Geometric Science of Information, pp. 327-334. Springer, 2013.
Betancourt, M. A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint arXiv:1701.02434, 2017.
Brofos, J. and Lederman, R. R. Evaluating the implicit midpoint integrator for Riemannian Hamiltonian Monte Carlo. In International Conference on Machine Learning, pp. 1072-1081. PMLR, 2021a.
Brofos, J. A. and Lederman, R. R. On numerical considerations for Riemannian Manifold Hamiltonian Monte Carlo. arXiv preprint arXiv:2111.09995, 2021b.
Brubaker, M., Salzmann, M., and Urtasun, R. A family of MCMC methods on implicitly defined manifolds. In Artificial Intelligence and Statistics, pp. 161-172. PMLR, 2012.
Bubeck, S., Eldan, R., and Lehec, J. Finite-time analysis of projected Langevin Monte Carlo. In Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS'15, pp. 1243-1251, Cambridge, MA, USA, 2015. MIT Press. URL http://dl.acm.org/citation.cfm?id=2969239.2969378.
Byrne, S. and Girolami, M. Geodesic Monte Carlo on embedded manifolds. Scandinavian Journal of Statistics, 40(4):825-845, 2013.
Cobb, A. D., Baydin, A. G., Markham, A., and Roberts, S. J. Introducing an explicit symplectic integration scheme for Riemannian Manifold Hamiltonian Monte Carlo. arXiv preprint arXiv:1910.06243, 2019.
Cousins, B. and Vempala, S. Gaussian Cooling and  $\mathrm{O}^* (\mathrm{n}^3)$  Algorithms for Volume and Gaussian Volume. ArXiv e-prints, September 2014.
Dalalyan, A. S. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society, Series B, 79(3):651-676, 2017.
Douc, R., Moulines, E., Priouret, P., and Soulier, P. Markov Chains. Springer, 2018.
Duane, S., Kennedy, A. D., Pendleton, B. J., and Roweth, D. Hybrid Monte Carlo. Physics letters B, 195(2):216-222, 1987.
Durmus, A. and Moulines, E. Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. The Annals of Applied Probability, 27(3):1551-1587, 06 2017. doi: 10.1214/16-AAP1238.
Dyer, M. and Frieze, A. Computing the volume of convex bodies: a case where randomness provably helps. Probabilistic combinatorics and its applications, 44:123-170, 1991.
Gelfand, A. E., Smith, A. F., and Lee, T.-M. Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. Journal of the American Statistical Association, 87(418): 523-532, 1992.
Girolami, M. and Calderhead, B. Riemann Manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2):123-214, 2011.
Lovász, L. and Vempala, S. The geometry of logconcave functions and sampling algorithms. *Random Struct. Algorithms*, 30(3):307-358, May 2007. ISSN 1042-9832. doi: 10.1002/rsa.v30:3. URL http://dx.doi.org/10.1002/rsa.v30:3.
Mok, K.-P. Metrics and connections on the cotangent bundle. In Kodai Mathematical Seminar Reports, volume 28, pp. 226-238. Department of Mathematics, Tokyo Institute of Technology, 1977.
Morris, B. J. Improved bounds for sampling contingency tables. *Random Structures & Algorithms*, 21(2):135-146, 2002.
Neal, R. M. et al. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2 (11):2, 2011.
Nesterov, Y. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2003.
Nesterov, Y. and Nemirovski, A. Multi-parameter surfaces of analytic centers and long-step surface-following interior point methods. Mathematics of Operations Research, 23(1):1-38, 1998.
Nesterov, Y. and Nemirovskii, A. Interior-point polynomial algorithms in convex programming. SIAM, 1994.
Pakman, A. and Paninski, L. Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians. Journal of Computational and Graphical Statistics, 23(2):518-542, 2014.
Roberts, G. O. and Rosenthal, J. S. Optimal scaling for various Metropolis-Hastings algorithms. Statistical science, 16(4):351-367, 2001.
Roberts, G. O. and Stramer, O. Langevin diffusions and Metropolis-Hastings algorithms. Methodology and computing in applied probability, 4:337-357, 2002.
Sasaki, S. On the differential geometry of tangent bundles of Riemannian manifolds. Tohoku Mathematical Journal, Second Series, 10(3):338-354, 1958.
Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. Split Hamiltonian Monte Carlo. Statistics and Computing, 24(3):339-349, 2014.
Strang, G. On the construction and comparison of difference schemes. SIAM Journal on Numerical Analysis, 5(3):506-517, 1968.
Tao, M. Explicit symplectic approximation of nonseparable Hamiltonians: Algorithm and long time performance. Physical Review E, 94(4):043303, 2016.
Thiele, I., Swainston, N., Fleming, R. M., Hoppe, A., Sahoo, S., Aurich, M. K., Haraldsdottir, H., Mo, M. L., Rolfsson, O., Stobbe, M. D., et al. A community-driven global reconstruction of human metabolism. Nature biotechnology, 31(5):419-425, 2013.
Zappa, E., Holmes-Cerfon, M., and Goodman, J. Monte Carlo on manifolds: sampling densities and integrating functions. Communications on Pure and Applied Mathematics, 71(12):2609-2647, 2018.
Organisation of the supplementary
This appendix is organized as follows. Appendix A summarizes general facts that will be useful for proofs. Appendix B and Appendix C provide additional details on Riemannian metrics and technicalities on self-concordance, respectively. The c-BHMC algorithm is presented in Appendix D with some results whose proofs are given in Appendix E. Appendix F presents general facts about numerical integration and specific facts on the integrators used in n-BHMC. Appendix G dispenses the proof of the result on implicit integrators of n-BHMC, stated in Section 4.1. Appendix H presents the modified version of n-BHMC, incorporating a step-size condition, for which we state the reversibility with respect to the target distribution in Section 4.2. Proof of this last result is given in Appendix I. A detailed comparison between n-BHMC and CRHMC (Kook et al., 2022a) is given in Appendix J. Finally, Appendix K provides more details on the experiments of Section 6.
Notation. For any Riemannian manifold  $(\mathsf{M},\mathfrak{g})$  and any  $z = (x,p)\in \mathrm{T}^{\star}\mathsf{M}$ , we denote by  $\| \cdot \| _z$  the norm (7) defined on  $\mathrm{T}^{\star}\mathsf{M}$  by  $\| (x',p')\|_z = \| x'\|_{\mathfrak{g}(x)} + \| p'\|_{\mathfrak{g}(x)^{-1}}$  for any  $(x',p')\in \mathrm{T}^{\star}\mathsf{M}$ . For any open subset  $\mathsf{U}\subset \mathbb{R}^d$  and any  $k\in \mathbb{N}$ , we denote by  $\mathrm{C}^k (\mathsf{U},\mathbb{R}^{d'})$  the set of functions  $f:\mathsf{U}\to \mathbb{R}^{d'}$  such that  $f$  is  $k$  times continuously differentiable.
A Useful facts and lemmas
We recall here some basic knowledge on linear algebra and probability, and state useful inequalities for our proofs.
Linear algebra reminders. For any matrices  $\mathrm{A}, \mathrm{B} \in \mathbb{R}^{d \times d}$ , we write  $\mathrm{A} \preceq \mathrm{B}$  if for any  $x \in \mathbb{R}^d$ ,  $x^\top (\mathrm{B} - \mathrm{A})x \geq 0$ . Any positive-definite matrix  $\mathrm{A} \in \mathbb{R}^{d \times d}$  induces a scalar product  $\langle \cdot, \cdot \rangle_{\mathrm{A}}$  on  $\mathbb{R}^d$ , defined by  $\langle x, y \rangle_{\mathrm{A}} = \langle x, Ay \rangle$ . This scalar product induces the norm  $\| \cdot \|_{\mathrm{A}}$  on  $\mathbb{R}^d$ , defined by  $\| x \|_{\mathrm{A}} = \sqrt{\langle x, x \rangle_{\mathrm{A}}} = \| \mathrm{A}^{1/2}x \|_2$ . For any positive-definite matrices  $\mathrm{A}, \mathrm{B} \in \mathbb{R}^{d \times d}$  and for any  $\alpha \geq 0$ ,  $\mathrm{A} \preceq \alpha \mathrm{B}$  is equivalent to  $\| \cdot \|_{\mathrm{A}} \leq \sqrt{\alpha} \| \cdot \|_{\mathrm{B}}$ . The canonical norm of  $\mathbb{R}^d$  induces the norm  $\| \cdot \|_2$  on  $\mathbb{R}^{d \times d}$ , defined by  $\| \mathrm{A} \|_2 = \sup_{\| u \|_2 = 1} \| Au \|_2$ . In particular, for any matrices  $\mathrm{A}, \mathrm{B} \in \mathbb{R}^{d \times d}$  and any vector  $x \in \mathbb{R}^d$ ,  $\| Ax \|_2 \leq \| A \|_2 \| x \|_2$  and  $\| AB \|_2 \leq \| A \|_2 \| B \|_2$ . Moreover, if  $A$  is non-negative-definite, then  $\| A \|_2 = \lambda(A)$ , where  $\lambda(A)$  is the largest eigenvalue of  $A$ , and  $\| A^\alpha \|_2 = \lambda(A)^\alpha$  for any  $\alpha > 0$ .
Probability reminders. In this section, we consider a smooth manifold  $\mathsf{M} \subset \mathbb{R}^d$ . We first recall the definition of reversibility (Douc et al., 2018, Definition 1.5.1) before stating general results on reversibility.
Definition 4. Let  $\mathrm{Q} : \mathrm{T}^{\star}\mathsf{M} \times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M}) \to [0,1]$  be a transition probability kernel and let  $\bar{\pi}$  be a probability distribution on  $\mathrm{T}^{\star}\mathsf{M}$ . Then,
(a)  $\mathbf{Q}$  is said to be reversible with respect to  $\bar{\pi}$  if for any  $f \in \mathrm{C}(\mathrm{T}^*\mathsf{M} \times \mathrm{T}^*\mathsf{M},\mathbb{R})$  with compact support
$$
\int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (z, z ^ {\prime}) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) = \int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (z ^ {\prime}, z) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z).
$$
(b)  $\mathrm{Q}$  is said to be reversible up to momentum reversal with respect to  $\bar{\pi}$  if for any  $f \in \mathrm{C}(\mathrm{T}^{\star}\mathsf{M} \times \mathrm{T}^{\star}\mathsf{M},\mathbb{R})$  with compact support
$$
\int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (z, z ^ {\prime}) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) = \int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (s (z ^ {\prime}), s (z)) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z),
$$
where we recall that for any  $(x,p)\in \mathbb{R}^d\times \mathbb{R}^d$ $s(x,p) = (x, - p)$
Lemma 5. Let  $\mathrm{Q} : \mathrm{T}^{\star}\mathsf{M} \times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M}) \to [0,1]$  be a transition probability kernel and let  $\bar{\pi}$  be a probability distribution on  $\mathrm{T}^{\star}\mathsf{M}$ . Assume that  $s_{\#}\bar{\pi} = \bar{\pi}$  and that  $\mathrm{Q}$  is reversible up to momentum reversal with respect to  $\bar{\pi}$ . Then  $\bar{\pi}$  is an invariant measure for  $\mathrm{Q}$ , i.e., for any  $f \in \mathrm{C}(\mathrm{T}^{\star}\mathsf{M},\mathbb{R})$  with compact support
$$
\int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (z ^ {\prime}) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) = \int_ {\mathrm {T} ^ {*} \mathsf {M} \times \mathrm {T} ^ {*} \mathsf {M}} f (z) \bar {\pi} (\mathrm {d} z).
$$
Proof. Let  $f \in \mathrm{C}(\mathrm{T}^{\star}\mathsf{M},\mathbb{R})$  with compact support. We have
$$
\begin{array}{l} \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f (z ^ {\prime}) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f (s (z)) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) \tag {Definition4} \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f (z) \mathrm {Q} \left(s (z), \mathrm {d} z ^ {\prime}\right) \left(s _ {\#} \bar {\pi}\right) (\mathrm {d} z) = \int_ {\mathrm {T} ^ {*} \mathrm {M}} f (z) \bar {\pi} (\mathrm {d} z) \quad (\text {m o m e n t u m r e v e r s a l o n} z), \\ \end{array}
$$
Lemma 6. Let  $\mathrm{Q} : \mathrm{T}^{\star}\mathsf{M} \times \mathcal{B}(\mathrm{T}^{\star}\mathsf{M}) \to [0,1]$  be a transition probability kernel and let  $\bar{\pi}$  be a probability distribution on  $\mathrm{T}^{\star}\mathsf{M}$ . Assume that  $s_{\#}\mathrm{Q} = \mathrm{Q}$  and that  $\mathrm{Q}$  is reversible with respect to  $\bar{\pi}$ . Then,  $\mathrm{Q}$  is reversible up to momentum reversal with respect to  $\bar{\pi}$ .
Proof. Let  $f \in C(\mathrm{T}^{\star}\mathsf{M},\mathbb{R})$  with compact support. We have
$$
\begin{array}{l} \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f (z, z ^ {\prime}) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f (z, s \left(z ^ {\prime}\right)) \left(s _ {\#} \mathrm {Q}\right) \left(z, \mathrm {d} z ^ {\prime}\right) \bar {\pi} (\mathrm {d} z) \quad \text {(m o m e n t u m r e v e r s a l o n} z ^ {\prime}) \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f \left(z ^ {\prime}, s (z)\right) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z) \tag {Definition 4} \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f \left(s \left(z ^ {\prime}\right), s (z)\right) \left(s _ {\#} \mathrm {Q}\right) \left(z, \mathrm {d} z ^ {\prime}\right) \bar {\pi} (\mathrm {d} z) \quad \text {(m o m e n t u m r e v e r s a l o n} z ^ {\prime}) \\ = \int_ {\mathrm {T} ^ {*} \mathrm {M} \times \mathrm {T} ^ {*} \mathrm {M}} f \left(s \left(z ^ {\prime}\right), s (z)\right) \mathrm {Q} (z, \mathrm {d} z ^ {\prime}) \bar {\pi} (\mathrm {d} z), \\ \end{array}
$$
Useful inequalities. The following inequalities hold:
(a) For any  $u\in [0,1 / 5]$ $(1 - u)^{-2}\leq 1 + 3u$  and  $(1 - u)^{-3}\leq 1 + 5u$
(b) For any  $u\in [0,1 / 2]$ $(1 - u)^{-1}\leq 1 + 2u$
(c) For any  $u\in [0,1],|(1 - u)^2 -1|\leq 3u$
(d) For any  $u\geq 0$  , we have  $1 - (1 + u)^{-1}\leq u$  and  $(1 - u)^{2} - 1\geq -2u$
B Details on Riemannian metrics
Let  $\mathsf{M}$  be a smooth  $d$ -dimensional manifold, endowed with a metric  $\mathfrak{g}$ . We recall that the Riemannian volume element corresponding to  $(\mathsf{M}, \mathfrak{g})$  is given in local coordinates by  $\mathrm{dvol}_{\mathsf{M}}(x) = \sqrt{\operatorname*{det}(\mathfrak{g})}\mathrm{d}x$ , where  $\mathrm{d}x$  is a dual coframe, (Lee, 2006, Lemma 3.2.). For any  $x \in \mathsf{M}$ , we respectively denote by  $\mathrm{T}_x\mathsf{M}$  and  $\mathrm{T}_x^\star\mathsf{M}$ , the tangent space at  $x$  and its dual space, i.e., the cotangent space at  $x$ . Note that  $\mathrm{T}_x\mathsf{M}$  and  $\mathrm{T}_x^\star\mathsf{M}$  are space vectors, and  $\mathrm{T}_x\mathsf{M}$  is endowed with the scalar product  $\langle \cdot, \cdot \rangle_{\mathfrak{g}(x)}$  by definition of the Riemannian metric. For clarity sake, we denote by  $v$  (resp.  $p$ ) an element of the tangent (resp. cotangent) space. We recall that the tangent bundle  $\mathrm{TM}$  and the cotangent bundle  $\mathrm{T}^{\star}\mathsf{M}$  are respectively defined by  $\mathrm{TM} = \sqcup_{x \in \mathsf{M}} \{x\} \cup \mathrm{T}_x\mathsf{M}$  and  $\mathrm{T}^{\star}\mathsf{M} = \sqcup_{x \in \mathsf{M}} \{x\} \cup \mathrm{T}_x^\star\mathsf{M}$ . These two sets are  $2d$ -dimensional manifolds.
Metric on the tangent bundle TM. Sasaki (1958) originally introduced on TM a Riemannian metric  $\hat{\mathfrak{g}}$ , which, among other properties, preserves the Euclidean metric induced by  $\mathfrak{g}$  on each tangent space. This metric is defined by
$$
\hat {\mathfrak {g}} = \left( \begin{array}{c c} \mathfrak {g} _ {i j} + v ^ {k} v ^ {l} \Gamma_ {i s} ^ {k} \Gamma_ {t j} ^ {l} \mathfrak {g} ^ {s t} & - \mathfrak {g} ^ {j s} v ^ {k} \Gamma_ {i s} ^ {k} \\ - \mathfrak {g} ^ {i s} v ^ {k} \Gamma_ {j s} ^ {k} & \mathfrak {g} _ {i j} \end{array} \right)  ,
$$
where  $\mathfrak{g}_{ij}$  and  $\mathfrak{g}^{ij}$  respectively refer to  $\mathfrak{g}$  and  $\mathfrak{g}^{-1}$ , and  $\Gamma$  corresponds to the Christoffel symbol. Since (TM,  $\hat{\mathfrak{g}}$ ) is a Riemannian manifold, the volume form on TM satisfies for any  $(x,v)\in \mathrm{TM}$
$$
\mathrm {d v o l} _ {\mathsf {T M}} (x, v) = \sqrt {\det  (\tilde {\mathfrak {g}} (x , v))} \mathrm {d} x \mathrm {d} v = \det  (\mathfrak {g} (x)) \mathrm {d} x \mathrm {d} v.
$$
Metric on the cotangent bundle  $\mathrm{T}^{\star}\mathsf{M}$ . Elaborating on the metric defined by Sasaki (1958), Mok (1977) showed that for any  $x\in \mathsf{M}$ ,  $\mathrm{T}_x^\star \mathsf{M}$  is naturally endowed with the scalar product  $\langle \cdot ,\cdot \rangle_{\mathfrak{g}(x)^{-1}}$  and proposed a Riemannian metric  $\mathfrak{g}^{\star}$  on  $\mathrm{T}^{\star}\mathsf{M}$ , which notably preserves this result on each cotangent space. This metric is closely related to  $\hat{\mathfrak{g}}$  and is defined by
$$
\mathfrak {g} ^ {\star} = \left( \begin{array}{c c} \mathfrak {g} _ {i j} + p ^ {k} p ^ {l} \Gamma_ {i s} ^ {k} \Gamma_ {t j} ^ {l} \mathfrak {g} ^ {s t} & - \mathfrak {g} ^ {j s} p ^ {k} \Gamma_ {i s} ^ {k} \\ - \mathfrak {g} ^ {i s} p ^ {k} \Gamma_ {j s} ^ {k} & \mathfrak {g} ^ {i j} \end{array} \right)  .
$$
Since  $(\mathrm{T}^{\star}\mathsf{M},\mathfrak{g}^{\star})$  is a Riemannian manifold, the volume form on  $\mathrm{T}^{\star}\mathsf{M}$  satisfies for any  $(x,p)\in \mathrm{T}^{\star}\mathsf{M}$
$$
\mathrm {d v o l} _ {\mathrm {T} ^ {*} \mathsf {M}} (x, p) = \sqrt {\det  (\mathfrak {g} ^ {\star} (x , p))} \mathrm {d} x \mathrm {d} p = \mathrm {d} x \mathrm {d} p.
$$
In contrast to the tangent bundle, the volume form on the cotangent bundle does not depend on  $\mathfrak{g}$ , which motivates to augment on  $\mathrm{T}^{\star}\mathsf{M}$  (instead of TM) any target measure  $\pi$  defined on  $\mathsf{M}$ .
C Properties of self-concordance
We first recall the definition of self-concordance and derive some of its properties in Lemmas 8 and 9. Definition 7 (Nesterov & Nemirovskii (1994)). Let  $\mathsf{U}$  be a non-empty open convex domain in  $\mathbb{R}^d$ . A function  $\phi: \mathsf{U} \to \mathbb{R}$  is said to be a  $\nu$ -self-concordant barrier (with  $\nu \geq 1$ ) on  $\mathsf{U}$  if it satisfies:
(a)  $\phi \in \mathrm{C}^3 (\mathsf{U},\mathbb{R})$  and  $\phi$  is convex,
(b)  $\phi (x)\to +\infty$  as  $x\to \partial \mathsf{U}$
(c)  $|D^3\phi(x)[h,h,h]| \leq 2 \| h \|_{\mathfrak{g}(x)}^3$ , for any  $x \in \mathsf{M}$ ,  $h \in \mathbb{R}^d$ ,
(d)  $|D\phi (x)[h]|^2\leq \nu \| h\|_{\mathfrak{g}(x)}^2$  ,for any  $x\in \mathsf{M}$ $h\in \mathbb{R}^d$
Self-concordance can be thought as a certain kind of regularity. Indeed, if  $\phi$  is a  $\nu$ -self-concordant barrier on a convex body  $\mathsf{U}$ , then  $D^2\phi$  is 2-Lipschitz continuous on  $\mathsf{U}$  with respect to the local norm induced by  $\mathfrak{g}$  (see Property (c)), and more restrictively,  $\phi$  is  $\nu$ -Lipschitz continuous with respect to the same norm (see Property (d)).
Lemma 8. Let  $\phi : \mathsf{U} \to \mathbb{R}$  be a  $\nu$ -self-concordant barrier with  $\mathfrak{g} = D^2\phi$ . Assume that  $\mathsf{U}$  is bounded. Then,  $\| \nabla \phi(x) \|_2 \to +\infty$  and  $\| \mathfrak{g}(x) \|_2 \to +\infty$  as  $x \to \partial \mathsf{U}$ .
Proof. Since  $\phi$  is convex, we have, for any  $(x,y)\in \mathsf{U}^2$
$$
\begin{array}{l} \phi (x) \leq \phi (y) + \nabla \phi (x) (x - y), \\ | \phi (x) | \leq | \phi (y) | + \| \nabla \phi (x) \| _ {2} \| x - y \| _ {2} \\ \leq | \phi (y) | + \| \nabla \phi (x) \| _ {2} \operatorname {d i a m} (\mathrm {U}), \\ \end{array}
$$
where we used Cauchy-Schwartz inequality in the second line. Using Property (b) of  $\phi$ , we obtain that  $\| \nabla \phi(x) \|_2 \to +\infty$  as  $x \to \partial \mathsf{U}$ . By combining this result with Property (d) of  $\phi$ , we obtain that  $\| \mathfrak{g}(x) \|_2 \to +\infty$  as  $x \to \partial \mathsf{U}$ .
According to Nesterov (2003) (see Lemma 4.1.2), Property (c) of self-concordance is equivalent to
$$
\left| D ^ {3} \phi (x) \left[ h _ {1}, h _ {2}, h _ {3} \right] \right| \leq 2 \prod_ {i = 1} ^ {3} \| h _ {i} \| _ {\mathfrak {g} (x)},
$$
for any  $x\in \mathsf{M}$  and any  $(h_1,h_2,h_3)\in \mathbb{R}^d\times \mathbb{R}^d\times \mathbb{R}^d$
Let  $u: \mathbb{R}^d \to \mathbb{R}^d$  be a linear operator. For any  $x \in \mathbb{R}^d$ , we define the operator norms  $\| u \|_{\mathfrak{g}(x)}$  and  $\| u \|_{\mathfrak{g}(x)^{-1}}$  by
$$
\left\| u \right\| _ {\mathfrak {g} (x)} = \sup  \left\{u (\mathfrak {g} (x) h): \| h \| _ {\mathfrak {g} (x)} = 1 \right\},
$$
$$
\left\| u \right\| _ {\mathfrak {g} (x) ^ {- 1}} = \sup  \left\{u \left(\mathfrak {g} (x) ^ {- 1} h\right): \| h \| _ {\mathfrak {g} (x) ^ {- 1}} = 1 \right\}.
$$
In addition, a set  $\mathsf{V} \subset \mathbb{R}^d$  contains no straight line if for any  $\mathsf{D}_{x_0} = \{tx_0 : t \in \mathbb{R}\}$  with  $x_0 \neq 0$ , we have  $\mathsf{V}^c \cap \mathsf{D}_{x_0} \neq \emptyset$ .
Lemma 9. Let  $\phi : \mathsf{U} \to \mathbb{R}$  be a  $\nu$ -self-concordant barrier with  $\mathfrak{g} = D^2\phi$ . Assume that  $\mathsf{U}$  contains no straight line. For any  $x \in \mathsf{U}$  and any  $r > 0$ , we denote by  $\mathsf{W}^0(x, r)$  the open Dikin ellipsoid of  $\phi$  at  $x$ , given by  $\mathsf{W}^0(x, r) = \{y \in \mathbb{R}^d \mid \|y - x\|_{\mathfrak{g}(x)} < r\}$ . Then, the following properties hold:
(a) For any  $x\in \mathsf{U}$  and any  $(h_1,h_2)\in \mathbb{R}^d\times \mathbb{R}^d$
$$
\left\| D ^ {3} \phi (x) \left[ h _ {1}, h _ {2} \right] \right\| _ {\mathfrak {g} (x) ^ {- 1}} \leq 2 \left\| h _ {1} \right\| _ {\mathfrak {g} (x)} \left\| h _ {2} \right\| _ {\mathfrak {g} (x)}.
$$
(b) For any  $x\in \mathsf{U},\mathsf{W}^0 (x,1)\subset \mathsf{U}$ , and for any  $y\in \mathsf{W}^0 (x,1)$
$$
\left(1 - \| y - x \| _ {\mathfrak {g} (x)}\right) ^ {2} \mathfrak {g} (x) \preceq \mathfrak {g} (y) \preceq \left(1 - \| y - x \| _ {\mathfrak {g} (x)}\right) ^ {- 2} \mathfrak {g} (x).
$$
(c) For any  $x\in \mathsf{U}$ , any  $y\in \mathsf{W}^0 (x,1)$  and any  $h\in \mathbb{R}^d$ , the following hold
$$
\left\| h \right\| _ {\mathfrak {g} (x)} \leq \left(1 - \left\| y - x \right\| _ {\mathfrak {g} (x)}\right) ^ {- 1} \left\| h \right\| _ {\mathfrak {g} (y)},
$$
$$
\left\| h \right\| _ {\mathfrak {g} (y)} \leq \left(1 - \left\| y - x \right\| _ {\mathfrak {g} (x)}\right) ^ {- 1} \left\| h \right\| _ {\mathfrak {g} (x)},
$$
$$
\left\| h \right\| _ {\mathfrak {g} (x) ^ {- 1}} \leq \left(1 - \left\| y - x \right\| _ {\mathfrak {g} (x)}\right) ^ {- 1} \left\| h \right\| _ {\mathfrak {g} (y) ^ {- 1}},
$$
$$
\left\| h \right\| _ {\mathfrak {g} (y) ^ {- 1}} \leq \left(1 - \left\| y - x \right\| _ {\mathfrak {g} (x)}\right) ^ {- 1} \left\| h \right\| _ {\mathfrak {g} (x) ^ {- 1}}.
$$