SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(a) First, Kook et al. (2022a) assume that the Hamiltonian dynamics given by  $H$  can be explicitly computed in continuous time (which is not the case in practice). They present the continuous version of RMHMC which incorporates the simplified constrained Hamiltonian  $H$ , see Algorithm 1. We refer to this algorithm as c-CRHMC. This algorithm is identical to our algorithm c-BHMC provided in Appendix D, apart from the involution checking step (see Line 4 in Algorithm 2). They state in Theorem 6 that the Markov kernel corresponding to one iteration of c-CRHMC satisfies detailed balance with respect to  $\bar{\pi}$ , thus ensuring that it preserves  $\bar{\pi}$ .
(b) Then, Kook et al. (2022a) provide a practical implementation of CRHMC, which is similar to n-BHMC, based on a time-discretization of the Hamiltonian dynamics for a given time-step  $h > 0$ . They first split the simplified constrained Hamiltonian  $H$  into  $H_{1}$  and  $H_{2}$ , by leveraging the non-separable aspect of  $H$  in  $H_{2}$ , as we do in Section 3.1. Then, they propose to use the first-order Euler method to solve the discretized ODE associated to  $H_{1}$ , as we also suggest, and the Implicit Midpoint Integrator (IMI) to solve the discretized ODE associated with  $H_{2}$ , while we implement the Generalized Leapfrog Integrator (GLI). We insist on the fact that these two second-order methods share the same theoretical properties (Hairer et al., 2006). Since IMI is implicit, they propose to approximate it with a numerical integrator  $\Phi_h$ , which is computed with a fixed-point method (as we do) detailed in Algorithm 3. Finally, they implement CRHMC which contains the same steps as n-BHMC apart from the involution checking step (see Line 8 in Algorithm 1): (i) refreshing the momentum with a symplectic scheme (Step 1 in both algorithms), (ii) solving the discretized ODE associated to  $H$  in three steps according to the splitting given by  $H_{1}$  and  $H_{2}$  (Step 2 in both algorithms), (iii) applying a Metropolis-Hastings filter (Step 3 in both algorithms), and (iv) operating a final momentum reversal (Step 4 in n-BHMC). They state in Theorem 8 that the Markov kernel corresponding to one iteration of CRHMC satisfies detailed balance with respect to  $\bar{\pi}$ , thus ensuring that it preserves  $\bar{\pi}$ .
J.2 Theoretical gaps in the reversibility of CRHMC Kook et al. (2022a)
First, we call into question the proof of the reversibility of c-CRHMC stated in (Kook et al., 2022a, Theorem 6). Indeed, Kook et al. (2022a) implicitly assume that the time-continuous Hamiltonian dynamics have a solution at any time, but do not provide any proof of this result. We emphasize here that this result is not trivial and possibly may not hold due to the pathological behaviour of the barrier at the border of M, see Proposition 14. In contrast, the involution checking step implemented in c-BHMC verifies this condition for the forward and the reversed dynamics after momentum reversal, which then allows us to derive the reversibility of c-BHMC, see Theorem 15.
Finally, we report some gaps in the proof of reversibility of CRHMC provided in (Kook et al., 2022a, Theorem 8). Indeed, while Kook et al. (2022a) claim that (Hairer et al., 2006, Theorem VI.3.5.) applies to CRHMC which contains the numerical integrator detailed in (Kook et al., 2022a, Algorithm 3), in fact (Hairer et al., 2006, Theorem VI.3.5.) only applies to the implicit integrator (IMI). This is a fundamental limitation of their theory. This theoretical confusion thus leads to numerical errors, as we detail in Section 6. In contrast, we present in Section 4 theoretical results on the implicit integrator (GLI), which allows us to derive reasonable assumptions on the numerical integrator  $\Phi_h$  to obtain reversibility of n-BHMC in Theorem 3. Contrary to Kook et al. (2022a), our proof relies on the properties of  $\Phi_h$  and highlights the critical role of the involution checking step.
K Experimental details
The numerical experiments presented in Section 6 are based on the MATLAB implementation of CRHMC provided by Kook et al. (2022a). We adapt this code for n-BHMC for sake of fairness. In particular, our implementation differs from CRHMC by the use of the Generalized Leapfrog integrator and the "involution checking" mechanism.
Details on experiments with synthetic data. In these experiments, the hypercube refers to the set  $[-1/2, 1/2]^d$  where  $d \in \{5, 10\}$ . We recall that the ground truth on the quantities we estimate is given by the Metropolis Adjusted Langevin Algorithm (MALA) (Roberts & Stramer, 2002) for the hypercube and the Independent Metropolis Hastings (IMH) sampler (Liu, 1996) for the simplex. We now give details on how the parameters of these two algorithms are chosen. For MALA, we use a constant step-size  $h = 0.05$  for both dimensions, which results in an average acceptance probability equal to 0.55 for  $d = 5$  and 0.44 for  $d = 10$ . For IMH, we use as proposal the uniform distribution, which is simply a Dirichlet distribution, resulting in an average acceptance probability of order 0.36. We recall that we use an adaptive step-size  $h$  in CRHMC and n-BHMC such that we obtain an average acceptance probability of order 0.5 in the MH filter. We now discuss the setting of the tolerance parameter  $\eta$  in n-BHMC. We choose  $\eta$  so that (i) the step-size  $h$  is at least greater than  $10^{-2}$  over the iterations of the algorithm and (ii) the average acceptance probability is roundly equal to 0.5. This heuristic results in defining (a) for the hypercube,  $\eta = 5$  if  $d = 5$  and  $\eta = 10$  if  $d = 10$  and (b) for the simplex,  $\eta = 10$  if  $d = 5$  and  $\eta = 200$  if  $d = 10$ .
Details on experiments with real-world data. For these experiments, we consider the exact same setting as in Kook et al. (2022a). In particular, we use the same adaptation strategies to update the step-size and the barrier functions.
Influence of the norm in (8). The norm chosen for the "involution checking step" is arbitrary and many options are available. In particular, one could design  $\|z^{(0)} - \Phi_h(z^{(1)})\|_2 > \eta$ , and thus pick the Euclidean norm in (8). However, while this approach should theoretically also reduce the bias in the method, we remark that the obtained Markov chains have very poor mixing time. This is due to the fact that in practice the update on the momentum is of order  $\|\mathfrak{g}(x)\|_2^{1/2}$  while the update on the position is of order  $\|\mathfrak{g}(x)^{-1}\|_2^{1/2}$ . Hence the Euclidean norm is ill-suited for the "involution checking step" and the proposal states are rejected a lot more near the boundaries, see Figure 3.
Figure 3: Outputs of n-BHMC after 25k iterations to sample from the uniform distribution over  $[-1,1]^2$  with  $\eta = 10^{-3}$ ,  $h = 0.8$ , using in (8) the "self-concordant" norm (left) or the Euclidean norm (right). Red samples are rejected and blue samples are accepted.