  Zigzag and other piecewise deterministic Markov process samplers have
attracted significant interest for their non-reversibility and other appealing
properties for Bayesian posterior computation. Hamiltonian Monte Carlo is
another state-of-the-art sampler, exploiting fictitious momentum to guide
Markov chains through complex target distributions. In this article, we uncover
a remarkable connection between the zigzag sampler and a variant of Hamiltonian
Monte Carlo exploiting Laplace-distributed momentum. The position and velocity
component of the corresponding Hamiltonian dynamics travels along a zigzag path
paralleling the Markovian zigzag process; however, the dynamics is
non-Markovian as the momentum component encodes non-immediate pasts. This
information is partially lost during a momentum refreshment step, in which we
preserve its direction but re-sample magnitude. In the limit of increasingly
frequent momentum refreshments, we prove that this Hamiltonian zigzag converges
to its Markovian counterpart. This theoretical insight suggests that, by
retaining full momentum information, Hamiltonian zigzag can better explore
target distributions with highly correlated parameters. We corroborate this
intuition by comparing performance of the two zigzag cousins on
high-dimensional truncated multivariate Gaussians, including a
11,235-dimensional target arising from a Bayesian phylogenetic multivariate
probit model applied to HIV virus data.
