Fixed-Distance Hamiltonian Monte Carlo
Anonymous Author(s)
AffiliationAddress email
Abstract
We propose a variation of the Hamiltonian Monte Carlo sampling (HMC) wherethe equations of motion are simulated for a fixed traversed distance rather thanthe conventional fixed simulation time.
This new mechanism tends to generateproposals that have higher target probability values.
The momentum distributionthat is naturally joint with our Fixed-Distance HMC (FDHMC), and keeps the proposal acceptance probability close to 1, is not Gaussian and generates momentumsthat have a higher expected magnitude.
This translates into a reduced correlationbetween the successive MCMC states and according to our experimental results,leads to a significant improvement in terms of the effective sample size per gradientwhen compared to the baseline HMC and No-U-Turn (NUTS) samplers.
Introduction
Markov chain Monte Carlo (MCMC) is an inference mechanism that approximates a target probability distribution by a sequence of states (a.k.a.
samples or draws ) (Chib and Greenberg, 1995).
Hamiltonian Monte Carlo (HMC) (Duane et al., 1987; Neal, 2011) is an MCMC algorithm where thecurrent state is considered as a particle (usually a unit mass) to which a randomly drawn momentumvector is assigned.
The state is also associated with a potential energy that is proportional to itsnegative log target probability density.
The next state is proposed via evolving the current stateaccording to a simulation of the equations of motion (using leapfrog integration) for a fixed timeduration.
If the simulation is accurate, the probability of accepting the proposal as the next state inthe Markov chain will be close to one.
This property makes HMC a powerful sampling tool wherethe proposal can be quite distant from the current state while being accepted with a high probability,leading to a comparatively rapid exploration of the target distribution.
In this work we further improveon HMC by highlighting two of its limitations and resolving them.
Limitation 1.
Along the trajectory that is traversed during any fixed-time simulation of the equationsof motion, it is more likely that a state with low target probability is proposed:
In HMC, the state isguided by the gradient information of the target density to move towards regions of high probability(low potential engery).
However, it moves through these regions with a high velocity and thusoscillates around them, spending more time in the surrounding low-probability regions (i.e. hills onthe potential energy) where it moves with low velocity.
As such, it is more likely that the state is in alow-probability region when the simulation time is over.
This bias is counter intuitive.
What makes the overall sampling process unbiased (in the sense that the MCMC chain converges tothe target distribution) relies on the HMC’s momentum distribution:
Note that a state that has a lowmagnitude momentum cannot move to a low-probability region (since at some level, its kinetic energywill be zero, the direction of the momentum vector will be reversed and the state will be pulled back)but a state with low-magnitude momentum can always move from a low-probability region to a highprobability region as its kinetic energy will only be increased.
Therefore, if the momentum distributionis concentrated around 0 , there is a strong bias towards high-probability regions.
Conversely, if themomentum distribution favours large magnitudes, the bias is less pronounced.
The exact distribution that generates “right amount of low-magnitude momentums” to counterbalance the bias that is inducedby Limitation 1, happens to be a 0 -mean Gaussian.
However using this distribution to generatelow-magnitude momentums causes another limitation for HMC:
Limitation 2.
HMC suffers from correlated samples in regions of high probability: A state that hasa low-magnitude momentum, cannot leave a high-probability region (as it does not have sufficientkinetic energy to lift it out of the potential energy valley).
During the simulation time, it only oscillates(or shivers) in its place leading to a proposal that is not distant from the current state.
Therefore,despite HMC’s nominal high proposal acceptance probability, it can also produce highly correlatedclose-by samples.
Note that algorithms such as No-U-turn sampling (NUTS) (Hoffman and Gelman, 2014) where thenumber of simulation leapfrog steps, and hence the simulation duration, is dynamically decided, donot solve Limitation 1 , because regardless of the number of the leapfrogs, or whether the state hasmade a U-turn or not, high-probability regions are traversed faster than low probability regions (andless intermediate states are generated from them) and therefore they are under-represented.
Regarding
Limitation 2 , a dynamically decided simulation duration, may save some computation by avoidingredundant oscillations for a state that is trapped in an energy valley, but it does not guarantee that thegenerated proposal is distant from the current state and hence limitation 2 is not addressed either.
The contribution of the present work is to systematically tackle both aforementioned HMC limitationsby simulating the equations of motion for a fixed traversed distance rather than a fixed evolutiontime.
An evolving state may spend a longer time in low-probability regions but due to its lowmomentum/velocity, the traversed distance (in each leapfrog step) is short.
Conversely, even thoughless time may be spent in high-probability regions, due to the higher momentum, the traverseddistance (per leapfrog step) is longer.
Therefore the fixed distance budget is exhausted more rapidlyin high-probability regions and the bias against these regions disappears.
This modification to the HMC algorithm also resolves Limitation 2.
Intuitively speaking, no counterbalancing is required because there is no bias towards regions of low probability in our FixedDistance HMC (FDHMC).
As such, in order to maintain the proposal acceptance probability close to1, FDHMC’s momentum distribution needs to generate values with higher expected magnitude thanthose generated from a Gaussian centred at 0 .
This modification translates into a faster exploration ofthe probability space and substantially less correlated samples.
To design FDHMC, we rely on the theory of Reversible Jump MCMC (RJMCMC) (Green, 1995).
There are recent works that see HMC in the light of RJMCMC (Levy et al., 2018;
Afshar et al.
, 2021)to design non-volume-preserving extensions.
But we are unaware of any existing variation of HMCwhere the traversed distance is fixed, or where the momentum vectors are drawn from a distributionother than normal, or where any solution is proposed for the two aforementioned limitations.
Afterproviding a brief background in Section 2 we will design FDHMC in sections 3 and 4.
According toour experiments provided in Section 5, FDHMC can outperform the baseline HMC and NUTS by asubstantial margin, in terms of the effective sample size per gradient.
Background
Our ultimate task is to approximate a target density , π Q ( q ) , defined on R n , with a set of states{ q ( c ) }
Nc =1 (also known as samples or draws ).
In Markov chain Monte Carlo (MCMC) framework,each sample, q ( c ) , is drawn conditioned on the previous sample, q ( c − 1) .
Nonetheless, the approximation is unbiased .
That is, if the number of draws N → ∞ , then the number of draws that are withinany region, A ⊂ R n , is proportional to the probability mass associated with that region, (cid:82) A π Q ( q ) d q .
Reversible Jump MCMC (Green, 1995) is an MCMC technique where to draw the next state, q ( c +1) :
(a) The current state, q ( c ) , is augmented by an auxiliary vector , r ( c ) ,
that is drawn from an arbitraryauxiliary distribution (that throughout, we assume, is continuous with a density function , π R ( r ) ,that is defined on R m , for some dimensionality m ).(b)
( q ( ∗ ) , r ( ∗ ) ) :
= F ( q ( c ) , r ( c ) ) is proposed as the next augmented state where F : R n + m → R n + mis an arbitrary differentiable mapping that is reversible , in the sense that F is the inverse of itself:
[Equation]
and only
[Equation]
(c) With Metropolis-Hastings-Green (MHG) acceptance probability:
[Equation]
the proposal, ( q ( ∗ ) , r ( ∗ ) ) , is accepted as the next augmented MCMC state, ( q ( c +1) , r ( c +1) ) , otherwisethe current state, ( q ( c ) , r ( c ) ) , is returned.(d) q
( c +1) is kept (as the next draw from the target density) and the auxiliary r ( c +1) , is discarded.
Hamiltonian Monte Carlo (HMC) (Duane et al., 1987) is a special case of RJMCMC where (a) theoriginal state, q , (that in this context, is referred to as the position vector) and the auxiliary vector(that is referred to as the momentum vector and is usually denoted by p ) have the same dimensions(i.e.
n = m ).
The auxiliary density function, π P ( p ) , is an n -variate normal distribution whichis usually assumed to be standard, N ( 0 n , I n × n )
The
, is the simulationof the equations of motion for the total time ϵL ,
L leapfrog steps, of duration ϵ , followedby of momentum vector: F ( q , p ) :
= g ◦ f 1 ,ϵ ◦ f 2 ,ϵ ◦ · · · ◦ f L,ϵ ( q , p ) ,where g ( q , p ) =
( q , − p ) and each leapfrog step f i,ϵ ( q [ i ] , p [ i ] ) =
( q [ i +1] , p [ i +1] ) is defined by thefollowing equations,
[Equation]
where the potential energy function U ( q ) :
= − log( π Q ( q )) , and ∇ U ( q ) is its gradient vector.
It canbe easily verified that this mapping, F ( q , p ) , is reversible.
Also, since F ( q , p ) is only composed ofshear mappings and a negation, it preserves the volume i.e. the absolute Jacobian determinant of thetotal transformation is equal to 1.
As such, MHG acceptance probability (2) simplifies to:
[Equation]
Evolution of a phase state , ( q , p ) , via the equations of motion conserves the Hamiltonian , U ( q ) +12 ∥
p ∥ 2 .
As such, if the leapfrog approximation is reasonably precise, HMC’s proposal acceptanceprobability is close to 1.
Seeing HMC as an instance of RJMCMC provides us with the insightrequired to design our own Fixed-distance HMC in the next sections.
Reversible leapfrogs that traverse a fixed distance
Designing a reversible fixed-distance leapfrog mechanism is challenging.
This problem is illustratedin Figure 1 where each phase state, ( q , p ) , is depicted by a node, q , to which an arrow, p , is attached.
An initial phase state ( q , p ) :
= ( q 0 , p 0 ) that goes through k leapfrog steps (each as in Figure 1a) evolves into ( q [ k ] , p [ k ] ) and traverses the total distance d [ k ] ( q , p ) :
= (cid:80) ki
=1 ∥ q [ i ] − q
[ i − 1] ∥ =(cid:80) ki
=1 ϵ ∥ p
[ i − 1] ∥ .
We want to restrict the total traversed distance to a fixed value, D .
In Figure 1-b,this distance budget is exhausted between the 4th and 5th leapfrog steps.
That is, d [4] ( q , p ) <
D <
d [5] ( q , p ) .
Suppose we return ( q [4] , − p [4] ) as the final state, F ( q , p ) =
( q ′ :=
q [4] , p ′ :
=− p [4] ) .
This process may seem reversible, as by 4 leapfrogs, ( q [4] , − p [4] ) is mapped back to ( q , p ) .
Nonetheless, in the backward direction, the termination condition may not happen at ( q , p ) .
Forinstance, in Figure 1-c, d [6] ( q ′ , p ′ ) <
D <
d [7] ( q ′ , p ′ ) , which suggests that F ( q ′ , p ′ ) =
( q ′ [6] , p ′ [6] ) .
This violates the required RJMCMC reversibility condition.
An alternative potential solution is toadd an extra leapfrog step with duration τ ′ :
= D − d [4] ( q 4 , p 4 ) ∥ p [4] ∥ <
ϵ to the forward pass which mapsthe state right to a point where the traversed distance, d
[ k ] ( q , p ) , is exactly equal to D .
That is,
F ( q , p ) =
( q [ x ] , − p [4] ) .
However, this process is not reversible either because the last forward stephas a duration τ ′ <
ϵ , and does not match the first leapfrog step (of the duration ϵ ) in the backwarddirection.
Our key insight is that by adding another leapfrog step of some duration, τ < ϵ , to thebeginning of the process, we can make the transition symmetric and reversible.
This algorithm isillustrated in Figure 2 and is described in Section 3.1.
(a)
A phase state ( q , p ) p q
Momentum half-step p ∗ ← p − ϵ 2 ∇ U ( q ) − ϵ 2 ∇ U ( q ) p ∗ p
Position full-step q [1] ← q + ϵ p ∗ p ∗ q q [1]
Momentum half-step p [1] ← p ∗ − ϵ 2 ∇ U ( q [1] ) − ϵ 2 ∇ U ( q [1] ) p ∗ p [1] q [1] (b) p p [1] p [4] q q [1] q [3] q [4] q [5] q
[ x ] (c) q ′ [7] p ′ =
− p [4] q ′ [5] q ′ [4] =
q q ′ [3] =
q [1] q ′ =
q [4] q ′ [6] q ′ [ x ]
3.1 Fixed-distance Leapfrog Mechanism (FD-Leapfrogs)
Given the leapfrog step size, ϵ , and the total evolution distance, D , as hyper-parameters, we definean ( n + 1) -dimensional
RJMCMC auxiliary vector ( p , τ ) where τ ∼ Unif (0 , ϵ ) .
We also define afunction F : R 2 n +
→ R 2 n +1 that maps ( q , p , τ ) to ( q ′ , p ′ , τ ′ ) as follows:
Initial position τ -step: Evolve the position, q , for time τ with fixed momentum, p .
q [1] :=
q
[Equation]
Initial momentum full-step:
[Equation]
Interchanged position and momentum full-steps: For i = 2 , . . .
k
, update the position andmomentum vectors interchangeably as follows:
[Equation]
where k is the maximum number of leapfrogs s.t.
the total traversed distance does not exceed D :d [ k ] ( q , p , τ ) ≤ D <
d [ k +1] ( q , p , τ ) , where d [ j ] ( q , p , τ
[Equation]
=
[Equation]
p
[Equation]
Final position τ ′ -step:
With momentum p [ k ] ,
the
remaining distance, d :
= D − d [ k ] ( q , p , τ ) , is
traversed in time
τ ′ :
= d
∥
p [ k ]
∥ , to reach the final position state, q ′ :
[Equation]
Negating the final momentum: ( q ′ , p ′ :
= − p
[ k ] , τ ′ ) is returned as F ( q , p , τ )
[Equation]
The relation between FD-leapfrogs and the ordinary leapfrog mechanism.
Traditionally, inleapfrog integration, the position and momentum vectors are updated at steps i and i + 12 , respectively.
However, by shifting the step indicator by half, we end up in an equally valid leapfrog integrator thatupdates momentums at i and positions at i + 1 2 , which is called position-Verlet integration (Batchoand Schlick, 2001)
(Compare with momentum-Verlet (3)):
[Equation]
Nonetheless, the first position update of the i -th leapfrog is the same as the second position updateof the ( i − 1) -th leapfrog.
Therefore, the ordinary position-Verlet leapfrog integration can also be
(a)
A phase state ( q , p ) q p
Position τ -step q [1] ← q + τ p t = τ q p p q [1]
Momentum full-step p [1] ← p − ϵ ∇ U ( q [1] ) − ϵ ∇ U ( q [1] ) p [1] p
Position full-step q [2] ← q [1] + ϵ p
[1] p [1] p [1] q [2] (b) t = τ q ′ t =
τ ′ q p p [5] = − p ′ p [1] p [2] p [3] p [4] q [1] q [2] q [3] q [4] q [5] q
[ k +1] (c) p ′
[2] p ′ [1] t = τ q ′ t = τ ′ p ′
[5] = − p p ′ q ′
[5] q ′ [1] q of the proposed algorithm is very similarto position-Verlet, with major differences being ( the durations of the and
evolutions, and τ
, are values in the interval (0 , ϵ ) rather than being fixed to ϵ 2 (albeit
( τ )
since τ ∼ Unif (0 , ϵ ) ).
(b) the number of interchanged position and momentum full-steps is decidedbased on the traversed distance rather than being fixed to ( L − 1) .
Reversibility.
The reversibility of FD-Leapfrogs is established by Lemma 3.1 and is illustratedin the example of Figure 2.
In the forward evolution (Figure 2-a), an augmented state ( q , p , τ ) ismapped to ( q ′ , p ′ , τ ′ ) and the total traversed distance is: D = τ
∥ p ∥
+ (cid:80) 4 i =1 ϵ ∥ p
[ i ] ∥ + τ ′ ∥ p [5] ∥(here, k = 5 ).
In Figure 2-b, the evolution starts from ( q ′ , p ′ , τ ′ ) .
It can be verified that p ′ =
− p [5] ,p ′ [1] = − p [4] , . .
, p ′ [5] = − p .
At this stage, (i.e. k ′ = 5 ), the (backward) traversed distance is:τ ′ ∥
p ′ ∥
+ (cid:80) 4 i =1 ϵ ∥ p ′
[ i ] ∥ which is equal to τ ′ ∥
p [5] ∥
+ (cid:80) 4 i =1 ϵ ∥ p
[ i ] ∥ .
The remaining distance is: τ ∥ p ∥ ,and by the current momentum, p ′ [5] = − p , it is traversed in time τ < ϵ .
Therefore, the interchangedfull-steps terminate, steps 4 and 5 of the algorithm are run and F ( q ′ , p ′ , τ ′ ) =
( q , p , τ ) is returned.
Lemma 3.1.
Fixed-distance Leapfrog Mechanism (FD-Leapfrogs) is reversible.
Being provided with a reversible mapping F ( q , p , τ ) =
( q ′ , p ′ , τ ′ ) , an HMC sampler with fixeddistance evolution mechanism is proposed in Section 4.
Fixed-Distance Hamiltonian Monte Carlo (FDHMC)
To design a RJMCMC based on the mapping, F ( q , p , τ ) = ( q ′ , p ′ , τ ′ ) , we need to compute the absolute Jacobian determinant, | ∂ ( q ′ , p ′ , τ ′ ) /∂
( q , p , τ ) | .
The Jacobian of a complicated transformationis typically calculated via decomposing the mapping into simpler steps, as the Jacobian determinantof a composition of functions is equal to the product of the Jacobian determinants of the parts.
Weobserve that F can be expressed as a composition of functions with form: Υ i ( q , p , ζ ) =
( q ∗ , p ∗ , ζ ∗ ) ,where ζ is an auxiliary variable with alternating semantics.
That is, ζ is initially set to time τ , then ζrepresents the remaining distance, d , and in the final steps, ζ represents time τ ′ :
Lemma 4.1.
The Absolute Jacobian determinant of the Fixed-distance Leapfrog Mechanism (FDleapfrogs), F ( q , p , τ ) =
( q ′ , p ′ , τ ′ ) , is ∥ p ∥
∥ p ′ ∥ .
Proof.
FD-Leapfrogs can be reformulated as:
[Equation]
p
[Equation]
τ
[Equation]
, with,
Υ 1 ( q
[Equation]
p
[Equation]
ζ )
[Equation] [Equation]
p
[Equation]
p
[Equation] [Equation]
Υ
[Equation]
[Equation] [Equation]
Υ
[Equation] [Equation]
Υ
[Equation]
Υ
[Equation]
p
[Equation]
) .Υ 1 is the initial position τ -step (Alg.
step 1).
Υ 2 returns the remaining distance as its last output.
Υ 3is a momentum full-step (Alg.
step 2).
Υ 4 is a position full-step.
Υ 5 updates the remaining distance.
As such, the composition, (Υ 3 ◦ Υ 5 ◦ Υ 4 ) , stands for an interchanged position and momentumfull-step, along with updating the remaining distance, d , (Alg. step 3) and is applied ( k − 1) timesto generate the output ( q [ k ] , p [ k ] , d ) .
Υ 6 computes the remaining time, τ ′ =
d / ∥ p [ k ] ∥ .
The finalposition τ ′ -step (Alg. step 4) is carried out by applying Υ 1 once more.
Finally, negating the finalmomentum (Alg. step 5) happens in Υ 7 .
Except Υ 2 and Υ 6 , all the above transformations preservevolume (because either they are shear mappings or a negation).
As such:
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)
( q , p ,ζ ) ← ( q [ k ] , p [ k ] , d ) · (cid:12)(cid:12)(cid:12)(cid:12) ∂
Υ 2 ( q , p , ζ ) ∂ ( q , p , ζ ) (cid:12)(cid:12)(cid:12)(cid:12) ( q , p ,ζ ) ← ( q [1] , p ,τ ) =
∥ p ∥
∥ p [ k ] ∥ =
∥ p ∥
∥ p ′ ∥ .
(10) □
4.1 Marginal momentum distribution
If we assume that FDHM has the same momentum distribution that the existing HMC has, i.e.π P ( p ) :
= N ( 0 , I ) , then by substituting (10) in (2), FDHMC’s acceptance probability becomes:
[Equation]
If the Hamiltonian dynamics is (roughly) exact, the above quantity is (almost) equal to min (cid:110) 1 ,
∥ p
[Equation]
Depending on the magnitude of the current and proposed momentums, this acceptance probability canbe quite low which is undesirable.
Nonetheless, the interesting point that we would like to highlightis that α ( · ) tends to reject proposals whose target probability is more than the target probabilityof the current state but always accepts the less probable proposals.
This rather counter-intuitivebehaviour can be justified as follows:
The role of the acceptance probability in the theory of MCMCis to counter-balance any bias that may exit in the proposal generation mechanism and to guaranteethat the draws are proportional to the target distribution.
For example, the proposal generation of a
Random Walk Metropolis-Hastings (RWMH) sampler (Metropolis et al., 1953) is totally independentof the target density.
As such, its corresponding acceptance probability, min { 1 , π Q ( q ′ ) /π Q ( q ) } ,favours the more probable proposals.
The reason that the acceptance probability of HMC (with exactdynamics) is 1 is that it is already unbiased.
To be more precise, the bias towards low probablestates (caused by Limitation 1) is canceled out by the bias towards high probable states (causedby Limitation 2).
By fixing the distance rather than time, we have resolved the first bias.
As such,the effect of the second bias prevails and is counter-balanced by α ( · ) .
By choosing the followingmomentum density:
[Equation]
the desirable acceptance probability of the existing HMC is re-established:
[Equation]
[Figure]
Figure 3 shows that the momentum density of FDHMC is strongly inclined towards generatingmomentums that have higher magnitudes.
In fact for FDHMC, π P ( 0 ) = 0 , while for HMC, π P ( 0 ) isthe peak of the density function.
This means that FDHMC does not suffer from the second bias of
HMC (i.e. Limitation 2) either, because with momentums that (in average) have higher magnitudes,the state will not be easily trapped in the valleys of the potential energy function.
4.2 Sampling the initial momentum
We need a fast way to directly draw a momentum (per FDHMC sample) from the distribution, π P ( p ) ,that is defined by (11).
We note that even though the support of this function is R n , it is totallysymmetric with respect to the angular coordinates and is only a function of the magnitude,
∥ p ∥ (see
Figure 3).
Lemma 4.2 implies that the momentum magnitude, m := ∥
p ∥ , is distributed as:
[Equation]
where m :=
∥
p ∥ , in which, χ ( m ; n + 1) ∝ m n exp( − m 2 / 2) , is a Chi distribution defined on the support: m ∈ [0 , ∞ )with the degree of freedom , ( n + 1) .
As such, the process of drawing a sample p ( c ) from π P ( p ) ∝ ∥ p ∥ · exp (cid:0) − 12 ∥ p ∥ 2 (cid:1) is as follows:
(a) Draw a direction vector, v ∈ R n , from a uniform n -sphere,
e.g. by normalising a draw from anormal distribution: v :=
s / ∥ s ∥ where s ∼ N ( 0 n , I n × n ) .(b)
Draw a magnitude, m , from a Chi distribution χ ( m ; n + 1) .(c)
p ( c ) is generated via scaling the drawn direction by the drawn magnitude: p ( c )
= m · v .
Lemma 4.2.
If a density function, π X ( x ) :
= π X ( x 1 , . . .
, x n ) (on R n ), is only a function of magnitude, ∥ x ∥ , then the random variable m =
∥ x ∥ , is distributed with density π M ( m ) ∝ m n − 1 · g ( m ) ,where g ( m ) :
= π X ( m, 0 , . . . , 0) =
π X (0 , m, 0 . . . , 0)
= . . .
=
π X (0 , . . .
, 0 , m ) .
Proof.
The CDF, Π M ( a ) , meaning the probability that m < a is:
[Equation]
This is equivalent to integrating over the density function, π X ( x ) , within the hypersphere, ∥ x ∥ ≤ m :
[Equation]
M
[Equation]
∞
[Equation]
I
[Equation]
integration on an n -sphere of radius a
[Equation]
drdϕ
[Equation]
dϕ n − 1 , with Jacobian terms due to the change of coordinates to spherical ∝
[Equation]
a
[Equation]
) · r n − 1 dr.
▷ The s ∼ N ( 0 n , I n × n ) m ∼ χ ( m ; n + 1) p ( c ) ← m · s ∥ s ∥
▷ where χ ( m ; n + 1) ∝ m n exp( − m 2 / 2) τ ( c ) ∼ Unif (0 , ϵ )
▷ duration of the initial fixed-momentum evolution q ← q ( c ) + τ ( c ) · p ( c ) ▷ initial evolution of the position vector for time τ ( c ) d
= D − τ ( c ) · ∥ p ( c ) ∥
▷ This is the remaining distance p ← p ( c ) − ϵ ∇ U ( q ) ▷ initial evolution of the momentum vector for time
ϵ ▷
while a position full-step evolution is possible without exceeding the remaining distance, d : while ϵ · ∥ p ∥ < d do q ← q + ϵ · p ▷ position full-step d ← d − ϵ · ∥ p ∥
▷ update the remaining distance p ← p − ϵ ∇ U ( q ) ▷ momentum full step τ ( ∗ ) ← d ∥ p ∥ ; q ( ∗ ) ← q + τ ( ∗ ) · p ; p ( ∗ ) ← − p ;
▷ constructing the final proposal if u ∼ Unif (0 , 1) <
π Q ( q ( ∗ ) )exp ( −∥ p ( ∗ )
∥ 2 / 2 ) π
Q ( q ( c ) )exp ( −∥ p ( c ) ∥ 2 / 2 ) then return q ( ∗ ) else return q ( c )
Comparing (13) and (14) shows that π M ( m ) ∝ g ( m ) · m n − 1 .
□
The complete FDHMC sampler is illustrated in Algorithm 1.
It can be seen that in terms of implementation complexity, this algorithm is quite comparable with the baseline HMC.
Nonetheless, asour experimental results will show, FDHMC significantly outperforms the baseline.
Experimental evaluation
We compare the performance of FDHMC against 3 baselines: (a) static HMC (Neal, 2011), andtwo versions of No-U-turn sampler (Hoffman and Gelman, 2014) (NUTS) including (b) Dynamic
Slice HMC (DSHMC), which is the original implementation (Hoffman and Gelman, 2014), and (c)
Dynamic Multinomial HMC (DMHMC) (Betancourt, 2017), which is the current version of NUTSused in STAN (Carpenter et al., 2017).
For tuning HMC, the initial total simulation duration, λ = ϵL , is set to 2.0 and the step-size is tunedby dual averaging (Nesterov, 2009; Hoffman and Gelman, 2014).
If dual averaging does not converge(which can show itself in a numerical error or a tuned L < 3 or L > 200 ) we halve the value of λ andrepeat this process until the convergence conditions are satisfied.
For tuning DSHMC and DMHMC,we rely on Mici probabilistic programming language (Graham, 2019).
The automated parameter tuning of FDHMC is presented in the supplementary material.
This includestuning the step-size, ϵ , via dual averaging and the following heuristic for tuning the fixed distance,
D : We observe that if D is too large, its correlation with the expected distance of the successivesamples, E [ ∥ q ( i ) − q ( i − 1) ∥ ] , is low.
As such, we primarily draw N = 500 samples, q ( i ) , from an
FDHMC sampler that is associated with a sufficiently large fixed-distance, D ∗ .
Then, we set D = 253 (cid:80)
Ni =2 ∥ q ( i ) − q ( i − 1) ∥ / ( N − 1) , as the final fixed-distance.
To decide the initial D ∗ , we set ϵ ∗ =
1and then repeatedly double or halve the value of ϵ ∗ till the acceptance probability of the Langevinproposal that corresponds a momentum with expected magnitude ∥ p ∥ = √ 2Γ( n/ 2+1) / Γ(( n +1) / 2)(i.e.
the mean of χ ( x, n + 1) ) and step-size ϵ ∗ crosses 0.5.
Then we set D ∗ :
= 10 ϵ ∗ .
Our experimental models include: (a) n -dimensional multivariate normal distributions (MVN)where n ∈ { 10 , 30 , 100 , 300 } and the covariance matrix is drawn from a Wishard distributionwhich has an identity scale matrix and its degree of freedom is equal to the model’s dimension, n .
[Equation]
where the dimension, n ∈ { 5 , 10 , 50 , 100 } , σ 2 = 1 and k = 3 .
(c)
The posterior density function,
[Table]
π Q ( α, β | X, y ) (15), associated with Bayesian Logistic Regression for classification whereeach data point contains a vector of predictors, x i , and the class label y i ∈ {− 1 , 1 } .
[Equation]
We set σ 2 = 1 and apply the model to the following three data sets that are available from the
UCI repository (Frank et al., 2011): (c.1) Australian Credit Approval (AusCr) data set where eachdata point have 14 predictors (as such, n = 15 ) and we only use the first 100 data points in thedata set.
(c.2) SPECT Heart (SPECT) data set with 267 data points, each with 22 predictors.
(c.3)
German Credit (GrCr) data set with 1000 data points, each with 24 predictors.
In these 3 data sets,we normalise all predictors, to have zero mean and unit variance.
For each sampling algorithm, 50 independent MCMC chains are run and the average Effective Sample
Size per gradient (ESS/grad) ±
95% condifence interval is reported in Table 1.
The length of eachchain is 1000 samples which are drawn after an initial 200 burn-in draws.
In each chain iteration,all samplers start from a same initial states and the ESS is approximated by following the detailsdescribed by Hoffman and Gelman (2014).
The results show that in 8 out of the 11 experiments, FDHMC outperforms the baselines and in thehigh-dimensional experimental models (e.g. MVN-300, FNNL-100), the difference is substantial.
Conclusion
The main insight of the present work is that the fixed-time evolution of a phase state via the equationsof motion is intrinsically biased towards generating proposals that have low target probability(Limitation 1).
In HMC, this bias is compensated by drawing momentums from a normal distributioncentered at 0 .
Many such sampled momentums have small magnitudes and do not let the the stateleave the high-probability regions.
This causes a (second) bias that compensates the first bias.
Nevertheless, this counterbalancing is obtained in the expense of more correlation between thesuccessive states that are drawn from the high probability regions (Limitation 2).
In the light of
RJMCMC theory, we modified the core of the HMC algorithm and designed a sampler where thestate is evolved for a fixed traversed distance rather than a fixed evolution time.
This dynamics isnot biased towards the proposals that have low target probability and is coupled with a momentumdistribution that generates momentums which have a higher expected magnitude.
According to ourexperimental results, this can lead to a better exploration of the space and a higher effective samplesize while maintaining the high proposal acceptance probability of the baseline HMC.
The proposed FDHMC can be easily implemented efficiently and can be combined with manyvariations of HMC.
More specifically, even though for simplicity throughout we focused on combining
FD-leapfrogs with the baseline HMC, this mechanism can be combining with any variation of HMCthat relies on Stormer–Verlet leapfrog integration e.g. the case where the mass matrice is not unit.