A UXILIARY V ARIATIONAL MCMC
Anonymous authors
Paper under double-blind review
A BSTRACT
We introduce Auxiliary Variational MCMC, a novel framework for learning MCMC kernels that combines recent advances in variational inference with insights drawn from traditional auxiliary variable MCMC methods such as Hamiltonian Monte
Carlo.
Our framework exploits low dimensional structure in the target distribution in order to learn a more efﬁcient MCMC sampler.
The resulting sampler is able to suppress random walk behaviour and mix between modes efﬁciently, without the need to compute gradients of the target distribution.
We test our sampler on a number of challenging distributions, where the underlying structure is known, and on the task of posterior sampling in Bayesian logistic regression.
Code to reproduce all experiments is available at https://github.com/AVMCMC.
I NTRODUCTION
Markov Chain Monte Carlo (MCMC) and Variational Inference (VI) are well-established approaches to approximating expectations under the complex distributions p that frequently arise in machine learning and statistics (Wainwright & Jordan, 2008; Brooks et al., 2011).
VI is usually fast and cheap but often places strong restrictions on the class of approximating distributions leading to irreducible bias.
MCMC on the other hand is asymptotically exact but may require an infeasible amount of computation to converge.
Given the complimentary strengths and weaknesses of the two methods it is natural to wish to combine them (De Freitas et al., 2001; Salimans et al., 2015).
The most naive pairing of the two methods, simply using a variational approximation q as the proposal distribution in a Metropolis-Hastings sampler (see for example Gamerman & Lopes (2006) or De Freitas et al.
(2001)), is known to scale poorly with the dimension of the target distribution (De Freitas et al., 2001) and mix inefﬁciently when p is multi-modal.
In this paper we suggest an alternative approach inspired both by the successes of black box variational inference (Ranganath et al., 2014) and auxiliary variable MCMC methods, such as Hamiltonian
Monte Carlo (HMC) (Duane et al., 1987; Girolami & Calderhead, 2011).
The key contributions of our work are :
• A general framework for marrying variational inference with Markov Chain Monte Carlo in a way likely to produce efﬁcient samplers.
• The use of the auxiliary variational method to capture latent low-dimensional structure in our target distributions and exploit this structure to suppress random walk behaviour.
• The extension of the Metropolis-Hastings algorithm to continuous mixture proposals.
• The introduction and demonstration of a speciﬁc instance of our framework, the Auxiliary Variational Sampler (AVS).
Our sampler takes advantage of ﬂexible distributions parameterized by neural networks that can be trained in a fully black-box manner.
A UXILIARY VARIATIONAL MCMC
The key idea behind Auxiliary Variational MCMC is to exploit structure present in the target distribution p ( x ) by ﬁrst ﬁtting a parameterized variational approximation in an augmented space.
This allows the sampler to leverage learned low-dimensional structure.
In the subsequent sections we introduce the auxiliary variational method and describe how it can be combined with a carefully chosen class of proposal distributions to construct an efﬁcient sampler.
2. M IXTURE PROPOSAL MCMC
To develop a valid MCMC algorithm we need to construct an ergodic Markov chain whose stationary distribution is our target distribution p ( x ) .
In order to do this we introduce a Metropolis-Hastings (Gamerman & Lopes, 2006) like algorithm with a specially chosen form of proposal distribution.
This proposal can be naturally combined with the auxiliary variational method which we will introduce in section 2.2.
We ﬁrst consider a mixture proposal distribution of the following form 1 :
[Equation]
We prove in supplement (6) that the following forms a valid MCMC sampling step:
Sample a from ˜ q
( a | x ) 2.
Sample x (cid:48) from ˜ q ( x (cid:48) | a ) 3.
Accept the candidate sample x (cid:48) with probability
[Equation]
otherwise reject x (cid:48) and deﬁne the new sample as a copy of the current x , namely x (cid:48)
= x .
It is worth noting that the above procedure is not equivalent to simply performing Metropolis-Hastings in the joint ( x, a ) space, but is a sampler in x alone, speciﬁed by the marginalized proposal distribution given by (1).
The two become equivalent only in the case that ˜ q ( a | x ) is constant, see supplement (6).
The mixture proposal can be extended to an arbitrary number of auxiliary variables as long as the acceptance ratio is adjusted accordingly.
For example with two auxiliary variables, the proposal becomes:
[Equation]
where ˜ q ( a (cid:48) | a ) is another arbitrary proposal distribution.
The acceptance probability is now given by:
[Equation]
For a proposal ˜ q ( a (cid:48) | a ) that is symmetric, ˜ q ( a (cid:48) | a ) =
˜ q ( a | a (cid:48) ) , this simpliﬁes to:
[Equation]
We now show how the mixture proposal can be naturally combined with the auxiliary variational method.
2. T HE AUXILIARY VARIATIONAL METHOD
The auxiliary variational method (Agakov & Barber, 2004; Ranganath et al., 2016) is a strategy for creating more expressive families of approximating distributions for use in variational inference.
Instead of minimizing the Kullback-Leibler divergence between our approximating distribution q ( x ) and our target p ( x ) , we instead minimize the divergence in an augmented space ( x, a ) with additional auxiliary variables a .
In doing this, we ﬁrst deﬁne a joint q φ ( x, a ) in the augmented space and a joint p ( x, a ) =
p θ ( a | x ) p ( x ) where θ and φ are parameters.
Marginalizing p ( x, a ) over a recovers p ( x ) by construction.
We are free to decide on the dimension and form (e.g. continuous, discrete or mixed) for a and our objective is the Kullback-Leibler divergence between the joint approximation q and joint target p :
[Equation]
Moving to the joint space allows us to tractably learn complex approximating distributions q φ ( x, a ) whose marginal, q φ ( x )
= (cid:82) q φ ( x | a ) q φ ( a ) da , may be intractable.
Initialize φ and θ (cid:46) Fit the variational distribution while Not Converged do { a 1 , . . .
, a N } ∼ q φ ( a ) { x 1 , . . .
, x N } ∼ q φ ( x | a n ) (cid:46) re-parameterized
L ← 1 N (cid:80) n log q φ ( x n | a n ) q φ ( a n ) p ( x n ) p θ ( a n | x n )
φ ← φ − η ∇ φ L θ ← θ − η ∇ θ L end a ∼ q φ ( a ) (cid:46) Mixture-Model MCMC sampling x 0 ∼ q φ ( x | a ) for t = 0 , . .
, T do a ∼ p θ ( a | x t ) a (cid:48) ∼ N (cid:0) a (cid:48) a, σ 2 a I (cid:1) x (cid:48) ∼ q φ ( x | a (cid:48) )
A ← p ( x (cid:48) ) p θ ( a (cid:48) | x (cid:48) )
q φ ( x t | a ) p ( x t ) p θ ( a | x t ) q φ ( x (cid:48) | a (cid:48) )
(cid:46) acceptance ratio u ∼ U [0 , 1] (cid:46) sample u from uniform distribution if u ≤ A then x t +1 = x (cid:48) else x t
+
=
x
t end end
The Auxiliary Variational Sampler.
The algorithm begins by ﬁtting a mixture variational distribution to the target p ( x ) by stochastic gradient descent, based on the auxiliary variational method.
After the auxiliary variational model is ﬁtted to p ( x ) the MCMC sampling phase begins and is a modiﬁed form of
Metropolis-Hastings sampling.
Whilst sampling in high-dimensional spaces is difﬁcult, in many cases of interest the high probability or typical regions lie close to much lower dimensional manifolds.
A key point of ﬁtting an auxiliary variational distribution is that, by constraining a to have dimension much lower than x , we can then exploit this low dimensional manifold to form a more efﬁcient proposal distribution, staying close to the manifold of signiﬁcant probability.
2. C OMBINING AUXILIARY VARIATIONAL INFERENCE AND MCMC
After ﬁtting our variational approximation to p ( x ) , we will have three variational distributions: q ∗ ( x | a ) , q ∗ ( a ) and p ∗ ( a | x ) that are the solution of the optimization problem given in (6).
These distributions approximately satisfy the following relationships:
[Equation]
which become exact iff the divergence in (6) becomes zero.
Here p ∗ ( a | x ) is a learned stochastic mapping from the high-dimensional target space to the low-dimensional auxiliary space and q ∗ ( x | a ) is a mapping in the opposite direction.
These learned mappings can be composed in a variety of ways to form marginal proposal distributions of the kind discussed in section 2.1 and also joint proposals.
We now discuss some natural combinations and argue that these may form good proposal distributions in practice.
2.3. N AIVE PROPOSAL DISTRIBUTIONS
A simple proposal constructed from our variational distribution is to perform Metropolis-Hastings in the joint ( x, a ) space with an independent proposal given by:
[Equation]
If the variational approximation is accurate such that q ( x, a ) ≈ p ( x ) p ( a | x ) then one might expect this to have high acceptance probability.
A potential downside of this scheme is that proposals are independent between time-steps and in high dimensions this may cause the acceptance ratio to become impractically low.
Another natural MCMC proposal distribution to consider is:
[Equation]
[Figure] where we have replaced the arbitrary proposal distribution of (1) with the optimal variational distributions learned by minimizing the joint divergence given in (6).
We might expect this to be a promising proposal distribution, with high acceptance probability, as it already approximately satisﬁes the stationarity criterion of our MCMC chain.
That is
[Equation]
However, if the variational distribution truly captures underlying structure in p ( x ) , it has been our experience that p ∗ ( a | x ) and q ∗ ( x | a ) become approximate inverses causing ˜ q ( x (cid:48) | x ) to resemble the identity mapping and thereby slowing mixing in the chain.
2.3. A N AUXILIARY RANDOM WALK PROPOSAL DISTRIBUTION
To avoid the above identity mapping issue and encourage the sampler to take large steps, we introduce an additional random perturbation in the auxiliary a -space.
That is we use a proposal distribution:
[Equation]
where ˜ q ( a (cid:48) | a ) =
N (cid:0) a (cid:48) a, σ 2 a I (cid:1) is a Gaussian proposal with mean a and isotropic covariance σ 2 a I .
The overall algorithm can be viewed as mapping from the high-dimensional x to the low-dimensional a , performing a random walk in the low dimensional auxiliary a -space and subsequently mapping back up to the high-dimensional target space x , see ﬁgure (1).
In high dimensions our target distribution is likely to have high probability only close to some lowdimensional manifold.
The traditional Random-Walk-Metropolis Algorithm proposes new samples x (cid:48) by perturbing the most recent sample in an arbitrary direction, usually based on a Gaussian proposal ˜
q ( x (cid:48) | x ) = N (cid:0) x (cid:48)
x, σ 2 x I (cid:1) ; however, in high-dimensional spaces almost all directions will correspond to steps off the manifold of high probability and thus out of the typical set.
In our case, we perform the random perturbation in the low-dimensional auxiliary space and our variational distribution q ∗ ( x (cid:48) | a (cid:48) ) ensures that this move remains within the manifold of high density.
We depict this process in ﬁgure (1) and provide explicit examples of the learned latent structure of real distributions in section 3.
Much like HMC, we are able to exploit the addition of auxiliary variables to encourage large moves of high probability in the target space but unlike HMC we do so by explicitly modelling the structure of the target distribution and do not require gradients of the target density.
We are able to take large steps because small perturbations in a can correspond to large perturbations within the manifold of high probability ; unlike methods that adapt to the local geometry of the probability manifold (Girolami & Calderhead, 2011; Strathmann et al., 2015), our variational ﬁt allows for larger, non-local moves in the x -space.
Note that, unlike HMC, in general there is no requirement that ( x, a ) are continuous random variables.
[Figure]
2. C HOOSING THE VARIATIONAL FAMILY
Within the above framework, we still have to decide on the structure of the variational distributions, q ( a, x ) and p ( a | x ) .
We take inspiration from recent successes in the generative modeling of complex data distributions (Kingma & Welling, 2014), and propose to parameterize our variational distributions using deep neural networks.
For continuous x we choose the following structure for each of our approximating distributions:
[Equation]
Where q φ ( x | a ) and p θ ( a | x ) are both diagonal Gaussian distributions whose means and covariances are parameterized by neural networks with parameters shared between the mean and covariance.
In some experiments we also choose p θ ( a | x ) to be a mixture of diagonal Gaussians (see the supplementary material).
Key to the ﬂexibility of the auxiliary variational method is that whilst q ( a, x ) can be evaluated point-wise, the marginal q ( x ) is a much richer approximating distribution whose density we typically cannot evaluate point-wise.
Whilst we can thus evaluate our joint approximating density we still can’t compute the objective:
[Equation]
However, we recognize that an unbiased estimator of the KL divergence can be obtained by sampling from q φ ( x, a ) .
We use the standard re-parameterization trick of Kingma & Welling (2014) to reduce the variance in the corresponding gradient estimator.
We refer to this version of the algorithm as the Auxiliary Variational Sampler (AVS).
The full algorithm is given in algorithm (1).
Reparameterisation requires the gradient of log p ( x ) ; if this is prohibitively expensive then the scorefunction estimator may be used (Kleijnen & Rubinstein, 1996).
Within this framework, different mixture proposals could be considered, based on the ﬁtted variational distribution, but our experience is that the Auxiliary Random Walk proposal is effective in our experiments.
E XPERIMENTS
To demonstrate the beneﬁt of ﬁtting an auxiliary variational approximation we ﬁrst test our sampler on a number of distributions with known low-dimensional structure.
We show how AVS is both able to recover the latent structure and exploit it for the purposes of sampling.
We then demonstrate the sampler on more realistic problems.
Ring density
The ﬁrst distribution is in two dimensions x
= ( x 1 , x 2 ) .
The distribution is a ring of high probability, centered at a ﬁxed distance from the origin, ﬁgure (2b).
We construct the ring density to have explicit latent structure by deﬁning the target distribution as:
[Equation]
Since this ring distribution is highly constrained, pure random walk sampling in x will not efﬁciently move around the ring.
However, provided we can ﬁnd a variational ﬁt to capture the ring structure, we expect our AVS sampler to be able to move more efﬁciently.
To train our AVS sampler, we ﬁt
[Figure]
an auxiliary variational approximation with a 1 dimensional continuous mixture (see supplementary for details).
We can examine the latent structure learned by the variational distribution by drawing samples from p ∗ ( a | x ) as we vary x
= (cos( θ ) , sin( θ )) for θ ∈ [0 , 2 π ] .
Looking at ﬁgure (3), we can see that the variational distribution has successfully captured the latent structure, automatically learning to map points of high probability in x onto distinct auxiliary variables.
Used with the mixture proposal (10) this results in an effective sampler that is able to take much larger step sizes than random walk Metropolis (RWM) as shown in ﬁgure (2).
Mixtures of Gaussians and Student T The second pair of densities we wish to sample from is a two dimensional mixture of Gaussians with highly separated means (with a distance of 20 between the means and a standard deviation of 1), ﬁgure (2), and a two dimensional mixture of Student T distributions.
These distributions have discrete latent structure and are challenging both for random walk Metropolis (RWM) (Gamerman & Lopes, 2006) and more advanced samplers such as HMC, which fail to ﬁnd more than one mode in any reasonable amount of time or can struggle with the heavy tails.
Our sampler has no problem ﬁnding both modes and, as shown by the plot of consecutive samples in ﬁgure (4), is able to hop between modes in single steps.
We use a 1-dimensional continuous auxiliary variable.
Posterior sampling in Bayesian models The low-dimensional distributions above demonstrate the ability of our sampler to learn and exploit latent structure.
Here we analyze the performance of our sampler on a more realistic problem: posterior sampling in Bayesian logistic regression.
We use the heart data-set used by Song et al.
(2017), which has 13 covariates and 270 data-points.
We tune all algorithms to maximize effective sample size (see supplement (8)) at convergence.
In the case of HMC we use a number of initial runs to select an appropriate step-size and then tune the number of leapfrog steps.
In all cases, we run a chain for 10000 burn-in steps and then draw 20000 samples to
[Figure] calculate diagnostics. In order to assess convergence and ensure the validity of the results, we monitor the Gelman-Rubin statistic (Gelman & Rubin, 1992) and visually inspect trace-plots of parameters.
3. E VALUATING PERFORMANCE
Evaluation of MCMC methods is notoriously difﬁcult (Gelman & Rubin, 1992).
Even diagnosing convergence is challenging and there is no clear agreement on the "correct" metric of performance.
In practice there are two factors that are of primary concern to an end-user:
How easy is the sampler to implement and tune?
and how computationally efﬁcient is the resulting sampler?
We evaluate our performance by comparison to RWM, HMC and two recently proposed neural adaptive samplers:
A-NICE-MC (Song et al., 2017) and L2HMC (Levy et al., 2018), discussed further in section 4.
For learned samplers there is an additional cost involved in training the sampler that may be offset by improved performance.
As a baseline, we include the independent proposal given by equation (8).
We choose here to focus our quantitative evaluation on the effective Sample Size (ESS), which measures how many independent samples would be equivalent to an MCMC sample, and the trainingtime for the learned samplers.
We choose not to focus our investigation on burn-in time since AVS, being initialized by the variational distribution, often has short burn-in relative to competitive methods.
For ESS calculations, we use the batch-means estimator applied to a single long chain as it has been shown to be more reliable than many alternatives (Thompson, 2010), exact details are given in the supplement.
For the experiments with mixtures of Gaussians we do not present ESS for HMC and RWM as both algorithms clearly fail to converge, getting trapped in one of the local modes.
For all distributions except the Mixture of Gaussians, the independent sampler also failed, getting quickly stuck in a region of high-probability where the acceptance rate dropped to near zero.
To ensure consistency in timing [Table] all methods were implemented using Tensorﬂow 1.10 (Abadi &
Agarwal, 2015) and run on a single Tesla K80 GPU.
Code to reproduce all experiments can be found at github.com/AVMCMC.
For L2HMC and A-NICE-MCMC, we perform a random search over hyper-parameters ensuring we include the parameters used in the original papers wherever possible.
The ﬁgures reported are the mean of 10 independent runs alongside their standard
[Table] [Table]
D ISCUSSION AND R ELATED W ORK
The results demonstrate that Auxiliary MCMC can be used to construct a practical sampler that is able to exploit low dimensional structure and mix between modes.
AVS produces competitive ESS, and initializing from a variational approximation reduces the need to burn-in.
AVS also offers a straightforward method to interpolate between pure VI and MCMC.
Though the more recent neural adaptive samplers have higher effective sample sizes, this comes at a high cost in training time that, in our moderately sized problems, overwhelms their beneﬁt.
It was also our experience that training L2HMC is very sensitive to correct tuning of many hyper-parameters.
In a search over 42 training conﬁgurations only 2 conﬁgurations converged to samplers able to mix between modes in the Mixture of Gaussian experiments.
The difﬁculty of tuning is hard to quantify and will no doubt vary with the experience of the practitioner.
We nonetheless see this as a possible barrier to the adoption of more complex adaptive methods and a possible explanation for the popularity of RWM.
We found qualitatively that AVS and A-NICE-MC were more straightforwards to tune with good performance from many hyper-parameter conﬁgurations.
When training time is included, the neural samplers often under-perform HMC but we expect this affect to diminish with data-size.
The work most similar to ours is Variational MCMC (De Freitas et al., 2001), in which a variational approximation is used as a proposal distribution in independent Metropolis-Hastings.
To overcome the poor scaling with dimension of the independent Metropolis algorithm, the authors interleave their variational proposal distribution with traditional RWM and make block proposals.
Although their use of a variational distribution aids the convergence of MCMC, their method still struggles to mix efﬁciently even in relatively low dimensional problems.
By introducing low dimensional auxiliary variables, we are able both to ﬁt a more accurate approximating distribution and to leverage the learned low dimensional structure to take very large steps in the target space, even hopping between modes.
In Salimans et al.
the authors also consider a combination of MCMC and auxiliary variational inference but take a different approach, choosing to use MCMC kernels to design more ﬂexible variational distributions rather than constructing an MCMC sampler at all.
There have also been other recent attempts to parameterize MCMC kernels with neural networks that don’t start from variational inference.
A-NICE-MCMC (Song et al., 2017), attempts to learn
MCMC transition kernels parameterized by volume preserving ﬂows (Dinh et al., 2015).
They craft an adversarial objective to train their kernels and use a discriminator that examines pairs of samples, in order to encourage fast mixing.
To ensure they have a valid sampler, they use a bootstrap procedure starting from an existing MCMC algorithm.
L2HMC (Levy et al., 2018) is an extension of HMC, that parameterizes a scaling and shift of Hamiltonian dynamics with neural networks and trains this parameterization to optimize the expected squared step-size.
They show that the introduction of ﬂexible neural networks to HMC vastly improves the ability to mix between modes but still require access to the gradient of the target density.
For any adaptive MCMC method, care has to be taken to preserve the ergodicity of the chain during adaptation (Andrieu & Thoms, 2008; Roberts & Rosenthal, 2009).
To overcome this, all of the above methods, including ours, stop adaptation before collecting any samples.
A potential problem with this strategy is that if the learned sampler is specialized to only a part of the distribution or the variational approximation has entirely missed regions of high probability, then such regions are unlikely to be explored.
A possible avenue for future work would be to investigate iterative improvement of our variational approximation with samples drawn from the true distribution rather than our variational approximation.
Another potential avenue for ensuring good coverage of the variational approximation, would be to combine our reverse KL objective, (14), with the KL divergence in the forward direction as this is known to encourage moment matching rather than mode seeking behavior (Minka, 2005).
C ONCLUSION
We introduced a novel framework for combining MCMC and Variational Inference that makes use of the auxiliary variational method to capture low-dimensional latent structure.
We have explored a particular black-box instance of this framework and demonstrated that it can be used to create a fast mixing sampler without the need to take gradients of the target distribution.
The method is competitive with other recent geometry-learning approaches and opens up additional avenues for exploring how to combine the best of variational inference and sampling.
S UPPLEMENTARY M ATERIAL M ETROPOLIS -H ASTINGS WITH A M IXTURE PROPOSAL
We deﬁne a mixture proposal using an auxiliary variable a as
[Equation]
We consider the transition kernel
[Equation]
It is trivial to check that this deﬁnes a valid distribution q ( x (cid:48) , a | x ) .
We wish to set f ( x (cid:48) , a, x ) such that p ( x ) is a stationary distribution of q ( x (cid:48)
| x ) .
That is
[Equation]
The right hand side of the above equation can be written as
[Equation]
) which simpliﬁes to
[Equation]
For the above to hold we require (changing the integration variable x (cid:48)(cid:48) to x and a (cid:48) to a )
[Equation]
Writing
[Equation]
) and considering the function
[Equation]
one can readily verify that
[Equation]
23) meaning that (20) is satisﬁed.
We can then sample from q ( x (cid:48) , a | x ) by ﬁrst sampling a from ˜ q
( a | x ) and then sampling from ˜ q ( x (cid:48) | a ) and accepting with probability f
( x (cid:48) , a, x ) .
That is :
Sample a from ˜ q
( a | x ) 2.
Sample x (cid:48) from ˜ q ( x (cid:48) | a ) 3.
Accept the candidate sample x (cid:48) with probability
[Equation]
otherwise reject x (cid:48) and deﬁne the new sample as a copy of the current x , namely x (cid:48)
= x .
The extension to the case of more than 1-auxiliary variable follows naturally using the same argument as above.
If we were to perform Metropolis-Hastings in the joint space of ( x, a ) then the acceptance probability would be given by:
[Equation]
which reduces to (24) if p ( a | x ) is a constant.
E XACT P ARAMETERIZATION OF THE VARIATIONAL DISTRIBUTIONS 7. L OW DIMENSIONAL EXAMPLES
For the mixtures of Gaussians and ring density experiments we used an auxiliary dimension ofand the target dimension was 2.
For the mixture of Gaussians, the variational distributions had the following form:
[Equation]
[Equation] [Equation]
Where the µ φ ( x ) and Σ φ ( a ) are 3 layer feed-forward neural networks with 10 neurons in each layer, all but the last layer were shared between them.
Similarly µ θ ( x ) and Σ θ ( x ) were also 3 layer feed-forward neural networks with all but the last layer shared.
We used tanh non-linearities in all but the ﬁnal layer which was simply linear.
In the case of the ring density experiments p θ ( a | x ) = Σ k π k N (cid:0) a µ kθ ( x ) , Σ kθ ( x ) (cid:1) was a mixture of 2 Gaussians, each parameterized as above.
The mixture weights π k were also learned and were paramterized as π 1 = σ ( τ ) , π 2 = 1 − π 1 to allow for unconstrained optimization.
7. R EGRESSION EXAMPLES
For Bayesian logistic regression, the target dimension was 14, the auxiliary dimension used wasdimensional and the number of hidden units was 300.
The structure was otherwise the same as for the low dimensional experiments.
C ALCULATION OF THE EFFECTIVE SAMPLE SIZE
The effective sample size is calculated as the reciprocal of the auto-correlation-time.
It is intended to represent the number of truly independent samples that would be equivalent to a correlated sample drawn using MCMC, in terms of the variance of estimated quantities.
One deﬁnition of the auto-correlation-time, for a 1-dimensional chain, is:
[Equation]
(28) where C τ = E [ x t , x t + τ ] is the lag- τ auto-correlation of the stationary converged MCMC chain.
Though multivariate deﬁnitions of the auto-correlation exist, it is common practice to report the lowest effective-sample-size across all dimensions.
For ease of comparison, we adopt this practice.
There are numerous methods for estimating ρ (Thompson, 2010).
It is worth noting however that simply estimating C τ from multiple chains and substituting the estimates into the above formula does not yield a consistent estimator except in very simple cases when the auto-correlations are guaranteed to be positive.
In general however, the auto-correlations can be of both negative and positive variance.
The estimator formed by substituting the empirically estimated auto-correlations does not have a variance that goes to 0 as the sequence length goes to inﬁnityThompson (2010).
Instead we use the batch-means estimator.
8. BATCH - MEANS ESTIMATOR
An estimator of the auto-correlation time is computed using "batch-means" (Thompson, 2010).
A single long sequence is split into m sub-sequences and the mean of each sub-sequence is calculated.
The auto-correlation-time is then estimated by using the ratio of the variance of the batch-means to the variance of the overall sequence.
The estimator is given by:
[Equation]
(29) where s 2 m is the variance of the batch-means and s 2 is the variance of the entire chain.
F URTHER E XPERIMENTAL D ETAILS 9. A BSOLUTE T RAINING T IMES
[Table]