Optimal Scaling for Locally Balanced Proposals in Discrete Spaces
Anonymous Author(s)
AffiliationAddress email
Abstract
Optimal scaling has been well studied for Metropolis-Hastings (M-H) algorithmsin continuous spaces, but a similar understanding has been lacking in discretespaces.
Recently, a family of locally balanced proposals (LBP) for discrete spaceshas been proved to be asymptotically optimal, but the question of optimal scalinghas remained open.
In this paper, we establish, for the first time, that the efficiencyof M-H in discrete spaces can also be characterized by an asymptotic acceptancerate that is independent of the target distribution.
Moreover, we verify, boththeoretically and empirically, that the optimal acceptance rates for LBP and randomwalk Metropolis (RWM) are 0 .
574 and 0 .
respectively.
These results also helpestablish that LBP is asymptotically O ( N 23 ) more efficient than RWM with respectto model dimension N .
Knowledge of the optimal acceptance rate allows one toautomatically tune the neighborhood size of a proposal distribution in a discretespace, directly analogous to step-size control in continuous spaces.
We demonstrateempirically that such adaptive M-H sampling can robustly improve sampling ina variety of target distributions in discrete spaces, including training deep energybased models.
Introduction
The Markov Chain Monte Carlo (MCMC) algorithm is one of the most widely used methods forsampling from intractable distributions [1].
An important class of MCMC algorithms is MetropolisHastings (M-H)
[2, 3], where new states are generated from a proposal distribution followed by a
M-H test.
The efficiency for M-H algorithms depends critically on the proposal distribution.
Forexample, gradient based methods, such as the Metropolis Adjusted Langevin Algorithm (MALA) [4],
Hamiltonian Monte Carlo (HMC) [5] and their variants [6, 7] substantially improve the performanceof M-H algorithms in theory and in practice, compared to naive Random Walk Metropolis (RWM),by leveraging gradient information to guide the proposal distribution [8].
Despite many advances, progress in gradient based methods has generally focused on continuousspaces.
However, [9] recently proposed a general framework of locally balanced proposals (LBP)for discrete spaces, where a proposal distribution is designed to utilize probability changes betweenstates.
Subsequently, [10] accelerated the sampler by using gradient information to approximate theprobability change.
In empirical evaluations, similar to gradient based samplers in continuous spaces,
LBP significantly outperforms RWM and other samplers in discrete spaces.
However, both [9] and[10] constrain the proposal distribution to lie within a 1-Hamming ball; i.e., only one site of the statevariable is allowed to change per M-H step.
Such a restricted update reduces the efficiency of thesampler.
[11] noticed this problem and modified the proposal distribution to allow multiple sites tobe changed per M-H step.
Although such larger updates significantly improve efficiency, [11] do not show how to determine the update size, leaving the number of sites updated in an M-H step as ahyperparameter to tune.
In continuous spaces, the scale of the proposal distribution is known to be a critical hyperparameterfor obtaining an efficient M-H sampler.
For example, consider a Gaussian proposal N ( x, σ 2 ) formodifying a current state x with scale σ .
If σ is too small, the Markov chain will converge slowlysince its increments will be small.
Conversely, if σ is too large, the M-H test will reject too high aproportion of proposed updates.
A significant literature has studied optimal scaling for gradient basedmethods in continuous spaces [12, 13, 8, 14], showing that the optimal scaling can be adaptively tunedw.r.t.
the acceptance rate, independent of the target distribution.
Such results suggest a direction forsolving the optimal scaling problem for LBP.
However, the underlying techniques for approximatinga diffusion process cannot be directly applied to LBP given its discrete nature.
In this work, we consider an asymptotic analysis as the dimension of the discrete model, N , convergesto infinity.
Starting with a product distribution, we prove that the asymptotic efficiency of LBP in discrete spaces is 2 R Φ( − 12 λ 1 R 32 /N ) with an asymptotic acceptance rate of 2Φ( − 12 λ 1 R 32 /N ) , wherethe scale R represents the number of sites to update per M-H step.
Therefore, the asymptoticallyoptimal scale of the proposal distribution is R
=
O
( N 23 ) with an asymptotically optimal acceptancerate of 0 . 574 , independent of the target distribution.
Moreover, for RWM in a discrete space, we showthat the asymptotic efficiency and acceptance rate are 2 R Φ( − 12 λ 2 R 1 2 ) and 2Φ( − 12 λ 2 R 1 2 ) , respectively.
Hence, the asymptotically optimal scale is O (1) and the asymptotically optimal acceptancerate is 0 . 234 for RWM.
By comparing LBP and RWM at their respective optimal scales, it can bedetermined that LBP is O ( N 23 ) more efficient than RWM.
These asymptotically optimal acceptance rates are robust in the following respects.
First, althoughthe initial derivation is established w.r.t.
product distributions, the result can be expanded to moregeneral distributions.
Second, the efficiency is not sensitive around the optimal acceptance rate.
Forexample, whereas 0 .
574 is the optimal acceptance rate for LBP, the algorithm retains high efficiencyfor acceptance rates between 0 . 5 and 0 .
Based on these observations, we propose an adaptive LBP(ALBP) algorithm that automatically tunes the update scale to suit the target distribution.
We validate these theoretical findings in a series of empirical simulations on the Bernoulli model, the
Ising model, factorized hidden Markov models (FHMM) and restricted Boltzmann machines (RBM).
The experimental outcomes comport with the theory.
Moreover, we demonstrate that ALBP canautomatically find near optimal scales for these distributions.
We also use ALBP to train deep energybased models (EBMs), finding that it reduces the MCMC steps needed in contrastive divergencetraining [15, 16], significantly improving the efficiency of the overall training procedure.
Background
Metropolis-Hastings Algorithm Let π denote the target distribution.
Given a current state x ( n ) , a
M-H sampler draws a candidate state y from a proposal distribution q ( x ( n ) , y ) .
Then, with probabilitymin (cid:110) 1 , π ( y ) q ( y,x ( n ) )
π
( x ( n ) )
q ( x ( n ) ,y ) (cid:111)
the proposed state is accepted and x ( n +1)
=
y ; otherwise, x ( n +1) =
x ( n ) .
In this way, the detailed balance condition is satisfied and the M-H sampler generates a Markov chainx 0 , x 1 , ... that has π as its stationary distribution.
Locally Balanced Proposal .
The locally balanced proposal (LBP) is a special case of the pointwiseinformed proposal (PIP), which is a class of M-H algorithms for discrete spaces [9] using the proposaldistribution Q g ( x, y )
∝ g ( π ( y ) /π
( x )) such that g is a scalar weight function.
Zanella [9] showsthat the family of locally balancing functions G = { g : R + → R + , g ( t ) =
tg ( 1 t ) , ∀ t > 0 } (e.g.g ( t ) = √ t or t t +1 ) is asymptotically optimal for PIP.
Hence, PIP with a locally balanced functionfor its weight function is referred to as LBP.
Despite having good proposal quality, PIP requiresthe weight g ( π ( z ) /π ( x )) to be calculated for all candidate states z in the neighborhood of x , whichresults in its high computational cost.
Grathwohl et al.
[10] propose to estimate the probability changeby leveraging the gradient, improving the scalability of LBP.
Locally Balanced Proposal with Auxiliary Path .
Sun et al.
[11] generalize LBP by introducingan auxiliary path sampler, which allows multiple sites to be updated per M-H step.
In particular,
Sun et al.
[11] sequentially selects the update indices without replacement, and uses these indices as auxiliary variables to keep the proposal distribution tractable while preserving the detailed balancecondition.
Although this can achieve significant improvements in empirical performance, Sun et al.[11] manually tune the update size per M-H step, and leave the optimal scale problem open.
Main Result
3.1 Problem Statement
We establish asymptotic limit theorems for two M-H algorithms in discrete spaces: the locallybalanced proposal (LBP) and random walk Metropolis (RWM).
Following previous work [12, 13,14, 17], we conduct our analysis on a product probability measure π .
In particular, for a state space
X = { 0 , 1 } N , we consider a factored target distribution
[Equation]
where each site is assumed to have a sufficiently large probability for being both 0 and 1 ; that is, for afixed ϵ ∈ (0 , 14 ) , we assume the target distribution belongs to:
[Equation]
where we denote a ∧ b = min { a, b } .
To measure the efficiency of the sampler, an ergodic estimatevaries with the objective function considered.
Alternatively, we use a natural progress estimate: theexpected jump distance (EJD).
Denote P θ as the transition kernel, d ( x, y ) as the Hamming distancebetween x and y .
For a M-H sampler parameterized by θ , its expected jump distance ρ ( θ ) andcorresponding expected acceptance rate a ( θ ) are
[Equation]
In continuous space, the limit of sampling process is a diffusion process, whose efficiency is determined by the expected squared jump distance (ESJD) [8].
In discrete space, the limit of the samplingprocess is a jump process and EJD is the correct metric (see Appendix B.1).
3.2 Locally Balanced Proposal
We consider the M-H sampler LBP- R , where R refers to flipping R indices in each M-H step.
Given a current state x , LBP-R calculates the weight w j for flipping index j as in PIP.
Since we areconsidering a binary target distribution of the form (1), we have
[Equation]
(4) where g is a locally balanced function.
Following [11], LBP-R select indices u r with probability 110 P ( u r = j ) ∝ w j sequentially for r = 1 , ..., R , without replacement.
The new state y is obtained byflipping indices u 1: R of x .
If we consider u as an auxiliary variable, the accept rate A ( x, y, u ) in the
M-H acceptance test can be written as
[Equation]
where W ( x, u ) =
N (cid:88) i =
[Equation]
w
[Equation]
R
[Equation]
r
[Equation]
w
[Equation]
r (5)
From theorem 1 in [11], the auxiliary sampler LBP-R satisfies detailed balance.
A M-H step of
LBP-R is summarized in Algorithm 1.
Algorithm 1: A
-H of LBP-R and ALBP 1 Given current state x ( n ) , current R t , initialize candidate set C
= { 1 , ..,
N } ; 2 for r =
..., R or r =
.., rounding
R t ) do Sample u r with P ( u r = j ) ∝ w j
( x ( n ) )1 { j ∈C} ; Pop u r out of the candidate set: C ← C\{ u r } ; 5 end 6 Obtain y by flipping indices u 1 , ..., u R of x ( n ) .
; 7 if rand(0,1) <
A ( x ( n ) , y, u ) then x ( n +1)
=
y else x ( n +1) =
x ( n ) ; 8 if t < T warmup then R t +1 ← R t
+ ( A ( x ( n ) , y, u ) − 0 . 574) ;
3.3 Optimal Scaling for Locally Balanced Proposal
We are now ready to state the first asymptotic theorem.
Theorem 3.1.
For arbitrary sequence of target distributions { π ( N ) } ∞ N =1 ⊂ P ϵ , the M-H sampler
LBP-R with a locally balanced weight function g obtains the following, if R =
⌊ lN 23 ⌋ ,
[Equation]
where Φ is the c.d.f.
of standard normal distribution and λ 1 only depends on π ( N )
[Equation]
The definition of λ 1 in (7) explains the motivation of restricting the target distributions in (2).
Infact, introducing the ϵ gives upper and lower bounds of λ 1 .
When all p j are arbitrarily close to 12 ,( w j (0) − w j (1)) 2 in numerator will be zero, so is λ 1 .
As a result, the acceptance rate will always be1 .
Else, when all p j are arbitrarily close to 0 or 1 , E x [ 1 N (cid:80)
Ni =
1 w i ( x i )] in denominator will be zero,and λ 1 will be infinity.
As a result, the acceptance rate will always be 0 .
So, we have to make themild assumption in (2) to assure the following asymptotic result holds.
A more detailed discussionabout ϵ is given in Appendix B.2.
Corollary 3.2.
The optimal choice of scale for R = lN 23 is obtained when the expected acceptancerate is 0 .
574 , independent of the target distribution.
Proof.
When R = lN 23 , denote z = λ 23 1 l , we have:
[Equation]
It means the optimal value of z is independent of the target distribution π ( N ) .
As Φ is known, we cannumerically solve z = 1 . 081 , and the corresponding expected acceptance rate is a = 0 .
Proof of Theorem 3.
Denote the current state as x and a new state proposed in LBP-R as y .
Consider the acceptance rate
A ( x, y, u ) in (5).
Using the fact that, if index j is not flipped then w j ( y ) =
w j ( x ) , we have:
[Equation]
where (9) takes advantage of the property of a locally balanced function.
Hence, the acceptance rate
A ( x, y, u ) can be simplified to:
[Equation]
From the definition in (5), we have W ( x, u ) =
W ( y, u ) .
Denote i
∧
j = min { i, j } and i ∨ j =max { i, j } , we have the following approximation:
Lemma 3.3.
Define W = E x,u [ W ( x, u )] .
We have: lim N → 0 (cid:80)
[Equation]
log
[Equation]
Ri
[Equation]
w ui
[Equation]
x
[Equation]
/W
[Equation]
x,u
[Equation]
ui
[Equation]
y
[Equation]
/W
[Equation]
y,u ) ) −( A + B ) = 0 , where
[Equation] [Equation]
To analyze A and B , we reverse the order of x and u .
In particular, instead of first sampling x ∼ π ( x ) ,then sampling u ∼ p ( x | u ) , we use a reversed order where we first determine the indices u , then thevalues of x u , and finally the values of x − u .
Lemma 3.4.
The joint distribution p ( x, u ) =
π ( x ) p ( u | x ) can be decomposed in the following form:
[Equation]
Denote j / ∈ u 1: r − 1 represents j ̸ = u
i for i = 1 , ..., r − 1 , the conditional probabilities are
[Equation]
With the conditional distribution in Lemma 3.4, we are able to give a concentration property of theterm B and show it is safe to ignore:
Lemma 3.5.
With a probability larger than 1 − O (exp( − N 12 )) , B = O (cid:0) N − 112 (cid:1) .
For term A , we use martingale central limit theorem with convergence rate [18] to bound the
Kolmogorov-Smirnov statistic.
Lemma 3.6.
When R = lN 23 , λ 1 defined as (7) , we have:
[Equation]
By (16), the expectation w.r.t.
A asymptotically equals to the expectation on N ( µ, σ 2 ) .
The finalstep to prove Theorem 3.1 is to exploit a property of the normal distribution.
Lemma 3.7.
If Z ∼ N ( µ, σ 2 ) , then we have:
[Equation] [Equation]
where Φ is the c.d.f.
of the standard normal distribution.
By Lemma 3.6, 3.7, we have the expectation of (10), which is the expected accept rate, equals to:
[Equation]
3.5 Optimal Scaling for Random Walk Metropolis
We denote the Random Walk Metropolis in discrete space as RWM- R , where R refers to flipping
R indices in each M-H step.
Under the Bernoulli distribution, a site is more likely to stay at highprobability position, so if we randomly flip a site, it is more likely to decrease its probability.
That is,intuitively, the acceptance rate will decrease exponentially as the scale R increases.
Consequently,the optimal scale for RWM- R should be O (1) .
Though this is not a rigorous proof, the constantscaling indicates that it will be hard to directly prove an asymptotic theorem for RWM- R .
To addressthis difficulty, we first restrict our target distribution to a smaller class of Bernoulli distributions
P ( β ) ϵ ⊂ P ϵ , which is formally defined as follows.
For a fixed ϵ ∈ (0 , 14 ) and a fixed β > 0 , define
[Equation]
When N is large, each p j will be very close to 12 .
In this way, the acceptance rate will not drop toofast when R is increased, and a non-constant R will be permitted.
This enables us to prove:
Theorem 3.8.
For arbitrary sequence of target distributions { π ( N ) } ∞ N =1 ⊂ P ( β ) ϵ , the M-H sampler
RWM-R obtains the following, if R = lN 2 β ,
[Equation]
where Φ is the c.d.f.
of the standard normal distribution and λ 2 only depends on π ( N ) .
[Equation]
Corollary 3.9.
The optimal scale R = lN 2 β is obtained when the expected acceptance rate is 0.234,independent of the target distribution.
The rate in Corollary 3.9 is proved for arbitrary β > 0 .
If we let β decrease to 0 , at β = 0 the optimalscale for RWM- R is O (1) while the optimal acceptance rate is 0 .
Also, we can notice that P ( β ) ϵconverges to P ϵ when β decrease to
and we are able to show the optimal scale of RWM in P ϵ is
O (1) , see details in Appendix B.3.
However, this limit is not mathematically rigorous, because
Theorem 3.8 and Corollary 3.9 only hold asymptotically, such that a smaller β requires a larger N .
Hence, when β decreases to 0 , N must approach infinity to satisfy the asymptotic theorem.
Althoughthere is this minor gap in the analysis, the conclusion nevertheless aligns very well with differenttarget distributions in the experiment section.
Adaptive Algorithm
Given knowledge of the optimal acceptance rate, one can design an adaptive algorithm that automatically tunes the scale of the M-H samplers.
For this purpose, we use stochastic optimization [19, 20]to adjust the scaling parameter R t to ensure that the statistic A t = a t − δ approaches 0 , where a t isthe acceptance probability for iteration t and δ is the target acceptance rate ( 0 . 574 for LBP and 0 . 234for RWM).
According to Theorem 3.1 and Theorem 3.8, the acceptance rate is a decreasing functionof the scaling R t .
Hence, we use the update rule:
[Equation]
(22) with step size η t = 1 .
We follow common practice and adapt the tunable MCMC parameters during awarmup phase before freezing them thereafter Gelman et al.
The computational cost for (22) isignorable comparing the total cost of a M-H step.
The algorithm boxes for ALBP and ARWM aregiven in Appendix C. More advanced implementations are possible, but it is out of the focus in thepaper.
We observe below that this simple approach is able to maintain the sampler robustly near theoptimal acceptance rate.
Related Work
Informed proposals for Metropolis-Hastings (M-H) algorithms have been extensively studied forcontinuous spaces [1].
The most famous algorithms are the Metropolis-adjusted Langevin algorithm(MALA) [4] and Hamiltonian Monte Carlo (HMC) [5].
MALA, HMC, and their variants [6, 7, 22,23, 24, 25, 26, 27] use the gradient of the target distribution to guide the proposal distribution towardhigh probability regions, which brings substantial improvements in sampling efficiency compared touninformed methods, such as random walk Metropolis (RWM) [2].
Informed proposals have also demonstrated recent success in discrete spaces.
[9] first gives a formaldefinition of the pointwise informed proposal (PIP) for discrete spaces, then proves that locallybalanced proposals (LBP), using a family of locally balanced functions as the weight function in PIP,are asymptotically optimal for PIP.
Following this work, [28] extended the framework to Markov jumpprocesses and introduced non-reversible heuristics to accelerate sampling.
Sansone [29] parameterizethe locally balanced function and tune it by minimizing a mutual information.
Grathwohl et al.[10] give a more scalable version of LBP for differentiable target distributions by estimating theprobability change through the gradient.
Despite strong empirical results, the LBP method of [9] onlyflips one bit per M-H step, since PIP has to restrict the proposal distribution to a small neighborhood,e.g. a 1-Hamming ball, due to its computational cost.
[11] generalize LBP to flip multiple bits in asingle M-H step, gaining significant improvement in sampling efficiency.
However, the scaling of theproposal distribution in [11] was manually tuned and the optimal scaling problem was left open.
For continuous spaces, the optimal scaling problem for informed proposals has been well studied.
A significant literature has already shown that the scale can be tuned with respect to the optimalacceptance rate [8], e.g. 0.234 for RWM [12], 0.574 for MALA [13], 0.651 for HMC [14], and 0.574 for Barker [17], by decreasing the scale so that the Markov chain converges to a diffusion process.
However, such a technique is not directly applicable to LBP given its discrete nature.
[30] make aninitial attempt on discrete space, however it assumes all dimensions satisfy independent, identical
Bernoulli distribution.
In this work, we have established for the first time the optimal scale for LBPand RWM in discrete spaces.
Experiments
The effectiveness of LBP has been extensively demonstrated in previous work, e.g. [9, 10, 11], incomparison to other M-H samplers for discrete spaces, such as RWM, Gibbs sampling, the Hamming
Ball sampler [31], and continuous relaxation based methods [32, 33, 34, 35].
Therefore, we focuson simulating LBP- R , with weight function g ( t ) =
t t +1 , and RWM- R to validate our theoreticalfindings.
More experiments, including different weight functions and comparison between "with"and "without" replacement versions of LBP are given in Appendix D.
Throughout the experiment section, we will use the gradient approximation [10].
That is to say, we estimate the change in probability of flipping index i is estimated by: ˜ dx i =
exp((
− 2 x i )( ∇ log π ( x )) i )
For the Bernoulli distribution, this is still exact and does not hinder the justification of the theoreticalresults.
For more complex models, this approximation makes the algorithms significantly moreefficient.
In particular, the gradient approximation only requires two calls of the probability functionand two calls of the gradient function.
Consequently, LBP with gradient approximation will takeabout twice time per update compared to RWM.
In our experiments, we observe that LBP and
GWG takes 1 . 2 ± 0 .
2 and 1 .
1 ± 0 .
1 more time per update, respectively, than RWM, across all targetdistributions.
We therefore omit reporting the detailed run time for each method.
6.1 Sampling from different target distributions
We consider four target distributions: the Bernoulli distribution, the Ising model, the factorial hidden
Markov model (FHMM), and the restricted Boltzmann machine (RBM).
For each model, we considerthree configurations: C1, C2, and C3 for smooth, moderate, and sharp target distributions.
To obtainperformance curves, we first simulate LBP- 1 and RWM- 1 for an initial acceptance rate a max .
Then,we adopt a max − 0 .
02 , ..., a max − 0 .
02 k , ... as a target acceptance rate.
For each rate, we use theadaptive sampler to obtain an estimated scale R , with which we simulate 100 chains and calculate thefinal real acceptance rate and efficiency.
In this way, we collect abundant data points to characterizethe relationship between acceptance rate and efficiency to facilitate the following analyses.
[Figure] [Figure]
Bernoulli Distribution .
We validate our theoretical results on Bernoulli distribution.
The probabilitymass function is given in (1).
For each configuration, we simulate on domains with three dimensionalities: N = 100 , 800 , 6400 .
The scatter plot for N = 800 is reported in Figure 1.
We alsoestimate λ in (7) and (21) and plot the theoretical efficiency curve in (5) and (20).
From Figure 1, wecan see that the simulation results align well with the theoretically predicted curves, and the optimalefficiencies were achieved at 0 .
574 for LBP and 0 .
234 for RWM for all configurations.
Ising Model .
The Ising model is a classical model in physics defined on a p × p square lattice graph( V p , E p ) (details in Appendix D.2).
For each configuration, we simulate on three sizes p = 20 , 50 ,100 .
We report the results for p = 50 in Figure 2.
For LBP, the optimal efficiencies are achieved ataround 0 .
5 , which is slightly less than 0 .
574 , although these values are close.
Thus we can say that the asymptotically optimal acceptance rate for LBP still applies to the Ising model.
For RWM, 0 .
234perfectly matches the acceptance rate where the optimal efficiencies are obtained.
Factorial Hidden Markov Model
The FHMM uses latent variables x ∈ X = { 0 , 1 } L × K tocharacterize time series data y ∈ R L (details in Appendix D.3).
Given y , we sample the hiddenvariables x from the posterior π ( x ) = p ( x | y ) .
For each configuration, we simulate in three sizes
L = 200 , 1000 , 4000 .
We report the results for L = 1000 in Figure 3.
One can observe that theseresults match the theoretical predictions very well.
[Figure] [Figure]
Restricted Boltzmann Machine .
A RBM is a bipartite latent-variable model that defines a distribution over binary data x ∈ { 0 , 1 } N and latent data z ∈ { 0 , 1 } h (details in Appendix D.4).
We train an
RBM on the MNIST dataset using contrastive divergence [15] and sample observable variables x .
We report the results in Figure 4.
For LBP, although RBM is much more complex than a productdistribution, its efficiency versus acceptance rate curves still match the theoretical predictions verywell.
For RWM, even using R = 1 will result in acceptance rates less than 0 .
234 for all configurations.
Although we cannot check what the optimal value is, we still observe that efficiency is an increasingfunction of the acceptance rate when the acceptance rate is less than 0 .
234 , as predicted by the theory.
Optimal Scaling and Efficiency .
We examine how optimal scaling R for LBP,
RWM and their relative efficiency ratio grow w.r.t.
the model dimension N .
In figure 5, we can see that both the optimal scaling and efficiency ratio are linear in loglog plot and the slopes are close to 23 across
Bernoulli, Ising, and FHMM.
The results matches the theories that the optimal scaling R
=
O
( N 23 ) for LBP, R = O (1) for
RWM, and the relative efficiency ratio LBP over RWM is O ( N 23 ) .
[Figure]
Size
[Figure]
Sampler
EJD
ESS
Time
EJD
ESS
Time
EJD
ESS
Time
EJD
ESS
Time
RWM0.
59.54 ARWM 1.
61.24 GRWM 1.
59.54 LBP1.
116.04 ALBP 78.
144.03 GLBP 78.
6.2 Adaptive Sampling
We have validated the theoretical findings regarding the optimal acceptance rates on various distributions.
In this section, we examine the performance of the adaptive sampler.
In addition to theexpected jump distance (EJD), we also report the effective sample size (ESS) 1 .
We compare the
[Figure]
adaptive sampler ALBP, ARWM with their single step version LBP-1, RWM-1, and grid searchversion GLBP, GRWM, where we tune the scaling R by grid search.
We give the sampling resultson Bernoulli model, Ising model, FHMM, and RBM with medium size and configuration C2 intable 1.
More results are given in Appendix D. We can see that the adaptive samplers are significantlymore efficient than single step samplers, especially for LBP.
Also, the adaptive samplers can robustlyachieve almost the same performance comparing to using grid search to find the optimal scaling.
6.3 Training Deep Energy Based Models
Learning an EBM is a challenge task.
Given data sampled from a true distribution π , we maximizethe likelihood of the target distribution π θ ( x ) ∝ e − f θ ( x ) parameterized by θ .
The gradient estimationrequires samples from the current model, which is typically obtained via MCMC.
The speed oftraining an EBM is determined by how fast a MCMC algorithm can obtain a good estimate of thesecond expectation.
We evaluate adaptive samplers by learning deep EBMs.
Following the setting in Grathwohl et al.[10], we train deep EBMs parameterized by Residual Networks [36] on small binary image datasetsusing PCD [16] with a replay buffer [37].
We compare two single step samplers and two adaptivesamplers, where LBP b uses g ( t ) =
t t +1 as weight function and LBP s uses g ( t ) = √ t as weightfunction.
When we allow them to run enough iterations in PCD, they are able to train EBMs in samegood quality.
To measure the efficiency of these samplers, we compare the minimum number of M-Hsteps needed in PCD in table 2.
We can see that adaptive samplers only need one half or even onefifth iterations compare to single step samplers.
We also present long-run samples from our trainedmodels via ALBP s in Figure 6.
Dataset
[Table]
Discussion
In this paper, we have addressed the optimal scaling problem for the locally balanced proposal (LBP)in [11].
We verified, both theoretically and empirically, that the asymptotically optimal acceptancerate for LBP is 0 . 574 , independent of the target distribution.
Moreover, knowledge of the optimalacceptance rate allows one to adaptively tune the neighborhood size for a proposal distribution in adiscrete space.
We verified the theoretical findings on a diverse set of distributions, and demonstratedthat adaptive LBP can improve sampling efficiency for learning deep EBMs.
We believe there is considerable room for future work that builds on these results.
For theoreticalinvestigation, the theory established under a strong assumption that the target distribution is a productdistribution, despite the results applies very well to more complicated distributions.
We believe theresults still hold under a weaker assumption that the target distribution has no phase transition.
Wealso believe it is possible to design a HMC style sampler for discrete spaces in the framework of[11] by using LBP as a block for the auxiliary path.
For empirical investigation, many real-worldproblems involve probability models of discrete structured data, such as syntax trees for naturallanguage processing [38], program synthesis [39], and graphical models for molecules [40].
Efficientdiscrete samplers should be able to accelerate both learning and inference with such models.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See Section 7(c)
Did you discuss any potential negative societal impacts of your work?
[No] The workis pure theory, don’t have negative impacts.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See supplemen-
tal material
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
The work does not
use existing assets.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Codebase is included in supplemental materials.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
you describe any potential participant risks, with links to Institutional Review
[Table]
Board
approvals,
applicable
[Table]
Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]
A Complete Proof
A.1 A concentration of W ( x, u )
Lemma A.1.
Define W = E x,u [ W ( x, u )] .
We have:
[Equation]
23) where C 2 is an absolute constant that only depends on the scalar ϵ in (2) .
Proof.
Define a martingale M n , n = 0 , 1 , ..., N + R .
We let M
=
When n ≤ N , it hasindependent increment
[Equation]
For n > N , it is defined as
[Equation]
26) where i / ∈ u 1: r − 1 means i ̸
=
u j for j = 1 , ..., r − 1 .
Since p i are controlled by ϵ in (2), we can find auniform bound
[Equation]
For 1 ≤ n ≤ N , we have
[Equation]
For 1 ≤ r ≤ R , we have
[Equation]
Hence, we can apply the Azuma-Hoeffding inequality:
[Equation]
Thus we prove the lemma.
The lemma indicates with high probability, for arbitrary δ >
[Equation] [Equation]
One observation of the proof is that, the concentration holds for arbitrary 0 ≤ R ≤ N .
For example,when R = N , W ( x, u ) ≡ W ≡ 0 , the concentration is still valid.
A.2 Lemma 3.
Proof.
Using Taylor’s series, we have
[Equation] [Equation]
Using Lemma A.1 and the property W ( x, u )
=
W ( y, u ) , with high probability, the first order termbecomes to:
[Equation]
Similarly, with high probability
[Equation]
the second order term becomes to:
[Equation]
Since R = lN 23 , denote i ∧ j = min { i, j } , i ∨ j = max { i, j } , with high probability, we have
[Equation]
Select 0 < δ < 112 , and the corresponding t = N δ , we have, for large enough N , the above equationdoes not hold with probability exponentially small, and the term o ( N 112 − δ ) can be ignored.
Hencewe prove the weak convergence.
A.3 Proof for Lemma 3.
Proof.
The distribution p ( u r | u 1: r − 1 ) can be approximated using the following tricks.
First, usinglemma A.1, with high probability, we have:
[Equation]
Derive the similar result for P ( u r = j | u 1: r − 1 ) .
Since we have R = lN 32 , for arbitrary 1 ≤ r ≤ R ,we have W has the same order as N .
Using the property of locally balanced function, wherep i w i (1) =
(1 − p i )
w i (0) , we have
[Equation]
Then, we use the identity:
[Equation]
hence, we have for the first step u 1 :
[Equation]
Recursively use this trick, for 1 ≤ r ≤ R = lN 23 we have:
[Equation]
Next, we calculate the conditional probability for x .
To simplify the notation, we denote P ( x j =1 | u, u r = j, x u 1: j − 1 ) to represented index j is selected at step u r , and not been selected in all previoussteps u 1 , ..., u r − 1 .
Also, we denote
[Equation] [Equation]
In this way, the conditional probability for x can be written as
[Equation] [Equation]
Since R = lN 23 , according to lemma A
[Equation] [Equation]
with high probability we have:
[Equation]
Using this approximation
[Equation]
we have:
[Equation] [Equation]
Thus we prove the lemma
[Equation]
A.4 A Property for the conditional distribution of u
The following result shows that marginal distribution for u 1 is a good approximation of the conditionaldistribution.
Proposition A.2.
For N large enough, the conditional distribution for u r = j given u 1: r − 1 can beapproximated by the marginal distribution of u
[Equation]
A.5 Proof for Lemma 3.5
Proof.
We first calculate its expectation using the conditional distribution derived in lemma 3.4.
Tosimplify the notation, we denote δ w ( i ) =
w u i (0) − w u i (1) for i
= 1 , ..., R and
[Equation]
i
[Equation] [Equation]
66) for i, j = 1 , ..., R , and k, l = 0 , 1 .
Then we have
[Equation]
The remaining expectation is with respect to u .
From proposition A.2, we know that the conditionalexpectation of u i can be estimated via the marginal distribution of u 1 .
In fact, when R = lN 23 , wehave:
[Equation] [Equation]
and similarly, we have:
[Equation]
Using these properties
[Equation]
we have
[Equation]
Hence
[Equation]
we prove that
[Equation]
The expectation of the B (12) is small.
To show it is save to ignore, we will prove the concentrationproperty.
Consider a function of x and u :
[Equation] [Equation]
where y is obtained by flipping indices u of x .
For changing x , we have:
[Equation]
where c j = 0 if j / ∈ u or c j = O ( R 2 N 2 ) if there exists r and u r = j .
For chaning u , we have
[Equation]
| F ( x 1 , ..., x N , u 1 , ..., u i , ..
[Equation]
83) where d i =
O ( R 2 N 2 ) for i = 1 , ..., R .
By McDiarmid’s inequality, we have:
[Equation] [Equation]
Hence, F ( x, u ) will concentrate to its expectation at scale O
( R 52 /N 74 ) .
Since R = lN 23 , withprobability larger than 1 − O (exp( − N 12 )) , B = O ( N − 112 ) .
A.6 Lemma
Proof.
To show that A weakly converges to a normal distribution, we use martingale central limittheorem.
Define a martingale M n , for n = 0 , 1 , ..., 2 R .
When n ≤ R , we let the process M n = 0and the filter F n as the σ -algebra determined by u 1 , ..., u n .
For R + 1 ≤ R + n ≤ 2 R , define
[Equation]
We first estimate the mean of the increment using the conditional probability derived in lemma 3.4.
Ifn ≤ R , the mean is obviously 0 , else
[Equation]
Then we estimate the variance of M n − M n − 1 .
We start with estimating the 2nd moment
[Equation] [Equation]
Then
[Equation]
we are able to calculate the variance:
[Equation]
We calculate the value of its mean µ and variance σ 2 .
[Equation]
[Equation]
Define µ 1 = E
[ w u 1 (1) + w u 1 (0)] .
For the first part in µ , using proposition A.2, we have
[Equation] [Equation]
[Equation]
Define σ 21 =
E
[( w u 1 (0) − w u 1 (1)) 2 ] , From lemma 3.4, we have
[Equation]
for the second term in µ , we have
[Equation]
for the variance σ 2 , we have:
[Equation]
Finally, we will decouple R with W .
Specifically:
[Equation]
Combine everything together, we have
[Equation]
Since R = lN 23 , we have the sum of the conditional variance is O (1) and the reminder is o (1) .
For a martingale, we need to check one more step.
We know | M n − M n − 1 | = 0 for n ≤ R .
Forn + R > R , we have:
[Equation] [Equation]
is uniformly bounded by a constant independent of N and R .
We denote
[Equation]
Then we can rewrite:
[Equation] [Equation]
[Equation]
By martingale central limit theorem, we have that:
[Equation] [Equation]
Furthermore, we use the convergence rate in [18], we have:
[Equation]
[Equation] [Equation]
Then we have the probability
[Equation] [Equation]
where
[Equation] [Equation]
where C δ is an absolute constant that only depends on δ .
We select δ = 12 , we have:
[Equation]
Since we consider R = lN 23 , we prove the lemma.
A.7 Proof of Lemma 3.7
Proof.
Assume Z ∼ N ( µ, σ 2 ) , then we have:
[Equation]
Specially, when µ = − 12 σ 2 , we have:
[Equation]
A.8 Proof for Theorem 3.
Proof.
In RWM-R, the proposal distribution is uniform, hence we only need to consider the probability ratio in the acceptance rate.
Given current state x and the picked indices u , the proposed state y isobtained by flipping indices u of x .
The acceptance rate is:
[Equation] [Equation]
Define the martingale M n , n = 1 , ..., 2 R .
For r = 1 , ..., R , we have M r = 0 and the filtration F r isdetermined by the σ -algebra of u 1 , ..., u R .
For R + 1 ≤ R + n ≤ 2 R , we have:
[Equation]
Hence, for n ≤ R , the increment is 0 .
For n + R > R , denote the mean of the increment is :
[Equation]
the variance of the increment is:
[Equation]
When N is large, we have p u n −
=
O ( N − β ) , hence
[Equation] [Equation]
is negative twice of the corresponding mean.
Since the indices u are uniformly picked, the conditionaldistribution of u r is:
[Equation]
Hence, we have the mean is
[Equation]
Similarly
[Equation]
we have the variance is:
[Equation]
When R =
O
( N 2 β ), the variance is at a constant order.
For a martingale, we also need to checkthe increments are uniformly bounded.
When n ≤ R , the increment is always 0 .
When R + 1 ≤
R + n ≤ 2 R , we have:
[Equation]
where C ( ϵ ) is a constant only determined by ϵ .
Hence, by martingale central limit theorem, we havethe distribution of M 2 R converges to a normal distribution.
Denote
[Equation] [Equation]
Then we can rewrite:
[Equation] [Equation]
Denote Z = (cid:80)
Rr =1 (1 − 2 x u r ) log p ur 1 − p ur .
By martingale central limit theorem, we have
[Equation] [Equation]
Furthermore, using the convergence rate in [18], we have:
[Equation] [Equation]
[Equation]
Then we have the probability
[Equation]
156) where
[Equation]
where C δ is an absolute constant that only depends on δ .
We select δ = 12 , we have:
[Equation] [Equation]
Hence, the expectation w.r.t.
[Equation]
Rr =1 (1 − 2 x u r ) log p ur 1 − p ur converges to the expectation w.r.t.
[Equation]
Using lemma 3.7, we have the acceptance rate converges to:
[Equation]
In RWM-R, the distance between the current state x and the proposed state y is always d ( x, y )
=
R ,hence we have:
[Equation] [Equation]
When R = ω ( N 2 β ) , we can give a concentration property.
Since the selection of u r is a martingale,we can apply Azuma-Hoeffding inequality:
[Equation]
Hence, When N is sufficiently large, with probability larger than 1 − O (exp( − 2 t 2 R 12 /N β )) , wehave:
[Equation]
For C > 0 independent with N, R .
A.9 Proof for Corollary 3.
Proof.
When R =
O
( N 2 β ) , denote z =
Rλ 22 /N 2 β
[Equation] [Equation]
[Equation] [Equation]
which means the optimal value of z is independent of the target distribution.
As Φ is known, wecan numerically solve z = 5 . 673 .
Hence the corresponding expected acceptance rate a = 0 .
234 ,independent with the target distribution, and the efficiency is Θ( N 2 β ) .
When R = ω ( N 2 β ) , withprobability 1 − O (exp( − 2 R/N β )) , the acceptance rate decrease exponentially fast, rendering o (1)jump distance.
For the remaining probability O (exp( − 2 R/N β )) , assuming all proposals are accepted,the efficiency is still bounded by:
[Equation] [Equation]
Hence, optimal efficiency is achieved when R =
O ( N 2 β ) .
B DiscussionB.1 Expected Jump Distance as the Metric to Tune the Scale
In this section, we want to convince the reader that the expected jump distance (EJD) is the correctmetric to evaluate the efficiency for samplers in discrete space.
To simplify the derivation, we considerthe distribution
[Equation]
We can notice that, compared to the target distributions considered in the main text (1), we assumethe target distribution is identical in each dimension.
Let the LBP chain, with R = lN 23 , being denoted as { x (1) , x (2) , ... } .
Since all dimensions areidentical, we only need to focus on the first dimension.
Denote w 1 = g
( π 1 ( x 1 =0)
π 1 ( x 1 =1) ) and w 0 =g
( π 1 ( x 1 =1) π 1 ( x 1 =0) ) .
From Lemma.
A.1, we can see that:
[Equation]
That’s to say, the probability ratio of x 1 = 0 and x 1 = 1 being flipped equals to their weight ratio.
Then we compare the acceptance rate in M-H test.
From the proof of the main theorem 3.1, we knowthe acceptance rate is determined by the term A defined in (11)
[Equation]
[Equation]
We can see that, when the first dimension is flipped in proposal, the difference of A is O ( N − 13 ) forx 1 = 0 and x 1 = 1 .
As a result, we have:
[Equation]
Now, we consider the one-dimensional process Z Nt = x 1 ( ⌊ tN 13 ⌋ ) .
The identical assumption impliesthat, the frequency for a site, for example the first dimension, being selected is lN − 13 .
We can easilysee that when N is large enough, Z Nt converges to a jump process Z t , whose generator we denote.
[Equation]
From the derivation above, we know that
[Equation]
Since the sketch of proof above shows that the ratio is independent with the parameter l , we have thefollowing important decomposition
[Equation]
[Equation]
where Q ( p ) is a matrix only depends on p and the locally balanced function g selected, and λ ( l ) is ascalar only depends on the parameter l .
Since Q ( p ) only depends on the target distribution, for any test functions f ( · ) , the inverse autocorrelation of the jump process is proportional to λ ( l ) .
When we tune l , the coefficient λ ( l )
=l · 2Φ( − λ 1 l 32 ) is the multiplication of the proposal frequency and the acceptance rate.
The valueλ 1 is defined in (7).
As a jump process, we don’t have to analytically compute the value of λ ( l ) , asλ ( l ) is proportional to the expected jump distance (EJD).
So, we can tune l by maximizing the EJD,without having to know the formulation of the target distribution.
Remark 1: The jump process in discrete space is different from the diffusion process in continuousspace.
For diffusion process, its velocity is characterized by the ESJD.
But for jump process, itsvelocity is characterized by the EJD.
That’s why Langevin algorithms tunes the step size via ESJD[13], but our LBP tunes the path length via EJD. ϵ 0 .0 .0 .0 . N    ϵ 0 .0 .0 .0 .
N Remark 2: To simplify the derivation, we assume that the target distributions have identical marginals.
For target distributions with non-identical marginal distribution, different dimensions i = 1 , ..., Ncan have different velocity
λ i ( l ) .
But the sampling process will still converge to jump process, andwe shall still use EJD to measure the efficiency.
B.2 The Choice of ϵ and the Optimal Acceptance Rate
The convergence of (6) does not depend on the value of ϵ in (2).
Based on the proof above, wecan know (6) converges at the rate O ( N − 112 ) .
But the convergence of the optimal acceptance ratedepends on the ϵ .
We can first consider two extreme cases for intuition.
When all p i are close to 12 ,λ 1 in (7) will be close to 0 and the optimal acceptance rate will be close to 1 ; when all p i are close to0 or 1 , λ 1 in (7) will be close to ∞ and the optimal acceptance rate will be close to 0 .
Hence, themain purpose to use fixed ϵ is to give upper and lower bounds for λ 1 in (7), such that the optimalacceptance rate can converge to 0 .
574 as in Corollary 3.2.
Next, we discuss how does the model dimension N in (1) needed in terms of ϵ to make sure theoptimal convergence to 0 .
When all p i have the extreme value determined by ϵ , using locallybalanced function g ( t ) = √ t , we can consider the following two situations:
• All | p i − 0 .
5 | = ϵ → 0 .
Then we have:
[Equation]
ϵ
[Equation]
When the expected acceptance is 0 . 574 , we need λ 1 l
=
O ( ϵl 32 ) equals to a constant, whichmeans l has the same order as ϵ − 23 .
Since we have R = lN 23 ≤ N , we need ϵ − 23 =
O ( N 13 ) .
As a result, we requires ϵ − 1 =
O ( N 12 ) , which basically means we need N ≥ ϵ − 2 to havethe optimal acceptance rate converges to 0 .
• All 0 .
5 − | p i − 0 .
5 | = ϵ → 0 .
We have:
[Equation]
When the expected acceptance is 0 . 574 , we need λ 1 l
=
O
( ϵ − 1 l 32 ) equals to a constant,which means l has the same order as ϵ 23 .
Since we have R = lN 23 ≥ 1 , we have l − 1 =
O ( N 23 ) .
As a result, we requires ϵ − 1 =
O ( N − 1 ) , which basically means we need N ≥ ϵ − 1to have the optimal acceptance rate converges to 0 .
So, both situations show that we need N ≥ C
ϵ − 2 , for some constant C ′ , to make sure the optimalacceptance rate converges.
In the main text, we assume ϵ is a constant and it guarantees Corollary 3.2holds.
We conduct extra numerical simulations to verify our results.
To simplify the experiments, we assumeall dimensions have the same configurations: p i = p .
We report the size of N needed to guaranteethat the optimal acceptance rate is 0 .
574 in Table 3 and Table 4.
B.3 Optimal Scale of RWM
When we assume the target distribution belongs to (2), the derivation of the optimal acceptancerate 0 . 234 is no longer valid.
But we can still show the optimal scale is R =
O (1) by proving theacceptance rate decreasing exponentially fast.
In particular, assume we use R = lN β in RWM.
Then the acceptance rate can be written as:
[Equation]
Consider the martingale M j , j = 0 , 1 , ..., R , such that M 0 = 0 and
[Equation]
By assumption in (2), we know that
[Equation]
And we have
[Equation]
By Azuma-Hoeffding lemma, we have
[Equation]
For β > 0 , R increases to infinity when N → ∞ .
In this case, log A ′ concentrates to a value
T ≤ R · 2 ϵ log 1 − 2 ϵ 1+2 ϵ and A ′ decreases exponentially fast.
Hence, the optimal scaling of RWM is
O (1) .
C Adaptive Algorithm
We give the algorithm box for ALBP:
Algorithm 2: Adaptive Locally Balanced Proposal 1: Initialize current state
2: Initialize scale
3: for = do 4:
Initialize candidate set
R ← probabilistic rounding of R t 6: for r=1, ..., R do 7:
Sample u r
P ( u r = j ) ∝ w j
x ( t ) )
1 { j ∈C} .
C ← C\{ u r } .
9: end for 10:
Obtain y by flipping indices u 1 , ..., u R of x ( t ) .
Compute acceptance rate A =
A ( x ( t ) , y, u ) .
12: if rand(0,1) <
A then 13: x (
=
y 14: else 15: x (
= x
) 16: end if 17: if t < T warmup then 18:
R t +1 ←
19: end if 20: end for
We give the algorithm box for ARWM: do
←
Uniformly sample
Obtain by indices
Compute acceptance rate A =
A ( x ( t ) , y, u ) .
9: if rand(0,1) <
A then 10: x ( t +1)
=
y 11: else 12: x
= x t ) 13: end if 14: if t < T warmup then 15:
R t +1 ← R t
+ ( A − 0 . 234) .
16: end if 17: end for
D Experiment Details
We consider five samplers:
• RWM: random walk Metropolis
• GWG( √ t ): LBP with replacement, same as algorithm 1 except for skipping line 5, weight
function g ( t ) =
√ t
• LBP( √ t ): LBP given in algorithm 1, weight function g ( t ) = √ t• GWG( t t +1 ):
LBP with replacement, same as algorithm 1 except for skipping line 5, weightfunction g ( t ) =
t t +
• LBP( t t +1 ): LBP given in algorithm 1, weight function g ( t ) =
t t +
For each sampler, we first start simulating with R = 1 to get an initial acceptance rate a max .
Then we adopt a max − 0 .
02 , a max − 0 .
04 , ..., a max − 0 .
02 k as the target acceptance rate, untila max − 0 .
02 k < 0 .
For each target acceptance rate a target , we use our adaptive sampler to getan estimated scaling R target .
Then we simulate 100 chains with scaling R target to get the expectedacceptance rate, expected jump distance, effective sample size ( a, d, e ) .
To measure the performance of the adaptive sampler, we compare three versions for each samplerabove.
In particular, for sampler X we have
• X-1, represents fixed scaling R = 1 version of the sampler.
• AX, represents the adaptive version of the sampler, whose target acceptance rate is selectedas 0 .
234 for RWM, and 0 .
574 for else.• GX, represents the grid search version of the sampler, where we always use the best results among all simulations for different target acceptance rates we mentioned above
D.1 Simulation on Bernoulli Model
The density function for Bernoulli distribution is:
[Equation]
We consider three configurations
• C1: p i is independently, uniformly sampled from [0 . 25 , 0 .
• C2: p i is independently, uniformly sampled from [0 . 15 , 0 . 85] .
• C3: p i is independently, uniformly sampled from [0 . 05 , 0 . 95] .
For each configuration, we simulate on three sizes:
• N = 100 , sample Markov chain x 1:10000 , use x 1:5000 for burn in, use x 5001:10000 to estimateexpected acceptance rate, expected jump distance, effective sample size.• N = 800 , sample Markov chain x
1:40000 , use x 1:20000 for burn in, use x 20001:40000 toestimate expected acceptance rate, expected jump distance, effective sample size.• N = 6400 , sample Markov chain x
1:100000 , use x 1:50000 for burn in, use x 50001:100000 toestimate expected acceptance rate, expected jump distance, effective sample size.
We give the scatter plot of ( a, d ) and ( a, r ) in figure 7.
And we examine the performance of ouradaptive algorithm in table 5 and table 6.
D.2 Simulation on Ising Model
Ising model is a classic model in physics defined on a p × p square lattice graph ( V p , E p ) .
That’s tosay, the nodes are indexed by { 1 , ..., p } 2 and an edge (( i, j ) , ( k, l )) exists if and only if one of thefollowing condition holds:
[Figure]
x p e c t e d
J u m p
D
i s t a
[Figure]
c
[Figure]
i v
[Figure]
m
[Figure] [Figure]
[Figure]
E x p e c t e d
J u m p
D
i s t a n c e
[Figure]
v
[Figure]
m
[Figure]
l
[Figure]
i z
[Figure]
• i =
k, j = l +
[Equation] [Equation]
The state space is X = {− 1 , 1 } V p and the target distribution is defined as:
[Equation]
Following [9]
[Equation]
we consider three configurations
• C1: α v is independently and uniformly sampled from ( − 0 .
4) if ( v 1 − p 2 ) 2 +( v 2 − p 2 ) {+
+}
≤p 2 2 , else α v is independently and uniformly sampled from ( − 0 . 4 , 0 . 2) ; λ = 0 .
1 .• C2: α v is independently and uniformly sampled from ( − 0 .
3 , 0 . 6) if ( v 1 − p 2 ) 2 +( v 2 − p 2 ) 2 ≤p 2 2 , else α v is independently and uniformly sampled from ( − 0 .
; λ = 0 .
C3: α v is
independently and uniformly sampled from
( − 0 .
8) if ( v 1 − p 2 ) 2 +( v 2 − p 2 ) 2 ≤
p 2 2 , else α v is independently and uniformly sampled from ( − 0 . 8 , 0 . 4) ; λ = 0 .
For each configuration, we simulate on three sizes:
• p = 20 , sample Markov chain x 1:10000 , use x 1:5000 for burn in, use x 5001:10000 to estimate
expected acceptance rate, expected jump distance, effective sample size.
[Table] [Table]
+1 )- 27.
ALBP( t t +1 ) 604.
GLBP( t t +1 ) 658.
GWG( √ t )- 26.
AGWG(
[Table]
GGWG
[Table]
LBP( [Table] t )- 28.
ALBP( [Table] t ) 598.
GLBP( [Table] t ) 636.
• p = 50 , sample Markov chain x
1:40000 , use x 1:20000 for burn in, use x 20001:40000 to estimate
expected acceptance rate, expected jump distance, effective sample size.
• p = 100 , sample Markov chain x 1:100000 ,
use x 1:50000 for burn in, use x 50001:100000 toestimate expected acceptance rate, expected jump distance, effective sample size.
We give the scatter plot of ( a, d ) and ( a, r ) in figure 8.
And we examine the performance of ouradaptive algorithm in table 8 and table 9.
D.3 Simulation on FHMM
FHMM uses latent variables x ∈ X = { 0 , 1 } L × K to characterize time series data y ∈ R L .
Denotep ( x ) as the prior for hidden variables x , and p ( y | x ) for the likelihood:
[Equation]
[Table] [Table]
Specifically, we have p ( x l, 1 ) = 0 .
1 , p ( x l,k = x l,k − 1 | x l,k − 1 )
= 0 .
8 independently ∀ l = 1 , ..., L and∀ k
= 2 , ..., K .
And we have all entries in W and b are independent Gaussian random variables.
Wesample latent variables x and sample y ∼ p ( y | x ) .
Then we simulate our samplers to sample the latentvariables x from the posterior π ( x )
=
p ( x | y ) .
We consider three configurations
• C1: σ 2 = 2• C2: σ 2 = 1• C3:
σ 2 = 0 .
For each configuration, we simulate on three sizes:
• L = 200 , K = 5 , sample Markov chain x 1:10000 , use x 1:5000 for burn in, use x 5001:10000 toestimate expected acceptance rate, expected jump distance, effective sample size.• L = 1000 , K = 5 , sample Markov chain x
1:40000 , use x 1:20000 for burn in, use x 20001:40000to estimate expected acceptance rate, expected jump distance, effective sample size.
[Figure]
x p e c t e d
J u m p
[Figure]
i s
[Figure]
a
[Figure] [Figure]
[Figure] [Figure]
c d
[Figure] [Figure]
i z
[Figure]
x
[Figure]
e c t e d
J u m p
D
i s t a n c e
[Figure]
c t i v e
[Figure]
i z
[Figure]
• L = 4000 , K = 5 , sample Markov chain x
1:100000 , use x 1:50000 for burn in, usex 50001:100000 to estimate expected acceptance rate, expected jump distance, effective samplesize.
We give the scatter plot of ( a, d ) and ( a, r ) in figure 9.
And we examine the performance of ouradaptive algorithm in table 11 and table 12.
D.4 Simulation on RBM
RBM is a bipartite latent-variable model, defining a distribution over binary data x ∈ { 0 ,
} Nand latent data z ∈ { 0 , 1 } h .
Given parameters W ∈ R h × N , b ∈ R N , c ∈ R h , the distribution ofobservable variables x is obtained by marginalizing the latent variables z :π ( x [Equation] ∝ exp( b T x ) h (cid:89) i =
(1 + exp
( W i x + c i ))
We train the RBM on MNIST dataset using contrastive divergence [15] in three configurations
• C1: h =
• C2: h =
• C3: h =
[Table] [Table]
LBP( t t +1 )- 36.
ALBP( t t +1 ) 37.
GLBP( t t +1 ) 37.
GWG( √ t )- 29.
AGWG( √ t ) 30. GGWG( √ t ) 30.
LBP( √ t )- 36.
ALBP( √ t ) 37.
GLBP( √ t ) 36.
For each configuration, we sample Markov chain x
1:40000 , use x 1:20000 for burn in, use x 20001:40000to estimate expected acceptance rate, expected jump distance, effective sample size.
We give the scatter plot of ( a, d ) and ( a, r ) in figure 10.
And we examine the performance of ouradaptive algorithm in table 14 and table 15.
[Figure]
x p e c t e d
J u m p
[Figure]
i s
[Figure]
a
[Figure]
i z
[Figure]
c t e d
[Figure]
a
[Figure]
ff e c t i v e
S a m p l e S
i z e
[Figure]
E x p e c t e d
J u m p
D
i s t a n c e
[Figure]
i v
[Figure]
i
[Figure] [Figure]
e c t i v e
[Figure]
a
[Figure]
p l e S i z e
[Figure]
[Table] [Table]
Size
[Table]
Sampler
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
LBP
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
ALBP
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
t t
[Table]
[Table] [Table]
[Table]
GLBP
[Table]
t t +1 ) 29.
[Table] [Table] [Table]
√ t )-
[Table]
AGWG
[Table]
√ t ) 5.
[Table]
GGWG
[Table]
t ) 5.
LBP( √ t )- 1.
ALBP
[Table]
√ t ) 11.
GLBP
[Table]
√ t )
[Table] [Table]
[Table]
GWG( √ t )- 109.
AGWG( √ t ) 94.
GGWG( √ t ) 96.
LBP( √ t )- 116.
ALBP( √ t ) 120.
GLBP( √ t ) 118.