Fast Doubly-Adaptive MCMC to Estimate the Gibbs Partition Function with Weak Mixing Time Bounds

1 Introduction . The Gibbs ( Boltzmann ) distribution is a family of probability distributions of exponential form . First introduced in the context of statistical mechanics [ 25 ] , Gibbs distributions are now ubiquitous in a variety of other disciplines , such as chemistry [ 24 , 31 ] , economics [ 1 , 54 ] and machine learning . Gibbs distributions are typically used to model the global state of a system as a function of a collection of interdependent random variables , each representing local states in the system . The dependencies ∗ r© indicates randomized ordering and equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . in the system are modeled by a Hamiltonian function , and the probability distribution is inversely proportional to exponent of the Hamiltonian scaled by the temperature ( see eq . ( 1 ) § 1.1 ) . Gibbs distributions provide potent statistical inference tools in many machine learning applications . They appear in probabilistic graphical models [ 41 , 49 , 51 ] , including restricted Boltzmann machines [ 43 , 66 ] , Markov random fields [ 40 , 47 ] , and Bayes networks [ 32 ] , and are applied in the analysis of images and graphical data [ 21 , 23 , 44 , 65 ] , topic modeling ( LDA ) [ 27 , 53 , 57 , 62 ] , and more [ 2 , 13 , 18 , 19 , 26 , 30 , 48 , 56 , 58 , 68 ] . A major obstacle in applying the Gibbs distribution in practice is the need to compute , or estimate , its partition function ( normalizing constant ) , henceforth written GPF . The partition function is defined over the Cartesian product of supports of a ( typically large ) number of variables , making exact computation intractable . Furthermore , due to interdependence of variables , exact sampling is not practically feasible , thus Markov-chain Monte-Carlo ( MCMC ) solutions for this problem have been extensively studied [ 6 , 22 , 29 , 34 , 37 , 39 , 42 , 52 , 63 , 67 ] . Like other MCMC methods , here various heuristics are used . The most well-known heuristics are the annealed importance sampling [ 39 , 52 , 64 ] or convergence diagnostics methods [ 10 , 11 , 20 , 61 ] . Unfortunately , these methods are often error-prone , as their correctness is only proven asymptotically , without rigorous mathematical analysis to bound their estimation error with finite samples . In fact , theoretical findings have shown that with no prior knowledge of relevant measures , such as the variance of importance weights in annealed importance sampling , or upper bounds on mixing or relaxation times for diagnostic methods , these methods are either unreliable or computationally intractable ( see [ 52 , section 4 ] or [ 8 , 33 ] ) . On the other hand , theoreticians study this problem by designing Fully Polynomial Randomized Approximation Schemes ( FPRAS ) ( see problem 1 ) . The state of the art FPRAS for estimating the GPF is a multi-stage algorithm involving a sequence of functions at various temperatures , such that the expectation of the product of these functions , or the product of the expectations of said functions , is the GPF . FPRAS ’ s are proven to produce ( approximate ) solutions w.h.p. , but their performance guarantees rely on available upper-bounds on various measures such as variances of estimators or mixing times of Markov chains . In static algorithms , these upper-bounds are given a priori , and adaptive2 algorithms estimate them dynamically , while increasing the sample size until desired properties are mathematically guaranteed . Thus , adaptive algorithms are less sensitive to looseness of known upper-bounds , more robust , often faster , and more easily applied to various settings . Most of the research on designing FPRAS ’ s for the GPF is focused on designing adaptive algorithms to produce sequences ( cooling schedules ) with minimum length while keeping the variances of estimators small ( thus removing the need to have a-priori known bounds on variances ) . In contrast , the computation of the sequence of mean estimates , which dominates the total computation cost , is done by black-box MCMC estimators , with a priori known upper bounds on the mixing times of the chains . These upper bounds are often loose , and improving them for particular models is a challenging active area of research [ 4 , 5 , 9 , 12 , 28 , 63 ] . In order to complement the adaptive cooling schedule and reduce dependence on a priori bounds on Markov chains ’ mixing times , it seems necessary to design an adaptive procedure with theoretical guarantees for MCMC-mean estimation . In this work we develop a doubly adaptive FPRAS , combining the adaptive cooling schedule with adaptive MCMC mean estimator that dynamically adapts the number of Markov chain steps to the observed underlying chain . Through rigorous theoretical analysis , we prove that our method outperforms the state of the art algorithms in several factors : ( 1 ) The computational complexity of our method is smaller ; ( 2 ) Our method is less sensitive to loose bounds on mixing times , an inherent component in these algorithms ; and ( 3 ) The improvement obtained by our method is particularly significant in the most challenging regime of high precision estimates . We demonstrate the advantage of our method in experiments run on classic factor graphs , such as voting and Ising models [ 5 , 7 , 15 ] . 1.1 Preliminaries and Prior Work . Assume a sample space Ω , Hamiltonian function H : Ω→ { 0 } ∪ [ 1 , ∞ ) , and inverse temperature parameter β ∈ R , referred to as inverse temperature . The Gibbs distribution on Ω , H ( · ) , and β is 2The usage of the word “ adaptive ” here refers to algorithms which draw samples progressively and adapt their sample complexity based on empirical estimates until desired conditions are met , as it has been used in [ 34 , 42 ] ( see § 1.1 ) , and should not confused with the work of [ 60 ] . then characterized by probability law ∀x ∈ Ω : πβ ( x ) . = 1 Z ( β ) exp ( −βH ( x ) ) . ( 1 ) Here Z ( β ) is the normalizing constant or Gibbs partition function ( GPF ) of the distribution , with Z ( β ) . = ∑ x∈Ω exp ( −βH ( x ) ) . ( 2 ) Estimating the GPF Z ( β ) , is computationally challenging , since typically the size of Ω is exponential in the number ofvariables , and the values of random terms in the sum have large variance ( due to the exponential ) . The following problem has been extensively studied , and is the focus of this paper . Problem 1 . Given a domain Ω , a Hamiltonian function H , and a parameter β , design a Fully Polynomial Randomized Approximation Scheme ( FPRAS ) for estimating the partition function Z ( β ) . = ∑ x∈Ω exp ( −βH ( x ) ) . In other words , for user-supplied ε , the task is to produce an estimate Ẑ ( β ) , such that with probability at least 1 − δ , we have ( 1 − ε ) Z ( β ) ≤ Ẑ ( β ) ≤ ( 1 + ε ) Z ( β ) , in time polynomial in 1/ε , ln ( 1/δ ) , and all other problem parameters ( e.g. , the number of vertices in an Ising model , or neurons in an RBM ) . All known scalable solutions to this problem rely on Monte-Carlo Markov-chain ( MCMC ) methods , and their execution cost is dominated by the total number of Markov chain steps they execute . We therefore follow past work , and analyze our algorithms in terms of number of the Markov chain steps . TPA-Based Adaptive Cooling Schedules Building on extensive earlier work [ 6 , 22 , 67 ] , the current state of the art is due to Huber and Schott [ 35 ] , with Kolmogorov ’ s sharper analysis [ 42 ] . They introduce the paired product estimator ( PPE ) , see definition 1.1 , and apply the tootsie-pop algorithm ( TPA ) to adaptively compute a near-optimal cooling schedule , i.e. , a sequence of inverse temperatures β0 < β1 < · · · < β ` −1 < β ` satisfying β ` = β , and that Z ( β0 ) is easy to compute , e.g. , β0 = 0 is often convenient , since Z ( 0 ) = |Ω| . We thus define Q . = Z ( β ) /Z ( β0 ) and estimate it using the paired product estimator . Definition 1.1 ( PPE [ 34 ] ) . Assume a cooling schedule β0 , β1 , . . . , β ` . For each pair ( βi , βi+1 ) in the schedule , we define two random variables , Xi ∼ πβi and Yi ∼ πβi+1 , all independent , and we then define fβi , βi+1 . = exp ( −βi+1−βi2 H ( Xi ) ) and gβi , βi+1 . = exp ( βi+1−βi 2 H ( Yi ) ) . It is easy to verify that E [ fβi , βi+1 ] = Z ( βi+βi+1 2 ) /Z ( βi ) , and E [ gβi , βi+1 ] = Z ( βi+βi+1 2 ) /Z ( βi+1 ) . We then define F .= ∏k i=1 fβi , βi+1 , G . = ∏k i=1 gβi , βi+1 . Letting µ̂ and ν̂ denote empirical estimates of E [ F ] and E [ G ] , respectively , the paired product estimator ( PPE ) is Q̂ .= µ̂/ν̂ . Denote by Vrel [ X ] . = E [ X 2 ] /E [ X ] 2 − 1 = V [ X ] /E [ X ] 2 the relative variance of a random variable X . The TPA schedule [ 35 , 36 ] is generated by an adaptive algorithm , which , by a proper setting of parameters , outputs a cooling schedule guaranteeing constant Vrel [ F ] and Vrel [ G ] ( see alg . 3 in the supplementary material ) . Kolmogorov [ 42 ] presents a tighter analysis of Huber ’ s TPA method , and proves that with slight modifications ( see alg . 4. in the Appendix ) the schedule has a shorter length , while preserving constant relative variance for the paired product estimators ( see thm . 1.1 ) . In this paper , we use Kolmogorov ’ s algorithm , and we denote it by TPA ( k , d ) . For completeness , both of Huber ’ s and Kolmogorov ’ s versions of TPA are presented in the Appendix . We will use the following result in our analysis : Theorem 1.1 ( [ 42 ] ) . Let Hmax . = maxx∈ΩH ( x ) , using TPA ( k , d ) , k = Θ ( logHmax ) and d = 16 to generate cooling schedule ( β0 , β1 , . . . , β ` ) . W.h.p. , we have ` = Θ ( log ( Q ) log ( Hmax ) ) and Vrel [ F ] +1 = ∏ ` i=1 ( Vrel [ fβi , βi+1 ] +1 ) = Θ ( 1 ) and Vrel [ G ] +1 = ∏ ` i=1 ( Vrel [ gβi , βi+1 ] +1 ) = Θ ( 1 ) . Kolmogorov [ 42 ] nearly matches known lower bounds when given oracle access to near-independent samples , but leaves open the possibility of better use of the dependent sequence of samples generated by MCMC chains . This fertile ground is ill-explored , since if an approximate sampling oracle draws samples by running a chain for T steps , there is a factor T potential improvement . MCMC Mean-Estimator Huber and Schott [ 35 ] assume unit-cost for exact sampling from each πβi , and Kolmogorov [ 42 ] extends their analysis to include the complexity of generating approximate samples with standard MCMC processes , assuming a priori upper-bounds on their mixing times . The main contribution of our paper is a specialized , adaptive , multiplicative MCMC-mean estimator for the TPA-based PPE . Our method is significantly more efficient than using standard black-box MCMC sampling for this problem , thus we improve the best-known method for estimating the GPF . LetM be an ergodic Markov chain with state space S and stationary distribution π . Let τmix ( ε ) denote the ε-mixing time ofM , and define τmix . = τmix ( 1/4 ) . Letting λ denote the second largest absolute eigenvalue ofM ’ s transition matrix , the relaxation time ofM is τrx . = ( 1− λ ) −1 , and it is related to the mixing time τmix , by ( τrx ( M ) − 1 ) ln ( 2 ) ≤ τmix ( M ) ≤ ⌈ τrx ( M ) ln ( 2/√πmin ) ⌉ [ 45 ] . Let T be an upper bound on max { τrx ( M ) , τmix ( M ) } . Consider any i.i.d . sampling concentration bound like Chebyshev ’ s , Hoeffding ’ s , or Bernstein ’ s inequalities [ 50 ] , with , say , sample complexity mε . Using MCMC as a black-box sampling tool , we obtain the same precision estimation guarantees , with a computational cost of mε · τmix ( ε/mε ) , which is equal to mε log ( mε · ε−1 ) · T in the absence of exact values for τmix . Other concentration bounds compute the average over the entire trace of a Markov chain , and their complexity is dependent on known upper-bounds on the relaxation time [ 14 , 38 , 46 , 50 , 55 ] , or function specific mixing time [ 59 ] . Note that since log ( 12ε ) ( τrx − 1 ) ≤ τmix ( ε ) ≤ log ( 1 επmin ) τrx , using these bounds is often more efficient , saving at least log ( mε ) steps . Recently , Cousins et al . [ 16 ] introduce a novel Markov chain statistical measure , the inter-trace variance . The inter-trace variance depends on both the function being estimated and the dependency structure between nearby samples in the chain , and unlike the mixing time , it can be efficiently estimated from data . By using progressive sampling , Cousins et al . show an additive MCMC mean estimator whose complexity is proved in terms of inter-trace variance and they show it it less sensitive to prior knowledge of the input parameters , such as relaxation time and trace variance . Unfortunately due to a few technical problems , their result can not directly be used with the TPA method . Thus , in order to obtain a doubly adaptive algorithm for problem 1 , we tailor their techniques to our setting , which requires developing new algorithms and analysis tools .

Summary: The authors propose a doubly "adaptive" MCMC algorithm to approximate the partition function in gibbs distributions. In related work, the partition function of Gibbs distributions are computed using paired product estimators [1]. Such estimators use a cooling schedule $[\beta_{(\cdot)}]$ to estimate two product functions $F$ and $G$ using stationary samples from the Gibbs distribution at each temperature. The ratio of these two products is then used to compute the partition function.  In the current work, the first "adaptive" refers to the adaptive TPA cooling schedule to decide $[\beta_{(\cdot)}]$. The second refers to the automatic number of steps used in the MCMC MeanEstimator which is used to estimate the two products mentioned above $F$ and $G$.  [1] - Huber, Mark. "Approximation algorithms for the normalizing constant of Gibbs distributions." The Annals of Applied Probability 25.2 (2015): 974-985.