Parallel Tempering With a Variational Reference
Authors: Nikola Surjanovic
Saifuddin Syed
Alexandre Bouchard-Cote
Trevor Campbell
Venue: NeurIPS 2022 Accept (2022)
Keywords: Bayesian inference
parallel tempering
variational inference
Markov chain Monte Carlo

Abstract: Sampling from complex target distributions is a challenging task fundamental to Bayesian inference. Parallel tempering (PT) addresses this problem by constructing a Markov chain on the expanded state space of a sequence of distributions interpolating between the posterior distribution and a fixed reference distribution, which is typically chosen to be the prior. However, in the typical case where the prior and posterior are nearly mutually singular, PT methods are computationally prohibitive. In this work we address this challenge by constructing a generalized annealing path connecting the posterior to an adaptively tuned variational reference. The reference distribution is tuned to minimize the forward (inclusive) KL divergence to the posterior distribution using a simple, gradient-free moment-matching procedure. We show that our adaptive procedure converges to the forward KL minimizer, and that the forward KL divergence serves as a good proxy to a previously developed measure of PT performance. We also show that in the large-data limit in typical Bayesian models, the proposed  method improves in performance, while traditional PT deteriorates arbitrarily. Finally, we introduce PT with two references---one fixed, one  variational---with a novel split annealing path that ensures stable variational reference adaptation. The paper concludes with experiments that demonstrate the large empirical gains achieved by our method in a wide range of realistic Bayesian inference scenarios.

Parallel Tempering With a Variational Reference
Nikola Surjanovic Department of Statistics
Saifuddin Syed
Department Statistics University of Oxford
University of British Columbia
nikola. stw.olaa.uranoiiseeetss ca
saifuddin, raiifuddin.sseddsttats... ox ac
Alexandre Bouchard-Côté
Trevor Campbell
Department of Statistics
Department of Statistics
University of British Columbia bouchard@stat.co ubc..cu.cccc ca
University of British Columbia
trevor ubc.
Abstract
Sampling from complex target distributions is challenging task fundamental to Bayesian inference. Parallel tempering (PT) addresses this problem by constructing a Markov chain on the expanded state space of a sequence of distributions interpo- lating between the posterior distribution and fixed reference distribution, which is typically chosen to be the prior. However, in the typical case where the prior and posterior are nearly mutually singular, PT methods are computationally prohibitive. In this work we address this challenge by constructing generalized annealing path connecting the posterior an adaptively tuned variational reference. The reference distribution is tuned to minimize the forward (inclusive) KL divergence to the posterior distribution using a simple, gradient-free moment-matching procedure. We show that our adaptive procedure converges to the forward KL minimizer, and that the forward KL divergence serves as good proxy to a previously developed of PT performance. We also show that in the large-data limit in typical Bayesian models, the proposed method improves in performance, while traditional PT deteriorates arbitrarily. Finally, we introduce PT with two references -ne fixed, one variational-wint a novel split annealing path that ensures stable variational reference adaptation. The paper concludes with experiments that demonstrate the large empirical gains achieved by our method in a wide range of realistic Bayesian inference scenarios.
Introduction
Parallel tempering (PT) is a popular approach to sampling from challenging probability distributions used in many scientific disciplines [1, 10, 5]. PT methods involve running Markov chain Monte Carlo (MCMC) on the expanded state space of a sequence of distributions that connect the target distribution of interest, T., to simple reference distribution, no. for which i..d. sampling tractable. The key innovation in PT is that the MCMC chain enables distributions along the path to swap states (or communicate). This communication enables i.i.d. draws from the reference mo aid in exploration of the challenging target Ti. Indeed, recent work has shown that the effectiveness of PT method is essentially characterized by the efficiency of this communication via the global communication barrier (GCB) from to Ti [36]. Intuitively, the GCB is low when the reference mo similar the target mi; in this case, the distributions along the path have substantial overlap and proposed swaps are generally accepted. The GCB is also inversely related to the restart rale, which quantifies how frequently i.i.d. samples from mo traverse the path to कı (a restart) [36].
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Basic Variational PT
Stabilized Variational PT
Standard PT
density
71
I
0.0
0.5
1.0
0.0
1.0
TT q q TO
parameter (x²)
Figure 1: Left box: visualization of the three main PT algorithms considered in this work. Nodes represent distributions interpolating between tractable reference distributions (bottom, either fixed or variational go), intractable distribution (top, #1, typically Bayesian posterior). Edges encode the structure of the possible swaps performed by the various PT algorithms. Right box: examples of path of marginal distributions ohtained on a Bayesian ODE parameter estimation problem with more than one latent variable for an mRNA transfection dataset [5. page 8]. The two modes shown) simultancously. In this examplc, the marginal of the prior places a small amount of mass on the second mode whereas the marginal of the variational reference places significant mass both modes. Because the variational reference covers both modes, the length of the annealing path from the reference to the target is shorter and it easier obtain samples from the target distribution using parallel tempering.
triviri to switch as they require changing other parameters (not
In the setting of Bayesian posterior inference a key application of PT, and the focus of this work the target m is the posterior distribution, and the reference distribution To is typically set to the prior. From the perspective of PT communication efficiency, this is poor choice in general; the prior often quite different from the posterior, resulting in a high GCB. As an extreme (but common) example, we show in this work that when the posterior distribution concentrates in the large-data limit, the restart rate with a fixed reference tends to zero and PT becomes computationally infcasible (Proposition 3.1). On the other hand, the posterior often exhibits regularities asymptotic normality in certain parameters, for example- that motivate the need for a choice of PT reference that can automatically adapt to the target to obtain computational gains.
In this work, we develop and analyze a novel PT algorithm that automatically adapts a variational reference distribution within a parametric family, Q {4s : é E D). This adaptive reference family addresses the shortcomings of using the prior as a reference: we show that in the large-data limit, the restart rate with an appropriate variational reference improves arbitrarily (Proposition 3.2). We find that even when one not the large data setting, our method can provide large empirical gains compared to fixed-refrennce PT in wide range of realistic Bayesian inference scenarios. The method is based on two major methodological contributions. First, we adapt the parameter ø to minimize the forward (inclusive) KL divergence KL( (//,) instead of directly taking gradients with respect to the GCB itself. This approach is particularly advantageous when Q is an exponential family: Theorem 3.5 shows that the forward KL is good surrogate of the GCB, and minimizing the forward KL amounts to matching moments, which involves no extra tuning effort from the user. We perform moment matching Theorem 3.4 identifies conditions that guarantee that the variational parameter estimate converges the optimum. Second, we combine two references-one fixed, variational-bar "gluing" two PT algorithms together (each based on one of the references, see Fig. 1). We demonstrate that this "stabilized" method necessary for obtaining a reliable PT algorithm: adaptation with just the variational reference alone can lead to "forgetting" the structure of the posterior distribution (e.g., multi-modality, as shown in Fig. 2). Although this requires more computational effort, show that under idealized conditions the restart rate of our adaptive method is no lower than half the restart rate of standard, fixed reference PT (Theorem 3.6) after accounting for the doubled computation time. In practice, it is often much better. Finally, the paper presents an extensive empirical study of the performance our method in a variety of real-world Bayesian models, including spatial models (sparse random field Poisson regression) and functional data analysis (Bayesian estimation of ODE parameters), among others. We find that our method can substantially increase the performance of PT.
simple iterative fashion, in rounds of increasingly many PT draws;

---

Target approximations random seeds
Trace plot for the random
Tempered restart rate
0.5
Standard PT
31
0.25
Catastrophic failure: mode "forgotten"
0.5
Basic Variational PT
31
0.25
0.5
0.15
Stabilized variational PT
31
0.25
Good seeds Poor seeds 0.11 (KS 0.1)
MCMC iteration
Figure 2: Comparison of three methods for Bayesian ODE parameter estimation with an mRNA transfection dataset [S, page 8J. Top row: standard PT succeeds in achieving a positive but small restart rate (a measure of PT efficiency; higher is better). Middle row: basic implementation of only one variational reference distribution leads to catastrophic failures in 3 out of 10 independent runs. The variational distribution sometimes "forgets" one modes (red denotes "bad" approximations with Kolmogorov-Sminn Smimmoo distance for the marginal posterior over the model parameter larger than 0.1) and becomes stuck in the other, which leads to an overestimation of the restart rate. Bottom row: using both fixed an adaptive reference addresses this issue and leads to markedly better performance in terms of restart rate compared to standard PT. All methods have comparable cost per iteration; all use the same variational reference family and run the same total number of chains and iterations.
Related work. The idea of adapting the tractable end-point of path of distributions has been explored most thoroughly in the context of estimation of normalization constants [24, 23, 39, 14). However, the importance of having both fixed and adaptive reference has not appeared in this literature. In the PT literature, the closest related work appears to be [31, section 5] and [7, section 4.1], which consider PT algorithmss with non-prior references. However, adaptive algorithm is proposed; the reference distribution is set to a fixed variational approximation in [31] and to an analytically tractable mode in [7]. Reference adaption has been considered in passing in the annealed importance sampling and sequential Monte Carlo literature, but without carefully considering the importance of stabilization discussed in our work, e.g. [15]. Another related line of work is the design of adaptive independence Metropolis- SectropoissHHsttings (IMH) proposals [9, 12, 32, 26]. In particular, the theoretical work studying convergence of adaptive IMH has recognized the usefulness of combining the adaptive reference with fixed reference [3, Section 7]. However, compared to the PT context, this combination is more straightforward to achieve in IMH via alternation of samplers. Finally, variational inference has previously been used to aid Monte Carlo methods, e.g., to learn proposals in sequential Monte Carlo [13], annealed importance sampling [34], and MCMC [25], and to form an initial reference for MCMC [17, 33]. None of these works connect the KL objective directly to the performance of PT algorithm as we in this paper.
Gaussian approximation
2 Background
This work builds on a recent state-of-the-at PT method, non-reversible parallel tempering (NRPT) [36]. The goal is to sample from a target distribution, IT, state space X. This is achieved by constructing a sequence N + distributions from Tı to a reference distribution To also on X. Given annealing schedule NN (B where 0 fo B1 < < AN with mesh size ||B = max, 13n Bn-1l, 1 the distribution for each chain n = O....,N N given by T3., where for all B E [0, 1). (x) x To 8 -(()) - (x)
3

---

Algorithm 1 Non-reversible parallel tempering (NRPT)
Require: Initial state Xo, annealing schedule Bv, # iterations T, annealing path n
In f 0 for n = ,......,N'
for = 1.2...., de
DpccalExploration// S. - if t is even Seve else Sodd
Local exploration kemels (e.g., HMC)
for n = 0, 0,I....,
do
An t- (x)
Acceptance probability from (1)
Tn (1-an)/T aT)(T Un + Unif(0, 1)
Store chain communication rejection rate estimates
if E St and Un S on then x" x² +- x², n1
Swap components n and n 1 of x
end for
X, t-2 x end for
Return: (x 1
In parallel tempering, we simulate a Markov chain X, = (X X that targets the product distribution on V given by π (x) TT (z²) TT, (x' ).. TaN (x as the unique invariant probability distribution of the Markov chain. Note that the target distribution Ti is marginal of ST, hence Monte Carlo averages based on X converge to the correct posterior expectations. Each iteration of PT involves two steps- local exploration and communicution as shown in Algorithm 1. The local exploration step involves applying, for each chain n, a πT, innvarint Markov kernel (e.g. Hamiltonian Monte Carlo). The communication step involves Metropolized swaps of states between adjacent chains; one alternates between swaps of chains n and n+ 1 for all n Sevnn followed by Soda. where Seven and Soda are the even and odd subsets of {O......V .N 1}. respectively. The deterministic alternation between even and odd swaps enjoys remarkable theoretical and empirical properties and ensures that the performance of parallel tempering does not degrade N increases [36]. Each proposed swap of component n and n + of (a ....,,NN) is accepted or rejected independently with probability
Tp, (x BBa4i (x") Ta (xx)
an (x) = 11
TT TB (x=+1)"
To characterize the performance of a parallel tempering method we study how often samples restart, i.e. travel from the reference To the target क [36]. Studying restarts isolates the effect of communication from the problem-specific characteristics of local exploration. (An improved local exploration kernel will of course improve overall performance.) When it is possible obtain ii.d. samples from To, the number of restarts is empirically found to be related to the effective sample size in the target distribution chain [36]. Formally, the restart rate T(Bx) from To to T with schedule By is the fraction of PT iterations that result in restart. The maximum value of T 1/2, since a communication swap is proposed with No at every other iteration of PT. If the local exploration is efficient (Assumption A.2) then r(Bx)= = 22+22 -N-1
I-ra ) [36, Corollary 1]. where = 1 - B[aa, [X)] and X ~ π. Asymptotically as N + cc. the round trip rate converges to a constant known as the asymptotic restart rate т: lim T rBBx = T (2 + 2A (To, m))
[36, Theorem 3], where A(TO, TI) is the global communication barrier (GCB) between no and mi
A(To, T1) 2//²
(z xu(2)²
E(|((Xg) ((x)))
{(x) = log
l1 X3,
T3.
(2)
The GCB can be estimated using the rejection rates ra for all adjacent pairs of chains, A(To, mi) EN Tn, where Tn = - Eia,, E[a,((X) for X ~ ग. with the approximation error decreasing to zero at a rate O(N-2) as the number of chains N increases [36, Section 5.2]. GCB values near zero imply that swaps are typically accepted and communication is efficient.
3 Parallel tempering with a variational reference
key degree of freedom one has when using PT is the reference distribution, un- Although the standard approach is to set To to the prior, Eq. (2) suggests that the GCB might be quite and

---

hence communication performance quite whor-when the prior and posterior differ significantly. which commonly occurs practice, Indeed, Proposition 3.1 motivates the importance of choosing the reference carefully: in a typical Bayesian model, as one obtains more data, the restart rate for the prior reference tends to zero. This result relies on Assumption B.3 in Appendix B, which stipulates standard technical conditions sufficient for, e.g., asymptotic consistency of the MLE, a Bernstein-von Mises result for asymptotic normality of the posterior [28], along with PT-specific assumptions such as efficient local exploration. We emphasize this result is motivation for our methodology; the proposed algorithms apply much more broadly and not only in the data limit.
Proposition 3.1 (Large-data restart rate, fixed reference). Consider data Y... = [Y:):= generated i..d. from a model with likelihood L(y(zo), To X C RR, satisfying the conditions in Assumption B.3. Denote mim to be the posterior conditioned on Ym. Then, in the large-data limit, the asymptolic restart rate Tm associated with the annealing path from na Ti, degrades arbitrarily, i.e, Ti 0 almost surely as m. + 00.
In this section, we demonstrate that allowing the reference be tunable addresses this issue.
3.1 Annealing paths with variational reference
Let Q = {as Φ E o) be a parametric family of distributions on X, and for each & E @ denote the annealing path from the reference distribution do the target Ti by
B E 10.1]; (x,)(x)) x (o(x))_P-
Note that for this modified annealing path, the target distribution remains the same although the reference may change. In the Bayesian framework, this means that the prior mo and posterior ı remain the same while the variational reference 1s is tuned. To ensure the variational reference family Q is compatible with the asymptotic PT theory developed in [36], we will assume Q is a PT-suitable family for the target TT. i..., each Os Q shares the same support at the target and satisfies some mild moment conditions (Assumption A. in Appendix A). We will also assume throughout that the fixed reference Wo is itself PT- for Ti P'T-suitability is sufficient to guarantee that the restart rate is inversely related to the GCB and that the schedule-tuning procedure from [36, Section 5.4) is justified.
3.2 Exponential variational reference family
The variational reference family Q should flexible enough match the target и reasonably well, but also simple enough to enable i.i.d. sampling, pointwise evaluation, and tractable optimization. Proposition 3.2 suggests that in the large-data limit, the family of multivariate Gaussian distributions often suffices. In particular, unlike the fixed prior reference-whore restart rate decays to zero in the large-data limit there exists a sequence of multivariate normal reference distributions so that the restart rate tends to its maximum value of 1/2. Note again that this large-data setting is just one instance in which a tunable reference helps; our method in this work applies much more broadly, and does not require Gaussian reference or rely on the asymptotic setup in Proposition 3.2. In particular, our method is advantageous in any setting where the GCB decreases compared to fixed-reference PT.
Proposition 3.2 (Large-data restart rate, variational reference). Consider the setting of Proposition 3. and suppose Q = [N(,, Σ) : 1 E Rd. E Rdxd Σ = E > 0} is a PT-suitable family for all targets 1im almost surely. Then for any fixed N > L, there exists a random sequence pmm E IRd, Em E IR dxd such that for any schedule By, in the large-data limit, the restart rate Tm (Bx) associated with Ti. N (Hmm Em) converges to the maximum possible value, Lee, for any schedule By we have Tm (Bx) 24 1/2 as m + 3c.
Proposition 3.2 motivates the use of tunable variational reference that can adapt the target, as opposed to fixed reference. In this work, we consider the general class of exponential reference families of full-rank where the distributions take the form qo (ar) = h((x) cxp(o) T 1y(x) - A(o)), for base density h, natural parameter é, sufficient statistic y), and log partition function A. Aside from being flexible enough to match posteriors arbitrarily well in the large-data limit, a key advantage an exponential reference family is that it is straightforward to fit: one can obtain the forward (inclusive) KI. divergence minimizer 40ka using a simple gradient- moment matching procedure because E, [7]. where Eg and E, denote expectations with respect to gs and Tı, respectively [16].
E In]
5

---

Indeed, under slightly more stringent technical assumptions in the setting of Propositions 3.1 and 3.2 namely Assumption B.9 Proposition 3.3 shows that we may use this forward KL as the reference sequence N (Jm: Em) for which the restart rate is asymptotically maximized. Assumption B.9 stipulates that the differences between the posterior mean and MI.E, as well as between the inverse Fisher information and scaled posterior variance, are not too large.
Proposition 3.3 (Large-data restart rate, moment matched reference). Consider the setting of Propo- sition 3.2 and suppose that Assumption B.9 also holds. Then, the conclusion of Proposition 3.2 holds if pm: Lm are set to the mean and variance of ni, conditioned on Y Ym, respectively.
3.3 Tuning the variational reference
In practice, we fit the exponential family reference iteratively by running NRPT for multiple tuning rounds r = ...... R: in each tuning round r we run T, = 2" iterations with variational parameter YT
f,. Using the generated states (X,,))=11> we obtain the parameter Φ for round r + 1 by solving
T (X T
S. [n]
Note that by relying on sufficient statistics, we are not required to keep in memory the MCMC trace to loop over MCMC samples when performing variational parameter optimization. For example, when Q is a Gaussian family, Eq. (3) simplifies to setting the mean vector and covariance matrix to the empirical mean and covariance obtained from the target chain samples X.N When full (non-diagonal) covariance matrix is used, one should start tuning when T, 2 d. We additionally use the samples from each round tune the annealing schedule By using the procedure from [36].
Theorem 3.4 shows that if the absolute spectral gap Gap(ó) [11] of the PT Markov chain with reference qá is bounded away from zero, and the number of iterations in each round tends to infinity at an appropriate rate, then ár will converge almost surely to the forward KI. minimizer ΦKL. Although Theorem 3.4 stipulates that 7) is bounded, this is technicality that is not required in practice.
Theorem 3.4 (Convergence of variational reference tuning). Suppose Q = {go Ø E D) is PT-suitable exponential family of full rank with sufficient statistic n(x)) bounded in x. Further assume that (KI exists and unique. Suppose each round of tuning starts at stationarity and there is k. > such that Gap(ò) 2 k > 0 for all @ and T, = (2(2") as r + sp. Then, (1): dr + ØKL almost surely as - 20; and (2): for all 0 < 3. almost surely there exists R(e) such that for all r 2 R(e),
3 [n] [v]l T: tc
3.4 Forward KL as surrogate objective
We now provide general theoretical justification for the minimization of the forward KL divergence as opposed to the global communication barrier, which is appealing as it enables a simple gradient- free moment matching procedure. First, note that when T Q. minimizing the KL divergence and the GCB is equivalent since they are both divergences [36]. Theorem 3.5 generalizes this to the more usual case where II & Q, demonstrating that the GCB the forward KI. minimum is bounded by quantities that depend on the flexibility of the variational family. In particular, provided that there exists Φ E such that the difference between log densities of the target n and reference 46o is bounded by a function g. then the GCB evaluated at the forward KL minimizer is bounded by a term involving expectations of g under the target and distributions do that are close to πT.
Theorem 3.5 (Forward KL proxy for the GCB). Suppose that Q = {uo & E Φ) is a PT-suitable exponential family of full rank. Let be any function such that for some do E & and for all E X, llog nı (z) - log 4ou (z) g(x).. Then, if ORL arg ming KL(TT (l96) exists and is unique, we have that A(gen, mi)
E [g] supseg Egggg) where n' {Ø Kl llgo) S (L (1(m)))0)))
We consider in Appendix D two simple examples to verify that the upper bound given by Theorem 3.5 small enough for practical purposes.

---

Algorithm 2 Variational PT
Require:
initial state # chains N- = 2NN, # total tuning rounds R. target wı, reference family Q {4a E Φ). initial reference parameter fixed reference ii Initialize annealing parameters uniformly with N₂- NV
N Bx
////N2//N,,. 2/,,,..,,
for r = ,,,....,,RR.............. Ro
T < 2"
>. Double the number of iterations in the next tuning round
T Concatenate ,o
> Concatenate paths using (4)
Bg Concatenate((g.w.55
Concatenate schedules using (5)
(x1) (r.) < NRPT By. s..
D PT with two references
Ba.N UpdaateSchecule((rr N-1 B3
b Tune annealing parameters for TT [36] > Tune annealing parameters for и [36]
By < (plate@Scedulle((r))))
BN)
Xa Xr 4- UpdateRefereece((:)]e
> Initialization for next round
Tune according to Eq. (3) another procedure
end for Return: (x1)
3.5
Stabilization with fixed reference
In Section 3.3 we provided result (Theorem 3.4) guaranteeing convergence of the adaptive scheme assuming the existence of an absolute spectral gap. In practice, the risk is that certain regions of + may significantly degrade the absolute spectral gap under the basic variational scheme discussed so far. For example, as shown in Fig. 2 (middle row), if the posterior multimodal, the variational reference may quickly center on one mode; because subsequent rounds of tuning use samples that depend on the variational reference, these samples may largely come from that one mode, causing the variational reference to remain trapped there for many tuning rounds.
To address this issue, we introduce parallel tempering with two reference distributions, using both the original (fixed) reference and variational reference, which we call "stabilized variational PT", illustrated in Fig. 1. We create an annealing path that starts fd proceeds along an annealing path to and then moves on new path from to To, connecting all three distributions. This modification adds significant robustness; as long as there are some restarts from the fixed reference, the target chain will escape the local optima and provide more accurate estimate of I used to tune Qs. In general, a well-tuned variational reference can provide a significant reduction GCB compared to just fixed path, but keeping the fixed reference ensures that the method will never do significantly worse than standard NRPT even if the variational reference tuning performs poorly (Theorem 3.6 below). Our variational PT algorithm with two references is presented in Algorithm in which UpdateSchedule refers to [36, Algorithms 2, 3].
To formalize the notion of a piecewise path, let T be the annealing path between q and T1, and let Th be the annealing path between the fixed reference mo and T1. We define the concatenated (piecewise) path म 0.
0 8 1
T 3
T
21
T2-23
3 1.
This new annealing path can be used within the NRPT Algorithm I as any other path. To tune the annealing schedule within each leg of PT with two references, we define the schedules Ba,N,
(Bo,n)m and B5 (B for the legs connecting 4s and mo Tı, respectively. Then, we define the concatenated schedule B.N = (Bo,n) N where N = No + N and
Bon 3
0 n S No N S n S
(5)
This concatenated schedule Bo, and path ne.9 are provided as input to the NRPT algorithm.
Finally, we provide an analysis of the worst-case performance of variational PT with two reference distributions. We show that the asymptotic restart rate of PT with two references is always greater than or equal to restart rate with either one of the two references alone. Because PT with two references requires twice the amount of computation, this amounts to a worst case of half the performance of regular PT with a fixed reference. practice we often find that including a variational reference substantially improves the PT restart rate.
7

---

Let F(B ) be the restart rate for T for the concatenated path, i.e. the rate at which samples from either reference 9s or TT reach the larget म.. Theorem 3.6 shows that if the Markov chain efficiently explores locally (Assumption A.2), then the restart rate of multiple-reference PT is the sum of the restart rate for Ti between d and To denoted T (B and r(Bx) respectively.
Theorem 3.6 (Restart rate of NRPT with two reference distributions). Let qá, To be PT-suitable references for the target Tí. Suppose the PT chains with references qs. with schedules .g.Ng NB respectively efficiently explore locally (see Assumption A.2). Then To Ta = T (B..N,, T(Bx) Moreover. if || 0, then
1
lim Fo (B) = N
2 -2 m1)
2A m1)
4 Experiments
We consider various Bayesian inference problems: 11 based on real data, and 4 based on synthetic data (see Table 1 in Appendix F for the details of each). The range of problem settings considered include spatial statistics, Bayesian ODE parameter inference, phylogenetic inference, and several distinct Bayesian hierarchical models. In all examples the variational reference is multivariate normal distribution with either diagonal estimated covariance matrix VPT_diag) or full covariance matrix (VPT_full) The code for the experiments is made publicly available: Julia code is available at httpe:
//github.com wtt.:////////////L////// Maaa-MML/Variiiionll and Blang code is at httpa: //tt://////tt/tt//// www.auu....uuueeeeeeeeeeeeeeeeee cww/UBC- Stat-XL/bll-ppe -pppe-e--e--
A distributed implementation is also under development at httpa: //github. figgghhb...oo// con/ Julia-lllaaal Temper inapprri///Piggoos. J1.
Experimental details can be found in Appendix F.
4.1 Comparative efficiency of variational PT families and PT baseline
We begin by comparing the communication efficiency of both VPT_diag and VPT_full to a state-of- the-art existing PT method, NRPT [36], which uses single, fixed reference. Note that there is also a computational trade-off between the two variational families (but we remind the reader that both still yield convergence of the target chain to the posterior). In particular, and hence potentially lower GCB- at the cost of a higher computational cost per iteration and more variational parameters to fit, while VPT_ has the same asymptotic computational cost per iteration as standard PT methods. We explore this tradeoff in Fig. 3.
offers more flexibility
We observe that both VPT_diag and fPl_full often substantially improve the restart rate compared to the NRPT baseline, and at worst achieve similar performance. The choice between fPl_full and VPT "PT_diag depends on the problem: for example, in low-dimensional problems such as Challem enger and Simple-min -mix, we observe that the full covariance in UPT_full is worth its additional cost per iteration. The situation is reversed in the Transfect PT variational family, we recommend VPT_ as safe default in light of Theorem 3.6 and of its computational cost per iteration asymptotically equivalent to NRPT.
problem. If one is pressed to select one
We also note that the tuning procedure converges relatively quickly. We show the number of restarts for the first 2.5% of computation time as insets in Fig. 3, and find that the number of restarts for the three methods can be distinguished early on. We also show in Fig. 4(a) that the GCB estimates converge in a small number of rounds for two additional representative problems (Lip Cancer and Vaccines). Note also that the GCB is substantially lower when variational reference is introduced.
4.2 Moment matching outperforms stochastic optimization
Next, we compare the proposed moment matching procedure described in Section 3 to several other stochastic optimization schemes to tune the variational reference. In contrast to moment matching- which is free of tuning saperr-praaeters we show in Appendix F.1 1.2 that stochastic optimization schemes require extensive hyper-parameter tuning. including step size schedule, choice of optimizer, and surrogate objective function. Moreover, we show in Fig. 5 (left) that it is difficult to specify a "default" hyper-parameter setting for stochastic optimization methods; a setting that works well on one problem (Rocket) generally will not work well on other problems (e.g.. Change Point, Titanic). contrast, moment matching performed well on all 14 problems we considered without requiring tuning. Consequently, our moment matching procedure is better candidate for integration
8

---

Challenger
Product
Simple Memple-MMxx
1590
Number ol metalte
Number restatt
Number at restars
1000
190
1.0
2009
400 Iseconds) Titanic
680
Time Transfection
Elliptic
Number of restats
Number of restorts
Number ot restarts
110
2.0
1009
25
565
100 (scconds)
150
206
2003
2003
2000
(secondo)
iseconds)
Figure 3: Number of restarts (higher is hetter) versus computation time in seconds for several models, with VPT full in green, VPT VPT_diag in red, and the NRPT baseline in blue. The two variational PT methods generally provide comparable better rate of restarts per second. Insets highlight the initial 2.5% of computation time, demonstrating that tuning of the variational references stabilizes quickly.
(a) Convergence of GCB estimates
(b) Markovian score climbing results
Up Cancer
Varches
Figure 4: (a) GCB VPT_ UPT_diag (red) and NRPT (blue) versus tuning round. (b, left) Tuning variational parameters using MSC with 10 replications (colours). The mean of the variational distribution for the model parameter 2 in Transfection model presented. The true marginal posterior distribution of æ bimodal (Fig. 2). We see here that MSC chooses one of the two modes for the estimation of the mean parameter. (b, right) Variational Gaussian approximation of the same parameter the Transfection model produced by 10 MSC runs with different seeds (colours). The aggregate of the different runs is shown an inset, ef. Fig. 2.
in probabilistic programming languages (PPLs), in which users do not expect be required to frequently change algorithmic tuning hyper-parameters. To illustrate this point, we have extended an existing open source PPL, Blang [6]. to include our method tuned via moment matching (code
available at https: con/UBC-Stat Stat--tttt-Mt-t---- MMMMMMM/lll-- (pt)).
4.3 Comparison to an externally tuned reference
We also compare moment- moment-matchnng matching variational reference tuning procedure to reference tuned by an existing procedure outside the PT context. In particular, we tested Markovian score climbing (MSC) [29] whih also optimizes the forward KL-for tuning a Gaussian reference with diagonal covariance. Details of MSC tuning, including sensitivity to stochastic optimization settings, can be found in Appendix F.11. The results for the Transfectias fentiiininn model presented in Fig. 4. In contrast to our stabilized moment matching approach (Fig. 2), tuning the reference using MSC in this example results in systematic catastrophic forgetting of one of the modes (Fig. 4 (b)) in all 10 replicates. We additionally refer readers to [21] for recent developments in score-based methods to minimize the forward KI. divergence.
4.4 Comparison of variational PT topologies
Finally, we compare the three algorithms introduced in Fig. (Basic Variational PT, Stabilized Variational PT and NRPT) on nine Bayesian inference problems (see Fig, 5, right).

---

Bocker
Uzange Picet
Taave
Ituree
Mavee
Tranet
Ptyro
App
Styhix
Srailind
Busid
PT
900
1000
5000 1000
5009
2003
1er02 Te400 1ee04 lancaco 1000 2000 10000
1000 2000 10000x0000
Bldget (number explorstion stepa)
lempered nestarts
Figure 5: Left: comparison of stochastic optimization and moment matching. An optimizer parameter setting that works well for the Rocket problem AAdam+FKL/SKL step size scale 0.1) does not generalize well to other problems Change-Point Titanic). contrast, moment matching reliably finds a well-tuned reference all 14 problems considered in this paper. Right: the same plot as in Fig. 2. but larger selection of models.
We set up the algorithmss so that the runtime per PT iteration has the same asymptotic complexity: this is done by selecting a diagonal covariance matrix for the Gaussian variational family and using the same total number of chains for all methods.
The results confirm the initial findings of Fig. 2: only the stabilized method always avoids catastrophic forgetting of modes. Moreover, in all but one example considered, we find that Stabilized Variational PT exhibits improved performance in terms of the number of restarts compared to the NRPT baseline (Fig. 5). The exception the 8-schools problem, in which the posterior is not well approximated by a diagonal Gaussian. Even in this case, as suggested by Theorem 3.6, the performance does not degrade by more than factor two. At the other end of the spectrum, for the Vaccines hierarchical model, the number of restarts increases >40-fold compared to the NRPT baseline, and for the spatial sparse conditional auto-regressive (CAR) model applied to the Lip Cancer problem, the performance jumps from zero restarts to 2300 restarts.
More results and details can be found in Appendix F.11.3, including alternative topological ar- rangements of variational PT algorithms, effective sample size per second results, well as global communication barriers for the problems considered in this section.
Conclusion
This paper addressed sampling from a complex target distribution within the parallel tempering framework by constructing generalized annealing path connecting the posterior to an adaptively tuned variational reference. Experiments in wide range of realistic Bayesian inference scenarios demonstrate the large empirical gains achieved by our method. Potential future work includes extending the gradient-free free tuning methodology to larger classes of variational families for the reference distribution. Further, heavy-tailed distributions can violate Assumption A. and is not clear without further examination what the implications are the convergence of the proposed algorithms. Therefore, another possible direction for future work is to develop methods suitable for heavy-taalld target distributions.
Acknowledgements
NS acknowledges the support of Vanier Canada Graduate Scholarship. ABC and TC acknowledge the support of an NSERC Discovery Grant. We also acknowledge use of the ARC Sockeye computing platform from the University of British Columbia.
References
[1] Kazunori Akiyama, Antxon Alberdi, Walter Alef, Juan Carlos Algaba, Richard Anantua, Keiichi Asada, Rebecca Azulay, Uwe Bach, Anne-Kathri Baczko, David Ball, et al. First Sagittarius
A Event Horizon Telescope results. IV. Variability, morphology, and black hole mass. The Astrophysical Journal Letters, 930(2):L15, 2022.
[2] Kazunori Akiyama, Juan Carlos Algaba, Antxon Alberdi, Walter Alef, Richard Anantua, Keichhi Asada, Rebecca Azulay, Event Horizon Telescope results. VII. Polarization of the ring. The Astrophysical Journal Letters, 910(1):L12, 2021.
Baczko, David Ball, Mislav Baloković, et al. First M87
10

---

[3] Christophe Andrieu and Éric Moulines, On the ergodicity properties of some adaptive MCMC algorithms. The Annals of Applied Probability, 16()):142--505, 2006.
[4] Guy Baele, Philippe Lemey, and Mare A. Suchard. Genealogical working distributions for Bayesian model testing with phylogenetic uncertainty. Systemalic Biology, 65(2):250-264, 2016.
[5] Benjamin Ballnus, Sabine Hug. Kathrin Hatz, Linus Görlitz, Jan Hasenauer, and Fabian Theis. Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems. BMC Systems Biology, 11(1):1-18, 2017.
[6] Alexandre Bouchard-Cöte. Kevin Chern, Davor Cubranic, Sahand Hosseini, Justin Hume, Matteo Lepur, Zihui Ouyang, and Giorgio Sgarbi. Blang: Probabilitistic Programming for Combinatorial Spaces. Journal of Statistical Software, 103:1-98, 2022.
[7] Ewan Cameron and Anthony Pettitt. Recursive pathways to marginal likelihood estimation with prior-sensitivity analysis. Statistical Science, 29(3):397-419 2014.
[8] Anand G Dabak and Don H. Johnson. Relations between Kullback-Leibler distance and Fisher information. 2002.
[9] Nando de Freitas, Pedro Højen- Michael Jordan, and Stuart Russell. Variational MCMC. In Uncertainty in Artificial Intelligence, pages 120-127, 2001.
[10] David Earl and Michael W Deem. Parallel tempering: theory, applications, and new perspec- tives. Physical Chemistry Chemical Physics, 7(23):3910-3916, 2005.
[11] Jianqing Fan, Bai Jiang, and Qiang Sun. Hoeffding's ' inequality for general Markov chains and its upplications to statistical learning. Journal of Machine Learning Research, 22(139):1-35, 2021.
[12] Jørund Gåsemyr. On an adaptive version of the Metropolis- algorithm with independent proposal distribution. Scandinavian Journal of Statistics, 30(1):159-173, 2003.
[13] Shixiang Gu, Zoubin Ghahramani, and Richard Turner. Neural adaptive sequential Monte Carlo. In Advances in Neural Information Processing Systems, volume 28, 2015.
[14] Iwona Hawryluk, Swapnil Mishra, Seth Flaxman, Samir Bhatt, and Thomas A. Mellan. Ref- erenced thermodynamic integration for Bayesian model selection: application to COVID-19 model selection. N0Xiv:2009.038511 2021.
[15] Jeremy Heng, Arnad Doucet, and Yvo Pokern. Gibbs flow for approximate transport with applications to Bayesian computation. Journal of the Royal Statistical Society: Series B, 83(1):156-187, 2021.
[16] Ralf Herbrich. Minimising the Kullback-Leibler divergence. Technical report, Microsoft Research, 2005.
[17] Matthew Hoffman. Learning deep latent Gaussian models with Markov Chain Monte Carlo. In International Conference on Machine Learning, pages 1510-1519, 2017.
[18] Richard Johnson. Asymptotic expansions associated with posterior distributions. The Annals of Mathematical Statistics, pages 851-864, 1970.
[19] Olav Kallenberg. Foundations of Modern Probability. Springer, 2021.
[20] Michael Karcher, Cheng Zhang, and Frederick A. Matsen IV. Variational Bayesian supertrees. arXiv:2104.1 1 191, 2021.
[21] Kyurae Kim, Jisu Oh, Jacob Gardner, Adji Bousso Dieng, and Hongseok Kim. Markov chain score ascent: A unifying framework of variational inference with Markovian gradients. In Advances Neural Information Processing Systems, 2022.
[22] Diederik P Kingma and Jimmy Ba. Adam: a method for stochastic optimization. In International Conference Learning Representations, 2015.

---

[23] Geneviève Lefebvre, Russell Steele, and Alain C. Vandal. A path sampling identity for computing the Leibler and J-divergences. Computational Statistics & Data Analysis, 14(7):1719-1731, 2010.
[24] Geneviève Lefebvre, Russell Steele, Alain C. Vandal, Sridar Narayanan, and Douglas L., Arood. Path sampling to compute integrated likelihoods: an adaptive approach. Journal of Computa- tional and Graphical Statistics, 18(2):415-43) 2009.
[25] Daniel Levy, Matthew Hoffman, and Jascha Sohl- Dickktein. Generalizing Hamiltonian Monte Carlo with neural networks. In International Conference on Learning Representations, 2018.
[26] Florian Maire, Nial Friel, Antonietta Mira, and Adrian Raftery. Adaptive incremental mixture Markov chain Monte Carlo. Journal of Computational and Graphical Statistics, 28(4):790-805 2019.
[27] Blažej Miasojedow, Eric Moulines, and Matti Vihola. An adaptive parallel tempering algorithm. Journal of Computational and Graphical Statistics, 22(3):649-666 2013.
[28] Jeffiey W Miller. Asymptotic normality, concentration, and coverage of generalized posteriors. Journal of Machine Learning Research, 22(168):1 2021.
[29] Christian A. Naesseth, Fredrik Lindsten, and David Blei. Markovian score climbing: variational inference with KL(pllq). In Advances in Neural Information Processing Systems, pages 15499- 15510, 2020.
[30] Radford M. Neal. Slice sampling. The Annals of Statistics, 31(3):705-767, 2003.
[31] UIrich Paquet, Ole Winther, and Manfred Opper. Perturbation Corrections in Approximate In- ference: Mixture Modelling Applications. Jourmal of Machine Learning Research, 10(43):1263- 1304, 2009.
[32] Hävard Rue, Ingelin Steinsland, and Sveinung Erland. Approximating hidden Gaussian Markov random fields. Journal of the Royal Statistical Society: Series B, 66(4):877-892, 2004.
[33] Francisco Ruiz and Michalis Titsias. A contrastive divergence for combining variational inference and MCMC. In International Conference on Machine Learning, pages 5537-5545, 2019.
[34] Tim Salimans, Diederik Kingma, and Max Welling, Markov chain Monte Carlo and variational inference: bridging the gap. In International Conference on Machine Learning, pages 1218- 1226, 2015.
[35] Thomas Severini. comparison the maximum likelihood estimator and the posterior mean in the single-parameter parameter case. Journal of the American Statistical Association, 86(416):997-0000 1991.
[36] Saifuddin Syed, Alexandre Bouchard-Côté, George Deligiannidis, and Arnaud Doucet. Non- reversible parallel tempering: scalable highly parallel MCMC scheme. Journal of the Royal Statistical Society: Series B, 84:321-350, 2021.
[37] Saifuddin Syed, Vittorio Romaniello, Trevor Campbell, and Alexandre Bouchard -ôtéé. Parallel tempering on optimized paths. International Conference on Machine Learning, volume 139, pages 10033-10042, 2021.
[38] Aad W van der Vaart. Asymptotic Statistics, volume 3. Cambridge University Press, 2000.
[39] Silia Vitoratou and Ioannis Ntzoufras. Thermodynamic Bayesian model comparison. Statistics and Computing, 27(5): 1755):166-11880 165-1180, 2017.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the pa- per's contributions and scope? [Yes] See the theoretical results (Section 3) and the experiments (Section 4).
(b) Did you describe the limitations of your work? [Yes] Limitations of the work are discussed in Section 3.5, where we assess the worst-case performance of the proposed method. We also note that chosen simulation examples are representative of various scenarios, including ones in which our method is comparable to a competitor.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Theoretical assumptions are described and referenced in the theorem statements of Section 3. They are laid out in full Appendix A and Appendix B.
(b) Did you include complete proofs of all theoretical results? [Yes] Complete proof's are provided in Appendices A-E.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material URL.)? [Yes] We include our code in public GitHub repositories. These repositories also contain the data or instructions for obtaining the data.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Experiments are described in detail in Appendix F.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] Instead error bars, results of experiments that were repeated multiple times were displayed graphically to allow for approximate visual estimates of uncertainty. See, for example, Fig. 3.
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] The description of the resources used is provided in Appendix F.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Appendix F for the list of models used and citations for the data sets.
(b) Did you mention the license of the assets? [Yes] Licenses mentioned in Appendix which summarizes the models and data sets used.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] We include our code public GitHub repositories. These repositories also contain the data or instructions for obtaining the data.
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [Yes] Licenses are described in Appendix F.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "xl8c3IDRA2l", "reviewer": "Reviewer_hxQQ", "summary": "This paper proposes to learn the prior distribution adaptively for parallel tempering. In particular, the prior distribution is tuned to optimize a proxy objective (forward KL divergence to the posterior) with a simple gradient-free moment matching procedure. In theory, the variational prior reference proves to outperform fixed reference, but in practice it may get stuck in a single mode, which the authors resolve by mixing the adaptive and fixed reference distributions. Empirically, the proposed method achieves a big gain over existing methods on Bayesian inference tasks.", "questions": "See some of my comments in the weaknesses section.\n\nOther questions:\n- The idea of tunning prior is quite natural and similar ideas have been discussed in Bayesian deep learning. So it is quite surprising to me that it has never been used before in parallel tempering. I'm curious about the authors' thoughts on that.\n- The authors mentioned that adding the fixed reference could help avoid the local optima and get a better estimate of the variational reference. However, the adaptive reference is still single-mode regardless. I wonder whether this would be an issue for harder problems.\n- Parallel tempering seems to be a promising sampling method, but I haven't seen many applications in the context of deep learning. By contrast, many other algorithms (including HMC and AIS) have been used widely in deep learning. I'm curious about the reasons and I wonder if the authors have tried on Bayesian deep learning problems.\n", "limitations": "The authors discussed the limitations in the paper and I don't see any negative societal impact of this work.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is very well written and easy to follow.\n- The introduced algorithm is intuitive and theoretically sound. In the large data limit, the moment-matched reference could achieve the best possible restart rate of 1/2.\n- The authors fixed the collapsed reference by adding fixed reference back in practice, which seems to work well empirically. To be fair, I'm not familiar with the datasets the authors used in the paper, so I don't know how convincing the empirical results are.\n\n\nWeaknesses:\n- Lack of discussions about the assumptions in theoretical analyses. For Propositions 3.1-3.3, the conclusions only hold under some assumptions mentioned in the Appendix. Adding some discussions or giving some intuitive explanations about the settings would be helpful for readers to understand the implications of all these propositions.\n- All the experiments are done on traditional inference problems with relatively toy models. In this case, I would expect sampling to be \"easy\". For models like deep neural networks, the posterior could be very complicated and I don't think the combination of a fixed and an adaptive reference would be enough.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "cLUju83TiI8", "reviewer": "Reviewer_BxuQ", "summary": "The authors proposed an improved version of the parallel tempering algorithm to solve the non-scalability issue with respect to the data size. In particular, the authors show that in the large-data limit, the restart rate degrades arbitrarily to 0, which strongly affects the communications between the chains associated with the target distribution and the prior distribution. To tackle that issue, the authors proposed to adopt variational inference based on the exponential family. Theories and experiments show much better restart rates.", "questions": "Before the exploration issue is elegantly solved, I still have concerns about the usefulness of this method to improve parallel tempering in big data.", "limitations": "NA", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros: \n\nI like the authors' insight on the weakness of parallel tempering with respect to the data size. Given a fixed schedule of parallel tempering, the communication efficiency does raise a concern in large-data limits. A major reason I am suspecting is that as the number of data points increases, the major mode becomes more dominant, which also inspires the authors to use a tunable prior based on variational inference.\n\nCons:\n\n1. I think the proposed method is not the right solution to tackle that issue. As is known that parallel tempering not only cares about communication efficiency (or restart rates) but also focuses on the exploration-exploitation trade-off. The current method seems to solve the issue of communication inefficiency, but the impact of exploration is not clear. If we don't know **how much exploration is sacrificed**, why not just adopt a prior that is close enough to the target distribution? In that way, we can maintain a large enough restart rate via the most vanilla method.\n\n2. The combination with a fixed reference further increases my concerns about this method in exploration, which has to resort to a different prior for exploration.\n\n3. Regarding the theories, I feel this paper is more suitable for a journal review.\n\n* I am familiar with Syed's JRSS-B'21 paper but the proof details of this work are not carefully checked.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "3XQ7q2490rd", "reviewer": "Reviewer_LMG3", "summary": "The work presents a new Parallel Tempering scheme that adapts a variational reference distribution within a parametric family. They adopt a parameter to minimize the forward KL divergence between the parametric family and the target distribution. They combine a fixed and an adaptive reference that leads to better restart rate performance than the baseline.", "questions": "My main question is regarding the scalability of the algorithm in deeper and bigger models. It would be interesting to see the method in those applications since the choice of Exponential distribution would make the scheme a good candidate for SG-MCMC like SGLD. But if this is not the scope of this work this is totally understandable and my decision is not conditioned on that.\n\nAlthough I come from the experimental part of Statistical Machine Learning community I like this work and I think it should be accepted in NeurIPS. This work fits the venue and could provide fruitful discussions between Parallel Tempering researchers.\n\n\nSome comments regarding the presentation:\n-I would put the figures closer to the references cause for the reader is very difficult to go back and forth in the text and try to find Figure and the corresponding text.\n-You need a conclusions paragraph.\n-Write more clear the advantages of the method in the introduction.\n-line 21: gold standard methodology for distribution approximation -> I would avoid using phrases as gold standard\n-should have legend on the figures and use the description to describe the plots and what the reader should focus on.\n", "limitations": "This is a theoretical work so negative societal impact is not discussed and the limitations are briefly but not clearly discussed in the main text (Subsection 3.5).", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-Interesting and witty idea combining a fixed and adaptive reference in the scheme.\n-Extensive theoretical analysis of the proposed scheme. The authors provide theoretical guarantees for the performance and convergence of the method.\n-Good presentation of the work.\n\nWeaknesses\n-A lot of toy experiments but not real world datasets. It would be interesting to see the method applied in a bigger model and a bigger dataset (like an image dataset MNIST, CIFAR10). \n-Structure is a bit odd since there are no conclusions and no discussion of limitations, future directions, societal impact. But again this is a theoretical work so societal impact is not applicable in this case. I would like to see the other stuff more in a separate paragraph though.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept