Accelerated Sampling with Stacked Restricted Boltzmann Machines
Authors: Jorge Fernandez-de-Cossio-Diaz
Clément Roussel
Simona Cocco
Remi Monasson
Venue: ICLR 2024 poster (2024)
Keywords: sampling
Restricted Boltzmann Machines
statistical physics
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: A study of accelerated sampling with stacked Restricted Boltzmann Machines, through analytical estimates of mixing times and clustering regimes

Abstract: Sampling complex distributions is an important but difficult objective in various fields, including physics, chemistry, and statistics. An improvement of standard Monte Carlo (MC) methods, intensively used in particular in the context of disordered systems, is Parallel Tempering, also called replica exchange MC, in which a sequence of MC Markov chains at decreasing temperatures are run in parallel and can swap their configurations. In this work we apply the ideas of parallel tempering in the context of restricted Boltzmann machines (RBM), a paradigm of unsupervised architectures, capable to learn complex, multimodal distributions. 
Inspired by Deep Tempering, an approach introduced for deep belief networks, we  show how to learn on top of the first RBM a stack of nested RBMs, using the representations of a RBM as ’data’ for the next one along the stack. In our Stacked Tempering approach the hidden configurations of a machine can be exchanged with the visible configurations of the next one in the stack. Replica exchanges between the different RBMs is facilitated by the increasingly clustered representations learnt by deeper RBMs, allowing for fast transitions between the different modes of the data distribution. Analytical calculations of mixing times in a simplified theoretical setting shed light on why Stacked Tempering works, and how hyperparameters, such as the aspect ratios of the RBMs and weight regularization should be chosen. We illustrate the efficiency of the Stacked Tempering method with respect to standard and replica exchange MC on several datasets: MNIST, in-silico Lattice Proteins, and the 2D-Ising model.

Published as a conference paper at ICLR 2024
ACCELERATED SAMPLING WITH STACKED RESTRICTED BOLTZMANN MACHINES
Jorge Fernandez-de--osioo-DDi
Clément Roussel* clement. roussel. ww1.ggoolyyecchiiuueu.ooooo org
Efdecdiicooude
Simona Cocco
Rémi Monasson
simona cocco@phys. fr
remi www.asncnnnnnys...oommm.m ens,
Laboratory of Physics of the Ecole Normale Supérieure. CNRS UMR 8023 & PSL. Research, Sorbonne Université. 24 rue Lhomond, 75005 Paris, France. (x) equal contribution
ABSTRACT
Sampling complex distributions is an important but difficult objective in various fields, including physics, chemistry, and statisties. An improvement standard Monte Carlo (MC) methods, intensively used in particular in the context of disor- dered systems, is Parallel Tempering, also called replica exchange MC, in which sequence of MC Markov chains decreasing temperatures and can swap their configurations. In this work we apply the ideas of parallel tempering in the context of restricted Boltzmann machines (RBM), paradigm of unsupervised architectures, capable to learn complex, multimodal distributions. Inspired by Deep Tempering, an approach introduced for deep belief networks, we show how to learn on top of the first RBM a stack of nested RBMs, using the representations of a RBM as 'data' for the next one along the stack. In our Stacked Tempering approach the hidden configurations of machine can be exchanged with the visible configurations of the next one the stack. Replica exchanges between the different RBMs is facilitated by the increasingly clustered representations learnt by deeper RBMs, allowing for fast transitions between the different modes of the data distribution. Analytical calculations of mixing times in simplified theoretical setting shed light on why Stacked Tempering works, and how hyperparameters. such the aspect ratios of the RBMs and weight regularization should be cho- sen. We illustrate the efficiency of the Stacked Tempering method with respect to standard and replica exchange MC on several datasets: MNIST, in-silico Lattice Proteins, and the 2D>Ising model.
in parallel
INTRODUCTION
General introduction. Sampling complex energy landscapes is an important goal in many scientific fields, including, but not limited to computational physics and statistical inference. Following the introduction of Monte Carlo (MC) methods by Metropolis in the late 40's, several approaches have been considered to speed up sampling. Among them cluster- algorithms employ non-local moves in the configuration space that are able to update large, possibly extensive number of microscopic variables at once (Swendsen & Wang 1987, Wolff 1989), but require detailed knowledge of the energy landscape for the construction of effective clusters. Another improvement of standard MC methods is parallel tempering (Swendsen & Wang 1986), which consists in simulating several copies of the system at higher temperatures, where energetic barriers are easier to cross. Exchanges of configurations between copies of the system at different temperatures the original system benefiting from the capability of regheteemerrture ones to quickly explore the configuration space, instead of getting indefinitely stuck in energetic valleys.
proposed, resulting in

---

Published as a conference paper at ICLR 2024
(a)
hiöden layer
(c)
/
4
3
6 8 4
0
4
MNIST
SA
Se
6
7
KMECCCIREVC KDEVGPLAGVI.
RCVIEADRLEG.
(b)
vibble laper
REYFRVNNINC.
MNIST (al digits)
P(W((v Ested lealinns
Pby
Docconatturt date
date
Lattice Proteins
20 lsing model
Figure 1: (a) RBM architecture, (b) Alternating Gibbs Sampling procedure. (c) Datasets used in the paper: MNIST digits (0/1 and all), Lattice Proteins (showing example sequences with high probability Pmu of folding into structures SA and SB. respectively), and 2-dimensional Ising model.
Contributions of this work. We hereafter report on an extension of the ideas of parallel tempering to data-driven modeling, which we called stacked tempering (ST). We use RBMs, paradigm of unsupervised architectures known be universal approximators of data distributions |Roxx & Bengio (2008). The distributions learnt by RBM may be complex and multimodal, and hence difficult to sample. We propose to learn stacks of nested RBMs, using the representations for the next one along the stack. Informally speaking, these RBMs learn more and more simplified versions of the data distributions, which become increasingly easier to sample with standard MC dynamics. We then couple the RBMs by allowing them to exchange configurations. These exchanges are made possible by the nested structure of the stack, i.e. the compatibility between the sizes of the layers of contiguous RBMs. We show numerically on three data sets, MNIST (LeCun 1998), in silico proteins (Shakhnovich & Gutin 1990), and the 2D Ising model Onsager (1944) that the resulting ST procedure is much faster than parallel tempering MC for sampling the learnt distributions. In addition, we present a mathemmtticlly tractable setup, in which the exponential (in the dimension of the data space) speed-up in sampling offered by ST with respect standard Gibbs sampling exactly calculated. We show how this speed up is intimately connected to how representations associated to data points evolve along the stack of RBM, depending on structural hyperparameters.
RBM as "data"
Relation to previous work, Neural network based algorithms are now widely applied to sample complex landscapes Wu et al. 20099) Noć et al. (2019); Gabrié al. (202T) Our work is inspired by the deep tempering procedure of Desjardins et al. (2014), introduced to accurately train deep belief
networks Hinton & Salakhutdinov (2006); Hinton al. (2006); Salakhutdinov & Murray (2008). In their original work, Desjardins et al. (2014) RBM layers with similar numbers of hidden units are all trained simultaneously with the aim of improving mixing of the MC chains used in the learning algorithm| Tieleman (2088) Contrary to deep tempering, stacked tempering trains RBMs one at time, and easier to implement in practice. By construction, RBMs at higher and higher depths in the stack hhaving fewer and fewer hidden units represent the data in an increasingly approximate way; their role is to extract more and more compressed encodings of the data distribution and help the bottom RBM sample its energy landscape. The idea that deep representations can be used for better mixing has been previously advocated Bengio et al. (2013) on a strong intuitive basis. To our knowledge the present work provides the first theoretical analysis (in minimal, overparametrized regime) supporting these intuitions, with a precise quantification of the mixing speed up offered by hierarchical clustering of deeper representations.
2 MODELS AND DATASETS
2.1 RESTRICTED BOLTZMANN MACHINES
Definitions. RBMs are undirected probabilistic models with two layers. A visible layer, which carries data points, is connected to a hidden (representation) layer through a weight matrix W, see Fig. a There are coupling between units within the same layer. The visible layer includes N units e. and the hidden layer is made of M units hy. For simplicity we hereafter assume v; = 0, (Bernoulli) 11 (similar to spins in statistical physics). The RBM model defines the joint probability
2

---

Published as a conference paper at ICLR 2024
distribution of a visible configuration {f4Je=1...N and hidden configuration h {hy p
N
M
N
M
1 e B(v, with E(v,h)= -- Σ P(v, h)
Uihh
Σ p=1
h (1)
Z
where Z is normalization constant known in physics the partition function, and the parameters Cy. and 9i represent biases acting on, respectively, units by and tq. The probability distribution P(v) of visible configuration can then be computed by marginalizing over the hidden-unit configurations:
1 - Z
P(v) P(y,
where (v) an effective energy function (see Appendix for an explicit expression).
Alternating Gibbs sampling. The bipartite nature of the RBM interaction graph suggests simple sampling algorithm, called Alternating Gibbs Sampling (AGS) and depicted in Fig. [Tb.. The basic observation underlying AGS is that the conditional distribution P(hlv) (respectively P(v[h) can be factorized over the hidden (respectively, visible) units. AGS consists of sampling from these two distributions in an alternated manner, for a large enough number of steps ensuring convergence to the correct equilibrium distribution P(v. h)). It is important to stress that the simplicity of AGS does not imply it is efficiently sampling the visible and hidden configuration spaces. AGS is generally not more efficient than standard Metropolis sampling of P(v) in the visible space, and is unable to sample distributions with multiple modes separated by large energy barriers (Roussel et al. (021).
The RBM is parametrized by its weights W = [Wip)) as well as the biases C₁ and Ii considered. All these parameters, generically denoted by θ must be learned from the training data, consisting of set of K samples {v"] J==I..... This is done through gradient ascent of the log- ligllikeihoodd of the data, LL(9) 1/ Ek=1 log P(v The generic expression for the gradients is
Training.
aLL 00
[BE(v) {
(OBLI)) d0
d0
data
model
where (-)aata denotes the expected value over the data {v*) k= K and (-) model over the model distribution P(v). The latter is an average over exponential number of terms and is thus untractable, and is estimated in practice with the Persistent Contrastive Divergence algorithm Tieleman (2008). Note that there is no inconsistency between the capability of training RBM, which directly relies on data to initialize the parallel dynamical chains in persistent constrative divergence, and the hardness of sampling it at later times, by exploring all the modes of the model distribution. We include regularization in the training procedure, e.g., by replacing (.T - LL(()) 7||0||2 in the case of Ta penalty over the model parameters.
2.2 DATASETS
We consider several datasets to illustrate the performance of our sampling procedure:
MNIST. is a large dataset of 28 x 28 pixel images of handwritten digits [eCunn 1998), see Fig. Ic). We binarize pixels (white or black). It is empirically known that transitions between very different digits, e.g. 0 and 1, are rarely observed when sampling a RBM trained on MNIST with AGS Béreux let al. 2023); Fernandez-de Cossio-Diaa -Daz al. (2023); Roussel et al. (02T). We test ST on MNIST and MNISTO/1, the restriction to O & digits only.
Lattice Proteins (LP). are synthetic protein models introduced to investigate protein design, that is, the search for sequences of amino acids likely to fold into a prescribed 3D structure. In the LP models introduced by Shakhnovich & Gutin (1990) and Mirny & Shakhnovich (200I) 27-long sequences v fold onto 3 3 x 3 lattice cubes. The probability Poat (Slv) that sequence folds into one of the 2 105 distinct structures S is an intricate function of the interactions between nearby amino acids on the structure, see Appendix for details. We choose two widely apart structures SA and Sg considered by Jacquin et al. (016)), see Fig. dd). Our dataset made of sequences v with high probabilities to fold in SA or in Sp. Lee. such that Poat (SAl) > 0.99 or Pnat(Snn v) > 0.99.
3

---

Published as a conference paper at ICLR 2024
Figure Principle of Stacked Tempering, illustrated with stack of three RBMs. Data are learnt and generated by the bottom RBM. Al- ternating Gibbs Sampling is used generate visible and hidden configu- rations in each RBM. At any step t the configurations vn+1 ble layer of the (n + 1)th RBM and h on the hidden layer of the n RBM can be exchanged, with prob- ability An (h,, v v see Eq.
n=
I AGS
swap attempts
t AGS
0
the visi-
swap attempts
0=1
AGS
2D Ising model. We consider a celebrated model of magnetism, the 2-dimensional Ising model on 32 32 grid with homogeneous interactions between closest neighbors Onsager (1944). This model is hard to sample close to the phase transition temperature, below which the magnetization is non zero. See Appendix for more details.
3 STACKED TEMPERING: PROCEDURE AND APPLICATIONS
We now present the Stacked Tempering (ST) procedure, in which nested set of RBMs recursively extract relevant collective modes of their predecessors, and, in turn, ease their sampling.
3.1 ARCHITECTURE AND TRAINING
The architecture supporting ST is sketched in Fig. 2 It consists of a stack of RBMs, whose widths such that the number M,, of hidden units of the n RBM equals the number N,, of visible units of the (n + 11)th one. This constraint is essential to make communication between RBM possible, as explained below. In addition, we choose the sizes of visible layers to decrease with the index n, i.e. N₂ > Nn-1, making deeper RBM 'simpler' than their predecessors in the stack.
The main RBM, which is trained through maximum likelihood on the data {v*] is the bottom one (n = 1). After training, a set of hidden representations {hf} of dimension M1 are obtained by sam- pling the conditional probabilities P1 (hlv*) for every 1 = 1, 1..., K. These hidden representations are then considered as training data configurations {vī = h for the next RBM, n = 2. This is possible since the dimension of the visible layer of the second RBM, N2, is equal to M1. After training of the second RBM set of representations {h?} are drawn, which are then used as 'data' {vš = h for training the third RBM. The process is iterated all the way up to the last RBM. In summary, the RBMs are trained sequentially, starting from the bottom one.
3.2 SAMPLING AND EXCHANGES
Each one of the RBM is sampled with AGS, independently of the others (Fig. (Fig.Ba)). In addition, after each AGS step I, an attempt is made to swap the hidden, h and visible, v n configurations of, respectively, the n and (n + 1)th RBM, see Fig, 2 a). This swap occurs with probability
(h vh+1) min (1, P (hL) Ph (v (v++1)
(4)
p (v (a
where ph and R++1 are the marginal distributions of the hidden and visible configurations of the nh and (n + 1)th RBM. The definition of An ensures that detailed balance is satisfied. Crucially it does not depend on the intractable normalizations (Z factors) of P P ph: which cancel out.
Due to the decrease of the layer width with depth (Fig. a)) and to weight regularization, each RBM expresses coarse-graine grained approximation of the landscape captured by the previous RBM. The deepest RBM can therefore be casily sampled, and in turn help the RBM below to undergo non local moves in its more complex landscape, without getting trapped in valleys.
4

---

Published as a conference paper at ICLR 2024
90
04)
Litiic prode
30
1000
PICMO
P1/10000
50-
curpling rape
sorqp evee
10000
UNIST
sampping Akpa
t0
500
1000
cumpligg ARRA
Figure 3: Application of Stacked Tempering to diverse datasets. (a,b,c) Cumulative number of transitions between modes during sampling MNISTO/I, Lattice Proteins, and 2D Ising model, with Stacked Tempering (ST), AGS, and Parallel tempering (PT, legend shows number of intermediate temperatures). Results are averaged over 100 MC chains. For MNISTO/1, perceptron classifier is used to determine the sampled digits, and transitions between O and are recorded. The æ-axis gives the number of Alternative Gibbs sampling (AGS) steps. Note that different algorithms perform different numbers of AGS per iteration, e.g., PT performs one AGS per intermediate temperature, and ST performs one AGS per stacked RBM. Autocorrelation times are reported in Table [S5] (d) shows a representative time-course of the classifier score during sampling, while (e) shows example sampled digits with AGS (top row) and with ST (below). Digits are displayed every 750 steps and the two dynamics are initialized with the same configuration. ST generates high-quality digits and mixes well between the two classes. (f.g) Representative (v))(v of sampled sequences for the two structures S SA, Sp. with ST. ST generates high-quality sequences, folding either on S or SB (Punt > 0.9) for over 80% of the samples, and mixes well between the two families of sequences. (h) For RBMs trained on full MNIST, deep convolutional neural network classifier LeCun et al. (1998). The matrix indicates digit pairs for which at least one transition is observed in average every 10 MC steps for ST (green) and AGS (black). ST is able to jump quicker between many more pairs of digits than AGS.
count the transitions between each pair of digits, using
3.3 APPLICATION TO MNISTO/1, MNIST, LATTICE PROTEINS, AND 2D ISING MODEL
We now illustrate the efficiency of ST with respect AGS and Parallel Tempering (PT). In PT the weights of the RBM are down-scaled by a parameter (called temperature) resulting in a smoother energy landscape. A series of RBMs different temperatures are allowed to exchange configurations to accelerate sampling. Our implemention follows Desjardins et al. (2010): :Cho et al. (2010). with uniform inverse temperatures between (the target distribution) and 0.
MNISTW/1. AGS is stuck in one of the two digit classes, see Fig. 3(a) Conversely, ST with stack
of four RBMs (N, 784, Mj N2 2000, M2 Na 100, M3 N₂ 25, M, 10) is able to frequently jump from one class to the other, see Fig. 3b. The number of digit switches increases linearly with sampling time, see Fig. cd.). We also implemented PT with increasing number of uniformly distributed intermediate temperatures (d). To compare all algorithms on the same time-scale, the z-axis counts the total number of Gibbs sampling steps on any RBM of the stack (PT uses one RBM per temperature, while DT uses one RBM in each layer).
MNIST. AGS is able to jump between specific (similar) pairs of digits, such as - 5, or - 3, as shown in Fig. 3h. Thus AGS takes long indirect paths to sample all digit classes. ST, 4 stacked RBMs with M₁ = 500, M2 = 200, M3 = 50, and M4 = 10 hidden units, is able to jump faster even many more pairs (even dissimilar) of digits.
Lattice Proteins. Three RBMs are used (N1 = 27 x 20 as amino acids can take 20 possible values, Mj N₂ 800, M2 N₂ = 50, M3 25). ST improves mixing between the sequence subspaces associated to structures S, A and Sn and generates high-quality sequences correctly folding either on one on the other structure, see Fig. e.f). The mixing properties of ST are much better than the
5

---

Published as a conference paper at ICLR 2024
ones of AGS and PT (Fig. 3 gg). AGS and PT almost never switch class (hence the plots overlap around zero).
2D Ising Model. A stack of 3 RBMs is used in ST, with 50, 20 and 1 hidden unit. We count the transitions in terms of the magnetization sign. ST performs cluster moves of large groups of spins efficiently, facilitated by the units learning about overall the magnetization of the lattice patches of nearby spins Mehta & Schwab (014); Fernandez-de Cossio-Diaz et al. (2023), as is evident from inspection of the weights of the stacked RBMs (see Appendix [C IJ,, whereas AGS and PT keep generating configurations with fixed magnetization sign (set by the initial configuration). See Fig. for example trajectories of the magnetization during sampling.
The intuitive picture of gradual simplification of landscapes along the stack of RBMs is supported by the applications MNISTO/1 and Lattice Protein structured data. The RBMs along the stacks progressively cluster the representations of the data, see Appendix Table [ST for the numbers of distinct representations various depths. Interestingly the cost of introducing additional temperatures in PT is not always offset by accelerated exploration. PT has difficulties in presence of separate modes, a phenomenon akin to critical slowing down second-order phase transitions, and requires a large number of temperatures near critical point Katzgraber et al. (2006).
4 THEORETICAL ANALYSIS
We now seek to characterize the conditions on the data distribution and on hyperparameters allowing ST to be efficient. To obtain a minimal and analytically tractable framework, we consider the case of strongly overparametrized RBM, ie. with layer widths N, M -> oo. Our analysis is organized along three steps: (I) the determination of the form of the coupling matrix through maximum fog-likelihood training; (2) the identification of the pattern separation/pattenn completion regime and phase transition depending on how much the RBM is regularized; (3) the calculation of the mixing and swapping times for stack of two RBMs, and the discussion of when and how much ST accelerates sampling.
4.1 A MINIMAL SETTING FOR ANALYZING RBM DATA- TO-A SOO--RRRREEENTTTTOO MAPPING
We consider the minimal case of K = 2 data configurations, v and v², with dot product = v². v²/N: Here, visible and hidden units are chosen to take #1 values. We learn these two data points with RBM, through maximization of the regularized log-likelihood,
NY
LI log P(v") + log P(v?)]
Σ W
2
over the weights Wi,., where 7 > 0 the regularization strength. As shown in Appendix E in the limit N, MM oo at fixed ratio a M the trained weights are of the form (after suitable ordering of the hidden units)
N
W_ if
isp5 1
WTe = X N
(6)
W+ 2 1 <pSM,
if
I+
where = hhh -hh-h//a is the overlap between the most likely hidden representations h' h2 of the data points v0 22 The values of W+ are chosen to maximize LL, resulting in the stationarity equations:
1 +
+ 2
7 W = tanh
W+ mint. my tanh
ooW
nr tanh (Wimı) (7)
2
These g-dependent weights can be injected back into the LL in Eq. 5 allowing us determine the value of E [-1++] +1] maximizing the log- lieeliik,,,ood, hereafter referred to as y y"(x,, (2).
4.2 REPRESENTATIONA REGIMES AND PIIASE TRANSITION
Our first aim to understand the tataatoo-reprseenter tion similarity y is related to the overlap between data points, æ. We first find that y is decreasing with a and increasing with y at fixed data overlap z, see Fig, aa.b). In other words, regularizing more the RBM, by decreasing either the amplitude of the weights (through higher La penalty 7) or
mapping, more precisely, how the representa-

---

extreme version of Published as a conference paper at ICLR 2024
(8
(9)
1.0
-0-0:0.05
y=60005
y=0.05
0 +0003
H0
Figure 4: Optimal overlap a.) (a) Behaviour of "² vs. at fixed 0.2 (b) Behaviour of y² vs. a at fixed x 0.5. (c) Behaviour y vs. at fixed y = 0.015; the dashed lines shows y = x.
the number of hidden units (equivalently, the aspect ratio a), produces more similar representations. For strong enough regularization, we obtain y" = 1, showing that the two representations become identical though the data points are not.
We then plot y as a function of x in Fig. c). We identify three different regimes. (i) For low- to-intermediate data overlaps and low regularization (small % large a),, y" is slowly increasing function of 2, with slope smaller than unity. The RBM has tendency to produce representations less correlated than the corresponding data. This regime can be referred to as pattern separation, vocable used the context computational neuroscience (Rolls) 2013). (ii) For low-to-intermmedite data overlaps and large regularization (large 7, small a),, y is quickly increasing function of z, with slope larger than unity. The RBM has a tendency to produce representations more correlated than the data. This regime is reminiscent of pattern completion (Rolls) 2013). (iii) For large data overlaps, Le. for & > Te. y" = 1: the RBM has mapped similar but distinct data onto a unique representation. This regime pattern completion. The value of xe as a function of a, 7 can be computed as discussed in Appendix
be referred
clustering and can be
The boundary between the pattern separation and completion regimes is a phase transition, clearly visible the abrupt change of slope of y" in Figs. [a.b). Details about the phase transition and the attached order parameter can be found in Appendix E This phase transition has concrete interpretation in terms of the generative diversity the RBM. Once the training done we can use our RBM to generate 'new' data through Alternating Gibbs Sampling. In the pattern completion regime, whatever the initial configuration = v or v² on the visible layer, the RBM will generate the same data distribution. In the pattern separation regime, the ergodicity of the visible configuration space broken, and the RBM will generate configurations v similar to the data point present at the initialization of the MC chain.
As final note let us emphasize that the representation regimes and phase transition identified here is not specific to the case K 2. The analytical caleulation of the log-likelihood LL done above can be extended to any finite K > 2 (while N, M are sent to infinity). Analysis of the extremization equations for K generic data points shows that, y progressively decreased, sequence of clustering-like phase transitions takes place, with more and more similar representations. The number of distinct representations varies from K to I regularization is made stronger.
4.3 ANALYTICAL CALCULATION OF THE AVERAGE MIXING AND SWAP TIMES FOR STACK OF TWO RBMs
We now analytically calculate the speed up offered by ST with respect to AGS in the case of stack of two RBM learning K data points. The two RBM are defined by aspect ratios d1 02 and regularization strengths 71, 72. see Fig. 5(a) We call x the overlap between the data points. The overlaps y1 between their representations in the first RBM, and y/2 between the representations of these representations in the second RBM can be determined as functions of the other parameters according to the previous Sections, with the results y1 = (x. (1, 71) and 1/2 y (V1, 02,,
Mixing time with AGS for a single RBM. We consider a single RBM, with data and representations overlaps equal to, respectively, x and y, aspect ratio and La penalty 7, as in Section 4.1 Extending
7

---

Published as a conference paper at ICLR 2024
(a)
(b)
10
(d)
(e)
a
AG8
W
Disss (2)
1
10"-
57
0.03
a,N
100-
0.00
0.02
0.01
swap
100
200
300
(c)
a,N
10'
0.0
0.5
1,0
0.1
02
0.3
W
1.0
100
Grond
Towx
0.5
12
0.-
N
0.0
data
0.0
0.5
1000
3000
N
Figure 5: Theoretical analysis of Stacked Tempering. (a). Stack of two RBMs. Teross are the mixing times of the Iwo machines, Tawop is the average time between swaps of h1 and V2. (b,c) Exponential scaling of the swap (b) and (c) mixing times as a function of the data dimension N. Two parameter settings (values in Appendix H) are shown each panel. (d,e) Average switching times (in log-scale) for ST and AGS, vs. (d) the data overlap at fixed wy 5, = 0.3, and (e) the penalty strength 7 of the top RBM, fixed x = 0.5, 02 = 0.5. Bottom panels show y/2 maximizing the likelihood of the top RBM.
the statistical mechanics approach that allowed us to determine the value of LL we can compute the free-energy barrier separating (when y < 1) the two minima of the energy landscape. This barrier, through Arrhenius law, gives access to the mixing time for switching from one minimum to another, Teroes N ((2,0,)) where the expression the barrier height is
B(I,a,7) = extr {a (a112 n.
(1-y) W- m_ n + 2 2
H(m + ( (1--) H(n)
(8)
with y y² (xr, a, 7). and where H(m) (m) is the binary entropy function. Detailed calculation can be found in Appendix G. Note that the time Tcross is exponential in N, showing that mixing is extremely slow, expected for AGS of multimodal data distribution (Roussel et al. (021). This asymptotic
result in very good agreement with numerical estimates for finite N, see Fig. ((c).
(I) The result above can be used to estimate the crossing times Teross and Teross of the two RBMs (when they are not coupled through replica exchange), with appropriate choices
(2)
parameters.
Swap time between two RBMs. We now study how the two RBMs in F which generate, respectively, configurations (vf, hi} and {vá, J. can occasionally exchange their configurations. The swap between h and v22 is accepted with probability A (hf, vÿ). see Eq, 4 The mean value
(A) this acceptance ratio can be computed, and inverse is an estimate of the characteristic time
Tswap between two replica exchanges (see Appendix F for detailed calculation), Tswap (A)
NG" to leading order in N, where
G min G(m+, G(m+,m,, m. m+, mmmmm))))))))))..m.........))m(. m-) G min
G(m+,m_, m_ m_,n,,n-)
and G(m,, m_ n4, n- is a free energy cost associated to swapping the configurations of the neighboring RBMs:
G(m+, m_ n.., n.) (n4, ((n,,n_)) m E (m+, m_) (m+,m_.) Smm,, m.)
(10)
where E/, Fy are the effective energies of the two RBMs at the contact layer, and S(L+, I+ ++ H(t_)t is an entropic contribution. This analytical prediction for Tswap is in excellent 2
=
agreement with numerical estimates, Fig. 5)...
Speed-up of ST with respect to AGS. We now combine the results above to determine the gain offered by ST over AGS. Sampling the bottom RBM (parameters a1, 71) with AGS will lead to switching time between modes of the order of TAGS T(1) e B(x,a..11) By comparison,
8

---

Published as a conference paper at ICLR 2024
sampling with ST will take time
TST max (T(0)s Tswap
e max (B1)).
(11)
Results for 7ST and TAGS are shown in Fig. d.e) for representative parameter values of a2 and 72- We observe the existence of an optimal speed up with ST, well below the value of the time required with AGS. This optimum implements trade-off between strong compression of the representations low acceptance rate of the swap attemps (obtained for instance for small ag or large Y/2 for the top RBM) and weak compression high acceptance (for large (22, small 7/2).
DISCUSSION
In this work, we have first introduced sampling algorithm for RBM. building upon the ideas of Parallel and Deep Tempering. Contrary to the latter Desjardins et al. (2014), our Stacked Tempering (ST) procedure is not a training algorithm for DBNs. By decreasing the number of hidden units with the depth along the stack, the RBMs encode increasingly simplified versions of the bottom energy landscape of interest, The fast mixing of high RBMs then propagates down through replica exchanges. We have shown, on varied datasets coming from machine learning, computational biology and statistical physics, that ST is more efficient than Alternate Gibbs Sampling and Parallel Tempering. even with much larger number of parallel chains.
In the second part of the paper we have provided analytical support to the observed gain in mixing time offered by ST, based on careful analysis of a stack with two RBMs. To obtain an analytically tractable framework, we have considered the case of largely overparametrized RBMs. This setup allowed us to determine transition between pattern separation and a pattern completion (clustering) regimes for the hidden representations, controlled by the hyperparameters of the RBMs. This transition has profound impact the average mixing and swapping times, emphasizing how the mapping between data and representations and the efficiency of sampling with ST are intimately interconnected. Let us also stress that, despite the minimal nature of this mathematical setting, we were able to obtain non-trivial analytical results in a field where exactly solvable models are not frequent. We also observe analogous clustering and separation regimes in real datasets (see Fig. S5 S6
An apparent limitation of our sampling procedure is the cost of training the stack of RBMs compared to other methods. In the architectures we consider, deeper RBMs have however less units and less rugged landscapes, and we find that their training converges faster (see Tables and and S4) Thus in practice, this additional training cost might not be significant. Another limitation of our work is that our theoretical analysis focuses on a over parameterized regime with few data points. This setting allowed us to derive exact analytical expressions that we believe yield insight into more general situations, but it remains unclear how it could be extended to other settings, for instance, when the number K of data points is comparable to their dimension N.
Our work could be extended in many directions. First, we could exploit the benefit of most efficient mixing provided by ST during the learning phase itself. Exchange efficiency across the stack could be used secondary training objective, e.g. through terms such log A between contiguous RBMs. These contributions, in addition to the log-likelssoods already considered in the training, would favor smoother energy landscape at the bottom of the stack, in manner akin to regularization. More generally, one would like to have a better understanding of how to optimize hyper-parameters of the stack (number of RBM layers, hidden units, regularization). Second, conceptually appealing feature of ST is that each single data point is associated to a sequence of representations, one per RBM in the stack. It is tempting to associate these nultirreprsssntatiion to multiple levels of adaptive Searse-grainig of the data, built by the RBMs in fully unsupervised way. A better characterization of these multiireppessenaaiions would be very interesting. From this point of view, we believe that our ST procedure, which lies half way between a single RBM and full DBN could offer a valuable path to better investigate the properties of deep unsupervised networks. Last of all, ST could applied to model physical systems, where it is desired to approximate or sample a known classical or quantum Hamiltonian Carlco et al. (2019); Carleo & Troyer (0177). or to aid training deep models like DBN.
Acknowledgments. We thank Alessio Giorlandino for useful discussions. JF&CD acknowledges support from Université PSL Al Junior Fellow program. JFdCD, CR. SC, RM acknowledge support from Grant No. ANR-19 Decrypted CF3000021-01.

---

Published as a conference paper at ICLR 2024
REFERENCES
Yoshua Bengio, Gregoire Mesnil, Yann Dauphin, and Salah Rifai. Better Mixing via Deep Represen- tations. International Conference on Machine Learning, pp. 552-560. PMLR, February 2013. URL
mlr. www.ir.p.eesseee / https////2///engiooo/
html ISSN: 1938-7228.
Nicolas Béreux, Aurélien Decelle, Cyril Furtlehner, and Beatriz Seoane. Learning a restricted boltzmann machine using biased monte carlo sampling. SciPost Physics, 14(3):032, 2023.
Giuseppe Carleo and Matthias Troyer. Solving the quantum many-body problem with artificial neural networks. Science, 355(6325):602-666, 2017.
Giuseppe Carleo, Ignacio Cirac, Kyle Crammer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, and Lenka Zdeborová. Machine learning and the physical sciences. Reviews of Modern Physics, 91(4):045002 2019.
KyungHyun Cho, Tapani Raiko, and Alexander Ilin. Parallel tempering is efficient for learning restricted boltzmann machines. In The 2010 international joint conference on neural networks (ijenn), pp. 1-8. IEEE, 2010.
Guillaume Desjardins, Aaron Courville, Yoshua Bengio, Pascal Vincent, and Olivier Delalleau. Parallel tempering for training of restricted Boltzmann machines. Proceedings of the thir- teenth international conference on artificial intelligence and statistics, pp. 155-152. MIT Press Cambridge, MA. 2010.
Guillaume Desjardins, Heng Luo, Aaron Courville, and Yoshua Bengio. Deep Tempering, [cs, stat/, October 2014. URL
arXiv: 1410.0123.
Jorge Fernandez-de Cossio-Diaz, Simona Cocco, and Rémi Monasson. Disentangling representations in restricted boltzmann machines without adversaries. Physical Review X, 13(2):021003, 2023,
Marylou Gabrié, Grant M Rotskoff, and Eric Vanden-Eijndee Efficient bayesian sampling using normalizing flows to assist markov chain monte carlo methods. arXiv preprint arXiv:2107.88001, 2021.
G. E. Hinton and R. R. Salakhutdinov. Reducing the Dimensionality of Data with Neural Networks. Science, 313(5786):504-507, July 2006. ISSN 0036-8075, 1095-9203, doi: 10. 126/science 1127647. URL https::/:///c
scwencemag org/ http://onnnntt33////eee lisher: American Association for the Advancement of Science Section: Report.
Pub-
Geoffrey E. Hinton, Simon Osindero, and Yee Yee-Whye Teh. fast learning algorithm for deep belief nets. Neural Comput, 18(7):1577-155, July 2006. ISSN 0899-7667. doi: 10.1162/neco.2006 18. 7.1527.
Hugo Jacquin, Amy Gilson, Eugene Shakhnovich, Simona Cocco, and Rémi Monasson. Benchmark ing inverse statistical approaches for protein structure and design with exactly solvable models. PLOS Computational Biology. 12(5):1-18, 05 2016. doi: 10.137 1/371/joumll.pcb 1oabb..10440 1004889. URL
https:://dii.... wtw.:ioorgg11... 1371 3137/////rral.. journal.... pwt.maallppppppppp..........a.rl 1004889.
Helmut G Katzgraber, Simon Trebst, David A Huse, and Matthias Troyer. Feedback-optimizeed parallel tempering monte carlo. Journal of Statistical Mechanics: Theory and Experiment, 2006 (03):,,3018, 2006.
Yann LeCun. The MNIST database of handwritten digits. http://yanm. lecun. com/exdb/nnist// 1998.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE,
1998.
Pankaj Mehta and David J. Schwab. An exact mapping between the Variational Renormalization Group and Deep Learning. arXiv:1410.38. [cond-mat, stat], October 2014. URL http: arXiv: 1410.3831.
10

---

Published as a conference paper at ICLR 2024
Leonid Mimy and Eugene Shakhnovich. Protein Folding Theory: From Lattice All-Atom Models. Annu. Rev. Biophys. Biomol. Struct., 30(1):311336, June 2001. ISSN 1056-8700. doi: 10.1146/
ammurew.biophys.30...300 URL https: www annual http://wwww.gg/////...11111 anturev. 30. .361
Publisher: Annual Reviews.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium
states of many-body systems with deep learning. Science, 365(6457) 2019.
Lars Onsager. Crystal statistics. i. two-dimensional model with an order-disorder transition. Phys.
Rev, 65:117-149, Feb 1944. doi: 10. 10.1103/hyyReev65..11 URL https:://ink.aap::
617
Edmund Rolls. The mechanisms for pattern completion and pattern separation in the hippocampus.
Frontiers in Systems Neuroscience, 7, 2013. ISSN 1662-5137 doi: 10.388///nns222133.00774 URL
https: / www. www.ooonnsssiioo wttp//tttggtge/eaareeoao.. .38.0.333333 3at.e//essse.....//////sssss.ss 2013 3.0070000
Clément Roussel, Simona Cocco, and Rémi Monasson. Barriers and dynamical paths in alternating
gibbs sampling of restricted boltzmann machines. Phys. Rev. E, 104:034109, Sep 2021. doi: 10. h0te://yyRReeE.10004441.11
URL www.nnk...s.ooo...........om...o org. ottp///////./ooooo///ooo/////o.. http://///.......oohee.eeeeeee
104. 034109
N. Le Roux and Y. Bengio. Representational Power of Restricted Boltzmann Machines and Deep Belief Networks. Neural Computation, 2016):1631-1449 June 2008. ISSN 0899-7667. doi: 10.1 wel11662eco.200..co 10008804-07-1111 Conference Name: Neural Computation.
Ruslan Salakhutdinov and lain Murray, On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning, ICML '08. pp. 872-879, New York, NY, USA, July 2008. Association for Computing Machinery, ISBN 978-1-60558-205-- 205-4. 16.99999666..1
doi: 10.1145/1390155 56. 1390266.
Eugene Shakhnovich and Alexander Gutin. Enumeration of all compact conformations of copolymers with random sequence of links. J. Chem. Phys.. 93(8):5977-577,, October 1990. ISSN 0021- 9606. doi: 10.
URL http://ppppcaaaaa.oo.ooorre///./
459480 Publisher: American Institute of Physies.
Robert H. Swendsen and Jian-Sheng Wang. Replica monte carlo simulation of spin-glasses. Phys.
Rev. Lett., 57:2607-2609, Nov 1986, doi: 10. 10.1000/hyy URL htthht:t/iiin
aps . 10g..oog//611/10.1110/hheeeeeeee
57.2607
Robert H. Swendsen and Jian- Wang. Nonuniversal critical dynamics in monte carlo simulations. htth::/::///// //link
Phys. Rev. Lett. 58:86888 Jan 1987. doi: 10.1 10.1103/hy.Reeeeett.88.8. 58.888.88 86
aps wpw.porg..a.......p..pp..p.....p
Tijmen Tieleman. Training restricted Boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learning, ICML '08, pp. 1064-1071, New York, NY, USA, July 2008. Association for Computing Machinery. ISBN 978-1-60558-205-4 doi: (0.1445/1391550 1390290. URL URL UUtp 1390290
Ulli Wolff. Collective monte carlo updating for spin systems. Phys. Rev. Lett., 62:361-364, Jan 1989. doi: 10. 103/PhysSevLeett.o
Dian Wu, Lei Wang, and Pan Zhang. Solving statistical mechanics using variational autoregressive networks. Physical review letters, 122(6):080602, 2019.
11


Reviews:
[{"review_id": "dpXiZ3yoGX", "reviewer": "Reviewer_4ET5", "summary": "The paper proposes a new sampling scheme for RBMs that utilizes a hierarchy of RBMs of decreasing latent and visible dimensions to encourage learning a smoother version of the data distribution that captures more global aspects of the distribution in a manner akin to temperature annealing in parallel tempering. Their training approach involves sampling the latent variables at a given layer using alternating (between the latent and visible variables) Gibbs sampling (AGS) and then using the latent variable samples as training data for the next layer up in the hierarchy. Sampling involves using AGS to generate latent variables in a layer and then swapping the latent variables of one layer with the visible variables of the next one up using an acceptance probability. Their experiments on MNIST, a protein folding problem and the 2-D Ising model demonstrate how their approach can sample modes much quicker than AGS and parallel tempering. They also give a theoretical analysis in the overparameterized regime showing how different settings of the hyperparameters (regularization weight and ratio of hidden-to-visible variables) induces different representational regimes of the model. They also perform an analysis of the mixing time of their scheme indicating how it matches empirical data", "questions": "- No questions that need clarifying", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The main strengths of the paper are the results which indicate their approach's superior sampling performance compared to other reasonable baselines of parallel tempering and AGS. \n- The explanation of their approach is clear to understand and is given succinctly.\n- The theoretical analysis gives good insight and interpretation of the results of their approach as well as the consequences of different settings of the hyperparameters. In addition, the fact that the relations established for the mixing times (as in Figure 5) match the empirical results adds confidence in the correctness of their analysis", "weaknesses": "- As their approach is intended to speed up sampling of RBMs a figure/table demonstrating how the real world sampling time is approved compared to the parallel tempering and AGS baselines would give the reader a better sense of how this method fares practically.\n- Adding figures similar to Fig 3(d-h) for the Ising model results in the main text would give the reader a better understanding of the results on this problem\n- A conceptual comparison to deep Boltzmann machines would be a great addition as it would make clear how their approach differs from sampling schemes for deep Boltzmann machines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pYLRk3m8HR", "reviewer": "Reviewer_fKHh", "summary": "This paper introduces a method called Stacked Tempering (ST) that applies the ideas of deep tempering to restricted Boltzmann machines (RBM). The authors propose to learn stacks of nested RBMs, where the representations of one RBM are used as \"data\" for the next one in the stack. By exchanging configurations between RBMs, the ST method allows for fast transitions between different modes of the data distribution. The paper provides analytical calculations of mixing times and demonstrates the efficiency of the ST method compared to standard Monte Carlo methods on several datasets.", "questions": "- Is it possible to extend this method for sampling from other energy-based models (e.g., deep Boltzmann machines)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper introduces a new approach called Stacked Tempering (ST) for sampling from Restricted Boltzmann Machines (RBMs).\n  - ST learns nested RBM stacks by using the representation of one RBM as the \"data\" for the next RBM.\n- Efficiency of the ST method is demonstrated through experiments on multiple datasets including MNIST, in-silico Lattice Proteins, and 2D-Ising model.\n- This paper provides the first theoretical analysis supporting the use of deep representations for improving mixing in RBMs, inspired by previous research on deep tempering.\n  - Obtained analytical results are interesting.", "weaknesses": "- Learning nested RBMs seems costly compared to simple parallel tempering approach where additional models are not required.\n- This method is only applicable to RBMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HoJOc7c9X2", "reviewer": "Reviewer_mREg", "summary": "The manuscript introduces a stacked RBM parallel tempering training, in which sampling is improved by swapping configurations of hidden and visible units in the vertical stack.", "questions": "Can the authors show the autocorrelation times of some observable (for example, on the Ising model, the magnetization at the critical point) ? It is hard to understand, from the proposed plots, whether the scheme introduced is faster than standard PT \n\nThe simplified theoretical analysis of mixing and swapping times is very interesting, but as far as I can see there is no direct comparison to simulation data. It would be desirable to have such a comparison, in order to understand whether the qualitative aspects of this analysis carry on to the more complex algorithm", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper introduces an interesting idea, that allows to sample more efficiently from RBMs, without the need to specify effective temperatures, as in Parallel Tempering", "weaknesses": "The numerical experiments do not allow to understand the improvement over more standard Parallel Tempering", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)