Learning Interpolations between Boltzmann Densities
Authors: Bálint Máté
François Fleuret
Venue: Accepted by TMLR ()

Abstract: We introduce a training objective for  continuous normalizing flows that can be used in the absence of samples but in the presence of an energy function. Our method relies on either a prescribed or a learnt  interpolation $f_t$ of energy functions between the target energy $f_1$ and the energy function of a generalized Gaussian  $f_0(x) = ||x/\sigma||_p^p$. The interpolation of energy functions induces an interpolation of Boltzmann densities $p_t \propto e^{-f_t}$ and we aim to find a time-dependent vector field $V_t$ that transports samples along the family $p_t$ of densities. The condition of transporting samples along the family $p_t$ is equivalent to satisfying the continuity equation with $V_t$ and $p_t = Z_t^{-1}e^{-f_t}$. Consequently, we optimize $V_t$ and $f_t$ to satisfy this partial differential equation. We experimentally compare the proposed training objective to the reverse KL-divergence on Gaussian mixtures and on the Boltzmann density of a quantum mechanical particle in a double-well potential.

Published in Transactions on Machine Learning Research (05/2023)
Learning Interpolations between Boltzmann Densities
Bälint Máté
balini. mote Seltiid.mtteuuuiigeeg Sunvige.
Deportment of Computer Science, Department of Physics University of Geneva
François Fleuret
flere
Department of Computer Science University of Geneva
Reviewed on OpenReview: https: // openreview. net/ formfo id- TH6YrEcbth
Abstract
We introduce training objective for continuous normalizing flows that can be used in the absence of samples but in the presence of energy function. Our method relies on either prescribed or learnt interpolation fe of energy functions between the target energy fi and the energy function of a generalized Gaussian fo(x) = ||///llp- The interpolation of energy functions induces an interpolation of Boltzmann densities Pt e and we aim to find a time-dependent vector field Vú that transports samples along the family Pt of densities. The condition of transporting samples along the family Pt is equivalent to satisfying the continuity equation with V2 and pt = Z, Z. Consequently, we optimize V2 and fa to satisfy this partial differential equation. We experimentally compare the proposed training objective the reverse KL-divergence on Gaussian mixtures and on the Boltzmann density of a quantum mechanical particle in double-well potential.
1 Introduction
We consider the task of estimating the expectation value (0()) of some observable O, under a proba- bility density p proportional to the unnormalized density e-1, where f f R2 IR is given energy function. In particular, we don't have access to true samples from p, all we have is the ability to evaluate f and its
derivatives for any 2 E R", A popular technique (Boyda et al. 2021 Albergo et al. 2021a a b 2022 Abbott
et a.. 2022 de Haan et al. 2021 Gerdes et al. 2022 Noé et al. 2018 Köhler et al. 2020 Nicoli et al.
2020 2021 for attacking this problem is to use normalizing flow to parametrize a variational density do and optimize the parameters 0 minimize the reverse KT-divergence
K KTlge.r] v] Eg (log qe (a:) l())())) EE Eg (log (x() f(x)) 2.
(1)
The use of normalizing flows for this problem is particularly attractive because qe be used as a proposal for importance sampling, [=()] Eg ( [x)]] to account for the inaccuracies of qo- Unfortunately, the reverse KT.-divergence is mode-seeking, making the training prone to mode-collapse (Fig. 2 To tackle this problem, several works have proposed alternative training objectives for normalizng flows. Vaitl et al.
(2022) introduce better estirnttors of the forward and reverse KL divergences, while Midgley et. al. (2022) use the a divergence instead of the reverse KL-divergence as their training objective. We propose yet another alternative based infinitesimal deformations of Boltzmann densitics (Pfau & Rezende, 2020
Mátó
& Fleuret 2022 This work was motivated by denoising diffuusio (Sohl-Dickslein el al. 2015 Ho el al. 2020
and score-base models (Song & Ermon 2019). Tt. also bears similarities to more recent works (Lipman et al.
2022 Albergo & Handen-Eijnded 2022 Liu et al.) 2022 Neklyudov et al. 2022) that generalize diffusion
models by relying on more flexible interpolations between the data and the base distribution.
The implementation of our experiments available https://gšthub. hasm/aalintatte/bbone hat.ssittttttttee.aammmaaaaiiitt

---

Published in Transactions on Machine Learning Research (05/2023)
The contributions of this work can be summarized follows
In $ we describe our method which relies on either a prescribed or a learnt interpolation ft of energy functions between the target energy fi and the energy function of a generalized Gaussian fo(=) ||z/a||. Given ft we optimize a vector field Vi transport samples along the family pe( x e fx(xx) of Boltzmann densities. After translating this condition to a PDE between V, and ft we propose to minimize the amount by which this PDE fails to hold.
First. find that in certain cases the linear interpolation improved performance over optimizing the reverse KL-divergence. We also show that in general this interpolation is insufficient.
f) if1 already leads to
Motivated by the failure mode of the linear interpolation, parametrize the interpolation with another neural network ft- (1 - fo thi + t(1 - -)) (1) and optimize Sf together with the vector field V.""
In g we run experiments on Gaussian mixtures and on the Boltzmann density of a quantum particle in a couble-well potential, and report improvements in KL-divergence, effective sample size, mode coverage and also training speed.
log Z
Figure 1: Interpolating in the space of unnormal- ized probability densities. The unnormalized target density (red) and its normalization (black) that we are trying to a flow to. The base density of the flow (green) lying on the subspace of normalized probability densities (orange). The interpolation connecting the base density to the unnormalized target (blue) and the induced interpolation of nor- malized densities (the projection of the blue trajec- tory to the "log Z = 0"'plane).
-
1211
f1
fo /Zo
log
Motivation
Consider the following multimodal density
( -8].1)+ 1) ([s -8].1) ( 8).1)).
(2)
where ((p, σ) denotes a normal density centered at with covariance matrix diag(o?). Fig. 2 shows the mode-collapse of a normalizing flow trained with the reverse KT-divergence on this target. The reason why mode collapse can happen the first place is the training objective itself. The mode seeking behavior of the reverse KL-divergence can be explained follows. As the sampling done according qo. the difference in ligliiiklssooss is weighted by the likelihood qa- This implies that if go completely ignores modes of p the log-likelihoods of p and qo not compared over regions that not covered by qo- In this paper we introduce a training objective for continuous normalizing flows that can be used to replace the reverse KL-divergence and investigate to what extent it solves the issue of mode collapse on multimodal targets.
t 0.00
0.17
0.33
0.50
0.67
t 0.83
t= 1.00
Modo-sccking nature of the reverse KL-divergence. The figures from left to right show how the latent gaussian is transformed by the continuous normalizing flow trained with the reverse KL objective. The blue blobs denote the target density (2) the arrows represent the vector field V,.
Figure 2:
2

---

Published in Transactions on Machine Learning Research (05/2023)
2 Background
Change of variables. Let pu(2) be probability density IR" and v IR" - R diffcomorphism. Pushing the densily po forward along y induces new probability densily p implicitly defined by
lo) p())) + log det dy
The term log det Jo (z)) measures how much the function vp expands volume locally at 2.
Continuous change of variables. Let now Vi be time-dependent vector field and v denote the diffeomorphism of flowing along the integral curves of V, from 0 to т. This family of diffeomorphisms generates a one- aue-parmette family of densities py. The amount of volume expansion a particle experiences along a trajectory 4, between and fo V VV))..) The log-likelihoods are then related by
log = logg-((M))) (V+2) + ! V V.(V,z)dt.
(4)
Rezende & Mohamed, 2015 Dinh et al. 2016) (continuous normalizing flows (Chen et al. 2018)) parametrize a subset of the space of all densities on R". They do this by first fixing base density Po and using a neural network that parametrizes the transformation (the vector field Vi). The (continuous) change of variables formula is then applied to compute the density induced by V (V)). Recently, generalizations of normalizing flows have also been studied (Nielsen et al. 2020 Huang al. 2020 Máté al. 2022).
Normalizing flows. Normalizing flows Tabaa & Turner 2013
Boltzmann densities. Let f RR - be an energy function with a finite normalizing constant Z f de=)( The function then induces a Boltzmann density over the configurations E R", p(x) ze ze-1(x) Conversely, given probability density function p: IR" R,,o the corresponding energy function can be recovered up to constant f logp log Z.
The continuity equation. One can describe a time-dependent probability density either with its time dependent density function pt or by reference (base) density Po and a time-dependent vector field V. The latter generates Pr by pushing po forward along the integral curves of V, from ( = to ( = The two descriptions are related by the continuity equation d.p: + V (PrV)) = 0, which, in the case of Boltzmann densitics pt- = Z, -- can be rewritten as follows,
0 = Sipr + V (p.Vi)
expanding Pt 2,- -
(5)
a, e --)) v V. 1Z
(6)
e -+00( a 7, (V))
(7)
e- -10,12 1) 2, 0,(e-))) 1/-. V/2 v V2 (Ve Vi)
(8)
e 12, 1
Z.le-Mv -V.V.- 1/- 1---le- f: (V.. Vi) factoring out pt c--
(9)
= Z, ee- Z,0,(Z,)) - 0.f. V. (VV..(V
(10)
-8: log Ze
where ) is the Euclidean scalar product between vectors. Since pt
l ft Z,-e
conclude
0.f. + (V Vi) V2 + log Z, = 0,
(11)
Lemma 1. Moreover, if any time-dependent energy function ft, time-dependent vector field V2 and spatially constant function C₁ salisfies
0.f. + (V Vz) V V.V4
(12)
then Ct necessarily equals O, log Z, with Z, f -r
3

---

Published in Transactions on Machine Learning Research (05/2023)
Proof. Let ft, Vt, Ce be such that they satisfy 12
We can then compute
Vi(e (a)
8,4, [(-O.ft)--hlddd
(V Vi) V.Vi)e fel)) d' + Je de-Selo)dr Ct (13)
I Iee di
0 log 74 =
Zz fe-fileddr
f f d=
where the last equality follows from the divergence theorem and the fact that e f((x) vanishes at infinity.
When numerically solving the continuity equation, Lemma enables to work with 12 instead of (11 and parametrize the triplet (f, V)C) with C' being a spatially constant function instead of estimating 0 log Zi
In what follows we only work with 12 and refer to it as the continuity equation.
3 Approximating the transport field
From here on, we will use the vector field Vi and the term "continuous normalizing flow" interchangeably. Our goal is to sample from target Boltzmann density Ptarget X e efarree by
defining a family energy functions ft, 0 S S interpolating between the larget energy fi Starget and the energy function of generalized Gaussian fo(x) |
2. finding a transport field such that (fa, V.) "solves" the continuity equation 12
If we succeed at both of these constructions, then we can obtain samples from Ptarget by sampling from Po α e ella//all; and let the samples follow the integral curves of V, from & = 0 to t = 1. Note that we turned the problem of learning a single density - into a continuous collection of problems, learning all the densities e fr for & S 1. It might seemn like that we just made the task more difficult, but the idea is that for any 0 S r 1 1, the density e e-f- is casier fit once f- alrcady fitted for some small e.
The pointwise continuity error
Regarding the second item of the above list, an analytical expression for V2 is not easy to find if we are given family of energy functions fr. This would amount to solving [12 which is difficult in general. Therefore will parametrize Vi with neural network and train it to minimize the amount by which the pair (fr, Vi) fails to satisfy the continuity equation. We begin by recalling the continuity equation,
0.f² + (V VA)
+,
(14)
where C is spatially constant function. In what follows, V. and C% parametrized by neural networks and are trained to minimize some monotonically incrcasing function 1 of the pointwise continuity error
E, = [0.f(()) + (V)) VV) (²)) - V V,((x) + C.).
(15)
The expression (( measures the incompatibility of f. and V at single (t,z) pair of coordinates, we will need to optimize some sort of integral of this pointwise error over both and
The continuity loss
Suppose that we have an interpolation of energy functions f.. We propose to train V and C to minimize the continuity error 11 along the trajectories of V. Formally, let qe be a parametric density parametrized by a continuous normalizing flow vo. We update the parameters to minimalize the integral of L(E) along the trajectories of the flow,
[()) Ez-po V/ 1 (Ear(te))) d
(16)
where (²(z) is given by transporting along the vector field V. between 0 and 7. We evaluate the integral 16p by discretizing time and using numerical ODE solvers.
In our experiments we tried L. R + (8 3,E E7, E E2 log(1 +8)).

---

Published in Transactions on Machine Learning Research (05/2023)
First approach: Linear interpolation
Arguably the simplest way to interpolate between a pair of functions (Jo, fi) is to set
(x(x) (1 ()f(() + (f)
O
(17)
The same interpolation is proposed by Wu et al. (2020) in the context of stochastic normalizing flows and is also common choice for annealed importance sampling (Neal 1998p. We will call the family (17) the linear interpolation. Fig. shows the evolution of the samples along a continuous normalizing flow trained with the continuity loss using the lincar interpolation. We observe that the same network trained with the continuity loss instead of the reverse KL-divergence can capture all modes of
t= 0.00
t=0.17 0.17
t= 0.33
t 0.50
t= 0.67
t t= 0.83
t= 1.00
Figure 3: The same continuous normalizing flow in Fig. 2 trained with the continuity loss using the linear interpolation. The top row shows how the larget density evolves under the predefined linear interpolation between fo x² /2 and log(EE. 2 while the bottom row shows how the samples from qe evolve along Vi as t is varied.
The issuc with the linear interpolation. Let us consider the density
V ([4 4.11) + ( .1)
(18)
This target does not enjoy the symmetry properties of the previous mixture, one of the modes is closer to the base and has lower relative weight than the other one. The top row of Fig. 4shows the linear interpolation to this target and the second row shows that this interpolation is insufficient to capture the mode which is further away. In a nutshell, the reason for this is that the linear interpolation does not preserve the relative weights of the modes as & is varied.
t= 0.00
t 0.17
t= 0.33
t= 0.50
t= 0.67
f t= 0.83
t= 1.00
Figure 4: The issue with the linear interpolation. The top row shows how the target density evolves under the predefined linear interpolation between fo a²2/8 and fi log(Fq. 18/, while the bottom shows how the samples from go evolve along V4 is varied.

---

Published in Transactions on Machine Learning Research (05/2023)
Interlude: good and bad interpolations
The reason why the linear interpolation can lead to problems is illustrated in Figures 5 andd G Figure 5 shows particular example where the linear interpolation in log-density space induces a "good" interpolation in density space. Figure O shows that for certain target functions the lincar interpolation log-density space induces an interpolation density space that is not "local" in the sense that probability mass gets moved between the modes. Such densities are difficult learn with the linear interpolation, even if in principle there exists vector field V2 that moves probability mass the correct way along any interpolation.
t == 0.00
t= 0.14
t= 0.29
t 0.43
t t= 0.57
t= 0.71
t= 0.86
t = 1.00
[1(xr
-0.-
-0,2
Figure 5: Linear interpolation between fo(2) 22/4 and fi = log(5e (x-3)² + +ffe-(++3)) (x+3)²), Linear interpo- lation in the log-density space (top row), and the induced interpolation in density space (middle row). The transport field V/2 which can be represented as a scalar-valued function for one-dimensional densities (bottom row). The thin gray line the bottom row denotes V = 0.
t == 0.00
t= 0.14
t= 0.29
t 0.43
0.57
t= 0.71
t= 0.86
t 1.00
-10
-50
-5.0
200
-500
Figure 6: Linear interpolation between fo(z) 2//4 and f1 = - Le (x-1)² + + (x+4))))) Linear interpo- lation in the log-density space (top row), and the induced interpolation in density space (middle row). The transport field Vú which can be represented as a rcalar-valucd function for one-dimensional densities (bottom row). The thin gray line in the bottom row denotes V, 0. Note how the relative weights of modes is not preserved f is varied, resulting in the exploding norm of V..

---

Published in Transactions on Machine Learning Research (05/2023)
Sanity check: interpolate along the diffusion process
Ho et al. 2020 learn to reverse a diffusion process that is in essence also a family probability densities p. on I" interpolating between the data density Pı (z) and the latent R-diimenionnal gaussian po = (0, Samples from Ps are of the form Vtx v1- where I ~ p1, Po- Gcometrically, the probability density Pt is obtained by first stretching pt by factor of vt and then convolving with the Gaussian kernel govi-
Diffusion Processes. Diffusion models (Sohl-Dickstein et al.
2015
d to nn pi (t 1/2x) G. +t P1 (t 1/2x)) (t (1(x)
G
(19)
P((#)
where we used X to denote the stretching by a factor of vt and Ge to denote the convolution with the Gaussian kernel govi-t. If = 1, both d (strotching by a factor of 1) and G. (convolving with a Dirac- delta) are just the identity. If = 0, then d collapses Pi to a Dirac-delta it with ((0, o) implying that Po o)) The attractive thing about the diffusion process is that it provides an interpolation between densities where the situation of Fig. is avoided. Loosely speaking, this not.
the orgin and G, convolves
interpolation is "local" in sense that that the linear interpolation
The family of Gaussian mixtures is closed under the diffusion process 19 we can even explicitly compute the time-evolution of Gaussian mixture and use to replace the linear interpolation. Fig. shows samples from a normalizing flow that was trained with the continuity loss using the hand-computed interpolation of diffusion process.
t= 0.00
t 0.17
= 0.33
f 0.50
t= 0.67
f 0.83
t 1.00
Figure 7: The same continuous normalizing flow as in Fig. with the continuity loss using the diffusion interpolation. The blue blobs denote the target density 18 a mixture of two Gaussians. The top row shows how the larget densily evolves under the predefined diffusion-like interpolation, while the bottom row shows how the samples from qo evolve along the flow varied.
Unfortunately, since all we given is the ability to evaluate the target energy pointwise, there is analytical formula for calculating the difffusion process of an arbitrary target density. Moreover, any numerical approach involves integration, that gets increasingly expensive as f gets smaller and the dimensionality of the problem gets larger. Nonetheless, this result demonstrates that minimizing the continuity error is a feasible approach, one just needs to be more careful about the interpolation between the latent and target energy functions. The authors could not find a prodefined interpolation that is 1) "local" in the still not formal, but intuitive sense and 2) easy to compute for arbitrary target energies. Instead, we propose to learn the family of densities between the base and the target.
2The Gaussian kernel G Gaussian density with mean and covariance matrix d 1))-

---

Published in Transactions on Machine Learning Research (05/2023)
Parametrizing the interpolation
We propose to neural network to parametrize the interpolation St as
f((z) (1 t)f(()) + tz)
t(1 (x),
0
(20)
where ff is parametrized by neural network. This parametrization ensures that the boundary conditions at E {0,1} are satisfied, and allows for flexibility the open time-interval (0, 1). The parameters of
the interpolation are trained together with the those of the flow (and those of Cτ) with the objective of
minimizing the continuity loss. Fig. g shows that this flexibility allows flow trained with the continuity loss to capture both modes of the distribution
18
t=0.00
t= 0.17
t= 0.33
t= 0.50
t= 0.67
t= 0.83
t= 1.00
Figure 8: The same continuous normalizing flow as in Fig. trained with the continuity loss using the trainable interpolation. The blue blobs denote the targel density (18), mixture of 2 Gaussians. The top row shows how the target density evolves under the learned interpolation, while the bottom row shows how the samples from qo evolve along Vi as t varied.
4 Experiments
To compare the continuity loss to the reverse KT-divergence train the same normalizing flow architecture by minimizing the 1) reverse KL objective and 2) the continuity loss. We run experiments on Gaussian mixtures and on the Boltzmann distribution of a quantum mechanical particle in a double-well potential.
Performance metrics
To quantify the results of the experiments, for each model we report a subset of the following metrics. For all runs we report the reverse KL-divergence (minus log 2),
E En (log (x) log p(x))
(21)
and the effective sample size,
ESS, = + ((z(a)//o(zz)) [(((a:)//noxx)))>
90.
(22)
where N is the number of samples. These metrics capture how good fit qo is for but only in those regions where samples available. They are therefore insensitive to mode collapse. To compensate for this, we compute the Hausdorff distance between the means of the modes M = {mi...,,m, m,) and batch of samples [,1,,.,,, from the model,
H HM, max min #EX
[|m-x|] 2113
(23)
8

---

Published in Transactions on Machine Learning Research (05/2023)
In the case of Gaussian mixtures, the means M are the means of mixture components whereas in the case of the quantum mechanical particle, the means are (sh1, $2, ΦN) = (a, a,...,, a) and (2,,,,,..., ΦN) = (b, b,...... ..... b) where a and b are the two local minima of (Φ)) (see $4.2 for details). The Hausdorff distance is a good metric for measuring mode coverage but is insensitive to the shape of the distributions. Finally, the forward KI-divergence (plus log 2),
Expp(logp(z) log 90 (x))
(24)
and effective sample size,
ESS, C. 20(x:)))) (0((2;)//(<;))
24 ~ P.
(25)
N (x)(( //p (x:))
provide the most accurate (both mode coverage and shape matching) description of the goodness of the fit. As they require samples from we only report them for the experiments on Gaussian mixtures.
4.1 Gaussian mixtures
In this section we consider two targets, those given by correspond well to what we observe in Figures 2
2 4 and
and
18
The metrics are reported in Table I and
g
Table I: Results of training the same flow architect with different objectives on the targets 2 and 18 Note that the target densities are normalized, i.e. log Z = 0. Mean and standard deviation values over 5 seeds are reported.
Energy function given by log(Eq 2
[[M, X) 4
Rev. KL 4
Rev.
ESS
Fw +
Fw. ESS 1
K pg). Cont. Loss with Linear Int. Cont. Loss with Trainable Int.
19.26 + .333
1.387 = .001
0.999 + .000
110.6 + 97.6
0.254 + .006 0006
0.060 .023
0.003 = .002
0.998
.001
0.001 + .001
0.998 .002
0.053 + .010
0.000 = .001
1.000 + .000
0.000 + .000
0.999 = .000
Energy function given by log [Eq 18
[[M, X) 4
Rev. KT.
Rev.
ESS
Fw.KL 4
Fw. ESS t
KT (T(go.p)
13.25 + .363
1.099 + .000
1.000
.000
74.38 + 30.3
0.332 + .007
Cont. Loss with Linear Int.
2.329 .124
1.081 = .004
0.928 + .880
37.68 6.99
0.336 = .008
Cont. Loss with Trainable Int.
0.045 + .022
0.001 = .001 0.999 .000
0.000 + .000
1.000 = .000
4.2 Quantum mechanical particle in double-well potential
In this section consider the trajectory of quantum mechanical particle moving in one-dimensional double-well potential V between 0 and T. We closely follow the experimental setup of Vaitl et al. 2202). The action associated to continous trajectory @(t) E R" 18m,0= t reads
S(ø(t)] = '- 2 V())dt,
V = 0
- 4
3
(26)
where \ are numerical parameters. After distretizing time, the discretised action of trajectory (N,))) is
Φ5-1 Φi - + V(o.)) AT,
2
S(ó1. .... 2N) = Σ
V(ő)= = - 0²
(27)
AT
where AT = T/N, m and the subscript + is to be understood modulo N. We replicate the choices of Vaitl et al. (2022) for T = 4, 1, {4,8, 16, 32, 64) and we use mass values m {1.50, 3.00, 4.50, 6.00). The goal is then, as before, sample trajectories (ø1, ...,N)) .N) from the Boltzmann density
1 5(01.....
2 = / e -(( (x).
p(1, ..., (N) = 7.
(28)
RN
9

---

Published in Transactions on Machine Learning Research (05/2023)
Heuristics. For larger values of m, the energy barrier between the wells of V gets greater, resulting in a bimodal, one-dimensional Boltzmann density e v()). (Fig. 9p. Intuitively, the second summand in 27 encourages the particle follow the unnor-
m 1.50
m 3,00
m
4.50
2 6.00
malized density -V(() at every time step, while the first one penalizes if the values of Φ consecutive time steps differ too much (ie. belong to different modes of e-V())). With the above interpretation we can argue that Ø; and 0i+1 are likely to be close, and every de is likely to belong to one of the modes of e-((oi) Then the density e -(xx)... has two modes, centered (a, a,..,, a) and (b, b,.,,)),, where and the local minima of V.
m - 20
Sensitivity to the mass parameter. Now we fix N 16 and vary the of particle E (1.50, 4.50, 6.00). We compare the contimuity loss with the trainable interpolation to the reverse KL objective. The quantitative results are summa- rized in Table In Figure 10 we compare e vV() to the histogram of flattened samples from the trained models. This makes sense since the action encourages the particle to follow the one-dimensional Boltzmann density of the potential (()) at every time slep. Note that these two densities are not supposed perfectly match, and Fig. 10 can only be used to detect mode-collapse of the flow.
Figure 9: Dependence of (Ø)) and of the Boltzmann
density e- V(d) on m.
m 1.50
= 3.00
= 4.50
= 6.00
L p)
Cont. Loss
Figure 10: Mode collapse of the reverse KL-divergence for higher values of m. The unnormalized density p()) x (o() (blue curve), compared to flattened samples di (orange histogram).
Table 2: Results of training the same flow architecture with different objectives the energy function (27h. Note that decrease of log x 0.7 in reverse KI-divergence corresponds to covering twice as much probability mass. Mean and standard deviation values over 3 seeds are reported.
m 1.50
3.00
H(M, X) J
Rev. KL, J
Rev. ESS
H(M,X)
Rev. KT. +
Rev. ESS
Kg(go.p)
0.717 = .06
-1.522 + .00
0.951 + .00
0.541 + .00
-18832 + .01
0.801 +: .11
Continuity Loss
0.678 + .02
-1.054 + .00
0.991 + .00
0.486 + .04
-1.835 + .00
0.979 + .00
m = 4.50
m 6.00
H(M, X) J
Rev. KL +
Rev. ESS
H(M,X) 4
Rev. KL $
Rev. ESS
KL(go.p)
13.61 - .49
-9.058 + .00
0.883 + .05
16.67 + .00
-22.49 + .01
0.935 + .01
Coutinuity Loss
0.335
-9765 =
0.966 .01
0.300 .00
-23.19
0.963 .01
10

---

Published in Transactions on Machine Learning Research (05/2023)
Sensitivity to the dimensionality and computational speedup. In this section fix relatively small mass value, m -1.50, resulting in unimodal Boltzmann density. We then train models at different time resolutions, N E 8, 16, 32, 64). For each N we trained two models, one with the reverse KL and one with the continuity loss. Figure 11 shows the cvolution of the (reverse) effective sample size and the continuity loss during the training process. Since the continuity loss a pointwise objective, contrary the reverse KI, it does not require backpropagation along the trajectories of the flow. On the other hand, the "baseline" reverse KL objective only requires parametrization of Vi and the computation of V Ve- In addition to this, the contimuity loss also requires to parametrizations of Ct and ft and the computation of Ö.S, ff and Cy. Overall, we observed speedup when switching to the continuity loss both in terms of number of optimization steps per unit time and also in terms of convergence speed (Figure
Nac
No12
Camitzitty Looe
3.6-
NinSi
0.4
X
X-64
10as
14
100.
510
10P
10P
training time
training stops
Figure 11: Evolution of the (reverse) effective sample size (left) and of the contimuity loss (right) during training for different dimensionality of the problem. The solid lines correspond to the models trained by optimizing the continuity loss, while the dashed lines denote the models trained by optimizing the reverse KL divergence.
Implementation and training details
Architectures. The parametrization of V2 and fa are given by weighted average of 4 MLPs. The weighting is done by evenly spaced RBF time-kernels, one for each model. In the case of the Gaussian targets the MLPs has two hidden layers with 64 neurons per layer, in the case the quantum particle the MLPs has hidden layers with 128 neurons per layer. Between the linear layers we use swish nonlinearities. The parametrization of C, consists of a single MLP with the same hyperparameters as the mixture components of the parametrizations of V, and fe- Importantly, our architecture is completely oblivious to the Z, C.,- symmetry of the lattice of quantum mechanical particle and to the Zz-symmetry of the double-well potential. We rely on automatic diffcrentiation to compute all derivatives. We lcave the exploitation of symmetrics and the use of architectures with analytic expressions for the divergence of V2 (Köhler et al. 2020 Gerdes et al. 2022) as well as for Vt and Ö,ft, for future work.
Optimization. We train with batch size of 256 using the Adam optimizer (Kingma & Ba 2017 and evaluate on batches of size 4096. The trajectories of the flow are computed by a 4th-order Runge-Kutt Kutta solver with 50 integration steps. The N = 64 and N = 32 runs in $4.2 are trained for 2.5 x 10 and 10 iterations, respectively. All other models are trained for 10a iterations. The initial learning rate of x 10-3 is annealed to following a cosine schedule.
Base Density. We use standard Gaussian base for the target (2) centered Gaussian with standard deviation for the target 18) and the generalized Gaussian proportional to e-=" in the experiments of 4.2
On the choice of the function L. The definition of the the continuity loss (16) involves a somewhat arbitrary choice of the function L. We compared the functions {E {E-r, E,E E² E E E + log(1 and ended up using 8 in all of our experiments, as empirically outperformed the other choices. The intuition is that the 1 norm provides good gradients when the error is large, while the 7,1 norm provides
11

---

Published in Transactions on Machine Learning Research (05/2023)
good gradients when the error is small and therefore their sum is expected to be superior to both of them. Our experiments support this intuition.
5 Summary and Closing remarks
We introduced an alternative training objective of continuous normalizing flows that uses an interpolation of energy functions. We've demonstrated empirically that the proposed objective is less prone to mode collapse than the KI-ddivegeecce when the target density has multiple modes and is computationally more efficient.
Two reasons for mode collapse. There can be, at least, two different. reasons for mode collapse when training with the reverse KL-divergence, First, if one of the modes so far away from the base distribution that it never gets visited by the flow. Second, if mode is visited during training by trajectories of the flow, but the flow still ignores it and fits the remaining modes of the density. It is important to distinguish these two scenarios as our proposed technique can help with the second kind, but not the first one. This is also the reason why the choice of the base density is important. When the base has a higher variance, a larger fraction of the space gets visited.
Reframing the method as a PINN. Our work naturally fits into the framework of Physics-informed neural networks (Raissi et al. 2019). What this work calls the pointwise continuity error, would be called the residual to the contimuity equation in the PINN literature. The core idea is essentially the same: the optmization of neural network satisfy a PDE.
6 Acknowledgement
The authors acknowledge support from the Swiss National Science Foundation under grant number CR- SII5 193716 "Robust Deep Density Models for High-Energy Particle Physics and Solar Flare Analysis (RODEM)". We also thank Samuel Klein, Jonas Köhler and Eloi Alonso for discussions, Saviz Mowlavi for mentioning the connection to PINNs, and Ricky T. Q. Chen for pointing out that 12) is just the continuity equation.
We would also like to express our appreciation of the Python ecosystem (Van Rossum & Drake Jr 1995 and its libraries that made this work possible. In particular, the experimental part of the paper relies heavily JAX (Bradbury et al., 2018), haiku (Hennigan et al. 2020), optax (Babuschkin et al. 2020). numpy
Harris et al. (020). hydra Yadan 2019), matplotlib (Hunter 2007). jupyter (Pérez & Granger 2007 and
weights&biasos BBiewald 2020
References
Ryan Abbott, Michael S. Albergo, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Gurtoj Kanwar, Sébastien Racanière, Danilo J. Rezende, Fernando Romero-López, Phiala E. Shanahan, Betsy Tian, and Julian M.
Urban. Cauge-equivariant flow models for sampling in lattice field theories with pseudofermions, 2022.
URL https://arxiv.org/att 00840000000022220004555555
Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. ar preprint XrXio:2209.155: 2022.
Michael S. Albergo, Denis Boyda, Daniel C. Hackett, Gurtej Kanwar, Kyle Cranmer, Sébastien Racanière, Danilo Jimenez Rezende, and Phiala E. Shanahan. Introduction to normalizing flows for lattice field theory, 2021a. URL httts:://arxat wwwaaarwwww..aaaaaeeeraeeaaaeaee
Michael Albergo, Gurtej Kanwar, Sé bastien Racanière, Danilo J. Rezende, Julian M. Urban, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, and Phiala Shanahan. Flow-based sampling for fermionic lattice field theories. Physical Review D, 104(11), dec 2021b. doi: 10.1103/physrevd. 1tt.:/reedd....11.......... 104.114507 kttpss//dof/ org...rg////a 100...110.phhhyareedd..o 104. 104.1145017.
URL
12

---

Published in Transactions on Machine Learning Research (05/2023)
Michael S. Albergo, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Gurtej Kanwar, Sébastien Racanière, Danilo J. Rezende, Fernando Romero-Tápez, Phiala E. Shanahan, and Julian M. Urban. Flow-based sampling in the lattice schwinger model at criticality, 2022. URL
11712
Igor Babuschkin, Kate Baumli, Alison Bell, Surya. Bhupatiraju, Jake Bruce, Peter Buchlovsky, David Bud- den, Trevor Cai, Aidan Clark, Ivo Danihelka, Antoine Dedieu, Claudio Fantacci, Jonathan Godwin, Chris Jones, Ross Hemsley, Tom Hennigan, Matteo Hessel, Shaobo Hou, Steven Kaplurowski, Thomas Keck, Iurii Kemaev, Michael King, Markus Kunesch, Lena Martens, Hamza Merzic, Vladimir Mikulik, Tamara Nor- man, George Papamakarios, John Quan, Roman Ring, Francisco Ruiz, Alvaro Sanchez, Rosalia Schneider, Eren Sezener, Stephen Spencer, Srivatsan Srinivasan, Wojeiech Stokówiee, Luyu Wang, Guangyao Zhou, and Fabio Viola. The DeepMind JAX Ecosystem, 2020. URL bttt:///iitull wtw.g.......oeeeeeeeeeeeeeeeeeee deepmidd
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https: hwtp:///wwwww..n..bbmm com/ Soft- ware available from wandb.com.
Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S. Albergo, Kyle Cranmer, Daniel C. Hackett, and Phiala E. Shanahan. Sampling using SU(n) gauge equivariant flows. Phys. Rev. D, 103:074504, Apr 2021. doi: S.08//hyyRevD..iee doi/10. (1103//hyyRev) 103. 074504
URL htthht/ss///ink aps. org/
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: compos- able transformations of Python-NumPy programs, 2018. URL http://github. com com/google/jay
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations, 2018. URL htttps//aaxxxv.oog//a///10... 07366
Pim de Haan, Corrado Rainone, Miranda C. N. Chong, and Roberto Bondesan. Scaling up machine learning for quantum field theory with equivariant continuous flows, 2021. URL https://arxiv. worpiiiiirrr//eo abs/ 02673
Laurent Dinh, Jascha Sohl-Dickster Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605, 08800, 2016.
Mathis Gerdes, Pim de Haan, Corrado Rainone, Roberto Bondesan, and Miranda C. N. Cheng. Learning lattice quantum field theories with equivariant continuous flows, 2022. URL http:///xxxvvvo./aaaba
2207.00283
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Pi- cus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585(7825):357-3883 September 2020. doi: (0.1088//41186--00-2---22 URL https: 2649-2.
//doi.org/11 org/100 otg/10.11//0000
Tom Hennigan, Trevor Cai, Tamara Norman, and Tgor Babuschkin. Haiku: Sonnet for JAX, 2020. URL
http://github. com//depmmin//dn-haall
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Newral Information Processing Systems, 33:6840-6855 2020.
Chin-Wel Huang, Laurent Dinh, and Aaron Courville. Augmented normalizing flows: Bridging the gap between generative flows and latent variable models. arXiu preprint arXiv:2002.07101 2020.
J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3):90-95 95, 2007. doi: N0.1099MCCSF.2077.555
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
13

---

hhtp:/hyeeee.......1 Published in Transactions on Machine Learning Research (05/2023)
Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: Exact likelihood generative learning for symmetric densities, 2020. URL https://arxiv http:///wwwww..../aabaaaseeee 20////0s0000.... 02425
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamn, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02 02.0777, 2022.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv: 03000, 2022.
Bálint Máté and François Fleuret.
Deformations of boltzmann distributions.
ar preprint
0tl::22100117773 2022.
Bálint Máté, Samuel Klein, Tobias Golling, and François Fleuret. Flowification: Everything is a normalizing flow. arXiv preprint 2r05: 15209, 2022.
Laurence Illing Midgley, Vincent Stimper, Grogor NC Simm, Bernhard Schölkopf, and Josó Miguel Hernández-Lobato. Flow annealed importance sampling bootstrap. arXiv preprinl ar 2xxi:2200800189 2022.
Radford M. Neal. Annealed importance sampling, 1998. URL https: http:://aarxww.oorraab///aaaaaa. http://wwwww.bb///y///ss//////00
Kirill Neklyudov, Daniel Severo, and Alireza Makhzani. Action matching: A variational method for learning stochastic dynamics from samples. arXiv preprint arXi::2910.0666; 2022.
Kim A. Nicoli, Shinichi Nakajima, Nils Strodthoff, Wojciech Samek, Klaus-Rober Müller, and Pan Kessel. Asymptotically unbiased estimation of physical observables with neural samplers. Physical Review (2), feb 2020. doi:
URL httpp////dol..rg/1..10022222222/
101.
023304
Kim A Nicoli, Christopher J Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima, and Paolo Stornati. Estimation of thermodynamic observables in lattice field theories with deep generative models. Physicol review letters, 126(3):032001, 2021.
Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vacs and flows. Advances in Neural Information Processing Systems, 33:12685- 12696, 2020.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators Boltananngemeeratorss-- sampllin equilibrum states of many-body systems with deep learning, 2018. URL http:///araav.oorrr/a///1111.../
Fernando Pérez and Brian F. Granger. IPython: system for interactive scientific computing. Compuling in Science and Engineering, 9(3):21-20 29, May 2007. ISSN 1521-9615. doi: 10.1109 D..1109/MCSE.2007..33 URL
https::/ipython org
David Pfau and Danilo Rezende. Intograble nonparametric flows. arXio preprint arXiu:2012 020055, 2020.
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning frame- work for solving forward and inverse problems involving nonlincar partial diffcrential equations. Journal of Computational Physics, 378:686-707, 2019. ISSN 0021-9991. doi:
10.045. URL httppp///ww..cccaeeeeeeeere../es
Danilo Rezende and Shakir Mohamed, Variational inference with normalizing flows. In International con- ference on machine learning, pp. 1530-1538. PMLR, 2015.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference Machine Learning, pp. 2256 2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neurra information processing systems, 32, 2019.
14

---

Published in Transactions on Machine Learning Research (05/2023)
Esteban G Tabak and Cristina V Turner. A family of nonparametric density estimation algorithms. Com- municalions on Pure and Applied Mathemalics, 1664))145-164, 2013.
Lorenz Vaitl, Kim Nicoli, Shinichi Nakajima, and Pan Kessel. Gradients should stay on path: better estimators of the reverse- and forward kl divergence for normalizing flows. Machine Learning: Science
and Technology, 344):04500 oct 2022. doi: (0008//6322213///ca/a URL https::/dx..do..rrg//
1088/2632-2155/2c99555
Guido Van Rossum and Fred L Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.
Hao Wu, Jonas Köhler, and Frank Noé. Stochastic normalizing flows, 2020.
Omry Yadan. Hydra framework for elegantly configuring complex applications. Github, 2019. URL
https://githhb com//abcebookkresssrddhyyyyaaa
15


Reviews:
[{"review_id": "gmPVS5YWIJ", "reviewer": "Reviewer_Hzj2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The paper is well written and pleasant to read, I liked to see controlled experiments to strengthen the story before section 4.\n\n- My personal opinion is that the overall approach makes sense theoretically.\n\n- There are real improvements on the practical side. \n\n\nWeaknesses:\n\n- There is no theoretical justification/ proofs on why the proposed method should work. Maybe the authors could consider a toy model for the parametrization and explain the behavior of the algo in a simple case? Or, maybe write a paragraph to explain why intuitively the proposed method should not be mode seeking (for example an intuitive explanation of what we observe in Eq 3)?\n\n- There is no discussion on how the ODE is discretized once the velocity field is learned.\n\n- The path joining $p_0$ to $p_1$ seems arbitrary, isn't there a more canonical one? Even the path that is learned (Eq 13) does not seem canonical to me. \n\n- The abstract is not well written. Too many uses of the word \"this\", it is not always clear what it refers to. Besides, \"minimize the amount ...\" is not clear at this stage of the reading.\n\n- Related to the above, in the deformation loss, the choice of L and the weights is not discussed. \n\n\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method to sample from the target distribution proportional to $\\exp(-f_1)$, where $f_1$ is a real valued function defined over a vector space. \n\nThe method relies on first sampling from an easy distribution $\\exp(-f_0)$ (e.g., a Gaussian) and then transporting the samples to the target. \nTo transport the samples to the target, the authors propose to consider a path $f_t, t \\in [0,1]$ from $f_0$ to $f_t$ and the associated distribution $p_t \\propto \\exp(-f_t)$. Since $p_t$ joins the easy distribution $\\exp(-f_0)$ to the target $\\exp(-f_1)$, the method relies on learning a velocity field $v_t$ of $p_t$, then the samples can be transported by solving the differential equation $x'(t)= v_t(x(t))$.\n\nThe main contribution of the paper is experimental: they show that the proposed method addresses the mode collapse behavior of concurrent methods based on optimizing the reverse KL. The proposed method is therefore more suitable when $f_1$ is not convex. \n\n ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Address the weaknesses mentioned above. These are mainly clarifications.\n\n- Define velocity field formally. I also wanted to establish Eq (4) from the definition of velocity field but I could not. Could you explain why Eq 4 holds?\n\n- Before the summary of contributions, explain what the advantages of the method are.\n\n- Rewrite the abstract.\n\n- Explain why an approximation is needed in Eq (21).\n\nSome questions:\n\n- When I saw Eq (7), I wanted to take the gradient to get rid of $C_t$. This new equation could be taken as the PDE. Any thoughts?\n\n- Instead of learning $f_t$, could we learn $\\nabla f_t$? This vector is known as the score and there are efficient methods to learn the score that were successfully used in the field of diffusion models.\n\n\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eo2EfL3LWU", "reviewer": "Reviewer_vNnM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper gives a very clear motivation and introduction of fitting a normalise flow model to a unnormalised density for the purpose of sampling. The examples are simple to understand, but perhaps too toy-ish for machine learning community. The main contribution is sound and I think it's simple but effective. \n\nThe weakensses are mainly the demonstrated significance of the idea, and some clarify. \n* One assumption in this work is that the gradient of the energy is unavailable, but no examples have shown this application. Does such scenario arise in physics? If so, show one, otherwise just use diffusion. \n* The main application is the phi-4 theory, which is not presented very clearly.\n* Relation to existing literature: some issues discussed in this work are related to energy-based or score-based methods in machine learning, and I encourage the authors to at least acknowledge the literature in writing (e.g. Song and Ermon 2019, Wenliang & Kanagawa 2021).\n* There are some design choices regarding the loss function, and there is no experiments showing the effect of this choice.\n* It would be great if one can demonstrate an example on moderate dimensional densities. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses sampling from an unnormalised model by fitting a continuous-time projection/action that pushes sample from an initial ( generalised Gaussian) distribution to the target energy function. The main contribution is the particular interpolation scheme, which is a combination of the usual linear interpolation between simple and target energies, and a novel a transient additive component that helps transporting the samples across different time steps. \n\nI apologise that my review is brief as I have very limited time, but I have understood the paper and please bear with me if anything is unclear. ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Critical: \n* Address my first weakpoint and answer the question: why not using diffusion?\n* Above eqn 8, the functions in footnote are not monotonically increasing.\n* Page 6, first para.: The issue isn't purely optimizability, Langevin dynamics would also work, and initialisation also matters. \n* Page 6, I don't see the purpose of the sanity check section, apart from saying: we don't want to evaluate gradients so don't use diffusion. Please clarifty the point here. \n* Page 6 eqn 12. The symbols St and Ct are overloaded, please change them. \n* Experiments should include some score-based methods (e.g. SVGD by Liu & Wang 2016), unless the example target density is non-differentiable, a case the authors seem to focus on, even if the current approach underperforms than this method. \n\n## Good to have\n1. Performance metrics should include maximum mean discrepancy (Gretton et al, 2012)\n2. Section 4.2 needs to be made friendly to more general machine learning audience, as well as an earlier mention of phi4 theory. If (19) isn't used later for experiments, don't write it down, just present (21). The intermiate discussions are very confusing for general ML readers. ", "broader_impact_concerns": "Not relevant", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tNXZECJQ7w", "reviewer": "Reviewer_t7Z6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Since I'm not an expert on the subject of normalizing flows, I would like to use this opportunity to ask the authors several possibly basic questions, before providing a review judging the strengths and weaknesses. I will return and modify this section as well as the request changes section based on the answers I receive. \n\nThe most important questions I have are about the training procedure. Firstly, can the authors provide a more clear description of the model that is being trained all in one place? I have a lot of confusion over this at the moment, and in particular, I would like address several of my detailed questions below: \n1. Is the restriction to training only interpolated models the only mechanism ensuring the flow reaches the right target? \n2. As a result, is the training loss essentially just ensuring the interpolation still forms a flow, where $V_t^\\theta, C_t^\\theta, f_t^\\theta$ are compatible? In particular, isn't $C_t$ determined by $f_t$ already, since it's arising from the normalizing constant? \n3. How is $q_\\theta$ parameterized by $V_t^\\theta$, and what is this object? \n4. What is $S$ in the definition of $w(z)$? \n5. I'm not an expert on the subject, but is the target $f_1$ always known for problems of this type? \n\n### Update: Feb 23 \n\nI want to thank the authors for providing quick answers to my questions. I believe at this point I have a better understanding of the paper, and I will provide a review in terms of the strengths and weaknesses. However, I will still refrain from providing a confident judgement of the paper, as I am not an expert on the subject of normalizing flows. \n\nStrengths \n1. The authors provided a new method for sampling using a flow based on technique, that is trained to ensure the flow structure is maintained. \n2. The approach avoids the explicit calculation of normalizing constants along the trajectory. \n3. Experiments show good behaviours of the algorithm. \n\nWeaknesses \n1. Perhaps it's just due to my lack of knowledge of this field, but I don't know how to compare the method the authors introduces to other comparable approaches. \n2. It's unclear how the parameters affect the computational difficulty of the method, e.g. the dimension of the problem, the distance between the different modes. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors introduced a new method of approximating a normalizing flow based on interpolating the energy functional. With experiments the authors showed that the method avoids collapsing on a single mode. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To be written later based on the answers I receive. \n\n### Update: Feb 23 \n\nI would like the authors to address the my comments in the weaknesses section by providing \n1. A comparison with alternative methods, and a discussion on how to judge between the different methods. \n2. Some additional experiments and discussion on how the performance or implementation difficulty changes depending on the parameters of the given distribution. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new method for approximating a normalizing flow based on interpolating the energy functional. The proposed method addresses the issue of collapsing to a single mode in normalizing flow models and has been shown to perform well in experiments. The method involves fitting a continuous-time projection/action that pushes samples from an initial distribution to the target energy function using a combination of linear interpolation and a novel transient additive component. There were some minor concerns and the revision has addressed all concerns. So I recommend accept. 


Decision: Accept (poster)