Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flows
Authors: Alberto Cabezas
Louis Sharrock
Christopher Nemeth
Venue: NeurIPS 2024 poster (2024)
Keywords: Sampling algorithms
Bayesian inference
MCMC
flow matching
continuous normalizing flows
Primary area: probabilistic_methods

Abstract: Continuous normalizing flows (CNFs) learn the probability path between a reference distribution and a target distribution by modeling the vector field generating said path using neural networks. Recently, Lipman et al. (2022) introduced a simple and inexpensive method for training CNFs in generative modeling, termed flow matching (FM). In this paper, we repurpose this method for probabilistic inference by incorporating Markovian sampling methods in evaluating the FM objective, and using the learned CNF to improve Monte Carlo sampling. Specifically, we propose an adaptive Markov chain Monte Carlo (MCMC) algorithm, which combines a local Markov transition kernel with a non-local, flow-informed transition kernel, defined using a CNF. This CNF is adapted on-the-fly using samples from the Markov chain, which are used to specify the probability path for the FM objective. Our method also includes an adaptive tempering mechanism that allows the discovery of multiple modes in the target distribution. Under mild assumptions, we establish convergence of our method to a local optimum of the FM objective. We then benchmark our approach on several synthetic and real-world examples, achieving similar performance to other state-of-the-art methods but often at a significantly lower computational cost.

Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flows
Alberto Cabezas"
Louis Sharrock"
Department of Mathematics and Statistics Lancaster University, UK
Department of Mathematics and Statistics
Lancaster University, UK
caabezzzggozallllllllaaaatteeee ac uk
1. basaarroc@llnccsterrrr ac.
Christopher Nemeth
Department of Mathematics and Statistics
Lancaster University, UK
c. neme anch@laanaaa castteer. ac.uk
Abstract
Continuous normalizing flows (CNFs) learn the probability path between refer- ence distribution and target distribution by modeling the vector field generating said path using neural networks. Recently, Lipman al. [45] introduced simple and inexpensive method for training CNFs in generative modeling, termed flow matching (FM). In this paper, we repurpose this method for probabilistic inference by incorporating Markovian sampling methods in evaluating the FM objective, and using the learned CNF improve Monte Carlo sampling. Specifically, we propose an adaptive Markov chain Monte Carlo (MCMC) algorithm, which combines a local Markov transition kernel with non-local, flow-infoe now-informed transition kernel, de- fined using CNF. This CNF is adapted on-the-fly using samples from the Markov chain, which are used to specify the probability path for the FM objective. Our method also includes an adaptive tempering mechanism that allows the discovery of multiple modes in the target distribution. Under mild assumptions, we establish convergence of method to local optimum of the FM objective. We then benchmark our approach on several synthetic and real-world examples, achieving similar performance to other stae-oof-the-ar methods, but often significantly lower computational cost.
Introduction
The task of sampling from probability distribution known only up normalization constant is funddaeental problem arising in a wide variety of fields, including statistical physics [51], Bayesian inference [25], and molecular dynamics [43]. In particular, let π r(dz) be target probability distribution on d with density ((x) with respect to the Lebesgue measure of the form'
+(x) 7
where a Rd - R4 is a continuously differentiable function which can be evaluated pointwise, and Z Squ #(x)dz an unknown normalizing constant. We are interested generating samples from the target distribution π in order to approximate integrals of the form .. [T]]] = E,, [x))], where f: R R.
'Equal contribution
In slight abuse of notation, we use π to denote both the target distribution and its density.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

A standard solution to this problem is Markov chain Monte Carlo (MCMC) [12, 64], which relies on the construction of Markov process which admits the target π as its invariant distribution. One of the most broadly applicable and widely studied MCMC methods is the Metropolis-Hasting (MH) algorithm [32], which proceeds in two steps. First, given a current sample J. a new sample y is proposed according to some proposal distribution ((-1zr). Then, this sample is accepted with probability axx 3) = min { {1, ा| Jaty m) }. This strategy generates a Markov chain with the desired stationary distribution and, under mild conditions on the proposal and the target, also ensures that the Markov chain is ergodic [65]. However, for high-dimensional. multi-modal settings, such methods can easily get stuck in local modes, and suffer from very slow mixing times [e.g.. 48).
Naturally, the choice of proposal distribution (-|2z) is critical to ensuring that MH MCMC algorithms explore the target distribution within a reasonable number of iterations. A key goal is to obtain pro- posal distributions with fast mixing times, which can be applied generically to any target distribution. This is particularly challenging in the face of complex, multi-modal (or metastable) distributions, which commonly arise in applications such as genetics [38], protein folding [41], astrophysics [22], and sensor network localization [37]. On the one hand, local proposals, such as those employed in the cetropolis-Adjuseee Langevin Algorithm (MALA) [66] or Hamiltonian Monte Carlo (HMC) [20, 56] struggle to transition between regions of high-probability, resulting in very long decorrelation times and few effective independent samples [e.g.. 49). On the other hand, global proposal distributions must be very carefully designed in order avoid high rejection rates, particularly in high dimensions 47].
Another popular approach to sampling is variational inference (VI) [10, 34, 61, 79], which obtains a parametric approximation To- (x)) T (xx to the target by minimising the Kullback-Leibler (KI.) divergence to the target over parameterized family of distributions Dj = {TO : E θ). State-of-the- art VI methods use normalizing flows (NFs), which consist of a sequence of invertible transformations between a reference and target distribution, to define flexible variational family [62]. There has also been growing interest in the use of continuous normalizing flows (CNFs), which define a path between distributions using ordinary differential equations [15, 27, 45J. CNF's avoid the need for strong constraints on the flow but, until recently, have been hampered by expensive maximum likelihood training.
In recent years, several works have sought hybrid methods which utilize NFs to enhance the per- formance of MCMC algorithms; see, e.g., [28] for a recent review. For example, NFs have been successfully used to precondition complex Bayesian posteriors, significantly improving the per- formance of existing MCMC methods [e.g.. 33, 39, 59, 68]. The synergy between local MCMC proposals and global, flow-informed proposals has also been explored, leading to enhanced mixing rates and effective estimation of multimodal targets [e.g.. 24, 67].
Our contributions In this paper, we continue this promising line of work, introducing new probabilistic inference scheme which integrates CNFs with MCMC sampling techniques. Our approach utilizes flow matching (FM), a scalable, simulation-free free training objective for CNFs recently introduced by Lipman et al. [45]. This enables, for the first time, the incorporation of CNFs into an adaptive MCMC algorithm. Concretely, our approach augments local, gradient-based Markov transition kernel with a non-local, flow-informed transition kernel, defined using a CNF. This CNF, and the corresponding transition kernel, are adapted on-the-fly fly using samples from the chain, which are used to define the probability path for the FM objective. Our scheme also includes an adaptive tempering mechanism, which is essential for discovering multiple modes in complex target distributions. Under mild assumptions, we establish that the flow-network parameters output by our method converge to local optimum of the FM objective. We then demonstrate empirically the performance of our approach on several synthetic and real-world examples, illustrating comparable or superior performance other state-of-the-ar sampling methods.
2 Preliminaries
Continuous Normalizing Flows A continuous normalizing flow (CNF) is a continuous-time generative model which is trained to map samples from base distribution po to a given target distribution [15]. Let in be time-dependent vector field that runs continuously the unit interval. Under mild conditions, this vector field can be used to construct time-dependent diffeomorphic map

---

called flow Φ 10 x R + R.. defined via the ordinary differential equation (ODE):
di (r(x) (x)))
Φ0(x) =
(2)
Given reference density Po R + R. and the flow 6, can generate probability density path p: [O, 1] x Rd - R__ as the pushforward of po under Φ. viz pt = [ for E [0, 1]. This yields, via the instantaneous change-of-variie variables formula (e.g., 14]
log (x log //" V V-va
where xi : (3)), and where V is the divergence operator, i.e. the trace of the Jacobian matrix. In modern applications, the vector field V; is often parameterized using neural network L. in which case the ODE in (2) referred to as a neural ODE [15]. In turn, this yields deep parametric model of for the flow @t, known as CNF [27].
Flow Matching One would typically like to learn a CNF which maps between a given reference density Po and a target density Given samples from the target, one approach is to maximize the log-likelihood F= log pf (a)] In practice, however, maximum likelihood training very slow as both sampling and likelihood evaluation require multiple network passes to solve the ODE in (2).
Flow Matching (FM) provides an alternative, simulation-free method for training CNFs [45]. Let Pa(x)) be a target probability density path such that Po is simple reference distribution, and P1 approximately equal to the target distribution. Let ny (ar) be vector field which generates this (z). Then the FM objective for the CNF vector field v (z) is defined as
[(0: m) 10
[1le (x) 2n (=)
In practice, we do not have direct access to the target vector field, i (1r) and so we cannot minimize (4) directly. However, shown in Lipman et al. [45, Theorem 2), it is equivalent to minimize the conditional flow-matching (CFM) loss
J(0: x) = R
Faautoo.EEarraa Eay Rexpe(-la)) (x) (x([)))))
(5)
where pr (+(1)) is conditional probability density path satisfying Po (o(x)))) po and pi (α|²1) 8 and vr ([(-(11) : Rd + _d is conditional vector field that generates pe(-(rr). There are various choices for pe(-(z1) and (-(1))) For simplicity, we here assume that the conditional probability path is Gaussian, viz (((1)=1)
A(x (x1), Se (x1 PIa), where j0, 1] Rd - Rd denotes time- dependent mean, and : [o, 1] x RR + R,, a time-dependent sealar standard deviation. For our experiments, we further adopt the optimal transport conditional probability path introduced in [45], setting m, (x1) (x, and sx(x)) ) = - (1 - amin) ) for some Omin < 1. In this case, the conditional
vector field assumes the particularly simple form (2(()))) 1-(1 1-(1-omin] Gmin
(1-Amin lz
Markovian Flow Matching
In this section, present main contribution, adaptive MCMC algorithm which combines non-local, flow-informed transition kernel trained via FM; local, gradicnt- based Markov transition kernel; and an adaptive annealing schedule. We begin by describing how CNFs can be used within MH MCMC algorithm.
3.1 MCMC with Flow Matching
Suppose, for now, that have access CNF (d))) trained (c.g.) via flow-matching, with corresponding vector field (v() rejo,)0, which generates a probability path (pf)=e(o.1) between a reference density Po and approximation of the target density Given point 25 Rd on the reference space, we can evaluate the log-density the pullback of the target distribution и as
0
10g(09177(20)) = log ((f(())))
V o (zo))d.
(6)
3

---

Under the assumption that the CNF approximately transports samples from Po to we expect that [oflsPo = π in the target space, and that [ Po in the reference space. Given that the reference distribution Po is chosen such that it is easy to sample from, this suggests the following strategy, sometimes referred as neural trasport MCMC or neutraMCMC [28, 33, 44, 59]. First, transform initial positions z from the target space to the reference space by solving
I0 lo (21) log po
21
/
o (x))) of (204) dt,
=
(7)
which integrates the combined dynamics of xy and the log-density of the sample backwards in time. Then, generate MCMC proposals yo in the reference space using any standard MCMC scheme which targets the pullback of the target distribution, as defined in (6). Finally, transform accepted proposals back target space using the forward dynamies, viz
[ logri(s)) log po (wo) = yo
[
el (we) ] dt. of (v())
(8)
This corresponds to using a transformation-inooree proposal in a Markov transition step, an approach which has been successfully applied using (discrete) normalizing flows [28, 33, 44, 59].
There are various possible choices for the proposal distribution on the reference space (see Appendix A). For example, [23] consider independent MH (IMH) proposal, where i.i.d. samples are drawn from the reference distribution. Here we focus on a flow-informed random-walk, motivated largely by its superior empirical performance in numerical experiments. This proposal performs particularly well on high- high-dimensional problems, where overfitting of the CNF can be corrected with stochastic steps, while exacerbated by independent proposals [39]. Concretely, our flow-informer random-walk transition kernel, summarized in Algorithm 2 (see Appendix A), can be written as
P(r, dy: π, 0) = a(x, y) (ddy/=> + (1 (d
(9)
where po (dylæ) is the distribution defined by the transition
xo x J. (d)(())) yo N copr), y so + L' (o)(
(10)
and axx, y) = min {1, (()) and b(x) = fga a(x (dyllx))) 7(2)pe(u(z)
Training the CNF Thus far, we have assumed that it is possible to train a CNF which maps samples from the reference distribution Pa to (an approximation of) the target distribution т. Clearly, however, the CFM objective not immediately applicable in our setting, since we do not have access to samples from the target π.
Tong et al. [72] propose two alternatives in this case: (i) use an importance sampling reweighted objective function, or (ii) use samples from a long-run MCMC algorithm (e.g.. MALA) as approximate target samples. Both of these approaches, however, have limitations. The former is unlikely succeed when the proposal distribution differs significantly from the target distribution, while the latter will only perform well when the chosen MCMC method mixes well.
In this paper, we adopt a different approach, updating the parameters of a CNF based on a dynamic estimate of the CFM objective obtained via adaptive MCMC algorithm. This is similar in spirit to other recent flow-in -nw-irnodeed MCMC algorithms [24, 35, 67], and the Markovian score climbing algorithm in [54].
3.2 Adaptive MCMC with Flow Matching
Overview Our adaptive MCMC scheme combines non-local, flow-informed transition kernel (e.g., a flow informed random-wall and local transition kernel (e.g.. MALA), which generate new samples from a sequence of annealed target distributions. These new samples are used to define a new estimate the CFM objective in (5), which is optimized to define new CNE, These steps are repeated until the samples converge in distribution to the target æ, and the flow-network parameters converge to local minima of the flow matching objective (see Proposition 3.1). This scheme, which we refer to Markovian Flow Matching (MFM), is summarized in Algorithm 1.

---

Sampling There is significant freedom regarding the choice of both the local and the non-local MCMC algorithms. In our experiments, we adopt the Metropolis- Mieropolis-Adjusted Langevin Algorithm (MALA) as the local algorithm. Thus, the local Markov kernel Q is given by
Q T) = (x, v)d(dy/)) + (1 - 6(x)))
(11)
where g(dy) z)) is given by
1 (ddy/=) x exp (-++ I1y TV log क (x)1l² dy.
(12)
and where, as elsewhere, (xx y) min {1, <(s)p(pla) / and b(x) Sa a(x, (dy(()))) In princi- ple, however, other choices such as HMC could also be used.
Meanwhile, for the non-local MCMC algorithm, we adopt the flow-informed random walk with non- local Markov kernel P defined in (9). Together, assuming alternate local and non-local steps, these two kernels define a Markov chain with Markov transition kernel R == P Q. given explicitly by R(z, dy; म, 0) = Je (z, d2; π P(, dy; ग, 0). In practice, the balance between local and non-local moves controlled by the hyperparameter li.. which sets the number of local steps before a global step.
Training Following each MCMC step, the parameters of the flow-informed Markov transition kernel P are updated based on a new estimate of I (6; .). To be precise, suppose we write Ne := Mo (-..; x, 8) for the distribution of the Markov chain with kernel R(-,;; म, 8) after & e N
steps, starting from initialization fo. where R = R o R... o R. Our objective function is then given by
J(0; = 1) Ee)pe(-))) (x) - oz (x(
(13)
For our choice of conditional probability path (i.e., the optimal transport path), we can in fact rewrite this objective as [45, Section 4.1]
[[; Mhy min)) = E,-2((0,1) un Exo-po [|e@ (dr(zol=1) - (oo -1(()))))))
(14)
where ve (x(21)= = T-a -
21-(1- and oi (x(2z) = (1 - (1 mmint) + tx In practice, we will
optimize Monte Carlo estimate of this objective, namely,
J(0; (K)) min))
(a. u. (dr, (26l) N
1
1
(15)
where (k)) are the samples from N chains of our MCMC algorithm after h E N iterations, t.d. 2 ~ Po. and 4, ~ (0, 1). The use N particles allow the state's 's mutation to N computing cores running in parallel at each iteration. Sampling steps can be run in parallel using modern vector- oriented libraries, before each particle is used to approximate the loss and update the parameters. Thus, the speedup gained by using more than one core scales linearly with the number of cores as long as there are as many cores as there are particles.
Annealing For complex (e.g., multimodal) target distributions, it can be challenging to learn CNF that successfully maps between the reference po and the target т. For example, if the locations of the modes of the target not known priori, and the MCMC chains more of the modes, it is unlikely that the local MCMC kernel, and therefore the trained flow, will ever discover these modes [e.g., 24, Section IV.C]. To alleviate this problem, one approach is to iteratively target a sequence of annealed densities (=)) which smoothly interpolate between a simple base distribution no (x: (e.g., standard Gaussian), and the target distribution T (I:) = xxx). This idea is central to other Monte Carlo sampling methods such as Sequential Monte Carlo (SMC) [18] and Annealed Importance Sampling (AIS) [55], well sampling methods used in coore-based generative modelling [e.g.. 70]. In our case, the annealed targets act as intermediary steps within the flow-informed MCMC scheme.
initialized far from one
standard way in which to construct the sequence ( is to use a geometric interpolation, defining
(x( TK (
(16)
5

---

Algorithm 1 Markovian Flow Matching
Input: target x. base To. vector field ug. reference Po. concentration mmin initial parameters 0
target ESS fraction a, iterations K, number of particles N, local kernel Q, flow-informed kernel
P. MCMC steps per resampling step ki. step sizes E1:K.
2: Output: flow-network parameters 0g ,.....NN (initialize samples)
3: Sample z0 ~ To for
4: for k = 1: K do
5:
if 3k-1
then
6:
3x solve (17) for 3k-1 < B S (update annealing lemperature)
7:
Ty = solve (16) (update annealing density)
8:
end if
9:
if'
mod ko + =-1=00 then
# ~ P( Tkk 0x-1) for i ,....... N (flowinffrmed Markov transition).
11:
else
12:
# ~
for I,...,N N (local Markov transition).
13:
end if
14:
02 = 0k- EY VoJ (ok min) (apdate flow-network parameters)
15:
end for
where Bo.k is sequence of temperatures which satisfies 0= 3u < <... 3x = [e.g.. 55]. In practice, it can be difficult to choose a good sequence of temperatures that provides a smooth transition between densities. One heuristic for adaptively setting this sequence is based the effective sample size (ESS). In particular, by setting the ESS to a user-specified percentage a of the number of particles N, the next temperature B& in the schedule can be determined by solving the recursive equation [9]
EN
80-1 [8)]
30-
( a
N 1= W
B. = inf BK-1 <35I:
(17)
84- (3)2
N
1=1
W
where w. (3) TK
| (x*) importance weights given the current temperature 3h In practice, find that the inclusion of this adaptive tempering scheme is essential in the presence of highly multimodal target distributions, enabling the discovery of modes which are not known priori.
Tu( I (x
)
( //mol (2 (
a vector of parameters 0x which defines a CNF (6t I]. Under the assumption that the parameter estimate converges, that is, 0g - 8 global as K + 00, where 4a global = arg mingee [6; π) is the global minimizer the CFM objective [f; x) in (5), this CNF guaranteed to generate a probability path (+f )ee(0.1) which transports samples from the reference po to the true target [c.g.. 45].
Convergence
The output of Algorithm
In practice, the objective I [0; σ) is highly non-convex, and thus is not possible to establish convergence result of this type without imposing unreasonably strong assumptions on the vector field (uf)rejoo))) This being said, is reasonable to ask whether 0g converges to local optimum of the CFM objective. We now answer this question in the affirmative. In particular, under mild regularity conditions, Proposition 3.1 guarantees that 0x - H* almost surely as K - oo, where 6* denotes local minimum of the CFM objective. This proposition closely mirrors [54, Proposition 1J. Its proof, which relies on classical result in [6, Theorem 3. 3.17], is provided Appendix
Proposition 3.1. Assume that Assumptions B.1 B.6 hold (see Appendix B). Assume also that (fx) is bounded sequence, which almost surely visits compact subset of the domain of attraction of 6- infinitely often. Then 0g + 8* almost surely.
Related work
In recent years, number of works have proposed algorithms which combine MCMC techniques with NFs; see, egg., [2, 28] for recent surveys. Broadly speaking, these algorithms fall into two distinct categories. NeutraMCMC methods leverage NFs as reparameterization maps which simplify

---

the geometry of the target distribution, before running (local) MCMC samplers in the latent space. This technique was pioneered in the seminal paper [59], and since been investigated in number of different works [e.g., 13, 33, 44, 54, 58, 68, 82, 84). Flow MCMC methods, meanwhile, utilize the pushforward of the base distribution through the NF as an (independent) proposal within an MCMC scheme. This approach was first studied by [3], and further extended in [4, 31, 57].
More recently, [24, 67] have introduced adaptive MCMC schemes which combine local MCMC samplers (e.g., MALA or HMC), with a non-local, flow-informed proposal (IMH or i-SIR); see also [35]. Our algorithm combines aspects of both neutraMCMC and flow MCMC methods and, unlike any existing approach, make use of a CNF (as opposed to discrete NF), by leveraging the conditional flow matching objective. The use of NFs within other Monte Carlo algorithms has also been the subject of recent interest. For example, [5, 50] consider augmenting SMC with NFs, while [19, 52] use NFs (or diffusion models) within AIS.
Although less directly comparable to our own approach, several other recent works have proposed to use (controlled) diffusion processes sample from unnormalized probability distributions. Such works include Zhang and Chen [85], who introduce the path integral sampler, Vargas et al. [75], who propose the denoising diffusion sampler, and Zhang et al. [83], who introduce generative flow samplers. Some other relevant contributions in this direction include [1, 8, 16, 60, 63, 69, 73, 74, 75, 76, 77, 78].
5 Experiments
In this section, we evaluate the performance of MFM (Algorithm 1) on two synthetic and two real data examples. Our method is benchmarked against four relevant methods. The Denoising Diffusion Sampler [DDS: 75] is a VI method which approximates the reversed diffusion process from reference distribution an extended target distribution by minimizing the KL divergence. Adaptive Monte Carlo with Normalizing Flows INFMMCMC; 24] is an augmented MCMC scheme which uses mixture of MALA and adaptive transition kemels learned using discrete NFs. Flow Annealed Importance Sampling Bootstrap [FAB; 52] is an augmented AIS scheme minimizing the mass-covering a-divergence with a = 2. Finally, Adaptive Tempered SMC (AT-SMC), i.c. the SMC algorithm described in [18] using a MALA transition kernel and a sequence of annealed distributions chosen adaptively by solving (17).
For each experiment, all MALA kernels use the same step size, targeting an acceptance rate of close to since we estimate expectations, e.g. in (14), using the current ensemble of particles, rather than single long chain. Following [85], we parameterize the vector field as
NN 03)u (x) NN(x, t; 01) + NN:(:; 02) x V log (z),
(18)
where the neural networks are standard MLPs with 2 hidden layers, using a Fourier feature aug- mentation for / [71], and where NN outputs a real value that reweights the vector field output using the time component. This architecture is also used by DDS [75, Section 4J. Meanwhile, FAB and NF-MCMC use rational quadratic splines [21]. Flows are trained using Adam [40] with a linear decay schedule terminating at ē 0. We report results for all methods averaged over 10 independent runs with varying random seeds. Code reproduce the experiments is provided at https///ithubb.cm/llbbbbbb
5.1 4-mode Gaussian mixture
Our first example is mixture of four Gaussians, evenly spaced and equally weighted, in two- dimensional space. The four mixture components have means (8, 8). (-8,8), (8, -8). (-8, -8). and all have identity covariance. This ensures that the modes sufficiently separated to mean that jumping between modes requires trajectories over sets with close to null probability. Given the synthetic nature of the problem, we can measure approximation quality using the Maximum Mean Discrepancy (MMD) [e.g., 29]; see Appendix C.1 for details. We can also include, as a benchmark, the results for approximation learned using FM with true target samples. Diagnostics for all models are presented in Table 1, and learned flow samples in Figure 1. Further algorithmic details and results are provided in Appendix C.2.
In this experiment, only our method (Figure 1a) and DDS (Figure Ic) learn the fully separated modes, reflecting the greater expressivity of CNFs in comparison to the discrete NFs used in, c.g., NF-MCMC
7

---

4-mode
16-mode
MMD
seconds
MMD
seconds
FM W/ π samples
3.69c-411..4e--
22.3 +: 0.64
1.35e-316.666
22.4 1.01
MFM &Q =
2.37e-3#2.29::
27.9 = 1.27
L.88-223366e:
28.2
MFM kQ 10
4.144-444.41e-4
117. 5.65
2.87e-3+9.66e-4
89.6+5.11
DDS
1.762-442..22-4
+14.+0.68
1.02e-1+4.10e-2
115. 0.64
NF-MCMC
5.8
72.0+111
8.05e-3+1142e-22
67.0 12.3
12.3
FAB
2.69 2.699-44:2.06-44
1.51e-3+11.0e-3
102. 4.32
AT-SMC
3.95e-242.90c-2
2.18
1.73e-215.30e-3
2.19 0.21
Table 1: Diagnostics for the two synthetic examples. MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
(Figure 1d). is worth noting that DDS provides closer approximation the real target than MFM and, notably, even FM trained using true target samples (top row). Given that both methods use the same network architecture but different learning objective, this suggests a potential limitation with the FM objective, at least when using this network architecture. This being said, MFM is notably more efficient than DDS (as well as the other methods) terms of total computation time. While this not a critical consideration in this synthetic, low-dimensional setting, it is significant advantage of MFM higher-dimensional settings involving real data (e.g., Section 5.3 and Section 5.4).
-18
45
11
15
(a) MFM kg== 10a
(b) FAB
(c) DDS
(d) NF-MCMC
Figure I: Comparison between MFM, FAB, DDS, and NF-MCMC. Samples from the target density for the 4-mode Gaussian mixture example.
5.2 16-mode Gaussian mixture
The second experiment is mixture of bivariate Gaussians with 16 mixture components. This is modification of the 4-mode example, with contrasting qualities that illustrate other characteristics of each of the presented methods. In this case, the modes are evenly distributed on |-16, 16]2, with random Dog-normal variances. The number of modes reduces the size of sets of (near) null probability between the modes, making jumping between them easier. To increase the difficulty of this model, all methods are initialized on a concentrated region of the sampling space. Diagnostics are presented in Table and learned flow samples in Figure Further details are provided Appendix C.3.
In this example, DDS collapses to the modes closest to the initial positions while our method captures the whole target. Since the modes are no longer separated by areas of near-zero probability, the discrete NF methods are now able to accurately capture the target density. In this case, FAB marginally outperforms MFM measured by the MMD, but this slight improvement in performance comes the cost of much higher run-time.
5.3 Field system
Our first real-world example considers the stochastic in [24], and described in Appendix C.5. This fundamental reaction-diffusion equation is central to the study of phase transitions in condensed matter systems. Incorporating random forcing terms or thermal fluctuations allows for a stochastic treatment of the dynamics, capturing the inherent randomness and uncertainties in physical systems. This model leads discretized target density
model [7], used benchmark

---

(a) MFM =o= 10
(h) FAB
(c) DDS
(d) NF-MCMC
Figure 2: Comparison between MFM, FAB, DDS, and NE-MCMC. Samples from target density for the 16-mode Gaussian mixture example.
which takes the form
d+1
b Σ (1 x2)2
log (x: = -B3
Σ
[2q T2 ) +
2
(19)
2A8
with As 4. and boundary conditions Tip Id+1 0. In our experiments, we take d 64. Meanwhile, following [24], other parameter values chosen to ensure bimodality a == 0.1, 1/a- = 10, and / = 20. The bimodality induced by the two global minima complicates mixing when using traditional MCMC updates. Learning the global geometry the target and using that information to propose transitions facilitates movement between modes. Unlike previous work [e.g.. 24], we deliberately choose not employ an informed base measure. Instead, we opt for standard Gaussian with no additional information. making the problem significantly more challenging. This choice illustrates the robustness of our approach.
#1:
Numerical diagnostics for each method are presented in Table 2. In this case, we use the Kernelized Stein Discrepancy (KSD) as measure of sample quality [e.g.. 26, 46]; see Appendix C.I for details. While this is not a perfect metric, it does allow us to qualitatively compare the different methods considered.
In this case, the tempering mechanism our method is crucial for ensuring that the learned flow does not collapse when plotting the samples generated in the grid in Figure 3. This experiment demonstrates the ability of our method capture complex multi-modal densities, even without informed base measure, a significantly lower computational cost (e.g.. 10-25x faster) than competing methods. Indeed, while FAB the best performing method in this experiment measured by the KSD, it failed to capture both of the modes in the target distribution, and required a much greater total computation time (see Table 2).
of the modes and instead explores both global minima. This is confirmed
worth noting that MFM (and the other two benchmarks, DDS and FAB) significantly outper- formed NF-MCME in this example, despite the similarities between MFM and NF-MCMC. While we tested various hyperparameter configurations for NF-MCMC, we were not able to find a setting that achieved comparable results in the absence of an informed base measure.
65
-04
43
422
-33
-18
(a) MFM kg= 102
(b) FAB
(c) DDS
(d) NF-MCMC
Figure 3: Comparison between MFM. FAB, DDS. and NF-MCMC. Representative samples from the target density for the Field system example.

---

5.4 Log-Gaussian Cox process
Bayesian inference for sighddimessionl spatial models is known be challenging. One such model is the pog-Gassian Cox process (LGCP) introduced in [53], which is used to model the locations of 126 Scots pine saplings in a natural forest Finland. See Appendix C.6 for full details. The target space is discretized to a M 40 x 40 regular grid, rendering the target dimension d = 1600. In Table 2, we report diagnosties for each algorithm.
In this case, the lack of multimodality in the target makes it good fit for non-tempered schemes. Similar to the previous example, NF-MCMC is unable to obtain an accurate approximation the target distribution. We suspect that this may be result of non-convergence: due to memory issues, it was not possible to run NE-MCMC (or FAB) for more than K 100h iterations. This also explains the (relatively) smaller run times of these algorithms in this example. By small margin, DDS provides the best approximation of the target, slightly outperforming MFM and FAB. Meanwhile, MFM provides good approximation to the target at a lower computational cost with respect to its competitors.
Field system
Loo--Gaussin Cox
K = 10
KSD U-stat.
KSD V-stat.
seconds
KSD U-stat.
KSD V-stat.
seconds
MFM KgO- K
2.61 + 2.00
20.9 2.49
52.3
1.311.25
1.13e-1:.05
28.1 + 0.24
117
MFM KQ 103
21.0 2.16
2.66
53.6 1.33
1.12e-1#.04
28.1 + 0.23
143 + 14.5
DDS
36.9
2400
7.59c-2:..0
2
3260
8.41
NF-MCMC
548 + 325
549 + 325
2000
7.55
89.0 238
215 + 46.4
FAB
+
1.78+
3880 7.19
1.55e-1+.06
52.3 +
1040 2.78
AT-MCC
1.61 + 2.33
18.4 2.35
4.13+030
1.39e-2+.01
25.0 + 0.12
6.11 + 0.44
Table 2: Diagnostics for the two real data examples. KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
Conclusion
Summary. In this paper, introduced Markovian Flow Matching, new approach to sampling from unnormalized probability distributions that augments MCMC with CNFs. Our method combines a local Markov kernel with non-local, flow-informed Markov kernel, which is adaptively learned during sampling using FM. It also incorporates an adaptive tempering mechanism, which allows for the discovery of multiple target modes. Under mild assumptions, we established convergence of the flow network parameters output by our algorithm to a local optimum of the FM objective. We also benchmarked the performance of our algorithm several examples, illustrating comparable performance to other state-of-the-ar methods, often at a fraction of the computational cost.
Limitations and Future Work. We highlight three limitations of our work. First, theoretical result established convergence of the flow network parameters obtained via MFM to a local minimum of the FM objective. Further work is required to understand how well these local minima generalize, in order to accurately quantify how accurately the corresponding CNF captures the target posterior. Second, we did not establish non-asymptotic convergence rates for our method. Finally, since it was not the main focus of this work, we did not explore in great detail other choices of architecture for the flow network. We expect that, for certain targets, this could have a significant impact the performance of MFM. Indeed, a promising avenue for further research lies in developing tailored CNFs designed for particular posterior distributions. This approach would go beyond the current practice of including the gradient of the log-posterior and instead exploit unique characteristics intrinsic to each model when constructing the flow.
Acknowledgments and Disclosure of Funding
LS and CN were supported by the Engineering and Physical Sciences Research Council (EPSRC), grant number EP/V022636/1. CN acknowledges further support from the EPSRC, grant numbers EP/S00L59X/1 and EP/Y028783/1.
10

---

References
[1] T. Akhound-Sadeglh J. Rector- Rector-Brooks, A. J. Bose, S. Mittal, P. Lemos, C.-H. Liu, M. Sendera,
S. Ravanbakhsh, G. Gidel, Y. Bengio, al. Iterated denoising energy matching for sampling from Boltzmann densities. In Proceedings of the 41st International Conference on Machine
Learning (ICML), 2024. 7
[2] M. S. Albergo and E. Vanden-Eijnden. Learning to sample better. arXiv preprint arXiv:2310. 1232, 2023. 6
[3] M. S. Albergo, G. Kanwar, and P. E. Shanahan. Flow-based generative models for Markov chain Monte Carlo in lattice field theory. Physical Review D, 100(3):04515,,2 2019.
[4] M. S. Albergo, G. Kanwar, S. Racanière, D. J. Rezende, J. M. Urban, D. Boyda, K. Cranmer,
D. C. Hackett, and P. E. Shanahan. Flow-based sampling for fermionic lattice field theories. Physical Review D, 104: 104:17, 2021. doi: 10.1 (0.1103/Physeev. 1004.............. 14507..
[5] M. Arbel, A. Matthews, and A. Doucet. Annealed flow transport Monte Carlo. In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021.
[6] A. Benveniste, M. Metivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Springer-Verlag Berlin, Heidelberg, 1990. doi: 10.1007/978-3--42-755999
[7] N. Berglund, G. D. Gesü, and H. Weber. An Kyrmers law for the stochastic Allen-Cahn equation in dimension two. Electronic Journal of Probability, 22:1 27, 2017. doi: 10.1214/ 17-EJP60.
[8] J. Berner, L. Richter, and K. Ullrich. An optimal control perspective on diffusion generative modeling. Transaction Machine Learning Research (TMLR), 2024. 7
[9] A. Beskos, Jasra, N. Kantas, and A. Thiery. On the convergence of adaptive sequential Monte Carlo methods. Annals of Applied Probability, 26(2): 2016. doi: 10.1214/ 15-AAP1113.
[10] D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational inference: a review for statisticians. Journal of the American Statistical Association, 112(518):859-877, 2017. doi: 10.1080 0114499917.1187773
[11] J. Bradbury, R. Frostig, P. Hawkins, M. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, VanderPlas, Wanderman- Milne,, and Q Zhang. JAX: composable transformations of Python+NummPy programs, 2018. URL http:// /ttp::////ttttt//g com/ google/jax. 18
[12] S. Brooks, A. Gelman, G. Jones, and X.-L. Meng. Handbook of Markov Chain Monte Carlo. Chapman and Hall/CRC, 1st edition, 2011 1. doi: 10.1201/b10905. 2
[13] A. Cabezas and C. Nemeth. Transport elliptical slice sampling. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS), 2023.
[14] R. Chen and Y. Lipman. Flow matching on general geometries. In Proceedings of the 12th International Conference Learning Representations (ICLR), 2024.
[15] R. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd Annual Conference on Neural Information Processing
Systems (NeurIPS), 2018. 2,
[16] V. De Bortoli, M. Hutchinson, P. Wirnsberger, and A. Doucet. Target score matching. arXiv preprint arXiv:2402.08667. 2024. 7
[17] I.. Del Debbio, J. Marsh Rossney, and M. Wilson. Efficient modeling of trivializing maps for lattice ø 4 theory using normalizing flows: a first look at scalability. Physical Review D. 104(9): 094507, 2021. doi:
2
[18] Del Moral, A. Doucet, and A. Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3):411-44 2006. 5.7
11

---

[19] A. Doucet, W. Grathwohl, A. G. Matthews, and H. Strathmann. Score-based diffusion meets annealed importance sampling. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. 7
[20] S. Duane, A. D. Kennedy, B. J. Pendleton, and D. Roweth. Hybrid Monte Carlo. Physics letters B, 195(2):216-22) 1987. doi:
(11188700-6608791199999 197-)9.11999 2
[21] C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Neural spline flows. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS), 2019. 7
[22] F. Feroz, M. Hobson, and M. Bridges. MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics. Monthly Notices of the Royal Astronomical Society, 2009. 2
[23] M. Gabrié, G. M. Rotskoff, and E. Vanden-Eijnden. Efficient Bayesian sampling using nor- malizing flows assist Markov chain Monte Carlo methods. In Proceedings of the 38th International Conference on Machine Learning (ICML): 3rd Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. 4
[24] M. Gabrié, G. M. Rotskoff, and E. Vanden-Eijnden. Adaptive Monte Carlo augmented with normalizing flows. Proceedings of the National Academy of Sciences, 119(10):2109420119,
2022. doi: 10. (0.1077/pnas.2109422 19. 2, 4, 5, 7, 8, 9, 20
[25] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995.
[26] Gorham and Mackey. Measuring sample quality with kernels. In Proceedings of the Proceedings of the 34th International Conference on Machine Learning (ICML), 2017. 9, 18
[27] W. Grathwohl, R. T. Chen, J. Bettencourt, I. Sutskever, and D. Duvenaud. FFJORD: Free-form continuous dynamies for scalable reversible generative models. In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2018. 2, 3, 19, 21
[28] L. Grenioux, A. Durmus, É. Moulines, and M. Gabrié. On sampling with approximate transport maps. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.
2, 4,
[29] A. Gretton, K. M. Borgwardt, J. Rasch, B. Schöolkopf, and A. Smola. A kemnl two-sample test. Journal of Machine Learning Research (JMLR), 13:723-773, 2012. 7, 18
[30] M. G. Gu and H. Kong. A stochastic approximation algorithm with Markov chain Monte- Carlo method for incomplete data estimation problems. Proceedings of the National Academy of Sciences, 95(13):7270-7274 1998. doi: 10. 00.1073/paas.5.13.7777 13..770. 17
[31] D. C. Hackett, C.-C. Hsich, M. S. Albergo, D. Boyda, J.-W. Chen, K.-F. Chen, K. Cranmer, G. Kanwar, and P. E. Shanahan. Flow-based sampling for multimodal distributions in lattice field theory. arXiv preprint arXiv:2107.00 00734, 2021.
[32] W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1)977-109 1970. doi: 10. 1001099/ooom//57...97 2
[33] M. Hoffman, P. Sountsov, J. V. Dillon, I. Langmore, D. Tran, and S. Vasudevan. Neutra-lizing bad geometry in Hamiltonian Monte Carlo using neural transport. In Proceedings of the 1st Symposium on Advances in Approximate Bayesian Inference (AABI), 2019. 2, 4,7
[34] M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic variational inference. Journal of Machine Learning Research, 14(1): 1303-13777, 2013. 2
[35] N. T. Hunt-Smith, W. Melnitchouk, F. Ringer, N. Sato, A. W. Thomas, and M. J. White. Accelcrating Markov chain Monte Carlo sampling with diffuusion models. Computer Physics Communications, 296: 109059, 2024. doi: 10.1016/j.cpc.2023 (0000/66...pc2222100999999999 4.7
[36] M. F. Hutchinson. A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Communications in -immutiii-- and Computation, 18(3): 1076, 1989. 19, 21
12

---

[37] A. Ihler, J. Fisher III. R. Moses, and A. Willsky. Nonparametric belief propagation for self- localization of sensor networks. IEEE Journal on Selected Areas in Communicationss, 23(4).
2005. doi: 10. 109/1SAC.2005
2
[38] S. T. Jensen, X. S. Liu, Q. Zhou, and J. S. Liu. Computational discovery of gene regulatory binding motifs: Bayesian perspective. Statistical Science, 19(1):188-200 284, 2004, doi: 10.1214/
0883423040000000007 2
[39] M. Karamanis, F. Beutler, J. A. Peacock, D. Nabergoj, and U. Seljak. Accelerating astronomical and cosmological inference with preconditioned Monte Carlo. Monthly Notices of the Royal Astronomical Sociery, 164(
2022. doi: 10. 1093/mmras/stuc2227.. 2,
[40] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
[41] S. Kou, J. Oh, and W. H. Wong. study of density of states and ground states in hydrophobic- hydrophilic protein folding models by equi-energy sampling. The Journal of Chemical Physics, 124(24), 2006. doi: 10.1063/1 10.1063/1.2206677. 2
[42] A. Lee. U-Statistics: Theory and Practice. Taylor & Francis, 1st edition, 1990. doi: 10.1201/ 9780203734520. 18
[43] B. Leimkuhler and C. Matthews. Molecular dynamics. Interdisciplinary Applied Mathematics, 39:443, 2015, doi: 10.10077978-3-3199 16375-8.
[44] S.-H. Li and L. Wang. Neural network renormalization group. Physical Review Letters, 121:
260601, 2018. doi: 10.1 Re.e 221.260601. 4,
[45] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In Proceedings of the 1 11th International Conference on Learning Representations (ICLR), 2023. 1, 2, 3, 6,
[46] Q. Liu, J. Lee, and M. Jordan. kernelized Stein discrepancy for goodness-of-ft tests. In Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016. 9, 18
[47] A. H. Mahmoud, M. Masters, S. J. Lee, and M. A. Lill. Accurate sampling of macromolecular conformations using adaptive deep learning and coarse- grarseeg representation. Journal of Chemical Information and Modeling, 62(7):1602-1617, 2022. doi: 10. 10.1021/acs.jimm.cc014
2
[48] O. Mangoubi, N. S. Pillai, and A. Smith. Does Hamiltonian Monte Carlo mix faster than random walk on multimodal densities? arXiv preprint arXiv:/808.033: 2018. 2
[49] O. Mangoubi, N. Pillai, and A. Smith. Simple conditions for metastability of continuous Markoy chains. Journal of Applied Probability, 58(1):83-105, 2021. doi: 10.1017/jp:.2020.8... 2
[50] A. Matthews, M. Arbel, D. J. Rezende, and A. Doucet. Continual repeated annealed flow transport Monte Carlo. In Proceedings of the 39th International Conference on Machine Learning (ICML), 2022. 7
[51] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6): 1087-1092, 1953.
[52] L. I. Midgley, V. Stimper, G. N. Simm, B. Schölkopf, and J. M. Hernández-Lobato. Flow annealed importance sampling bootstrap. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023. 7, 18, 20
[53] J. Møller, A. R. Syversveen, and R. P. Waagepetersen. Log Gaussian Cox processes. Scan- dinavian Journal of Statistics, 25(3):451-485 1998. doi: (0.1111/146--94990000 10, 22
[54] C. Naesseth, F. Lindsten, and D. Blei. Markovian score climbing: variational inference with (plll) Proceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS), 2020. 4, 6, 7. 17
13

---

[55] R. M. Neal. Annealed importance sampling. Statistics and Computing, 11:125-1133 2001. doi:
10. 10.1023/A110099222150225 5,
[56] R. M. Neal et al. MCMC using Hamiltonian dynamics. Handbook of Markov chain Monte Carlo, 2(1,) 2011.
[57] K. A. Nicoli, S. Nakajima, N. Strodthoff, W. Samek, K.-R. Müller, and P. Kessel. Asymptotically unbiased estimation of physical observables with neural samplers. Physical Review E, 101 023304, 2020. doi: 10. 10.1103/hyyReEE..01..000 00..02304.4 7
[58] Noé. S. Olsson, J. Köhler, and H. Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning, Science, 365(6457), 2019. doi: 10.1 126/science. aaw 147. 7,20
[59] M. D. Parno and Y. M. Marzouk. Transport map accelerated Markov chain Monte Carlo. SIAM/ASA Journal on Uncertainty Quantification, 6(2):645-682, 2018. doi: 10.1137/
17M1134640. 2, 4,7
[60] A. Phillips. H.-D. Dau, M. J. Hutchinson, V. D. Bortoli, G. Deligiannidis, and A. Doucet. Particle denoising diffusion sampler. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.
[61] R. Ranganath, S. Gerrish, and D. Blei. Black box variational inference. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS). 2014. 2
[62] D. Rezende and S. Mohamed. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning (ICML), 2015. 2
[63] L. Richter, J. Berner, and G.-H. Liu. Improved sampling via learned diffusions. In Proceedings of the 12th International Conference on Learning Representations (ICLR), 2024. 7
[64] C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer-Verlag New York, 2nd edition, 2004. doi: 10.
2
[65] G. O. Roberts and Rosenthal. General state space Markov chains and MCMC algorithms. Probability Surveys, 1:20- 71, 2004. doi: 10. 10.12114/10 4/544//44/9 100000024 2
[66] G. O. Roberts and R. L. Tweedie. Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, 2(4):341-363, 1996.
[67] S. Samsonov, E. Lagutin, M. Gabrié, A. Durmus, A. Naumov, and E. Moulines. Local-global MCMC kernels: the best of both worlds. In Proceedings of the 36th Annual Conference Neural Information Processing Systems (NeurIPS), 2022. 2, 2,4,
[68] C. Schönle and M. Gabrié. Optimizing Markov chain Monte Carlo convergence with normalizing flows and Gibbs sampling. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems (NeurIPS): Al for Science Workshop, 2023. 2,7
[69] M. Sendera, M. Kim, S. Mittal, P. Lemos, I.. Scimeca, J. Rector-Brooks. A. Adam, Y. Ben- gio, and N. Malkin. Improved off off-policy training of diffusion samplers. arXiv preprint arXiv:2402.05099 2024.7
[70] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS), 2019. 5
[71] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ra- mamoorthi, J. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In Proceedings of the 34th Annual Conference Neural Information Processing Systems (NeurIPS), 2020. 7
[72] A. Tong, N. Malkin, G. Huguet, Y. Zhang. J. Rector-Brooks, K. Fatras, G. Wolf, and Y. Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research (TMLR), 2024. 4
14

---

[73] B. Tzen and M. Raginsky. Theoretical guarantees for sampling and inference in generative models with latent diffusions. In Proceedings of the 32nd Annual Conference on Learning Theooy (COLT), 2019. 7
[74] F. Vargas and N. Nüsken. Transport, variational inference and diffusions: with applications to annealed flows and Schrödinger bridges. Proceedings of the 40th International Conference on Machine Learning (ICML): Workshop New Frontiers in Learning. Control, and Dynamical Systems, 2023. 7
[75] F. Vargas, W. Grathwohl, and A. Doucet. Denoising diffusion samplers. In Proceedings of the 1 11th International Conference Learning Representations (ICLR), 2023." 7
[76] Vargas, A. Ovsianas, D. Fernandes, M. Girolami, N. D. Lawrence, and N. Nüsken. Bayesian learning via neural Schrödinger-Fölliiee Föllmer flows. Statistics and Computing, 33(1):3, 2023. doi: 10.1007/s1 1222-1222222222 10172-5.
[77] Vargas, T. Reu, and Kerekes. Expressiveness remarks for denoising diffusion models and samplers. In Proceedings of the 5th Symposium on Advances in Approximate Bayesian Inference (AABI), 2023. 7
[78] Vargas, S. Padhy, D. Blessing, and N. Nüsken. Transport meets variational inference: controlled Monte Carlo diffusions. In Proceedings of the 12th International Conference Learning Representations (ICLR), 2024. 7
[79] M. J. Wainwright, M. I. Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends@ in Machine Learning, 1(1-2):1-305, 2008. doi: 10.1561/ 2200000001. 2
[80] K. W. K. Wong, M. Gabrié, and D. Foreman-Mackey, flowMC: Normalizing flow enhanced sampling package for probabilistic inference in JAX. Journal of Open Source Software, 8(83): 5021, 2023. 18
[81] H. Wu, J. Köhler, and F. Noé. Stochastic normalizing flows. Advances in Neural Information Processing Systems, 33:5933- -994,, 2020. 20
[82] B. J. Zhang, M. Marzouk, and K. Spiliopoulos. Transport map unadjusted Langevin algo- rithms. arXiv preprint arXiv:2302.07227 2023. 7
[83] D. Zhang, R. T. Q. Chen, C.-H. Liu, Courville, and Y. Bengio. Diffusion generative flow samplers: Improving learning signals through partial trajectory optimization. In Proceedings of the 12th International Conference on Learning Representations (ICLR), 2024. 7
[84] I.. Zhang, D. M. Blei, and C. A. Naesseth. Transport score elimbing: variational inference using forward kl and adaptive neural transport. Transactions on Machine Learning Research (TMLR), 2023.7
[85] Q. Zhang and Chen. Path integral sampler: a stochastic control approach for sampling. In Proceedings of the 10th International Conference Learning Representations (ICLR), 2022. 7
15

---

A Flow-Informed Markov Chain Monte Carlo Methods
Algorithm 2 Flow-informed Random Walk Metropolis Hastings
1: Input: initial z, target n, vector field of, flow parameters
2: Output: x/
3: Jopt 2.38/ Vd
4: (1(x) = It=1 t- x
? (d)(z))
30 A log ((
5:
T
+
(or())) dt
6:
~ (-12200 popt)
of (c)
7:
A lo1)(
y/1
3/0
of (v) dt
8: a min 1,
( expl p(u))
m(mn) expl log p(m)))
9: With probability a make x' t y1 else x'
Algorithm 3 Flow-informed Independent Metropolis Hastings
1: Input: initial x, target density n, vector field of, reference density po. flow parameters 0.
2: Output: x'
3: (1(
u= t- σ
No A log plg)((u))
1 di V (de(1))
(o((1))
4:
0
5:
@0(x) = #1=0 ~ Po
6:
log p(11)
To log Pa (zo)
-V of (z)())) dl
e (e((z))
1, (Z expp expl log p(uo))
7: a + min
exp(logp(a) (
8: With probability a make x' +- I1 else + I
Algorithm 4 Flow-informed Conditional Importance Sampling
1: Input: initial æ, target density æ, vector field of, reference density go, flow parameters 6, number of importance samples K.
2: Output: x'
3: (1(")
M1=1 f- T
of (dr( V oe (1))
Uo A log p(oo
4:
0
(u))
5: wi Po(Mo) expl
log p(uo]]
6:
x: (0)
< I
7: for 1: do
8:
2o 00(2)) Tz=0 90
+ {
(()()))) dl
21 log p(x)))
E0 log log
9:
-V of
(x11)
10:
+ exp(log((z1))
11:
[k) + x
end for
13: Choose K' with probability P(K = k) x wy. then make x' < (K))
16

---

B Proof of Proposition 3.1
Our proof follows closely the proof of [54, Proposition 1]. Let 0* be a minimizer of the CFM objective in (4), which we recall is given by
[(0; = - Px
[1le] (x) (x(()))) = Eayrr (00) (20)
where we have defined j(0, z1) = E00 (x) [1l)((x) oe [x(2))))1]] Now, consider the ordinary differential equation (ODE) given by
d (t) = V m))
((0) Po. 120
(21)
dt
We say that 0 is stability point of this ODE if, given the initial condition 0(0) = 0, the ODE admits the unique solution 0(t) = for all t 2 0. Naturally, the minimizer 6* is a stability point of this ODE, since Vo TT) 8=8- O. Meanwhile, we call e the domain of attraction of 6* if, given the initial condition ((0) E e, the solution ((t) E for all 2 O, and (tt) converges to H as t + oc.
Let Ia E _²= and S dd be an open subset of Rd Let θ S Ed be an open set in 18 and O, S e be compact subset of Ô. Consider the Markov transition kernel M := : a Q given by cycle of ko repeated transitions of MALA transition kernel and a flow-informed RWMH transition kernel, viz
Mr (2, dy) - f..)=--.. dz (z)) (( dr 0) P(Tkg dy; π,, 0)- (22)
This transition kernel is r-invariant since both P and Q are r-invariant. In addition, let M n, (x, dy) be the repeated application of this Markov transition kernel, namely,
Ma o dy) = f./ Mx.e Mx,,) dz2) ... Ma.o
dy).
(23)
Following [30, 54), we impose the following assumptions, for some sufficiently large positive real number 9 > 1.
Assumption B.1 (Robbins-Monro Condition). requirements:
The step size sequence (=x) satisfies the following
80
Σ Sk
00;
Σ
x oo.
(24)
Assumption B.2 (Integrability). There exists a constant C₁ > 0 such that for, any 0 θ, x E X and
k 121 1,
f Iv1)) dy) < (1(1+121)
(25)
Assumption B.3 (Convergence of the Markov Chain). For each O, holds that
lim sup 1 + | / (1 10000 sE
o dy) - n (dy) = 0.
(26)
Assumption B.4 (Continuity in 0). There exists a constant C2 such that for all 0, 0' E O
/f (1+ (MM dy) M (z, dy)) C2 8 (1 + lx:1)
(27)
Assumption B.5 (Continuity in xt). There exists a constant C3 such that for all æ x2 E X,
(0 / (1 + (y] (M (21, dy) M (12, dy)) C3|21 #2 | | (28)
sup #ge
Assumption B.6 (Conditions on the Objective Function). For any compact subset O_ C θ there exist positive constants p, K1, K2, K3 and 1/2 such that for all 6, 0' E O and xr 11, 42 E X,
Vaj(0, x1) K2 10+1),
(29)
(0 Voj(A, x/) K₂ 21 s/1(11 + |m 12 1x2)1")
(30)
V vej(O, Vaj(0', 20) < Ka 01 1tt).
(31)
With the above assumptions, the result follows from Theorem of [30] by setting x - 31, NT - Mg 2(0,z) + v1).
17

---

C Additional Experimental Details
Code for the numerical experiments is written in Python with array computations handled by JAX [11]. The implementation of relevant methods for comparison is sourced from open source repositories: DDS using franciscovargas/denoo [80], and FAB using lollcat/fab-jax [52]. All experiments are run on an NVIDIA V100 GPU with 32GB of memory. In the following subsections, we will give more details on the modelling and hyperparameter choices for each experiment, along with additional results.
diffusion samplers, NF-MCMC using kazewonggflowMMC
C.1 Diagnostics
Let π and // be two probability measures, Let F denote the unit ball in reproducing kernel Hilbert space (RKHS) H. associated with the positive definite kernel k Fd id + RR Then the maximum mean discrepancy (MMD) between ı and v is defined as [29, Section 2.2]
MMD (T, v) || mm133
(32)
where m is the mean embedding of æ, defined via [ = (T)m=)H for all f E H. Using standard properties of the RKHS, the squared MMD can be written as [29, Lemma 6]
MMD (T, v) = En [k(2, x')]
21 go [k v)] + Bywasr [k(y, 3/)l
(33)
Thus, given samples (#:) ~ v and ( v, an unbiased estimate of the squared MMD can be computed as
1
m
7
V12
MMD (T, m(m 1)
k 25))
k Mi)
m
$
1=1
m
m
k Mj).
(34)
m(m (m 1)
j
For a kernel k, the kernel Stein discrepancy (KSD) between т and v is defined as the MMD between and v. using the Stein kernel h₂ associated with k which is defined as
ki (x, x') = V. Vor((r,')) + Vak(z, V log + Vark(( Va logoo(())) (x) + k(x, x V, log Var log (x))
(35)
and satisfies the Stein identity E, [2, )]] = O. We thus have that
(T v) = MMD? (T, v)
(36)
= Ex,xr KKr (I, x')l 2B [K (I, 4)1 + Eyruyro (n(v.'1)) Ey-rer~~ ~y [ky (y. v''l-
(37)
(38)
We obtain estimates of the KSD by using U-statistics or V-statistics. In particular, unbiased estimate of KSD? (T, is given by the U-statistic [42]
KSD
kn (Vis vi)-
(39)
n(n 1)
Alternatively, we can estimate KSD v) using biased (but non-negative) V-statistic of the form [46, Section 4]
KSDA,
k (Vi, v().
(40)
n2
In all of our numerical experiments, we calculate the U- and V- statistics using the inverse multi- quadratic kernel k(xr,r') (1 (r x')x (x 2'))² due to its favourable convergence properties
[26, Theorem 8), setting 2.
18

---

C.2 4-mode Gaussian mixture
For this experiment, all methods use N = 128 parallel chains for training and 128 hidden dimensions for all neural networks. Methods with a MALA kernel use a step size of 0.2, and methods with splines use 4 coupling layers with 8 bins and range limited to [-16, 16].
In Table 3, we present results for K 10 iterations. Since MFM is much more efficient than other methods, we also report results for a great number of total iterations. Table contains results for K 5 10 iterations for MFM and AT-SMC. In the main text, we present results for K 5 103 learning iterations for MFM and AT-SMC, and K = 10 iterations for the other algorithms, since this renders the total computational cost of all algorithms somewhat comparable.
For both choices of K, we also present results using Hutchinson's trace estimator (HTE) [27, 36] to calculate the MH acceptance probability in the flowiifformed Markov transition kernel. As expected, its effect on sample quality becomes more apparent as ke increases. However, its effect on computation time is less significant than in larger dimensional examples.
K 10
1 log π
KSD U-stat.
KSD V-stat.
MMD
seconds
FM w/ samples
4.20
0.08
6.85e-3:13.755.
7.16e-313.776.3
1.90e-311.1
6.46 E 0.32
MFM Kop
K
4.55 0.16
7.62e-3.17.46:
7.98s-317.45e-1
3.00e-312.00e3
10.4 1 0.66
MFM Ro-
10a
-4.50 + 0.14
3.36e-34:3.15
18
5.71e-3+:3.19e
2.03e:3+2.126-3
12.1 + 0.65
HTTE
4.52 0.15
4.77e-3::1.93e-
5.12e-3:11.33e-3
2.27c-31:1.30--
13.9 0.33
MFM ki
10
1.49
7.01e-3++3.9e:
7.366-3-+3349e-3
1.62e-3+7.61e--
24.8 1.38
w/ HTB
1..12
1.10e-216.80e--
1.14e-2.6.81e--
2.64e-311.03e-3
30.0 1 1.70
DDS
-4.22
9.89e44+1.05es
1.30e-3:t1.05e
1.76x-4+22.22-4
114. 0.68
NF-MCMC
0.21
1800-2::1
1.83e-2:1144--2
5.85e-34:3991c3
72.0 + 11.7
FAB
4.67 0.16
2.31e-3:11.19..3 2.69e-3:11.2ee3
2.69e-412.06e--
101. 1 3.24
AT-SMC
0.00
3.95e-342.06e-3
4.30e-34:200e--
2.986-2+4.08e-2
1.38 + 0.08
Table 3: Diagnostics for the 4-mode Gaussian mixture with K= 103. Eieilepo log ग is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow; MMD is the Maximum Mcan Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
K 10
[o log π
KSD U-stat.
KSD V-stat.
MMD
seconds
FM w/ π samples
-4.22 - 0.04
1.50e-3::6338
1.81e-346.33e-4
3.69e-4+1.84e4
22.3 0.64
MFM Kg- = K
-4.47 + 0.05
3.15e-34:2. 10e-3
3.50e-342.1 10e.3
2.37e 3.-
27.9 + 1.27
MFM lig = 10a
3.61e-34:2.07e-3
3.96e-3442.07e-3
1.05e-348.90e.
39.2 + 1.74
4.48 0.09
3.50e-3.12.22--3
3.86e-3112.22--
1.88e-311.966-3
41.2 1 1.65
MFM Kog 10
-4.44 0.07
3.152:34:2228--3
3.49c-342.28--3
8.13e-444.4ee4
117. 5.65
HTT
1.46 0.06
4.800-313.176-4
5.15e-313.170-3
1.37e-319.65e-
147. 9.44
AT-SMC
4..48 1 0.04
4.07e-3:1..248
4.42e-311.244.-
3.95e-212.90e.22
2.18 1 0.26
Table 4: Diagnostics for the 4-mode Gaussian mixture with K = 5- 103. E log π is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow; MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
C.3 16-mode Gaussian Mixture
Like the 4-mode example, all methods use N = 128 parallel chains for training and 128 hidden dimensions for all neural networks. Methods with a MALA kernel use step size of 0.2, and methods with splines use 4 coupling layers with 8 bins and range limited to [-16, 16). In Table 5 we present results for K = 10 iterations. In Table 6, we provide results for MFM and AT-SMC for K = x 10 learning iterations. In the main text, we present results for K = 5 10 learning iterations for MFM and AT-SMC and K = 10 iterations for all other algorithms, which yields a more comparable total computation time.
19

---

K = 10
Bis log
KSD U-stat.
KSD V-stat.
MMD
seconds
FM samples
-7.09
2.94e-2+1.322.2
2.99-22t1.32e-2
1.89:
6.67 + 0.19
Ki= K
6.93 0.47
1.6f-22:1.14e22
1.71e-2f1.156.22
3.71e:244.8ee3
10.4
MFM Ro 10a
-7.21 0.73
1.00=2+9.31e-3
1.85e-2+94433-3
181e-228.9703
11.4
HTTE
7.34 0.881
2.10e-218.65e-3
2.15e-2.1..7e--
1.74e-218.126-3
13.0 0.83
MFM li
10
+.25
2.95e-2#11003e22
3.01e-2#1.044.2
1.066-2+2.76e-
20.3
w/ HTE
0.88
3.17-22+1.77-2
3.23e-2=1.99e-2
1.30e-2+:3333-3
22.5 1.81
DDS
5.86 0.20
6.65e-3#6.69e-3
0.94e-3:+6.69e-3
1.02=-1+4.10-2
115. + 0.64
NE-MCMC
3.74
1.23e-2.1.71e-22
1.26e-211..72--2
8.05e-311.42--2
67.0 1 12.3
FAB
5.89 + 0.28
4.22e-3-:3.31e3 4.58e-343.33e.3
1.51e-3+
102. + 4.32
AT-SMC
5.91 = 0.07
2.07e-341.03e-3
2.380-341.04e-3
3.72x2244.45e-
1.36 + 0.20
Table Diagnosties for the 16-mode Gaussian mixture with K 10 Bioilepo log is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow; MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95 % confidence intervals over 10 independent runs.
K = 5 10a
1g log π
KSD U-stat.
KSD V-stat.
MMD
seconds
FM w/ samples
-5.74 0.11
2.91e33+1.283 3
3.26e-3+1.24e.
1.35-3+6.66e
22.4 + 1.01
MFM ko = K
-6.09 0.10
3.00e:3+7.87e4
3.34e-3+7.95e-4
1.8%--233.67.3
28.2 + 2.84
MFM ko 10a
-5.90 - 0.08
3.37e-342.00e..
5.74c-342201e--
2.98e-341.370-3
34.8 1.95
w/ HTB
-5.88 0.12
5.43e-31:2.32--3
5.80e-311.33e.1
3.86e-31:1.233-1
38.7 1 2.53
MFM kg = 10
5.98 0.13
3.48--312.07e--
5.86e-3.12.09e-3
2.87c-319.67c4
89.6 1 5.19
w/ HTE
-5.92 - 0.09
9. 9.18e-3:t4.48.
9.57e-3+44.0e.
3.58e:3+1.00e-
110. + 6.77
AT-SMC
5.84 = 0.05
2.096-348.03e-
2.40e-348.56e-4
1.736-245.30e-
2.19 + 0.21
Table 6: Diagnostics for the 16-mode Gaussian mixture with K 5 10a. Bioii log π is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow; MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
C.4 Many Well
We also present a synthetic problem approximating the 32-dimensional Many Well distribution given by the product of 16 copies of the 2-dimensional Double Well distribution [e.g.. 52, 58, 81],
1 2² + constant,
log logp(xr: (11, x2)
- 1 Gz
(41)
where each copy of the Double Well is evaluated on different pair of the 32 inputs. The 32- dimensional Many Well has 2 = 65536 modes, one for each possible choice mode each the 16 copies of the double well. We can obtain exact samples from the Many Well by sampling each independent copy of the Double Well.
Like previous synthetic examples, all methods use N = 128 parallel chains for training and 128 hidden dimensions for all neural networks. Methods with MALA kernel use a step size of 0.1, and methods with splines use 4 coupling layers with 8 bins and range limited to [-16, 16). Table presents results for
10 iterations. In Table 8, we provide results for MFM and AT-SMC for K = 5 x 103 learning iterations. In the main text, we present results for K = 5 10 learning iterations for MFM and AT-SMC and K = 10 iterations for all other algorithms, which yields more comparable total computation time.
C.5 Field system
The stochastic Allen-Cahn equation is defined in terms of random field ø |0, 1] + satisfying the following stochastic partial differential equation [e.g., 24, Section V]:
d① 82 Øt ds² a - )²) /23 128-in(t,, s).
(42)
20

---

K = 10
1 P0 log
KSD U-star.
KSD V-saat
MMD
seconds
w/ samples
86.6
17.4 6.24
6.25
1.21
0.31
Kq K
101. 0.70
0.12
0.17
2.27:8=1.377.
11.1
MFM RQ- 10a
0.70
0.12
0.13
2.28o-8=1.370-
120. 17.6
w/ HTE
101.1..70
0.11
2.26e-811.3768
35.6 1.95
MFM ko 10
101.
0.11
0.76
2.23e-8=1.37-
+9.0
w/ HTE
101.
0.11 0.09
0.76
2.27c-8=1.66c-
259. 17.6
DDS
139.
0.13 0.14
0.61 0.13
1.49e-5=2.03e-
227. = 0.91
NF-MCMC
1.61
1
2.29 0.66
4.796-7-2.41o-7
184. 1 44.1
FAB
137.
4.79e.3+2.55ee
4.32e:114.144.2
3.87e-7=2.90-7
304 3.64
AT-SMC
130. 2.93
0.02 0.03
0.37 0.03
1.75e-5=1.19e-
1.35 0.36
Table Diagnosties for the many well with K 103. oi #Po log the Monte Carlo approxi- mation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow; MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
Bo log π
KSD U-stat.
V-stat.
MMD
scconds
FM w/ samples
98.6 + 5.32
2.57 + 4.07
4.56 + 4.07
-1 -1.13e-9:2.88
25.6 0.48
MFM lig = K
101. + 0.64
1.02e-119.03e2
7.64e:119.78e22
2.28e-8+:1.8
29.7 1.03
MFM kQ 10a
101. - 0.63
1.02e-149.05e>
7.61e-149.7de-2
2.28o-8+1..5e.
587. 27.0
w/ HTE
102. 0.78
1.02e-148.32e2
7.62e-1#8.91e-2
2.27c-8+11.8e-
154. 7.51
Ko
102. 0.81
9.03e-2.1.224e-2
7.63e-118.15e-2
2.27c-8.11.57e-8
5130. 104.
2.20
3.86e-142.669e
1.05 1125:...0 0.27
2.21e-811..57.8
1190. 27.6
AT-SMC
130. 3.17
2.44e-245.99-2
5.79c-146.76e-2
1.52e-5#1.00e-
2.59 0.27
Table 8: Diagnostics for the many well with K 5 103, pe log is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow: MMD is the Maximum Mean Discrepancy between real samples from the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
where a > 0 is parameter, 33 is the inverse temperature, s E [0, 1] denotes the spatial variable, and 17 is spatiotemporal white noise. We impose Dirichlet boundary conditions throughout, that 6(s 0) o(s = 1) O.
The associated Hamiltonian, reflecting a spatial coupling term penalizing changes in ø, takes the form:
do 0s
1 4a
(s))
U. [b) 3
ds.
(43)
2
At low temperatures, this coupling induces aligmment of the field in either the positive or negative direction, leading to two global minima, Ø+ and é_, with typical values of +1.
For this example, all methods for all neural networks. Methods with a MALA kernel use a step size of 0.0001, and methods with splines use 8 coupling layers with bins and range limited [-5, 5). Results for K 10s learning iterations are presented in Table In the main text, we present results for kq = 102 for MFM.
N 1024 parallel chains for training and 256 hidden dimensions
Interestingly, in digh-dimmenioonl problems such this and the following, the version of the algorithm using Hutchinson's 's trace estimator [27, 36) to calculate the MH acceptance probability has little apparent effect on the approximation quality. It does, however, have significant impact on the computation time.
C.6 Log-Gaussian Cox process
The original 10 x 10 square meter plot standardized to the unit square. We discretize the unit square [0, 1|2 into a M 40 x 40 regular grid. The latent intensity process A = (Am me is specified as Am exp(Xx (X,),, where X {Xm)mEM is Gaussian process with a constant mean po E IR and exponential covariance function En [m, n) o² exp (-Im r2l((403)) for m, M,
21

---

K 10
1 logm π
KSD U-stat.
KSD V-stat.
seconds
MFM KiQ K
74.7 + 5,67
2.61 2.00 2.00
20.9 + 2.49
52.3 1.23
MFM Ki = K/10
69.6 6.07
2.50
2
61.7
4.37
w/ HTE
6.11
2.17
21.0
53.6 1.33
MFM KQ = K/100
4.19
2.34
2.39
180
w/ HTE
11.6
3.84
2.26
71.5
DDS
17.4
15.2
18.0 36.9
2400 8.65
NF-MCMC
9.62
548 + 325
549 325
2000 + 15.6
FAB
0.14
0.14
1.78
0.42
3880 7.19
AT-SMC
63.5 + 9.91
1.61 + 2.33
18.4 + 2.35
4.13 + 0.30
Table 9: Diagnostics for the field system with K 10 Bioilaro log is the Monte Carlo approxi-
mation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernel Stein Discrepancy U- and V-statistics between the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
i.. X (Wool Σo) for 1 [L.............. 1 F2 with dimension d 1600. The chosen parameter values are o² = 1.91. 3 = 1/33,, and log 126) - 22, corresponding to the values estimated in [53]. The number of points in each grid cell Y J E N40x40 are modelled as conditionally independent and Poisson distributed with means a.m.
(() = II
exp(Im3m - exp((mm)).
(44)
me[1:30]²
where a = 1/40² represents the area of each grid cell. For this example, all methods use N 128 parallel chains for training and 1024 hidden dimensions for all neural networks. Methods with MALA kernel use a step size of 0.01, and methods with splines use 8 coupling layers with 8 bins and range limited [-10, 10). Results for K 10a learning iterations are presented in Table 10. In the main text, we present results for kQ = 10 for MFM. We were unable to run NE-MCMC and FAB for K 1 iterations because of memory issues; instead, we present results for K 10 iterations only for the models using discrete normalizing flows.
K = 10a
Bioiap log
KSD U-stat.
KSD V-stat.
seconds
MFM KQ K
1960 = 10.8
1.13e-14-5.88-
28.1 + 0.246
1.19
MFM KQ
100
1960 = 10.8
1.14e-1=5.13e.
28.1 0.241
w/ HTE
1960 = 12.2
1.12e-1+4.93e-2
28.1 + 0.236
143 14.5
MFM Kg
102
1960 = 12.6
1.15e-115.126-2
17300 9970
w/ HTE
1960 = 11.1
28.1 1.15e-115.17--2
3
DDS
1850 = 8.59
7.222-242.240-2
24.7 0.08
3260 - 8.41
NF-MCMC
1410 = 53.8
11.8 7.55
89.0
215 - 46.4
FAB
3070 = 80.7
1.55e-1+6.12--2
52.3 + 2.02
1040 + 2.78
AT-SMC
1910 4.21
1.39e-2+1.14e-2
25.0 + 0.12
6.11 + 0.44
Table 10: Log Gaussian Cox point process diagnostics for K 10 where Eiolaaro log is the Monte Carlo approximation of the log-target density using the learned flow to generate samples; KSD U-stat and V-stat are the Kernelized Stein discrepancy's U- and V-statistics between the target and samples generated from the learned flow. Results are averaged and empirical 95% confidence intervals over 10 independent runs.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims made the abstract and the introduction provide an accurate
reflection of the theoretical, methodological, and experiment contributions in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The final section includes discussion of the limitations of this work, including the computational efficiency of the proposed method, situations in which the proposed method may be outperformed by other baselines, and hyperparameters which may strongly influence the performance of the proposed method.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
23

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
Our theoretical result (Proposition 3.1 in Section 3) is accompanied by full
set of assumptions and complete proof (in Appendix B). Any results upon which the proof relies are properly cited. In addition, all the theorems, formulas, and proofs the paper are numbered and cross-referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper includes full details of the algorithm used to obtain the experimental results in the paper, including all of the hyper-parameter settings for each of the different numerical experiments. Our code is also made publicly available on GitHub (see Section 5).
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
24

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our code is made publicly available on GitHub (see Section 5).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/ public/ souuidee//ooeSuuiisssiiii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. htabbiic/guddee//dddubbibiiii (oooeoubmssiiiono ssiiisolioyyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all of the training and test details, including hyperparameter settings, train-test splits, choice of optimizer, etc., necessary to understand and reproduce experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report error bars for all of our numerical experiments (see Section 5). In the text, we provide full details regarding how these error bars are calculated, well as the factors which vary between each experimental run.
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We include full details of the type of compute workers used in the main text (see Appendix C). We also include details the running time for each of the main numerical experiments, for both proposed method and the relevant baselines.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: / /tt.rinuu..pp/// cc cc/ppblic////eecccocccceccblpclt /Ethiiiiiie ceciicellellleee
Answer: [Yes]
Justification: The research conducted in the paper conforms with NeurIPS Code of Ethics in every respect, including with regards to Potential Harms Caused by the Research Process, Societal Impact and Potential Harmful Consequences, and Impact Mitigation Measures. All of the authors have read and reviewed the NeurlPS Code of Ethics to ensure that the research conducted in this paper conforms with the guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
26

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper contains foundational research on new method for approximate statistical inference. It is not tied to any particular applications and, as such, these are not directly discussed in the main text.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not include any datasets or models associated with high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
27

---

The creators and original owners of assets used in the paper (e.g., code for relevant baselines) are properly credited (see Section 5). We both cite the original papers that produced these assets, and include URLs to the code.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcon thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release any new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
28

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments any research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "HqdI8f8jen", "reviewer": "Reviewer_YCu1", "summary": "This paper aims to use continuous normalizing flows (CNFs) to define the proposal distribution in a MCMC framework. While the use of flow models for MCMC proposals is not entirely new, this paper introduces an interesting training procedure that iteratively updates the learned CNF model while performing MCMC. Existing works either (i) make use of an importance weighted training objective, or (ii) first perform long-run MCMC to obtain high-quality samples. Instead, this work makes use of the Flow Matching training objective to fit to its current MCMC samples, where samples are obtained from a MCMC involving the CNF itself.", "questions": "Generally, while the proposed MCMC algorithm as a whole makes sense, it is composed of multiple components. In order to better understand each proposed component, it would be good to have ablation experiments analyzing the behavior of each component.\n\n- Given that there is a series of local MCMC updates, it would be good to compare against MCMC with just the Q transition kernel. This can help answer what exactly is the change in improvement from including a flow-informed transition kernel.\n\n- I think what would be really interesting is a plot of the acceptance rate between the P (flow-informed) and Q (local) kernels. I would imagine that at the beginning P has poor acceptance rates because the CNF is poorly trained, while after the MCMC chain is run for long enough times, its acceptance rates are much higher. Meanwhile, the acceptance rate of Q should be constant but perhaps lower than the acceptance rate of P? Regardless, it would be interesting to see such a plot to better understand the behavior of the proposed algorithm.\n\n- If you turn off the annealing, how much worse does the approach perform?\n\n### Minor comments\n- The likelihood notation L(D|x)  when discussing annealing (Eq 16) is not used anywhere else (and also not used in the experiments since they directly define pi(x). I felt this to be confusing because as it is currently written, it may seem that the annealing trick is restricted to only Bayesian inference settings. I would imagine the annealing simply takes L(x) = pi(x) / pi_0(x) for some pi_0(x). It might be good to clarify this?\n- There are 3 instances of \"NeutraMCMC\" which should probably be \"Neural MCMC\" ?", "limitations": "The paper is written clearly with objective analyses, with no excessive beautification of the proposed method. It's nice.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Overall, I feel this paper is a combination of good solid ideas that individually are not entirely new, but packaged in a way that makes sense.\n- The writing is also clear and to the point.\n- Experiments seem to be carried out on some standard benchmark potential functions.\n- The paper repeatedly mentions that the proposed method is favorable to competing neural MCMC approaches based on wallclock time, being significantly faster in some cases.", "weaknesses": "- I feel the set of experiments is \"too standard\". Given that Bayesian inference is such a long field, it would be make a paper much stronger if it can show meaningful improvement on real unsolved problems where exploration is key.\n- There is a need to add ablation experiments that provide better intuition regarding what benefit the flow-informed Markov transition is doing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to the reviewer for their positive remarks and their constructive feedback. We provide a detailed point by point response to their review below.\n\n**Weaknesses**\n- **I feel the set of experiments is too standard...** Thanks for raising this comment. While we agree with the reviewer that the inclusion of real-world unsolved inference problems would further strengthen this work, we believe that this is beyond the scope of the paper. Indeed, we would argue that the set of numerical experiments presented are very comparable to other recent works in the literature [1,2,3]. Not only this, but they demonstrate the robust performance of our method in a range of challenging examples (e.g., multimodal targets, high-dimensional targets, etc.). \n- **There is a need to add ablation experiments...** We agree with the reviewer that some additional ablation experiments would help to elucidate the contribution of different components of our algorithm. In our latest revision of the paper, we now include several such experiments. We provide full details in our responses to the points raised in the 'questions' section of the review (see below). \n\n**Questions**\n- **It would be good to compare against MCMC with just the Q transition kernel...**  This comparison is actually already present for all of our existing numerical experiments. In particular, results for running the algorithm with just the Q transition kernel correspond to the results with k_Q=K (see, e.g., Table 3 - 8 in Appendix C). We have now added an additional remark at the start of the numerical experiments section to clarify this, and moved these results from the appendix to the main paper. \n- **I think what would be interesting is a plot of the acceptance rate...** We are grateful to the reviewer for this suggestion, and agree that it would be interesting. The acceptance rate of Q is targeting 1 since we estimate expectations, e.g. in (14), using the current ensemble of particles, rather than a single long chain (see L244). The acceptance rate of P certainly increases as the algorithm runs; in particular, it often jumps to high values, before stabilising to a lower value. We can include plots to illustrate this behaviour in an appendix in the revised version of the paper.\n- **If you turn off the annealing, how much worse does the approach perform?** Thanks to the reviewer for raising this interesting question. The relative performance of the algorithm with or without annealing is dependent on (i) the task at hand and (ii) the initialisation of the particles. In particular, if the initialisation of the particles doesn’t cover one or more of the target modes, then these modes are much less likely to be explored without annealing. In general, without the annealing strategy, one shouldn't expect the algorithm to find states in modes distinct from initialisation. See also the related discussion in [3, Section IV.C]. On the other hand, when at least one particle is initialised in each of the the metastable basins (i.e., the modes) of the target distribution, annealing is much less important. To confirm this behaviour, we have run our algorithm with and without annealing for one of the multimodal two-dimensional experiments described in Section 5. These results, which indeed confirm the behaviour discussed above, are included in an appendix in the revised version of our paper.\n\n- **The likelihood notation...** The use of the likelihood notation in (16) is indeed not used elsewhere, and we agree with the reviewer that this notation may suggest the use of annealing is only restricted to the setting of Bayesian inference. As the reviewer correctly points out, this is not the case. We have now rewritten this section in to cover the general setting. \n- **There are 3 instances of neutraMCMC...** The term \"NeutraMCMC\" is actually correct here. This terminology originates in [4]; see also [5, Section 2.2]. \n\nMany thanks again to the reviewer for their useful feedback. We hope that we have been able to fully answer their questions and that our responses will increase their confidence in this paper. \n\n**References**\n\n[1] Midgley et al., 2023. Flow Annealed Importance Sampling Bootstrap. ICLR 2023.\n\n[2] Vargas et al., 2023. Denoising Diffusion Samplers. ICLR 2023.\n\n[3] Gabrie et al., 2022. Adaptive Monte Carlo Augmented with Normalizing Flows. PNAS.\n\n[4] Hoffman et al., 2019. Neutra-lizing Bad Geometry in Hamiltonian Monte Carlo using Neural Tranport. AABI 2018.\n\n[5] Grenioux et al., 2023. On Sampling with Approximate Transport Maps. ICML 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KTNOzltg3Z", "reviewer": "Reviewer_eFeW", "summary": "The authors propose a novel method for incorporating flow matching with MCMC; entailing constructing a Markov kernel as a mixture of regular MCMC step and flow step (from data space (\\pi_1) to prior (\\pi_0, typically Gaussian) and back with some added noise. The flow guarantees a likelihood and can hence be used for accept/ reject corrections. \n\nIn this sampling setting, one does not have access to samples from the data and hence the flow is trained from MCMC samples jointly within sampling.", "questions": "My understanding is that the flow update pushes samples into a simpler prior / latent type space then back to the data space. Is it possible to include Markov transitions in the prior space i.e. in the simpler (Gaussian) distribution, which are easier and encourage non local updates (in data space)? The addition of Gaussian noise is a bit like this I suppose but has anything else been considered?\n\n\nDoes this actually require flow matching? I feel any diffusion model would also have be usable here, it can be trained in the same way and one can use the probability flow ODE to have tractable likelihoods and if needed similar to flow matching. There are many similarities between the two models, especially for Gaussian marginal, but diffusions seem to be better performing generative models still, see e.g. Lipman 2022 (cited) vs Karras 2022 (https://arxiv.org/abs/2206.00364) in terms of generative performance. If just using Langevin dynamics without accept/reject correction then the added stochasticity of diffusion samplers may also help, plus the benefit of Langevin dynamics corrector steps along the diffusion paths could possibly be used (Song 2021).", "limitations": "See weaknesses.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "I think the authors did a great job on this paper:\n- The method is simple and well explained\n- Experiments are thorough with appropriate baselines including recent DDS, good performance and code is available\n- The authors clearly articulate limitations, which I appreciate\n- Prior and related work is discussed and not hidden", "weaknesses": "I worry how long it takes the flow network to converge. Simulating the flow back and forward to generate samples at regular frequency during training could be quite slow if involving a large number of network evaluations for the flow. And if the MCMC locally does not mix then the flow will not see many samples. I understand tempering may help resolve this but at what cost?\n\nSimilarly, I imagine the samples from the flow could be quite far from the target distribution if the flow was not pre-trained or initialized to be close to some nearby distribution.\n\nThe idea itself is quite straightforward. This is not necessarily a weakness in itself, I am surprised something similar has not been considered which am unaware of. I understand there are many similar schemes that are cited using non-local Markov updates with normalizing flows (discrete), just not using flow matching (e.g. https://arxiv.org/abs/2105.12603) and other works from the same authors. I appreciate the authors have discussed and cited these. This however limits the novelty of the methodological contribution slightly in my opinion.\n\n\nMinor presentation:\n- I struggle to understand from Figure  what the ground truth samples should look like. I assume like the samples from the author's method. It would help readers to show the density to know what the ground truth should be, similar to the other experiments.\n- Similarly Table 2, show in bold the top performing items to help readers\n\nSome other relevant works using diffusion / flow for sampling:\n- Target score matching, Bortoli et al 2024 https://arxiv.org/abs/2402.08667\n- Iterated Denoising Energy Matching for Sampling from Boltzmann Densities, Akhound-Sadegh et al 2024\nand although cited the work of Phillips et al 2024 (already cited) could be used as a baseline given code is available (also in jax)\n- CRAFT and AFT  (Annealed flow transport Monte Carlo, 2021) may also be good baselines to include, code is available in jax for those too.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback and constructive comments. We provide a detailed point by point response to their review below. \n\n**Weaknesses**\n- **I worry how long it takes the flow network...** The reviewer is correct that flow-informed MCMC samples are relatively expensive to compute, due to the need to simulate the CNF. However, the rest of the algorithm is very inexpensive. Moreover, our empirical results illustrate that we do not require a very large number of flow-informed transition steps for the flow network to converge to a good approximation of the target distribution, even when the flow network is not initialised close to the target distribution. It is worth noting that, given particular computational constraints, one can balance the (more expensive) flow-informed global updates with the (much less expensive) local updates by varying the value of $k_Q$ (see Appendix C for some indicative results). \n- **Similarly, I imagine samples from the flow could be quite far from the target...if the flow was not pre-trained...** The reviewer raises an interesting point with regards to pre-training of the flow, or initialisation of the flow network. In cases where one has some a priori information about the target distribution (e.g., the location of the modes), our algorithm could instead use a reference distribution which utilises this information (e.g., a mixture of Gaussians, centred at these modes). This is possible using the general formulation of flow matching given in, e.g., [1], which allows for the arbitrary specification of reference and target distributions and may further improve the convergence rate of the flow network in our algorithm. However, in all of our experiments, where a standard Gaussian was used as the reference density, (samples output by) the flow network converged to the target within a reasonable number of iterations.\n- **Minor presentation...** Thanks for pointing out these issues. We have now included the ground truth samples in this figure and ensured that the best-performing method is shown in bold in Table 2 (and all other tables).\n- **Some other relevant works...** Thanks for pointing out these papers. We have now added [2] and [3] to our discussion of related work. Regarding baselines for our numerical experiments, we would argue that the baselines we have already included cover the most relevant methods in the literature. Nonetheless, we agree that including results for the methods in [4, 5, 6] would also be useful. Given the time constraints, we have not been able to re-run all of our numerical experiments for these methods during the rebuttal period. However, we will look to include these experiments in the revised version of our paper. During the rebuttal period, we *have* obtained results for all of our numerical experiments for an adaptively tempered SMC scheme, which are included in the PDF in our global response to all reviewers. \n\n**Questions**\n- **My understanding is the flow pushes samples into a simpler...** The reviewer is correct that other Markov transitions are also possible in the latent space, and we have now added a remark in Section 3 to ensure this is clear. For example, there is also the option to do independent MH (instead of RWMH). However, we found that the added noise of the RWMH transition kernel helped with exploration of the state space, and avoiding mode collapse. Although, due to space constraints, we are unable to include these results in the PDF attached to our global response to all reviewers, we will include this comparison in an appendix in the revised version of the paper. Other options that use local gradient information are also possible, but these would require even more evaluations of the pullback target density (and its gradient). This requires numerically solving the ODE, which is by far the most expensive part of the algorithm.\n- **Does this actually require flow matching?** The reviewer is correct that one could use a diffusion model within our algorithm, using the probability flow ODE to evaluate the likelihood of the samples under the model. On the one hand, this would allow for other sampling schemes (e.g., stochastic samplers, predictor-corrector schemes, etc.) as the reviewer suggests. It would certainly be interesting to investigate further whether the use of such samplers has any benefit in terms of algorithmic performance, although we feel that this is beyond the scope of this work. On the other hand, the use of (conditional) flow matching arguably allows for more flexibility in terms of the specification of the probability path, as well as arbitrary source distributions [1].\n\nMany thanks again to the reviewer for their useful feedback. We hope that we have been able to fully answer their questions and that our responses will increase their confidence in this paper. \n\n**References**\n\n[1] Tong et al., 2024. Improving and generalizing flow-based generative models with minibatch optimal transport. TMLR. \n\n[2] Bortoli et al, 2024. Target Score Matching. arXiv.\n\n[3] Arkhound-Sadegh et al., 2024. Iterated denoising energy matching for sampling from Boltzmann densities. ICML 2024.\n\n[4] Phillips et al, 2024. Particle Denoising Diffusion Sampler. ICML 2024.\n\n[5] Arbel et al., 2021. Annealed Flow transport Monte Carlo. ICML 2021.\n\n[6] Matthews et al., 2022. Continual Repeated Annealed Flow Transport Monte Carlo. ICML 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ik0y0opMNe", "reviewer": "Reviewer_HJhZ", "summary": "A method called Markovian flow matching (MFM) for training neural ODEs (continuous normalizing flows) by flow matching to sample from distributions given as unnormalized density functions is proposed. The method obtains samples on which to perform FM training by an MCMC procedure that alternates two kinds of kernels: Metropolis-adjusted Langevin in the target space (as has been done in past work) and Gaussian proposals in the latent space, i.e., the pullback of the target by the learned ODE. An annealing scheme is additionally used. MFM is evaluated on toy low-dimensional tasks, a synthetic density from statistical physics, and a high-dimensional data-derived density (log-Gaussian Cox process).", "questions": "Minor:\n- L25: You probably want to specify \"a new sample $y$\" and in the subsequent lines clarify that the \"mild conditions\" are also needed for the target to be the *only* stationary distribution.\n- L77: Dash between \"change\" and \"of\" instead of hyphen.\n- L86: Wrong placement of parentheses.\n- L246: I do not believe [78] introduced this parametrisation -- it was in [80] (and possibly in earlier work as well).\n- I don't understand equation (18), which involves $x_{65}$ in the first sum although the dimensionality is 64. Is there is mistake, or should the indices be interpreted modulo $d$?\n- L308: Typo in \"comparable\".", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea to do MCMC in the latent space of a CNF by pullback along the flow map is original, as far as I know.\n  - There are two general families methods for training continuous-time models to sample Boltzmann densities: using maximum likelihood on (possibly reweighted) approximate samples (e.g., FAB and the proposed MFM) and using distribution-matching objectives (e.g., DDS, PIS), as well as hybrid approaches.\n  - It's interesting to see progress on making methods of the first kind efficient, and to see them combined with CNFs. In the past, e.g., in [Tong et al. \"Improving and generalizing flow-based generative models\", D.2], FM has been used with MCMC, but only in the target space.\n- No serious problems with the writing, including the global structure of the paper and the exposition of preliminaries.", "weaknesses": "- Some math bugs (imprecise exposition):\n  - L72: $v_t$ being **any** time-dependent vector field does not imply existence of a diffeomorphic flow map (even if it is continuous in $t$, if that is how we interpret \"runs continuously in the unit interval\"). One also needs integrability conditions.\n  - L75: $p_0$ has to be strictly positive for the path to take values in $\\mathbb{R}^+$ and for the discussion the discussion that follows to make sense.\n- Comparison with SDE models: Approximate (MCMC) samples can also be used to train diffusion models (neural SDEs):\n  - First, this could be done either by minimising a variational bound on the log-likelihood, similar to what is done here for CNFs -- is it possible to do such a comparison? At present the SDE baseline (DDS) is not using MCMC, only \"on-policy\" forward exploration.\n  - Second, methods such as that in [Sendera et al. \"Improved off-policy training of diffusion samplers\"] use MCMC on the target density to obtain samples for training with a distribution-matching, not maximum-likelihood, objective.\n- The main weakness is that the experiments are not very comprehensive or convincing:\n  - The 2D GMMs are toy problems; all methods, including DDS, should find the modes when appropriately tuned on such problems, and so it is hard to draw any conclusions from the results.\n  - On the field system, MFM, although much faster than FAB, is performing **much** worse in the metrics used.\n  - On LGCP: \n    - The use of this task for benchmarking samplers of Boltzmann densities is questionable in the first place, since there is no known ground truth (even the exact normalising constant is diagreed upon) -- see, for example, the aforementioned [Sendera et al., B.1] for a discussion of inconsistent or irreproducible evaluation in past work on this dataset.\n    - Why use the nonstandard metric (KSD)? Most past work has used estimates of the log-partition function, which here is reported in the appendix.\n    - On that more standard metric, MFM is significantly underperforming (Table 8).\n- On both the field system and LGCP, how are we to know if the proposed method is useful on this problem without a point of comparison? I would strongly suggest a long-run HMC, MALA, and SMC (for example, run for the same wall time as the shortest and longest of the ML methods evaluated) as baselines.\n- I would also strongly suggest evaluations on a more comprehensive set of benchmarks that are common in past work on samplers of Boltzmann densities (as in the FAB and DDS papers, among others).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks to the reviewer for their thorough engagement with our work and for their constructive feedback. We provide a detailed point-by-point response below.\n\nWe hope that we have been able to address the reviewers' questions, and that they will consider increasing their score based on our response and changes to the paper. We look forward to engaging with them during the discussion period, and welcome any further questions.\n\n**Weaknesses**\n- **Some math bugs...**  We agree we should have been more precise here. We have now amended L72 and L75 to include the required conditions, as suggested by the reviewer.\n- **Comparison with SDE models...** Thanks for pointing this out. With regards to *minimising a variational bound on the log-likelihood using (MCMC) samples,* the problem here is that we need to solve the ODE numerically both for generating MCMC samples and evaluating the gradient of the log-likelihood. This means such a method would be very inefficient since the ODE solver is the most expensive part. This is one of the motivations for using flow matching; we don't need to solve the ODE, just evaluate the vector field. With regards to *off-policy training*, we were unaware of the literature the reviewer cited [1]. We agree that it would be nice to (empirically) compare our method with an off-policy method, although this was not possible during the rebuttal period. We have updated the related work section to include this reference and will aim to include this as a benchmark in the revised version of the paper.\n- **The main weakness is that the experiments are not very comprehensive...** \n\t- With regards to the two-dimensional experiments, similar experiments are a standard benchmark in similar papers, including recent works such as [2,3]. In the 16-mode Gaussian mixture experiment (Section 5.2), we were unable to find a hyperparameter configuration in which DDS captured all of the modes. \n\t- With regards to the field system, while FAB does outperform our method with respect to KSD, FAB does *not* in fact capture both of the modes (see Figure 3).  Indeed, of the considered methods, only our approach (MFM) captures both of the modes in this example. In addition, the runtime for MFM is *significantly* lower than that of FAB (see the discussion in Section 5.3). \n\t- With regards to LGCP, we were unaware of the discussion regarding the suitability of this task as a benchmark for sampling methods [1, Section B.1], and are grateful to the reviewer for bringing this to our attention. It is worth noting that this benchmark has been used widely in several other recent papers in this field, hence its inclusion in our paper [e.g., 4]. In terms of our use of the KSD (and the MMD), we do not think these are particularly non-standard metrics. Their use is largely justified by [5] and subsequent work; see also recent works including [6,7,8] or [9] where the KSD or MMD are used as metrics for assessing sample quality. Regardless, in our updated results, included in the PDF in the global response, our method in fact performs similarly to SMC when measured in terms of $\\\\smash{\\\\mathbb{E}_{[\\\\phi_1^{\\theta}]{\\\\#}p_0} [\\log \\\\pi]}$. These results are obtained by including a time-dependent weight in our neural network for the vector field (as in DDS) and illustrate that there is further room for improvement in our method by exploring other neural network architectures. This is a key strength of our approach: it allows the flexibility to use any vector field architecture (since the architecture is entirely unconstrained for CNFs), which is not the case for DDS. We leave a more detailed investigation of this to future work.\n- **On both the field system..., how are we to know if the proposed method is useful...without a point of comparison?** We agree that the inclusion of an additional 'ground truth' benchmark would be useful here. We have now generated additional results for these experiments using (adaptively tempered) SMC. These results are included in the PDF in our global response. We will include these results in the revised version of the paper, with the caveat that there is some debate on the quality of such a benchmark in such a high-dimensional problem. \n- **I would... suggest evaluations on a more comprehensive set of benchmarks...**  We have now also added results for the 'many wells' experiments in [2]. With the inclusion of the experiment, we would argue that our numerical experiments cover a significant number of the experiments considered in other recent papers of a similar flavour (e.g., FAB, NF-MCMC, DDS). In particular, the 16-mode example and the 'many wells' experiment are considered in [2], the field system example in [10], and the LGCP in [5].\n\n**Questions**\n- **Minor**. Thanks for pointing these out. We have amended the manuscript as per all of your suggestions. Regarding (18), your second suggestion (i.e., that the indices should be interpreted modulo $d$ is correct). We have added a short remark to clarify this.\n\n**References**\n\n[1] Sendera et al., 2024. Improved Off-Policy Training of Diffusion Samplers. arXiv.\n\n[2] Midgley et al., 2023. Flow Annealed Importance Sampling Bootstrap. ICLR 2023.\n\n[3] Arkhound-Sadegh et al., 2024. Iterated denoising energy matching for sampling from Boltzmann densities. ICML 2024.\n\n[4] Vargas et al., 2023. Denoising Diffusion Samplers. ICLR 2023.\n\n[5] Gorham et al., 2017. Measuring Sample Quality with Kernels. ICML 2017.\n\n[6] Nemeth et al., 2021. Stochastic Gradient Markov Chain Monte Carlo. JASA.\n\n[7] Chehab et al., 2024. A Practical Diffusion Path for Sampling. Chehab et al., 2024. ICML 2024: SPIGM Workshop.\n\n[8] Maurais et al., 2024. Sampling in Unit Time with Kernel Fisher–Rao Flow. ICML 2024.\n\n[9] Blessing et al., 2024. Beyond ELBOs: A Large Scale Evaluation of Variational Methods for Sampling. arXiv.\n\n[10] Gabrie et al., 2022. Adaptive Monte Carlo Augmented with Normalizing Flows. PNAS.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wMvvoETvpR", "reviewer": "Reviewer_Sj6c", "summary": "The proposed method falls in the wider category of models that integrate MCMC methods with normalizing flows. Specifically, the method expands on the neutra-MCMC model, where the key idea is to run MCMC in a simpler reference space rather than in the (more complicated) target one. The main difference is that previous work uses discrete normalizing flows while the proposed approaches employs the Flow Matching framework and hence relies on a continuous velocity field, which is provided with a continuous Normalizing Flow.", "questions": "1. Some work analyzes the performance of methods combining MCMC with NFs methods in terms of (i) how easy is it to sample from multi modal distributions and (ii) scalability with dimensions [1]. They found that neutra-MCMC (similarly to the proposed method) is efficient in multi-modal settings but struggles more in high dimensions, compared to methods that use Flows as proposal for MH. Did you observe a consistent behaviour with their analysis? For instance I would have expected NF-MCMC to perform better in the Log-Gaussian Cox experiment. Or did the proposed method outperform \"flow-MCMC\" methods also when inflating the dimensions even more? (e.g. in the experiments in Section 5.3 and 5.4)\n2. The use of CNF are motivated by the availability of the velocity field, which is required by the Flow Matching objective. However, other neutra-MCMC methods work very well also with discrete NFs. CMF are definitely more expressive than standard discrete flows that are commonly used in the literature (mainly real NVP), but one could also use more powerful discrete NFs like [2]. I was wondering whether the authors could discuss the benefits of the proposed approach compared to neutra-MCMC with discrete NFs.\n3. The proposed method uses a hybrid between local and global updates, similarly to [3] (which is cited in the paper). The main difference it the use of a CNF (for the Flow Matching objective) instead of NFs. So I think it would be very relevant to compare the proposed method against [3]. Also the authors of [3] use very naive layers (real NVP, which consist of basic shift and scale transforms), while more flexible layers like [2] might exclude that the difference in performance is to be connected to the expressiveness of the flows. Similarly, the NF-MCMC uses very simple NFs layers (always real NVP).\n4. Even though the Flow Matching objective could allow to learn complicated distributions, in the end the authors use a very simple velocity field to transform the conditional distribution. Doesn't this limit the expressiveness of the learnt distribution?\n5. In line 218 do the authors mean FAB instead of DDS?\n\n\n[1] Grenioux et al, On Sampling with Approximate Transport Maps, ICML 2023\n\n[2] Perugachi-Diaz et al., Invertible DenseNets with Concatenated LipSwish, NeurIPS 2021\n\n[3] Samsonov, Local-Global MCMC kernels: the best of both worlds, NeurIPS 2022", "limitations": "Limitations are clearly stated.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- the authors propose an interesting way to use the Flow Matching framework in the context of Flow-enhanced MCMC samplers\n- I also found interesting the use of a sample-based annealing scheme (which is however not novel). The authors claim it works particularly well for multi-modal distributions, which is definitely a very relevant setting", "weaknesses": "- the novelty aspect of the proposed method seem limited to extend current formalism of MCMC - Normalizing Flows (NFs) methods (specifically neutra-MCMC) to the specific architecture of Continuous Normalizing Flows (CNF). However, I don't see a priori why the (continuous) CNF should be preferable discrete NFs (which are even more expressive than CNF [1]) if not because the flow matching objective requires access to the velocity field provided by CNF\n\n[1] Perugachi-Diaz et al., Invertible DenseNets with Concatenated LipSwish, NeurIPS 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their positive comments, and for their detailed and constructive feedback. We provide a point by point response to their review below. \n\nWe hope that we have been able to address the reviewers' questions, and that they will consider increasing their score based on our response and changes to the paper. We look forward to engaging with them during the discussion period, and welcome any further questions.\n\n**Weaknesses**\n- **The novelty aspect of the proposed method seem limited...** - We are grateful to the reviewer for pointing out [1], although we would contend that [1] does not show discrete NFs are more expressive than CNFs. While the residual LipSwish architecture in [1] is more expressive than the standard coupling flow used by discrete NFs, it is still constrained to a specific architecture which cannot be freely adapted to a specific model of interest. A significant strength of our method is that it can use any architecture and, in particular, one which can be tailored to the target distribution at hand. This being said, we agree that an interesting avenue for future work would be a more detailed evaluation of how CNFs perform in comparison to the most expressive discrete NFs (e.g., LipSwish).\n\n**Questions**\n\n**1.** Thanks for highlighting the recent benchmarking paper [2]. This paper did indeed influence our choice to compare our method to NF-MCMC rather than a neutra-MCMC method (e.g., transport elliptical slice sampling [3]). In particular, as the reviewer notes, [2] shows that NF-MCMC generally scales better with dimension than neutra-MCMC. Given this, we thought that NF-MCMC would be a fairer comparison in high-dimensional experiments. Interestingly, in our experiments, we found that NF-MCMC scaled worse than FAB, as well as worse than MFM and DDS. We suspect that MFM (our method) outperforms NF-MCMC in these high-dimensional settings as MFM is not a \"pure\" neutra-MCMC method. In particular, we combine a Markov transition kernel which targets the pullback of the target measure under our CNF (similar to other neutra-MCMC methods) with a local transition kernel which targets the target measure directly (similar to, e.g., [4]) and an adaptive annealing strategy.\n\n**2.** As noted in our response above (see \"weaknesses\"), one strength of our method (and, more generally, of CNFs) is that the neural network architecture is entirely unconstrained. In particular, there is the possibility of tailoring the architecture to the model (i.e., the target distribution) at hand, since we are not constrained to a specific Jacobian. This is not the case for discrete NFs,  even more expressive discrete NFs with more exotic architectures such as the one proposed in [1].\n\n**3.** We agree with the reviewer that [5], which combines a flow-informed global transition kernel (i-SIR) and a local transition kernel, is a very relevant reference. It is worth noting that the algorithmic template in [5] is very similar to the one in [4], which was (to our knowledge) the first paper to combine a flow-informed global transition kernel (IMH) with a local transition kernel (MALA). Interestingly, among the stated reasons that [4] focus on i-SIR for their global update as opposed to IMH (as proposed in [5]) is that \"theoretical guarantees can be obtained for i-SIR whereas IMH is more difficult to analyze\" and \"IMH and i-SIR (as a multiple-try MCMC) are expected to have similar performances for comparable computational budget\" [4, Section 4]. Given this, we felt that including the method proposed in [5] (NF-MCMC) as a comparator in our numerical experiments was sufficient. With regards to the neural network architecture used for the methods in [4, 5], we used the simple layers (realNVP) implemented in their paper (and in the accompanying software). We believe that this is a fair comparison, given that we also use a relatively simple neural architecture for our CNF. While beyond the scope of this paper, we agree with the reviewer that an interesting avenue for future work would be to investigate whether using more flexible layers for the discrete NFs used in (e.g.,) [4,5] would enhance the performance of these methods.\n\n**4.** The reviewer is correct that, throughout our experiments, we use a relatively simple parametrisation of the vector field. While this does limit the expressiveness of the learned distribution, we found in our experiments that it was sufficient to achieve comparable performance to competing methods, often for a fraction of the computational cost. The design of novel CNF architectures, tailored to specific models, is a non-trivial question, beyond the scope of this paper. However, this would be a very interesting direction for future work. Importantly, there are *no* restrictions on the neural network architecture utilised by our method, so there is significant scope for exploring other architectures which may further improve performance. To illustrate this point, in our global response (see attached PDF), we provide additional results when using a neural network which includes an additional time-dependent weighting (similar to DDS). These results show a significant improvement over the results reported in the original submission, and illustrate the scope for additional improvement based on further refinements of the neural network used by the CNF.\n\n**5**. Thanks to the reviewer for pointing this out, we did indeed mean FAB in L281. This has now been corrected!\n\n**References**\n\n[1] Perugachi-Diaz et al., 2023. Invertible DenseNets with Concatenated LipSwish. NeurIPS 2021\n\n[2] Grenioux et al., 2023. On Sampling with Approximate Transport Maps. ICML 2023. \n\n[3] Cabezas et al., 2023. Transport Elliptical Slice Sampling. AISTATS 2023.\n\n[4] Gabrie et al., 2022. Adaptive Monte Carlo Augmented with Normalizing Flows. PNAS.\n\n[5] Samsonov et al., 2022. Local-Global MCMC Kernels: The Best of Both Worlds. NeurIPS 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Summary**

Many thanks to all of the reviewers for their positive feedback about the paper, as well as their detailed and constructive comments. We provided a detailed point-by-point response to each of the reviewers' specific comments in the individual responses below.

**Additional Results**

We attach to this global response a set of additional numerical results, which we have generated based on the feedback of one or more of the reviewers. In particular, the attached PDF contains:

- New results for all of the numerical experiments using a slightly different parameterisation of the vector field, namely, $\text{NN}^*(t; \theta_3) v_t^\theta(x) = \text{NN}(x, t; \theta_1) + \text{NN}(t; \theta_2) \times \nabla \log \pi(x)$, where the neural networks are standard MLPs with 2 hidden layers, using a Fourier feature augmentation for $t$, and $\text{NN}^*$ outputs a real value that reweights the vector field output using the time component. These new results show a significant improvement over the results in the original submission, and illustrate the scope for additional improvements in the performance of MFM (our algorithm) based on further refinements to the design of the neural network architecture used to parameterise the vector field. These results appear in Tables 1,2,4,5 in the PDF.
    
- New results for an additional benchmarking experiment, namely, the "Many Well" experiment considered in [1,2,3]. These appear in Table 3 in the PDF.
    
- New results for all of the numerical experiments for an adaptively tempered sequential Monte Carlo (SMC) algorithm (AT-SMC). These appear in the bottom row of all of the tables in the PDF.
    
- Results for all of the numerical experiments when running MFM with just the 𝑄 transition kernel. These results appear in the row labelled "MFM $k_Q = K$" in the PDF.
    

**References**

[1] Noé et al., 2019. Boltzmann generators: Sampling Equilibrium States of Many-Body Systems with Deep Learning. Science.

[2] Wu et al., 2020. Stochastic Normalizing Flows. NeurIPS 2020.

[3] Midgley et al., 2023. Flow Annealed Importance Sampling Bootstrap. ICLR 2023.

Meta-review: This paper suggests using flow matching trained CNFs to build MCMC proposals. The main novelty seems to be the incorporation of the flow matching training of the CNF simultaneously with the MCMC process, using generated MCMC samples. 

Reviewers mostly agreed this paper has merit and offers an interesting contribution; has clear exposition; presents a reasonable set of experiments supporting the paper main claims (i.e., method is comparable-to-favorable and often much faster than alternative MCMC sampling baselines); limitations are clearly presented; prior work is well explained and positioned.

Regarding limitations of the paper: Reviewers asked about missing ablations and some limitation in the extent of experiments demonstrated, which the authors addressed, at least partially (which is reasonable within the rebuttal time frame) in the rebuttal, including a new set of additional numerical results; the convergence time of flows due to the need to sample from the model during training, which the authors addressed by confirming this is indeed an issue, yet still worthwhile overall as number of iteration is not large; missing some relevant work and discussion, which the authors promised to include in their revision.

Overall, this paper seems to pass the bar for acceptance, and we strongly encourage the authors to imply the revisions discussed in their rebuttal in the camera ready version of this paper.
Decision: Accept (poster)