Efficient Regression-based Training of Normalizing Flows for Boltzmann Generators
Authors: Danyal Rehman
Oscar Davis
Jiarui Lu
Jian Tang
Michael M. Bronstein
Yoshua Bengio
Alexander Tong
Joey Bose
Venue: ICLR 2026 Poster (2026)
Keywords: Normalizing Flows
Generative Models
Optimal Transport
Flow Matching
AI for Science
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A new regression-based approach to training normalizing flows

Abstract: Simulation-free training frameworks have been at the forefront of the generative modelling revolution in continuous spaces, leading to large-scale diffusion and flow matching models. However, such modern generative models suffer from expensive inference, inhibiting their use in numerous scientific applications like Boltzmann Generators (BGs) for molecular conformations that require fast likelihood evaluation. In this paper, we revisit classical normalizing flows in the context of BGs that offer efficient sampling and likelihoods, but whose training via maximum likelihood is often unstable and computationally challenging. We propose Regression Training of Normalizing Flows (RegFlow), a novel and scalable regression-based training objective that bypasses the numerical instability and computational challenge of conventional maximum likelihood training in favour of a simple $\ell_2$-regression objective. Specifically, RegFlow maps prior samples under our flow to targets computed using optimal transport couplings or a pre-trained continuous normalizing flow (CNF). To enhance numerical stability, RegFlow employs effective regularization strategies such as a new forward-backward self-consistency loss that enjoys painless implementation.  Empirically, we demonstrate that RegFlow unlocks a broader class of architectures that were previously intractable to train for BGs with maximum likelihood. We also show RegFlow exceeds the performance, computational cost, and stability of maximum likelihood training in equilibrium sampling in Cartesian coordinates of alanine dipeptide, tripeptide, and tetrapeptide, showcasing its potential in molecular systems. Code available at: https://github.com/danyalrehman/RegFlow.

Published as a conference paper at ICLR 2026
EFFICIENT REGRESSION-BASER TRAINING OF NOR- MALIZING FLOWS FOR BOLTZMANN GENERATORS
Danyal Rehman 1,2.3; Oscar Davis", Jiarui Lu Jian Tang Michael Bronstein 4.5
Yoshua Bengio ! Alexander Tong" 22 Avishek Joey Bose .4,7f -4.77
'Mila, Université de Montréal, "Massachusetts Institute of Technology UUniversity of Oxford, AATTHRR, HEC Montréal, 'Imperial College London
ABSTRACT
Simulation- celllattio--feeeeeee training frameworks have been at the forefront of the generative modelling revolution in continuous spaces, leading to large-scale diffusion and flow matching models. However, such modern generative models suffer from expensive inference, inhibiting their use in numerous scientific applications like Boltzmann Generators (BGs) for molecular conformations that require fast likelihood evaluation. In this paper, revisit classical discrete-time normalizing flows in the context of BGs that offer efficient sampling and likelihoods, but whose training via maximum likelihood is often unstable and computationally challenging. We propose REGRESSION TRAINING OF NORMALINING FLOws (REGFLOW), a novel and scalable agression-based training objective that bypasses the numerical instability and computational challenge of conventional maximum likelihood training in favour of simple fz-regression objective. Specifically, REGFLLOW maps prior samples under our flow to targets computed using optimal transport couplings pre-trained continuous normalizing flow (CNF). To enhance numerical stability, REGFLOW employs effective regularization strategies such as wwwarddbackwardd consisensst loss that enjoys painless implementation. Em- pirically, we demonstrate that REGFLOW unlocks a broader class of architectures that were previously intractable to train for BGs with maximum likelihood. We also show REGFLOW exceeds the performance, computational cost, and stability of maximum likelihood training in equilibrium sampling in Cartesian coordinates of alanine dipeptide, tripeptide, and tetrapeptide, showcasing its potential in molecular systems. Code is available https: //gi thub. com/ danyalrehaaaaaaaaa Regg/ low.
INTRODUCTION
The landscape of modern simulation- free generative models in continuous domains, such as diffusion mod- els and flow matching, has led to ofate-of-the-art generative quality across a spectrum of domains (Betker et al., 2023; Brooks et al., 2024; Huguet et al., 2024; Geffner et al., 2025). Despite the scalability of simulation-free training, generating samples and computing model likelihoods from these model families requires computationally expensive inference-often hundreds of model thllu- the numerical simulation of the learned dynamical system. 'he search for efficient inference schemes has led to a new wave of approaches that seek to learn one-step generative models, either through distillation (Yin et al., 2024; Lu and Song, 2024; Sauer et al., 2024; Zhou et al., 2024), shortcut training (Frans et al., 2024), or Inductive Moment Matching (IMM) (Zhou et al., 2025) methods that are able retain the impressive sample quality of full simulation. However, many highly sensitive applications- instance, in the natural sciences (Noé al., 2019; Wirnsberger al., 2020)-require more than just high-fidelit samples: they also necessitate accurate estimation of probabilistic quantities, the computation of which can be facilitated by having access cheap and exact model likelihoods.
Table I: Overview of various generative models and their relative tradc-offs with respect to the number of inference steps, ability to provide exact likelihoods, and training objective for learning.
Method
One-step
Exact likelihood
Regression training
CNF (MLE)
Flow Matching
Shorteut (Frans et al., 2024)
X
IMM (Zhou al., 2025)
NF (MLE)
REGFLOW (ours)
"Correspondence to: danyal. rehman@mille la. quebec
I Equal advising

---

Published as a conference paper at ICLR 2026
Consequently, for one-step generative models to successfully translate to scientific applications, they must additionally provide faithful one-step exact likelihoods that can be used to compute scientific quantities of interest, e.g., free energy differences (Rizzi et al., 2021), using the generated samples.
Given their intrinsic capacity to compute exact likelihoods, classical normalizing flows (NF) have re- mained the de facto method for generative modelling in scientific domains (Tabak and Vanden-Eijnden, 2010; Tabak and Turner. 2013; Dinh et al.. 2016; Rezende and Mohamed, 2015). For example. in tasks such as equilibrium sampling of molecules, the seminal framework of Boltzmann Generators (Noé et al., 2019) 9) pairs a normalizing flow with an importance sampling step. Consequently, rapid and exact likelihood evaluation critical both for asymptotically debiasing generated samples in such high-impact applications and for refining them via annealed importance sampling (Tan et al., 2025a;b).
Historically, NFs employed in conventional generative modelling domains (such as images) are trained with the maximum likelihood estimation (MLE) objective, which has empirically lagged behind the ex- pressiveness, scalability, and ease of training of modern continuous normalizing flows (CNFs) trained with regression-
objectives like flow matching and stochastic interpolants (Peluchetti, 2023; Liu, 2022: Lipman et al., 2023; Albergo and Vanden-Eijnden 2023). A key driver of the gap between classi- cal flows and CNFs can be attributed the MLE training objective itself, which computes the change- of-variable formula for gradient ascent on the log-likelihood function with invertible architectures. As result, architectures have balance ease of optimization with expressivity, with highly flexible archi- tectures being highly prone to heing numerically unstable (Xu and Camphell, 2023; Andrade, 2024). For instance, in the context of Boltzmann Generators, this tension between MIE training and invertible architectures has led to BGs that use classical flows underfitting target molecular systems in compari- son to BGs that employ flow matching (Klein et al., 2023). However, despite the expressive power of CNFs, inference still requires expensive numerical simulation-exet exact likelihood requires simulation of the divergence, a second-order derivative. This raises the natural motivating research question:
Q. Does there exist a performant training recipe for BGs with classical NFs beyond MLE?
Present work. In this paper, we answer in the affirmative. We investigate how to train an invertible neural network to directly match predefined invertible function and build BGs with classical flows. We introduce REGRESSION TRAINING OF NORMALIZING FLOWS (REGFLOW), a novel regression- based training objective for classical normalizing flows that marks significant departure from the well-established MLE training objective. Our key insight is that access to coupled samples from any invertible map is sufficient to train a generative model with regression objective. As result, we can train a classical flow by learning to match in ée-regression the pre-computed noise-data pairings given by existing both non-parametric or parametric-inveeeeee invertible maps. As result, training REGFLOW provides similar benefits to NF training as flow matching does to continuous NFs but with the new unlocked benefit that inference provides exact likelihoods in a single step -..., without numerical simulation of the probability flow ODE and thus is significantly cheaper than a CNF.
To train BGs using REGFLOW, we propose a variety of couplings to facilitate simple and efficient training. We propose endpoint targets that are either: (1) outputs of a larger pretrained CNF; or (2) the solution to a pre-computed OT map done offline as a pre-processing step. To enhance training stability we also include a series of regularizers, and in particular, new forward-backward self-coonsstencyy regularizer that completely removes the need for computing the computationally-ee expennsive Jacobian determinant that is needed in MLE training. In each case, the designed targets are the result of already invertible mappings, which simplifies the learning problem for NF's and enhances training stability. Empirically, we deploy BG-based REGFLOW flows on learning equilibrium sampling for short peptides in alanine di-, tri-, and retrapeptide, and find even previously discarded NF for BGs, such as affine coupling (Dinh et al., 2016) or neural spline flows (Durkan et al., 2019), can outperform their respective MLE-trained counterpart. In particular, we demonstrate that in scientific applications where MLEE training is unsuccessful, the same BG model trained using RFGFL.OW provides higher fidelity proposal samples and likelihoods. Finally, we demonstrate a completely new method of performing Targeted Free Energy Perturbation (Wimsserrger et al., 2020) that avoids costly energy evaluations with REGFLOW that are not possible with MLE training of normalizing flows.
2 BACKGROUND AND PRELIMINARIES
Generative models. A generative model can be seen as an (approximate) solution to the distribution matching problem: given two distributions Pn and pl. the distributional matching problem seeks to find a push-forward map fo : Rd - Rd that transports the initial distribution to the desired endpoint P1- [Jo] # (po). Without loss of generality, we set Pprior- == po to be tractable prior (typically
2

---

Published as a conference paper at ICLR 2026
standard normal) and take Pdata := P1 the data distribution, from which we have empirical samples. We now turn our attention to solving the generative modelling problem with modelling families that admit exact log-likelihood, log pz(z), where pe = [ff] # (Po), with particular emphasis on normalizing flows (Tabak and Vanden- -ijndenn,,,de, 2010; Tabak and Turner, 2013; Dinh et al., 2014; 2016; Rezende and Mohamed, 2015; Papamakarios et al., 2021).
2.1 CONTINUOUS NORMALIZING FLOWS
A CNF models the generative modelling problem as a (neural) ODE a ft,(())) = v2 (fr, (x+)). is the smooth generator and forms the solution pathway to (neural) ordinary differential equation (ODE) with initial conditions fo(2a) = Zig. Furthermore, 2r,0 : 10, 1] x IRd + d is the time-dependent velocity field associated with the (flow) map that transports particles from Po to Pt. A CNF is an invertible map up to numerical precision, and as result, we can compute the exact log- likeeeoe,,,,d, log Pt, a (x+). using the instantaneous change of variable formula for probability densities (Chen et al., 2018). The overall log-likelihood of a data sample, Zo. under the model can be computed as follows:
(z)).
Here, fo [0. 1] IR - Rd (l,
log P1 log Po J, V vC,
Maximizing the model log- ligelikei in eq. (1) offers one possible method to train CNF's but incurs costly simulation. Instead, modern scalable methods to train CNF's employ flow matching (Lipman et al., 2023; Albergo and Vanden- Vanden-Eijnden, 2023; Tong et al.. 2023; Liu et al., 2023), which learns ve,0 by regressing against the (conditional) vector field associated with designed target conditional flow everywhere in space and time, e.g., constant speed conditional vector fields.
Numerical simulation. In practice, the simulation of a CNF conducted using a specific numerical integration scheme that can impact the likelihood estimate's 's fidelity in eq. (1). For instance, an Fuler integrator tends to overestimate the log-likelihood (Tan al., 2025a), and thus it is often preferable to utilize integrators with adaptive step size, such as Dormand- Dormand-Prince(43) Prince(4)5 (Hairer et al., 1993). In applications where estimates of the log- likelihood suffice, it is possible to employ more efficient estimators such as Hutchinson's 's trace estimator to get unbiased-yet higher variance-estimate of the divergence. Unfortunately, as we demonstrate in $3.1, such estimators are too high variance to be useful for importance sampling even in the simplest settings, and remain too computationally expensive and unreliable in larger scientific applications considered this work.
One-step maps: Shorteut models. One way to discretize an ODE is to rely on the self- consistency property of ODEs, also exploited in consistency models (Song et al., 2023), namely that jumping At in time can be constructed by following the velocity field for two half steps (At/2). This is the core idea behind shortcut models (Frans et al., 2024) that are trained at various jumps by conditioning the vector field network on the desired step-size At. Precisely, 2. (2z) = ft 2t) + (1'+ 22 where z'+Ai. = Σ₁ + ft (z At) In their extreme, shortcut models define a one-step mapping which has been shown to generate high-quality images, but it remains an open question whether these models can reliably estimate likelihoods.
2.2 NORMALIZING FLOWS
The generative modelling problem can also be tackled using time-agnostic generators. One such prominent example Normalizing Flows (NFs) (Tabak and Vanden- Eijnnen, 2010: Tabak and Turner, 2013; Dinh et al., 2016; Rezende and Mohamed, 2015), which parameterize diffeomorphisms (continuously differentiable bijective functions, with continuously differentiable inverse). fe _Rd + Rd. For arbitrary invertible maps fe. computing the change in log probability is prohibitively expensive with cost that scales with (dd)) Consequently, it is popular build fo using a composition of M elementary diffeommrphisms, each with an easier to compute Jacobian determinant: fo = -1 o o... o fo (Papamakarios et al., 2021). Through function composition, simple invertible blocks can lead to flows that are universal density approximators (Teshima et al., 2020; Ishikawa et al., 2023; Kong and Chaudhuri, 2021: Zhang al., 2020; Bose et al., 2021), and the resulting MLE objective for training is simply:
M-1
log det Ofia(z;)) Oz;
(x = Pooo) - Σ
Pu = I).
(2)
1=0
3

---

Published as a conference paper at ICLR 2026
Boltzmann Generators. A Boltzmann Generator (BG) (Noé et al., 2019) combines a normalizing flow model, pa, with an importance samplingg correction to produce i..dd samples from a target Boltz- mann distribution Praggt- The normalizing flow defines tractable proposal density pa(z) from which we draw K independent points xz(²) ~ pa, i [K]. For each sample, we evaluate an unnormalized importance weight, which allow any observable o(x) to be consistently estimated under the target measure Ptarget using self- self-normalized importance sampling (SNIS) (Liu, 2001; Agapiou et al., 2017);
ΣK w (a(*) @ (xx()) w (xx()) exp(- cxp(-E(=())/Fpg)
E Purge ( (1:) 2
Ex=1 w(x(0)
Pe (xx(1)
where {(x) denotes the potential energy and TyT are the Boltzmann constant and temperature re- spectively. The normalized weights v (z()) = w (z()) )// Σ, w (z(1) can also be used to resample the generated configurations, yielding unbiased i.i.d. draws from the desired Boltzmann distribution.
3 REGRESSION TRAINING OF NORMALIZING FLOWS
We seek to build one-step transport maps that both push forward samples 30 ~ Do to 21 Pl. and also permit exact likelihood evaluation. Such a condition necessitates that this learned map is bijective function-i.e. an invertible map-and enables us compute the likelihood using the change of variable formula. While using an MLE objective is always a feasible solution to learn this map, is often not scalable solution for both CNFs and classical NFs. Beyond architectural choices and differentiating through a numerical solver, learning flows using MLE is intuitively harder as the process of learning must simultaneously learn the forward mapping, fe, and the inverse mapping, fg without knowledge of pairings (1) T20, 21) from coupling.
To appreciate this nuance, consider the set of invertible mappings I and the subset of flows F C I, that solve the generative modelling problem. For instance, there may exist multiple ODEs (possibly infinitely many) that push forward pn to pl. It is clear then that the MLE objective allows the choice of multiple equivalent solutions I E F. However, this precisely what complicates learning fo. as certain solutions are harder to optimize since there is no prescribed coupling π ( 21 ) for noise To, and data targets æ1- That to say, during MLE optimization of the flow fo. the coupling æ evolves during training as it is learned in conjunction with the flow, which can often be significant challenge to optimize when the pairing between noise and data is suboptimal.
Regression objectives. In order to depart from the MLE objective, we may simplify the learning problem by first picking a solution f E F and fixing the coupling π T'xoo.²1 21) induced under this choice, i.e. Pı = [ (po). Given privileged access to f*, we can form simple regression objective that approximates this in continuous time using our choice of learnable flow:
((0)= E, Er, [1.f2.((z)) - fi (=))))
where (zo, xt" (2r, and 24 pe(-[20, 21) is drawn from known conditional noising kernel such as a Gaussian distribution. We note that the regression objective in eq. (4) is more general than just flows in I, and, optimality, the learned function behaves like fr" on the support of po. under mild regularity conditions. We formalize this intuition more precisely in the next proposition.
Proposition 1. Suppose that ft is invertible for all that (ft)-1 is continuous for all t. Then, as C(0) + 0, it holds that ((ft)-1 o f(.o) - for almost all (with respect to po) x.
The proof for proposition 1 can be found in $A, and illuminates that solving the original generative modelling problem via MLE can be re-cast as matching problem to a known invertible function f. Indeed, many existing generative models already fit into this general regression objective based on the choice of f", such as conditional flow matching (CFM) (Tong et al., 2023), rectified flow (Liu et al., 2023), and (perfect) shorteut models (Frans et al., 2024). This proposition also shows why these models work as generative models: they converge in probability to the prespecified map.
3.1 WARMUP: ONE-STEP GENERATIVE MODELS WITHOUT LIKELIHOOD
As there exist powerful one-step generative models in image applications, it is tempting to consider whether they can be used for BG applications requiring likelihoods. As warmup, we investigate the use of current state-of-the-an one-step generative models in shortcut models (Frans et al., 2024) and In- ductive Moment Matching (IMM) (Zhou al., 2024) through simple experiment (see $B for details).
4

---

Published as a conference paper at ICLR 2026
(a) Non-invertible shortcut. (b) Non-invertible IMM. (c) IMM with an NE. (d) Ground truth.
Figure/ 1: Evaluation of LMM and shortcut models with exact likelihood on the synthetic checkerboard experiment. Depictions are provided of the 2D histograms after self- normalizmzzn importance sampling is used.
Synthetic experiments. We instantiate both model classes on a simple generative modelling problem, where the data is checkerboard density. In fig. 1, plot the results and observe, that non-invertible shortcuts and IMM models are imperfect at learning the target and are unable to be corrected to Poymth after resampling. However, when IMM is used to train an NF (Durkan et al., 2019), we see samples that almost perfectly match Paynth but but such an approach not scalable ($B.2).
This puts spotlight on counter-intuitive question given proposition 1: Why do shortcut models have incorrect likelihoods? While proposition implies pointwise convergence of So to f", this does not imply convergence or regularity of the gradients of fo. and thus shortcut models can still achieve high quality generations without the need to provide faithful likelihoods.
Insufficiency of uniform convergence. One-step maps are trained to converge pointwise to Sa - f on sub-domain D S ddd However, this does not imply pointwise convergence of gradients foe + Vf* For instance, consider the following toy example: fm (x) = sin(mz) + x and (*(x) = I. As m + oo, fm converges uniformly to f"; however, the gradient fm = cos does not converge. Importantly, this means that while fo would produce increasingly accurate generations, its likelihoods derived through eq. (2) may not converge to those of the base model.
3.2 TRAINING NORMALIZING FLOWS USING REGRESSION
We now outline our REGFLOW framework to train a one-step map for a classical NF. To remedy the issue found shorteut models and IMM in section 3.1, we judiciously choose fo to be an already exactly invertible mapping -ie., classical NE. Since NFs are one-step maps by construction, eq. (4) instantiated using simple regression objective follows:
(() = B (1))) ff + RAR Ezo.7 || x111 112 X,
(5)
where R is regularization strategy and X, E IR+ is the strength of regularization. Explicit in eq. (5) is the need to procure one-step targets 2 = fi (za) from known invertible mapping fr. We outline the choice of such functions in $3.3. We also highlight that the one-step targets in eq. (5) differ from the typical flow matching objective where the continuous targets fi = S (1. (2ro xx1) ) (see $A.3 for discussion). Consequently, for NFs that universal density approximators (Teshima et al., 2020; Kong and Chaudhuri, 2021; Zhang et al., 2020), the learning problem includes a feasible solution.
Training recipe. We provide the full training pscudocode in algorithm 1. In practice, we find that ft is often ill-conditioned, with the target distribution often centered around some lower-dimensional subspace of IRd similar to prior work (Zhai et al., 2024). This may cause fo to become numerically ill-conditioned. To combat this, we use three tricks to maintain numerical stability. Specifically, we regularize the loss function, add small amounts Gaussian noise to the target distribution similar to Hui et al. (2025); Zhai et al. (2024), and, finally, add weight decay to optimizer.
Speedup from uni-directional flow training and inference. For some flow types these are roughly equivalent (RealNVP or Jet) in computation time. However, for some flows such as autoregressive flows (e.g. NSF), the network f(x) is substantially faster to evaluate than its inverse f (z). In standard maximum likelihood training of normalizing flows, the model trained with passes from data to noise. This is then reversed during generation with passes from noise to data. In REGFLOW, inference and training can be done from noise to data. This means substantially faster inference can be achieved by training autoregressive flows where the fast direction is oriented from noise to data.
Regularization Strategies. In principle, classical normalizing flows can be trained using a standalone regression objective that directly maps latents to data. In practice, we observe that regression training
5

---

Published as a conference paper at ICLR 2026
alone impact numerical invertibility- similar phenomenon to that observed in MLE- normalizing flows (Xu and Campbell, 2023; Andrade, 2024). This adversely impacts re-weighted samples as the NF becomes increasingly numerically unstable. To remedy this, we introduce two regularization strategies, one using the tog-determinant of the Jacobian (see eq. 6). while the other does not, resembling cycle-consistency loss using
regularization (see eq. 7):
Llog-dat (z)) z1 l12 + Ar (log |ee (J)(2)))))
(6)
Loud-bwd (z)) 21 112 xfllfo (fo(a))) -
(7)
The first regularization strategy uses the same log determinant that is needed in the change of variable formula, which comes at no additional computational cost for the architectures we experiment with. Intuitively, this penalizes the flow map from collapsing to a point as it regularizes against sharp mass placements, which what determinant geometrically computes. The second regularizer is new newardd-bacwward self-consistency regularizer that ensures invertibility at the output level, but at double the computational cost. However, interestingly, since it does not require the Jacobian, opens up potential directions for less constrained architectures. For our purposes, we find both of these regularizers accomplish our aim of avoiding collapse and maintaining invertibility.
Algorithm 1 REGRESSION TRAINING OF NORMALIZING FLOWS
Input: Prior Po. empirical samples from pr, regularization weight A, noise scale Xn network fo
1: while training do
2:
(ro,
(xo, x1)
Sample batches of size b i...d. from the dataset & Add scaled noise to targets Loss with regularization
3:
T1 f- 11 + An E, with ~ NN I)
4:
C(0) [ 21 X,R
5:
9 + Upddteee VoC(A))
6:
return fe
3.3 REGFLOW TARGETS
To construct useful one-step targets in REGFLOW, we must find a discretization of an invertible function-e.g, an ODE solution-al longer time horizons. More precisely, we seek discretization of an ODE such that each time point + At where the regression objective evaluated corresponds true invertible function fi Consequently, if we have access to invertible map such that directly regress our parametrized function one-step map, fo. (2,) 21- This motivates the search and design of other invertible mappings that give us invertibility at longer time horizons, for which we give two examples next.
t + At =
Optimal transport targets. Optimal transport in continuous space between two distributions defines a continuous and invertible transformation expressible as the gradient of some convex function (Villani, 2021; Peyré and Cuturi, 2019). This allows us to consider the invertible OT plan:
fot arg miny /
s.t. Te (po) = P1
(8)
where e RRd x Rd + R is the OT cost and T Rd - Rd is transport map. We note that this map is interesting as it requires no training; however, exact OT runs in (n²)) time and 0(n²) space, which makes it challenging to scale to large datasets. Furthermore, OT-CFM (Tong et al., 2023), which uses mini-batches to approximate the OT-plan. Nevertheless, in applicable settings, full batch OT acts
highlight that this differs from
one-time offline pre- pre-processing step for training fe.
Reflow targets. Another strategy to obtain samples from an invertible map is to use pretrained CNF, also known as reflow (Liu, 2022). Specifically, we have that:
fietow (xo) = To + f of (x.)dt = = 21.
In other words, the one-step invertible map is obtained from a pre-trained CNF v;, from which we collect dataset of noise- target pairs, effectively forming x² (1). We now prove that training on reflow targets with REGFLOW reduces the Wasserstein distance to the pt.

---

Published as a conference paper at ICLR 2026
Proposition Let Profoos be a pretrained CNF generated by the vector field vi, real numbers (Lo)eejo.) such that vi is L Li-Lipschitz for all E [0. 11, and NF fe trained using Eq. 5 by regressing against Fregton (2o), where 20 ~ NN(,,)) Then, writing Pe = Law (f6 (ro)), we have:
(o)) S K exp (o (ndl) -e, K2 f=/('
<(((E((1(())) Ur (xx)) & dl, (10)
( (((lo(((aa)
where is the l2 approximation error between the velocity field of the CNF and the ground truth generating field vi. e² = Enrg.. ([.ietoo(()))
50 (x0())))
The proof for proposition 2 is provided in $A. Intuitively, the first term captures the approximation error of the pretrained CNF to the actual data distribution Pt, and the second term captures the approximation gap between the flow trained using REGFLOW to the reflow targets obtained via Prefoow
While these two cases represent interesting instantiations of f there exist many other possible procedures for obtaining f*. We investigate the theoretical properties for f* in appendix A.4 to provide guidance for those who wish to investigate other targets.
Table Quantitative results on alanine dipeptide (ALDP), tripeptide (AL3), and tetrapeptide (AL4) reported as mean standard deviation over three seeds.
Dipopride DP:
Tripeprite (AI-3)
Taragepride (A144)
Algurilann 1
ESS
S:W.
1-5%
ESS
E-W.
7-35
ESS
T-W,
NSF (MLE))
0,65
0.022
13,797 + 2.73
1.243 + 6.103
0024 = 0.044
17. 37.96-22 +: 1.21
1,655 = 0.180
0.816 =: 8,013
30 856 1.050
3885 #: 0.400
NSP (RauFLo))
0.005 = 0.004
0,501
6013
0951 1: 0,854
0051 = 0.018
0853 = 0.305
1.577
=: 0,140
0,011
0000
3.277
0.546
2 :: 0,102
NVP
X20
21e3
X20
X30
(RaGFLOw)
0,005
0
3,241
1,81
0.215
0,013
2.177
let (MLL)
<lesk
>-0
clod
blel
2-1)
<le-4
>10
Jet (RacFLow)
0.055 = 0.0%6
4,193 E 1016
0,801 0,976
0,976
<le4
3,644 a 0.358
<le.
230
4 EXPERIMENTS
We evaluate NFs trained with REGFL.OW three molecular systems: alanine dipeptide (ALDP), alanine tripeptide (AI.3), and alanine tetrapeptide (AL.4.. These peptides are a standard benchmark for tesing generative models in computational chemistry. We asses the models two key tasks: equilibrium conformation sampling and targeted free energy prediction (TFEP) (Wirnsherger et al., 2020). Through these experiments, show that REGFLOW outperforms the conventional maximum likelihood estimation (MLE) training for NFs in these scientific applications.
Setup. We test three different architectures: RealNVP with a residual network parametrization (Dinh et al., 2016), neural spline flows (NSF) (Durkan et al., 2019), and Jet (Kolesnikov et al., 2024), across three different molecular systems (ALDP, AL3, and AL4) of increasing size and compare the performance of the same invertible architecture trained using MLE, and using REGFLOW. We report: Effective Sample Size (ESS); the I-Wasserstein distance on the energy distribution; and the 2- 2-Wasserstein distance on the main dihedral angles as described in &C with additional results in $D.
Main results. We report our main quantitative results in table 2 and observe that REGFLOW with reflow targets consistently outperforms MLE training of NFs across all architectures on both E-W, and T-W; metrics, and slightly underperforms MLE training on ESS. However, this can be justified by the mode collapse that happens in MLE training as illustrated in the Ramachandran plots for alanine dipeptide fig. 2, which artificially increases ESS. Examining the energy histogram plots in fig. 2 we observe that NFs trained using REGFLOW more closely match the true energy distribution. We also illustrate these improvements across metrics when using OT targets over reflow, as shown Appendix fig. 9. Our results clearly demonstrate that REGFLOW is often a compelling alternative to MLE training in BGs for all analyzed NF architectures, and allows training of architectures that were previously untrainable with MLE training.
REGFLOW leads to faster training and inference. We note that NEs trained with REGFLOW are far faster at com- puting likelihoods compared to their MLE-traine' coun- terparts, except for cases where the NF has an analytical inverse (Res-NVP, Jet) due to the reversal of the flow. For autoregressive flows like NSF-where the reverse pass is slow to compute--we we observe the maximum benefit:
Table 3: Inference efficiency comparisons.
Time to compute likelihoods for 200k points.
Modek +
Dipeptide (ALDP)
Algorithm
MLE
REGFLow
CFM
Speed Up
277.00
8.18
NiA
RV-
3.61
3.51
N/A
1.03x
Ja
67.63
60.43
N/A
1.11x
CNF
N/A
NIA
26969.30
N/A
7

---

Published as a conference paper at ICLR 2026
Table 4: Training time comparison between MLE and REGFLOW for alanine dipeptide.
Metric
MLE
REGFLow
OT
CNF
&-W: 7.090
10h10
6h54
7h23
T-W 1.368
12h17
7h32
7h56
Table S: ALDP with various regularization strategies.
Models +
Dipeptide (ALDP)
Algoritimn
ESS T
E-W;
T-W
(MLE)
0.055
0.012
2.713
1223
-0.03
NSF (REGFLOW reg)
0.032
1: 0.008
0.604
0.045
1.083
00.09
NSF (REGFLOW w/ logdkts)
0.036
+ 0.007
0.519
2 0.021
0.958
000074
XSF (REaPLoW w/ fwd-bwd)
0.055 0.004
0.501
0.011
0.951
0.054
Res-NVP (MLEE)
10
30
Re--NVP.KKKGFLL ow wio reg)
0
+ 0.010
2.948
0.457
1.179
0.218
Res-NVP (RSCPPPRR w/ logdens)
0.032
+ 0.008
2,310
0.411
0.7%6
0.109
Rei-NVP (RSGFFL (wd--ood)
0.035
0.008
0.812
0.121
Jet (MLE)
10-4
10a
30
Jet (REGFLOW wlo reg)
0.053 +: 0.007
9.707
L.843
1.224
= 0.181
Jet (REGFLOW w/ logdlets)
0.051 + 0.004
6.349
1.412
0,872
00.655
(REOFLOW Ewd-bwod)
0.055
0.006
0,801
0.076
REGFLOW enables nearly x4x speedup in inference compared to the equivalent MLE-trained NF, as seen in Tab. 3. We also compare performance relative to CNFs, which require integrating the divergence of the vector field-thi makes likelihood evaluation extremely expensive compared to discrete NFs. We observe that CNF inference with likelihoods is approximately 450x more expensive than our slowest NF (Jet) and x more expensive than our fastest NF (Res-NVP).
MLE
Rog
Ground 1ath
NedFice
0.22
Sax daa
0.22
Nomalized Deraty
0200
Fegerel beeiigttt
032
Progese nnl coseigrrett
0.00
Free energy
0.00
2.0
0.04
0.00
3.50
30
110
300
100
41-9)
Figure 2: Energy distributions and resampled Ramachandran plots for alanine dipeptide. (left to right): Energy distribution most best MLE-trained NF; energy distribution of best REGFLOW; ground truth MD data torsion angle distribution, best MLE-trained model Ramachandran plot; best REGFLOW Ramachandran plot.
Next, we contrast the training times between MLE and REGFLOW, accounting for: (1) CNF training or OT map pre-computation; (2) sample generation from the CNF; and (3) REGFLOW training until its performance exceeds MLE. Across all settings, REGFLOW consistently outperforms MLE, Specifically, we observe that achieving superior performance on E-W1 requires ~27% less time with REGFLOW, while on T-W,, the speedup closer ~35%. We also compare the training times between MLE and REGFLOW across all peptide systems. In fig. 3, we illustrate how the energy varies during training using REGFLOW; the dotted lines symbolize the best energy using the MLE-trained NSF on the validation set. Here, we see that the crossover between REGFLOW and MLE occurs after 1h20, 1h20, and ~244 for REGFLOW to outperform MLE on the dipeptide. tripeptide, and tetrapeptide, respectively. Conversely, MLE training took ~ 10h10 ~ Ih20, and ~ ~ 1h40 using the dipeptide, tripeptide, and tetrapeptide, respectively. These studies further validate the potential for REGFLOW to serve as an efficient and effective alternative to MLE.
Alternative regularization strategies. We inves- tigate the impact of different regularization strate- gies to prevent numerical collapse for REGFLOOW in table 5. We consider regularization (w/o reg), regularization of the magnitude of the log deter- minant of the Jacobian (w/ logdets), and direct invertibility penalization fforward-backward)) For our usecase, the Jacobian comes at no extra cost and is therefore the most efficient. The forward- backward regularizer enforces cycle consistency by performing a forward pass of the NF, followed by reverse pass the same generated samples, and
Thirspepaade
10a
Eneegyonn, .e09
204
10"
10:2
10
1.
2b
Time,
hours
Figure 3: Training time required for REGFLOOW to outperform the most performant MLE model (NSF).
8

---

Published as a conference paper at ICLR 2026
computing the &g distance between the reconstructed priors. This is at least twice expensive the logdet regularization for our use case, however does perform quite well, and interestingly opens up the possibility for more flexible architectures. All regularizations outperform MLE, and the logdet regularization offers the best tradeoff between performance and speed for our usecase, so we use that regularization for the remainder of our experiments.
Ablations. In table 6, we report REGFLOW using OT targets and various amounts of generated re- flow targets- unique advantage of using reflow as the invertible map. As observed, each target choice improves over MLE, outside of ESS for NSF. Importantly, we find that using more sam- ples in reflow consistently improves performance metrics for all architectures. In fig. 4, we show how performance increases with the number re- flow samples and we ablate the impact of regular- ization. We find performance improvements with increasing regularization, up to around 10-6 X, I 10-5. Regularizing beyond this is suffi- cient to ensure empirical invertibility based on validation loss of Liwd-buk < 10-A, but hampers generation performance.
Table 6: Ablations target types and amount of reflow targets ALDP.
Darasets
Dipeptide (ALDP)
Algoriitm J
ESS 1
E-W; 1
T-W
NSF (MLE)
0.055
13.80
1.243
(ReoFLOW
0.016
17.39
KSP (ReGFLOW 10.4M CNF)
0.035
0.501
0.951
NSF (REGFLoW OT)
0.003
0.604
2,019
Res-NVP (MIFF
110-4
> 10"
>.
Res-NVP (RuGFLow @ 100k CNF)
0,009
46.93
1,155
(REGFLOW
CNF)
2.104
Res-NVP (RuGFLoW @ OT)
0.006
0.699
1.969
(MLE)
100
Jet (REGFLOW @ 100K CNF)
0.017
31.42
1,081
Ast (REGFLOW @ 10.4M CNF)
0.051
4.193
0.801
(REGFLOW OT)
2.534
10a
NSF
THE
1,05
NSF (RogFlow)
NSF (Rey Flow)
2.50
0
NSF (RegFlow)
Res-NVh (RegFlow)
Res NVP (RegFlow)
Res NVP (RegFlow)
(RegFlow)
100
(Regfflo)
2:25
(RegFlow)
1.00
E-W-]
0.95
1.75
1.50
1.25
1.00
0.80
Reflow Samples, Na
Reflow Samples, N.mton
Regularization,
Figure 4: Left and center: Ablations demonstrating performance improvements with increasing number of reflow samples. Right: Increasing regularization improves T-W; up certain point, beyond which numerical invertibility guaranteed the regression objective, and subsequently, sample quality, adversely impacted.
Targeted Free Energy Perturbation. Ac- curate caleulations of the free energy dif- ference between two metastable states of a physical system is both ubiquitous and of profound importance in the natural sciences. One approach to tackling this problem Free Energy Perturbation (FEP) which exploits Zwanzig's identity: EA [e BAUT = "AAP
0.30
Free energy lhg
Free Energy Difference
0.25
0.20
0.15
oe o
0.10
0.05
where AF = FB FA is the Helmholtz free energy difference between two metastable states A and B (Zwanzig, 1954). Targeted Free Energy Perturbation (TFEP) improves over FEP by using NFs to learn an invertible
Res-Mip
jet
oir cu
Figure 5: Left: The 3planar and or conformation states; Right: REGFLOW's ability to learn free energy differences. map using MLE to increase the distributional overlap between states and (Wirnsberger et al., 2020; Moqvist et al., 2025); however, this can be challenging for several reasons. NFs are difficult to learn, especially when the energy function expensive to compute, or the states occupy small areas.
We propose
TFEP method that does not require energy function evaluations during training. By using REGFLOW, we can train the NF solely based on samples from states A and B. This enables TFEP, where energy evaluations may be costly-a new possibility that is distinct from NFs trained using MIEE To demonstrate this application of REGFLOW we train an NF solely from samples from two modes of ALDP (see fig. 5) and use OT targets which avoid any energy function evaluation.

---

Published as a conference paper at ICLR 2026
We include reference using the DiT CNF trained to map between meta-stable achieves similar predictions, albeit taking nearly three orders of magnitude longer to compute. We find achieve high-quality free energy estimation in comparison ground truth Molecular Dynamics (MD) using only samples during training, as illustrated in fig. 5. We believe this is a promising direction for future applications of free energy prediction.
also
5 RELATED WORK
Exact likelihood generative models. NFs are generative models with invertible architec- tures (Rezende and Mohamed, 2015; Dinh et al., 2016) that produce exact likelihoods for any given points, Common models include RealNVP (Dinh et al., 2016), neural spline flows (Durkan et al., 2019), and Glow (Kingma and Dhariwal, 2018). Jet (Kolesnikov et al., 2024) and TarFlow (Zhai et al., 2024) are examples transformer-based normalizing flows. Aside from Jet and Tarflow, NEs have generally underperformed compared to diffusion models and flow matching methods (Ho et al., 2020; Lipman et al., 2023; Albergo et al., 2023; Liu, 2022), partly due to the high computational cost of evaluating the log-determinants of Jacobians each training step.
Few-step generative models. To avoid costly inference, few-step generative models were introduced as methods to accelerate the simulation of diffusion and CNFs. Common examples include DDIM (Song et al., 2022) and consistency models (Song et al.. 2023), which introduced a new training procedure that ensured the model's endpoint prediction remained consistent. Recently, flow maps (Boffi et al.. 2024; 2025; Song and Dhariwal, 2023; Lu and Song, 2024; Geng et al., 2024; 2025; Sabour et al., 2025) have improved upon this paradigm. Other lines of work proposed related but different training objectives, generalizing consistency training (Frans et al., 2024; Zhou et al., 2025; Kim al., 2024; Heek al., 2024). Beyond diffusion and FM, residual networks (He et al., 2015) are a class of neural networks that are invertible if the Lipschitz constant of fo is at most one (Behrmann et al., 2019). The oeg-determinant of the Jacobian then approximated by truncating series of traces (Behrmann et al., 2019) an approximation improved in Chen et al. (2020).
6 CONCLUSION
In this work, we present REGFLOW, method for generating high-quality samples alongside exact likelihoods in a single step. Using base coupling between the dataset samples and the prior, provided by either pre-computed optimal transport or base CNF, we can train classical NF using simple regression objective that avoids computing Jacobians at training time, as opposed to typical MLE training. In theory and practice, we have shown that the learned model produces faithful samples, the likelihoods of which empirically allow us to produce statc-of-the-ar results on several molecular datasets, using importance-samplinn
resampling. Limitations include the quality of the proposal samples, which substantially improve on MLE-trained NFs, but are not on par with state-of-the-re CNFs or variants thereof. Moreover, while producing accurate and high high-quality likelihoods, they do not, in theory, match those of the base coupling, which can be desirable property.
10

---

Published as a conference paper at ICLR 2026
7 ACKNOWLEDGEMENTS
DR received financial support from the Natural Sciences and Engineering Research Council's (NSERC) Banting Postdoctoral Fellowship under Funding Reference No. 198506. OD is supported by both Project CETI and Intel. AJB is partially supported by an NSERC Postdoctoral Fellowship and by the EPSRC Turing AI World-Leading Research Fellowship No. EP/X040062/1 and EPSRC AI Hub No. EP/Y028872/1. JT acknowledges funding from the Canada CIFAR AI Chair Program and the Intel-Mila partnership program. The authors acknowledge funding from UNIQUE, CIFAR, NSERC, Intel, and Samsung. The research was enabled in part by computational resources provided by the Digital Research Alliance of Canada (https:/ /alalaaaanncann ca), Mila (https://mila. quebec), and NVIDIA
ETHICS STATEMENT
This paper is primarily methodological, presenting theoretical developments without direct exper- imental implementation or associated ethical considerations; however, we advise due caution for future beneficiaries of our work in their potentially sensitive application domains.
REPRODUCIBILITY STATEMENT
We have made numerous efforts ensure the reproducibility of our work. The main paper provides detailed descriptions of the proposed methods and evaluation protocols. Additionally, in our Ap- pendix, include extensive details data normalization, the MD datasets used for training, model architectures and sizes, training configurations, and our choice of regularization hyperparameters. Further, all assumptions and methodological choices explicitly documented, and we plan to publicly release all the developed code upon acceptance.

---

Published as a conference paper at ICLR 2026
REFERENCES
S. Agapiou, O. Papaspiliopoulos, D. Sanz-Alonso, and A. M. Stuart. Importance sampling: Intrinsic dimension and computational cost. Statistical Science, pages 405-431, 2017.
M. S. Albergo and E. Vanden-Eijnden. Building normalizing flows with stochastic interpolants. International Conference on Learning Representations (ICLR), 2023.
M. S. Albergo, N. M. Boffi, and E. Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprin! 2303.08797, 2023.
D. Andrade. Stable training of normalizing flows for high-dimensional variational inference. arXiv preprint arXiv:240.. 16408, 2024.
Behrmann, W. Grathwohl, R. T. Q. Chen, D. Duvenaud, and J.-H. Jacobsen. Invertible residual networks, 2019. URL
/aapa////111..99900 111.00995.
Benton, G. Deligiannidis, and A. Doucet. Error bounds for flow matching methods. arXiv preprint arXiv:2305. 16860, 2023.
Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, Zhuang, J. Lee, Y. Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dalle-3. pdf. 2(3):8, 2023.
N. M. Boffi, M. S. Albergo, and E. Vanden-Eijnden. Flow map matching. arXiv preprint arXiv:2406. 00500000007..... 2, 2024.
N. M. Boffi, M. S. Albergo, and Vanden-Eijnden. How build a consistency model: Learning flow maps via self iif-dditillation, 2025. URL httpss//arxii hwtp://wwwrg.///////////// org/abs/2505 18825.
A. J. Bose, M. Brubaker, and I. Kobyzev. Equivariant finite normalizing flows. arXiv preprini I:Xiv:2100088644 2021.
T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, C. Ng. R. Wang, and A. Ramesh. Video generation models as world simulators. 2024. generationn denerration-moelll models-as- -sor-lllsllldd www.wwlddsimlltloooooooooooooooo simulators.
URL https://opent
R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud. Neural ordinary differential equations. Neural Information Processing Systems (NeurIPS), 2018.
R. T. Q. Chen, J. Behrmann, D. Duvenaud, and J.-H. Jacobsen. Residual flows for invertible generative modeling, 2020. URL
wttp://ww..oggbb///g/ee//// 02735.
L. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. arXiv preprint
2014.
L. Dinh, J. Sohl-Dickstein, and S. Bengio. Density estimation using real nvp. arXiv preprint vrXiv:1605.0850 2016.
C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Neural spline flows. Advances in neural information processing systems, 32, 2019.
C. Ermann, R. Ke, and C.-.. Schönlieh. iunets: Fully invertible u-nets with learnable up- and downsampling, 2020. URL
httpariwv..ogg/////eeeee/ 05220.
K. Frans, Hafner, S. Levine, and P. Abbeel. One step diffusion via shortcut models. arXiv preprint arXiv:2410.1 12557, 2024.
Geffine,, Didi. Z. Zhang, D. Reidenbach, Z. Cao, J. Yim, M. Geiger, C. Dallago, E. Kucukbenli. A. Vahdat, et al. Proteina: Scaling flow-based protein structure generative models. arXiv preprint 007000. 2025.
Z. Geng, A. Pokle, W. Luo, J. Lin, and J. Z. Kolter. Consistency models made easy, 2024. URL https://axxiv.org/bbb.... 14554
12

---

Published as a conference paper at ICLR 2026
Z. Geng, M. Deng, X. Bai, J. Z. Kolter, and K. He. Mean flows for one-step generative modeling, 2025. URL /ttps://arxiv. wwtpeeewwwwioo.r//erreeeeeeeeee org/abs/2505. 13447.
D. Ghamari, Hauke, R. Covino, and P. Faccioli. Sampling rare conformational transitions with quantum computer. Scientific Reports, 12(1):16336, 2022.
E. Hairer, S. P. Nørsett, and G. Wanner. Solving Ordinary Differential Equations I: Nonstiff Problems, volume 8 of Springer Series in Computational Mathematics. Seringer-Verlage 2nd edition, 1993. ISBN 978-3-540-56677
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition, 2015. URL https: / http///aaivvo.oobbrra////e/////e 512.03385.
Heek, E. Hoogeboom, and T. Salimans. Multistep consistency models, 2024. URL https:
//arxiv.org orgg/ http//bb//2..0000
J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models, 2020. URL https: 11239.
//
G. Huguet, J. Vuckovic, K. Fatras, E. Thibodeau-Laufer, P. Lemos, R. Islam, C.-H. Liu, J. Rector- Brooks, T. Aahhund-Sadeggh M. Bronstein, et al. Sequence Sequence-augmentte augmented se (3)-flow matching for conditional protein backbone generation. arXiv preprint arXiv:2405.20313, 2024.
K.-H. Hui, C. Liu, X. Zeng, C.-W. Fu, and A. Vahdat. Not-so-optimal transport flows for 3d point cloud generation. ICLR, 2025.
M. F. Hutchinson. stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in KaatssicssSimulled and Computation, 18(3):1059-1076, 1989.
I. Ishikawa, T. Teshima, K. Tojo, K. Oono, M. Ikeda, and M. Sugiyama. Universal approximation property of invertible neural networks. Journal of Machine Leurning Research, 24(287) 2023.
D. Kim, C.-H. Lai, W.-H. Liao, N. Murata, Y. Takida, T. Uesaka, Y. He, Y. Mitsufuji, and S. Ermon. Consistency trajectory models: Learning probability flow ode trajectory diffusion, 2024. URL https://arxiv.or wwtp://g//abb////// 02279.
D. P. Kingma and P. Dhariwal. Glow: Generative flow with invertible 1xl convolutions, 2018. URL
nttps://arxins www./oog///////.o..oooooooo. /a0s///8001 8078877.0 03039.
L. Klein, A. Krämer, and F. Noé. Equivariant flow matching. Advances in Neural Information Processing Systems, 36:59886-59910 2023.
A. Kolesnikov, A. S. Pinto, and M. Tschannen. Jet: A modern transformer-based normalizing flow. arXiv preprint arXiv:2412./5/29 2024.
Z. Kong and K. Chaudhuri. Universal approximation of residual flows in maximum mean discrepancy. arXiv preprint arXiv:2103.055 05793,, 2021.
Y. Lipman, Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. International Conference on Learning Representations (ICLR), 2023.
J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, 2001.
Q. Liu. Rectified flow: A marginal preserving approach to optimal transport. arXiv preprint 2rXiv:2209. 14577, 2022.
X. Liu, C. Gong, and Q Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. International Conference on Learning Representations (ICLR), 2023.
C. Lu and Y. Song. Simplifying, stabilizing and scaling continuous- timeeuuuuu---eeeeee consistency models. arXiv preprint arXiv:2410.11083 2024.
S. Moqvist, W. Chen, M. Schreiner, F. Nuske, and S. Olsson. Thermodynamic interpolation: A generative approach to molecular thermodynamics and kinetics. Journal of Chemical Theory and Computation, 21(5):2535-2555 2025.
13

---

Published as a conference paper at ICLR 2026
F. Noé, S. Olsson, J. Köhler, and H. Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457)ceaww1 110t 2019.
G. Papamakarios, E. Nalisnick, D. Rezende, S. Mohamed, and B. Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57): 1-64, 2021.
Peluchetti. Non-denoising forward-time diffusions. arXiv preprint arXiv:2312.14588 2023.
G. Peyré and M. Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning, 1 1(55-6)335-660)) 2019.
D. Rezende and S. Mohamed. Variational inference with normalizing flows. In International conference on machine learning, pages 1530-1538. PMLR, 2015.
A. Rizzi, P. Carloni, and M. Parrinello. Targeted free energy perturbation revisited: Accurate free energies from mapped reference potentials. The journal of physical chemistry letters, 12(39): 9449 9454, 2021.
A. Sabour, S. Fidler, and K. Kreis. Align your flow: Scaling continuous-time flow map distillation, 2025. URL https://arxits http:////aarri.ooraa////////// 14603.
A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach. Adversarial diffusion distillation. In European Conference Computer Vision, pages 87-103. Springer, 2024.
J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models, 2022. URL https://axxii.. org/abs/2011 02502.
Y. Song and Dhariwal. Improved techniques for training consistency models, 2023. URL https://arxin arg/abs//311 14189.
Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models, 2023. URL https://///: org/ 230/2 01 469.
E. G. Tabak and C. Turner. A family of nonparametric density estimation algorithms. Communica- tions on Pure and Applied Mathematics, 66(2):145-164 2013.
E. G. Tabak and E. Vanden-Eijnden. Density estimation by dual ascent of the log-likelihood. Communications Mathematical Sciences, 2010.
C. B. Tan, J. Bose, C. Lin, L. Klein, M. M. Bronstein, and A. Tong. Scalable equilibrium sampling with sequential boltzmann generators. arXiv preprint arXiv:2502. 184622, 2025a.
C. B. Tan, M. Hassan, L. Klein, S. Syed, D. Beaini, M. M. Bronstein, A. Tong, and K. Neklyudov. Amortized sampling with transferable normalizing flows. arXiv preprint arXiv:250.. 18175, 2025b.
T. Teshima, I. Ishikawa, K. Tojo, K. Oono, M. Ikeda, and M. Sugiyama. Coupling- invertible neural networks are universal diffeomorphism approximators. Advances in Neural Information Processing Systems, 33:3362-3373, 2020.
A. Tong, N. Malkin, G. Huguet, Y. Zhang, J. fector-Brooks K. Fatras, G. Wolf, and Y. Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.0044
2023.
C. Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
P. Wirnsberger, A. J. Ballard, G. Papamakarios, S. Abercrombie, S. Racanière, A. Pritzel, D. Jimenez Rezende, and C. Blundell. Targeted free energy estimation via learned mappings. The Journal of Chemical Physics, 153(14), 2020.
Z. Xu and T. Campbell. Embracing the chaos: analysis and diagnosis of numerical instability in variational flows. Advances in Neural Information Processing Systems, 36:32360-32386. 2023.
T. Yin. M. Gharbi, R. Zhang, E. Shechtman, Durand, W. T. Freeman, and T. Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 1661-6663,, 2024.
14

---

Published as a conference paper at ICLR 2026
S. Zhai, R. Zhang, P. Nakkiran, D. Berthelot, J. Gu, H. Zheng, T. Chen, M. A. Bautista, N. Jaitly, and Susskind. Normalizing flows are capable generative models. arXiv preprint arXiv:2412.06329, 2024.
H. Zhang, X. Gao, J. Unterman, and T. Arodz. Approximation capabilities of neural odes and invertible residual networks. In International Conference on Machine Learning, pages 11086- 11095. PMLR, 2020.
L. Zhou, S. Ermon, and J. Song. Inductive moment matching. arXiv preprint arXiv:2503.07565, 2025.
M. Zhou, H. Zheng. Z. Wang, M. Yin, and H. Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. Forty-first International Conference on Machine Learning, 2024.
R. W. Zwanzig. High-temperature equation of state by perturbation method. nonpolar gases. The Journal of Chemical Physics, 2208):1420-1422 08 1954. ISSN 0021-9606. doi: 10. 106/01 1740409. URL https::///ooii. 11440/ 1740409.
1063/1
15

---

Published as a conference paper at ICLR 2026
A PROOFS
A.1 PROOF OF PROPOSITION
We first recall proposition below.
Proposition 1. Suppose that St is invertible for all 1, that (ft)-1 continuous for all t. Then, 1 o ft,o)(a)) - for almost all (with respect to po) x.
(0) + 0, it holds that ((f;)=1
To prove proposition 1, we first prove the following lemma, which is essentially the same as the proposition, but it abstracts out the distribution of It, which depends on Zo. Iı, and t.
Lemma 1. For fanctions (Jn)n21 and g. where 9 is invertible and has a continuous inverse, Bo ~ po, if MSE(Jn g) = Emp (zo) - g( + O, then lit 9 (fn(z)) = x for almost all (with respect to po)
Proof. Let Y, = (2)) (200)l22. We know that lim, E = 0 (as it corresponds to the MSE), which implies that lim Var(Y.) O. Consequently, Yn for
constant E IR. Moreover, by Jensen's inequality and the convexity of x + x= we find that (E[Y,))² S EYY meaning that 0. This implies that lim, (n 2(>)
almost everywhere, and thus that lim, fn (2r) = ((x). Finally, since g is continuous, we can apply the function to both sides of the limit to find that lim 9
(aat--oooonfeeeeeee (x) x, almost everywhere.
It suffices to apply the above lemma to 24 pe( Fo x1 (r To))o(Fo)-
A.2 PROOF OF PROPOSITION
We now prove proposition 2. The proposition reuses the following regularity assumptions, as introduced in Benton al. (2023), which recall verbatim below for convenience:
(Assumption 1)
Let Vhue be the true generating velocity field for the CNF with field v" trained using flow matching. Then the true and learned velocity v² are close in l and satisfy:
So E,, (llu, (1+.) vf a+)) [[]]dl < K
(Assumption 2) For each x E Ied and E [0, 1), there exists unique flows (fr,)) and (fs.t).me))E(()) starting at f(r..) = x and f(s.s),tmu with velocity fields of (Ix) and tre (x),, respectively. Additionally, f" and fime continuously differentiable in x, and t.
(Assumption 3) The velocity field oī (Ix.) is differentiable in both and and also for each E [0, there exists a constant L, such that of (xx) is L--Lipschitz in x.
Let Preflo be a pretrained CNF generated by the vector field vf, real numbers (Le)seto,1 such that oi Li-Lipschitz for all [0, 1], and NF J trained using Eq. 5 by regressing against frggou (zo), where 2o ~ (0, Then, writing nf = Law (f (2o)), we have:
Proposition 2.
(2(p1.Po) S K exp (L. Ledt) -c, K KN f.' ([1u (xxe) Vtr
L
(x=) (x
t di, (10)
where K is the l2 approximation error between the velocity field of the CNF and the ground truth generating field u?, e² Exo.. (ijipoo (xo)
Proof. We begin by first applying the triangle inequality to Wz(p1, pa) and obtain:
Wz(p1,Po) Po) I W2(p1, Pretiow) + W2 PPreelow.PP p.
(11)
The first term is an error in Wasserstein-2 distance between the true data distribution and reflow targets, which is still a CNF. straightforward application of Theorem in Benton et al. (2023)
gives bound on this first Wasserstein-2 distance
W2 (PtLProtow)) S K exp Lodtt (Lott)
(12)
A sharper bound can be obtained with additional assumptions, demonstrated in Benton al. (2023), but it is not critically important in context.
16

---

Published as a conference paper at ICLR 2026
To bound W22 Predtow Po), recall that the following inequality holds (2La((X),,aw(X) 1 E | Y [2] for any two random variables X and Y. In our case, these random variables are Pictow = Law (ictoo((a)) and pof = Laww( (2o)). This gives:
W2 W2(Pretoow Pá Euo. 21
[ (2o) f& (zo) [1
22
(13)
Combining eq. (12) and eq. (13) achieves the desired result and completes the proof.
Wa(p1. $ K exp (Lo Lidt) ((dtt)
+ Ezg.21 [| (xo) fol (zo (zo))l2
27
(14)
Note that the bound on W2 (Prefooow Pe ) is effectively the square-root of the REGFLOW objective and thus optimization of the NF using this loss directly minimizes the upper bound to W2 (pp.p))))
A.3 REGFLOW IN CONTINUOUS TIME
Current state-of-the-art CNFs are trained using "flow matching" (Lipman et al., 2023; Albergo and Vanden-Fijnden 2023; Liu et al., 2023), which attempts match the vector field associated with the flow to a target vector field that solves for mass transportation everywhere in space and time. Specifically, we can cast conditional flow matching (CFM) (Tong et al., 2023) from the perspective of REGFLOW. To see this explicitly, consider a pre-specified probability path, (x and the & following fi.m or (xt)). However, since it is generally computationally challenging to sample from Pt directly, the marginalization trick is used to derive an equivalent objective with conditional Jietm We note that REGFLOW requires Si,eim to be invertible therefore this assumes regularity on 0
P. (1e). This is generally satisfied by adding small amount of noise the following. We present this simplified form for clarity.
(xx) pe( z (ro. x1), Pt ( 0(zii (1 t)xo + tx
(15)
Then setting fi.efm
p. Pe (x x1) it is easy to show that:
a
|olooar) 01 (1
2 |ewoter) p)pe(tr)) 1 +C,
[())=
E 2i
= Er
E 11
De (xx)) - Ji, (zx))l
with independent of (Lipman et al., 2023), and x is a loss weighting, which fits within the REGFLOW framework in the continuous-time setting with the last equality known as target/end -point prediction.
A.4 REQUIREMENTS FOR REGFLOW TARGETS
In practice, REGFLOW deals with discrete couplings. Any discrete coupling is usable for training RegFlow long there exists invertible function on the continuous domain which agrees with it. This leaves us with easily verifiable necessary and sufficient properties for the base coupling. Specifically, let N (2n, 21) denote the coupling between empirical point sets 20, 21 in Rd.
Proposition 3. If Tt is a permutation and there does not exist i + j such that xa = xá or a = ai_
Then there exists invertible function f RRd - Rd such that for all zq E Dn. f(x)) I and for all oi F1, (z+) z/-
Proof. We proceed by constructing an example f which satisfies the necessary properties. First we denote f xo + Ii the discrete invertible function mapping the point set zo to the point set ø1.
Let
(x) (x)
if . E 2D
f(x)
if E else
This function is invertible on R and satisfies the necessary properties the proposition on the domains of do and æ1.
17

---

Published as a conference paper at ICLR 2026
7
S
5
9
3
8
6
7
7
2
7
9
7
O
9
7
9
5
Q
H
Y
R
2
7
7
F
Y
Q
Z
9
5
O
/
5
O
6
O
2
3
G
6
G
O
6
S
O
Z
6
D
2
6
3
6
C
2
4
2
S
(a) 1-step.
(b) 2-step.
(c) 4-step.
(d) 8-step.
(c) 66-stpp.
Figure 6: Generations of IMM trained with an iUNet with variable number of steps.
This proposition establishes necessary and sufficient conditions for training valid REGFL.OW between zy and 21- This property is quite simple to obtain in practice. For both OT-coullings and CNF-couplings is almost by definition a permutation. The only trouble is if there exist duplicate points. This is a measure-zero event in continuous space, and therefore is not an issue.
In fact, any random permutation matrix satisfies these conditions. Which leads to the perhaps more interesting question of what are the properties of a "good" coupling т. In some sense we are looking for x that are "good" couplings that are somehow "easy" learn and generalizes well when trained with the REGFLOW procedure in given setting.
In this work we used REGFLOW to improve the training speed and convergence of the same normal- izing flow architectures that are normally trained using maximum likelihood (MLE).
We believe the classic MLE objective would help when it is difficult to find a good coupling for the given architecture, dataset, and REGFLOW learning framework. In this work we established that there exist settings where OT and CNF-couplings outperform the MLE objective. We leave it to future work to study the optimal couplings in given setting.
B ADDITIONAL BACKGROUND
B. INDUCTIVE MOMENT MATCHING
Introduced in Zhou et al. (2025), Inductive Moment Matching (IMM) defines a training procedure for one-step generative models, based diffusion/flow matching. Specifically, IMM trains models to minimize the difference in distribution between different points in time induced by the model. As a result, this avoids direct optimization for the predicted endpoint, in contrast to conventional diffusion.
More precisely, let fa Rd x [o, 112 - Rd,, (I. s, t) + fo(x, s, t) be a function parameterized by A. IMM minimizes the following maximum mean discrepancy (MMD) loss:
[(0,) E..,.0001 [w]ss 0)MMD* (po.. 1.(elr) r S 1, with 8, ~0, 1) iid, 2 0 is a weighting function, 21 is sample from the target distribution, zig ~ AO 1), x. is some interpolation between zo and æ1 at time (typically, using the DDIM interpolation (Song et al., 2022)), the subscript E N of parameter 9 refers to its training step, and MMD is some MMD function based on a chosen kernel (typically, Laplace). Essentially, the method uses as target the learned distribution of the previous step at higher time to train the current distribution at lower times. With a skip paramcterization, the higher time distribution is by construction close to the true solution, as pe (I, 2,)) s p(z, z,,) when r R S, and I, is known. (Or, in other terms, fo (z, S, r R 5) RJ x with the skip parameterization.) When the distributions match (when the loss is zero), MMD* model's 's and the target distribution's 's respective moments all match.
(16)
where 0 S
= O, and so the generative
This training procedure allows for variable-step step sampling. For chosen timesteps, (ts) one can sample from P1.e by sampling my N( I) and performing the steps:
Itss1 + DD (Zx, t441, ti), Fe, ti, (1+1).
(17)
where DDIM is the DDIM interpolant.
18

---

Published as a conference paper at ICLR 2026
0
3
2
7
3
2
2
3
+
7
2
4
4
1
3
4
8
6
4
4
4
3
2
7
7
X
3
7
4
73
5
J
9 5 4
4
3
2
9
7
3
4
!
7
4
8
3
0
5
7
x
{
f
i
2
3
(a) Using the ResFlow architecture proposed in Chen al. (2020).
(b) Using the TarFlow architec-
(c) Using the TarFlow architec- ture (Zhai al., 2024), 16.
ture (Zhai al., 2024), 4.
Figure 7: One-step generation results with a Lipschitz-constraince constrained (ResFlow) model and an invertible model (TarFlow) for IMM. The m parameter is the group size in IMM used to approximate the MMD.
B.2 INDUCTIVE MOMENT MATCHING NEGATIVE RESULTS
We detail in appendix B.1 the Inductive Moment Matching (IMM) framework (Zhou et al., 2025). Observing the sampling procedure, which we give in eq. (17), one can make this procedure invertible by constraining the Lipschitz constant of the model, or by using invertible model. For the first case, we use the "Euler" (skip) parameterization alongside the DDIM interpolation, it is shown that the reparameterized model go be written
(18) Moreover, 0 s 1, and so if the Lipschitz constant of fe is strictly less than one, then the overall model is invertible, using the argument of residual flows (Behrmann et al., 2019); so the change of variables formula applies follows (using the time notation of IMM/diffusion):
Vx, s,
90(I, s,t) = x- (s fo s,t).
log pi (z) log (o
Σ log [(t++1 - t.) det (t
(19)
The difficulty of evaluating the log-determinant of the Jacobian remains. Note, however, that we do need find the inverse of the function to evaluate the likelihood of generated samples, since we know each (i The second path (of using an invertible model) is viable only for one-step sampling with no skip parameterization (which, according to Zhou al. (2025), tends under-perform, empirically), since the sampling procedure then boils down to 21 = f(xo, 1, 0) for zy ~ NO, I).
While both approaches succeeded in synthetic experiments, they fail to scale to datasets such as MNIST, the results of which include here in fig. 6 and in fig. 7. We have tried iUNet (Etmann normalizing
et al., 2020) and TarFlow (Zhai et al., 2024), an invertible UNet and a Transformer-bast flow, respectively, for invertible one-step models; and we have tried the ResFlow architecture in (Chen et al., 2020) for the Lipschitz-conssraanedd approach. As observed, TarFlow fails to produce images of high quality; iUNets produced significantly better results, albeit still not sufficient, especially for the one-step sampling, which is the only configuration that guarantees invertibility; the Lipschitz-constrained ResFlow entirely failed to produce satisfactory results, although the loss did diminish during training. In general, an even more important limitation is the difficulty of designing invertible Lipschitz-constrained models for other data types, for instance, 3D coordinates. Perhaps further research on the architectural side could allow for higher performance with invertible sampling.
C EXPERIMENTAL DETAILS
C. METRICS
The performance metrics considered across the investigated flows were the effective sample size, ESS, Wasserstein-1 energy distance, E-W,, and the Wasserstein-2 distance on dihedral angles, T-W,.
Note that we have adapted IMM's notation our time notation, with noise at time zero, and clean data time one.
19

---

Published as a conference paper at ICLR 2026
Effective Sample Size (ESS). We compute the effective sample size (ESS) using Kish's formula, normalized by the number of samples generated:
N 1
wi
ESS (!ws)
(20)
N
2i=1 wi
where w; is the unnormalized weight each partiele indexed by over N particles. Effective sample size measures the variance of the weights and approx how many more samples would be needed compared an unbiased sample. For us, this captures the local quality of the proposal relative to the ground truth energy. It does not rely on a ground truth test set; however, it is quite sensitive and may be misleading in the case of dropped modes or incomplete coverage, as it only measures agreement on the support of the generated distribution.
Wasserstein-1 Energy Distance (E-Wi). The Wasserstein-1 I energy distance measures how well the generated distribution matches some ground truth sample (often generated using MD data) by calculating the Wasserstein-1 distance between the energy histograms. Specifically:
(-Wi(I, min
yldT(z, (2, y),
(21)
J.yy
where is a valid coupling of p(x) and p(y). For discrete distributions of equal size, can be thought of as a permutation matrix. This measures the model's ability to generate very accurate structures as the energy function values. When the bond lengths have minor inaccuracies, the energy can blow up extremely quickly.
requires extremely accurate bond lengths obtain reasonable energy
Torus Wasserstein (T-W22) The torus Wasserstein distance measures the Wasserstein-2 dis- tance on the torus defined by the main torsion angles of the peptide. That is for a peptide of length there are 2(1 1) torsion angles defining the dihedrals along the backbone of interest ((0), wh)), (02, v (Ør, 4/)).. We define the torus Wasserstein distance over these backbone angles as:
T-V2(p.g)² min [ cT( u)² dr
(22)
where is valid coupling between p and q, and (x, u)² is the shortest distance on the torus defined by the dihedral angles:
2(L-1) (Dihedrals(z)) - Dihedrals(y); +) mod 2z
C7 (F.
Σ
(23)
2=
The torus Wasserstein distance measures large scale changes and is quite important for understanding mode coverage and overall macro distribution. We find REGFLOW does quite well in this regard.
C.2 ADDITIONAL DETAILS ON EXPERIMENTAL SETUP
To accurately compute the previously defined metries, 250k proposal samples were drawn and re- re-weighted for alanine dipeptide, tripeptide, and tetrapeptide.
Data normalization. We adopt the same data normalization strategy proposed in (Tan al., 2025a), in which the center of mass of each atom is first subtracted from the data, followed by scaling using the standard deviation of the training set.
Exponential moving average. We apply exponential moving average (EMA) the weights of all models, with a decay of 0.999, as commonly done in flow-based approaches to improve performance.
Training details and hardware. All models were trained on NVIDIA L40S 48GB GPUs for 5000 epochs, except those using OT targets, which were trained for 2000 epochs. Convergence was noted earlier in the OT experiments, lcading to early stopping. The total training time for all models is summarized table 7. The time taken to compute the OT map is also provided; since computing the OT map independent of the feature dimension, but only on the number of data points used, the compute time was relatively consis- tent across all datasets. A total of 100k points was used for training the CNF, performing MLE training, and computing the OT map.
Table 7: REGFLOW training time (in hours) ALDP, AL3, and AL4.
Model
ALDPP
A1.3
AL.4
OT map
3.6
3.8
3.8
DIT CNF
27.6
40.7
48.6
NSF
21.0
23.8
26.8
Res-NVP
15.7
15.6
15.0
Jet
19.1
19.2
20.1
20

---

Published as a conference paper at ICLR 2026
Reflow targets. Ablations done to investigate the influence of synthetic data quantity all metrics. For all benchmarking performed against MLE training, the largest amount of synthetic data was used. For ALDP, AL3, and AL4, this constituted 10.4M, 10.4M, and 10M samples, respectively.
Determinant regularization. During REGFLOW, it was initially ob- served that as proposal sample quality improved. the re-weighted sam- ples progressively deteriorated across all metrics due to the models becoming numerically non non-invertible. This was partially addressed by adding regularization to the loss in the form of log determi- nant penalty. Sweeps were conducted using multiple regularization weights ranging between 10-7 and 10 to prevent hampering sam- ple performance. The amount of regularization added was function of the flow and dataset. The final weights are summarized in table 8.
Table Regularization weights used across datasets and flows.
Model
ALDP
AL3
AL4
NSF
10-6
10-5
10-5
Res-NVP
10-3
10-5
10-6
Jet
10-3
10-
10-3
Target noise. To discourage numerical con-invertibility of the trained flows, Guassian noise was also introduced to the target samples. Experiments were conducted with noise magnitudes of 0.01, 0.05, 0.1, and 0.25, with final value of 0.05 being selected for use across models and datasets.
REGFLOW implementation details. A summary of all trained model configurations is provided in table 9. To maintain fair comparison, the configurations reported below were unchanged for MLE training and REGFLOW. Adam was used as the optimizer with learning rate of 5 10 and a weight decay of 0.01. We also included a varying cosine schedule with warmup in line with the approach suggested in (Tan al., 2025a).
Table 9: Model configurations for the DiT CNF, NSF, Res-NVP, and Jet across all datasets (ALDP, AL3, AL4).
A dash (-) indicates the parameter is not applicable to the respective model.
Model
hidden features
transforms
layers
blacks per layer
conditioning dim.
heads
dropout
parameters (M)
DIT CNF
768
128
12
0.1
46.3
NSF
256
24
76.8
Res-NVP
512
6
0.1
80.6
432
0.1
Quality of CNF targets. To maximize the likelihood that models trained with REGFLOW have the potential to outperform MLE, securing high-quality targets is essential. In line with this pursuit, a CNF with a diffusion transformer backbone was used. In fig. 8, the true data and the CNF proposal are shown, where can be seen that the learned energy distributions across all three peptides are nearly perfect. Re-weighted samples are not included as obtaining likelihoods from the CNF requires estimating the trace of the divergence, which often an expensive operation with large time and memory cost. Although many unbiased approaches for approximating the likelihood exist (Hutchinson, 1989), these methods are typically unusable for Boltzmann Generators due to their variance, which can introduce bias into the weights needed for importance sampling.
DIT CNF (ALDP)
DIT CNF (AL3)
DIT CNF (AL4)
Normalized Density
Proposal
True data
Normalized Density
0.06
Proposal
True data
Normalized Density
0.05
True data Proposal
0.06
0.05
0.03
0.04
0.03
0.02
0.02
0.02
0.01
0.01
0.00
-50
50
100
-200
-150
-100
-50
50
100
E(x)
8(x)
8(x)
Figure 8: True energy distribution and learned proposal using the DiT based CNF. "The re- weighted proposal is not present because it too computationally expensive to compute for a sufficient number of points.
21

---

Published as a conference paper at ICLR 2026
(RegFlow w/ OT Targels)
Res ReP (RegFlow Targets)
Jet (RegFlow Targets)
data
0.12
0.12
Normalized Density
Proposal
Nlrmalized Density
Proposal
Normalized Density
Proposal
0.10
Proposal (rewsighted)
0.10
Proposal (reweigttedd
0.10
Proposal trewe ghted)
0.08
0.08
0.08
0.06
0.06
0.04
0.04
0.04
0.02
0.02
0.02
-50
50
100
0.00
-50
0
100
-50
100
8(x)
E(x)
E(x)
Figure 9: Energy distribution of the original and re-weighted samples, as well as the true data, when using 100,000 OT targets on ALDP (left: NSF (REGFLOW); center: Nes-NVP (REGFLOW); right: Jet (REGFLOW)).
D ADDITIONAL. RESULTS
D.1 REGFLOW PERFORMANCE USING OT TARGETS
Optimal transport targets. In addition using reflow targets from a pre-trained CNF, we pre- compute an OT map to obtain an invertible pairing between source and target samples. We combine this map with REGFLOW training, and report results in fig. 9 for alanine dipeptide. Here, demonstrate an example of where REGFLOW training goes beyond distillation and can serve as an effective approach at training classical normalizing flows on diverse invertible maps.
D.2 PERFORMANCE ON LARGER PEPTIDES
Alanine tripeptide and alanine tetrapeptide We demonstrate the learned distributions of the two pairs of dihedral angles that parameterize alanine tripeptide and tetrapeptide using our best MLE-trained and REGFLOW flows fig. 11 and fig. 12. The inability to capture the modes using MLE is elucidated, where multiple modes appear to blend together in both sets of dihedral angles in fig. 11. Conversely, using REGFLOW, most modes are accurately captured and the general form of the Ramachandran plots conforms well to that of the true distribution obtained from MD. The findings observed with alanine tripeptide are even more pronounced with alanine tetrapeptide, where certain modes are entirely missed when MLE-trainee flows used, however, most modes are accurately captured, and the density distribution is in strong agreement with the ground truth data. These findings clearly demonstrate the utility of a regression- basoddd--aadddoee training objective over conventional MLE for applications to equilibrium conformation sampling of peptides.
in fig. 12. With REGFLOW,
In fig. 10, we demonstrate that the energy distribution of the re-weighted samples using REGFLOW, which yields more favourable energy distribution over MLE-trained flows. For the tripeptide, the results are in strong agreement with MD. For the tetrapeptide, the re-weighted samples are superior than their MLE counterparts, but have room for improvement in matching the true energy distribution.
AL3 (MLE)
AL3 (RegFlow)
AL4 (MLE)
AL4 (RegFlaw)
Tox date
The data
Tue daca
Tru data
Norralized Den st
Iepeaa
Praparal
Normalized Demsily
0.10
Prppoaa
Mormalized Densty
5.10
Prepesal
0.00
tsseiighto
manegttd
0.03
0.02
0.08
0.00
0.00
0.08
5,06
0.06
Normalizze
0.03
0.03
10.44
0.54
0.02
0.02
0.03
0.02
Figure 10: Energy distribution of original and re-weighted samples generated for the most performant MLE and REGFLOW models on alanine tripeptide (left and center left) and alanine tetrapeptide (center right and right).
D.3 GENERATED SAMPLES OF PEPTIDE CONFORMATIONS
Samples of generated peptides. Below we provide sample conformations of alanine dipeptide generated using both MLE training and REGFLOW in fig. 13. In addition, we include sample molecules of the larger peptides, obtained through REGFL.OW training as well in fig. 14.
22

---

Published as a conference paper at ICLR 2026
Ground Truth MD
MLE
RegFlow
Free energy kuT
2.0
0.0
-1
Free energy kgT
7
Figure 11: Ramachandran plots for alanine tripeptide (left: ground truth, middle: best MLE-trained flow, right:
best REGFLOW flow). REGFLOW captures most modes, while MLE-trained flows struggle.
Ground Truth MD
MLE
RegFlow
Free energy ka
2.0
π
SP0
SPo
:Po
4.0
Free energy knl
2.0
91
Free energy knl
0.0
0
0
2
4
Figure 12: Ramachandran plots for alanine tetrapeptide (left: ground truth, middle: best MI.E-trained flow, right: best REGFLOW flow). REGFLOW captures most modes, while MLE-trained flows struggle.
D.4 TARGETED FREE ENERGY PERTURBATION
Generating regression targets. Using the available MD data, two conformations of alanine dipeptide were selected: Splanar and oR (Ghamari et al., 2022). The (ó, 1b) ranges for the 3planar conforma-
23

---

Published as a conference paper at ICLR 2026
Figure Generated conformations alanine dipeptide across various flow-hased methods (left: NSF w/ ML.;; center left: NSF w/ REGFLOW: center right: Res-NVP w/ REGFLOW; right: Jet w/ REGFLOW.
Figure 14:
Generated samples of larger peptides using NSF (REGFLOW) (left: ALDP; center: AL3; right: AL4).
tion were chosen as (-2.5, 2.2) and (2.3, 2.6), and for the oR conformation as (-1.45, 1122) and (-0.7, -04),, respectively. The dataset was then truncated to 82,024 source-target conformation pairs, which were used to compute the OT pairing and generate an invertible map. These pairs were subse- quently trained using REGFLOWW with the same model configurations and settings outlined in table 9.
24


Reviews:
[{"review_id": "887A9L7Rce", "reviewer": "Reviewer_Upe7", "summary": "This paper describes REGFLOW, an approach for training a normalizing flow that performs better than traditional maximum likelihood training.  The approach regresses to predetermined flows, either from another model or precomputed optimal transport couplings. This approach results in substantial better normalizing flow models than traditional training.", "questions": "Why are NFs trained with REGFLOW substantially faster at computing likelihoods?\n\nHow does increasing the number of OT couplings improve performance? What if the OT is approximate?\n\nWhat is the basis for the statement that beyond a certain level of regularization that numerical invertability is guaranteed? Is this an empirical statement, or is there are proof (if the former, than perhaps an alternate wording would be more accurate).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper addresses a chronic problem with normalizing flows.  The described regression loss is intuitive an simple to implement (once couplings have been determined).  The approach results in dramatic improvements compared to MLE training using the same models and data.  Sensitivity to some parameters (e.g. regularization) is explored. I appreciate the evaluation of free energy.  The paper is well written and easy to follow.", "weaknesses": "Although the improvement compared to NF models is extreme, the results aren't necessarily state-of-the-art compared to other models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WMVZiDdgX7", "reviewer": "Reviewer_fKR3", "summary": "This manuscript outlines a new approach to train one-step generative models, specifically normalizing flows, which allow exact sample likelihood evaluations. This work is important as likelihood evaluation is a bottleneck for applications of NF in the sciences, as most prominently outlined here: Boltzmann Generators. The idea outlined in this paper is simple and effective: train a regular NF against a pre-specified flow (e.g. either a pre-trained Continuous NF, or a pre-computed OT flow). This allows for faster and more stable training and more efficient sample likelihood evaluations in most cases.\n\nWhile the method seems like it still has some ways to go to be ready for prime-time, I find that the paper, overall, is an interesting conceptual step. Consequently, i am willing to increase my score if the concerns below are addressed.", "questions": "- How do the authors envision this approach scale to larger systems? Comparing figures 7 and 8 it seems like the RegFlows miss important details that might be important for performance, and one would expect this to only increase with system size.\n- Are observables --- e.g. free energies --- computed under the DiT-based CNF meaningfully different from those under the reweighed values from the presented RegFlow approach?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Conceptually clear and well written manuscript. Numerous insightful comments about normalizing flows.\n- Well thought-out experiments and evaluations. All fairly standard in the field now, but still well done.\n- Clear performance gain, in terms of compute-time, over most of the included baselines.", "weaknesses": "- Claims and attribution. The proposed TFEP method is closely related to the ambient thermodynamic interpolant approach by Moqvist et al https://arxiv.org/abs/2411.10075 \n- Sample quality and scaling. ESS remain fairly low. Scaling to tetra peptides is nice several other recent works e.g. https://arxiv.org/abs/2502.18462 demonstrate scaling to significantly larger systems. \n- Lack of error estimates on evaluation statistics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lslsqBfI8I", "reviewer": "Reviewer_dEB8", "summary": "This paper proposes to train normalizing flows by regression to alleviate the training instability in traditional MLE training to transform samples from a prior $q$ to a target distribution $p$, ie $x_0\\sim q$ and $x_1=T(x_0)\\sim p$. To enable regression-based training, it requires that there exists an invertible map between $x_0$ and $x_1$. Finding an exact solution is intractable, and therefore the author proposes two different approximations: 1. through coupling $(x_0, x_1)$ with optimal transport; 2. through coupling $(x_0, x_1)$ by training an additional CNF and letting $x_1$ be generated from $x_0$ through the trained CNF.\n\nThe author justifies the usage of their CNF-coupling by showing the error bound wrt wasserstein distance between the learned distribution and the target distribution. The experimental results also showcase the effectiveness of the proposed method, through various of peptide tasks.\n\nIn summary, the reviewer gives a weak acceptance.", "questions": "1. Can the author also show the training time comparison analogous to table 4 on other benchmarks?\n\n2. Table 3 is a bit confusing. Why are the inference times of the same NF but different training methods (MLE/RegFlow) different?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is simple and effective, which improves the training stability of NFs\n\n2. The author provides mathematical justification for using the CNF-coupling, by showing the wasserstein distance between trained model and target.\n\n3. Though the CNF-coupling sounds expensive at the first glance, the author shows that it requires much less computational overhead in table 4.\n\n4. The experimental results are good", "weaknesses": "1. The main concern lies in the invertibility of coupling, as both OT-coupling and CNF-coupling are approximations. It would be great if the author could elaborate on when this approximation would be broken and, in such a case, how the classic MLE objective would help. Intuitively speaking, if $p_0$ and $p_1$ are too separate, the true velocity might be less smooth and the (t-dependent) Lipschitz constants can be large, which means the Wasserstein bound can be very loose.\n\n2. The experiments in this paper focus on training NFs and then doing importance sampling to get equilibrium samples. However, in the plots, such as Figure 2, the author only shows the reweighted energy histogram but not the resampled Ramachandran plots. The reviewer thinks it is important as well to show the rama-plots with recalibrated samples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)