Verifier-Constrained Flow Expansion for Discovery Beyond the Data
Authors: Riccardo De Santi
Kimon Protopapas
Ya-Ping Hsieh
Andreas Krause
Venue: ICLR 2026 Poster (2026)
Keywords: flow models
diffusion models
exploration
verifiers
scientific discovery
Primary area: generative models

Abstract: Flow and diffusion models are typically pre-trained on limited available data (e.g., molecular samples), covering only a fraction of the valid design space (e.g., the full molecular space). As a consequence, they tend to generate samples from only a narrow portion of the feasible domain. This is a fundamental limitation for scientific discovery applications, where one typically aims to sample valid designs beyond the available data distribution. To this end, we address the challenge of leveraging access to a verifier (e.g., an atomic bonds checker), to adapt a pre-trained flow model so that its induced density expands beyond regions of high data availability, while preserving samples validity. We introduce formal notions of strong and weak verifiers and propose algorithmic frameworks for global and local flow expansion via probability-space optimization. Then, we present Flow Expander (FE), a scalable mirror descent scheme that provably tackles both problems by verifier-constrained entropy maximization over the flow process noised state space. Next, we provide a thorough theoretical analysis of the proposed method, and state convergence guarantees under both idealized and general assumptions. Ultimately, we empirically evaluate our method on both illustrative, yet visually interpretable settings, and on a molecular design task showcasing the ability of FE to expand a pre-trained flow model increasing conformer diversity while preserving validity.

Published as a conference paper at ICLR 2026
VERIFIER-CONSTRAINED FLOW EXPANSION FOR DISCOVERY BEYOND THE DATA
Riccardo De S, Kimon Protopapas' ; Ya-Ping Hsieh', Andreas Krause
EETH Zürich, 2ETH Al Center
{rdesanti, kproteoopppottptppp.....p krausea (ethzz ch, (yaping hsiih) Heinf www.e.ethe.... ch
ABSTRACT
Flow and diffusion models are typically pre-trained on limited available data (e.g., molecular samples), covering only a fraction of the valid design space (e.g.. the full molecular space). As consequence, they tend to generate samples from only a nar- row portion of the feasible domain. This is a fundamental limitation for scientific discovery applications, where one typically aims sample valid designs beyond the available data distribution. To this end, we address the challenge of leveraging access verifier (e.g., an atomic bonds checker), to adapt a pre-trained flow model so that its induced density expands beyond regions of high data availability, while preserving samples validity. We introduce formal notions of strong and weak verifiers and propose algorithmic frameworks for global and local flow expunsion via probability-spac optimization. Then, we present Flow Expander (FE), scalable mirror descent scheme that provably tackles both problems by verifier-constrained entropy maximization over the flow process noised state space. Next, provide thorough theoretical analysis of the proposed method, and state convergence guarantees under both idealized and general assumptions. Ultimately, we empirically evaluate method both illustrative, yet visually interpretable settings, and on a molecular design task showcasing the ability of FE to expand pre-trained flow model increasing conformer diversity while preserving validity.
INTRODUCTION
Recent years have seen major progress in large-scale generative modeling. In particular, flow (Lipman et al., 2022; 2024) and diffusion models (Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Ho et al., 2020) now produce high-fidelity samples and have been applied successfully across domains such chemistry (Hoogeboom et al., 2022), biology (Corso et al., 2022), and roboties (Chi et al., 2023). These models are typically trained via divergence minimization objectives, such as score (Song et al., 2020) or flow matching (Lipman et al., 2022), approximate the distribution induced by training data (c.g., molecular samples), which typically only cover tiny subset of the full valid design space. As a consequence, pre-trained generative models concentrate their density over a narrow region of valid designs, and are unlikely to generate valid samples beyond areas of high data availability. This is a fundamental limitation for scientific discovery tasks such as material design and drug discovery. where one typically wishes to generate valid designs beyond the data distribution. In particular, limited coverage of the valid design space leads to irreducible sub-optimality gap in generative optimization (De Santi et al., 2025b; Uchara et al., 2024h; Li et al., 2024) problems, where one aims to generate samples maximizing a task-specific utility function X R. (e.g., binding affinity for protein docking). as illustrated in Figure 1.
f
Limited coverage of the valid design space leads to gen- erating sub-optimal samples for downstream optimization tasks.
Figure 1:
Prior work has addressed this issue through nanifold-explorttion schemes that re-balance a pre- trained model's density over diverse, promising modes (e.g., De Santi al., 2025a;b; Celik et al., 2025). However, the validity signal learned by a pre-trained flow model diminishes outside high-data
"Equal contribution. Corresponding author: rdesant @@ethz.chh

---

Published as a conference paper at ICLR 2026
regions. Therefore, seemingly promising low- proobababillity modes that such methods would further explore, could turn out to be invalid. This highlights the need to inject further validity information into the exploration process via external verifier (Botta et al., 2025; Wang et al., 2024): formally, function u X' + (0. } that provides data-specific validity signal. Luckily, there exists more-or-less accurate verifiers for wide variety of real-world discovery applications, such atomic-hond check- ers for drug discovery (e.g.. O' O'Boyle et al., 2011), protein folding predictors for protein design (c.g.. Jumper et al., 2021), as well as physies-based simulators for mechanical and material design (c.g.. Kresse & Furthmüller, 1996). Motivated by these insights, in this work we advance flow and diffusion- based design space exploration methods (De Santi et al., 2025a;b) by asking the following question:
How can we leverage given verifier to adapt a flow or diffusion model to generate designs beyond high data-availability regions while preserving validity?
Answering this would contribute to the algorithmi-thheoreice foundations of generutive exploration, and enable applications of flow-basee waw-based exploration schemes in diverse scientific discovery tasks.
Our approach We address this challenge by formally introducing two verifier types. A strong verifier is function X' - {0, 1} that characterizes validity exactly (i.e., v(x) = 1 iff x: is valid). A weak verifier instead acts as a filter: it rejects certain invalid designs but misses others (formally v(zx 00 = x is invalid). While the former is arguably rare in scientific discovery applications, the latter is ubiquitous. For instance, most molecular checkers examine specific constraints (e.g.. atomic bonds, graph topology, or conformer geometry), ruling out certain invalid samples, but without guaranteeing validity. We show that strong verifiers allow to adapt pre-trained model to globally expand over the entire valid design space, While this is not the case for weak verifiers, they can also be leveraged for a more conservative, local expansion. To this end, we introduce mathematical frameworks for global and local flow expansion via veriffer-constrained entropy maximization (Sec. 3). Next, we propose Flow Expander (FE), a scalable mirror descent scheme acting over the flow process noised state space that provably tackles both problems by sequentially alternating expansion and projection steps (Sec. 4). We provide theoretical guarantees for FE, showing convergence results under both idcalized and general assumptions via mirror-flow theory (Sec. 5). Ultimately, we evaluate our method on both illustrative, yet visually interpretable settings, and a molecular design task, showcasing the ability of FE to expand a pre-trained flow model to increase molecular conformer diversity while better preserving validity than current flow-based exploration schemes (Sec. 6).
Our contributions In this work, we provide the following contributions:
A formalization of Global and Local Flow Expansion via verifier-constraiig entropy maximization, which formally capture the practically relevant problem of expanding the coverage of a pre-trained flow or diffusion model by integrating information from an available strong or weak verifier (Sec. 3).
Flow Expander (FE), a principled probability-space optimization scheme that provably solves both problems introduced via constrained entropy maximization over the flow noised state space (Sec. 4).
Noised Space Exploration (NSE), noised state space unconstrained exploration scheme, obtained as by product, that outperforms existing flow-based methods for high-dim. exploration (Sec. 4).
A theoretical analysis of the proposed algorithm providing convergence guarantees under both simplified and realistic assumptions via mirror-flow flow theory (Sec. 5).
An experimental evaluation of FE, showcasing its practical relevance on both visually interpretable illustrative settings, and molecular design task aiming to increase conformer diversity. (Sec. 6).
2 BACKGROUND AND NOTATION
Mathematical Notation. Using X' C Rd to refer to the design space (an arbitrary set), we denote the set of Borel probability measures on X' with P(X),, and the set of functionals over the set of probability measures P(X) as F(X). Given an integer N, we define [N] = NJ.
Flow-based Generative Modeling. Generative models aim to approximately replicate and sample from data distribution Pdata. Flow models tackle this problem by modeling flow, which incre- mentally transforms samples Xo = zo from a source distribution Po into samples X; = x from the target distribution Pdata (Lipman et al., 2024; Farebrother et al., 2025). Formally, flow is time- dependennd map vo: : [0. 1] x Rd - R such that 3) : (t, x) + (x). A generative flow model is then defined as continuous time Markov process [Xchosegı generated by applying flow th to Xo. ie. X, = (X,), t E [o. 1] such that X, = 1/ (Xo) ~ pdata- In the context of flow modeling, the flow de is defined by a velocity field u [0, 1] Rd - Rdd which is vector field implicitly defining
2

---

Published as a conference paper at ICLR 2026
P1
une
Pay
Tpree
Bay
noise
distriburion
W
Po
Pz ?
(b) Strong and weak verifier sets Figure 2: (2a) Pre-trained and globally expanded flow model inducing densities p and optimal
(a) Global flow expansion
density pj. (2b) Valid design space Ω strong and weak verifiers S2 i E [3], and their compositions.
& via the following ordinary differential equation (ODE), typically referred to as flow ODE:
d dt (x) = u, (x), vo(z) =
(1)
We write (e(00 refer the probability path of marginal densities of the flow model, i.e., X, (Xn) Pt, and denote by p" the probability path of marginal densities induced by the velocity field u. In practice, Flow Matching (FM) (Lipman et al., 2024) can be used to estimate a velocity field u² s.t. the probability path pd satisfies Pa = Po and pi Pdatas where po denotes the source distribution, and Pdota the target data distribution. Typically FM is rendered tractable by defining P`¹ as the marginal of conditional density pi (- x1), e.g..
X₂ Xo. X1 = K2 + WxX1
(2)
and we l). Then v² can be learned by regressing onto the conditional velocity field u(-|z1) (Lipman et al., 2022). Interestingly, diffusion models (Song & Ermon, 2019) (DMs) admit an equivalent ODE-based formulation with identical marginal densities to their original SDE dynamics (Lipman al., 2024, Chapter 10). Consequently, while in this work we adopt the notation of flow models, our contributions carry over directly to DMs.
where Ko = w1 = 1 and 11 = wi 0 (e.g., Ki =
Reinforcement Learning. We formulate continuousttim reinforcement learning (RL) as a specific class of finite-hrrizon optimal control problems (Wang et al., 2020; Jia & Zhou, 2022; Treven et al., 2023; Zhao et al., 2024). Given a state space X' and an action space A, we consider the transition dynamics governed by the following ODE:
Continuous-time
d
dt (x) at (1)(
where a A is selected action. We consider state space X = Rd x [0, 1]. and denote by (Markovian) deterministic policy a function गy (X2) = т (X,,t) E A mapping a state (x,t) E X' to an action A such that a, π (X,,t) t), and denote with p. the marginal density at time t induced by policy æ. Considering the continuous-time reinforcement learning formulation above, a velocity field ure can be interpreted an action process a, == w (X,, l), where af is determined by continuous-time RL policy via at" = ThE (Xe, t) (De Santi et al., 2025a). Therefore, we can express the flow ODE induced by a pre-trained flow model by replacing a, with arre in Eq. 3, and denote the pre-trained model by its policy prre, which induces density 1 = pT" approximating Pdata-
3 PROBLEM STATEMENT: GLOBAL AND LOCAL FLOW EXPANSION
Given a pre-trained flow model are inducing density of that covers sufficiently only limited region of the valid design space' Ω C Rd (e.g., molecular space, see Fig. 1), we aim to adapt it by leveraging verifier (e.g., an atomic bonds checker) to compute a model n," inducing a process:
(e(x) = (,(w with at = =xxxt))
dt
such that its density P == pi is more uniformly distributed over the valid design space than p pre. To this end, first denote by verifier a scalar function X' - {0, 1), and indicate by Ω2 = {I E X v(x)= = }} the verifier-set induced by v. Next, we classify any verifier v as strong or weak depending on the relationship between its verifier-set Q, and the valid design space Ω.
In this work, we consider the valid design space to be unknown, yet bounded set.
3

---

Published as a conference paper at ICLR 2026
Definition 1 (Strong Verifier). We denote by strong verifier a function :: X {0,1) s.t. Qt = S2
By Def. 1, (xx 1 > Ω hence a strong verifier fully characterizes the valid design space Z2
3.1 AN IDEALIZED PROBLEM: GLOBAL FLOW EXPANSION VIA STRONG VERIFIERS
Given a pre-trained flow model पhe and strong verifier X - {0. 1) as defined within Def. one can capture the problem of computing a new flow model x such that its marginal density pi = pi covers Ω uniformly via the following verifier-constrained entropy maximization problem.
Global Flow Expansion via Verifier-Constrained Entropy Maximization
π E arg max H(p)) subject to s.l. pÍ E P(L.)
(5)
In this formulation, the constraint p pl p enforces that the marginal density t matches the pre-trained model marginal, and H E F (ft) denotes the differential entropy functional expressed as:
- f du log
dp dx J P(2,)
H(u) =
(6)
where St is the bounded verifier-set induced by Crucially, Problem computes flow model x* inducing the density pi with maximum entropy among all densities supported 2,. Therefore, the optimal density pT according to Problem 5 conresponds to the uniform density over the entire valid design space Ω2, i.e., pi (S2)
uniforms are the entropy-mane maximizing densities sets and f2 = Ω due to v being a strong verifier. Notably, Problem does not depend on the prior generative model pPre_ In fact, since the strong verifier fully characterizes the valid design space Q. prior information not required to compute the maximally explorative, yet valid flow model n²
bounded
Problem 5 provides sharp data-free objective for verifier-based flow/diffusion model learning, well capturing the ideal goal of a uniform prior over the valid design space for subsequent use in down- stream tasks. Nonetheless, discussed in Sec. strong verifiers are arguably rare in most scientific discovery applications (e.g., material design, drug discovery). Towards overcoming such limitation, in the following we sharpen our notion of verifier to one that ubiquitous in real-world discovery tasks.
3.2 REALISTIC FRAMEWORK: LOCAL FLOW EXPANSION VIA WEAK VERIFIERS
We first relax the notion of strong verifier introduced in Def. the following one of weak verifier.
Definition 2 (Weak Verifier). We denote by weak verifier a function X {0. 1) Q2 Ω.
As Fig. 2b illustrates, Def. requires only the one-sided condition 2(z) strong verifiers, it does not guarantee e(z) = = E Ω (i.e., V cannot certify membership in Q). Instead, represents a superset Q2 5 Ω and effectively acts as filter. Moreover, multiple weak verifiers (1s) can be combined, yielding Ω, = n, Prg, which is typically tighter to Ω for more diverse verifiers U/ c.g., checking atomic bonds, molecular graph topology, and conformer geometry.
= x & Ωt unlike
Given this new realistic notion of verifier, the global flow expansion Problem 5 would evidently no longer compute the desired flow model. In fact, for weak verifier it holds Q. Ω, therefore the optimal flow density p" = (S2,)) would generate invalid designs over 2p \ Ω2 as shown in Fig. 2b. Moreover, weak verifiers typically induce unbounded verifier sets, which would even render Problem 5 ill-posed. To address these issues, introduce the local flow expansion problem, which aims to locally expand the prior flow model xP"e by integrating information from both v and Pre.
Local Flow Expansion via KL-regularized Cerifie-CConstrainer Entropy Maximization
E arg max H(pf) - a. (DKi(pille"" subject to are T:ppppo
s.l. pi E P(P.))
(7)
Here, the weak verifier v acts as a filter preventing the entropy term from driving exploration into verifier-rejected regions. Since cannot detect all invalid areas, expansion must remain conservative and leverage the validity signal encoded in the prior model. This is achieved via the a-weighted KL divergence between the density p induced by the fine-tuned model, and pere Crucially, this term enforces T² to preserve prior validity signal, thus preventing x from allocating density in regions unlikely according pre even if valid according to the weak verifier. For sufficiently large o, the
4

---

Published as a conference paper at ICLR 2026
density induced by the expanded flow model x² stays arbitrarily close to the prior in probability space- hence local expansion. In practice, the choice of a should reflect the degree of risk-aversion versus sevelng toward the discovery task hand, well the quality of the weak verifier v (i.e., how tightly Ω2 approximates Qt. Interestingly, in the limit of Q, - Ω a should clearly be set to 0, which naturally retrieves the presented global flow expansion Problem
sub-case.
FLOW-EXPANDER SCALABLE GI.OBAL. AND LOCAL EXPANSION
In the following, we propose Flow Expander (FE), which provably solves the global and local flow expansion problems (see Eq. and 7). To this end, we first lift their formulations from the probability space associated to the last time-step marginal pi to the entire flow process Q = {pF (o
Flow Expansion via Verifier-Constrannee Noised Space Entropy Maximization
π E arg max L (Q")) = S.' (xgr(pT) dt subject to T:ph =po
[o(x]] = xm
E
Under this unifying formulation, 9, P(X) + R is a functional over densities p. induced by flow æ. We note that under general regularity assumptions, optimal policy x² for Problem 8 is optimal also for the global and local flow expansion problems (see Eq. and 7) if the functional G is defined as:
(((PT) HP??)
Ge(pT) H(p)) 04 DRI(P? Pe
Global Expansion
Local Flow Expansion (o:: = o)
Before introducing FE, we first recall the standard notion of first variation of G over a space of proba- bility measures (ef. Hsieh al., 2019). A functional G F(X), where P(X) - IR, has first vari- ation at E P(X') if there exists a function 6G(x) E F(X) such that for all μ E P(x) it holds that:
gp + ep') 9(x) + e/yl (g)()))()))) + o(c)-
where the inner product is an expectation. Intuitively, 8g(x) can be interpreted as an infinite- dimensional gradient over probahility measures. Given this concept of first variation, FE solves Prob- lem 8 by computing a process Q* at each iteration k E [K]. via the following mirror descent step:
(MD Step) Constrained and Regularized Process Surprise Maximization
1 Q E arg max (6C(Q*=1). Q) दe DeL (QIIQ Q:po=p5 k-1
s.t. E [v(x)] = (10)
ppppl
While the MD step in Eq. 10 might seem abstract, the following Lemma 4.1 hints at more practical formulation of the above through the lens of stochastic optimal control (Fleming & Rishel, 2012).
Lemma 4.1 (First Variation Flow Process Functionals). For objectives defined the form of Eq. 8, we have:
(6C(Q)). Q) = =f(² [89](p()] dt.
(11)
q
Lemma 4.1 factorizes (BC(Q* Q) into an integral over the flow process of terms fz) == X,59, (pt (z). Crucially, this demopccoopooition allows to rewrite the MD step (Eq. 10) as the fol- lowing standard constrained control-affine optimal control problem2 (Domingo-Enrich et al., 2024):
Constrained and Regularized Process Surprise Maximization via Fine- Eine-Tuning
min E
2 |T (11 100--J-(4 fx,,,, di s.1. E [b(x)] = 1, with fe () 7x89,(0t))1 mmpp
Concretely, compute flow * inducing Q² (Eq. 10) via EXPANDTHENPROIEET (see Alg. 4), which decouples constrained optimization into sequential expansion and projection steps:
We leave standard dynamical system constraints (e.g., Equation 13 Domingo-Fnrich al., 2024) as implicit.
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 EXPANDTHEN
1: Input: who²; current flow model, V., 89: gradients of functional grad., 7ki inverse update step-size, (e integral weighting coefficients, a: verifier, YJe: fine-tuning strength
2: Expansion step:
FINETUNINGSOLVVE(("""" Vv, Va,dGe, des
(12)
3: Projection step:
FINETUNINGSOLVEERFF logo, m.)
(13)
Output: Fine-tuned policy
Algorithm 2 Flow Expander (FF)
1: Input: x² pre-trained flow model, {a+ reto.]] KI-regularivation coefficients, (==) inverse
update step-sizes, (Ae inttegal weighting coefficients, v: verifier, {m) [n=))=>> projection strengths
2: Init: To == pre
3: for k = 1,2,.... ..... K do
4: Set:
81 gh-
dgelee (1)
Global FE (G-FE)
(14)
ai (8, -st
Local FE (L-FE)
5: Fine-tune Th-1 into Th via Algorithm 4:
- EXPANDTHENPPOJECTT*** Vo, na (e
6: end for
7: Output: policy क TK
Expansion step. The unconstrained expansion step is performed over the noised state space, which can be tackled by extending established control (or RL) based methods for fine-tuning with the running cost (e( = 71.9, (Ir), effectively computing a process Q* such that:
Q k arg max Q) Q:po=55"
Dk/ Q*1
(15)
7
Projection step. Given Q the projection step adapts the flow n to enforce the constraint Eq. 10 via reward-guided fine-tuning (e.g., Uehara et al., 2024a, Sec. 8.2):
le E arg max E [log u(x)] DKL QQI(²*)
(16)
Q:po=rl" 2opp1
The EXPANDTHEN PROJECT scheme in Alg. 4 solves optimization problem 10, i.e., returns a flow model N inducing a process Q* that is solution to 10. Formally, the following holds:
Proposition
Qa E arg max (BC(Q* ", ))QQ) Q) DKL (Q11Q=1) nx
1
s.t. E a0
[v[x)]=1
(17)
Q:po=p5
Notice that this step could alternatively be performed via constrained reward-guided fine-tuning (e.g.. Gutjahr al., 2025). Finally, present Flow Expander (FE) in Alg. 2, which effectively approx- imates the mirror descent scheme presented above by iteratively applying EXPANDTHENPROJиCT
Complete algorithm execution. FE takes inputs: pre-trained flow model Tre KL- regularization coefficients {re the number of iterations K, inverse step-sizes (16
K (Tark=1 1
integral weighting coefficients (>x Jeeoo,1) verifier and a projection strength schedule {mh 1R R=
At each E [K] FE computes the gradient (w.r.t x)) of the first variation at the previous policy mk-1+ (line 4). Then, FE computes the flow model ग& via the EXPANDTHENPROJEET
ie.. Vg (pT scheme (see Alg. 4), which takes in input the current flow model n k-1 the computed gradients, and
the verifier v, and returns the updated flow model æ Ultimately, FE returns the final policy πk.
Closed-form gradient expressions. FE operates using trajectory reward gradients V., 5 (p.))) In fact, while such rewards difficult to estimate, their gradients admit close-form expres- sions (De Santi et al., 2025a) that can be approximated via available quantities:
V.HH(p))= =
xe 8H(pT) a, VzòDKu(p pre -5 at (sT pre
(18)
Global Flow Expansion
Local Flow Expansion (a:

---

Published as a conference paper at ICLR 2026
Algorithm 3 Noised Space Exploration (NSE)
1: Input: x=T pre-trained flow model, (e KL-regularization coefficients, {7x update step-sizes. (Ae (Ejoa) iittegrll weighting coefficients
inverse
2: Init: To == xure
3: for k = 1, K do
4: Set:
V Zz
(6] s
(19)
Expansion step, fine-tune Wx-1 into Tr via:
π FINETUNINGSOLVER(TT"
V.. 6be, 7k)
(20)
end for 7: Output: policy ग ==
and can simply be plugged into any first-order fine-tuning solver yielding a scalable method. Eq. 18 depends on the score sf (x) V log (z), obtained from the score network or flow field (Domingo- Enrich et al., 2024). Unlike prior flow-based exploration methods using only the divergent terminal score L (De Santi al., 2025a;b), FE uses scores across time and allows to downweight & - via A
Beyond verifier-constrained settings, noised-space exploration also yields a stronger unconstrained ex- ploration scheme. We denote by Noised Space Exploration (NSE) the variant of FE obtained by remov- ing the projection step; pseudocode is given in Alg. 3. As shown Sec. 6, NSE stabilizes diffusion- and flow-based exploration in higher dimensions, substantially outperforming existing methods.
GUARANTEES FOR FLOW-EXPANDER
We aim show that FE admits provable guarantees ensuring reliable behavior in practice. To this end, we leverage the flexible framework of constrained mirror descent, a classical optimization method that has recently found successful applications in sampling and generative modeling (Karimi et al., 2024; De Santi et al., 2025a;b). We analyze two regimes. First, an idcalized setting, where each step of Eq. 10 can be computed exactly leading to sharp step-size prescriptions and fast, polynomial convergence rates. Then, realistic setting, where each MD step can only be solved approximately for which we show asymptotic convergence the optimal solution under mild noise and bias assumptions.
Idealized setting. We state that the exact updates case admits finite-time convergence guarantee:
Theorem 5.1 (Convergence guarantee in the idealized process-level setting). Consider the ob- jective L defined Equation (8), and let X fo dydt. Let (Q*) be the iterates generated by Equation (10) with Tik 1/x* for all l E [K]. Then
X* K
L(Q") - L(Q <
DKL(Q" QPM0),
(21)
where Q" arg max (()).
General setting. Recall that Q* k the k-th iterate of FE. In realistic scenarios, however, Eq. 10 can only be solved approximately, so we interpret the update as approximating the idcalized iteration:
Of E arg max ( Q) DKL D (*11
s.t.
E [m(z)]
(22)
Q:po=p
z=-p1
To measure deviations from these idealized iterates, let Ta be the filtration up to step k and decompose the oracle into its bias and noise components:
by E. ( sC(Q)) TT]] == = 0C( SC(Q.) by.
(23)
Here, by captures systematic error while U₁ is conditionally mean-zero. Under mild assumptions noise and bias (see Assumptions E. to E.2), we obtain the following guarantee.
Theorem 5.2 (Convergence guarantee in the general process-level setting (informal)). Suppose
the oracle satisfies finite- sitteevariance noise and vanishing bias, and let the step-sizes {7k} follow the Robbins-Monra rule (Ex Tkk satisfy
Σx th 30). Then the iterates {Q*) generated by FE
Q* - Q' a.s.,
(24)
where Q" E arg maxq L(Q).
7

---

Published as a conference paper at ICLR 2026
00
05
10
Enirg
Pre-trained samples
(h) G-FE S-MEME
(c) G-FE VS CONSTR
(d) Entropy Validity
GEL
30
Itesttoon
heeaton
(c) G-FE Expansion K=1 (f) G-FE Projection K=1 (g) Entropy evaluation
(h) Validity evaluation
Figure 3: (top) Global FE (G-FE) expands the pre-trained flow model (3a) into x² (violet, 3b), increasing coverage (i.e., entropy), while preserving validity (i.e., red ellipse interior). Compared with
the unconstrained exploration S S-MEME method, and constrained generation (CONSTR), Global FE (G-FE) shows cest-of-bothh woolds behaviour: achieving near-optimal entropy and validity (Fig. 3d).
EXPERIMENTAL EVALUATION
We analyze the ability of Global FE (G-FE) and Local FE (L.-FE) to expand flow model densities while preserving validity of generated samples, and compare their performance against recent flow-based exploration methods, namely FDC (De Santi al., 2025b), and S-MEME (De Santi et al., 2025a), as well as a standard constrained generation scheme, denoted by CONSTR (Sec. 8.2 Uchara et al., 2024a). We present experiments on two visually interpretable settings, followed by molecular design task aiming to increase conformer diversity (more details are provided in Apx. G).
Global Flow Expansion via Strong Verifier. We run G-FE on a pre- model pPP to globally expand its density pf over the valid design space (red ellipse in Figs. 3a-3c, 3e-3f). As shown in Fig. 3b and 3c, G-FE (violet) run with y= and K = 10, expands into previously uncovered areas (lower right), staying within the valid region. In comparison, S- M-MEME (black, Fig. 3b) predictably fails to restrict density to the valid region (light red area). Symmetrically, CONSTR (see Fig. 3c, orange) confines density to the valid space but fails to expand it. Fig. 3d shows that G-FE explores nearly as much as S-MEME (i.e., 1.97 vs. 2.17 entropy), while retaining significantly higher validity: 0.99 against 0.73 of S-MEME. Remarkably, G-FE preserves the same degree of validity of CONSTR while exploring significantly more (1.97 versus 0.72 entropy). Figs. 3e-3f show the first expand-then- project steps of G-FE and Figs. 3g-3h show entropy and validity estimates with 95% CI over 5 seeds for G-FE and all baselines. In summary, G-FE achieves both near-optimal exploration and validity.
Local Flow Expansion via Weak Verifier. We consider pre-trained flow model The whose density pt is concentrated in a central high-density region, with lowproobayility promising modes on either side (see Fig. 5a). Crucially, while the two right-most modes are valid, the left one is not. We fine-tune Tre via L-FE for K = 8 iterations and modes - i.e., perform mode discovery (De Santi et al., 2025b; Morshed Boddeti, 2025). As shown in Fig. 5b FDC, a KL-regularized entropy maximization scheme, predictably increases diversity over plausible modes by redistributing density to the invalid left one. L-FE, however, leverages weak verifier (gray circled area in Fig. 5c) prevent Figure 4: Entropy-Validijg allocating more density to that invalid region, and even removes density from that region. 5a, top). Effectively, L-FE uses the weak verifier perform form of mode selection, i.e., filtering out invalid modes during the expansion process. As shown in Fig. 4, Local FE (L-FE) achieves high entropy (i.e., 1.67 versus 1.17 and 1.58 of L-FE), while preserving high validity, namely 0.89 compared to 0.74 of FDC, almost fully preserving the prior model's 's validity of 0.9.
0.99 to expand its induced density over diverse
arr
0.10
L-FE increases molecular conformer diversity for de-novo design on QM9, In this experiment, we aim to increase diversity of molecular conformers in a molecular design task. We run FE on Fol CTMC (Dunn & Koes, 2024) pre-trained on QM9 dataset (Ramakrishnan et al., 2014). Our weak verifier a filter excluding molecules for which any two atoms are closer than 0.975 Angstroms (A), and validity is evaluated via RDKit (RDKit) sanitization paired with the aforementioned check.
8

---

Published as a conference paper at ICLR 2026
9:0
306
FRC
HRC
(a) Pre-trained samples
(b) FDC samples
(c) L-FE samples
PRE FDC
LIF
PRE FOC
Cantomon 1002
6.7
PDC
0.0
(d) QM9 visual diversity
(c) C. Vendi
Validity
(f) Conformer Vendi
Validity
Figure 5: (top) L-FE (yellow, 5c) expands the prainen flow model The (green, 5a) over promising
yet verifier-filtered modes, while FDC (blue, 5b) expands x over all plausible modes leading to increased density in invalid regions (left mode in Fig. 5b). (bottom) FE increases visual (5d), and quantitative diversity (5f), while preserving higher validity than FDC (5e-5g)
We evaluate diversity of molecular conformers by conformer VENDI (Friedman Dieng, 2022) metric (see Apx. G.2) capturing diversity over sampled conformers via their fingerprints. L-FE, run for K 5 iterations and a 9, quantitatively increases diversity compared to the pretrained model (Fig 5e, VENDI of 100 vs 89). This is visually shown in Fig 5d, histogram plot of 1-dim PCA projection of molecular fingerprints (see Apx. G for further details). In particular, L-FE (violet) expands the pre-trained flow model to explore promising and verifier-cee verifier-certified modes of the pre-trained model density (see Fig. 5d). Crucially, L-FE achieves a similar degree conformer diversity (100 103) to FDC, an unconstrained exploration scheme, while preserving significantly higher sample validity, i.e., 81% vs 69%, as shown in Figs. 5e, 5f, and 5g.
L-FE increases molecular conformer diversity for de-novo design on GEOM-Drugs. In this exper- iment, we aim to increase the diversity of generated molecular conformers in a molecular design task with drug-like molecules. We run FE FlowMol CTMC (Dunn & Koes, 2024) pre-trained GEOM- Drugs (Axelrod & Gomez-Bommarell 2022). As in the previous experimental setting, the weak veri- fier employed a filter excluding molecules for which any two atoms are closer than 0.975 Ångstroms (Å), and validity is evaluated via the RDKit (RDKit) sanitization operation paired with the aforemen- tioned check. We evaluate diversity of molecular conformers by conformer VENDI (Friedman & Dieng, 2022) metric (see Apx. G.2) capturing diversity over sampled conformers via their fingerprints. L-FE, for K iterations, (82% vs 72%) than the pre-trained model, as shown in Fig. 6a. Similarly, L-FE induces higher diver- sity (529 508) than FDC, a recent diffusion-based unconstrained exploration method (De Santi et al., 2025b), while preserving significantly higher sample validity, i.c., 82% vs 66%, as shown in Fig. 6a.
1/9, and 1 5, achieves higher diversity (529 476) and validity
Moreover, within Apx. G.5.4, we report an ablation study for the proposed method parameters. Note that L-FE performs consistently better than NSE. Since NSE corresponds to L-FE with 0, this result illustrates the working mechanism and importance of the projection step (i.e., 1) > 0). Interestingly, G-FE, which is equal to L-FF with a, = O, shows performance on par with I.FFE for very conservative parameters ( 0.0002, K 3), while gradually degrading for less conservative parametrizations. This behaviour is likely due to the implicit KL-regularization between iterates within the mirror descent update step (see Eq. 10), which implies prior regularization for small K.
NSE achieves higher exploration performance against current methods. We evaluate NSE, the unconstrained exploration variant of L-FE obtained by removing the projection step (see Alg. 3 in Sec. 4 for further details), to perform flow-based design space (unconstrained) exploration. We consider FlowMol CTMC (Dunn & Koes, 2024) pre- pre-trained on GEOM -DDugg dataset (Axelrod & Gomez- Bombarelli, 2022), and report in Fig. 6b the results for NSE with K and 0. We observe that

---

Published as a conference paper at ICLR 2026
Runders
4,00
PM2
3376
2000
FDC
-00
XX3
XIo
200
Contoree Wendi
Contoomee Venci
(a) I.-FE on GEOM-Drigs
(h) NSE FDC
(c) Computational Cost (d) GEOM-drigs molecule
Figure 6: (6a) L-FE (violet) achieves higher diversity and validity than FDC on GEOM-Drugs. (6b) NSE, the unconstrained variant of L.-FE, exhibits superior performance compared with FDC, a state-of-the-art inffusiio-based unconstrained exploration method. (6c) Computational cost comparison of L-FE, FDC, and NSE. (6d) Representative drug-like molecule generated via L-FE.
NSE achieves consistently higher diversity (i.e., 519 vs 508) and validity (i.e., 74% vs 66%) against FDC (De Santi et al., 2025b), state-of-the-an method for flow-based unconstrained exploration.
L-FE and NSE have computational costs comparable to current exploration schemes. We report in Fig. 6c a comparison of computational cost, measured via the method runtime (seconds [s]) of L-FB and NSE compared against FDC (De Santi et al., 2025b). One can notice that although the schemes proposed within this work (i.e., L-FE and NSE) perform exploration over the entire flow process noised state space, they do not incur in significantly higher computational cost compared with FDC.
7 RELATED WORK
Diffusion and flow based design space exploration Recent works introduced methods for flow based design space exploration via maximization of entropy functionals (De Santi et al., 2025a;b) or approximations (Celik et al., 2025). While these methods explore by leveraging information from a prior model, FE directs exploration either (i) exclusively via a verifier (i.e., global expansion, see 5), or (ii) combining verifier information with prior validity cues (i.e., local expansion, see 7). Moreover, while current schemes explore only the last time-step state space, we lift the exploration task to the en- tire flow process, providing principled solution to the score divergence problem mentioned in Sec. 4.
Maximum state entropy exploration. Maximum state entropy exploration, introduced by Hazan et al. (2019), tackles the pure-exploration problem of maximizing the entropy of the state distribution induced by a policy over a dynamical system's 's state space (c.g., Lee et al., 2019; Mutti et al., 2021; Guo et al., 2021; De Santi et al., 2024a). The flow expansion problems (Eq. and 7) are closely related, with př representing the state distribution induced by policy π over a subset of the flow process state space (i.e., for time-step L 1). Recent studies have tackled maximum entropy exploration with finite sample budgets (e.g., De Santi et al., 2024b; Prajapat et al., 2023; Mutti et al., 2023; 2022b;a). which could be relevant for future work, e.g., design space exploration under limited samples constraint.
Sample diversity in diffusion models generation. A well-known limitation of flow-based generation limited sample diversity. This problem has been recently addressed by numerous studies (e.g., Corso et al., 2023; Um et al., 2023; Kirchhof et al., 2024; Sadat et al., 2024; Um & Ye, 2024; Klarner et al., 2024). Crucially, such methods are complementary to ours. In fact, they can be applied to promote diverse sampling from the expanded model produced by FE. In particular, whereas these works aim to maximize diversity of a fixed diffusion model or flow model, we aim to sequentially fine-tune pre-trained flow model so that its induced density is permanently expanded over the valid design space. Moreover, our formulations (Eq. 5, 7) and FE scheme increase diversity while integrating validity signal from chosen verifier.
CONCLUSION
This work tackles the fundamental challenge of leveraging verifier (e.g.. an atomic bonds checker), to expand a pre-trained model's 's density beyond regions of high data availability, while preserving validity of the generated samples. To this end, we introduce notions of strong and weak verifiers and cast global and local flow expansion as probability spaace optimization problems. We present Flow Expander (FE), a scalable mirror-descent scheme that provably solves both problems via verifier-constrained entropy maximization over the flow process noised state space. We provide thorough analysis showing convergence guarantees for FE under idealized and general assumptions by employing recent mirror-flow flow theory. Ultimately, we empirically evaluate a molecular design task showcasing the ability of FE to increase molecular conformer diversity while preserving better levels of validity than current flow and diffusion- bassd exploration methods.
method on both illustrative settings, and
10

---

Published as a conference paper at ICLR 2026
9
ACKNOWLEDGEMENTS
This publication was supported by the ETH AI Center doctoral fellowship to Riccardo De Santi. The project has received funding from the Swiss National Science Foundation under NCCR Catalysis grant number 180544 and NCCR Automation grant agreement 5INF40 180545.
10 REPRODUCIBILITY STATEMENT
We acknowledge that our work is documented sufficiently to ensure reproducibility of our results. We provide implementation details of all algorithms and procedures, such as: complete pseudocode in Appendix F, as well as hyperparameter choices and hardware requirements in Appendix Section G. We also give detailed account of our experimental setup in Section 6, including explanations of metrics and procedures used to evaluate our algorithms in Appendix G. Finally, our implemented version of FE, leverages the well-established Adjoint Matching (Domingo-Ena) Enrich al., 2024) method as an oracle.
REFERENCES
Simon Axelrod and Rafael Gomez-Bombarelli, Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data, 9(1):185, 2022.
Heinz H Bauschke, Jérôme Bolte, and Marc Teboulle. A descent lemma beyond lipschitz gradient continuity: first-order methods revisited and applications. Mathematics of Operations Research, 42(2):330-348, 2017.
Michel Benaim. Dynamics of stochastic approximation algorithms. In Seminaire de probabilites XXXIII, pp. 1-68. Springer, 2006.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint 2rXi::2305.13301, 2023.
Edoardo Botta, Yuchen Li, Aashay Mehta, Jordan Ash, Cyril Zhang, and Andrej Risteski. On query complexity of verifier-assisted language generation. arXiv preprint arXiv:2502.12123, 2025.
Onur Celik, Zechu Li, Denis Blessing, Ge Li, Daniel Palenicek, Jan Peters, Georgia Chalvatzaki, and Gerhard Neumann. Dime: Diffusion-basee maximum entropy reinforcement learning. arXiv preprint arXiv:2502.02316. 2025.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shu- Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023.
Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusson steps, twists, and turns for molecular docking, arXiv preprint arXiv:2210.01776, 2022.
Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, and Tommi Jaakkola. Particle guidance: non-iid diverse sampling with diffusion models. arXiv preprint 1.Xiv:2310.131000 2023.
Riccardo De Santi, Federico Arangath Joseph, Noah Liniger, Mirco Mutti, and Andreas Krause. Geometric active exploration in markov decision processes: the benefit of abstraction. arXiv preprint arXiv:2407.13366 2024a.
Riccardo De Santi, Manish Prajapat, and Andreas Krause. Global reinforcement learning: Beyond lin- ear and convex rewards via submodular semi-graan emi-gradient methods. arXiv preprint 2rXiv:2407.09990 2024b.
Riccardo De Santi, Marin Vlastelica, Ya-Ping Hsieh, Zebang Shen, Niao He, and Andreas Krause. Provable maximum entropy manifold exploration via diffusion models. In International Conference Machine Learning, 2025a.
11

---

Published as a conference paper at ICLR 2026
Riccardo De Santi, Marin Vlastelica, Ya-Ping Hsieh, Zebang Shen, Niao He, and Andreas Krause. Flow density control: Generative optimization beyond entropy- regulairegu fine-tuning. In Advances in Neural Information Processing Systems (NeurlPS), December 2025b.
Carles Samingg--Enrich Michal Drozdzal, Brian Karrer, and Ricky TQ Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. arXiv preprint arXiv:2409.0880) 008.0, 2024.
Ian Dunn and David Ryan Koes. Mixed continuous and categorical flow matching for 3d de novo molecule generation. ArXiv, pp. arXiv-2404, 2024.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 16679858-79885, 2023.
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni, Rémi Munos, Alessandro Lazaric, and Ahmed Touati. Temporal difference flows. arXiv preprini arXiv:2503.09817 2025.
Wendell H Fleming and Raymond W Rishel. Deterministic and stochastic optimal control, volume 1. Springer Science & Business Media, 2012.
Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.02411 2022.
Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Alaa Saade, Shantanu Thakoor, Bilal Piot, Bernardo Avila Pires, Michal Valko, Thomas Mesnard, Tor Lattimore, and Rémi Munos. Geometric entropic exploration. arXiv preprint arXiv:2101.02055, 2021.
Sven Gutjahr, Riccardo De Santi, Luca Schaufelberger, Kjell Jorner, and Andreas Krause. Constrained molecular generation via sequential flow model fine-tuning. In ICML 2025 Generative Al and Biology (GenBio) Workshop. 2025.
Elad Hazan, Sham Kakade, Karan Singh, and Abby Van Soest. Provably efficient maximum entropy exploration. In International Conference Machine Learning, 2019.
Jean-Baptiste Hiriart-Urruty and Claude Lemaréchal. Fundamentals of convex analysis. Springer Science & Business Media, 2004.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp. 8867-8887. PMLR, 2022.
Ya-Ping Hsieh, Chen Liu, and Volkan Cevher. Finding mixed nash equilibria of generative adversarial networks. In International Conference on Machine Learning, pp. 28 18-988 PMLR, 2019.
Yanwei Jia and Xun Yu Zhou. Policy evaluation and lemporal-difference learning in continuous time and space: A martingale approach. Journal of Machine Learning Research, 23 23(154):1-35 2022.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-588)
2021.
Mohammad Reza Karimi, Ya-Ping Hsieh, and Andreas Krause. Sinkhorn flow as mirror flow: A continuous-time framework for generalizing the sinkhorn algorithm. In International Conference on Artificial Intelligence and Statistics, pp. 4 4186-4194. PMLR, 2024.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), ICLR (Poster), 2015. URL http: dbbp.. uni -trittrerr de / db/
conf/iclr/icl r201 5. html #K i ngmaB14
12

---

Published as a conference paper at ICLR 2026
Michael Kirchhof, James Thortoo,, Pierre Ablin, Louis Béthune, Eugene Ndiaye, and Marco Cuturi. Sparse repellency for shielded generation in text-to-image diffusion models. arXiv preprint 002.022400.0002, 2024.
Leo Klarner, Tim G. J. Rudner, Garrett M. Morris, Charlotte M. Deane, and Yee Whye Teh. Context- guided diffusion for out-of-distribution molecular and protein design. In Proceedings of the 41 International Conference on Machine Learning, ICML'2 '2. JMLR.org. 2024.
G. Kresse and J. Furthmüller. Efficient iterative schemes for ab initio total-energy calculations using plane-way wave basis set. Physical Review B, (1116):11119-11860 1996. doi: 10. 1..103//hysRev.. 11.11
Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdinov. Efficient exploration via state marginal matching. arXiv preprint arXiv:190.005277 2019.
Zihao Li, Hui Yuan, Kaixuan Huang, Chengzhuo Ni, Yinyu Ye, Minshuo Chen, and Mengdi Wang. Diffusion model for data-driven black-box optimization. arXiv preprint arXiv:2403. 13219, 2024.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint 2.X0:2210000277,, 2022.
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:2412.06266 2206664, 2024.
Haihao Lu, Robert M Freund, and Yurii Nesterov. Relatively smooth convex optimization by first-order methods, and applications. SIAM Journal Optimization, 28(1):333-333) 354, 2018.
Panayotis Mertikopoulos, Ya-Ping Hsieh, and Volkan Cevher. A unified stochastic approximation framework for learning in games. Mathematical Programming, 203(1))559-609, 2024.
Alexander Mielke and Jia-Jie Zhu. Hellinger-kantoroviich gradient flows: Global exponential decay of entropy functionals. arXiv preprint arXiv:2501. 2025.
Mashrur Morshed and Vishnu Boddeti. Diverseflow: Sample-effcieen diverse mode coverage in flows. In Proceedings of the Computer Vision and Putlern Recognition Conference, pp. 23303- 23312, 2025.
Mirco Mutti, Lorenzo Pratissoli, and Marcello Restelli. Task-agnostic exploration via policy gradient of a non-parametric state entropy estimate. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9028-9036, 2021.
Mirco Mutti, Riccardo De Santi, Piersilvio De Bartolomeis, and Marcello Restelli. Challenging com- mon assumptions in convex reinforcement learning. Advances in Neural Information Processing Systems, 0114449-45022 2022a.
Mirco Mutti, Riccardo De Santi, and Marcello Restelli. The importance of non-markovianity in maximum state entropy exploration. In International Conference on Machine Learning, pp. 16223-16239, PMLR, 2022b.
Mirco Mutti, Riccardo De Santi, Piersilvio De Bartolomeis, and Marcello Restelli. Convex reinforce- ment learning in finite trials. Journal of Machine Learning Research, 24(250):1-44 2023.
Noel M O' O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geoffrey R Hutchison. Open Babel: An open chemical toolbox. Journal of Cheminformatics, 3(1):33, 2011. doi: 10.1 11188/1758-2946---33
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2888 2011.
Manish Prajapat, Mojmír Mutný, Melanie N Zeilinger, and Andreas Krause, Submodular reinforce- ment learning. arXiv preprint arXiv:2307. 13372. 2023.
13

---

Published as a conference paper at ICLR 2026
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
RDKit. Rdkit: Open-source cheminformatics. https://wwwt wwwtt//tt/wwww.t../.mo.......... MM-DD.
rdkit org, Accessed: YYYY-
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pp. 400-407, 1951.
Scyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Oumar Hilliges, and Romann M. Weber. Cads: Unleashing the diversity of diffusion models through condition-annedt sampling, 2024. URL
https://arxe org/ abss 2310.17347.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13356 2020.
Lenart Treven, Jonas Hübotter, Bhavya Sukhija, Florian Dorfler, and Andreas Krause. Efficient ex- ploration in continuous-time model-based reinforcement learning. Advances in Neural Information Processing Systems, 15:42119-4221 2023.
Masatoshi Uehara, Yulai Zhao, Tommaso Biancalani, and Sergey Levine. Understanding rein- forcement learning-basse based fine-tuning of diffusion models: A tutorial and review, 2024a. URL ab//////////0 13734.
https: / /ttp//aaavvv///////////e//...av
Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Sergey Levine, and Tommaso Biancalani. Feedback efficient online fine-tuning of diffusion models. arXiv preprint 1rXi::2442..1339 2024b.
Soobin Um and Jong Chul Ye. Self-guided generation of minority samples using diffusion models. In European Conference on Computer Vision, pp.
Springer, 2024.
Soobin Um, Suhyeon Lee, and Jong Chul Ye. Don't play favorites: Minority guidance for diffusion models. arXiv preprint arXhv:2301.12334, 2023.
Haoran Wang, Thaleia Zariphopoulou, and Xun Yu Zhou. Reinforcement learning in continuous time and space: A stochastic control approach. Journal of Machine Learning Research, 21 21(198):1-344 2020.
Zihan Wang, Yunxuan Li. Yuexin Wu, Liangchen Luo, Le Hou, Hongkun Yu, and Jingbo Shang. Multi-step problem solving through a verifier: An empirical analysis on model-induced process supervision. arXiv preprint arXiv:2402.00 02658, 2024.
David Weininger. Smiles, a chemical language and information system. 1. introduction to methodol- ogy and encoding rules. J. Chem. Inf. Comput. Sei., 28(1):31-366 1988.
Hanyang Zhao, Haoxian Chen, Ji Zhang, David D Yao, and Wenpin Tang. Scores actions: framework of fine-tuning diffusion models by continous-time reinforcement learning. arXiv preprint arXiv:2409.088: 2024.
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
CONTENTS
Derivation of Gradients of First Variation
16
C Proof of Proposition 1
17
D Proof for Theorem 5.1
19
Proof for Theorem 5.2
20
F Detailed Example of Algorithm Implementation
23
G Experimental Details
24
G.I Illustaaivve Examples Experimental Details
24
G.2 Conformer VENDI
24
G.3 PCA Projection for Fig. 5d
24
G.4 Validity Computation in Molecular Experiments
25
G.5 Practical Details for Experiments Molecules
25
H Update Step Reparametrization
31
15

---

Published as a conference paper at ICLR 2026
B DERIVATION OF GRADIENTS OF FIRST VARIATION
In this section we present derivations the results in equation 14 relating the gradient of the first variation of the trajectory rewards to the score function. We derive the result for L-FE, the result for G-FE follows as a subcase.
First, recall the trajectory rewards for L-FE:
V2, (p Vz, ((H(p)) ar DxL (p/ lp
(25)
V2, (2,())())) - an V., 8D (pt lppt')
(by linearity)
(26)
Thus it suffices to show derivations for V (p((7) and n., 8DxL (PT (p Starting with the entropy functional, recalling its definition H(pf) So pł (ar log P (xr)do have:
V... 8H((F)
V v. (1 log pÑ
(27)
V x, log p?
(28)
(29)
Similarly for the second term:
V., 8D 8D
(pF
Vz, př log pÝ
(30)
2x, (log pÝ 1 log pt
(31)
87 s
(32)
16

---

Published as a conference paper at ICLR 2026
C PROOF OF PROPOSITION 1
In this section we show that the optimization problem in 10 can be decomposed into an unconstrained expansion step followed by projection into the constrained set. We start by defining the following processes:
1 DKI D (Q/1Q=))
Q' E arg max (6L(Q" Q:po=pó
E xm
[o(x)]
(33)
7
1
Q E arg max (BC(Q)-1), Q) DEL (Q/QQ=))
(34)
Q:Po=pp A-1
7k
Q* E arg min Qipo
DxL (Q/ s.t.
[x(x)] =
(35)
Tmp
Q" E arg max [logv(x)] Q:po=pó ampp
Q
(36)
letting pr: Pr. Pr 1i refer to their respective marginal densities time Note that Q the output of the projection step in 15, and that Q is the output of the projection step in 16. The following Lemma asserts that solving the optimization problem in equation 10 equivalent solving the expansion step of 34 followed by the formal information projection step of 35.
Lemma C.I. Let Q be the process associated with the previous iterate - and let Q and Q be defined as above. Then Qf Q².
Proof. First, note that the processes Qt-1 and O satisfy the following relationship (see e.g. Domingo-Enrice Enrich (2024) equation 22):
di
log IQ (X) (x)====(("=1))) + const.
(37)
which implies the following equality for arbitrary process q (taking the expectation and rearrang- ing):
( k- Q)
DKL = DKL (Q]@") - const.
(38)
Therefore the equation below holds for any arbitrary set of processes A:
1
arg mrg.maa( --1),
DKL (Q[QQ// = arg min DEL (Q(((*)
(39)
QEA
7k
9EA
and thus also holds for the set A (Q s.t. po Po k-1 and Enam [u(x)) 1}: the set of feasible solutions to 10.
Finally, the following Lemma reformulates the information projection step in 35 as the fine-tuning objective in 34:
Lemma C.2. Let Q and Q be defined as above. Then Q = Q*
Proof. Recall the definition of F
Q E arg max [log v(x)
1 DxL Q+11
(40)
Qupa=ph-1 pmp
7k
17

---

Published as a conference paper at ICLR 2026
and note that the expectation in the first term is finite only if u(z) # 0, P1 a.s., in which case it
vanishes. Thus the maximizer must belong the set {Q Po Po Rapp [u(x)) = 1} effectively
k-1
turnng the first term into a constraint.
18

---

Published as a conference paper at ICLR 2026
D PROOF FOR THEOREM 5.1
Theorem 5.1 (Convergence guarantee in the idealized process-level setting). Consider the objective
L defined in Equation (8), and let X = -Jo Ardl. Let {Q²} be the iterates generated by Equation (10)
with 7/. = 1/X for all E [K]. Then
X* [(Q') =(Q%)
s K
(er(Q""1Q0T))
(21)
where Q" E arg max (Q)).
Proof. Fix an initial reference measure Q = Q", and define the function
(QQ) DkL. (Q Q)
(41)
which measures the Kullback-Leibler divergence Q from this reference. This choice of Q will in the framework of mirror descent with relative smoothness (Bauschke
serve as the reference et al., 2017; Lu et al., 2018). The key point is that the objective L in Equation (8) is not necessarily smooth in the classical sense, but it X -smooohh relative to Q.
To formalize this, let D2 Q Q') denote the Bregman divergence generated by Q. By definition, Do QQQ) Q(Q') (áQ(Q'), Q Q*)
direct computation shows that when the KL divergence from fixed reference measure, the Bregman divergence reduces exactly to another KI. divergence:
Do Q') Dri. (Qki.QQ1) Q')
This equivalence will allow us to leverage classical properties relative entropy in the convergence analysis.
Next, consider the mirror descent iterates {Q*) for minimizing (-2)². smoothness, we have
By the definition of relative
(42) Here, the first inequality follows directly from the smssmoothness of (-L) relative to Q, defined in Equation (41). Intuitively, this is generalization of the standard quadratic upper bound used in classical smooth optimization, but with the Bregman divergence replacing the squared Euclidean norm.
(Q)) s + (8(=))())))))))))))))))))))))))) Q - + Q
We can refine this bound further by applying the three-point inequality of the Bregman divergence (Lu et al., 2018, Lemma 3.1). Let us define a linearized function
6 (Q) = /
and let 2 = Q x= = = Then the three- point identity gives
(8( ((-C)(Q*=1)) Q* - Q - (8(- k-1 "),, Q kh-1) + DD Q² X* Delu, Q*),
(43)
for any reference point μa Combining Equation (42) and Equation (43) yields
(Q*) s (-)) (8(-2)(Q*))) Q²-1) + X* DDDa,, @=11)==+-----00 X* Delu, Q*).
(44)
Finally, we can telescope this inequality over h: == 1,..,, K. Using the monotonicity of (-))(Q²)
along the iterates and the non-tegatyvity of the Bregman divergence De we obtain (Lu et al., 2018):
K
(--)((h-(--0))
s S Qº) >'Dolu, Q*) XD Q"),
(45)
for any Q. Dividing both sides by and rearranging gives a simple ergodic convergence rate:
(-)) - (-C) (=
(46)
K
which shows that the iterates converge at an O(1/K) rate in terms of the relative entropy.
O
We adopt the standard convention of convex minimization rather than concave maximization, which explains the negative sign in the formulation.
19

---

Published as a conference paper at ICLR 2026
E PROOF FOR THEOREM 5.2
To establish our main convergence theorem, we impose a few auxiliary assumptions that are widely used in the analysis stochastic approximation and gradient flows. These assumptions mild and typically satisfied in practical applications.
Assumption E.1 (Precompactness of Dual Iterates). The sequence of dual variables (62(0)) is precompact in the Lo topology.
Precompactness ensures that the interpolated trajectories of the dual iterates remain within a bounded region in function space. This property is crucial for applying convergence results based on asymptotic pseudotrajectories, and similar precompactness assumptions have appeared in the literature on stochastic approximation and continuous-time interpolations of discrete dynamics (Benaïm, 2006; Hsieh et al., 2019; Mertikopoulos et al., 2024).
In our dinee-dimensional parameter space, (E. essentially requires that the sequence of iterates produced by the solver remains in bounded set. This is very mild requirement: it is satisfied as soon as the solver does not diverge numerically (e.g., no exploding parameters or NaNs), which is exactly what we observe in all our experiments. Moreover, standard practices such as bounded initialization, weight decay, and gradient clipping can be viewed as explicit mechanisms that enforce this boundedness.
Assumption E.2 (Noise and Bias Control). The stochastic approximations in the updates satisfy, almost surely, the following conditions:
lb
00
+ 00
(47)
E (1bxll1e ||Ux|120e] < co.
(48)
nollbxllo < C0.
(49)
These conditions are standard the Robbins Mobbins-Monro framework (Robbins Monro, 1951; Benaim, 2006; Hsieh al., 2019). They guarantee that the bias of the stochastic updates vanishes asymp- totically, and that the cumulative effect of the noise remains controlled. Together, they ensure that the stochastic perturbations do not prevent convergence of the iterates to the optima of the target objective.
With these assumptions in place, we are ready restate the main result and present its proof.
Theorem E.1 (Convergence guarantee in the general trajectory setting (rigorous)). Suppose the oracle satisfies Assumptions E to E.2, and let the step-sizes {ru follow the Robbins-Monro rule (E 7k 00, Σx 2h < xc). Then the iterates {Q*) generated by FE satisfy
Q² -" a.s.,
(50)
where Q" E arg maxq ((Q).
As in the proof of Theorem 1, fix an initial reference measure Proof.
Q
and define the relative entropy functional
((Q)) = DKL QQ
(51)
Correspondingly, we introduce the initial dual variable
dQ
ho = (Q(Q") log dQ
where dQ denotes the Radon Radon-Nikodym derivative of Q with respect to Q This dual representation JQ encodes the convex geometry of the problem.
20

---

Published as a conference paper at ICLR 2026
Continuous-time mirror flow. We now consider the continuous-ting time mirror flow dynamics
h: 8( ((-))
(MF)
Q = 6 (h+),
where Q" denotes the Fenchel conjugate of the relative entropy functional. Explicitly, we recall that
y (h) logo E[e"],
which follows from the variational characterization of the Kullback-Leibles divergence (Hsieh et al., 2019; Hiriart-Urruty & Lemaréchal, 2004).
piscrrte-to-continuuss interpolation. connect the discrete algorithm with the flow equation MF, introduce interpolation of the iterates. Define the linearly interpolated process h(t) by
7k 7.
h(1)= = h
7 k++
(h 1h++1 h²), h" = 82(0))
ar
(Int)
where Or are the step sizes. This construction yields a continuous-time trajectory (h(4)) that faithfully tracks the discrete iterates the limit of vanishing step sizes.
Asymptotic pseudotrajectories. We recall the notion of asymptotic pscudotrajectory (APT), which provides the precise mathematical bridge between discrete stochastic processes and determin- istic flows.
Let denote the flow map associated with equation MF; that is, (x.(f) the solution of equation MF at time h when initialized at f.
Definition 3 (Asymptotic Pseudotrajectory (APT)). A trajectory h(t) is called asymptotic pseu- dotrajectory (APT) of equation MF for every finite horizon T 0,
lim t-000 oShST
sup
||h[l + (x((il)) =
Intuitively, this condition requires that the interpolated sequence asymptotically shadows the exact flow on every bounded time interval.
Limit set characterization. The central result of Benaīm (2006) asserts that the long-ten terg-teemmn behavior of an APT is governed by the internally chain transitive (ICT) sets of the limiting flow.
Theorem E.2 (APT Limit Set Theorem (Benaim, 2006, Thm. 4.2)). hf) is a precompact APT of equation MF, then almost surely its limit set lies within the set of internally chain- thain-transitive (ICT) points of the flow.
Reduction of the convergence proof. With these tools, the convergence analysis reduces to verifying two key claims:
(C1) Under Assumptions E. to E.2, the interpolated sequence (h(t)) indeed forms precompact APT of equation MF.
(C2) The set ICT points the flow equation MF coincides with the set of stationary points of C.
Verification of Claim (C1). Precompactness follows directly from Assumption E. which guaran- tees uniform tightness the sequence of measures and hence compactness of their trajectories the weak topology. In addition, standard arguments from stochastic approximation (Hsieh et al., 2019; Benaïm, 2006; Mertikopoulos et al., 2024) yield the following quantitative estimate: for every finite horizon T 0, there exists a constant C(T) such that
|h + h) (h(t)))) < [(T) 1, I,T ((T) (T))
sup T
21

---

Published as a conference paper at ICLR 2026
where A(t - 1) denotes the cumulative effect of noise the interval [t 1, + T 1], while b(T) and y(T) capture, respectively, the bias and step-size contributions. This bound quantifies the deviation of the interpolated process from the deterministic mirror flow equation MF.
APT approximation. Under the noise and bias conditions Assumption E.2, both perturbations vanish asymptotically:
lim (t - +,1 lim b(T) 0, 1-22
1-000
uniformly over bounded horizons T. Consequently, the discrepancy in the above bound vanishes in
the limit, and the interpolated process h(t) shadows the continuous-time flow arbitrarily well.
Altogether, these arguments show that (t) indeed precompact asymptotic pseudotrajectory of the mirror flow.
Verification of Claim (C2). The flow equation MF is precisely the continuous-time mirror de- a gradient flow in the spherical
scent dynamics associated with (-L), which is known Hellinger- Bittinger-Kattrovvide geometry (Mielke & Zhu, 2025). As such, (-L) acts strict Lyapunov function for the system: along any non-stationary trajectory. di
d (-:) < O. By (Benaïm, 2006, Corollary 6.6), every precompact APT converges to the set stationary points of the Lyapunov function. Since the objective function L is the relative entropy, and hence strictly convex, its stationary point coincide with global minimizer.
Conclusion.
Combining (C1) and (C2) with Theorem E.2, we deduce that the interpolated process (t) converges almost surely to the set of minimizers of (-f). which readily implies that the original sequence (Q*) inherits the same convergence guarantee.
22

---

Published as a conference paper at ICLR 2026
F DETAILED EXAMPLE OF ALGORITHM IMPLEMENTATION
In this section we provide comprehensive pseudocode of an example implementation for the two FINETUNINGSOLVER subprocedure in Alg. implemented using a variation of Adjoint Matching (AM) which is introduced comprehensively in Domingo-Enrich et al. (2024), although we provide pseudocode below for completeness. We note that in principle one could substitute for any other linear fine-tuning method.
Before presenting the implementations, we shortly clarify some relevant notation. The algorithm makes explicit use of the interpolant schedules and introduced in equation 1. We note that in flow model literature they are more commonly known as a₂ and Bt. We denote by uhe the velocity field corresponding to the pre-trained policy The and likewise use ufnee for the velocity field corresponding to the fine-tuned policy. In short, FINETUNINGSOLVER first samples trajectories, which are then used to approximate the solution of surrogate ODE whose marginals are used regression targets for the control policy (see Domingo-Enne Earaggh-Errccheeee et al. (2024) Section 5 for a full discussion). We note that FINETUNINGSOLVER can be used for objectives with and without trajectory rewards, simply by setting trajectory rewards to zero.
Algorithm Adjoint Matching for fine-tuning Flow Matching models (FINETUNINGSOLVERR
Require: w pre-trained FM velocity field, (V gradients of trajectory
rewards, {>e frejo.1]: (optional) trajectory reward weights, y: fine-tuning strength 1: Initialize fine-tuned vector fields: finetune = u with parameters -} do Sample m trajectories X = (X1)cE(o....)) with memoryless noise schedule a(I) =
2: for .O......i .
3:
2k0 11 -k)). egg.
Xeth X₁ h 2ug finetune (Xt,t)
de we
X.) + vho(()ee Et, ~ ~(,, Xo NO
(51)
4: For each trajectory, solve the lean adjoint ODE backwards in time from 1 to 0, e.g.:
ârيk= = āz + hā, Vx, 22 base (Xe,))
xx) Vx, â 7 V (X1).
wi
(52)
5: Note that X, and a, should be computed without gradients, i.c., X; stopgrad(X;), ā = stopgrad(ā.).
6: For each trajectory, compute the following Adjoint Matching objective:
Lidi Match (0)
Σ
2 o())
finelume (Xe, f) huse vu t)) tz(
(53)
(E(O...,,1AA)
7: Compute the gradient (eC and update 0 using favorite gradient descent algorithm.
8: end for
Output: Fine-tuned vector field
fineuuee
Crucially, we employ the fine-tuning oracle in Alg. 8 also implement the projection step within Sec. 4, as indicated within Alg. 4. Moreover, in the case of a non-differentiable (weak or strong) verifier, the projection step can be implemented via a 0-th order RL-based fine- tne-tuning method, e.g., (Fan et al., 2023; Black et al., 2023), which induce the same closed- foom solution as Alg. 8.
23

---

Published as a conference paper at ICLR 2026
G EXPERIMENTAL DETAILS
G.1 ILLUSTRATIVE EXAMPLES EXPERIMENTAL DETAILS
Numerical values in all plots shown within Sec. 6 are means computed over diverse runs of via different seeds. Error bars correspond to 95% Confidence Intervals. For the following comparisons, we aimed tune each algorithm parameters so that the method would work well in the specific illustrative example.
Pre-trained models. The pre-trained models appearing in Sec. 6. in the context of illustrative examples, are learned synthetically generated data, via standard learning procedures. In particular, in Sec. 6 we always show samples generated by such pre-trained models.
Global Flow Expansion.
For G-FE, we use A, = 1iffftt>> - 0.05, and X 1.2 otherwise, "hk (1+3(k-1]) 7f 2
1.5
and K 10.
For CONSTR employ 17 2.
0.315 For S-MEME employ 7 (I+3(E-1)) and K 10 and use s7(2) sT-e(x) with 0.02 discussed in Sec. 4.
Local Flow Expansion. The models used act on 2-dim state (I1, 22). of which is shown only the coordinate in the process-level figures reported in Sec. Since we use as oracle AM, which requires differentiable gradient, consider binary verifier (shown in Fig. 5c in grey), which we smoothen, thus rendering it differentiable and approximate. Notice that differentiability is not required by FE, but is rather implementation detail due to the specific oracle used (i.e., AM Miningo-Enrich et al., 2024), see Sec. F for further details). In particular, there exist several analogous oracles that do require function differentiability (e.g., Fan al., 2023).
For L-FE we employ K 8, A 0 variance, i.e., de / 2(kg ( ke)), otherwise; 7k 0.3, 7) = 0.1.
0.015, and proportional with the process
For FDC K 8, 7kk 0.06 and use (x) (x) with 0.02 discussed in Sec. 4.
G.2 CONFORMER VENDI
We begin with a detailed explanation our diversity metric: conformer VENDI. In general, VENDI (Friedman & Dieng, 2022) is diversity metric operating arbitrary inputs based pairwise distance kernel k X' x X' - [0, 1]. For list of inputs 21...., In and a symmetric pairwise distance kernel VENDI defined
n
V VSI = exp (- Σ
A log Ax)
(52)
where A are eigenvalues of the distance matrix K with KG₃ k(Z,, xz)). In our work, we use molecular fingerprinting method combined with kernel simply defined the Euclidean distance between fingerprints, thereby inducing a kernel on molecules themselves. The particular fingerprinting method is defined the sorted list pairwise atomic distances. Formally, for molecule with N atoms at positions 01,... aN we first compute the matrix of pairwise distances Aij = ||a,
aj l2, 1 iiisj & N, which is then sorted as A Ayyo--y/iNNN--u yielding the fingerprint Ā E N(N-))
G.3 PCA PROJECTION FOR FIG. 5D
In this section we explain the dimensionality reduction method used to generate the plot in 5d. We first generated 25000 molecules from both the pre-trained model prr (yielding Drs and the fine-tuned
24

---

Published as a conference paper at ICLR 2026
model x² (yielding Dinntuuedd), which was computed by the L-FE algorithm The for K iterations with a = 9. We then fingerprinted each set molecules using the method described above, and fit a 1-dim PCA on the fingerprints for D using SCIKIT-LEARN (Pedregosa et al., 2011), which was then used to transform both D and Dflnctuned into 1-dimensional vectors. Fig. 5d corresponds histogram plot of each of the resulting sets of vectors.
G.4 VALIDITY COMPUTATION IN MOLECULAR EXPERIMENTS
In the context of our experiments molecules the concept of validity is defined through pipeline of several checks, defined below. Our validity function passes through each one sequentially, returning an invalid result if any fail, and a valid result only if all checks pass.
First, we attempt to samitize each molccule using RDKit's ' (RDKit) CHEM. SANITIZEMOL function. As an added check, we test if it is possible to convert the molecule and back from SMILES (Weininger, 1988) notation.
2. We then iterate over each atom in the molecule, checking for any implicit hydrogens (our model must generate explicit hydrogens as FlowMol (Dunn & Koes, 2024) does) or any radical electrons which would make the molecule invalid.
3. Finally perform our weak verifier check, filtering out molecules for which any two atoms are closer than 0.9Å. Details on this weak verifier are explained below.
The final validity check is evaluate the weak verifier on molecules that pass the previous steps. The weak verifier itself is evaluated by first computing the vector of pairwise distances between atoms Á (see discussion in Section G.2 above), then taking the minimum element Ag and checking if is lower than 0.9A, in which case molecule is classified as invalid. Including this check in the validity function guarantees by construction that the weak verifier satisfies Definition 2, since failing the weak verifier check implies failing the validity check as well.
G.5 PRACTICAL DETAILS FOR EXPERIMENTS ON MOLECULES
In this section we discuss the practical choices behind the molecular design experiments discussed in Section 6. We start with a discussion of hyperparameter settings, followed by some implementation techniques adapting the verifier feedback for first-order solver, and finally discuss hardware and platform used for training.
G.5.1 HYPERPARAMETE CHOICES
For our experiments molecules we use the FlowMol CTMC model from Dunn & Koes (2024) trained on the QM9 dataset as a pre-trained model. We run each algorithm (L-FE and FDC) with the following parameters:
= 5 iterations
Regularization strength of =
Decreasing stepsize of "
70 1+
with 7 0.00001
For the trajectory reward weighting (L-FE only) we use d = or v 2nd @ K4 ki). ensuring A as for stability discussed at the end of section
For both, we clip the score near the end of the trajectory as sf s7 suin t,1-c) for 0.005.
We fix the number of atoms in generated molecules (for model training and metric calcula- tion) to 10, in order to simplify metric calculations.
When using Adjoint Matching (AM) (Domingo-Enrich et al., 2024) to implement the subroutines of any algorithm we use N = 4 iterations, we sample batch of m trajectories of length 40 at each iteration and update the parameters # using Adam Kingma & Ba (2015) with a learning rate of 0.00055. We note that since FlowMol CTMC is mixed categorical and continuous flow model, only use AM to update the parameters corresponding to the continuous outputs of the model, i.c., the atom positions.
25

---

Published as a conference paper at ICLR 2026
G.5.2 HYPERPARAMETER CHOICES FOR EXPERIMENTS ON GEOM
In order to the performance of our model more practical setting, we performed additional experiments using the GEOM dataset. We again use a FlowMol CTMC model from Dunn & Koes (2024) as pre-trained model, however this time using checkpoints from training on the GEOM dataset. Note that for simpler hyperparameter search we use the reparametrization discussed in section H. The optimal parameter set each algorithm (L-FF, G-FE, NSE, and FDC) is not identical in this setting, therefore we first report the hyperparameters in common before listing the differences for each algorithm below:
Common hyperparameters for L-FE, G-FE, NSE, FDC:
3 iterations
Constant (adjusted) stepsize 7i- To. although the magnitude fo differs for each algorithm For the trajectory reward weighting (L-FE, G-FB and NSE) we use X o,
2 - ko), ensuring X - 0 as + for stability as discussed at the end of section 4
mmin(t for For all methods, we clip the score near the end of the trajectory as sf S 0.005.
We fix the number of atoms in generated molecules (for model training and metric calcula- tion) to 30, in order to simplify metric calculations.
Per-algorithm hyperparameter variations:
L--F:
0.4
t 0. 0.0005 (constant)
a 5.0 (constant)
NSE:
0.0
7x 0.0002 (constant)
FDC:
= 0.9
- 7x = 0.0005 (constant)
When using Adjoint Matching (AM) (Domingo-Enrinn et al., 2024) implement the subroutines of any algorithm usign the GEOM FlowMol model we use N = 4 iterations, we sample batch of m trajectories of length 40 each iteration and update the parameters 0 using Adam Kingma Ba (2015), with a learning rate of 0.0001. We note that since FlowMol CTMC is mixed categorical and continuous flow model, we only use AM to update the parameters corresponding to the continuous outputs of the model, i.e., the atom positions.
G.5.3 SMOOTHING THE WEAK VERIFIER
Since we use Adjoint Matching for all fine-tuning tasks we need all rewards to be differentiable. While our weak verifier formally defined as v(x) + x respects the minimum atom separation bound of 0.9A, we use the following differentiable approximation using sigmoid soft indicator function:
N(N-1)/2
v(x)= N(N 1)/2
1
Σ
exp 0.9)
(53)
exp - 0.9)
where A, are the pairwise atomic distances introduced in Section G.2 above. This alternative verifier differentiable and provides gradient feedback everywhere and therefore can be used in Adjoint Matching.
26

---

Published as a conference paper at ICLR 2026
G.5.4 ABLATION STUDY
We report an ablation study of the key hyperpurameters a, 7k. and T/K- Notice first that since L-FE is generalisation of both NSE and G-FE, we recover NSE by setting instead setting a = The following plot shows a comparison of the Pareto fronts for each method, all compared against FDC. Furthermore, at the end of this section, we report results of running the projection step alone (effectively setting 7/k 0).
0, and recover G-FE by
LFFE
G-FE
304
222
2003
1252
1000 1750
2000
1250
Contomer
Figure 7: Comparison of different parametrization of L-FE, G-FE, NSE, FDC, for K = 3. We compute Conformer Vendi and validity over 5000 samples, and average results over 8 runs for each parameterization with 8 different seeds.
In Fig. we report all the Pareto dominant points chosen for each method evaluated in the following ranges of hyperparameters (using the reparametrized notation from Appendix H):
3: 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95
7 (constant): 0.0001, 0.0002, 0.0003, 0.0004, 0.0005, 0.001
(constant): 0.0, 0.1, 0.5, 1.0, 2.0, 5.0
For each combination of the hyperparameters above, we run = 3 iterations of each method (L-FE, G-FE and NSE) and average the results across 8 seeds. Conformer Vendi and validity always computed for batches of 5000 samples. For all other hyperparameterss refer to the previous section G.5.2 discussing general hyperparameter choices for the GEOM model. To build the plot above, we simply drop all points that are Pareto dominated (i.c. same/greater Conformer Vendi and same/greater validity) by some other point, and plot the remaining points. We also drop all points with Conformer Vendi lower than the pre-trained model. The results shown in Fig. clearly demonstrate the ability each method to trade off validity for higher diversity (measured by Conformer Vendi). However notice that only G-FE and L-FE can significantly increase validity due their use of verifier information. Overall, notice that both L-FE and G-FE outperform all other methods almost uniformly, and NSE uniformly outperforms FDC (both unconstrained exploration algorithms). L-FE is especially effective at retaining or even increasing validity: significantly dominates all other methods in terms of validity in the range of 475 to 550 Conformer Vendi (top left of Fig. 7).
27

---

Published as a conference paper at ICLR 2026
G.5.5 TABLES OF RESULTS
We report the numerical results (value and confidence interval) for each point in Fig. 7 in the tables below. For each method we report the mean and confidence interval corresponding to each point from left to right, and report the parameterization (3, % and 1j if applicable) used to generate the point. We acknowledge the size of the confidence intervals increases drastically in the regime of more exploratory points (higher Conformer Vendi): increasing stability of exploration methods in that regime remains an open problem. Still, we note that I.-FE dominates other methods most significantly when the Conformer Vendi increase is modest (less than 550 Conformer Vendi), and in that regime the confidence intervals are reasonably concentrated.
Table I: Values for L-FE reported in Fig 7 with 95% confidence intervals
%
%oo
Mean Validity (95% CI)
Mean Conformer Vendi (95% CI)
0.3
0.0002
5.0
0.89 (0.87 + 0.91)
481.35 (460.24 = 500.42)
0.6
0.0005
5.0
0.85 (0.79 0.89)
500.11 (460.72 = 546.91)
0.4
0.0005
5.0
0.83 (0.76 + 0.89)
519.05 (454.45 585.99)
0.1
0.0004
5.0
0.82 (0.70 0.90)
529.47 (465.92 = 629.29)
0.2
0.0004
5.0
0.74 (0.53 = 0.88)
531.90 (452.14 633.92)
0.2
0.0005
5.0
0.70 (0.53 0.86)
563.75 (472.19 = 683.79)
0.2
0.0004
2.0
0.68 =
581.26 (467.44 728.66)
0.3
0.0005
5.0
0.68 (0.46 0.85)
796.64 (475.55 + 1371.21)
0.2
0.0005
2.0
0.65 (0.43 = 0.84)
831.91 (496.49 = 1423.91)
0.1
0.0004
2.0
0.62 (0.39 + 0.82)
900.08 (515.31 + 1591.61)
0.1
0.0005
5.0
0.60 (0.33 + 0.83)
1382.46 (516.37 2493.28)
0.4
0.0005
0.5
0.58 (0.31 + 0.81)
1682.32 (560.86 = 3320.25)
0.1
0.0005
2.0
0.37 (0.13 0.64)
1956.33 (684.10 10 3322.31)
0.1
0.0005
0.5
0.37 (0.12 + 0.64)
2068.02 (866.88 = 3366.56)
Values are mean (95% confidence interval). Confidence intervals computed using bootstrapping and shown to
two decimal places.
Table 2: Values for G-FE reported in Fig 7 with 95% confidence intervals
%
Mean Validity (95% CI)
Mean Conformer Vendi (95% CI)
0.0002
0.5
0.76 (0.69 = 0.83)
485.65 (442.52 +: 524.71)
0.0005
5.0
0.69 (0.46 0.86)
890.19 (434.36 + 1731.52)
0.0004
0.5
0.57 (0.33 + 0.79)
1004.79 (531.74 + 1790.78)
0.0004
1.0
0.52 (0.27 = 0.77)
1125.94 (521.23 = 2182.77)
0.0004
0.1
0.48 (0.22 = 0.72)
1676.67 (533.47 = 28888.04
0.0005
1.0
0.46 (0.22 + 0.71)
1706.53 (664.86 = 3100.05)
0.0005
2.0
0.35 (0.12 0.60)
1898.72 72 (796.05 = 3221.332
0.0005
0.5
0.31 (0.08 = 0.56)
2245.62 (1135.37 + 3500.43)
Values are mean (95% contidence interval). two decimal places.
Confidence intervals computed using bootstrapping and shown to
28

---

Published as a conference paper at ICLR 2026
Table 3: Values for NSE reported in Fig with 95% confidence intervals
B
%
Mean Validity (95% CT)
Mean Conformer Vendi (95% CT)
0.0
0.0002
0.74 (0.68 0.81)
518.92 (505.70 531.90)
0.4
0.0004
0.68 (0.58 + 0.77)
529.85 (484.34 + 580.24)
0.3
0.0004
0.68 (0.55 0.79)
537.83 (491.25 + 588.98)
0.1
0.0004
0.60 (0,42 = 0.77)
639.04 (501.23 1 857.000
0.2
0.0004
0.57 (0.37 0.74)
902.23 (534.29 1559.95)
0.1
0.0005
0,48 (0.25 = 0.69)
1678.29 (669.60 = 3024.30)
0.0
0.0005
0.19 (0.02 = 0.40)
2151.14 (1256.65 + 3162.47)
Values (95% confidence interval). Confidence intervals computed using bootstrapping and shown
two decimal places.
Table Values for FDC reported Fig with 95% confidence intervals
B %
Mean Validity (95% CI)
Mean Conformer Vendi (95% CT)
0.4
0.0001
0.71 (0.67 0.74)
484.34 (461.47 + 506.75)
0.5
0.0002
0.71 (0.63 = 0.77)
496.06 (467.29 = 523.97)
0.7
0.0002
0,70 (0.64 = 0.75)
502.49 (483.86 = 526.78)
0.5
0.0001
0.69 (0.65 = 0.74)
505.84 (468.66 2 538.20)
0.2
0.0003
0.67 (0.58 = 0.77)
539.29 (504.70 = 558.655)
0.9
0.001
0.56 (0.49 = 0.62)
601.06 (531.19 + 699.07)
0.3
0.0004
0.54 (0.34 + 0.72)
664,45 (531.77 847.98)
0.5
0.0005
0,46 (0.30
674.63 (544.65 = 824.76)
0,4
0.0005
0.41 (0.25 0.58)
768.98 (599.68 975.95)
0.1
0.0004
0,41 (0.24 0.56)
1084.21 (574.88 = 2062.43)
0.1
0.0005
0.40 (0.15 = 0.66)
1774.38 (828.37 + 2904.44)
0.2
0.0005
0.30 (0.07 0.56)
1948.54 54 (954.87 + 3004.34)
Values (95% confidence interval). Confidence intervals computed using bootstrapping and shown
decimal places.
G.5.6 ABLATION: PURE EXPAND AND PROJECT STEPS
In order to isolate the effects of the EXPAND and PROJECT steps, we report the result of running each for K iterations Fig. 8. Fig. shows the Pareto optimal points of the following hyperparameter combinations:
EXPAND step (NSE, black circles in Fig. 8)
3: 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95
To: 0.0001, 0.0002, 0.0003, 0.0004, 0.0005
PROJECT step (green circles in Fig. 8)
1/6 "/u constant set to: 0.0, 0.1, 0.5, 1.0, 2.0, 5.0
We remark again that running only the EXPAND step without the PROJECT step is equivalent to running NSE. For each parameter combination above we average results over seeds. Notice that the EXPAND and PROJECT steps have the reverse effect of each other: the EXPAND step trades off validity for increased diversity (Conformer Vendi) where as the PROJECT step increases validity at a cost of marginally reduced diversity.
29

---

Published as a conference paper at ICLR 2026
ML
12:0
Figure 8: Illustrating the result of isolating the EXPAND step (NSE) and the PROJECT step
G.5.7 COMPARISON WITH INFERENCE TIME FILTERING
In this section, we consider an additional comparison of the proposed methods against a baseline corresponding FDC plus inference- time filterin. We report the results in Fig. 9 below. In partieular, we consider the following algorithms:
FDC (De Santi et al., 2025b) inference-tima filtering (blue Fig. 9)
NSE (Alg. 3 time filtering (black in Fig.
L-FE
In particular, for the first two cases above, given model fine-tuned using FDC or NSE, we filter the samples generated inference time by using the same weak verifier employed by L-FE. We that this schemes induce a closed-form solution formally mathematically equivalent to conditional sampling, with the condition that the samples are in Ω2 Thus, this schemes effectively amounts to simulating perfect projection step, albeit at the price of costly per-sample inference- time filtering (e.g., via rejection sampling). We calculate the Conformer Vendi and validity on each set of 5000 molecules, and report the results in Fig. 9 below. As one can notice, NSE filtering shows superior performance compared against FDC + filtering, while L-FE shows slightly less validity than NSE + filtering, while being significantly cheaper sampling time, which is needed requirement for certain generative modeling applications. Compared with FDC + filtering, L-FE shows superior exploration capabilities and slightly lower validity. Nonetheless, it might be possible parametrize L-FE to achieve lower diversity and higher validity, similarly to FDC + filtering. In particular, these two algorithms theoretically have the same closed-form solutions, and the concrete differences amount the high per-sample cost of inference-time filtering, and the superior exploration capabilities of L.-FE and NSE over FDC likely due noised space exploration, discussed in Sec. 2.
Each point in Fig. 9 corresponds to different Pareto optimal parametrization (3, y and if applicable) for each method. Each method run for K 3 iterations, with results averaged across seeds. See section G.5.2 for details about other hyperparameters.
G.5.8 HARDWARF
We ran all our experiments using a single NVIDIA RTX 2080Ti GPU per run (QM9 experiments) single NVIDIA RTX 4090 GPU per (GEOM experiments).
30

---

Published as a conference paper at ICLR 2026
fittring SStoring
140
9--
0.75
contermer vind
Figure L-FE compared to NSE and FDC with inference-time filtering for different parameterizations. We note that L-FE maintains competitive performance with both methods despite having no filter applied the output samples, and NSE still outperforms FDC when both are corrected with an inference-time filter.
H
UPDATE STEP REPARAMETRIZATION
Recall the expression for the gradient of running costs in the Local Flow Expander algorithm (Alg. equation 14):
v., 88(p8) d v, ((H(p() (p:
(54)
Ae(- +a(s? -s,"))
(55)
=>>e((a 1)s7 -ass
(56)
which are then multiplied by the stepsize у at each iteration, resulting in the following expression being plugged into the Adjoint Matching algorithm as the gradient of the running cost:
V = -m>iaaaa + 1)s7 - a.s
(57)
While has an intuitive interpretation the regularization strength in objective 7, has the unfortunate side-effect of scaling the magnitude of the running cost which could potentially have the opposite effect. Indeed, notice that as - oo the running costs explode. For practical applications seems more suitable to reparametrize the running cost as follows:
v fe -s _.s
(58)
for 3 = a+1 E [0. 1]. absorbing a (a + 1) factor into the new stepsize 7y:
i = (a + 1)7k
(59)
Note that this parametrization as expressive as before but easier to tune as disentangles effect of the stepsize and the regularization strength.
31


Reviews:
[{"review_id": "cvF7mVTzbG", "reviewer": "Reviewer_Kx6q", "summary": "This paper proposes Flow Expander (FE), a framework that expands the support of pretrained flow or diffusion models under verifier constraints. The method formulates verifier-constrained entropy maximization as a mirror-descent optimization problem over probability measures, consisting of two alternating steps: an Expansion step that increases entropy and explores new modes, and a Projection step that enforces validity through a soft-verifier function. The authors provide theoretical convergence guarantees and demonstrate the approach on toy 2D examples and QM9 molecular conformers. The work is theoretically well-motivated, connecting flow-based generative modeling with constrained optimization, but its empirical validation and baseline clarity remain limited.", "questions": "**Separation from S-MEME.**\nMany components of the proposed Flow Expander (e.g., entropy maximization, mirror-descent formulation, convergence proof) appear closely aligned with S-MEME. Could the authors clearly specify which elements are newly introduced in this paper (e.g., verifier projection) and which are inherited from S-MEME or related prior works?\n\n**Applicability to discrete domains.**\nThe proposed formulation assumes a continuous variable space, where functional gradients and mirror-descent updates are well defined.\nHow could this framework extend to **discrete or combinatorial domains** such as molecular graphs or protein sequences, where the notion of a variational gradient is not clearly defined?\nIn particular, can the *soft-verifier* idea be adapted to these settings, which are arguably more relevant to real-world drug or material discovery?", "limitations": "", "rating": 4, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "**Strong motivation and relevance.**\nThe paper tackles an important limitation of pretrained flow and diffusion models—namely, their inability to explore beyond the data manifold while maintaining sample validity. The idea of integrating verifier-based constraints into generative model fine-tuning is timely and relevant for scientific design applications (e.g., molecular or material generation).\n\n**Principled formulation.**\nThe proposed *Flow Expander* framework is grounded in a clear optimization principle: verifier-constrained entropy maximization. Casting this as a mirror-descent problem in the space of probability measures provides a mathematically elegant and unified view of exploration under validity constraints.\n\n**Theoretical completeness.**\nThe paper presents a solid convergence analysis. The proofs, though dense, follow established mirror-descent theory and offer formal justification for the proposed update rule.\n\n**Potentially general concept.**\nThe introduction of a *soft-verifier* mechanism is conceptually interesting and, if properly extended, could provide a practical interface between learned generative models and rule-based or simulation-based validity filters used in real-world design workflows (e.g., high-throughput screening, physical constraints, or chemical property checks).", "weaknesses": "**Limited and unconvincing experiments.**\nThe empirical evaluation is restricted to 2D toy examples and a small-scale QM9 conformational generation task. These setups are insufficient to demonstrate practical effectiveness or scalability of the proposed method. The results primarily serve as proof-of-concept demonstrations rather than evidence of real-world impact or generalization capability.\n\n**Unclear and unverifiable baselines.**\nThe paper cites Uehara et al. (2024, Section 8.2) as the source of the “CONSTR” baseline. However, that section contains only a theoretical corollary without any algorithmic description, implementation details, or experimental setup, making it impossible to reproduce or verify the reported baseline results. Moreover, many components of the proposed method appear to be directly borrowed from S-MEME (e.g., the mirror-descent formulation, entropy maximization objective, and theoretical convergence argument), yet the paper does not clearly delineate which parts are newly introduced and which are adapted from prior work. This lack of transparency in baseline selection and methodological novelty significantly undermines the credibility and reproducibility of the experimental claims.\n\n**Severe clarity and notation issues.**\nThe paper suffers from significant readability and presentation problems:\n1. Misuse of notation and inconsistent subscripts/superscripts.\n2. line (234) incorrectly states $\\delta\\mathcal{G}(\\mu)\\in F(\\mathcal{X})$, though the functional derivative should be a function over $\\mathcal{X}$.\n3. Algorithm 4 is referenced but missing.\n4. Symbols such as $F(\\mathcal{X}), \\mathcal{G}_t$, and $\\mathcal{L}(Q)$ are repeatedly overloaded, making the derivations unnecessarily hard to follow. These issues significantly reduce the accessibility of an otherwise theoretically interesting paper.\n\n**Applicability and experimental realism.**\nThe introduction of *soft-verifiers* is an interesting and potentially powerful concept, as it could, in principle, allow high-throughput filtering strategies—commonly used in drug and material discovery—to be incorporated into generative models. However, the paper does not provide convincing examples where this mechanism becomes practically important. Beyond toy examples, the molecular conformational experiments hold limited chemical relevance: in conformational space, the goal is typically to reproduce the Boltzmann distribution rather than to impose external validity filters, so the verifier concept is of marginal utility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YKSvNnIF5t", "reviewer": "Reviewer_oQPY", "summary": "Flow and diffusion models are typically pre-trained on limited available data. As a result, they tend to generate samples from only a narrow portion of the feasible domain.\nTo address this limitation, the authors assume access to a verifier and propose adapting a pre-trained flow model so that its induced density expands beyond regions of high data availability. They pose the key question:\n“How can we leverage a given verifier to adapt a flow or diffusion model to generate designs beyond high data-availability regions while preserving validity?”\nThe authors consider two types of verifiers:\n•\tStrong verifier: a function nu: X -> {0,1} that characterizes validity exactly, i.e., nu(x)=1 if and only if x is valid.\n•\tWeak verifier: a function that acts as a filter—it rejects some invalid designs but may fail to detect others (formally, nu(x)=0 => x is invalid).", "questions": "•\tIn the description of Continuous-time Reinforcement Learning, states and actions have been defined, but I think the definition of reward is missing.\n\n•\tLine 175: If Omega_v is not a bounded set, then without further constraints, there is no maximum entropy distribution. Is there a simple way to generalize Problem 5 to this unbounded setting?\n\n•\tInstead of maximizing entropy, can we get reasonably good results by simply maximizing the variance of the distribution instead?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Major contributions: \n•\tFlow Expander (FE), a principled probability-space optimization scheme\n•\tA theoretical analysis of the proposed algorithm\n•\tAn experimental evaluation of FE\n\nIt is a well-written paper, with new ideas and interesting results. \n\nI think many researchers in our community will appreciate this paper.", "weaknesses": "•\tThe paper is somewhat dense and not always easy to follow.\n\n•\tThe numerical experiments are somewhat limited. I appreciate both the illustrative examples and the results on the molecular design task, but I wish the paper included more high-impact, real-world examples where verifiers exist.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K89oNNPXVx", "reviewer": "Reviewer_MbXc", "summary": "This paper addresses the problem that pre-trained generative models (flows, diffusion) tend to sample from a narrow part of the valid design space, which is a key limitation for scientific discovery. The authors propose to \"expand\" the model's density by leveraging an external verifier. The paper formalizes this into two problems: Global Flow Expansion (using a perfect strong verifier) and Local Flow Expansion (using an imperfect weak verifier). \n\nTo solve these, the paper introduces Flow Expander (FE), a mirror descent algorithm. A key idea is to formulate the optimization objective over the entire noised state space of the flow process ($Q^\\pi = \\{p_t^\\pi\\}_{t \\in [0,1]}$) rather than just the final time-step $p_1^\\pi$. This is claimed to provide a principled way to avoid score divergence issues near $t=1$. The paper provides theoretical convergence guarantees and presents experiments on 2D illustrative tasks and a molecular conformer generation task (QM9).", "questions": "* Can you provide a direct comparison against the FDC + post-hoc rejection sampling baseline? This would be very helpful to demonstrate that your complex constrained optimization is superior to a simple filter.\n* How do you propose to use FE with a truly black-box, non-differentiable verifier (e.g., a hard RDKit sanitization check)? This seems to be the most common and important use case.\n* Please address the question that your L-FE experiment parameters ($\\alpha=9$, $\\gamma_k \\approx 0.00001$) are so conservative the model is \"not moving\" from the pre-trained state. Can you provide a hyperparameter ablation study for $\\alpha$ and $\\gamma_k$ to show how the diversity/validity trade-off changes?\n* To help us understand the method, can you provide an ablation showing the results of running only the EXPAND step and only the PROJECT step for $K$ iterations?\n* Can you provide an experiment for Global Flow Expansion on a real-world dataset, not just a 2D toy problem?\n\nI would be willing to raise my score if the authors can thoroughly address the concerns raised in the weaknesses and questions section. In particular, the concerns regarding the experiments (ablation, large-scale dataset).", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "* Addresses how to leverage pre-trained generative models to explore novel and valid regions of a design space, moving beyond the original data distribution.\n* Formalizes the problem into Global Flow Expansion (using a strong verifier) and Local Flow Expansion (using a weak verifier).\n* Lifts the optimization objective from the final time-step ($p_1$) to the entire noised state space ($Q^\\pi$) to theoretically mitigate the score divergence problem that occurs as $t \\to 1$.", "weaknesses": "* One weakness is the use of potentially uninformative baselines. The paper compares FE (a \"search + constraint\" method) against \"search-only\" (S-MEME/FDC) and \"constraint-only\" (CONSTR). This comparison is not fully informative, as FE is designed to outperform them. A fair and important baseline would be unconstrained exploration (FDC/S-MEME) followed by post-hoc rejection sampling using the verifier. Without this, the practical value of FE's complex optimization is unknown.\n* The method's reliance on a differentiable verifier is a considerable practical limitation. The chosen solver (Adjoint Matching, Alg. 3) requires gradients from $\\log v(x)$. Most real-world verifiers for scientific discovery (e.g., RDKit SanitizeMol, physics simulators) are black-box and non-differentiable. The paper's workaround (smoothing a simple function in App G.1, G.5.2) does not solve this general problem, limiting the method's applicability.\n* The experimental validation is not fully convincing. The 2D experiments can be sensitive to tuning, and the QM9 dataset is too small-scale to demonstrate robustness.\n* There is a notable absence of hyperparameter ablation studies for the key parameters ($\\alpha, \\gamma_k, \\eta_k$).\n* This is particularly concerning for the L-FE molecular experiment. The chosen parameters ($\\alpha=9$, $\\gamma_k=0.00001/(1+k)$) are so conservative that the effective step size is $\\tilde{\\gamma}_k \\approx 0.0001$ and the KL weight $\\beta=0.9$. This suggests the model is moving very little from the pre-trained state. The claimed high validity (81%) is likely just the original model's high validity, not a product of the algorithm.\n* As discussed in Appendix H, $\\alpha$ and $\\gamma_k$ are entangled, jointly determining the effective step size $\\tilde{\\gamma}_k$. This implies the method is likely quite sensitive to hyperparameter tuning, but this important aspect is not analyzed.\n* The computational cost appears very high. The algorithm requires $\\approx 2 \\times K \\times N$ full model fine-tuning runs (e.g., $2 \\times 8 \\times 4 = 64$ in the L-FE experiment). This may be impractical for large-scale problems, and the paper makes no analysis of this cost or scalability against other baselines.\n* A potential theory-practice gap exists. The convergence guarantees (Thm 5.2) rely on assumptions (E.1, E.2) about the solver, but there is no proof or justification that the actual solver used (Adjoint Matching, Alg. 3) satisfies these assumptions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NsxOeU1jtU", "reviewer": "Reviewer_e76w", "summary": "This paper proposes Flow Expander (FE), a general framework for expanding the support of pre-trained flow models to cover the valid design space more uniformly. The method uses verifiers—functions that check the validity of samples (e.g., physical or chemical constraints)—to guide exploration and improve the generative model’s coverage beyond its initially narrow region.", "questions": "See the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Introduces a method to expand flow models using strong or weak verifiers to expand diversity.\n- The scheme allows practical, gradient-based fine-tuning compatible with existing flow and diffusion models.", "weaknesses": "- What is $\\lambda_t$ and $\\gamma_t$ in Eq. 8 and 9? Are these weightings and discount factors? And how are they chosen?\n- How do you parametrize the verifier function $v(x)$, is it modeled like a Gumbel-Softmax? Also, I wonder if it's strict to have $v(x)$ to be bounded, is the method extendable if one has an unbounded $v(x)$?\n- I am not sure about the downstream applications of this method in drug discovery, as currently it's only restricted to the bounded verifier objective to check validity or bonds. Usually, this can also be done via BO in latent space by embedding priors with GP or just generally with rejection sampling in latent space to get valid molecules only, similar to an explore and exploit-based method. If the $v(x)$ can be extended to the unbounded domain such that one can use molecular properties to guide it, then it would be a good contribution.\n- Does the model incorporate whether the verifier is noisy?\n- Could FE be viewed as iteratively reshaping an implicit energy landscape defined by verifier penalties?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)