Devil is in the Details: Density Guidance for Detail-Aware Generation with Flow Models
Authors: Rafal Karczewski
Markus Heinonen
Vikas K Garg
Venue: ICML 2025 poster (2025)
Keywords: Diffusion models
likelihood
flow matching
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: We analyze the connection between log-density and image detail in flow models and provide tools for detail-aware sampling.

Abstract: Diffusion models have emerged as a powerful class of generative models, capable of producing high-quality images by mapping noise to a data distribution. However, recent findings suggest that image likelihood does not align with perceptual quality: high-likelihood samples tend to be smooth, while lower-likelihood ones are more detailed. Controlling sample density is thus crucial for balancing realism and detail. In this paper, we analyze an existing technique, Prior Guidance, which scales the latent code to influence image detail. We introduce score alignment, a condition that explains why this method works and show that it can be tractably checked for any continuous normalizing flow model. We then propose Density Guidance, a principled modification of the generative ODE that enables exact log-density control during sampling. Finally, we extend Density Guidance to stochastic sampling, ensuring precise log-density control while allowing controlled variation in structure or fine details. Our experiments demonstrate that these techniques provide fine-grained control over image detail without compromising sample quality. Code is available at https://github.com/Aalto-QuML/density-guidance.

Devil is in the Details: Density Guidance for Detail-Aware Generation with Flow Models
Rafal Karczewski!
Markus Heinonen
Vikas Garg 12
Abstract
higher lkelihood
likelihood
Diffusion models have emerged a powerful class of generative models, capable of produc- ing high-quality images by mapping noise to data distribution. However, recent findings sug- gest that image likelihood does not align with perceptual quality: high- lighllikelihood samples tend to be smooth, while loker-likelhoodd ones are more detailed. Controlling sample density thus crucial for balancing realism and detail. In this paper, we analyze an existing technique, Prior Guidance, which scales the latent code to influ- ence image detail. We introduce score alignment, a condition that explains why this method works and show that can be tractably checked for any continuous normalizing flow model. We then propose Density Guidance, principled modi- fication of the generative ODE that enables ex- act log-density control during sampling. Finally, we extend Density Guidance to stochastic sam- pling, ensuring precise log-density control while allowing controlled variation in structure or fine details. Our experiments demonstrate that these techniques provide fine-grained control over im- age detail without compromising sample quality. Code is available at https: / //gi //ggtttuu thub. com/ Aa to weoo-uuiddeensttt--eee ginpynguiddd.dd...ooc dance.
Figure 1. Density guidance controls the amount of detail. Im- ages sampled from the condition and latent code with differ- ent strengths of guidance. Top: StableDiffusion v2.1 (Rombach et al., 2022). Bottom: EDM2 (Karras al., 2024b).
Remarkably, the density po correlates negatively with the amount of detail: within the typical samples (Nalisnick et al., 2019) Po higher-density images tend to lack detail and be smooth, while lower-density images tend be richly textured and detailed (Sehwag et al., 2022) (See Fig. 1). Outside the typical samples, extremely low density leads to broken images (Karras et al., 2024a), while extremely high density strips detail to the point of resembling sketch drawings or blurs (Karczewski et al., 2025).
1. Introduction
Diffusion models are a family generative models that learn to map noise data distribution Po, which allows re- alistic image sampling (Ho et al., 2020; Song et al., 202 2021b;a; Vahdat et al., 2021). In the quest towards high-fieeliyy sam- pling is natural to ask whether perceptual quality of images aligns with their likelihood Po( (Karczewski et al.. 2025y?
Somewhat surprisingly the common sampling strategies in flow models do not optimise for sample density (Karras et al., 2022). Recently, Karczewski et al. (2025) proposed an approach towards controlling the sample density by biasing the sampling towards the extremely high likelihood regions of p(æy xt). and demonstrated that these correspond to unrealistic images. Their work is limited in three ways.
'Dparrment of Computer Science, Aalto University. Fin- land Yai Ltd. Correspondence Rafai Karczewski fi.
<rafal
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Density guidance
The approach is only derived for SDE models with linear drift. The exact procedure is not tractable so the authors resort to approximations in practice. Finally, it only allows targeting the highest possible likelihoods, which do not produce realistic images. This highlights the need for a more general approach that allows fine-grained control over sample density while preserving realism.
2.1. Continuous normalizing flows
Continuous normalizing flows (CNFs) (Chen et al., 2018) are probabilistic models specified by a tractable prior dis- tribution pr at terminal time T and ordinary differential equation (ODE)
dx. = )a:
In this paper, we build upon prior observations that scaling the latent code affects image detail (Song al., 2021b). We refer to this method as Prior Guidance and we provide theoretical explanation for this phenomenon by introducing score alignment, a condition under which Prior Guidance provably increases or decreases log-density We show that this condition often holds in practice.
which samples by integrating from ær pr at t T to = 0 by following the time-varying vector field u, R + RD with solution
T == = 2T [ u, (xx
(2)
The flow family encompasses many popular generative frameworks, including diffussinn/scre-basee models (Song et al., 2021b), flow matching (Lipman et al., 2023; Tong et al., 2024), rectified flows (Liu et al., 2023), stochastic in- terpolants (Albergo & Vanden-Eijnden 2023), consistency models (Song et al., 2023), and the denoising probabilistic models at continuous limit (Kingma al., 2021): the vector field un is the denoiser.
Beyond this analysis, we introduce Density Guidance, novel procedure that allows explicit control over the log- density of generated samples. Assuming knowledge of the score function, we derive an alternative ODE that guarantees the log-density of the trajectory evolves exactly as specified. Empirically, we show that this method achieves similar results to Prior Guidance.
In CNF we can evaluate the log-likelihood of sample moving according to Eq. I with (Chen et al., 2018):
Finally, we extend Density Guidance to incorporate stochas- tic sampling. This enables precise control over the log- density of generated samples even when randomness is in- troduced. By injecting noise at different stages of the genera- tion process, we can selectively influence variations in high- level structure (e.g., shape and composition) or fine-grained details. Our experiments demonstrate that this stochastic extension allows for enhanced diversity while preserving control over the desired level of detail.
p. = div u,(x;))
(3)
dt
where Pr is the marginal density of a process defined by Eq. Karezewski et al. (2025); Skreta al. (2025) gener- alized this formula to enable tracking of the marginal pa for sample following different direction dæ; (. )dt as d log p((:)) div (xx,) dt
In summary, in this paper we
(4)
+ V (oo) (z(())))
us u((xz))
introduce Score Alignment, condition that explains how latent code scaling affects image detail and can be tractably checked for any CNF model - section 3;
At ú, uc this reduces back to Eq. 3. See Appendix B for detailed derivations.
2.2. Diffusion models
derive a modification of the generative ODE that en- ables exact log-density control during sampling Den- sity Guidance- - section 4:
A notable case of flow models are diffusion models given by forward process Pt (x (20) = (a,,0, of ID), equiva- lently. by stochastic differential equation (SDE)
extend Density Guidance to stochastic sampling, retain- ing exact log-density control while allowing controlled variation in structure or details- section 5.
dx. f(()x;dt + +())d
(5)
with drift [(1)= lo diffusion g² = 20 de
log #t ds
and Wiener process W, A CNF with drift
2. Background
1 u (P-ODE ) = f()) 799 5² (w))
(6)
Let x a
E RDD We assume spatial gradient V à E RD, divergence div = Ea 8zg E IR, (ag ..... Hep
score
a
shares marginals P: with Eq. 5 when pr are shared (Song et al., 2021b). This Probability- ODE (PF-ODE) an efficient, deterministic, sampler in diffusion models.
and Laplacian continuous time E (0, T] between data Po and noise pr.
E.
2² 0uy E R operators. We assume
2

---

Density guidance
t
isso
v log Pr (r(y))
8ko Üxp
v (og
xy
(x()))
SA a S 9000
Figure 2. Prior Guidance and Score Alignment (SA). Prior guidance works by moving ær (noise) in the direction of V log pr and decoding to 2u (image). The purpose of this is increase or decrease log (o(o)), which is inversely related to the level of detail in Zb. SA a condition that ensures prior guidance is effective by requiring the alignment of score vectors across time steps. Red arrows represent tangents the curves: V log pr(er) is the tangent the left curve ær, and its push-forward via dea is the tangent dær the decoded curve at æo. SA states that the transformed tangent vector must align with V log (wo) such that the angle S 90" (non-negative dot product).
2.3. Stochastic sampling and likelihood
as A log (x))) = div V lo we use the Hutchinson's ss trick (Hutchinson, 1989; Grathwohl et al., 2019) using a single Rademacher test vector.
Eijkelboom et al. (2024) pointed out that any CNF can be cast as an SDE model via the score function V log (m:):
dx; -< ( dt +t) (7)
3. Scaling the latent code - when and why?
In this section evaluate whether the simple latent rescal- ing approach of Song et al. (2021 is sufficient to control for sample density, and demonstrate its shortcomings.
where W is the Wiener process going backward in time and the process defined by Eq. 7 shares marginals with Eq. 1 for any choice of the variance term &p as long as they share pr- Karczewski et al. (2025) demonstrated that for SDE models one can also track the evolution of marginal
Song et al. (2021b) observed that scaling the latent noise Ey (O, I) down with the amount of detail in the decoded images æ PF-ODE (ettoDD((xx))) (BxT). while preserving the image semantics (See Fig. 2). Inter- estingly, Karczewski et al. (2025) recently showed that the log-density log Po (xx) assigned by a diffusion model corre- lates with the amount of detail in the generated image. Most diffusion models use a Gaussian noise pr = -N ID), whose score V log pT (x()) = -TT simply reduces the la- tent norm towards zero. Thus (down)scaling equivalent to maximizing log pr.
monotonically decreases
d log (x.)) = dt + (tt) (o:)) aw, (8)
where
F(t,x)) div (x) 1
(t)
T is
These two observations suggest a simple hypothesis:
2.4. Neural network approximations
In all our experiments, we assume access to pre-trained neural networks to approximate u, and V log Pt. When du; estimating Jacobian- vector products such as dw v, we lever- age automatic differentiation compute these efficiently with single network pass. To estimate divergence, such
Prior guidance: To increase (decrease) log Po it suffices move ær in the positive (negative) direction of V pr and then decode.
ue(x)) IIn Eijkelboom et al. (2024), the drift V log pa (x:). This is caused by different conventions. We follow the diffusion literature (Song et al., 2021b), where time flows backward, i.e. dt < O and the Wiener process runs in reverse during sampling. In Fijkelhoom al. (2024) sampling from to and dt
We are interested in studying whether (steepest) log pr in- crease in latent æт leads to monotonic increase in log Po of the decoding (x(x)) Eq. 2. To formalise this notion, we assume a latent curve c : [0, 1] + R² at t = T, whose tangent is given by the score (((s) logpr(c(s)).
3

---

Density guidance
monotonic curve decoding has
Correlation with PNG(x))
0.75
d
log Po [ 2 0.
Vs,
(10)
ds
0.50
which is equivalent to Score Alignment (Appendix C):
0.25
dco
SA :
v log Po (2o)
v log pr lr 2 0,
(11)
0.00
dæt
P(())
decoding score
push--forward score E
-0.25
(x.
for all 2T IR.,, where (((xT) = dm, log Pr (xT). Jæy Appendix C.2 we show that Eq. 11 always holds when u, is linear in æ, in trivial diffusion models. We also show that it does not hold in general by providing counterexample. See Fig. 2 for visualisation.
0
T
Figure 4. Negative lo correlates well with image com- pression size, while the recently proposed intrinsic dimensionality measure FLIPD (Kamkari et al., 2024) correlates weakly. Exper- iment performed latent diffusion model EDM2 trained on ImageNet 512 x 512 resolution (Karras et al., 2024b).
To evaluate the SA Eq. 11 we need to solve for vo. We use sensitivity equations (i.c. forward differentiation)
d
UA (Ir+) (are) 8ie
dt.
(12)
the amount of detail (or information) in the image mea- sured by the size of PNG compression of the decoded image PNG( In Fig. 4 we show on high resolution latent dif- fusion model EDM2 (Karras et al., 2024b) that - log Pt (xx, correlates with PNG( more strongly, reaching a maxi- mum of 84%, compared 29% achieved by FLIPD. Fur- thermore, we observed that the correlation of log pr (3x, with PNG((z)) is the strongest not for t 0, but rather = 0.6 corresponding to log SNR(t) = log = 1. This suggests that for detail manipulation with Prior Guidance, verification of the SA condition Eq. 11 should be done up to this value of ( rather than / = 0.
Vt
with by = V log prg((())) to describe the evolution, and specifically, to solve vo (wr) (Baydin et al., 2018).
EDM2 ImageNet 512-512
UNet CIFAR-10 32:32
V. pog,, Vv.
100
80-
10000
40-
20-
100000
What if score is unknown? To verify SA, one needs to solve Eq. 12 and estimate v log (o The latter is straightforward for diffusion models, but not in the general case of Eq. 1. Remarkably, it is also possible to evalu- ate v. V log Pu (xz) for any flow model without estimating the score itself. Concretely, in Appendix C.1. show that for W/ == Po
Figure Nearly all æy satisfy the positive score alignment of Eq. 11 across models and datasets.
Empirical demonstration To empirically verify whether SA holds, one sample large batch of æT Pr and solve Eq. 12 from t = T to = 0² and check whether oo V log po (2r) 2 0. We demonstrate this on two models, a VP-SDE model trained on CIFAR-10 10 (Karczewski et al., 2025), and EDM2, a conditional latent diffusion trained on ImageNet512 (Karras et al., 2024b). We find that for CIFAR 97% of the latent codes satisfy the equation and 100% for EDM2 (Fig. 3). This shows that, in most cases, scaling the latent code ær impacts V log (o(z)) monotoni- cally, and thus explains the visual effect of low-level feature manipulation (Fig. 1). See Appendix M for more samples.
d:w div( du (zx))t). Therefore, in absence of the score function, one can es- timate Wo by augmenting Eq. 12 with We initialized at wr = | log (x
xx V: Wi div
ue (2ee) Oue (Cee)
d
dt.
(13)
( cha
In Fig. 6 we present the SA verification algorithm. To em- pirically validate Eq. 13, we used a VP-SDE CIFAR-10 diffusion model (Karczewski et al., 2025), sampled 256 latent codes and solved Eq. 13 from / = T to / corre- sponding to log SNR(t) = 1. This is score-based based model and thus we can compare the ground truth w' V log P. (a,) with the estimated wi We found that their correlation was at 98.8%. See Fig. 5.
Log-density vs FLIPD Kamkari et al. (2024) recently proposed FLIPD- a method for measuring local intrin- sic dimension and argued that correlates strongly with
In practice we solve until f e for small 0.

---

Density guidance
9000
Algorithm Score Alignment verification
8000
0.988
1: input: Flow ut, latent ær E R., step size > 0
2: initialize vr = V (oggr((rr)) = T. WT = Ur
7000
3: while t > 0 do
6000
4: dæ + ur(w:)) dv JVP Rt. Ut)
5000
6: E + Uniform -1, 1]
Rademacher variables
4000
7:
dw t JVPPU æz, E)
Hutchinson's 's trick
4000
5000
6000
7000
8000
9000
8:
It t It dt - dæ
(x,))
9:
Us t D; dt du
10: Wi t W. dt dw
Figure 5. The augmented sensitivity equations of Eq. 13 accu- rately tracks the score alignment (SA).
11: 1 dl
12: end while
13: if V log (a(o)) known then
14: return V log po(o)) on
4. Density Guided Sampling
15: else
16: return wp
In section 3 we discussed ways to determine whether scaling the latent code corresponds changing log Pu (2z)). In par- ticular, showed that the necessary SA condition Eq. 11 does not always hold. Furthermore, the prior guidance does not allow choosing the desired sample log-density.
17: end if
Figure 6. Score A Alignment Verification. When the score V log po is known, Eq. 12 applies, and the highlighted steps (correspond- ing to Eq. 13) can be omitted. We provide JAX implementation in Listing 1.
We now present an approach for sampling zo with explicit control of log Po (2o). Suppose that we require an instanta- neous density changes over time,
Eq. 16 into
d log (x)) (xx.) dt
constraint:
RR
(14)
1 - v Vlog loggo(<x));
ü ūr(w))) [(1)xx
(5((zz))
for predetermined by To achieve this, choose ODE dæ; = dtd such that its density change from Eq. 4 satisfies
(17)
which can readily be used for sampling for any 6.
(xx, = v logp:(x;))*) (zz()) -ur(2) (2xe) div u. (xx). (15)
The question is: How to choose by? Notably, we cannot simply push b, to be arbitrarily high or low, since it will fall off the diffusion manifold, leading to nonsense decodings.
Whenever V log Po + 0, Eq. 15 has multiple solutions of u for any by- We choose ū that is closest to which uniquely gives (See Appendix D)
4.1. Explicit quantile matching
Suppose that we want æpp to have a pre-defined value of log-density log (xo) = e R. which is equivalent
(z))) = (x) + div (x) + (x)) V log. (16)
V log Pe (2)
[0 by (z logpr(#T)
score hias (x)
(18)
Density guidance: Eq. 16 steers the sample away from the original trajectory towards desired likeli- hood.
If this holds for b, the Eq. 16 will generate a sample æn with the log-density However, not all choices of b equally good. In practice, and V log Pt approximated with neural networks, and their predictions are only accurate when æ; is in the typical region of (Nalisnick et al., 2019).
When b, = div ue we reduce to the canonical sampler ú, = u, Since using Eq. 16 requires knowing the score function, we assume from this point on that u, is given by the PF-ODE of diffusion Eq. 6, which transformed by
Suppose that the target value e the q 'tth quantile of log Po- where 9 E [o, 1). A simple strategy is to choose b, such that the sample æ; remains on the same quantile g over all times
5

---

Density guidance
we want (o(z)) evolve. However, the challenge is to determine what are the reasonable values of by so that logPt does not deviate from what is typical. We tackle this problem by analyzing the stochastic view of CNFs. Specifically, for Øt ~ Pt. Eq. 7 says that when evaluating Lt-dt. we can add random noise and stay within the typical region pt-dt as long as we correct for it by subtracting the score from the drift. Furthermore, Eq. 8 says how log pt changes under this stochastic evolution.
7500
d (0.99)
6, (0.01)
5000
Typical region
log pu (xc)
2500
2500
5000
0.0
0.2
0.4
0.6
0.8
1.0
Concretely, the average change in log-density, when adding noise of strength cptt) is given by
Figure 7. Quantiles and typical values of log p. (xx) for a diffusion model trained CIFARIO.
1 E [d p((2))] -diiv (div un(x)) ( ((t)(((()))
dt,
(20)
"fax" "whistle"
where l((x)= = A log (æ) + |/ lo (x) E R. Interest- ingly, in Appendix F, we empirically (and theoretically in simplified cases) show that in diffusion models gih(mss)
losc elsst distributiin
long distribution
v2D ap-
proximately follows NO, 1) when 2 ~ Pt and dimension high. A reasonable choice is then
V2D 1 - 2
b((x)) div u. ux(
$ (q).
(21)
of
20000 40000
0
20000
40000
log po(sa))
where & is the cumulative distribution function W 1) and q is the desired quantile. We found that choosing yp match the diffusion strength in Eq. 5, i.e. y g works well in practice. Thus, in our experiments we use
Figure& 8. Different classes have different likelihoods. Left: Dis- tributions of log likelihoods of 16 randomly selected classes from ImageNet. Right: Distributions of lggliiklihoods for "fox" and "whistle" differ significantly.
V2D oi
b8 (x) = div (x) - -9- g
+ (q).
(22)
and pr. Let (g) denote the g-th quantile of log Then
After plugging this definition of to Eq. 16, we get
log (mt) (ogpp((ar) 1/¹ br(x)) = (1))), (19)
1 po-ODE (xx) f(t)x, (t) (2) logp((z;), (23)
u
which is satisfied for (xx) == 4 (g). The quantile func- tion (2) can be estimated by sampling K independent samples or pr, estimating log (ax)) with Eq. 3 and finding empirical quantiles for target values of q. We visual- ize oi (0.99) and de (0.01) in Fig. 7 estimated for a Variance- Preserving (VP) SDE diffusion model with linear -NR-SNR schedule trained on CIFAR1O. We experimentally verify the accuracy of explicit quantile matching Appendix
which is equivalent to simply rescaling the score by
2DD (g)
Tx( = | (ogp(():))
(24)
We call Eq. 23 Rensity-Guided Sampling (DGS). Impor- tantly, DGS comes at extra cost since the score is eval- uated at each sampling step anyway. Note that, as shown in Fig. 4, the correlation of log Pe (xx)) with image detail is the strongest for t* R log SNR -1) and thus in our ex- periments we only use guidance the [T, t*] interval. In Fig. we show samples generated with DGS with different values of q. Interestingly, the samples are perceptually very similar to those from Prior Guidance (Appendix M). See Appendix J for more samples and quantitative results.
4.2. Implicit quantile matching
considerable drawback of explicit quantile matching is the need estimate Φ; This becomes especially problem- atic for conditional generation, where the distribution of log can differ significantly for different classes (Fig. 8). For applications such as text-to-iman to--mage, this would require estimating the distribution of log Pt for every possible text prompt, which is not feasible.
Conditional generation Whenever a conditional score function V log Pr (=/cond)) is available, where cond can be any condition (class, text, etc.), one need only replace the score function with the conditional one in Eq. 23.
The Eq. 15 gives a recipe for altering the flow based on how
6

---

Density guidance
07
0
Figure Density Guidance controls the amount of detail. Samples generated with Eq. 23 using the EDM2 model (Karras al., 2024b).
5. Stochastic density guidance
and
In previous sections we discussed two methods for control- ling log (o())) during ODE sampling form Eq. 1. How- ever, has been reported that adding stochasticity during sampling can improve sample quality (Song al., 2021b; Karras al., 2022). Neither of the previously discussed methods supports stochastic sampling. Recently Karczewski et al. (2025) lifted the first roadblock towards this by show- ing how to evaluate log)o(#o) ) for an SDE. We now ask: Is it possible to also control log (o(p)) during stochastic sampling?
V log (r
lo (x)
Pi(x) = ID
(28)
V log (xx)
is the "score-orthogonal" projection, which ensures that the log px (ax.) changes deterministically even though is stochastic. In Appendix G we provide the formula Eq. 120 the SDE drfit that exactly achieves log Pr (æ;) = by (dz for any choice of b, and u4, which we omit here for presentation clarity. In Appendix we experimentally demonstrate that stochastic sampling, provided the number of sampling steps is large enough.
obtain exact likelihoods even for
Recall stochastic CNF sampler with noise strength yo:
dx: (u:(z=)- 1 (() +t+ (7)
The Eq. 25 allows increasing variation in the samples by injecting noise to DGS Eq. 23 whilst maintaining the desired evolution of the log-density Furthermore, since is known that diffusion models first generate high-level features and then the details (Ho al., 2020; Deja al., 2022; Wang & Vastola, 2023), DGS can be combined with stochasticity by introducing noise at specific stages of the generation process, allowing for controlled variation in either high- or low-level features while preserving the desired level of detail. We demonstrate this approach Fig, 10 and provide more samples in Appendix K.
In Appendix G, we show that, similarly to density guidance Eq. 23, it can be altered to enforce the desired evolution of log-density over time. Specifically, suppose that u, = PF-ODE and that we require dlogg pe(a.) (x.)) for b u,
defined in Eq. 22. Then, the stochastic process
dæ: u SDE (ax) )dl + ((.)).
(25)
approximately satisfies dt
pl(a
bx()))), where
6. Related Work
u
DDDG-SD EDE
(x)) u.
ARG-ODE/A
(x)
(26)
Schwag et al. (2022) proposed a method for generating sam- ples from low-density regions of diffusion models. However, due to the intractability of likelihood in diffusion models, their approach relies on approximations. Subsequent work
(z) (x)
+ 5 (1) V log (x)
(27)
correction for adited stochusticity

---

Density guidance
Deterministic guidance
Adding noise early (high-level variation)
Adcling noise late (low-level variation)
High detail
Low detail
Figure 10. Stochastic density guidance increases variation in generated samples whilst maintaining the desired level of detail. Samples generated with Eq. 25. Adding stochasticity carly in the sampling process changes high-level features, whereas adding noise later, only affects lower-level detail.
by Karczewski al. (2025) demonstrated that likelihood is, in fact, tractable even in stochastic diffusion models, chal- lenging the need for such approximations. Building this, our proposed methods provide explicit likelihood control for both deterministic sampling-vv -via Prior Guidance (section 3) and Density Guidance (section 4)- and stochastic sampling through Stochastic Density Guidance (section 5).
Yu al. (2023) introduced Riemannian Langevin Dynam- ics, SDE with non-diagonal diffusion matrix, similar in structure our Stochastic Density Guidance (section 5), However, a key distinction is that our diffusion matrix is projection onto the orthogonal complement of the sub- space spanned by the score function. As a result, is not positive definite and cannot serve as a Riemannian metric tensor, making our approach fundamentally different in its mathematical formulation and behavior.
Song al. (2021b) observed that scaling the latent code alters the amount of detail in deterministically generated images. While this phenomenon has been widely acknowl- edged, we provide rigorous analysis (section 3) and prove that it is a direct consequence of Score Alignment Eq. 11, which guarantees that scaling leads to monotonic change in likelihood of the generated image, Poggo Further- more, we introduce tractable numerical tools (Fig. 6) that can verify whether any given CNF model (not necessarily score-based) exhibits this behavior.
Recently, Kamkari al. (2024) proposed method for mea- suring local intrinsic dimension, which, in the case of im- ages, corresponds to the amount of detail present. However, we show that negative log is a more effective measure of image detail and provide empirical comparisons in Fig. 4. Moreover, while Kamkari et (2024) focus on measuring image detail, our methods enable direct manipulation of allowing for finer control over generative model outputs.
Karras et al. (2024a) proposed auto gutid-g as method for improving sample quality by targeting high-density re- gions. However, Karezewski et al. (2025) found that the highest-density regions in diffusion models contain cartoon- like or blurry images, which raises concerns about the ef- fectiveness of purely maximizing likelihood. In contrast, we introduce multiple cost-free methods for explicitly con- trolling the likelihood of generated samples. Additionally, while Karras et al. (2024a) observed that scaling the score function leads to oversimplified images, we demonstrate that DGS Eq. 23 enables effective control over image de- tail- both increasing and decreasing whhn the scaling is adapted both temporally and spatially Eq. 24.
7. Conclusion
In this paper, we introduced methods for controlling sample density in flow models, enabling manipulation of image detail through likelibood-guided sampling. We provided a theoretical explanation of latent code scaling by introducing score alignment, a condition that can be tractably checked for any CNF model. Building on this, we derived Density Guidance, principled modification of the generative ODE that allows for exact log-density control during sampling. Finally, we extended this approach to stochastic sampling, demonstrating that it retains precise detail control while allowing controlled variation in image structure and detail.
8

---

Density guidance
Our findings deepen the understanding of likelihood in flow models and provide practical tools for better sample control.
Finlay, C., Gerolin, A., Oberman, A., and Pooladian, A.-A. Learning normalizing flows from Sattrop-KKattoyyy potentials. arXiv preprint vrXiv:2006.06033, 2020.
Impact Statement
Grathwohl, W., Chen, R. T., Bettencourt, J., Sutskever, I., and Duvenaud, D. FFJORD: Free-form continuous dy- namics for scalable reversible generative models. In ICLR, 2019.
This paper presents work that advances the understanding and controllability of sample density diffusion-based gen- erative models. By introducing techniques for precise log- density control, our work contributes to improved inter- pretability and fine-grained control over image generation. While these advancements could enhance applications in cre- ative and scientific domains, they also raise considerations around synthetic media generation and potential misuse. However, our contributions primarily aim improving the- oretical understanding and control in generative modeling, without introducing new ethical risks beyond those already associated with generative AI.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. In NeurlPS, 2020.
Hutchinson, M. F. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Com- munications in
and Computation,
1989.
Itô, K. On formula concerning stochastic differentials. Nagoya Mathematical Journal, 3:55-65, 1951.
Acknowledgements
Kamkari, H., Ross, B. L., Hosseinzadeh, R., Cresswell, J., and Loaiza- Loaiza-Ganem, G. A geometric view data complexity: Efficient local intrinsic dimension estimation with diffusion models. In NeurIPS, 2024.
This work was supported by the Finnish Center for Artificial Intelligence (FCAI) under Flagship R5 (award 1501 1052). RK thanks Paulina Karezewska for her help with preparing figures. VG acknowledges the support from Saab-WASP (grant 41 1025), Academy of Finland (grant 342077), and the Jane and Aatos Erkko Foundation (grant 7001703).
Karczewski, R., Heinonen, M., and Garg, V. Diffusion models as cartoonists! The curious case of high density regions. In ICLR, 2025.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022.
References
Albergo, M. S. and Vanden-Eijnden, E. Building normaliz- ing flows with stochastic interpolants. In ICLR, 2023.
Karras, T., Aittala, M., Kynkäänniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding diffusion model with bad version of itself. In NeurIPS, 2024a.
Baydin, A., Pearlmutter, B., Radul, A., and Siskind, J. Au- tomatic differentiation in machine learning: A survey. JMLR, 2018
Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., and Laine, S. Analyzing and improving the training dynamics of diffusion models. In CVPR, 2024b.
Chen, R., Rubanova, Y., Bettencourt, J., and Duvenaud, D. Neural ordinary differential equations. In NeurlPS, 2018.
Kingma, D., Salimans, T., Poole, B., and Ho, J. Variational diffusion models. In NeurIPS, 2021.
de Jong, P. A central limit theorem for generalized quadratic forms. Probability Theory and Related Fields, 75(2):261- 277, 1987.
Lipman, Y., Chen, R., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In ICLR, 2023.
Deja, K., Kuzina, A., Trzcinski, T., and Tomezak, On ana- lyzing generative and denoising capabilities of diffusion- based deep generative models. In NeurlPS, 2022.
Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.
Mittal, A., Soundararajan, R., and Bovik, A. C. Making "completely blind" image quality analyzer. IEEE Signal processing letters, 2012.
Dockhorn, T., Vahdat, A., and Kreis, K. Score-based genera- tive modeling with crimie Langevin diffusion. In ICLR, 2022.
Nalisnick, E., Matsukawa, A., Teh, Y. W., and Lakshmi- narayanan, B. Detecting out-of distribution inputs to deep generative models using typicality. In NeurIPS Bayesian Deep Learning workshop, 2019.
Eijkelboom, F., Bartosh, G., Naesseth, C. A., Welling, M., and van de Meent, J. Variational flow matching for graph generation. In NeurlPS, 2024.

---

Density guidance
Rombach, R., Blattmann, A., Lorenz, D., Esser, and
Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
Sami, M., Hasan, M. M., Dawson, J., and Nasrabadi, N. Hf-diff: High-frequency perceptual loss and distribu- tion matching for one-step diffusion-based image super- resolution. arXiv, 2024.
Sehwag, V., Hazirbas, C., Gordo, A., Ozgenel, F., and Can- ton, C. Generating high fidelity data from low-density regions using diffusion models. In CVPR, 2022.
Skreta, M., Atanackovic, L.,, Bose, J., Tong, A., and Neklyu- dov, The superposition of diffusion models using the itô density estimator. In ICLR, 2025.
Song. J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In ICLR, 2021a.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score- baree-baeddde generative modeling through stochastic differential equations. In ICLR, 2021b.
Song, Y., Dhariwal, P. Chen, M., and Sutskever, Consis- tency models. In ICML, 2023.
Tong, A., FATRAS, K., Malkin, N., Huguet, G., Zhang, Y., Rector-Brooks, J., Wolf, G., and Bengio, Improv- ing and generalizing flow-based generative models with minibatch optimal transport. TMLR, 2024.
Vahdat, A., Kreis, K., and Kautz, J. Score-based generative modeling in latent space. NeurIPS, 2021.
Wang, B. and Vastola, J. Diffusion models generate images like painters: an analytical theory of outline first, details later. arXiv preprint arXiv:2303.02249. 2023.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. Image quality assessment: from eтror visibility to struc- tural similarity. IEEE transactions on image processing, 2004.
Xu, C., Cheng, X., and Xie, Y. Normalizing flow neural networks by JKO scheme. In NeuriPS, 2024.
Yu. H., Hartmann, M., Williams, B., and Klami, A. Scalable stochastic gradient Riemannian Langevin dynamics in non-diagonal metrics. TMLR, 2023.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang. O. The unreasonable effectiivnness of deep features as perceptual metric. In CVPR, 2018.
Zhi, W. Lai, T. Ott, I.., Bonilla, F. V., and Ramos, Learning ODEs via diffeomorphisms for fast and robust integration, In /CML. 2022.
10

---

Density guidance
A. Auxiliary results
A.1. Constrained optimization
In multiple sections, we will be solving constrained optimization problems, which can be written in the following way. Suppose v E D. of O, any y RD and a e R. The problem we will encounter is
min wER 2 æ yl1
(29)
s.t. x= a.
We solve this by introducing the Lagrangian [(x, A) x yl? + X(xTvx a) for R. By setting
dC On 0, get
y + \v = 00=
Av.
(30)
To find À we substitute for æ in the constraint and find
yd = x y" Combining the two, we get that the solution is given by
(31)
2
0"
x y +
|100
v.
(32)
A.2. Divergence-gradien identity
We will make use of an identity connecting the gradient of the divergence with the divergence Jacobian vector product.
Lemma A.I. Let f R + RDD with continuous 2-nd order derivatives and E IR. Define g(æ) = div f(x)
ED af' di: (xx) and G(x) = am (x)v. Then g RD + R is a scalar function and G RD + RD is a vector function satisfying
af
Vg(z)= div G(x).
(33)
Equivalently, we write it as
0f
(V div f(x))) div dx
(x)v
(34)
Proof.
D 0g
05t Ox,
a Dæ,
as
a
as dx
Vg(x))
dzj
(z)
Σ
dr,
(z)
vn Σ
dz,"" 1/
Σ
dx,
(x)
j=1
5=1
j=1
D 0 (x) div G(z).
dzi
A.3. Optimality of projection
In Appendix G we will be interested in finding linear operator A DDD satisfying Av 0 for some v, so that the distance between A and the identity ID is minimal. The following lemma provides solution.
Lemma A.2. Ler 0 * v E R.D. The solution of
| Ipll A D
min
(35)
s.f. Av 0,
where |||| can be either the spectral or Frobenius norm, is given by the projection matrix
A DPT = P Ip
(36)
|
||20||

---

Density guidance
Proof. First that for any A RDxD satisfying Ar O, have
w (A In) w
In)
v0 - 4) |
| Ipll 2 A - Tpll = max
2
1.
(37)
w=0
| 2
|
On the other hand P - ID= = =(10) [|
which has only single non-zero eigenvalue À = 1 and thus
||ll|
P 1P-Ip|le
IDIFF
|P
Iplla
(38)
and
Pe = 0
||0||2 O.
(39)
Therefore A P satisfies Av and minimizes both ||A In) 12 and || Ipllr.
B. Derivation of CNF density evolutions
We reproduce the contimuous-time normaling flow (CNF) density evolution Chen et al. (2018),
d log pi (x.)) div u4 (x:,,,
(40)
dt
and the generalised CNF density evolution of Karczewski et al. (2025),
log (x (x:;)) + V (o:) (ūr(x:) uu (x:)),
(41)
dl
with unified derivation.
We assume a time-dependent particle E R. evolving through continuous time E R governed by an ordinary differential equation (ODE)
dæ: (x:),
(42)
dt
where (x)) R R (x:)).
R. deeendepn vector field that maps any state vector æ to its time derivative vector
CNF density evolution We are interested in the time evolution of the spatiotemporal log- likelihoddoo log (xx, for particles evolving under ODE. We write the log density total derivative wrt time by using the chain rule
llg (x(e))
dpe(xx)) dt (x.)
(43)
dt
pe(x))
(Op((z)) Øt
dæc dt
(44)
pr(w))
Ix
which describes the density evolution of a particle moving under flow. We assume that the ODE is continuous-timg time normalizing flow, where the density conserved over time. This is described by the continuity equation (Finlay et al., 2020; Xu al., 2024)
Op: + V. (p((oc)))((xe) 0,
(45)
12

---

Density guidance
which describes the change in particle density result of vector field ut transporting the particles, location æ. By substitution we obtain
d dloog (m)t)
1
(- (-V (p(( + Vp(( u1(z)))
(46)
dt
p((x))
p((x))) (- Pe( (1,)) Vpp(zx)) ux(x)) + Vpe( ue uz(z))
(47)
((()))) Pe(ae) ) (x)()))
(48)
.V.uu
(49)
div (are).
(50)
Generalised CNF density evolution Next, we derive the evolution of log- density log of a particle that is moving in some non-canonical direction, ie. a; = (x4)) o (xx)). Notably, the continuity equation remains with the u, as we are describing the particle density in the marginal induced by the original transport u₁ We obtain
d (((t)
(-v (p)()))))e((r)) Vpe(are) (..)))
(51)
dt
(xt.)
(:(a))) (x,)) (pr()))) un (x))) + (pr()t)) (n(
(52)
Pe(axs)
(r(x))) (x,) Vpc(xx)) (ūr(zz) u((x)))
(53)
pr())))
1
- (x(())) + (xx+)
(pt) (ūx((x)) - uz(z))
(54)
div (x;)) + V log po (ū( ue(xi))
(55)
which is the generalised instantaneous change of density.
C. Derivation of Score Alignment
In this section, we prove the Score Alignment condition, a necessary and sufficient condition for prior guidance to be effective controlling log Formally, assume a latent curve e [o, 1] + RDD following the score at / = T, (ss) = v Po (x)(((s)) In Gaussian prior pr the curve becomes a line of scaled latents Ty. The log Po is monotonic on the decoded curve when
d ds
0,
Vs e (0,1).
(56)
The chain rule gives the derivative
d
log (ppo(()(((s)) = (log)(( 2a(((s)) ds Ozo
(57)
ds
(logm((o((((s)) (c(s))c(s)
(58)
dxt
(log)( T day dzr
(c(s) V Plogsr((ss))
(59)
Therefore, for Eq. 56 to hold for curve passing through some arbitrary Ør E RR D at some point & E (0, 1) it must hold
Ozo (xT) log 0,
V log (=o
(60)
OaT
where in Eq. 11 we omit the ør in the parentheses for brevity.
13

---

Density guidance
C.I. Score alignment time evolution
In this subsection, we derive Eq. 13, i.c. how the SA condition can be checked without knowing V log for fort- < T. Let be latent curve following the score and passing through æy, i... (-e,E) + R., c'((s) V Vlogpn (lggpr(((s)) and c(0) ZT. Define
(t, s) = (x for E [0, SE (-E,E)
(61)
The SA condition at ær (Eq. 60) is given by
Orb
Ozu dvr
(0.0) 0) V VlogPo(Z((ZT))
(TT
logpr
(62)
Os
Note that
35b (T.0) = ds
log (p(p((((s)) = logpr(c(s))
V log pr (xT))
(63)
0s
a00
Therefore Eq. 62 can be equivalently written as
au Os 00 0)
ab (T,0)
O Us
Ob (T.0) (oogpr()))) + ["d
8%
as +
(0.0)
(1. 0)dt,
(64)
Otos
where we applied the fundamental theorem of calculus. We arrived at seemingly more complex formula. However, can now swap the order of the derivatives (assuming that log E (IR x R) and u E C² (R x R))):
0
(
④b (
010s 8)
OsHl. (1, 8) = 0s 01 (1,8)
(65)
0y is given by Eq. 3:
Oib (1, 8) log log = div (c(x))
(66)
Øt
dt
and by the chain rule and denoting V div u, the gradient of the scalar function ++ div (z)):
2% d OsHl
(t, 8)
div u, (x:
(67)
ds
(xx (c(())
(68)
ds
dz. (s)
= (V (iv
(69)
dzo
Ox.
(Vdiv
(c( V log (c(s))
(70)
Oao
After setting 0 get
S Osôt
Ow: (xT) V (rg(r(())
(t, (V div (x,
(71)
Dxo
=v((x))
(V div (xx n (x(T))
(72)
(34)
(Ou: (xx (z:))o)
div
(73)
Ox
where æ; (xT))) and v. v.(xT) ). After plugging into Eq. 64:
0& (0. 0) || log as
1
1
(Jun (m:))o dl dx
div
(74)
which after plugging into Eq. 62 becomes
dxy
0
V log Po (x)o) (xo))
V logpr(#T) ||V log pT (xT))
div
(z:) dt
(75)
det
and crucially, the score function for t < T does not appear in the RHS, which can be estimated purely from derivatives of u4.
14

---

Density guidance
VE SDE (a,- = 1)
VP SDE (a of 1)
COT SDE (0z
0e = 1)
2
Xy X1
Ny. = X]
Ny 1.22X
Xy 1.22%
Xy Rg
Xy 2g
1.22
1.22
log log (xg))(()) (x,)
0-
-1
-6-
1,25%
-2
-2-
Xy Xg
Xy 1.22
T
T
T
Figure 11. Prior Guidance ineffective due unsatisfied SA condition. For some latent codes: æt drz1 scaling the latent code by 1.22 increases log po(o), while for others the same scaling leads to decrease in log (x (ær = drzı). It can be seen that the blue lines cross, while the orange lines not. The same behavior was observed regardless of which SDE was used.
C.2. Score alignment holds in linear models
In this subsection, we show that the score alignment Eq. 11 holds whenever uy(x,) )) is linear in æ. Such models are for example dinear-drift diffusion models with Gaussian data distribution Po- Score alignment then an immediate consequence of Eq. 75. Specifically, when u, is linear in then tht ds does not depend and thus
iru dx
div
0
(76)
and
Oar
f Odt (og)r()))) 0. (77)
V log (xo)
V (ogpr(ar)) = V log pr (1T) +
dxr
C.3. SA does not always hold
We provide a simple example, for which the Score Alignment condition fails and thus Prior Guidance does not lead to monotonic changes in log (xo)) We study a 2-dimensional Gaussian mixture distribution with three components: po " (M,,, 0.005l2), where we randomly chosen).
[-0.3502, -0.6207]* = -0.4828, 1.0680)" and μュ = [-0.7789, 0.7565J (μ4
We found two latent codes 2 = [1.3166, -0.2252 and z2 1- 0.15504 -02115]) exhibiting inconsistent behaviour. Specifically, when æт = ZT scaling up by 1.22 decreases log po (zo), while for æt σT the same scaling increases log (=))) We visualize this in Fig. 11 with solid lines corresponding to decoding æy and the dashed lines the decodings of 1. 1.222r. This behavior was consistent regardless of which SDE was used.
D. Derivation of Density Guidance
In this section we derive Eq. 16. From Eq. 15, we see that if æ, following trajectory given by dx, = (x;x))dt, then
d dl be(x))
v lo (ū, - ux(x)) ((())) + div (xx))
(78)
dl
ü,
= (x(e)) + div ui (x:) + un (xx) (og
(79)
Whenever V (x. O. then RHS is satisfied only when (xx,) div (xx). In other words, when the score function vanishes, the infinitesimal change in log po(p)) is the same and equal to div u, (x.)) regardless of the choice of u4.
Assume now that V log + 0. For fixed (t, xx), we can treat the condition in Eq. 78 as a linear equation with w = úz (x:) being the unknown quantity we want to solve for. It is single equation with D variables (dimensionality of w), i.e., it does not have a unique solution. We can choose one that satisfies additional criteria out of all possible
15

---

Density guidance
32 sampling steps
64 sampling
128 sampling
312 sampling
1024 sampling steps
9000
Obtained log
8000
7000
6000
8000
4000
2600
size
2100
2200
2000
5000
4000
Target Po
Target log
Target log Pu
Target log
Target log
Figure 12. Explicit Quantile Matching achieves exact likelihoods when step size goes zero. For numbers of sampling steps, the correlation hetween the desired logpe Po and the obtained log Po is above 99%.
solutions. Specifically, we choose solution that diverges from the original trajectory (xx)) the least. We therefore solve the following constrained optimization problem
min w WER
(x
(80)
s.t. w V (x b((x)) + div ux(x)) + ux(x)) V lo
which is treated in Appendix A.1. The solution
w = (x; + (x()))) + div (xx))) + us (xx, log - V logp: u
ue (2re)
v log Pt (xx)
(81)
V (x;))
= (xr) + (x( + div (x())
V log);
(82)
V log (ar,)
which matches Eq. 16. E. Explicit quantile matching
To demonstrate claims made in Section 4.1, we performed density guidance with explicit quantile matching on CIFAR-10 Specifically, estimated the quantile function d, described in Section 4.1 by sampling K3 samples ør pr and solving the PF-ODE (Eq. 6) from t T to 0 in 1024 Euler steps. For all samples, estimated the marginal log-density at each step log pe (mx) with Eq. 3 and defined the quantile function di empirical quantiles of log (m..). We then define bz(x) Φt. which we estimate with moving average of finite difference estimates.
We found that the difference between the desired values of log-density and the obtained ones goes to zero as we decrease the discretization error (increase the number of sampling steps). Interestingly, for lower number of sampling steps, even though we do not obtain exact desired values of likelihood, the correlation between the desired values and the obtained ones remains above 99%, even for few 32 Euler sampling steps. This means that for all values of the number of sampling steps.
monotonic relationship between the target log Po and the amount of detail (PNG size). Please see Fig. 12. As "ground truth" log Po (z)) estimate, we used Eq. 3 for encoding Zo to Dr with the PF-ODE (Eq. 6) in 1024 Euler steps.
'We tested K [16, 64, 128, 256, 512, 1024] and found that using 128 is enough
correlation between the
desired value log-iensity the obtained one above 99%.
16

---

Density guidance
F. Asymptotic behaviour of A (()) + | lo))(2>))12
In this section, discuss observation that proved useful in determining typical values pe(m)) in diffusion models. Specifically, we observed that for some distributions po after diffusing into Pt via the forward process p(æ,(20) Ma, of ID), the following holds
h(x) of (x)) + (o
d a 1),,
(83)
v2D
D-+00
where D denotes the dimension of the distribution pt. and 4" denotes convergence in distribution.
F.1. Single data point
We begin by showing Eq. 83 for the simplest possible case, where po 8mg
In that pt NN and
V po
of
A (o))
of
Since x = a; + TrE for NN(0, ID) and our expression becomes
h(x) of (- - + 1|=)))
E,(e -
(84)
V2D
v2D
Since {e}}, are d. random variables with x² distribution, we have that E[c9] 1, Var[e?] 2, and the claim follows from the central limit theorem.
F.2. Non-isotropic Gaussian distribution
When pr is Gaussian, but with non-diagonal covariance an analogous result holds when the covariance matrix satisfies some additional conditions. We begin with useful lemma.
Lemma F.1 (Quadratic CLT (de Jong, 1987)). Suppose A= [ais] E R RDD is real symmetric matrix with eigenvalues A1. Ap. Let f=y.)= be independent variables such that Ej N 1).
If
maxjsD X2 D
lim
= =0,
(85)
Ejen X2
then
8" AE -Tr(A) N
(86)
D
Let (u, Σ) for Σ satisfying the following conditions. Denoting Σ = LL", and E = -- with A Ai...., Ap eigenvalues of Σ, we assume
lim maxjsD X? ΣjSD X7
0
(87)
Note that for = of we have E = of course holds. Then
Ip. d 77. and all the above conditions becomes limp-+oo 1 0, which of
h(x) = A pa + V (=) d N 1).
(88)
2 Log
D-+02
For N(, E), we can represent μ + Le for E LIT and e NN ID). In this case, we have
V loggr(xx = レー x) = - -
2 log) -
A loggr(x) -TrEE
17

---

Density guidance
Note that |/V log (r)))l = e Se, where E ( Since Σ- (12 and Tr(AB) Tr(BA), we have
A log pt()) Tr(>)) and |ل||| r |ー || 4 log (x We can now write
A Po + (ogg
e' Tr(E) d ب N(011
h(x=)
(89)
22 V2 (
/2|lE|F
D-200
from Lemma F.1.
F.3. Gaussian Mixture
Usually, the distributions we are interested in can be represented as pa k Σ== Ozy, where (at)) C RDD is the data set. We show that in this case Eq. 83 also holds. In that case, Pt 1/ Ex=N Nu of ID), where Me ax We will use the following identity, which holds for any p(a:):
A logp(x)) + V log logg((>)11 Ap(x)
(90)
p(z)
In the Gaussian mixture case (denoting Pk= = (u,,, IDD. we have
dr P((x)
1
Σ
Pk (x
41 x²
E
(p( - x²)
K
of
Kot
k
and
82 0(x P((x)))( Koū dr
1
Σ
a (1) x²) o? p()( = -pa(=)(ut- - -x/2
Kot
Therefore, we have
h(x) of (A + log (o
(x)
-- Ex (x(
D
(91)
V2D
v2D
where we wx((x) (a:) == (a) In Theorem we show that h(x:) $ ((0, 1). We additionally verify this hypothesis numerically.
p(=)-
Specifically, we set the number of components K 128 and sample {uu] from ( of I We then sample N 16384 samples 2y (ar))) and evaluate corresponding values of h(x) with Eq. 91. We repeat this experiment for three values of 0: [0.5, 1, 10). To test whether the distribution of h(x) approaches / 1) for larger D, we repeat this experiment for D = 2 for m = 6, 7, 12 and evaluate the p-value of a normality test on h(x) 4. We see that for D greater than x 1000, the distribution of f(x)) is close to NO, 1) as evidenced by p-value being greater than the commonly used significance threshold a 0.05. Please see Fig. 13.
E4. Image data
In this section we study Po being CIFAR-10 image data distribution and V P: being approximated with a neural network. Specifically, we uniformly sample different times E (0, T] and corresponding noisy samples æ; ~ pr()). Then, we estimate V loggt (ar) using model and A (x)) x div (oggt(()) using the Hutchinson's trick. Finally, we plot the cumulative density function of 1/ 1). If h(æ;) (o00, 1), then क (h((m())) ~ (((0 1) for all t E (0, T). Indeed, this is precisely the observed behaviour for two different choices of the forward process: ai + of 1 (VP-SDE) and a, o, 1 (CFM) confirming that this finding also holds for high dimensional image data. See Fig. 14.
& (--(((z;)) for h(xz) estimated using Eq. 83, where
G. Stochastic density guidance
In this section, we derive the stochastic density guidance method. A central tool this section is Itô's lemma (Itô, 1951), a generalization of the total derivative to stochastic processes.
"https://docs scipy. http:///www..toee.t////////////e http://ssapppstttss..ssss nornal teat.
18

---

Density guidance
0.8
0.5
1.0
0.6
10.0
0.05
p-value
0.4
0.2
0.0
10a
103
dimension
Figure 13. h(x) approaches ([(p 1) for larger D..
04 07
1.0
1.0
0.8
(M(())))))
0.6
(M(x)
0.6
0.4
0.4
0.0
0.0
0.0
0.2
0.4
0..6
0.8
1.0
0.0
0.2
11.4
0.6
0.8
1.0
Figure 14. h(w:) approximately follows (/ for various definitions of the diffusion process.
19

---

Density guidance
Lemma G.I (Itö's Lemma). Let da, pu(1, a. )dld G(I, ze dd be D-dimensional It process with jt R R + RR., G: IR TRD DDD and W the Wiener process in RDP For smooth fianction h IR X R.D -> IR, it holds that hitt, via) also an Itō process with the following dynamics
dh(t, xx) = ( ( (t, x4) + u(t,
0h
Oh dx
TT (t, + Tr (G(t, ((tt, at),
Ot
(92)
0h
(t, G(t,
dx
where h is the Hessian matrix of h worl a. In the case when G(I, ( the dynamics simplify to
[b dh(t, ( (1, m4) (tt,
Oh (1, at) + (
1 (1) mi) ds + e(t) dx
0h (t. v.) dW,
(93)
Ox
In contrast to the total derivative for deterministic processes, the forward and reverse-time dynamics are not the same for stochastic processes. We now prove the reverse-time Itô's lemma, which will be useful in our derivation as our convention that sampling happens backward time, from to
Corollary 1 (Reverse-time Itô's lemma). Let dæ; plt, x4 2t)dt + G(t, x. ).W, di O. W the Wiener process running backwards in time from t T to t 0 and μ and G are as in Lemma G.I. Then
(1, xx2)
0h (1, on) + pull, ax1)
ah
(1, v.) Tr
0t
Ox
(94)
Oh
(t, x;,²² x,),,,
with the modifications coming from time-reversal highlighted in blue.
Proof. Let T Since ds dd,, the dynamics of a can be equivalently written (Dockhorn et al., 2022):
da., (n(T rds + G(T s.v.))d JD.,)
(95)
for the standard Wiener process W and ds > O. Now let h(s, x)) = h(T - -,, x). Applying Itô's lemma to yields
dh(s, x.) =
Oh
(s, xx) -M(T- s, dx
Oh (s. (s)
1 Tr (G(T-
((T--saa s.
s, s.2.)) ds
Øs
Oh
+ dæ
(8, " TGT - x. JdW
Öh
Oh
(T - s, 1 Tr (G - s, x. s,x.) s, x.)) ds
(T s, x.) - M(T s, xx)
Ot
dx
0h (T s, ) T G(T - s, x. ddW.
+ dx
(96)
The claim follows from switching back to running backward in time <- T
Recall Eq. which describes stochastic sampling from CNF model
1 +())) (())dW
da: = (2xz) - 2
(7)
for which, we know how the log-density evolves (Eq. 8):
1 d log ((x.)) (- div div (x) (t) (ologri +
1 (()) dt v(t)) TdW. (8)
We now ask: how can modify the stochastic dynamics (Eq. 7) so that d dog pr (#t) (xx))
(97)
20

---

Density guidance
for some given by Suppose that is following
dx, = (z;))dt + ((),,,)
(98)
for some and G. To evaluate the change log-density we will use Corollary applied ii(t, 2) = p: (a)):
1 log
(xx) (x4) 'V Vlogg((x)) 1 Tr
dloog((<..) =
Øt
(99)
+ V po Gtt, x2 d.W
Since we assumed that dllogi( = (x))), the stochastic component of dloog pa must vanish, i.e. pe v p( ( G(I, x) There are many G that satisfy this condition including a trivial G 0. However, stan- dard stochastic sampling (Eq. 7) assumes isotropic noise, ie. G(t, x) <p and want match that closely as possible. An optimal solution (Lemma A.2) to this problem is the projection ((t, ((()) for:
| [[Vog (1 ( [o
(
V Flogp(()))
log):(
(a(x) = ID
(100)
Clearly (a (a) Furthermore, since P. is projection matrix, it also holds that (:(x) Pr( Now can plug this into Eq. 99 and we obtain
dlog (x:) = ( Log Pt (xx) + (xx))) log))(x;) - - (P(=
(-
(101)
0L
Using the symmetry and idempotency of Pt, and properties of the trace (linearity and Tr(AB) = Tr(BA)), we have
Tr (P)()) V (og)(()))((()) Tr (z) logp(()))
(102)
= Tr (V"
Tr log p: (V pa (a=)
(103)
(o) v (z) (z
(104)
Vlogg, 2
Alogp:(x) - R (x),, V lo
(105)
where
v lo 2 log (o)).(#)
R(V2 log (xr), V Vlogp((>))
(106)
V l: (2)
represents the Rayleigh quotient of the Hessian evaluated at V log (z). Furthermore, from Eq. 45, we have
Olog Pt (x) = div (x)) V log (ogp((2)) " (xx)
(107)
Øt
Combining these, we get
be(x;) = d prox))(
div u, (x,) + (x: (" (ū,( us (xx))
dt
(108)
2 (²) (A lo (x) R(V2 (z), V Vlog)(())))
Any (x) satisfying Eq. 108 guarantees the desired evolution of log-density, However, we wish to minimize the discrepancy from the new drift (x,) and the one from Eq. 7, which guarantees sampling from the correct distribution: u (xx)) } y² (t)V log (ax)). Therefore, we solve the constrained optimization problem:
min l ux(x)) 1 MERD
2 ( (t)
(109)
s.t. à is a solution of Eq. 108.
21

---

Density guidance
This is problem setting discussed and solved in Appendix A.1 with
a = u
y (x())) (1) (og
(110)
v = V Vlog
a = (xte + div (x+))) + ur (Vlg::)))Voopr(r)) =1) (A logp:(<)) - RRV=RR(le log V Vlogp:()))
After substituting
ü, (xx)) y + vz
(111)
|101
= (m() (t) 2 logg
(112)
(((2)))) + di + (x, log Pr x.)) logg: (unfass
4 log (xx)
V log (x)))
(113)
A (l)) log (x)) Vlog
+ -/22()
(114)
V Po
be (xx) div un (z)) + (t)l
=v(())) 2 [t) (x;) +
V Po Pt (x;)
(115)
V log (xx)) 1
log R R log Pa (x). V log (x
+ -22(t)
V (x))
(116)
V lo (1,
=x() 5 2222 (x.)) + be (xx) + div uu x
1
log))
5 ( V (t,)))
(117)
V log (xx) 2
2 (t) (z) R(V log pa (z)) V Io (x:)
v (lggp((x;)
(118)
I ( ux( be(xx)) + div (x:)) + 1/2 (t) (A (o - RR logpt (og Vlogg(
(log)(())) (119)
Vlogp,
The solution is
(x;)) (x,))) bx(s)) + div (xx)) + (1) (A lo (x) R
Vlogma v (og)) (120)
log p (x
which exactly matches Eq. 15 when ip expected. Now suppose that u, u. pF-ODe and b is defined as in Eq. 22.
1 V2D Φ (4)
(x)) = div ux())
- (1) of
(22)
Then the drift becomes
1 DG-ODE (x: 2 2
A Alogp( R(V2 lo V (x) log (x
(x) u
(121)
| (z:))
Practical approximation The Laplacian, A (e)),, is given by the trace of the Hessian 2 log (z(),, which corre- sponds to the sum of its eigenvalues. In contrast,
R. R(V2 (x)), llo V Vlogpt
7 72 log pa (x) log (x
V log (x)
represents the Rayleigh quotient of the Hessian evaluated at V log This quantity is bounded absolute value by the largest absolute eigenvalue the Hessian, i.c., its spectral norm. Intuitively, when the eigenvalues the Hessian are relatively uniform- such as when it is close to a scaled identity matrix-ti the Laplacian scales linearly with the dimension,
22

---

Density guidance
32 sampling Steps
51 sampling
128 samping steps
512 samplig steps
1024 sampling sleps
4006 samping steps
(f = 0.1ggt)
Obtained
8000
2000
( = 0.5g(()
Obtained logrt
6000
2000
(P()) = 0.9g(t)
Ostzined
6000
2000
4000
Target log P
Target log
Target log Pa
Targel log
Target log
Target log
Figure 15. Explicit Quantile Matching obtains exact likelihoods even for stochastic sampling. Top row: small amount of noise; Middle row: medium amount of noise; Bottom row: large amount of noise. The higher the amount of noise sampling, more steps need to be taken for the difference between desired log po and obtained log to go zero.
whereas the Rayleigh quotient remains bounded by constant. This suggests that in high-dimensional settings, the Laplacian dominates.
Empirically, we verified this intuition by estimating both quantities on real data. Specifically, we used VP-SDE model trained on CIFAR-10 10 (32 32 resolution, D 3072) (Karczewski et al., 2025) and a VE-SDE model trained on ImageNet (64 64 resolution, D Pt. (used 8192 and 16384 samples respectively) and found that the ratio negligibly small in practice
12288) (Karras et al., 2022) and sampled uniformly values of [0,T] and corresponding
R 2 log (x)), V p:
0.0003 0.00006
for CIFAR-10
~
(122)
A Ao
0.00006 0.00003
for ImageNet64
This confirms that, in practice, R (og log (x:)
negligible compared to A (z)). Therefore, in practice,
we use
A log Vlogp.()).
DG-SDE u
(x) = u
DG-ODE (x) ク (t)
(123)
log (1
correction for added stochasticity
H. Explicit quantile matching with stochastic sampling
In this section, we repeat the experiment from Appendix E, where we define the desired log-density evolution b, (x) ar di. where o; is the empirical quantile function (see Section 4.1). However, instead of density guidance, we perform stochastic density guidance, i.e. sample with the density guided SDE:
dx: u, (x, + (())))(
(124)
where ú, is defined in Eq. 120, with the exception that we set We experimented with different definitions of şo, which controls the strength the noise injection, specifically, we tested (p()) = rt(t) for r = [0.1, 0.5, 0.9], where g is the diffusion strength the forward process Eq. 5. As expected, as the amount noise increases, the required number of steps to take to achieve exact likelihoods increases, Please see Fig. 15.
(x)), V (x)):())) to zero as explained in Eq. 122.
23

---

Density guidance
I. JAX implementation of Score Alignment verification
Import jax
import jax. random as jr
Import numpy as jnp
def aug drift (u, model_ modei_key, key, _p__ke jr.spiit (key, eps jr. rademacher (epa_key, (x.size,), dtype-jnp. 3loat32) def (xx):
key):
return (t, rerrshhpee (x. shape), key-model_key) flatten def du_dv(x_)
11
u_pred, du dv pred jax. (u, (x_,), (v,))
12
return h.predd (x. shape), du_áv_pzed return du_dv fx.aatte))))
13
15
def aug_driftt__omeeal (u, x, v, t, key);
16
model_key. eps eps_key jr.split (key, 2)
17
epa = jr. rademachee (eps_key, (x.size,), dtype=jnp. float32) def (xx):
utt, resehh (x. (x. key=model key) flatten def du dv (x
20
21
u pred,, du dv pred = jax. ivp (u, (x,),, (v,)) return di_åv_pred, r.pred.reshape (x. shape)
22
def div_du_dv(x_)
du_dv_pred, du_dv_eps, u_pred jax. (du_dv, (x,)), (eps,), has_aux-True) return u u_pred, du su__v__pred -jnp. sum du du_dv_eps)
25
26
return div_du_dv (r__du_d((x.flatteee))
def score aocraaaligmeentteeitiioooann x_T, V_T, dt, key, eps=1c-2, usc_omega=Palse)
t T
30
x = X_I v - v_r
31
omega jnp.sum(v_") **2) if use_omega else None
while eps:
key, subkey jr.split (key)
35
if use ase_omega:
dx, dv, domega = drift w_ (u, t, subkey) omega át domega
else:
39
dx, dv aug_drift(u, (u, 8, v, t, subkey)
40
x - dt dx
v - dt dv
t
43
if use_omegar
44
return omega
else:
return jnp. (v, score_fn(ess, key))
Assuming score_fn is known
Listing 1. JAX Implementation of Score Alignment Verification
J. Quantitative analysis of prior and density guidance
In this section we provide more samples and a quantitative analysis showing that we can reliably control log-density of generated samples and thus amount of detail as measured by PNG file size. Please see: Fig. 16 for results for StableDiffusion. Fig. 17 for EDM2, and Fig. 18 for EDM2 with classifier-free guidance.
K. More Stochastic Density Guidance samples
We generate more samples using Eq. 25 with the EDM2 model in two scenarios: adding noise early: (p(t) = 0.2g(t) for log SNR(t) < -4 and sp(t) 0 otherwise; and adding noise late: ppt) = 0.3g(t) for log SNR(t) > -3 and (t) otherwise. To demonstrate that have fine-grained control over image detail, we do for various values the
24

---

Density guidance
= 0:25
0.33
0:45
0.55
0.63
0.75
Density Guidance
PNG size
|x, 194.19
|x. 193.31
x
192.4
x
191.37
x
190.69
x
189.82
Prior Guidance
PNG size
Figure Stable Diffusion v2.1 samples. PG and DG monotonically control the amount of detail measured PNG file size.
<=
= 0.4
6=
=
0.75
Density Guidancee
sice
[xy]| 130..9
xr||
|
127.13
Prior Guidance
PNG sto
Ixzil
[xyl
Figure 17. EDM2 samples. PG and DG monotonically control log-density and amount of detail.
==
= 0.4
== 0.6
2 = 0.75
Density Guidance
PNG
[xrl|
xrl|
|x+|
Xr||
Prior Guidance
PNG
Figure 18. EDM2 samples with classifier-free guidance. Both PG and DG effective when CFG is used.
25

---

Density guidance
S
Density Guidance
Food carly
noge bit
carly
Prior Guidance
sole lite
roes
Figure 19. Density guided samples in stochastic sampling. We chose two starting random seeds and for cach of them generated 4 random samples with two strategies: adding noise early in the generation (altering high-level detail), or late (low-level detail). For the inner sampling loop, used the same random seeds across all runs.
Density Guidance
Prior Guidance
Regular Samples
Optimal Density Guidance
Optimal Prior Guidance
18
Figure 20. Density and Prior Guidance improve perceptual quality measured by NIQE score. Left: NIQE scores (lower is better) for both Density and Prior Guidance on the EDM2 model for various values of the hyperparameters (g and ||æ> || respectively) with regular samples reference in grey. Right: Representative examples of best-scoring hyperparameters. NIQE score seems favor lower-detail images.
hyperparameters. See Fig. 19 for a visualization. We compare this for Prior Guidance, which is not principled for stochastic sampling, because the larger the amount of noise, the less information dr carries about the final generated sample wo.
L. Connection with perceptual metrics
There exist various metrics measuring the perceptual quality of image generation models. Most popular include LPIPS (Zhang et al., 2018) and SSIM (Wang et al., 2004). However, these are reference-based quality measures, meaning that they require the ground truth to compare to, which is not available in unconditional image generation. We therefore used NIQE (Mittal et al., 2012), which is a non-reference perceptual quality measure, reported to strongly correlate with human judgement. It provides a single number per image, which indicates whether an image has been distorted (a lower number higher quality). was used, e.g., by Sami et al. (2024) to evaluate super-resolution diffusion models.
M. Prior Guidance samples
In Fig. 21 we show samples generated with the EDM2 ImageNet 512 x 512 model (Karras et al., 2024b). Specifically, we randomly sampled latent code æт OTE for e [N Ip) and scaled it to have specific values of the norm. Since log (T(x)) = C - 1 |l and || ~ (D), we choose the values of the target squared norm to be quantiles of (D) for q E [0.001, 0.999] for xr to remain in the typical region of pr. Higher values of q mean higher norm, i.c. lower log pr (xr), and are thus decoding produces more detailed images.
26

---

Density guidance
Given that the Score Alignment holds for the EDM2 model (Fig. 3), can see that scaling the latent code (Prior Guidance) effective in controlling log po (20) and thus the image detail. We additionally include samples for StableDiffusion v2.1 (Rombach et al., 2022) in Fig. 22.
N. Gaussian Mixture asymptoties
Lemma N.1. Let f R + R, [(x) - exp( 1 Then for all x E R
[(x) 1
(125)
Proof.
1
1
(1
f(x) = 2
exp
-
2
I + exp
-
22
exp
(-2))
2
1
2
fx))) for x < < and (z) for x > Therefore for it E R
f(x) - f(2) = exp(-1)2= <
6
Lemma N.2. For any and
a
2
+ x
2 4a.
(126)
r
Proof.
0 S
2
a²
2a
x²
( + (=+x)²
4a
x22
Theorem 1. Let X E R be drawn from a mixture of K Gaussian components:
Pr(Y=k)=mm) X NN TTD);
(127)
where 2j for i # (note that this assumption is not restrictive, because when different components share the mean, the mixture can be rewritten with distinct μ and updated T). Define
1 11l12
M(X)= /2D
Σ
wy (X)
D
(128)
a²
where
1 || ,1 11
w)(x)) Σ K
T exp
20
(129)
2 || m= Tm exp
pm
Then h(X) $ 1), 1) as D x
Proof. Step 1: It suffices to show h(X) / (Y = k)
N(0, 1). Indeed,
K T
(t(x)(1) E [x)
Σ
T&B (x(( Y =
k=1
Jem
Thus each conditional law converges to (0, 1) (point-wise convergence of characteristic functions), so does the unconditional mixture.
Step 2: Rewrite X 14 oe. Conditioning Y k. we have X 14 + ae, e N(0, Ip). Thus
XX 1
| D d N(01))
||e||
2
X
22
v 2D
27

---

Density guidance
Step 3: Define remainder Rp. Set
K Σ
||X - -1y11
(X)
||e|| R.D.
o²
1=1
so
||E||2 D
RD V2D
h(X)
+
V2D
Using Slutsky's theorem, suffices to show RD/ 22/v2D= 4 O, which a weaker condition than convergence in probability. thus suffices show
R +
(130)
v2D
Step 4: Showing Rp/v2D & 0. Let
AJA = 14
+ 0.
0
Note
| μب o²
||e + , 1|2 ||e|| + 2e D + ||,,,11 ||=|12 by
for by 2E Det j| 1? and by since A 0. Hence
K
RD
Σ
wy (X) |2E + |A = -u
==1
j
Also:
Ty exp
20 2
[12]
J exp
[- (1 + bb)]]
"y exp( bb))
W/ (X)
Σ
Tm exp
20 r pm 11
K
Tm exp
1- [1|=|12 + bm)]
Σ K
-m=1 Tm exp( (5ml
m==1
Then
Σ jkt Ty expppppppp -b,))
RD
Tk Empz Tm exp( (mm))
We will now separate the sum in the numerator of RD into positive and negative bj. Define J+ = {j * klb; 2 0} and
J- = Li + kiby < 0):
EjE+ Ty eep(pp bb;bb
Ejer- Tyy exp((}jb))b
RD
Th Emohh Tm exp(-3Jun) +
Thk + Emyt Tm
From Lemma N.I, exp(-1b,)) and thus Ejer Tr achieve upper bounds):
S
Therefore (note shrinking denominators
Ejeni Tyy exeep((bbr
Ejes Tj eexp(111 bbb )b5
RDl =
TTk + Emppe Tm exp( (b kmm)
Th + Ematk Tm expl 2 bm)) ΣJEJ- Ty exp( bb;) Th + Emsrk Tm cxp( 5m))
1
Kk +
Emsr Tm exp( (xp(-}bm)
1 ΣjEJ T T exp((t/) |b,b
+
Tk.
T + EmeJJ exp( 5mbm)
ΣjeJ Ri exp( exx((=bb,)155
+ Σ ,51byb.
Th + EmEJ- exp( bm)) Thk
jEJ
28

---

Density guidance
3, exp(-
Lge By exp( }bz) met Eme- Rm expl - 2bm)
where w #x +x+ -Emes-- um exp(- 2bm)
and jEJJ Wy
S 1. Thus
Σ و
Σ
Wi max bby S max 1رط max( min bj. 0), j+k
jEJ
JEJ
jEJ
jEJ-
where the last equality comes from the definition of J In summary
RDl - + max
min ,,, 0). 8fk
Now, for any 8 0:
Pr
RDD 22D
6) Pr (RD > 8v2D) < Sr
mmx
- 1--miin bj. 0) 8v2D j/k
T
Now choose large enough so that a = óv2D 1 > 0. Then
max mraxx(mmin/l min by. 0) a -Pr (3 by <a) - Σ
Pr
Pr (b, < -a).
1
Auu |
Now plugging in the definition of by and using the fact that Z
.N(0, 1).
-|A 1)
Pr (b < aa)= Pr (2e" , || < <a) Pr Z <<< -
||
And using the standard Gaussian tail bound Pr(Z < -t) = =P > 1) S exp( -2)) for >
Pr (b, < -a) S exp (-1 || + | 9) S exp
(-
1
(-29)
2
where the last inequality comes from Lemma N.2. Therefore
Pr
P
Pr(b, < -a) < Σ exp
--2a = (K exp
exp (-2 ((VV2-
1
22
Th.
D-x0
jz
j/
We have shown that for all 8 > 0
RD
lim Pr D-x0
8 0,
(131)
V2D
which means
RD
0.
2DD D
29

---

Density guidance
0.001
0.995
Figure 21. Prior Guidance controls the detail when Score Alignment holds. Samples generated with the EDM2 latent diffusion model (Karras et al., 2024b) for different values of quantiles for the x² distribution. See Appendix for details.
0.001
0.999
Figure 22, Samples with Prior Guidance on StableDiffusson v2.1 (Rombach al., 2022)
30


Reviews:
[{"review_id": "PkfTlWiSKh", "reviewer": "Reviewer_8ij3", "summary": "The paper proposes a novel method, Density Guidance, to control the level of detail in generated images of flow models. It addresses the observsed mismatch between image likelihood and perceptual quality. The samples of high-likelihood are usually overly smooth, while the low-likelihood ones are more detailed. The author analyze the Prior Guidance adn introduce score alignment. They then propose Density Guidance to enable explicit log-density control by modifying the generative ODE. The model is further extended to stochastic sampling, enabling precise log-density control while allowing controlled variation in structure or fine details. The experimental results demonsrate taht the proposed method can adjust image detail while maintaining the image quality.\n\n### update after rebuttal\nThe rebuttal has addressed my concerns regarding the perceptual evaluation and ablation study. The authors have provided satisfactory answers about the relationship between perceptual metrics and score alignment, as well as clarified the distinctions between ODE sampling and stochastic sampling approaches. Based on these clarifications, I have decided to increase my score to 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer for their positive comments and constructive feedback. We address the raised points below.\n\n**File with new figures:** https://anonymous.4open.science/r/DensityGuidance-20E6/Density_guided_sampling___Rebuttal.pdf\n\n> Discuss some papers about perceptual quality metrics and detail-preservation.\n\nCertainly. In the camera-ready revision we will include a discussion on perceptual quality including the reference-based metrics such as the ones suggested (LPIPS, FID, SSIM), as well as no-reference-based, such as NIQE [3].\n\n> The user study or perceptual evaluation is missing. LPIPS, FID, SSIM or user study could be added to strengthen the claims.\n\nThank you for this suggestion. The metrics proposed are \"Reference-based\" metrics, which compare the generated images to reference ones. LPIPS and SSIM score a single image against a single reference image, while FID compares the set of generated images to the set of \"real\" images. The issue with LPIPS and SSIM is that, for a given generated image, we do not have a corresponding \"ground truth\" image to compare to. FID would have been more suitable for our use-case, however it is computationally expensive, requiring generating tens of thousands images [1], which was beyond our coomputational budget. It has also been reported to not always agree with human judgement [2].\n\nWe thus propose to use NIQE [3], which is a \"no-reference\" image quality evaluation metric, which is reported to correlate strongly with human judgement. It provides a single number per image, which indicates whether an image has been distorted (a lower number - higher quality). It was used e.g. by [4] to evaluate super-resolution diffusion models.\n\nWe evaluated Density and Prior Guidance for EDM2 model, and the now included State-of-the-art model FLUX. In Figure 14 you can see that:\n* For EDM2 model: guided samples can obtained better (lower) NIQE scores than regular samples (gray area);\n* For FLUX model: regular samples already score optimally.\n\nAfter a visual inspection of optimally scoring guided samples (as measured by NIQE) in Figure 15, we noticed that NIQE actually prefers images with significantly less detail than regular samples. For the FLUX model (Figure 16), there were no perceptual differences between regular samples and best NIQE scoring ones.\n\n> The paper introduces [...] score alignment, density guidance, and stochastic density guidance, but it does not conduct a ablation study to evaluate the contributions of each component.\n\nThank you for this question. We take this opportunity to clarify:\n* Score alignment (SA) is a novel framework to verify whether a known procedure (Prior Guidance) will be effective in practice\n* (Stochastic) Density Guidance is a novel algorithm proposed by us, which is principled and can be used with any diffusion model (regardless of whether SA holds)\n\nThat said, we now performed an extensive evaluation of Prior and Density Guidance, both quantitative and qualitative, including novel models. Please see our response to Reviewer ppbq for more details.\n\n> How does density guidance compare to methods that use explicit perceptual loss functions, such as LIPIS for controlling detail?\n\nThe difference between Density Guidance (DG) and models trained with perceptual loss functions is two-fold. First, DG can be used, without any finetuning or retraining, and for no extra cost, on models which were trained without any perceptual losses. Second, it provides a way to control the generations. One can generate images with either high or low level of detail, depending on the use-case. Models trained with perceptual losses do not have that capability.\n\n> What is the relationship between preceptual metrics and score alignment?\n\nScore Alignment guarantees that Prior Guidance effectively changes log-density in deterministic sampling. We show in Figure 14 how that can impact the perceptual metrics.\n\n> How different does the model perform in ODE sampling the stochastic sampling with Density Guidance?\n\nPlease see our response to Reviewer ppbq for the details on the comparison on different modes of sampling.\n\nWe thank the Reviewer again for their useful suggestions that helped improve our work. We hope that our clarifications and additional experiments addressed all concerns and ask for a reconsideration of the score.\n\n---\n\n[1] Heusel et al. \"GANs trained by a two time-scale update rule converge to a local Nash equilibrium.\" (NeurIPS 2017)\n\n[2] Liu et al. \"An improved evaluation framework for generative adversarial networks.\" (arXiv 2018)\n\n[3] Mittal et al. \"Making a “completely blind” image quality analyzer\" (IEEE Signal processing 2012)\n\n[4] Sami et al. \"HF-Diff: High-Frequency Perceptual Loss and Distribution Matching for One-Step Diffusion-Based Image Super-Resolution.\" (arXiv 2024)", "claims_and_evidence": "The paper make several claims\n1. Density Guidance enables explicit log-dencity control. It is justified by a derivation modifying the generative ODE.\n2. Score Alignment explains prior guidance. It is supported by a theoretical analysis.", "methods_and_evaluation_criteria": "The proposed method is well-justified for controlling image detail in flow models. The use of score alignment to explain prior guidance provide a good insight. The evaluation contrains comparison of generated images, analyses between log-density and perceptual metrics and quantitative evaluation of proposed method, which are appropriate.", "theoretical_claims": "The theoretical analysis about score alignment and density guidance looks sound. The authors provide detailed derivations in the appendix.", "experimental_designs_or_analyses": "The experiments effectively validate the proposed method. It conducts the experiments on CIFAR-10 and ImageNet datasets. The proposed method is compared to stable diffusion and edm2. The paper also evaluates the relationship between log-density and perceptual detail.", "relation_to_broader_scientific_literature": "The work is related to the literature on diffusion models and normalizing flows. It connects well to prior fingdings on the relationship between likelihood and image detail.", "essential_references_not_discussed": "It may be better to discuss some papers about perceptual quality metrics and detail-preservation.", "other_strengths_and_weaknesses": "Strengths:\n1. Good theoretical contribution. The paper introduces score alignment, which explains the relationship between prior guidance and image detail. This provides a solid theorecial insight for the observation in prior work.\n2. The method is well-motivated. The paper proposes density guidance, which enables log-density control insteand of heuristic modification. It can be used in ODE framework of diffusion models and extended to stochastic sampling, which allows controoled variation. The method does not need any additional training.\n3. Comprehensive expriments. The paper validate the method on CIFAR-10 and ImageNet. The method is compared to Prior Guidance and demonstrate better control over image detail. \n\nWeaknesses:\n1. The user study or perceptual evaluation is missing. LPIPS, FID, SSIM or user study could be added to strengthen the claims.\n2. Lack of a comprehensive ablation study. The paper introduces multiple modifications, including score alignment, density guidance, and stochastic density guidance, but it does not conduct a ablation study to evaluate the contributions of each component.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. How does density guidance compare to methods that use explicit perceptual loss functions, such as LIPIS for controlling detail?\n2. What is the relationship between preceptual metrics and score alignment?\n3. How different does the model perform in ODE sampling the stochastic sampling with Density Guidance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aDpw943OqI", "reviewer": "Reviewer_J9Xk", "summary": "This work introduce a method to control the sampling density in diffusion models. The main contribution is using score alignment to scale and control the sampling guidance, which works for both deterministic and stochastic sampling. \n\nThe experiments demonstrate that density guidance and its stochastic extension provide fine-grained control over image details.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the Reviewer for their support of our work. Below we address the raised concern.\n\n**File with new figures:** https://anonymous.4open.science/r/DensityGuidance-20E6/Density_guided_sampling___Rebuttal.pdf\n\n> I would appreciate it if the authors could provide practical advice on how the proposed density guidance might improve current sampling methods.\n\nThe Density Guidance works on any diffusion model without retraining, or finetuning and requires no extra cost during sampling. We now ran a new experiment and measured the predictions of the generated samples using NIQE [1], which is a metric for image quality assessment reported to correlate strongly with human judgement.\n\n### Potential applications of detail control\n\nWe believe that potential applications of the presented methods are image editing, where the user might want to control the amont of detail in the image. From [2] we also know that highest densities contain cartoon-like images, and thus user can have fine grained control over the spectrum between realistic images and cartoons describing the same scene. People also acknowledge that image generation can be used for “aiding designers in producing striking scenes for video games” [article](https://news.mit.edu/2025/ai-tool-generates-high-quality-images-faster-0321) and we believe that detail control can become an addition to that toolkit.\n\nWe also note that there has been interest among the practitioners in explicitly controlling the amount of detail in image generation:\n\n* Modifying Stable Diffusion to generate less detail [Thread](https://www.reddit.com/r/StableDiffusion/comments/1e4spmo/how_to_generate_images_with_less_detail/)\n* Modifying Stable Diffusion to generate more detail  [Thread](https://www.reddit.com/r/comfyui/comments/1866y53/how_do_i_get_more_detail/)\n\nFinally, we would also like to point out that the density-guidance is derived to control the log-density of the generated samples. We know from prior literature [2], that for image data, this correlates with image detail. Perhaps in domains different than images, controlling log-density may be desirable for other purposes. Density Guidance can be used for that as well. Investigation of domains other than image data is out of scope for this work but certainly an interesting direction that we hope this work could pave way for.\n\nWe hope our response has addressed the Reviewer's concerns, and that the additional experiments provided in the uploaded file further strengthen your support for our submission, which we hope will be reflected in an updated score.\n\n---\n\n[1] Mittal et al. \"Making a “completely blind” image quality analyzer\" (IEEE Signal processing 2012)\n\n[2] Karczewski et al. \"Diffusion Models as Cartoonists: The Curious Case of High Density Regions\" (ICLR 2025)", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "yes", "experimental_designs_or_analyses": "yes.\nHowever, the experiment only demonstrates the side effects of density sampling. I have not observed any positive impact of density sampling on sampling quality.", "relation_to_broader_scientific_literature": "This paper may enhance the community’s understanding of the sampling process, and some of the perspectives presented are quite interesting.", "essential_references_not_discussed": "N.A", "other_strengths_and_weaknesses": "This paper introduces a density guidance for both deterministic and stochastic processes, enabling precise control over the likelihood (log-density) during the sampling process. Additionally, this work provides solid theoretical foundations that can enhance the community’s understanding of the sampling process. Furthermore, the study uncovers some interesting phenomena: high-density generated results may appear relatively blurry, while lower-likelihood samples introduce more intricate details.\n\nOne concern I have is that I have not observed any positive impact of the authors’ proposed solution on existing sampling techniques. While I acknowledge the authors’ contribution and the effectiveness of the density guidance, I would appreciate it if the authors could provide practical advice on how the proposed density guidance might improve current sampling methods.\n\nHowever, I must still express that I am inclined to accept this work.", "other_comments_or_suggestions": "n.a", "questions_for_authors": "I hope the authors can offer suggestions on how density guidance could be used to optimize the current sampling process.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rZCYzhmPq3", "reviewer": "Reviewer_reGE", "summary": "This paper studies the control of the amount of details in samples from diffusion models. The authors first established a theoretical framework (Score alignment) to explain a trick to increase sample details in prior literature. Then, the authors explore a suite of methods that can be used to control the exact scale of amount of details in the generated samples, both in the deterministic case (Density guidance) and in stochastic case. Theoretical claims are proved on simplified cases and some qualititave anlaysis is performed. Experiments on SD2.1 and EDM are performed to show the real-world use cases.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer for their thorough evaluation of our work and very insightful questions! We address the raised points below.\n\n**File with new figures:** https://anonymous.4open.science/r/DensityGuidance-20E6/Density_guided_sampling___Rebuttal.pdf\n\nGlossary:\n* Density Guidance - DG\n* Prior Guidance - PG\n* Score Alignment - SA\n* Classifier-free guidance - CFG\n\n> **Q1:** Only two models analyzed for SA, and no theoretical guarantee.\n\nThis is true that there is no theoretical guarantee for SA. There cannot be such a guarantee as we show:\n  * In Fig 3, right - for the CIFAR model, SA does not hold for 3% of the latent codes (line 209)\n  * In Appendix C.3, we provide an example of a Gaussian mixture (exact scores known), where SA does not hold.\n\nThis emphasizes the point from the paper: **SA does not always hold.** This in part motivates the DG approach, because PG is not always guaranteed to work.\n\nWe discuss FLUX at the end of the response.\n\n> **Q2:** no quantitative evidence of Explicit Quantile Matching.\n\nPlease see our response to Reviewer ppbq.\n\n> **Q3:** Can you prove that Eq 21 implies Eq 18?\n\nWe actually do not make that claim. Eq 21 is based on results in Appendix D, which **we have now extended by a rigorous proof in the Gaussian Mixture case** (Theorem 1). The motivation is to keep the samples in the typical regions of $p_t$, but we do not guarantee exact quantiles.\n\n> **Q4:** Few StableDiffusion samples. Also, semantic changes visible.\n\nWe have included more levels for Stable Diffusion, with PNG sizes (Fig 19). Regarding semantic changes - this is true, and can be even more drastic as in the train example with PG in Fig 19. We do not guarantee that only the low-level features change in all cases. In the extremes semantic changes can happen as well. However, it is consistent with the amount of detail as measured by PNG.\n\n> **Q5:** Quantitative analysis of DG\n\nPlease see our response to Reviewer ppbq.\n\n> **Q6:** Only two levels in Fig 10. Does stochastic guidance differ from PG?\n\nWe added more samples and levels in Fig 21 for both DG and PG. Perceptually, both seem to be monotonically controlling the detail, but we argue in the response to Q5: DG is more accurate.\n\n> **Q7:** What are practical application scenarios?\n\nDue to character limit, please refer to our response to Reviewer J9Xk, where we discuss potential applications.\n\n> **Q8:** When might the method fail?\n\nA potential issue is applying DG in low dimensions. We use the fact that $h(x)$ is approximately Gaussian (Appendix D). This only holds when the dimensionality is large.\n\nAnother approximation we make is discussed in lines 1147-1161. We justify it on two datasets. If one wants to increase the accuracy further, Eq 119 can be used instead, which makes no approximations. It comes at a cost of one additional Jacobian-Vector-Product.\n\n> **Q9:** Would the method work with CFG?\n\nAs explained in the CFG paper, CFG can be interpreted as classifier guidance with an implicit classifier. This means that CFG is just a regular diffusion process, but with a different base distribution, favouring a certain class. Thus, all our results apply without changes, just with a redefined target distribution.\n\nWe sampled with an EDM2 model with CFG and found consistent behaviour with other models. Please see Fig 22.\n\n>L175, \"$v_t$\" is not discussed \n\nWe denote by $v_t$ the score pushed forward from $T$ to $t$. We refer to it later in the text, e.g. in Eq 12. We will make it more explicit in the final revision.\n\n> No reference for Eq. 3.\n\nThe reference is Chen et al. 2018. We will make it more explicit.\n\n>I didn't find such claims in Song et al. 2021b\n\nIt can be seen in the Appendix in Figure 6 - it is not referenced in the main text. Authors call it \"temperature rescaling\" (reducing norm of embedding).\n\n> Will the methods work for SOTA models like FLUX?\n\nWe have included new results with FLUX.1[dev]. Fig 23 shows samples with PG and DG, and Fig 24 shows a DG image with PNG and TIFF filesize comparison. Fig 25 shows the coupling of PNG and TIFF filesizes over guidance.\n\nFLUX shows different behavior to other models. Images are richer, but detail variations from guidance are milder. The weaker DG effect can be attributed to the FLUX model coupling a latent diffusion on 16x64x64 space with a strong decoder to 3x768x768, whose effect on logp is unknown. We only control the latent portion of the model. FLUX is undocumented and unpublished.\n\nThe effect of DG on PNG filesizes also becomes inconsistent: adding more semantic detail doesn’t necessarily increase filesize, possibly due to the images already being highly realistic and rich in patterns. Furthermore, PNG is only an approximation of the true information content of the image. We include comparison to TIFFs, which shows more consistent coupling between filesize and detail.\n\nWe thank the Reviewer again for their high-quality review that significantly contributed to improving our submission.", "claims_and_evidence": "**Major claims made in Sec. 3**: \n\n- The trick of \"scaling latent code\" will work because (1) it decreases the likelihood of $x_T$, (2) decreasing likelihood of $x_T$ correlates to the decrease of likelihood of $x_0$, and (3) the likelihood of $x_0$ correlates to the amount of details. (1) is supported by the prior Gaussian distribution. (2) is supported by the Score Alignment condition, which is partially and qualitatively shown for selected models. (3) is supported by a previous literature. \n  - **(Q1)** The major complaint here is on (2). Only two models (VP-SDE and EDM2) are analyzed qualitatively. It would be great if the authors could analyze state-of-the-art models (e.g., Stable diffusion XL, FLUX). Moreover there is no theoretical guarantee for this property.\n- Likelihood correlates well with amount of details in the image, which is shown through the correlation between the image compression size and the sample likelihood.\n\n**Major claims made in Sec. 4**\n\n- Explicit quantile matching enables sampling images with an exact likelihood of $c$. This is mostly supported with theoretcial proof in Appendix D,  and a claim from prior literature that sampling from typical regions will produce accurate predictions.\n  - **(Q2)** The major complaint here is that this claim is not supported empirically. Practically, the authors propose to sample $K$ times to estimate the quantile function. However it is not clear on how large $K$ should be in practice. Moreover, there is no empirical results on whether using this method indeed produce samples with a likelihood of the exact value. It would be great if we can see results of this algorithm to be applied to real-world unconditional generators, and whether the produced samples indeed have an altered amount of detail and whether those samples indeed have an exact likelihood expected.\n- Implicit quantile matching works similarly for conditional generators. This is supported by some sparse theoretical results in Appendix D. and qualitative results in Fig. 9.\n  - **(Q3)** I could not fully understand why using $b_t$ as defined in Eq. 21 would guarantee the condition in Eq. 18. I can get some intuitions from the fact in Eq. 20, yet is there a rigorous proof on this?\n  - **(Q4)** The results in Fig. 9 are interesting. However, the result for stable diffusion is sparse: there are only two levels shown, so we could not really tell whether the method is really achieving a fine-grained control over amount of detail. Moreover in SD2.1 it can be seen that there are changes in semantic contents, especially in the train example. \n  - **(Q5)** There are no quantitative analysis on the generated samples' amount of details. It would be beneficial if we can see a plot or a table measuring both the image compression size (in PNG as in Fig.4) and the likelihood of $K$ samples in different levels.\n\n**Major claims made in Sec. 5**\n\n- Eq 25. extends the above sampling procedure to stochastic process. This is supported by the proof in Appendix F and by experiments in Fig. 10.\n  - **(Q6)** There are only two levels of details in Fig. 10. Similarly, we could not tell whether the proposed method works to control specific level or details. (Will a simple prior guidance do very similar thing?)", "methods_and_evaluation_criteria": "The methods proposed in this paper are sound from the description. The evaluation mostly makes sense except for some minor issues, as discussed in Q2, Q4, Q5, and Q6.", "theoretical_claims": "I didn't fully check the proofs in the appendix.", "experimental_designs_or_analyses": "Yes, I checked most experimental designs and analyses. They are mostly adequate except for some minor issues as discussed in Q2, Q4, Q5 and Q6.", "relation_to_broader_scientific_literature": "This paper studies models from recent studies in diffusion models (Song et al. 2021) and flow models (Lipman et al. 2023, Liu et al. 2023) and leverages some insights from prior literature (Song et al. 2021b). This paper is strongly based on the findings in (Karczewski el al. 2024).", "essential_references_not_discussed": "Relevant literatures are adequately discussed to the best of the reviewer's knowledge.", "other_strengths_and_weaknesses": "Beyond the strengths and weaknesses already discussed (Q1-Q6):\n\nStrengths:\n\n- The paper is well-written and well-organized. Detailed discussions and proofs are presented for most claims.\n- Extensive studies are conducted for the research question proposed. The contribution is solid and extensive, and the paper explores the proposed framework in various settings including conditional/unconditional generation and stochastic generation.\n- The problem studied in this paper is interesting. It may potentially be insightful to inspire reseach in related fields.\n\nWeaknesses:\n\n- There are some issues with the validation and experiment design, as in Q1-Q6.\n- **(Q7)** What would be a practical application scenario of the proposed technique? When would a user be interested in a fine-detailed control over the amount of details of the generated image?\n- **(Q8)** There is no discussion on when the method will fails to achieve the desired properties. There are some approximations in the theoretical proofs of the method, so it would be great if the authors could analyze the scenarios where the method will fail.\n- **(Q9)** How would the method be used together with Classifier-free guidance, which is the de facto methods to perform conditional generation for diffusion models?", "other_comments_or_suggestions": "Minor issues and comments:\n\n- In L175, it seems that \"$v_t$\" is not discussed in Eq. 11. \n- There should be reference for Eq. 3.", "questions_for_authors": "Minor questions:\n\n- L140(right), I actually didn't find such claims explicitly stated in Song et al. 2021b. Could you point out the exact location of such a claim?\n- Will the proposed method be able to work for state-of-the-art models like Stable Diffusion XL or FLUX?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rcSlg9ykfB", "reviewer": "Reviewer_ppbq", "summary": "This paper introduces a collection of methods for controlling likelihood of samples generated by a flow/diffusion model. Authors provide a comprehensive review of prior work on density control, in particular providing a more formal analysis of latent scaling [Song 2021]. They further introduce density guidance - a method for sampling with explicit likelihood control through an alternative ODE formulation ensuring the sample stays in a pre-defined quantile over time. They further introduce a stochastic variant of density guidance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the Reviewer for their time and efforts to scrutinize our submission. We address the raised concern below.\n\n**File with new figures:** https://anonymous.4open.science/r/DensityGuidance-20E6/Density_guided_sampling___Rebuttal.pdf\n\n>The practical utility of the proposed method is a bit unclear due to lack of evaluation. It would be great to understand whether the theoretical claims could be properly validated? E.g. why not follow methodology from [Karczewski'25] for quantiative/qualitative analysis?\n\nThank you for this suggestion, we have now included an extensive evaluation of the proposed methods building on the methodology of [Karczewski’25]. Specifically:\n\n### Explicit Quantile Matching (EQM)\n\nWe estimated the quantile function for the CIFAR model as described in line (311 left). We tested $K = [16, 32, 64, 128, 256, 512, 1024]$ and found that using $K=128$ is enough to ensure a correlation between the desired value of log-density and the obtained one is above 99%. Based on the estimated quantile function $\\phi_t$, we estimate $b_t = \\frac{d}{dt}\\phi_t$ with a moving average of the finite difference estimates.\n\nFurthermore, we found that the difference between the desired values of log-density and the obtained ones goes to zero as we decrease the discretization error (increase the number of sampling steps). Interestingly, for lower number of sampling steps, even though we do not obtain exact desired values of likelihood, the correlation between the desired values and the obtained ones remains above 99%, even for as few as 32 Euler sampling steps. This means that for all values of the number of sampling steps, we saw a monotonic relationship between the target $\\log p_0$ and the amount of detail (PNG size). Please see Figure 17.\n\nFinally, we also show that **we can obtain exact values of likelihoods even when sampling stochastically** by using results from Appendix F, and the Euler–Maruyama algorithm. We tested different amounts of added noise: $\\varphi(t)= r g(t)$ for $r=[0.1, 0.5, 0.9]$. As expected, as the amount of noise increases, the required number of steps to take to achieve exact likelihoods also increases. Please see Figure 18.\n\n### Prior Guidance vs Density Guidance vs Stochastic Density Guidance\n\nWe quantitatively compared Density Guidance (DG), and Prior Guidance (PG) using the EDM2 model. We measures the correlation between the hyperparameter ($q$ for Density guidance and $||x_T||$ for prior guidance) and the obtained $\\log p_0$. We found 66% for DG and 68% for PG. \n\nFurthermore, we compared DG and PG with stochastic sampling. I.e.using Eq 25 for Stochastic Density Guidance (SDG), and Eq 7 for “Stochastic Prior Guidance” SPG (i.e. regular stochastic sampling after rescaling the latent code). We tested two scenarios:\nAdding noise early: $\\varphi(t)=0.2g(t)$ for $\\log SNR(t) < -4.03$, and $\\varphi(t)=0$ otherwise;\nAdding noise late $\\varphi(t)=0.3g(t)$ for $\\log SNR(t) > -3$, and $\\varphi(t)=0$ otherwise.\n\nWe found the correlation between the hyperparameter and the obtained $\\log p_0$ to be 50% for SDG and 25% for SPG. We summarize all correlations in the table below.\n\n||Density Guidance|Prior Guidance|\n|-|-|-|\n|Deterministic Sampling|66%|68%|\n|Stochastic Sampling|50%|25%|\n\nFor DG the drop in correlation from deterministic to stochastic sampling can be explained by the same reasoning as for the EQM, i.e. stochastic sampling requires significantly more sampling steps to achieve the desired levels of $\\log p_0$ (Figure 18).\n\nFor PG, stochastic sampling is not principled, i.e. the more noise we add during sampling, the less information is contained in the starting point $x_T$. For example, if $\\varphi(t)=g(t)$ for all $t$, then the process is the Reverse SDE, and $p(x_0|x_T)$ does not depend on $x_T$, and thus scaling the latent code has no effect on the final sample. Hence the need for Density Guidance for stochastic sampling. \n\nPlease see Fig 20 for details on the evaluation of log-densities and corresponding PNG files sizes, and Fig 21 for the visualization of the stochastic samples.\n\n### Additional Experiments\n\nWe have also added the following:\n* Analysis of the impact of Guidance on perceptual metric NIQE (Fig 14, more details in the response to Reviewer 8ij3)\n* More samples and quantitative results with Stable Diffusion (Fig 19)\n* Results with models using Classifier-Free Guidance (Fig 22, more details in the response to Reviewer reGE, Q9)\n* A new State-of-the-art model FLUX (Fig 23-25, more details in the response to Reviewer reGE, under SOTA models)\n* A rigorous proof of the hypothesis we posed in Appendix D about the asymptotic behaviour of $h(x)$ for the Gaussian Mixture (Theorem 1 in the uploaded file)\n\nWe thank the Reviewer again for their constructive feedback, which strengthened our claims and improved the quality of our submission. We hope that we have adequately addressed the concerns, and you will consider raising your score.", "claims_and_evidence": "The paper does deliver on the theoretical claims, but the experimental claims are not very extensively validated (e.g. empirical evaluation of prior vs density vs stochastic density guidance are a handful of qualitative examples).", "methods_and_evaluation_criteria": "There is almost no coherent evaluation.", "theoretical_claims": "I checked the claims in the main paper leading to Eq.(24), assuming that proves in supp are correct, they do seem to be meaningful and consistent with existing work [Karczewski'ICLR2025]", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "This paper explores an interesting property of the flow/diffusion models that has been noticed recently [Karczewski'25] and explains ad-hoc techniques commonly used in score-based generative models [Song'25].", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "+ Overall the paper introduces a significant contribution to understanding the properties of generative diffusion models, both in terms of providing theoretical insight to existing sampling techniques, and in terms of novel methods for density control during generation.\n- The practical utility of proposed method is a bit unclear due to lack of evaluation.", "other_comments_or_suggestions": "Although this is mostly theory-focused work, it will be beneficial to get minimal quant/qual validation, even if on toy data.", "questions_for_authors": "Apart from several images, experimental validation is missing, would be great to understand whether the theoretical claims could be properly validated? E.g. why not follow methodology from [Karczewski'25]  for quantiative/qualitative analysis?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a method for controlling the level of detail in images generated by flow-based models. It first analyzes the phenomenon where high-likelihood samples appear overly smooth, and low-likelihood ones are more detailed. They explain this with a new condition called score alignment, which ensures that scaling the latent noise vector reliably affects the output image's log-likelihood. Building on this, they modify the generative ODE to explicitly steer samples toward a target likelihood, and further extend it to stochastic sampling. Their approach enables fine-grained control over image detail without sacrificing sample quality.

The paper received 2 weak accept, and 2 accept. All the reviewers acknowledged that this paper is well-motivated, with good theoretical contribution. One reviewer raised concerns for validation and experiment design, who later was satisfied with the rebuttal. While the authors' initial response to reviewer `J9Xk`’s question on practical guidance was partial, the ACs confirmed that the subsequent clarification adequately addressed how Density Guidance can be used to control fine-grained detail during sampling.

Given the paper's solid quality and positive feedback, ACs agreed on acceptance.
Decision: Accept (poster)