Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search
Authors: Yuta Oshima
Masahiro Suzuki
Yutaka Matsuo
Hiroki Furuta
Venue: NeurIPS 2025 poster ()
Keywords: Inference-Time Search
Text-to-Video Generation
Diffusion Models
Alignment
Primary area: applications

Abstract: The remarkable progress in text-to-video diffusion models enables the generation of photorealistic videos, although the content of these generated videos often includes unnatural movement or deformation, reverse playback, and motionless scenes. Recently, an alignment problem has attracted huge attention, where we steer the output of diffusion models based on some measure of the content's goodness. Because there is a large room for improvement of perceptual quality along the frame direction, we should address which metrics we should optimize and how we can optimize them in the video generation. In this paper, we propose diffusion latent beam search with lookahead estimator, which can select a better diffusion latent to maximize a given alignment reward at inference time. We then point out that improving perceptual video quality with respect to alignment to prompts requires reward calibration by weighting existing metrics. This is because when humans or vision language models evaluate outputs, many previous metrics to quantify the naturalness of video do not always correlate with the evaluation. We demonstrate that our method improves the perceptual quality evaluated on the calibrated reward, VLMs, and human assessment, without model parameter update, and outputs the best generation compared to greedy search and best-of-N sampling under much more efficient computational cost. The experiments highlight that our method is beneficial to many capable generative models, and provide a practical guideline: we should prioritize the inference-time compute allocation into enabling the lookahead estimator and increasing the search budget, rather than expanding the denoising steps.

Inference-Time Text-to-Vides Alignment with Diffusion Latent Beam Search
Yuta Oshima' Masahiro Suzuki!
Yutaka Matsuo! Hiroki 2uruta
'The University of Tokyo Google DeepMind
yuta. www.maweebaabbbbmmmmmmoo t. u-tokyo......
ac
Abstract
The remarkable progress in text-to-video diffusion models enables the generation of photorealistic videos, although the content of these generated videos often includes unnatural movement deformation, reverse playback, and motionless scenes. Recently, an alignment problem has attracted huge attention, where we steer the output of diffusion models based Because there large room for improvement of perceptual quality along the frame direction, should address which metrics should optimize and how we can optimize them in the video generation. In this paper, we propose diffiusion latent beam search with lookahead estimator, which select better diffusion latent to maximize a given alignment reward at inference time. We then point out that improving perceptual video quality with respect to alignment to prompts requires reward calibration by weighting existing metrics. This is because when humans vision language models evaluate outputs, many previous metrics quantify the naturalness of video do not always correlate with the evaluation. We demonstrate that our method improves the perceptual quality evaluated on the calibrated reward, VLMs, and human assessment, without model parameter update, and outputs best generation compared to greedy search and best-of-N sampling under much more efficient computational cost. The experiments highlight that our method is beneficial many capable generative models, and provide a practical guideline: we should prioritize the inference-time compute allocation into enabling the lookahead estimator and increasing the search budget, rather than expanding the denoising steps."
measure of the content's goodness.
Introduction
The remarkable progress in text-to-video diffusion models enables photorealistic, high-resolution video generation [1-4]. Many future applications are anticipated, such as creating novel games [5], movies [6], simulators to control real-world robots [7]. However, the detailed contents of the generated video often include unnatural movement or deformation, reverse playback, and motionless scenes, which should not happen in the real world. For instance, simulating factual physics in the generated video is still challenging [8, 9]. Recently, it has attracted lot of attention to steering the output of diffusion models based reward evaluation, quantifying the goodness of the content, which studied as an alignment problem [10, 11]. There is a large room for improvement of perceptual quality along the frame direction in the video, and align models with our preferences, we should address which metrics to optimize and how to optimize them.
1 Work done as advisory role only. Weebsite: h/stes//sites. wwwgle...ooooooooooo.ooo.oooe... wwwp/oie/ttv-dlll.l..coe "Code: https://github wtt.uu...oooeee//e htm//hhhm001114////Diiiffooooo
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Diffusion Latent Bear Search
Lookahead Estimator
MSRVTT-test
beessing 3xep
intal Lewer
Latert Evelustion Procees
Gemini Calibrated Reward
BoN
0.62
DLBS
c=t+1
Ri-1
Lookaheed
IT
0.61
DL8S-LA
gr 3]
0.60
0.59
t---11
Exp-1
0.58
:
r==
1.
1.17 1.28
0.57
130
2
2
212
2
Accepsed Lasom
Rejoctedl Letent
Evaluatiio
Inference Time (second)
Figure I: (Lefft) Diffusion latent beam search (DLBS) seeks better diffusion path over the reverse process; sampling K latents per beam and possessing beams for the next step, which mitigates the effect from inaccurate arginax. Lookahead (LA) estimator notably reduces the noise at latent reward evaluation by interpolating the rest of the time steps from the current latent with deterministic DDIM. (Right) DLBS achieves much better computationall-ffiiii
than best-of-N (BoN), achieving higher performance gains under the same execution time. LA estimator (DLBS-LA) could remarkably boost efficiency only with marginal overhead on top DLBS,
In this paper, we propose Diffussion Latent Beam Search (DLBS) with lookahead estimator, an inference-time search over the reverse process (Figure I; Left), which can select a better diffusion latent to maximize given alignment reward. A lookahead estimator reduces the noise in the reward estimate, and a beam search robustly explores the latent paths, avoiding inaccurate argmax operations.
We then point out that the improvement of perceptual video quality, considering the alignment to prompts, requires reward calibration of existing metrics [12]. When evaluating outputs using capable vision language models [13, 14] or human raters, many previous metrics for quantifying video naturalness do not always correlate with them. Optimal reward design for measuring perceptual quality highly depends on the degree of dynamics described in evaluation prompts. We design weighted linear combination of multiple metrics, which is calibrated to perceptual quality and improves the correlation with VLMM/hmmnn preference.
We demonstrate that DLBS can induce high-quality outputs based on the calibrated reward, AI, and human feedback (Figure 2), without model parameter update, and realize the best generation under much more efficient computational cost compared to greedy search [15, 11] and best-of-N sampling [16. 17]. The experiments also highlight that our method is beneficial to many SoTA models (e.g.. Latte [18]. CogVideoX [19], and Wan 2.1 [20]), and provide practical guideline that we should prioritize the inference- time compute allocation into enabling the lookahead estimator and increasing the search budget, rather than expanding the denoising steps.
2 Preliminaries
Latent Diffusion Models Latent diffusion models [21, 18] are a special class of diffusion proba- bilistic models [22, 23], and popular choices for high-resolution text-to-video generation [24, 25, 4], which considers the diffusion process in embedding space. Let Xy be a video and encode it as Zo = Exc(Xo) using VAF [26]. Continuous-time forward diffusion process can be modeled as solution to stochastic differential equation (SDE) [27]: dz f(z. t)dt + g(f)dw, where Zn ~ Pz(z) the latent as initial condition while pt is the marginal distribution of 2, f: Ed x R - Rd is the drift coefficient, g: IR. IR is the diffusion coefficient, and w E Rd is d-dimensional standard Wiener process. f(: ) and g(-) are designed appropriately for the marginal distribution to reach Pr (z) R N/ I) /. > T [28]. Reverse diffusion process generates samples Zn through the follow- ing reverse-time SDE: dz = [[(z, f) V
2 log Pt (2)]dt+ g(t) where dt here is an infinitesimal negative time step from T to 0 and w E Id is a standard reverse-time Wiener process. We start this with Z (0, I). This SDE induces the marginal distribution on the data pd (z) (i.e., pre- pre-trained diffusion models). While we omit the notation for simplicity, we consider the text-to-video generation problem, where the diffusion process is conditioned text prompts c.
Alignment for Text-to- Video Diffusion Models this paper, we define the alignment problem in text-to-video generation increasing the probability of generating perceptually good video for

---

"Under rainbow, kicks up spray fast-flowing river."
Latte
GS (K 32)
DLBS-LA (KB 8,T
"dog puts paws together"
Latte
BoN (KB 32)
+ DLBS-LA (KB 8,7"
dogs esch other, suddenly skidding sharp
CogVideoX-58
DLBS-LA 8,7"
person hoverboard colliding with board stopping abruptly."
Wan 2.1-148
DLBS-LA (KB 8,7"
Figure 2: Comparison of text-to-vidco results between DLBS-LA, base models, and other sampling methods on SoTA models (Latte (18), CogVideoX [19], and Wan 2.1 [20]). DLBS-LA produces more dynamic, natural, and prompt-aligned vidcos than basclines.
humans, such as max Elp(O Xo, c)] where O {0, 1] represents if the generated video No conditioned on c perceptually higher quality or not. The common assumption is such a probability depends on proxy scalar reward function r e) such p(O 1 Xoo x exp(3-1r(xx e)) with E R. and then the problem comes down to reward maximization. The proxy reward function may input the generated video Xo and prompt
Alignment as Stochastic Optimal Control Previous works formulate such a reward maximization
problem from the view of stochastic optimal control [29, 11, 30). where aim to find an additional
drift term :)::.) for the following reverse SDE: dz = f(zz, () v())Vv, log Pa (z) + u(z. 1)]dl g(()d.. For convenience, V log (v( because stochastic control is often based on the standard flow time (t 0 + T), and then the original SDE is re- re-written dy [[(2, t) u(z, t)] + g(f)dw, where dt here an infinitesimal time step and dw is a standard Wiener process.
adopt the change-of-variables as := Zr-t and f(v, () = f(v,
Because the alignment problem comes down to reward maximization, the objective in stochastic control literature is
u argmax E ((UT)) A T [u(v,.)l dl J g(t)²
u
where r'(-) == r(Dec(-)) evaluates the latent in the video space and A > O. E[1 is taken over sampling process above. In stochastic control, the optimal value function is known to defined as,
(((T)) /
T [ s)l2 ds ds|p, = g(s)2
(t) =Ep
where pi (v) 1 exp( (w) A !r? Tre (v). and obtain the optimal drift u" (v, = g( V of (v)
[31].
This optimal value function is the solution of stochastic Hamilton-Jacol Jacobi-Bellman equation [32] according to this Feynman- Kac formula [33, 34]:
" (1((2)
cxp ('((T)) |es -
exp
=
(3)
A
and then we obtain a tractable form of the optimal drift term as:
r' ('(UT)
u' (v, t) = V log Eprr
exp
1e =
(4)
The intuition here that the optimal drift pulls the current latent v, while following the pre-trained reverse SDE, into the region achieving higher reward at time T".

---

3 Diffusion Latent Beam Search
We first provide a unified view of existing inference-time alignment methods through several practical approximations of optimal drift u tv), (Section 3.1). To mitigate errors from approximations, we propose a novel search algorithm, diffusion latent beam search with lookahead estimator (Section 3.2).
3.1 A Unified View on Practical Approximations
While Equation 4 has a relatively tractable form, it is still computationally expensive, since the expectation requires complete diffusion sampling to evaluate the latent at each time step and face numerical instability. Previous alignment methods rely on multiple-step practical approximations.
Step. 1: Jensen's Inequality First, when assuming =((r) is almost deterministic (this might hold when t - T), Jensen's 's inequality yields the following approximation by exchanging log and E[:). which can be considered as a certain form of classifier guidance [35]:
(vi.t) 9(1) V.E (r (
(5)
Step. 2: Tweedie's Formula To avoid the computationally expensive expecta- tion, the expected reward is further ap- proximated as E,r= (r'(v)) r' (iTT)) where VTT = E Pr 1 u] is one-step approximation of poste- rior mean [36], which can be calculated only with the current latent without full diffusion path. Therefore, the op- timal drift term can be seen solely depending on the current time step t:
Algorithu Diffusion Latent Beam Search (DLBS) with Stochastic DDIM
Input: latent diffusion model eq., reward function r', noise scheduling parameter
(de))mo (gtt-pp number beams B, number of candidates K -~ Initial B beams
v| x
1: Z...... 1.
fort= tt= T do
3: for to do
+Compute the posterior mean of 1
5:
K/. vo.. (2] vI (1(c(((())))
< vor-Thuu v ae-t (22)(())
end for
if > 1 then
for j =
to B do
10:
Sample next candidate latents
I1:
2/_ x/_ axe with c!...., of (1 1)
u" (v,, g()) (UrT)
12:
Estimate the clean sample from noisy lntent
(6)
13:
s21 oie-1 vor-i (x] /I or 1x) 1))
14:
end for
15: Search higher reward beams from K B Intents
Such computationally tractable drift term has been leveraged for previ- ous inference-time alignment methods via approximate guidance or twisted sequential Monte Carlo (SMC) [37].
16: budget :- [(x!' 1. 1. " (x s8
17:
for y' to B do argiaxx Stasgee (2
18:
19:
budget budget \ 1(x angnnnsy end for
20:
21:
j (1..... B) + 3' b Reset selected B intices
However, as the approximated poste-
22:
end if
rior mean Prycin intermediate steps is noisy, evaluation with the reward func-
23:
end for
24:
return: xo www.an.eeeeee............ of) ((())
tion for clean data r')) may not pro-
vide a reliable signal [38]. Moreover, Equation 6 requires the reward gradient, which is not applicable to non-differentiable rewards, such as AI feedback, and is also not suitable for modalities whose reward gradient imposes huge computational cost in practice, such as video.
Step. 3: Converting Reward Gradient into argmax The usage of reward gradient can be converted
into argmax operator [11, 39, 40]. The intuition here is that since the optimal drift in Equation induces the diffusion latent to the direction where it maximizes the reward, we replace such maximization with a zeroth-order search. The SDE is approximated as:
dv = f(v,t)dt + g(t) dw" where dw* argmax (DT)))
Note that the current diffusion latents v, and posterior mean DTJ are sampled by following the standard Wiener process dw. This approximation is leveraged for inference- time alignment via greedy search [11, 39] or SMC [41] of diffusion latents. However, greedy search can result in sub-optimal generation affected by inaccurate reward estimate r' (((Dr) due to its noisy input. Moreover, it can be challenging to obtain an accurate density ratio term required in SMC for a high-dimensional domain, such video generation.

---

3.2 Mitigating Approximation Errors via Beam Search
Existing practical algorithms based on these three approximations, such greedy search [9], fall into sub-optimal generation due to the erroneous reward evaluation with a noisy estimate of the posterior mean [36], and argmax operator based on them. To resolve the error accumulation, we propose a simple yet robust modification, diffusion latent beam search (DLBS) with lookahead estimator. To clearly describe the practical implementation, we use the notation of discrete-time diffusion process in the rest of the section (see Appendix E for the continuous- timee diffusion process).
Practical Implementation We summarize the detailed sampling procedure of DLBS in Algorithm 1. For the diffusion sampler, we use stochastic DDIM [42] with a decreasing sequence {ar]== E (0, 1]", noise level 7), and noise schedule dz = n/( (1 0t- 1) - ae) v 1 a which is equivalent DDPM [23] when 7) = 1.0. We initialize B latent beams from the Gaussian distribution (Line 1.1), sample K latents per beam in the next time step (Line I.11), and then compute the one-step estimation of the posterior mean (Line 1.13). DLBS evaluates the estimator of posterior mean Zujt-1 with reward function (Line 1.18) and selects Top-B-rewarded latent beams instead of Top-1 (i.e., argmax) from K B candidates (Line 1.19). which is iterated over entire reverse process from = T to f= = 0. DLBS can possess latent beams more widely than greedy search under the same budget, which mitigates error propagation due to the approximated diffusion latent evaluation.
Lookahead Estimator The other source of ap- proximation errors than the argmax operator is a one-step estimator of the posterior mean 2oje-1 from Tweedie's 's formula, which is still noisy, es- pecially in earlier time steps, and leads to inac- curate reward evaluation. To reduce errors in reward evaluation, we propose a lookahead (LA) estimator - which is estimated by running -tes deterministic DDIM (1 < T << T) while equally interpolating the rest of time steps from the current latent 2 to Zo (Algorithm 2). While requiring additional denoising steps. its cost is almost the same as naive DLBS because most computational costs come from when decode zo (i.e., reward evaluation). Theoretically, enlarging the lookahead steps T" monotonically tightens the upper bound on the reward- reward-approste approoiimaiooon error (see Appendix F). Empirically, modest horizon (T' = 2,3, 6) delivers substantial search improvements (Figure 9; Left), but the marginal gains saturate, so pushing T' further yields little additional benefit (see Appendix M.4).
Algorithm Lookahead (LA) with Deterministic DDIM
Input: Intent diffusion model fg. current diffusion Intent Z,-1- number of lookabead steps T' [<<c T)
Ron s"-step deterministic DDIM starting from 7--1
2: i(e) 6 {t- 1-..... 1.....++100 (t- 111--... 1.- (+)))))),
3: Select new lonkahead noise schedule (6.)""> for T'sstep
interpolation of the rest of original for hra0
Zur: := Ze-:
5: Fojtery Mien - (T-piieattp)))
Vige
& for a = 2 to do
7: Xita-1) Va.-1ko Haa vI teolaaa)))))
Bojito-) Ric--s vI (olstel
Vo.
end for
I0: returm: (z, 1: 1: o Latent and T.A. estimator
4 Calibrating Reward to Preference Feedback
Human evaluation is one of the most valuable assessments for generative models, yet gathering human feedback at scale prohibitively costly. A practical approach to reduce the time and cost is to leverage AI feedback from VLMs [43], which has been shown to modestly align with human judgment on video quality [44-46] (see Appendix K). In this work, we assume that the VLM evaluation works as oracle, and we align model outputs with the preferences of VLMs, which is reasonable due to their capability and the cost to be saved. Our qualitative and quantitative evaluations also confirm that the highly rated video by VL.Ms is generally good for us.
However, because alignment via inference-time search requires massive reward evaluation queries, we still need to build more tractable proxy rewards that do not rely on humans or external VLM APIs. The question here is what metrics for perceptual video quality can improve the feedback from VLMs. Because the criteria of videos preferred by humans are multi-objective, maximizing a single metric may lead to undesirable generation due to over-optimization. For instance, focusing exclusively on temporal consistency or frame-by-free frame quality metrics can unintentionally reduce the video's 's motion magnitude (see Appendix H). this section, we first review the possible video quality metrics (Section 4.1), evaluate the Pearson correlation between these and the VLM feedback score, and then propose a reward calibration (Section 4.2), aiming to align the existing video rewards to VL.S by considering their weighted linear combination through the brute-force search of coefficients.
5

---

-0.05.p -0.00.p.0.00 0.06 8 -0.01, 101.p-00000 0.60
p=0.14,-<000
8 p=0.1.,00000
_ p=.00.0000001
R=0.41, p=0001
R=0466
Gemiri Scote
DEVIL- high
Subject Condistency
Smoothness
Dynamic Degree
Aesthetic Quality
Imaging Quality Vixe Consistency Calibrated Reward
Figure 3: 2D-histogram and correlation between reward functions for perceptual video quality [12] and Al feedback from Gemini [14]. single reward (e.g., subject consistency; blue) is often not aligned well with preference from Gemini, which happens for all the prompt sets with different dynamics grades (see Figure 13). The calibrated reward, a linear combination of perceptual metrics via brute-force search (green), achieves the best Pearson correlation coefficient in all settings (statistically significant with <, 0.01).
4.1 Metric Reward for Perceptual Video Quality
Following Huang et al. [12), we select six base reward functions for perceptual video quality (see Appendix C):
Subject Consistency quantifies how consistently the subject appears across video frames with DINO [47].
Motion Smoothness leverages the motion prior in AMT [48] to evaluate whether the generated video's motion is smooth and physically plausible.
Dynamic Degree quantifies the overall magnitude of dynamic object movement by estimating optical flow [49] for each pair of consecutive frames.
Aesthetic Quality measures compositional rules, color harmony, and the overall artistic merit of each video frame with LAION aesthetic predictor [50].
Imaging Quality assesses low-level distortions (e.g., over-exposure, noise, blur) in each frame with MUSIQ predictor [51].
Text- Video Consistency captures how closely the content video aligns with a prompt with
ViCLIP [52].
To reflect the multi- dimensional aspect of preferred videos, we model the calibrated reward function r" (-.)) as a weighted linear combination of video quality met-
Reward Calibration
Sublect
Motion
Dynamic
Assthetic
Imaging
Text Vidco
rics: r* *(xo, c) Ens wir,(xx (Xo, /)/ Σ MM We
DEVIL medium
DEVIL
MSRVTTest
The coefficient w; is determined by maximizing the Pearson correlation with preference feedback. We heuristically conduct a brute-force search within a reasonable range (Section 4.2).
The coefficient calibrated reward w, with feedback from Gemini. Each set of prompts, which has different dynamics grade, requires distinct mixture of perceptual video qualities.
Figure 4:
Experimental Setup We leverage Gemini- 1.5 [14] and GPT-4o [13] as automated raters for generated videos. We provide a prompt and generated video as inputs, instructing VLMMs assign discrete scores (from to 10) based on overall visual quality (e.g., clarity, resolution, brightness, and aesthetic appeal), the appropriateness of motion for either static dynamic scenes, the smoothness and consistency of shapes and motions, and the degree of aligmment with prompt (see Appendix I).
We select four prompt sets from two distinct datasets (see Appendix G). DEVIL [53] classifies its prompts into five categories depending on the dynamics grade, each further divided by subject type (e.g., cat, horse, truck, nature, etc.). We focus three of the five dynamics grades (high, medium, and static) and select one prompt randomly from each subject- subdiviii within a chosen category. We also draw 30 random captions from the test split of MSRVTT [54], widely used a video benchmark.
We generate 64 videos per prompt from pre-trained I Latte [18] using the DDIM sampler with T 50 and 1 0.0 to examine the correlation among AI feedback and perceptual quality metrics. We also prepare candidates for the calibrated reward by choosing the combination of weights w; E {0.0, 0.25, 0.5, 0.75, 1.0} and use those later rank them based on the correlation with AI feedback.
4.2 Correlation and Reward Calibration
Figure illustrates the 2D-histogram and the corresponding correlation between each metric and feedback from Gemini. Relying on single metric often yields low correlation, which supports

---

DEVIL-high
DEViLmediumm
DEVIL static
MERVITT-tst
DEVIL
DEVILmedium
612
628
9:4
(Budget 5cN/5-111
(Rudget
K-0 Dadgee
GS/3-13
BsAIX-11
(URS)bbwt)
Figure 5: (Left) Comparison among diffusion latent beam search (DLBS), best-of-N (BoN), and greedy search (GS). We measure the performance in terms of combinational reward calibrated to Gemini. DLBS improves all the calibrated rewards the best as the search budget K B increases (especially KB = 16, 32), while BoN and GS, in some cases, eventually slow down or saturate the performance. Notably, an LA estimator with small search budget (K B 8, T" 6) is comparable to or even outperforms DLBS (K B 32). (Right) Optimal balance between the number of latent K and the number of beams B under the same budget. For instance, as we increase the budget K 16, 32, we peak around K = 4,8, 8, 16, which is about 550 of the budget.
the multifaceted nature of perceptual video quality. See Appendix J. for further results, where we can see that the relative importance of each metric depends on the dynamics grade of the prompts: in highly dynamic DEViL-high, the dynamic degree correlates more strongly with VLMs than consisteney metrics. Conversely, subject consistency and motion smoothness play more prominent roles in less-dynamic DEVIL-medium or DEVIL-static. Because the aesthetic score focuses on frame-by-frame visual quality, tends to correlate strongly with VLM in low-motion scenarios. In high-motion scenarios, in contrast, rapid movements and frequent transitions often introduce motion blur or abrupt changes in composition, reducing the frame-level aesthetic quality and thus weakening its correlation with VLMs.
Reward calibration, weighted linear combination of these metrics, yields the highest correlation with Gemini (Figure 3, green). We select the best coefficients among brute-force candidates, based on the correlation with Gemini, for each set of prompts with different dynamics grade (Figure 4). Prompts with a high dynamics grade, i.e., DEViL-high, place greater weight on the dynamic degree. In contrast, prompts that describe slight motion, i.e., DEVIL-medium and DEVIL-statie, place a smaller weight on it. In addition, Appendix L presents results from best-of-64 sampling with a single metric or calibrated reward, where single metric often leads to over-optimization. This highlights the importance of reward calibration, appropriately weighting multiple criteria, as aligning with a request in the prompt.
5 Inference-Time Text-to-Video Alignment
Experimental Setup We use the same prompts and Gemini-/GPT-calibaate rewards as in Section 4. We compare the following inference-time search methods with a noise level 1) = 1.0 for DDIM:
Best-of-N Sampling (BoN): We initialize B latents and they follow the reverse process indepen- dently (K = 1). At = O, we evaluate the reward and select the best.
Greedy Search (GS): At each denoising step, we select the best-rewarded diffusion latent
(B = 1) from K candidates sampled in a reverse process.
* DLLS:: Given the budget K B, we sweep possible combinations in terms of power of 2 (e.g..
K = 8, B = 2), and report the best results except for the case with K = 1 and B = 1.
DLBS-LA: We combine DLBS with a lookahead estimator from the 6-step deterministic DDIM.
Our experiments aim to assess: (1) scaling the search budget and computational costs for efficient resource allocation (Section 5.1); (2) evaluating alignment performance with feedback from hu- mans and VLMs (Section 5.2); (3) assessing scalability to capable SoTA models (Section 5.3); (4) quantitative analysis on the diversity of generated video (Section 5.4); (5) validating that DLBS is complementary to fine-tuning methods (Section 5.5); (6) performing detailed ablations LA steps T", and robustness to diverse and complex prompts (Section 5.6).
5.1 Scaling Search Budget and Computational Cost
Figure 5 (Left) measures the combinatorial reward calibrated to Gemini while increasing the search budget K B E (1, 4,88 8, 16, 32). DLBS improves all the calibrated rewards the best as K B increases (especially K = 16, 32), while BoN and GS eventually slow down or saturate the performance in some cases. See Appendix M.I for results with GPT calibrated reward and Appendix M.2 for the results with K B = 64, where we still observe the improvement.
7

---

MSRVTT-test
MSRVTT-test
CogVideoX very--hry
2.1 MovieGen
Callbrato Roward
Calbrated Reward
Belake21
0.38
Gernini
0.55
0.8ST
Gem
0.48
Dus
LU
0.30
0.58
0.56
0.44
2 211 2
2
2
2
2
2
2
16
Inference Compute (NFE)
Inference Time (second)
< (Budgel)
K B (Budget)
Figure 6: (Left) DLBS achieves alignment performance gains more efficiently than BoN and GS under the same number function evaluations (NFE) or execution time. Increasing the search budget provides larger improvements under an equivalent computational cost than scaling the number of diffusion steps (DLBS-T: KB 8, T 100, 200). Employing the LA estimator (DLBS-A) further amplifies these gains. only with marginal overhead, yielding remarkably better efficiency than BoN GS. (Right) DLBS and DLBS-LA LA help the latest SoTA models, CogVidcoX-BB [19] and Wan 2.1-14B [20], improve the generated video quality.
DEVIL-high
DEViL-static
DEVIL-high
DEVIL-medium
DEVIL-stalee
MERVTT-EsSt
7.0
VidenScore
2.55
Score
6.5.
2.5
6.0-
2,05
2.45
2.4
5.0
5.0
8oN
DLBS
DURS
DLOS
K+ (Budgee)
K 6 (Budgee))
Figure 7: (Left)) DLBS calibrated reward can improve the original preference feedback from VLMs. As in
Figure 5, we use each calibrated reward (Figure 4) for DLBS and evaluate the quality with Gemini or GPT-4o. (Right) DLBS on calibrated reward also improves another qualitative metric, the most, VideoScore [55], which is not involved in a reward calibration.
Figure 5 (Right) demonstrates the scaling trend of DLBS, proportional to the search budget, under various choices of K. The results show that there is an optimal balance hetween the number of latent K and the number of beams B under the same budget. For instance, as we increase the budget to KB = 16, 32, we peak around K = 4, 8, 16, which is about 25-50% of the budget. This implies that balancing possession and exploration of diffusion latents in DLBS helps search for the best outputs robustly. See Appendix M.3 for further results.
We also analyze how alignment performance scales with different DLBS configurations under fixed computational budgets, using the number of function evaluations (NFE) and wall-clock time as cost measures. As shown in Figure 6 (Left),, DLBS consistently outperforms BoN and GS across both budgets, demonstrating superior efficiency in utilizing compute. Adding the LA estimator further amplifies this advantage, offering substantial performance gains with minimal overhead. In contrast, increasing the number of diffusion steps T (DLBS-T; KB = 8, T E (100, 200)) results in only marginal improvements despite the higher computational cost. Our results suggest clear strategy for inference-time budget allocation: prioritize enabling the LA estimator and increasing the search budget K / leads to substantial performance gains, while increasing the diffusion steps T provides limited benefit relative to its computational cost.
5.2 Evaluation with AI and Human Feedback
As discussed in Section 4, we obtain manageable reward function through the reward calibration, which reduces the cost for frequent evaluation queries in inference-time search. While DLBS efficiently improves the calibrated reward (Figure 5; Left), natural question is whether DLBS can improve an actual assessment by VLMs or humans by optimizing their calibrated rewards. We first use each calibrated reward for DLBS, then evaluate the quality using discrete scores (from 1 to 10) from Gemini or GPT-4o. Figure (Left) demonstrates that DLBS maximizing calibrated rewards can improve the original preference feedback from VLMs, we grow the search budget.
Next, we evaluate with VideoScore [55], a metric trained on human judgments that evaluates videos across five quality criteria. Figure 7 (Right) shows that DLBS significantly improves the quantitative evaluation based on human evaluation. Lastly, we perform pairwise comparisons between DLBS-LA (KB = 8, T" = 6, NFE 2500) and BoN (KB = 64, NFE = 3200) by three human evaluators (Figure 8; Left). The results confirm that, whatever models or prompts we choose, the quality of content generated by DLBS-LA consistently outperforms that of baseline despite requiring fewer NFEs. This emphasizes that our proposed method, integrating reward calibration and beam search in a latent space, effectively enhances perceptual video quality. See Appendix M. for the details.
8

---

Win Rate DLBS-LA against BoN
Calbbrated Reward
DEViL-high
DEViL-static
MSRVTT=test
Divt Leca
0.51
582
Gemina
0.86
Letn
0.75
5.62
0:72-
0.52
We'
0.30
8,200 2225 0.ta 272
0.71
0,350 6.315 9.43 3.425
0.755 6.150 0.02
6:36 0.32 0.31
0%
25% 30%
73%
102%
VICLIP MPD
VICUP MPD
VICLIP MPD
VICLI MPD
Win
Lose
BoN(K=1)
GS(B=1)
DLBS
DLBS-LA
Figure 8 (Left) The pairwise comparisons of DLBB-LA (K = 8, T = 6, NFF = 2500) and BoN (KB = 64,
NFF = 3200) by three human raters, when DIBBS-A searches effectively for the hest latents in diffusion process. (Right) Alignment- diversity tradeoff (KB 32). The mean pairwise distance (MPD) of ViCLIP embeddiing is used as a measure of diversity. DLBS and DLBS-LA (T' 6) achieve high performance while maintaining higher diversity.
Table I: Performance of DLBS with DPO finetuned VideoCrafter2 on DEVIL-hip and MSRVTT-test datasets. While DPO alone yields marginal improve- ments, combining it with DLBS leads notable gains, demonstrating the compatibility inference- time search with fine-tuning approaches.
Callbrgtee salutatdd Roseard
MSRVTT-test
S Reward Reward
1,003 pmoppe from MouleGes
2.80
0.400
E:3318
Camini
0.50
Mark
Genini
0.375
2.50
0.350-
0.325-
Figure 9: (Left) Comparison of different LA steps "" on MSRVTT-test (KB = 8). The performance improves as the number of L.A steps increases. (Right) Perceptual quality comparison on a large and complex prompt set (1,003 prompts from Movie Gen Video Bench [57]). DLBS-LA can generalize to diverse prompts.
Method
DEVIL-high MSRVTT-test
VideoCrafter2
0.337
0,555
+ DPO
0.335
0.556
+ DPO & DLBS
0.359
0.576
5.3
Scaling Model Parameters and Capabilities
We scale up the base diffusion model from Latte to the latest SoTA models, such as Cog VideoX- SB [19] and Wan 2.B-14B [20], and evaluate if DLBS can improve the generation quality from those larger models. Note that we use DDE-DDMSSllve+++ [56] for Wan 2.1 experiments. Since base models more capable here,
adopt more challenging prompts from DEVIL-very-high! (22 prompts) and Movie Gen Video Bench [57] (20 prompts). See Appendix G and Appendix J.2 for the details. Figure (Right) shows that our methods achieve significant improvements in calibrated reward for both models. As shown in Figure 8 (Left), human evaluation also supports that our proposed method could generally work well with any text-to-video models, even with more capable models in the future. See Appendix M.9 for results with maximum frame length, and Appendix M. 10 for further results with CogVi VideoX and Wan 2.1.
5.4 Alignment-Diversity Tradeoff
Alignment for diffusion models can steer desirable outputs, but it is said that the diversity of generated samples or the performance of original models often degrade [10, 37]. While inference-time search does not change or degrade the model itself, we here compare the diversity of samples among BoN, GS, DLBS. and DLBS-LA. We measure the sample diversity
the mean pairwise distance of ViCLIP [52] embeddings (see Appendix D). Figure 8 (Right) reveals that DLBS and DLBS-LA achieve high performance with higher diversity than BoN or GS. This exhibits benefit from the wider possession and exploration of diffusion latents in DLBS and DLBS-LA.
5.5 DLBS is Compatible with Finetuning
In image generation, Ma et [17] has shown that allocating additional computation at inference time can be more effective than relying solely on post-training approaches. We find similar trend in video generation. To examine this, we apply a representative fine-tuning method, VideoDPO [58], to VideoCrafter2 [3]. As shown in Table 1, VideoDPO alone brings negligible improvement over the baseline. However, when combined with DLBS, the performance increases substantially on both DEViL-high and MSRVTT-test. These results indicate that DLBS is complementary to post- training methods, enabling further performance gains even after fne-tuning See Appendix M.7 and Appendix M.8 for results comparing DLBS with other alignment methods.

---

5.6 Ablation Study
Lookahead Steps for Reward Estimate We scale up LA steps T' to obtain an accurate reward estimate. We use MSRVTT-test (KB = 8) and Gemini reward for experiments. Figure 9 (Left) shows that the number of LA steps increases, the performance improves more. Even T 2.3 significantly outperforms the posterior mean, which is often used in prior works [15, 11, 41]. This is because the sub-optimal performance of inference-tine search comes from the approximation errors, and LA estimator can notably reduce them. As shown in Figure 5 (Left), DLBS-LA (KB 8, T 6) achieves comparable or even outperforming results with DLBS (K B 32). It is quite beneficial to spend computation to estimate the reward accurately. See Appendix M.4 for further discussions. In addition, an ablation study on diffusion steps is shown in Appendix M.5.
A Large and Complex Prompt Set To confirm the robustness of our approach on large and highly diverse prompt set, we compare GS B=1) with DLBS-LA (K T' with all the prompts in Movie Gen Video Bench [57], which comprises 1,003 prompts (Figure 9; Right). We observe that DLBS-LA consistently delivered substantially higher alignment rewards, demonstrating that DLBS generalizes effectively to complex prompt distributions. Additionally, we also assess reward transferability by applying weights calibrated on DEVIL-high and DEVII--medium MSRVTT-test prompts, consistently improving scores (see Appendix J.4).
6 Related Works
Classifier guidance [35, 59] has been the most popular to enhance text-content alignment. On top
of that, recent works [11, 39] leverage reward or external feedback inference time by selecting better latents [60]. which probably achieve higher rewards during the reverse process. Kim et al. [61] propose twisted SMC [37] with reward gradient, which is not suitable for non-differentiable feedback and for domains such as video, where reward gradient needs a huge memory cost. Gradient-fres methods [62, 17] such as SMC [41] greedy search [15] often exhibit sub-optimal results affected by inaccurate reward estimates from noisy latents. Yeh et al. [63] uses ODE to estimate the reward, but it highly depends on Karras sampler [28] avoid numerical instability. In contrast, we address the error propagation from inaccurate reward estimates with beam search and lookahead estimator via deterministic DDIM, which is more popular and stable. Our methods work more scalably when allocating more computation budget at inference time. See Appendix N for further related works.
7 Discussion and Limitation
Our reward calibration assumes that VLMs serve as a proxy for human evaluation, and we demonstrate both qualitative and quantitative improvements in video quality through evaluations by VLMs and human raters. In future work, incorporating more specialized and accurate evaluators (e.g., reward models that focus on physical laws [9]) could enable more fine- iin-igrined analysis. In practice, often do implicit or explicit best-of-N sampling for video generation. In contrast, DLBS-LA exhibits much better computational efficiency. Spending more computation at inference time significantly improves perceptual quality, but it is orthogonal compared to speeding up the sampling process via distillation [64, 65], architecture changes [66], or parallel sampling [67]. We believe both high-quality and speedy sampling have practical needs and should be balanced.
8 Conclusion
This paper studies which metrics we should optimize and how to optimize them for better text- to-video generation. We point out that feedback from humans or capable VLMs reflects multiple dimensions of video quality, so optimizing an existing metric alone insufficient; rather, we should calibrate the reward by combining. Our DLBS with LA estimator reduces the error propagation from the inaccurate reward estimate. We demonstrate that DLBS is the most scalable, efficient, and robust inference-time search that significantly improves video quality under the same computational costs. We hope our work encourages more uses of inference-time computation for text-to-video models.
10

---

Acknowledgements
We thank Po-Hung Yeh, Shohei Taniguchi, Kuang-Huei Lee, Arnaud Doucet, Heiga Zen, Robin Scheibler, and Yusuke Iwasawa for their support and helpful discussion the initial idea of this work. HF was supported by JSPS KAKENHI Grant Number JP22J21582 (by March 2025), and MS was supported by JSPS KAKENHI Grant Number JP23H04974. We also appreciate the funding support from Google Japan.
References
[2] Google DeepMind. Veo 2, 2024. URL wwwmiidee//oorl.../
[1] OpenAl. Sora, 2024. URL htt.httt///ppeaii.coom wwt.cooommmmcc
veo-2/.
[3] Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. arXiv preprini 2rXiv:2401.09047, 2024.
[4] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 2023.
[5] Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holded Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, al. Genie: Generative interactive environments. arXiv preprint arXiv:2402.1539) 2024.
[6] Junchen Zhu, Huan Yang, Huiguo He, Wenjing Wang. Zixi Tuo, Wen-Huang Cheng, Lianli Gao, Jingkuan Song, and Jianlong Fu. Moviefactory: Automatic movie creation from text using large generative models for language and images. In Proceedings of the 31st ACM International Conference on Multimedia, pages 9313-9319, 2023.
[7] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators, 2024. URL https://openai. com/research/ video-gent cetooo-gnrraiionnoe connerationmmodess--ae aasss-aswwwe as-warld-------------ao--- was-world-simulaae sim-latorss......
[8] Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physies-grounded image-to-vider generation. In European Conference on Computer Vision (ECCV), 2024.
[9] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang. and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. arXiv preprint wxXiv:2406.03520, 2024.
[10] Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback. arXiv preprini arXiv:2302 /2192, 2023.
[11] Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with non-differentiable rule guided diffusion, 2024.
[12] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
[13] OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.

---

[14] Gemini Team. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. 2023.
[15] Jaemin Kim, Bryan S Kim, and Jong Chul Ye. Free* guide: Gradient-free path integral control for enhancing text-to-video generation with large vision-language models. arXiv preprint arXiv:2411.17441 2024.
[16] Ziyi Zhang, Sen Zhang. Yibing Zhan, Yong Luo, Yonggang Wen, and Dacheng Tao. Confronting reward overoptimization for diffusion models: A perspective of inductive and primacy biases. In Forty-first International Conference on Machine Learning. 2024.
[17] Nanye Ma, Shangyuan Tong. Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, and Saining Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.097:2 2025.
(18) Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048 2024.
[19] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, others, and J. Tang. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint 266072 2024.
[20] WanTeam, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang. Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang. Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.2000
2025.
[21] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. arXiv preprint arXiv:2112.10522. 2022.
[22] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 2256-2265, 2015.
[23] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851, 2020.
[24] Yingqing He. Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fieeltyy long video generation. arXiv preprint arXiv:2211. 1. 2022.
[25] Andreas Blattmann, Robin Rombach, Huan Ling. Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[26] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1rXiv:1312.6114, 2013.
[27] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
(28) Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Flucidating the design space of diffusion-based generative models. arXiv preprint arXiv:2206.003: 00364, 2022.
12

---

[29] Masatoshi Uchara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine- tuning of continuous-time diffusion models as rntropy-regularized control. arXiv preprint arxiv:2402. 15194, 2024.
[30] Carles Domingo-Enrin Michal Drozdzal, Brian Karrer, and Ricky T. Q. Chen. Adjoint matching: Fine-tuning flow and diffission generative models with memoryless stochastic optimal control. arXiv preprint arXiv:2409.0886 08861, 2024.
[31] Michele Pavon. Stochastic control and nonequilibrium thermodynamical systems. Applied Mathematics and Optimization, 19:187-202, 1989.
[32] Lawrence C. Evans. Partial differential equations. American Mathematical Sociery, 19, 2022.
[33] Bernt Øksendal. Stochastic differential equations. Springer, 2003.
[34] Pierre Moral. Feynman-kaa formulae: genealogical and interacting particle systems with applications. Springer, 2004.
[35] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, 2021.
[36] Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. The Eleventh Interna- tional Conference on Learning Representations, 2023.
[37] Luhuan Wu, Brian L. Trippe, Christian A. Naesseth, David M. Blei, and John P. Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. arXiv preprint arXiv:2306.1779 17775, 2024.
[38] Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang. Ji Li, and Liang Zheng. Step-aware preference optimization: Aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.00 043 14, 2024.
[39] Xiner Li, Yulai Zhao, Chenyu Wang. Gabriele Scalia, Gokcen Eraslan, Surag Nair. Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, and Masatoshi Uchara. Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding. arXiv preprint MrXiv:2448.082555 2024.
[40] Arpit Bansal, Mong-Min Chu, Avi Schwarzschidd, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. arXiv preprint 27Xiv:2302.07121, 2023.
[41] Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.066
2025.
[42] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[43] Xun Wu, Shaohan Huang, and Furu Wei. Multimodal large language model is a human-aligned annotator for text-to-image generation. arXiv preprint arXiv:2404.151000 2024.
[44] Sanghyeon Na, Yonggyu Kim, and Hyunjoon Lee. Boost your own human image generation model via direct preference optimization with ai feedback. arXiv preprint arXiv:2405.202 2024.
[45] Xun Wu, Shaohan Huang, Guolong Wang, Jing Xiong, and Furu Wei. Boosting text-to-video generative model with MLLMs feedback. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[46] Hiroki Furuta, Heiga Zen, Dale Schuurmans, Aleksandra Faust, Yutaka Matsuo, Percy Liang, and Sherry Yang. Improving dynamic object interactions in rext-to-video generation with ai feedback. arXiv preprint arXiv:2412.02617, 2024.
13

---

[47] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv: 14294, 2021.
[48] Zhen Li, Zuo- Zuo-Liang Zhu, Ling-Har Hao Han, Qibin Hou, Chun-Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[49] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. arXiv preprint arxiv:2003. 12039, 2020.
[50] LAION-AI. ceathhetic--rediitoo
aesthetic-predictor 2022.
URL https:///ittub..comm/
[51] Junjie Ke, Qifei Wang, Yilin Wang. Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. arXiv preprint (rXiv:2108.05997 2021.
[52] Hu Xu, Gargi Ghosh, Yao Huang, Dmytro Okhonko. Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. Videoclip: Contrastive pre-training for zero- shot video-text understanding. arXiv preprint arXiv: 109.4444094 2021.
[53] Mingxiang Liao, Hannan Lu, Xinyu Zhang, Fang Wan, Tianyu Wang, Yuzhong Zhao, Wang- meng Zuo. Qixiang Ye, and Jingdong Wang. Evaluation of text-to-video generation models: A dynamics perspective. arXiv preprint
2024.
[54] Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vit: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattem recognition, pages 5288-52966 2016.
[55] Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang. Quy Due Do Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Yuchen Lin, and Wenhu Chen. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv: 15252, 2024.
[56] Chuhui Lu, Yuchen Zhou, Feng Bao, Jian Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022.
[57] Meta Movie Gen team. Movie gen: A cast of media foundation models. https//aaxivoor//as22410....
2024.
[58] Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei. Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni prefereeee alignment for video diffusion generation. arXiv preprint arXiv22422.14167, 2024.
[59] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. vrXiv:2207.12598 2022.
arXiv preprint
[60] Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent optimization improves classifier guidance. arXiv preprint 2tti:2303313705 2023.
[61] Sunwoo Kim, Minkyu Kim, and Dongmin Park. Test-time alignment of diffusion models without reward over-optimization. In The Thirteenth International Conference on Learning Representations, 2025.
[62] Hongkai Zheng, Wenda Chu, Austin Wang, Nikola Kovachki, Ricardo Baptista, and Yisong Yue. Ensemble kalman diffusion guidance: A derivative- ffree method for inverse problems, 2024.
[63] Po-Hung Yeh, Kuang-Huei Lee, and Jun- Mun-Chen Chen. Training-fres free diffusion model alignment with sampling demons. arXiv preprint arXiv:2410.05760, 2024.
[64] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. arXiv preprint 222.x:2220.0014.. 2022.
14

---

[65] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consisteney models, arXiv preprint arXiv:2303.01469 2023.
[66] Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, and Yutaka Matsuo. Ssm meets video diffusion models: Efficient long-term video generation with structured state spaces. arXiv preprint arXiv:2403.07711, 2024.
[67] Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. In seirth- Conference on Neural Information Processing Systems, 2023.
[68] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Jourmal of Machine Learning Research, 21(140):1-67, 2020. URL http://jmlr.. http://ww..rggppraaaaeee prrg/papers/221/00- 074 html.
[69] Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023.
[70] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint 20XI::0010.11929: 2020.
[71] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00022 2021.
[72] Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. Perceptual quality assessment of smartphone photography. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[73] G. Li, Y. Huang, T. Efimov, Y. Wei, Y. Chi, and Y. Chen, Accelerating convergence of score- based diffusion models, provably. arXiv preprint arXiv:2403.03852, 2024. URL https: / //arxiv. www.orgg///////ooooo//..o..eee// /403. 03852.
[74] Jay Zhangjie Wu, Guian Fang. Haoning Wu, Xintao Wang, Yixiao Ge, Xiaodong Cun, David Jun- hao Zhang, Wei Liu, Yuchao Gu, Rui Zhao, Weisi Lin. Wynne Hsu, Ying Shan, and Mike Zheng Shou. Towards better metrie for text-to-video generation. arXiv preprint arXiv:2401.0778/ 2024.
[75] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
[76] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint axXiv:2106.09685 2021.
[77] Özgün Çięek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: Learning dense volumetric segmentation from sparse annotation. arXiv preprint arXiv:1606.06650, 2016.
[78] Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavanzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine- tuning text-to-image diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[79] Yinan Zhang, Erie Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. arXiv preprint arXiv:2401.12244, 2024.
15

---

[80] Huizhuo Yuan, Zixiang Chen, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning of diffusion models for rext-to-image generation. arXiv preprint arXiv:2402.10210, 2024.
[81] Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Qimai Li, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. arXiv preprint axXh::231.133331 2023.
[82] Bram Wallace, Mcihua Dang, Rafacl Rafailov, Lingi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong. Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. arXiv preprint 1rxiv:3311.12900 2023.
[83] Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun SHUM, and Tong Zhang. RAFT: Reward ranked finetuning for generative foundation model alignment. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
[84] Mihir Prabhudesai, Russell Mendonca, Zheyang Qin, Katerina Fragkiadaki, and Deepak Pathak. Video diffusion alignment via reward gradients. arXiv preprint urXiv:2407.08737. 2024.
[85] Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion models on differentiable rewards. In The Twelfth International Conference on Learning Repre- sentations, 2024.
[86] Jiachen Li. Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sugato Basu, Wenhu Chen, and William Yang Wang. T2v-turoo: Breaking the quality bottleneck of video consistency model with mixed reward feedback. arXiv preprint arXiv:2405. 11.4405..77000 18750, 2024.
[87] Zipeng Qi, Lichen Bai, Haoyi Xiong, and Zeke Xie. Not all noises are created equally:diffusion noise selection and optimization. arXiv preprint arXiv:2407. 140444 2024.
[88] Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Jaewon Min, Minjae Kim, Wooscok Jang, Hy- oungwon Cho. Sayak Paul, SconHwa Kim, Eunju Cha, Kyong Hwan Jin, and Seungryong Kim. A noise is worth diffusion guidance. arXiv preprint arXiv:2412.03895 2024.
[89] Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint arXiv:2411.09502, 2024.
[90] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions Image Processing, 13(4):600-612, 2004.
[91] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. arXiv preprint 160iv:1606.03490 2016.
[92] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as perceptual metric. In CVPR, 2018.
[93] Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. 2019.
arXiv preprint
[94] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton. Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield- Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedback. arXiv preprint 1rXiv:2212.000 2022.
16

---

[95] Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXivv:2009.0 00267, 2023.
[96] Hiroki Furuta, Kuang-Huu Huei Lee, Shixiang Shane Gu, Yutaka Matsuo, Aleksandra Faust, Heiga Zen, and Izzeddin Gur. Geometric- preference optimization for soft preference labels. arXiv preprint (XXi:244990069)) 2024.
[97] Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, and Rongrong Ji. X-clip: End-to- end multi-grained contrastive learning for video-text retrieval. arXiv preprint arXiv:2207.07285, 2022.
17

---

A Broader Impacts
Our work contributes the progress of text-to-video by focusing on improving the perceptual quality and fidelity of generated videos, specifically addressing issues like unnatural movement, deformation, and temporal inconsistencies, through the inference-time alignment algorithm. Such advancements hold immense potential for revolutionizing creative fields and enabling new applications in gaming, filmmaking, and robotics.
On the other hand, the ability to generate highly realistic videos raises concerns about the potential for misuse in creating deceptive content, including deepfakes and misinformation. Like other generative models, such as large language models, text-to-video models, and their inference-times time search, may inherit and amplify biases present in the training data due to the misalignment. This might lead the generation of videos that perpetuate harmful stereotypes or underrepresent certain groups.
Lastly, by focusing inference- time alignment, method promotes more of computational resources at test time. On one side, this may increase the environmental footprint for running large generative models and on the other side, detailed recipe can contribute to designing efficient use of resources and reducing the footprint associated with training. We believe that discussing this aspect crucial as the scale of these models continues grow.
B Implementation Details
Code Our implementation for the experiments are available at https:///itthu www/shim0m11 DyV-Diffuiion- Search.
Models Our experiments cover three text-to-video diffusion models.
Latte [18]: a T5-conditioned latent diffusion transformer with 1.1 B parameters, built on PixArt- a [68, 69]. CogVideot [19]: larger DiT-based model with 2B or 5B parameters. We mainly used VSg
Wan 2.1 [20]: a DiT-based flow model with 1.3B or 14B parameters. We use Wan 2.1-1.3B for reward calibration and Wan 2.1-14B for inference-time search experiments.
pyperparameters
Latte: DDIM scheduler with linear noise schedule (Brar = 1.0 10-4, Bend 2.0 10-2) and classifier-fie guidance scale Widg = 7.5.
CogVideoX DDIM scheduler with the original settings and W/dg 6.0. Owing to computational resource constraints, we limit the frame length to 17 per sample."
Wan 2.1: DPMSolver++ with guidance scale Wefg 5.0. For the same computational reasons, the spatial resolution is limited to 832 x 480 and the frame length to 33 per sample.
Hardware configuration
Latte: FP16 inference on single NVIDIA A100 (40 GB), batch size
Cog Vi BF16 inference on a single NVIDIA 100 (40 GB), batch size 1.
Wan 2.1: FP16 inference on four NVIDIA H100s (80 GB each), batch size 1.
Al-feedback endpoints
We use API endpoints: gemini- 1. wwt.iniii..--------.--- 1o-po-0022 and 2ptt-4o-2224-11
"The experiments reported in Appendix M.9 conducted with different number of frames.
18

---

C Details of Metric Rewards
Subject Consistency We adopt the subject consistency metric proposed in VBench [12] to quantity how consistently subject is depicted across consecutive video frames. Concretely, for each frame i in a video, we extract a feature representation d; using DINO [47] with a ViT-B/16 [70] backbone. Let (d, d,) denote the cosine similarity between the features d, and dy. Then, VBench defines the subject consistency metric as follows:
Raubject
(d), de) + (d--1: di)).
(8)
T
2
DINO, which is trained in self-supervised manner using unlabeled images and image augmentations, does not explicitly suppress intra-class variations. As a result, it remains particularly sensitive to identity shifts within the same subject, making it well-suited for evaluating subject consistency across frames.
Motion Smoothness We adopt the caane-interpolation-biiee VBench [12] to assess whether a generated video's motion smooth and physically plausible. In particular, this metric leverages the motion prior from AMT [48], employing its AMT-S variant for frame reconstruction. Concretely, let [f. f, .a......fee fen denote the frames of a generated video. We f2n]. and rely on AMT-S to reconstruct the missing frames [f,, fs. f We then compute the Mean Absolute Error (MAE) between these reconstructed frames and the original odd-numbered frames, denoting this measure by Ramoothocss. Finally, following the normalization scheme introduced in VBench, we define:
metric originally proposed in
remove each odd-numbered frame to obtain a lower-frame-rae sequence
255 Rsmoothness
smoothness-nom
255
which ensures that the final score lies in the range [0, 1]. with higher values indicating smoother motion. This measure leverages the motion prior in AMT to evaluate whether the generated video's motion is smooth and physically plausible. We remove each odd-numeered frame, then use AMT-S to reconstruct those frames based on short-term motion assumptions.
Dynamic Degree This measure quantifies the overall magnitude of dynamic object movement. Let be the total number frames in the generated video. For each pair of consecutive frames and ++ 1, we estimate the optical flow v, using RAFT [49], compute its norm |vell, and sum these values across all frames:
Rdynamics
|vell-
(10)
We then apply logarithmic transformation Raynamics and divide by 16:
Rdynamics-rescaled log Rdynamies)
(11)
16
This rescaling helps ensure that the value range of Roynamics-scaled is roughly comparable to other metrics in evaluation.
This criterion evaluates compositional rules, color harmony, and overall artistic merit on a per-frame basis. Concretely, for each frame i in a video, we extract CLIP image embedding c image using the CLIP ViT-L/14 model [71]. We then feed emmage into the LAION aesthetic predictor [50], which assigns a raw rating r [0, 10). normalize these scores to [0, range, set
Aesthetic Quality
(12)
Let be the total number of frames. The final aesthetic reward is then obtained by taking the average of the normalized ratings across all frames:
Raesthetic
(13)
19

---

Because the LAION aesthetic predictor leverages CLIP embeddings instead of raw images, it captures higher-level features related composition, color harmony, and artistic appeal.
Imaging Quality This indicator assesses low-level distortions (e.g., over-exposure, noise, blur) in each generated frame. We adopt the MUSIQ predictor [51], trained the SPAQ dataset [72]. The frame-wise score is normalized to [o, I] by dividing by 100, and the final video score is the mean of these normalized values across all frames in the same way Equation 12 and Equation 13.
Text-Vides Consistency This measure captures how closely generated video's content aligns with its text prompt. We employ ViCLIP [52], model pre-trained fine-tuned to handle temporal relationships, to embed both the video frames and the text. Since ViCLIP computes embeddings from 8-frame inputs, sample 8 frames from each video. Let v viddo denote the resulting video embedding and ylxt denote the text embedding. We then define the final alignment as the cosine similarity between these embeddings:
10M video-text dataset and
P Reyeoossisencyy
video ylett
(14)
D Details of Sample Diversity
We measure the sample diversity as the mean pairwise distance of VICLIP [52] embeddings to quantify the diversity in videos, inspired by approach for evaluating diversity in images [61]. Specifically, given N generated video samples, first extract VICLIP embeddings viieo, for each sample i. The pairwise diversity score is then computed the mean pairwise distance:
D Dridso-diversity
(11(vviddee videoo, (+) vyilen.())))
(15)
N(N 1)
Here, (i), (i)) denotes the cosine similarity between the VICLIP embeddings of two generated videos and j. This formulation is similar to Equation 14, but in the case of pairwise distance computation, we take the pairwise mean of 1 (cosine similarity) to obtain a diversity measure.
20

---

E Algorithms with Continuous-time Diffusion Process
In this section, we present our algorithms for continuous-time diffusion process. For Wan 2.1 [20]. we integrated the proposed search algorithm into DPMSolver++ [56], widely used continuous-ting solver. Algorithms and present the pseudocode. Although we present the first-order variant for clarity, the procedure extends straightforwardly to higher-order formulations. Throughout this section, we adopt the notation of Lu et [56].
Algorithm Diffusion Latent Beam Search (DLBS) with SDE-DPMSolver++
Input: signal prediction latent diffusion model 2g. reward function r', time steps {t, 1 noise scheduling parameter {at, [0t {or,) 1. number of beams B, number of candidates K
1: zi 2 ~ (0, I) > Initial B beams
2: for
M do
3: for to B do
4: Update one step to produce z!,
5:
de md + +a,, (1-e 220-22)200
01,-1
6:
end for
7:
if < then
8:
for j to
B do
9:
Sample K next candidate latents
10:
2, = L + +o,ee -Ei_, with cl,_, ot ~ I(0.))
11:
> Estimate the clean sample from noisy latent
12:
5 tarle,
Jin 2 o,
-rx - (z,)))
13:
end for
14:
> Search higher-reward beams from K latents 1. "..,,. (z
15:
budget := (z!) (trlt,))
KB
2 2KB (xrlt,))) }}
for j' B do
17:
z = argmax, 2 Ebudget (2 tarlt))
18:
2argmax) budget = budget {zi, tm end for
19:
20:
j E (1, B} j Reset selected B indices
21: end if
end for
23:
return: ZIM = argmax,
f Elet clt a r' (z(
Algorithm 4 Lookahead (LA) with DPMSolver++
Input: signal prediction latent diffusion model zg, current diffusion latent 2,, number of lookahead steps M'(<< M)
1: > Run -step deterministic DPMSolver++ starting from Z " 2: s(u) E fs...., M' + M )...,,,, M s M
M² M. M}
3: Select new lookahead noise schedule {ä,, M for -'tstep interpolation of the rest of original
{ar,) M M do
4: Zt = Zt,
5: for
destu) i1 6: Tesco Paca- Jescu 1)
alss (e 1)z0 ())))))))))
7: end for
8: return: (ze, 2tx Axcon ) Latent and LA estimator
21

---

Theoretical Analysis on Lookahead Estimator
Consider the Lookahead estimator described in Algorithm 2, which obtains the state (0) by performing 7" steps DDIM (or another diffusion-based sampling) with 1/ = 0.0. Our goal is to show that, as 7" grows, the reward estimate r' (Z))(()))) converges to r' (zo). thereby improving estimation accuracy.
Let Z be state in the latent space from which wish recover the initial latent Zg. By applying T" steps of DDIM with = 0.0, we obtain an approximation 20jā10)- From prior work [73], the error Zoftt0) zoll scales as follows:
Oa/T)
(DDIM),
2 ((O) zoll -
OL/vTT)
(DDMM),
((1((T)))
(an n-th order solver).
Hence, increasing 7" yields progressively better approximation of Zg-
Assume Zo is the latent representation at time = 0. By the Continuous Mapping Theorem, if Z0) + Zo as T" co, then for any continuous function f. we have
(()))((0) - f(zo).
Setting f(-) = =+),),, where y' is reward model, yields
r' (Z))))())) + r'()),
T 0o.
We further assume that the reward model r'() Lipschitz continuous with Lipschitz constant L. Then for any two latent states Zo and Zö. the reward estimates satisfy
r (za) r' (z+)) < L||z. zb
Hence, the order of the error in r' (2) tracks the order of the error in Zij((o) itself. Explicitly,
r' (Z - r' (zo) < L ( zoll.
implying that an (1/7T") (or better) approximation in latent space implies an O(1TT") (or corre- spondingly better) approximation in the reward space.
As T" increases, Zojz(u) converges to Zo, and consequently (zo)))))) converges to r'(z.. Because the reward model is Lipschitz continuous, this convergence ensures that the error in reward estimation decreases at the same order as the eтror of the latent approximation. Therefore, employing the LA estimator with larger T" yields more accurate reward estimate.
22

---

G List of Prompts
MSRVTT-tstt
singing stage one person being she
someone is filming parked car in the parking lot
cat food it babics rabbit
mario game with bombs
someone is browsing set of games on their console
game being played
holds very large stick
ycllow-baired girl is explaining about game
ship sailing
water
10. with blonde hair and black shirt talking
11. buffalo attacking a man people dancing
playing
13. child playing vidco
14. person is showing how to fold paper
15.
sitting
couch
16. man inside of a car is using his finger to point
17. man waters his plants
18. the symmetrical cone japan famous symbol
19. an indoor soccer game
20. japanese monkey bothing in spring with pleasant music
21. images of motorcycles being shown
22. someone is serving food in the restaurand
competition type
24. woman the news is talking about story
25. this phone review vidco
standing
game
27. person is filming white car interior seat
28. video clips from movie
with blue and white shirt walking around
30. person making something in the kitchen
DEVIL-high
baokshelf collapses books flying everywhere, creating chaos in the quiet
Swift scenes of sandstorm engulfing desert oasis, with dunes shifting and palm trees hending in the relentless wind.
A chaotic seene cowboys rounding up cartle during stampede.
Suklenly, hits the city, rain pouring down like tarreni, making rivers the streets.
WWI biplanes in doglight with canvas wings ripping, dramatic cloud backdrop, dltraedetiiled.
snow, creating large cloud powder: Amidst chunderstomm, lightning bolt strikes bicycle. setting it ablaze with crackling energy and lighting up the dark, rainy street.
mountains, erupts
A single cagle dives extremely fast, snatching fish from the water.
big wave flips, landing upside down.
10. A drives through wall of fire in daring escape.
11. The cat tore across the living room, jumping over toys and furniture to catch the mouse.
cow jumps
landing pond a big splash.
13. Twa dogs chase each other, suddenly skidding around sharp comer.
14. A storm sweeps un elephant into riging river, carrying away swiftly.
15. Racing the
giraffe charges
the horizon, shadows stretching long.
16. Against the wind, a lone horse gallops, mame strearning behind.
17. Jumping over gorge, the motoreycle lands just time on the other side.
18. thief sprints away from the with the police in hot pursuit.
19. The icc cracks beucath their feet, making the sheep skid and slide, rushing to solid ground.
20. Lightning strikes as a train blasts horn, cutting through stormy night.
21. truck speeds
devert, dust clouds swirling behind it.
22. Under rainhow, zebra kicks spray of water as it crosses a fast- fast-flowing river.
23

---

DEVIL-medium
London heathrow, united kingdom 05 12 2019: super-telpphoto plane accelerates down rumway through hear shimmer A coal dj teddy bear with sunglasses on top of turniable with video static
Aerial view. cute girl in the cost drive on country mad on the bicycle
Brown pelican flying flight fall hay hurbor in ecuador
fishing boatt,
ocean, thailand.
filled yellow school bas with over over-sized sized black wheels drives through flooded area with red lights on and gets splattered with mud
7. St. petersburg, russin circa march, 2015: vehicles drive city ringroad at evening time. st. petersburg ring road main route encireling the city
cal manages hang on dangling abject
Taking cow milk cheese with fork 4k footage
10. passes and out of view
1930s: elephant rours, mon
elephant. elephants through jungle.
gun, throws gun on ground, runs away.
12. the bahy giraffe roomed on and then camera shakes
13. Cowboys drive group of horses at farming enterprise.
14. 4k couple watching film at home & jumping with shock the action
are reading themselves start mini-motorbike
16. Macao beach with stone mountains acrial view from drone. travel destination. summer vacation. dominican republic
17. Male bexer resting and sweating after boxing training
18. Wild tulips in meadow background sky, suarise. bonfire. quict spring moming in the steppe.
19. Sheep eating grass punata and potosi, bolivia.
20. arctic town norway ca july 2018: train station bailding and tilt up
21. woman is describing different sets of tubes and hoses in the back of white pick truck which is parked on the side of a street with cars going by the background
22. Istanbal, turkey- - october 2018: commuters inside istanbul metro wagon travelling towards taksim station
DEVIL-static
airplane with red body shown first time. up stuffed
when you can see the first view full bike
second bird lands on feedersecond bird lands on feeder red boat first
realistic footage. public transport passengers transportation parking, city bus stop video
Tourist
on urhan highway bridge background.
black under the blue sign.
dog puts paws logether
10. white horse standing beside red colored wearing girl dress standing with stick bending down knee displaying screen
11. Blurred conference
with audience 4k video
we orange
right
13. A woman and man. holding gift.
14. A tranquil tableau of the old rod barn stoed weathered and iconic against the backdrop of the countryside
15. black numbers 1758 bottemno train
16. large white
travels through
followed two trucks and ascends gray road through mountains
17. view of big city from balcony view of big city from balcony
24

---

DEVIL-very-hun
Classical stylc of a horse partaking an ancient chariot race, seenes switching quickly from cheering crowds to closc-ups of intense wheel clashes.
High-speed shots of volcanic eruption engulfing tropical island, with lava fountains spewing molten rock and the environment transforming from idyllic paradise to hellish landscape of ash and fire.
3. A thrilling scene of rural mountain biking extreme sports, starting from the early morning cycling adventure, transitioning to the intense chase through fields and forests, and ending with cheers and celebrations under the sunset.
4. Neon-li lit streets pulse with energy as vehicles engage in high-octane pursuit, transitioning seamlessly from chaos to calculated evasion.
Against the backdrop of setting son, the chase intensifies, each turn a beartbeat away from capture.
A fighter jet dodging rapid anti air gunfire, quick maneuvers, tricer rounds visible.
Bear escaping collapsing cave, meks tumhling, dust rising, (masterpiece)), (hest quality), 8k, high detailed, ultra-detailed, bear, ((dark rocky textures)), sprinting, (echoing rumble), sudden movement
7. A courier on a bike weaves through traffic at breakneck speed, namroly avoiding cars and pedestrians in a rush to make deliveries on time.
8. A hummingbird rapidly darting between vibrant flowers lush garden, with quick cuts various close-up shoes showeasing its rapid wing movement and agility.
9. Venctian gondola chase scene, narrow canals, historic buildings, urgent escape, ((mnasterpiece)). ((best quality).. 8k. high detailed, uliru detailed, gondola, ((twisling canals)). (ancient architecture), (urgeni puidling)) cinematic chuse..
10. Futuristic sparts cars rucing on a vertical loop track against a sci (i cityscupe, cars defying gravity, ((speed truils)). (dizzyynng heights), (spectacular crushes), the thrill of cutting. edge technology.
11. Cat rapidly zigzaggngg acroes a roftap, avoiding swooping hirds under stommy sky.
12. A sequence of a cow performing acrohatic stunts over series of colorful, abstract platforms that morph shapes.
13. The dog bursts through thicket, darting from loggy forest to a steep hillside, rocks crumbling under its paws as it charges towards a roaring river below.
14. Thundering across a vast desert plain, the elephants race over dunes and dodge sandstorms, before swiftly travensing through a rocky
canyon, bounding over boulders and leaping across narrow ravines.
15. A giraffe navigating city during a robot uprising, with quick cuts showing chaotic hattles, explosions, and futuristic technology high-stakes escape secnario.
16. horse leading wild stampede across stormy beach with crashing. depicted with swift, sweeping eamera moves, cinematic composition.
17. Intense motorcycle escape from volcanic cruption. with tramsitions from lava lava-filled landscapes to asb-clouded skies.
18. futuristic robot uprising. (llssers liring)). metallic explosions, debris, ((screaming civilians)), dystopian cityscupe.
19. Sheeps engaging in high-speed pursuit through a eyberpunk city, the scene rapidly transitioning between neon-li lit streets, bustling marketplaces, and lowering skyserapers.
20. A pulse pounding sequence
tmin harreling through treachemoss storm, scene transitioning hetween lightning. skies torrents of rain to flooded tracks and collapsing bridges.
21. track rushing away from treacherous mountain pass during a blizzard, with sudden avalanches and rockslides adding the danger.
22. A zebra sprinting across the basy lanes of Times Square in New York City, with seene transitions occurring quickly as it moves from iconic billboads bustling sidewalks filled with tourists.
MovieGen
1. A green monster made of plants walks through an airport.
2. A marble goes through glass cup. breaking into pieces.
A droplet of water falling
hot surface, instantly evaporating into wisp sleum that swirls gracefully into the air. Johannesbury South Africa during a beautiful sunset.
man wearing green and sun taking pleasant
A person on a hoverboard colliding with wall, the board stopping abrupily.
6. A toy robot wearing blue jeans and a white t-shirt taking a pleasant stroll in Johannesburg South Africa during winter storm. In a marathon race, female athlete gradually sprints abead of the male athletes.
wenager eating a slice pizza, cheese stretching as they pull away.
9. A man in suit lights monsters.
10. A dog made of ice melts completely in a hot summer day.
1I. A truck right alongside a flowing river, capturing the movement of the water and the surrounding forest.
12. group of skateboarders perform tricks on ramps and rails skate park, showcasing their skills.
13. A air balloon descending back to the ground.
14. Chef chopping onions in the kitchen for the preparation of the dish.
15. Zoom in shot the face of young woman sitting on bench in the middle of empty school gym.
16. The couple runs hand hand release sky lantem, then watches drift upward into night sky, carried by the wind with the stars shining above.
17. Acrial view shot of cloaked figure elevating the between dkyscrapers.
18. softhall player sliding safely into second hase.
19. A giraffe lifeguard ourfit, sitting atop high chair watching over crowded pool.
20. A speed skater accelerating during a short track race.
25

---

H Detailed Analysis on Reward Function for Perceptual Video Quality
Figure 10 shows that different metrics in reward functions for perceptual video quality often exhibit negative weak correlations. For example, dynamic degree tends to be negatively correlated with many other metrics, indicating that optimizing exclusively for one metric can either reduce motion dynamics or undermine temporal consistency and aesthetic quality. These findings underscore the need to balance potentially conflicting reward functions, rather than prioritizing any single one in isolation, and emphasize the importance of carefully calibrated approach to evaluating generated videos.
DEViL-high
DEVIL-medium
DEVIL-static
MSRVTT-ted T-st
0:12
0:32
1:22
0:4T
0G2
2-B
1:20
E13
6:50
0-44
C:44
CDN
320
6.40
6:1
6:55
6.7
8:50
4:79
A:50
Bukr
6.14
000
Caleado mg
Figure 10: Correlation between reward functions for perceptual video quality.
I Prompt of AI Feedback
Prompt for Al Feedback from VLMs
You helpful assistant that evaluates the quality of generated video from textual prompt.
Compare the text prompt and generated video and evaluate the quality (visual quality, proper dynamics, etc...) of the video.
First explain the reasoning, then present the final assessment. Start the reasoning with 'Reasoning:
After explaining the reasoning, present the final assessment with AAssessmet:" '
Your final Assessment' should single-number score from to 10, not as a fraction.
When evaluating, consider the following points:
Visual Quality: Evaluate the clcarness, resolution, brightness, aesthetic appeal of the video.
Dynamics: Evaluate whether the video demonstrates appropriate dynamics, ensuring it avoids excessive movement in situations meant to be static or insufficient movement situations intended to he dynamic.
Smoothness, Consistency, and Naturalness: Assess the smoothness, consistency, and naturalness shape and motion objects, animals, and humans.
Contents: Evaluate whether the video content aligns with the given text prompt.
Textual Prompt: (instruction]
Video: [video_file _ile]
26

---

Further Results for Calibrating Reward to Preference Feedback
J.1 Basic Prompts
Figure 13 and Figure 14 show the two-dimensional histogram and correlation between reward function and AI feedback from Gemini [14] and GPT-4o [13], and Figure 11 represents the coefficient of calibrated reward designed for GPT-4o. The relative weighting assigned to the dynamic degree changes according to the dynamics grade of the prompt. Specifically, prompts with a high dynamics grade, i.e. DEVIL-high, place greater weight on the dynamic degree. In contrast, prompts that describe slight motion, i.e., DEVIL-medium and DEVIL-static, place a smaller weight on it. This behavior matches the pattern observed in reward calibration with Gemini (Figure 4). GPT-4o exhibits a stronger inclination toward dynamics than Gemini.
Subject Motion
Dynamic Degree
Aesthetic Quality
Imaging Quality
Text-Video Consistency
Consistency
Smoothness
1.6
0.4
0.2
DEVIL-high
DEVIL-medium
DEVIL-static
MSRVTT-test
Figure 11: Coefficients of calibrated reward with GPT-4o.
J.2 Challenging Prompts
This section describes the reward calibration procedure and results for two challenging prompt sets, DeViL--vey--iihh and MovieGen, which were introduced to evaluate our method with larger T2V models, such as Cog VideoX [19] and Wan 2. [20]. Following the methodology for reward calibration with Latte (see Section 4), we generated Consistent with observations in Appendix J.1, using solely text-video consistency is insufficient to fully capture AI feedback from Gemini (Figure 15). We choose the combination weights w, to maximize correlation with Gemini's evaluations. The coefficients of calibrated weights are shown in Figure 12.
videos per prompt using Wan 2.1-1.3B [20].
Subject Consistency
Motion Smoothness
Dynamic Degree
Aesthetic Quality
Imaging Quality
Text-Video Consistency
1.0
0.8
0.4
0.2
DEVIL-very-high
MovieGen
Figure 12: Coefficients of calibrated reward with Gemini.
27

---

-0.05, 0.06
-0.01, 0.08
0.68
2214,,14 000.0
R 0.14,p <001
8= -0.10,
2=0.41
DEVIL high Gemini
62
0.12, 0.01
0.100 0.00 0.01
p.1..p
p.2.00. 0.001
8= 0.06.p 0.03
R=1.16, p 0.01
00.00,0p0000
DEVIL medium
03
92
XX
82
A- p 0.01
0.15, P- 0.01
0.055, 0.10
0.05,
0x
-0.04, 0.16
30
0.34
0.34 p 0.01
0.3.3 p0000000
DEVIL statc Gemini
92
A- p.366,xx00..
R- 0.27.px0..00
.B 0.00,px0.00
R 0.400pxx.000
R- 0.13,p 0.01
0.3..2 px 0.01
-00.11,p00.00
MSRV
Subject Consistency Motion Smoothness
Dynamic Dogree
Aasthetic Quality
Imaging Quality
Text Video Consistency
Calbrated Roward
Figure 13: 2D 2D-histogram and correlation between reward function and Al feedback from Gemini.
-0.33, p-0.0.00.1
f= -0.24 0.01
0.000 p
-0.30, p 0.01
R 00.00-0.011
-0.45.pp0000 =.000
DEVIL high
GPT
60
52
DEVIL momimmiiu GPT
-0.15.pc0..0
0.02 0.42
0.10, 0.01
p.000.p 0.89
p.20.
k=0.07.pp p 0.01
m0 07. 0.001
0A 03
92
18
14
04
12
04
-0. 00.000,,00
0.01
R=
-0.05,p 0.12
p-
=024..=0000
-0.05, p= 0.13
8=0.30 0.01
8 8 B0..0.0000000
DEVIL stabic
34 31
16
66
02
CA 16
32
93
02
03
34 15
- 0.222,2 00.00
Rm0.10 0.0001
0.208,p
-0.05,
0.04
0.3..2 p 0.01
10.002,ppp0000 0.00
MSR/TT-te
37
F.0 Degree
02
82 Consistency
Consistency
Smoothness
Quality
Imaging Quality
Figure 14: 2D-histogram and correlation between reward function and feodback from GPT-4o.
-0.16,px00
A.2029
DEVIL vory
Gemini Score
0.0100, <0.03
-0.00,066
p-.00
0.2..5, p...p..x0000
0.05. 0.07
0 px0.00
0.30ppx000
Geminn Score
MovieSon
300 3R0
Subject Consistency
Dynamic Degree
Quality
Imaging Quality
Consistency
Calibrated Reward
Figure 15: 2D-histogram and correlation between reward function and AI feedback from Gemini for challenging prompt sets, DeviL-very-high and MovieGen.
28

---

J.3 Cost of Reward Calibration
As described in Section 4.1 and Appendix J.2, we generate 64 videos per prompt using pre-trained Latte and Wan 2.1 models. Compared to naively querying VLMs at every inference step, our calibration approach is substantially more cost-efficient, since the VLM queries are amortized through one-time weight estimation. Table 2 summarizes the difference in per-prompt query count and execution time when applying DLBS (KB 32). These results demonstrate that reward calibration reduces the number of VLM queries, making large-scale search with DLBS computationally feasible.
Table 2: Comparison of query count and execution time between naive VLM queries during search and reward calibration. Assuming 15 seconds per VL.M query.
Method
Query Count
Exec. Time (sec)
Querying VLMs during Search (KB 32) Reward Calibration
T-=50 KB=32 1600
2 102,400
64
960
J.4
Generalization of Reward Calibration across prompts
Video generation inherently involves trade-offs between fundamental properties such as dynamics and consistency (Appendix H), which may require category-speciite calibration for optimal performance. However, despite these domain-specinf requirements, we hypothesize that calibrated rewards can generalize to some extent across different datasets, as they are based on shared principles of perceptual quality. To the out-of-domain transferability, we conducted additional experiments applying the reward weights calibrated on DEVIL-high and DEVIL-medium to MSRVTT-test prompts (Table 3). We used Latte [18] as a base model and evaluated the results using VideoScore [55], human preference-trained evaluator, measuring five key metrics along with their corresponding average scores. With the DEVIL-hiig reward, we can enhance other metries while maintaining dynamics. DEVIL-medium reward, which is closer domain to MSRVTT-test, shows a different trade-off pattern. While it slightly reduces dynamics, significantly improves other metries and achieves higher average score than the MSRVTT-test reward, demonstrating higher transferability.
Table 3: Out-of domaiin prompt generalization. Rewards calibrated on DEVIL- hiiLL-iih/mmddunn applied to MSRVTT. prompts. All metrics derived from VideoScore [55]. VQ Visual Quality: TC Temporal Consistency; Dynamic Degree; T2V Align. Text- to videeo Alignment; Factual Consistency.
(R: Reward, P: Prompt)
VQ
TC
DD
T2V Align.
FC
Average
Latte
2.32
2.01
2.91
2.67
2.07
2.40
+ DLBS (KB = 8) with R=MSRVTT, P-MSRVTT
2.50
2.27
2.88
2.74
2.28
2.53
with R-DEVIL- P-MSRVTT
2.49
2.26
2.89
2.73
2.31
2.54
with R=DEVIL-high, P=MSRVTT
2.36
2.03
2.90
2.68
2.10
2.42
29

---

K Correlation between VLM and Human Evaluation
As mentioned in prior research [44-46], evaluation from VLMs such as Gemini and GPT-4o exhibits high correlation with human assessment compared to other existing metrics. As experiment, we measured the correlation between the Al feedback from these VLMs and human labels in the TVGE dataset [74]. As shown in Figure 16, Gemini achieved correlation of 0.49, and GPT-4o achieved 0.51. Consequently, optimizing for these VLM rewards is valid way to improve human-perceived quality, rather than merely "gaming" the metrics.
R 0.49, p < 0.01
R = 0.51, < 0.01
4.-
4,5
TVGE Score
35-
3.5
25
2.5
1.5
10
1.0
Gemini Score
GPT Score
Figure 16: Correlation between VL.M outputs and human labels the TVGE dataset.
For deeper analysis of failure cases, qualitatively examined the top 5% outliers between human preference labels in the TVGE dataset and VLM (Gemini) evaluation (Figure 17). As far as we observed, VLM sometimes makes subtle mistakes, but we did not any critical failures.
"the words KEEP OFF THE GRASS' on a sign next lawn"
KEEP OF IF THE GASS
KEEP OF IT THE GASS KEEP OF IF THE GASS KEEP OF IT THE GAS
KEEP OF IT THE GAS
"Four friends have picnic, enjoying six sandwiches and two bottles of juice"
(ashion portrait shoot of girl in colorful glasses, breeze moves her hair"
Figure 17: Misaligned cases of Gemini-baan based evaluation with human preferences. (Top) For prompts specifying text, such as "the words 'KEEP OFF THE GRASS' on sign next to a lawn," the VLM was significantly harsher than human evaluation text rendering (VLM: 2/10, Human: 4.5/5). (Middle) For prompts specifying quantities, such as "Four friends have a picnic, enjoying six sandwiches and bottles juice, the VLM was more lenient than human evaluation (VLM: 8/10, Human: 1.5/5). (Bottom) For "fashion portrait shoot of girl in colorfal glasses, a breeze moves her hair," despite missing arms in the generated person, the VLM was misled by distracting background patterns, possibly mistaking them for -urtain-like elements that obscure the arms behind the background (VLM: 8/10, Human: 1.2/5).
30

---

L Qualitative Evaluation of Calibrated Reward
We provide best-of-64 videos by individual rewards and VLM calibrated rewards in Figure 18. Videos selected solely on single metric can over-optimize one aspect while neglecting others, whereas those chosen via VLM-calibrated rewards exhibit a more balanced quality. For instance, videos chosen solely based on temporal consistency (i.e., subject consistency and motion smoothness) or frame-by-frann quality (i.e., aesthetic quality, imaging quality) tend to lack dynamic movement, whereas those selected based on dynamic degree often lose temporal consistency. Evaluations relying on single metric also fail reflect the given prompt in some cases. Text-video consistency, which often exhibits high correlation with VLM-based evaluation among individual metrics (Figure 3), relatively effective in capturing the overall quality of a video. However, it may overlook certain aspects, such as frame-wise artifacts. In contrast, videos selected using VLM-calibrated rewards exhibit a more balanced overall quality.
"A storm sweeps elephant into raging river, carrying away swiftly."
Dynamic Degree
Aesthetic Quality
Gemini Calibrated Reward
with aerial
destination.
dominican republic
Subject Consistency
Imaging Quality "black car under the blue sign."
GPT Calibrated Reward
US
CU
O urril 7
Motion Smoothness
Dynamic Degree
Gemini Calibrated Reward
yellow- girl is explaining about game"
Subject Consistency
Text-Video Consistency
GPT Calibrated Reward
We select the video with the highest reward out 64 randomly generated candidates each prompt, drawn from DEVIL-high, DEVIL-medium, DEVIL-static, and MSRVTT-tsst (arranged from top to bottom). Videos chosen using VLM-calibrated rewards achieve more balanced quality compared to those relying single metric. For instance, when subject consistency, motion smoothness, aesthetic quality serves the sole selection criterion, the resulting videos often lack dynamic movement, whereas prioritizing dynamic degree can compromise temporal consistency, Moreover, annle-metric evaluations may occasionally fail to align with the intended prompt.
Figure 18:
31

---

M Further Results for Diffusion Latent Beam Search
M.1
Scaling Search Budget with GPT-4o Calibrated Reward
We measure the performance using reward calibrated to GPT-4o (Figure 19). DLBS improves all the calibrated rewards the best the search budget K increases (especially KB 16, 32), while BoN and GS, in some cases, eventually slow down or saturate the performance. Notably, an LA estimator with a small search budget (KB T 6) comparable to even outperforms DLBS (K B 32).
DEVIL-high
DEVIL-medium
DEVIL-static
MSRVTT-test
0.325
0.46
0.35
0.275
0.44
0.225
0.58
0.30
0.42
0.40
16
(Budget)
K B (Budget)
K (Budget)
(Budget)
BoN(K=1)
GS(B=1)
DLBS
DLBS-LA
Figure Inference-time search reward calibrated GPT-4o.
M.2
Scaling Search Budget to Larger Regimes
Figure 20 and Figure 21 show the performance of inference-time search DEVIL-medium and MSRVTT-test that includes the results with KB 64. We can observe that the increasing trends still continue.
DEVIL-medium
DEVIL-medium
Calibrated Reward
BoN(K=1)
0.74
GS(B=1)
0.72
Gemini
DLBS
Budget=1
0.72
DLBS-LA
Budget=2
0.70
Budget=4
0.70
Budget=8
Budget=16
Budget=32
0.68
0.68
Budget=64 64
1
2
4
8
16
32
64
1
2
4
8
16
32
64
*
(Budget)
Figure 20: Inference-time search on reward calibrated to Gemini including KB = 64.
MSRVTT-test
MSRVTT-test
Calibrated Reward
0.60
0.62
BoN(K=1)
GS(B=1)
Gemini
0.60
DLBS
Budget=1
DLBS-LA
0.58
Budget=2
Budget=4
0.58
Budget=
0.56
Budget=16
0.56
Budget=32
Budget=64
8 16
32
64
1
8
16
32
64
K B (Budget)
K
Figure 21: Inference-time search reward calibrated Gemini including KB 64.
32

---

M.3 Full Results for Scaling Trend of DLBS
Figure 22 demonstrates the scaling trend of DLBS, proportional to the search budget, under various choices of K. The results show that there is an optimal balance between the number of latent K and the number of beams B under the same budget. For instance, as we increase the budget to KB 16, 32, we have a peak around K 4, 8, 16, which is about 25-50% of the budget. This implies that balancing possession and exploration of diffusion latents in DLBS helps search for the best outputs robustly.
DEVIL-high
DEVIL-medium
DEVIL-static
MSRVTT-test
Calltratt Baward
0.39
0.72
Gemini
0.38
0:58
0.37
Eizpen2
0.70
0.
0u
0.48
0.36
0uspe-12
0.46
Eu528-10
0.56
Codpet--2
0.35
0.44
0.26
0.60
0:43
GPT
Calitrated
Bicpe-2
0.59
0udget-2
Eudpe-2
0.42
Kodgetid
0.22
Duspe-15
Raspted
0.57
Duspe-22
Euppe-10
0:41
Codpet-24
16
32
16
32
16
0.40
16 32
BoN(K=1)
GB(B=1)
(LBS(best)
Figure 22: DLBS can improve the performance in any prompts or reward, as we increase the search budget
KB E [1, 2, 4,8, 16, 32}. In addition, we can see an optimal balance between the number of latent K and the number of heam B under the same budget. For instance, we increase the budget to K B 16, 32, we have peak around K 4,8, 16, which is about 25-50% of the budget.
M.4 Further Analysis on Lookahead Estimator
Figure 23 (Left) demonstrates that increasing the number of reward estimation steps T in the LA estimator leads to improved reward prediction performance for during the denoising process. This finding suggests that extending the LA steps enables a more effective search based on accurate reward predictions, particularly in the early stages of the denoising. As shown in Figure 23 (Right), enlarging the look-ahead horizon increases the reward gain to T' 6; beyond this point, e.g., T' no significant benefit while multiplying the computational cost. Accordingly, we fix 7" 6 in the main experiments, as it captures nearly all the attainable gains at minimal cost. These results were obtained on Latte [18] using DDIM sampler.
offers
MSRVTT-test
MSRVTT-test
Mean Absolute Error
Posterior Mean (step=1)
Gemini Calibrated Reward
0.08
(Rewards)
steps
0.60
LA Steps 6
0.59
Posterior Mean (step=1)
LA stops
0.04
steps 2
0.58
steps 3
0.02
steps
0.57
LA steps 20
0.00
1
50
40
30
20 10
Denoising Timestep t
K
Figure 23: (Left) Comparison of the reward estimation error for different LA steps T'. We evaluate the reward predicted by the LA estimator, which projects z, to Zojā(0) in T' steps (Algorithm 2) and computes r' (()))))))) against the actual reward obtained by projecting u, to zo in / steps using a DDIM sampler (1) 1.0) and evaluating r' (zo). (Right) Impact of T' on search performance. Reward improves rapidly up to T" = 6 but saturates thereafter; using T' = 20 offers no measurable gain. These results show that a modest T" is sufficient in practice.
33

---

To verify that this behavior is not specific to DDIM sampler, we conducted the same ablation with an SDE- +Peooe+t++ [56] on Wan 2.1-1.3B [20] (Figure 24). Note that the notation follows Algorithm 4. Specifically, M' in the SDE-DPMSolver++ setting corresponds to the 7" used with the DDIM sampler. We observed the same pattern shown Figure 23. Increasing the look-ahead horizon M' monotonically improves the LA estimator's reward prediction. Search reward gain up to roughly M' 6, after which gains saturate. For example, M' 12 yields no measurable improvement while incurring substantially higher cost. This cross-sampler and cross-model consistency provides practical guideline for choosing M'; a modest horizon (Rs 6) captures nearly all attainable benefit.
MovieGen
MovieGen
Mean Absolute Error
0.20
Postertor Mean
Calibrated Reward
0.38
(step=1)
(Rewards) rdd
0.37
LA Steps 3
Gemini
steps
0.36
Posterior (step=1)
0.10
0.35
LA steps 2
LA steps 3
0.05
0.34
LA steps 6
LA steps 12
0.33
50
40
30 20
10
Denoising Timestep 5
Figure 24: (Left) Comparison of the reward estimation error for different LA steps M'. We evaluate the reward predicted by the LA estimator, which projects 2,, to (( in M' steps (Algorithm 4) and computes steps using SDE- DPMSolver++ [56] and evaluating r' (zær). (Right) Ablation study of M' on search performance with SDE- DPMSolver++ [56] Wan 2.1-B.3B [20]. Reward improves rapidly up M' 6 but saturates thereafter; using M' 12 offers no measurable gain. These results show that a modest M' sufficient practice.
r' r' ( ), against the actual reward obtained by projecting zı, Zty in (M
34

---

M.5 Ablation Study for Diffusion Steps
Scaling Diffusion Steps Figure 25 (Left) shows the performance when increasing the number of denoising steps T. Since DDIM exhibits fast convergence [42], BoN with a larger T does not improve the reward much. DLBS improves performance when scaling denoising steps to T 200 more than BoN, which implies that DLBS benefits from larger computational resources in denoising. However, as Figure (Left) indicates, these gains are smaller than those obtained by widening the beam budget K B or leveraging the LA estimator.
Range Diffusion Steps for Search We investigate which range of diffusion steps DLBS should be applied to for effective search. Unlike Kim et al. [15], which applies GS only during the initial 5-10 steps, our results in Figure 25 (Right) show that applying DLBS throughout all steps leads to substantially better performance. This suggests that applying DLBS entirely is more effective than focusing on the early stage.
MSRVTT-test
MSRVTT-test
Gemini Calibrated Reward
0.59
Calibrated Reward
0.58
0.57
t 0]
0.58
Gemini
t t 45)
0.56
t [E050,44 40)
t (40,0,30 30]
0.57
T=50
T=100
0.55
t [30. 20]
E [20,2,10 10]
T-200
(30,
0.54
1
2
4
8
1
4
K
Figure 25: (Left) Scaling the denoising steps T. (Right) Range of denoising steps [50, apply search methods. While Kim et al. [15] applics GS in the first 5-10 steps, DLBBS over the entire diffusion steps yiclds the largest improvement.
M.6 Ablation Study for in DDIM scheduler
Figure 26 illustrates how varying the value of 7) in DDIM influences search performance. Here, controls the degree of randomness in the DDIM scheduler: = 0.0 corresponds to the deterministic version of DDIM, while y 1.0 is equivalent DDPM. As yj decreases below 1.0, performance in terms of the final reward diminishes, presumably because lowering the randomness in the sampling process narrows the scope of exploration.
MSRVTT-test
Gemini Calibrated Reward
0.57
0.56
1=1.0
n=0.5
n
0.55
2
4
8
K
Figure 26: Comparison among different v DDIM sampler.
35

---

M.7 Comparison with Gradient-Bass Search Methods
Applicability to diverse reward models One advantage of our zero-order search framework is its applicability to reward models for which computing gradients is computationally prohibitive (e.g., large-scale VLMs) or fundamentally impossible (e.g.. human evaluators or external API- based models). To demonstrate this, we simulated search with human feedback by employing VideoScore [55], a VL.M-bsee reward model trained on human evaluations, as the reward function. As shown in Table 4, DLBS-LA with VideoScore as the evaluator achieved substantial improvements over the vanilla baseline, suggesting that eegh-performance VLMs or human evaluators can, in principle, be directly incorporated reward functions in our framework.
Table 4: Results of DLBS-LA with VideoScore as the evaluator, illustrating its applicability to reward models for which computing gradients is computationally prohibitive (c.g., large-scale VLMs) or fundamentally impossible (e.g., human evaluators or external API-based models).
Method
VideoScore
Latte
2.40
+ DLBS-LA (KB = 4, 6) with VideoScore
2.69
Efficiency in time and memory We further compared DLBS with DAS [61], a first-order gradient- based method based on Sequential Monte Carlo. Experiments were conducted on Stable Diffusion 1.5 [21] with LAION Aesthetic V2 [50] as the reward model, using an NVIDIA RTX 6000 Ada (48GB). Table 5 shows that under the assumption of equal exccution time, DLBS (a zero-order method) takes the lead because it can have a larger search budget, which refers to the number of particles used for search, i.e., KB for DLBS and N for DAS. Our observation that a zero-order method achieves better performance than a first-order method under the equal execution time aligns with prior findings inference-tine search for image generation [41].
Gradient- search methods also exhibit significant increases in memory usage due gradient computations required for the reward function and the VAE decoder. In other words, gradient- based methods are actually inefficient in terms of memory cost. The results shown in Table are based on a single evaluator and single frame (i.e., image generation). Note that for video generation, as mentioned in Figure 3, a single evaluator metric does not correlate well with perceptual quality, necessitating the combination of multiple evaluators, which roughly multiplies the memory requirements for gradient calculation. Additionally, video generation models do not decode just one frame from the VAE (maximum frames are 81 for Wan 2.1 [20] and 49 for Cog [19]). The gradient increase would be significantly larger in video generation than in image generation, making grasee methods almost impossible in practice. For reference, the memory usage of vanilla Latte, DLBS-LA, and DAS in Table 6.
Table Comparison between DLBS and DAS Stable Diffusion 1.5 with LAION Aesthetic V2 reward. DLBS (a zero-order method) achieves better performance than DAS (a first-order method) under equal exccution time.
Method
Score
Time (sec)
Memory (GB)
SD 1.5
5.81
2
4.3
DAS (N = 8)
6.59
108
14.2
DLBS (K 8, B 2)
6.63
103
5.9
DAS (N 166)
6.68
220
14.2
DLBS (K 16, B 2)
6.69
209
5.9
Table Memory usage of search methods on Latte. The memory advantage of DLBS-LA (a zero-order method) becomes more critical in video generation.
Method
Latte
DLBS-LA
DAS
Memory Usage (GB)
16.3
38.9
-48.0 (OOM)
36

---

M.8 Comparison with Other Inference- Time Search Methods
Concurrently with our work, inference-time search based on zero-order sequential Monte Carlo (SMC) has been proposed. We include a comparison with FK Steering [41] in Figure 27, where the resampling mechanism in the SMC-based methods does not occur frequently enough, preventing them from surpassing BoN performance, in our text-to-video experiments.
MSRVTT-test
Calibrated Reward
0.62
BoN(K=1)
GS(B=1)
Gemini
0.60
DLBS
DLBS-LA
FK Steering
0.58
0.56
1
2
8
16
32
64
K B (Budget)
Figure 27: Comparison with FK steering [41].
M.9 Scalability Long Videos
To demonstrate that our method is scalable even when extending the frame count to the model's s maximum, we conducted experiments with the maximum frames for Cog VideoX55BB (49 frames, seconds) and Wan 2.1-1.3B (81 frames, seconds). The reward, which was calibrated using 33-frame, 2-second videos generated by Wan 2.1-1.3B, was applied as-is. As shown in Table and Table 8, we confirmed that even with longer frames, the reward values could be improved more efficiently than the BoN baseline.
Table 7: Scalability results on CogVideoXs with 49 frames, 6 seconds using DEVIL-very-high prompts.
Method
KB
Reward
Inference Compute (NFE)
CogVideoX-50
0.429
50
+ BoN
64
0.474
3200
128
0.478
6400
+ DLBS
16
0.481
1200
32
0.490
2400
DLBS-LA
0.497
2500
16
0.517
4900
Table 8: Scalability results on Wan 2.1-1.3B with 81 frames, 5 seconds using MovieGen prompts.
Method
KB
Reward
Inference Compute (NFE)
Wan 2.1-1.3B
0.313
50
+ BoN
128
0.357
6400
256
0.360
12800
+ DLBS
16
0.357
1200
32
0.371
2400
+ DLBS-LA
0.373
2500
16
0.393
5000
37

---

M.10 DLBS with Larger Text-to-Video Models
We have tested our method on VideoCrafter2 [3] (1.9B) and Cog -5g 2B [19] and Wan 2.1-14B, 1.3B [20]. Our experiments confirm that our DLBS and DLBS-LA yield significant improvements, indicating their effectiveness can be observed in larger video generation models.
DEViL-very-high
MovieGen
DEVIL-very-high
MovieGen
0.375
0.48
0.30
06-8-31
Gemini
0.48
0.350
2LB5
0.34
0.46
0.46
0.325
0.92
0.300
0.28
K (Budget)
(Budget)
K
Figure 28: Inference-time search with Vode [19].
DEViL-very-high
MovieGen
DEViL-very-high
MovieGen
Callbrated Roward
0.50
0.36
2023-11
0.46
Gemini
0.46
2485
0.34
0.44
0.32
0.42
Eutga--
K (Budgeti
(Budget)
Figure 29: Inference-time search with Cog Videoid [19].
DEVi
MovieGen
DEVIL-very-high
MovieGen
Calbrated Reward
0.51
0.36
Gernii
0.50
0.36
1485
SABS
0.34
0.48
0.48
Eodget-d Cudget-dd
0.47
32
0.32-
K
(Budgot)
(Budgot)
Figure Inference-time search with Wan 2.1-14B [20].
DEVIL-very-high
MovieGen
DEVIL-very-high
MovieGen
Calbrated Rowward
0.38
0.50
0.36
Gemini
0.34
0.48
1.34
DUBS LA
Gudpetid
0.32
0.46
0.30
Eodact-d1
(Budget)
(Budget)
Figure 31: Inference-time search with Wan 2.1-1.3B [20].
MSRVTT-test
MSRVTT-test
DEViL-high
DEVIL-high
Caltbated Reward P
Go2--1
Gemin
0.58
0.58
Gamin
ELBS
0.57
0.57
0.56
0.56
2:dge
Judgen
K- B (Budget)
K- B (Budget)
Figure 32: Search with VideoCrafier2 [3].
Figure 33: Search with Cog VideoX-5B [19].
38

---

M.11 Further Results in AI and Human Evaluation
We show full results of evaluations using VideoScore [55]. metric trained on human judgments that evaluates videos at 8 fps across five dimensions and outputs scores ranging from 1.0 to 4.0 (see Figure 34). Under this metric, videos generated with our DLBS consistently outperformed those generated without search and with BoN.
Visual Quality
Temporal Consistency
Dynamic Degree
Text-to-video Alignment
Factual Consistency
DEVIL-hige Bivh 24-
Average
2.5
3.05
2.8
3.00
2.45
DLBS
K5m1
K8m1
DON
Bob
DUSS
Bob
DUBS
Visual
Temporal Consistency
Dynamic Degree
Text-to-video Alignment
Factual Consistency
DEVILmediimm
Quality
Average
2.7
26
25
2.4
2.8
2.3
DLBS
D0.95
K8m1
KBm3
BON
DLBS
KB-1
Boh
DUDS
Roh
DlBs
Visual Quality
Temporal Consistency
Dynamic Degree
Text-to-video Alignment
Factual Consistency
DEVIL-static
2.9
2.6
Average
2.7
2.4
2.8
2.5
2.7
2.4
DLBS
KB--
KB-1
6:M
DLBS
DUPS
Visual Quality
Temporal Consistency
Dynamic Degree
Text-to-video Alignment
Factual Consistency
IMSVVTT-tss
Average
2.3
2.4-
2.2
2.85
DL6S
0125
10-1
K0-1
B0W
0U65
KB-1
BoN
0150
10-0
BoNk
Figure 34: DLBS on calibrated reward also improves another qualitative metric, most, VideoScore [55], which is not involved in reward calibration.
We also show additional results of human judgment by three human evaluators (see Figure 35). These experiments confirmed that, regarding human preference (win rate), content generated with our search strategy consistently outperformed content produced without search.
Win Rate of DLBS-LA against GS (KB-1)
Win Rate of DLBS-LA against GS (KB-1)
CogVideoX
DEViL-high
DEVIL-Very high
Latte
CogVideoX
DEViL-medium
Latte DEViL-tatic
Wan DEVIL-very, high
Wan 2.1 MovieGen
0%
25%
50%
75%
100%
0%
25%
50%
75%
100%
Win
Lose
Win
Lose
Figure 35: Human evaluation results. We searched vidcos using the Gemini calibrated reward and asked three human evaluators to compare outputs from GS (K B = 1) and DLBS-LA (KB = T" = 6). "Win" indicates that the video generated by DLBS-LA was preferred.
M.12 Qualitative Results for DLBS
Qualitative results are shown in https::/sitte wwogle..co/vvee//////.e//
39

---

M.13 DLBS for Image Generation
We adopt PixArt-a [69] as our base text-to-image generation model. For evaluation, we directly reuse the prompt set of 45 common animal categories from prior works [75, 63]. As reward model, we employ the LAION aesthetic predictor [50] assess image quality.
Animal Images
Animal Images
Aesthetic Quality
BoN(K=1]
6S[B=1)
DIBS
0.75
Budget=1
DLBS-LA
Budget=2
Budget=4
0.70
Budget=8
0.7
Budget=16
Budget=33
0.65
0udget=64
1
4
16
64
1
16
64
K * B (Budget)
K
Figure 36: Inference-tine search with PixArt-ar [69].
Figure 37: Qualitative Results in inference-time search with PixArt-a a [69]. (Left) GS (KB 1). (Right)
DLBS-LA (KB 32, 6).
N Extended Related Works
Aligning Diffusion Models via Finetuning Aligmment by finetuning text-conditioned models has been investigated for image [10] and video [46, 45] generation. Typically, LoRA [76] in backbone model [77] finetuned through policy gradient [75, 78, 79], direct preference optimization [80, 44, regression [83], or direct reward gradient [84, 85, 45, 86). Some train an extra model for better initial noise space [87-89]. In contrast, we focus on the search over the denoising process at inference time, which does not require any model updates and may not degrade the original performance.
81, 38, 82), neward-weight
Evaluation of Text-to- Video Generation While there are several conventional metrics for video generation (or the one repurposed from image generation) such SSIM [90], IS [91], LPIPS [92], or FVD [93), those are not always suitable to evaluate how the quality of contents in video is, which is much more emphasized in text-to-video generation [74]. It has been a long-standing challenge to comprehensively and semantically evaluate the dynamics of contents or physical commonsense in generated videos [9, 53J. To deal with that, VBench [12] has recently been proposed as a suite of holistic evaluations for text-to-vides generation to reflect the perceptual aspect of the quality, such as consistency, smoothness, aesthetics of contents, or text-video alignment. Moreover, inspired by the success in LLMs [94-96]. we could leverage VLMs, which become more capable these days, as proxy of human evaluation of the contents [45]; by finetuning CLie-based models [55, 71, 97], or prompting GPT-4o [1 [13] or Gemini [14]. Our paper adopts AI feedback from VL.Ms as an alternative human rater, and proposes a recipe to calibrate reward to other sources of feedback (such as Al or human feedback), by considering linear combination of fine-graine metrics.
40

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
Our abstract and introduction appropriately include the claims made in the
paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
41

---

Justification:
See section 7.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[NA]
Justification: Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe the details of experiments Section 5 and other necessary information in Appendix B.
Guidelines:
42

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Our experiments are based on the open-source dataset [53, 54,
57J. Our experimental code is shown in https: http://anooooussaaaa/m 4open.scieecce/r science/r/ 12V-Diffusion- Seenn-Sarch-55777
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (tttps://nips.ca
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / httpppscccppubbcc/ccccccccs /es/iidSu///oommmiiiii /Cooe/ubdeeeaeeeee ssi aasionPoll. onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
43

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the details of experiments in Section 5 and other necessary information in Appendix B.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the statistical significance of the Pearson correlation coefficient in Figure 3 and Appendix J.1. We also report the reward values averaged over multiple prompts to reduce the variance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix B.
44

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compule required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: ptttps://eeuriss wwtpehuww....ppppppppeeteetctcce cttpe/ttthiciiellecccceceecaaaee ines?
Answer: [Yes]
Justification: Ethics.
We believe our research conforms, in every respect, with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Appendix A.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
45

---

Answer: [NA]
Justification: This paper does not include new datasets or pre-trained models that pose a risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have appropriately cited the papers of existing assets used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. condaataeess has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: he paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
46

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
We only recruited participants for user experiments to validate the effective- ness of our model, where they were asked to choose from generated videos. No human participants were involved in the dataset construction or model training process.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our experiment solely involves mcasurement and does not entail behavioral manipulation; therefore, we did not apply for IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: We LLMS only for writing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: /ttpss:///euip...eeee wwwperiisscc///oniio http://o/eeeennc//////////////// for what should or should not be described.
47


Reviews:
[{"review_id": "2PCli8ssU0", "reviewer": "Reviewer_3Psp", "summary": "The paper addresses a challenge of perceptual misalignment in text-to-video diffusion models, where generated videos often contain unnatural motion, temporal inconsistencies or static scenes. The authors propose Diffusion Latent Beam Search (DLBS) with lookahead estimator, an inference-time method that optimizes video quality by selecting a better diffusion latent to maximize a given alignment reward. DLBS leverages beam search over diffusion latents guided by a lookahead estimator to reduce noise in reward evaluation. The authors also provide a technique for reward calibration by leveraging AI feedback from VLMs. The higher a strong VLM estimates the generated video quality, the better visual quality it would have from a human point of view. Six base reward functions for perceptual video quality are used: Subject Consistency, Motion Smoothness, Dynamic Degree, Aesthetic Quality, Imaging Quality and Text-Video Consistency. At the experimental stage the authors demonstrate that DLBS is the most scalable, efficient and robust inference-time search that significantly improves video quality under the same computational costs.", "questions": "I would advice to make additional comments on generalization of VLM-Human correlation and reward calibration to improve the paper strengths. Ablations on solvers (e.g., DPMSolver++) or nu values would clarify optimization choices instead of just using deterministic DDIM (nu=0) at the experimental setup.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths.\n\nThe authors provide a strong vision to better denoising way in latent space which is the core contribution of the paper. The ablations and correlations are thorough. Human evaluations (pairwise comparisons) and diverse prompts (DEVIL, MSRVTT, MovieGen) strengthen validity.\n\n\nWeaknesses.\n\nWhile VLMs are used as human proxies, their correlation with human evaluation is moderate (nearly 0.5 correlation). Relying on VLMs without deeper analysis of failure cases (e.g., prompt ambiguity) provides additional risks for reward over optimization and highly biasing. The computational cost is not presented at the experimental stage. The rewards use fixed weights per prompt category after calibration, but prompt adaptation is unexplored. \n\nClarity. The text is well-structured and mostly clear. Figures effectively illustrate the method and results. The pseudocode and appendices sections aid reproducibility.\n\nSignificance. The paper addresses a critical gap in text-to-video generation. The inference time approach is model-agnostic, scalable (tested up to 14B params) and deployable without additional training or fine-tuning. The efficiency claims show readiness to real-world use.\n\nOriginality. DLBS adapts a well-known beam search paradigm to diffusion latents, while reward calibration and lookahead estimation offer novel solutions to reward noise and misalignment.", "was_revised": true, "final_justification": "I will remain my score according to the authors’ answers", "rebuttal": "We thank the reviewer for the constructive feedback.\n\n**> W1**\n\n> Relying on VLMs without deeper analysis of failure cases (e.g., prompt ambiguity) provides additional risks for reward over optimization and highly biasing.\n\nFirst, for a deeper analysis of failure cases, we qualitatively examined the top 5% outliers between human preference labels used in **Appendix J** and VLM (Gemini) linear regression predictions. \n- For prompts specifying text, such as \"the words 'KEEP OFF THE GRASS' on a sign next to a lawn,\" the VLM was significantly harsher than human evaluation on text rendering (VLM: 2/10, Human: 4.5/5).\n- For prompts specifying quantities, such as \"Four friends have a picnic, enjoying six sandwiches and two bottles of juice,\" the VLM was more lenient than human evaluation (VLM: 8/10, Human: 1.5/5).\n- For \"fashion portrait shoot of a girl in colorful glasses, a breeze moves her hair,\" despite missing arms in the generated person, the VLM was misled by distracting background patterns, possibly mistaking them for curtain-like elements that obscure the arms behind the background (VLM: 8/10, Human: 1.2/5). \n\nAs far as we observed, VLM sometimes makes subtle mistakes, but we did not see any critical failures.\n\nNext, to avoid the biases from specific VLMs, we have analyzed preference trends for GPT-4o and Gemini. In **Appendix I.1**, we present reward calibration weights using GPT-4o and compare them with Gemini calibration results (**Figure 4**), analyzing that GPT-4o tends to emphasize dynamics more than Gemini in the trade-off between dynamics and consistency.\nTo make this clearer, we calculated the proportion of dynamics metrics (i.e., Dynamic Degree), temporal consistency (i.e., Subject Consistency, Motion Smoothness, and Aesthetic Score), T2V alignment (i.e., Text-to-Video Consistency) in the calibrated reward weights using Gemini and GPT-4o. \n| Metric | VLM | DEVIL-high | DEVIL-medium | DEVIL-static | MSRVTT-test |\n|---|---|---|---|---|---|\n| Dynamics | Gemini | 0.167 | 0.125 | 0.000 | 0.167 |\n|  | GPT-4o | 0.333 | 0.250 | 0.250 | 0.273 |\n| Temporal Consistency | Gemini | 0.167 | 0.750 | 0.400 | 0.583 |\n|  | GPT-4o | 0.000 | 0.500 | 0.250 | 0.364 |\n| T2V Alignment | Gemini | 0.667 | 0.125 | 0.600 | 0.250 |\n|  | GPT-4o | 0.667 | 0.250 | 0.500 | 0.364 |\n\nWhile GPT-4o emphasizes dynamics more and Gemini prioritizes temporal consistency, both show similar text alignment weighting.\n\nLastly, regarding concerns about reward overoptimization, we confirmed in the **Appendix L.7** that diversity is not reduced by reward optimization, and we have verified that our method improves performance on both automatic evaluations, different from calibrated reward to be optimized, and human evaluations (**Section 5.2**).\n\n**> W2**\n\n>  The computational cost is not presented at the experimental stage.\n\n- For search algorithms\n\n    We provide the types and numbers of GPUs used in the **Appendix B.4**. \n    For Latte, we present computational costs (please see **Figure 6; Left**). \n    For CogVideoX and Wan 2.1, we newly demonstrate the relationship between cost and performance when searching with maximum frame counts (please see **Q2 from Reviewer 3kkj**).\n\n- For reward calibration\n\n    As mentioned in **Section 4.1**, we generate 64 videos per prompt from pre-trained Latte and Wan 2.1.\n    Our reward calibration approach is significantly more cost-efficient than naively querying VLMs every time during search. This is demonstrated in the following comparison between the per-prompt query count of reward calibration and that of naive VLM-based reward evaluation under DLBS with Wan 2.1 1.3B ($KB=32$).\n\n    | Method | Query Count | Exec. Time (sec) |\n    |---|---|---|\n    | Reward Calibration | $64$ | $\\approx$ 960 |\n    | Querying VLMs during Search ($KB=32$) | $(T=)50 \\times (KB=)32 = 1600$ | $\\approx$ 102,400 |\n\n**> W3**\n\n> The rewards use fixed weights per prompt category after calibration, but prompt adaptation is unexplored.\n\nOur experiments have shown that optimizing an existing single reward is not aligned with video preferences (**Figure 3**), and that we should consider the weighted combination of existing rewards to be a better proxy for judging preferable videos. Because the important metrics for a preferred video can differ among the prompts (e.g., focusing on dynamic movement, aesthetic static scene, etc), we have suggested that reward calibration should be done per prompt categories, where they have similar dynamics. The individual prompt is the smallest unit of the prompt categories, so our experiments support that it is possible to adapt rewards into specific prompts.\n\nMoreover, although we fixed the weight of rewards per category in the paper,  we here demonstrate that expanding prompt coverage enables more flexible reward design.\nWhile the paper shows reward calibration performed separately on each prompt set (DEVIL-high, medium, static, and MSRVTT-test), we now conduct a unified calibration approach. \nWe combine all four prompt sets into a single dataset of 91 prompts and perform reward calibration on this comprehensive collection. \nThe resulting calibrated reward is then applied to evaluate performance on each individual prompt set, demonstrating improved generalizability across diverse prompt types.\n\n| Method/VideoScore | DEVIL-high | DEVIL-medium | DEVIL-static | MSRVTT-test |\n|---|---|---|---|---|\n| Vanilla | 2.50 | 2.59 | 2.62 | 2.40 |\n| DLBS ($KB=8$) + Unified Reward | 2.60 | 2.65 | 2.68 | 2.54 |\n\nIn summary, we believe that our reward calibration can achieve either specialization and generalization, depending on the given prompts and how we want to optimize the reward.\n\n**> Q1** \n\n> I would advice to make additional comments on generalization of VLM-Human correlation and reward calibration to improve the paper strengths.\n\n- Generalization of VLM-Human correlation\n\n    The validity of using VLM evaluation as a proxy for human evaluation is not solely supported by the moderate correlation between evaluations. \n    Most importantly, **Section 5.2** demonstrates that search using VLM-calibrated rewards successfully improves Human Preference and Human Preference-trained Reward Models (VideoScore) across multiple prompt sets and models. \n\n    To further strengthen this argument, we conducted human evaluations comparing DLBS-LA ($KB=8$, $T’=6$, $\\mathrm{NFE}=2500$) and BoN ($KB=64$, $\\mathrm{NFE}=3200$) in **W3 from Reviewer 3kkj**. Because the calibrated rewards can work as proxy functions for human evaluation, DLBS-LA outperforms BoN in human evaluation despite requiring fewer NFEs.\n\n    These results support the effectiveness of using VLM evaluation as a substitute for human evaluation in improving human evaluation outcomes.\n\n    For discussion of trend in Gemini and GPT-4o, please refer to **W1**.\n\n- Generalization of reward calibration\n    - Generalization for prompts\n\n        We first note that video generation inherently involves trade-offs between fundamental properties such as dynamics and consistency (**Appendix G**), which may require category-specific calibration for optimal performance. \n        However, despite these domain-specific requirements, we hypothesize that calibrated rewards can generalize to some extent across different datasets, as they are based on shared principles of perceptual quality. \n         To test the out-of-domain transferability, we conducted additional experiments applying the calibrated reward weights calibrated on DEVIL-high and DEVIL-medium to MSRVTT-test prompts. \n        We evaluated the results using VideoScore, a human preference-trained evaluator, measuring five key metrics along with their average scores.\n\n        | (R: Reward, P: Prompt)| Visual Quality | Temporal Consistency | Dynamic Degree | T2V Alignment | Factual Consistency | Average |\n        |---|---|---|---|---|---|---|\n        | **Vanilla** | 2.32 | 2.01 | **2.91** | 2.67 | 2.07 | 2.40 |\n        | **DLBS (KB=8)** | | | | | | |\n        | + R=MSRVTT, P=MSRVTT | **2.50** | **2.27** | 2.88 | **2.74** | 2.28 | 2.53 |\n        | + R=DEVIL-med, P=MSRVTT | 2.49 | 2.26 | 2.89 | 2.73 | **2.31** | **2.54** |\n        | + R=DEVIL-high, P=MSRVTT | 2.36 | 2.03 | ***2.90*** | 2.68 | 2.10 | 2.42 |\n\n        With the DEVIL-high reward, we can enhance other metrics while maintaining dynamics.\n        DEVIL-medium reward, which is a closer domain to MSRVTT-test, shows a different trade-off pattern: while it slightly reduces dynamics, it significantly improves other metrics and achieves a higher average score than the MSRVTT-test reward, demonstrating higher transferability.\n\n        Moreover, in the original paper, we have confirmed that generalization for unseen in-domain prompts, using the same reward, can improve performance on unseen prompts (please see **Figure 8; Right**).\n\n    - Generalization for video length\n\n        Our method is scalable even when extending the frame length to the model's maximum.\n        To demonstrate this, we conducted additional experiments with the maximum frames for Wan 2.1 1.3B and CogVideoX-5B (please see **Q2 from Reviewer 3kkj**). \n        The reward, which was calibrated using 33 frames, 2s videos generated by Wan 2.1 1.3B, was applied as-is.\n        As a result, we confirmed that even with longer frames, the reward values could be improved more efficiently than the BoN baseline.\n\n    - For generalization for models and solver types, please see **Q2**.\n\n**> Q2**\n\n> Ablations on solvers (e.g., DPMSolver++) or nu values would clarify optimization choices instead of just using deterministic DDIM ($nu=0$) at the experimental setup.\n\nWhile used DDIM with $\\eta=0$ for Latte calibration, DPMSolver++ was employed for Wan 2.1 calibration (please see **Appendix B.3** for detailed settings). \nNotably, we confirmed that rewards collected from Wan 2.1 1.3B with DPMSolver++ can improve human evaluation scores for CogVideoX 5B with DDIM solver (**Section 5.3, Appendix L.9**)."}, {"review_id": "xIVeRP88ob", "reviewer": "Reviewer_rNpU", "summary": "The paper tackles the problem of alignment in text-to-video diffusion, making generated videos both follow a text prompt and high perceptual quality in inference time.\nThe Diffusion Latent Beam Search and a lookahead estimator are proposed to obtain a suitable sample at each denoising step.\nThe authors also introduced a weight linear combination of multiple metrics that aligned with perceptual quality and human preference.", "questions": "1. The LA estimator selects $T^\\prime=6$. Does it work for different models? Does it hold for different samplers?\n2. Beam size and candidate count interact with complexity. Have you explored any dynamic or adaptive strategies to adjust ($B, K$) on-the-fly based on intermediate reward?", "limitations": "The beam search can reduce the inference cost compared to best-fo-N. But the still takes tens of seconds per video on high-end GPUs, limiting real-time or large-scale use.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strength:\n1. the paper proposed a beam search for inference-time alignment in the latent space.\n2.  systematic study of six perceptual metrics correlations with VLM/Human preferences.\n3. extensive experiments on different parameters (e.g., $B, H, T^\\prime$)\n4. Effectively improve the performance across different diffusion model variants.\n\n---\nweakness:\n1. calibration relies on a strong VLM (e.g., Gemini, GPT). Without these models, it might be challenging to construct reliable metrics.\n2. efforts of tuning ($B, K, T^\\prime$) can be expensive.", "was_revised": true, "final_justification": "This paper is the first to investigate inference time beam search for diffusion video generation, presenting extensive experiments that demonstrate its superior quality compared to naive generation. It also explores strategies for performing beam search efficiently. I recommend accepting this paper.", "rebuttal": "We appreciate the reviewer’s thoughtful feedback.\n\n**> W1** \n> calibration relies on a strong VLM (e.g., Gemini, GPT). Without these models, it might be challenging to construct reliable metrics.\n\nAs explained in **Section 4**, while our current approach uses VLM feedback for reward calibration, this is primarily for practical demands rather than a fundamental necessity. \n\nHuman evaluation remains the gold standard for video quality assessment, but collecting human feedback at the required scale for search is prohibitively expensive and time-consuming. \n\nA practical approach to reduce time and cost is leveraging AI feedback from VLMs, which has shown modest alignment with human judgment on video quality (**Appendix J**). \nSince alignment via inference-time search requires massive reward evaluation queries, we need tractable proxy rewards that avoid reliance on humans or external VLM APIs for deployment.\nExisting lightweight metrics for video naturalness (e.g., Subject Consistency, Dynamic Degree) often fail to correlate with human/VLM evaluations (**Figure 3**), motivating our reward calibration approach that weights existing metrics using VLMs. \nWhile human labels would be ideal for calibration, collecting the extensive metric evaluations required would be even more costly and time-consuming than gathering preference labels from VLM APIs.\n|| Human | VLM | Lightweight Metrics |\n|---|---|---|---|\n| Alignment with Human Eval. | ◎ | ○ | △ |\n| Cost | △ | ○ | ◎ |\n| Exec. Speed | △ | ○ | ◎ |\n\n\nCrucially, to demonstrate the feasibility of reward calibration without VLM APIs, we performed calibration on MSRVTT-test prompts using VideoScore (a reward model trained on human evaluations) and searched accordingly, showing that our approach can work in principle without external VLM dependencies.\n\n| Method | VideoScore |\n|---|---|\n| Vanilla | 2.40 |\n| DLBS-LA ($KB=4$, $T’=6$) + VideoScore Calibrated Reward | **2.60** |\n\n**> W2**\n\n> efforts of tuning (B,K,T′) can be expensive.\n\n- ($K$, $B$)\n\n    In **Appendix L.3** and **L.9**, we present ablation studies on (B,K) allocation across multiple prompts and models. \n    As described in **Appendix L.3**, we found that the optimal allocation typically peaks around 25%-50%. \n    We also demonstrate that this pattern holds not only for video generation but also for image generation (**Appendix L.12**). \n    These findings provide practical guidance, suggesting that exploration can be focused within this specific range rather than exhaustively searching the entire parameter space.\n\n- $T'$\n\n    We comprehensively validated across multiple prompt sets, models, and samplers that adopting $T' \\approx 6$ achieves performance improvements, providing clear guidelines (please see **Appendix L.4** and **Q1**).\n\n**> Q1** \n\n> The LA estimator selects $T′=6$. Does it work for different models? Does it hold for different samplers?\n\nTo further verify that the choice of $T'=6$ is effective beyond DDIM solvers, we conducted an ablation study with SDE-DPMSolver++ for Wan 2.1 1.3B, varying $T'$ across $1, 2, 3, 6$, and $12$. Even in this case, performance improved efficiently up to $T'=6$, achieving nearly identical performance to $T'=12$, which requires twice the computational cost. \nThis supports that the choice of $T'=6$ works for different solvers (As we show in DDIM in **Appendix L.4** of the paper).\n\n| $T'$ | 1 | 2 | 3 | 6 | 12 |\n|---|---|---|---|---|---|\n| Reward | 0.351 | 0.360 | 0.365 | **0.376** | **0.380** |\n\nMoreover, as shown in **Figure 6**, we confirmed that `DLBS-LA` with $T'=6$ achieves superior performance compared to methods without LA estimator (i.e., `DLBS`, `GS`, `BoN`) across different models (Wan 2.1, CogVideoX, Latte). \n\n**> Q2**\n\n> Have you explored any dynamic or adaptive strategies to adjust ($B$,$K$) on-the-fly based on intermediate reward?\n\nWe initially considered dynamic or adaptive strategies that tune the allocation of ($B$,$K$) across diffusion timesteps, such as starting with a larger $B$ in early steps and transitioning to emphasize $K$ as the process progresses. However, we did not prioritize this direction because of the following reasons; fIrst, the performance gains from incorporating the lookahead estimator (please compare `DLBS` and `DLBS-LA` in **Figure 5; Left**) are significantly larger than those from tuning ($B$,$K$) allocation (**Figure 5; Right**) (please see **Appendix L.3, L.9** for further results). \nWe also compared against Sequential Monte Carlo-based methods that adaptively decide whether to search or retain previous candidates at each diffusion step, but these showed weaker performance improvements than DLBS (**Appendix L.8**). Adaptive meta-planning on how to allocate $KB$ budget would be an interesting future direction."}, {"review_id": "9cDnnibi2P", "reviewer": "Reviewer_srve", "summary": "This paper introduces latent beam search at inference time to align the generated videos with input prompts. There are two main contributions: 1) It proposes latent beam search and lookahead estimator for inference-time scaling. It samples K latents and selecting B beams by evaluating the reward through lookahead estimator. 2) It proposes a reward calibration algorithm to better evaluate the quality of generated videos.", "questions": "1) Computational inefficiency and limited improvement. According to Figure.1 (right) and Figure.6 (left), with $2^5$x increase in inference computation time, the improvement is only around 10%. Given the additional computation required, I think the improvement is far from sufficient. Moreover, although the proposed method shows improved performance with increased inference time, the simplest BoN baseline also exhibits a similar trend, and the improvement over BoN is also limited (less than 10%). Therefore, I believe the proposed method gives very limited improvement in test-time scaling performance.\n2) Beam search is still random search. At each timestep, the method still introduces random noise to generate different beam candidates. This is essentially not different from Best of N (BoN), except that it expands the space of random search. BoN's randomness lies in the initial latent, while beam search introduces randomness at every timestep. I believe the effectiveness of such random search is inherently limited. Although the authors use classifier guidance to show that the search process is moving toward the target, the zero-order nature of the random search seems not enough. If it were possible to compute first- or higher-order gradients with respect to the target and update accordingly, I believe that would be more meaningful.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1) The problem explored in this paper: how to perform test-time scaling in diffusion models is a very meaningful one. The proposed beam search method is straightforward and solid.\n2) This paper's writing is good and the method is easy to understand.\n\nWeaknesses:\nSee questions.", "was_revised": true, "final_justification": "The authors adrress my concern in computational inefficiency and the difference between BoN, zero-order search and first-order search.\n\nOverall, I believe that the test-time scaling of diffusion models is a very interesting and important academic problem. While there is still much to be addressed, I think this paper takes at least a small step in the right direction. I will raise my score and lean towards accepting it. Thank you once again for the authors' continued responses.", "rebuttal": "We appreciate the reviewers’ insights and address each point below.\n\n**> Q1**\n\n> I believe the proposed method gives very limited improvement in test-time scaling performance.\n\nWhile 10% reward increase is seemingly small, it represents a strong perceptual quality improvement. Here, we conduct human evaluations comparing DLBS-LA (KB=8, T’=6, NFE=2500) and BoN (KB=64, NFE=3200).\nDLBS-LA significantly outperforms BoN in human evaluation despite requiring fewer NFEs.\n\n| Model | Prompt Set | Win Rate of DLBS-LA against BoN | Reward (DLBS-LA) | Reward (BoN) |\n|---|---|---|---|---|\n| Latte | DEVIL-medium | **69.70%** | 0.726 | 0.705 |\n| Latte | MSRVTT-test | **65.56%** | 0.607 | 0.578 |\n| CogVideoX-5B | DEVIL-very-high | **69.70%** | 0.500 | 0.476 |\n| Wan 2.1 14B | MovieGen | **60.00%** | 0.378 | 0.357 |\n\nMoreover, in the original paper, the approximately 10% reward improvement achieved by DLBS has been confirmed to enhance human preference and human preference-trained evaluation models (VideoScore) across multiple prompt sets and models (**Section 5.2**). We also have confirmed that DLBS is superior to  Best-of-N (BoN) when using VideoScore across multiple datasets (**Figure 7; Middle**). \n\n**> Q2** \n\n> I believe the effectiveness of such random search is inherently limited. Although the authors use classifier guidance to show that the search process is moving toward the target, the zero-order nature of the random search seems not enough.\n\nOne clear advantage over gradient-based methods is the ability to use reward models where gradients are either computationally prohibitive (e.g., large-scale VLMs that are not scalable for backpropagation) or fundamentally impossible to compute (e.g., human evaluators or external API-based models). \nTo simulate the search with human feedback, we conducted a search using VideoScore, a VLM-based reward model trained on human evaluations, and evaluated the results with human feedback. \nThe following results imply that, in principle, high-performance VLMs available only through APIs (Gemini, GPT-4o) or actual humans can also be used as reward functions.\n\n| Method | VideoScore |\n|---|---|\n| Vanilla | 2.40 |\n| DLBS-LA ($KB=4$, $T’=6$) + VideoScore | **2.69** |\n\nFurthermore, to address reviewer concerns about the efficiency of our proposed method when using gradient-computable reward models, we compare our DLBS against  DAS [1], a first-order gradient-based search method based on Sequential Monte Carlo, where N represents the number of particles, and compare the metric improvements relative to runtime. \nRuntime measurements were conducted on NVIDIA RTX 6000 Ada (48GB). \nDue to computational cost constraints as described later, we compared using SD 1.5 with LAION Aesthetic V2 as the reward model. \n\n| Method | Score | Time (s) | Memory (GB) |\n|---|---|---|---|\n| Vanilla | 5.81 | 2 | 4.3 |\n| DAS N=8 | 6.59 | 108 | 14.2 |\n| DLBS K=8, B=2 | **6.63** | **103** | **5.9** |\n\nDLBS achieves competitive results compared to first-order gradient-based search methods within similar computational time.\n\nIn addition, gradient-based search methods exhibit significant memory usage increases due to computations for the reward function and VAE decoder. In other words, gradient-based methods are actually inefficient in terms of memory cost.\nFurthermore, these results are for a single evaluator with a single frame (i.e., image generation). \nNote that for video generation, as mentioned in **Figure 3**, a single evaluator metric does not correlate well with perceptual quality, necessitating the combination of multiple evaluators, which roughly multiplies the memories for back propagation $N_{\\mathrm{reward}}$ times.\nAdditionally, video generation models do not decode just one frame from the VAE (maximum frames are 81 for Wan 2.1 and 49 for CogVideoX).\nThe gradient increase would be significantly larger in video generation than in image generation, making gradient-based methods almost impossible in practice.\n\nFor reference, the memory usage of vanilla Latte, DLBS-LA, and DAS is as follows:\n| Method | Vanilla | DLBS-LA | DAS |\n|---|---|---|---|\n| Memory Usage (GB) | 16.3 | 38.9 | > 48.0 (OOM) |\n\n[1] Kim et al. Test-time Alignment of Diffusion Models without Reward Over-optimization. ICLR2025."}, {"review_id": "bNx8JqaxQt", "reviewer": "Reviewer_3kkj", "summary": "This paper presents Diffusion Latent Beam Search (DLBS), a gradient-free inference-time search algorithm for aligning text-to-video diffusion model outputs with perceptual quality and prompt intent. The method is developed based on the beam search, which can select a better diffusion latent to maximize a given alignment reward at inference time. To address the noisy evaluation of intermediate latents, the authors propose a lookahead estimator using deterministic DDIM steps to better approximate the final output. Additionally, the paper highlights a significant issue in current evaluation: existing perceptual metrics (e.g., smoothness, consistency) correlate poorly with human or VLM preferences. To solve this, the authors introduce a reward calibration step, learning weighted combinations of existing metrics to better match Gemini/GPT feedback.", "questions": "1. Can the calibrated reward weights learned on DEVIL-high be applied to other datasets (e.g., MSRVTT) with minimal degradation? Or must each dataset have its own calibration?\n2. How does DLBS perform when applied to longer or higher-resolution video generation tasks (e.g., >8s or >512×512)?\n3. Does DLBS reduce diversity due to beam pruning toward high-reward paths? Could you include diversity metrics to evaluate this issue?", "limitations": "The authors discuss some limitations in Section 7.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The proposed DLBS requires no model retraining, is plug-and-play, and integrates well with any existing text-to-video diffusion model.\n* The recognition that no single perceptual metric aligns well with VLM or human feedback is important and convincingly demonstrated. The calibrated reward design adds real value.\n* The method is benchmarked across datasets (MSRVTT, DEVIL, MovieGen), prompt difficulty levels (static, medium, dynamic), and model scales (Latte, CogVideoX, Wan 2.1). Includes both automatic and human feedback.\n\nWeaknesses:\n* While DLBS integrates known components (beam search, lookahead via DDIM), the method is more of a well-engineered combination. Its novelty lies more in system design and reward calibration.\n* The improvements hinge significantly on the quality of the calibrated reward. Yet, the brute-force search for weights, though effective, may not generalize well or be feasible without VLM feedback access.\n* Although using VLMs (Gemini, GPT-4o) is practical, it inherits their limitations. The assumption that VLM preferences fully reflect human preference is only partially validated. Human evaluations are relatively limited in scope.\n* While DLBS improves reward-aligned outputs, its impact on sample diversity is under-explored. Could it lead to mode collapse or reduced creativity?\n* The learned weight combinations for calibrated rewards differ across datasets. It’s unclear how sensitive performance is to these weights, and whether transfer across domains is viable.", "was_revised": true, "final_justification": "The additional experiments and clarifications are appreciated and have addressed most of my concerns.\n\nHowever, I still have concerns about the sensitivity of performance to the reward weight configuration. While the authors show some transferability, the paper still lacks a clear strategy or guideline on how to select or adapt reward weights in practice.\n\nConsidering the overall quality of the paper, I will maintain my positive score.", "rebuttal": "We thank the reviewer for careful reading and detailed feedback.\n\n**> W1**\n\n> While DLBS integrates known components (beam search, lookahead via DDIM), the method is more of a well-engineered combination.\n\nWe believe that our contributions and implications go beyond a simple combination of existing components; beam search and lookahead via DDIM.\n\nFirst, we provide a unified view of existing inference-time alignment methods through several practical approximations of optimal drift $\\mathbf{u}^∗(\\mathbf{ν}_t, t)$ (**Section 3.1**). To mitigate these approximation errors, we introduce DLBS with an integrated lookahead estimator (**Section 3.2**).\n\nNext, we provide theoretical (**Appendix E**) and empirical (**Appendix L.4**)  analysis of approximation errors for the lookahead estimator, demonstrating that using a few steps (e.g., 6 steps) rather than traditional single-step lookahead (i.e., Tweedie’s formula) can reduce reward estimation errors. \n\nMoreover, through a comprehensive comparison of budget allocation, we show that lookahead achieves larger performance gains from search than increasing beam search budget or diffusion denoising steps (**Section 5.1**).\n\nLastly, we demonstrate that the lookahead estimator is not limited to DDIM, but can be applied to other samplers as well—for example, SDE-DPMSolver++ used with Wan 2.1 (**Appendix D**).\n\n**> W2** \n\n> The improvements hinge significantly on the quality of the calibrated reward. The brute-force search for weights, though effective, may not generalize well or be feasible without VLM feedback access.\n\n- \"not generalize well\"\n\n    First, we have demonstrated generalization across models and schedulers. \n    We confirmed that rewards collected from Wan 2.1 1.3B with DPMSolver++ can improve human evaluation scores for CogVideoX 5B with DDIM solver (**Section 5.3**).\n    \n    Next, we have also confirmed that for in-domain prompts, using the same reward can improve performance on unseen prompts (please see **Figure 8; Right**).\n    \n    Lastly, for out-of-domain prompts and video length generalization, please see **Q1** and **Q2**, respectively.\n\n- \"be feasible without VLM feedback access\"  \nAs explained in **Section 4**, while our current approach uses VLM feedback for reward calibration, this is primarily for practical demands rather than a fundamental necessity. \n\n    Human evaluation remains the gold standard for video quality assessment, but collecting human feedback at the required scale for search is prohibitively expensive and time-consuming. \n\n    A practical approach to reduce time and cost is leveraging AI feedback from VLMs, which has shown modest alignment with human judgment on video quality (**Appendix J**). Since alignment via inference-time search requires massive reward evaluation queries, we need tractable proxy rewards that avoid reliance on humans or external VLM APIs for deployment. Existing lightweight metrics for video naturalness (e.g., `Subject Consistency`, `Dynamic Degree`) often fail to correlate with human/VLM evaluations (**Figure 3**), motivating our reward calibration approach that weights existing metrics using VLMs. While human labels would be ideal for calibration, collecting the extensive metric evaluations required would be even more costly and time-consuming than gathering preference labels from VLM APIs.\n    || Human | VLM | Lightweight Metrics |\n    |---|---|---|---|\n    | Alignment with Human Eval. | ◎ | ○ | △ |\n    | Cost | △ | ○ | ◎ |\n    | Exec. Speed | △ | ○ | ◎ |\n\n    Crucially, to demonstrate the feasibility of reward calibration without VLM APIs, we performed calibration on MSRVTT-test prompts using VideoScore (a reward model trained on human evaluations) and conducted search accordingly, showing that our approach can work in principle without external VLM dependencies.\n    | Method | VideoScore |\n    |---|---|\n    | Vanilla | 2.40 |\n    | DLBS-LA ($KB=4$, $T’=6$) + VideoScore Calibrated Reward | **2.60** |\n\n**> W3**　\n\n> The assumption that VLM preferences fully reflect human preference is only partially validated. Human evaluations are relatively limited in scope.\n\nMost importantly, **Section 5.2** demonstrates that search using VLM-calibrated rewards successfully improves Human Preference and Human Preference-trained Reward Models (VideoScore) across multiple prompt sets and models. \nThis supports the effectiveness of using VLM evaluation as a substitute for human evaluation in improving human evaluation outcomes.\n\nTo strengthen this argument, we conducted human evaluations comparing DLBS-LA ($KB=8$, $T’=6$, $NFE=2500$) and BoN ($KB=64$, $NFE=3200$).\nWhile the calibrated rewards are proxy functions for human evaluation with limited responsiveness, DLBS-LA outperforms BoN in human evaluation despite requiring fewer NFEs.\n\n| Model | Prompt Set | Win Rate of DLBS-LA against BoN | Reward (DLBS-LA) | Reward (BoN) |\n|---|---|---|---|---|\n| Latte | DEVIL-medium | **69.70%** | 0.726 | 0.705 |\n| Latte | MSRVTT-test | **65.56%** | 0.607 | 0.578 |\n| CogVideoX-5B | DEVIL-very-high | **69.70%** | 0.500 | 0.476 |\n| Wan 2.1 14B | MovieGen | **60.00%** | 0.378 | 0.357 |\n\n\n**> Q1 & W5** \n\n> Can the calibrated reward weights learned on DEVIL-high be applied to other datasets (e.g., MSRVTT) with minimal degradation?\n\nTo address concerns about reward transferability, we first note that video generation inherently involves trade-offs between fundamental properties such as dynamics and consistency (**Appendix G**), which may require category-specific calibration for optimal performance. \nHowever, despite these domain-specific requirements, we hypothesize that calibrated rewards can generalize to some extent across different datasets, as they are based on shared principles of perceptual quality.\nTo test this generalization capability, we conducted additional experiments applying the calibrated reward weights calibrated on DEVIL-high and DEVIL-medium to MSRVTT-test prompts. \nWe evaluated the results using VideoScore, a human preference-trained evaluator, measuring five key metrics along with their average scores.\n| (R: Reward, P: Prompt)| Visual Quality | Temporal Consistency | Dynamic Degree | T2V Alignment | Factual Consistency | Average |\n|---|---|---|---|---|---|---|\n| **Vanilla** | 2.32 | 2.01 | **2.91** | 2.67 | 2.07 | 2.40 |\n| **DLBS (KB=8)** | | | | | | |\n| + R=MSRVTT, P=MSRVTT | **2.50** | **2.27** | 2.88 | **2.74** | 2.28 | 2.53 |\n| + R=DEVIL-med, P=MSRVTT | 2.49 | 2.26 | 2.89 | 2.73 | **2.31** | **2.54** |\n| + R=DEVIL-high, P=MSRVTT | 2.36 | 2.03 | ***2.90*** | 2.68 | 2.10 | 2.42 |\n\nWith the DEVIL-high reward, we can enhance other metrics while maintaining dynamics.\nDEVIL-medium reward, which is a closer domain to MSRVTT-test, shows a different trade-off pattern: while it slightly reduces dynamics, it significantly improves other metrics and achieves a higher average score than the MSRVTT-test reward, demonstrating higher transferability.\n\nMoreover, although we fixed the weight of rewards per category in the paper,  we demonstrate that expanding prompt coverage enables more flexible reward design as in **W3 from Reviewer 3Psp**.\n\nWe believe that, while our reward calibration is designed for specialization to specific prompt sets, it can also achieve generalization to some extent. In particular, we can also do reward calibration against all the prompt sets jointly, which yields a good reward that works.\n\n**> Q2** \n\n> How does DLBS perform when applied to longer or higher-resolution video generation tasks (e.g., >8s or >512×512)?\n\n- Longer video generation   \n\n    Our method is scalable even when extending the frame length to the model's maximum. First, note that the maximum frame length and duration of open-source SoTA video DiT models are: Wan 2.1: 81 frames, 5s (16fps), CogVideoX: 49 frames, 6s (8fps), Hunyan Video [1]: 129 frames, 5s (24fps). Therefore, experiments with >8s duration are difficult to conduct.  \n\n    To demonstrate that our method is scalable even when extending the frame count to the model's maximum, we conducted additional experiments with the maximum frames for Wan 2.1 1.3B and CogVideoX 5B. \n    The reward, which was calibrated using 33 frames, 2s videos generated by Wan 2.1 1.3B, was applied as-is.\n    As a result, we confirmed that even with longer frames, the reward values could be improved more efficiently than the BoN baseline.\n\n    - CogVideoX 5B, 49 frames (6s), DEVIL-very-high prompts\n    | Method | $KB$ | Reward | Inference Compute (NFE) |\n    |---|---|---|---|\n    | Vanilla | 1 | 0.429 | 50 |\n    | BoN | 64 | 0.474 | 3200 |\n    | BoN | 128 | 0.478 | 6400 |\n    | DLBS | 16 | 0.481 | 1200 |\n    | DLBS | 32 | 0.490 | 2400 |\n    | DLBS-LA | 8 | **0.497** | 2500 |\n    | DLBS-LA | 16 | **0.517** | 4900 |\n\n    - Wan 2.1 1.3B, 81 frames (5s), MovieGen prompts\n    | Method | $KB$ | Reward | Inference Compute (NFE) |\n    |---|---|---|---|\n    | Vanilla | 1 | 0.313 | 50 |\n    | BoN | 128 | 0.357 | 6400 |\n    | BoN | 256 | 0.360 | 12800 |\n    | DLBS | 16 | 0.357 | 1200 |\n    | DLBS | 32 | 0.371 | 2400 |\n    | DLBS-LA | 8 | **0.373** | 2500 |\n    | DLBS-LA | 16 | **0.393** | 5000 |\n\n- Higher-resolution video generation  \n\n    The resolution of the CogVideoX & Wan experiments conducted in **Section 5.3** is 832 * 480 > 512 * 512.\n    The experimental configurations are shown in **Appendix B**.\n\n**> Q3 & W4**\n\n> Does DLBS reduce diversity due to beam pruning toward high-reward paths?\n\nPlease see **Figure 24** in **Appendix L.7**, where we have reported the reward–diversity trade‑off. \nDLBS achieves high performance while maintaining higher diversity than either BoN or GS. \nThis exhibits a benefit from the wider possession and exploration of diffusion latents in DLBS, compared to BoN or GS.\n\n[1] Hunyuan Foundation Model Team. HunyuanVideo: A Systematic Framework For Large Video Generative Models. arXiv2025."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field, with 4 Boarderline Accept recommendations. All the reviewers appreciate that this paper proposes a novel approach to investigate inference time beam search for diffusion video generation, with reasonable technical improvements and relatively comprehensive experiments. Thus, this paper achieves the NeurIPS acceptance requirement. AC highly urges the authors to go through the detailed comments carefully to polish the writing and provide extra experimental details, so as to ensure the final acceptance.
Decision: Accept (poster)