Title: Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search
Venue: NeurIPS 2025
Accepted: yes
Keywords: Inference-Time Search

Text-to-Video Generation

Diffusion Models

Alignment
Abstract: The remarkable progress in text-to-video diffusion models enables the generation of photorealistic videos, although the content of these generated videos often includes unnatural movement or deformation, reverse playback, and motionless scenes. Recently, an alignment problem has attracted huge attention, where we steer the output of diffusion models based on some measure of the content's goodness. Because there is a large room for improvement of perceptual quality along the frame direction, we should address which metrics we should optimize and how we can optimize them in the video generation. In this paper, we propose diffusion latent beam search with lookahead estimator, which can select a better diffusion latent to maximize a given alignment reward at inference time. We then point out that improving perceptual video quality with respect to alignment to prompts requires reward calibration by weighting existing metrics. This is because when humans or vision language models evaluate outputs, many previous metrics to quantify the naturalness of video do not always correlate with the evaluation. We demonstrate that our method improves the perceptual quality evaluated on the calibrated reward, VLMs, and human assessment, without model parameter update, and outputs the best generation compared to greedy search and best-of-N sampling under much more efficient computational cost. The experiments highlight that our method is beneficial to many capable generative models, and provide a practical guideline: we should prioritize the inference-time compute allocation into enabling the lookahead estimator and increasing the search budget, rather than expanding the denoising steps.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Diffusion Latent Beam Search (DLBS), a gradient-free inference-time search algorithm for aligning text-to-video diffusion model outputs with perceptual quality and prompt intent. The method is developed based on the beam search, which can select a better diffusion latent to maximize a given alignment reward at inference time. To address the noisy evaluation of intermediate latents, the authors propose a lookahead estimator using deterministic DDIM steps to better approximate the final output. Additionally, the paper highlights a significant issue in current evaluation: existing perceptual metrics (e.g., smoothness, consistency) correlate poorly with human or VLM preferences. To solve this, the authors introduce a reward calibration step, learning weighted combinations of existing metrics to better match Gemini/GPT feedback.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The proposed DLBS requires no model retraining, is plug-and-play, and integrates well with any existing text-to-video diffusion model.\n* The recognition that no single perceptual metric aligns well with VLM or human feedback is important and convincingly demonstrated. The calibrated reward design adds real value.\n* The method is benchmarked across datasets (MSRVTT, DEVIL, MovieGen), prompt difficulty levels (static, medium, dynamic), and model scales (Latte, CogVideoX, Wan 2.1). Includes both automatic and human feedback.\n\nWeaknesses:\n* While DLBS integrates known components (beam search, lookahead via DDIM), the method is more of a well-engineered combination. Its novelty lies more in system design and reward calibration.\n* The improvements hinge significantly on the quality of the calibrated reward. Yet, the brute-force search for weights, though effective, may not generalize well or be feasible without VLM feedback access.\n* Although using VLMs (Gemini, GPT-4o) is practical, it inherits their limitations. The assumption that VLM preferences fully reflect human preference is only partially validated. Human evaluations are relatively limited in scope.\n* While DLBS improves reward-aligned outputs, its impact on sample diversity is under-explored. Could it lead to mode collapse or reduced creativity?\n* The learned weight combinations for calibrated rewards differ across datasets. It’s unclear how sensitive performance is to these weights, and whether transfer across domains is viable.\n\nQUESTIONS:\n1. Can the calibrated reward weights learned on DEVIL-high be applied to other datasets (e.g., MSRVTT) with minimal degradation? Or must each dataset have its own calibration?\n2. How does DLBS perform when applied to longer or higher-resolution video generation tasks (e.g., >8s or >512×512)?\n3. Does DLBS reduce diversity due to beam pruning toward high-reward paths? Could you include diversity metrics to evaluate this issue?\n\nLIMITATIONS:\nThe authors discuss some limitations in Section 7.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces latent beam search at inference time to align the generated videos with input prompts. There are two main contributions: 1) It proposes latent beam search and lookahead estimator for inference-time scaling. It samples K latents and selecting B beams by evaluating the reward through lookahead estimator. 2) It proposes a reward calibration algorithm to better evaluate the quality of generated videos.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1) The problem explored in this paper: how to perform test-time scaling in diffusion models is a very meaningful one. The proposed beam search method is straightforward and solid.\n2) This paper's writing is good and the method is easy to understand.\n\nWeaknesses:\nSee questions.\n\nQUESTIONS:\n1) Computational inefficiency and limited improvement. According to Figure.1 (right) and Figure.6 (left), with $2^5$x increase in inference computation time, the improvement is only around 10%. Given the additional computation required, I think the improvement is far from sufficient. Moreover, although the proposed method shows improved performance with increased inference time, the simplest BoN baseline also exhibits a similar trend, and the improvement over BoN is also limited (less than 10%). Therefore, I believe the proposed method gives very limited improvement in test-time scaling performance.\n2) Beam search is still random search. At each timestep, the method still introduces random noise to generate different beam candidates. This is essentially not different from Best of N (BoN), except that it expands the space of random search. BoN's randomness lies in the initial latent, while beam search introduces randomness at every timestep. I believe the effectiveness of such random search is inherently limited. Although the authors use classifier guidance to show that the search process is moving toward the target, the zero-order nature of the random search seems not enough. If it were possible to compute first- or higher-order gradients with respect to the target and update accordingly, I believe that would be more meaningful.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper tackles the problem of alignment in text-to-video diffusion, making generated videos both follow a text prompt and high perceptual quality in inference time.
The Diffusion Latent Beam Search and a lookahead estimator are proposed to obtain a suitable sample at each denoising step.
The authors also introduced a weight linear combination of multiple metrics that aligned with perceptual quality and human preference.

STRENGTHS AND WEAKNESSES:
strength:
1. the paper proposed a beam search for inference-time alignment in the latent space.
2.  systematic study of six perceptual metrics correlations with VLM/Human preferences.
3. extensive experiments on different parameters (e.g., $B, H, T^\prime$)
4. Effectively improve the performance across different diffusion model variants.

---
weakness:
1. calibration relies on a strong VLM (e.g., Gemini, GPT). Without these models, it might be challenging to construct reliable metrics.
2. efforts of tuning ($B, K, T^\prime$) can be expensive.

QUESTIONS:
1. The LA estimator selects $T^\prime=6$. Does it work for different models? Does it hold for different samplers?
2. Beam size and candidate count interact with complexity. Have you explored any dynamic or adaptive strategies to adjust ($B, K$) on-the-fly based on intermediate reward?

LIMITATIONS:
The beam search can reduce the inference cost compared to best-fo-N. But the still takes tens of seconds per video on high-end GPUs, limiting real-time or large-scale use.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses a challenge of perceptual misalignment in text-to-video diffusion models, where generated videos often contain unnatural motion, temporal inconsistencies or static scenes. The authors propose Diffusion Latent Beam Search (DLBS) with lookahead estimator, an inference-time method that optimizes video quality by selecting a better diffusion latent to maximize a given alignment reward. DLBS leverages beam search over diffusion latents guided by a lookahead estimator to reduce noise in reward evaluation. The authors also provide a technique for reward calibration by leveraging AI feedback from VLMs. The higher a strong VLM estimates the generated video quality, the better visual quality it would have from a human point of view. Six base reward functions for perceptual video quality are used: Subject Consistency, Motion Smoothness, Dynamic Degree, Aesthetic Quality, Imaging Quality and Text-Video Consistency. At the experimental stage the authors demonstrate that DLBS is the most scalable, efficient and robust inference-time search that significantly improves video quality under the same computational costs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths.\n\nThe authors provide a strong vision to better denoising way in latent space which is the core contribution of the paper. The ablations and correlations are thorough. Human evaluations (pairwise comparisons) and diverse prompts (DEVIL, MSRVTT, MovieGen) strengthen validity.\n\n\nWeaknesses.\n\nWhile VLMs are used as human proxies, their correlation with human evaluation is moderate (nearly 0.5 correlation). Relying on VLMs without deeper analysis of failure cases (e.g., prompt ambiguity) provides additional risks for reward over optimization and highly biasing. The computational cost is not presented at the experimental stage. The rewards use fixed weights per prompt category after calibration, but prompt adaptation is unexplored. \n\nClarity. The text is well-structured and mostly clear. Figures effectively illustrate the method and results. The pseudocode and appendices sections aid reproducibility.\n\nSignificance. The paper addresses a critical gap in text-to-video generation. The inference time approach is model-agnostic, scalable (tested up to 14B params) and deployable without additional training or fine-tuning. The efficiency claims show readiness to real-world use.\n\nOriginality. DLBS adapts a well-known beam search paradigm to diffusion latents, while reward calibration and lookahead estimation offer novel solutions to reward noise and misalignment.\n\nQUESTIONS:\nI would advice to make additional comments on generalization of VLM-Human correlation and reward calibration to improve the paper strengths. Ablations on solvers (e.g., DPMSolver++) or nu values would clarify optimization choices instead of just using deterministic DDIM (nu=0) at the experimental setup.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}