T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback

Authors: jiachen  li
weixi  feng
tsu-jui  fu
xinyi  wang
sugato  basu
wenhu  chen
william yang wang
uc santa barbara
  google
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 22:42:33

Abstract: Diffusion-based text-to-video (T2V) models have achieved significant success but continue to be hampered by the slow sampling speed of their iterative sampling processes. To address the challenge, consistency models have been proposed to facilitate fast inference, albeit at the cost of sample quality. In this work, we aim to break the quality bottleneck of a video consistency model (VCM) to achieve **both fast and high-quality video generation**. We introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. Notably, we directly optimize rewards associated with single-step generations that arise naturally from computing the CD loss, effectively bypassing the memory constraints imposed by backpropagating gradients through an iterative sampling process. Remarkably, the 4-step generations from our T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and Pika. We further conduct human evaluations to corroborate the results, validating that the 4-step generations from our T2V-Turbo are preferred over the 50-step DDIM samples from their teacher models, representing more than a tenfold acceleration while improving video generation quality.

Main paper: 
 introduction: 
 Diffusion model (DM) [Sohl-Dickstein et al., 2015 , Ho et al., 2020] has emerged as a powerful framework for neural image [Betker et al., 2023 , Rombach et al., 2022 , Esser et al., 2024 , Saharia et al., 2022] and video synthesis [Singer et al., 2022 , Ho et al., 2022a , He et al., 2022 , Wang et al., 2023b , Zhang et al., 2023] , leading to the development of cutting-edge text-to-video (T2V) models like Sora [Brooks et al., 2024] , Gen-2 [Esser et al., 2023] and Pika [Pika Labs, 2023] . Although the iterative sampling process of these diffusion-based models ensures high-quality generation, it significantly slows down inference, hindering their real-time applications. On the other hand, existing opensourced T2V models including VideoCrafter [Chen et al., 2023 [Chen et al., , 2024] ] and ModelScopeT2V [Wang et al., 2023c] are trained on web-scale video datasets, e.g., WebVid-10M [Bain et al., 2021] , with varying video qualities. Consequently, the generated videos often appear visually unappealing and fail to align accurately with the text prompts, deviating from human preferences.Efforts have been made to address the issues listed above. To accelerate the inference process, Wang et al. [2023a] applies the theory of consistency distillation (CD) [Song et al., 2023 , Song and Dhariwal, 2023 , Luo et al., 2023a] to distill a video consistency model (VCM) from a teacher T2V model, enabling plausible video generations in just 4-8 inference steps. However, the quality of VCM's generations is naturally bottlenecked by the performance of the teacher model, and the reduced number of inference steps further diminishes its generation quality. On the other hand, to align generated videos with human preferences, InstructVideo [Yuan et al., 2023] draws inspiration from image generation techniques [Dong et al., 2023 , Clark et al., 2023 , Prabhudesai et al., 2023] and proposes backpropagating the gradients of a differentiable reward model (RM) through the iterative video sampling process. However, calculating the full reward gradient is prohibitively expensive, resulting in substantial memory costs. Consequently, InstructVideo truncates the sampling chain by limiting gradient calculation to only the final DDIM step, compromising optimization accuracy. Additionally, InstructVideo is limited by its reliance on an image-text RM, which fails to fully capture the transition dynamic of a video. Empirically, InstructVideo only conducts experiments on a limited set of user prompts, the majority of which are related to animals. As a result, its generalizability to a broader range of prompts remains unknown.In this paper, we aim to achieve fast and high-quality video generation by breaking the quality bottleneck of a VCM. We introduce T2V-Turbo, which integrates reward feedback from a mixture of RMs into the process of distilling a VCM from a teacher T2V model. Besides utilizing an image-text RM to align individual video frames with human preference, we further incorporate reward feedback from a video-text RM to comprehensively evaluate the temporal dynamics and transitions in the generated videos. We highlight that our reward optimization avoids tackling the highly memoryintensive issues associated with backpropagating gradients through an iterative sampling process. Instead, we directly optimize rewards of the single-step generations that arise from computing the CD loss, effectively bypassing the memory constraints faced by conventional methods that optimize a DM [Yuan et al., 2023 , Xu et al., 2024a , Clark et al., 2023 , Prabhudesai et al., 2023] .Empirically, we demonstrate the superiority of our T2V-Turbo in generating high-quality videos within 4-8 inference steps. To illustrate the applicability of our methods, we distill T2V-Turbo (VC2) and T2V-Turbo (MS) from VideoCrafter2 [Chen et al., 2024] and ModelScopeT2V [Wang et al., 2023c] , respectively. Remarkably, the 4-step generation results from both variants of our T2V-Turbo outperform SOTA models on the video evaluation benchmark VBench [Huang et al., 2024] , even surpassing proprietary systems such as Gen-2 [Esser et al., 2023] and Pika [Pika Labs, 2023 ] that are trained with extensive resources. We further corroborate the results by conducting human evaluation using 700 prompts from the EvalCrafter [Liu et al., 2023] benchmark, validating that the 4-step generations from T2V-Turbo are favored by human over the 50-step DDIM samples from their teacher T2V models, which represents over tenfold inference acceleration and enhanced video generation quality.Our contributions are threefold:• Learn a T2V model with feedback from a mixture of RMs, including a video-text model. To the best of our knowledge, we are the first to do so. • Establish a new SOTA on the VBench with only 4 inference steps, outperforming proprietary models trained with substantial resources. • 4-step generations from our T2V-Turbo are favored over the 50-step generation from its teacher T2V model as evidenced by human evaluation, representing over 10 times inference acceleration with quality improvement. 
preliminaries: 
 Diffusion models (DMs). In the forward process, DMs progressively inject Gaussian noise into the original data distribution p data (x) ≡ p 0 (x 0 ) and perturb it into a marginal distribution p t (x t ) with the transition kernel p 0t (x t |x 0 ) = N (x t |α(t)x 0 , β 2 (t)I) at timestep t. α(t) and β(t) correspond to the noise schedule. In the reverse process, DMs sequentially recover the data from a noise sampled from the prior distribution p T (x T ) := N (x T |0, β 2 (T )I). The reverse-time SDE can be modeled by an ordinary differential equation (ODE), known as the Probability Flow (PF-ODE) [Song et al., 2020a] :EQUATIONwhere µ(•) and σ(•) are the drift and diffusion coefficients, respectively, with the following properties:EQUATIONThe PF-ODE's solution trajectories, when sampled at any timestep t, align with the distribution p t (x t ).Empirically, a denoising model ϵ θ (x t , t) is trained to approximate the score function -∇ log p t (x t ) via score matching. During the sampling phase, one begins with a sample x T ∼ p T (x T ) and follows the empirical PF-ODE below to obtain a sample x0 .EQUATIONIn this paper, we focus on diffusion-based T2V models, which operate on the video latent space Z and train a denoising model ϵ θ (z t , c, t) conditioned on the text prompt c, where z t is obtained by perturbing the image latent z = E(x), ∈ Z and E is a VAE [Kingma and Welling, 2013] encoder. The T2V models employ Classifier-Free Guidance (CFG) [Ho and Salimans, 2021] to enhance the quality of conditional sampling by substituting the noise prediction with a linear combination of conditional and unconditional noise predictions for denoising, i.e., εθ (z t , ω, c, t)= (1+ω)ϵ θ (z t , c, t)-ωϵ θ (z t , ∅, t),where ω is the CFG scale. After the completion of the inference process, we can generate a video by x0 = D(z 0 ) with the VAE decoder D corresponding to E.Consistency Distillation. Conventional methods [Ho et al., 2020 , Song et al., 2020b] generate their samples by solving the PF-ODE sequentially, leading to DM's slow inference speed. To tackle this problem, consistency models (CM) [Song et al., 2023, Song and Dhariwal, 2023] propose to learn a consistency function f : (x t , t) → x ϵ to directly map any x t on the PF-ODE trajectory to its origin, where ϵ is a fixed small positive number. And thus, the consistency function f has the following self-consistency propertyEQUATION) where x t and x ′ t are from the same PF-ODE. We can model f with a CM f θ . When tackling the PF-ODE of a T2V model that operates on the video latent space Z, we aim to learn a video consistency model (VCM) [Luo et al., 2023a , Wang et al., 2023a] EQUATIONwhere c skip (t) and c out (t) are differentiable functions with c skip (ϵ) = 1 and c out (ϵ) = 0, and F θ is modeled as a neural network. We can distill a f θ from a pre-trained T2V DM by minimizing the consistency distillation (CD) [Song et al., 2023 , Luo et al., 2023a ] loss as belowEQUATIONwhere d( We follow the LCM paper [Luo et al., 2023a] to use DDIM [Song et al., 2020b] as the ODE solver Ψ and defer the formula of the DDIM solver to Appendix A.3 Training T2V-Turbo with Mixed Reward FeedbackIn this section, we present the training pipeline to derive our T2V-Turbo. To facilitate fast and high-quality video generation, we integrate reward feedback from multiple RMs into the LCD process when distilling from a teacher T2V model. Figure 2 provides an overview of our framework. Notably, we directly leverage the single-step generation ẑ0 = f θ z t n+k , ω, c, t n+k arise from computing the CD loss L CD (6) and optimize the video x0 = D(ẑ 0 ) decoded from it towards multiple differentiable RMs. As a result, we avoid the challenges associated with backpropagating gradients through an iterative sampling process, which is often confronted by conventional methods optimizing DMs [Clark et al., 2023 , Xu et al., 2024a , Yuan et al., 2023] .In particular, we leverage reward feedback from an image-text RM to improve human preference on each individual video frame (Sec. 3.1) and further utilize the feedback from a video-text RM to improve the temporal dynamics and transitions in the generated video (Sec. 3.2). 
optimizing human preference on individual video frames: 
 Chen et al. [2024] achieve high-quality video generation by including high-quality images as singleframe videos when training the T2V model. Inspired by their success, we align each individual video frame with human preference by optimizing towards a differentiable image-text RM R img . In particular, we randomly sample a batch of M frames {x 1 0 , . . . , xM 0 } from the decoded video x0 and maximize their scores evaluated by R img as belowEQUATION)Existing image-text RMs [Wu et al., 2023a , Xu et al., 2024a , Kirstain et al., 2024] are limited to assessing the alignment between individual video frames and the text prompt and thus cannot evaluate through the temporal dimensions that involve inter-frame dependencies, such as motion dynamic and transitions [Huang et al., 2024 , Liu et al., 2023] . To address these shortcomings, we further leverage a video-text RM R vid to assess the generated videos. The corresponding objective J vid is given belowJ vid (θ) = E x0,c [R vid (x 0 , c)] , x0 = D f θ z t n+k , ω, c, t n+k . (9)To this end, we can define the total learning loss L of our training pipeline as a linear combination of the L CD in (6), J img in (8), and J vid in (9) with weighting parameters β img and β vid .EQUATIONTo reduce memory and computational cost, we initialize our T2V-Turbo with the teacher model and only optimize the LoRA weights [Hu et al., 2021 , Luo et al., 2023b] instead of performing full model training. After completing the training, we merge the LoRA weights so that the per-step inference cost of our T2V-Turbo remains identical to the teacher model. We include pseudo-codes for our training algorithm in Appendix B. 
experimental results: 
 Our experiments aim to demonstrate our T2V-Turbo's ability to generate high-quality videos with 4-8 inference steps. We first conduct automatic evaluations on the standard benchmark VBench [Huang et al., 2024] to comprehensively evaluate our methods from various dimensions (Sec. 4.1) against a broad array of baseline methods. We then perform human evaluations with 700 prompts from the EvalCrafter [Liu et al., 2023] to compare the 4-step and 8-step generations from our T2V-Turbo with the 50-step generations from the teacher T2V models as well as the 4-step generations from the baseline VCM (Sec. 4.2). Finally, we perform ablation studies on critical design choices (Sec. 4.3).Settings. We train T2V-Turbo (VC2) and T2V-Turbo (MS) by distilling from the teacher diffusionbased T2V models VideoCrafter2 [Chen et al., 2024] and ModelScopeT2V [Wang et al., 2023c] , respectively. Similar to both teacher models, we conduct our training using the WebVid10M [Bain et al., 2021] datasets. We train our models on 8 NVIDIA A100 GPUs for 10K gradient steps without gradient accumulation. We set the batch size of training videos to 1 for each GPU device. We employ HPSv2.1 [Wu et al., 2023a] as our image-text RM R img . When distilling from VideoCrafter2, we utilize the 2nd Stage model of InternVideo2 (InternVid2 S2) [Wang et al., 2024] as our video-text RM R vid . When distilling from ModelScopeT2V, we set R vid to be ViCLIP [Wang et al., 2023d] .To optimize J img (8), we randomly sample 6 frames from the video by setting M = 6. For the hyperparameters (HP), we set learning rate 1e -5 and guidance scale range [ω min , ω max ] = [5, 15] . We use DDIM [Song et al., 2020b] as our ODE solver Ψ and set the skipping step k = 20. For T2V-Turbo (VC2), we set β img = 1 and β vid = 2. For T2V-Turbo (MS), we set β img = 2 and β vid = 3. We include further training details in Appendix A.We evaluate our T2V-Turbo (VC2) and T2V-Turbo (MS) on the standard video evaluation benchmark VBench [Huang et al., 2024] to compare against a wide array of baseline methods. VBench is designed to comprehensively evaluate T2V models from 16 disentangled dimensions. Each dimension in VBench is tailored with specific prompts and evaluation methods.Table 1 compares the 4-step generation of our methods with various baselines from the VBench leaderboard1 , including Gen-2 [Esser et al., 2023] , Pika [Pika Labs, 2023] , VideoCrafter1 [Chen et al., 2023] , VideoCrafter2 [Chen et al., 2024] , Show-1 [Zhang et al., 2023] , LaVie [Wang et al., 2023b] , and ModelScopeT2V [Wang et al., 2023c] . Table 4 in Appendix further compares our methods with VideoCrafter0.9 [He et al., 2022] , LaVie-Interpolation [Wang et al., 2023b] , Open-Sora [Open-Sora, 2024], and CogVideo [Hong et al., 2022] . The performance of each baseline method is directly reported from the VBench leaderboard. To obtain the results of our methods, we carefully follow VBench's evaluation protocols by generating 5 videos for each prompt to calculate the metrics. WeTable 1 : Automatic Evaluation on VBench [Huang et al., 2024] . We compare our T2V-Turbo (VC2) and T2V-Turbo (MS) with baseline methods across the 16 VBench dimensions. A higher score indicates better performance for a particular dimension. We bold the best results for each dimension and underline the second-best result. Quality Score is calculated with the 7 dimensions from the top table. Semantic Score is calculated with the 9 dimensions from the bottom table.Total Score a weighted sum of Quality Score and Semantic Score. Further details can be found in Appendix C. Both our T2V-Turbo (VC2) and T2V-Turbo (MS) surpass all baseline methods with 4 inference steps in terms of Total Score, including the proprietary systems Gen-2 and Pika. 1 , the 4-step generations of both our T2V-Turbo (MS) and T2V-Turbo (VC2) surpass all baseline methods on VBench in terms of Total Score. These results are particularly remarkable given that we even outperform the proprietary systems Gen-2 and Pika, which are trained with extensive resources. Even when distilling from a less advanced teacher model, ModelScopeT2V, our T2V-Turbo (MS) attains the second-highest Total Score, just below our T2V-Turbo (VC2). Additionally, our T2V-Turbo breaks the quality bottleneck of a VCM by outperforming its teacher T2V model, significantly improving over the baseline VCM.To verify the effectiveness of our T2V-Turbo, we compare the 4-step and 8-step generations from our T2V-Turbo with the 50-step DDIM samples from the corresponding teacher T2V models. We further compare the 4-step generations between our T2V-Turbo and their baseline VCMs when distilled from the same teacher T2V model. We leverage the 700 prompts from the EvalCrafter [Liu et al., 2023] video evaluation benchmark, which are constructed based on real-world user data. We hire human annotators from Amazon Mechanical Turk to compare videos generated from different models given the same prompt. For each comparison, the annotators need to answer three questions: Q1) Which video is more visually appealing? Q2) Which video better fits the text description? Q3) Which video do you prefer given the prompt? Appendix D includes additional details about how we set up the human evaluations.Figure 3 provides the full human evaluation results. We also qualitatively compare different methods in Figure 4 . Due to limited space, we include additional qualitative comparison results in Appendix F. Notably, the 4-step generations from our T2V-Turbo are favored by humans over the 50-step generation from their teacher T2V model, representing a 12.5 times inference acceleration with improving performance. By increasing the inference steps to 8, we can further improve the visual quality and text-video alignment of videos generated from our T2V-Turbo, reflected by the fact that our 8-step generations are more likely to be favored by the human compared to our 4-step generations in terms of all 3 evaluated metrics. Additionally, our T2V-Turbo significantly outperforms its baseline VCM, demonstrating the effectiveness of our methods, which incorporate a mixture of reward feedback into the model training. R vid = ViCLIP T2V-Turbo (VC2) R vid = InternVid S2 T2V-Turbo (VC2) R vid = ViCLIP T2V-Turbo (MS) R vid = InternVid S2 T2V-Turbo (MS) 
related work: 
 Diffusion-based T2V Models. Many diffusion-based T2V models rely on large-scale image datasets for training [Ho et al., 2022a , Wang et al., 2023c , Chen et al., 2023] or inherit weights from pre-trained text-to-image (T2I) models [Zhang et al., 2023 , Blattmann et al., 2023 , Khachatryan et al., 2023] .The scale of text-image datasets [Schuhmann et al., 2022] is usually more than ten times the scale of open-sourced video-text datasets [Bain et al., 2021 , Wang et al., 2023d] and with higher spatial resolution and diversity [Wang et al., 2023c] . For example, Imagen Video [Ho et al., 2022b] discovers that joint training on a mix of image and video datasets improves the overall visual quality and enables the generation of videos in novel styles. Models trained with WebVid-10M [Bain et al., 2021] like ModelScopeT2V [Wang et al., 2023c] or VideoCrafter [Chen et al., 2023] also treat images as a single-frame video, and use them to improve video qualities. LaVie [Wang et al., 2023b] initialize the training with WebVid-10M and LAION-5B and then continue the training with a curated internal dataset of 23M videos. To overcome the data scarcity of high-quality videos, VideoCrafter2 [Chen et al., 2024] proposes to disentangle motion from appearance at the data level so that it can be trained on high-quality images and low-quality videos. The data limitation of high-quality videos and aligned, accurate video captions has been a longstanding bottleneck of current T2V models. In this paper, we propose to combat this challenge by leveraging reward feedback from a mixture of RMs.Accelerating inference of Diffusion Models. Various methods have been proposed to accelerate the sampling process of a DM, including advanced numerical ODE solvers [Song et al., 2020b , Lu et al., 2022a ,b, Zheng et al., 2022 , Dockhorn et al., 2022 , Jolicoeur-Martineau et al., 2021] and distillation techniques [Luhman and Luhman, 2021 , Salimans and Ho, 2021 , Meng et al., 2023 , Zheng et al., 2023] . Recently, Consistency Model [Song et al., 2023 , Luo et al., 2023a] is proposed to facilitate fast inference by learning a consistency function to map any point at the ODE trajectory to the origin. Li et al. [2024] proposes to augment consistency distillation with an objective to optimize image-text RM to achieve fast and high-quality image generation. Our work extends it for T2V generation, incorporating reward feedback from both an image-text RM and a video-text RM.Vision-and-language Reward Models. There have been various open-sourced image-text RMs that are trained to mirror human preferences given a text-image pair, including HPS [Wu et al., 2023b,a] , ImageReward [Xu et al., 2024a] , and PickScore [Kirstain et al., 2024] , which are obtained by finetuning a image-text foundation model such as CLIP [Radford et al., 2021] and BLIP [Li et al., 2022] , on human preference data. However, to the best of our knowledge, no video-text RMs, e.g., T2VScore [Wu et al., 2024] , that mirrors human preference on a text-video pair has been released to the public. In this paper, we choose HPSv2.1 as our image-text RM and directly employ the video foundation models ViCLIP [Wang et al., 2023d] and InterVid S2 [Wang et al., 2024] that are trained for general video-text understanding as our video-text RM. Empirically, we show that incorporating feedback from these RMs can improve the performance of our T2V-Turbo.Learning from Human/AI Feedback has been proven as an effective way to align the output from a generative model with human preference [Leike et al., 2018 , Ziegler et al., 2019 , Ouyang et al., 2022 , Stiennon et al., 2020 , Rafailov et al., 2024 , Xu et al., 2024b] . In the field of image generation, various methods have been proposed to align a text-to-image model with human preference, including RL [Sutton and Barto, 2018 , Li et al., 2020 , 2023a] based methods [Fan et al., 2024 , Prabhudesai et al., 2023 , Zhang et al., 2024] and backpropagation-based reward finetuning methods [Clark et al., 2023 , Xu et al., 2024a , Prabhudesai et al., 2023] . Recently, InstructVideo [Yuan et al., 2023] extends the reward-finetuning methods to optimize a T2V model. However, it still employs an image-text RM to provide reward feedback without considering the transition dynamic of the generated video. In contrast, our work incorporates reward feedback from both an image-text and video-text RM, providing comprehensive feedback to our T2V-Turbo. 
conclusion and limitations: 
 In this paper, we propose T2V-Turbo, achieving both fast and high-quality T2V generation by breaking the quality bottleneck of a VCM. Specifically, we integrate mixed reward feedback into the VCD process of a teacher T2V model. Empirically, we illustrate the applicability of our methods by distilling T2V-Turbo (VC2) and T2V-Turbo (MS) from VideoCrafter2 [Chen et al., 2024] and ModelScopeT2V [Wang et al., 2023c] , respectively. Remarkably, the 4-step generations from both our T2V-Turbo outperform SOTA methods on VBench [Huang et al., 2024] , even surpassing their teacher T2V models and proprietary systems including Gen-2 [Esser et al., 2023] and Pika [Pika Labs, 2023] . Our human evaluation further corroborates the results, showing the 4-step generations from our T2V-Turbo are favored by humans over the 50-step DDIM samples from their teacher, which represents over ten-fold inference acceleration with quality improvement.While our T2V-Turbo marks a critical advancement in efficient T2V synthesis, it is important to recognize certain limitations. Our approach utilizes a mixture of RMs, including a video-text RM R vid . Due to the lack of an open-sourced video-text RM trained to reflect human preferences on video-text pairs, we instead use video foundation models such as ViCLIP [Wang et al., 2023d] and InternVid S2 [Wang et al., 2024] as our R vid . Although incorporating feedback from these models has enhanced our T2V-Turbo's performance, future research should explore the use of a more advanced R vid for training feedback, which could lead to further performance improvements. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We conform, in every respect, with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the broader impact of our work in Sec. G.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes] Justification: We have discussed this in Appendix G.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have make sure that we have followed all the rules.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: We include the pseudo-codes for training our T2V-Turbo in Algorithm 1. We use the red color to highlight the difference from the standard (latent) consistency distillation [Luo et al., 2023a , Song et al., 2023] . Encoding training data into latent space:D z = {(z, c) | z = E(x), (x, c) ∈ D} θ -← θ repeat Sample (z, c) ∼ D z , n ∼ U[1, N -k] and ω ∼ [ω min , ω max ] Sample z t n+k ∼ N α (t n+k ) z; σ 2 (t n+k ) I ẑΨ,ω tn ← z t n+k + (1 + ω)Ψ z t n+k , t n+k , t n , c -ωΨ z t n+k , t n+k , t n , ∅ x0 = D f θ z t n+k , ω, c, t n+k J img (θ) = E x0,c M m=1 R img (x m 0 , c) J vid (θ) = E x0,c [R vid (x 0 , c)] L CD = d f θ z t n+k , ω, c, t n+k , f θ - ẑΨ,ω tn , ω, c, t n L (θ, θ -; Ψ) ← L CD -β img J img (θ) -β vid J vid (θ) θ ← θ -η∇ θ L (θ, θ -) θ -← stop_grad (µθ -+ (1 -µ)θ) until convergenceTable 4 : Automatic Evaluation on VBench [Huang et al., 2024] . We compare our T2V-Turbo (VC2) and T2V-Turbo (MS) with baseline methods across the 16 VBench dimensions. A higher score indicates better performance for a particular dimension. We bold the best results for each dimension and underline the second-best result. Quality Score is calculated with the 7 dimensions from the top table. Semantic Score is calculated with the 9 dimensions from the bottom table. Total Score a weighted sum of Quality Score and Semantic Score. Both our T2V-Turbo (VC2) and T2V-Turbo (MS) surpass all baseline methods with 4 inference steps in terms of Total Score, including the proprietary systems Gen-2 and Pika. Figure 5 shows the user interface displayed to the labelers when conducting our human evaluations. Each method generate videos of 16 frames using the 700 prompts from EvalCrafter [Liu et al., 2023] .For our T2V-Turbo (VC2), we collect its 4-step and 8-step generations and compare them with the 50-step DDIM samples from its teacher VideoCrafter2. For our T2V-Turbo (MS), we collect its 2-step and 4-step generations and compare them with the 50-step DDIM samples from its teacher ModelScopeT2V. We also compare the 4-step generations between our T2V-Turbo their baseline VCM, demonstrating the significant quality improvement of our methods.As mentioned in Sec. 4.2, we hire labelers from Amazon Mechanical Turk platform and form the video comparison task as many batches of HITs. Specifically, we choose labelers from Englishspeaking countries, including AU, CA, NZ, GB, and the US. Each task needs around 30 seconds to complete, and we pay each submitted HIT with 0.2 US dollars. Therefore, the hourly payment is about 24 US dollars.We note that the data annotation part of our project is classified as exempt by Human Subject Committee via IRB protocols. In this section, we provide the full ablation results performed in Table 3 , which can be found in Table 5 . We further examine the effect of different choices of R img . In the initial stage of our project, we train VCM (VC2) + R img with several different image-text RMs, including HPSv2.1 [Wu et al., 2023a] , PickScore [Kirstain et al., 2024] , and ImageReward [Xu et al., 2024a] . We collect the 4-step generations from each method and qualitatively compare them with the 4-step generation from the baseline VCM. As shown in Figure 6 , incorporating reward feedback from any of these R img leads to quality improvement over the baseline VCM (VC2). It is worth noting that HPSv2.1 and PickScore are fine-tuned from CLIP with human preference data. Therefore, learning from CLIP might also lead to better performance than the baseline VCM. We provide additional qualitative comparisons between our T2V-Turbo, the baseline VCM, and their teacher T2V models in Figures 7, 8 , 9, and 10.The prompts for the top two and bottom two rows in Figure 1 are given below:• With the style of low-poly game art, A majestic, white horse gallops gracefully across a moonlit beach. • Kung Fu Panda posing in cyberpunk, neonpunk style.The ability to create highly realistic synthetic videos raises concerns about misinformation and deepfakes, which can be used to manipulate public opinion, defame individuals, or perpetrate fraud. Addressing these concerns requires robust regulatory frameworks and ethical guidelines to ensure the technology is used responsibly and for the benefit of society. Therefore, we are committed to installing safeguard when releasing our models. Specifically, we will require users to adhere to usage guidelines. Despite the challenges, the impact of our T2V-Turbo is profound, offering a scalable solution that significantly enhances the accessibility and practicality of generating high-quality videos at a remarkable speed. This innovation not only broadens the potential applications in fields ranging from digital art to visual content creation but also sets a new benchmark for future research in T2V synthesis, emphasizing the importance of human-centric design in the development of generative AI technologies. Answer: [Yes] Justification: We include our codes in the supplementary materials. We have released our models and codes in https://github.com/Ji4chenLi/t2v-turbo.Guidelines:• The answer NA means that paper does not include experiments requiring code. Justification: We include the information in Appendix A.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).https://huggingface.co/spaces/Vchitect/VBench_Leaderboard 


Decision: The paper received two borderline rejects, one weak accept and two accepts. The concerns were primarily centered around lack of technical novelty since the paper's summary can be distilled to "applying reward models to distillation". But all the reviewers point out the impressive results with the approach using very few steps. The paper applies multiple image and video reward models during consistency distillation to avoid memory bottlenecks. It also showcases that a true video reward model (which don't exist publicly) is not needed, and using video-text models as reward models also boosts performance. The AC agrees with the majority of reviews and feels that given the simplicity of implementation and the impressive results, the paper merits acceptance.

Reviews: review: paper_summary: paper_summary: The paper proposes T2V-Turbo, a model aiming to achieve both fast and high-quality text-to-video generation by breaking the quality bottleneck of a video consistency model (VCM). It integrates mixed reward feedback from one image and one video reward model into the consistency distillation process of a teacher T2V model. The 4-step generations from T2V-Turbo outperform SOTA methods on the VBench benchmark and are favored by humans over the 50-step DDIM samples from the teacher model, achieving over ten-fold inference acceleration with quality improvement.
strength_weakness: strength_weakness: - The method mainly combines the consistency distillation in the Video Consistency Model (VCM) with multiple reward models. Although it has achieved good results in solving the existing problems of the T2V model, this combination is relatively conventional and may be somewhat lacking in originality.
- The citation format of this paper should adhere to the NeurIPS standard, the authors misuse \citet throughout the paper, especially in Sections 2 and 5, making it hard to read smoothly.
questions: questions: - In Line 228, the authors write “InternVid2 S2 outperforms ViCLIP in several zero-shot video-text retrieval task”.  While in Table 1, we know that ModelScopeT2V (MS) has less total score than VideoCrafter2 (VC2), which means that MS is a relatively weaker model than VC2, then why choose ViCLIP as the video RM for MS rather than using a stronger InternVid2 S2?
limitations: limitations: The authors have adequately addressed the limitations of the work in the paper. They acknowledge the lack of an open-sourced video-text reward model trained to reflect human preferences on video-text pairs and discuss the potential use of a more advanced video reward model in the future. They also mention the concerns about misinformation and deepfakes raised by the ability to create highly realistic synthetic videos and commit to installing safeguards when releasing the models, such as requiring users to adhere to usage guidelines.
paper_summary: paper_summary: The paper introduces T2V-Turbo to enhance the quality of video consistency models in text-to-video generation. The authors address the slow sampling speed of diffusion-based T2V models and the low quality of generated video by integrating feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. This integration allows for the optimization of single-step generations, bypassing the memory constraints of backpropagating gradients through iterative sampling processes. T2V-Turbo demonstrates significant improvements in both speed and quality, achieving high performance on the VBench benchmark and surpassing leading models such as Gen-2 and Pika.
strength_weakness: strength_weakness: - **Technical contribution**: The proposed approach is simply a mixture of previous works, i.e., consistency distillation and reward feedback learning, which deflates the technical contribution of the paper. However, I acknowledge that this is one of the pioneering approaches in text-to-video generation.
- **Reward models**: While the proposed method heavily depends on the reward models, the reward models that the paper used are actually not designed to function as reward models. This is because of the absence of video-text reward models in comparison to image generation (as the author mentioned in their limitations). However, it is a more right approach to first design a good reward model for video generation than developing a video generation model to align with such reward models.
questions: questions: - Regarding reward models for video generation. Could the author provide more details about how the reward models should be designed in order to obtain better video generation or evaluating video generation models?
- It seems like only consistency distillation for VC2 degrades the performance (Table 1), while the score remains the same for ModelScope. What if we do not use consistency distillation, and only use reward feedback learning for VC2 or MS? Then this may have a higher score than T2V-Turbo? Suppose it is true, then what if we distill after the feedback learning? To summarize my points, does the joint training of feedback learning and consistency distillation is better than sequential fine-tuning?
limitations: limitations: The paper highlights the limitation of relying on existing video-text reward models that are not explicitly trained to reflect human preferences on video-text pairs. Instead, the authors use video foundation models like ViCLIP and InternVid S2 as substitutes. While incorporating feedback from these models has enhanced T2V-Turbo’s performance, the authors acknowledge that the development of more advanced video-text reward models could further improve the results. Additionally, the complexity of integrating mixed reward feedback into the CD process could hinder the broader adoption and scalability of the approach.
paper_summary: paper_summary: This paper aims to achieve a video consistency model with both fast and high-quality generation. Specifically, the authors introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. The differentiable reward models consists of an image-text reward model and a video-text reward model. Experiment results verify the effectiveness of proposed method.
strength_weakness: strength_weakness: - The novelty is limited. The paper seems to simple combine the video consistency model and the differentiable reward models. Different from previous works, the authors utilize a mixture of reward models, where a video-text reward model is additionally added to encourage the diffusion model to better model the temporal dynamics.  However, to me, this contribution is quite incremental. 

- For the paper presentation, I think it is better to show what the text prompt is in Fig. 1 to help reader better understand the difference between different models. 

- For the quantitative comparison, it can be seen that the proposed method actually doesn't perform the best for most of the evaluation metrics. I am not sure if comparing total score (i.e., a weighted sum of Quality Score and Semantic Score) only is enough to show the superiority of proposed method.
questions: questions: See the weakness part.
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: This paper presents a distillation method for text-to-video models. In short, it builds upon latent consistency models (more specifically, adapting the paper "Reward Guided Latent Consistency Distillation" from image to video models). The method involves the usual consistency model objective, in addition to that feedback from both an image-text reward model (e.g. HPSv2.1) and a video-text model (e.g. InternVideo) is used to enhance the quality of the generated videos. Overall, the proposed approach not only is a able to few-step video generation (4/8), but is also outperforming existing open-source video generators such as ModelScope and VideoCrafter.
strength_weakness: strength_weakness: The most obvious weakness about this paper is that it is a direct application of Li et al. ("Reward Guided Latent Consistency Distillation") to video models. As far as I can understand, the changes involve replacing SD2.1 with VideoCrafter/ModelScope and incorporating an additional video-text RM in addition to the image-text RM. Similarly, the main difference I can see with Wang et al. 2023 (VideoLCM) is the utilization of the reward objectives. 

Given that previous works have shown the efficacy of Latent Consistency Models for video generation, and the benefits of reward models for LCM distillation in images, it's not entirely novel to combine these 2 ideas into a single distilled video model. 

[Minor] I don't find too many qualitative examples, and I couldn't find more videos in the supplementary (apart from the code). It would definitely be nicer if there were more examples that were provided.
questions: questions: Overall, I definitely lean towards accepting the paper. Despite being "obvious", I think that having an open-weight model+implementation for this would definitely help research in the field. However, I would really like the authors to point out if I have missed out details in their contributions or any other novel aspects of their work, since that is the major shortcoming I see currently.
limitations: limitations: Limitations are sufficiently discussed.
paper_summary: paper_summary: The paper presents T2V-Turbo, a training strategy where additional reward models are introduced during the consistency distillation process to enhance the T2V consistency model's quality. With such enhancement, the trained model achieves favorable results in both VBench and human evaluations.
strength_weakness: strength_weakness: 1. Regarding the technical contribution, the proposed method seems like a straightforward extension of the video consistency model. The idea of adding direct supervision on the clean samples in consistency distillation has also been explored in previous works, e.g., [1];
2. It's hard to evaluate the motion quality improvements according to the static frames in the paper. It's highly recommended to include the corresponding mp4 files;
3. I wonder which reward model contributes more to quality improvement from Fig.4 row 1 and Fig.4 row 3? This cannot be inferred from Tab.2 and it's recommended to present some qualitative results in the ablation study;
4. The paper lacks in-depth discussions on the choice and influence of the image/video reward models. For instance, would it be better to CLIP/GAN-discriminator compared to the human preference score? Why do MS and VC2 perform differently when combined with InternVidS2?  Section 4.3 only provides empirical results.

[1] Adversarial Diffusion Distillation
questions: questions: 1. Why use different video reward models for ModelScope and VideoCrafter?
2. What leads to different hyperparameter choices in L164-165?
limitations: limitations: The authors have discussed the limitations.

Comments: title: On the technical difficulty of learning from a video-text RM
Clarification on the technical contribution
Re: Rebuttal
New Results for Your Review!
Further clarification
Re: technical significance
Clarification on our technical significance
Re: Rebuttal
Follow-up the discussion
Follow-up the discussion
Thank you!
Thanks for the clarification
Rebuttal by Authors
Anonymous website link with generated videos
comment: Dear Reviewer MMwk,

We would like to bring your attention to **the technical difficulty of learning from a video-text RM**. We regret not emphasizing this challenge sufficiently in our original manuscript. Unlike learning from an image-text RM, obtaining feedback from a video-text RM $\mathcal{R}\_\text{vid}$ demands significantly more memory. For instance, using models like ViCLIP and InternVid2 S2 requires sampling a batch size of 8 frames from the generated video clips. Since we work with a latent video generator, we must enable gradients while decoding these video frames from latent vectors to allow for passing from  $\mathcal{R}\_\text{vid}$ to the video generator. Consequently, this computational process becomes nearly impossible to fit within a 40GB A100 GPU if we also need to pass gradients through an iterative sampling process.

Our method addresses this challenge by cleverly leveraging single-step generation from consistency distillation. This approach is crucial, as even with single-step generation, we nearly max out the 40GB memory of the A100 GPU—let alone handling gradients through an iterative sampling process.

As the discussion period draws to a close, we kindly ask the reviewer to **reconsider their rating of our paper by focusing on our core scientific contributions**.

Best regards,

The Authors
Dear Reviewer Kq5y,

Thank you for your response. We want to further **clarify our technical contributions**.

First, **we regret not emphasizing the technical challenge of learning from a video-text reward model (RM) sufficiently** in our original manuscript. Unlike learning from an image-text RM, obtaining feedback from a video-text RM $\mathcal{R}\_\text{vid}$ demands significantly more memory. For instance, using models like ViCLIP and InternVid2 S2 requires sampling a batch size of 8 frames from video clips. Since we work with a latent video generator, we must **enable gradients** during the decoding of these video frames from latent vectors to allow for passing from the $\mathcal{R}\_\text{vid}$ to the video generator. Consequently, this computational process becomes nearly impossible to fit within a 40GB A100 GPU if we also need to pass gradients through an iterative sampling process.

To tackle this challenge, our method cleverly takes advantage of the single-step generation arising from consistency distillation, which is crucial for learning from $\mathcal{R}_\text{vid}$. Notably, even by operating with single-step generation, we almost fully utilize the 40GB memory of the A100 GPU.

Additionally, we want to emphasize that **the technical simplicity of our method should not be seen as a drawback**. On the contrary, being technically simple yet highly effective is a distinct advantage of our approach.

In this paper, we address the core challenges of video generation: 1) improving generation quality, 2) reducing inference time, and 3) alleviating the intensive memory and computational cost. Empirically, we achieve significant results by outperforming SOTA video-generation systems on standard benchmarks. **We hope the reviewer can evaluate our paper based on our core scientific contribution**.

Looking forward to your response.

Best regards,

The Authors
I appreciate the authors' detailed rebuttal and the additional results. They address some of my concerns.

However, my major concern about the technical contribution still exists. Especially when comparing Fig. 2 in this paper and Fig. 2 in VideoLCM, the major technical contribution is adding a reward loss on the clean samples, which seems straightforward without too many challenges. Therefore, I tend to maintain my original score.
Dear Reviewer Kq5y,

We would like to kindly invite you to review the results we have submitted during the rebuttal period. We have thoroughly addressed your concerns and included videos corresponding to the static frames in our paper. Thanks to the support of our Area Chair, we are able to share an [anonymous website](https://spangled-blanket-128.notion.site/Qualitative-results-of-T2V-Turbo-1290f9ec0fb34685918438d7ba590e83#69781d1bb2a048c69ce6c51441ecde50) containing all these videos. We hope that you will reconsider your ratings in light of these new results : )

Best regards,

The Authors
Dear Reviewer MMwk,

Thanks for your response.

Firstly, we would like to emphasize that the technical simplicity of our method should not be seen as a drawback. On the contrary, **being technically simple yet highly effective is a distinct advantage of our approach**. In this paper, we address the core challenges of video generation: 1) improving generation quality, 2) reducing inference time, and 3) alleviating the intensive memory and computational cost when aligning a video generator.

Current state-of-the-art proprietary video generation systems, such as Gen-2 and Pika, require **several minutes** to produce a short video clip. In contrast, our T2V-Turbo can generate high-quality videos **within 5 seconds**, making it significantly more suitable for real-time applications. T2V-Turbo achieves both fast and high-quality video generation. As validated on VBench, its 4-step generation process surpasses the performance of proprietary systems like Gen-2 and Pika, which rely on extensive resources. To the best of our knowledge, we are the first to simultaneously address these two contradictory aspects—**speed and quality**—within the same video-generation framework.

Secondly, we would like to reiterate the significance of our mixture of RMs design, which allows us to break the quality bottleneck in VCM without access to a $\mathcal{R}\_\text{vid}$ trained to mirror human preference. Our ablation study in Table 2 empirically provides valuable scientific insights into the effectiveness of different RMs.

While leveraging feedback from $\mathcal{R}\_\text{img}$ alone (VCM + $\mathcal{R}\_\text{img}$) is sufficient to match the visual quality (**Quality Score**) of our T2V-Turbo, the additional feedback from $\mathcal{R}\_\text{vid}$ further enhances text-video alignment, resulting in higher **Semantic Score**. We provide qualitative evidence in the attached PDF. To the best of our knowledge, we are the first to improve video generation with feedback from video-text RMs. We believe that future advancements in video-text RMs will further improve the performance of our methods.

Thank you again for your time and effort in providing valuable feedback on our work. We hope our clarifications will lead to a more favorable evaluation of our contribution.

Best,

The Authors
Thanks for the authors' further clarification. 

After reading the authors' clarification, I further checked the paper. My concern still exists. To me, the joint optimization adopted in the paper is also straightforward. I didn't see any gap or technical difficulty to perform the joint optimization. As claimed in Fig. 2 and Eq. (10) of the paper, the paper just simply integrates reward feedback into the VCD procedures (i.e., adding the RM loss terms to the original VCD loss term) . 

Thus, I still think the paper or the method lacks technical novelty. I will keep my original rating.
Dear Reviewer MMwk,

Thank you for your response. We want to clarify the **technical issues** we solved in this paper and emphasize our technical significance.

1. Traditional reward learning methods for video generation, such as InstructVideo, suffer from intensive memory costs due to backpropagating gradients through a diffusion model (DM)'s iterative sampling process. Our method solves this issue by leveraging the single-step generation arising from consistency distillation.

2. Our method significantly reduces the training time required to align a video generation model, achieving a 4x increase in efficiency in wall-clock training time compared to InstructVideo.

We also wish to address a **major understanding**. Our approach **does NOT simply replace the DM with a VCM** from a conventional reward learning method. A straightforward combination of VCM and differentiable RM requires either 1) distilling a VCM from a pretrained DM and subsequently aligning the VCM with a differentiable RM, or 2) aligning a DM with a differentiable RM before distilling a VCM from the aligned DM.

Neither of these sequential methods achieves the same memory reduction as our approach because they still involve backpropagating gradients through the iterative sampling processes of either the DM or the VCM. Furthermore, these methods are not as computationally efficient as ours due to their two-phase training pipelines.

We sincerely appreciate your feedback and hope that this clarification will encourage you to reconsider your evaluation of our work.

Best regards,

The Authors
Thanks for the authors' detailed response. However, the response still does not clarify my concerns about the novelty. To me, the method proposed in this paper is a combination of VCM and differentiable RM. I understand that compared with differentiable RM methods, it is different as it introduces VCM instead of using conventional diffusion models. I also understand that combining the two techniques does bring benefits and achieve superior results. However, I didn't see this paper solving any novel technical issues when combining the two techniques. To me, the engineering significance of this paper is greater than its technological innovation. Thus, I tend to keep my original rating.
Dear Reviewer MMwk,

We greatly appreciate your insightful feedback, which has significantly contributed to the clarity and enhancement of our work. We have carefully addressed your comments in our response, worked to resolve any misunderstandings, and included multiple video examples for qualitative comparisons to illustrate the effectiveness of our method.

We kindly request that you revisit our paper in light of our response and clarifications, and consider whether these updates might lead to a reevaluation of your rating.

Best regards,

The Authors
Dear Reviewer Kq5y,

Your feedback has been invaluable in helping us clarify, improve, and refine our work. We have diligently addressed your comments in our response, made every effort to dispel misunderstandings, and provided various video examples for qualitative comparisons to demonstrate the effectiveness of our method.

We kindly ask you to revisit our paper in light of our response and consider whether the changes and clarifications we have provided might warrant a reconsideration of your rating.

Best regards,

The Authors
Dear Reviewer 61hZ,

Thank you for raising your score! Your feedback has been invaluable in helping us improve the presentation of our work. We are pleased that our rebuttal has clarified the contributions of our work.

Thanks and best regards,

The Authors
I thank the authors for providing additional videos, and also clarifying the contributions of the method. Based on this, I am now raising my score.
Dear Reviewer x94N,

Thank you again for your positive feedback on our work!

Best regards,

The Authors
Thank you for the detailed rebuttal and the extensive qualitative examples provided. I acknowledge the computational burden involved in T2V generation and the meticulous nature of evaluating video generation models. This paper contributes valuable strategies for improving T2V generation and offers a robust framework for evaluating video generation models, which will be beneficial for other researchers in the field. After careful consideration, I will maintain my original score.
We thank the reviewer for the positive feedback on our work! Please find our detailed feedback below.

> The method mainly combines the consistency distillation in the Video Consistency Model (VCM) with multiple reward models. Although it has achieved good results in solving the existing problems of the T2V model, this combination is relatively conventional and may be somewhat lacking in originality.

We would like to highlight that our method cleverly leverages the single-step generation arising from consistency distillation. By optimizing the rewards of the single-step generation, our method avoids the highly memory-intensive issues associated with passing gradients through an iterative sampling process. In contrast, previous methods, such as InstructVideo [1], require backpropagating gradients through diffusion model's iterative sampling process, resulting in substantial memory costs.

Additionally, we emphasize the importance of our mixture of RMs design, which enables us to break the quality bottleneck in VCM without access to a $\mathcal{R}\_\text{vid}$ trained to mirror human preference. By learning from an image-text RM $\mathcal{R}\_\text{img}$ and a video foundation model $\mathcal{R}\_\text{vid}$, such as ViCLIP and InternVid2 S2, we achieve both fast and high-quality video generation. Notably, the 4-step generations from our T2V-Turbo achieve the SOTA performance on VBench, surpassing proprietary models, including Gen-2 and Pika. To the best of our knowledge, we are the first to improve video generation using feedback from video-text RMs. We believe that future advancements in video-text RMs will further enhance the performance of our methods.

> The citation format of this paper should adhere to the NeurIPS standard, the authors misuse \citet throughout the paper, especially in Sections 2 and 5, making it hard to read smoothly.

We thank the reviewer for pointing out the issues! We will make sure to fix the citation issues in our revised manuscript!

> why choose ViCLIP as the video RM for MS rather than using a stronger InternVid2 S2?

Our main purpose is to **examine our methods with a diverse set** of $\mathcal{R}\_\text{vid}$. As we did not have access to a $\mathcal{R}\_\text{vid}$ trained to reflect human preference on video, we instead chose to experiment with video foundation models, including ViCLIP and InternVid2 S2. Notably, we conduct a comprehensive ablation study in Table 3, presenting results for our T2V-Turbo (VC2) and T2V-Turbo (MS) when setting $\mathcal{R}\_\text{vid}$ to both ViCLIP and InternVid2 S2. This approach allows for a thorough assessment of our methods across different $\mathcal{R}\_\text{vid}$.
Dear reviewers, 

As requested in the reviews, the authors, after consultation with me, have shared an anonymous website containing video results. The videos are available at https://spangled-blanket-128.notion.site/Qualitative-results-of-T2V-Turbo-1290f9ec0fb34685918438d7ba590e83#69781d1bb2a048c69ce6c51441ecde50. Please consider this part of the rebuttal response.

Best,
Your AC