翻译下面的那，然后给出解释3.4.2 Improving Inference Efficiency To sample videos efficiently, we use the Euler sampler with a unique t-schedule tailored to our model. Empirically, we found that Euler outperforms higher-order solvers like midpoint (Atkinson, 1991) or adaptive solvers like Dopri5 (Dormand and Prince, 1980). We observed that reducing the number of inference steps for video generation is more challenging than for image generation due to the additional time dimension, i.e., the quality and prompt alignment of the generated motion are more sensitive to the number of inference steps compared to static images. For example, videos generated with 250, 500, or 1000 linear steps show noticeable differences in scene composition and motion quality. While techniques such as distillation (Salimans and Ho, 2022; Kohler et al., 2024) can be used to speed up the model inference, they require additional training. Next, we show a simple inference-only technique that can lead up to ⇠ 20⇥ speed up with a few lines of code. We found that we can closely approximate the quality of an N-step video generation process with merely 50 steps by implementing a linear-quadratic t-schedule. This approach follows the first 25 steps of an N-step linear schedule and then approximates the remaining N 25 steps with 25 quadratically placed steps. For example, a video generated with 1000 linear steps can be precisely emulated by 25 linear steps followed by 25 quadratic steps, whereby the linear steps are identical to the first 25 linear steps of a 1000-step linear schedule. The linear-quadratic strategy is predicated on the observation that the first inference steps are pivotal in setting up the scene and motion of the video. This is visualized in Figure 10, where we plot the average change between the input and output of each transformer block at every inference step. Similar behavior is observed in the diffusion-based fast video model PAB (Zhao et al., 2024), where the average per-step difference of attention blocks follows a U-shaped pattern, compared to the L-shaped curve in Figure 10. Since most changes occur in the first solver steps, taking over the first linear steps of an N-step schedule followed by much bigger steps is enough to approximate the full N-step result. The quadratic spacing of the latter steps is crucial, as it emphasizes the importance of the early stages in the flow-matching sequence. In practice, we use a 50-step linear-quadratic schedule emulating N = 250 linear steps for optimal results.
3.5 Evaluation In this section, we explain how we evaluate the text-to-video quality of Movie Gen Video and other models. Our goal is to establish clear and effective evaluation metrics that identify a model’s weaknesses and provide reliable feedback. We explain the different text-to-video evaluation axes and their design motivations in Section 3.5.1. We introduce our new benchmark, Movie Gen Video Bench, in Section 3.5.2. Throughout this work, we use human evaluation to assess the quality of generated videos across various evaluation axes. When evaluating each axis, we conduct pairwise A/B tests where expert human evaluators assess two videos side-by-side. Evaluators are instructed to choose a winner based on the axis being measured, or to declare a tie in case of no clear winner. We include a discussion on the motivation and reliability of using human evaluations and on existing automated metrics in Section 3.5.3. 3.5.1 Evaluation Axes Evaluating text-to-video generation presents unique challenges compared to text-to-image tasks, primarily due to the added complexity of the temporal dimension. For a video to be considered high quality, it must stay faithful to the provided text prompt, maintain a high visual quality across frames without noticeable flaws, and be visually appealing with a photorealistic style. To assess these factors, we evaluate the quality of generated videos across three main axes: (1) Text-alignment, (2) Visual quality, and (3) realness and aesthetics. Each axis, along with their fine-grained sub-axes, is described in details below and summarized in Table 4. Text-alignment. This axis measures how well a generated video aligns with the provided prompt. An input prompt can include wide-ranging descriptions of subject appearance, motion, background, camera motion, lighting and style, visual text, etc. Human evaluators are asked to pay close attention to these specific aspects and select the video that aligns more closely with the prompt. To provide more nuanced feedback, evaluators Text-alignment Visual quality Realness & Aesthetics Subject & Motion alignment Overall Frame consistency Motion Completeness Motion Naturalness Realness Aesthetics Table 4 Evaluation axes for text-to-video generation. We evaluate video generations across 3 axes, each of which is composed of multiple fine-grained sub-axes. Text-alignment evaluates the ‘alignment’ between the input text prompt and the video. Visual quality, Realness & Aesthetics evaluate the quality of the generated video independent of the input text prompt.
are also asked to specify their reasoning based on two orthogonal sub-axes: Subject match: This measures alignment of subject appearance, background, lighting and style; and Motion match: This measures the alignment of motion-related descriptions. Visual Quality. Compared to visual quality in text-to-image generation, much of the perceived quality in generated videos stems from the quality of motion – a video-specific dimension. Therefore, in text-to-video visual quality evaluation, we focus on measuring the model’s ability to generate consistent, natural, and sufficient amounts of motion in the output videos. To capture these critical aspects, we propose the following four sub-axes, which we outline below. • Frame consistency: This metric assesses the temporal consistency of generated content. Violations of frame consistency can manifest as morphing-like artifacts, blurred or distorted objects, or content that abruptly appears or disappears. We consider frame consistency a crucial measure of the model’s ability to understand object framing and relationships in motion, as inconsistencies or distortions often arise when the model fails to accurately represent interactions between objects or their environment. Additionally, frame consistency reflects the model’s capacity to handle challenging tasks, such as prompts that require fast-moving content, e.g., in sports scenarios, where maintaining consistent appearance is especially difficult; or reasoning about occlusions, e.g., objects re-appearing after being occluded. • Motion completeness: This measures whether the output video contains enough motion. A lack of motion completeness may occur when the prompt involves out-of-distribution or unusual subjects (e.g., monsters, ghosts) or real-world objects performing unusual activities (e.g., people flying, pandas playing piano). Due to limited training data for such scenarios, the model may struggle to generate enough amount of motion, resulting in either static videos or those with only camera movement. Motion completeness evaluates the magnitude of motion in the video. A win on this axis indicates a greater amount of motion, even if it includes distortion, fast motion, or appears unnatural. • Motion naturalness: This metric assesses the model’s ability to generate natural and realistic motion, demonstrating a solid understanding of real-world physics. It covers aspects such as natural limb movements, facial expressions, and adherence to physical laws. Motion that appears unnatural or uncanny will be penalized. • Overall quality: For a given pair of videos being compared, the above three metrics might not result in the same winner. To resolve this, we introduced the overall quality sub-axis, where human evaluators are asked to pick the winning video that has better “overall” quality given the previous three sub-axes. This is a holistic metric that asks the human annotators to use their perception and to balance the previous signals to capture overall how good a generated video is. Realness & Aesthetics. Realness and aesthetics evaluate the model’s ability to generate photorealistic videos with aesthetically pleasing content, lighting, color, style, etc. We ask human evaluators to evaluate along two sub-axes: • Realness: This measures which of the videos being compared most closely resembles a real video. For fantastical prompts that are out of the training set distribution (e.g., depicting fantasy creatures or surreal scenes), we define realness as mimicking a clip from a movie following a realistic art-style. We additionally ask the evaluators to select a reason behind their choice i.e., “subject appearance being more realistic” or “motion being more realistic”. • Aesthetics: This measures which of the generated videos has more interesting and compelling content, lighting, color, and camera effects. Again, we ask the evaluators to provide details justifying their choice from “content being more appealing/interesting”, and “lighting/color/style being more pleasing”.