NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generation
Authors: Xiaokun Feng
Haiming Yu
Meiqi Wu
Shiyu Hu
Jintao Chen
Chen Zhu
Jiahong Wu
Xiangxiang Chu
Kaiqi Huang
Venue: ICLR 2026 Poster (2026)
Keywords: long video gengration model;  video gengration benchmarks
Primary area: datasets and benchmarks

Abstract: With the rapid development of foundation video generation technologies, long video generation models have exhibited promising research potential thanks to expanded content creation space. Recent studies reveal that the goal of long video generation tasks is not only to extend video duration but also to accurately express richer narrative content within longer videos. However, due to the lack of evaluation benchmarks specifically designed for long video generation models, the current assessment of these models primarily relies on benchmarks with simple narrative prompts (e.g., VBench). To the best of our knowledge, our proposed NarrLV is the first benchmark to comprehensively evaluate the Narrative expression capabilities of Long Video generation models. Inspired by film narrative theory, (i) we first introduce the basic narrative unit maintaining continuous visual presentation in videos as Temporal Narrative Atom (TNA), and use its count to quantitatively measure narrative richness. Guided by three key film narrative elements influencing TNA changes, we construct an automatic prompt generation pipeline capable of producing evaluation prompts with a flexibly expandable number of TNAs. (ii) Then, based on the three progressive levels of narrative content expression, we design an effective evaluation metric using the MLLM-based question generation and answering framework. (iii) Finally, we conduct extensive evaluations on existing long video generation models and the foundation generation models that underpin them. Experimental results demonstrate that our metric aligns closely with human judgments. The derived evaluation outcomes reveal the detailed capability boundaries of current video generation models in narrative content expression.

Published as a conference paper at ICLR 2026
NARRLV: TOWARDS A COMPREHENSIVE NARRATIVE- CENTRIC EVALUATION FOR LONG VIDEO GENERATION
Xiaokun Fengl.2.3, Haiming Yu? Meiqi Wu
12.3,
Shiyu Hu Jintao Chen
Chen Zhu Jiahong Wu 3
Xiangxiang Chu
Kaiqi Huang!
1 School of Artificial Intelligence, UCAS 2 CASIA 3 AMAP, Alibaba Group
School of Computer Science and Technology, UCAS
5 School of Physical and Mathematical Sciences, NTU PKU
Project Page: https: //mmmmmmmmmm/ .aa.am....ithabbb...b...... www.t..thh........ i
ABSTRACT
With the rapid development of foundation video generation technologies, long video generation models have exhibited promising research potential thanks to expanded content creation space. Recent studies reveal that the goal of long video generation tasks is not only to extend video duration but also accurately express richer narrative content within longer videos. However, due to the lack of evaluation benchmarks specifically designed for long video generation models, the current assessment of these models primarily relies on benchmarks with simple narrative prompts (e.g., VBench). To the best of our knowledge, proposed NarrLV is the first benchmark to comprehensively evaluate the Narrative expression capabilities of Long Video generation models. Inspired by film narrative theory, (i) we first introduce the basic narrative unit maintaining continuous visual presentation in videos as Temporal Narrative Atom (TNA), and use its count to quantitatively mea- sure narrative richness. Guided by three key film narrative elements influencing TNA changes, we construct an automatic prompt generation pipeline capable of producing evaluation prompts with a flexibly expandable number of TNAs. (ii) Then, based on the three progressive levels of narrative content expression, we de- sign effective evaluation metric using the MLLM- question generation and answering framework. (iii) Finally, we conduct extensive evaluations on existing long video generation models and the foundation generation models. Experimental results demonstrate that our metric aligns closely with human judgments. The derived evaluation outcomes reveal the detailed capability boundaries of current video generation models in narrative content expression.
INTRODUCTION
Video generation has consistently been regarded long-ten research goal (Xing al., 2024), from the earliest techniques with subtle motion effects (Vondrick et al., 2016) to recent foundation models like Wan (Wang et al., 2025) that achieve high-fidelity dynamic video generation. Given that these models are limited to producing short videos, recent studies have shifted focus toward designing long video generation models (Xing et al., 2024). Benefiting from a broader content creation space, long video generation models (Waseem & Shahzad, 2024) show greater potential to meet practical needs in areas such as film production and world simulation (Cho et al., 2024; Mao al., 2025).
Some approaches (Kim et al., 2024; Zhao et al., 2025) have incorporated innovative designs into denoising models, enabling foundation video generation models (Chen al.. 2024a; Yang al., 2024b) to produce more frames. However, the goal of long video generation goes beyond extending video duration. It critically involves accurately and appropriately conveying richer narrative content in extended videos (Waseem & Shahzad, 2024; Bansal et al., 2024a). Existing long video models often focus on leveraging temporally evolving narrative texts to guide video generation across different time segments, thereby enhancing the narrative content in the generated videos. Models such as FreeNoise (Qiu et al., 2024), Presto (Yan et al., 2024), and Mask (Qi et al., 2025) emphasize
Work done during the internship at AMAP, Alibaba Group.
Corresponding author
t Project lead

---

Published as a conference paper at ICLR 2026
efficient interaction between segmented texts with diverse narrative semantics and corresponding video clip features, reflecting the field's pursuit of generating narrative-rich onn-duration videos.
Unlike the rapid development of long video generation models, the evaluation benchmarks for this task appear somewhat lagging. Early models like NUWA-XL (Yin et al., 2023), Loong (Wang et al., 2024b), and FreeNoise (Qiu al., 2024) used conventional metrics (FID (Heusel et al., 2017), FVD (Unterthiner et al., 2019), CLIP-SIM (Radford et al., 2021)), which are often misaligned with human judgment (Otani al., 2023). To address this, numerous benchmarks (Huang et al., 2024a; Liu et al., 2024b; Ling et al., 2025; Chen et al., 2025e) for video generation have been proposed, yet there is still a lack of benchmarks specifically designed for long video generation. This leads to recent models, such as Prestro (Yan et al., 2024), GLC-Diffusion (Ma et al., 2025), and SynCoS (Kim et al., 2025), typically being evaluated on general benchmark, VBench (Huang et al., 2024a). Although VBench encompasses a wide range of evaluation dimensions, its prompts generally consist of brief narratives, limiting its effectiveness in assessing the models' ability to convey rich narrative content.
To evaluate the Narrative expression capabilities of Long Video generation models, we propose novel benchmark, NarrLV, inspired by film narrative theory (Verstraten, 2009). Firstly, to quantify the abstract concept of narrative content richness, we define the smallest narrative unit maintaining continuous visual presentation as a Temporal Narrative Atom (TNA). The number of TNAs serves as a quantitative measure of narrative richness, as illustrated by the prompts shown in Fig. I (a). Fig. 1 (b) shows that representative benchmarks (Huang et al., 2024a; Wang et al., 2024a; Feng et al., 2024a) concentrate on prompts with only small number of TNAs in a narrow range (please see App. A.1 for details), which limits their evaluation to simple narratives with limited richness. To thoroughly assess the full narrative capabilities of long video generation models, we construct an innovative prompt suite that can flexibly expand narrative content richness. Specifically, based on the 6D principles of film narratology (Cutting, 2016; Hu et al., 2023a), we identify three key dimensions affecting TNA changes: scene attributes, object attributes, and object actions. Subsequently, we use the Large Language Model (LL.M) (Yang et al., 2024a) to establish an automatic prompt generation pipeline capable of generating test prompts that cover a wide range of TNA numbers.
Corresponding to the prompt suite focused on narrative content, we design an effective evaluation metric following progressive narrative expression paradigm (Chatman & Chatman, 1980; Roberts et al., 1996; Cowie, 2013). From the basic elements of scenes and objects to the narrative units they form, our metric encompasses three evaluative dimensions: narrative element fidelity, narrative unit coverage, and narrative unit coherence. Considering the flexible and diverse nature of narrative content, our implementation leverages the MLL.M-based (Bai et al., 2025; Hurst et al., 2024) question generation and answering framework (Hu et al., 2023b; Yarom et al., 2023; Cho et al., 2023a). which can create extensible question sets according varying TNA numbers. Finally,
conduct
comprehensive evaluations of existing long video generation models (Bansal et al., 2024b; Kim et al., 2024; Qiu et al., 2024; Lu et al., 2024; Zhao et al., 2025) and the foundation models (Wang et al., 2025; Kong et al., 2024; Yang et al., 2024b; Zheng et al., 2024b; Lin et al., 2024) they are often built upon. The experimental results show that our metries align well with human preferences and provide detailed insights into the narrative expression boundaries of current models.
Our key contributions are as follows: (i) light of the lack of benchmarks for long video generation models, we propose NarLLV, a novel benchmark focusing on narrative content expression capabilities. (ii) Inspired by film narrative theory, NarLLV comprises a thorough prompt suite with flexibly expandable narrative content, and effective evaluation metric based on progressive narrative expression. (iii) We conduct comprehensive evaluations of existing long video and foundation generation models using our metrics, which demonstrate high alignment with human preferences.
2 RELATED WORKS
Long video generation models. Owing to high computational costs in video feature processing (Kim et al., 2024), foundation video generators (e.g., VideoX (Yang et al., 2024b) and Wan (Wang et al., 2025)) typically produce short videos. Comparatively, long video generation models generally refer to those capable of generating longer videos than these foundation models (Zhao et al., 2025; Kim et al., 2024; Lu et al., 2024). In practice, most long video models are extensions of short-video foundation models. FreeLong (Lu et al., 2024) generates more frames by balancing the feature frequency distribution for long videos. RIFLEx (Zhao et al., 2025) achieves 3x extension
2

---

Published as a conference paper at ICLR 2026
of video duration by adjusting temporal position encoding. In addition to pursuing longer video durations, long video generation tasks also focus on accurately conveying richer narrative content in extended videos. Specifically, videos generated at different time intervals need to be guided by textual narratives that evolve over time (Zhou et al., 2024; Tian et al., 2024; Bansal et al., 2024a; Qiu et al., 2024; Qi et al., 2025). These temporally changing textual descriptions form rich narrative content and pose new challenges for model design. FreeNoise (Qiu et al., 2024) progressively injects segmented texts regarding object movement evolution into different denoising steps. Presto (Yan et al., 2024) proposes an innovative segmented cross- attention strategy, directly facilitating the interaction between latent features of long videos and segmented narrative texts. Addressing the current lack of benchmarks specifically designed for long video generation models, we develope a novel benchmark, Va focused on narrative expression capabilities.
Video generation evaluation. The growing capabilities of video generation models continually introduce new de- mands for effectively evaluating the generated videos (Liu et al., 2024a). Early evaluations primarily rely on generic metrics (e.g., FID (Heusel et al., 2017), IS (Salimans et al., 2016), FVD (Unterthiner et al., 2019), which of- ten exhibit significant deviations from human perception (Huang et al., 2024a; Otani et al., 2023) and provide lim-
pproon rediig a cary
del
changin from brovn
ThA+ Jun Kayeta
Dew' buthen Nea pood
ited insight into model capabilities (Zheng et al., 2025). To better evaluate various model capabilities, several spe- cialized benchmarks have been proposed. For instance, VBench (Huang et al., 2024a) defines 16 evaluation di- mensions based on video quality and video-condition consistency. DEVIL (Liao et al., 2024) emphasizes video dynamism; TC-Bench (Feng et al., 2024a) evaluates tem- poral compositionality; and VMBench (Ling et al., 2025) thoroughly assesses motion quality. StoryEval (Wang al., 2024a) is another related benchmark that evaluates event-level story presentation capability using prompts of 2 to 4 consecutive events. However, all these benchmarks primarily object short-duration models. As shown in Fig. 1, their prompts contain relatively few TNAs with a narrow distribution, making them insufficient for testing models on complex, extended narrative content. In contrast, NarLLV is designed to fill this gap by providing a platform to evaluate the generative capacity of long video generators under rich and comprehensive narrative content.
Theak for sarks)
porkam an the
Nuronn than
Duikety
Sticky Ale
the aniin rindy me
Figure I: (a) Prompt examples with vary- ing numbers of TNAs. (b) Comparison of TNA count distributions across different benchmark.
NARRLV
The overall framework of our NarrLV is illustrated in Fig. 2. First, building on film narrative theory (Verstraten, 2009; Cutting, 2016), we introduce the Temporal Narrative Atom (TNA) as unit to measure the richness of narrative content and identify three key dimensions (Hu et al., 2023a) that influence its count. Subsequently, we develop an automated prompt generation pipeline capable of producing evaluation prompts with a flexibly expandable number of TNAs. The resulting prompt suite enables comprehensive assessment of the model's s generation capability across various levels of narrative content richness. Finally, leveraging the MLLM-based question generation and answering framework (Hu et al., 2023b; Yarom et al., 2023; Cho et al., 2023a), we construct a comprehensive evaluation metric founded on the three progressive levels of narrative content expression. In the following sections, we will provide detailed introductions to each component.
3.1 PRELIMINARIES OF FILM NARRATIVE THEORY
Film narratology (Verstraten, 2009; Kuhn, 2009) is a discipline dedicated to the study of narrative structures and expressive techniques in films. To evaluate video generation models with emphasis on narrative expression, we draw upon relevant theories from this field. First, the richness of narrative content is an abstract concept. To facilitate its quantification, it is necessary to define a basic unit for measuring narrative richness (McKee, 2005). Drawing from the definition of Beat in film narratology (McKee, 2005), we define the smallest narrative unit in continuous visual expression the Temporal
3

---

Published as a conference paper at ICLR 2026
(a) Extensible TNA-Driven Prompt Suite
(b) Evaluated Models
TNA change factors
Oiject Pair Swi
Prompt generation pipeline
Civen muskaation promple
Film Narrative
Scene attribute
Object attr ibuie
1. Sampled scene-dbject
Evalustion
Object
2. Given TNA change factor
prompts
Video generation models
3. Given TNA Numbers
Long
Founsaiion video
penuraion
(c) Progressive Narrative N Evaluation Metric
Evaluation dimensions
Metric calculation
Generated videos
Scene,
Narrative clement fidelity
Evaluation prompts
X Generated videos
Object
constitule
2. Narrative unit coverage
Dimension-
Human Preference Alignment
Narrative
specific
Anewera
unit
Narrative coherence
queetions
MIL
&Metric results
Well-Aligned
Figure 2: Framework of our NarrLV. (a) Our prompt suite is inspired by film narrative theory and identifies three key factors influencing Temporal Narrative Atom (TNA) transitions. Based on these, we construct a prompt generation pipeline capable of producing evaluation prompts with flexibly adjustable TNA counts. (b) Our evaluation models include long video generation models and the foundation models they often rely on. (c) Based on the progressive expression of narrative content, we conduct evaluations from three dimensions, employing an HLLM--based question generation and
answering framework for calculations. Our metric is well-aligned with human preferences.
Narrative Atom (TNA). Fig. presents several prompt examples containing different numbers of TNAs. Evidently, the greater the number of TNAs, the richer the corresponding narrative content.
Following this, a naturally arising question is: what factors influence the number of TNAs? The 6D principles of film narrative (Cutting, 2016; Hu al., 2023a) divide the narrative content into six critical elements based on spatiotemporal and causal relationships in video: total frame, temporal continuity, spatial continuity, scene, action, and object. In the context of video generation tasks, the total frame count, i.e., video length, is determined by the inherent characteristics of the generation model. Regarding temporal and spatial continuity, existing generation models typically assume setting of continuous spatio-temporal change (Cho et al., 2024; Liu et al., 2024a). Specifically, when constructing training datasets, they explicitly exclude samples that are spatio-temporally discontinuous due to factors like shot cuts (Kong et al., 2024). Therefore, the adjustable factors that can alter the narrative richness are limited to scene, object, and action. Based on this, we identify three key variable factors influencing the number of TNAs: scene attributes, object attributes, and object actions, formalized as = [an lact, fan) These factors are similar to the temporal composition factors mentioned in TC-Bench (Feng al., 2024a). However, unlike TC-Bench which primarily focuses on two TNAs, our prompt suite emphasizes the flexible extensibility of TNA count.
3.2 EXTENSIBLE TNA-DRIVEN PROMPT SUITE
key feature of our benchmark is the introduction of prompts that enable flexible TNA extensibility thoroughly assess the narrative expression capabilities of video generation models. To achieve this goal while minimizing the time-consuming and labor-intensive manual design processes (Huang et al., 2024a; Feng al., 2024a), we develop an automatic prompt generation pipeline based on the LLM (Yang et al., 2024a). Considering that scenes and objects are the primary factors influencing TNA numbers, our pipeline first aggregates a comprehensive set of scene-object pairs. Then, we sample specific scene-object instances and utilize the LLM to generate specific test prompts by integrating their potential attribute and action evolution.
Acquisition of scene-object pair set. To ensure that our test prompts closely align with the video content that users typically focus on, our data source includes the recently released and user-focused dataset VideoUFO (Wang & Yang, 2025), which effectively reflects real-world applicability scenarios (Wang& & Yang. 2024). Additionally, we incorporate the latest DropletVideo (Zhang et al., 2025) dataset, which features rich narrative content. Specifically, we randomly sample 100k text prompts from VideoUFO- IM (Wang& & Yang, 2025) and DropletVideo- IM (Zhang et al., 2025), respectively. Subsequently, we employ an LLM (Yang et al., 2024a) analyze these 200k prompts individually,
4

---

Published as a conference paper at ICLR 2026
Generated videca
Eveluation prompt
Metric calculation
On an esports competition stage player weannng headphones is seated intentty front computer setup.
Dise soan bestar piver"
kodyy
Doom bucle
jreisally, the player rapiilly types
the keyboard.
Next, the player suddenly stands
Norrice wd sovenge
Doer siter ooeen regit aet ofi
Then, the player removes
headdphaees
plaves soddy heiig lima"
sinally the player raises both
Nammis and
solerence
+000
victory.
Figure 3: Illustration of our metric evaluation process. Given an evaluation prompt, different video generation models produce corresponding video outputs. Coneurrently, based on the semantic information within the prompt, judgment questions concerning different evaluation dimensions are generated, resulting in evaluation outcomes for the generated videos. Better viewed with zoom-in.
extracting the scene and major object list o corresponding to each text. Next, we merge the object lists under the same scene to obtain the final scene-object pairs Sg. For instance, in basketball court scene, the object list includes basketballs, players, referees, and other related objects. Ultimately, the aggregated so constitutes our scene-object pair set So (see App. A.2 for for detailed implementation and statistical analysis).
Automatic prompt generation. As shown in the pipeline of Fig. 2 (a), we first extract a specific scene-ohject instance 8, from So. Then, we randomly sample 1 to objects from the object list in So. Next, we specify the TNA change number and the TNA change factor f. utilizing an LLM to incorporate the potential attribute/action evolution process. For detailed prompt instructions, please refer App. A.3. Finally, we obtain a test prompt Pf, corresponding to and f, formalized as:
(8or n.)
LLM
Pf.0:
where 8 E So, I E F, n E Nena].
Post- post-processsig. Based on the aforementioned pipeline, we can quickly generate large-scale prompts encompassing different TNA change factors and numbers. Considering the rising computational costs of video generation models (e.g., Wan2.1-14B (Wang et al., 2025) requires about 10 minutes to produce video on H20 GPU), it is necessary to perform post-processing carefully select small yet representative prompt suite (Huang et al., 2024a). First, for scenc-object pair set So with large quantities, we categorize them into 14 major categories (see App. A.2 for more details). For instance, under the sports venue category, there are subsets for football fields, basketball courts, etc.
Under each factor f and number n. we select 3 8o from each major category, ultimately obtaining 20 evaluation prompts (p',, 1 In addition, we temporarily set the maximum TNA number Nene 6, and observe that this range can already reveal some insightful conclusions (see Sec. 4.2). With change factors, we evaluate the models under 20 x 6 x 3 360 prompts. It is important to note that our prompt generation pipeline has good extensibility. For longer video generation the future, we can follow the same process to obtain prompts with broader TNA distribution.
3.3 PROGRESSIVE PEERATIVE-EXPRESSIEEE EVALUATION METRIC
To systematically evaluate the narrative quality of long video generation, we introduce three core metrics- Narraatvv Element Fidelity, Narrative Unit Coverage, and Narrative Unit Coher- ence-grounded in audiovisual storytelling principles (Chatman & Chatman, 1980; Roberts et al., 1996; Cowie, 2013; Diniejko, 2010). These dimensions provide a rational approach for assessing narrative expression by progressively focusing on the basic elements of scenes and objects and the temporal narrative units they form.
Furthermore, given the inherently flexible and diverse nature of narrative content, traditional task- specific models (Hinz et al., 2020; Cho et al., 2023b), due to their limited generalization capabilities, find it challenging to perform effective evaluations. Hence, we adopt the recently popular MLLM- based question generation and answer framework (Cho et al., 2023a; Yarom et al., 2023; Hu et al., 2023b). As shown in Fig. 2 (b), given a evaluation prompt Pf.n. the video generation model m produces video that requires evaluation. Based on the semantic information in pf. utilize an LLM to generate the dimension-speciin question set Q. Then, using the generated video v, we employ the MILMM to answer each question in Q, resulting in an answer set A, Finally, the evaluation
5

---

Published as a conference paper at ICLR 2026
results R are derived mapping from A. This can be formalized as:
(prm)) " U, (pym)) LLM Q, (Q,v) MILLM, - A-R. R.
(2)
Corresponding to the three evaluation dimensions mentioned above, our evaluation question set Q comprises three categories: Qŭa. Qcov. and Qenh- Our three evaluation dimensions are represented as Rad. Reov. and Resh. For some uncertain questions, during the process of deriving A from (Q, v). we observe that the MLLM tends to produce inconsistent answers across multiple repetitions for the same input. Moreover, the degree of uncertainty a question directly influences the inconsistency of its answers (please refer to App. B.1 for more details). Thus, for the same (Q; u) input, we instruct the MLLM to provide answers consecutively five times and use the proportion of a specific answer
among these five as the final result, i.e., [Q, v) MILM, A|x5 + R. Fig. illustrates the calculation process for each of our evaluation dimensions, as detailed below:
Narrative element fidelity (Rea). To determine whether the generated video accurately conveys the narrative content of the prompt Ps.*, it is first essential to examine the generation of basic narrative elements represented by the scene and major objects in Pf." (Chatman & Chatman, 1980). Thus, in
Qña, we initially extract the following narrative elements based the initial description in Pf.mi scene category, scene attributes. object categories, object attributes, object actions, and initial layout of objects within the scene. Elements missing in the prompt are ignored automatically. For each included element, we generate corresponding binary judgment questions qia- with answers Ond in [yes, no]. As depicted in Fig. 3, these questions form the set Qua = {ghh Niw where the number of questions Nna is determined by the number of narrative elements Ps.n-
the step (p)..) LLM
k=1"
Ana]xs + Ria processing. For each question hho the MILM provides answers fa Ji=1 through five iterations. We calculate the proportion of positive answers "pos (ie.,, yes) in the set [dnd =t=1 as the score răa for that question. Finally, by computing the mean of all has derive the final Ria:
Next, we perform the [(Ria, u)
MLLM
k.
5.115
Naa
11, if where 8(x, u) = (0, otherwise
Σ 8(a apos),
Red
(3)
Naa
Narrative unit coverage (Hear)) For the narrative elements evaluated by REa their temporal evolution forms the TNAs that encompass different narrative contents. Thus, Rew is primarily used to assess the coverage of the TNAs involved in the prompt Pf. by the generated video v. the step
(p)..) I.M Qcov, we first extract the TNA list corresponding pf,n. Then, we generate a judgment question Yoow for each TNA regarding existence, forming the question set Qcov = (o k= the number of questions Neov is determined by n, meaning the scope of the questions expands along with the expansion of TNAs. For the calculation of Reov. we employ the same approach as Eq. 3.
) where
Qeah: first extract the TNA list corresponding to Py.-- Then, we sequentially select pairs of adjacent TNA contents and generate judgment questions Yooh regarding the existence of transitions between them. This forms the question set Q a |Nob 1e= where Noch also determined by n. Based this question set, we apply the calculation method from Eq. 3 to obtain R'sh- Additionally, considering that the existence of TNAs is a prerequisite for determining transitions between them, we introduce the proportion of TNA existence Pina. which, along with Rooh determines the final Rgoh:
Narrative unit coherence (Rehh). For the step (pf..) LLM
1 (Resh + Pena), where O(x) 2
{ 0, otherwise (4)
if > 0
Or -Too)) Reoh
Pina
Noov
Here, we consider TNA to exist corresponding A exceeds the threshold Teow.
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS.
Evaluation models, Our evaluation focuses on text-to-video models, a fundamental scenario in video generation (Li et al., 2019; Singer al., 2022). First, scope includes recently open-

---

Published as a conference paper at ICLR 2026
(a) Evalustion results of the foundation video gemeration models
0.9
0.8
0.6
Open
0.3
0.7
0.0
0.5
0.3
0.4
THA Nambers
Numbbara (b) Evaluation results
Numbers
results of the long video generation models
0.3
ite
0.8
NPO TALE
0.0
0.6
0.40
0.6
0.5
0.35
0.5
0:30
0:2
0.3
0.2
0.2
THA Numbers
TNA Numbers
Figure 4: Evaluation results across three evaluation dimensions. Evaluated models include: (a) foundation video generation models and (b) long video generation models.
sourced long video generation models: TALC (Bansal et al., 2024b), FIFO-Diffusion (Kim et al., 2024), FreeNoise (Qiu et al., 2024), FreeLong (Lu et al., 2024). FreePCA (Tan et al., 2025), and RIFLEx (Zhao et al., 2025). Additionally, considering that many long-video generation models are derived from foundation video generation models, we find it necessary include some of the latest mainstream open-source models. These include 1a (Wang et al., 2025), Hunyuan Video (Kong et al., 2024), CogVideoX1.55-5 (Yang et al., 2024b), Open-Sora 2.0 (Zheng et al., 2024b), and Aipen-SoraPaae V1.3 (Lin et al., 2024). For the implementation details, please refer to App. C.
Human annotation. To analyze the alignment between our metric and human perception of narrative content expression, we perform human preference labeling on a large set of generated videos. Given a prompt Pf.. and the models be evaluated {m, we randomly select two different models (m.r my). where x A y. to generate the corresponding video pairs (Vz: Vyy) for preference comparison. Corresponding to the three progressive dimensions in our metric, each video pair includes three questions (see App. D.1 for more details). Since n = 1 does not involve transition coherence between TNAs, we select test prompts within the range of n E [2, 6]. Additionally, for each prompt, we select two video pairs, ultimately forming 600 pairs (i.e., 1.8k questions) that require annotation. For each pair, we invite three human annotators. To ensure the correct understanding of the annotation task, we provide detailed training instructions to the annotators prior to the annotation process.
Implementation settings. In our prompt suite construction pro- cess, we utilized - (Yang et al., 2024a), which excels in text analysis and instruction-folloiing capabilities, to extract scene and object elements from 200k text prompts. For the prompt generation pipeline, we chose GPT-4o (Hurst et al., 2024). For our evaluation metric, we employ the latest Qwen2.5-VL-72B. Instruct (Bai et al.. 2025) as our MLLM. For the video input, we extract visual input by sampling 2 frames per second to feed into the MILMM. The threshold Teuv is set to 0.3. All experiments were conducted on machines equipped with 8 x H20 GPUs.
Recults on the number of TNA expressions
"oundation medda
Upper bouns
Ner
TMA Numbers
4.2 EVALUATION RESULTS
Figure 5: Evaluation the num- ber of TNA expressions Neep-
Building the NamLV benchmark, perform a series of eval- uations (see App. D.2 for calculation details.) and distill four key observations regarding current video generation models.
(i) Richer narrative semantics in text prompts weaken the model's representation of narrative units, while its ability to represent basic elements remains
7

---

Published as a conference paper at ICLR 2026
Evaluation results of the VideoCraft-basee video generation models
0.25
VinoCrt Pransaing
Aidinn Fret
0.71
0.20
0,450
0.63
0,42
0.67
0,400
0.53
0.375
0.50
0.150
Mumbers
TNA Numbers
Figure 6: Evaluation results across three evaluation dimensions. Evaluated models include the foundation video generation model VideoCraft and the extended long video generation models (i.e., FIFO-Diffusion, FreeLong. FreeNoise and FreePCA).
relatively unaffected. As shown in Fig. 4, we present the performance of foundation and long video generation models across three evaluation dimensions. As the number of TNAs increases, metrics for narrative units, namely Rew and Rooh. exhibit a noticeable downward trend, while the metric for narrative elements, Ría, fluctuates within small range. This suggests that even with text enriched in narrative content, the model is able to extract key elements for generation. However, constructing narrative content that evolves over time using these elements remains a challenge.
(ii) Current models can only represent a very limited number of narrative units. Considering that Rewv reflects the average generation rate of TNAs, we introduce new metrie Nexp Rew x n, which represents the number of TNAs that the model can effectively express. As shown in Fig. 5, with the increase in TNA numbers, Neep for both types of models shows very slow increase, with the gap to the upper bound gradually widening. Therefore, when applying existing models, it is advisable that the number of TNAs contained in a given prompt does not exceed 2.
(iii) The foundation model determines the narrative expression capability of the long video generation models derived from it. Existing long video generation models are typically constructed by introducing specially designed modules onto the foundation model. For instance, MFFO-Diffusion. FreeLong, FreePCA and FreeNoise are all derived from VideoCraft (Chen et al., 2023; 2024a). Fig. illustrates their performance. Interestingly, these models showease similar capabilities in narrative elements (i.e., Raa). However, terms of narrative unit expression capabilities (i.e., Rao and Rach), all long video models outperform VideoCraft, demonstrating the effectiveness of existing long video module designs. Nevertheless, the Rew and Resh of these long video models are quite similar, indicating that the capability of long video generation models largely depends on the foundation model employed. Although existing long video models perform less effectively than the latest foundation models (as shown Fig. 4 and Fig. 5), these foundation models provide broad research opportunities for the advancement of long video generation.
(iv) The impact of TNA change factors. As shown in Tab. 1, we summarize the subsets cor- responding to three factors (i.e., Satts fact- tan), and calculate the model's performance the three evaluation dimensions. With respect to narrative element generation (Rea), the model demonstrates superior average performance on the initial ob- ject action (tact) compared to the other two factors (san. tat). How- ever, for narrative units (Reovs Reoh), the model's performance poorest along the object action factor (faz)). This indicates that the model excels in accurately generating object action, but struggles with achieving diverse action variations.
Table 1: Comparison of model scores across three change factors under various metrics.
Rnd
Reov
Rech
Model
9ul
fast
8an
fan.
faxt
8ati
fyt.
faxs
Foundation video generation models
Wan
74,9
77.8
82.5
68.8
72.7
70.3
50.1
52.4
54.5
Hunyuan Video
74.4
77.2
76.9
64.3
64.6
57.9
44.7
44.2
40.8
Cog VideoX
67.3
69.9
69.1
62.9
60.2
58.6
44.5
38.9
43.1
Open-Sora
71.6 71.4
76.8
59.0
63.2
56.7
41.4
44.1
41.1
Sonn-Sor--Paan
68.5 67.8
73.6
59.3
60.6
52.7
38.9
39.0
35.2
Long video generation models
RIFLEx
59.6 62.4
67.8
56.1
59.4
52.7
39.2
39.9
39.2
FreeL.ong
74.4 72.3
76.3
56.7
64.2
52.8
38.2
42.2
35.7
FreeNoise
77.6 71.5
74.5
58.5
63.0
51.2
40.7
43.1
34.4
FreePCA
69.6 67.8
72.3
55.7
60.5
53.2
37.1
40.4
35.8
FIFO-Diffusion
71.3 68.4
75.0
58.9
61.2
53.1
39.1
40.3
35.5
TALC
38.0 37.1
40.4
31.0
33.0
31.6
21.9
23.4
21.7
Mean
67.9 67.6
71.4
57.4
60.3
53.7
39.6
40.7
37.9
8

---

Published as a conference paper at ICLR 2026
4.3 ADDITIONAL ANALYSIS
Statistical analysis of our prompts suite. Fig. 1 presents a statistical distribution of TNA numbers for our prompts compared other representative benchmark prompts. Clearly, our prompt suite covers a broader and more uniform range of TNA numbers, facilitating a comprehensive evaluation of video generation models' narrative expression capability. Additionally, as shown in Fig. we perform word cloud analysis on 600 meticulously selected prompts. It is evident that words like suddenly, next, and finally; which pertain to the progression of narrative content, hold significant weight, aligning with our carrative-centrie evaluation objectives.
Table Comparison of metrics across different benchmarks. Consist-n/3 denotes the subset with n consistent results out of three annotations.
Table 3: Ablation results on our metric. Consist- n/3 denotes the subset with n consistent results out of three annotations.
Consist-2/3
Consist-3/3
Consist-2/3
Consist-3/3
Metric
# Variation
Rsa
Reoy
Reah Raa Reor
Rech
Raa
Rew Rech
Raa
Rews Rech
VBench-2.0 0.33 0.32 0.28 0.31 0.27 0.29
baseline
0.63 0.67 0.67 0.81 0.80 0.79
StoryEval 0.41 0.51 0.51 0.55 0.55 0.56
2 1-response
0.61 0.63 0.64
0.81 0.77 0.78
3-responses
0.62 0.66 0.67
0.81 0.78 0.80
Ours
0.63 0.67 0.67 0.81 0.80 0.79
4 adjust MLLM 0.65 0.63 0.64
0.78 0.72 0.75
Analysis of alignment with human judgments. We use the video preference dataset annotated by three human participants, selecting data where two or three participants choose the same answers, which form subsets Consist-2/3 and Consist-3/3, and consider these annotations as groundtruth. Then, we analyze the evaluation accuracy of our metric and related metrics in align- ment with this groundtruth (see App. D.1 1 for more details). The results in Tab. 2 indicate high level of alignment between our metric and human perception, ensuring the reliability of the above evaluation conclusions. We compare our metric with the recent benchmarks involving narrative content evaluation, i.e., VBench- 2.0 (plot) (Zheng et al., 2025) and StoryEval (Wang al., 2024a). Unlike VBench2.0, which uses video descriptions to make judg- ments, and StoryEval, which requires the model to assess all narrative units at once, our progressive, question-driven approach demonstrates significant performance advantage.
beginn
Color
oright
nexit
black
ght
sucia
dooer n
bark
marr tafa carsing
whi S C nee
shife
green- stand yellow
clear
Figure 7: Word cloud analysis results of our prompt suite.
Ablation analysis of metric design. Tab. (#1) presents the alignment accuracy of our metric with human judgments. Tab. 3 (42) and Tab. 3 (43) represent using MLLM to generate answers once and three times, respectively. As the frequency of responses increases, the accuracy correspondingly improves. However, when comparing Tab. 3 (#3) with Tab. 3 (#1), which uses 5-responses, accuracy shows signs of convergence. Hence, we choose the 5-responses approach. Finally, Tab. 3 (#4) denotes the replacement of the Owen2.5-VL--7B with @wen2.5-VLL-2BB. The results indicate that reduction in MLLM capacity adversely affects accuracy.
Results Intar-Frama Feeture
Feature-level visualization analysis. In addition analyzing the generated result videos, we also aim to provide explanations from an intermediate feature level. Specifically, we introduce metric D,. defined as the average feature distance between consecutive frames. We obtain measurement results using the Wan2.1-145 under 6 TNAs and show the results in Fig. 8. Intuitively, an increase in the number of TNAs leads to more information- rich video, resulting in corresponding increase in inter-frame distances. However, due to the limited amount of information that can be conveyed within unit of time, D, ultimately shows converging trend. For implementation details, See App. D.3.
Aan
Numbers
Figure 8: Analysis results on inter-frame feature distance Dr.
To intuitively understand the narrative expression capability of the model, we present in App. C.2 shows video generation results for prompts with different TNA counts and change factors.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
To accommodate the pursuit of long video generation models for expressing rich narrative content extended durations, propose NarrLV, novel benchmark dedicated to comprehensively assessing the narrative expressiveness of long video generation models. Inspired by the film narrative theory, we introduce prompt suite with flexibly extendable narrative richness and effective metric based on progressive narrative content expression. Consequently, we conduct extensive evaluations of existing long video generation models and the foundation generation models they typically depend on. Experimental results reveal the capability boundaries of these models across various narrative expression dimensions, providing valuable insights for further advancements. Moreover, our metric shows high consistency with human judgments. We hope this reliable evaluation tool can facilitate future assessments of long video generation models.
ACKNOWLEDGMENTS
This work is jointly supported by the National Science and Technology Major Project (No. 20.22220001 (6403) and the Strategic Priority Research Program of Chinese Academy of Sciences (No.XDA27010201),
REFERENCES
Rudolf Arnheim. Film as art. Univ of California Press, 1957.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge. Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 2, 22
Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Sypektor, Aditya Grover, and Kai- Wei Chang. Tale: Time-aligned captions for multi-scene text-to-video generation. arXiv preprint arXiv:2405.04682. 2024a. 1, 3, 26, 34
Hritik Bansal, Yonatan Bitton. Michal Yarom, Idan Szpektor, Aditya Grover, and Kai-Wer Chang. Tale:
Time-aligned captions for multi-scene text-to-video generation. arXiv preprint arXiv:2405.04682 2024b. 2, 7, 28
Seymour Benjamin Chatman and Seymour Chatman. Story and discourse: Narrative structure in fiction and film. Cornell university press, 1980. 2, 5,
Chubin Chen, Sujie Hu, Jiashu Zhu, Meiqi Wu, Jintao Chen, Yanxun Li, Nisha Huang, Chengyu Fang,
Jiahong Wu, Xiangxiang Chu, et al. Taming preference mode collapse via directional decoupling alignment diffusion reinforcement learning. arXiv preprint arXiv:2512.24144 2025a. 26
Chubin Chen, Jiashu Zhu, Xiaokun Feng, et al. S -g-giidance: Stochastic self guidance for training-free enhancement of diffusion models. arXiv preprint arXiv:2508. /2880, 2025b. 26
Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma, et al. Skyreels-v2: Infinite-lnngth film generative 2025c. 26, 27
model. arXiv preprint ar XrXiv:2504.133
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang. Xiaodong Cun, Shaoshu Yang. Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafterl: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310. 195/2, 2023.
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
7310-7320, 2024a. 8, 26, 29
Honghao Chen, Yurong Zhang, Xiaokun Feng, Xiangxiang Chu, and Kaiqi Huang. Revealing the dark secrets of extremely large kernel convnets on robustness. arXiv preprint arXiv:2407.08972 2024b. 21
10

---

Published as a conference paper at ICLR 2026
Honghao Chen, Xingzhou Lou, Xiaokun Feng, Kaiqi Huang, and Xinlong Wang. Unveiling chain of step reasoning for vision- asion-language models with fine- grained rewards. arXiv preprint arXiv:2509.19003, 2025d. 21
Lin Chen. Exploring the impact of short videos society and culture: An analysis of social dynamics and cultural expression. Pacific International Journal, 6(3):115-118, 2023. 35
Rui Chen, Lei Sun, Jing Tang, Geng Li, and Xiangxiang Chu. Finger: Content aware fine-grained evaluation with reasoning for ai-generated videos. arXiv preprint arXiv:2504. 10358, 2025e. 2
Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine- ane-grained evaluation for text-to-image generation. arXiv preprint arXiv:2310 20. 0. 18235, 2023a. 2, 3, 5, 21
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3043-3054 2023b. 5
Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang. Sora an agi world model? a complete survey on text-to-video generation. arXiv preprint arXiv:2403.05133, 2024. 1,4
Xiangxiang Chu, Renda Li, and Yong Wang. Usp: Unified self-supervised pretraining for image generation and understanding, arXiv preprint arXiv:2503.06132. 2025. 21
Elizabeth Cowie. The popular film progressive text discussion of coma. In Feminism and film theory, pp. 104-140. Routledge, 2013. 2,5
James E Cutting. Narrative theory and the dynamics of popular movies. Psychonomic bulletin & review, 23:1713-1743, 2016. 2, 3,
Juntao Dai, Tianle Chen, Xuyao Wang, Ziran Yang, Taiye Chen, Jiaming Ji, and Yaodong Yang. Safesora: Towards safety alignment of text2video generation via a human preference dataset. Advances in Neural Information Processing Systems, 37:17161-17214 2024. 35
Andrzej Diniejko. Introduction to the Study of Literature and Film in English. Uniwersytet Warsza- wski, 2010.
Thomas Doherty. Hollywood and Hitler, 1933-1939 Columbia University Press, 2013.
Weixi Feng, Jiachen Li, Michael Saxon, Tsu-jui Fu, Wenhu Chen, and William Yang Wang. Te-bench: Benchmarking temporal compositionality in text-to-video and image-to-video generation. arXiv preprint 406.022406.08666, 08656, 2024a. 2, 3,4, 17
Xiaokun Feng. Shiyu Hu, Xiaotang Chen, and Kaiqi Huang. A hierarchical theme recognition model for sandplay therapy. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 241-252. Springer, 2023. 20
Xiaokun Feng. Xuchen Li, Shiyu Hu, Dailing Zhang, Jing Zhang, Xiaotang Chen, Kaiqi Huang, et al. Memvlt: Vision-language tracking with adaptive memory- prompts. Advances in Neural Information Processing Systems, 37:14903-14933, 2024b. 27
Xiaokun Feng, Shiyu Hu, Xuchen Li, Dailing Zhang. Meiqi Wu, Jing Zhang, Xiaotang Chen, and Kaiqi Huang. Atetrack: Aligning target-context cues with dynamic target states for robust vision- language tracking. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 19850-19861, 2025a. 22
Xiaokun Feng, Dailing Zhang, Shiyu Hu, Xuchen Li, Meiqi Wu, Jing Zhang, Xiaotang Chen, and Kaiqi Huang. Cstrack: Enhancing rgh-x tracking via compact spatiotemporal features. arXiv preprint arXiv:2505. 19434, 2025b. 27
Xiaokun Feng, Dailing Zhang, Shiyu Hu, Xuchen Li, Meiqi Wu, Jing Zhang, Xiaotang Chen, and Kaiqi Huang. Enhancing rision-languse tracking by effectively converting textual cues into visual cues. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1-5. IEEE, 2025c. 27
11

---

Published as a conference paper at ICLR 2026
Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. Advances in Neural Information Processing Systems, 14423885-23889 2021. 29
Xiangwei He and Lijuan Fang. Regulatory challenges synthetic media governance: Policy frameworks for ai ai-generated content across image, video, and social platforms. Journal of Robotic Process Automation, Al Integration, and Workflow Optimization, 9(12):36-54 2024. 35
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 2,3
Tobias Hinz, Stefan Heinrich, and Stefan Wermter. Semantic object accuracy for generative text- to-image synthesis. IEEE transactions on pattern analysis and machine intelligence, 44(3): 1552-1565, 2020.
Shiyu Hu, Dailing Zhang, Xiaokun Feng, Xuchen Li, Xin Zhao, Kaiqi Huang, et al. A multi-modal global instance tracking benchmark (mgit): Better locating target in complex spatio-temporal and causal relationship. Advances in Neural Information Processing Systems, 36:25007-25030, 2023a. 2.4
Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 20406-20417, 2023b. 2, 5,
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 21877-21818, 2024a. 2,
Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang. Yaohui Wang, Xinyuan Chen, Ying-Cong Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench++: Comprehensive and versatile benchmark suite for video generative models. arXiv preprint arXiv:2411.13500, 2024b. 35
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card, arXiv preprint arXiv:2410.21221 2024. 2, 17
Amelia Katirai, Noa Garcia, Kazuki Ide, Yuta Nakashima, and Atsuo Kishimoto. Situating the social issues of image generation models in the model life cycle: sociotechnical approach. Al and Ethics, pp. 1-18, 2024. 35
Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint 47X1v.2405.11473, 2024. 1, 7, 22, 26, 28, 29, 33, 34
Subin Kim, Seoung Wug Oh, Jui-Hsien Wang, Joon- Young Lee, and Jinwoo Shin. Tuning-free multi- event long video generation via synchronized coupled sampling, arXiv preprint 0rX1v:2503.08605. 2025. 2
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600 2024. 2, 4, 7, 22, 28,
Markus Kuhn. Film narratology: Who tells? who shows? who focalizes? narrative mediation in self-reftexive fiction films. Point of View, Perspective, and Focalization. Modeling Mediacy in Narrative, pp. 278-278, 2009.
Sheng Li, Zhiqiang Tao, Kang Li, and Yun Fu. Visual to text: Survey of image and video captioning. IEEE Transactions Emerging Topics in Computational Intelligence, 3(4):297-312, 2019. 6
12

---

Published as a conference paper at ICLR 2026
Xuchen Li, Xiaokun Feng, Shiyu Hu, Meiqi Wu, Dailing Zhang, Jing Zhang, and Kaiqi Huang. Dil: Diverse text generation for visual language tracking based on llm. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7283-7292 2024a. 17
Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, and Kaiqi Huang. Dtvit: A multi-modal diverse text benchmark for visual language tracking based on Ilm. arXiv preprint vrXiv:2410.02492. 2024b. 17
Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, and Kaiqi Huang. How texts help? a fine-grained evaluation to reveal the role of language in vision-language tracking. arXiv preprini arXiv:2411.15000 2024c. 17
Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, and Kaiqi Huang. Visual language tracking with multi-modal interaction: A robust benchmark. arXiv preprint arXiv:2409.0880 08887, 2024d. 17
Mingxiang Liao, Qixiang Ye, Wangmeng Zuo, Fang Wan, Tianyu Wang, Yuzhong Zhao, Jingdong Wang, Xinyu Zhang, et al. Evaluation of text-to-video generation models: A dynamics perspective. Advances in Neural Information Processing Systems, 37:109790-109816, 2024.
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.0013 2024. 7, 28, 33
Xinran Ling, Chen Zhu, Meiqi Wu, Hangyu Li, Xiaokun Feng, Cundian Yang, Aiming Hao, Jiashu Zhu, Jiahong Wu, and Xiangxiang Chu. Vmbench: A benchmark for perception-aligned video motion generation. arXiv preprint arXiv:2503. 10076, 2025. 2.3
Xiao Liu, Xinhao Xiang, Zizhong Li, Yongheng Wang, Zhuoheng Li, Zhuosheng Liu, Weidi Zhang, Weiqi Ye, and Jiawei Zhang. A survey of ai-generated video evaluation. arXiv preprint arXiv:2410. 19884, 2024a. 3.4
Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang. Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b. 2
Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Freelong: Training-free long video generation with spectralblend temporal attention. arXiv preprint 1rX1v:2407.19918, 2024. 7, 26, 28, 29, 33, 34
Yongjia Ma, Junlin Chen, Donglin Di, Qi Xie, Lei Fan, Wei Chen, Xiaofei Gou, Na Zhao, and Xun Yang. Tuning-free long video generation via global-local collaborative diffusion. arXiv preprint arXiv:2501.05484, 2025. 2
Fangyuan Mao, Aiming Hao, Jintao Chen, Dongxia Liu, Xiaokun Feng, Jiashu Zhu, Meiqi Wu, Chubin Chen, Jiahong Wu, and Xiangxiang Chu. Omni-effects: Unified and spatially-comtrollabtee visual effects generation. arXiv preprint 250.::::000007881,, 2025.
Robert McKee. Story. Dixit, 2005.
Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin Satoh. Toward verifiable and reproducible human evaluation for text-to- image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14277-14286, 2023. 2,3
Tianhao Qi. Jianlong Yuan, Wanquan Feng, Shancheng Fang, Jiawei Liu, SiYu Zhou, Qian He, Hongtao Xie, and Yongdong Zhang. Mask Dual mask-based diffusion transformer for multi- scene long video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 1,3
Haonan Qiu, Menghan Xia, Yong Zhang. Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2024. 1, 2, 3, 7, 22, 26, 28, 29, 33, 34
13

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PmLR, 2021. 2
Daniel SL Roberts, Paul S Cowen, and Brenda E MacDonald. Effects of narrative structure and emotional content on cognitive and evaluative responses to film and text. Empirical Studies of the Arts, 14(1):33 47, 1996. 2,5
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, 2016.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209. 14792, 2022.
Jiangtong Tan, Hu Yu, Jie Huang, Jie Xiao, and Feng Zhao. Freepea: Integrating consistency information across long-short frames in training-free long video generation via principal component analysis. In Proceedings of the Computer Vision and Putlern Recognition Conference, pp. 27979- 27988, 2025.
Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, et al. Magi-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505. 2025. 26, 27
Ye Tian, Ling Yang. Haotian Yang, Yuan Gao, Yufan Deng, Xintao Wang, Zhaochen Yu, Xin Tao, Pengfei Wan, Di ZHANG, et al. Videotetris: Towards compositional text-to-video generation. Advances in Neural Information Processing Systems, 37:29489-29511 2024. 3
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marein Michalski. and Sylvain Gelly. FVD: A new metric for video generation. In International Conference on Learning Representations Workshop, 2019. 2,
Peter Verstraten. Film narratology. University of Toronto Press, 2009.
Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. Advances in neural information processing systems, 29, 2016.
Ang Wang. Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprini urXiv:2503. 24314, 2025. 1, 2, 5, 7, 22, 28,
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elscope text-to-video technical report. arXiv preprint (xXiv:2338.005777 2023. 26
Wenhao Wang and Yi Yang. Vidprom: A callionn-salle real prompt-gallery dataset for text-oo-video diffusion models. Advances in Neural Information Processing Systems, 101666618666660 2024.
Wenhao Wang and Yi Yang. Videoufo: A million-scale user-focused dataset for text-to-video generation. arXiv preprint arXiv:2503.011 01739,, 2025. 4, 18
Yiping Wang, Xuchai He, Kuan Wang, Luyao Ma. Jianwei Yang. Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Is your world simulator a good story presenter? a consecutive events-based benchmark for future long video generation. arXiv preprint 2xXi:22412.16211, 2024a. 35
17,
Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models. arXiv preprint arXiv:2410.02757, 2024b. 2
Zhanyu Wang, Longyue Wang, Zhen Zhao, Minghao Wu, Chenyang Lyu, Huayang Li, Deng Cai, Luping Zhou, Shuming Shi, and Zhaopeng Tu. Gpt4video: A unified multimodal large language model for cndructiio-followwe understanding and safety-aware generation. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 3907-3916, 2024c. 35
14

---

Published as a conference paper at ICLR 2026
Faraz Waseem and Muhammad Shahzad. Video is worth a thousand images: Exploring the latest trends in long video generation. arXiv preprint arXiv:2412.1866 2024.
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17096 2023, 23
Meiqi Wu, Jiashu Zhu, Xiaokun Feng, Chubin Chen, Chen Zhu, Bingze Song, Fangyuan Mao, Jiahong Wu, Xiangxiang Chu, and Kaiqi Huang. Imagerysearch; Adaptive test-time search for video generation beyond semantic dependency constraints. arXiv preprint arXiv:2510.14847, 2025a. 26
Meiqi Wu, Bingze Song, Ruimin Lin, Chen Zhu, Xiaokun Feng, Jiahong Wu, Xiangxiang Chu, and Kaiqi Huang. Latent temporal discrepancy as motion prior: A loss-weighting strategy for dynamic fidelity in 12v. arXiv preprint arXiv:2601.20504 2026. 26
Weijia Wu, Zeyu Zhu, and Mike Zheng Shou. Automated movie generation via multi-agent cot planning. arXiv preprint arXiv:2503.07314. 2025b. 26, 27
Junfei Xiao, Ceyuan Yang, Lvmin Zhang, Shengqu Cai, Yang Zhao, Yuwei Guo, Gordon Wetzstein, Maneesh Agrawala, Alan Yuille, and Lu Jiang. Captain cinema: Towards short movie generation. arXiv preprint arXiv.2507.18334 2025. 26,
Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. survey on video diffusion models. ACM Computing Surveys, 57(2): 2024.
Xin Yan, Yuxuan Cai, Qiuyue Wang, Yuan Zhou, Wenhao Huang, and Huan Yang. Long video diffusion generation with segmented cross-attention and content-rich video data curation. arXiv preprint arXiv:2412.01316, 2024. 1,
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. 2024a. 2,4,7, 18
Likun Yang, Xiaokun Feng, Xiaotang Chen, Shiyu Zhang, and Kaiqi Huang. See your heart: Psychological states interpretation through visual creations. arXiv preprint 10Xiv:230..10276. 2023. 20
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408. 06072, 2024b. 1, 2, 7, 22, 26, 28, 33
Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roce Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 16:1611166,,, 2023. 2, 3, 5,
Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, et al. Nuwa-xl: Diffusion over diffusion for extremely long video generation. arXiv preprint ar ar3iv:2303. 22222333.2233 /2346, 2023. 2
Runze Zhang, Guoguang Du, Xiaochuan Li, Qi Jia, Liang Jin, Lu Liu, Jingjing Wang, Cong Xu, Zhenhua Guo, Yaqian Zhao, et al. Dropletvideo: A dataset and approach to explore integral spatio-temporal consistent video generation. arXiv preprint arXiv:2503.0m 060550 2025. 4, 18
Canyu Zhao, Mingyu Liu, Wen Wang. Weihua Chen, Fan Wang, Hao Chen, Bo Zhang, and Chunhua Shen. Moviedreamer: Hierarchical generation for coherent long visual sequence. arXiv preprint 1rXiv:2407.16655, 2024. 26,
Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, and Jun Zhu. Riflex: A free lunch for length extrapolation in video diffusion transformers. arXiv preprint arXiv:2502.1588. 2025. 1, 2, 7, 22, 26, 28,
15

---

Published as a conference paper at ICLR 2026
Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. VBench-2.08 2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arX1n:2503.21755, 2025. 3, 3,
Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma. Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, et al. tideogen-of-thogghe Step-by-step generating multi-shot video with minimal manual intervention. arXiv preprint arXiv:2412.02259 2024a. 26, 27
Zangwei Zheng, Xiangyu Peng. Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arKiv:2412. 20404, 2024b. 2, 7, 28, 33
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 40(6))
2017. 20
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Con- sistent self-attention for long-range image and video generation. Advances Neural Information Processing Systems, 37:110315-1 1105400 2024. 3
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A
MORE DETAILS ON OUR PROMPT SUITE
In this section, will provide comprehensive overview of the implementation details of prompt suite.
A.1 STATISTICAL ANALYSIS OF TNA NUMBERS IN EXISTING BENCHMARKS
Fig. presents a statistical analysis of the number of TNAs in existing representative benchmarks such VBench (Huang et al., 2024a), TC-Bench (Feng al., 2024a), and StoryEval (Wang et al., 2024a). For StoryEval (Wang et al., 2024a), since provides an event list corresponding to each prompt, and each event element in this list is TNA of interest, consider the length of this list as the number of TNAs contained in each prompt. However, for VBench (Huang et al., 2024a) and TC-Bench (Feng al., 2024a), which lack corresponding structured representations, evaluation and analysis studies based on LLMs (Li et al., 2024a;d;b;c), and employ GPT-4o (Hurst al., 2024) to perform this analysis task. Specifically, we employ the following instruction to analyze each text prompt to determine its corresponding number of TNAs.
follow recent
The prompt instruction for analyzing the number of TNAs the text.
As an expert in video narrative structure analysis, please analyze the given text based on the Temporal Narrative Atom (TNA). TNA is the minimal narrative unit in video generation that maintains continuous visual representation. can be further understood through the following examples:
"A man running" + TNA count is I, as there is one continuous action.
2. "A person stands up from chair and starts walking" - TNA count is 2, due to two actions ("stands up" + "walking"),
3. "A room changes from bright to dim" - TNA count is 2, due two environmental attributes ("bright" + "dim").
# Task Description
Given text, please analyze the number of TNAs contained in this text.
Example Demonstration
Input: A chameleon changes from brown to green Output: 2
Based on the information provided above, please help me analyze the following text and only output the final result.
Input: (User-provided information]
Table Al: The major scene categories we study and their corresponding examples.
Scene Category Artificial Landscape
Examples
Garden, Fountain, Tree Nursery, Rice Field, Wheat Field,
Hayfield, Cornfield, Vineyard, Lawn
Dining & Food Venue
Restaurant, Kitchen, Diner, Cafeteria, Fast Food Restau- rant, Café, Dessert Shop, Food Court, Beer Hall
Commercial & Retail
Clothing Store, Bookstore, Jewelry Store, Gift Shop, Hard- ware Store, Pharmacy, Grocery Store, Pet Store, Shoe
Store
Residential & Lodging
Apartment Building, Beach Villa, Cottage, Cabin, Man- sion, Prefab Home, Treehouse, Mountain Lodge, Igloo Airport, Raft, Bus Stop, Subway Station, Train Station, Parking Lot, Parking Garage, Highway, Port
Transportation Hub
17

---

Published as a conference paper at ICLR 2026
Sports Venue
Soccer Field, Basketball Court, Baseball Field, Tennis Court, Golf Course, Race Track, Gymnasium, Volleyball
Court, Boxing Ring
Industrial & Production Facility
Car Factory, Assembly Line, Repair Shop, Oil Rig, Indus- trial Zone, Energy Facility, Landfill, Warehouse, Assem-
bly Line
Public Facility & Service
Fire Station, Police Station, Courthouse, Embassy, Post
Office, School, Library, Lecture Hall, Science Muscum
Arts & Entertainment
Art Gallery, Art Studio, Music Studio, Cinema, TV Studio,
Nightclub, Carousel, Arcade, Amusement Park
Architectural Structure
Bridge, Arch, Corridor, Viaduct, Dam, Moat, Pavilion,
Gazebo, Porch
Cultural & Religious Site
Church, Mosque, Temple, Synagogue, Mausoleum, Ceme-
tery, Castle, Pagoda, Palace
Gaming & Virtual Environment
Game Scene, Sandbox Environment, Sci-Fi Scene, Ani-
mation Scene, VR/AR Enhanced Environment
Natural Geography
Forest, Rainforest, Desert, Beach, Coast, Glacier, Volcano,
Canyon, Monolith
Other Special Scene
Military Base, Catacomb, Archacological Dig, Battlefield, Trench
A.2 MORE DETAILS ON THE SCENE-OBIECT PAIR SET
Scenes and objects are the primary factors influencing TNA that we focus on, and they play significant role in constructing our evaluation prompts. For 200k text prompts from VideoUFO (Wang Yang, 2025) and DropletVideo (Zhang et al., 2025), we utilize Qwen2.5-32B, -25---2-tttnttrett (Yang et al., 2024a) to extract the list of scenes and main objects corresponding to each text prompt. The prompt instruction used is as follows:
The prompt instruction for analyzing the scene-object pair in the text.
As an expert in video narrative structure analysis, please analyze the essential elements of the text description related the video clip, to extract the corresponding scene and main objects set.
# Task Description
For a given text description about a video clip, you need to analyze its corresponding scene categories and list of main objects.
The scene category may appear directly in the text description. For texts without direct provision, you need to infer based on semantic content. b. For the analysis of the main objects, please ignore some unimportant redundant information, such as subtitles the video or OCR content on objects.
# Example Demonstration
Input: The video opens with person standing in dark room, surrounded by various digital screens displaying data and charts. The screens are colorful and dynamic, with different types of graphs and icons. The person appears to be in virtual augmented reality environment, as indicated by the holographic elements and the way the screens interact with the space. As the video progresses, the person turns around and looks at a smartphone, which displaying message that reads "LET'S KICKSTART THE FUTURE." Output: ("Virtual Augmented Reality Environment": screens"])
["Person", "Smartphone",, "Digital
Based on the information provided above, please help me analyze the following text prompt strictly following the above JSON format and only outputting this JSON. Input: (User-provided information)
18

---

Published as a conference paper at ICLR 2026
of Scarnes
2000
1000
Number Object Categprie
Figure Al: Statistical distribution of the number of object categories across different scenes.
5.6%
8.6%
8.1%
7.5%
Categories
Artificial Landscape
Dining & Food Venue
8.0%
Commercial & Retail
Residential & Lodging
Transpartation Hub
Sports Venue
5.0%
Production Facility
Facility
8.6%
Arts Entertainment
Architectural Structure
7.5%
Cultural & Religious Site
Gaming & Virtual Environment
5.6%
Natural Geography
Other Special Scene
6.4%
6.9%
6.9%
Figure A2: Statistical distribution of scene theme categories in our evaluation prompts (pie chart).
19

---

Published as a conference paper at ICLR 2026
For each text prompt, extract and obtain the corresponding scene and list of main objects. Subsequently, we merge objects within the same scene and record the frequency of occurrence for each object. Hence, each scene and its associated object list are considered 8o. forming our scene-object set So = { After aggregation, we obtained 16k such 80- We have compiled statistics the number of object categories under different scenes, shown in Fig. A1,
scene-object pair
Due to the high computational cost of video generation, is challenging to evaluate each specific scene comprchensively. Given the similarity among many scenes, have classified these scenes into 14 major categories. Tab. Al presents the names of these 14 categories along with examples of representative scenes. Considering that human-related scenes are more complex and diverse than natural scenes (Zhou et al., 2017; Feng et al., 2023; Yang et al., 2023), these categories are primarily constructed around human-related scenes. Although our evaluation prompts can encompass all of these 14 major scene categories, thus ensuring the diversity scenes in our evaluation prompts. shown in Fig. A2, we present the proportion of each scene theme category within our evaluation prompts pic chart. The balanced distribution across different scene categories indicates that our evaluation prompts are highly diverse.
impractical cover every individual scene,
A.3 MORE DETAILS ON THE AUTOMATED PROMPT GENERATION PIPELINE
As illustrated in Fig. 2 (a) and Eq. 1, we utilize the sampled scene-object pair information the specified TNA number n, and the factor for TNA change f. to generate a specific evaluation prompt Pfn using GPT-4o. The prompt instruction we employ is as follows:
The prompt instruction for evaluation prompt generation.
As expert in video narrative structure analysis, please analyze the given text based the Temporal Narrative Atom (TNA). TNA is the minimal narrative unit in video generation that maintains continuous visual representation. examples:
be further understood through the following
"A man running" TNA count is 1, there is one continuous action.
2. "A person stands up from chair and starts walking" - TNA count is 2, due to two actions ("stands up" "walking").
3. "A room changes from bright to dim" TNA count is 2, due to two environmental attributes ("bright" + "dim").
The reasons for TNA change in video narrative are primarily:
1. 1.Scene attribute changes 2. 2.Object attribute changes 3.Object action changes
# Task Description
Your task is to generate a video segment description resulting in (User-specified TNA count) TNAs due /User-specified TNA change factor) based on provided scene information and main objects:
Imagine an initial scene based on the provided scene information and objects. From this, describe the scene's is overall attribute style (e.g., "overall grayish scene," "overall sunny") and position layout of main objects in the scene.
The number of provided objects is 1. Evaluate the reasonableness of including the object in the scene based on scene type. If unreasonable, the object may be omitted. b. Extra objects may be introduced to meet the imagined scene requirements, but the total number of objects should not exceed 3.
2.Based on the initial scene, generate narrative content due to [User-specified TNA change factor) resulting in (User-specifiedd TNA count) TNAs.
a. If the TNA change factor is "scene attribute changes," consider the potential attribute categories of scene and design reasonable attribute evolution process.
b. If the TNA change factor is "object attribute changes," consider the potential attribute categories of object and design a reasonable attribute evolution process.
20

---

Published as a conference paper at ICLR 2026
If the TNA change factor is "object action changes," consider the potential action categories of object and design reasonable action evolution process.
3. Consolidate the initial scene description and subsequent TNA evolution into one text.
The final text should contain two parts: the initial scene and object layout description, followed by the TNA evolution description. Each part can be expressed in various forms.
Object layout description should introduce all potential objects, including those potentially involved in the TNA evolution description.
Object state and action description should be concise and clear.
d. The TNA count in the video segment text should match the specified count, and the type of TNA change should match the specified type.
Example Demonstration
For generating video content descriptions due to (User-specified TNA change factor) with TNA count of {User-specified TNA count), here are reference examples:
<Examples of text for specific TNA count and TNA change factor Based on the above prompt, please help generate the textual description for the following input. Note: Only output the final description text without additional explanations. Input: (User-provii providdd information]
For the aforementioned <Examples of text for specific TNA count and TNA change factor taking TNA count of 3 and the change factor "object action changes" example, provide the following example:
The evaluation prompt examples for 3 TNAs with the "object action changes" factor.
1. Examplel:
Input:
Scene:
Undersea
Object:
Coral
Output: In the tranquil undersea world, vibrant corals spread out with a turtle hovering just above. Initially, the sea turtle slowly descends toward the corals. Then, the turtle stops and rests coral. Finally, the turtle starts swimming upwards.
Example2:
Input:
Scene: Bench
Object: Person
Output: In the gentle afternoon sunlight, person sits quietly on bench, reading book. Then, the person puts the book away. Finally, the person stands up from the bench.
B MORE DETAILS ON OUR METRIC
In this section, we provide further implementation details of our evaluation metric.
B.1 DISCUSSION ON MILM ANSWERS TO UNCERTAIN QUESTIONS
Our evaluation metric computation employs a recently widely-adopted MLLM-based question generation and answering framework (Cho al., 2023a; Yarom et al., 2023; Hu al., 2023b), which leverages the powerful content understanding capabilities (Chu et al., 2025; Chen et al., 2025d) of MLLMs to perform robust evaluation (Chen et al., 2024b). For each question, we have an MLLM respond five times, and use the proportion of specific answer among these five responses as the final outcome. We utilize this method because we have found that MLLMs tend to produce inconsistent answers across multiple repetitions for uncertain questions. Moreover, the degree of uncertainty of question directly influences the inconsistency of its answers. As illustrated Fig. A3, we present three questions concerning two video frame images, with each question requiring Q
21

---

Published as a conference paper at ICLR 2026
Q3:1s the scene
Do:e the Q2:Does the feature cozy chair?
characterized by the sun shining brightly in a blue sky?
feature potted plants?
Yes, Yes, Yes, Yes, Yes
No, No, No, No
Yes, No, Yes, No
Yes, Yes, Yes,
Yes, Yes, Yes, Yes, Yes
Yes, Yes, Yes, Yes, Yes
Yes, Yes
Figure A3: An example illustrating the inconsistent responses of MILM uncertain questions. For QI and Q2, MLLM provides consistent answers five times due to the clarity of judgment based on video frames. However, for Q3, the uncertainty present in the top image results inconsistent responses from MILM.
(Bai et al., 2025) to provide answers five times. The first two questions about the existence of objects yield consistent answers across all five responses from the MLLM due to the clear determination possible from the frame images. However, the third question concerning scene attributes shows inconsistency answers based on top image, indicating uncertainty, whereas the bottom image, providing clear basis for the question, results in completely consistent answers from the MLLM.
Reproducibility of the proposed metrics. As mentioned previously, MILMSs may produce inconsis- tent answers the same question, and this variance is associated with the inherent ambiguity of the question. Therefore, we use the mean multiple responses as the final answer. To assess the impact of this randomness (i.e., non-zero temperature) on result reproducibility, we conducted random error analysis under different sampling counts. Specifically, for a given sample count n, we draw n responses for each evaluation and repeat this process three times. The mean absolute error between each pair among the three sets of results then calculated and taken as the measure random error. As shown in Tab. A2, the random error decreases monotonically with increasing and eventually stabilizes. When using five samples, the random error falls below 0. 1,, indicating that our evaluation results are highly reproducible.
Table A2: Mean absolute error under different sample counts for various models.
Model
1
2
5
Wan (Wang al., 2025)
0.0031
0.0025
0.0013
0.0009
0.0009
Cog VidexX (Yang et al., 2024b)
0.0035
0.0019
0.0017
0.0008
0.0007
Hunyuan Video (Kong et al., 2024)
0.0024
0.0020
0.0013
0.0009
0.0008
RIFLEx (Zhao et al., 2025)
0.0044
0.0025
0,0013
0.0008
0.0008
FreeNoise (Qiu et al., 2024)
0.0038
0.0022
0.0014
0.0010
0.0009
FIFO-Diffusion (Kim et al., 2024)
0.0027
0.0024
0.0013
0.0009
0.0009
B.2 MORE DETAILS ON THE IMPLEMENTATION OF OUR METRIC
The overall computation process of our progressive narrative-expressive evaluation metric is presented in Sec. 3.3. Here, we provide additional implementation details. Firstly, the calculation of narrative element fidelity (Rna), it is expected that the information the scene and main objects of interest (Feng et al., 2025a) is well generated at the initial frame. Therefore, the step [( v) MLLM, Ana) x5. only use v containing the initial frame image. Additionally, considering that the aesthetic
22

---

Published as a conference paper at ICLR 2026
quality of the generated video affects narrative effectiveness various levels (Amheim, 1957; Doherty, 2013), we incorporate the aesthetic score of the initial video frame as fixed offset, treating aesthetic questions part of the question set and integrating it into the final metric calculation across the three metric dimensions. Specifically, we utilize the latest aesthetic evaluation model, Q-align (Wu et al., 2023), and map its aesthetic score model is used for this aesthetic question, it needs to be answered only once by the model.
to 1 range. Since dedicated aesthetic evaluation
Additionally, given an evaluation prompt, we use GPT-4o automatically generate corresponding questions. First, we utilize GPT-4o to organize the evaluation prompt into structured text. For the first evaluation dimension that focuses on scene and object elements, the structured text includes information on "Seene Type," "Main Object Category," "Initial Scene Attributes," " and "Main Object Layour." " For the second and third evaluation dimensions that focus on narrative unit information, the structured text contains list information derived from various TNA evolution states. The prompt instruction for implementing this operation is as follows:
The prompt instruction for structured text extraction of the evaluation prompt.
As an expert in video narrative structure analysis, please analyze the given text based on the Temporal Narrative Atom (TNA). TNA is the minimal narrative unit in video generation that maintains continuous visual representation. can be further understood through the following examples:
1. "A man running" - TNA count is 1, as there is one continuous action.
2. "A person stands up from chair and starts walking" - TNA count is 2, due actions
(" ("stands up" + "walking"),
"A room changes from bright to dim" TNA count 2, due two environmental attributes ("bright" + "dim").
The reasons for TNA change in video narrative are primarily: Scene attribute changes
2. Object attribute changes
3. Object action changes
# Task Description
You will be provided with text list describing the TNA evolution process of video narrative content. Your task is to analyze the scene category and main objects, and from that, organize the evolution process concerning specific TNA change factors.
Regarding the text list, here are some additional introductions: Each element describes only one TNA state, meaning the length of the list equals the TNA count of this entire video narrative content.
b. The elements in the list evolve sequentially over time; the first element in the list contains the initial scene description, also indicating the scene and main objects of the entire video narrative content, as well as the initial position layout information of the main objects in the scene.
c. The subsequent elements in the list focus primarily on specific TNA evolution descriptions. d. Accompanying this list is the reason for TNA evolution for the entire timeline narrative content, being one of the three reasons mentioned above.
2. First, analyze the scene category, main object category, initial attributes of the scene, and initial position layout information of the main objects in the scene, mainly based on the initial scene description in the first element of the list.
3. Then, based on the subsequent list elements reflecting the TNA evolution situation and the reason for TNA change, extract the evolution states of the scene or object. The number of extracted evolution states should match the number of list elements.
4. The final output should follow specific JSON format. Please refer to the format in the examples below for precise output.
23

---

Published as a conference paper at ICLR 2026
# Example Demonstration
Example1:
Input: Text narrative content: ["At the foot of the hill, lush vegetation thrives in the afternoon sunlight.", ", "Suddenly, rain begins to pour down, enveloping the entire scene." "]
'TNA Change Reason: Scene Attribute Changes
TNA Count: 2
Output: "Scene Type": "foot of the hill", "Main Object Category": ["lush vegetation",, "Initial
Scene Attributes": "warm afternoon sunlight". "Main Object Layout": "lush vegetation thrives at the foot of the hill.", "TNA Evolution States": ["warm afternoon sunlight", "rain enveloping the entire scene"]
2. Example2:
Input:
Text narrative content: ("On the balcony, there is potted plant next watering can, and stands on the balcony gazing into the distance.", ", "The man begins to water the potted plant on the balcony.", "The man retreats back into the house, leaving the balcony."] TNNA Change Reason: Object Action Changes
TNA Count:
Output: "Scene Type": "balcony", "Main Object Category": ["potted plant", "watering can". "man"J, "Initial Scene Attributes": null, "Main Object Layout": "a potted plant next to watering can, and a man stands on the balcony gazing into the distance", "TNA Evolution States": ["the man stands on the balcony gazing into the distance", "the man waters the potted plant", "the man retreats back into the house"]
Based on the above prompt, please assist me in extracting the structured text for the following input. Note: Only output the final structured text without additional explanations. Input: (User-provided information
Subsequently, based on this structured text, we utilize GPT-4o to generate corresponding judgment questions. The prompt instruction used is as follows:
The prompt instruction for question generation.
As expert in video narrative structure analysis, please analyze the given text based the Temporal Narrative Atom (TNA). TNA is the minimal narrative unit in video generation that maintains continuous visual representation. can be further understood through the following examples:
"A man running" - TNA count is 1, as there is one continuous action.
2. "A person stands up from chair and starts walking" - TNA count is 2, due to two actions ("stands up" + "walking").
3. "A room changes from bright to dim" - TNA count is 2, due to two environmental attributes ("bright" + "dim").
The reasons for TNA change video narrative are primarily:
Scene attribute changes
2. Object attribute changes Object action changes
Task Description
You will be provided with JSON data analyzing the elements of the video narrative content. Your task is to generate a series of corresponding questions based on this JSON data. Each key-value pair corresponds to specific questions to be generated as follows:
1. "Scene Type": Corresponds to the scene location where the entire video narrative occurs. The question template to be generated is: "Does the scene take place in the xx(scene name)?"
24

---

Published as a conference paper at ICLR 2026
2. "Main Object Category": Corresponds to the main objects involved in the entire video narrative. The question template to be generated is: "Does the scene feature xx(main object name) Note: If multiple main objects are involved, corresponding question needs to be generated for each object separately.
3. "Initial Scene Attributes": Corresponds to the initial attributes of the scene in the entire video narrative. The question template be generated is: "Is the scene characterized by xx(initial scene attribute)?"
4. "Main Object Layout": Corresponds to the positional layout information of the main objects in the entire video narrative. The question template to generated is: "Is the xx positioned as xxx(object and its positional layout information)?"
a. Note: If there is a positional layout relationship between multiple objects, a corresponding question needs be generated for each layout relationship separately.
5. "TNA Evolution States": Corresponds to the information the evolution of TNA states. This is a list data type, e.g., [TNA state I, TNA state 2, xxx]. The method of generating questions and the corresponding template a. First, for each TNA state, generate a question separately, e.g., "Does the video contain any segments that xx(TNA state 1)?". "Does the video contain any segments that xx(TNA state 2)?"...
follows:
This type of question matches the number of elements in the list.
Then, generate questions regarding the transition between adjacent TNA states. For adjacent TNA state I and 2, the question is: "Does the scene transition from Xxx(TNA state 1) to Xxx(TNA state 2) over time? To judge this question, must first be determined that the scene exhibits Xxx(TNA state I) at a certain time and subsequently exhibits Xxx(TNA state 2), with clear transition process over time.
6. When generating specific questions, analyze the specific content to make adjustments. Feel free adjust the templates to ensure the questions flow smoothly. The final output must fol- low specific JSON format for structured output. Refer to the examples below for the exact format.
# Example Demonstration
Examplel:
Input:
-NNA Element Information: { "SScene Type": "foot of the hill". "Main Object Category": ["lush "Initiaa
vegetation"], "Initial Scene Attributes": "warm afternoon sunlight", "Main Object Layout": "lush vegetation thrives at the foot of the hill.", "TNA Evolution States": ["warm afternoon sunlight", "rain enveloping the entire scene"]
TNA Change Reason: Scene Attribute Changes
TNA Count: 2
Output: { "Scene Type": ["Does the scene take place at the foot of the hill?"], "Main Object Category": ["Does the scene feature lush vegetation?"), "Initial Scene Attributes": ["Is the scene characterized by warm afternoon sunlight?"), "Main Object Layout": ["Does the lush vegetation thrive the foot of the hill?"], "TNA Evolution StatesO": D'Does the video contain any segments showing warm afternoon sunlight?", "Does the video contain any segments where rain engulfs the entire scene?"), "TNA Evolution Statesl": ["Does the scene transition from warm afternoon sunlight to rain enveloping the entire scene over time? To judge this question, it must first be determined that the scene exhibits warm afternoon sunlight at a certain time and subsequently exhibits rain enveloping the entire scene, with clear transition process over time." "]
2. Example2:
Input:
TNA Element Information: [ "Scene Type": "balcony", "Main Object Category": ["potted plant", "watering can", "mn"", "Initial Scene Attributes": null, "Main Object Layout": "a potted
plant next watering can, and a man stands on the balcony gazing into the distance". "TNA Evolution States": ["the man stands on the balcony gazing into the distance", "the man waters the potted plant", "the man retreats back into the house"]
25

---

Published as a conference paper at ICLR 2026
TNA Change Reason: Object Action Changes
-NNA Count:
Output: "Scene Type": ("Does the scene take place balcony?"), "Main Object Category":
("Does the scene feature a potted plant?", "Does the scene feature a watering can?", "Does the scene feature man?"", "Initial Scene Attributes": null, "Main Object Layout": ["Is the potted plant positioned next to a watering can?", "Is the man standing on the balcony gazing into the distance?"), "TNA Evolution States0": ["Does the video contain any segments showing the man standing on the balcony and gazing into the distance?", "Does the video contain any segments showing the man watering the potted plant?", "Does the video contain any segments of the man retreating back into the house?"], "TNA Evolution States ["Does the man's is action transition from standing on the balcony gazing into the distance watering the potted plant over time? To judge this question, it must first be determined that the man is standing the balcony gazing into the distance at a certain time and subsequently waters the potted plant, with a clear transition process over time.", "Does the man's action transition from watering the potted plant to retreating back into the house over time? To judge this question, it must first be determined that the man is watering the potted plant at certain time and subsequently retreats back into the house, with clear transition process over time.")
Based on the above prompt. please assist me in extracting the structured text for the following input. Note: Only output the final structured text without additional explanations. Input: (User-provided information }
C MORE DETAILS ON OUR EVALUATED MODELS
In this section, we will provide further implementation details regarding evaluated models and visualize some evaluation results.
C.I ADDITIONAL INTRODUCTION TO THE EVALUATED MODELS
We present the video duration, frame rate, and resolution information of the evaluated models in Tab. A5, with all data obtained based on the configuration of the official code. Due to the robust gencral-purpose generative capacity of foundation video generation models, most methodological improvements in video generation are built upon these pretrained models (Chen et al., 2025b;a; Wu et al., 2026; 2025a). Long video generation models follow the same paradigm. For TALC (Bansal et al., 2024a), is implemented based on the foundation model ModelScopeTT2V (Wang et al., 2023). For FreeLong (Lu et al., 2024), FreeNoise (Qiu et al., 2024), and FIFO-Diffusion (Kim et al., 2024), we adopted the official implementation based on VideoCraft2 (Chen et al., 2024a). For RIFLEx (Zhao et al., 2025), opted for twofold duration extension approach based on CogVideoX-SB (Yang et al., 2024b).
Recent advancements in long-vides generation and multi-shot generation models have demonstrated substantial potential for producing rich narrative content. For example, to enable infinite-framt video generation, Skyreel-v2 (Chen et al., 2025c) introduces novel diffusion model architecture and an effective multi-stage training scheme. Similarly, MAGI-1 (Teng et al., 2025) achieves chunk-level autoregressive generation through sophisticated noise intensity design, facilitating flexible streaming long video synthesis. To enrich narrative content in generated long videos and support multi-shot scenarios, Captain Cinema (Xiao al., 2025) proposes an innovative top-down keyframe planning and bottom-up video synthesis mechanism, enabling the generation of short movies. MovieDreames (Zhao et al., 2024) adopts a similar hierarchical framework, leveraging the advantages of autoregressive and diffusion models to produce highly consistent keyframes, which are then extended to high-quality long videos. In contrast, VGOT (Zheng et al., 2024a) and MovieAgent (Wu et al., 2025b) utilize existing powerful content creation tools to construct intuitive and effective multi-agent frameworks. Specifically, VGOT employs a structured, step-by-step framework to address three core challenges: narrative fragmentation, visual inconsistency, and transition artifacts. MovieAgent develops an effective automatic film generation method based on a script and character bank by implementing systematic multi-agent thain-of-thought planning.
26

---

Published as a conference paper at ICLR 2026
We attempt include these models part of our evaluation. However, because Captain Cinema (Xiao et al., 2025) and MovieDreamer (Zhao et al., 2024) are not open-source, we are unable to perform quantitative assessments them. In addition, MovieAgent (Wu al., 2025b) requires specially constructed character bank as input, which is incompatible with the input format defined in our benchmark, rendering it unsuitable for evaluation within our framework. Taking these factors into account, we evaluate Skyreel-v2 (Chen et al., 2025c), MAGI-1 (Teng et al., 2025), and VGOT (Zheng al., 2024a), and report the results in Tab. A7.
Furthermore, we analyze the computational efficiency of several representative foundation and long video generation models on an H20 GPU. Specifically. we evaluate the number of parameters in their key denoising networks (Params). the computational cost per forward operation (FLOPs), the time required for each forward operation (T), and the total number of forward steps needed for complete generation process (Steps). The results in Tab. A6 indicate that recent foundation video generators, such as Wan2.1-14B and Hunyuan Video, possess extremely large parameter counts and correspondingly high computational costs. Additionally, current long-vides models-inclludin all built upon the same early foundation model (VideoCraft), resulting in identical parameter counts. However, their porwad FLOPs differ due to each model employing a distinct strategy for long-video feature modeling.
FreeLong, FreeNoise, and FIFO-Diffusion
Due to the substantial computational costs involved, existing long video generation models generally face challenges in generating significantly longer videos, and are typically limited to producing videos approximately to 3 times the duration of their foundation counterparts. Unlike conventional video generation approaches, FIFO-Diffusion employs a unique denoising mechanism that enables the recursive generation of longer videos without significant increase in computational cost. We extended its official default setting from 10 seconds to 60 seconds to analyze our benchmark's 's evaluation capability for minute-long videos. Tab. A3 shows that increasing the video duration led to an improvement in the model's 's narrative capability (the mean score increased from 0.57 to 0.59). We speculate that this mainly because longer videos provide more space for content creation, thereby enabling the model to express narratives more effectively.
Table A3: Performance of FIFO-Diffusion on 10-second and secsen video generation.
Rad
Resr
Reah
Model
Mean
8au
fanl
fact
fau.
fas
Ball
fun
fact
FIFO-Diffus (10s) 0.75 0.74 0.79 0.59 0.61 0.53 0.39 0.41 0.35 0.57 0.59
FIFO-Diffusion (60s) 0.75 0.73 0.78 0.61 0.65 0.58 0.41 0.43 0.41
C.2 VISUALIZATION OF EVALUATION RESULTS
To intuitively understand the narrative expression capability of the model, we present the video generation outcomes corresponding to prompts under different TNA counts and change factors, as shown in Fig. A6, Fig. A7 and Fig. A8. Intuitively, the increase in video length brings more challenges to the model (Feng et al., 2024b; 2025b;c), highlighting that there remains substantial room for improvement in the generative capabilities of existing long-video generation models.
Table A4:
Analysis of answer consistency across different questions. Consist-n/3 denotes the subset with consistent answers out of three annotations.
Metric
Consist-1/ 1/3
Consist-2/3
Consist-3/3
Rua (Q1)
81
361
158
Reov (2)
69
305
226
Ranh (Q3)
73
309
218
D MORE DETAILS ON THE EXPERIMENTS
In this section, we provide additional implementation details regarding our experiments.
27

---

Published as a conference paper at ICLR 2026
Videot:
Video::
Prompt:
ielotinshll sipedto within soiea, soiich demoneirrstee butter gentration
Inttal
helrtth
order foot srowering the folowing qusdione, the mone elensent
videed
Videot
Game
OZ: cuastion frcusss antth narratio invelvedd the
Information comsuonncing this proipt en folow::
Pheas andlyo genarstect greater nambbrr
Soace Tyme Seiling
Tagaat Cangrog
Sea.
Vileoo
Viduot
Same
D3: This cunaic bocea di thi allon botwoon bealc mumatiiccaat
redkedd bejproppho "haaa anddyur wiicc garaeskee eidoo codorroeass groater number foon basusen boan camntiva
perzze dece
Videod
Samn
Figure A4: Interface for human preference annotation. From left to right, the interface includes pair of videos be compared, an evaluation prompt with corresponding structured element information, as well as three multiple-choice questions to be answered.
Table A5: Information on duration, frame Rate, and resolution of videos generated by our evaluation models.
Model
Duration
Frame Rate
Resolution
Wan (Wang et al., 2025)
5
16 FPS
1280 x 720
Hunyuan Video (Kong et al., 2024)
5
24 FPS
1280 x 720
Cog VideoX (Yang et al., 2024b)
5
16 FPS
1360 x 768
Open-Sora (Zheng al., 2024b)
5
24 FPS
336 192
Open-Sora-Plan (Lin et al., 2024)
5
18 FPS
640 x 352
RIFLEx (Zhao al., 2025)
12s
8 FPS
720 480
FreeLong (Lu et al., 2024)
12s
10 FPS
512 x 320
FreeNoise (Qiu al., 2024)
6s
10 FPS
512 320
DiffuDifusion (Kim et al., 2024)
10s
10 FPS
512 x 320
2n s, if n < 5 8 otherwise
TALC (Bansal al., 2024b)
FPS
256 256
D.1 ANALYSIS OF METRIC ALIGNMENT WITH HUMANS
As introduced Sec. 4.1, we conduct human preference annotations, which lay the foundation for subsequent analysis of the alignment between our metric and human perception. The human annotation interface is shown in Fig. A4. For each video pair, we provide the corresponding text prompt description. Additionally, to facilitate annotation, we also provide annotators with the structured information extracted from the evaluation prompts (see App. B.2). Based on this information, annotators required complete three judgment questions sequentially, which directly correspond to our three evaluation dimensions.
Our annotations were conducted by trained graduate-level researchers (Master's and PhD students) with backgrounds in computer vision. Each video pair was independently annotated by three an- notators, and the process was supervised by two experts in video generation. For each annotation, annotators reviewed the full video and answered the corresponding questions, which took approxi- mately minutes per video depending on its length and complexity. Annotators were assigned reasonable daily workload, and all annotation tasks were compensated at the standard research assistant rate.
Statistical analysis of the annotation results reveals that some video pairs have situations where three annotators choose three different answers. This means each option is selected maximum of once, and we denote this subset as Consist-1/3. Additionally, three participants selecting the same answers as Consist-2/3 and Consist-3/3. The sample sizes shown in Tab. A4. Due to its poor consistency,
denote subsets with two or
corresponding these three subsets perform experimental analysis on the Consist-1/3 subset. For Consist-2/3 and Consist-3/23 we analyze the alignment between our metric and human preference. As indicated in Tab. 2, for the subset
do not
28

---

Published as a conference paper at ICLR 2026
with higher human consistency (Consist-3/3)), our metric also shows better alignment with human preference.
To intuitively illustrate the effecctiveness of our metrie, present qualitative examples in Fig, A5 showing evaluation results for different video pairs generated from the same prompt. These examples indicate that the differences in metric scores are well aligned with perceptual quality.
IRB review. Previous studies (Geirhos et al., 2021) have demonstrated that experiments solely involving interaction with computer systems (i.e., screen and mouse) pose no risk to participants and therefore do not require IRB approval. Since our experiment follows the same procedure, we did not seek IRB review.
thester set up for grand perfermance, stage is brightly 1it lights dranatic setting, perforeance begins, Lights cetliing.
shadows
Rea 25.7 Rus
Rea 02.7
Row 80.2
V.S.
Ros 68.1
sex'1 autddor hetoyry with portee plants a rory rhate, the shlors brightly clesn hlue sky. time passss, cleuds roll in, cascing the bulcony into glozmy, shadomol Jirsspter
55.3
35.9
53.9
V.S.
Ru 82.8
74.7
70.0
diely 1it tabletop, wolel green poised beside colluction dice model
nóniature
sudienly,
85.3
45.9
50.3
00.6
40.2
V.S.
Rua 27.2
bright Forest under clear blue wtth backpack strolls letsurely. Then, eoment. Shartly ofter, the takes out water froe his backsock.
17.4 Rux 30.3
73.6
Ros 26.6
V.S.
43.5
65.3
trightly stape, female hast sitting chate, spaaking. Then, bast stands up the mixes goodyye audierce. Finally.
stage.
Roy 06.7
Rey 46.7
85.6
Rus Biv1
Rua. 73,5
sunry afterrcon, a little girl stants her hare garden, Subsequently, she walls isto the gardon. Then ahe pisks hardhui or tresh ripwers, Finally, she waits ost or the garcen witt the tlovers.
Bea"
Ruc-
Rise
V.S.
Figure AS: Evaluation prompts and corresponding generated video pairs. The results shown on both sides are calculated by our evaluation metric across three dimensions.
D.2 MORE DETAILS ON THE EVALUATION RESULTS ANALYSIS.
Our evaluation results involve three different dimensions: TNA count n E [1, TNA change factors E [San, lact, fau]. and our metric R E RRa,, Reovy Rech) For each evaluation model, there 6 x 3 x 3 evaluation result data, denoted as A. We present all these evaluation results for foundation video generation models and long video generation models in Fig. A9 and Fig. A10, respectively. Although these results provide detailed display of model performance across various dimensions, they do readily facilitate the derivation of corresponding conelusions. The key observations presented in Sec. 4.2 are synthesized based on these evaluation results. Next, we introduce the specific process of this synthesis:
For observation (i), we focus on the variations of the three metric indicators under different TNA counts. Thus, the results Fig. are obtained by averaging A over the three TNA change factors. Observation (ii) focuses on the TNA expression quantity Nesp- which is constructed based on the Reov indicator and also averaged over the three TNA change factors. Furthermore, the results shown in Fig. 5 are statistically derived for both foundation video generation models and long video generation models. The solid lines represent the medians, and the shaded areas are determined by the 5th and 95th percentiles. Observation (iii) focuses the variation in the three metric indicators for VidcoCraft-base models (Chen et al., 2024a; Kim et al., 2024; Lu et al., 2024; Qiu et al., 2024) under different TNA counts. The caleulation method in Fig. 6 is the same as that used in observation (i).
29

---

Published as a conference paper at ICLR 2026
Thk-1:
gently rloating background.
Than2: hill, vegetation thrives ware sunlight. Suddenly, begins to pour down, enveloping the entire scene.
TMa+3: In
helicopter
surrounded
greeney. Suddenly, downpour occurs, envelaping scene. Subsequently, the snow, Dtanketing the forest and grassland with thick layer of white.
TM-4; In barren with rugged, rocky walls and dry, dusty ground. Suddenly, sandstorn approaches, passes,
swirling begins to rain, and the ground darkens as it absorbs the water. Finelly, rain stops and the sun breaks through the clouds, lesving the canyon brighter with glistening vet rocks.
reducing visibility
TMee:
sun, Susdenly, clouds gather cast over the landsrape, creating cooler, din atmosphere. Then, sandstorm sweeps through, reducing visibility and covering the ground with thin layer of sand, After the storr passes, rare, gentle rain begins to fall, brincing refreshing wetness to the scene, Finally, as the vibrant orange purple, casting tong
painting
transforeed
Thee:: Beside stretch
flowing tall and bushes under scene transitions
bright sun, passes, sunlight gradually dininishes,
warn golden slow spreading across the area, Then, night falls, casting deepening blue twi light Soon, the sky becomes completely dark, and the entire scare is il illurinsted by the soft silver light of the moon. As Finally,
appreaches, bringing vibrant
brushed delicate grassy
fully,
Figure A6: Evaluation prompts and corresponding generated videos under varying TNA numbers (1 to 6) induced by scene attribute change factors. The viewing order of video frames from left to right, top to bottom.
30

---

Published as a conference paper at ICLR 2026
witland, green paln stands by scene progresses, the pala trees' leaves change from green brown.
mArd: yellowing leaf flosts the surface of the river. Gradually, the yellow leaf changes Finally, green leaf
TMad: A brown rushroos is growing in comp acadow, Grasually, the color of these rushroces changes froz browt to rod. Then, the cotor of the rush roces changes from red to purple. Finally, the aushroons turn blue.
Initially,
tangaroo's readish-braae. Suddenlyy ly, changes readish-brow brown. next, the color transitions froe dart brown to gray. Then, the gray fur shifts white. Fisally, the fur charges froe white to black.
TMAr6: At lively beach party, person is danc ing gleafally near the shoreline. Initially, the person's face
is adorned with bright, cheerfut repressonn Suddenly, confuston. contempl latiaationn that, face changes to deternination. Fimilly, expression into ane of sheer joy.
Figure A7: Evaluation prompts and corresponding generated videos under varying TNA numbers (1 to 6) induced by target attribute change factors. The viewing order of video frames is from left to right, top bottom.
31

---

Published as a conference paper at ICLR 2026
TMA-1:
Lsnd tall
TMA+2: In the tranzuil ocean brimming with corat, sea turtle glides slowly above the coral. subsequent by, the turtle piece of coral.
clear strells teisurely. Then, TMM-3:
TWked: In a dark, narrm alley, a man with flasst ighht walks slovly forward. Suddenly, stops and looks around cautiously, ly. Then, the nan bends down to pisk up a stone from the ground. Finally, he stands up and cont inues walking forward.
TMA+5:
Juncla, surrounded by green leaves,
banana trees as nonkey nimbly
wcaves betveen the branches. Then, the monkey stoos and sits on branch of a rubber tree. text,, it picks banara fron the tree. After that, the rankey puels the banana and begins to eat. Finally, the cankey tasses may the banana peet and continues to Leep betveen the trees.
rurst. landscape rolling green proup peop stands circle, chatzing haspily. Suddenly, they
thgether nearby farahouse. Then, group stops front the faratouse to adntre view. that, one person waves their arss gather everyane's 's attent inn. Next, the group sits down on the grass for leisurely pienie. Finally, the group stanes up and starts playing gane or catch.
Figure A8: Evaluation prompts and corresponding generated videos under varying TNA numbers (1 to 6) induced by target action change factors. The viewing order of video frames is from left to right, top to bottom.
32

---

Published as a conference paper at ICLR 2026
(a) Evaluation results of foundation video generation models regarding the attrbuute factor.
2.00
Ain
open 201o
0.60
3.60
0.0
0.50
0.73
RCOM
0.7
0.70
0.6
0.45
0.40
3.60
0.50
0.50
TMA Bumbe's
foundation video genaration ragarding attribute change factor.
0.90
0.9
Capuitace
2.00
Caplicecd
2.50
0.65
2.59
3.50
0.10
0.45
0.60
3.40
0.5
0.30
0.0
3.90
3.50
0.25
TMA Numbers
THA Numbers
TIA Numbers
(c) Evaluation results of the foundation video generation models regarding objact change factor.
3.50
Ain
1pan with
0,7
Win
Open Stno
Con
0.8
3.00
0.5
35.75
Rone
0.6
0.4
0.60
0.59
0.5
Figure A9: Evaluation results across three evaluation dimensions and three TNA change factors. The evaluated models comprise mainstream foundation video generation models (Wang et al., 2025; Yang et al., 2024b; Zheng al., 2024b; Lin et al., 2024; Kong et al., 2024).
Observation (iv) focuses on the variation in the three metric indicators under different TNA change factors. Therefore, the results in Tab. 1 are obtained by averaging A over the six TNA change ranges.
Table A6: Computational efficiency of different video generation models. Params, FLOPs, T, and Steps denote the number of parameters, computational cost per forward operation, time per forward operation, and total number of forward steps per generation, respectively.
Model
Params (G)
FLOPs (T)
T (s)
Steps
Wan2.1-14B (Wang et al., 2025)
14.3
904.9
111.6
50
Hunyuan Video (Kong et al., 2024)
12.8
351.6
131.8
50
Open-Sora-Plan (Lin et al., 2024)
2.8
100.8
3.5
100
FreeL.ong (Luu al., 2024) FreeNoise (Qiu et al., 2024)
1.4
51.9
13.7
1.4
23.5
6.6
50
FIFO-Diffusion (Kim et al., 2024)
1.4
5.9
1.1
800
D.3 IMPLEMENTATION DETAILS OF FEATURE- VISUALIZATION ANALYSIS
In addition to evaluating based on the final video generation results, we introduce the inter-frame feature average distance metric D Sec. 4.3, which facilitates analysis at the intermediate feature level. Specifically, for given diffusion- based video generation model, we select the video latent N, space features Z {z:) 1-1 the last denoising timestep, where Ny denotes the number of video
33

---

Published as a conference paper at ICLR 2026
Table A7: Performance of latest video generation models.
Rad
Rçcv
Righ
Model
Lan
lact
Sall
Lin
Luct
Lan
last
SkyReels-V2 V2
72.7 69.0
75.0 65.5
68.9 59.9
46.9
51.5
43.1
MAGI
72.5
73,4
77.9
58.7
58.6
53.6
39.5
39.3
39.3
VGOT
83.0
79.1
79.0
70.1
72.0
52.0
51.5
56.2
35.2
Evaluation the long generation regarding the change factor.
0.8
og
0.7
Freduun
Fraslomy
0.8
0.7
0.5-
0.4-
Bru
Reok
0.2
TMA Kumbers
ThA Namber
TNA Numbers
(b) Evalustion results the long video generation models regarding the otject attribute change factor.
mg
MA
0.7
0.4
Rew
Ronk
0.
3.4
0.2
TMM Kumbers
Numbers
Number
(c) Evalustion results of the long video generation models regarding the object action change factor.
0.9
Preditido
Rew
0.5
0.6
0.5
3.4
0.4
0.2
Figure A10: Evaluation results across three evaluation dimensions and three TNA change factors. The evaluated models comprise mainstream long video generation models (Kim et al., 2024; Qiu et al., 2024; Lu al., 2024; Zhao et al., 2025; Bansal al., 2024a).
frames. Then, Dy obtained through the following operation:
N EM Eラ (z: )²)²
Dy
(AI)
(N₂)
This metric represents the average inter-frame feature distance for each video. For the results shown in Fig. 8, we select 15 prompts under each TNA for evaluation to ensure the reliability of the assessment outcomes. The solid lines represent the means, and the shaded areas are determined by the 30th and 70th percentiles.
E LIMITATIONS AND BROADER IMPACT
In this work, we propose a novel benchmark, NarLVV, which aims to comprehensively assess the narrative expressiveness of long video generation models. Currently, our evaluation primarily focuses
34

---

Published as a conference paper at ICLR 2026
open-source text-to-video models, which represent the fundamental task setting in the video generation domain. In the future, we intend to continually expand the scope of our evaluation models to include image-to-vide to-idooo models and cutting-edge open-source models. It is worth noting that, utilizing our established evaluation platform, we can directly test these models without requiring complex additional design.
Our NarLLV effectively reveals the narrative expressiveness of video generation models. Similar to many technologies centering around generative models, this work carries potential societal implica- tions that warrant careful consideration (Katirai et al., 2024; Chen, 2023). Specifically, the models we assess with stronger narrative expression capabilities might facilitate the creation of deceptive or harmful video content. However, as advancements in video generation safety and regulatory technologies continue (He Fang, 2024; Wang et al., 2024e; Dai et al., 2024), we believe these negative impacts will be progressively mitigated.
F
USAGE OF LARGE LANGUAGE MODELS
Consistent with recent benchmarks (Zheng et al., 2025; Huang et al., 2024b; Wang et al., 2024a) in the field of video generation, we explore the integration of large language models (LLMs) into the design of benchmarks to enable automated evaluation. Specifically, existing LLMs are utilized as tools both the construction of prompt generation pipelines and the implementation of MLLM-based question answering metrics. Detailed configurations of the employed LLMs provided in the main text (please refer to Sec. 4. Moreover, we have employed GPT-4o to assist with the language polishing of this manuscript during its preparation.
35


Reviews:
[{"review_id": "7H9l2nlmmY", "reviewer": "Reviewer_gGXr", "summary": "The paper indicates that current video generation models have advanced in producing long-duration clips, while their narrative capabilities have not been systematically evaluated by recent benchmarks. This paper proposes NarrLV, the first benchmark for long-form video narrative, which features dedicated metrics including element fidelity, unit coverage, and unit coherence. NarrLV is also presented as an end-to-end framework capable of automatically generating prompts and conducting evaluations. Comprehensive comparisons across 11 SOTA models through NarrLV reveal insights into the relation between prompt semantics and narrative units.", "questions": "Following up on the previously mentioned weakness, could the authors provide an illustration of the thematic distribution of the testing prompts? A visualization, such as pie chart, would be helpful.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper serves a complementary role to VBench. As mentioned by the authors, the VBench series focuses on quality metrics such as controllability, commonsense, and physical plausibility, but notably does not cover narrative metrics.\n- The definition of TNA is novel and the proposed metrics are both reasonable and appropriate. Inspired by film narrative theory, the authors define TNA as the basic elements of video content. Building upon TNA, the three metrics quantitatively measure diversity, fidelity, and coherence. This contrasts with previous solutions, which involved simply throwing frames into VLMs and directly asking them about these abstract concepts. Therefore, NarrLV provides a much more convincing evaluation framework.", "weaknesses": "- This paper does not evaluate more advanced, closed-source models, such as Sora2 and Kling. However, given the significant cost and access constraints associated with these models, this omission is understandable and acceptable for an academic research paper\n- While Section 3.2 describes the prompt generation pipeline in detail, it lacks a sufficient analysis of the resulting prompts. For example, it is recommended that the authors conduct a statistical analysis of the theme categories within the generated prompts. This would help researchers interpret the evaluation results more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iyesOYGr3i", "reviewer": "Reviewer_ixiX", "summary": "This paper aims to address the lack of adequate evaluation methods in the field of long video generation, specifically for assessing a model's ability to generate narrative content. The authors propose a new benchmark named NarrLV, the core idea of which is to draw from film theory to define the Temporal Narrative Atom to quantify the richness of narrative content. Authers further introduce an automated prompt generation pipeline and a question-answering evaluation system. The proposed metric encompasses three evaluative dimensions: narrative element fidelity, narrative unit coverage, and narrative unit coherence. The authors claim through experiments that this metric demonstrates high consistency with human judgment.", "questions": "Please see weaknesses", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The problem of evaluating the narrative capabilities of long videos is an important and unresolved challenge in the current field.\n- The authors' attempt to construct a systematic, automated evaluation framework is a direction worth exploring.\n- The experimental results show a strong correlation with human preferences, which increases the benchmark's credibility as a proxy for human evaluation.", "weaknesses": "**1. Significant Omission of Long Generation Method**\n\nFirst, I question the rationale for evaluating base models like WAN. on a \"long video\" benchmark. These models are fundamentally designed to generate short video clips. Evaluating them on tasks far outside their intended design (i.e., long video, which I would argue implies a duration of several minutes) does not seem to yield meaningful insights and may be an unfair comparison.\n\nMy primary criticism is the benchmark's almost exclusive focus on training-free methods. This has led to a very large oversight: the paper almost completely ignores the entire category of multi-shot long video generation. This area represents the true state-of-the-art in generating coherent, long-form narratives and sequences.\n\nTo be a comprehensive and valuable benchmark for the community, it is essential to include, evaluate, or at the very least, thoroughly discuss these more advanced methods. I strongly recommend the authors incorporate the following highly relevant works, which are central to the long video problem:\n\n- Captain Cinema: Towards Short Movie Generation\n- Skyreels-v2: Infinite-length film generative model\n- Moviedreamer: Hierarchical generation for coherent long visual sequence\n- MovieAgent: Automated Movie Generation via Multi-Agent CoT Planning\n- VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention\n\nEven for methods that are not open-source, a benchmark paper has a responsibility to cite and situate them within the landscape. Without addressing these hierarchical, agent-based, and multi-shot approaches, the paper's current contribution as a long video benchmark feels incomplete and overlooks the most relevant research.\n\n**2. Concerns on TNA**\n\nTNA simplifies \"narrative\" into a series of discrete physical state changes (attributes, actions). This ignores higher-level narrative elements like causal logic, character motivation, emotional development, or cinematic language. Therefore, NarrLV evaluates something more akin to \"sequential instruction following\" rather than true \"storytelling.\"\n\nThe upper limit for TNA in the experiments is only 6. While this reveals the bottlenecks of current models, it is far from the ultimate goal of long video. The benchmark's effectiveness in handling more complex, longer sequences (e.g., more than 10 TNAs) has not been validated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hpQcy9RSdk", "reviewer": "Reviewer_Ev65", "summary": "This work proposed a novel benchmark, NarrLV, to comprehensively evaluate the narrative expression capabilities of long video generation models. The evaluation is centered on elements inspired by film narrative theory, and an evaluation metric based on an MLLM-based QA framework. Experimental results identify the strengths and limitations of existing video generation models, where the insights can fuel future research.", "questions": "- See Weakness 1\n- In line 242, why does the paper evaluate the models under 20x6x3 = 360 prompts? It is unclear to me what the 360 prompts actually cover. \n- Please elaborate on the alignment between the human preference annotations and the MLLM-based classification metrics. Specifically, since human annotation is based on subjective pairwise preference and the MLLM determines objective fulfillment of conditions (fidelity, coverage, coherence), how can we confidently derive that the metric is aligned with human annotation?\n- Please provide more information about the annotator. Specifically, their background, workload, and duration spent on each annotation task. Did the annotator get paid for the annotation task?\n\nSuggestions:\n- For Figure 2, it would be more intuitive to replace (b) Evaluated Models with a video generation step. I.e., given the (generated) prompt. A video generation model will generate a corresponding video, which will be used in the next stage (evaluation). The current listed models are specific to this work, and I believe the figure can be more general.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Inspired by film narrative theory, the novel benchmark, NArrLV, defines the smallest narrative unit as Temporal Narrative Atom (TNA), which is a quantitative measure of narrative richness in generated video. It further identifies three key dimensions, i.e., scene attribute, target attribute, and target action, that influence the TNA. NarrLV contains a prompt suite, which can flexibly generate prompts with a desired TNA number.\n- For evaluation, this work follows a progressive narrative expression paradigm that focuses on narrative element fidelity, narrative unit coverage, and narrative unit coherence, and proposes an MLLM-based QA framework and the ground truth TNAs.\n- Evaluation makes several observations of the existing video generation models. The quantitative findings allow researchers to make informed decisions in model improvement in future work.", "weaknesses": "- W1: I acknowledge that the narrative prompt suite is highly valuable. In lines 190-192, it describes the factors that influence the number of TNA is similar to TC-Bench. Can the author articulate the differentiating factor between TC-Bench and NarrLV? Specifically, I would like to understand what the novelty of NarrLV is that enables it to extend to prompts with a higher number of TNA counts.\n- W2: I acknowledge that the proposed benchmark suite and evaluation focus on the narrative expressiveness of the video generation models. However, it would be more comprehensive to understand other related metrics such as hallucinations, content coherence, and image quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XkdgVkDrsd", "reviewer": "Reviewer_5Gfb", "summary": "This paper introduces NarrLV, a benchmark aimed at evaluating the narrative expressiveness of long video generation models. Drawing from film narratology, it defines a new unit called the Temporal Narrative Atom (TNA) to quantify narrative richness. The benchmark consists of:\n\n* An LLM-generated prompt suite with variable TNA counts to control narrative complexity.\n\n* An MLLM-based question–answering metric that evaluates three aspects: narrative element fidelity, narrative unit coverage, and narrative unit coherence.\n\nExperiments benchmark several recent video generation models (e.g., Wan, HunyuanVideo, FreeLong, FIFO-Diffusion) and report that NarrLV correlates well with human judgments.", "questions": "* Since all prompts are LLM-generated, how do the authors ensure they reflect real-world user prompts? Why not use or compare with real datasets like VidProM?\n\n* The appendix lacks model details. Can the authors provide inference settings (duration, FPS, resolution) and confirm all models were tested under equal conditions?\n\n* How stable are the MLLM-based metrics across different question templates or MLLMs? Any variance or seed analysis to support reliability claims?\n\n* Can the authors show prompt–video examples or visual comparisons to verify that the metrics align with perceptual quality?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* Multi-dimensional metric: The three-level evaluation (fidelity, coverage, coherence) provides a more nuanced understanding of narrative quality.\n\n* Quantitative validation: The authors include human–metric alignment tests showing reasonable consistency, supporting reliability.\n\n* Potential for extensibility: The benchmark can, in principle, scale with longer prompts or future models.", "weaknesses": "* Synthetic prompts: All prompts are LLM-generated, which limits real-world representativeness and linguistic diversity. Real datasets such as VidProM (Wang & Yang, NeurIPS 2024) already offer large-scale, human-written long video prompts but are neither cited nor used.\n\n* Missing qualitative examples: The paper lacks visual examples of generated videos as supplementary material, reducing interpretability and reader engagement.\n\n* Limited practical insight: The analysis mainly confirms intuitive findings (e.g., models struggle with multi-event coherence) without providing new conceptual insights or design implications for improving generation models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)