CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models

## Abstract

Abstract Despite significant advances in video synthesis, research into multi‑shot video generation remains in its infancy.
Even with scaled-up models and massive datasets, the shot transition capabilities remain rudimentary and unstable, largely confining generated videos to single-shot sequences.
In this work, we introduce CineTrans , a novel framework for generating coherent multi‑shot videos with cinematic, film‑style transitions.
To facilitate insights into the film editing style, we construct a multi-shot video-text dataset Cine250K with detailed shot annotations.
Furthermore, our analysis of existing video diffusion models uncovers a correspondence between attention maps in the diffusion model and shot boundaries, which we leverage to design a mask‑based control mechanism that enables transitions at arbitrary positions and transfers effectively in a training-free setting.
After fine-tuning on our dataset with the mask mechanism, CineTrans produces cinematic multi-shot sequences while adhering to the film editing style, avoiding unstable transitions or naive concatenations.
Finally, we propose specialized evaluation metrics for transition control, temporal consistency and overall quality, and demonstrate through extensive experiments that CineTrans significantly outperforms existing baselines across all criteria.

## 1 Introduction

Endowed with extensive pre-training and sophisticated architectures, diffusion models (Blattmann et al., 2023; Ho et al., 2020; Rombach et al., 2022; Song et al., b) have demonstrated promising capabilities in generating videos with high visual quality and strong consistency comparable to real videos (Wan et al., 2025; Kong et al., 2024; Yang et al., 2024; Li et al., 2024). Particularly in the domain of text-to-video generation (T2V) (Wang et al., 2024; Yang et al., 2024; Li et al., 2023; Cho et al., 2024), the breakthrough has attracted considerable attention, highlighting the potential of diffusion models in mastering video creation (Wang et al., 2024; Xu et al., 2024; Chen et al., 2023a; 2024a). However, generating multi-shot videos with cinematic editing style and movie-like transitions from a brief user input continues to be a significant challenge.

Existing work on long video generation can be broadly categorized into two aspects. First, larger models trained on massive datasets advance the capability of video interpretation and generation, enhancing the visual fidelity and maximum video length. (Kong et al., 2024; Wan et al., 2025; Yang et al., 2024).
Second, techniques such as conditioning improve consistency across concatenated samples, enabling longer outputs through clip stitching (Zheng et al., 2024; He et al., 2023; Xie et al., 2024; Zhao et al., 2024). While both approaches partially support multi-shot sequence generation, they still exhibit notable limitations.
Large-scale models primarily focus on single-shot videos due to the scarcity of shot transitions in their training datasets (Wang et al., 2023b; Ju et al., 2025; Wang et al., 2023a).
Cinematic transitions are not guaranteed, let alone at precisely controlled positions. Moreover, the high computational cost and extended training time undermine the efficiency of this approach.
Meanwhile, generating individual shots separately and concatenating them requires substantial manual intervention, and ignores prior knowledge from cinematic multi-shot dataset, resulting in cuts that often misalign with real-world editing styles.
Additionally, many recent works often target narrow contexts, such as facial consistency (Zheng et al., 2024) or specific animated series (Dalal et al., 2025), which constrains their applicability to general video generation.

While the aforementioned methods achieve certain multi-shot effects, very few explicitly focus on cinematic transitions within diffusion models. In this work, we propose CineTrans, a framework that produces multi-shot videos with cinematic transitions as shown in Figure [1](https://arxiv.org/html/2508.11484v1#S0.F1).
To support this, we develop a refined data processing pipeline that processes raw footage into a 250K video-text dataset. Our split-stitch procedure groups semantically related clips and removes gradual transitions, and we employ transition-aware models to annotate hierarchical captions. The resulting Cine250K provides frame-level shot labels and temporally-dense captions, preserving film editing style and making it well-suited for multi-shot video generation, while proving effective in enabling more natural shot transitions and stronger inter-shot consistency.

To analyze how diffusion-based models handle cinematic multi-shot sequences, we dive deep into the attention patterns, examining attention maps across the temporal dimension.
We find that the attention maps have strong impact on the intra-shot and inter-shot frames.
This insight clarifies the underlying mechanism of shot transitions in diffusion models. Building on this, we introduce a mask mechanism featuring strong correlations within shots and weak correlations between shots, achieving controlled cinematic transitions and enabling zero-shot multi-shot generation.

As shown in Figure [2](https://arxiv.org/html/2508.11484v1#S1.F2), our proposed CineTrans framework functions through two key aspects.
First, through an analysis of the attention maps, we prove that the mask mechanism aligns with the diffusion model’s inherent understanding of cinematic multi-shot sequences.
The application of mask enables strong intra-shot frame correlations in attention module, facilitating precise frame-level cinematic transitions, which remains effective even in a training-free setting.
Second, the constructed Cine250K encapsulates prior knowledge of film editing. Fine-tuning on this dataset equips CineTrans with the ability to generate cinematic transitions that conform to this style, rather than simply concatenating semantically similar clips. Consequently, CineTrans is able to producing cinematic multi-shot videos aligned with film-editing conventions.

We evaluate the model on a series of prompts with specified transitions and conduct a comprehensive analysis using multiple metrics from different perspectives.
Experimental results demonstrate that CineTrans achieves finer shot transition control and stronger consistency compared to other multi‑shot generation methods, without compromising overall quality.

Our contributions are summarized as follows:

- •
We developed a dataset of 250K video-text pairs, complete with frame-level shot labels and hierarchical annotations, which facilitates video diffusion models for generating cinematic transitions and consistency between shots.
- •
We analyze attention maps in diffusion models for multi-shot video generation and observe a strong connection between attention probabilities and shot transitions. Building on this insight, we introduce a mask mechanism that enables cinematic transitions within diffusion models, leading to the CineTrans framework, which is effective even in a training-free setting.
- •
We propose a series of comprehensive metrics tailored for cinematic multi-shot video generation and evaluate CineTrans, demonstrating its ability to control cinematic transitions, enhance temporal consistency, and preserve overall quality.

Figure: Figure 2: Overview of CineTrans. Existing video generation models focus primarily on single-shot video. The multi-shot video generation cases often follow several failures, remaining unstable and uncontrollable. Observations of these multi-shot cases reveal a structured pattern in attention layers. Based on this insight, we introduce a mask mechanism and fine-tune the model with our constructed dataset Cine250K, resulting in significantly improved performance.
Refer to caption: x2.png

## 2 Related Work

Diffusion-based Video Generation Diffusion-based approaches, built on the iterative denoising framework of Latent Diffusion Model (Rombach et al., 2022), utilize scaled-up model (Kong et al., 2024; Wan et al., 2025; Yang et al., 2024; Blattmann et al., 2023; Chen et al., 2023a; 2024a; He et al., 2022; Lin et al., 2024; Polyak et al., 2024) and large video datasets (Miech et al., 2019; Bain et al., 2021; Zellers et al., 2021; Xue et al., 2022; Wang et al., 2023a; b; Chen et al., 2024b; Xiong et al., 2025) to generate high-quality, prompt-adherent videos with extended durations.
Due to their strong semantic understanding, certain pretrained models (Kong et al., 2024; Wan et al., 2025) can preliminarily generate multi-shot videos when provided with transition-specified prompts, but at the expense of vast computational resources, long training times, and unreliable, imprecise transitions. In contrast, our work offers frame-level stable cinematic transitions, seamlessly applied to the diffusion-based framework.

Multi-Shot Video Generation.
Recent work has explored multi-shot video generation, which can be categorized into two main approaches.
The first generates each shot separately and then concatenates them, focusing on consistency between the generated shots.
Animate-a-Story (He et al., 2023) uses motion structure retrieval for plot-aligned clips guided by text prompts. (Chen et al., 2023b) proposes a mask-based diffusion model to generate a smooth transition between shots. Zhuang et al. (2024) attempts to utilize the language model to generate prompts for different shots of a video.
DreamFactory (Xie et al., 2024) employs an LLM-based framework with multi-agent collaboration and keyframe iteration design to ensure consistency and seamless transitions. MovieDreamer (Zhao et al., 2024) adopts a hierarchical autoregressive architecture for global coherence, and VGoT (Zheng et al., 2024) uses keyframes and identity-preserving embeddings to enforce temporal and character consistency.
While these consistency-driven approaches are effective to some extent, they do not leverage real multi-shot video datasets and tend to overlook complex relationships between shots, including variations in camera angles and scenes, which are crucial in human-edited videos.
The second approach modifies model architectures to generate multi-shot content directly. Test-Time training (Dalal et al., 2025) adds a layer for long multi-shot generation but is tied to a specific animated series, limiting generalization. Mask^2DiT (Qi et al., 2025) applies shot-wise semantic masking yet focuses on text condition injection and assumes fixed shot durations.
ShotAdapter (Kara et al., 2025) and LCT (Guo et al., 2025) introduce transition tokens and specialized positional encodings, respectively, yet exhibit low consistency or require large-scale training.
In contrast, our method generates cinematic multi-shot videos in a single pass with flexible frame-level control, achieving strong performance even in a training-free setting and demonstrating generalizability across diverse scenarios.

## 3 Problem Formulation

Diffusion models (Ho et al., 2020; Song et al., a) are a class of generative models learning to reverse a diffusion process, which learn to corrupt data via a predefined noisy Gaussian process and then invert that corruption through a trained neural network. Video diffusion models simply apply this same forward-reverse framework across temporal frames, yielding coherent video sequences $\mathbf{F}=\{f_{1},f_{2},\ldots,f_{N}\}$, where each element $f_{t}$ represents a video frame.
A key component in these models is integrating the attention mechanism (Vaswani et al., 2017), which allows latent variables to focus on each other’s relevant information and can be formalized as:

$$ $\mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{softmax}\left(\dfrac{\mathbf{QK}^{T}}{\sqrt{d_{k}}}\right)\mathbf{V},$ (1) $$

where $\mathbf{Q}$, $\mathbf{K}$, $\mathbf{V}$ are the query, key, and value matrices, and $d_{k}$ is the key dimensionality.

Cinematic multi-shot video generation task also follows the framework mentioned above, aiming to generate videos with cinematic transitions that are both aesthetically pleasing and compliant with the text prompts. Additionally, with the introduction of multi-shot, the core challenges include:

- •
Generation of cinematic transitions.
The generated sequence $\mathbf{F}=\{f_{1},f_{2},\ldots,f_{N}\}$ can be divided into $M$ sub-intervals, where each $\mathbf{F}_{m}=\{f_{i_{m}},f_{i_{m}+1},\ldots,f_{i_{m+1}-1}\}$ contains frames from index $i_{m}$ to $i_{m+1}-1$, for $m=1,2,\ldots,M$, $M$ is the specified shot count, and $i_{1}=1$, $i_{M+1}=N+1$. Within sub-intervals, frames maintain visual continuity, while noticeable changes at the boundaries create cinematic transitions.
- •
Consistency within and across shots.
Within a shot, maintaining visual consistency is crucial for continuity. In contrast, inter-shot consistency emphasizes high-level semantic similarity over low-level details, i.e., maintaining consistency across shots despite substantial compositional differences. This means that its evaluation is not confined to specific attributes, such as composition or facial features, but is instead guided by film-editing conventions, ensuring applicability to general scenarios.

## 4 Dataset

Figure: Figure 3: Dataset curation pipeline. The raw video is split into several clips and then selectively stitched based on semantic features. A selection process then chooses high‑quality multi‑shot videos. After initial assembly, gradual changes are removed. Finally, a language model is used to annotate each video with a general caption and each shot with its shot caption, yielding temporally dense annotations.
Refer to caption: x3.png

A video with cinematic transitions integrates multiple clips while preserving consistency.
To capture film editing prior knowledge for multi‑shot sequences, we introduce Cine250K, a dataset for cinematic video generation.
As shown in Figure [3](https://arxiv.org/html/2508.11484v1#S4.F3), starting from 633K richly edited videos from Vimeo(^1^11https://vimeo.com), we design a multi-stage preprocessing pipeline to construct the annotated multi‑shot dataset.

First, the transition points are identified by Pyscenedetect (Castellano, 2024), resulting in fragmented segments. Adjacent segments with high similarity, measured by ImageBind (Girdhar et al., 2023) features, are then stitched together according to predefined rules to assemble an initial collection of 16M clips containing shot transitions. We filter this collection by aesthetic score, duration, and shot count to form the preliminary video dataset.
Subsequently, since shot transitions can be categorized into instantaneous hard cuts and gradual changes that blur segment boundaries, we apply TransNetV2 (Soucek & Lokoc, 2024) to detect and remove all gradual transition frames which do not clearly belong to any single shot. This yields unambiguous segments with precise shot labels, including exact start and end frame indices.
In the final step, each video receives both a general caption produced by LLaVA-Video-7b-Qwen2 (Zhang et al., 2024) for temporally-dense descriptions of cinematic transitions, and separate shot captions from LLaVA-NeXT (Liu et al., 2024).

Following the pipeline, Cine250K offers high‑aesthetic videos, precise shot labels, and hierarchical captions, thereby supplying rich prior knowledge for cinematic multi‑shot video generation and facilitating the production of videos with authentic film editing style.
In Table [1](https://arxiv.org/html/2508.11484v1#S4.T1), we compare Cine250K with other datasets. As a multi-shot video dataset with detailed shot labels, the high-quality content of Cine250K can significantly facilitate research and exploration in multi-shot video generation.
The details are presented in Appendix [A](https://arxiv.org/html/2508.11484v1#A1) and [B](https://arxiv.org/html/2508.11484v1#A2).

**Table 1: Comparison of Cine250K and other video-text datasets.**
| Dataset | #Videos | Avg video len | Avg text len | Multi-shot | shot label | Aesthetic | Resolution |
| --- | --- | --- | --- | --- | --- | --- | --- |
| InternVid (Wang et al., 2023b) | 234M | 11.7s | 17.6 | ✗ | - | High | 720P |
| LVD-2M (Xiong et al., 2025) | 2M | 20.2s | 88.7 | ✗ | - | Medium | Diverse |
| OpenVid-1M (Nan et al., 2024) | 1M | N/A | N/A | ✗ | - | High | Diverse |
| Panda70M (Chen et al., 2024b) | 70.8M | 8.5s | 13.2 | ✓ | ✗ | Medium | 720P |
| LLaVA-Video-178K (Zhang et al., 2024) | 178K | $\sim$ 40.4s | $\sim$ 300 | ✓ | ✗ | High | Diverse |
| Shot2Story20K (Han et al., 2023) | 20K | 16s | 201.8 | ✓ | ✓ | Medium | 720P |
| Shot2Story134K | 134K | N/A | N/A | ✓ | ✓ | Medium | 720P |
| Cine250K (Ours) | 250K | 10.75s | 148.79 | ✓ | ✓ | High | 720P |

## 5 Methodology

In this section, we introduce our insight and method for the proposed CineTrans. Section [5.1](https://arxiv.org/html/2508.11484v1#S5.SS1) presents our observations in the attention maps of diffusion models, highlighting the differences between intra-shot and inter-shot correlations. Based on this observation, we introduce a block-diagonal mask mechanism in detail in Section [5.2](https://arxiv.org/html/2508.11484v1#S5.SS2) for cinematic multi-shot video generation. In Section [5.3](https://arxiv.org/html/2508.11484v1#S5.SS3), we discuss the implementation of inference.
An overview of the methodology is shown in Figure [4](https://arxiv.org/html/2508.11484v1#S5.F4).

### 5.1 Frame correlation in attention module

Recently, owing to the increased model size, dataset scale, and computational resources, some diffusion models (Kong et al., 2024; Wan et al., 2025) have demonstrated preliminary capabilities in generating multi-shot videos. However, how diffusion models internally model transitions within a video remains unclear and warrants further exploration. We hypothesize that, for correlation between two adjacent frames, there is a significant difference between transition points and non-transition points. The transition point involves a substantial shift, while a non-transition point requires continuity to maintain visual coherence, making them inherently divergent.

In video diffusion models, the denoising process captures temporal correlations through the attention module. Specifically, the video latent representation is flattened into a sequence of tokens.
Tokens from different frames participate in the calculation of the attention maps in specific layers, enabling the pretrained model to generate continuous, high-quality video segments.
As a result, attention maps are a valuable tool for analyzing frame correlations.

We explore and visualize frame-wise attention maps in the context of the multi-shot video generation cases.
As shown in Figure [4(a)](https://arxiv.org/html/2508.11484v1#S5.F4.sf1), it demonstrates strong correlations for intra-shot frames and weak correlations for inter-shot frames.
More specifically, the attention probability matrix exhibits a block-diagonal structure, with each block corresponding to a shot. Figure [4(a)](https://arxiv.org/html/2508.11484v1#S5.F4.sf1) also illustrates the quantitative differences in attention probabilities across various layers of the diffusion models, highlighting the variations between intra-shot and inter-shot correlations.
To further quantify this observation, we compute the average ratio of mean intra-shot to inter-shot attention probabilities (26.68) and assess its correspondence with the ground-truth shot boundaries via Pearson correlation, yielding r=0.71 (p$<$0.01).
This suggests the potential of leveraging the attention maps to guide the generation of multi-shot videos.

Figure: (a) A case of generated multi-shot video within diffusion model.
Refer to caption: x4.png

### 5.2 Mask mechanism

Building on our observation, we introduce a mask mechanism, a simple strategy that operates on the attention probability in text-to-video diffusion models. Specifically, we construct an attention mask $\mathcal{M}$ for the visual tokens in the attention module at specific layers as follows:

$$ $\mathcal{M}_{ij}=\begin{cases}0&\text{if }i,j\in\text{same shot}\\ -\infty&\text{if }i,j\notin\text{same shot}\end{cases}$ (2) $$

The mask matrix is subsequently added to the attention score in Eq. [3](https://arxiv.org/html/2508.11484v1#S5.E3), effectively weakening the correlations across different shots:

$$ $\mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{softmax}\left(\dfrac{\mathbf{QK}^{T}}{\sqrt{d_{k}}}+\mathcal{M}\right)\mathbf{V}.$ (3) $$

As a result, the final attention probabilities form block-diagonal matrices, with cinematic transitions occurring at predefined positions. As shown in Figure [4(b)](https://arxiv.org/html/2508.11484v1#S5.F4.sf2), transition positions are specified in advance, and the mask matrix is constructed accordingly, enabling precise control over the process. With the block-diagonal mask mechanism, diffusion models can generate cinematic multi-shot videos with fine-grained control.

The mask mechanism functions through two aspects.
First, it aligns with the phenomenon observed in Section [5.1](https://arxiv.org/html/2508.11484v1#S5.SS1), conforming to diffusion models’ inherent understanding of cinematic multi-shot sequences.
Second, unmasked layers enable full-frame token attention, allowing each token to attend to others across different shots, thereby establishing high-level semantic consistency.
This facilitates the effective utilization of multi-shot video datasets, thereby generating multi-shot videos that align with film editing.
We present the impact of applying the mask at different layers on the results in Appendix [D.2](https://arxiv.org/html/2508.11484v1#A4.SS2).

### 5.3 Implementations

Visible-First‑Frame Attention. Beyond the block‑diagonal pattern detailed in Section [5.1](https://arxiv.org/html/2508.11484v1#S5.SS1), we additionally observe that, in certain attention layers, all visual tokens correlate strongly with the first temporal latent.
This finding suggests a pronounced reliance on initial video information. To exploit this observation, we introduce a Visible-First‑Frame mechanism within the attention layers, augmenting the mask design from Section [5.2](https://arxiv.org/html/2508.11484v1#S5.SS2) to improve consistency in multi‑shot video generation. Further details are provided in Appendix [C.1](https://arxiv.org/html/2508.11484v1#A3.SS1).

Customization. Building on our mask mechanism, we guide a diffusion model originally designed for single-shot generation to execute user-specified transitions, thus producing genuine multi-shot videos. For instance, by incorporating LoRA (Hu et al., 2022) weights, we achieve zero-shot generation of multi-shot sequences with enhanced consistency and user-defined styles or character attributes, even though those weights are originally trained on single‑shot videos.

Figure: (a) A sunset ocean panorama gradually shifts to a warm close‑up of sunlit clouds. (shot count: 3)
Refer to caption: x6.png

## 6 Experiment

In this section, we present the implementation details, evaluation settings, and results. Section [6.1](https://arxiv.org/html/2508.11484v1#S6.SS1) describes the details of CineTrans and baselines, which are evaluated on a series of metrics designed for the cinematic multi-shot video generation task. The metrics and results are presented in Section [6.2](https://arxiv.org/html/2508.11484v1#S6.SS2). Section [6.3](https://arxiv.org/html/2508.11484v1#S6.SS3) demonstrates that CineTrans performs well due to the components we propose.

**Table 2: Quantitative results. The best and runner-up are in bold and underlined.**
| Method | Transition Control Score↑ | Inter-shot Consistency | Intra-shot Consistency | Aesthetic Quality↑ | Semantic Consistency↑ |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Semantic | Visual | Subject↑ | Background↑ |  |  |  |  |  |  |
| Score↑ | Gap↓ | Score↑ | Gap↓ |  |  |  |  |  |  |
| StoryDiffusion<br>+CogVideoXI2V | - | 0.5214 | 0.4966 | 0.5660 | 0.3605 | 0.9783 | 0.9713 | 0.6296 | 0.2091 |
| HunyuanVideo<br>+Cinematron | 0.3787 | 0.5631 | 0.3764 | 0.6053 | 0.2855 | 0.9606 | 0.9721 | 0.5978 | 0.2082 |
| HunyuanVideo | 0.2111 | 0.5723 | 0.4075 | 0.5436 | 0.3485 | 0.9476 | 0.9633 | 0.6042 | 0.2064 |
| Wanx2.1-T2V-turbo | 0.2355 | 0.6431 | 0.3002 | 0.6516 | 0.2333 | 0.9332 | 0.9590 | 0.6324 | 0.2046 |
| CogVideoX | 0.0324 | 0.5150 | 0.5915 | 0.6248 | 0.2226 | 0.9310 | 0.9582 | 0.5509 | 0.2061 |
| \rowcolorblue!10<br>CineTrans-Unet (Ours) | 0.8598 | 0.8095 | 0.2444 | 0.7247 | 0.1457 | 0.9598 | 0.9725 | 0.5747 | 0.2224 |
| \rowcolorblue!10<br>CineTrans-DiT (Ours) | 0.7003 | 0.7858 | 0.1552 | 0.7874 | 0.1901 | 0.9673 | 0.9775 | 0.6508 | 0.2109 |

**Table 3: Quantitative results for ablation study. The best are in bold.**
| Method | Transition Control Score↑ | Inter-shot Consistency | Intra-shot Consistency | Aesthetic Quality↑ | Semantic Consistency↑ |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Semantic | Visual | Subject↑ | Background↑ |  |  |  |  |  |  |
| Score↑ | Gap↓ | Score↑ | Gap↓ |  |  |  |  |  |  |
| CineTrans-Unet Ablation |  |  |  |  |  |  |  |  |  |
| w/o Mask, w/o Tuning | 0 | - | - | - | - | 0.9615 | 0.9702 | 0.5901 | 0.2110 |
| w/o Mask, w/ Tuning | 0.2398 | 0.7900 | 0.3279 | 0.7226 | 0.2148 | 0.9582 | 0.9718 | 0.5711 | 0.2077 |
| w/ Mask, w/o Tuning | 0.6168 | 0.7962 | 0.4336 | 0.8186 | 0.3000 | 0.9616 | 0.9719 | 0.5764 | 0.2196 |
| \rowcoloryellow!20<br>CineTrans-Unet | 0.8598 | 0.8095 | 0.2444 | 0.7247 | 0.1457 | 0.9598 | 0.9725 | 0.5747 | 0.2224 |
| CineTrans-DiT Ablation |  |  |  |  |  |  |  |  |  |
| w/o Mask, w/o Tuning | 0.2051 | 0.5924 | 0.3421 | 0.5274 | 0.3574 | 0.9153 | 0.9523 | 0.6322 | 0.2063 |
| w/o Mask, w/ Tuning | 0.2112 | 0.6532 | 0.3422 | 0.6087 | 0.3312 | 0.9213 | 0.9526 | 0.6345 | 0.2019 |
| w/ Mask, w/o Tuning | 0.6564 | 0.7838 | 0.1772 | 0.7844 | 0.1943 | 0.9618 | 0.9746 | 0.6556 | 0.2093 |
| \rowcolorpink!20<br>CineTrans-DiT | 0.7003 | 0.7858 | 0.1552 | 0.7874 | 0.1901 | 0.9673 | 0.9775 | 0.6508 | 0.2109 |

### 6.1 Implementation details

We implement CineTrans-Unet based on LaVie (Wang et al., 2024), and the mask mechanism is applied to the last six layers. We finetune the model on Cine250K using a total batch size of 128 for 20,000 steps. The learning rate is set to $1\times 10^{-4}$. CineTrans‑DiT extends Wan2.1‑T2V‑1.3B (Wan et al., 2025) by integrating a mask mechanism applied to transformer layers 7-28 and is released in two variants. The training‑free variant augments the block‑diagonal mask with the Visible-First‑Frame Attention described in Section [5.3](https://arxiv.org/html/2508.11484v1#S5.SS3), enabling the generation of high‑quality multi‑shot videos.
The second variant applies LoRA fine-tuning (rank=64) on top of the former, using a total batch size of 256 for 2,800 steps. All experiments are conducted on NVIDIA A100 GPUs.
Moreover, we apply LoRA weights to CineTrans-DiT on top of the training-free version to enable model customization, and the results are shown in Figure [6](https://arxiv.org/html/2508.11484v1#S6.F6).

Figure: Figure 6: Training-free results of model customization using LoRA weights.
Refer to caption: x8.png

For baseline comparison, we select three categories: large‑scale T2V diffusion model, multi‑shot model, and customization model. CogVideoX1.5‑5B (Yang et al., 2024), HunyuanVideo Kong et al. (2024), and Wanx2.1‑T2V‑turbo(^2^22https://tongyi.aliyun.com/wanxiang/) leverage large‑scale pretraining and thus possess strong semantic understanding. Each of these models is prompted with a general instruction specifying the desired shot count. StoryDiffusion (Zhou et al., 2024) first produces a sequence of semantically consistent images following both the general prompt and individual shot‑specific prompts, which CogVideoXI2V (Yang et al., 2024) then expands into a video. As a customization model, Cinematron(^3^33https://civitai.com/api/download/models/1494601?type=Model&format=SafeTensor) offers dedicated transition capabilities and employs the same sampling procedure as Hunyuan.

Figure: (a) Convergence of consistency distribution.
Refer to caption: x9.png

### 6.2 Evaluation

To comprehensively evaluate the multi‑shot video generation task, we design 100 prompts spanning four categories using GPT‑4o (Achiam et al., 2023), each incorporating cinematic transitions. For each initial general description, we manually annotate the shot count and then generate captions for each shot, thus constructing a complete prompt series. During evaluation, each method employs its required prompt format (multi‑prompt or single‑prompt) for sampling. To ensure a fair comparison, when using only the general description, the shot count is explicitly included in the text.

Metrics. We propose a set of metrics to evaluate the generated videos from three perspectives: transition control, temporal consistency, and overall video quality.
First, we evaluate transition control by performing shot segmentation (Soucek & Lokoc, 2024), comparing the detected number of shots in the generated video against the prompt’s specified count, and computing the Transition Control Score as detailed in the Appendix [E](https://arxiv.org/html/2508.11484v1#A5).
Second, temporal consistency encompasses both intra-shot and inter-shot consistency. For intra-shot coherence, following VBench (Huang et al., 2024), we treat each shot as a video and compute both subject and background consistency, averaging the results across all shots.
For inter-shot coherence, we evaluate from both semantic and visual perspectives. Semantically, we extract features for each shot using ViCLIP (Wang et al., 2023b) and compute cosine similarities. Visually, following VBench-Long, we compute the similarity between the middle frames of each shot to derive both subject and background similarity scores, which are then averaged to obtain the final inter-shot visual consistency. Furthermore, as discussed in Section [3](https://arxiv.org/html/2508.11484v1#S3), inter-shot consistency prioritizes high-level coherence aligned with film editing conventions, rather than low-level pixel similarity. Therefore, a high inter-shot consistency score, approaching the theoretical maximum, may actually indicate high pixel-level similarity, which violates the prior of multi-shot video design. To address this limitation, we introduce an auxiliary metric: Consistency Gap with Film Editing. For inter-shot consistency metrics, we collect reference score distributions from 1000 randomly selected multi-shot videos that have undergone professional film editing. The Jensen-Shannon Distance between the score distribution of generated videos and that of the reference set is computed as the Consistency Gap. Figure [7(a)](https://arxiv.org/html/2508.11484v1#S6.F7.sf1) illustrates the convergence of the consistency distribution for our selected reference set. This metric complements the raw consistency score by quantifying the deviation from natural film-editing style, thus providing a more comprehensive evaluation of video consistency.
Finally, following VBench, we assess overall video quality using aesthetic quality and overall consistency, which also evaluate the visual and semantic aspects, respectively.

Results. The quantitative comparison is shown in Table [3](https://arxiv.org/html/2508.11484v1#S6.T3).
Even in the training‑free setting, our proposed mask mechanism delivers strong transition control. In terms of consistency, CineTrans achieves high scores and closely aligns with film‑editing style, which is also shown in Figure [7(b)](https://arxiv.org/html/2508.11484v1#S6.F7.sf2). Because aesthetic quality is largely determined by the base model, CineTrans‑Unet performs slightly worse in this regard, whereas CineTrans‑DiT exhibits superior results.
Furthermore, as shown in Table [4](https://arxiv.org/html/2508.11484v1#S6.T4), the user study results demonstrate the superiority of our method in terms of user preference.

For qualitative results, as shown in Figure [5](https://arxiv.org/html/2508.11484v1#S5.F5), we compare the generated results of different methods. Our method demonstrates a remarkably frame-level transition control capability while preserving coherence across different shots. Even without fine‑tuning, CineTrans‑DiT exhibits strong performance, demonstrating the transferability of the framework. In contrast, large‑scale pretrained models fail to adhere to specified shot counts or fully misinterpret the concept of cinematic transitions. Similarly, both customized models and existing multi‑shot models show low temporal consistency.

**Table 4: Results of User Study**
| Model | Transition Control | Consistency |
| --- | --- | --- |
| CineTrans-DiT (ours) | 4.60 $\pm$ 0.50 | 4.15 $\pm$ 0.75 |
| CineTrans-Unet (ours) | 4.75 $\pm$ 0.44 | 4.10 $\pm$ 0.72 |
| StoryDiffusion+CogVideoXI2V | - | 3.95 $\pm$ 0.76 |
| HunyuanVideo+Cinematron | 3.60 $\pm$ 1.95 | 3.80 $\pm$ 0.68 |
| HunyuanVideo | 3.45 $\pm$ 1.88 | 3.50 $\pm$ 0.76 |
| CogVideoX | 2.50 $\pm$ 1.24 | 3.05 $\pm$ 0.60 |
| Wanx2.1-T2V-turbo | 3.25 $\pm$ 1.77 | 3.45 $\pm$ 0.69 |

### 6.3 Ablation Studies

Figure: (a) Effects of Visible-First-Frame Attention.
Refer to caption: x11.png

We conduct ablation studies to assess the impact of the mask mechanism and fine-tuning process. The results shown in Table [2](https://arxiv.org/html/2508.11484v1#S6.T2) demonstrate that our proposed components effectively enhance performance. In terms of inter‑shot visual consistency, the fine‑tuned model yields a lower Consistency Score, reflecting increased compositional variation between shots introduced by training on film‑edited multi‑shot videos. This effect is further corroborated by a reduced Consistency Gap, indicating closer alignment with the film‑editing style. The slight decline in Aesthetic Quality after fine‑tuning may be attributed to aesthetic domain differences between Cine250K and the original training set of base model. Furthermore, Figure [8](https://arxiv.org/html/2508.11484v1#S6.F8) illustrates how fine‑tuning and Visible-First‑Frame Attention enhance video consistency and enable stable generation.

## 7 Conclusion

In this paper, we introduce a novel framework CineTrans for cinematic multi-shot video generation and construct a comprehensive dataset Cine250K with detailed shot annotations.
Through the analysis of attention maps in video diffusion models, we identify a strong connection between attention probabilities and cinematic transitions.
Based on this observation, we propose a novel mask mechanism that enables fine-grained control over cinematic transitions, thus leading to CineTrans framework which transfers successfully in a training-free setting. Extensive experiments validate the effectiveness of CineTrans across multiple evaluation metrics, demonstrating improved transition control, temporal consistency, and overall video quality.
Our work demonstrates the potential of diffusion models for multi-shot video generation, offering a new perspective for directly generating movie-like videos and paving the way for future research on controllable video synthesis.

## References

- Achiam et al. (2023)
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*, 2023.
- Bain et al. (2021)
Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman.
Frozen in time: A joint video and image encoder for end-to-end retrieval.
In *International Conference on Computer Vision*, pp.  1728–1738, 2021.
- Blattmann et al. (2023)
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.
Stable video diffusion: Scaling latent video diffusion models to large datasets.
*arXiv preprint arXiv:2311.15127*, 2023.
- Castellano (2024)
Brandon Castellano.
Pyscenedetect: Intelligent scene‐cut detection and video splitting tool (version 0.6.4).
Python package and tool, 2024.
URL [https://github.com/Breakthrough/PySceneDetect](https://github.com/Breakthrough/PySceneDetect).
Release version 0.6.4 (June 10, 2024).
- Chen et al. (2023a)
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al.
Videocrafter1: Open diffusion models for high-quality video generation.
*arXiv preprint arXiv:2310.19512*, 2023a.
- Chen et al. (2024a)
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan.
Videocrafter2: Overcoming data limitations for high-quality video diffusion models.
In *Computer Vision and Pattern Recognition*, pp.  7310–7320, 2024a.
- Chen et al. (2024b)
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al.
Panda-70m: Captioning 70m videos with multiple cross-modality teachers.
In *Computer Vision and Pattern Recognition*, pp.  13320–13331, 2024b.
- Chen et al. (2023b)
Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu.
Seine: Short-to-long video diffusion model for generative transition and prediction.
In *ICLR*, 2023b.
- Cho et al. (2024)
Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang.
Sora as an agi world model? a complete survey on text-to-video generation.
*arXiv preprint arXiv:2403.05131*, 2024.
- Dalal et al. (2025)
Karan Dalal, Daniel Koceja, Jiarui Xu, Yue Zhao, Shihao Han, Ka Chun Cheung, Jan Kautz, Yejin Choi, Yu Sun, and Xiaolong Wang.
One-minute video generation with test-time training.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp.  17702–17711, 2025.
- Girdhar et al. (2023)
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra.
Imagebind: One embedding space to bind them all.
In *Computer Vision and Pattern Recognition*, pp.  15180–15190, 2023.
- Guo et al. (2025)
Yuwei Guo, Ceyuan Yang, Ziyan Yang, Zhibei Ma, Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang.
Long context tuning for video generation.
*arXiv preprint arXiv:2503.10589*, 2025.
- Han et al. (2023)
Mingfei Han, Linjie Yang, Xiaojun Chang, and Heng Wang.
Shot2story20k: A new benchmark for comprehensive understanding of multi-shot videos.
*arXiv preprint arXiv:2312.10300*, 2023.
- He et al. (2022)
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen.
Latent video diffusion models for high-fidelity long video generation.
*arXiv preprint arXiv:2211.13221*, 2022.
- He et al. (2023)
Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, et al.
Animate-a-story: Storytelling with retrieval-augmented video generation.
*arXiv preprint arXiv:2307.06940*, 2023.
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
In *Neural Information Processing Systems*, volume 33, pp.  6840–6851, 2020.
- Hu et al. (2022)
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.
Lora: Low-rank adaptation of large language models.
*ICLR*, 1(2):3, 2022.
- Huang et al. (2024)
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al.
Vbench: Comprehensive benchmark suite for video generative models.
In *Computer Vision and Pattern Recognition*, pp.  21807–21818, 2024.
- Ju et al. (2025)
Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan.
Miradata: A large-scale video dataset with long durations and structured captions.
In *Neural Information Processing Systems*, volume 37, pp.  48955–48970, 2025.
- Kara et al. (2025)
Ozgur Kara, Krishna Kumar Singh, Feng Liu, Duygu Ceylan, James M Rehg, and Tobias Hinz.
Shotadapter: Text-to-multi-shot video generation with diffusion models.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp.  28405–28415, 2025.
- Kong et al. (2024)
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.
Hunyuanvideo: A systematic framework for large video generative models.
*arXiv preprint arXiv:2412.03603*, 2024.
- Li et al. (2024)
Chengxuan Li, Di Huang, Zeyu Lu, Yang Xiao, Qingqi Pei, and Lei Bai.
A survey on long video generation: Challenges, methods, and prospects.
*arXiv preprint arXiv:2403.16407*, 2024.
- Li et al. (2023)
Xin Li, Wenqing Chu, Ye Wu, Weihang Yuan, Fanglong Liu, Qi Zhang, Fu Li, Haocheng Feng, Errui Ding, and Jingdong Wang.
Videogen: A reference-guided latent diffusion approach for high definition text-to-video generation.
*arXiv preprint arXiv:2309.00398*, 2023.
- Lin et al. (2024)
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al.
Open-sora plan: Open-source large video generation model.
*arXiv preprint arXiv:2412.00131*, 2024.
- Liu et al. (2024)
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee.
Llava-next: Improved reasoning, ocr, and world knowledge, January 2024.
URL [https://llava-vl.github.io/blog/2024-01-30-llava-next/](https://llava-vl.github.io/blog/2024-01-30-llava-next/).
- Miech et al. (2019)
Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic.
Howto100m: Learning a text-video embedding by watching hundred million narrated video clips.
In *International Conference on Computer Vision*, pp.  2630–2640, 2019.
- Nan et al. (2024)
Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai.
Openvid-1m: A large-scale high-quality dataset for text-to-video generation.
*arXiv preprint arXiv:2407.02371*, 2024.
- Polyak et al. (2024)
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al.
Movie gen: A cast of media foundation models.
*arXiv preprint arXiv:2410.13720*, 2024.
- Qi et al. (2025)
Tianhao Qi, Jianlong Yuan, Wanquan Feng, Shancheng Fang, Jiawei Liu, SiYu Zhou, Qian He, Hongtao Xie, and Yongdong Zhang.
Maskˆ 2dit: Dual mask-based diffusion transformer for multi-scene long video generation.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp.  18837–18846, 2025.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Computer Vision and Pattern Recognition*, pp.  10684–10695, 2022.
- Song et al. (a)
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
In *International Conference on Learning Representations*, a.
- Song et al. (b)
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
Score-based generative modeling through stochastic differential equations.
In *International Conference on Learning Representations*, b.
- Soucek & Lokoc (2024)
Tomás Soucek and Jakub Lokoc.
Transnet v2: An effective deep network architecture for fast shot transition detection.
In *ACM International Conference on Multimedia*, pp.  11218–11221, 2024.
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
In *Neural Information Processing Systems*, volume 30, 2017.
- Wan et al. (2025)
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al.
Wan: Open and advanced large-scale video generative models.
*arXiv preprint arXiv:2503.20314*, 2025.
- Wang et al. (2023a)
Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu.
Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation.
*arXiv preprint arXiv:2305.10874*, 2023a.
- Wang et al. (2024)
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al.
Lavie: High-quality video generation with cascaded latent diffusion models.
*International Journal of Computer Vision*, pp.  1–20, 2024.
- Wang et al. (2023b)
Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al.
Internvid: A large-scale video-text dataset for multimodal understanding and generation.
*arXiv preprint arXiv:2307.06942*, 2023b.
- Xie et al. (2024)
Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F Bissyand, and Saad Ezzini.
Dreamfactory: Pioneering multi-scene long video generation with a multi-agent framework.
*arXiv preprint arXiv:2408.11788*, 2024.
- Xiong et al. (2025)
Tianwei Xiong, Yuqing Wang, Daquan Zhou, Zhijie Lin, Jiashi Feng, and Xihui Liu.
Lvd-2m: A long-take video dataset with temporally dense captions.
In *Neural Information Processing Systems*, volume 37, pp.  16623–16644, 2025.
- Xu et al. (2024)
Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang.
Easyanimate: A high-performance long video generation method based on transformer architecture.
*arXiv preprint arXiv:2405.18991*, 2024.
- Xue et al. (2022)
Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, and Baining Guo.
Advancing high-resolution video-language representation with large-scale video transcriptions.
In *Computer Vision and Pattern Recognition*, pp.  5036–5045, 2022.
- Yang et al. (2024)
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al.
Cogvideox: Text-to-video diffusion models with an expert transformer.
*arXiv preprint arXiv:2408.06072*, 2024.
- Zellers et al. (2021)
Rowan Zellers, Ximing Lu, Jack Hessel, Youngjae Yu, Jae Sung Park, Jize Cao, Ali Farhadi, and Yejin Choi.
Merlot: Multimodal neural script knowledge models.
*Neural Information Processing Systems*, 34:23634–23651, 2021.
- Zhang et al. (2024)
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li.
Video instruction tuning with synthetic data.
*arXiv preprint arXiv:2410.02713*, 2024.
- Zhao et al. (2024)
Canyu Zhao, Mingyu Liu, Wen Wang, Weihua Chen, Fan Wang, Hao Chen, Bo Zhang, and Chunhua Shen.
Moviedreamer: Hierarchical generation for coherent long visual sequence.
*arXiv preprint arXiv:2407.16655*, 2024.
- Zheng et al. (2024)
Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, et al.
Videogen-of-thought: A collaborative framework for multi-shot video generation.
*arXiv preprint arXiv:2412.02259*, 2024.
- Zhou et al. (2024)
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou.
Storydiffusion: Consistent self-attention for long-range image and video generation.
*Advances in Neural Information Processing Systems*, 37:110315–110340, 2024.
- Zhuang et al. (2024)
Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, and Yali Wang.
Vlogger: Make your dream a vlog.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp.  8806–8817, June 2024.

## Appendix A Details of Dataset Construction

Section [4](https://arxiv.org/html/2508.11484v1#S4) describes the Cine250K construction pipeline. In this section, we provide a detailed account of different stage.

### A.1 Method of Splitting and Shot Segementation

During the Splitting stage, videos are initially segmented using PySceneDetect (Castellano, 2024). In the subsequent gradual-change removal stage, TransNetV2 (Soucek & Lokoc, 2024) is employed to detect and handle gradual transitions, yielding the final shot boundaries and associated shot labels. This pipeline is selected because Pyscenedetect operates exclusively on the CPU, offering greater processing efficiency than Transnetv2; however, for a pre-segmented video, Transnetv2 achieves higher shot segmentation accuracy and can handle gradual transitions more effectively than Pyscenedetect.

For Transnetv2, Figure [9](https://arxiv.org/html/2508.11484v1#A1.F9) presents an example of detection. Transnetv2 provides single-frame-prediction and all-frame-prediction, which are designed to handle hard cuts and gradual transitions, respectively.
The single-frame-prediction refers to the probability of an individual frame being a transition. In contrast, the all-frame-prediction predicts all frames involved in a transition, essentially estimating the probability of a frame being part of a gradual change.
We apply threshold-based filtering to exclude frames with a high probability of being transition frames and obtain shot labels. Table [5](https://arxiv.org/html/2508.11484v1#A1.T5)
presents the threshold settings.

Figure: Figure 9: An example of transition detection by Transnetv2. The green line represents the result of single-frame-prediction, while the blue line represents the result of all-frame-prediction.
Refer to caption: img/Transnetv2_example.png

**Table 5: Threshold settings for dataset construction.**
| parameter | value |
| --- | --- |
| Pyscenedetect splitting | 27 |
| $\alpha$ | 0.9 |
| $\beta$ | 0.7 |
| $\gamma$ | 0.8 |
| Transnetv2 single-frame-threshold | 0.45 |
| Transnetv2 all-frame-threshold | 0.50 |

**Table 6: Shot Segmentation Accuracy of different methods.**
| Method | Pyscenedetect | Transnetv2 |
| --- | --- | --- |
| Shot Segmentation Accuracy | 65.50% | 87.00% |

Figure: Figure 10: An example of shot segmentation by Pyscenedetect and Transnetv2. In the figure, Transnetv2, after the removing gradual changes step, accurately segments the two shots, while Pyscenedetect fails to identify the gradual changes. The transition frames caused by gradual changes are marked with dashed lines.
Refer to caption: x13.png

To quantitatively compare accuracy on shot segmentation, we randomly select 200 videos covering multiple categories, each containing multiple shots. We then apply both Pyscenedetect and Transnetv2 for shot segmentation and compare the results manually. A correctly segmented video is assigned 1; otherwise, 0. The shot segmentation accuracy is then calculated accordingly, as demonstrated in Table [6](https://arxiv.org/html/2508.11484v1#A1.T6). A specific example is shown in Figure [10](https://arxiv.org/html/2508.11484v1#A1.F10).

### A.2 Details of Stitching Phase

During the Stitching stage, adjacent segments exhibiting high semantic similarity are merged to form a single video, thereby constructing multi-shot video collections. Specifically, we extract semantic features from the first and last frames of each segment using ImageBind (Girdhar et al., 2023) and quantify their similarity via Euclidean distance. For the $i$-th segment $C^{i}$, we denote the semantic features of its first and last frames as $C_{\mathrm{first}}^{i}$ and $C_{\mathrm{end}}^{i}$, respectively, with their distance represented as $\mathrm{dis}(C_{\mathrm{first}}^{i},C_{\mathrm{end}}^{i})$. Based on the distance, video segments are processed sequentially and either stitched or filtered according to the following criteria.

- •
For a segment $C^{i}$, if $\mathrm{dis}(C_{\mathrm{first}}^{i},C_{\mathrm{end}}^{i})>\alpha$, the segment is filtered.
- •
For $C^{i}$, if $C^{i-1}$ is absent (either non-existent or filtered), $C^{i}$ is treated as a video’s beginning.
- •
For $C^{i}$, if $\mathrm{dis}(C^{i-1}_{\mathrm{end}},C^{i}_{\mathrm{first}})<\beta$ and $\mathrm{dis}(C^{i-1}_{\mathrm{first}},C^{i}_{\mathrm{end}})<\gamma$, then $C^{i-1}$ and $C^{i}$ are stitched to form a new segment.

Here, $\alpha$ restricts significant changes within a clip, $\beta$ enables stitching of semantically similar transitions or originally continuous segments, and $\gamma$ ensures consistency between the video’s beginning and end. After processing all segments sequentially, each resulting segment is considered a complete video, forming a preliminary dataset of videos with shot transitions.

## Appendix B Statistic of Cine250K

As we present in Section [4](https://arxiv.org/html/2508.11484v1#S4), Cine250K is a carefully curated multi-shot video dataset with detailed captions. This section presents its overall statistics.
As shown in Figure [11](https://arxiv.org/html/2508.11484v1#A2.F11), the average video duration is 10.75s, and the average caption length is 148.79. Most videos contain 2 to 3 shots. It is important to note that although duration and shot count filtering are applied during data processing, TransnetV2 (Soucek & Lokoc, 2024) is later utilized to remove gradual changes and re-identify shots. As a result, the final shot count and duration do not strictly adhere to the initial filtering criteria. After reidentification, 87.99% of the videos contain 2 to 5 shots. Figure [11](https://arxiv.org/html/2508.11484v1#A2.F11) presents the shot distribution for videos with 1 to 10 shots, which represents 99.90% of all videos.

Figure: Figure 11: Statistics of Cine250K. To facilitate observation, the figure presents the shot distribution for videos containing 1 to 10 shots. The caption word count distribution only considers data with fewer than 500 words.
Refer to caption: x14.png

Regarding video categories, following Vimeo’s classification, the dataset is divided into 10 categories. Among them, Travel and Documentary have relatively higher proportions. Overall, the distribution of categories is fairly balanced.

In Figure [12](https://arxiv.org/html/2508.11484v1#A2.F12), we select several example video-text pairs to demonstrate the specific characteristics of the cinematic multi-shot sequences in the dataset and the style of captions.

Figure: Figure 12: Example video-text pairs in Cine250K.
Refer to caption: x15.png

## Appendix C Additional Details of Implementation

### C.1 Visible-First-Frame Attention

Figure: Figure 13: Visualization of the temporal-domain attention maps of Wan2.1 when generating multi-shot videos. Certain layers exhibit a pronounced focus on the first temporal slice, which motivates our Visible-First-Frame Attention mechanism.
Refer to caption: x16.png

In Section [5.3](https://arxiv.org/html/2508.11484v1#S5.SS3), we introduce the implementation detail dubbed Visible-First-Frame Attention, which serves to further stabilize the mask mechanism, particularly within DiT architectures, as demonstrated in Figure [8(a)](https://arxiv.org/html/2508.11484v1#S6.F8.sf1). This mechanism is also motivated by our analysis of DiT’s attention maps during multi-shot video generation. As illustrated in Figure [13](https://arxiv.org/html/2508.11484v1#A3.F13), in certain layers all visual tokens assign high attention probabilities to tokens originating from the first frame (owing to temporal compression in the VAE, this in fact corresponds to the first latent temporal slice), suggesting that the initial frame assumes a special function in the diffusion model’s denoising process. In light of this observation, we modify our mask matrix so that its first column is set entirely to zero, thereby effecting the Visible-First-Frame Attention mechanism, which can be formulated as:

$$ $\mathcal{M}_{ij}=\begin{cases}0&\text{if }j=1\,\text{or}\,i,j\in\text{same shot}\\ -\infty&\text{if }j\neq 1\,\text{or}\,i,j\notin\text{same shot}\end{cases}$ (4) $$

### C.2 Multi-prompt

In Section [6.2](https://arxiv.org/html/2508.11484v1#S6.SS2), we note that some methods employ a multi-prompt strategy at inference, i.e., each shot is prompted by its own text description. CineTrans-DiT also supports this capability. In addition to the primary mask matrix between video tokens, we introduce an additional mask between text and video tokens, following Qi et al. (2025), to enable precise semantic control over each shot. It is worth noting that this mask is applied only to the attention layers governing text–video token interactions, and is distinct from our proposed mask mechanism, which operates on video–video token interactions.

## Appendix D Additional Results

### D.1 The specific details of the attention probabilities.

In Section [5.1](https://arxiv.org/html/2508.11484v1#S5.SS1), we explore the frame correlations in the case of cinematic multi-shot video generation in diffusion models, where attention modules in certain layers exhibit a block-diagonal structure, i.e., strong correlations for intra-shot frames and weak correlations for inter-shot frames. In this section, we will discuss the specific details of the attention maps across different architectures, layers, and timesteps.

As for the architecture, we investigate both the temporal-spatial-decoupled framework and the full attention framework. The temporal-spatial-decoupled framework applies the temporal attention module directly to the time sequences, where tokens at different spatial locations do not interact with each other. In contrast, the full attention framework operates on all tokens of the video, allowing correlations across both temporal and spatial dimensions simultaneously. To focus on frame correlations, we group and average the tokens in the full attention framework by the frames before conducting further analysis. In Figure [14](https://arxiv.org/html/2508.11484v1#A4.F14) and Figure [15](https://arxiv.org/html/2508.11484v1#A4.F15), we use Wang et al. (2024) and Kong et al. (2024) as representatives of different architectures and present the attention maps across different layers and timesteps in both qualitative and quantitative manners.

As timesteps increase, the discrepancy between intra-shot and inter-shot attention probabilities grows in the temporal-spatial-decoupled framework, whereas it remains stable in the full attention framework. For different layers, the temporal-spatial-decoupled framework exhibits noticeable differences across most layers, whereas the full attention framework shows disparity primarily in the earlier layers. In summary, both frameworks demonstrate significant differences between intra-shot and inter-shot attention probabilities. This finding further substantiates the prevalence of strong intra-shot and weak inter-shot correlations, supporting the proposed approach.

Figure: (a) Visualization of temporal attention maps in temporal-spatial-decoupled framework. The attention probability matrices across most layers exhibit a block-diagonal pattern.
Refer to caption: x17.png

Figure: (a) Result of temporal attention probabilities in temporal-spatial-decoupled framework. In most layers, the probabilities of tokens within a shot and those across shots exhibit a noticeable difference, which seems to increase as the denoising process progresses.
Refer to caption: x19.png

### D.2 Impact of Mask Layers on Results

Figure: (a) Results of CineTrans-Unet.
Refer to caption: x21.png

For CineTrans-Unet and CineTrans-DiT, the mask mechanism employs different masking layers, a design choice motivated by observations of the attention maps when diffusion models generate multi-shot videos. In this section, we examine the effects of applying masks to different layers on the quality of the generated videos, thereby demonstrating the rationale behind our masking strategy.

As shown in Figure [16(a)](https://arxiv.org/html/2508.11484v1#A4.F16.sf1), CineTrans-Unet exhibits noticeable visual distortions when the mask is applied to all layers or only to the early layers. In severe cases, some parts of the video become indistinct or heavily degraded. When no mask is applied or when the mask is applied only to the middle layers, cinematic transitions do not emerge clearly. Applying the mask to the later layers enables effective control over transitions without significantly affecting visual quality, suggesting it as the optimal strategy. This suggests that, in the spatial-temporal-decoupled architecture, correlations in the earlier layers primarily influence visual quality, while the later layers may regulate the consistency between adjacent frames. Therefore, applying the mask mechanism to the last six layers is demonstrated to be effective.

As shown in Figure [16(b)](https://arxiv.org/html/2508.11484v1#A4.F16.sf2), the proportion of layers requiring masking in CineTrans-DiT is larger than in CineTrans-Unet, and both the early and late layers need to retain fully visible attention. Masking all layers to this architecture may lead to low inter-shot consistency and severe visual degradation at the transitions. Conversely, masking only the earlier or later layers fails to effectively guide the transitions. Moreover, when fewer layers are masked, the inter-shot differences are reduced, which deviates from the convention of multi-shot video. These observations suggest that the current strategy of masking the middle layers achieves the optimal balance, enabling precise control over transitions while preserving a reasonable degree of consistency.

### D.3 Qualitative Evaluation for Ablation Studies

Figure: Figure 17: Qualitative results of ablation studies.
Refer to caption: x23.png

The quantitative results of the ablation studies are presented in Section [6.3](https://arxiv.org/html/2508.11484v1#S6.SS3), and this section provides a supplementary qualitative analysis.

As shown in Figure [17](https://arxiv.org/html/2508.11484v1#A4.F17), CineTrans can stably control the generation of transitions. In contrast, models without the mask mechanism, whether finetuned or not, are almost incapable of generating cinematic transitions. Direct application of the mask mechanism without further finetuning, i.e., training-free results, does indeed generate transitions.
However, further training can help the model produce shot transitions that better align with the film editing style, exhibiting higher aesthetic quality and improved consistency.

## Appendix E Evaluation

### E.1 Prompt Details

In Section [6.2](https://arxiv.org/html/2508.11484v1#S6.SS2), we present a set of prompts generated by GPT-4o (Achiam et al., 2023) to evaluate the performance of multi-shot video generation. Figure [18(a)](https://arxiv.org/html/2508.11484v1#A5.F18.sf1) provides the prompt used to guide GPT.

Given the long inference time of video generation models, we design 100 prompts for evaluation, covering multiple categories. For these prompts, the transition guidance can be divided into two types. One type explicitly specifies a significant semantic change, such as prompts that include phrases like The video transition to, clearly indicating shot changes. The other type implicitly guides the shot transition, where the prompt does not explicitly differentiate content changes between shots, but rather provides an overall description of the video. This type of prompt design accounts for the fact that some cinematic multi-shot videos only involve switching camera angles without significant semantic change. The multi-shot guidance in such cases primarily relies on the prompt’s opening phrase: The multi-shot video consists of {num_shot} shots. Figure [18(b)](https://arxiv.org/html/2508.11484v1#A5.F18.sf2) presents examples of these two types of prompts, and Figure [18(c)](https://arxiv.org/html/2508.11484v1#A5.F18.sf3) shows the word cloud distribution for this set of prompts. Table [7](https://arxiv.org/html/2508.11484v1#A5.T7) outlines the categories of the prompts. It is worth noting that, for methods requiring multi-prompt inference, we employ GPT-4o to expand the general prompt into shot-specific captions according to the designated shot count.

**Table 7: Details of prompt categories for evaluation.**
| category | count |
| --- | --- |
| scenary | 34 |
| architecture | 10 |
| human | 25 |
| object | 31 |

Figure: (a)
Refer to caption: x24.png

### E.2 Metric details

In Section [6.2](https://arxiv.org/html/2508.11484v1#S6.SS2), we establish evaluation metrics from three aspects: transition control, temporal consistency, and overall video quality. This section will provide a detailed explanation of the specific definitions of these metrics.

Figure: Figure 19: Overview of the metric design. We devise evaluation measures along three complementary dimensions: transition control, temporal consistency, and overall video quality.
Refer to caption: x26.png

Transition Control. For transition control, we define the Transition Control Score to measure whether the shot count in the generated video aligns with the specified. Specifically, the generated video is first processed by Transnetv2 for shot segmentation, yielding the number of shots, denoted as $s_{\mathrm{generated}}$, while the prompt specifies a target shot count, $s_{\mathrm{specified}}$. The Transition Control Score can be computed according to Equation [6](https://arxiv.org/html/2508.11484v1#A5.E6).

$$ $x=\dfrac{s_{\mathrm{generated}}}{s_{\mathrm{specified}}}$ (5) $$

$$ $\mathrm{Transition\,Control\,Score}=\dfrac{x^{k}}{e^{k(x-1)}}$ (6) $$

Figure [19](https://arxiv.org/html/2508.11484v1#A5.F19) visualizes the calculation method of Equation [6](https://arxiv.org/html/2508.11484v1#A5.E6) for the Transition Control subpanel. In practice, when $x<1$, $k$ is set to 2, and when $x\geq 1$, $k$ is set to 1.6. Given that the prompts used for evaluation all specify multiple transitions, the score is set to 0 when the generated video consists of a single shot. When the shot count in the generated video matches the specified value, the score is recorded as 1. More generally, the score is determined based on the absolute difference from the specified value.

Temporal consistency. In terms of temporal consistency, we consider both intra-shot consistency and inter-shot consistency. Intra-shot consistency treats each shot as a separate video and calculates the metric between adjacent frames using a method similar to that in VBench (Huang et al., 2024). The focus of this section is on inter-shot consistency. As for frame extraction, we use the middle frame of each shot for calculation. However, if the video does not generate multiple shots, inter-shot consistency cannot be evaluated. As shown in Table [2](https://arxiv.org/html/2508.11484v1#S6.T2), the original LaVie (Wang et al., 2024) lacks the ability to generate transitions, and therefore its inter-shot consistency metric does not have a corresponding value.

Regarding the metric definition, inter-shot consistency cannot directly serve as the final evaluation metric. In multi-shot video generation, the goal is to ensure that the generated video aligns with the editing style of film-edited videos. High consistency would imply pixel-level similarity, which may contradict the multi-shot nature of real video editing. To address this, we extract 1000 film-edited videos as a validation dataset and compute their inter-shot consistency as a reference set. The final metric is then determined by the Jensen-Shannon Distance (JSD) between the inter-shot consistency of the generated video and that of film-edited videos, as shown in Equation [9](https://arxiv.org/html/2508.11484v1#A5.E9). A lower JSD indicates a closer alignment with the reference distribution.

$$ $M=\frac{1}{2}(P+Q)$ (7) $$

$$ $\mathrm{JS}(P\,\|\,Q)=\dfrac{1}{2}D(P\,\|\,M)+\dfrac{1}{2}D(Q\,\|\,M)$ (8) $$

$$ $\mathrm{JSD}=\sqrt{\mathrm{JS}(P\|Q)}$ (9) $$

Videos whose inter-shot consistency distribution aligns with film editing styles are considered to exhibit higher inter-shot consistency performance, while those deviating from film editing practices are regarded as having lower performance. This defines the evaluation metric based on inter-shot consistency.

## Appendix F Limitation

### F.1 Failure Case

Figure: Figure 20: Failure case with low consistency, which probably results from insufficient training.
Refer to caption: x27.png

The role of the mask mechanism in controlling the occurrence of transitions is relatively stable. Even in cases where the generated videos does not exactly match the expected shot count, this can be attributed to relatively small compositional differences between shots, which, despite being perceptible to the human eye as content jumps, cannot be detected by the shot segmentation model. However, CineTrans still exhibits failure cases with low consistency, as illustrated in Figure [20](https://arxiv.org/html/2508.11484v1#A6.F20), which may be due to insufficient training or the need for more detailed character ID annotations in the dataset.

### F.2 Future Work

Although CineTrans has achieved promising results in cinematic multi-shot video generation using the mask mechanism, there are still limitations to be addressed, along with several promising directions for future research.

- •
While the mask mechanism enables control over the occurrence of shots, the specification of camera viewpoints could be made more controllable, for example, enabling changes in shooting perspective within a scene that are more consistent with the film production pipeline. Therefore, achieving higher content consistency and more precise control over camera viewpoints will be a key future direction, potentially requiring the incorporation of 3D information as prior knowledge.
- •
Multi-shot videos could also be extended to greater lengths. Incorporating auto-regressive generation into the video generation pipeline represents a promising approach for producing longer multi-shot videos.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 10
presentation: 4
questions: 1. Layer selection sensitivity: Appendix E.2 suggests the best results with late layers for UNet and middle layers for DiT. Can you provide a brief heuristic or automated criterion for selecting mask layers on a new backbone, and quantify robustness to layer shifts?

2. Boundary robustness: At inference, how sensitive is control to small timestamp jitter in the provided shot boundaries, and do you recommend padding a few frames around boundaries to avoid failure cases like Figure 20?

3. Metrics specification: Appendix F mentions exact definitions. For completeness, would it be possible to include the precise formula for Transition Control Score and Consistency Gap in the main text or a boxed definition?
soundness: 4
strengths: - Clear, well-motivated mechanism: The block-diagonal mask is explicitly defined and integrated into the attention logits, with principled alignment to measured intra- vs inter-shot attention structure.

- Strong empirical gains in control: Transition Control Score improves markedly over large T2V and multi-shot baselines while preserving quality.

- Thoughtful evaluation design: The paper evaluates transition control, intra-/inter-shot consistency, and aesthetic quality, including a novel Consistency Gap aligned to a film-edited reference set.

- Ablations that justify design: Impact of masking different layer ranges is analyzed for both UNet and DiT architectures, guiding the chosen masking strategy.

- **High-quality supplementary material: Appendix is detailed, code and logs are included, and an HTML project page with videos aids understanding and reproducibility.** Kudos on the hard work!
summary: The paper proposes CineTrans, a masked-attention mechanism for controllable multi-shot, film-style transitions in video diffusion models, together with Cine250K, a curated multi-shot dataset. A simple block-diagonal mask added to attention logits enforces strong intra-shot and weak inter-shot correlations, aligning with observed attention patterns; the method works training-free and improves further with fine-tuning, achieving strong transition control and competitive quality on comprehensive metrics.
weaknesses: - Over-hard masking; missed opportunity for temporal scheduling: Equation (2) uses a binary mask with 0 on same-shot pairs and $-\infty$ across shots, which hard-zeros inter-shot attention in Equation (3). While effective, this may induce abrupt changes (also visible in the shared videos). A time-dependent or diffusion-step-dependent penalty could yield smoother transitions, e.g., replacing $-\infty$ by $-\alpha(t)$ that reaches $-\infty$ for a couple of time steps, this ramps near shot boundaries and across denoising steps, then saturates, before relaxing post-transition. This would preserve some cross-shot context pre-cut and reduce artifacts at boundaries. Please consider and, if possible, report a small ablation of annealed masks; if not possible, then consider it as a suggestion for future work.

- Related work gap: Prior work on long video generation with temporal control, e.g., VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis [ICLR’25], is not discussed. Please position CineTrans relative to these temporal-control strategies and clarify the conceptual differences between mask-based control and these schedules. Additionally, multiple prior works have considered Masked Attention for various applications (not just video generation). It would be a good idea to discuss a few of these related works as well, briefly.  

References:

[ICLR25] Li, Yumeng, et al. "VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis." The Thirteenth International Conference on Learning Representations.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. Using the mask strategy requires knowing between which frames a shot transition occurs. How does this paper determine the frame length of different shots when generating multi-shot videos? Is it manually set, or controlled by hyperparameters? If it is manually set, how do you ensure that a complete action can be generated within a single shot without being interrupted?
soundness: 3
strengths: 1. This paper contributes a large multi-shot dataset of 250K videos with frame-level shot boundaries and hierarchical captions.
2. The proposed method is simple and easy to follow.
3. The paper is well-structured and easy to read.
summary: This paper proposes a novel framework for generating coherent multi-shot videos with film-style transitions by introducing a block-diagonal attention mask and a “Visible-First-Frame” mechanism in video diffusion models, enabling precise shot boundaries and stable temporal consistency. Additionally, this paper constructs a 250K video–text dataset with frame-level shot labels to support video diffusion models in generating cinematic transitions and maintaining inter-shot consistency. This paper proposes a series of comprehensive metrics to evaluate the results of multi-shot video generation.
weaknesses: 1. The method in this paper only adds a mask mechanism between shots to ensure content consistency, but this makes it difficult to maintain fine-grained consistency across different shots, especially for background regions, as shown on the left side of Figure 5. Table 1 (Intra-shot Consistency) also demonstrates that the improvement in consistency achieved by this method is limited compared to the baseline.
2. Although the paper claims to achieve cinematic transitions, the proposed method only supports hard cuts, which limits its range of applications.

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 2
questions: 1. Robustness to Ambiguous or Overlapping Shot Boundaries: How does CineTrans perform when shot boundaries are ambiguous or overlap semantically/visually (e.g., with crossfade-like transitions)? Is the mask mechanism brittle or robust under these settings?
2. Mask Relaxation Variants: Have the authors experimented with soft masks or probabilistic boundary definitions? Are there gains in aesthetic quality, or does this impair transition control?
3. Aesthetic Quality Decline After Fine-Tuning: What is the hypothesized cause of the decline in Aesthetic Quality (Table 2) after fine-tuning? Is it dataset-related or architectural?
soundness: 2
strengths: The work introduces a novel mechanism that discovers and leverages a block-diagonal pattern in attention maps for transition control, directly inspiring an improved architecture; it further uses a mask-based strategy to align model internals with multi-shot video structure, enabling precise, shot-wise editing. A new large-scale dataset, Cine250K, fills a key gap with rich annotations (including frame-level labels and semantic stitching) and follows film-editing conventions. Extensive experiments provide thorough ablations and comparisons on transition control, intra- and inter-shot consistency, and aesthetic quality, complemented by insightful visual analyses. The approach transfers cleanly to customized or pre-trained diffusion backbones and improves both transition control and consistency in training-free and fine-tuned variants, with clear, intuitive examples and visual ablations highlighting advantages over existing methods.
summary: The paper introduces CineTrans, a diffusion-based framework for generating coherent multi-shot videos with cinematic transitions, aiming to bridge the gap between single-shot and multi-shot video generation prevalent in recent works. To support this, the authors construct Cine250K, a large-scale, multi-shot video-text dataset with detailed shot annotations and hierarchical textual descriptions. CineTrans leverages an observed block-diagonal attention pattern for shot boundaries in diffusion models, and implements a mask-based control mechanism that enables frame-level control over cinematic transitions—even in a training-free regime. The approach is empirically validated against several strong baselines, showing superior transition control, temporal consistency, and overall quality via both standard and custom metrics.
weaknesses: 1. Lack of Analysis on Limiting Scenarios: The paper demonstrates strong results on curated prompts and the Cine250K distribution, but does not critically examine or quantify limitations outside this scope, e.g., severe domain shifts, failure cases, or fundamental breakdowns of mask-based control when transition points are ambiguous or overlap.
2. Limited Theoretical Rigor or Insights: While the empirical demonstration of attention map patterns is clear (see Figure 4), the work lacks theoretical analysis or ablation to clarify why these attention correlations emerge (e.g., role of architecture, dataset, or objective). Are these properties universal, or dataset/model-dependent? This affects reproducibility and generalizability.


Decision: Accept (Poster)