SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
Authors: Junsong Chen
Yuyang Zhao
Jincheng YU
Ruihang Chu
Junyu Chen
Shuai Yang
Xianbang Wang
Yicheng Pan
Daquan Zhou
Huan Ling
Haozhe Liu
Hongwei Yi
Hao Zhang
Muyang Li
Yukang Chen
Han Cai
Sanja Fidler
Ping Luo
Song Han
Enze Xie
Venue: ICLR 2026 Oral (2026)
Keywords: Video Diffusion Model
Primary area: generative models

Abstract: We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720×1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at a remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation:  (1) Linear DiT: We leverage linear attention as the core operation, which is more efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block-wise autoregressive approach for long video generation by employing a constant-memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at a fixed memory cost, eliminating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, we explore effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1\% of the cost of MovieGen. Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models (e.g., Wan 2.1-1.3B and SkyReel-V2-1.3B) while being 16x faster in measured latency. Moreover, SANA-Video can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x} speedup). In summary, SANA-Video enables low-cost, high-quality video generation. Code and model will be publicly released.

Published as a conference paper at ICLR 2026
SANA-VIDERS EFFICIENT VIDEO GENERATION WITH BLOCK LINEAR DIFFUSION TRANSFORMER
Junsong Chen Yuyang Zhao"*, Jincheng Yu Ruihang Chu', Junyu Chen',
Shuai Yang', Xianbang Wang Yicheng Pan*, Daquan Zhou', Huan Ling',
Haozhe Liu Sanja Fidler?, Ping Luo', Song 2an Enze Xie
Hongwei Yi² Hao Zhang?, Muyang Li Yukang Chen', Han Cai?,
HKU NNVIDIA 'IIT 'THU PKU KAUST
Project Page: http://nwlabbsgituuub.ooooo
SANA
SANA
SANA
Physical
Simulation
(a). Generation results from our models
Linear Attention
SANA-Video-28
Causal Full Attention
Wan2.1-1.38
SkyReelv2 1.3B
568
100
OOM
93
Wanz.1 1.38
403
46
CogVideoX
400
26
35
58
20
SANA-Video
2B
36
16x
7.2
7.2
7.2
72
5
0
148
295
443
590
30
10
Generation latency on 720x1280x81 (c). GPU VRAM video length video with default inference settings
(d). Generation latency video
on 480x832 resolution
length (s) on 48DxB32 resolution
Figure 1: An overview generated videos and inference latency and memory of SANA- Video. The generation latency is measured under 50 denoising steps. Linear attention is more efficient for video generation and our block linear attention maintains a fixed memory requirement for long videos.
ABSTRACT
We introduce SANA-Video, a small diffusion model that can efficiently gener- ate videos up 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation: (1) Linear DřT: We leverage linear attention the operation, which is
efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block- wise autoregressive approach for long video generation by employing constant- memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at fixed memory cost, elim- inating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, explore effective data filters and model train- ing strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1 % of the cost of MovieGen. Given its low cost, SANA- Video achieves com- petitive performance compared to modern ofate-of-the- th-the--rt the small diffusion models (e.g., Wan 2, 1-1.3B and SkyyReel-V2-1.33 while being 16x faster measured la-

---

Published as a conference paper at ICLR 2026
tency, Moreover, SANA-Vidde can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x speedup). In summary. SANA- enables low-cost, high-quality video generation. Code and model will be publicly released.
INTRODUCTION
Video generation is currently a highly active field, fueling applications that range from creative content production and digital live streaming virtual product displays. Recent large-scale models from industry labs, such as Veo3 (DeepMind, 2025), Kling (Kuaishou, 2024), Wan (Wang et al,, 2025a) and Seedance (Gao al., 2025), have demonstrated remarkable performance in generating high-fidelity video content. However, this quality comes at the cost of immense computational complexity. Video generation is exceptionally token-extensive task; for instance, producing single 5-second video at 720p resolution with a model like Wan 14B (Wang et al., 2025a) requires to process over 75,000 tokens, taking 32 minutes on H100 GPU. This sheer volume of data leads to prohibitive training costs and extremely slow generation speeds, rendering these powerful models impractical for widespread research and application. Even with large cost, generating long video (> 10 s) is hard to realize with these large models due to the full-sequence processing operation. Recent works (e.g., MAGI-1 (Teng et al., 2025) and SkyReelv2 (Chen et al., 2025a)) explores the long video generation but the efficiency is strictly constrained by the vanilla attention and KV cache. Given these challenges, a pivotal question arises: Can we develop a high qualitya and high- rig-rresotution video generator that is computationally efficient and runs very fast on both cloud and edge devices?
This paper proposes SANA- Video, a small diffusion model designed for both efficient training and rapid inference without compromising output quality. In stark contrast to the massive resource quirements of contemporary models, SANA- Video's 's training is remarkably cost-effective, requiring only 64 NVIDIA H100 GPUs for 12 days, which represents little as 1% of the training cost of MovieGen (Polyak et al., 2024) and 10% of that of OpenSora (Zheng et al., 2024). This efficiency extends to inference, where SANA-Vides can generate a 5-sccond, 720p video in just 36 seconds on a NVVIDI H100 GPU. By drastically reducing the computational barrier, SANA- Video makes high- quality video generation more accessible and practical for broader range of users and systems. The improvements mainly lie in three key components.
Linear DIT. We extend SANA (Xie al., 2025a) linear DiT design to the video domain, addressing the significant computational bottleneck of traditional self-attention (O(N3), as shown in Fig. 1(d). By replacing all attention modules with our efficient linear attention, we reduce complexity to (N), which is crucial for high-resolution video generation and leads to 4x acceleration on 720p video. To enhance our model for video, we make two key improvements. We first integrate Rotary Position Embeddings (RoPE) (Su et al., 2024) to improve long-context modeling. In Sec. 3.2, we detail our exploration of the optimal placement for RoPE and how we address the training instability it can introduce. Additionally, connection. This design allows us to effectively leverage pre-trained image models and efficiently adapt them for video generation by aggregating temporal features.
introduce ID temporal convolution to the Mix-FFN via shortcut
Block Linear Attention with KV Cache The success of SANA- in long video generation is mainly inspired by the attribute of causal linear attention (Katharopoulos al., 2020). Based on our reformulation of the causal linear attention operation, we reduce the KV cache to a small and fixed memory, along with fixed computational cost for each new token. This natively supports long- context operations. Based on the block linear attention module, we introduce a two-stage autore- gressive model continue-training paradigm, including autoregressive block training with monotoni- cally increasing SNR sampler and the improved self-forcing specially for our long context attention operation, leading to efficient, long, and high quality video generation.
Efficient Data Filter and Training. The low training cost is mainly attribute to three aspects: the powerful pre-trained text-to-image (T2I) model, efficient data filtering, and the efficient training strategy. First, SANA-Video is continue pre-trained from SANA (Xie et al., 2025a;b)-1.6h T21 model with the modification for spatio-temporal modeling (Sec. 3.2). Second, we collect data from diverse data source and design specific data filtering criterion for each data source. In addition, strong VLM (Bai et al., 2025a) serves as our video captioner, producing highly detailed captions (80-100 words), including subject category, color, appearance, actions, expressions, surrounding
2

---

Published as a conference paper at ICLR 2026
environment, camera angles, etc. Third, with the high-quality video-text pairs, train SANA- Video in multiple stages from low resolution to high resolution and finally leverage human preferred data for SFT, ensuring the model can efficiently learn the motion and aesthetic appearance.
In conclusion, our model achieves a latency that over 13 x faster than the sate-oofthheart Wan2.1 for 720p video generation (Fig. I(b)), while delivering competitive results across many benchmarks. Additionally, we quantize and deploy our SANA- Video on RTX 5090 GPUs with the NVFP4 preci- sion, where takes just 29 seconds generate 5s 720p video. We hope our model can be efficiently used by everyday users, providing a powerful foundation model for fast video generation.
2
PRELIMINARIES
2.1 VIDEO DIFFUSION MODEL
Following SANA (Xie et al., 2025a), we use Rectified Flows (RFs) (Esser et al., 2024) with SNR sampler as the training objective in Eq. 1. Here, c is the conditional embedding, 9 is the model weights, and u(x" t, c; denotes the output velocity predicted by the diffusion model. v(x) 3 the target velocity. In this paper, our SANA-Vides is a unified framework for Text-to- Im-age (T2I), Text- to- video (T2V), and Image-to- Video (I2V) generation by varying condition embeddings. Specifi- cally, for T2I and T2V, c is the text prompt and x is the image video. For I2V, we use first frame and text prompt as condition c. By setting the noise the first frame to zero, SANA-Vin Video can re- alize 12V without any model modification. Therefore, the joint training of T2I, T2V, and I2V makes SANA-Video a unified framework that can perform all tasks with a single model.
Ecgaa || Iu (t,;)) u(x)]]]
2.2 AUTOREGRESSIVE LONG VIDRO GENERATION
Autoregressive diffusion models combine token/block wise autoregressive chain-rule decomposi- tion with denoising diffusion models, emerging as a promising direction for long sequence genera- tion like language (Arriola et al., 2025) and video generation (Yin al., 2025; Chen et al., 2025a; Huang et al., 2025). Specifically, for a sequence of blocks T1:N = (x1, "2,-..,......................... IN)) the gen- eration process is product of block distribution using the chain rule p(11.N) IT, p( (r))))), with each block distribution p(=r) modeled using diffusion process (Eq. 1). This approach leverages the strengths of both autoregressive models and diffusion models to capture sequential dependencies and enable block-wise, high-quality generation.
SANA-VIDES
Scaling video generation to higher resolutions and longer sequences dramatically increases the num- ber of tokens, making the O(N2) complexity of self-attention major bottleneck in computation, speed, and memory. This underscores the need for efficient linear attention in video generation. Building upon SANA Linear DiT (Xie et al., 2025a), we introduce Linear Video DiT (Fig. 2(a)) for video generation by integrating two key components: Rotary Position Embeddings (RoPE) and a temporal ID convolution within the Mix-FFN. These designs keep SANA's macro architecture as well as additional temporal modeling (Fig. 2(b)), allowing us to leverage a pre-trained image model and efficiently adapting it into powerful video model through continuous pre-training. In addition to the short video generation, we introduce block linear attention module for efficient long video generation. With the re-formulation of linear atlention, the block linear attention module and causal Mix-FFN keep a constant- memoryy KV cache and linear computational cost for long video. Based on this KV cache, we design two stage post-training paradigm to unlock the infinite length generation ability, leading to high-quality and efficient long video generation model.
3.1 TRAINING STRATEGY
Stagel: VAE adaptation on text-to-image (T2I) Training video DiT models from scratch is issturce-ittessive due the mismatch between image and video VAEs. We address this by first ef- ficiently adapting existing T2I models to new video VAEs. Specifically, We leverage different video VAEs in generating videos of different resolution. For 480P videos, VAEs with high-compression
3

---

Published as a conference paper at ICLR 2026
Linear Attention
Mix-FFN
SALLA
SHOUP
Temporal 3x1x1 Condlayer
MatMM
Blocky
Block
Block
Bloei
Cannal liserr Adeesiso
Tused
x Conviayeer
(a). Block-wise autoregressive training pipeline
Mathful
h
RoPE
RDFE
SILU
Linear Aun
Cross Alto
ReLU
RelU
3
Mix
nice
Ficn
Convlayer
H. 3]
Linear
Linear
Linear
ConvLayer
Acyburpst sian SAMA"
Rewriter
mmall LLM
Time Emb
(b). Architecttrr overview of our SANA-Video.
(c). Linear DiT module
(ig. A high-level block-wise autoregressive training pipeline based on our block causal KV cache. (Details Sec. 3.3). Fig.(b) Our model pipeline, containing Autoencoder, Re-writer, Linear DiT, and text encoder. Fig.(c) The detailed design of the added 3D RoPE in linear attention and the temporal convolution in our Linear DiT's Mix-FFN.
Figure 2: Overview of SANA-
ratio limits the overall performance, and thus we adopt Wan-VAR (Wang et al., 2025a). For 720P high-resolution video, introduce video VAE, DCAE-V, which provides higher compression ratio for more efficient generation (details in Sec. 3.5). The adaptation of both VAEs is highly effi- cient, converging within 5-10k training steps, further demonstrating the strong generalization ability of our Linear DiT.
Stage2: Continue pre-training from T2I model. Initializing video Linear DiT from pre-trained T2I model (Xie et al., 2025a) is an efficient and effective way to leverage the well-learned visual and textual semantic knowledge. Therefore, from the first stage and introduce additional designs to model long-context and motion informa- tion (Sec. 3.2). The additional temporal designs are tailor-made for linear attention, improving the locality of attention operation. The newly added layers are zero-initialized with skip connection, which minimizes their influence on the pre-trained weights during early training. After this identity initialization, SANA- -ideo is trained in cearse-to-fine manner. It first trains on low-resolution, short videos (e.g., 192P 2.5 seconds) before moving to higher resolution, longer videos (e.g.. 480P seconds) with different data filtering criteria (Appendix E). This coarse-to-fine approach efficiently encourages SANA-Video fast learn dynamic information with abundant data and then refine de- tails using less, but higher-quality, data.
initialize SANAAVides with model adapted
Stage3: Autoregressive block training. The continued pre-training makes SANA-Video an effi- cient small diffusion model, primarily for high-resolution 5-second video generation. To enable the generation of much longer videos, we analyze the attributes of linear attention in Sec. 3.3 and pro- pose constant- memory block KV cache for autoregressive generation. Building this design, conduct autoregressive block training in two steps: we first train the autoregressive module and then address exposure bias with our improved self-foreing block training (Sec. 3.4). This process results in high-quality, efficient model for long video generation.
3.2 EFFICIENT LINEAR DIT PRETRAINING
SANA- Video adopts the SANA (Xie et al., 2025a) as the base architecture and innovatively tailors the I Linear Diffusion Transformer blocks to handle the unique challenges of T2V tasks, as depicted in Fig. 2. Several dedicated designs are proposed as follows:
Linear Attention in Video DIT. Our work extends the SANA (Xie et al., 2025a) architecture by integrating Rotary Position Embeddings (RoPE) (Su et al., 2024) into its efficient ReLU (ø) linear attention blocks. This integration crucial for enhancing the model's 's ability to handle the sequential and spatial relationships in high-quality video generation. The core of our design lies in applying RoPE after the ReLU activation, specifically as RoPE(ReLU(Z)). as shown in Fig. 2. This order is critical because it prevents the ReLU kernel from filtering the positional information encoded
4

---

Published as a conference paper at ICLR 2026
Me-aar DK PastLEE QK
Trahing Loss
Linear Altention No PK:
Linear with ((RoPE)))
Linear with
Vanila Altentio from Wan
RoPE)ACS)) [Ours)
Training Steps
(a) Attention map comparison
(b) ost-RooE QK Sum
Figure 3: Analysis of Linear Attention with RoPE. (a) Visual comparison of attention maps. First two plots compare vanilla softmax attention (Wan) to our linear attention without positional encod- ing. The latter two plots show our method's effect: applying RoPE after the ReLU kernel results in a sparser, more localized attention pattern. (b) Training loss for the QK sum (Eq. 2 denominator). Re- moving RoPE from the denominator (green line) ensures training stability, as discussed in Sec. 3.2.
by RoPE. As Fig. 3 shows, this design results in attention maps with clear focus on local re- gions, which is essential for capturing fine-grained video details. However, applying RoPE directly to queries and keys (as in vanilla attention) can make the linear attention mechanism numerically unstable (Su, 2021) due to the difference between softmax and ReLU similarity functions. The RoPE transformation can change the non non-negative nature of the ReLU output, potentially causing the denominator in the standard linear attention formula (Eq. 2) become zero. To solve this, modify the calculation: while the numerator includes RoPE on the queries and keys, we remove RoPE from either the key the query in the denominator. This ensures the denominator remains positive, guaranteeing training stability (Fig. 3 (b)) while still benefiting from positional encoding.
O ( RoPF(
RoPF(>(Q.)) ROPFLOCK,))V))
RoPE( N ()
( ((K))T)
where On, Qi K; and V, denote the output, query, key and value of the ith token.
Mix-FFN with map in SANA- with the softmax attention map in Wan2.1 (Wang et al., 2025a). We observe that linear attention is much denser and less focused on local details compared to softmax attention. SANA (Xie et al., 2025a) ameliorates the locality problem in image generation with the convolu- tion Mix-FFN. Building upon the Mix-FFN, we enhance it with temporal 1D convolution. The temporal convolution with a shortcut connection is appended to the end of the block (Fig. 2(b)), enabling seamless temporal feature aggregation while preserving initialization. The module helps capture local relationships along the temporal axis, resulting in better motion continuity and consis- tency in generated videos. As evidenced in our ablation study (Fig. 5(a)), this addition Ieads to a significantly lower training loss and improved motion performance.
Mixture. As shown in Fig. 3, we compare the linear attention
3.3 BLOCK LINEAR ATTENTION
This section outlines key components enabling efficient long-video generation. Inspired by the inherent attribute of causal linear attention (Katharopoulos et al., 2020), we explore the constant- memory global KV cache in our block linear attention module, which supports long-context atten- tion with small, fixed GPU memory. Based on this module, we introduce a two-stage autoregressive model continue training paradigm: autoregressive block training with monotonically increasing SNR sampler and an improved self-forcing method for our long-context attention.
3.3.1 BLOOC LINEAR ATTENTION WITH KV CACHE
Table 1: For a sequence with N tokens E RIXD, memory and compute costs are compared among three attention types. Causal linear attention shows best efficiency while maintains global memory.
Metrie
Causal Full Attention
Causal Local Attention
Causal Linear Attention
Memory
O(N x D)
O(W x D) O(W x D)
O(D*)
Comp. Cost (N th token)
O(N x D)
O(D*)
Comp. Cost (N tokens)
O(N x D)
O x W x D)
O(N x D2)
5

---

Published as a conference paper at ICLR 2026
Tokon
Meek
Cache
Compute
0r V.
Block
Q:
Block
Block
Tokenu
Zero Phrdingg
X
KM
GP
Cashed Taken
Block
Blook
Cumulaatve Sum
Token:
00
Cy
Tokon,
Vanilla Attention
Linear Attention
Causal Linear Attention
(a) Block Causal Linear Attention
(b) Block Causal Mix-FFN
Figure 4: Overview Block Linear Attention. (a) We compare the attention compute mechanism among vanilla attention, linear attention and causal linear attention. (b) The illustration of block causal Mix- -FN in processing the adjacent blocks.
Limitation of Causal Vanilla Attention. In view of the training objective (Eq. 1), block-wise causal attention is required to implement autoregressive generation. Recent works (Huang et al., 2025; Chen et al., 2025a; Teng et al., 2025) use combination of full attention within a block and causal attention to previous blocks. To reduce computational costs, they leverage KV cache, which is effective but comes with memory overhead. For each new token E R with N cached conditional tokens, it requires O(N D) memory to store the cache and O(N x D) FLOPs for the attention computation. However, since the computational and memory costs grow linearly, these methods (Huang et al., 2025; Chen et al., 2025a; Teng et al., 2025) often restrict the attention window to local scope during long video generation. While this maintains stable cost, it comes the expense of losing global-context information.
KV cache in Block Linear Attention. In contrast to the dramatically increased computational and memory cost in causal vamilla attention, linear attention (Katharopoulos et al., 2020) has significant efficiency advantage, naturally supporting long video generation with global attention while main- taining constant memory. Consider the causal attention setting, linear attention (Eq. 2) output for the ith token can be re-formulated as:
0, (QQ0 ((K)))
aQa (E V (QQ (E d(K))" + o(K)")
(QQ (E S
.(Q.) EE) o(K)")
(QQ)) [Ej] o(K))* + tOKK)
where Sj = =6(K)) denotes the attention state for the jth token. We omit RoPE here for sim- plicity. Obviously, long the cumulative sum of state ΣE=1 Sy and the cumulative sum of keys Σj=l o(K))² are stored, only the attention state for the ith token S; E DxD is required to com- pute. Therefore, the memory cost only Σj=1 SJ E RDD and Σj= d (())) E Dx1 taking O(D²) in total, and the computational cost is only O(D?). In Table and Fig. 4(a), we compare the memory and computational cost among causal full attention, causal local attention and our causal linear attention. Since N > W >> D, causal linear attention achieves the best efficiency and can still maintain global memory in long video generation.
Block Causal Mix-FFN. In addition linear attention, our proposed temporal-spatial Mix-FFN enhances locality using convolutional layers. To support long video generation, this module must also operate causally. We ensure causal processing during both training and inference with two operations, as illustrated in Fig. 4(b). First, to prevent information leakage from subsequent blocks during training, we append an all-zero token ('Zero Padding' E 1 XD) the end of each block E R T'' x Second, our causal temporal convolution (kernel size 3) requires the last frame of the preceding block. We address this by caching the last token of each block ('Token_ IR xxw b) and prepending it to the next. Overall, our causal linear DiT module keeps fixed memory cache, containing cumulative sum of attention states and keys from all previous frames for attention, along with the last frame of the previous block for Mix-FFN.
3.4 LONGSANA
Autoregressive Block Training. The continue training of the autoregressive SANA-Video variant, L.e. LongSANA, begins with the pre-trained 5s SANA-Video model. To align with the pre-trained

---

Published as a conference paper at ICLR 2026
model's distribution, propose monotonically increasing SNR sampler. Specifically, we ran- domly select a block and sample a timestep for it with the SNR sampler (Esser et al., 2024). Then the timesteps for the remaining blocks are sampled via propagated probability (Sun et al., 2025), ensuring all the timesteps are monotonically increasing, i.e., later blocks have larger timestep than early blocks. This proposed timestep sampler offers two key advantages. First, the monotonically increasing timesteps have a much smaller sampling space than random timesteps, which results in faster convergence and better performance. Second, applying the SNR sampler to a randomly selected block guarantees that every block is trained with sufficient information.
Addressing Exposure Bias with Long Training. However, monotonically increasing SNR sampler cannot address a severe problem in autoregressive generation, i.e. exposure bias, where condition blocks are ground truth during training but are generated content during inference, leading to error accumulation and limiting performance in long video generation. Self-Forcing (Huang et al., 2025) aims to address this issue a vanilla attention DiT' model with autoregressive rollout. Limited by the increasing VRAM requirement of causal vanilla attention (Fig. I(c) and Table 1), Self-Forcing uses local attention within a designed window size. Consequently, it sets the length of self-generated content to be the same as the pre- pre-trained model (i.e., 5s). Later on, LongLive (Yang al., 2025a) explores streaming long training on 1 minute video, but it still limits to the local attention with sink due the complexity of full attention. In contrast the full attention, the block linear attention in LongS ANA supports a long-context global KV cache with a small and constant GPU memory. This allows us to further extend LongLive with global attention when self generatiinn a much longer video (e.g., min), which better aligns the conditioning signals between training and inference and keeps better temporal consistency. The inference details are illustrated in Alg. 1.
3.5 DEEP COMPRESSION VIDEO AUTOENCODER
SANA-Vice achieves high efficiency and quality for 480P video generation using Wan- VAE. How-
ever, even with our efficient linear attention, the generation speed for 720P videos is 2.3x slower. This efficiency drop is even more severe for full attention DiT models (4x for Wan 2.1 1.3B), inspir- ing us to explore a more efficient VAE that can compress more tokens. We fine-tune DCAE (Chen et al., 2024c) into DCAE-V, with spatial down- wwwn--smllingggg factor of F 32, a temporal factor of T' 4, and channels C' = 32. The number of latent channels aligns with our pre-trained T21 model, enabling fast adaptation from an image to video model in the same latent space.
The concurrent Wan2.2-5B model also achieves 32 times spatial compression, by combining a VAE with a spatial down sampling factor of 16 and patch embedding compression of 2. The advantages of DCAE-V over Wan2.2- VAE are twofold. First, DCAE-V's 32 latent channels align with our pre- trained T21 model, which improves convergence speed. Second, to achieve the same compression ratio, Wan2.2-VAE would require the model to predict much larger latent dimension (192 vs. 32 in DCAE-V), a task that is difficult for a small diffusion model (Details in Appendix D.I). As shown in Table 3, DCAE-V exhibits reconstruction performance comparable to other state-of-the-an VAEs like Wan2.1 (Wang et al., 2025a), Wan2.2 (Wang et al., 2025a), and LTX-VViee (HaCohen et al., 2024). This high compression allows our model to achieve performance on par with much larger models (e.g., Wan2. and Wan2.2-5BB while demonstrating significant acceleration, as shown in Table 2. Specifically, SANA-Vided can generate 720P 5s video within just 36 seconds, which is 53x acceleration over Wan2. 1an2.1-14B 1-B. When compared to Wan2.2-5B, which shares the same compression ratio as ours, SANA-Video achieves 3.2x acceleration.
Table 2: Latency on H100 GPU and VBench Table 3: Reconstruction capability of different Autoencoders on Panda-70M 192p resolution.
evaluation on 720 1280 x 81 resolution videos.
Models
Latencyis)
Total
Quality
Semantic
Autoencoder
Ratio
PSNR
SSIM
LPIPS
2.-14B
1897
85.77
75.58
FSTIC16 (Wan2. I-VAE)
0.95
Wan-2. 1-1.3B
400
83.38
85.67
74.22
F16T4C48 (Wan2.2-VAE)
21
35.61
0.96
0.01
Wan-2.2-58
116
83.28
85.03
76,28
F32TSC128 (LTX-VAE)
64
32.26
0.3
0.04
SANA-Video-2B
36
84.05
84.63
81.73
F32T4C32 (Our DCAE-V)
128
33.25
0.94
7

---

Published as a conference paper at ICLR 2026
Table 4: Comprehensive comparison of our method with SOTA approaches in efliciency and performance on VBench. The speed is tested on one H100 GPU with BF16 Precision. Latency: Measured with a batch size 1, on for fair comparison. We highlight the best, second best, and third best entries.
video, using the model's default inference steps
Evaluation scores
Methods
Latency (8)
Speedup
#Params (B)
Total
Quality
Semantic / 12V
Text-to-Vidde
MAGI-1 (Teng et al., 2025)
435
1.1x
4.5B
79.18
82.04
67.74
Step-Video
Video (Ma et al., 2025)
246
2.0x
30B
81.83
84.46
71.28
Cog 1.5 (Yang al., 2024)
111
4.4x
5B
82.17
82.78
79.76
SkyReels- V2 (Chen al., 2025a)
132
3.7x
1.3B
82.67
84.70
74.53
Open-Sora-2.0 (Peng al., 2025)
465
1.0x
14B
84.34
85,40
80.12
Wan2.1-14B (Wang al., 2025a)
484
1.0x
14B
83.69
85.59
76.11
Wam2.1-1.33 1-1.B (Wang al., 2025a)
103
4.7x
1.3B
83.31
85.23
75.65
SANA-Video
60
8.0x
2B
83.71
84.35
81.35
Image-to- Video
MAGI-1 (Teng et al., 2025)
435
1.1x
4.5B
89.28
82.44
96.12
Step-Video- ViV (Ma et al., 2025)
246
2.0x
30B
88.36
81.22
95.50
CogVideoX-5b-12 (Yang al., 2024)
111
4.4x
5B
86.70
78.61
94.79
Hunyuan Video-12V (Kong al., 2024)
210
2.3x
13B
86.82
78.54
95.10
Wan2.1-14B (Wang al., 2025a)
493
1.0x
14B
86.86
80.82
92.90
SANA-Video
8.2x
79.65
96.40
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS.
Pipeline Settings. For DiT model, best utilize the pre-trained text-to-image model SANA (Xie et al., 2025a), our SANA- 2B is almost identical to those of the original SANA (Xie et al., 2025a), including the diffusion transformer model and small decoder-only text encoder. For 480P videos, we leverage Wan2.1-VAB (Wang et al., 2025a) autoencoder. For 720P high-resolution video generation, we fine-tune the DCAE (Chen et al., 2024c) into a video deep compression au- toencoder (DCAE-V) to facilitate more efficient training and inference. Our final model is trained on 64 H100 GPUs for approximately 12 days. Details are Appendix C.I.
4.2 PERFORMANCE COMPARISON AND ANALYSIS
The comprehensive efficiency and performance comparison among SANA-Vider with state- of-the-art is illustrated in Table 4. We adopt VBench (Zhang al., 2024) the performance evaluation metric and the generation latency of a 480P 81-frame video as efficiency metric. As shown in Table SANA- exhibits remarkable latency of 60 seconds, marking it the fastest model compared. This translates to a throughput that is 7.2x faster than MAGI-I and over 4x faster than Step-Video. In terms of comparison, SANA-Video achieves Total Score of 83.71 on text-to-video generation, comparable with large model Open-Sora-2.0 (14B) and outperforming Wan2.1 (1.3B). In addition, SANA- Video achieves 88 0200 Total Score on image-to-vine -o-ii-to- generation, outperformance large DiT models Wan2.1 (14B) and Hunyuan Video-12V (11B). Furthermore, SANA- achieves the best semantic I2V score across all the methods, demonstrating strong vision-text semantic alignment.
4.3 ABLATION STUDIES
We then conduct ablation studies on the crucial architectural modifications discussed in Sec. 3.2. As shown in Fig. 5, we provide training loss curves and latency profiles on H100 GPUs.
Linear Attention Module. We incorporate three key designs to enhance our linear attention model. First, we integrate 3D RoPE to focus linear attention on local features (Fig. 3). This improves perfor- mance, evidenced by significantly lower training loss (Fig. 5(a)). Second, to address differences between linear and vanilla attention, we introduce a Temporal Mix-FFN module. Its train- ing loss curve (Fig. 5(b)) demonstrates that ID temporal convolution layer significantly enhances
8

---

Published as a conference paper at ICLR 2026
Bock
Block
Temporal Conv Temporal Canv
Random
Toahing Locs
RoPE
Traniing
spoctiup
spasdupe
Increasing
0.1
10000
20000
30000
Traning Steps
Training Staps
192p
400p
720
(a) 3D RoPE
(b) Temporal Conv
(c) Attention Type Profile
(d) Timestep S Settings
Figure 5: SANA- Video configuration ablation studies. (a) Training loss curves with and without 3D RoPE. (b) Training loss curves with and without temporal 2D Convolution. (c) Latency compar- ison of BAANA-Vide between linear and full attention. (d) Comparison of monotonically increasing versus random timestep sampling in autoregressive block training.
performance. Finally, our linear attention design provides a significant efficiency advantage. As Fig. 5(c) shows, model's lateney becomes lower at higher resolutions, achieving 2 speedup at 480P and 4x at 720P, proving its superior efficiency for high-resolution video generation.
Monotonically Increasing SNR Sampler. We compare the proposed monotonically increasing SNR sampler with random timestep sampling in the autoregressive block training. As shown in Fig. 5(d) (two columns are from different blocks), monotonically increasing SNR sampler achieves better quality and more consistency across blocks.
We also provide quantitative ablations studies for 3D RoPE, temporal conv and monotonically in- creasing SNR sampler in Table 5. All the three design is crucial for achieving our high-quality, efficient and long video generation.
Table Quantitative ablation studies on VBench.
Model
Total Score
Quality Score
Semantic Score
w/o Temporal Conv
80.94
82.63
74.18
Temporal Conv
76.15
w/o 3D RoPE
81.19
82.68
75.22
w/ 3D RoPE
82.79
83,89
78.38
Random Steps Increasing Steps
82.00
83.13
77.51
83.70
84,43
80.78
Long Video Generation. We compare SANA-Video with previous autoregressive video genera- tion methods on VBench, as shown in Table 6. SANA-Video achieves comparable performance with Self-Forcing (Huang et al., 2025) while outperforming SkyReel-V V2 (Chen al., 2025a) and Caus Vid (Yin et al., 2025).
Table 6: Comparison of autoregressive video genera- tion methods on VBench.
1:20
71
87
2,4%
53
Model
Total Score f
Quality Score f
Semantic Score
43
50
CausVid
81.20
84.05
69,80
1200g1200<1
SkyReels-V2
82.67
84.70
74.53
50-Step Latency RTX 5090
Self- Forcing
84.31
85.07
81.28
Figure 6: Latency comparison of our model on BF16 and NVFP4 precision.
SANA-Video
83.70
80.78
APPLICATIONS AND DEPLOYMENT
As pre-training model, SANA- Video can be casily extended to multiple applications of vidco gen- eration. First, adapt SANA-Vicd to several world model applications (Fig. 1 and Appendix F): embodied AI, autonomous driving and game generation. (Details are in Appendix F). Second, we quantize model to NVFP4 for efficient inference.
On-Device Deployment with 4-Bit Quantization. quantize SANA-Video from BF16 NVFP4 format using SVDQuant (Li et al., 2024a). To balance
To facilitate efficient edge deployment, we

---

Published as a conference paper at ICLR 2026
efficiency and fidelity, selectively quantize the following layers: the QKV and output projec- tions in self-attention, the query and output projections in cross-attention, and the 1xl convolutions in feed-forward layers. Other components (normalization layers, temporal convolutions, and KV projections in cross-attention) are kept at higher precision to preserve semantic quality and prevent compounding errors. As shown in Fig. 6, this strategy reduces the end-to-end generation time for a 720p 5-second video from 71 to 29 on single RTX 5090 GPU, achieving a 2.4x latency speedup while maintaining quality indistinguishable from the BF16 baseline.
RELATED WORKS
Video generation has advanced rapidly with diffusion models, evolving from texi-to-image exten- sions with temporal layers (Singer et al., 2022; Wu et al., 2023b) to latent video diffusion (Blattmann al.. 2023a), temporal VAEs, and large-scale diffusion transformers such as Sora (Brooks al., 2024). Parallel work investigates hybrid autoregressive-diffsii
strategies to endow visual models with long-term planning, including block-wise and token-level hybrids (Li al., 2024b; Hu al., 2024; Deng et al., 2024), later extended to video with temporal planners (Liu et al., 2024a) and diffusion forcing (Chen et al., 2025a; 2024a). An alternative line research targets efficiency in training and inference through architectural design. Approaches include linear (Xie et al., 2025a) and state-space attention mechanisms (Liu et al., 2024b) for image generation, as well as mamba- based designs (Wang et al., 2025b; Gao et al., 2024) and spatio-temporal factorization (Chen al., 2023a; Ho et al., 2022; Wang et al., 2025c; Singer et al., 2022) for video generation. Collectively, these methods highlight the ongoing push toward scalable, efficient, and general-purpose video dif- fusion models. more comprehensive related work section is provided in the Appendix B.
7 CONCLUSION
In this paper, we introduce SANA-Video, small diffusion model that can efficiently generate high resolution, high-quality and long videos a remarkably fast speed and low hardward requirement. The significance of SANA-Vide lies in the following improvements: linear attention as the core operation, leading to remarkable efficiency improvement block linear attention with costant- memory KV cache, supporting minute-long video generation with fixed memory cost; effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs. With such a small cost, SANA-Vided showcases 16x faster latency but competitive performance with modern sate-of-thee-eee small diffusion models.
loken-extensive video generation task;
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Accelerate: An Extension of PyTTrch for enabling easy deployment of models for large-scale train- ing, 2022. URL hwtpt:h/ttththhhiee www.coo/////ooooo heom/hhgginggeeee www.ggaccacccccceeeeaeec
Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang. Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autore- gressive and diffuusin language models. arXiv preprint arXiv:2503.09573 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang. Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar X1vv25502.13923 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.1392. 2025b.
Bowen Baker, 1lge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (ypt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 35:24639-24654, 2022.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023a.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22563-22577 2023b.
G. Bradski. The OpenCV Library. Dr. Dobb's 's Journal of Software Tools, 2000.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufci Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, al. Video generation models as world simulators. OpenAl Blog. 1(8):1, 2024.
Brandon Castellano. PySceneDetect. PySceneDetect.
www.:/thhu...ooeeaaaaaaee URL http:////gttabb..ocm//eettthooo/
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 17:24081-2241255 2024a.
Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Juncheng Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengchen Ma, et al. Skyreels-v2: Infinite-length film generative model. arXiv preprint arXiv: 13044, 2025a.
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafterl: Open diffusion models for high- quality video generation. arXiv preprint arXiv:2310. 19512, 2023a.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXiv:2310.00422, 2023b.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to-stron training of diffusion transformer for 4k text-to-image generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2024b.
11

---

Published as a conference paper at ICLR 2026
Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. arXiv preprint arXiv:2410.10733, 2024c.
Shoufa Chen, Chongjian Ge, Yuqi Zhang, Yida Zhang, Fengda Zhu, Hao Yang. Hongxiang Hao, Hui Wu, Zhichao Lai, Yifei Hu, et al. Goku: Flow based video generative foundation models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 23516-23527, 2025b.
AgiBot World Colosseum contributors. Agibot world colosseum.
https://github. com/
OpenDrivetabl wet.e//eiaaaaaaaaaeaaaa Aoi 2024.
Google DeepMind. Veo http://tttssseeeeppnnnnggge////t
2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Shi Guang, and Haoqi Fan. Causal diffusion transformers for generative modeling. arXiv preprint vrXiv:2412.12095, 2024.
FFmpeg Developers. FFmpeg. 2025.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et Scaling rectified flow transformers for high-resolution image synthesis. In /CML, 2024.
Yu Gao, Jiancheng Huang, Xiaopeng Sun, Zequn Jie, Yujie Zhong, and Lin Ma. Matten: Video generation with mamba-attention. arXiv preprint arXiv:2405.03003. 2024.
Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang. Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. Seedance 1.0: Exploring the boundaries video generation models. arXiv preprint arXiv:2506.099 09113, 2025.
songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming- Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22930-22941, October 2023.
Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-context autoregressive video modeling with next-frame prediction. arXiv preprint arXiv:2503. 19325, 2025.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.0011 2024.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303: 2022.
Jinyi Hu, Shengding Hu, Yuxuan Song, Yufei Huang, Mingxuan Wang, Hao Zhou, Zhiyuan Liu, Wei-Ying Ma, and Maosong Sun. Acdit: Interpolating autoregressive conditional modeling and diffusion transformer. arXiv preprint arXiv.2412.07720, 2024.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap autoregressive video diffusion. arXiv preprint arXiv:2506.08 08009, 2025.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are mns: Fast autoregressive transformers with linear attention. In ICML, 2020.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.
Kuaishou. Kling ai. http://p///dtth.....o..bbaaeeeee
2024. 2024.
Black Forest Labs. Flux. https:
flux, 2024.
12

---

Published as a conference paper at ICLR 2026
Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun- Yan Zhu, and Song Han. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. arXiv preprint 21X1v:2411.05007, 2024a.
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 56424-56445, 2024b.
Xingyang Li Muyang Li*, Tianle Cai, Haocheng Xi, Shuo Yang. Yujun Lin, Lvmin Zhang, Songlin Yang. Jinbo Hu, Kelly Peng. Maneesh Agrawala, Ion Stoica, Kurt Keutzer, and Song Han. Radial attention: (n log n) sparse attention with energy decay for long video generation. NeurIPS, 2025.
Haozhe Liu, Shikun Liu, Zijian Zhou, Mengmeng Xu, Yanping Xie, Xiao Han, Juan C Pérez, Ding Liu, Kumara Kahatapitiya, Menglin Jia, al. Mardini: Masked autoregressive diffusion for video generation at scale. arXiv preprint 0rXiv:2410.20222 2024a.
Songhua Liu, Weihao Yu, Zhenxiong Tan, and Xinchao Wang. Linfusion: 1 gpu, 1 minute, 16k image. 2024b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 2017.
Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-12v technical report: The practice challenges, and future of video foundation model. arXiv preprint urXiv:2502.10244 2025.
William Peebles and Saining Xie. Scalable diffuuion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195 4195-4205, 2023.
Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, al. Open-sora 2.0: Training commercial-level video generation model in $200 k. arXiv preprint MxXiv:2503309642 2025.
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching Yao Chuang, et al. Movie gen: A cast of media founda- tion models. arXiv preprint arXiv:2410.13720 2024.
Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan Yuille, and Liang-Chieh Chen. Flowar: Scale- wise autoregressive image generation meets flow matching. arXiv preprint arXi::441.115205, 2024.
Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan Yuille, and Liang- Chen. Beyond next- token: Next-x prediction for autoregressive visual generation. arXiv preprint arXiv:2502.20388, 2025.
Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7br Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685, 2025.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation data. arXiv preprint arXiv:2209. 14792, 2022.
generaationnno without text-video
Jianlin Su. Transformer upgrade path: 2. rotary position encoding with comprehensive advantages. pp. 12966-12977, Mar 2021.
Jianlin Su. Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, Yi Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with auto-regressive diffusion. In CVPR, 2025.
13

---

Published as a conference paper at ICLR 2026
Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, et al. Magi-l: Autoregressive video generation at scale. arXiv preprint arXi::2505.12211, 2025.
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint WxXv:2503.20314. 2025a.
Hongjie Wang, Chih- Yao Ma, Yen-Cheng Liu, Hou, Tao Xu, Jialiang Wang, Felix Juefei-Xi Yaqiao Luo, Peizhao Zhang, Tingbo Hou, et al. Lingen: Towards high-resolution minute-length texe-to-video generation with linear computational complexity. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2578-2588, 2025b.
Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to-video diffusion models. In NeurlPS, 2024.
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Compuler Vision, 133(5):3059-307) 2025c.
Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 201544,, 2023a.
Jay Zhangjie Wu, Yixiao Ge. Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufci Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7623-7633, 2023b.
Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity. In ICML, 2025.
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. SANA: Efficient high-resolution text-to-image synthesis with linear diffusion transformers. In ICLR, 2025a.
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, et al. Sana 1.5: Efficient scaling of training-time time and inference- time compute in linear diffusion transformer. In JCML, 2025b.
Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1 (111::1991-13999 2023.
Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509 22622, 2025a.
Shuo Yang, Haocheng Xi, Yilong Zhao, Muyang Li, Jintao Zhang, Han Cai, Yujun Lin, Xiuyu Li, Chenfeng Xu, Kelly Peng, al. Sparse videogen2: Accelerate video generation with sparse attention via semantic-aware permutation. NeurIPS, 2025b.
Zhuoyi Yang. Jiayan Teng. Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408..660 2024.
Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.
14

---

Published as a conference paper at ICLR 2026
Fan Zhang, Shulin Tian, Ziqi Huang, Yu Qiao, and Ziwei Liu. Evaluation agent: Efficient and promptable evaluation framework for visual generative models. arXiv preprint arX0v:2412.09645, 2024.
Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. In ICML, 2025a.
Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504. /2626, 2025.
Peiyuan Zhang, Yongqi Chen, Haofeng Huang, Will Lin, Zhengzhong Liu, Ion Stoica, Eric Xing, and Hao Zhang. Vsa: Faster video diffusion with trainable sparse attention. NewrPPS, 2025b.
Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhenghong Liu, and Hao Zhang. Fast video generation with sliding tile attention. ICML, 2025c.
Yifu Zhang, Hao Yang, Yuqi Zhang, Yifei Hu, Fengda Zhu, Chuang Lin, Xiaofeng Mei, Yi Jiang, Zehuan Yuan, and Bingyue Peng. Waver: Wave your way to lifelike video generation. arXiv preprint arXiv:2508. 1576111111111 2025d.
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. 2024.
arXiv preprint arXiv:2412.2044
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modaa model. arXiv preprint arXiv:2408. 11039, 2024.
Daquan Zhou, Weimin Wang. Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11016 2022.
Lianghui Zhu, Zilong Huang, Bencheng Liao, Jun Hao Liew, Hanshu Yan, Jiashi Feng, and Xing- gang Wang. Dig: Scalable and efficient diffusion models with gated linear attention. 2024.
15

---

Published as a conference paper at ICLR 2026
A
LLM USAGE
Our use of large language models (LLMs) was limited to editorial assistance to improve the clarity and readability of this manuscript. Specifically, these tools were used to refine grammar and phras- ing, enhance the logical flow between sections, and condense overly verbose passages for concise- ness. Crucially, all original research ideas, experimental designs, and data analyses were conceived and executed by the authors; the LLM did not contribute to any scientific or methodological content.
B FULL RELATED WORK
B.1 VIDEO DIFFUSION MODEL
Video generation has become a rapidly growing focus in generative AL. Modern approaches typically VAE to compress videos into latent space, where diffusion conditieed on text, images, or both-learns to generate content. Early studies, such as Make-A-Vides (Singer et al., 2022), PYoCo (Ge et al., 2023) and Tune-A-Video (Wu et al., 2023b), adapted text-to-image models with additional temporal layers to enable video generation. Works like, MagicVideo (Zhou et al., 2022), SVD (Blattmann al., 2023a) and Latent Video Diffusion (Blattmann et al., 2023b) played pioneering roles in scaling latent diffusion approaches. However, the limited compression rate of VAEs has hindered their ability to generalize long video sequences. A major breakthrough came with Sora (Brooks et al., 2024), which introduced a temporal VAE to compress temporal dimensions alongside spatial ones, while adopting transformer-based backbone (Peebles & Xie, 2023) at scale. Recent efforts have pushed this framework further. For instance, Wan 2.2 (Wang et al., 2025a) incorporated a sparse MoE architecture that routes different diffusion steps to specialized experts, while VE03 (DeepMind, 2025) extended the paradigm by integrating audio, achieving state-of-the- art performance. The success of MovieGen (Polyak et al., 2024), Seaweed (Seawead et al., 2025), Goku (Chen et al., 2025b), and Waiver (Zhang et al., 2025d) further demonstrates the potential of video generation and its broad impact on practical applications. These developments underscore video generation as of the most dynamic and competitive frontiers in generative Al community.
B.2 AUTO-REGRESSEE DIFFUSION MODEL
Auto-regressive generation dominates the text domain, while diffusion models have become the stan- dard for visual generation. Recent research explores how to combine these paradigms to duplicate the long-term planning capacity of large language models in vision generation. A straightforward solution (Zhou et al., 2024) is to jointly train an autoregressive (AR) model for text and a diffusion model for vision, but this leaves the visual side reliant solely diffusion without benefiting from AR modeling. Inspired by block diffusion (Arriola et al., 2025), several works (Li et al., 2024b; Hu al., 2024; Deng et al., 2024; Ren al., 2025; 2024) explore AR-diffusion hybrids: MAR (Li et al., 2024b) disentangles the two, letting AR predict conditions and diffusion reconstruct tokens; ACDiT (Hu al., 2024) integrates them via block- diffusion with autoregression across blocks, while CausalDiffusion (Deng et al., 2024) extends this to token-level autoregression. Extending these ideas to video is natural since frames form temporal chunks. FAR (Gu et al., 2025) gener- ates each frame autoregressively; MarDini (Liu et al., 2024a) employs an AR planner to provide frame-level conditions, with diffusion recovering pixels for tasks such as video interpolation, video extension, and image-to-video generation. Beyond this, MAGI (Teng et al., 2025) and Skyreel (Chen et al., 2025a) remove the dual-model design, training under the strategy of diffusion forcing (Chen et al., 2024a), where later frames are assigned higher noise levels, thereby enabling infinite autore- gressive inter-chunk prediction and high-quality inner-chunk diffusion generation. More recently, self-forcing (Huang et al., 2025) highlights gap between training (real data diffused with noise) and inference (model-generated conditions), and proposes rollout-based training to align the two, leading to more robust long-term prediction.
B.3 EFFICIENT ATTENTION FOR MULTIMODAL GENERATION.
Diffusion Transformers (DiT) have emerged as the mainstream architecture for visual content gener- ation. Representative models include PixArt-2 (Chen et al., 2023b), Stable Diffusion 3 (SD3) (Esser et al., 2024), and Flux (Labs, 2024), the latter demonstrating the potential of scaling DIT to 12B
16

---

Published as a conference paper at ICLR 2026
parameters for high-resolution image synthesis.. To address the computational challenges of vanilla attention (O(22)); )), various methods have replaced with ehear-complexity mechanisms. For in- stance. DiG (Zhu et al., 2024) uses gated linear attention, PixArt-E (Chen et al., 2024b) designs key-value token compression, while LinFusion (Liu et al., 2024b) involves Mamba-hased structure and SANA (Xie al., 2025a) employs ReLU linear attention approaches to reduce computational overhead. With the rise of video generation, the computational demands of standard quadratic atten- tion have become major bottleneck. To address the high computational cost of 3D video attention, many existing works employ factorized spatial and temporal attention to reduce complexity (Chen al., 2023a; Ho et al., 2022: Wang et al., 2025c; Singer et al., 2022). Other methods reduce at- tention complexity by selectively skipping certain token interactions (Xi et al., 2025; Yang et al., 2025b; Li* et al., 2025; Zhang et al., 2025 Simultaneously, other models, such as Mamba- based architectures (Wang et al., 2025b; Gao et al., 2024), have explored state-space models and complextyy designs for efficient video generation. However, these methods either retain some quadratic complexity due to global self-attention layers or are limited to local attention. In contrast, our model maintains a constant-meme memory KV cache with global attention mechanism, enabling the generation of high-quality, minute- length videos.
C MORE IMPLEMENTATION DETAILS
C.I PIPELINE CONFIGURATION
As detailed in Table 7, our SANA-Video- model supersedes the original SANA (Xie et al., 2025a) architecture, including the diffusion transformer and a small decoder-only text encoder, to best uti- lize the pre-trained text-to-image model's weights. However, we introduce several key modifications to support video generation. We increase the FFN dimension from 5600 to 6720 and the head dimen- sion from 32 to 112 to accommodate 3D RoPE, and we add temporal convolution in the Mix-FFN module to enhance motion performance. To effectively capture latent features from both images and videos, our approach uses different VAES based on resolution. For 480P videos, we leverage a Wan2.1-VAR (Wang et al., 2025a) to prioritize reconstruction quality with a lower compression rate (F8T4C16). In contrast, for high-resolution 720P videos, we fine-tune the DCAE (Chen et al., 2024c) into a more aggressive deep compression autoencoder, DCAE-V (F32T4C32), to facilitate more efficient training and inference. For conditional feature extraction, we follow SANA by using small decoder-only LLM for efficient text processing. For our training strategy, we also employ multi-aspect augmentation to enable arbitrary aspect ratio generation and facilitate image-video joint training, allowing the model to generate both images and videos from a single architecture. The AdamW optimizer (Loshchilov & Hutter, 2017) is utilized with a weight decay of 0.03 and a con- stant learning rate of 5c-5. We use Accelerate FSDP (acc, 2022) for efficient sharded data parallel training. Our final model is trained on 64 H100 GPUs for approximately 12 days.
C.2 DETAILED TRAINING BREAKDOWN
In this subsection, we provide a detailed breakdown of the data scale, composition, and training steps for the three main stages of the SANA-Vidd Video training pipeline. Our sources of video data include public high quality datasets, Pexels and Artlist, as well as internal dataset. We design filtering and captioning pipeline (Appendix H),, and the overall data scale after filtering is O(10M).
Stage 1: VAE Adaptation on Text-to-Image (T2I). This stage aims to adapt the VAE for robust representation learning before tackling the video domain. In this stage, we use approximately 10M internal images and train the model for 20K iterations with a batch size of 512.
Stage 2: Continue Pre-training of T2V Model. This stage aims at transferring learned image knowledge to video generation, establishing initial motion and short-term temporal coherence grad- ually from low resolution (192p) to high resolution (480p 720p). In this stage, we use all the filtered video text pairs from our data pipeline (approximately 10M 5-second length video pairs). The LinearDiT model is trained for 200K iterations with batch size of 128.
Stage 3: Autoregressive Block Training (Long Video Fine- Fine-Tuning). This stage aims at fine- tuning the LinearDiT to generate long videos in an autoregressive manner, enabling few-step and minute-Length generation. In this stage (Sec. 3.4, we first use approximately IM 5-second length short video samples fine-tune the causal linear attention and causal MixFFN for 10K iterations.
17

---

Published as a conference paper at ICLR 2026
Algorithm Block Linear Diffusion Inference with Linear KV Cache
Require: KV cache
Require: Denoise timesteps ffi,..., trr. noise scheduler vp
Require: Number of blocks M
Require: Block-wise diffusion model Go (C#* returns cumulative of state Σ 8, cumulative of key Σ "(K)" and conv cache f)
1: Initialize model output Xe f-
2: Initialize KV cache KV [None, None)
mori=l.....
do
4: Initialize Viy (0,
5: for j T ,...., 1......d.. do
6: Set 6b + Go(zly i KV)
7:
if; j = then
8:
Xo append
Update Cache KV &- GX (f0s O, KV)
10:
else
11:
Sample ~ 1)
Set ri < ( Eイー-1)
13:
end if
14: end for
15: end for
16:
return Xe
Subsequently, we leverage self-rollout long training with few-step distillation (DMD (Huang al., 2025)) for 9K steps to further improve performance and efficiency. The long training uses 200K prompts from VidProM dataset (Wang & Yang, 2024).
Table 7: Architecture details of the proposed SANA- SANe-Video.
Model
Width
Depth
FEN
#Heads
#Param (M)
SANA-Vices Video-2B
2240
20
6720
20
2056
C.3 BLOCK LINEAR ATTENTION DURING INFERENCE
We follows Self- Salf-Forcing (Huang et al., 2025) for autoregressive inference, with the KV cache up- date based on our design (Alg. 1). Specifically, we first initialize KV cache as empty and start to denoise the first block. After fully denoised, the attention state Σ9 S, cumulative sum of keys Ee d(K)T and conv cache f will be stored. For the remaining blocks (e.g., n-th block), they will use the existing KV cache to denoise the latent until clean and then update the cumulative attention state Σ" S and cumulative new cache. Such update leverages the global while keeping the memory constant and small, making the long video generation efficient and effective.
of keys Σd ((K)". Also, cache will be replaced with the
D MORE RESULTS
Please refer to our project page https://wnaassggituuiiooSn/n/// for the qualitative compari- son and our generation results.
D. VAE COMPARISON
In Sec. 3.5, we analyze the differences and performance of various video VAEs. To select the VAE that best suits our small diffusion model, we conducted generalization experiment. We hypothesize that VAE with better reconstruction ability under perturbation will be better fit, the diffusion model's 's output during inference may be slightly different from the clean latent distribution seen during VAE training. Specifically, we add Gaussian noise the encoded latent before decoding
18

---

Published as a conference paper at ICLR 2026
Wanz 1on2.1-1.39
Wan2.2-S5
SANA- SANA-Video -Vieo
Prompt: Photorealistc move scene stil. medium shot flamingo with iridescent pink feathers and elegant saoles omate its neck gracefully curved, leg exlended, highlighting intricate body spotlight illuminates iridescent
performs flawless pirouette
polished wood Background: photorealistic, opulent opera house interoor, slightly blurred. Dramatic, theatrical lighting enhances elegance and drama.
Wan2.1-1.35
Wan2.2-5B
SANA-Video
dark vintage the easel. Suddenly, flames erupt his canvas, black moke rising into a gloomy sky. The background vast green field stretching to blurred horizon, symbolizing creation and destruction. Medium shot, arist in focus amidst the chaos.
Prompt
expansive foggy wilderness, calmly painting
Figure 7: Qualitative comparison among T2V methods. SANA-Vide has comparable motion control and video-text semantic alignment with state-of-the-art small diffusion models.
it, setting a/ = 3t ez, where z ~ (0, I). As the results Table show, our DCAE-V per- forms much more robustly under different noise levels. This demonstrates its superior reconstruction generalization, making the ideal choice for our small diffusion model.
Table Performance comparison of different VAE models on 1000 samples from Panda-70M with different noise perturbation levels.
Model
latent shape
psnr
ssim
Ipips.
Wan2. IVAE (c 0)
16, T/4, H/8, W/8
34.41
0.95
0.01
Wan2.2VAF, (e = 0)
48, T/4, H/16, W/16
35.61
0.96
DCAE-V (E 0)
32, T/4, H/32, W/32
33.25
0.94
0.03
Wan2. IVAE (e 0.1)
16, T/4, H/8, W/8
28.61
0.89
0.06
Wan2.2VAE (e = 0.1)
48, T/4, H/16, W/16
30.12
0.92
0.04
DCAE-V (e 0.1)
32, T/4, H/32, W/32
31.91
0,93
0.04
Wan2. IVAE (c 0.2)
16, T/4, H/8, W/8
24.25
0.78
0.16
Wan2.2VAE (c = 0.2)
48, T/4, H/16, W/16
25.94
0.84
0.10
DCAE-V (f 0.2)
32, T/4, H/32, W/32
29.34
0.90
0.05
19

---

Published as a conference paper at ICLR 2026
SANA-Video SkyReel-22 LTX-Video SANA-Video SkyReel-V2 LTX-Video
Prompt The individual's expression shifts from neutral to intense The camera remains steady, focusing the person's face and upper body.
Prompt majestic brown cow across a dusty field under clear blue sky. The camera captures the animal from low angle motion effect enhances the fluidity of the cow's movement...
slow-
Figure 8: Qualitative comparison among I2V methods. SANA- Video has better motion control and video-text semantic alignment.
D.2 QUALITATIVE COMPARISON
Text-to-Video Generation. We compare the text-to-video generation results with current state- of-the-art small diffusion models W-n2.1-1.3B (Wang et al., 2025a) and Wan2.2-5B (Wang et al., 2025a). As shown in Fig. 7, SANA- Video has comparable semantic understanding, great motion control, and high aesthetic quality.
Image-to- Video Generation. We compare the image-to-video generation results with small dif- fusion models (I.3B). As shown Fig. 8, LANA-Vides has the best semantic understanding ability ("camera re- mains steady" instruction in the first case) as well as the best motion control ("slow-motion effect" instruction in the second case) and moderate motion magnitude (first case).
(HaCohen et al., 2024) (2B) and SkyReelv2-12V (Chen et al., 2025a)
D.3 MORE 12V RESULTS
Our SANA-Video is unified framework that can perform T2I, T2V and 12V with a single model. We visualize the I2V generation results in Fig. 9. The first column is the reference image and the remaining columns are the generated video. Our MANA-Vider can generate semantic consistent and temporal smooth videos based on the first frame.
D.4 INFLUENCE OF MOTION SCORE
As mentioned in our data pipeline (Sec. E), we use the average optical flow value represent the motion magitude, which is called motion score in our paper. The motion score is added to the text prompt to better control the motion. In Fig. 10, compare the impact of motion score in the I2V task, which is more clear with the same reference image. By increasing the motion, SANA- Video generate videos with larger but still consistent motion.
20

---

Published as a conference paper at ICLR 2026
Reference Frame
Generation Frames
A fluffy Ragdoll is stirring pot the stove with wooden spoon...
Two young men stand together bustlngg city street at night, ta laking selfie...
A man is skydiving, suspended mid-air against a backdrop of fluffy...
serene valley unfolds beneath cloudy sky, with river winding...
Figure 9: Visualization of image-to-video generation. SANA-Video can keep consistent with the first frame while generating realistic motion.
Motion30 Motion10
Figure 10: The impact of motion score 12V task. Higher motion score lead larger motion.
D.5 LONGSANA VISUALIZATION
In Fig. 11. provide example of our 1-minute long video generation. LongSANA is able to generate motion consistent and semantically aligned long videos.
D.6 LONG VIDEO EVALUATION
We further conduct quantitative evaluation on VBench-Long (Zhang et al., 2024) to verify the ef- fectiveness of LonASANA in long video generation. We compare LongSANA with previous state- of-the-art art methods on 30-second video generation in Table 9, Our LongSANA achieves the best semantic and total scores. addition, SANA-Viden the fastest method that can generate videos in real-time with 27.5 FPS, demonstrating the efficiency and effectiveness of LongSANA when han- dling long video sequences.
E DATA PROCESSING PIPELINE
To build our training dataset, collect public real and synthetic data and implement multi-stage filtering paradigm. First, we use DySceneDetect (Castellano) and FFmpeg (Developers, 2025) to cut vidcos into single-scene short clips. For each video clip, analyze its aesthetic and motion quality, as well as providing detailed captions. Specifically, the motion quality is measured by Unimatch (Xu et al., 2023) (optical flow) and VMAF (Peng et al., 2025) (pixel difference), and
21

---

Published as a conference paper at ICLR 2026
0-10s
10-20s
20-30s
30-40s
40-50s
50-60s
white Arctic fox gracefully across fallen log dense forest. The fax's fur is pristine white, and moves with agility and purpase, its tail held high. camera follows fox closely, capturing its fluid movements and the serene beauty of the woodland setting. The fox's ears perked up, and its eyes focused ahead, embodying the spirit freedom and wildness.
Figure 11: Long video visualization of LongSANA.
Table 9: Comparison of long video generation methods on the VBench-Long (Zhang et al., 2024) benchmark. All compared methods generate 30s videos for evaluation.
Model
Total Score 1 Quality Score
Semantic Score
FPS
SkyReels-V V2 (Chen et al., 2025a)
75.29
80.77
53.37
0.49
FramePack (Zhang & Agrawala, 2025)
81.95
83.61
75.32
0,92
Self Forcing (Huang et al., 2025)
81.59
83.82
72.70
17.00
SANA-Video
82.29
83.10
79.04
27.50
only clips with moderate and clear motion kept. Furthermore, the average optical flow used as a representation of motion magnitude, injecting into prompt for better motion controllability. Aesthetic quality measured by pre-trained video aesthetic model (DOVER (Wu al., 2023a)) and key frame saturation obtained with OpenCV (Bradski, 2000), where low aesthetic score and over-saturated videos are removed. Finally, we collect approximately 5,000 human preferred high- quality videos based stringent motion and aesthetic criteria. The SFT data is collected with diverse but balanced motion and style categories, which can further improve the overall performance. As shown in Fig. 12, the details of this data processing pipeline are discussed as follows.
Scene Detection and Shot Cut. In the pre-training stage, we focus on generating 5-second short videos with 16 FPS a specific scene. However, the raw videos are commonly long and tains more than one scene. Therefore, we cut the raw videos to small video shots with two steps: PySceneDet (Castellano) split the scenes and FFmpeg (Developers, 2025) split videos into short clips.
Motion Filtering. Our pre-training dataset comes from multiple sources, and each source of data differs not only style but also in motion. Motion that is too fast or too slow degrades the motion performance of SANA-Video. Following Zheng al. (2024), apply Unimatch (Xu et al., 2023)
22

---

Published as a conference paper at ICLR 2026
Public DaOa
Scene Cut
Data Noquis ton
Row Data
Text- Cips
Captioning
Saturation
WIE Embods
Human Swlection
Syrthwic Dats
Figure 12: Data filtering paradigm of SANA-Video
User
You are a vidoo captioning specialist whese goalls generate high quality English promats byrefaning to the detalla of user's input videns. Your complete, expressive, and cattural-anundiigg caption
within video, produce carefully anelyne accunately conseys seans. caption should presenve or original intent and meaning of the video while enhancing its clarity and descriptive richness, Strictly adhere to the formatting of the examples proviced.
User
Task Requirements:
1. You need to deseribe the the main subject of the video datail, incluting thiir applerance, actions, expressions, and the surrounding enviromment.
amphasize
information the input different angles.
3. Your output should convey natural movement attributes, incorporating netural actions related to descr subject
catagory, using simple and direct vorba as much as possible.
4. You should reference the detaited information in the video, such cheracter actions, cloching, backgr ounds, and
amphasioe the
photo.
the output prompt around words. 6. No mattar mattarwhat what languago the usar inputs, you must always output English.
Example the Engliih prompt:
1. Japanese fresh film-style photo of young Fast Asian girl with double bmids sitting by the bost. The girl wears white square collar puff steeve dress, decorated with plects and buttons, She has foir skin, delicate features, and slightly matancholiis eyes, staringii direectllyy at camera. Her hair falls naturally, with bangs covering part of her forehead. She rests her hands
with hints mountains, some dry plants. The photo vintage texture, madium shot aeated portiait.
appearing
relased, background
blurred outdoor
sky,
2. An anime illustration in vibrant painting style of white giri with cat ears holding a folder, showing slightly discatisfled expression. She has long dark purple hair and red eyes, wearing a dark gray skirt and light top with wht bold Chinese characters that
(2hang)) The background with foint outlines some fumiiurre visible. A pink hovers above her head, smooth Japanceo cel shading styla. close up from a sightl elevated perspective.
lightyllow indoor
3. CG game concapt digital art featuring huge arocodile with its mouth wide opan, with trees and thorss growing on and grayis-wwhtt, resembling Its mouth agape, the crocadie reveals pink tongue sharp teoch. The background features
back. The crocadile's and thorny protrusions. dusk sky with some distant trees, giving the overall scane dark and cold atmosphera. close-up up from law angle.
with
4. In the style of an American drama promotional postar, Watter White sits metal foiding chair wearing yellow Fnglish
him, glasses, staring forward, dressed in yellow jumpsult, knaas, exuding a calm and confident demeanor. The backg aund shaws on abanconed, dim factory with light filtering through the windows. There's is notceable grainy texture, A medium shot with straight on close up of the character.
plastic storage baves. "Breaking Red"
by pilas hands resting
Directly ourput the English
cat leaps off ballcory. mid-air jump with its legs extended and flowing behind, from low-anglle view, capturing the moment just as lneves the arige.
Qwen-2.5-VL
Figure 13: An overview of the captioning pipeline.
and Vmaf score the motion of each video. Unimatch can evaluate the optical flow of two given images of the same shape. We select frames from each video every 0.5 seconds, reshape them into 320x576, and calculate the average optical flow over all selected frames. Vmaf, on the other hand, simply computes the pixel difference of two images; the Vmaf over all consecutive frames and normalize them. Due to the variance of different video sources, analyze the motion scale and set the appropriate motion range individually, ensuring data has moderate and clear motion. During pre-training, we also append Motion score: {unimatch value) to the text prompt to help control the motion magnitude of the generated videos (Fig. 10).
FFmpeg (Developers, 2025) to compute
Aesthetics Many Imagee works have proven that high aesthetic data can improve the training efficiency of an image generation model (Chen et al., 2023b). We believe that this also applies to
23

---

Published as a conference paper at ICLR 2026
the video generation model. We use Dover (Wu et al., 2023a) to each video for its aesthetics. Dover produces three different scores: aesthetic score, technical score, and overall score, among which we use the overall score as the filter metric.
Saturation We also observe that some of our data, especially synthetic data and real data converted from HDR to SDR, has unnatural color, appearing in high saturation. To prevent these data from damaging the output quality of SANA-Vides we use OpenCV (Bradski, 2000) to compute satu- ration score of each video. We select frames from each video every 0.5 seconds, convert their color representation from RGB to HSV, where the "S" channel in HSV color representation stands for saturation. By averaging the "S" channel over all pixels and frames, obtain the saturation score of a video. We keep only videos with saturation score lower than threshold set to a reasonable value for each data source.
Captioning (Wang et al., 2025a) shows that LLM rewritten prompts can produce more accurate and detailed prompts within the same distribution, and thus make the model easier to learn, and consequently enhance the model's performance. Moreover, for synthetic data with existing prompts, replacing their prompts with LLM rewritten ones helps reduce the misalignment between the original prompts and their synthetic output. Following (Wang et al., 2025a), we use Qwen-2.5-VL. (Bai et al., 2025b) to caption our data as shown in Fig. 13.
SFT Data For our final stage of SFT training, selected approximately 5,000 high-quality videos based on stringent criteria for motion and aesthetics. The motion requirement is fulfilled by the pres- ence of either distinet object motion, camera motion, both. Ideal object motion is characterized by a moderate magnitude and a clearly focused action that is free from occlusions. Similarly, any camera movement must be stable and smooth, without jittering, maintain 3D consistency. The aesthetic criteria are equally comprehensive. Beyond technical qualities like balanced brightness and natural color, we prioritize videos with appealing overall content and layout, demonstrated by thoughtful composition and engaging subject matter. Following this filtering process, the videos were classified into four motion categories (human activities, animal activities, other objects, natu- ral or urban scenes) and three aesthetic styles (realistic, cartoon, cinematic). This strategic sampling across diverse categories is crucial for ensuring both the model's high performance and the breadth of its capabilities. The influence of fine-tuning on the SFT data is illustrated in Fig. 14, where both the aesthetic details (the eyes in the first example), and the motion realism (the pipe of the second example) will be improved.
w.O. SFT
SFT
Figure 14: Analysis of the influence of SFT, Fine-tuning on the human preferred SFT data can improve the video details and adherence the laws of physics.
WORLD MODEL
Robolks
Drking
MineCroft
Figure 15: Visualization of world model task generation.
24

---

Published as a conference paper at ICLR 2026
We fine-tune SANA- Video several downstream tasks to demonstrate the potential of applying SANA-Video to world model related generation: embodied AI, autonomous driving and game gen- eration.
World Model for Embodied AI. The first important downstream task for video generation is em- bodied AI, where SANA-Vider can be used to generate simulation data for robot training. In this task, we leverage AgiBot (contributors, 2024) as the training data, which contains synchronized views of different camera views. The head-front view is adopted the target videos and filtered with our data pipeline. The generation results are shown in the first row of Fig. 15.
World Model for Autonomous Driving. Video generation model is also good simulator for autonomous vehicle scenarios, and SANA-Vides can be used to generate diverse and realistic driving scenes. In this task, we fine-tune SANA-Vidh on internal driving data, using the front camera with 30 FOV. The generation results are shown in the second row of Fig. 15.
World Model for Game Generation. We explore downstream game generation create interactive video games. Specifically, we use VPT (Baker et al., 2022) as the training data, containing screen recording videos of players playing Minecraft. The raw videos are cut and processed following our data pipeline in Appendix E. In addition, we train small classifier to identify low-quality data in the scenario. The generation results are shown in the third row of Fig. 15.
25


Reviews:
[{"review_id": "ogRfOBpnHH", "reviewer": "Reviewer_Ai4Y", "summary": "This paper presents SANA-Video, an extension of SANA for video generation. The main idea is two-fold: it adopts a Linear Diffusion Transformer similar to SANA, but extends it with Block Linear Attention and a constant-memory KV cache, enabling block-wise autoregressive generation for synthesizing long videos. The authors also propose DCAE-V, a video autoencoder that achieves a much higher spatial compression ratio (32×) than existing VAEs (e.g., Wan-VAE with 8×). In addition, they introduce several techniques to improve training efficiency and effectiveness — such as autoregressive block training with self-forcing, which combines the benefits of self-forcing and a global KV cache. As a result, the model can handle much longer temporal contexts (e.g. 1 minute) while maintaining high generation quality. They also propose a quantization techniques for better deployment.", "questions": "See my weaknesses above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper is generally well-written and easy to follow.\n- Sampling cost has been a really critical issue in video diffusion models, and thus, I think this paper is very well motivated.\n- Compared with existing popular video diffusion models (e.g., WAN), the efficiency improvement is decent while generating similar-quality videos.", "weaknesses": "While I appreciate the strong improvements over existing video diffusion models, I have the following concerns:\n- Technical novelty: The main contributions seem to be extensions or combinations of existing techniques. For example, linear attention was introduced by SANA for text-to-image generation, and block-wise attention has already been employed in several recent long video generation works. I wouldn’t say this warrants rejection, but it also makes it difficult to strongly recommend the paper.\n- Scaling behavior: Given that the training cost of SANA-Video is not prohibitively high, I wonder if the authors could provide scaling-law analysis for the proposed architecture. For instance, if we train a larger model (say, 5B parameters) using the same data and setup, can we expect further improvements over the 2B variant? While efficiency is a strong point, the model quantitatively underperforms larger video models in most metrics. I’m curious whether this is due to the smaller model size or if there are architectural bottlenecks that limit performance even at larger scales.\n- Minor: Figure 1c is great, but it would be even more informative to include a version with full attention blocks for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JLfEPZwu33", "reviewer": "Reviewer_NYaf", "summary": "SANA-Video is a small diffusion model that swaps standard attention for linear attention and adds a constant-memory KV cache with block-wise autoregressive inference. The combo lets it generate minute-long, up-to-720x1280 videos efficiently, with low training cost and fast inference. \nCore contributions: 1. Linear DiT for video: extends SANA's linear-attention DiT to video. 2. Block Linear Attention with a constant-memory KV cache: Reformulates causal linear attention so long videos can be generated block-wise with a fixed KV state and per-token cost. 3. Low cost, competitive results, and practical deployment.", "questions": "Please see weaknesses section.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Speed and efficiency at scale.\nIt replaces vanilla self-attention with linear attention (as in SANA-Image), reducing the quadratic complexity with respect to tokens to (near) linear. This significantly cuts compute for large spatiotemporal token counts and yields faster video generation without heavily inflating parameters. In practice, that means higher throughput and shorter time-to-first-frame on the same hardware.\n2. Long-video generation with bounded memory.\nDuring long-horizon synthesis, a standard KV cache grows with sequence length and quickly exhausts VRAM. This paper proposes a constant-memory KV cache together with block-wise linear attention, enabling block-autoregressive generation that keeps the memory footprint essentially flat while still carrying forward global context. The result is more stable, minute-scale generation on a single GPU.\n3. Competitive quality at lower cost.\nThe training recipe is designed for efficiency (e.g., linear attention, staged curricula), and the inference stack is optimized for speed. Reported latencies are up to 16× faster than comparable small-parameter baselines, while maintaining competitive visual quality and motion consistency. For deployment, this translates into lower operational cost and better user-perceived responsiveness.\n4. Strong text–video alignment at high resolution.\nThe model targets high-resolution outputs (e.g., 720p) and emphasizes faithful grounding to the prompt across long clips. In addition to object/style fidelity, it improves temporal adherence—keeping actions, scenes, and entities consistent over time—so the final videos look coherent rather than drifting from the original description.", "weaknesses": "1. On minute-scale clips, the model often locks into one scenario with minimal variation in composition, lighting, or action. This can be less helpful in real-world use cases (e.g., advertising, storytelling, education), where users expect diversity, pacing changes, and clear scene evolution.\n2. Potential but unverified claim. Intuitively, this approach should help produce higher-resolution videos—even if clip duration must be shortened to keep compute in check. However, the experimental section does not report targeted evaluations or head-to-head comparisons at higher resolutions, so this benefit remains unsubstantiated. One possibility is that linear attention imposes practical limits (e.g., on spatial detail retention, kernel choice, or memory/computation trade-offs) that make rigorous high-res testing less favorable or harder to standardize.\n3. Although the results are impressive, the contribution of this work on the technical side is somewhat limited. Their two main components have been already explored in the literature, linear DiT in [1] and Constant-Sized KV Caches in [2]. So combining them is not really a significant novel idea.\n[1] Liu et. al. LinFusion: 1 GPU, 1 Minute, 16K Image. 2024\n[2] Ravi Ghadia et. al. Dialogue Without Limits: Constant-Sized KV Caches for Extended Response in LLMs. ICML 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n1qjJ3ilGQ", "reviewer": "Reviewer_CMDt", "summary": "This paper introduces SANA-Video, an efficient and lightweight video diffusion model capable of generating high-quality videos up to 720p resolution and minute-level durations on consumer-grade GPUs. Its core designs include:\nLinear DiT Architecture,\nBlock Linear Attention with Constant-Memory KV Cache, \nHighly Compressive Video VAE (DCAE-V).\n\nExperiments demonstrate that SANA-Video achieves 720p video generation speeds 53× faster than Wan2.1-14B and 3.2× faster than Wan2.2-5B, while delivering comparable or superior quality to mainstream compact models (especially in semantic alignment) on VBench.", "questions": "1. The latent channel number of DCAE-V is 32, lower than Wan2.2-VAE's 48. Have you tried training DCAE-V with higher channel number?\n\n2. Training Techniques for DCAE-V: How does DCAE-V achieve such strong reconstruction performance? Are there any special techniques used during training (e.g., dataset or algorithm tweaks)?\n\n3. 480p Inference: When generating 480p resolution videos, do you use DCAE-V or Wan VAE?\n\n4. Concerns on High Compression VAE: Does the high compression rate of DCAE-V impact the generation of fine details?\n\n5. Quantized Deployment: Does color distortion or motion stuttering occur after NVFP4 quantization?\n\n6. Improving Quality: Is it possible to further reduce the quality gap with Wan2.1-14B by scaling up Linear DiT for higher resolutions?\n\n7. Long-Video Generation Performance: How does the model perform on long-duration video generation tasks?\n\n8. The method achieves high semantic alignment scores on the VBench dataset. What could be the reasons for this? Is there a potential trade-off between semantic alignment and generation quality?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Engineering Innovation: The combination of Linear DiT and Block Causal KV Cache is highly practical, effectively solving memory and speed bottlenecks in long-video generation.\n\n2. Low Training Cost: Trained in only 12 days on H100 GPUs, significantly reducing costs compared to similar works (e.g., OpenSora, MovieGen), thereby lowering research entry barriers.\n\n3. Outstanding Inference Efficiency: Generates 720p videos in just 36 seconds on H100 GPUs and 29 seconds on RTX 5090 GPUs (with NVFP4 quantization), showing deployment feasibility.\n\n4. Comprehensive Experiments: Covers a wide range of tasks, including Text-to-Video (T2V), Image-to-Video (I2V), long video generation, ablation studies, VAE comparisons, and quantized deployment analysis.", "weaknesses": "1. Generation Quality Needs Improvement: Quality scores are not high enough.\n\n2. Lack of Detailed VAE Analysis: Details on training and inference for the VAE are missing, along with its performance on high-frequency details—e.g., whether grid-like artifacts are observed.\n\n3. No Open-Source Code or Model: The claimed results and efficiency are hard to verify without access to code or model weights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xEwWAJ9sh9", "reviewer": "Reviewer_M6uh", "summary": "This paper targets efficient, high-resolution, and minute-long video generation and introduces SANA-Video, a 2 B-parameter diffusion model that replaces quadratic self-attention with linear attention and a constant-memory KV cache. Extensive experiments on benchmarks with 480p and 720p  resolution show the great efficiency and performance of SANA-Video. The manuscript is clearly structured, well illustrated.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper addresses a timely and important problem—computational bottlenecks in long, high-resolution video generation—and offers a practical solution deployable on edge GPUs, such as RTX 5090. The authors propose a novel block linear attention with constant-memory KV cache, elegantly combining linear attention and autoregressive training to unlock arbitrarily long videos without memory explosion. Experiments are good, including latency, memory, and the VBench benchmark; the 12-day 64-GPU training cost is convincingly shown to be <1 % of MovieGen while outperforming larger models. The writing is accessible, with intuitive figures, algorithm pseudocode, and some ablation studies.", "weaknesses": "I believe the main weakness of this paper lies in the lack of in-depth ablation studies and analysis regarding the proposed technical innovations. Although Tables 2, 4, and 5 present quantitative comparisons between SANA-Video and existing SOTA models, the paper severely lacks detailed ablations that isolate and validate the contributions of the proposed methods. While Figure 3 and Figure 6 provide a simple comparison of loss convergence, this analysis is limited in two ways. First, they only reflect the change of training loss, which do not adequately capture model performance on validation or test data. Second, loss convergence is not necessarily correlated with the final generation quality of the model to my experience. Therefore, I recommend that the authors conduct comprehensive ablation experiments to validate their claimed innovations. Without such evidence, the paper appears to mainly extend the SANA T2I model to video training. Although this extension may indeed yield a faster and better-performing model for both academia and industry, it lacks sufficiently validated technical insights that could meaningfully inform future research. To better align with ICLR’s emphasis on technical novelty and rigorous validation, I encourage the authors to enrich the paper with more thorough experimental analyses focusing on their proposed innovations. \n\nIn addition, the authors only provide a description of the data filtering strategy but do not specify the detailed information about the datasets used at each stage, including their sources, size, quality, and distributions. This omission raises concerns about the reproducibility of the work. For example, if one is provided with the SANA-T2I model and the exact data preprocessing pipeline, it remains unclear how he could reproduce the reported results in Table 4 using 64 H100 GPUs within 12 days.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)