Training-Free Efficient Video Generation via Dynamic Token Carving
Authors: Yuechen Zhang
Jinbo Xing
Bin Xia
Shaoteng Liu
Bohao PENG
Xin Tao
Pengfei Wan
Eric Lo
Jiaya Jia
Venue: NeurIPS 2025 poster ()
Keywords: Video Generation
Training-Free
Efficient Generation
Diffusion Transformers
Primary area: deep_learning
TL;DR: TLDR: Jenga accelerates video diffusion models up to 8.83× without retraining, maintaining generation quality while reducing runtime from minutes to seconds.

Abstract: Despite the remarkable generation quality of video Diffusion Transformer (DiT) models, their practical deployment is severely hindered by extensive computational requirements. This inefficiency stems from two key challenges: the quadratic complexity of self-attention with respect to token length and the multi-step nature of diffusion models. To address these limitations, we present Jenga, a novel inference pipeline that combines dynamic attention carving with progressive resolution generation. Our approach leverages two key insights: (1) early denoising steps do not require high-resolution latents, and (2) later steps do not require dense attention. Jenga introduces a block-wise attention mechanism that dynamically selects relevant token interactions using 3D space-filling curves, alongside a progressive resolution strategy that gradually increases latent resolution during generation. Experimental results demonstrate that Jenga achieves substantial speedups across multiple state-of-the-art video diffusion models while maintaining comparable generation quality (8.83$\times$ speedup with 0.01\% performance drop on VBench). As a plug-and-play solution, Jenga enables practical, high-quality video generation on modern hardware by reducing inference time from minutes to seconds---without requiring model retraining.

Training-Free Efficient Video Generation via Dynamic Token Carving
Yuechen Zhang'
Jinbo Xing'
Bin Xia
Shaoteng Liu'
Bohao Peng'
Xin Tao
Pengfei Wan
Eric Lo Jiaya Jia
'CUHK 2HKUST
KKuaishou Technology 'SmartMore
Project page: https:// fttts:://julannjaaeee github. httecpp..//ppropcss//eegggg
225 7.20x
Runyuan T2V 8CPt/ 38s 41.
358b/
Arc'ndacy 7thx 2.12
Wan2-1 //34/ 4.70e
Prosuge Two pandas discussing scudemic paper
dennieing timester
fower gencrate
moce tolen, genarate details, sparse amentton
attention
bey volue: detalle preserved, content degraded
Attention can be sparse to generate details
(b) Core idea: reducing token interactions
Figure I: Jenga generates high-quality videos with an efficient DiT inference pipeline. (a): Extremely sparse attention can preserve details in generated videos. (b): We minimize token interactions via dynamic sparse attention with progressive resolution design. We present videos generated by Jenga (sub-sampled 48 frames) among different models, marked with DiT latency and relative speedup rate. Please use Adobe Acrobat Reader for live video visualization.
Abstract
Despite the remarkable generation quality of video Diffusion Transformer (DiT) models, their practical deployment severely hindered by extensive computational requirements. This inefficiency stems from two key challenges: the quadratic complexity of self-attention with respect to token length and the multi-step nature of diffusion models. To address these limitations, we present Jenga, novel infer- ence pipeline that combines dynamic attention carving with progressive resolution generation. Our approach leverages two key insights: (1) early denoising steps do not require high-resolution latents, and (2) later steps do require dense atten- tion. Jenga introduces a block wise attention mechanism that dynamically selects relevant token interactions using 3D space-filling curves, alongside progressive resolution strategy that gradually increases latent resolution during generation. Experimental results demonstrate that Jenga achieves substantial speedups across multiple state-of-the-arr video diffusion models while maintaining comparable generation quality (8.83x speedup with 0.01% performance drop on VBench), As plug-and and-play solution, Jenga enables practical, high-quality video generation
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

modern hardware by reducing inference time from minutes to seconds witthut requirigg model retraining.
1 Introduction
The advancement of Latent Diffusion Models [1, 2, 3, 4, 5, 6, 7] has significantly propelled the development of image and video generation. Recently, Diffusion Transformers (DiT) [8. 9, 10, 11, 12, 13, 14, 15, 16] have emerged as the predominant architecture for foundation models due to their inherent scalability and superior generative capabilities. As high-resolution video generation techniques continue to advance and DiT-based models scale to unprecedented sizes, the computational efficiency of generating high-quality content has become critically important. For example, generating mere 5-second 720P video using Hunyuan Video [12] on single NVIDIA H800 GPU requires approximately 27 minutes, severely limiting its practical applications in real-world scenarios.
This challenge stems from two orthogonal factors: (1) Self-Attention versus massive token length N. The continuously increasing token length for high-resolution generation causes computational bottleneck due to the ( computational complexity of self-attention in Transformers. Even with efficient attention mechanisms [17], self-attention in Hunyuan Video [12] still consumes 77.8% of the total processing time. (2) The multi-step nature of Diffusion models. The denoising process requires forwarding through the DiT architecture times, introducing T-fold computational overhead compared to non-diffusion models [18, 19] of similar specifications.
To address these challenges, various approaches have been explored. One branch focuses on operator-based acceleration, particularly attention optimization, to eliminate computational bot- tlenecks. STA [20], CLEAR [21], and SVG [22] predefine head-aware attention sparsity patterns in temporal or spatial dimensions. However, these approaches inadequately account for dynamic variations in attention patterns across inputs and achieve only modest speedup ratios (1.5-2x), ), insuf- ficient for practical deployment. Orthogonal approaches optimize the diffusion generation pipeline through distillation [23, 24, 25, 26]. quantization [27, 28, 29], or feature reuse [30, 32J. However, distillation incurs significant training costs while often degrading output quality. Similarly, feature reusing and quantization methods also face limitations in achieving adequate acceleration ratios necessary for practical applications.
Based on the two orthogonal factors identified, we propose Jenga, progressive, fully sparse inference pipeline with dynamic, generalizable Attention Carving kernel. Studies have shown that the diffusion denoising process progresses from low to high-frequency generation [33, 34], where earlier steps establish content structures while later steps refine details. The core idea of Jenga is: Early denoising steps do not require high-resolution latents, while later steps do not require dense full attention. Once video content is established, the inherent redundancy in video latents means that not every token must participate in attention computations; at high resolutions, attention inherently sparse, and fine details can be generated without full attention. Accordingly, Jenga designs a device-ffiendll Attention Carving kernel that decomposes latents into contiguous latent blocks using space- ppace-filling curves, and employs block-wise attention to selectively compute key-value pairs, creating an efficient attention mechanism. As illustrated in Fig. 1 (a), video details can be preserved when we only keep 1 1% key-value blocks using Attention Carving.
Generating content layouts does not require huge latent inputs, so we introduce a multi- multi-stage Pro- gressive Resolution (ProRes) strategy that generates video through phased resizing and denoising of latents, effectively reducing the token interactions. Under this strategy, we face the challenge of generating resolution-dependen variations in the field of view that affect content richness. For example, low-resolution generation focuses on zoomed-in details rather than global scenes. To coun- teract this, introduce text-attention amplifier that reduces local neighborhood focus, enhancing condition information utilization, and producing more content-infon cnntenttifformatiee results, which are similar to generating content directly using high-resolution.
As illustrated in Fig. (bb) Jenga is a combination of two complementary techniques: ProRes handles robust content generation with lower resolutions, while Attention Carving processes sparse attention, reducing token interactions. Like optimally arranged real-world Jenga blocks, these techniques deliver efficient, high-quality video generation with high block sparsity. Empowered by Jenga, we achieve impressive results across multiple state-of-the-art DiT-based video diffusion models. For instance, obtain 4.68-8 3.83 x speedup on Hu [12] while maintaining comparable performance

---

on VBench [35]. Similarly, we demonstrate significant acceleration on Hunyuan Video-12V (4.43x), the distilled model VccVideo [25] (2. (2. and Wan2.1 1.3B [13] (4.79x). Further, when deployed on an 8x H800 GPU computing node, Jenga reduces the DiT inference time to 39 seconds for Hunyuan Video and 12 seconds for Video.
Our contributions are threefoid: (1) we propose a novel dynamic block-wise attention carving approach that enables high-efficiency sparse attention computation for video generation; (2) we introduce Progressive Resolution, which decouples the content generation and detail refinement stages, reducing token interactions and achieving further acceleration; and (3) as a plug-and-pii inference pipeline, Jenga achieves unprecedented speedup across various modern video DiT architectures.
2 Related Works
Efficient attention design in Transformers represents a critical research direction focused on mitigating computational demands arising from the quadratic O(N²) complexity relative to the token sequence length N. In Language Models, efficient attention methods like MInference [36], HIP [37, 38]. MoBA [39], and NSA [40, 41, 42, 43] adopt partial or hierarchical key-value selections for efficient long-context understanding. To process dense vision features, efficient attention designs are also adopted in ViT and Diffusion Models, including linear attention [44, 45] and cascade attentions [46]. All these approaches aim to reduce the number of tokens actively participating in attention computations, thereby achieving acceleration and decreasing memory requirements,
Efficient video generation has garnered substantial research interest concurrent with the rapid evolution of video Diffusion Transformers (DiTs) [8, 11, 12, 47, 13]. Early acceleration techniques focused on reducing sampling steps, primarily through step distillation methodologies [25, 26] or training-free approaches that leverage step-wise feature reuse, such TeaCache [31] and RAS [32, 48J. Bottleneck Sampling [49] employs a variable resolution strategy across different sampling stages, thereby utilizing fewer tokens during intermediate computational phases. Complementary to step reduction strategies, various efficient attention mechanisms for DiT's have emerged, including CLEAR [21], STA [20], and SVG [22], which operate on the fundamental assumption of localized attention distribution patterns. While this localization assumption preserves consistent attention structures, inherently constrains the model's capacity for long-range feature aggregation. Recent advancements in block-wise attention architectures, such as Sparge/Attn [50, 27] and AdaSpa [51], implement selective processing based on block-level mean values, achieving approximately two-fold acceleration in video generation pipelines. Nevertheless, their optimization potential remains limited by rigid block partitioning structures and attention sparsity parameters that require further finetune.
3 Jenga: Tofen-Efffcient Optimization for Video Diffusion Transformers
Latent Diffusion Models (LDMs) [1] learns reverse a noise corruption process, transforming random noise into clean latent-space samples. At time step t E (o...., the model predicts latent state z, conditioned on 1f+11 (a (2++1) (z ne (1 t). of where 0 represents the model parameters, Ho denotes the predicted mean, and is the predetermined standard deviation schedule. For Diffusion Transformers [8], during each timestep f, the model processes noisy visual latent tokens Ie together with tokenized conditional embeddings (e.g., text prompt), predicting the noise component c added at that timestep. scheduler [52] then guides the progressive denoising process to compute the next denoised state _ = scheduler(xre, e, f), gradually yielding fully denoised video latent Ti which is then converted back to pixel space with a pre-trained VAE decoder.
The overview of our method is illustrated in Fig. 2. Jenga aims to minimize the computational complexity by reducing the number of tokens processed in each operation within video DiTs [8]. This is achieved through two primary optimizations: (I) enhancing the efficiency of the self-attention mechanism (Sec. 3.1) and (2) streamlining the inference pipeline (Sec. 3.2). In video DiT, we typically process N, numel 2.)) = thw visual tokens, where t, h, and w represent the temporal length, height, and width of the video latent 2g in the latent space, after the visual patch embedding layer, 2, patchemb(zx).
3.1 Block-Wisse Attention Carving
As observed in [20, 50], the proportion of time spent on self-attention operations within transformer forward passes becomes increasingly dominant as the number of tokens grows. The 3D full-attention
3

---

Space tilling
GPU
m threads
Imagination
Lan 730P
Each Mick: fully
penolo
paralai atention
Wileoo
Bioci-vtsee artention
Rest
skop shep
Soan plig
540P
3D Latwat Biock
Partiion
Maak
Black-utte
Block-wtan atlection Zernel
26.) skg
Content
Sampling timeetege
Twetille
Carring
Outaide DIT: Peagesiire Resolutiin
Figure 2: Overview of Jenga. The left part illustrates the attention carving. A 3D video latent is partitioned into local blocks before being passed to the Transformer layers. A block-wise attention is processed to get a head-aware sparse attention is performed. The right part illustrates the Progressive Resolution strategy. The number of tokens and timesteps is compressed to ensure an efficient generation.
masks. In each selected block, dense parallel
Siffer Labett Couitton
Kent Bock
Moan pooling
G(-)
Block attention
tae Ordeciat
Rilling Chiise (SFC)
Racurdie SEC
Impertser Made (hould coleedlted
Canditizn be calourate
Aipreeyy 30 www.ggoooooot be Ealoolined
Re-Ordering
Figure 3: Attention Carving (AttenCarve). Here we illustrate toy example of a x 4 4 latent, where m = 8 latent items form block. Left: The latent 3D re-ordering and block partition via space filling curves (SFC). Right: After the block-wise attention in Eq. (3), Mask, combined with the pre-computed Condition Mask and Adjacency Mask, block-wiss dense attention mask is passed to the customized kernel for device-efficient attention.
construct the Importance
mechanism in video transformers can be represented in its most fundamental form as:
Attention (Q,, K,, Vi) = softmax (Q:KTT/ /V Vd.
where Qi KG, V; E Nxd, represent the query. key, and value features for the attention head respectively. We define d as the embedding dimension, and as the number of attention heads with dy d/h. = N, + N₂ denotes the total number of tokens, comprising N,, vision tokens and N, condition tokens. In the context of video diffusion models, this attention operation incurs significant computational overhead due its quadratic complexity O(N2) concerning the token count across spatial and temporal dimensions.
Due the inherent redundancy in video latents, direct approach to improve efficiency is reduce the number of key-value pairs each query attends to. We adopt block-wise coarse key-value selection method, as shown in Fig. 3. FlashAttention [53, 17] and other GPU-optimized approaches [50. 20] uniformly divide Q and KV into M blocks with = N /M tokens each, corresponding to m parallel threads the attention computation, to compute exact attention results across all A12 blocks through parallel processing. For simplicity, we assume N,, and N, are padded lengths divisible by Our objective therefore to reduce KV pairs the block level. First, to obtain tokens with higher internal similarity within 3D blocks, we reorder the 1D vision tokens 2dww (flattened along thw dimensions) into block-wise order 2bik before subsequent partitioning. The reordering and its inverse process are represented by:
Ablk G (Zun) 4dh (ZmL)
where G(-) represents an index permutation function implemented via the Generalized Hilbert re- ordering [54, 55, 56], a toy example of which is illustrated in the left part of Fig. 3. Compared with vanilla linear hwt ordering, this space-filling curve (SFC) ordering ensures that tokens in proximity within 2bik effectively preserve their 3D neighborhood relationships from the original space. Thus, this approach enables uniform partitioning directly in the flattened dimension when computing attention operations.

---

stage
stage p-1
Tanasig Ship
Cess reuse Tao1
delp dkip slip
And video
ellentous scort
slip skop
Aest
Timestep Skip & Multi Stage Leteat Rescale
Teat Alteltiionnnn Ampliieer
debull mustiction
Direct denotany
Ratd Vaea (TV)
Law
Test attection affect the BOV, rideos see geneested from 3602 initial stage
High
Figure 4: Progressive Resolusion (ProRes). Left: A brief illustration of stage switch and timestep skip. Before the rescale in stage s, we revert the latent clean state f.. then re-noise on the upsampled clean latent. Right & Bottom: We add bias on the video-text attention score, to enable scalable Field of View (FOV) low-resolution content generation.
For K V--llock selection, we build a one-hot block-wise 2D mask B E RMMM for each attention head represent the selection result of the block-sparse attention. It union of three masks, as shown in the right part of Fig. 3: (1) Importance Mask Brop. For importance-
block attention selection, inspired by MoBA [39] from large language models, we employ block-wise mean values compute an attention probability map that roughly identifies which block pairs require attention computation. Specifically, for the reordered inputs, we express the relevance between blocks R using:
softmax /vdc),
where (:) is a mean-pooling operator each block of size m. Then for the i-th query block, we set rate k, and keep the top kM key-value blocks in R. Meanwhile, for each query block, we set constraint to fulfill the cutoff approximate accumulated probability. This means after all &M blocks are selection, we still need to select blocks for some block-head combination with top probabilities, until the accumulated probability meets a cutoff softmax probability threshold p, defined by Je R[]]]] > p. This constraint is set to avoid global context lost, especially for some attention heads to aggregate global information.
(2) Condition Mask. Boadd
fi > No/m Vj N./m), where i.j are mask indices in query- key block dimensions. This means all condition-relater attentions should be fully computed. (3) Adjacency Mask. B Bajaa
(i)a(ii))), which represents whether i-th and j-th blocks are adjacent in the 3D thw space. The adjacency mask is beneficial fixing border artifacts between spatially adjacent blocks. In Jenga, B and Buija are pre-computed, and only determined by the resolution and the partition function g. The final selection array defined as the union of three one-hot masks, Badja.
B Brop J Boond
For the block-wise attention, we skip the computation of indices that BiJ hence achieve an attention complexity O(N'N), in which N' = m E B/M is the average number of selected tokens.
3.2 Progressive Resolution
Block-wise Attention Carving significantly reduces the latency of each DiT forward pass, but since diffusion sampling is an iterative process, compressing the number of tokens at the diffusion pipeline level is also crucial for accelerating generation. Leveraging the coarse-to-fine nature of diffusion denoising [33, 57], we decompose the generation inference process of T timesteps into S stages, starting from low resolution R {t. h, WI, d} and progressively increasing the resolution each stage until reaching the final target resolution Rs {t. h, w, d), where r represents the latent patch size and d is the channel dimension. The stage switch is illustrated in the left part of Fig. 4. At the end of each intermediate stage at timestep f, we predict the clean latent at Ag E RR and resize a higher resolution Rs+1, then re-noised following an approach similar to [49]. The progressive resolution process between stages is defined as:
It-1 (1 o,) U(2)) ozē, where 26 It o,6 E~ 1)
(4)
5

---

Here (()) is a latent upsample function in 3D space, for which we employ area interpolation. é, is the prediction at timestep i, and o is the time-dependent standard deviation in the scheduler [52]. By reducing resolution, the earlier stages involve significantly fewer tokens in inference, while the denoising at higher resolutions ensures the generated videos maintain high-quality details.
Text-Attention Amplifier. Unlike bottleneck-style sampling [49], ProRes determines video content and structure during the low-resolution stage, without preserving the original resolution in the initial stage. While Video DiT generates coherent low-resolution videos, we observe that the Field of View (FOV) degrades with decreasing resolution, effectively transforming ProRes into a super-resolution process on videos with a constrained FOV. We illustrate this phenomenon in Fig. 4, which occurs because tokens at lower resolutions disproportionatel attend to their spatial neighborhoods.
To maintain a stable FOV across resolutions, we introduce a text-attention amplifier with a resolution- dependent bias / that "hypnotizes" the model in the first low-resolution stage by enhancing text- attention weights, thereby reducing the focus on spatial neighborhoods. This concept is illustrated in Figs. 2 and 4. Specifically, when processing vision query block qu and a condition key block ko in attention, the biased vision-condition attention score is calculated as: qukk + 3 where B = -p lo (ungee(R)))))) (R.)//mml(()()) (ns) is computed based on the token count ratio between the current stage resolution R2 and the target resolution Rg, with serving as balancing factor.
Case-Agnostic Timestep Skip. Timestep reduction is one of the most common optimization directions in efficient diffusion pipelines. Methods like TeaCache [31. 32] approximate outputs by caching input features to dynamically determine which steps can be skipped. However, in practical implementation, we observe that TeaCache's skip mechanism is effectively a static timestep scheduler, rather than truly case-wise dynamic step skipping approach. Therefore, we employ fixed timestep skip setting (23 steps, same as TeaCache-fasl that samples more densely at the beginning and end while sampling sparsely in the middle, eliminating the additional computation overhead of TeaCache.
4 Experiments
4.1 Implementation Details.
Settings. Our experiments are primarily conducted on the Hunyuan Video [12] architecture with a 50-step configuration. All generated Hunyuan Video vidcos maintain resolution of 125 x 720 x 1280, corresponding to a patchified video latent size of t x h x w = 32 x 45 x 80, approximately 115K tokens. Unless specified, all experiments are performed on one NVIDIA H800 GPU.
For Attention Carving block partitioning. we employ Generalized Hilbert [54] as G(-) with a block
size of m 128. We implement the Attention Carving kernel using Triton [58] and adopt
progressive top-K selection strategy when computing the importance mask: h = 0.3 at stage 1, and = 0.2 for subsequent stages. The probability threshold is set to p = 0.3. When calculating the adjacency mask Budja- it incorporates a 26-neighborhood provide two basic configurations Base and I (straight 720P) and stages (starting with 540P, 50% steps each stage). We also introduce a 2-stage Jenga-Flash setting, which applies smaller A values in both stages to further enhance efficiency. The balancing factor of the text-attention amplifier is set to p O.5. After timestep skipping, 23 of the original 50 timesteps are retained, while additional steps will be added after the stage-swite process. We adopt TeaCache-stylee style [31] latent reuse, where features are reused before the image unpatchiffy layers. Comprehensive details are provided in Appendix B.
3D latent space. For ProRes stages, corresponding to implementations using
Multi-GPU Adaptation. Our method seamlessly integrates into multi-GPU parallel processing configurations. We have implemented adaptations based on xDiT [59, 60) within our approach using the Hunyuan [12] framework. This enables parallel processing of attention operations across the head dimension, while all operations except patchification are parallelized across multiple GPUs along the token dimension. Utilizing an 8-GPU parallel configuration, Jenga-Flash Flash achieves a further 6.28% x speedup (245s + 35s) with identical computational operations, which is also 5,8x faster than the official 8-GPU implementation in Hunyuan Video [12]. Detailed latency results are shown in Tab. 2b. We provide specifications of this implementation in Appendix B.2.
Distilled Model & Image-to- Video. Jenga demonstrates considerable generalizability across diffu- sion model architectures. It not only achieves substantial acceleration ratios in Text-to-Vinn Video (T2V) models [12, 13], but our adaptive attention carving technique can also be effectively implemented in models refined through step-distillation [25, 26] with 3 speedup. Furthermore, when applied

---

Frongt Antivated Suture cloee-op op durry moneer inoelng badice meelin candle
diadee
Pisepe A petni ditt with bamass freed growing withis has tay rnd pondas nisning srwind.
Froeepe ennggartuuuc
octopoc Higgeat cmnaling
(A) Visual Comparttons
foloww behand
survarded
Nee 2454
Dirsiler TOV
Teuge Suags,
Enamples ef Settings
Figure 5: Qualitative comparisons. (a): Jenga maintains strong semantic performance while producing high-quality videos. (b): Examples across multiple Jenga settings, also demonstrate how the text-amplifier stabilizes Field of View (FOV) across different initial resolutions.
to Image-to- Video (I2V) models [12], our approach achieves 4.43x speed improvement in I2V generation [12] tasks even without employing ProRes. Detailed results are shown in Tab.
Evaluation and Metries. For speed assessment, we report the diffusion time consumed specddiicellllll the DiT forward pass time-ae as the VAE decoding component remains constant across all con- figurations. We also report FLOPs and step-wise FLOPs to provide an intuitive comparison of computational complexity. For qualitative evaluation, we employ the widely adopted Cl.IP-based metric CLIPScore [61] to measure text-video alignment, and utilize the comprehensive benchmark suites VBench [35] and VBench-12V [62] with their original full-set prompts. We evaluate each prompt with fixed random seed to ensure both evaluation consistency and statistical reliability. Additionally, we conducted a user study to assess human preference rates between Jenga and various efficient generation baselines, including direct comparison with vanilla inference.
4.2 Comparisons
Attention Efficiency. We benchmarked our Attention Carving (AttenCarve) approach against state- of-the-art training- free attention optimization methods, specifically MInference [36], CLEAR [21], and SVG [22], shown in Tab. 1. From a theoretical perspective, CLEAR (3D local window) and SVG (spatial-temporal windows) can be viewed specialized instances of our more general Jenga framework. establish robust block-selection baseline, we adapted MInference [36] for video generation by removing causal masks and modifying selection optimizations. Jenga's 's dynamic block selection mechanism more effectively identifies crucial key-value pairs in video content while preserving important local information aggregation. Consequently, AttenCarve achieves superior acceleration ratios (2.17x) with reduced computational requirements while maintaining higher generation quality, particularly in terms of semantic adherence, compared to existing approaches.
Sampling Efficiency. We further compared our Progressive Resolution (ProRes) approach with TeaCache [31] in Tab. 1. We observe that ProRes and timestep skipping represent orthogonal solutions that address different aspects of efficient sampling. By incorporating ProRes, we achieve a significant reduction in step-wise FLOPs while maintaining high-quality video outputs. Qualitative evaluations confirm that our Progressive Resolution strategy effectively preserves generated video quality while substantially improving computational efficiency (3.28x x speedup).

---

Table Evaluation results on Hunyuan Video [12]. We report evaluations of the baseline (row 1). attention optimization methods (row 2-4), pipeline optimization methods (row 5-8), and the combined results of Jenga (row 9-11). Here VBench-Q and VBench-S stand for Quality and Semantic metrics in VBench [35]. Best and the second best scores are highlighted.
Computation Loads
Quality Evaluation [35, 61]
Latency Speed
Methods
NFB
PFLOPs)
PFLOPS/ seps
VBench
VBench- Q1
VBench St
CLIP score,
DIT time,
Speedup
HunyuanVideo [12]
50
534.44
10.68
82.74%
85.21%
72.84%
30.67
1625s
1.00x
CLEAR (=33) (21]
479.97
82.68%
86.06%
69.17%
1848s
0.89
MInference 136 Iooeomal
50
187.79
3,76
83.36%
85.41%
75.16%
30.73
815s
x
SVG [22]
50
243.36
4.86
83. 11%
85.87%
72.07%
30.63
988s
1.64x
AttenCarve
50
163.04
3,26
83.42%
85.31%
75.85%
30.60
748s
2.17x
TesCache-slow [31]
31
331.35
10.68
82.53%
85.64%
70.09%%
30.42
9677
1.68X
TeaCache fast [31]
23
245.84
10.68
82.39%
85.51%
69.91%
30.39
703s
2.31%
ProRes
50
353.21
82.85%
86.20%
69.43%
1075a
1.51
ProRes-timeskip
24
162.29
6.76
82.57%
85.78%
69.73%
30,13
495s
3.28
Attenarrve +
84.65%
76.98%
83.12%
485s
3.35x
Jenga-Base
23
75.49
3.28
83.34%
85.19%
75.92%
30.59
3478
4.68
Jenga-Tuue
24
47.77
1.99
83.07%
84,47%
77.48%
30.78
225s
7.22x
Jenga-Flush
24
32,97
1.37
82.73%
84.01%
77.58%
30.77
184s
8.83x
Table 2: Model adaptation & parallel computing. All latencies are DiT forward time. We evaluate VBench [35] T2V models and VBench-I2V [62] for I2V models.
(a) Jenga HunyuanVideo-127 Video-12V [12] (row 1-4) and I f13] (row 5-7), while we report timestep skip result as the efficiency baseline.
(b) Jenga on distilled model [25] (row 1-4) and multi- GPU inference (row 5-7). For multi-GPU, benchmark results are the same as Jenga-Flash in Tab. 1.
NFF
VBench
latency
speedup
Methods
GPU VBench
CLIP
latency
speodup
HunywanIZV [12]
50
87.49%
1499s
1,00X
AccVideo [25]
83.82%
31.23
161s
1,00x
+ TimeSkip
23
87.67%
720s
2,08
Jenga
83.29%
31.12
51s
3.16x
+ Jenga
23
87.75%
338s
4.43x
Jenga-8GPL
8
83.29%
31.12
76
23.00x
1-.2.1-1.38 [13]
50
83.28%
115s
1,00
GPU spoddap
279x
4 10x
8 8xx
VBench
+ TeaCache fast [31]
15
82.63%
34s
3.48%
Hunyuan [12]
1625s
844s
440s
225s
82.74%
Jenga
82.52%
17s
6.52x
Jenga-Flach
107s
82.73%
Visual Quality
21.8% 8%%
66.0%
25.7%
59.0%
14.0%
41. 11.
47.9%
10.33
Semantic Quality
35.95
M0.8%
2.58
21.05
13.0%
43.0%
51.1%
5.08
Overall Quality
8F
52.0%
10.2%
25
$8.05
14.0%
$3.05
48.0%
0.8%
Junge Seae (347a) va HanyaanVodee (825a)
Flush (2354) TeaCoohe Out (713k)
Jenge Elaak (225))
SVG
(905g)
Tetter Same Wore
6.50
Figure 6: User study. We report pair- wise preference rates for visual, semantic, and overall quality.
Qualitative Evaluation. By orthogonally combining AttenCarve and ProRes with different stage configurations, we developed three variants of Jenga: Jenga-Bat Base (1-stage), lenga-Turbo (2-stage), and Jenga-Flash (2-stage, higher sparsity). These variants effectively balance generation quality and speed, achieving 4.68-8.82x acceleration while maintaining high-quality outputs. Notably, both Jenga-Base and Jenga-Turbo surpass the baseline on VBench [35] metrics, with particularly significant improvements in the semantic score (72.84% - 77.48%). This demonstrates that our approach not only accelerates inference but can also enhance the semantic coherence of generated videos. It is worth highlighting that when combining AttenCarve with timestep skipping alone Jenga-Bass)) our quality metrics were not negatively affected. Jenga's 's focus on key information selection improves semantic performance. We provide visualization cases in Fig. 5. Mcanwhile, our static case-agnos agnostic timestep skip schedule performs similar behaviour to the TeaCache in both Hunyuan Video and Hunyuan VIdeo-I2V [12]. Results are reported in Tabs. 1 and 2a.
User Study. We conducted a user study employing the standard win-rate methodology to evaluate our approach. Questionnaires were constructed, each containing 12 randomly selected videos generated using Sora prompts [63]. The videos were presented in randomized order, and participants were asked to evaluate them along three dimensions: visual, semantic, and overall quality. We collected
8

---

Table 3: Ablation Studies. All latencies are DiT forward time.
(a) Cutoff probability and multi-stage selection rates,
(b) Block partition & block selection masks (left and right are two separate tables, row 1-3), stage numbers (row 4- 7), and text amplifier bias (row 8- 8-10). VB-QU BB-Q/S represents VBench Quality and Semantic.
We report VBench latency results. The second col- umn and the head row represent drop rates in the first and second stages, respectively.
select k
0.3
0.2
0.1
partition
VBench
laiency 229s
selection w/o Bant w/o Buis
VBench
lalency
0.4
82.70% / 283
82.59% / 242s
82.35% / /166
hwt linear
82.82%
81.82% 82.42%
221s
0.5
0.3
82.98% / 266s
82.75%
/ 232s
82.43% /20s
SFC
83.07%
225s
222s
83.07% 253s
82.89% 222s
82.60%
stage number
VBench
VB-Q
VB-S
latency
speed
0.4
82.90% / 277s
82.61%
82.61%
205
(T20P)
83.34%
85.19%
75.92%
347s
4.68 x
82.87% 262s
83.07%
225s
82.96%
195s
(500-7005
83.07%
84.47%
77.48%
225s
7.22%
0.2
82.88% / 252x
82.85%
82.73%
184s
80.53%
81.66%
76.00%
157s
10.35x
0.4
82.60% / 260s
82.47%
2376
82.42%
/205s
bias fuctor
-0.5
0.0
0.5
1.0
1.5
82.87% 261s
82.85%
2276
82.8 84% 116s
VBench
82.06%
82.40%
83.07%
82.87%
82.80%
0.2
83.01 248s
82.85% 2222 82.67% 183s
CLIP score
30.32
30.60
30.78
30.94
31.05
(c) Ablation study on mask selection strategy. We analyze the contribution of different mask components overall performance. The "No Mask" baseline uses
(d) Ablation studies on key hyperparameters. (Top): Pro- gressive resolution design with varying low-res step ratios. (Bottom): Cutoff probability threshold analysis.
ProRes and timestep skip with full attention.
mask type
Bup
Bane
Bajs
VBench
latency
speed
Res Step
10%
30%
70%
90%
No Musk (FA)
Full
Full
Full
82.57%
495s
1.00X
Traj-Tmmsteep
987
947
883
767
437
Topk only
Part
81.35%
220s
2.25x
VRench
82.36%
82.35%
83.07%
81 11.03%
78. 78.74%
TopK, SPC
Part
81.41%
220s
2.25x
Latency
286s
253s
225s
207s
169%
Prob. Constraint
81.87%
223s
2.22x
wio Adjacency
82.42%
222s
2.23x
Cutoff Probability
0.0
0.3
0.7
0.9
w/o Condition
<
81.82%
221s
2.24x
VRench
82.85%
83.07%
82.75%
92.8
82.59%
Importance
77.41%
140h
3.54x
Latency
227s
225s
232s
271s
330s
All Mask
83.07 %
225s
2,20X
Effective Sparsity
80.3%
80.1%
78.5%
72.3%
57.5%
total of 70 completed feedback forms, with results presented in Fig. 6. The findings demonstrate that our method is perceptually indistinguishable from multiple efficient generation baselines [22, 12, 31] when subjected to human evaluation.
4.3 Ablation Study and Discussions
To rigorously validate the effectiveness of proposed method, we conducted comprehensive ablation studies on both Attention Carving and Progressive Resolution, with results in Tab. 3.
Attention Carving. As shown in Tab. 3a, we ablated selection rates k and truncation probability p. Our results demonstrate robust performance even with a smaller (82.73% for 0. 1-02 selection rate). The findings reveal a gradual decline in both latency and generation quality selection rates increase in the second stage, while & in the first stage has minimal impact on latency. The probability constraint enhances global information gathering, illustrated in Fig. 7, but large cutoff' value (p = 0.5) disrupts the selection balance among attention heads, leading to slight performance degradation. Tab. 3b ablates ritentrreorder and block selection strategies. Our experiments revealed that conventional linear partitioning can introduce shift artifacts in videos. Furthermore, this scanning approach disregards locality and consequently requires more blocks than space- filling curve (SFC) partitioning, resulting in marginally increased latency. Fig. 7 and Tab. 3b also validate the effectiveness of incorporating the adjacency mask Budja and condition mask Beond demonstrating their necessity.
For cutoff probability analysis in Tab. 3d, lower values (0.0-0.3) achieve higher effective sparsity (80.3%-80. 1.0...1... (6) by selecting the most important blocks, while higher values approach full attention behavior. The effective sparsity significantly exceeds theoretical selection rates, revealing the inherently local nature of video attention patterns.
Mask Selection Strategy. Tab. 3c provides comprehensive analysis of different mask components' contributions to overall performance. The results demonstrate that while the importance mask B alone achieves significant speedup (2.25x), incorporating probability constraints and space-filling curve (SFC) ordering provides marginal improvements. The condition mask B and adjacency mask Badja prove essential for maintaining generation quality, with their removal causing noticeable performance degradation. Notably, removing the importance mask entirely leads to substantial quality

---

Accumnllate
Allloct
mode
Gradient
Figure 7: Qualitative results for ablations. Left: Missing Adjacency Mask Bdjja causes grid effects on block borders. Right: Cutoff probability p helps gain global contexts.
loss (77.41% % vs. 83.07%), confirming its critical role in preserving video generation fidelity while enabling efficient sparse attention.
Progressive Resolution. The ablation studies presented in Tab. 3b demonstrate the effectiveness of our multi-stage approach. We found that a 2-stage configuration maintains strong generation quality, while increasing to = stages introduces some quality degradation due to latent alignment challenges. Nevertheless, the 3-stage variant still delivers satisfactory quality while achieving 10.35x speedup. Additionally, we evaluated the impact of various text-attention amplifier scales on generation quality. As shown in Tab. 3b, excessively high amplifier values introduce more global context and a shift in softmax distribution, resulting in some quality reduction. However, appropriately scaled amplifiers enhance content richness without compromising generation quality.
Resolution Scheduling. Tab. 3d reveals key insights about trajectory- resolution scheduling. The denoising trajectory serves as guidance for optimal resolution scheduling, with the 50% low- resolution step configuration achieving the best balance between quality (83.07%) and efficiency (225s). We observe dramatic quality degradation when low-resolution steps exceed 50%, while too few low-resolution steps also reduce quality, validating that low-resolution content generation requires higher attention density to establish proper structure.
Limitation Analysis & Future Works. While Jenga demonstrates compelling efficiency gains, some limitations remain. Foremost is maintaining latent alignment during resolution transitions- latent resizing offers computational advantages over pixel-domain operations (after VAE processes), but occasionally produces boundary artifacts. We found that these artifacts can be mitigated using detailed and comprehensive prompts. Our current implementation employs non-adaptive SFC block partitioning without leveraging semantic context for token importance, presenting clear improvement opportunity. Future work could integrate learnable attention carving strategies during training rather than applying them post-hoc, potentially yielding optimal token selection while preserving Jenga's s efficiency benefits. Detailed limitations are discussed in Appendix C.I.
Conclusion
In this paper, we introduce Jenga, training-free free inference pipeline that addresses computational bottlenecks in DiT-based video generation by dynamically managing token interactions. Our approach combines block-wise Attention Carving with Progressive Resolution, effectively decoupling content generation from detail refinement to significantly reduce computational complexity while preserving generation quality. Extensive experiments demonstrate substantial speedups up to 8.83x across leading models, including text-to-vine viveo,, image-to-vee video, and step distilled models. As plug-an and -nd--pay solution requiring no model retraining, Jenga represents a significant advancement toward making high-quality video generation more practical and accessible for real-workes applications.
10

---

Acknowledgements This work was supported in part by the Research Grants Council under the Areas of Excellence scheme grant AoE/E-601/22-R, This work is partially supported by Hong Kong General Research Fund (14208023, 14206825), Hong Kong AoF/P-404/16 18, and the Centre for Perceptual and Interactive Intelligence (CPIT) Ltd under InnoHK supported by the Innovation and Technology Commission.
References
[1] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. CVPR, pages 10684-10695, 2022. 2,
[2] Patrick Fsser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873-12883, 2021. 2
[3] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, al. Photorealistic text-to- image diffusion models with deep language understanding. NeurIPS, 35:36479 364944, 2022. 2
[4] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33:6840- 6851, 2020.
[5] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang. Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. Improving image generation with better captions. Technical report, OpenAI, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. NeurlPS, 34:8780-8794, 2021.
[8] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pages 4195-4205, 2023.
3
[9] Xin Ma. Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan- Bang-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. TMLR, 2025.
[10] Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang. Easyanimate: A performrfomm long vidco generation method based transformer architecture. arXiv preprint or Xiv:2405. 18991, 2024. 2
[11] Zhuayi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvidcox: Text-to-video diffusion models with an expert transformer. arXiv preprint 2oXx::2408.06722 2024. 2,
[12] Weijie Kong, Qi Tian, Zijian Zhang. Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint wxXix:2212.000 03603, 2024. 2. 3, 6, 7, 8, 9, 25, 26, 27, 31
[13] Ang Wang, Baole Ai, Bin Wen, Chaojic Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 14, 2025, 2, 3, 6, 8, 25.31 31
[14] Black Forest Labs. Flux. https: / /gtgttgtttt//////////gg/.... http://hhh..ccmmobaaooooooeeeeee
fwwerooreotoeooor Sobresttlabs/Flle ffxu, 2024. 2
[15] Jinho Xing, Long Mai, Cusuh Ham, Jiahui Huang, Aniruddha Mahapatra, Chi-Wing Fu, Tien-Tsin Wong. and Feng Liu. Motioncanvas: Cinematic shot design with controllable image-to-video generation. arXiv preprint X50: 04299, 2025. 2
[16] Yuechen Zhang, Yaoyang Liu, Bin Xia, Bohao Peng, Zexin Yan, Eric Lo, and Jiaya Jia. Magic mirror: Id-preserved video generation in video diffusion transformers. arXiv preprint arXiv:2501.0300 2025. 2
[17] Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.086)/ 2023. 2, 4, 23, 26, 29
[18] Ian Goodfellow, Jean Pouget-Abade Mehdi Mirza, Bing Xu, David Harde-Farleyy Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139 144, 2020. 2
11

---

[19] Minguk Kang, Jun-Yan Zhu, Richard Zhang. Jaesik Park. Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In CVPR, pages 10124-10134, 2023.
[20] Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhenghong Liu, and Hao Zhang. Fast video generation with sliding tile attention. arXiv preprint S00i::2502.0450077 2025. 2, 4, 30
[21] Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Clear: Conv-like linearization revs pre-trained diffusion transformers up. arXiv preprint arXiv:241..66112 2024. 2, 3, 7, 8, 26, 30, 31
[22] Hancheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity, arXiv preprint arXiv:2502.01776, 2025. 2, 3. 7, 8, 9, 26. 29,
[23] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. JCML, 2023. 2
[24] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In CVPR, pages 14297-14306, 2023.
[25] Haiyu Zhang, Xinyuan Chen, Yaohui Wang, Xihui Liu, Yunhong Wang, and Yu Qiao. Accvideo: Acceler- ating video diffusion model with synthetic dataset. arXiv preprint arXiv:2503. 29.2225033.942,, 2025. 2, 8, 25, 26,
[26] Hangliang Ding, Dacheng Li, Runlong Su, Peiyuan Zhang, Zhijie Deng, lon Stoica, and Hao Zhang. Efficient-vdit: Efficient video diffusion transformers with attention tile, 2025. 2, 3,
[27] Jintao Zhang, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen, et al. Sageattention: Accurate 8-bit attention for plug-ande plug-and-play inference acceleration. arXiv preprint arXiv:2410.02367, 2024, 2,
[28] Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang. Zhen Dong, Daniel Kang. Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In ICCV, pages 17535-17545, 2023. 2
[29] Chenglin Yang, Celong Liu, Xueqing Deng, Dongwon Kim, Xing Mci, Xiaohui Shen, and Liang- Chen, 1.58-bit flux. arXiv preprint arXiv:2412. 18655,, 2024. 2
[30] Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Accelerating diffusion models through block caching. In CVPR, pages 621 1-6220, 2024. 2
[31] Feng Liu, Shiwei Zhang. Xiaofeng Wang. Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time to cache for video diffusion model. arXiv preprint 1rXiv:2411.19108, 2024. 2, 3, 6, 7, 8, 9, 26, 30,31 31
[32] Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjic Chen, and Linfeng Zhang. From reusing to forecasting: Accelerating diffusion models with taylorseers. arXiv preprint arXiv:2503.0692.. 2025. 2, 3,
[33] Chenlin Meng. Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Jun Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. ICLR, 2022. 2,5
[34] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen Or. Prompt-to- prompt image editing with cross attention control. ICLR, 2023. 2
[35] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang. Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, pages 21807-21818, 2024. 3, 7, 8, 31, 34
[36] Huiqiang Jiang, Yucheng Li. Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir Abdi, Dongsheng Li, Chin-Yew Lin. Minference 1.0: Accelerating pre- pre-filling for long-context Ilms via dynamic sparse attention. NeurlPS, 37:52481- 15, 2024. 3, 7, 8, 23, 24, 26, 31
[37] Heejun Lec, Geon Park, Youngwan Lee, Jina Kim, Wonyoung Jeong, Mycongjac Jeon, and Sung Ju Hwang. Hip attention: Sparse sub- quadratic attention with hicrarchical attention pruning. arXiv e-prints, pages arXiv 2ily-240,2222 2024. 3
[38] Heejun Lee, Geon Park, Jaduk Suh, and Sung Ju Hwang. Infinitehip: Extending language model context to 3 million tokens on a single gpu. arXiv preprint arXiv:2502.
2025. 3
[39] Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong. Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, et al. Moba: Mixture of block attention for long-contex 1lss. arXiv preprint arXiv:2502.13189 2025. 5,
12

---

[40] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. ICLR, 2020. 3
[41] Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models. ICLR, 2023. 3
[42] Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. Quest: Query-aware sparsity for efficient long-context Ilm inference. arXiv preprint arXiv:2406. 10774, 2024.
[43] Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang. Zhiping Xiao, et al. Native sparse attention: Hardware- aligned and natively trainable sparse attention. arXiv preprint arXiv:2502. / 1089, 2025. 3
[44] Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolttion image synthesis with linear diffusion transformers. arXiv preprint arXiv:2410. 10629, 2024.
[45] Dongchen Han. Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. Flatten transformer: Vision transformer using focused linear attention. In ICCV, pages 5961-5971, 2023. 3
[46] Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuging Yang, Han Hu, and Yixuan Yuan, Efficientvit: Memory efficient vision transformer with cascaded group attention. In CVPR, pages 14420- 14430, 2023. 3
[47] Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming. Xiaoniu Song, Xing Chen, et al. Step-video-12v technical report: The practice, challenges, and future of video foundation model. arXiv preprint arXiv:2502. 10248, 2025. 3
[48] Ziming Liu, Yifan Yang, Chengruidong Zhang, Yiqi Zhang, Lili Qiu, Yang You, and Yuqing Yang. Region-adaptive sampling for diffusion transformers. arXiv preprint ar0iv: 10389, 2025. 3
[49] Ye Tian, Xin Xia, Yuxi Ren, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Yunhai Tong, Ling Yang, and Bin Cui. Training-free diffusion acceleration with bottleneck sampling, arXiv preprint arXiv:2503. 18940, 2025. 3, 5,5, 6, 23
[50] Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. arXiv preprint #xXi:25022.88133 2025. 3, 4, 29
[51] Yifei Xia, Suhan Ling. Fangcheng Fu, Yujie Wang, Huixia Li, Xuefeng Xiao, and Bin Cui. Training- free and adaptive sparse attention for efficient long video generation. arXiv preprint arXiv:2502.21073 2025.
[S2] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. ICLR, 2023. 3,6 6
[53] Tri Dao, Dan Fu. Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. NeurIPS, 35:16344-16359, 2022. 4
[54] Jakub Červený and contributors. Gilbert: Space-filling curve for rectangular domains arbitrary size. fttps://ggiee tht.bb..hhu.oog com/ jakubcerveny gilrtr,, 2025. Accessed: April 16, 2025. 4, 6, 24
[55] Hans Sagan. Space-filling curves. Springer Science & Business Media, 2012.
[56] Xiaoyang Wu, Jiang, Peng-Shua Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang. Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In CVPR, pages 1840-4851) 2024.
[57] Shilong Zhang, Wenbo Li, Shoufa Chen, Chongjian Ge, Peize Sun, Yida Zhang, Yi Jiang, Zehuan Yuan, Binyue Peng, and Ping Luo. Flashvideo: Flowing fidelity to detail for efficient high-resolution video generation. arXiv preprint arXiv:2502.05179 2025. 5
[58] Philippe Tillet. Introducing triton: Open-source gpu programming for neural networks. ht https:// openai...
6
[59] Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. xdit: an inference engine for diffusson transformers (dits) with massive parallelism. arXiv preprint arXiv:2411.01733 2024. 6, 26
[60] Hao Liu, Matci Zaharia, and Pieter Abbecl. Ring attention with blockwise transformers for ncar-infinite context. arXiv preprint arXiv:2310.0r8 10.01889, 2023. 6
[61] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. EMNLP, 2021. 7.8
13

---

[62] Ziqi Huang, Fan Zhang. Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si. Yuming Jiang, al. Vbench++: Comprehensive and versatile benchmark suite for video generative models. arXiv preprint urXiv:2411.13503 2024. 7.8
[63] OpenAL. Video generation models world simulators.
8
[64] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:12706; 2024. 22, 29
[65] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. JCML, 2024.
[66] Juechu Dong, Boyuan Feng. Driss Guessous, Yanbo Liang, and Horace He. Flex attention: programming model generating optimized attention kernels. arXiv preprint arXiv:2412.05496 2024. 26
[67] Jumxian Guo, Haotian Tang, Shang Yang. Zhekai Zhang, Zhijian Liu, and Song Han. Block Sparse 2024.
Attention, https:/ /ttppsh.///ttt.b.. http///gttaaaa...m......o www.ai...oomiitteeeeeee han 1ab/ BBook- Apparse-Attento
[68] Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe Song, and Zhanyu Ma. Demofusion: Democratising high-resolution image generation with no CVPR, pages 6159-6168, 2024. 27
[69] Zhen Yang, Guibao Shen, Liang Hou, Mushui Liu, Luozhou Wang, Xin Tao, Pengfei Wan, Di Zhang, and Ying-Cong Chen. Rectifiedhr: Enable efficient high-resolution image generation via energy rectification. arXiv preprint arXiv:2503.000 025337, 2025. 27
[70] Revital Dafner, Daniel Cohen-Or, and Yossi Matias. Context-based space filling curves. Computer Graphics Formm, volume 19, pages 209-218. Wiley Online Library, 2000. 28
[71] Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision. NeurIPS, 37:66658666655, 2024. 29
[72] Alexandros Stergiou and Ronald Poppe. Adapool: Exponential adaptive pooling for information--rtaining
downsampling. T7P, 32:251-2666 2022. 29
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction part claim the paper's 's scope (efficient video
generation) and contribution training-free efficient video generation method).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification: The paper do discuss the limitation of the work, and will provide an analysis in the Supplementary.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper is not discussing theoretical method, most of the related theoreti- cal result are correctly referenced, the novel part of the method also correctly explained with assumptions (scopes) noted.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
16

---

Justification: We includes the core part of our method with implementation details in both the main paper and the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We will open-source the code as committed.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nips) cc/
public/ guides / /CooeSubmiii ss ssionPolicy) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new spen-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. cc/
Cete//odeeuubaa isimiissionn innPooPoll oonPllccy) liycy) for more details. The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
17

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We specified all details in the main paper and the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropriate
information about the statistical significance of the experiments?
Answer: [No]
Justification: Due to the page limit and computation cost, we do not include error bars the main paper. We will report the error bar mainly in the latency results in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, report detailed computer resources.
18

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /re:://neuur /neurrpuu...p// cc
Answer: [Yes]
Justification: Yes.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the social impacts in the supplementary.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
19

---

Answer:
[NA]
Justification:
Our method is training- ffrenn free and is based on the current open- sourced models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers s w thoode. com/ da t asst has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: The method is training-frees free.
Guidelines: The paper does not release assets.
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
20

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification:
We report the instructions and screenshots the user study in the supplemen-
tary.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //tp/r/eeippp///e/ cc/ LLM) for what should should not described.
wttplppp.....ooooee ccc//onfeeences /205/
2025/
21

---

Appendix
Training-Free Efficient Video Generation via Dynamic Token Carving
Overview of the Supplementary Material
HTMML /indes.hmm
T
Appendix File (This PDF)
HTML website
batp::/julianjuaneee. wwt.hh..ioooooocttetteoeoeeeeeee projec/ssjietceennnnnee. (open your browser)
This appendix contains extended discussions, technical details of Jengs, and additional qualitative and quantitative that supplement main pager.
contains extensive video results, comparisons, ablation studies, and limitation analyses that cannat be fully represented document.
It consiets 4 parts.
A. Technical Details
C. Discussions and Analysis
Are there any trikks used this method?
An algorithmic emplementation is provided help
Limitation and alternative solutions.
understand the full pipeline Jenga
What attention patterns looks real cases?
Block partition details
In-depth speed and computation analysts.
How to push the limit the speedup ratio?
examples analysis
B. Implementation Details
detailed parameter selection difforent settings
(HuryuanTVV, Hunyuan12V, Wan model series).. Detailed implement iemplmmnttaion for Multi-GP GPU adaptation Adaptation details for 12V models and distilled models. study.
D. More Results
VBench result on full 16 dimensious. More visualizations.
Figure 8: Overview of the Supplementary. We hope all readers enjoy this work in detail. We summarize common possible questions and important technical points here to arrange the supple- mentary. We strongly recommend that all readers open the link bt https: httpp:://///aa////aeea/aeaaaaaa/ julianjuaner github. po/projects/ jengga/////egga//
jenga/ in your browser for video result visualizations.
A Algorithmic Implementation
For
more comprehensive understanding of the method component of Jenga, we provide pseudo-code algorithmic workflows Algorithm 1 (Progressive Resolution), Algorithm (Attention Carving pipeline), and Algorithm 3 (building block mask B).
A.1 Details in Pipeline and ProRes
In the Progressive Resolution algorithm, we highlight three key technical details that were fully elaborated in the main text.
Frequency re-ordering. Prior to each attention layer, input latent patches undergo positional embedding operations such as RoPE [64], which typically establish frequency maps based on the standard thw ordering. Since we employ G to re-order the latents, we similarly apply foik = G(f) re-order the frequency components across different dimensions, ensuring alignment with the latent ordering. As this operation is performed only once per stage, its computational overhead is negligible.
Ordering back before unpatchify. Since the block selection in AttenCarve occurs after patchi- fication, and both
patchify and unpatchify operations need to be performed in the thw space,
22

---

10: end if Algorithm 1 Progressive Resolution Framework for Jenga Video Generation
Require: Text prompt c, stage number S, resolutions R1,..., Rs, block size m, block selection rates ky...., ks, cutoff probability p. text-amplifier p. timestep lists ......, T'y
Ensure: Diffusion model Ma. flo--machhing scheduler
1: Text tokens: 2. LM(()
2: for s to S do
3: Initial noise é~ (0, E 2 + a
4: Compute block reordering G and adjacency masks Buga
5: Remap positional frequencies: feik getFreg(Hss,)
6:
for t in T, do
7:
Reorder tokens: 2, f ((patchfi((xe))
8:
Apply sparse attention: 2, +- Ma(zi, Te, kn, P. Suik, Budja)
9:
Restore order: & + (mnpacchiy(g-(xxx)))
10:
Denoise step: 2z-1 +- scheduler E2,
end for
if > then
13:
Predict clean latent: 30 f- 2t Jcfe
14:
Resolution transition: Jt-1 o ((()) t
15:
Reset text amplifier: + for >1
16:
Increase sampling shift: a < a +
17:
end if
18:
end for
return Final prediction æn
Algorithm 2 Block-Sparse Attention with Con- ditional Enhancement
Algorithm 3 Build Block-wise Attention Mask
Require: Query Qus Key K, top-k, probability threshold visual blocks Me. adjacency mask
Require: Query Q, Key K, Value V. top-k, block
size m. text blocks Mc, probability threshold p,
Bulja
adjacency mask Badja
Ensure: Black selection mask B Q, K BlockPPol(Q.)). BlockPool( (loc)PP mean pooling per block.
Ensure: Attention output
Get visual blocks M, [N/m] Me
2: if Me > then
2: Block attention scores: S QKT /
3: Extract Qu from first vision blocks x M to-
3: Convert to probabilitics: R + softmax
kens
4: Sort probabilities: Raccted I + sort R, desc = True)
4: B BuildMask(Q.., K, k. Me U Buda
5: O,, AtteCCarve Qavemui K. V,B)
5: C cccmuuum (Racete
6: end if
6: N₂ <- max (sum(C C S p) + +1, M.)
7:
if Mc 0 then
7: Initialize: Bup + zeros B, H, M., Mota)
8: Extract Q. from remaining tokens
8: Fill Bup using indices 1 1 i, ;, 0: N&]
9: Oc + FullAttention(C (Q,, K, V): Text blocks
sec all.
9: Beund <- {i Vjj M.}
10: B + B Budja Beand
11:
return concat (O., Oc)
11: return
we must execute reordering after patchification. Subsequently, before unpatchification, we apply the inverse operation G from Eq. (2), ensuring that all transformations are performed in the appropriate space.
Scheduler re-shift. Following the re-noise process in Eq. (4), although theoretically we maintain the same noise strength, the clean state žj still exhibits a discrepancy from the true distribution. To address this, we employ approach similar to BottleNeck Sampling [49, 65], progressively increasing the timestep shift factor of the rectified flow scheduler across stages.
A.2 Details in AttenCarve
The implementation of AttenCarve builds upon the official codebase of block-wise MInfer- ence [36]. To enhance attention efficiency, we decoupled the vision and text query blocks as Q = concat(Q... Qc), and applied FlashAttention2 [17] directly to the condition blocks. For the cut- off probability constraint when constructing the importance mask Bop. formulate the optimization
23

---

Algorithm 4 Block-Sparse Attention with Text Amplification Kernel
Require: Query Q, Key K, Value V. sequence lengths, qk scale, text amplifier p. text block start index, block
mask B, block dimensions
Ensure: Output features
1:
program id(() // Current query block
2: off nff_hz- t program_id(1)
Batch head index
3: Load sequence length and check bounds
4: Initialize offsets for data loading
5: Load query block y and scale by qk scale
6: Initialize accumulators m. + -oc, 1
7: for block idx = NUM_ WIM_BBOCKS
do 8: is is_vlld block <- B(off_hz,, [off__h,,star__a start_m block_idwy
9: is valid block then
10:
Load key-value block
offset block idx x BLOCK N Compute attention scores qk <- q k²
12:
Apply sequence length mask to qk II Apply text amplification
13:
14:
text 1_tex__blook block <- block_idx 2 text_ text_block_stteeeeeeeeeeeeeeeeee block_start
15:
gk qk p is__ext________ block else qk
16:
Compute attention weights p exp(ak max((q)) Update accumulators with standard attention updates end if
17:
19:
end for
20:
Normalize: acc + acc/4,
21:
Write results output
problem as minimizing the number of selected blocks:
Buoli]] Bupli
min
subject to
R >
(5)
JEB
To satisfy this constraint, our implementation employs sort-then-greedt -ee-eerl---llllettt approach. For block index selection operations, we leverage vectorized indexing techniques to circumvent large-scale for loops, thereby substantially improving computational efficiency. In line 2 of Algorithm 3, we address an omission the original Eq. (3) by explicitly incorporating the dimension de in multi-head attention. Additionally, implemented several engineering optimizations based the MInference [36] block selection mechanism, including replacing the original einsum operations with CUBLAS-optimize torch. bmm () functions for enhanced latency performance.
A.3 Index Re-Order and Block Partition
Block 4450
(428, 023, 467. a48,
Each color represents block
Adjacent Latents (Neighhorrhoode)
Visualization (32"45'80)
Block 900
Figure 9: A real block partition example. We adopt resolution-indeeendenn Space-Filling Curve (SFC) [54] to accommodate a wider range resolutions compared to static 3D partitions. The right portion illustrates the local adjacent blocks using a look-up mask Badja-
To provide readers with a better understanding of the block partition characteristics in Jenga, beyond the toy example in Fig. 3, we demonstrate the Space-Filling Curve (SFC) implementation in a real 720P video latent space Fig. 9. We employ Generalized Hilbert curves to overcome the limitation of standard Hilbert curves, which are only suitable for (2". 2n 2n) 3D spaces. It is important to

---

Table Detailed parameters. We report the eтror bars for DiT latency measurements. The bolded steps indicate the additional steps required during stage transitions.
AttenCarve
ProRes
Performance
Settings
NFF
list
It"
step ratio
latency
VBench
Hunyuan Video [12]
50
R
=[32 80)
1625
15x
82.74%
Jenga-Base
23
10.3, 0.21
0.3 1
R 11.0, 1.0
[0-24, 25-49]
0.5
171
347
83,34%
Jenga-Turbo
24
[0.3, 0.2]
033 2
R x [0.75, 1.0]
[0-24, 25-49]
0.5
[7.9]
225 Gs
83.07%
Jengo Flash
24
0.2]
0.3
2
1t x [0.75, 1.0]
[0:24, 25
0.5
[7.9]
184 1: 3x
82.73%
Jengo-3Sung 3Suge
24
[0.3, 0:2]
0.3 3
R x [0.5, 0.75, 1.0]
[0-14, 15-24, 25-49]
0.5
[7,9,
157 1: 3x
80.539
Hunyuan Video-12V
R²
=132,
JJnga
23
[0.3, 0.2]
0.3
2t [1.0,
[0-24, 25:49]
0.0
[7]
338
87.75%
AccVideo [25)
R =[32, 44, 78)
161 + 1.
83.84%
+ Jenga-Baze
[0.3, 0.2]
0.3
Rº 11.0, 1.0
25-491
0.5
171
76 = 25
83.39%
+ Jenga--Tuiil
[0.3, 0.2]
0.3
R x (0.75, 1.0]
[0-24, 25-49]
0.5
[7]
=
83.29%
W-n2.1-1.3B [13]
50
R =[20, 30, 52]
115 3s
83.28%
+ Jenga Baxe
1.5
[0.2, 0.1]
0.9
12 [1.0, 1.0]
[0-24, 25-49]
0.0
[7]
24 :: 2s
82.68%
+ Jergu-Turbo
15
[0.2, 0.11
0.9 1
R² x [0.75, 101
[0-24, 25-491
0.0
171
17 = 28
82.52%
note that each block in Jenga is not a regular rectangular prism, but rather a local cluster of tokens that are naturally partitioned. This design provides Jenga with minimal constraints regarding video wetennsionswiithhoe requiring padding along physical dimensions, it only necessitates that the total token count thw be divisible by the block count The continuity property of SFC in the original space also ensures certain degree of semantic similarity among tokens within each block.
We further demonstrate how to utilize the Adjacency Mask Bajja to identify blocks that are spatially adjacent in 3D space based their SFC representation. As illustrated, for block 450, by identifying the blocks to which neighboring tokens belong, we located 20 adjacent blocks that are subsequently incorporated into the attention computation for the current block.
B Implementation Details
B.1 Detailed Parameter Settings
In Tab. 4, we provide a comprehensive list of almost all key parameters used in this work. It worth noting that although Jenga-Bane employs a single-stage pipeline, utilized different drop rates (0.7, 0.8) at different timesteps, effectively dividing our steps into two segments. We discovered that using a higher cutoff probability (i.e., p 0.9) in Wan2.1 [13] significantly improved results without incurring additional computational time, suggesting the presence of few attention heads that concentrate on global features. We briefly describe our ProRes adaptation specifically for Hunyuan Video [12] (i.e., Jenga-Base). We will implement ProRes adaptation for Wan2.1 [13] in the future.
B.2 Multi-GPU Adaptation
Re-Order Flatten, LN.,
"K Blocks
Grath All tol
[X
DM MC
modaact
cudae2
cudsati
: [N,, N/G, d]
Order Back Unflatten,
Video [thw]
Gather to
State A: FEN, Projection Layer
State B: Head-Split Attention (Jenga kemnll)
Figure 10: Multi-GPU adaptation in Jenga. We highlight the computation for each GPU in yellow.
25

---

For multi-GPU parallelism, we adapted our approach based on the xDiT [59] foundation used in Hunyuan Video. As illustrated in Fig. 10, we implemented parallelization across G GPUs. The paral- lelism within Transformer blocks remains consistent with the original implementation (i.e., state A: parallelization along the token dimension before and after attention, and state B: parallelization along the head dimension within attention). We modified the corresponding LongContextAttentton interface to make AttenCarve compatible with this parallel paradigm. Additionally, we discovered that when utilizing multi-GPU parallelism, the block selection process becomes the performance bottleneck. As explained in Appendix A.2, employing more efficient torch. brm operations significantly accelerates multi-GPU execution (reducing processing time from 77s to 34s with 8 GPUs).
For parallelism outside transformer blocks, since we have naturally serialized tokens using SFC, we can directly partition them according to their SFC indices before feeding them into state A. This straightforward implementation also eliminates the previous requirement that latent sizes be divisible by G' along specific dimensions.
B.3
Image-to-Vine Video & Distilled Model
For Image-to- Video [12] adaptation, two specific details warrant clarification. Since this model performs specialized modulation operations on image conditions (latent at t(0)), we provide an additional leven mask (m),, m = if 0, else 0} when inputting tokens into the model. This enables decoupled modulation operations the re-ordered latents. Additionally, the condition mask Boand incorporates both text conditions and conditioning features from the first frame. Given that the first frame already contains the overall content of the video, we did not implement the text-attention amplifier.
For the distilled model Acc Video [25], which inherently requires fewer sampling steps, we employed single-stage Jenga-Base setting as detailed in Tab. 4. Other configurations, including multi-GPU implementation, remain consistent with our Hunyuan Video setup.
B.4 Compared Baselines
To establish a uniform evaluation standard, we standardized the test prompts, utilized the more widely adopted FlashAttention2 [17], and maintained consistent input video dimensions across experiments. Below are the specific configurations for comparison methods beyond the baseline:
CLEAR [21J. We implemented based on the original FlexAttentin [66] with a 3D radius r = 32. When calculating FLOPs, since CLEAR does not account for GPU parallelism capabilities, we used the actual block sparsity (11.1% % instead of the theoretical 56%) to compute effective FLOPs. Combined with the kernel optimization overhead of Attentioo itself, the resulting generation speed could not even surpass the baseline.
Minference [36]. As explained in Sec. 4.2, we enhanced the block-wiss attention mechanism from MInference. We removed the causal mask designed for LLMs and implemented selection rate of & 0.3. Notably, several approaches similar to MInference exist, such block-sparse attention [67] and MoBA [39], which employ essentially identical methodologies.
SVG [22]. We utilized SVG's original implementation and resolution, incorporating its optimized RoPE and Normalization kernels with sparsity setting of 0.2.
TeaCache [31]. We employed the official thresholds (0.1 1 for slow, 0.15 for fast configurations). For 1an2.1, we set the threshold to 0.2 and enabled the use_ret_ster parameter, which provided further acceleration while preserving result quality.
B.5 Details about User Study
Fig. 11 presents the Google Form questionnaire and anonymous website interface used to display video assets in our user study. We randomly sampled 12 prompts from a pool of 63 paired results and randomized the left-right ordering of videos within each comparison pair. To ensure data quality, we excluded invalid responses with completion times less than 5 minutes or greater than I hour. We also removed 3 submissions exhibiting highly homogeneous selection patterns (e.g., consistently choosing the "left video" or "same" " for all comparisons). The results from the remaining 70 valid questionnaires are presented in Fig. 6.
26

---

Study Efficient Generation Methods
User Study on Efficient Video Generation Methods
andira
haddy
videos, please choose
Right Video roch q unition eva visual quality (temsporal consissency, aesthetic qualltty semanoic quality Finolly prefenence,
pare dud with ciatico fired poving aedaercams
(prompt consistencyf
quality
youngg
the
WA Wiee
Day VIRK
(a) Google Form for user study
(b) HTML for video preview
Figure 11: User study, (a): Questionnaire form example using Google Form. (b): Anonymous video preview website for live comparison.
C Discussions and Analysis
C.1 Limitation Analysis & Alternative Solutions
Tallint
Dynamic scone:
Cleas boundary: latent mis Lgoment
Complex textane: leaa artifact
Figure 12: Some failcases. We present two potential failure cases that may occur when using more stages (S 3), well scenarios where this setting more suitable.
Table 5: Results with different prompt formats. Generation with enhanced prompts eliminate quality degradation and boost multi-stage results (comparable video quality with 10.35 x speedup).
Hunyuan [12] 1.00x
Jenga-Turee (2-stage) 7.22
Jenga-3Stage (3-stage) 10.35
Prompt Total VBBench Q VBench-S
VBeech
VBBench
VBench
VBench
Standard
82,74%
85.21%
72.84%
83.07%
84.47%
77.48%
80.53%
81.66%
76.00%
Enhanced
82.61%
83.98%
77.11%
83.29%
84.22%
79.57%
82.34% a234
83 6.65 6539
77 07.08% amo
As discussed in Sec. 4.3, Jenga faces certain challenges when implementing Progressive Resolution (ProRes). Several studies [68, 69] have examined the disparities between latent-space resizing and pixel-space resizing. Even with substantial re-noising (ar, > 0.9), we cannot guarantee that edges in
27

---

the pixel space will be perfectly denoised in the final result. Since our work focuses on transformer acceleration, we opted against using the VAE decode-resize--nnon approach, as tiled decode-encode operations during stage transitions would introduce additional latency of nearly 50 seconds. Fig. 12 illustrates some failure cases and usage scenarios our current solution in 3-stage Jenga (results shown in Tab. 3b, 10.35x faster). We observed that generation quality occasionally deteriorates in static scenes or scenarios with clear boundaries (as well as in the Image-to-Videe these issues tend to diminish when generating more complex textures scenes with intricate motion patterns. We validated both the baseline and multi-stage results on VBench using enhanced prompts, shown in Tab. 5. This enables users obtain satisfactory video results with significant acceleration when using more complex prompts (such as Sora-style prompts, as demonstrated Fig. 5 (b), the SUV case).
scenario), However,
Beyond the training-based improvements discussed Sec. 4.3, another promising direction for optimization is developing enhanced block partition methods. While the current SFC approach possesses many desirable properties, remains fundamentally static. Extending context-based SFC approaches [70] into 3D video latent space could potentially yield better utilization of block selection.
C.2 Block Selection: Attention Patterns
Local Faiteen
Latation- Aocare
Semantic- (Renr)
Figure 13:
Attention patterns. Visualization of attention distributions across different layers and timesteps for the first block (at the corner position) containing 128 latent items.
We visualize the block-aware attention scores in Fig. 13. Our analysis reveals four key characteristics in the attention patterns: (1) In shallow layers, most patterns exhibit strong locality features, (2) attention patterns highly correlate with position, forming stripe or planar distributions. deeper attention patterns emerge, where attention shifts according to the vidco's s semantic content. (4) Simultancously, we observe hybrid patterns combining the three aforementioned characteristics, as well global patterns with attention sinks. Our cut-off probability threshold specifically designed to capture information from these latter heads. These visualized patterns not only demonstrate the inherent sparsity characteristics of attention mechanisms but also highlight the necessity for dynamic block selection in our approach.
layers of the model, (3) semantic-awee
C.3 Resolution-Awart Field of View
In addition to the influence of the text-attention amplifier on Field View (FOV) demonstrated in Figs. and 5, we present additional examples in Fig. 14 showing dynamic FOV changes achieved by adjusting the factor p. observed that in certain scenarios, not utilizing the text-attention amplifier results in an overly localized focus, ultimately reducing the content coverage in the frame. By introducing the bias parameter we can exert a degree of control over different field-of-vien ranges.
28

---

Promutt: Temple Heghaesnus, its Doste gaene stnicallyy against backdrop quiet
Pronph: sccelarating gain speed
low
high
Figure 14: Dynamic FOV. We demonstrate the impact the balancing factor p on field of view in both static and dynamic scenes. Additional ablation examples presented in the HTML supplement.
720P
115200
Pre-attention
0.059a
$40P
63360
Attention
0.445s
Atten profile
380P
28 160
Attestit
0.037a
(x) Number of Inkons
Attin
Blockswise
720P
708s
Pru Pre-atteention
0.059s
0.132s
360P
68c
Poet Attention
0,037s
2.80%
Bloek Selection
Text Amention 1.13%
3.78%
Othece
steps)
inference
Figure 15: Latency analysis. (a, b) Visual token counts and generation times at different resolutions. (c) Acceleration of AttenCarve vs. FlashAttention2 [17]. (d) Time breakdown across AttenCarve components.
C.4 Speed Analysis & Additional Overheads
In this section, we provide an in-depth analysis of our method's latency. First, as illustrated in Fig. 15 (a)-(b), demonstrate the necessity of directly reducing token count by adjusting resolution. At 360P, only 1/4 of the input tokens, the generation speed achieves a 10x improvement compared to 720P. In Fig. 15 (c), specifically evaluate the acceleration achieved by AttenCarve compared to FlashAttention22 [17], which achieves 3.7 x speedup in attention computation. Furthermore, Fig. 15 (d) provides a detailed time breakdown across different components of AttenCarve, showing that Block selection introduces only 2.8% computational overhead. Additionally, we analyzed the memory efficiency our approach. Without any specialized optimizations, when generating 720P videos, Jenga introduces minimal additional memory overhead of only 3.7% (71.84 + 74.49 GiB).
Despite the series of optimizations in Jenga, numerous improvements. These include incorporating quantization optimizations mentioned in SVG [22] and StargeAtttn [50], well kemnel optimizations for RoPE [64] and normalization operations. From hardware perspective, adapting PlashAttention33.. ture shows significant speed enhancement potential. Additionally, parallelization and sparsification strategies for the VAE component have not been fully explored. These directions represent promising areas for future engineering optimizations and continued investigation in our work.
remain for potential performance
[71] attention kernels on the Hopper architec-
C.5 Structural Fidelity Evaluation and Comparative Analysis
While our primary evaluation focuses on VBench for comprehensive video quality assessment, we recognize the value of complementary metrics that capture different aspects of video quality. Structural fidelity metrics like FVD (Fréchet Video Distance) provide additional validation of our method's effectiveness by evaluating the distance between training data distribution and inference data distribution. As a training-free method, we adopt two evaluation protocols: (1) the official VBench test results with different sampling seeds, and (2) the high-quality video dataset Inter4K [72] real data distributions to evaluate FVD.
29

---

Table Structural fidelity evaluation with FVD metrics. We compare Jenga with complementary acceleration methods across different models, demonstrating independent effectiveness.
Method
VBench
Latency
Speed-Up
FVD-ImeerK
FVD-Hunyuan
HunyuanVidos
82.74%
1625s
1.00
600
144
AttenCarve Only
83.42%
748s
2.17x
448
164
ProRes Only
82.85%
1075s
1.51
620
191
ProRes Skip
82.57%
495s
3.28
630
203
AlvenCarve ProRes
83.12%
485.
3.35
542
127
Jenga: Turbo
83.07%
225s
7,22
583
141
VBench
Latency
Speed-Up
FVD-Inter4K 4
FVD-Wand
Wan2. 1.33B
83.28%
115s
1.00
788
182
TesCachhe-fatt
34s
232
AttenCarve Only
82.96%
71s
1.62
674
169
AttenCarve ProRes
83.56
52s
2.21 x
AttenCarve Skip (Jenga-Base)
82.80%
246
4.79
702
184
Jenga-Turie
82.52%
178
6.52
548
194
Table 7: Comparison of different partitioning strategies. We evaluate computational overhead for 720x 180x videos with block_size=122 demonstrating SFC's superior efficiency.
Mask Type
STA Tiled Local (6.8.8)
Optimized Local (3,8.16)
3D-SFC
Padding Tokens
19,440
7,920
112
Additional MatMul Computation
35.32%
13.78%
0.19%
Independent Acceleration Analysis. To demonstrate that Jenga achieves substantial acceleration independently and can be combined with orthogonal methods, we conducted comprehensive compar- isons with TeaCache [31], a feature reuse technique. Tab. shows results on both Hunyuan Video and 1an2.1 models, revealing that our acceleration is fundamentally independent of feature caching techniques.
The results demonstrate that Jenga preserves video generation quality comparable to baselines while achieving superior acceleration. For Hunyuan Video, Jenga-Turbo achieves an FVD score of 141 versus 144 for the baseline, maintaining structural fidelity while delivering 7.22x speedup. Similarly, on Wan2.1, our method achieves competitive FVD scores across different evaluation protocols. The FVD results align with VBench trends, confirming that our acceleration techniques do not compromise generation quality.
Importantly, while TeaCache focuses on reusing computed features across timesteps, Jenga reduces computation through progressive resolution and selective attention while boosting generation quality. This orthogonality means the methods can potentially be combined for even greater acceleration benefits. Our results show that Jenga's acceleration is fundamentally independent of feature caching techniques, providing complementary approach to efficient video generation that maintains high structural fidelity.
Design Choice: Static SFC vs. Adaptive Partitioning, Our choice of static Space-Filling Curve (SFC) construction is motivated by several key considerations that balance effectiveness and com- putational efficiency. First, SFC block partition exhibits inherent locality properties that align with the predominantly local characteristics of attention computation in high-resolution video data. The generalized Hilbert curves naturally possess local-neighborhood properties where neighbors on the ID curve correspond to neighbors in 3DD space, validated in Tab. 3b where SFC improves both latency and performance while reducing line-wise drifting artifacts compared to linear partitioning.
Second, SFC demonstrates remarkable parameter insensitivity compared to hand-crafted strategics. While local window approaches (as in STA [20] and CLEAR [21]) restrict models to specific resolutions with dimensional padding severely impacting performance, SFC's ID nature makes it insensitive to resolution and block size parameters. As shown in Tab. 7, our 3D-SFC requires only 112 padding tokens and 0.19% additional computation, compared to 19,440 tokens and 35.32% overhead for STA's tiled local windows, demonstrating superior efficiency.
Furthermore, adaptive partitioning faces fundamental challenges in text-to-video generation. Since generation starts from pure noise, extracting meaningful semantic information for adaptive token partitioning in early denoising timesteps is problematic. We experimented with dynamic approaches,
30

---

including changing SFC dimension scanning directions during interleaved forward attention passes enhance block interactions (similar Swin-window shift). This approach yielded no significant quality improvements while introducing substantial processing overhead 20s per video) due to memory discontinuity and defragmentation costs. Our static SFC approach avoids these overheads entirely through pre-computation while maintaining the locality benefits essential for efficient sparse attention. While this represents a limitation with room for future improvement, the current design effectively balances computational efficiency with generation quality.
D
Additional Results
D.1
Detailed Benchmarks
Table 8: Detailed VBench [35] results. We omit the percentage symbol % for better preview.
Quality Metries
Somantic Metrics
backgrooudd consistency
subject comsistency
componi flickering
amoothness
spatial relationship
overall consistency
aesthetic quality
imaging quality
dynamic degree
multiple objects
appearance style
tempora style
abject class
coler
Methods
HonyuanViden [12]
196.59
98.06
82.03
93.75
86
20.51
CLEAR [21]
97.15
97.82
99.61
99.57
63.03
68.88
45.83
58.59
48.89
92.00
93.27
69.41
44.18
20.97
22.61
26.36
Minference [36]
94.90
97.66
99.41
99.47
61.62
69.78
65.27
75.00
83.08
88.00
93.75
77.18
42.28
20.80
23.08
27.17
SVG1221
97.75
99.55
61.78
61.11
74.52
90.36
77.25
20.20
AttenCarve
95.94
97.85
99.30
99.18
62.47
69.09
70.83
86.71
73.02
93.00
90.67
75.45
47.17
19.50
23.43
26.36
TeaCache-slow [31]
96.70
97.89
99
61.54
72.09
20.05
TeuCache fast [31]
96.68
97.79
99.32
99.50
61.42
68.59
56.94
64.08
64.71
90.00
85,99
71.22
36.26
20.12
23.12
25.77
ProMes
96.16
97.58 99.72
99.55
63.75
70.36
70.83
82.81
55.15
89.00
88.24
67,26
26.10
20.46
21.89
26.79
ProRes-timeskip
95.57
97.68
99.54
62.93
68.97
72.22
76.95
59.19
90.00
11
29.04
20.66
21.75
Jenga Base
95.09 97.86
99.31
99.18
62.47
69.09
72,22
86.71
73.02
88.00
90.67
75.45
47.17
19.51
23.43
26.36
Jenga-Turbo
93.42 96.85
99.31
98.85
63.89
66.64
77.78
94.14
66.91
94.00
95.31
73.76
50.37
19.85
23.74
27.98
Jenga-Flash
92.75
19
99.27
98.57
62.29
85.71
73.61
63.60
90.00
99.26
71.97
56.25
20.27
24.43
28.05
VceVide [25]
95.92
97.53
99.35
99.28
61.40
67.98
58.33
89:40
76.30
88.00
92.
80.29
$1.09
20.49
24.43
26.73
+Jenga
95.36
96.97
61.38
90.37
1-.2.1-1.3B [13]
96.46
98.40
99.52
98
64.08
67.36
59.72
75.00
47.64
82.00
81.87
71.49
23.11
19.82
23.68
23.59
TeaCache [3]]
96.40
98.25
98.70
62.03
76.39
82.47
19.83
22.99
Jenga
95.40
97.92
99.44
98.55
63.13
65.37
61.11
74.76
53.89
78.00
88.42
70.08
26.53
20.25
23.34
34
23.49
Quality Metries
12V Semantic Metries
Total
backgroand consistency
background consistency
subject consistency
motice smoothness
subject comsistency
aosthetic quality
imaging quality
dynamic degroo
Quality Score
camera motion
Total Score
12V Score
Methods
Vidoo-2VVioo-12V [12]
95.67
96.39
99.21
61.55
70.37
21.14
78.30
51.38
98.90
99.38
96.67
87.49
timeskip
95.75
61.93
21.54
Jenga
93.99
95.75
99.00
60.84
70.43
40.65
79.31
49.80
98.43
99.14
96. 18
87.74
Tab. 8 provides comprehensive evaluation results across all 16 dimensions of VBench [35]. As shown, Jenga achieves notable advantages in multiple semantic score dimensions while maintaining high performance in quality metrics.
Regarding detailed results in Tab. 8, there are two key points to clarify. First, we discovered that compared to the static local patterns used in CLEAR [21], our query/head-aware dynamic patterns significantly enhance the dynamic degree of generated results (45.8355 > 70.83%). Overall, Jenga introduces larger motion amplitude at the quality level, while presenting some trade-offs in subject
31

---

consistency when the selection rate is small Venga-Flashh At the semantic level, Jenga demonstrates substantially better semantic adherence across multiple dimensions (color, object class, scene, and overall consistency).
D.2 More Visual Results
We showease additional results of Jenga in different settings, as illustrated in Fig. 16, and Fig. 17. We recommend viewing the video files in the provided HTML to better evaluate the effectiveness of our method.
E
Social Impacts
This paper introduces novel framework for efficient video generation that is based current pretrained Diffusion Transformers. Although this application has potential to be misused by mali- cious actors for disinformation purposes, significant advancements have been achieved in detecting malicious generation. Consequently, we anticipate that our work will contribute to this domain. In forthcoming iterations of our method, we intend to introduce the NSFW (Not Safe for Work) test for detecting possible malicious generations. Through rigorous experimentation and analysis, our objective is enhance comprehension of video generation techniques and alleviate their potential misuse.
32

---

Jenga-Bacn Base 347s, 4.68x
Several giant woely menamoths approich trending thrrugh anosy
Excreme closs of 24 old warannk eye blinking, stancing in Marrakech during magic
close vieu ofa glaas sphere that has zen gurden. within it. There a semall sphene zon creating the
Jenga- Turbo 225s, x
dose-cp Victoria showcaaus iss striking blue plumage and rnd cheetnn
3D vtbbant, enchanted torest...
The camera tollows behind varrage black
diri radd surcounded by gine trees stepp mountain slops, dust locks up
Jenga-Flas Flash 184s, x
Hlstatical Bootago of Faliformia during the gold
muvie trailer featuring the adventunes of the year old
space man weasing wool knitied matoreyele
cinematic shot 35mm film, virid coloes.
young ploce cloud reading
Jenga-Super 157s, 10.35
Photorealistic closeup video of two pizute shipa buttling eath other as they snil inside
gergeemely rendened pagerrrac world ofa cural rocd rit with colarfal Est:
drone sruund beautifal histuric church built on rocly onteropping along the Amalh
Figure 16: Visualization results. From top to bottom, each three videos from the same setting.
33

---

H2 Jennn Jenga 338s, 4.43x
An serial view of faturitiic building coerlesicing
pans
young smalling pink flower
pesson riding a
seagull is Bytng towarda panson hand
Wenne 24s, x.79
A is shaking head
A eate happy Corgl playtig park, suzari, up
storm vacouming beach.
sailing lesurelly along Vincent van Gegh
AceVideo=Jende Jenga 76s, x
abark owtmming the ocwan, Seaturing sheady and smoots: peespertive
happy pask, racking Focus
couch and a potted plant
matoncycle slmwing
Figure 17: Visualization results for model adaptations. Prompts are from VBench [35].
34


Reviews:
[{"review_id": "gHHHFySbDj", "reviewer": "Reviewer_G2mf", "summary": "This paper proposes Jenga, a training-free inference acceleration framework for video diffusion transformers (DiTs). It addresses two primary bottlenecks in DiT-based video generation: the quadratic complexity of self-attention and the multi-step nature of diffusion. Jenga achieves acceleration through two core components:\n\n1. Attention Carving: A dynamic block-wise sparse attention mechanism using 3D space-filling curves to reduce redundant token interactions.\n\n2. Progressive Resolution (ProRes): A multi-stage generation strategy where early denoising occurs at low resolutions and progressively upsamples latents, reducing the total number of processed tokens.\n\nThe framework is shown to be broadly applicable across text-to-video, image-to-video, and distilled models, achieving up to 8.83× speedup on state-of-the-art video diffusion models like HunyuanVideo, AccVideo, and CogVideoX, with minimal degradation in generation quality (e.g., less than 0.01% drop on VBench).", "questions": "1. Clarification of Block-Level Attention Mechanism: In Figure 2 and Section 3.1, it's unclear how frequently the block-wise attention mask is updated. More clarification is needed on the roles and update frequencies of the importance mask, condition mask, and adjacency mask—specifically, which are static and which contribute most to performance. Additionally, please elaborate on why the condition and adjacency masks are solely determined by resolution.\n\n2. Connection Between Trajectory Curvature and Resolution Scheduling: The paper introduces curvature analysis to guide block selection, while the progressive resolution strategy (ProRes) appears independently designed. Is there any correlation between curvature changes in the sampling trajectory and the decision points for resolution upsampling?\n\n3. Applicability to Low-Step or Consistency Models: Has Jenga been evaluated in low-step regimes (e.g., 4–8 denoising steps) or on consistency-based models like SD3? These increasingly popular settings may challenge assumptions in both ProRes and the sparsity design of Attention Carving.\n\n4. Adaptivity of Sparsity Masks: The current block selection uses fixed thresholds (e.g., top-k, softmax cutoff). Could adaptive or learned sparsity mechanisms further improve the balance between quality and efficiency, especially across diverse prompt distributions or content styles?", "limitations": "The paper does address some limitations, especially regarding resolution transitions and heuristic design, but the discussion could be more thorough. In particular:\n\n1. Lack of Generalization to Consistency and Low-Step Models: The current method assumes a moderately long sampling process (e.g., 24–50 steps). It may not generalize to ultra-fast samplers like consistency models or low-step DMs, where resolution transitions and sparsity patterns can break down.\n\n2. Static Heuristics for Sparsity: The block selection (top-k, threshold-based) is fixed per stage. While effective, this limits adaptability to different prompts or domains and may underperform in highly dynamic content.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. Proposes a training-free, plug-and-play framework for efficient video generation that achieves up to 8.83× speedup across various DiT architectures, including HunyuanVideo and AccVideo, without model modification or retraining.\n\n2. Introduces Attention Carving, a dynamic, block-wise sparse attention mechanism guided by 3D space-filling curves and masks based on block importance, adjacency, and condition relevance—offering a clean and GPU-friendly design.\n\n3. Leverages a Progressive Resolution (ProRes) strategy that denoises at lower resolutions in earlier stages and progressively increases resolution, aligning with the coarse-to-fine nature of diffusion processes.\n\n4. Includes a novel text-attention amplifier to stabilize the field of view (FOV) during early low-resolution steps, which would otherwise generate zoomed-in or truncated content.\n\n5.  strong empirical results across text-to-video, image-to-video, and distilled models, maintaining semantic quality and CLIP scores even under aggressive speedup (e.g., Jenga-Flash, 8.83×).\n\n## Weaknesses\n\n1. While the paper compares against cache-based methods, Jenga does not reuse intermediate states or features, which limits its benefit for very low-step (e.g., <8) samplers or consistency models, where reuse is more effective.\n\n2. The block-wise attention masking procedure, though efficient, is still heuristic-based and uses static top-k and probability thresholds. No adaptive or learned sparsification is explored.\n\n3. he motivation for combining curvature estimation with ProRes is underdeveloped: how the geometry of the sampling trajectory informs resolution transitions isn’t fully explained or connected.\n\n4. The novelty is somewhat incremental, as both sparse attention (e.g., SpargeAttn) and progressive resolution (e.g., Bottleneck Sampling) have prior art. Jenga’s contribution lies more in effective integration than in introducing new algorithmic ideas.", "was_revised": true, "final_justification": "After reading the authors' rebuttal and other reviews, I choose to maintain my recommendation.", "rebuttal": "Dear reviewer G2mf,\n\nThank you for your feedback and comments. We appreciate the thoroughness with which you have reviewed our manuscript, as evidenced by your insightful and valuable comments. We are grateful for your positive assessment of our work, particularly your recognition of its novelty and results. We will address your comments below and incorporate the necessary revisions in the updated paper.\n\n## Q1 Clarification of Block-Level Attention\nWe address your concerns by breaking them down into the following specific points:\n\n### Q1.1 How frequently the block-wise attention mask is updated?\n\nThe table below provides detailed information about the number of updates and computation timing for each mask type in an $L$-layer DiT model with $T$ actual denoising timesteps across $S$ resolution stages in ProRes.\n\n|Mask Type|Importance|Condition|Adjacency|\n|-|-|-|-|\n|Number of updates|$LT$|$S$, switch each stage|$S$, switch each stage|\n|When to compute|Start of attention| Before denoise |Before denoise|\n\nAttention Carving directly replaces the normal Full Attention. This means the head-block-wise attention mask $\\mathbf{B}$ is updated in **per-layer attention**. If we perform inference on a 50-step model with 60 layers, it will update 50×60=3000 times. This design choice is motivated by the fact that attention patterns may vary significantly across different heads, layers and timesteps, as visualized in **Appendix Fig. 13**. We have experimented with reusing masks across different timesteps, but this approach failed to generate satisfying video results (VBench < 80%). We will include this explanation in the revision for better clarification. Additionally, while the block-selection operation (determining the importance mask $\\mathbf{B}_\\text{top}$) is performed layer-wise, we demonstrate in Appendix C.4 and Fig. 15 that **the computational overhead of block selection is minimal** (less than 3% of the total sparse attention computation).\n\nRegarding the Condition Mask and Adjacency Mask, as stated in L162-163, \n\n> In Jenga, Condition Mask and Adjacency Mask are pre-computed, and only determined by the resolution and the partition function $\\mathcal{G}$.\n\nThese two masks are static and precomputed as the resolution-aware SFC ($\\mathcal{G}$) is predetermined. The only updates occur during stage switches when the latent resolution changes.\n\n### Q1.2 Which Mask Contributes Most to Performance?\nIn Table 3(b) and Figure 7, we provided an ablation study on SFC selection and mask selection, demonstrating the effectiveness of our mask selection scheme. Here we present a more comprehensive ablation study to provide better validation of our method. The \"No Mask\" baseline is the result with ProRes and timestep skip.\n|Mask Type|Importance|Condition|Adjacency|VBench|Latency|\n|-|-|-|-|-|-|\n|No Mask (FA)|Full|Full|Full|82.57%|495s|\n|TopK only|✓ (partially)|||81.35%|220s|\n|TopK, SFC|✓ (partially)|||81.41%|220s|\n|TopK, Prob. Constraint, SFC|✓|||81.87%|223s|\n|w/o Adjacency|✓|✓||82.42%|222s|\n|w/o Condition|✓||✓|81.82%|221s|\n|w/o Importance||✓|✓|77.41%|140s|\n|All Mask|✓|✓|✓|83.07%|225s|\n\nThe above results demonstrate that: (1) Each mask component contributes meaningfully to quality or efficiency, (2) **The importance mask provides the most significant impact on both metrics**, (3) The combination of all three masks achieves the optimal balance between visual quality and computational efficiency. The \"w/o Importance\" case shows the most dramatic speedup but significant quality degradation, confirming that **content-aware adaptive block selection** is crucial for maintaining generation quality.\n\n## Q2 Connection Between Trajectory Curvature and Resolution Scheduling\n\nThe denoising trajectory serves as a guidance mechanism for optimal resolution scheduling. We conducted an ablation study on resolution scheduling to better understand the relationship between trajectory curvature and optimal transition timing:\n|Low-Res Step|10%|30%|50%|70%|90%|\n|-|-|-|-|-|-|\n|Traj-Timestep|987|947|883|767|437|\n|VBench|82.36%|82.35%|83.07%|81.03%|78.74%|\n|Latency|286s|253s|225s|207s|169s|\n* The experiment uses Jenga-Turbo's basic setting, containing one low-res > high-res stage transition.\n\nThe experimental results reveal key insights about trajectory-guided resolution scheduling. Due to the timestep shift mechanism in DiT, more computational steps are allocated to the early stages of the trajectory. Even in the final 10% of timesteps, the actual trajectory still contains approximately 43.7% of the total steps.\n\nWe observe a dramatic quality degradation when low-res steps exceed 50% of the total timesteps, while **introducing latent misalignment artifacts**, confirming that multi-stage generation requires sufficient high-resolution denoising to maintain video quality. Meanwhile, when stage 1 (low-res) steps are too few, quality also decreases. This aligns with our motivation (L48-49) that low-res content generation requires a higher attention density to establish proper content structure. This finding validates why ProRes and AttenCarve work synergistically--they are not merely independent methods but complementary components that address different phases of the generation process.\n\nThe 50% low-res step configuration achieves the best balance between quality (83.07%) and efficiency (225s), demonstrating that trajectory curvature analysis effectively guides optimal resolution scheduling decisions.\n\n## Q3 Discussion on Consistency and Low-Step Models\nWe may not have clearly explained that **AccVideo is actually a low-step model (5 NFE)** derived from HunyuanVideo. In Table 2(b), we adopt Jenga on AccVideo with single and multi-GPU integration, achieving a 2.12× acceleration with only AttenCarve with comparable VBench and CLIPScore performance. Our method emphasizes the orthogonality with step-reduction (distilled model like AccVideo) and feature reuse (TeaCache) without affecting generation quality.\n|Method|VBench|CLIP|Latency|Speed-Up|\n|-|-|-|-|-|\n|AccVideo|83.82%|31.23|161s|1.00×|\n|AccVideo + AttenCarve (Paper Reported)|83.39%|31.07|76s|2.12×|\n|AccVideo + AttenCarve + ProRes|83.29%|31.12|51s|3.16×|\n\nMeanwhile, based on our discussion in Q2 regarding ProRes and trajectory analysis, as resolution switching fundamentally depends only on the corresponding trajectory timestep, we believe **few-step diffusion models can directly apply ProRes for further acceleration**. The table above demonstrates that using a two stage ProRes can maintain quality while providing 3.16× speedup even in few-step scenarios. This reinforces the orthogonal nature of our approach--it can complement existing few-step models without interference.\n\n## Q4 Adaptivity of Sparsity Masks\n\nAs discussed in Q1, the importance mask is updated adaptively in each attention forward pass, with adaptation for each attention head in our implementation. This means the block selection operation is online and adaptive.\n\nAlthough SFC is pre-determined by resolution, it has several benefits (a) adaptive resolution capability and (b) inherent local nature (which motivates our \"Jenga\" analogy). For a more detailed discussion on static or adaptive block partition, please refer to response in **cBBb Q2**.\n\nAlthough $k$ and threshold $p$ are pre-defined, the block selection for each query block remains adaptive across different attention heads based on the current content. As Jenga achieved a higher Dynamic Degree score in VBench in **Appendix Table 6**, we may not directly treat the high sparsity to the degradation in generating highly dynamic contents.\n \nAs a pure training-free method, current version avoids the need for offline per-head optimal search procedures (a fitted static sparsification, but not adaptive with input feature) that methods like STA[1] require. Jenga's significantly smaller search space makes it easily adaptable to new models without extensive hyperparameter tuning. It would be a future direction to design a metric to determine adaptive or learnable sparsity.\n\n## Q5 Novelty is Incremental?\n\nWe acknowledge that there are previous works in sparse attention and multi-resolution sampling and discussed them in L92-93, L98-101. However, our contribution represents an advance beyond existing approaches:\n\n**Fundamental Motivation Difference:** Jenga's core motivation differs fundamentally from previous work. We treat resolution scaling and attention carving as a unified framework, both targeting **token interaction reduction**. The central challenge is balancing large attention sparsity with content generation quality. Our unified approach addresses this systematically.\n\n**Comparison with SparseAttn:** SparseAttn achieves only moderate sparsity (0.5) with extra tuning, compared to Jenga's high sparsity (0.75). This substantial improvement in sparsity-quality trade-off represents a meaningful technical contribution. AttenCarve's higher sparsity is achieved by the introduction of ProRes, which pre-generates (noisy) content in a low-res space.\n\n**Comparison with Bottleneck Sampling:** The BottleNeck design requires additional computation in the first stage and achieves only sub-optimal speedup ratios, even with some timestep reduction. Our goal is to achieve extreme speedup ratios through systematic optimization.\n\n**Technical Novelty:** Our method introduces several technical innovations: (1) 3D SFC-based block partitioning and adaptive block selection with hybrid masks, (2) Text-attention amplifier that addresses FOV problems in training-free low-res generation. **Most importantly, ProRes solves the content degradation when adopting high sparsity attention across all timesteps, maintaining both efficiency and quality throughout the generation process.** While technical directions may share similarities, we believe Jenga represents a technically solid contribution that combines strong practical utility in video generation.\n\n[1] P. Zhang .et.al., Fast Video Generation with Sliding Tile Attention, 2025"}, {"review_id": "snsjVLR8gD", "reviewer": "Reviewer_N3G7", "summary": "This paper proposes a novel training-free and efficient method for video generation. The approach is based on two key insights: (1) early denoising steps do not require high-resolution latents; and (2) later steps do not require dense attention. Based on these observations, the authors introduce a block-wise attention mechanism using 3D space-filling curves, which effectively reduces the number of key-value blocks in attention. Additionally, they present a multi-stage Progressive Resolution strategy that uses varying latent resolutions across different diffusion timesteps, thereby reducing token interactions throughout the process. By integrating a few-step approach, the method achieves 4–8× acceleration on HunyuanVideo-T2V without compromising video quality.", "questions": "See the weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Compared to prior methods with hand-crafted attention strategies, the proposed dynamic Token Carving mechanism more effectively identifies crucial key-value pairs in video content. It also preserves global context through accumulated probability.\n\n2. The method maintains video generation quality while significantly improving inference efficiency.\n\n3. Comprehensive experiments are conducted to validate the effectiveness of the proposed approach.\n\nWeaknesses:\n1. How are the diffusion timesteps for different stages determined in the Progressive Resolution strategy? What impact do different transition points have on the final video quality?\n\n2. In line 287, the paper states that \"a large cutoff value (p = 0.5) disrupts the selection balance among attention heads, leading to slight performance degradation.\" Could the authors elaborate on the mechanism behind this observation? A more detailed explanation or visualization would be helpful.\n\n3. What is the speedup achieved when using only AttenCarve and ProRes (without applying timeskip)?", "was_revised": true, "final_justification": "This paper accelerates the inference speed of VDM by incorporating sparse attention and progressive resolution denoising. Compared to existing sparse attention strategies, the proposed method achieves higher sparsity while maintaining generation quality. Moreover, when combined with current approaches, it achieves an 8× speed-up. Therefore, I believe this paper meets the standard for acceptance.", "rebuttal": "Dear reviewer N3G7,\n\nThank you for your valuable feedback and constructive comments. We are grateful for your **clear positive** assessment of our work, particularly your recognition of its dynamic token carving strategy and comprehensive experiments. We will address your comments below and incorporate the necessary revisions in the updated paper.\n\n## Q1 Stage Ablation for Progressive Resolution\n\nThank you for this insightful question. This is indeed an important ablation study we conducted but did not systematically report. We provide supplementary experiments in the table below:\n\n|Low-Res Step|10%|30%|50%|70%|90%|\n|-|-|-|-|-|-|\n|Traj-Timestep|987|947|883|767|437|\n|VBench|82.36%|82.35%|83.07%|81.03%|78.74%|\n|Latency|286s|253s|225s|207s|169s|\n* The experiment uses Jenga-Turbo's basic setting, containing one low-res > high-res stage transition.\n\nWe have included a detailed analysis in **G2mf Q2** as follows:\n>The experimental results reveal key insights about trajectory-guided resolution scheduling. Due to the timestep shift mechanism in DiT, more computational steps are allocated to the early stages of the trajectory. Even in the final 10% of timesteps, the actual trajectory still contains approximately 43.7% of the total steps.\n\n>We observe a dramatic quality degradation when low-res steps exceed 50% of the total timesteps, while **introducing latent misalignment artifacts**, confirming that multi-stage generation requires sufficient high-resolution denoising to maintain video quality. Meanwhile, when stage 1 (low-res) steps are too few, quality also decreases. This aligns with our motivation (L48-49) that low-res content generation requires a higher attention density to establish proper content structure. This finding validates why ProRes and AttenCarve work synergistically--they are not merely independent methods but complementary components that address different phases of the generation process.\n\n>The 50% low-res step configuration achieves the best balance between quality (83.07%) and efficiency (225s), demonstrating that trajectory curvature analysis effectively guides optimal resolution scheduling decisions.\n\nMeanwhile, ensuring sufficient high-resolution steps is indeed a critical factor for maintaining generation quality in Jenga. This is partially a consequence of adhering to our training-free methodology. Some works like FlashVideo propose that \"high resolution generation is relatively easy to optimize into few steps.\" Training a few-step detail generation model is certainly possible; however, such approaches require substantial data and computational resources, and may be superseded by stronger foundation generation models. Our training-free approach provides a more sustainable and generalizable solution that can readily adapt to newer and more powerful foundation video generation models without requiring retraining.\n\n## Q2 The Impact of Cutoff Probability\n\nThis is a valuable question. The cutoff probability threshold $p$ is one of the key hyperparameters that controls the sparsity-quality trade-off in our attention carving mechanism. \n\nWe conducted systematic experiments to understand its impact in HunyuanVideo, with a Jenga-Turbo setting.\n\n|Cutoff Probability $p$|0.0|0.3|0.5|0.7|0.9|1.0 (Full Attention)|\n|-|-|-|-|-|-|-|\n|VBench|82.85%|83.07%|82.75%|82.18%|82.59%|82.57%|\n|Latency|227s|225s|232s|271s|330s|495s|\n|Effective Sparsity|80.3%|80.1%|78.5%|72.3%|57.5%|0.0%\n|STD Across Heads|0|545|19001|23546|70551|0|\n|Diff with Full Attention|0.141|0.136|0.132|0.115|0.074|0|\n* The latency results may have some fluctuations (not exceeding 5s), but the overall trend is correct.\n\nFrom these results, we derive the following analysis: Higher $p$ values make the model results approach Full Attention. We computed the standard deviation (STD) of selected block quantities across different heads and the average difference between AttenCarve and full attention. Due to varying feature extraction capabilities of different heads (visualized in Appendix C.2), the STD increases with higher $p$ values. We also observe that the local nature of attention is revealed: Effective Sparsity is significantly greater than the theoretical average selection value (57.5% vs. 10%).\n\nWe believe that both low and high cutoff probability values are reasonable in this context: With high $p=0.9$, covering more probability regions ensures a certain degree of sparsity while allowing each head to retain most of the original features. Although $p=0.9$ does not achieve optimal performance in HunyuanVideo, it closely approaches the full attention baseline (82.57%, same as ProRes + timeskip in **Table 1**) and achieves the best results in Wan2.1. \n\nWith lower $p=0.3$, AttenCarve selects the most important blocks, with the removal of long-tailed features in attention **balancing the block selections among heads (with a small STD)** and boosting performance. We guarantee higher sparsity while partially preserving the most important components in certain global heads.\n\nIntermediate $p$ values (0.5, 0.7) introduce relatively unimportant and partial features in attention heads (evidenced by the dramatic STD increase across heads without fully preserving global head representations), thereby reducing sparsity without improving results. We will incorporate this discussion into the revision, and thank you for prompting us to explore these ablation phenomena more deeply.\n\n## Q3 Module-Wise Speed-Up Ablations\n\nWe provided individual results for AttenCarve and ProRes in Table 1. AttenCarve and ProRes achieve speedups of 2.17× and 1.51× respectively (3.28× when combined with timestep skip). For a module-wise evaluation, we provide detailed parameter configurations in Table 4 in the Appendix. Below is an excerpt from Table 1, with additional module combinations. We use a two-stage basic setting (Jenga-Turbo), where attention sparsity is also applied in a two-stage manner in AttenCarve experiments.\n\n|Method|VBench-Q|VBench-S|VBench|Latency|Speed-Up|\n|-|-|-|-|-|-|\n|Baseline|85.21%|72.84%|82.74%|1625s|1.00×|\n|AttenCarve Only|85.31%|75.85%|83.42%|748s|2.17×|\n|ProRes Only|86.20%|69.43%|82.85%|1075s|1.51×|\n|ProRes + Step Skip|85.78%|69.73%|82.57%|495s|3.28×|\n|AttenCarve + ProRes|84.65%|76.98%|83.12%|485s|3.35×|\n|Jenga-Turbo (Combined)|84.47%|77.48%|83.07%|225s|7.22×|\n\nThe results demonstrate that **the acceleration ratio can essentially be viewed as the product of individual method speedups**. The synergistic combination of these approaches enables Jenga to achieve high acceleration ratios while maintaining video generation quality. The organic integration of ProRes and AttenCarve working together produces even greater performance gains than the sum of their individual contributions would suggest."}, {"review_id": "EQNZAID4oH", "reviewer": "Reviewer_iQ7G", "summary": "The paper presents Jenga, a plug-and-play, training-free method to accelerate video diffusion Transformers, reducing inference time from minutes to seconds. Contributions include the introduction of block-wise attention carving for sparse computation, as well as the incorporation of progressive resolution for further acceleration. Additionally, Jenga is applied during inference without requiring any training.", "questions": "1.\tRegarding novelty: It is recommended that the authors reframe the motivation and its correspondence with the proposed method to better highlight the novelty of the approach.\n2.\tRegarding acceleration performance: The authors are encouraged to provide more detailed explanations. If the acceleration is based on TeaCache, the speedup appears to be modest.\n3.\tRegarding metrics: It is suggested that the authors include objective metrics such as FVD to provide a more comprehensive evaluation.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper has a well-structured format and the illustrations are visually appealing.\n2.\tThe training-free design enhances its generalizability.\n3.\tThe design of the masks in the method is quite clever and insightful.\n4.\tThe experiments are conducted across multiple methods, demonstrating the generalizability of Jenga.\n\nOverall, the design of the mask and the complete experiments make me inclined to accept this work.\n\n---\n\nWeakness:\n\n1.\tThe observation that early denoising steps in diffusion do not require high-resolution latents is not novel and has been previously discussed in the literature. The progressive resolution design appears more heuristic than principled. Additionally, there seems to be a potential inconsistency between the claim in Lines 45–47 that early steps establish global structure, and the observation in Line 60 that low-resolution generation tends to emphasize local, zoomed-in details. Clarifying this relationship would strengthen the motivation for ProRes.\n2.\tThe sparse attention mechanism proposed in Jenga shares similarities with prior work such as CLEAR [1] and SVG [2], which also adopt token or block sparsity. While the proposed block-wise dynamic selection is interesting, it may not represent a fundamentally new direction compared to these existing techniques.\n3.\tIn Table 2(a), the acceleration improvement of Jenga over TeaCache for Wan2.1 appears relatively modest (from 34s to 24s). It would be helpful to clarify whether this result is achieved by applying Jenga on top of TeaCache or independently. This distinction is important for evaluating the true incremental benefit of the method.\n4.\tThe evaluation currently focuses primarily on perceptual and semantic alignment metrics such as VBench and CLIPScore. However, the absence of structural fidelity metrics (e.g., FVD[3]) makes it difficult to assess the overall visual consistency and realism of the generated videos. Including such measures would provide a more comprehensive assessment of generation quality.\n\n[1] Liu S, Tan Z, Wang X. CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up[J]. arXiv preprint arXiv:2412.16112, 2024.\n\n[2] Xi H, Yang S, Zhao Y, et al. Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity[J]. arXiv preprint arXiv:2502.01776, 2025.\n\n[3] Unterthiner T, Van Steenkiste S, Kurach K, et al. Towards accurate generative models of video: A new metric & challenges[J]. arXiv preprint arXiv:1812.01717, 2018.", "was_revised": true, "final_justification": "The author's response resolved my confusion and I decided to maintain my rating.", "rebuttal": "Dear reviewer iQ7G,\n\nThank you for your valuable feedback and constructive comments. We are grateful for your positive assessment of our work, particularly your recognition of Jenga's generalizability and comprehensive experiments. We will address your comments below and incorporate the necessary revisions in the updated paper.\n\n## Q1 The Novelty & Motivation of Jenga \n\n### Q1.1 ProRes: Low-to-High Resolution\nWe agree that multi-stage resolution progression is not entirely novel in isolation. However, several challenges arise in training-free pipelines, including the abnormal FOV problem discussed in L60. Jenga is the first training-free open-sourced progressive generation pipeline for Video Diffusion Transformers. As stated in **G2mf Q5**, most importantly, **ProRes solves the content degradation when adopting high sparsity attention across all timesteps, maintaining both efficiency and quality throughout the generation process.**\n\n**Regarding the inconsistent expression**, `early steps` and `generate low resolution contents` represent distinct concepts. In high-resolution generation, early steps produce global content while maintaining normal field of view (FOV). Similarly, in low-resolution generation, early steps also generate global content encompassing overall layout and structures. However, low-resolution content inherently introduces a \"zoom-in\" FOV degradation, where the generated layout and structures appear zoomed-in, presenting only partial views rather than the intended global perspective. We will clarify this distinction in our revision.\n\n### Q1.2 AttenCarve: Sparse Attention is Not New\nWe acknowledge that utilizing sparsity for attention acceleration is not fundamentally novel. Our work does not claim to be \"the first sparse attention design for video generation.\" Instead, we focus on how to effectively leverage sparse attention through principled design choices and achieve higher sparsity levels while maintaining quality.\n\nCurrent widely-used attention approaches suffer from limitations: they are either static (lacking content adaptivity, **cBBb Q2**) or achieve only low sparsity levels ($\\sim$0.5). Our method introduces SFC-based block partitioning and cutoff probability mechanisms to address resolution-dependent challenges and achieve substantially higher sparsity ($\\sim$0.75-0.80) without quality degradation. The technical contribution lies in the systematic combination of these components to create a practical, training-free acceleration framework that works across different resolutions and model architectures.\n\n### Q1.3 Complementary Nature of ProRes and Attention Carving\n\nThe core innovation of Jenga lies in recognizing that **attention sparsity and progressive resolution are fundamentally complementary** techniques that address different aspects of the computational bottleneck in video diffusion models:\n\n**1. Temporal Complementarity**: Based on diffusion denoising theory, the generation process naturally progresses from low to high-frequency content. ProRes leverages this by using lower resolutions during early timesteps when content structure is being established, while Attention Carving becomes most effective during later timesteps when high-resolution details are refined but full attention is no longer necessary.\n\n**2. Spatial Complementarity**: ProRes reduces the **total number of tokens** (addressing the N in O(N²) complexity), while Attention Carving focuses on reducing the **density of token interactions in high resolution** (addressing the practical computation within the O(N²) framework). This dual approach creates a multiplicative effect on acceleration.\n\n**3. Content-Generation Constraints and Resolution Trade-offs**: During content generation phases, AttenCarve cannot employ extremely high attention sparsity levels without compromising content quality, thus limiting its acceleration potential. However, content generation fundamentally does not require high-resolution inputs. ProRes strategically exploits this by reducing token interactions during content generation through lower resolutions, while preserving the ability to apply higher sparsity levels during high-resolution detail refinement when content structure is already established. This creates a **sparsity-resolution scheduling**: conservative sparsity at low resolution (content generation) and aggressive sparsity at high resolution (detail refinement), maximizing acceleration while maintaining generation quality.\n\n**4. Orthogonality**: Unlike competing acceleration methods that often interfere with each other, ProRes and Attention Carving operate on orthogonal dimensions--resolution scaling vs. attention sparsity--allowing them to be combined without negative interactions. This explains why our combined approach achieves 7.22× speedup compared to individual contributions of 3.28× (ProRes) and 2.17× (Attention Carving).\n\n## Q2 Structural Fidelity Metrics\n\nThank you for raising this important point about evaluation metrics. We acknowledge that structural fidelity metrics like FVD (Fréchet Video Distance) provide additional validation of our method's effectiveness. While our primary evaluation focuses on VBench for comprehensive video quality assessment, we recognize the value of complementary metrics that capture different aspects of video quality. We also provide detailed VBench-Q quality assessments in **Appendix Table 6**, including video consistency, visual quality, smoothness, dynamic degree, etc.\n\nWe conducted additional experiments with FVD to provide a more complete evaluation. As FVD evaluates the distance between **training data distribution** and **inference data distribution**, as a training-free method, we adopt 1. the **official** VBench test result (with different sampling seeds) and 2. the high-quality video dataset Inter4K as real data distributions to evaluate FVD.\n\n**The tables in Q3 further compare methods using FVD metrics**. The results show that Jenga preserves video generation quality comparable to the baseline, achieving an FVD score of 141 versus 144 in terms of structural fidelity. The FVD results align with the VBench trends observed across different methods.\n\n## Q3 TeaCache & Jenga: Independent Results (with FVD metrics)\n\nWe appreciate your interest in understanding the relationship between our work and complementary acceleration techniques like TeaCache. The experiments with TeaCache demonstrate that Jenga achieves substantial acceleration independently and can be combined with other orthogonal methods, including feature reuse and few-step models (**G2mf Q3**).\n\n|Method|VBench|Latency|Speed-Up|FVD-Inter4K ↓|FVD-Hunyuan ↓|\n|-|-|-|-|-|-|\n|HunyuanVideo|82.74%|1625s|1.00×|600|144|\n|AttenCarve Only|83.42%|748s|2.17×|448|164|\n|ProRes Only|82.85%|1075s|1.51×|620|191|\n|ProRes + Skip|82.57%|495s|3.28×|630|203|\n|AttenCarve + ProRes + Skip (Jenga-Turbo)|83.07%|225s|7.22×|583|141|\n|**AttenCarve + ProRes**|83.12%|485s|3.35×|542|127|\n\nHere we also report updated performance on the Wan2.1 model. Due to time constraints, we only tested AttenCarve + Skip on the Wan series in the submission. We have updated the Jenga-Turbo setting, which further boosts efficiency.\n\n|Method|VBench|Latency|Speed-Up|FVD-Inter4K ↓|FVD-Wan2.1 ↓|\n|-|-|-|-|-|-|\n|Wan2.1-1.3B|83.28%|115s|1.00×|788|182|\n|TeaCache-fast|82.63%|34s|3.48×|738|232|\n|AttenCarve + Skip (Jenga-Base)|82.68%|24s|4.79×|702|184|\n|AttenCarve Only|82.96%|71s|1.62×|674|169|\n|AttenCarve + ProRes + Skip (Jenga-Turbo)|82.52%|17s|**6.52×**|548|218|\n|**AttenCarve + ProRes**|82.86%|52s|2.21×|565|214|\n\nAs discussed in **Q2 and G2mf Q3**, our results demonstrate that Jenga's acceleration is fundamentally independent of feature caching techniques like TeaCache. **We believe that further improvement on the basis of a mature acceleration method is both challenging and necessary.** One reason for the less significant speedup ratio is that the native input resolution (480P videos) is relatively small, resulting in a smaller proportion of attention computation. However, the reduction in the timestep dimension is linear. While TeaCache focuses on reusing computed features across timesteps, Jenga reduces computation through progressive resolution and selective attention, while also boosting generation quality. This orthogonality means the methods can be combined for potentially even greater acceleration benefits without quality degradation.\n\n**Future Integration:** We plan to explore combinations with TeaCache and other orthogonal efficient generation methods in future work, such as token purning, SageAttention (quantized attention), etc.\n\n[1] Stergiou, Alexandros and Poppe, Ronald, AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling, 2021"}, {"review_id": "AxDczOYCWB", "reviewer": "Reviewer_cBBb", "summary": "This paper presents a training-free and efficient video generation method called Jenga. It involves two aspects: dynamic attention carving and progressive resolution generation, which is inspired by the observed insight (1) early denoising steps do not require high-resolution latents, and (2) later steps do not require dense attention. The experiments are extensively conducted and the performance is superior, for example, x8.83 speedup with 0.01% performance drop on VBench.", "questions": "I am a little bit confused about the title and content. It seems the the title is highly related with attention carving. But Jenga involves progressive resolution generation. The two parts, attention carving and progressive resolution generation, are independent. Could the author explain it? In my view, it could be simply a combination of the two tricks. \n\n\nIt seems that the authors partially ablate different masking strategies. It could be better to increasingly add different masking strategies.", "limitations": "The authors have highlighted the limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nClear to follow\n\nExtensive and logical experiment\n\nNice illustration\n\nWeaknesses:\n\nLatent alignment and artifacts during resolution transitions\n\nUses static space-filling curves (SFCs) for latent block partitioning\n\npurely heuristic and post-hoc attention carving, without adaptability", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer cBBb,\n\nThank you for your valuable feedback and constructive comments. We are grateful for your positive assessment of our work, particularly your recognition of its clarity and comprehensive experiments. We will address your comments below and incorporate the necessary revisions in the updated paper.\n\n## Q1 Latent Alignment May Introduce Artifacts\n\nWe acknowledge this limitation in our discussion (L302-305) and provide a **detailed discussion in Appendix C.1**. Indeed, latent alignment requires certain trade-offs as a training-free method, since pure pixel-level decode-encode operations consume approximately 40 seconds, which is unacceptable for a video generation acceleration framework. Nevertheless, we provide several mitigation strategies and discussions:\n\n**1. High noise levels minimize alignment impact from resize+renoise operations.** During early denoising timesteps when noise levels are substantial (883/1000 of the actual denoising trajectory, in **G2mf Q2**), the resize and renoise operations have minimal impact on latent alignment since the semantic content is not yet well-formed.\n\n**2. Noise shift compensation.** Discussed in **Appendix L339-342**, we introduce adaptive noise shift adjustments to maintain consistency across resolution transitions, helping preserve the underlying latent structure.\n\n**3. Enhanced semantic conditioning.** By incorporating richer semantic conditions during the alignment process (i.e., using enhanced prompts, with results in **Appendix Table 5** as validation), we can better preserve content coherence across different resolutions.\n\n**4. Faster VAE integration.** Future work can leverage faster VAE variants to reduce the computational overhead of pixel-level operations while maintaining alignment quality.\n\n## Q2 SFC is Static\n\nYour observation is accurate. SFC construction is completed before denoising and only depends on resolution and dimension ordering (e.g., hwt or thw, representing different orderings). As acknowledged in our limitation section (L305-307), this approach has room for improvement. We explain the rationale behind Jenga's SFC choice and challenges with adaptive partitioning:\n\n**1. SFC block partition exhibits inherent locality properties.** Attention computation for high-resolution data predominantly exhibits local characteristics (e.g., STA, SeedVR). SFC naturally possesses local-neighborhood properties, where neighbors on the 1D curve correspond to neighbors in 3D space. We validated in Table 3(b) that SFC improves both latency and performance while reducing line-wise drifting artifacts.\n\n**2. SFC demonstrates remarkable parameter insensitivity.** Our Generalized Hilbert curves maintain locality while being insensitive to parameters. Hand-crafted block partition strategies (local windows in STA & CLEAR, Swin windows in SeedVR) restrict models to specific resolutions, with dimensional padding severely impacting performance. SFC's 1D nature makes it insensitive to resolution and block_size parameters. The table below shows additional padding and computational overhead for 720×1280×129f videos with block_size=128:\n\n|Mask Type|STA Tiled Local (6, 8, 8)|Optimized Local (3, 8, 16)|3D-SFC|\n|-|-|-|-|\n|Padding Tokens|19440|7920|112|\n|Additional MatMul Computation|35.32%|13.78%|0.19%|\n\n\"Optimized Local\" represents the optimal GPU-friendly block partitioning, still significantly larger than Jenga's 3D SFC.\n\n**3. Adaptive Partition: Cannot be defined at arbitrary timesteps.** Text-to-video generation starts from pure noise, making it challenging for adaptive token partitioning to extract meaningful semantic information. Determining adaptive partitions through latent semantic correspondence in early denoising timesteps is problematic.\n\n**4. Benefits of static approaches: Frequent reordering degrades generation speed.** We experimented with changing SFC dimension scanning directions during interleaved forward attention passes to enhance block interactions (similar to Swin-window shift). This approach yielded no significant quality improvements while introducing substantial processing overhead (~20s per video). Token reordering operations cause memory discontinuity and defragmentation overhead. **Adaptive block partition strategies would face similar challenges**, with additional overhead for constructing the adaptive strategy itself--an overhead that static SFC avoids entirely through pre-computation.\n\n## Q3 Adaptability of the Attention Carving\n\nFollowing our discussion on block partitioning in Q2, we acknowledge that Attention Carving employs heuristic strategies. In practice, during Attention Carving, within each attention head, the block selection for each query block is adaptive and content-dependent. The importance mask $\\mathbf{B}_\\text{top}$ is dynamically computed based on the current latent features and query-key relationships, making the carving process responsive to the evolving content during generation. While the SFC partitioning strategy is static, the actual attention patterns and block selections are adaptive to the semantic content at each timestep and layer, ensuring that the most relevant spatial-temporal relationships are preserved during the carving process.\n\nAs a pure training-free method, the current version avoids the need for offline tuning procedures. Jenga's significantly smaller search space makes it easily adaptable to new models without extensive hyperparameter tuning. It would be a future direction to design a metric to determine adaptive sparsity.\n\n## Q4 Title: \"Token Carving\", It Seems Not Related With ProRes?\n\nProRes fundamentally reduces token interactions as well. By reducing resolution (and thus the input token count), ProRes effectively reduces token interactions. Both ProRes and Attention Carving share the same core motivation: **reducing computational complexity through strategic token interaction reduction**. ProRes achieves this through progressive resolution scaling, while Attention Carving accomplishes this through selective attention masking. The \"carving\" metaphor aptly describes both processes--ProRes \"carves\" away lower-resolution details in the early stage, while Attention Carving \"carves\" away less important attention connections. Together, they form a unified framework for efficient video generation that operates across both input token dimensions and attention pattern dimensions.\n\n## Q5 Two Methods are Independent (similar to iQ7G Q1.3)\n\nWhile ProRes and Attention Carving operate independently, the core innovation of Jenga lies in recognizing that **attention sparsity and progressive resolution are fundamentally complementary** techniques that address different aspects of the computational bottleneck in video diffusion models:\n\n**1. Temporal Complementarity**: Based on diffusion denoising theory, the generation process naturally progresses from low to high-frequency content. ProRes leverages this by using lower resolutions during early timesteps when content structure is being established, while Attention Carving becomes most effective during later timesteps when high-resolution details are refined but full attention is no longer necessary.\n\n**2. Spatial Complementarity**: ProRes reduces the **total number of tokens** (addressing the N in O(N²) complexity), while Attention Carving focuses on reducing the **density of token interactions in high resolution** (addressing the practical computation within the O(N²) framework). This dual approach creates a multiplicative effect on acceleration.\n\n**3. Content-Generation Constraints and Resolution Trade-offs**: During content generation phases, AttenCarve cannot employ extremely high attention sparsity levels without compromising content quality, thus limiting its acceleration potential. However, content generation fundamentally does not require high-resolution inputs. ProRes strategically exploits this by reducing token interactions during content generation through lower resolutions, while preserving the ability to apply higher sparsity levels during high-resolution detail refinement when content structure is already established. This creates a **sparsity-resolution scheduling**: conservative sparsity at low resolution (content generation) and aggressive sparsity at high resolution (detail refinement), maximizing acceleration while maintaining generation quality.\n\n**4. Orthogonality**: Unlike competing acceleration methods that often interfere with each other, ProRes and Attention Carving operate on orthogonal dimensions--resolution scaling vs. attention sparsity--allowing them to be combined without negative interactions. This explains why our combined approach achieves 7.22× speedup compared to individual contributions of 3.28× (ProRes) and 2.17× (Attention Carving).\n\n## Q6 Ablation on Mask Strategies\n\nIn Table 3(b) and Figure 7, we provided an ablation study on SFC selection and mask selection. We acknowledge that more comprehensive ablation studies (i.e., incrementally adding strategies) would better validate our method's effectiveness. Here we present a more detailed ablation study to provide better validation of our method:\n\n|Mask Type|Importance|Condition|Adjacency|VBench|Latency|\n|-|-|-|-|-|-|\n|No Mask (FA)|Full|Full|Full|82.57%|495s|\n|TopK only|✓ (partially)|||81.35%|220s|\n|TopK, SFC|✓ (partially)|||81.41%|220s|\n|TopK, Prob. Constraint, SFC|✓|||81.87%|223s|\n|w/o Adjacency|✓|✓||82.42%|222s|\n|w/o Condition|✓||✓|81.82%|221s|\n|w/o Importance||✓|✓|77.41%|140s|\n|All Mask|✓|✓|✓|83.07%|225s|\n\nThis comprehensive ablation demonstrates that: (1) Each mask component contributes meaningfully to both quality and efficiency, (2) The importance mask provides the most significant impact on both metrics, (3) The combination of all three masks achieves the optimal balance between generation quality and computational efficiency.\n\nFor additional details regarding mask-related implementations, please refer to our response in **G2mf Q1.1**."}]

Author rebuttal:

Meta-review: This paper introduces Jenga, a tranining-free method for video DiTs which consists of attention carving and progressive resolution. Reviewers are satisfied with the novelty, performance and inference efficiency of the proposed method as well as the writing of the paper. Conerns raise by reviewers are addressed. In the end, the reviewers all recommend accept. After including the newly added important results as promised, I think this paper will meet the standard and therefore recommend an acceptance.
Decision: Accept (poster)